2024-05-07T02:03:28.038 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-07T02:03:28.038 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_pdonnell@teuthology --archive /home/teuthworker/archive/pdonnell-2024-05-07_01:13:22-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7695123 --name pdonnell-2024-05-07_01:13:22-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/suites/dbench}} --verbose -- /home/teuthworker/archive/pdonnell-2024-05-07_01:13:22-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7695123/orig.config.yaml 2024-05-07T02:03:28.097 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-07T02:03:28.207 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-05-07_01:13:22-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7695123 branch: wip-pdonnell-testing-20240503.163550-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/suites/dbench}} email: pdonnell@redhat.com first_in_suite: false job_id: '7695123' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: pdonnell-2024-05-07_01:13:22-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240503.163550-debug ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: bd4e00a39d766328b458e4c953f4d722fed96a31 subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: bd4e00a39d766328b458e4c953f4d722fed96a31 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240503.163550-debug sha1: 366c9130c497dee9e73178650cf710ed21269b18 owner: scheduled_pdonnell@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1842 sha1: bd4e00a39d766328b458e4c953f4d722fed96a31 sleep_before_teardown: 0 subset: 478/512 suite: fs suite_branch: wip-pdonnell-testing-20240503.163550-debug suite_path: /home/teuthworker/src/github.com_batrick_ceph_366c9130c497dee9e73178650cf710ed21269b18/qa suite_relpath: qa suite_repo: https://github.com/batrick/ceph suite_sha1: 366c9130c497dee9e73178650cf710ed21269b18 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi019.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMExTw6uTcQ/LDLck8NATCpkcpMwEYT+RWGfuCXeLD0RI0/uVnULJaFft0JA3DigNOV7Mv4W4PB3Ws7Rt1rEChc= smithi027.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNmuBCEQzBakC2p701jvGrHf2GNbVNRidlwag4DGI3+QDp6ivApD5TdsnF8kERWUrhuA0W2Q30cqf0HPhNJ4GZA= smithi064.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJPPQB5lzfuaGlCVXiaJCgSaV7HuDbtLw4hpOTnLsSiabhRbvPXRoJWA8y2KFELtO4jPEtIaQup9Avy9rInFoJE= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-07_01:13:22 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-07T02:03:28.207 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_batrick_ceph_366c9130c497dee9e73178650cf710ed21269b18/qa; will attempt to use it 2024-05-07T02:03:28.208 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_batrick_ceph_366c9130c497dee9e73178650cf710ed21269b18/qa/tasks 2024-05-07T02:03:28.208 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-07T02:03:28.210 INFO:teuthology.task.internal:Checking packages... 2024-05-07T02:03:28.234 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'bd4e00a39d766328b458e4c953f4d722fed96a31' 2024-05-07T02:03:28.234 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-07T02:03:28.234 INFO:teuthology.packaging:ref: None 2024-05-07T02:03:28.234 INFO:teuthology.packaging:tag: None 2024-05-07T02:03:28.234 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240503.163550-debug 2024-05-07T02:03:28.234 INFO:teuthology.packaging:sha1: bd4e00a39d766328b458e4c953f4d722fed96a31 2024-05-07T02:03:28.234 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240503.163550-debug 2024-05-07T02:03:28.379 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3522.gbd4e00a3 2024-05-07T02:03:28.381 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-07T02:03:28.388 INFO:teuthology.task.internal:no buildpackages task found 2024-05-07T02:03:28.388 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-07T02:03:28.464 INFO:teuthology.task.internal:Saving configuration 2024-05-07T02:03:28.485 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-07T02:03:28.492 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-07T02:03:28.515 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi019.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-07_01:13:22-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7695123', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-07 01:57:12.361367', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMExTw6uTcQ/LDLck8NATCpkcpMwEYT+RWGfuCXeLD0RI0/uVnULJaFft0JA3DigNOV7Mv4W4PB3Ws7Rt1rEChc='} 2024-05-07T02:03:28.540 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi027.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-07_01:13:22-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7695123', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-07 01:57:12.359806', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNmuBCEQzBakC2p701jvGrHf2GNbVNRidlwag4DGI3+QDp6ivApD5TdsnF8kERWUrhuA0W2Q30cqf0HPhNJ4GZA='} 2024-05-07T02:03:28.560 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi064.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-07_01:13:22-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7695123', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-07 01:57:12.362690', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJPPQB5lzfuaGlCVXiaJCgSaV7HuDbtLw4hpOTnLsSiabhRbvPXRoJWA8y2KFELtO4jPEtIaQup9Avy9rInFoJE='} 2024-05-07T02:03:28.560 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-07T02:03:28.566 INFO:teuthology.task.internal:roles: ubuntu@smithi019.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-07T02:03:28.566 INFO:teuthology.task.internal:roles: ubuntu@smithi027.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-07T02:03:28.567 INFO:teuthology.task.internal:roles: ubuntu@smithi064.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-07T02:03:28.567 INFO:teuthology.run_tasks:Running task console_log... 2024-05-07T02:03:28.664 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f958833c0d0>, signals=[15]) 2024-05-07T02:03:28.664 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-07T02:03:28.671 INFO:teuthology.task.internal:Opening connections... 2024-05-07T02:03:28.671 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi019.front.sepia.ceph.com 2024-05-07T02:03:28.673 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-07T02:03:28.748 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi027.front.sepia.ceph.com 2024-05-07T02:03:28.749 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-07T02:03:28.820 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi064.front.sepia.ceph.com 2024-05-07T02:03:28.821 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-07T02:03:28.895 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-07T02:03:28.902 DEBUG:teuthology.orchestra.run.smithi019:> uname -m 2024-05-07T02:03:28.920 INFO:teuthology.orchestra.run.smithi019.stdout:x86_64 2024-05-07T02:03:28.920 DEBUG:teuthology.orchestra.run.smithi019:> cat /etc/os-release 2024-05-07T02:03:28.978 INFO:teuthology.orchestra.run.smithi019.stdout:NAME="CentOS Stream" 2024-05-07T02:03:28.979 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION="9" 2024-05-07T02:03:28.979 INFO:teuthology.orchestra.run.smithi019.stdout:ID="centos" 2024-05-07T02:03:28.979 INFO:teuthology.orchestra.run.smithi019.stdout:ID_LIKE="rhel fedora" 2024-05-07T02:03:28.979 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION_ID="9" 2024-05-07T02:03:28.979 INFO:teuthology.orchestra.run.smithi019.stdout:PLATFORM_ID="platform:el9" 2024-05-07T02:03:28.979 INFO:teuthology.orchestra.run.smithi019.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-07T02:03:28.979 INFO:teuthology.orchestra.run.smithi019.stdout:ANSI_COLOR="0;31" 2024-05-07T02:03:28.979 INFO:teuthology.orchestra.run.smithi019.stdout:LOGO="fedora-logo-icon" 2024-05-07T02:03:28.979 INFO:teuthology.orchestra.run.smithi019.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-07T02:03:28.979 INFO:teuthology.orchestra.run.smithi019.stdout:HOME_URL="https://centos.org/" 2024-05-07T02:03:28.979 INFO:teuthology.orchestra.run.smithi019.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-07T02:03:28.979 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-07T02:03:28.979 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-07T02:03:28.980 INFO:teuthology.lock.ops:Updating smithi019.front.sepia.ceph.com on lock server 2024-05-07T02:03:29.003 DEBUG:teuthology.orchestra.run.smithi027:> uname -m 2024-05-07T02:03:29.021 INFO:teuthology.orchestra.run.smithi027.stdout:x86_64 2024-05-07T02:03:29.021 DEBUG:teuthology.orchestra.run.smithi027:> cat /etc/os-release 2024-05-07T02:03:29.079 INFO:teuthology.orchestra.run.smithi027.stdout:NAME="CentOS Stream" 2024-05-07T02:03:29.079 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION="9" 2024-05-07T02:03:29.079 INFO:teuthology.orchestra.run.smithi027.stdout:ID="centos" 2024-05-07T02:03:29.079 INFO:teuthology.orchestra.run.smithi027.stdout:ID_LIKE="rhel fedora" 2024-05-07T02:03:29.079 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION_ID="9" 2024-05-07T02:03:29.080 INFO:teuthology.orchestra.run.smithi027.stdout:PLATFORM_ID="platform:el9" 2024-05-07T02:03:29.080 INFO:teuthology.orchestra.run.smithi027.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-07T02:03:29.080 INFO:teuthology.orchestra.run.smithi027.stdout:ANSI_COLOR="0;31" 2024-05-07T02:03:29.080 INFO:teuthology.orchestra.run.smithi027.stdout:LOGO="fedora-logo-icon" 2024-05-07T02:03:29.080 INFO:teuthology.orchestra.run.smithi027.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-07T02:03:29.080 INFO:teuthology.orchestra.run.smithi027.stdout:HOME_URL="https://centos.org/" 2024-05-07T02:03:29.080 INFO:teuthology.orchestra.run.smithi027.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-07T02:03:29.080 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-07T02:03:29.080 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-07T02:03:29.080 INFO:teuthology.lock.ops:Updating smithi027.front.sepia.ceph.com on lock server 2024-05-07T02:03:29.103 DEBUG:teuthology.orchestra.run.smithi064:> uname -m 2024-05-07T02:03:29.122 INFO:teuthology.orchestra.run.smithi064.stdout:x86_64 2024-05-07T02:03:29.122 DEBUG:teuthology.orchestra.run.smithi064:> cat /etc/os-release 2024-05-07T02:03:29.180 INFO:teuthology.orchestra.run.smithi064.stdout:NAME="CentOS Stream" 2024-05-07T02:03:29.180 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION="9" 2024-05-07T02:03:29.180 INFO:teuthology.orchestra.run.smithi064.stdout:ID="centos" 2024-05-07T02:03:29.180 INFO:teuthology.orchestra.run.smithi064.stdout:ID_LIKE="rhel fedora" 2024-05-07T02:03:29.180 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION_ID="9" 2024-05-07T02:03:29.180 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM_ID="platform:el9" 2024-05-07T02:03:29.180 INFO:teuthology.orchestra.run.smithi064.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-07T02:03:29.180 INFO:teuthology.orchestra.run.smithi064.stdout:ANSI_COLOR="0;31" 2024-05-07T02:03:29.180 INFO:teuthology.orchestra.run.smithi064.stdout:LOGO="fedora-logo-icon" 2024-05-07T02:03:29.180 INFO:teuthology.orchestra.run.smithi064.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-07T02:03:29.180 INFO:teuthology.orchestra.run.smithi064.stdout:HOME_URL="https://centos.org/" 2024-05-07T02:03:29.180 INFO:teuthology.orchestra.run.smithi064.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-07T02:03:29.180 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-07T02:03:29.181 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-07T02:03:29.181 INFO:teuthology.lock.ops:Updating smithi064.front.sepia.ceph.com on lock server 2024-05-07T02:03:29.203 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-07T02:03:29.216 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-07T02:03:29.228 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-07T02:03:29.228 DEBUG:teuthology.orchestra.run.smithi019:> test '!' -e /home/ubuntu/cephtest 2024-05-07T02:03:29.231 DEBUG:teuthology.orchestra.run.smithi027:> test '!' -e /home/ubuntu/cephtest 2024-05-07T02:03:29.233 DEBUG:teuthology.orchestra.run.smithi064:> test '!' -e /home/ubuntu/cephtest 2024-05-07T02:03:29.251 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-07T02:03:29.256 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-07T02:03:29.256 DEBUG:teuthology.orchestra.run.smithi019:> test -z $(ls -A /var/lib/ceph) 2024-05-07T02:03:29.288 DEBUG:teuthology.orchestra.run.smithi027:> test -z $(ls -A /var/lib/ceph) 2024-05-07T02:03:29.293 DEBUG:teuthology.orchestra.run.smithi064:> test -z $(ls -A /var/lib/ceph) 2024-05-07T02:03:29.321 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-07T02:03:29.430 INFO:teuthology.run_tasks:Running task kernel... 2024-05-07T02:03:29.444 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-05-07T02:03:29.444 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-05-07T02:03:29.444 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-05-07T02:03:29.444 DEBUG:teuthology.orchestra.run.smithi019:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-07T02:03:29.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-07T02:03:29.462 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-05-07T02:03:29.692 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-05-07T02:03:29.692 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-05-07T02:03:29.693 DEBUG:teuthology.orchestra.run.smithi019:> uname -r 2024-05-07T02:03:29.710 INFO:teuthology.orchestra.run.smithi019.stdout:5.14.0-437.el9.x86_64 2024-05-07T02:03:29.710 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-05-07T02:03:29.711 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-07T02:03:29.711 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum upgrade -y linux-firmware 2024-05-07T02:03:36.396 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - BaseOS 1.6 MB/s | 8.1 MB 00:04 2024-05-07T02:03:50.929 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - AppStream 1.4 MB/s | 19 MB 00:13 2024-05-07T02:03:56.412 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - CRB 3.4 MB/s | 6.2 MB 00:01 2024-05-07T02:03:57.830 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - Extras packages 75 kB/s | 16 kB 00:00 2024-05-07T02:03:59.767 INFO:teuthology.orchestra.run.smithi019.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-05-07T02:04:05.298 INFO:teuthology.orchestra.run.smithi019.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-05-07T02:04:07.474 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-05-07T02:04:07.475 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2024-05-07T02:04:07.476 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-05-07T02:04:07.584 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-05-07T02:04:07.585 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-05-07T02:04:08.195 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-05-07T02:04:08.195 DEBUG:teuthology.orchestra.run.smithi019:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-05-07T02:04:08.197 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi019.front.sepia.ceph.com to complete... 2024-05-07T02:04:21.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07 02:04:21 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-07T02:04:21.951 INFO:teuthology.orchestra.run.smithi019.stderr:FINISHED --2024-05-07 02:04:21-- 2024-05-07T02:04:21.951 INFO:teuthology.orchestra.run.smithi019.stderr:Total wall clock time: 14s 2024-05-07T02:04:21.951 INFO:teuthology.orchestra.run.smithi019.stderr:Downloaded: 1 files, 341M in 13s (25.6 MB/s) 2024-05-07T02:04:21.954 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-05-07T02:04:21.954 DEBUG:teuthology.orchestra.run.smithi019:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-07T02:04:24.133 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying... ######################################## 2024-05-07T02:04:24.244 INFO:teuthology.orchestra.run.smithi019.stdout:Preparing... ######################################## 2024-05-07T02:04:24.352 INFO:teuthology.orchestra.run.smithi019.stdout:Updating / installing... 2024-05-07T02:04:32.640 INFO:teuthology.orchestra.run.smithi019.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-05-07T02:05:12.650 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi019.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-07T02:05:12.650 DEBUG:teuthology.orchestra.run.smithi019:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-07T02:05:12.738 INFO:teuthology.orchestra.run.smithi019.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-05-07T02:05:12.738 INFO:teuthology.orchestra.run.smithi019.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-05-07T02:05:12.738 INFO:teuthology.orchestra.run.smithi019.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-05-07T02:05:12.738 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-05-07T02:05:12.738 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-05-07T02:05:12.738 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-05-07T02:05:12.738 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-05-07T02:05:12.739 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-05-07T02:05:12.739 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-05-07T02:05:12.739 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-07T02:05:12.739 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-07T02:05:12.739 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-07T02:05:12.739 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-07T02:05:12.739 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-07T02:05:12.739 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-07T02:05:12.739 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-07T02:05:12.739 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-07T02:05:12.739 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-07T02:05:12.739 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-07T02:05:12.739 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-07T02:05:12.739 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-07T02:05:12.740 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-07T02:05:12.740 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-07T02:05:12.740 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-07T02:05:12.740 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-07T02:05:12.740 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-07T02:05:12.740 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-07T02:05:12.740 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-07T02:05:12.740 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-07T02:05:12.740 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-05-07T02:05:12.740 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-05-07T02:05:12.740 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-05-07T02:05:12.740 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-07T02:05:12.740 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-05-07T02:05:12.740 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-07T02:05:12.740 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-07T02:05:12.741 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-05-07T02:05:12.741 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-05-07T02:05:12.741 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-05-07T02:05:12.741 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-05-07T02:05:12.741 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-05-07T02:05:12.741 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-05-07T02:05:12.741 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-05-07T02:05:12.741 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-05-07T02:05:12.741 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-07T02:05:12.741 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-05-07T02:05:12.741 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-05-07T02:05:12.741 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-05-07T02:05:12.741 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-05-07T02:05:12.741 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-05-07T02:05:12.741 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-05-07T02:05:12.742 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-05-07T02:05:12.742 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-05-07T02:05:12.742 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-05-07T02:05:12.742 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-05-07T02:05:12.742 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-05-07T02:05:12.742 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-05-07T02:05:12.742 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-05-07T02:05:12.742 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-05-07T02:05:12.742 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-05-07T02:05:12.742 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-05-07T02:05:12.742 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-05-07T02:05:12.742 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-05-07T02:05:12.742 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-05-07T02:05:12.742 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-05-07T02:05:12.743 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-05-07T02:05:12.743 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-05-07T02:05:12.743 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-05-07T02:05:12.743 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-05-07T02:05:12.743 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-05-07T02:05:12.743 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-05-07T02:05:12.743 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-05-07T02:05:12.743 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-05-07T02:05:12.743 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-05-07T02:05:12.743 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-05-07T02:05:12.743 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-05-07T02:05:12.743 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-05-07T02:05:12.743 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-05-07T02:05:12.743 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-05-07T02:05:12.744 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-05-07T02:05:12.744 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-05-07T02:05:12.744 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-05-07T02:05:12.744 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-05-07T02:05:12.744 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-05-07T02:05:12.744 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-05-07T02:05:12.744 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-05-07T02:05:12.744 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-05-07T02:05:12.744 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-05-07T02:05:12.744 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-05-07T02:05:12.744 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-07T02:05:12.744 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-05-07T02:05:12.744 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-05-07T02:05:12.744 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-05-07T02:05:12.745 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-05-07T02:05:12.745 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-05-07T02:05:12.745 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-05-07T02:05:12.745 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-05-07T02:05:12.745 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-05-07T02:05:12.745 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-05-07T02:05:12.745 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-05-07T02:05:12.745 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-05-07T02:05:12.745 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-05-07T02:05:12.745 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-05-07T02:05:12.745 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-05-07T02:05:12.745 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-05-07T02:05:12.745 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-05-07T02:05:12.745 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-05-07T02:05:12.745 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-05-07T02:05:12.746 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-05-07T02:05:12.746 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-05-07T02:05:12.746 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-05-07T02:05:12.746 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-05-07T02:05:12.746 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-05-07T02:05:12.746 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-07T02:05:12.746 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-07T02:05:12.746 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-05-07T02:05:12.746 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-05-07T02:05:12.746 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-05-07T02:05:12.746 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-05-07T02:05:12.746 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-05-07T02:05:12.746 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-05-07T02:05:12.746 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-05-07T02:05:12.747 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-05-07T02:05:12.747 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-05-07T02:05:12.747 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-05-07T02:05:12.747 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-05-07T02:05:12.747 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-05-07T02:05:12.747 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-05-07T02:05:12.747 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-07T02:05:12.747 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-07T02:05:12.747 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-05-07T02:05:12.747 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-05-07T02:05:12.747 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-05-07T02:05:12.747 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-05-07T02:05:12.747 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-05-07T02:05:12.747 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-05-07T02:05:12.747 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-05-07T02:05:12.748 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-05-07T02:05:12.748 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-05-07T02:05:12.748 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-05-07T02:05:12.748 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-05-07T02:05:12.748 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-05-07T02:05:12.748 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-05-07T02:05:12.748 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-05-07T02:05:12.748 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-05-07T02:05:12.748 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-05-07T02:05:12.748 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-05-07T02:05:12.748 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-05-07T02:05:12.748 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-05-07T02:05:12.748 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-05-07T02:05:12.748 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-05-07T02:05:12.748 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-05-07T02:05:12.748 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-05-07T02:05:12.748 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-05-07T02:05:12.749 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-05-07T02:05:12.749 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-05-07T02:05:12.749 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-05-07T02:05:12.749 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-07T02:05:12.749 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-07T02:05:12.749 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-05-07T02:05:12.749 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-05-07T02:05:12.749 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-05-07T02:05:12.749 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-05-07T02:05:12.749 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-05-07T02:05:12.749 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-05-07T02:05:12.749 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-05-07T02:05:12.749 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-05-07T02:05:12.749 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-05-07T02:05:12.749 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-05-07T02:05:12.750 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-05-07T02:05:12.750 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-07T02:05:12.750 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-05-07T02:05:12.750 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-05-07T02:05:12.750 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-05-07T02:05:12.750 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-05-07T02:05:12.750 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-05-07T02:05:12.750 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-05-07T02:05:12.750 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-05-07T02:05:12.750 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-07T02:05:12.750 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-05-07T02:05:12.750 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-05-07T02:05:12.750 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-05-07T02:05:12.750 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-05-07T02:05:12.751 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-05-07T02:05:12.751 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-05-07T02:05:12.751 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-07T02:05:12.751 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-07T02:05:12.751 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-07T02:05:12.751 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-05-07T02:05:12.751 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-07T02:05:12.751 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-05-07T02:05:12.751 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-07T02:05:12.751 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-07T02:05:12.751 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-07T02:05:12.751 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-07T02:05:12.751 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-07T02:05:12.751 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-07T02:05:12.751 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-05-07T02:05:12.752 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-05-07T02:05:12.752 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-05-07T02:05:12.752 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-05-07T02:05:12.752 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-05-07T02:05:12.752 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-05-07T02:05:12.752 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-05-07T02:05:12.752 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-07T02:05:12.752 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-07T02:05:12.752 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-07T02:05:12.752 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-07T02:05:12.752 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-07T02:05:12.752 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-05-07T02:05:12.752 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-05-07T02:05:12.752 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-05-07T02:05:12.752 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-07T02:05:12.752 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-07T02:05:12.753 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-07T02:05:12.753 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-07T02:05:12.753 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-07T02:05:12.753 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-07T02:05:12.753 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-05-07T02:05:12.753 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-05-07T02:05:12.753 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-07T02:05:12.753 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-05-07T02:05:12.753 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-05-07T02:05:12.753 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-05-07T02:05:12.753 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-05-07T02:05:12.753 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-07T02:05:12.753 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-07T02:05:12.753 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-07T02:05:12.753 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-05-07T02:05:12.753 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-05-07T02:05:12.753 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-05-07T02:05:12.754 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-07T02:05:12.754 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-05-07T02:05:12.754 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-05-07T02:05:12.754 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-05-07T02:05:12.754 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-07T02:05:12.754 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-05-07T02:05:12.754 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-05-07T02:05:12.754 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-07T02:05:12.754 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-05-07T02:05:12.754 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-07T02:05:12.754 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-05-07T02:05:12.754 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-05-07T02:05:12.754 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-05-07T02:05:12.754 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-05-07T02:05:12.754 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-05-07T02:05:12.755 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-05-07T02:05:12.755 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-05-07T02:05:12.755 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-05-07T02:05:12.755 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-05-07T02:05:12.755 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-05-07T02:05:12.755 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-05-07T02:05:12.755 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-05-07T02:05:12.755 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-05-07T02:05:12.755 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-05-07T02:05:12.755 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-05-07T02:05:12.755 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-05-07T02:05:12.755 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-07T02:05:12.755 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-05-07T02:05:12.755 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-05-07T02:05:12.755 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-07T02:05:12.755 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-05-07T02:05:12.755 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-05-07T02:05:12.756 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-05-07T02:05:12.756 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-05-07T02:05:12.756 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-05-07T02:05:12.756 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-05-07T02:05:12.756 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-05-07T02:05:12.756 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-05-07T02:05:12.756 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-05-07T02:05:12.756 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-05-07T02:05:12.756 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-05-07T02:05:12.756 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-07T02:05:12.756 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-07T02:05:12.756 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-05-07T02:05:12.756 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-05-07T02:05:12.756 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-05-07T02:05:12.757 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-05-07T02:05:12.757 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-05-07T02:05:12.757 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-05-07T02:05:12.757 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-05-07T02:05:12.757 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-05-07T02:05:12.757 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-07T02:05:12.757 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-07T02:05:12.757 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-05-07T02:05:12.757 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-05-07T02:05:12.757 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-05-07T02:05:12.757 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-05-07T02:05:12.757 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-05-07T02:05:12.757 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-05-07T02:05:12.757 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-05-07T02:05:12.757 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-05-07T02:05:12.758 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-05-07T02:05:12.758 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-07T02:05:12.758 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-05-07T02:05:12.758 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-07T02:05:12.758 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-07T02:05:12.758 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-07T02:05:12.758 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-07T02:05:12.758 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-07T02:05:12.758 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-07T02:05:12.758 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-07T02:05:12.758 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-07T02:05:12.758 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-07T02:05:12.758 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-05-07T02:05:12.758 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-05-07T02:05:12.758 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-05-07T02:05:12.758 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-07T02:05:12.759 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-05-07T02:05:12.759 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-05-07T02:05:12.759 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-05-07T02:05:12.759 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-05-07T02:05:12.759 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-05-07T02:05:12.759 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-07T02:05:12.759 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-05-07T02:05:12.759 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-05-07T02:05:12.759 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-05-07T02:05:12.759 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-05-07T02:05:12.759 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-05-07T02:05:12.759 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-05-07T02:05:12.759 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-05-07T02:05:12.759 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-05-07T02:05:12.759 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-05-07T02:05:12.759 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-05-07T02:05:12.759 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-05-07T02:05:12.760 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-05-07T02:05:12.760 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-05-07T02:05:12.760 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-05-07T02:05:12.760 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-05-07T02:05:12.760 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-05-07T02:05:12.760 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-05-07T02:05:12.760 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-05-07T02:05:12.760 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-05-07T02:05:12.760 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-07T02:05:12.760 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-05-07T02:05:12.760 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-05-07T02:05:12.760 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-05-07T02:05:12.760 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-05-07T02:05:12.760 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-05-07T02:05:12.760 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-05-07T02:05:12.760 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-05-07T02:05:12.760 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-05-07T02:05:12.761 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-05-07T02:05:12.761 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-05-07T02:05:12.761 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-05-07T02:05:12.761 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-05-07T02:05:12.761 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-05-07T02:05:12.761 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-05-07T02:05:12.761 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-05-07T02:05:12.761 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-05-07T02:05:12.761 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-05-07T02:05:12.761 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-05-07T02:05:12.761 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-05-07T02:05:12.761 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-07T02:05:12.761 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-05-07T02:05:12.761 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-05-07T02:05:12.761 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-07T02:05:12.762 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-05-07T02:05:12.762 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-05-07T02:05:12.762 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-05-07T02:05:12.762 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-05-07T02:05:12.762 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-05-07T02:05:12.762 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-05-07T02:05:12.762 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-05-07T02:05:12.762 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-05-07T02:05:12.762 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-05-07T02:05:12.762 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-05-07T02:05:12.762 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-05-07T02:05:12.762 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-05-07T02:05:12.762 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-05-07T02:05:12.762 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-05-07T02:05:12.762 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-07T02:05:12.763 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-05-07T02:05:12.763 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-05-07T02:05:12.763 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-05-07T02:05:12.763 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-05-07T02:05:12.763 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-05-07T02:05:12.763 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-05-07T02:05:12.763 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-05-07T02:05:12.763 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-05-07T02:05:12.763 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-05-07T02:05:12.763 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-05-07T02:05:12.763 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-05-07T02:05:12.763 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-05-07T02:05:12.763 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-05-07T02:05:12.763 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-05-07T02:05:12.763 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-05-07T02:05:12.763 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-05-07T02:05:12.763 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-05-07T02:05:12.764 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-05-07T02:05:12.764 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-05-07T02:05:12.764 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-05-07T02:05:12.764 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-05-07T02:05:12.764 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-05-07T02:05:12.764 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-05-07T02:05:12.764 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-05-07T02:05:12.764 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-05-07T02:05:12.764 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-05-07T02:05:12.764 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-05-07T02:05:12.764 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-05-07T02:05:12.764 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-05-07T02:05:12.764 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-05-07T02:05:12.764 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-05-07T02:05:12.764 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-05-07T02:05:12.764 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-05-07T02:05:12.765 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-05-07T02:05:12.765 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-05-07T02:05:12.765 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-05-07T02:05:12.765 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-05-07T02:05:12.765 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-05-07T02:05:12.765 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-05-07T02:05:12.765 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-05-07T02:05:12.765 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-05-07T02:05:12.765 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-05-07T02:05:12.765 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-05-07T02:05:12.765 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-05-07T02:05:12.765 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-05-07T02:05:12.765 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-05-07T02:05:12.765 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-05-07T02:05:12.765 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-05-07T02:05:12.765 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-07T02:05:12.765 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-07T02:05:12.766 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-07T02:05:12.766 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-07T02:05:12.766 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-07T02:05:12.766 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-07T02:05:12.766 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-07T02:05:12.766 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-07T02:05:12.766 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-07T02:05:12.766 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-07T02:05:12.766 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-07T02:05:12.766 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-07T02:05:12.766 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-05-07T02:05:12.766 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-05-07T02:05:12.766 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-05-07T02:05:12.766 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-07T02:05:12.767 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-05-07T02:05:12.767 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-05-07T02:05:12.767 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-05-07T02:05:12.767 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-05-07T02:05:12.767 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-05-07T02:05:12.767 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-05-07T02:05:12.767 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-05-07T02:05:12.767 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-05-07T02:05:12.767 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-05-07T02:05:12.767 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-05-07T02:05:12.767 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-05-07T02:05:12.767 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-05-07T02:05:12.767 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-05-07T02:05:12.767 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-05-07T02:05:12.767 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-05-07T02:05:12.767 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-05-07T02:05:12.767 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-05-07T02:05:12.768 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-05-07T02:05:12.768 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-05-07T02:05:12.768 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-05-07T02:05:12.768 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-05-07T02:05:12.768 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-05-07T02:05:12.768 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-05-07T02:05:12.768 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-05-07T02:05:12.768 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-05-07T02:05:12.768 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-05-07T02:05:12.768 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-05-07T02:05:12.768 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-07T02:05:12.768 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-07T02:05:12.768 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-05-07T02:05:12.768 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-07T02:05:12.768 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-07T02:05:12.768 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-07T02:05:12.769 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-07T02:05:12.769 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-07T02:05:12.769 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-07T02:05:12.769 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-07T02:05:12.769 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-07T02:05:12.769 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-07T02:05:12.769 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-07T02:05:12.769 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-07T02:05:12.769 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-07T02:05:12.769 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-07T02:05:12.769 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-07T02:05:12.769 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-07T02:05:12.769 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-07T02:05:12.769 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-07T02:05:12.769 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-07T02:05:12.770 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-07T02:05:12.770 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-07T02:05:12.770 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-07T02:05:12.770 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-07T02:05:12.770 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-07T02:05:12.770 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-05-07T02:05:12.770 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-05-07T02:05:12.770 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-07T02:05:12.770 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-05-07T02:05:12.770 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-05-07T02:05:12.770 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-05-07T02:05:12.770 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-05-07T02:05:12.770 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-07T02:05:12.770 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-05-07T02:05:12.770 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-07T02:05:12.771 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-07T02:05:12.771 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-07T02:05:12.771 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-07T02:05:12.771 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-07T02:05:12.771 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-05-07T02:05:12.771 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-05-07T02:05:12.771 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-07T02:05:12.771 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-05-07T02:05:12.771 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-07T02:05:12.771 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-05-07T02:05:12.771 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-07T02:05:12.771 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-05-07T02:05:12.771 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-07T02:05:12.771 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-05-07T02:05:12.771 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-05-07T02:05:12.771 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-07T02:05:12.772 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-05-07T02:05:12.772 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-07T02:05:12.772 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-05-07T02:05:12.772 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-07T02:05:12.772 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-05-07T02:05:12.772 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-07T02:05:12.772 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-05-07T02:05:12.772 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-07T02:05:12.772 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-05-07T02:05:12.772 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-05-07T02:05:12.772 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-07T02:05:12.772 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-07T02:05:12.772 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-05-07T02:05:12.772 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-05-07T02:05:12.772 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-05-07T02:05:12.772 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-07T02:05:12.773 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-05-07T02:05:12.773 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-05-07T02:05:12.773 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-05-07T02:05:12.773 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-05-07T02:05:12.773 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-07T02:05:12.773 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-05-07T02:05:12.773 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-07T02:05:12.773 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-07T02:05:12.773 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-07T02:05:12.773 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-05-07T02:05:12.773 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-07T02:05:12.773 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-05-07T02:05:12.773 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-05-07T02:05:12.773 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-05-07T02:05:12.773 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-07T02:05:12.773 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-05-07T02:05:12.774 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-05-07T02:05:12.774 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-05-07T02:05:12.774 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-05-07T02:05:12.774 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-05-07T02:05:12.774 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-05-07T02:05:12.774 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-05-07T02:05:12.774 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-05-07T02:05:12.774 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-07T02:05:12.774 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-07T02:05:12.774 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-07T02:05:12.774 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-07T02:05:12.774 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-07T02:05:12.774 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-07T02:05:12.774 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-07T02:05:12.774 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-07T02:05:12.775 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-07T02:05:12.775 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-05-07T02:05:12.775 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-05-07T02:05:12.775 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-05-07T02:05:12.775 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-05-07T02:05:12.775 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-05-07T02:05:12.775 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-05-07T02:05:12.775 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-05-07T02:05:12.775 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-05-07T02:05:12.775 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-05-07T02:05:12.775 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-05-07T02:05:12.775 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-05-07T02:05:12.775 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-05-07T02:05:12.775 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-05-07T02:05:12.776 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-05-07T02:05:12.776 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-05-07T02:05:12.776 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-05-07T02:05:12.776 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-05-07T02:05:12.776 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-05-07T02:05:12.776 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-05-07T02:05:12.776 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-07T02:05:12.776 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-05-07T02:05:12.776 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-05-07T02:05:12.776 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-05-07T02:05:12.776 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-05-07T02:05:12.776 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-05-07T02:05:12.776 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-05-07T02:05:12.776 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-05-07T02:05:12.776 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-05-07T02:05:12.776 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-05-07T02:05:12.777 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-05-07T02:05:12.777 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-05-07T02:05:12.777 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-05-07T02:05:12.777 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-05-07T02:05:12.777 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-05-07T02:05:12.777 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-05-07T02:05:12.777 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-05-07T02:05:12.777 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-05-07T02:05:12.777 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-05-07T02:05:12.777 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-05-07T02:05:12.777 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-05-07T02:05:12.777 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-05-07T02:05:12.777 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-05-07T02:05:12.777 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-05-07T02:05:12.777 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-07T02:05:12.778 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-05-07T02:05:12.778 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-05-07T02:05:12.778 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-05-07T02:05:12.778 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-05-07T02:05:12.778 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-05-07T02:05:12.778 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-05-07T02:05:12.778 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-05-07T02:05:12.778 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-07T02:05:12.778 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-05-07T02:05:12.778 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-05-07T02:05:12.778 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-05-07T02:05:12.778 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-05-07T02:05:12.778 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-05-07T02:05:12.778 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-05-07T02:05:12.778 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-07T02:05:12.778 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-07T02:05:12.779 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-05-07T02:05:12.779 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-05-07T02:05:12.779 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-05-07T02:05:12.779 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-05-07T02:05:12.779 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-05-07T02:05:12.779 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-07T02:05:12.779 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-05-07T02:05:12.779 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-07T02:05:12.779 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-07T02:05:12.779 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-07T02:05:12.779 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-05-07T02:05:12.779 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-05-07T02:05:12.779 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-05-07T02:05:12.779 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-05-07T02:05:12.779 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-05-07T02:05:12.780 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-07T02:05:12.780 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-05-07T02:05:12.780 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-05-07T02:05:12.780 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-05-07T02:05:12.780 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-05-07T02:05:12.780 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-05-07T02:05:12.780 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-07T02:05:12.780 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-07T02:05:12.780 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-07T02:05:12.780 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-05-07T02:05:12.780 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-07T02:05:12.780 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-05-07T02:05:12.780 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-05-07T02:05:12.780 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-05-07T02:05:12.780 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-05-07T02:05:12.780 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-07T02:05:12.780 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-07T02:05:12.781 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-07T02:05:12.781 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-07T02:05:12.781 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-05-07T02:05:12.781 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-07T02:05:12.781 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-07T02:05:12.781 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-07T02:05:12.781 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-07T02:05:12.781 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-05-07T02:05:12.781 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-07T02:05:12.781 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-05-07T02:05:12.781 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-07T02:05:12.781 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-05-07T02:05:12.781 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-07T02:05:12.781 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-05-07T02:05:12.781 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-07T02:05:12.781 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-05-07T02:05:12.782 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-07T02:05:12.782 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-07T02:05:12.782 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-07T02:05:12.782 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-05-07T02:05:12.782 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-05-07T02:05:12.782 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-07T02:05:12.782 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-07T02:05:12.782 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-07T02:05:12.782 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-07T02:05:12.782 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-07T02:05:12.782 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-05-07T02:05:12.782 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-07T02:05:12.782 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-07T02:05:12.782 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-05-07T02:05:12.782 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-07T02:05:12.783 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-07T02:05:12.783 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-07T02:05:12.783 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-07T02:05:12.783 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-07T02:05:12.783 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-05-07T02:05:12.783 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-07T02:05:12.783 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-07T02:05:12.783 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-05-07T02:05:12.783 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-05-07T02:05:12.783 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-07T02:05:12.783 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-05-07T02:05:12.783 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-07T02:05:12.783 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-07T02:05:12.784 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-07T02:05:12.784 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-07T02:05:12.784 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-07T02:05:12.784 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-07T02:05:12.784 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-07T02:05:12.784 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-07T02:05:12.784 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-07T02:05:12.784 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-07T02:05:12.784 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-05-07T02:05:12.784 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-05-07T02:05:12.784 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-07T02:05:12.784 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-05-07T02:05:12.784 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-05-07T02:05:12.785 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-07T02:05:12.785 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-07T02:05:12.785 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-07T02:05:12.785 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-07T02:05:12.785 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-07T02:05:12.785 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-07T02:05:12.785 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-05-07T02:05:12.785 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-07T02:05:12.785 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-07T02:05:12.785 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-05-07T02:05:12.785 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-05-07T02:05:12.785 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-05-07T02:05:12.785 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-07T02:05:12.786 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-05-07T02:05:12.786 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-05-07T02:05:12.786 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-05-07T02:05:12.786 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-07T02:05:12.786 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-05-07T02:05:12.786 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-07T02:05:12.786 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-05-07T02:05:12.786 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-07T02:05:12.786 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-05-07T02:05:12.786 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-07T02:05:12.786 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-05-07T02:05:12.786 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-07T02:05:12.786 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-05-07T02:05:12.786 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-07T02:05:12.786 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-05-07T02:05:12.787 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-07T02:05:12.787 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-05-07T02:05:12.787 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-07T02:05:12.787 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-07T02:05:12.787 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-07T02:05:12.787 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-07T02:05:12.787 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-07T02:05:12.787 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-05-07T02:05:12.787 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-05-07T02:05:12.787 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-07T02:05:12.787 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-07T02:05:12.787 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-07T02:05:12.787 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-05-07T02:05:12.787 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-07T02:05:12.788 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-07T02:05:12.788 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-07T02:05:12.788 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-07T02:05:12.788 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-07T02:05:12.788 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-07T02:05:12.788 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-05-07T02:05:12.788 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-07T02:05:12.788 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-05-07T02:05:12.788 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-07T02:05:12.788 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-07T02:05:12.788 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-05-07T02:05:12.788 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-07T02:05:12.788 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-07T02:05:12.788 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-05-07T02:05:12.788 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-07T02:05:12.789 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-05-07T02:05:12.789 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-07T02:05:12.789 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-05-07T02:05:12.789 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-07T02:05:12.789 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-07T02:05:12.789 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-05-07T02:05:12.789 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-07T02:05:12.789 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-07T02:05:12.789 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-07T02:05:12.789 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-07T02:05:12.789 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-07T02:05:12.789 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-05-07T02:05:12.789 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-07T02:05:12.789 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-05-07T02:05:12.789 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-07T02:05:12.790 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-07T02:05:12.790 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-07T02:05:12.790 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-07T02:05:12.790 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-05-07T02:05:12.790 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-07T02:05:12.790 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-05-07T02:05:12.790 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-07T02:05:12.790 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-05-07T02:05:12.790 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-07T02:05:12.790 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-05-07T02:05:12.790 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-07T02:05:12.790 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-07T02:05:12.790 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-05-07T02:05:12.791 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-07T02:05:12.791 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-07T02:05:12.791 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-07T02:05:12.791 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-07T02:05:12.791 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-05-07T02:05:12.791 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-07T02:05:12.791 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-07T02:05:12.791 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-07T02:05:12.791 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-05-07T02:05:12.791 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-07T02:05:12.791 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-07T02:05:12.791 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-07T02:05:12.791 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-07T02:05:12.792 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-05-07T02:05:12.792 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-07T02:05:12.792 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-05-07T02:05:12.792 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-07T02:05:12.792 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-05-07T02:05:12.792 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-07T02:05:12.792 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-07T02:05:12.792 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-05-07T02:05:12.792 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-07T02:05:12.792 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-07T02:05:12.792 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-05-07T02:05:12.792 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-07T02:05:12.792 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-05-07T02:05:12.792 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-05-07T02:05:12.793 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-07T02:05:12.793 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-05-07T02:05:12.793 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-07T02:05:12.793 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-05-07T02:05:12.793 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-05-07T02:05:12.793 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-07T02:05:12.793 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-05-07T02:05:12.793 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-05-07T02:05:12.793 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-05-07T02:05:12.793 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-05-07T02:05:12.793 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-07T02:05:12.793 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-07T02:05:12.793 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-07T02:05:12.793 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-05-07T02:05:12.794 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-05-07T02:05:12.794 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-05-07T02:05:12.794 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-05-07T02:05:12.794 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-05-07T02:05:12.794 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-07T02:05:12.794 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-07T02:05:12.794 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-05-07T02:05:12.794 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-05-07T02:05:12.794 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-07T02:05:12.794 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-07T02:05:12.794 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-07T02:05:12.794 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-05-07T02:05:12.794 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-05-07T02:05:12.795 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-05-07T02:05:12.795 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-05-07T02:05:12.795 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-05-07T02:05:12.795 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-05-07T02:05:12.795 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-05-07T02:05:12.795 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-05-07T02:05:12.795 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-05-07T02:05:12.795 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-05-07T02:05:12.795 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-05-07T02:05:12.795 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-05-07T02:05:12.795 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-05-07T02:05:12.795 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-05-07T02:05:12.795 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-05-07T02:05:12.795 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-05-07T02:05:12.795 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-05-07T02:05:12.796 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-05-07T02:05:12.796 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-07T02:05:12.796 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-05-07T02:05:12.796 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-07T02:05:12.796 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-07T02:05:12.796 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-07T02:05:12.796 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-07T02:05:12.796 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-05-07T02:05:12.796 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-05-07T02:05:12.796 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-05-07T02:05:12.796 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-05-07T02:05:12.796 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-05-07T02:05:12.796 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-05-07T02:05:12.796 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-05-07T02:05:12.796 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-05-07T02:05:12.797 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-05-07T02:05:12.797 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-05-07T02:05:12.797 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-07T02:05:12.797 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-07T02:05:12.797 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-07T02:05:12.797 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-05-07T02:05:12.797 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-07T02:05:12.797 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-05-07T02:05:12.797 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-05-07T02:05:12.797 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-05-07T02:05:12.797 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-05-07T02:05:12.797 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-07T02:05:12.797 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-05-07T02:05:12.797 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-05-07T02:05:12.798 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-05-07T02:05:12.798 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-07T02:05:12.798 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-05-07T02:05:12.798 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-05-07T02:05:12.798 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-05-07T02:05:12.798 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-05-07T02:05:12.798 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-05-07T02:05:12.798 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-05-07T02:05:12.798 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-05-07T02:05:12.798 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-05-07T02:05:12.798 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-05-07T02:05:12.798 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-05-07T02:05:12.798 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-05-07T02:05:12.798 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-05-07T02:05:12.798 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-05-07T02:05:12.799 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-05-07T02:05:12.799 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-07T02:05:12.799 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-05-07T02:05:12.799 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-07T02:05:12.799 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-05-07T02:05:12.799 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-05-07T02:05:12.799 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-05-07T02:05:12.799 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-05-07T02:05:12.799 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-07T02:05:12.799 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-05-07T02:05:12.799 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-05-07T02:05:12.799 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-07T02:05:12.799 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-05-07T02:05:12.799 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-05-07T02:05:12.800 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-07T02:05:12.800 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-05-07T02:05:12.800 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-07T02:05:12.800 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-07T02:05:12.800 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-07T02:05:12.800 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-07T02:05:12.800 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-07T02:05:12.800 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-07T02:05:12.800 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-07T02:05:12.800 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-07T02:05:12.800 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-07T02:05:12.800 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-05-07T02:05:12.800 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-05-07T02:05:12.800 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-07T02:05:12.800 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-05-07T02:05:12.801 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-07T02:05:12.801 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-07T02:05:12.801 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-07T02:05:12.801 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-07T02:05:12.801 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-07T02:05:12.801 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-07T02:05:12.801 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-07T02:05:12.801 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-07T02:05:12.801 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-07T02:05:12.801 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-05-07T02:05:12.801 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-05-07T02:05:12.801 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-07T02:05:12.801 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-07T02:05:12.801 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-05-07T02:05:12.801 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-05-07T02:05:12.801 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-07T02:05:12.802 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-07T02:05:12.802 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-05-07T02:05:12.802 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-05-07T02:05:12.802 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-05-07T02:05:12.802 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-05-07T02:05:12.802 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-05-07T02:05:12.802 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-05-07T02:05:12.802 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-05-07T02:05:12.802 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-05-07T02:05:12.802 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-07T02:05:12.802 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-05-07T02:05:12.802 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-05-07T02:05:12.802 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-07T02:05:12.802 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-07T02:05:12.802 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-07T02:05:12.802 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-07T02:05:12.803 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-07T02:05:12.803 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-07T02:05:12.803 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-07T02:05:12.803 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-07T02:05:12.803 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-07T02:05:12.803 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-07T02:05:12.803 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-05-07T02:05:12.803 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-07T02:05:12.803 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-07T02:05:12.803 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-07T02:05:12.803 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-07T02:05:12.803 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-07T02:05:12.803 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-07T02:05:12.803 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-05-07T02:05:12.803 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-07T02:05:12.804 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-07T02:05:12.804 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-07T02:05:12.804 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-07T02:05:12.804 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-07T02:05:12.804 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-07T02:05:12.804 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-07T02:05:12.804 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-07T02:05:12.804 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-07T02:05:12.804 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-07T02:05:12.804 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-07T02:05:12.804 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-07T02:05:12.804 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-05-07T02:05:12.804 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-05-07T02:05:12.804 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-05-07T02:05:12.804 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-05-07T02:05:12.805 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-05-07T02:05:12.805 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-05-07T02:05:12.805 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-05-07T02:05:12.805 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-05-07T02:05:12.805 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-07T02:05:12.805 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-05-07T02:05:12.805 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-05-07T02:05:12.805 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-07T02:05:12.805 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-07T02:05:12.805 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-05-07T02:05:12.805 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-05-07T02:05:12.805 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-07T02:05:12.805 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-05-07T02:05:12.805 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-05-07T02:05:12.805 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-07T02:05:12.805 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-05-07T02:05:12.806 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-05-07T02:05:12.806 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-05-07T02:05:12.806 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-07T02:05:12.806 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-05-07T02:05:12.806 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-07T02:05:12.806 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-05-07T02:05:12.806 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-05-07T02:05:12.806 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-07T02:05:12.806 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-05-07T02:05:12.806 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-07T02:05:12.806 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-07T02:05:12.806 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-07T02:05:12.806 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-07T02:05:12.806 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-07T02:05:12.807 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-05-07T02:05:12.807 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-05-07T02:05:12.807 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-07T02:05:12.807 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-07T02:05:12.807 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-07T02:05:12.807 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-07T02:05:12.807 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-05-07T02:05:12.807 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-05-07T02:05:12.807 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-07T02:05:12.807 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-05-07T02:05:12.807 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-05-07T02:05:12.807 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-07T02:05:12.807 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-07T02:05:12.807 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-05-07T02:05:12.807 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-05-07T02:05:12.808 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-05-07T02:05:12.808 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-05-07T02:05:12.808 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-05-07T02:05:12.808 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-05-07T02:05:12.808 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-05-07T02:05:12.808 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-05-07T02:05:12.808 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-05-07T02:05:12.808 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-05-07T02:05:12.808 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-07T02:05:12.808 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-07T02:05:12.808 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-05-07T02:05:12.808 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-05-07T02:05:12.808 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-05-07T02:05:12.808 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-07T02:05:12.808 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-05-07T02:05:12.808 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-05-07T02:05:12.809 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-05-07T02:05:12.809 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-07T02:05:12.809 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-05-07T02:05:12.809 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-05-07T02:05:12.809 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-07T02:05:12.809 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-07T02:05:12.809 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-07T02:05:12.809 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-05-07T02:05:12.809 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-07T02:05:12.809 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-05-07T02:05:12.809 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-07T02:05:12.809 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-05-07T02:05:12.809 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-05-07T02:05:12.809 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-07T02:05:12.809 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-05-07T02:05:12.810 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-05-07T02:05:12.810 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-05-07T02:05:12.810 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-07T02:05:12.810 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-07T02:05:12.810 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-05-07T02:05:12.810 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-07T02:05:12.810 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-05-07T02:05:12.810 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-05-07T02:05:12.810 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-07T02:05:12.810 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-07T02:05:12.810 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-07T02:05:12.810 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-07T02:05:12.810 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-07T02:05:12.810 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-05-07T02:05:12.810 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-05-07T02:05:12.811 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-05-07T02:05:12.811 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-05-07T02:05:12.811 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-07T02:05:12.811 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-05-07T02:05:12.811 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-07T02:05:12.811 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-05-07T02:05:12.811 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-05-07T02:05:12.811 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-05-07T02:05:12.811 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-05-07T02:05:12.811 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-05-07T02:05:12.811 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-07T02:05:12.811 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-05-07T02:05:12.811 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-07T02:05:12.811 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-05-07T02:05:12.811 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-05-07T02:05:12.811 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-05-07T02:05:12.811 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-05-07T02:05:12.812 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-05-07T02:05:12.812 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-05-07T02:05:12.812 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-07T02:05:12.812 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-05-07T02:05:12.812 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-05-07T02:05:12.812 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-05-07T02:05:12.812 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-07T02:05:12.812 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-07T02:05:12.812 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-07T02:05:12.812 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-07T02:05:12.812 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-07T02:05:12.812 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-07T02:05:12.812 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-07T02:05:12.812 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-07T02:05:12.812 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-07T02:05:12.813 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-07T02:05:12.813 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-07T02:05:12.813 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-07T02:05:12.813 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-07T02:05:12.813 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-07T02:05:12.813 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-07T02:05:12.813 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-07T02:05:12.813 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-07T02:05:12.813 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-07T02:05:12.813 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-05-07T02:05:12.813 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-05-07T02:05:12.813 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-07T02:05:12.813 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-05-07T02:05:12.813 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-05-07T02:05:12.813 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-05-07T02:05:12.814 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-05-07T02:05:12.814 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-05-07T02:05:12.814 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-07T02:05:12.814 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-05-07T02:05:12.814 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-05-07T02:05:12.814 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-07T02:05:12.814 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-05-07T02:05:12.814 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-05-07T02:05:12.814 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-07T02:05:12.814 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-05-07T02:05:12.814 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-07T02:05:12.814 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-05-07T02:05:12.814 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-05-07T02:05:12.814 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-05-07T02:05:12.814 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-05-07T02:05:12.815 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-05-07T02:05:12.815 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-05-07T02:05:12.815 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-05-07T02:05:12.815 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-05-07T02:05:12.815 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-05-07T02:05:12.815 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-07T02:05:12.815 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-05-07T02:05:12.815 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-05-07T02:05:12.815 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-05-07T02:05:12.815 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-07T02:05:12.815 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-05-07T02:05:12.815 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-05-07T02:05:12.815 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-05-07T02:05:12.815 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-05-07T02:05:12.816 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-05-07T02:05:12.816 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-05-07T02:05:12.816 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-05-07T02:05:12.816 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-05-07T02:05:12.816 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-05-07T02:05:12.816 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-05-07T02:05:12.816 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-05-07T02:05:12.816 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-07T02:05:12.816 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-05-07T02:05:12.816 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-05-07T02:05:12.816 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-05-07T02:05:12.816 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-07T02:05:12.816 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-07T02:05:12.816 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-05-07T02:05:12.817 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-05-07T02:05:12.817 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-07T02:05:12.817 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-05-07T02:05:12.817 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-05-07T02:05:12.817 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-05-07T02:05:12.817 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-05-07T02:05:12.817 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-05-07T02:05:12.817 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-05-07T02:05:12.817 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-05-07T02:05:12.817 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-07T02:05:12.817 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-05-07T02:05:12.817 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-05-07T02:05:12.817 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-05-07T02:05:12.817 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-07T02:05:12.817 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-07T02:05:12.818 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-05-07T02:05:12.818 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-07T02:05:12.818 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-07T02:05:12.818 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-07T02:05:12.818 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-05-07T02:05:12.818 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-05-07T02:05:12.818 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-05-07T02:05:12.818 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-05-07T02:05:12.818 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-07T02:05:12.818 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-05-07T02:05:12.818 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-07T02:05:12.818 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-07T02:05:12.818 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-07T02:05:12.818 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-05-07T02:05:12.818 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-05-07T02:05:12.819 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-05-07T02:05:12.819 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-05-07T02:05:12.819 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-05-07T02:05:12.819 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-05-07T02:05:12.819 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-05-07T02:05:12.819 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-05-07T02:05:12.819 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-05-07T02:05:12.819 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-05-07T02:05:12.819 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-05-07T02:05:12.819 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-05-07T02:05:12.819 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-05-07T02:05:12.819 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-05-07T02:05:12.819 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-07T02:05:12.819 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-05-07T02:05:12.819 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-05-07T02:05:12.820 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-07T02:05:12.820 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-07T02:05:12.820 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-05-07T02:05:12.820 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-07T02:05:12.820 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-05-07T02:05:12.820 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-05-07T02:05:12.820 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-07T02:05:12.820 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-05-07T02:05:12.820 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-05-07T02:05:12.820 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-07T02:05:12.820 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-07T02:05:12.820 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-07T02:05:12.820 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-07T02:05:12.820 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-07T02:05:12.820 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-07T02:05:12.821 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-07T02:05:12.821 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-05-07T02:05:12.821 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-07T02:05:12.821 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-07T02:05:12.821 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-07T02:05:12.821 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-07T02:05:12.821 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-07T02:05:12.821 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-05-07T02:05:12.821 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-05-07T02:05:12.821 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-07T02:05:12.821 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-07T02:05:12.821 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-07T02:05:12.821 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-05-07T02:05:12.821 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-05-07T02:05:12.821 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-07T02:05:12.822 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-07T02:05:12.822 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-05-07T02:05:12.822 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-07T02:05:12.822 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-05-07T02:05:12.822 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-05-07T02:05:12.822 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-07T02:05:12.822 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-05-07T02:05:12.822 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-05-07T02:05:12.822 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-05-07T02:05:12.822 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-05-07T02:05:12.822 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-05-07T02:05:12.822 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-05-07T02:05:12.822 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-07T02:05:12.822 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-05-07T02:05:12.823 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-05-07T02:05:12.823 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-05-07T02:05:12.823 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-05-07T02:05:12.823 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-05-07T02:05:12.823 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-07T02:05:12.823 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-07T02:05:12.823 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-05-07T02:05:12.823 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-05-07T02:05:12.823 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-05-07T02:05:12.823 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-05-07T02:05:12.823 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-05-07T02:05:12.823 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-05-07T02:05:12.823 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-07T02:05:12.823 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-07T02:05:12.824 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-07T02:05:12.824 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-07T02:05:12.824 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-07T02:05:12.824 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-07T02:05:12.824 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-07T02:05:12.824 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-07T02:05:12.824 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-07T02:05:12.824 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-07T02:05:12.824 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-07T02:05:12.824 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-05-07T02:05:12.824 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-05-07T02:05:12.824 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-07T02:05:12.824 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-07T02:05:12.824 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-07T02:05:12.824 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-05-07T02:05:12.825 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-05-07T02:05:12.825 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-07T02:05:12.825 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-07T02:05:12.825 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-05-07T02:05:12.825 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-05-07T02:05:12.825 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-07T02:05:12.825 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-07T02:05:12.825 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-05-07T02:05:12.825 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-05-07T02:05:12.825 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-05-07T02:05:12.825 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-07T02:05:12.825 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-07T02:05:12.825 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-07T02:05:12.825 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-07T02:05:12.826 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-05-07T02:05:12.826 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-07T02:05:12.826 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-05-07T02:05:12.826 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-07T02:05:12.826 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-07T02:05:12.826 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-07T02:05:12.826 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-05-07T02:05:12.826 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-05-07T02:05:12.826 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-05-07T02:05:12.826 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-07T02:05:12.826 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-05-07T02:05:12.826 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-05-07T02:05:12.826 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-05-07T02:05:12.826 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-05-07T02:05:12.826 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-07T02:05:12.826 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-05-07T02:05:12.827 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-05-07T02:05:12.827 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-07T02:05:12.827 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-05-07T02:05:12.827 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-05-07T02:05:12.827 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-05-07T02:05:12.827 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-05-07T02:05:12.827 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-05-07T02:05:12.827 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-05-07T02:05:12.827 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-05-07T02:05:12.827 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-05-07T02:05:12.827 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-05-07T02:05:12.827 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-05-07T02:05:12.827 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-05-07T02:05:12.827 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-05-07T02:05:12.827 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-05-07T02:05:12.828 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-05-07T02:05:12.828 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-05-07T02:05:12.828 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-05-07T02:05:12.828 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-05-07T02:05:12.828 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-05-07T02:05:12.828 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-05-07T02:05:12.828 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-05-07T02:05:12.828 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-05-07T02:05:12.828 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-07T02:05:12.828 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-05-07T02:05:12.828 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-05-07T02:05:12.828 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-05-07T02:05:12.828 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-05-07T02:05:12.828 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-05-07T02:05:12.828 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-05-07T02:05:12.828 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-05-07T02:05:12.828 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-05-07T02:05:12.829 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-05-07T02:05:12.829 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-05-07T02:05:12.829 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-05-07T02:05:12.829 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-05-07T02:05:12.829 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-05-07T02:05:12.829 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-05-07T02:05:12.829 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-05-07T02:05:12.829 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-05-07T02:05:12.829 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-05-07T02:05:12.829 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-05-07T02:05:12.829 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-05-07T02:05:12.829 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-05-07T02:05:12.829 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-05-07T02:05:12.829 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-05-07T02:05:12.829 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-05-07T02:05:12.829 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-05-07T02:05:12.830 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-05-07T02:05:12.830 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-05-07T02:05:12.830 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-05-07T02:05:12.830 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-07T02:05:12.830 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-05-07T02:05:12.830 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-07T02:05:12.830 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-05-07T02:05:12.830 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-07T02:05:12.830 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-05-07T02:05:12.830 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-05-07T02:05:12.830 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-05-07T02:05:12.830 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-05-07T02:05:12.830 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-05-07T02:05:12.830 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-05-07T02:05:12.830 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-05-07T02:05:12.831 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-05-07T02:05:12.831 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-05-07T02:05:12.831 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-05-07T02:05:12.831 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-05-07T02:05:12.831 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-05-07T02:05:12.831 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-05-07T02:05:12.831 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-05-07T02:05:12.831 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-05-07T02:05:12.831 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-05-07T02:05:12.831 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-05-07T02:05:12.831 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-05-07T02:05:12.831 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-05-07T02:05:12.831 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-05-07T02:05:12.831 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-05-07T02:05:12.832 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-05-07T02:05:12.832 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-05-07T02:05:12.832 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-05-07T02:05:12.832 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-05-07T02:05:12.832 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-05-07T02:05:12.832 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-05-07T02:05:12.832 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-05-07T02:05:12.832 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-05-07T02:05:12.832 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-05-07T02:05:12.832 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-05-07T02:05:12.832 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-05-07T02:05:12.832 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-05-07T02:05:12.832 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-05-07T02:05:12.832 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-05-07T02:05:12.832 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-05-07T02:05:12.833 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-05-07T02:05:12.833 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-05-07T02:05:12.833 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-05-07T02:05:12.833 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-05-07T02:05:12.833 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-05-07T02:05:12.833 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-05-07T02:05:12.833 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-05-07T02:05:12.833 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-05-07T02:05:12.833 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-05-07T02:05:12.833 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-05-07T02:05:12.833 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-05-07T02:05:12.833 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-05-07T02:05:12.833 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-05-07T02:05:12.833 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-05-07T02:05:12.833 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-05-07T02:05:12.834 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-05-07T02:05:12.834 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-05-07T02:05:12.834 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-05-07T02:05:12.834 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-05-07T02:05:12.834 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-05-07T02:05:12.834 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-05-07T02:05:12.834 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-05-07T02:05:12.834 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-05-07T02:05:12.834 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-07T02:05:12.834 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-05-07T02:05:12.834 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-05-07T02:05:12.834 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-05-07T02:05:12.834 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-07T02:05:12.834 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-05-07T02:05:12.834 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-07T02:05:12.835 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-05-07T02:05:12.835 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-07T02:05:12.835 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-05-07T02:05:12.835 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-07T02:05:12.835 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-07T02:05:12.835 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-07T02:05:12.835 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-05-07T02:05:12.835 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-05-07T02:05:12.835 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-05-07T02:05:12.835 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-05-07T02:05:12.835 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-05-07T02:05:12.835 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-05-07T02:05:12.835 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-05-07T02:05:12.835 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-05-07T02:05:12.835 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-05-07T02:05:12.836 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-05-07T02:05:12.836 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-05-07T02:05:12.836 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-05-07T02:05:12.836 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-05-07T02:05:12.836 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-05-07T02:05:12.836 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-05-07T02:05:12.836 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-05-07T02:05:12.836 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-05-07T02:05:12.836 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-05-07T02:05:12.836 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-05-07T02:05:12.836 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-05-07T02:05:12.836 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-05-07T02:05:12.836 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-05-07T02:05:12.836 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-05-07T02:05:12.836 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-05-07T02:05:12.837 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-05-07T02:05:12.837 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-05-07T02:05:12.837 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-05-07T02:05:12.837 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-05-07T02:05:12.837 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-05-07T02:05:12.837 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-07T02:05:12.837 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-05-07T02:05:12.837 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-05-07T02:05:12.837 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-05-07T02:05:12.837 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-05-07T02:05:12.837 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-05-07T02:05:12.837 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-05-07T02:05:12.837 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-05-07T02:05:12.837 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-05-07T02:05:12.837 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-05-07T02:05:12.838 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-05-07T02:05:12.838 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-05-07T02:05:12.838 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-05-07T02:05:12.838 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-05-07T02:05:12.838 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-05-07T02:05:12.838 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-05-07T02:05:12.838 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-05-07T02:05:12.838 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-05-07T02:05:12.838 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-07T02:05:12.838 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-05-07T02:05:12.838 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-05-07T02:05:12.838 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-05-07T02:05:12.838 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-05-07T02:05:12.838 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-05-07T02:05:12.839 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-05-07T02:05:12.839 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-05-07T02:05:12.839 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-05-07T02:05:12.839 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-05-07T02:05:12.839 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-05-07T02:05:12.839 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-05-07T02:05:12.839 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-05-07T02:05:12.839 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-05-07T02:05:12.839 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-07T02:05:12.839 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-05-07T02:05:12.839 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-05-07T02:05:12.839 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-05-07T02:05:12.839 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-05-07T02:05:12.839 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-05-07T02:05:12.839 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-05-07T02:05:12.840 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-05-07T02:05:12.840 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-05-07T02:05:12.840 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-05-07T02:05:12.840 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-05-07T02:05:12.840 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-05-07T02:05:12.840 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-05-07T02:05:12.840 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-05-07T02:05:12.840 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-05-07T02:05:12.840 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-05-07T02:05:12.840 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-05-07T02:05:12.840 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-05-07T02:05:12.840 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-05-07T02:05:12.840 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-05-07T02:05:12.840 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-05-07T02:05:12.840 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-05-07T02:05:12.841 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-05-07T02:05:12.841 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-07T02:05:12.841 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-07T02:05:12.841 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-07T02:05:12.841 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-07T02:05:12.841 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-07T02:05:12.841 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-07T02:05:12.841 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-07T02:05:12.841 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-07T02:05:12.841 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-07T02:05:12.841 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-07T02:05:12.841 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-07T02:05:12.841 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-07T02:05:12.842 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-07T02:05:12.842 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-07T02:05:12.842 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-07T02:05:12.842 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-07T02:05:12.842 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-07T02:05:12.842 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-07T02:05:12.842 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-07T02:05:12.842 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-07T02:05:12.842 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-05-07T02:05:12.842 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-07T02:05:12.842 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-07T02:05:12.842 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-07T02:05:12.842 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-05-07T02:05:12.842 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-05-07T02:05:12.842 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-05-07T02:05:12.843 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-05-07T02:05:12.843 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-05-07T02:05:12.843 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-05-07T02:05:12.843 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-05-07T02:05:12.843 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-05-07T02:05:12.843 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-05-07T02:05:12.843 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-05-07T02:05:12.843 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-05-07T02:05:12.843 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-05-07T02:05:12.843 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-05-07T02:05:12.843 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-05-07T02:05:12.843 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-05-07T02:05:12.843 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-05-07T02:05:12.843 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-05-07T02:05:12.843 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-05-07T02:05:12.844 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-05-07T02:05:12.844 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-05-07T02:05:12.844 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-07T02:05:12.844 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-05-07T02:05:12.844 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-07T02:05:12.844 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-05-07T02:05:12.844 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-05-07T02:05:12.844 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-05-07T02:05:12.844 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-05-07T02:05:12.844 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-05-07T02:05:12.844 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-05-07T02:05:12.844 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-05-07T02:05:12.844 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-05-07T02:05:12.844 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-05-07T02:05:12.844 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-05-07T02:05:12.845 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-05-07T02:05:12.845 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-05-07T02:05:12.845 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-07T02:05:12.845 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-07T02:05:12.845 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-07T02:05:12.845 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-07T02:05:12.845 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-07T02:05:12.845 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-07T02:05:12.845 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-07T02:05:12.845 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-07T02:05:12.845 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-07T02:05:12.845 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-07T02:05:12.845 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-07T02:05:12.845 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-07T02:05:12.845 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-07T02:05:12.845 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-07T02:05:12.846 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-07T02:05:12.846 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-07T02:05:12.846 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-07T02:05:12.846 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-07T02:05:12.846 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-07T02:05:12.846 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-07T02:05:12.846 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-07T02:05:12.846 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-05-07T02:05:12.846 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-05-07T02:05:12.846 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-05-07T02:05:12.846 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-05-07T02:05:12.846 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-05-07T02:05:12.846 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-05-07T02:05:12.846 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-05-07T02:05:12.846 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-05-07T02:05:12.847 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-05-07T02:05:12.847 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-05-07T02:05:12.847 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-05-07T02:05:12.847 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-05-07T02:05:12.847 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-05-07T02:05:12.847 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-05-07T02:05:12.847 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-05-07T02:05:12.847 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-05-07T02:05:12.847 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-07T02:05:12.847 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-05-07T02:05:12.847 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-05-07T02:05:12.847 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-05-07T02:05:12.847 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-05-07T02:05:12.847 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-05-07T02:05:12.847 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-07T02:05:12.847 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-05-07T02:05:12.847 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-05-07T02:05:12.847 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-05-07T02:05:12.848 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-07T02:05:12.848 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-07T02:05:12.848 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-07T02:05:12.848 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-07T02:05:12.848 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-07T02:05:12.848 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-07T02:05:12.848 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-07T02:05:12.848 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-07T02:05:12.848 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-07T02:05:12.848 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-07T02:05:12.848 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-07T02:05:12.848 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-07T02:05:12.848 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-07T02:05:12.848 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-07T02:05:12.848 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-07T02:05:12.848 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-07T02:05:12.849 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-07T02:05:12.849 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-07T02:05:12.849 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-07T02:05:12.849 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-07T02:05:12.849 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-07T02:05:12.849 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-07T02:05:12.849 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-07T02:05:12.849 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-07T02:05:12.849 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-05-07T02:05:12.849 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-05-07T02:05:12.849 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-07T02:05:12.849 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-05-07T02:05:12.849 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-05-07T02:05:12.849 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-05-07T02:05:12.849 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-05-07T02:05:12.850 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-07T02:05:12.850 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-05-07T02:05:12.850 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-05-07T02:05:12.850 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-05-07T02:05:12.850 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-05-07T02:05:12.850 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-05-07T02:05:12.850 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-05-07T02:05:12.850 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-05-07T02:05:12.850 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-05-07T02:05:12.850 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-05-07T02:05:12.850 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-05-07T02:05:12.850 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-05-07T02:05:12.850 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-05-07T02:05:12.851 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-05-07T02:05:12.851 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-05-07T02:05:12.851 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-07T02:05:12.851 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-07T02:05:12.851 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-07T02:05:12.851 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-07T02:05:12.851 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-07T02:05:12.851 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-07T02:05:12.851 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-07T02:05:12.851 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-07T02:05:12.851 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-07T02:05:12.851 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-07T02:05:12.851 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-07T02:05:12.851 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-07T02:05:12.851 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-07T02:05:12.852 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-07T02:05:12.852 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-07T02:05:12.852 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-05-07T02:05:12.852 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-07T02:05:12.852 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-07T02:05:12.852 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-07T02:05:12.852 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-07T02:05:12.852 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-07T02:05:12.852 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-07T02:05:12.852 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-07T02:05:12.852 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-07T02:05:12.852 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-07T02:05:12.852 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-07T02:05:12.852 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-07T02:05:12.852 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-07T02:05:12.853 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-07T02:05:12.853 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-07T02:05:12.853 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-05-07T02:05:12.853 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-05-07T02:05:12.853 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-07T02:05:12.853 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-07T02:05:12.853 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-07T02:05:12.853 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-07T02:05:12.853 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-07T02:05:12.853 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-07T02:05:12.853 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-07T02:05:12.853 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-07T02:05:12.853 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-07T02:05:12.853 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-07T02:05:12.854 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-07T02:05:12.854 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-07T02:05:12.854 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-07T02:05:12.854 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-05-07T02:05:12.854 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-07T02:05:12.854 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-05-07T02:05:12.854 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-05-07T02:05:12.854 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-07T02:05:12.854 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-07T02:05:12.854 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-05-07T02:05:12.854 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-05-07T02:05:12.854 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-07T02:05:12.854 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-07T02:05:12.854 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-05-07T02:05:12.855 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-05-07T02:05:12.855 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-07T02:05:12.855 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-05-07T02:05:12.855 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-07T02:05:12.855 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-07T02:05:12.855 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-05-07T02:05:12.855 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-05-07T02:05:12.855 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-05-07T02:05:12.855 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-07T02:05:12.855 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-05-07T02:05:12.855 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-07T02:05:12.855 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-05-07T02:05:12.855 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-05-07T02:05:12.855 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-05-07T02:05:12.855 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-05-07T02:05:12.856 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-05-07T02:05:12.856 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-05-07T02:05:12.856 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-05-07T02:05:12.856 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-05-07T02:05:12.856 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-05-07T02:05:12.856 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-05-07T02:05:12.856 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-05-07T02:05:12.856 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-05-07T02:05:12.856 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-05-07T02:05:12.856 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-05-07T02:05:12.856 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-07T02:05:12.856 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-07T02:05:12.856 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-07T02:05:12.856 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-05-07T02:05:12.857 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-05-07T02:05:12.857 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-05-07T02:05:12.857 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-05-07T02:05:12.857 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-07T02:05:12.857 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-05-07T02:05:12.857 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-05-07T02:05:12.857 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-07T02:05:12.857 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-05-07T02:05:12.857 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-05-07T02:05:12.857 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-07T02:05:12.857 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-05-07T02:05:12.857 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-07T02:05:12.857 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-05-07T02:05:12.857 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-07T02:05:12.857 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-07T02:05:12.857 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-05-07T02:05:12.857 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-05-07T02:05:12.858 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-05-07T02:05:12.858 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-05-07T02:05:12.858 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-05-07T02:05:12.858 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-05-07T02:05:12.858 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-05-07T02:05:12.858 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-05-07T02:05:12.858 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-05-07T02:05:12.858 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-05-07T02:05:12.858 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-05-07T02:05:12.858 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-05-07T02:05:12.858 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-05-07T02:05:12.858 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-05-07T02:05:12.858 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-05-07T02:05:12.858 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-05-07T02:05:12.859 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-05-07T02:05:12.859 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-05-07T02:05:12.859 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-05-07T02:05:12.859 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-05-07T02:05:12.859 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-05-07T02:05:12.859 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-05-07T02:05:12.859 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-05-07T02:05:12.859 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-05-07T02:05:12.859 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-05-07T02:05:12.859 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-05-07T02:05:12.859 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-05-07T02:05:12.859 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-05-07T02:05:12.859 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-05-07T02:05:12.859 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-05-07T02:05:12.859 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-05-07T02:05:12.859 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-05-07T02:05:12.859 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-05-07T02:05:12.860 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-05-07T02:05:12.860 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-05-07T02:05:12.860 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-05-07T02:05:12.860 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-05-07T02:05:12.860 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-05-07T02:05:12.860 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-05-07T02:05:12.860 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-05-07T02:05:12.860 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-05-07T02:05:12.860 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-05-07T02:05:12.860 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-05-07T02:05:12.860 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-05-07T02:05:12.860 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-05-07T02:05:12.860 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-05-07T02:05:12.860 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-05-07T02:05:12.860 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-05-07T02:05:12.860 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-05-07T02:05:12.861 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-05-07T02:05:12.861 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-05-07T02:05:12.861 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-05-07T02:05:12.861 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-05-07T02:05:12.861 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-05-07T02:05:12.861 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-05-07T02:05:12.861 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-05-07T02:05:12.861 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-05-07T02:05:12.861 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-05-07T02:05:12.861 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-05-07T02:05:12.861 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-05-07T02:05:12.861 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-05-07T02:05:12.861 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-05-07T02:05:12.861 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-05-07T02:05:12.861 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-05-07T02:05:12.862 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-05-07T02:05:12.862 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-05-07T02:05:12.862 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-05-07T02:05:12.862 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-05-07T02:05:12.862 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-05-07T02:05:12.862 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-05-07T02:05:12.862 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-05-07T02:05:12.862 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-05-07T02:05:12.862 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-05-07T02:05:12.862 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-05-07T02:05:12.862 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-05-07T02:05:12.862 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-05-07T02:05:12.862 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-05-07T02:05:12.862 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-05-07T02:05:12.862 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-05-07T02:05:12.863 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-05-07T02:05:12.863 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-05-07T02:05:12.863 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-05-07T02:05:12.863 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-05-07T02:05:12.863 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-05-07T02:05:12.863 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-05-07T02:05:12.863 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-05-07T02:05:12.863 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-05-07T02:05:12.863 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-05-07T02:05:12.863 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-05-07T02:05:12.863 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-05-07T02:05:12.863 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-05-07T02:05:12.863 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-05-07T02:05:12.863 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-05-07T02:05:12.863 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-05-07T02:05:12.864 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-05-07T02:05:12.864 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-05-07T02:05:12.864 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-05-07T02:05:12.864 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-05-07T02:05:12.864 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-05-07T02:05:12.864 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-05-07T02:05:12.864 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-05-07T02:05:12.864 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-05-07T02:05:12.864 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-05-07T02:05:12.864 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-05-07T02:05:12.864 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-05-07T02:05:12.864 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-05-07T02:05:12.864 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-05-07T02:05:12.864 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-05-07T02:05:12.864 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-05-07T02:05:12.865 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-05-07T02:05:12.865 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-05-07T02:05:12.865 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-05-07T02:05:12.865 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-05-07T02:05:12.865 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-05-07T02:05:12.865 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-05-07T02:05:12.865 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-07T02:05:12.865 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-07T02:05:12.865 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-05-07T02:05:12.865 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-05-07T02:05:12.865 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-07T02:05:12.865 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-05-07T02:05:12.865 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-05-07T02:05:12.865 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-05-07T02:05:12.865 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-05-07T02:05:12.866 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-05-07T02:05:12.866 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-05-07T02:05:12.866 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-07T02:05:12.866 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-07T02:05:12.866 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-05-07T02:05:12.866 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-07T02:05:12.866 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-07T02:05:12.866 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-05-07T02:05:12.866 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-07T02:05:12.866 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-05-07T02:05:12.866 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-05-07T02:05:12.866 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-05-07T02:05:12.866 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-07T02:05:12.866 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-05-07T02:05:12.867 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-05-07T02:05:12.867 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-05-07T02:05:12.867 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-05-07T02:05:12.867 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-05-07T02:05:12.867 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-05-07T02:05:12.867 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-05-07T02:05:12.867 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-05-07T02:05:12.867 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-05-07T02:05:12.867 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-05-07T02:05:12.867 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-05-07T02:05:12.867 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-05-07T02:05:12.867 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-05-07T02:05:12.867 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-05-07T02:05:12.867 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-05-07T02:05:12.867 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-05-07T02:05:12.872 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-05-07T02:05:12.872 DEBUG:teuthology.orchestra.run.smithi019:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-07T02:05:12.896 INFO:teuthology.orchestra.run.smithi019.stdout:postinstall scriptlet (using /bin/sh): 2024-05-07T02:05:12.896 INFO:teuthology.orchestra.run.smithi019.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-07T02:05:12.897 INFO:teuthology.orchestra.run.smithi019.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-05-07T02:05:12.897 INFO:teuthology.orchestra.run.smithi019.stdout:fi 2024-05-07T02:05:12.897 INFO:teuthology.orchestra.run.smithi019.stdout:for file in vmlinuz System.map config; do 2024-05-07T02:05:12.897 INFO:teuthology.orchestra.run.smithi019.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-05-07T02:05:12.897 INFO:teuthology.orchestra.run.smithi019.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-05-07T02:05:12.897 INFO:teuthology.orchestra.run.smithi019.stdout: fi 2024-05-07T02:05:12.897 INFO:teuthology.orchestra.run.smithi019.stdout:done 2024-05-07T02:05:12.897 INFO:teuthology.orchestra.run.smithi019.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-05-07T02:05:12.897 INFO:teuthology.orchestra.run.smithi019.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-05-07T02:05:12.897 INFO:teuthology.orchestra.run.smithi019.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-05-07T02:05:12.897 INFO:teuthology.orchestra.run.smithi019.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-05-07T02:05:12.897 INFO:teuthology.orchestra.run.smithi019.stdout:fi 2024-05-07T02:05:12.897 INFO:teuthology.orchestra.run.smithi019.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-05-07T02:05:12.897 INFO:teuthology.orchestra.run.smithi019.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-05-07T02:05:12.898 INFO:teuthology.orchestra.run.smithi019.stdout:fi 2024-05-07T02:05:12.898 INFO:teuthology.orchestra.run.smithi019.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-07T02:05:12.898 INFO:teuthology.orchestra.run.smithi019.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-07T02:05:12.898 INFO:teuthology.orchestra.run.smithi019.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-05-07T02:05:12.898 INFO:teuthology.orchestra.run.smithi019.stdout:fi 2024-05-07T02:05:12.899 DEBUG:teuthology.orchestra.run.smithi019:> sudo rpm -qi grub2-tools 2024-05-07T02:05:12.972 INFO:teuthology.orchestra.run.smithi019.stdout:Name : grub2-tools 2024-05-07T02:05:12.973 INFO:teuthology.orchestra.run.smithi019.stdout:Epoch : 1 2024-05-07T02:05:12.973 INFO:teuthology.orchestra.run.smithi019.stdout:Version : 2.06 2024-05-07T02:05:12.973 INFO:teuthology.orchestra.run.smithi019.stdout:Release : 68.el9 2024-05-07T02:05:12.973 INFO:teuthology.orchestra.run.smithi019.stdout:Architecture: x86_64 2024-05-07T02:05:12.973 INFO:teuthology.orchestra.run.smithi019.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-07T02:05:12.973 INFO:teuthology.orchestra.run.smithi019.stdout:Group : Unspecified 2024-05-07T02:05:12.973 INFO:teuthology.orchestra.run.smithi019.stdout:Size : 8264254 2024-05-07T02:05:12.973 INFO:teuthology.orchestra.run.smithi019.stdout:License : GPLv3+ 2024-05-07T02:05:12.973 INFO:teuthology.orchestra.run.smithi019.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-07T02:05:12.973 INFO:teuthology.orchestra.run.smithi019.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-07T02:05:12.974 INFO:teuthology.orchestra.run.smithi019.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-07T02:05:12.974 INFO:teuthology.orchestra.run.smithi019.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-07T02:05:12.974 INFO:teuthology.orchestra.run.smithi019.stdout:Packager : builder@centos.org 2024-05-07T02:05:12.974 INFO:teuthology.orchestra.run.smithi019.stdout:Vendor : CentOS 2024-05-07T02:05:12.974 INFO:teuthology.orchestra.run.smithi019.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-07T02:05:12.974 INFO:teuthology.orchestra.run.smithi019.stdout:Summary : Support tools for GRUB. 2024-05-07T02:05:12.974 INFO:teuthology.orchestra.run.smithi019.stdout:Description : 2024-05-07T02:05:12.974 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:05:12.975 INFO:teuthology.orchestra.run.smithi019.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-07T02:05:12.975 INFO:teuthology.orchestra.run.smithi019.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-07T02:05:12.975 INFO:teuthology.orchestra.run.smithi019.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-07T02:05:12.975 INFO:teuthology.orchestra.run.smithi019.stdout:hardware devices. 2024-05-07T02:05:12.975 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:05:12.975 INFO:teuthology.orchestra.run.smithi019.stdout:This subpackage provides tools for support of all platforms. 2024-05-07T02:05:12.976 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-07T02:05:12.976 INFO:teuthology.task.kernel:Updating grub on smithi019 to boot 6.9.0-rc3-g806ec36b2112 2024-05-07T02:05:12.976 DEBUG:teuthology.orchestra.run.smithi019:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-07T02:05:13.669 INFO:teuthology.orchestra.run.smithi019.stderr:Generating grub configuration file ... 2024-05-07T02:05:15.015 INFO:teuthology.orchestra.run.smithi019.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-07T02:05:15.055 INFO:teuthology.orchestra.run.smithi019.stderr:done 2024-05-07T02:05:15.058 DEBUG:teuthology.orchestra.run.smithi019:> mktemp 2024-05-07T02:05:15.087 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/tmp.9I48jfMBYM 2024-05-07T02:05:15.087 DEBUG:teuthology.orchestra.run.smithi019:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.9I48jfMBYM 2024-05-07T02:05:15.153 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 0666 /tmp/tmp.9I48jfMBYM 2024-05-07T02:05:15.291 DEBUG:teuthology.orchestra.remote:smithi019:/tmp/tmp.9I48jfMBYM is 6KB 2024-05-07T02:05:15.312 DEBUG:teuthology.orchestra.run.smithi019:> rm -fr /tmp/tmp.9I48jfMBYM 2024-05-07T02:05:15.327 DEBUG:teuthology.orchestra.run.smithi019:> sudo /bin/ls /boot/loader/entries || true 2024-05-07T02:05:15.388 INFO:teuthology.orchestra.run.smithi019.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-07T02:05:15.389 INFO:teuthology.orchestra.run.smithi019.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-07T02:05:15.389 INFO:teuthology.orchestra.run.smithi019.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-07T02:05:15.389 INFO:teuthology.orchestra.run.smithi019.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-05-07T02:05:15.390 DEBUG:teuthology.orchestra.run.smithi019:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-05-07T02:05:15.567 DEBUG:teuthology.orchestra.run.smithi019:> sudo shutdown -r now 2024-05-07T02:05:45.570 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-07T02:05:45.570 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-05-07T02:05:45.571 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-07T02:06:45.573 DEBUG:teuthology.orchestra.remote:timed out 2024-05-07T02:06:54.574 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-05-07T02:06:54.575 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-07T02:06:54.924 DEBUG:teuthology.orchestra.run.smithi019:> true 2024-05-07T02:06:55.644 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-05-07T02:06:55.644 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-05-07T02:06:55.644 DEBUG:teuthology.orchestra.run.smithi019:> uname -r 2024-05-07T02:06:55.689 INFO:teuthology.orchestra.run.smithi019.stdout:6.9.0-rc3-g806ec36b2112 2024-05-07T02:06:55.689 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-05-07T02:06:55.690 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-07T02:06:55.690 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-07T02:06:56.692 DEBUG:teuthology.parallel:result is None 2024-05-07T02:06:56.693 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-07T02:06:56.700 INFO:teuthology.task.internal:Creating test directory... 2024-05-07T02:06:56.700 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-07T02:06:56.706 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-07T02:06:56.708 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-07T02:06:56.750 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-07T02:06:56.757 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-07T02:06:56.767 INFO:teuthology.task.internal:Creating archive directory... 2024-05-07T02:06:56.767 DEBUG:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-07T02:06:56.771 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-07T02:06:56.774 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-07T02:06:56.822 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-07T02:06:56.828 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-07T02:06:56.828 DEBUG:teuthology.orchestra.run.smithi027:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-07T02:06:56.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-07T02:06:56.854 DEBUG:teuthology.orchestra.run.smithi064:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-07T02:06:56.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-07T02:06:56.871 DEBUG:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-07T02:06:56.876 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-07T02:06:56.897 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-07T02:06:56.925 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-07T02:06:56.945 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-07T02:06:56.959 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-07T02:06:56.961 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-07T02:06:57.116 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-07T02:06:57.160 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-07T02:06:57.165 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-07T02:06:57.172 INFO:teuthology.task.internal:Configuring sudo... 2024-05-07T02:06:57.172 DEBUG:teuthology.orchestra.run.smithi019:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-07T02:06:57.177 DEBUG:teuthology.orchestra.run.smithi027:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-07T02:06:57.180 DEBUG:teuthology.orchestra.run.smithi064:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-07T02:06:57.256 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-07T02:06:57.265 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-07T02:06:57.265 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-07T02:06:57.272 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-07T02:06:57.275 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-07T02:06:57.318 DEBUG:teuthology.orchestra.run.smithi019:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-07T02:06:57.395 DEBUG:teuthology.orchestra.run.smithi019:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-07T02:06:57.480 DEBUG:teuthology.orchestra.run.smithi019:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-07T02:06:57.554 DEBUG:teuthology.orchestra.run.smithi019:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-07T02:06:57.622 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-07T02:06:57.622 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-07T02:06:57.702 DEBUG:teuthology.orchestra.run.smithi027:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-07T02:06:57.747 DEBUG:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-07T02:06:57.804 DEBUG:teuthology.orchestra.run.smithi027:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-07T02:06:57.889 DEBUG:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-07T02:06:57.933 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-05-07T02:06:57.933 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-07T02:06:58.001 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-07T02:06:58.046 DEBUG:teuthology.orchestra.run.smithi064:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-07T02:06:58.096 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-07T02:06:58.181 DEBUG:teuthology.orchestra.run.smithi064:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-07T02:06:58.224 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-07T02:06:58.224 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-07T02:06:58.292 DEBUG:teuthology.orchestra.run.smithi019:> sudo service rsyslog restart 2024-05-07T02:06:58.298 DEBUG:teuthology.orchestra.run.smithi027:> sudo service rsyslog restart 2024-05-07T02:06:58.301 DEBUG:teuthology.orchestra.run.smithi064:> sudo service rsyslog restart 2024-05-07T02:06:58.334 INFO:teuthology.orchestra.run.smithi027.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-07T02:06:58.373 INFO:teuthology.orchestra.run.smithi064.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-07T02:06:58.402 INFO:teuthology.orchestra.run.smithi019.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-07T02:06:58.748 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-07T02:06:58.756 INFO:teuthology.task.internal:Starting timer... 2024-05-07T02:06:58.756 INFO:teuthology.run_tasks:Running task pcp... 2024-05-07T02:06:58.769 INFO:teuthology.run_tasks:Running task selinux... 2024-05-07T02:06:58.777 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-07T02:06:58.777 DEBUG:teuthology.orchestra.run.smithi019:> sudo service auditd rotate 2024-05-07T02:06:58.927 INFO:teuthology.orchestra.run.smithi019.stdout:Rotating logs: 2024-05-07T02:06:58.933 DEBUG:teuthology.orchestra.run.smithi027:> sudo service auditd rotate 2024-05-07T02:06:59.010 INFO:teuthology.orchestra.run.smithi027.stdout:Rotating logs: 2024-05-07T02:06:59.012 DEBUG:teuthology.orchestra.run.smithi064:> sudo service auditd rotate 2024-05-07T02:06:59.066 INFO:teuthology.orchestra.run.smithi064.stdout:Rotating logs: 2024-05-07T02:06:59.068 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-07T02:06:59.068 DEBUG:teuthology.orchestra.run.smithi019:> /usr/sbin/getenforce 2024-05-07T02:06:59.130 INFO:teuthology.orchestra.run.smithi019.stdout:Permissive 2024-05-07T02:06:59.131 DEBUG:teuthology.orchestra.run.smithi027:> /usr/sbin/getenforce 2024-05-07T02:06:59.155 INFO:teuthology.orchestra.run.smithi027.stdout:Permissive 2024-05-07T02:06:59.155 DEBUG:teuthology.orchestra.run.smithi064:> /usr/sbin/getenforce 2024-05-07T02:06:59.187 INFO:teuthology.orchestra.run.smithi064.stdout:Permissive 2024-05-07T02:06:59.187 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi019.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi027.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi064.front.sepia.ceph.com': 'permissive'} 2024-05-07T02:06:59.188 DEBUG:teuthology.orchestra.run.smithi019:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-07T02:06:59.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-07T02:06:59.256 DEBUG:teuthology.orchestra.run.smithi027:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-07T02:06:59.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-07T02:06:59.284 DEBUG:teuthology.orchestra.run.smithi064:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-07T02:06:59.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-07T02:06:59.311 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-07T02:06:59.311 DEBUG:teuthology.orchestra.run.smithi019:> sudo /usr/sbin/setenforce permissive 2024-05-07T02:06:59.412 DEBUG:teuthology.orchestra.run.smithi027:> sudo /usr/sbin/setenforce permissive 2024-05-07T02:06:59.441 DEBUG:teuthology.orchestra.run.smithi064:> sudo /usr/sbin/setenforce permissive 2024-05-07T02:06:59.468 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-07T02:06:59.479 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-07T02:06:59.496 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-07T02:06:59.497 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi019.front.sepia.ceph.com,smithi027.front.sepia.ceph.com,smithi064.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-07T02:12:41.340 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi019.front.sepia.ceph.com'), Remote(name='ubuntu@smithi027.front.sepia.ceph.com'), Remote(name='ubuntu@smithi064.front.sepia.ceph.com')] 2024-05-07T02:12:41.342 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-05-07T02:12:41.343 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-07T02:12:41.450 DEBUG:teuthology.orchestra.run.smithi019:> true 2024-05-07T02:12:41.602 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-05-07T02:12:41.602 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-05-07T02:12:41.603 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-07T02:12:41.678 DEBUG:teuthology.orchestra.run.smithi027:> true 2024-05-07T02:12:41.760 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-05-07T02:12:41.760 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-05-07T02:12:41.761 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-07T02:12:41.837 DEBUG:teuthology.orchestra.run.smithi064:> true 2024-05-07T02:12:41.915 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-05-07T02:12:41.916 INFO:teuthology.run_tasks:Running task clock... 2024-05-07T02:12:41.927 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-07T02:12:41.927 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-07T02:12:41.927 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-07T02:12:41.935 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-07T02:12:41.935 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-07T02:12:41.937 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-07T02:12:41.938 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-07T02:12:41.969 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-07T02:12:41.984 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-07T02:12:41.987 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-07T02:12:42.003 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-07T02:12:42.009 INFO:teuthology.orchestra.run.smithi027.stderr:sudo: ntpd: command not found 2024-05-07T02:12:42.021 INFO:teuthology.orchestra.run.smithi027.stdout:506 Cannot talk to daemon 2024-05-07T02:12:42.028 INFO:teuthology.orchestra.run.smithi064.stderr:sudo: ntpd: command not found 2024-05-07T02:12:42.028 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-07T02:12:42.035 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-07T02:12:42.039 INFO:teuthology.orchestra.run.smithi064.stdout:506 Cannot talk to daemon 2024-05-07T02:12:42.049 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-07T02:12:42.054 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-07T02:12:42.067 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-07T02:12:42.068 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-07T02:12:42.110 INFO:teuthology.orchestra.run.smithi027.stderr:bash: line 1: ntpq: command not found 2024-05-07T02:12:42.114 INFO:teuthology.orchestra.run.smithi027.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-07T02:12:42.114 INFO:teuthology.orchestra.run.smithi027.stdout:=============================================================================== 2024-05-07T02:12:42.114 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-07T02:12:42.114 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-07T02:12:42.114 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-07T02:12:42.114 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-07T02:12:42.125 INFO:teuthology.orchestra.run.smithi064.stderr:bash: line 1: ntpq: command not found 2024-05-07T02:12:42.129 INFO:teuthology.orchestra.run.smithi064.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-07T02:12:42.129 INFO:teuthology.orchestra.run.smithi064.stdout:=============================================================================== 2024-05-07T02:12:42.129 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-07T02:12:42.129 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-07T02:12:42.129 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-07T02:12:42.129 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-07T02:12:42.140 INFO:teuthology.orchestra.run.smithi019.stderr:sudo: ntpd: command not found 2024-05-07T02:12:42.175 INFO:teuthology.orchestra.run.smithi019.stdout:506 Cannot talk to daemon 2024-05-07T02:12:42.218 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-07T02:12:42.262 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-07T02:12:42.410 INFO:teuthology.orchestra.run.smithi019.stderr:bash: line 1: ntpq: command not found 2024-05-07T02:12:42.418 INFO:teuthology.orchestra.run.smithi019.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-07T02:12:42.418 INFO:teuthology.orchestra.run.smithi019.stdout:=============================================================================== 2024-05-07T02:12:42.418 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-07T02:12:42.418 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-07T02:12:42.418 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-07T02:12:42.418 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-07T02:12:42.419 INFO:teuthology.run_tasks:Running task install... 2024-05-07T02:12:42.428 DEBUG:teuthology.task.install:project ceph 2024-05-07T02:12:42.428 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'bd4e00a39d766328b458e4c953f4d722fed96a31'}} 2024-05-07T02:12:42.428 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bd4e00a39d766328b458e4c953f4d722fed96a31'} 2024-05-07T02:12:42.428 INFO:teuthology.task.install:Using flavor: default 2024-05-07T02:12:42.434 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-07T02:12:42.434 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-07T02:12:42.434 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bd4e00a39d766328b458e4c953f4d722fed96a31', 'tag': None, 'wait_for_package': False} 2024-05-07T02:12:42.435 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bd4e00a39d766328b458e4c953f4d722fed96a31 2024-05-07T02:12:42.436 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bd4e00a39d766328b458e4c953f4d722fed96a31', 'tag': None, 'wait_for_package': False} 2024-05-07T02:12:42.437 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bd4e00a39d766328b458e4c953f4d722fed96a31 2024-05-07T02:12:42.438 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bd4e00a39d766328b458e4c953f4d722fed96a31', 'tag': None, 'wait_for_package': False} 2024-05-07T02:12:42.438 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bd4e00a39d766328b458e4c953f4d722fed96a31 2024-05-07T02:12:42.581 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/flavors/default/ 2024-05-07T02:12:42.581 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3522.gbd4e00a3 2024-05-07T02:12:42.589 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/flavors/default/ 2024-05-07T02:12:42.589 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3522.gbd4e00a3 2024-05-07T02:12:42.657 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/flavors/default/ 2024-05-07T02:12:42.658 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3522.gbd4e00a3 2024-05-07T02:12:42.695 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-07T02:12:42.695 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-07T02:12:42.695 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-07T02:12:42.706 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-07T02:12:42.706 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-07T02:12:42.706 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-07T02:12:42.734 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-07T02:12:42.735 DEBUG:teuthology.orchestra.run.smithi064:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bd4e00a39d766328b458e4c953f4d722fed96a31/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-07T02:12:42.793 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-07T02:12:42.794 DEBUG:teuthology.orchestra.run.smithi019:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bd4e00a39d766328b458e4c953f4d722fed96a31/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-07T02:12:42.802 DEBUG:teuthology.orchestra.run.smithi064:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-07T02:12:42.827 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-07T02:12:42.827 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-05-07T02:12:42.827 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-07T02:12:42.857 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-07T02:12:42.857 DEBUG:teuthology.orchestra.run.smithi027:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bd4e00a39d766328b458e4c953f4d722fed96a31/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-07T02:12:42.879 DEBUG:teuthology.orchestra.run.smithi064:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-07T02:12:42.926 DEBUG:teuthology.orchestra.run.smithi027:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-07T02:12:42.928 DEBUG:teuthology.orchestra.run.smithi019:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-07T02:12:42.945 INFO:teuthology.orchestra.run.smithi064.stdout:check_obsoletes = 1 2024-05-07T02:12:42.946 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum clean all 2024-05-07T02:12:43.005 DEBUG:teuthology.orchestra.run.smithi027:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-07T02:12:43.047 DEBUG:teuthology.orchestra.run.smithi019:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-07T02:12:43.072 INFO:teuthology.orchestra.run.smithi027.stdout:check_obsoletes = 1 2024-05-07T02:12:43.074 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum clean all 2024-05-07T02:12:43.128 INFO:teuthology.orchestra.run.smithi019.stdout:check_obsoletes = 1 2024-05-07T02:12:43.133 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum clean all 2024-05-07T02:12:43.183 INFO:teuthology.orchestra.run.smithi064.stdout:53 files removed 2024-05-07T02:12:43.211 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-07T02:12:43.315 INFO:teuthology.orchestra.run.smithi027.stdout:53 files removed 2024-05-07T02:12:43.342 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-07T02:12:43.468 INFO:teuthology.orchestra.run.smithi019.stdout:53 files removed 2024-05-07T02:12:43.507 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-07T02:12:43.897 INFO:teuthology.orchestra.run.smithi064.stdout:ceph packages for x86_64 203 kB/s | 84 kB 00:00 2024-05-07T02:12:44.027 INFO:teuthology.orchestra.run.smithi027.stdout:ceph packages for x86_64 203 kB/s | 84 kB 00:00 2024-05-07T02:12:44.273 INFO:teuthology.orchestra.run.smithi064.stdout:ceph noarch packages 36 kB/s | 12 kB 00:00 2024-05-07T02:12:44.394 INFO:teuthology.orchestra.run.smithi027.stdout:ceph noarch packages 37 kB/s | 12 kB 00:00 2024-05-07T02:12:44.477 INFO:teuthology.orchestra.run.smithi019.stdout:ceph packages for x86_64 157 kB/s | 84 kB 00:00 2024-05-07T02:12:44.648 INFO:teuthology.orchestra.run.smithi064.stdout:ceph source packages 5.6 kB/s | 1.9 kB 00:00 2024-05-07T02:12:44.745 INFO:teuthology.orchestra.run.smithi027.stdout:ceph source packages 6.0 kB/s | 1.9 kB 00:00 2024-05-07T02:12:45.061 INFO:teuthology.orchestra.run.smithi019.stdout:ceph noarch packages 24 kB/s | 12 kB 00:00 2024-05-07T02:12:45.637 INFO:teuthology.orchestra.run.smithi019.stdout:ceph source packages 3.9 kB/s | 1.9 kB 00:00 2024-05-07T02:12:46.712 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - BaseOS 4.2 MB/s | 8.1 MB 00:01 2024-05-07T02:12:47.940 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - BaseOS 2.5 MB/s | 8.1 MB 00:03 2024-05-07T02:12:49.351 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - BaseOS 2.2 MB/s | 8.1 MB 00:03 2024-05-07T02:13:03.045 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - AppStream 1.3 MB/s | 19 MB 00:15 2024-05-07T02:13:05.640 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 19 MB 00:16 2024-05-07T02:13:07.277 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 19 MB 00:16 2024-05-07T02:13:08.429 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.2 MB 00:01 2024-05-07T02:13:10.772 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - Extras packages 15 kB/s | 16 kB 00:01 2024-05-07T02:13:12.759 INFO:teuthology.orchestra.run.smithi027.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-05-07T02:13:13.161 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - CRB 3.3 MB/s | 6.2 MB 00:01 2024-05-07T02:13:13.648 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - CRB 1.4 MB/s | 6.2 MB 00:04 2024-05-07T02:13:15.024 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - Extras packages 99 kB/s | 16 kB 00:00 2024-05-07T02:13:15.028 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 16 kB 00:00 2024-05-07T02:13:16.045 INFO:teuthology.orchestra.run.smithi064.stdout:Extra Packages for Enterprise Linux 24 MB/s | 21 MB 00:00 2024-05-07T02:13:17.654 INFO:teuthology.orchestra.run.smithi019.stdout:Extra Packages for Enterprise Linux 8.7 MB/s | 21 MB 00:02 2024-05-07T02:13:18.082 INFO:teuthology.orchestra.run.smithi027.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-05-07T02:13:20.425 INFO:teuthology.orchestra.run.smithi027.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-07T02:13:20.425 INFO:teuthology.orchestra.run.smithi027.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-07T02:13:20.436 INFO:teuthology.orchestra.run.smithi027.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-07T02:13:20.437 INFO:teuthology.orchestra.run.smithi027.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-07T02:13:20.438 INFO:teuthology.orchestra.run.smithi027.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-07T02:13:20.439 INFO:teuthology.orchestra.run.smithi027.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-07T02:13:20.441 INFO:teuthology.orchestra.run.smithi027.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-07T02:13:20.443 INFO:teuthology.orchestra.run.smithi027.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-07T02:13:20.444 INFO:teuthology.orchestra.run.smithi027.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-07T02:13:20.508 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-05-07T02:13:20.529 INFO:teuthology.orchestra.run.smithi027.stdout:======================================================================================= 2024-05-07T02:13:20.529 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2024-05-07T02:13:20.529 INFO:teuthology.orchestra.run.smithi027.stdout:======================================================================================= 2024-05-07T02:13:20.529 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-05-07T02:13:20.529 INFO:teuthology.orchestra.run.smithi027.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-07T02:13:20.529 INFO:teuthology.orchestra.run.smithi027.stdout: ceph x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 6.6 k 2024-05-07T02:13:20.529 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-base x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 5.4 M 2024-05-07T02:13:20.529 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-fuse x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 863 k 2024-05-07T02:13:20.529 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 147 k 2024-05-07T02:13:20.529 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 1.5 M 2024-05-07T02:13:20.530 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 142 k 2024-05-07T02:13:20.530 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 3.5 M 2024-05-07T02:13:20.530 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 7.4 M 2024-05-07T02:13:20.530 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-rook noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 49 k 2024-05-07T02:13:20.530 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-radosgw x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 10 M 2024-05-07T02:13:20.530 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-test x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 43 M 2024-05-07T02:13:20.530 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-volume noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 273 k 2024-05-07T02:13:20.530 INFO:teuthology.orchestra.run.smithi027.stdout: cephadm noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 763 k 2024-05-07T02:13:20.530 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-mirror x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 224 k 2024-05-07T02:13:20.530 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-shell x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 27 k 2024-05-07T02:13:20.530 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-top noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 18 k 2024-05-07T02:13:20.530 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-07T02:13:20.530 INFO:teuthology.orchestra.run.smithi027.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-07T02:13:20.530 INFO:teuthology.orchestra.run.smithi027.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-07T02:13:20.531 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs-devel x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 32 k 2024-05-07T02:13:20.531 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs2 x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 756 k 2024-05-07T02:13:20.531 INFO:teuthology.orchestra.run.smithi027.stdout: librados-devel x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 127 k 2024-05-07T02:13:20.531 INFO:teuthology.orchestra.run.smithi027.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-07T02:13:20.531 INFO:teuthology.orchestra.run.smithi027.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-07T02:13:20.531 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-07T02:13:20.531 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-07T02:13:20.531 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-07T02:13:20.531 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-07T02:13:20.531 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cephfs x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 162 k 2024-05-07T02:13:20.531 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rados x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 321 k 2024-05-07T02:13:20.531 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rbd x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 298 k 2024-05-07T02:13:20.531 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rgw x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 100 k 2024-05-07T02:13:20.532 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-fuse x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 85 k 2024-05-07T02:13:20.532 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-mirror x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 3.0 M 2024-05-07T02:13:20.532 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-nbd x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 170 k 2024-05-07T02:13:20.532 INFO:teuthology.orchestra.run.smithi027.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-07T02:13:20.532 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrading: 2024-05-07T02:13:20.532 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-07T02:13:20.532 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-07T02:13:20.532 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-07T02:13:20.532 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-07T02:13:20.532 INFO:teuthology.orchestra.run.smithi027.stdout: librados2 x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 3.3 M 2024-05-07T02:13:20.532 INFO:teuthology.orchestra.run.smithi027.stdout: librbd1 x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 3.1 M 2024-05-07T02:13:20.532 INFO:teuthology.orchestra.run.smithi027.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-07T02:13:20.532 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-07T02:13:20.533 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2024-05-07T02:13:20.533 INFO:teuthology.orchestra.run.smithi027.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-07T02:13:20.533 INFO:teuthology.orchestra.run.smithi027.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-07T02:13:20.533 INFO:teuthology.orchestra.run.smithi027.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-07T02:13:20.533 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-common x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 20 M 2024-05-07T02:13:20.533 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 34 k 2024-05-07T02:13:20.533 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mds x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 2.3 M 2024-05-07T02:13:20.533 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 274 k 2024-05-07T02:13:20.533 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mon x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 4.9 M 2024-05-07T02:13:20.533 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-osd x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 18 M 2024-05-07T02:13:20.533 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 16 k 2024-05-07T02:13:20.533 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-selinux x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 25 k 2024-05-07T02:13:20.533 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-07T02:13:20.533 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-07T02:13:20.534 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-07T02:13:20.534 INFO:teuthology.orchestra.run.smithi027.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-07T02:13:20.534 INFO:teuthology.orchestra.run.smithi027.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-07T02:13:20.534 INFO:teuthology.orchestra.run.smithi027.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-07T02:13:20.534 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-07T02:13:20.534 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-07T02:13:20.534 INFO:teuthology.orchestra.run.smithi027.stdout: libcephsqlite x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 158 k 2024-05-07T02:13:20.534 INFO:teuthology.orchestra.run.smithi027.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-07T02:13:20.534 INFO:teuthology.orchestra.run.smithi027.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-07T02:13:20.534 INFO:teuthology.orchestra.run.smithi027.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-07T02:13:20.534 INFO:teuthology.orchestra.run.smithi027.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-07T02:13:20.534 INFO:teuthology.orchestra.run.smithi027.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-07T02:13:20.534 INFO:teuthology.orchestra.run.smithi027.stdout: libradosstriper1 x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 497 k 2024-05-07T02:13:20.535 INFO:teuthology.orchestra.run.smithi027.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-07T02:13:20.535 INFO:teuthology.orchestra.run.smithi027.stdout: librgw2 x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 5.2 M 2024-05-07T02:13:20.535 INFO:teuthology.orchestra.run.smithi027.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-07T02:13:20.535 INFO:teuthology.orchestra.run.smithi027.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-07T02:13:20.535 INFO:teuthology.orchestra.run.smithi027.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-07T02:13:20.535 INFO:teuthology.orchestra.run.smithi027.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-07T02:13:20.535 INFO:teuthology.orchestra.run.smithi027.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-07T02:13:20.535 INFO:teuthology.orchestra.run.smithi027.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-07T02:13:20.535 INFO:teuthology.orchestra.run.smithi027.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-07T02:13:20.535 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-07T02:13:20.535 INFO:teuthology.orchestra.run.smithi027.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-07T02:13:20.535 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-07T02:13:20.535 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-07T02:13:20.536 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-07T02:13:20.536 INFO:teuthology.orchestra.run.smithi027.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-07T02:13:20.536 INFO:teuthology.orchestra.run.smithi027.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-07T02:13:20.536 INFO:teuthology.orchestra.run.smithi027.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-07T02:13:20.536 INFO:teuthology.orchestra.run.smithi027.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-07T02:13:20.536 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-07T02:13:20.536 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-argparse x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 45 k 2024-05-07T02:13:20.536 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-common x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 135 k 2024-05-07T02:13:20.536 INFO:teuthology.orchestra.run.smithi027.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-07T02:13:20.536 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-07T02:13:20.536 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-07T02:13:20.536 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-07T02:13:20.536 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-07T02:13:20.537 INFO:teuthology.orchestra.run.smithi027.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-07T02:13:20.537 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-07T02:13:20.537 INFO:teuthology.orchestra.run.smithi027.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-07T02:13:20.537 INFO:teuthology.orchestra.run.smithi027.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-07T02:13:20.537 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-07T02:13:20.537 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-07T02:13:20.537 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-07T02:13:20.537 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-07T02:13:20.537 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-07T02:13:20.537 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-07T02:13:20.537 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-07T02:13:20.537 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-07T02:13:20.537 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-07T02:13:20.537 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-07T02:13:20.538 INFO:teuthology.orchestra.run.smithi027.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-07T02:13:20.538 INFO:teuthology.orchestra.run.smithi027.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-07T02:13:20.538 INFO:teuthology.orchestra.run.smithi027.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-07T02:13:20.538 INFO:teuthology.orchestra.run.smithi027.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-07T02:13:20.538 INFO:teuthology.orchestra.run.smithi027.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-07T02:13:20.538 INFO:teuthology.orchestra.run.smithi027.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-07T02:13:20.538 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-07T02:13:20.538 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-07T02:13:20.538 INFO:teuthology.orchestra.run.smithi027.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-07T02:13:20.538 INFO:teuthology.orchestra.run.smithi027.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-07T02:13:20.538 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-07T02:13:20.538 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-07T02:13:20.538 INFO:teuthology.orchestra.run.smithi027.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-07T02:13:20.538 INFO:teuthology.orchestra.run.smithi027.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-07T02:13:20.539 INFO:teuthology.orchestra.run.smithi027.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-07T02:13:20.539 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-07T02:13:20.539 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-07T02:13:20.539 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-07T02:13:20.539 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-07T02:13:20.539 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-07T02:13:20.539 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-07T02:13:20.539 INFO:teuthology.orchestra.run.smithi027.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-07T02:13:20.539 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-07T02:13:20.539 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-07T02:13:20.539 INFO:teuthology.orchestra.run.smithi027.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-07T02:13:20.539 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-07T02:13:20.539 INFO:teuthology.orchestra.run.smithi027.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-07T02:13:20.539 INFO:teuthology.orchestra.run.smithi027.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-07T02:13:20.540 INFO:teuthology.orchestra.run.smithi027.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-07T02:13:20.540 INFO:teuthology.orchestra.run.smithi027.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-07T02:13:20.540 INFO:teuthology.orchestra.run.smithi027.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-07T02:13:20.540 INFO:teuthology.orchestra.run.smithi027.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-07T02:13:20.540 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-07T02:13:20.540 INFO:teuthology.orchestra.run.smithi027.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-07T02:13:20.540 INFO:teuthology.orchestra.run.smithi027.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-07T02:13:20.540 INFO:teuthology.orchestra.run.smithi027.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-07T02:13:20.540 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-07T02:13:20.540 INFO:teuthology.orchestra.run.smithi027.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-07T02:13:20.540 INFO:teuthology.orchestra.run.smithi027.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-07T02:13:20.540 INFO:teuthology.orchestra.run.smithi027.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-07T02:13:20.540 INFO:teuthology.orchestra.run.smithi027.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-07T02:13:20.540 INFO:teuthology.orchestra.run.smithi027.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-07T02:13:20.540 INFO:teuthology.orchestra.run.smithi027.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-07T02:13:20.541 INFO:teuthology.orchestra.run.smithi027.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-07T02:13:20.541 INFO:teuthology.orchestra.run.smithi027.stdout:Installing weak dependencies: 2024-05-07T02:13:20.541 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-07T02:13:20.541 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-07T02:13:20.541 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-05-07T02:13:20.541 INFO:teuthology.orchestra.run.smithi027.stdout:======================================================================================= 2024-05-07T02:13:20.541 INFO:teuthology.orchestra.run.smithi027.stdout:Install 148 Packages 2024-05-07T02:13:20.541 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrade 8 Packages 2024-05-07T02:13:20.541 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-07T02:13:20.541 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 219 M 2024-05-07T02:13:20.542 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-05-07T02:13:21.358 INFO:teuthology.orchestra.run.smithi027.stdout:(1/156): ceph-19.0.0-3522.gbd4e00a3.el9.x86_64. 21 kB/s | 6.6 kB 00:00 2024-05-07T02:13:21.451 INFO:teuthology.orchestra.run.smithi064.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-05-07T02:13:21.892 INFO:teuthology.orchestra.run.smithi027.stdout:(2/156): ceph-fuse-19.0.0-3522.gbd4e00a3.el9.x8 1.6 MB/s | 863 kB 00:00 2024-05-07T02:13:21.984 INFO:teuthology.orchestra.run.smithi027.stdout:(3/156): ceph-immutable-object-cache-19.0.0-352 1.6 MB/s | 147 kB 00:00 2024-05-07T02:13:22.417 INFO:teuthology.orchestra.run.smithi027.stdout:(4/156): ceph-common-19.0.0-3522.gbd4e00a3.el9. 14 MB/s | 20 MB 00:01 2024-05-07T02:13:22.551 INFO:teuthology.orchestra.run.smithi027.stdout:(5/156): ceph-mds-19.0.0-3522.gbd4e00a3.el9.x86 4.0 MB/s | 2.3 MB 00:00 2024-05-07T02:13:22.585 INFO:teuthology.orchestra.run.smithi027.stdout:(6/156): ceph-mgr-19.0.0-3522.gbd4e00a3.el9.x86 9.0 MB/s | 1.5 MB 00:00 2024-05-07T02:13:22.727 INFO:teuthology.orchestra.run.smithi027.stdout:(7/156): ceph-base-19.0.0-3522.gbd4e00a3.el9.x8 3.2 MB/s | 5.4 MB 00:01 2024-05-07T02:13:23.418 INFO:teuthology.orchestra.run.smithi027.stdout:(8/156): ceph-mon-19.0.0-3522.gbd4e00a3.el9.x86 5.6 MB/s | 4.9 MB 00:00 2024-05-07T02:13:23.519 INFO:teuthology.orchestra.run.smithi027.stdout:(9/156): ceph-selinux-19.0.0-3522.gbd4e00a3.el9 252 kB/s | 25 kB 00:00 2024-05-07T02:13:23.795 INFO:teuthology.orchestra.run.smithi064.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-07T02:13:23.795 INFO:teuthology.orchestra.run.smithi064.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-07T02:13:23.806 INFO:teuthology.orchestra.run.smithi064.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-07T02:13:23.807 INFO:teuthology.orchestra.run.smithi064.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-07T02:13:23.808 INFO:teuthology.orchestra.run.smithi064.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-07T02:13:23.809 INFO:teuthology.orchestra.run.smithi064.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-07T02:13:23.810 INFO:teuthology.orchestra.run.smithi064.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-07T02:13:23.813 INFO:teuthology.orchestra.run.smithi064.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-07T02:13:23.813 INFO:teuthology.orchestra.run.smithi064.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-07T02:13:23.881 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-05-07T02:13:23.896 INFO:teuthology.orchestra.run.smithi064.stdout:======================================================================================= 2024-05-07T02:13:23.896 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-05-07T02:13:23.896 INFO:teuthology.orchestra.run.smithi064.stdout:======================================================================================= 2024-05-07T02:13:23.896 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-05-07T02:13:23.896 INFO:teuthology.orchestra.run.smithi064.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-07T02:13:23.896 INFO:teuthology.orchestra.run.smithi064.stdout: ceph x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 6.6 k 2024-05-07T02:13:23.896 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-base x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 5.4 M 2024-05-07T02:13:23.896 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-fuse x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 863 k 2024-05-07T02:13:23.896 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 147 k 2024-05-07T02:13:23.896 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 1.5 M 2024-05-07T02:13:23.896 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 142 k 2024-05-07T02:13:23.896 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 3.5 M 2024-05-07T02:13:23.896 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 7.4 M 2024-05-07T02:13:23.896 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-rook noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 49 k 2024-05-07T02:13:23.897 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-radosgw x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 10 M 2024-05-07T02:13:23.897 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-test x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 43 M 2024-05-07T02:13:23.897 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-volume noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 273 k 2024-05-07T02:13:23.897 INFO:teuthology.orchestra.run.smithi064.stdout: cephadm noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 763 k 2024-05-07T02:13:23.897 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-mirror x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 224 k 2024-05-07T02:13:23.897 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-shell x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 27 k 2024-05-07T02:13:23.897 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-top noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 18 k 2024-05-07T02:13:23.897 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-07T02:13:23.897 INFO:teuthology.orchestra.run.smithi064.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-07T02:13:23.897 INFO:teuthology.orchestra.run.smithi064.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-07T02:13:23.897 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs-devel x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 32 k 2024-05-07T02:13:23.897 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs2 x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 756 k 2024-05-07T02:13:23.897 INFO:teuthology.orchestra.run.smithi064.stdout: librados-devel x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 127 k 2024-05-07T02:13:23.897 INFO:teuthology.orchestra.run.smithi064.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-07T02:13:23.897 INFO:teuthology.orchestra.run.smithi064.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-07T02:13:23.898 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-07T02:13:23.898 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-07T02:13:23.898 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-07T02:13:23.898 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-07T02:13:23.898 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cephfs x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 162 k 2024-05-07T02:13:23.898 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rados x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 321 k 2024-05-07T02:13:23.898 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rbd x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 298 k 2024-05-07T02:13:23.898 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rgw x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 100 k 2024-05-07T02:13:23.898 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-fuse x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 85 k 2024-05-07T02:13:23.898 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-mirror x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 3.0 M 2024-05-07T02:13:23.898 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-nbd x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 170 k 2024-05-07T02:13:23.898 INFO:teuthology.orchestra.run.smithi064.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-07T02:13:23.898 INFO:teuthology.orchestra.run.smithi064.stdout:Upgrading: 2024-05-07T02:13:23.898 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-07T02:13:23.898 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-07T02:13:23.899 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-07T02:13:23.899 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-07T02:13:23.899 INFO:teuthology.orchestra.run.smithi064.stdout: librados2 x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 3.3 M 2024-05-07T02:13:23.899 INFO:teuthology.orchestra.run.smithi064.stdout: librbd1 x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 3.1 M 2024-05-07T02:13:23.899 INFO:teuthology.orchestra.run.smithi064.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-07T02:13:23.899 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-07T02:13:23.899 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-05-07T02:13:23.899 INFO:teuthology.orchestra.run.smithi064.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-07T02:13:23.899 INFO:teuthology.orchestra.run.smithi064.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-07T02:13:23.899 INFO:teuthology.orchestra.run.smithi064.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-07T02:13:23.899 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-common x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 20 M 2024-05-07T02:13:23.899 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 34 k 2024-05-07T02:13:23.899 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mds x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 2.3 M 2024-05-07T02:13:23.899 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 274 k 2024-05-07T02:13:23.899 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mon x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 4.9 M 2024-05-07T02:13:23.900 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-osd x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 18 M 2024-05-07T02:13:23.900 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 16 k 2024-05-07T02:13:23.900 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-selinux x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 25 k 2024-05-07T02:13:23.900 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-07T02:13:23.900 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-07T02:13:23.900 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-07T02:13:23.900 INFO:teuthology.orchestra.run.smithi064.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-07T02:13:23.900 INFO:teuthology.orchestra.run.smithi064.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-07T02:13:23.900 INFO:teuthology.orchestra.run.smithi064.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-07T02:13:23.900 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-07T02:13:23.900 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-07T02:13:23.900 INFO:teuthology.orchestra.run.smithi064.stdout: libcephsqlite x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 158 k 2024-05-07T02:13:23.900 INFO:teuthology.orchestra.run.smithi064.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-07T02:13:23.900 INFO:teuthology.orchestra.run.smithi064.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-07T02:13:23.900 INFO:teuthology.orchestra.run.smithi064.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-07T02:13:23.900 INFO:teuthology.orchestra.run.smithi064.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-07T02:13:23.900 INFO:teuthology.orchestra.run.smithi064.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-07T02:13:23.901 INFO:teuthology.orchestra.run.smithi064.stdout: libradosstriper1 x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 497 k 2024-05-07T02:13:23.901 INFO:teuthology.orchestra.run.smithi064.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-07T02:13:23.901 INFO:teuthology.orchestra.run.smithi064.stdout: librgw2 x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 5.2 M 2024-05-07T02:13:23.901 INFO:teuthology.orchestra.run.smithi064.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-07T02:13:23.901 INFO:teuthology.orchestra.run.smithi064.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-07T02:13:23.901 INFO:teuthology.orchestra.run.smithi064.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-07T02:13:23.901 INFO:teuthology.orchestra.run.smithi064.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-07T02:13:23.901 INFO:teuthology.orchestra.run.smithi064.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-07T02:13:23.901 INFO:teuthology.orchestra.run.smithi064.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-07T02:13:23.901 INFO:teuthology.orchestra.run.smithi064.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-07T02:13:23.901 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-07T02:13:23.901 INFO:teuthology.orchestra.run.smithi064.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-07T02:13:23.901 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-07T02:13:23.901 INFO:teuthology.orchestra.run.smithi064.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-07T02:13:23.901 INFO:teuthology.orchestra.run.smithi064.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-07T02:13:23.901 INFO:teuthology.orchestra.run.smithi064.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-07T02:13:23.901 INFO:teuthology.orchestra.run.smithi064.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-07T02:13:23.902 INFO:teuthology.orchestra.run.smithi064.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-07T02:13:23.902 INFO:teuthology.orchestra.run.smithi064.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-07T02:13:23.902 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-07T02:13:23.902 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 45 k 2024-05-07T02:13:23.902 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-common x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 135 k 2024-05-07T02:13:23.902 INFO:teuthology.orchestra.run.smithi064.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-07T02:13:23.902 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-07T02:13:23.902 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-07T02:13:23.902 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-07T02:13:23.902 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-07T02:13:23.902 INFO:teuthology.orchestra.run.smithi064.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-07T02:13:23.902 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-07T02:13:23.903 INFO:teuthology.orchestra.run.smithi064.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-07T02:13:23.903 INFO:teuthology.orchestra.run.smithi064.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-07T02:13:23.903 INFO:teuthology.orchestra.run.smithi064.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-07T02:13:23.903 INFO:teuthology.orchestra.run.smithi064.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-07T02:13:23.903 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-07T02:13:23.903 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-07T02:13:23.903 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-07T02:13:23.903 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-07T02:13:23.903 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-07T02:13:23.903 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-07T02:13:23.903 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-07T02:13:23.903 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-07T02:13:23.903 INFO:teuthology.orchestra.run.smithi064.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-07T02:13:23.903 INFO:teuthology.orchestra.run.smithi064.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-07T02:13:23.904 INFO:teuthology.orchestra.run.smithi064.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-07T02:13:23.904 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-07T02:13:23.904 INFO:teuthology.orchestra.run.smithi064.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-07T02:13:23.904 INFO:teuthology.orchestra.run.smithi064.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-07T02:13:23.904 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-07T02:13:23.904 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-07T02:13:23.904 INFO:teuthology.orchestra.run.smithi064.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-07T02:13:23.904 INFO:teuthology.orchestra.run.smithi064.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-07T02:13:23.904 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-07T02:13:23.904 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-07T02:13:23.904 INFO:teuthology.orchestra.run.smithi064.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-07T02:13:23.904 INFO:teuthology.orchestra.run.smithi064.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-07T02:13:23.904 INFO:teuthology.orchestra.run.smithi064.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-07T02:13:23.904 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-07T02:13:23.904 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-07T02:13:23.905 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-07T02:13:23.905 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-07T02:13:23.905 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-07T02:13:23.905 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-07T02:13:23.905 INFO:teuthology.orchestra.run.smithi064.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-07T02:13:23.905 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-07T02:13:23.905 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-07T02:13:23.905 INFO:teuthology.orchestra.run.smithi064.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-07T02:13:23.905 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-07T02:13:23.905 INFO:teuthology.orchestra.run.smithi064.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-07T02:13:23.905 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-07T02:13:23.905 INFO:teuthology.orchestra.run.smithi064.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-07T02:13:23.905 INFO:teuthology.orchestra.run.smithi064.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-07T02:13:23.905 INFO:teuthology.orchestra.run.smithi064.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-07T02:13:23.905 INFO:teuthology.orchestra.run.smithi064.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-07T02:13:23.905 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-07T02:13:23.906 INFO:teuthology.orchestra.run.smithi064.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-07T02:13:23.906 INFO:teuthology.orchestra.run.smithi064.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-07T02:13:23.906 INFO:teuthology.orchestra.run.smithi064.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-07T02:13:23.906 INFO:teuthology.orchestra.run.smithi064.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-07T02:13:23.906 INFO:teuthology.orchestra.run.smithi064.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-07T02:13:23.906 INFO:teuthology.orchestra.run.smithi064.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-07T02:13:23.906 INFO:teuthology.orchestra.run.smithi064.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-07T02:13:23.906 INFO:teuthology.orchestra.run.smithi064.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-07T02:13:23.906 INFO:teuthology.orchestra.run.smithi064.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-07T02:13:23.906 INFO:teuthology.orchestra.run.smithi064.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-07T02:13:23.906 INFO:teuthology.orchestra.run.smithi064.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-07T02:13:23.906 INFO:teuthology.orchestra.run.smithi064.stdout:Installing weak dependencies: 2024-05-07T02:13:23.906 INFO:teuthology.orchestra.run.smithi064.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-07T02:13:23.906 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-07T02:13:23.906 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-05-07T02:13:23.906 INFO:teuthology.orchestra.run.smithi064.stdout:======================================================================================= 2024-05-07T02:13:23.907 INFO:teuthology.orchestra.run.smithi064.stdout:Install 148 Packages 2024-05-07T02:13:23.907 INFO:teuthology.orchestra.run.smithi064.stdout:Upgrade 8 Packages 2024-05-07T02:13:23.907 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-07T02:13:23.907 INFO:teuthology.orchestra.run.smithi027.stdout:(10/156): ceph-osd-19.0.0-3522.gbd4e00a3.el9.x8 13 MB/s | 18 MB 00:01 2024-05-07T02:13:23.907 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 219 M 2024-05-07T02:13:23.908 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-05-07T02:13:23.970 INFO:teuthology.orchestra.run.smithi027.stdout:(11/156): cephfs-mirror-19.0.0-3522.gbd4e00a3.e 3.3 MB/s | 224 kB 00:00 2024-05-07T02:13:24.021 INFO:teuthology.orchestra.run.smithi019.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-05-07T02:13:24.087 INFO:teuthology.orchestra.run.smithi027.stdout:(12/156): ceph-radosgw-19.0.0-3522.gbd4e00a3.el 7.4 MB/s | 10 MB 00:01 2024-05-07T02:13:24.112 INFO:teuthology.orchestra.run.smithi027.stdout:(13/156): cephfs-shell-19.0.0-3522.gbd4e00a3.el 190 kB/s | 27 kB 00:00 2024-05-07T02:13:24.170 INFO:teuthology.orchestra.run.smithi027.stdout:(14/156): libcephfs-devel-19.0.0-3522.gbd4e00a3 380 kB/s | 32 kB 00:00 2024-05-07T02:13:24.196 INFO:teuthology.orchestra.run.smithi027.stdout:(15/156): libcephfs2-19.0.0-3522.gbd4e00a3.el9. 8.8 MB/s | 756 kB 00:00 2024-05-07T02:13:24.246 INFO:teuthology.orchestra.run.smithi027.stdout:(16/156): libcephsqlite-19.0.0-3522.gbd4e00a3.e 2.1 MB/s | 158 kB 00:00 2024-05-07T02:13:24.271 INFO:teuthology.orchestra.run.smithi027.stdout:(17/156): librados-devel-19.0.0-3522.gbd4e00a3. 1.6 MB/s | 127 kB 00:00 2024-05-07T02:13:24.355 INFO:teuthology.orchestra.run.smithi027.stdout:(18/156): libradosstriper1-19.0.0-3522.gbd4e00a 4.5 MB/s | 497 kB 00:00 2024-05-07T02:13:24.455 INFO:teuthology.orchestra.run.smithi027.stdout:(19/156): python3-ceph-argparse-19.0.0-3522.gbd 451 kB/s | 45 kB 00:00 2024-05-07T02:13:24.555 INFO:teuthology.orchestra.run.smithi027.stdout:(20/156): python3-ceph-common-19.0.0-3522.gbd4e 1.3 MB/s | 135 kB 00:00 2024-05-07T02:13:24.706 INFO:teuthology.orchestra.run.smithi027.stdout:(21/156): librgw2-19.0.0-3522.gbd4e00a3.el9.x86 12 MB/s | 5.2 MB 00:00 2024-05-07T02:13:24.723 INFO:teuthology.orchestra.run.smithi027.stdout:(22/156): python3-cephfs-19.0.0-3522.gbd4e00a3. 969 kB/s | 162 kB 00:00 2024-05-07T02:13:24.773 INFO:teuthology.orchestra.run.smithi027.stdout:(23/156): python3-rados-19.0.0-3522.gbd4e00a3.e 4.7 MB/s | 321 kB 00:00 2024-05-07T02:13:24.823 INFO:teuthology.orchestra.run.smithi027.stdout:(24/156): python3-rbd-19.0.0-3522.gbd4e00a3.el9 2.9 MB/s | 298 kB 00:00 2024-05-07T02:13:24.849 INFO:teuthology.orchestra.run.smithi027.stdout:(25/156): python3-rgw-19.0.0-3522.gbd4e00a3.el9 1.3 MB/s | 100 kB 00:00 2024-05-07T02:13:24.915 INFO:teuthology.orchestra.run.smithi027.stdout:(26/156): rbd-fuse-19.0.0-3522.gbd4e00a3.el9.x8 933 kB/s | 85 kB 00:00 2024-05-07T02:13:25.041 INFO:teuthology.orchestra.run.smithi027.stdout:(27/156): rbd-nbd-19.0.0-3522.gbd4e00a3.el9.x86 1.3 MB/s | 170 kB 00:00 2024-05-07T02:13:25.166 INFO:teuthology.orchestra.run.smithi027.stdout:(28/156): rbd-mirror-19.0.0-3522.gbd4e00a3.el9. 9.6 MB/s | 3.0 MB 00:00 2024-05-07T02:13:25.192 INFO:teuthology.orchestra.run.smithi027.stdout:(29/156): ceph-grafana-dashboards-19.0.0-3522.g 228 kB/s | 34 kB 00:00 2024-05-07T02:13:25.233 INFO:teuthology.orchestra.run.smithi027.stdout:(30/156): ceph-mgr-cephadm-19.0.0-3522.gbd4e00a 2.1 MB/s | 142 kB 00:00 2024-05-07T02:13:25.453 INFO:teuthology.orchestra.run.smithi064.stdout:(1/156): ceph-19.0.0-3522.gbd4e00a3.el9.x86_64. 15 kB/s | 6.6 kB 00:00 2024-05-07T02:13:25.642 INFO:teuthology.orchestra.run.smithi027.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3522.gbd4e0 7.9 MB/s | 3.5 MB 00:00 2024-05-07T02:13:25.776 INFO:teuthology.orchestra.run.smithi027.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3522.gbd 2.0 MB/s | 274 kB 00:00 2024-05-07T02:13:25.884 INFO:teuthology.orchestra.run.smithi027.stdout:(33/156): ceph-mgr-rook-19.0.0-3522.gbd4e00a3.e 456 kB/s | 49 kB 00:00 2024-05-07T02:13:26.001 INFO:teuthology.orchestra.run.smithi027.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3522.gb 141 kB/s | 16 kB 00:00 2024-05-07T02:13:26.135 INFO:teuthology.orchestra.run.smithi027.stdout:(35/156): ceph-volume-19.0.0-3522.gbd4e00a3.el9 2.0 MB/s | 273 kB 00:00 2024-05-07T02:13:26.218 INFO:teuthology.orchestra.run.smithi027.stdout:(36/156): ceph-mgr-diskprediction-local-19.0.0- 7.5 MB/s | 7.4 MB 00:00 2024-05-07T02:13:26.260 INFO:teuthology.orchestra.run.smithi027.stdout:(37/156): cephadm-19.0.0-3522.gbd4e00a3.el9.noa 6.0 MB/s | 763 kB 00:00 2024-05-07T02:13:26.294 INFO:teuthology.orchestra.run.smithi027.stdout:(38/156): cephfs-top-19.0.0-3522.gbd4e00a3.el9. 241 kB/s | 18 kB 00:00 2024-05-07T02:13:26.603 INFO:teuthology.orchestra.run.smithi064.stdout:(2/156): ceph-fuse-19.0.0-3522.gbd4e00a3.el9.x8 751 kB/s | 863 kB 00:01 2024-05-07T02:13:26.604 INFO:teuthology.orchestra.run.smithi019.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-07T02:13:26.605 INFO:teuthology.orchestra.run.smithi019.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-07T02:13:26.625 INFO:teuthology.orchestra.run.smithi019.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-07T02:13:26.627 INFO:teuthology.orchestra.run.smithi019.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-07T02:13:26.628 INFO:teuthology.orchestra.run.smithi019.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-07T02:13:26.629 INFO:teuthology.orchestra.run.smithi019.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-07T02:13:26.632 INFO:teuthology.orchestra.run.smithi019.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-07T02:13:26.635 INFO:teuthology.orchestra.run.smithi019.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-07T02:13:26.637 INFO:teuthology.orchestra.run.smithi019.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-07T02:13:26.711 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-05-07T02:13:26.733 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2024-05-07T02:13:26.733 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2024-05-07T02:13:26.733 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2024-05-07T02:13:26.733 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-05-07T02:13:26.733 INFO:teuthology.orchestra.run.smithi019.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-07T02:13:26.733 INFO:teuthology.orchestra.run.smithi019.stdout: ceph x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 6.6 k 2024-05-07T02:13:26.733 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-base x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 5.4 M 2024-05-07T02:13:26.733 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-fuse x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 863 k 2024-05-07T02:13:26.733 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 147 k 2024-05-07T02:13:26.734 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 1.5 M 2024-05-07T02:13:26.734 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 142 k 2024-05-07T02:13:26.734 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 3.5 M 2024-05-07T02:13:26.734 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 7.4 M 2024-05-07T02:13:26.734 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-rook noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 49 k 2024-05-07T02:13:26.734 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-radosgw x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 10 M 2024-05-07T02:13:26.734 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-test x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 43 M 2024-05-07T02:13:26.734 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-volume noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 273 k 2024-05-07T02:13:26.734 INFO:teuthology.orchestra.run.smithi019.stdout: cephadm noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 763 k 2024-05-07T02:13:26.734 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-mirror x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 224 k 2024-05-07T02:13:26.734 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-shell x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 27 k 2024-05-07T02:13:26.734 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-top noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 18 k 2024-05-07T02:13:26.734 INFO:teuthology.orchestra.run.smithi019.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-07T02:13:26.735 INFO:teuthology.orchestra.run.smithi019.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-07T02:13:26.735 INFO:teuthology.orchestra.run.smithi019.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-07T02:13:26.735 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs-devel x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 32 k 2024-05-07T02:13:26.735 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs2 x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 756 k 2024-05-07T02:13:26.735 INFO:teuthology.orchestra.run.smithi019.stdout: librados-devel x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 127 k 2024-05-07T02:13:26.735 INFO:teuthology.orchestra.run.smithi019.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-07T02:13:26.735 INFO:teuthology.orchestra.run.smithi019.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-07T02:13:26.735 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-07T02:13:26.735 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-07T02:13:26.735 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-07T02:13:26.735 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-07T02:13:26.735 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cephfs x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 162 k 2024-05-07T02:13:26.735 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rados x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 321 k 2024-05-07T02:13:26.736 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rbd x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 298 k 2024-05-07T02:13:26.736 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rgw x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 100 k 2024-05-07T02:13:26.736 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-fuse x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 85 k 2024-05-07T02:13:26.736 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-mirror x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 3.0 M 2024-05-07T02:13:26.736 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-nbd x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 170 k 2024-05-07T02:13:26.736 INFO:teuthology.orchestra.run.smithi019.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-07T02:13:26.736 INFO:teuthology.orchestra.run.smithi019.stdout:Upgrading: 2024-05-07T02:13:26.736 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-07T02:13:26.736 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-07T02:13:26.736 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-07T02:13:26.736 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-07T02:13:26.736 INFO:teuthology.orchestra.run.smithi019.stdout: librados2 x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 3.3 M 2024-05-07T02:13:26.736 INFO:teuthology.orchestra.run.smithi019.stdout: librbd1 x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 3.1 M 2024-05-07T02:13:26.737 INFO:teuthology.orchestra.run.smithi019.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-07T02:13:26.737 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-07T02:13:26.737 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-05-07T02:13:26.737 INFO:teuthology.orchestra.run.smithi019.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-07T02:13:26.737 INFO:teuthology.orchestra.run.smithi019.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-07T02:13:26.737 INFO:teuthology.orchestra.run.smithi019.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-07T02:13:26.737 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-common x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 20 M 2024-05-07T02:13:26.737 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 34 k 2024-05-07T02:13:26.737 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mds x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 2.3 M 2024-05-07T02:13:26.737 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 274 k 2024-05-07T02:13:26.737 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mon x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 4.9 M 2024-05-07T02:13:26.737 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-osd x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 18 M 2024-05-07T02:13:26.737 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 16 k 2024-05-07T02:13:26.737 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-selinux x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 25 k 2024-05-07T02:13:26.738 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-07T02:13:26.738 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-07T02:13:26.738 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-07T02:13:26.738 INFO:teuthology.orchestra.run.smithi019.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-07T02:13:26.738 INFO:teuthology.orchestra.run.smithi019.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-07T02:13:26.738 INFO:teuthology.orchestra.run.smithi019.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-07T02:13:26.738 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-07T02:13:26.738 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-07T02:13:26.738 INFO:teuthology.orchestra.run.smithi019.stdout: libcephsqlite x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 158 k 2024-05-07T02:13:26.738 INFO:teuthology.orchestra.run.smithi019.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-07T02:13:26.738 INFO:teuthology.orchestra.run.smithi019.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-07T02:13:26.738 INFO:teuthology.orchestra.run.smithi019.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-07T02:13:26.738 INFO:teuthology.orchestra.run.smithi019.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-07T02:13:26.738 INFO:teuthology.orchestra.run.smithi019.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-07T02:13:26.738 INFO:teuthology.orchestra.run.smithi019.stdout: libradosstriper1 x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 497 k 2024-05-07T02:13:26.739 INFO:teuthology.orchestra.run.smithi019.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-07T02:13:26.739 INFO:teuthology.orchestra.run.smithi019.stdout: librgw2 x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 5.2 M 2024-05-07T02:13:26.739 INFO:teuthology.orchestra.run.smithi019.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-07T02:13:26.739 INFO:teuthology.orchestra.run.smithi019.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-07T02:13:26.739 INFO:teuthology.orchestra.run.smithi019.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-07T02:13:26.739 INFO:teuthology.orchestra.run.smithi019.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-07T02:13:26.739 INFO:teuthology.orchestra.run.smithi019.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-07T02:13:26.739 INFO:teuthology.orchestra.run.smithi019.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-07T02:13:26.739 INFO:teuthology.orchestra.run.smithi019.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-07T02:13:26.739 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-07T02:13:26.739 INFO:teuthology.orchestra.run.smithi019.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-07T02:13:26.739 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-07T02:13:26.739 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-07T02:13:26.739 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-07T02:13:26.739 INFO:teuthology.orchestra.run.smithi019.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-07T02:13:26.740 INFO:teuthology.orchestra.run.smithi019.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-07T02:13:26.740 INFO:teuthology.orchestra.run.smithi019.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-07T02:13:26.740 INFO:teuthology.orchestra.run.smithi019.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-07T02:13:26.740 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-07T02:13:26.740 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-argparse x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 45 k 2024-05-07T02:13:26.740 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-common x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 135 k 2024-05-07T02:13:26.740 INFO:teuthology.orchestra.run.smithi019.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-07T02:13:26.740 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-07T02:13:26.740 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-07T02:13:26.740 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-07T02:13:26.740 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-07T02:13:26.740 INFO:teuthology.orchestra.run.smithi019.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-07T02:13:26.740 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-07T02:13:26.740 INFO:teuthology.orchestra.run.smithi019.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-07T02:13:26.740 INFO:teuthology.orchestra.run.smithi019.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-07T02:13:26.741 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-07T02:13:26.741 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-07T02:13:26.741 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-07T02:13:26.741 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-07T02:13:26.741 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-07T02:13:26.741 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-07T02:13:26.741 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-07T02:13:26.741 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-07T02:13:26.741 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-07T02:13:26.741 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-07T02:13:26.741 INFO:teuthology.orchestra.run.smithi019.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-07T02:13:26.741 INFO:teuthology.orchestra.run.smithi019.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-07T02:13:26.742 INFO:teuthology.orchestra.run.smithi019.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-07T02:13:26.742 INFO:teuthology.orchestra.run.smithi019.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-07T02:13:26.742 INFO:teuthology.orchestra.run.smithi019.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-07T02:13:26.742 INFO:teuthology.orchestra.run.smithi019.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-07T02:13:26.742 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-07T02:13:26.742 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-07T02:13:26.742 INFO:teuthology.orchestra.run.smithi019.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-07T02:13:26.742 INFO:teuthology.orchestra.run.smithi019.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-07T02:13:26.742 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-07T02:13:26.742 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-07T02:13:26.742 INFO:teuthology.orchestra.run.smithi019.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-07T02:13:26.742 INFO:teuthology.orchestra.run.smithi019.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-07T02:13:26.742 INFO:teuthology.orchestra.run.smithi019.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-07T02:13:26.742 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-07T02:13:26.743 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-07T02:13:26.743 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-07T02:13:26.743 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-07T02:13:26.743 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-07T02:13:26.743 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-07T02:13:26.743 INFO:teuthology.orchestra.run.smithi019.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-07T02:13:26.743 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-07T02:13:26.743 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-07T02:13:26.743 INFO:teuthology.orchestra.run.smithi019.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-07T02:13:26.743 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-07T02:13:26.743 INFO:teuthology.orchestra.run.smithi019.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-07T02:13:26.743 INFO:teuthology.orchestra.run.smithi019.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-07T02:13:26.743 INFO:teuthology.orchestra.run.smithi019.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-07T02:13:26.744 INFO:teuthology.orchestra.run.smithi019.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-07T02:13:26.744 INFO:teuthology.orchestra.run.smithi019.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-07T02:13:26.744 INFO:teuthology.orchestra.run.smithi019.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-07T02:13:26.744 INFO:teuthology.orchestra.run.smithi019.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-07T02:13:26.744 INFO:teuthology.orchestra.run.smithi019.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-07T02:13:26.744 INFO:teuthology.orchestra.run.smithi019.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-07T02:13:26.744 INFO:teuthology.orchestra.run.smithi019.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-07T02:13:26.744 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-07T02:13:26.744 INFO:teuthology.orchestra.run.smithi019.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-07T02:13:26.744 INFO:teuthology.orchestra.run.smithi019.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-07T02:13:26.744 INFO:teuthology.orchestra.run.smithi019.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-07T02:13:26.744 INFO:teuthology.orchestra.run.smithi019.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-07T02:13:26.744 INFO:teuthology.orchestra.run.smithi019.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-07T02:13:26.745 INFO:teuthology.orchestra.run.smithi019.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-07T02:13:26.745 INFO:teuthology.orchestra.run.smithi019.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-07T02:13:26.745 INFO:teuthology.orchestra.run.smithi019.stdout:Installing weak dependencies: 2024-05-07T02:13:26.745 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-07T02:13:26.745 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:13:26.745 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-05-07T02:13:26.745 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2024-05-07T02:13:26.745 INFO:teuthology.orchestra.run.smithi019.stdout:Install 148 Packages 2024-05-07T02:13:26.745 INFO:teuthology.orchestra.run.smithi019.stdout:Upgrade 8 Packages 2024-05-07T02:13:26.745 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:13:26.746 INFO:teuthology.orchestra.run.smithi064.stdout:(3/156): ceph-immutable-object-cache-19.0.0-352 1.1 MB/s | 147 kB 00:00 2024-05-07T02:13:26.746 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 219 M 2024-05-07T02:13:26.746 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-05-07T02:13:26.878 INFO:teuthology.orchestra.run.smithi064.stdout:(4/156): ceph-base-19.0.0-3522.gbd4e00a3.el9.x8 2.9 MB/s | 5.4 MB 00:01 2024-05-07T02:13:26.928 INFO:teuthology.orchestra.run.smithi027.stdout:(39/156): ceph-test-19.0.0-3522.gbd4e00a3.el9.x 13 MB/s | 43 MB 00:03 2024-05-07T02:13:26.953 INFO:teuthology.orchestra.run.smithi027.stdout:(40/156): fuse-2.9.9-15.el9.x86_64.rpm 116 kB/s | 80 kB 00:00 2024-05-07T02:13:27.029 INFO:teuthology.orchestra.run.smithi064.stdout:(5/156): ceph-mgr-19.0.0-3522.gbd4e00a3.el9.x86 10 MB/s | 1.5 MB 00:00 2024-05-07T02:13:27.346 INFO:teuthology.orchestra.run.smithi064.stdout:(6/156): ceph-mon-19.0.0-3522.gbd4e00a3.el9.x86 15 MB/s | 4.9 MB 00:00 2024-05-07T02:13:27.445 INFO:teuthology.orchestra.run.smithi027.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 515 kB/s | 253 kB 00:00 2024-05-07T02:13:27.504 INFO:teuthology.orchestra.run.smithi027.stdout:(42/156): libquadmath-11.4.1-3.el9.x86_64.rpm 339 kB/s | 195 kB 00:00 2024-05-07T02:13:27.537 INFO:teuthology.orchestra.run.smithi027.stdout:(43/156): libgfortran-11.4.1-3.el9.x86_64.rpm 646 kB/s | 803 kB 00:01 2024-05-07T02:13:27.621 INFO:teuthology.orchestra.run.smithi064.stdout:(7/156): ceph-mds-19.0.0-3522.gbd4e00a3.el9.x86 2.6 MB/s | 2.3 MB 00:00 2024-05-07T02:13:27.638 INFO:teuthology.orchestra.run.smithi027.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 797 kB/s | 106 kB 00:00 2024-05-07T02:13:27.663 INFO:teuthology.orchestra.run.smithi027.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-05-07T02:13:27.755 INFO:teuthology.orchestra.run.smithi027.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-05-07T02:13:27.788 INFO:teuthology.orchestra.run.smithi027.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 1.7 MB/s | 215 kB 00:00 2024-05-07T02:13:27.906 INFO:teuthology.orchestra.run.smithi019.stdout:(1/156): ceph-19.0.0-3522.gbd4e00a3.el9.x86_64. 21 kB/s | 6.6 kB 00:00 2024-05-07T02:13:27.914 INFO:teuthology.orchestra.run.smithi027.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 2.6 MB/s | 1.2 MB 00:00 2024-05-07T02:13:28.031 INFO:teuthology.orchestra.run.smithi027.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 918 kB/s | 107 kB 00:00 2024-05-07T02:13:28.089 INFO:teuthology.orchestra.run.smithi027.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 2.0 MB/s | 689 kB 00:00 2024-05-07T02:13:28.140 INFO:teuthology.orchestra.run.smithi027.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 2.6 MB/s | 943 kB 00:00 2024-05-07T02:13:28.156 INFO:teuthology.orchestra.run.smithi027.stdout:(52/156): elfutils-libelf-devel-0.190-2.el9.x86 171 kB/s | 21 kB 00:00 2024-05-07T02:13:28.207 INFO:teuthology.orchestra.run.smithi027.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-05-07T02:13:28.249 INFO:teuthology.orchestra.run.smithi027.stdout:(54/156): flexiblas-3.0.4-8.el9.x86_64.rpm 301 kB/s | 33 kB 00:00 2024-05-07T02:13:28.316 INFO:teuthology.orchestra.run.smithi027.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 167 kB/s | 18 kB 00:00 2024-05-07T02:13:28.321 INFO:teuthology.orchestra.run.smithi064.stdout:(8/156): ceph-osd-19.0.0-3522.gbd4e00a3.el9.x86 18 MB/s | 18 MB 00:00 2024-05-07T02:13:28.358 INFO:teuthology.orchestra.run.smithi027.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 102 kB/s | 11 kB 00:00 2024-05-07T02:13:28.380 INFO:teuthology.orchestra.run.smithi064.stdout:(9/156): ceph-selinux-19.0.0-3522.gbd4e00a3.el9 435 kB/s | 25 kB 00:00 2024-05-07T02:13:28.398 INFO:teuthology.orchestra.run.smithi019.stdout:(2/156): ceph-fuse-19.0.0-3522.gbd4e00a3.el9.x8 1.7 MB/s | 863 kB 00:00 2024-05-07T02:13:28.441 INFO:teuthology.orchestra.run.smithi027.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-05-07T02:13:28.483 INFO:teuthology.orchestra.run.smithi027.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 361 kB/s | 45 kB 00:00 2024-05-07T02:13:28.608 INFO:teuthology.orchestra.run.smithi027.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.6 MB/s | 3.0 MB 00:00 2024-05-07T02:13:28.642 INFO:teuthology.orchestra.run.smithi027.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 3.6 MB/s | 582 kB 00:00 2024-05-07T02:13:28.667 INFO:teuthology.orchestra.run.smithi027.stdout:(61/156): librdkafka-1.6.1-102.el9.x86_64.rpm 2.9 MB/s | 662 kB 00:00 2024-05-07T02:13:28.734 INFO:teuthology.orchestra.run.smithi027.stdout:(62/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 202 kB/s | 25 kB 00:00 2024-05-07T02:13:28.768 INFO:teuthology.orchestra.run.smithi027.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 377 kB/s | 47 kB 00:00 2024-05-07T02:13:28.793 INFO:teuthology.orchestra.run.smithi027.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-05-07T02:13:28.823 INFO:teuthology.orchestra.run.smithi019.stdout:(3/156): ceph-immutable-object-cache-19.0.0-352 346 kB/s | 147 kB 00:00 2024-05-07T02:13:28.860 INFO:teuthology.orchestra.run.smithi027.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-05-07T02:13:28.886 INFO:teuthology.orchestra.run.smithi027.stdout:(66/156): openblas-0.3.21-2.el9.x86_64.rpm 301 kB/s | 35 kB 00:00 2024-05-07T02:13:29.194 INFO:teuthology.orchestra.run.smithi027.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 5.2 MB/s | 1.6 MB 00:00 2024-05-07T02:13:29.319 INFO:teuthology.orchestra.run.smithi027.stdout:(68/156): postgresql-contrib-13.14-1.el9.x86_64 7.1 MB/s | 911 kB 00:00 2024-05-07T02:13:29.428 INFO:teuthology.orchestra.run.smithi027.stdout:(69/156): postgresql-private-libs-13.14-1.el9.x 1.2 MB/s | 133 kB 00:00 2024-05-07T02:13:30.112 INFO:teuthology.orchestra.run.smithi027.stdout:(70/156): openblas-openmp-0.3.21-2.el9.x86_64.r 3.6 MB/s | 4.7 MB 00:01 2024-05-07T02:13:30.404 INFO:teuthology.orchestra.run.smithi027.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 3.5 MB/s | 1.0 MB 00:00 2024-05-07T02:13:30.504 INFO:teuthology.orchestra.run.smithi027.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 875 kB/s | 87 kB 00:00 2024-05-07T02:13:30.549 INFO:teuthology.orchestra.run.smithi019.stdout:(4/156): ceph-base-19.0.0-3522.gbd4e00a3.el9.x8 1.8 MB/s | 5.4 MB 00:02 2024-05-07T02:13:30.757 INFO:teuthology.orchestra.run.smithi019.stdout:(5/156): ceph-mds-19.0.0-3522.gbd4e00a3.el9.x86 1.2 MB/s | 2.3 MB 00:01 2024-05-07T02:13:30.829 INFO:teuthology.orchestra.run.smithi064.stdout:(10/156): ceph-radosgw-19.0.0-3522.gbd4e00a3.el 3.2 MB/s | 10 MB 00:03 2024-05-07T02:13:30.946 INFO:teuthology.orchestra.run.smithi064.stdout:(11/156): cephfs-mirror-19.0.0-3522.gbd4e00a3.e 1.9 MB/s | 224 kB 00:00 2024-05-07T02:13:31.055 INFO:teuthology.orchestra.run.smithi064.stdout:(12/156): cephfs-shell-19.0.0-3522.gbd4e00a3.el 250 kB/s | 27 kB 00:00 2024-05-07T02:13:31.141 INFO:teuthology.orchestra.run.smithi019.stdout:(6/156): ceph-mgr-19.0.0-3522.gbd4e00a3.el9.x86 2.5 MB/s | 1.5 MB 00:00 2024-05-07T02:13:31.155 INFO:teuthology.orchestra.run.smithi064.stdout:(13/156): libcephfs-devel-19.0.0-3522.gbd4e00a3 317 kB/s | 32 kB 00:00 2024-05-07T02:13:31.556 INFO:teuthology.orchestra.run.smithi064.stdout:(14/156): ceph-test-19.0.0-3522.gbd4e00a3.el9.x 14 MB/s | 43 MB 00:03 2024-05-07T02:13:31.589 INFO:teuthology.orchestra.run.smithi064.stdout:(15/156): libcephfs2-19.0.0-3522.gbd4e00a3.el9. 1.7 MB/s | 756 kB 00:00 2024-05-07T02:13:31.606 INFO:teuthology.orchestra.run.smithi064.stdout:(16/156): libcephsqlite-19.0.0-3522.gbd4e00a3.e 3.1 MB/s | 158 kB 00:00 2024-05-07T02:13:31.665 INFO:teuthology.orchestra.run.smithi064.stdout:(17/156): librados-devel-19.0.0-3522.gbd4e00a3. 1.7 MB/s | 127 kB 00:00 2024-05-07T02:13:31.726 INFO:teuthology.orchestra.run.smithi064.stdout:(18/156): libradosstriper1-19.0.0-3522.gbd4e00a 4.1 MB/s | 497 kB 00:00 2024-05-07T02:13:31.790 INFO:teuthology.orchestra.run.smithi064.stdout:(19/156): python3-ceph-argparse-19.0.0-3522.gbd 704 kB/s | 45 kB 00:00 2024-05-07T02:13:31.840 INFO:teuthology.orchestra.run.smithi064.stdout:(20/156): python3-ceph-common-19.0.0-3522.gbd4e 2.7 MB/s | 135 kB 00:00 2024-05-07T02:13:31.891 INFO:teuthology.orchestra.run.smithi064.stdout:(21/156): python3-cephfs-19.0.0-3522.gbd4e00a3. 3.2 MB/s | 162 kB 00:00 2024-05-07T02:13:31.958 INFO:teuthology.orchestra.run.smithi064.stdout:(22/156): python3-rados-19.0.0-3522.gbd4e00a3.e 4.7 MB/s | 321 kB 00:00 2024-05-07T02:13:32.016 INFO:teuthology.orchestra.run.smithi064.stdout:(23/156): python3-rbd-19.0.0-3522.gbd4e00a3.el9 5.0 MB/s | 298 kB 00:00 2024-05-07T02:13:32.066 INFO:teuthology.orchestra.run.smithi019.stdout:(7/156): ceph-common-19.0.0-3522.gbd4e00a3.el9. 4.4 MB/s | 20 MB 00:04 2024-05-07T02:13:32.075 INFO:teuthology.orchestra.run.smithi064.stdout:(24/156): python3-rgw-19.0.0-3522.gbd4e00a3.el9 1.7 MB/s | 100 kB 00:00 2024-05-07T02:13:32.142 INFO:teuthology.orchestra.run.smithi064.stdout:(25/156): rbd-fuse-19.0.0-3522.gbd4e00a3.el9.x8 1.3 MB/s | 85 kB 00:00 2024-05-07T02:13:32.154 INFO:teuthology.orchestra.run.smithi027.stdout:(73/156): python3-babel-2.9.1-2.el9.noarch.rpm 3.6 MB/s | 6.0 MB 00:01 2024-05-07T02:13:32.263 INFO:teuthology.orchestra.run.smithi027.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 2.2 MB/s | 244 kB 00:00 2024-05-07T02:13:32.334 INFO:teuthology.orchestra.run.smithi064.stdout:(26/156): ceph-common-19.0.0-3522.gbd4e00a3.el9 2.7 MB/s | 20 MB 00:07 2024-05-07T02:13:32.371 INFO:teuthology.orchestra.run.smithi027.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-05-07T02:13:32.426 INFO:teuthology.orchestra.run.smithi064.stdout:(27/156): rbd-mirror-19.0.0-3522.gbd4e00a3.el9. 11 MB/s | 3.0 MB 00:00 2024-05-07T02:13:32.451 INFO:teuthology.orchestra.run.smithi064.stdout:(28/156): rbd-nbd-19.0.0-3522.gbd4e00a3.el9.x86 1.4 MB/s | 170 kB 00:00 2024-05-07T02:13:32.471 INFO:teuthology.orchestra.run.smithi027.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 256 kB/s | 26 kB 00:00 2024-05-07T02:13:32.477 INFO:teuthology.orchestra.run.smithi064.stdout:(29/156): ceph-grafana-dashboards-19.0.0-3522.g 679 kB/s | 34 kB 00:00 2024-05-07T02:13:32.510 INFO:teuthology.orchestra.run.smithi064.stdout:(30/156): ceph-mgr-cephadm-19.0.0-3522.gbd4e00a 2.4 MB/s | 142 kB 00:00 2024-05-07T02:13:32.516 INFO:teuthology.orchestra.run.smithi019.stdout:(8/156): ceph-mon-19.0.0-3522.gbd4e00a3.el9.x86 2.8 MB/s | 4.9 MB 00:01 2024-05-07T02:13:32.572 INFO:teuthology.orchestra.run.smithi027.stdout:(77/156): python3-jsonpointer-2.0-4.el9.noarch. 191 kB/s | 19 kB 00:00 2024-05-07T02:13:32.608 INFO:teuthology.orchestra.run.smithi019.stdout:(9/156): ceph-selinux-19.0.0-3522.gbd4e00a3.el9 278 kB/s | 25 kB 00:00 2024-05-07T02:13:32.680 INFO:teuthology.orchestra.run.smithi027.stdout:(78/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.6 MB/s | 172 kB 00:00 2024-05-07T02:13:32.747 INFO:teuthology.orchestra.run.smithi027.stdout:(79/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 1.1 MB/s | 4.4 MB 00:03 2024-05-07T02:13:32.786 INFO:teuthology.orchestra.run.smithi064.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3522.gbd4e0 11 MB/s | 3.5 MB 00:00 2024-05-07T02:13:32.789 INFO:teuthology.orchestra.run.smithi027.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 319 kB/s | 35 kB 00:00 2024-05-07T02:13:32.861 INFO:teuthology.orchestra.run.smithi064.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3522.gbd 3.6 MB/s | 274 kB 00:00 2024-05-07T02:13:32.919 INFO:teuthology.orchestra.run.smithi064.stdout:(33/156): ceph-mgr-rook-19.0.0-3522.gbd4e00a3.e 850 kB/s | 49 kB 00:00 2024-05-07T02:13:32.978 INFO:teuthology.orchestra.run.smithi064.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3522.gb 282 kB/s | 16 kB 00:00 2024-05-07T02:13:33.131 INFO:teuthology.orchestra.run.smithi027.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 733 kB/s | 250 kB 00:00 2024-05-07T02:13:33.153 INFO:teuthology.orchestra.run.smithi064.stdout:(35/156): librgw2-19.0.0-3522.gbd4e00a3.el9.x86 3.5 MB/s | 5.2 MB 00:01 2024-05-07T02:13:33.178 INFO:teuthology.orchestra.run.smithi064.stdout:(36/156): ceph-volume-19.0.0-3522.gbd4e00a3.el9 1.3 MB/s | 273 kB 00:00 2024-05-07T02:13:33.237 INFO:teuthology.orchestra.run.smithi064.stdout:(37/156): cephfs-top-19.0.0-3522.gbd4e00a3.el9. 313 kB/s | 18 kB 00:00 2024-05-07T02:13:33.338 INFO:teuthology.orchestra.run.smithi064.stdout:(38/156): cephadm-19.0.0-3522.gbd4e00a3.el9.noa 4.1 MB/s | 763 kB 00:00 2024-05-07T02:13:33.482 INFO:teuthology.orchestra.run.smithi027.stdout:(82/156): python3-oauthlib-3.1.1-5.el9.noarch.r 635 kB/s | 222 kB 00:00 2024-05-07T02:13:33.542 INFO:teuthology.orchestra.run.smithi019.stdout:(10/156): ceph-radosgw-19.0.0-3522.gbd4e00a3.el 6.9 MB/s | 10 MB 00:01 2024-05-07T02:13:33.604 INFO:teuthology.orchestra.run.smithi064.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 219 kB/s | 80 kB 00:00 2024-05-07T02:13:33.617 INFO:teuthology.orchestra.run.smithi019.stdout:(11/156): cephfs-mirror-19.0.0-3522.gbd4e00a3.e 3.0 MB/s | 224 kB 00:00 2024-05-07T02:13:33.673 INFO:teuthology.orchestra.run.smithi027.stdout:(83/156): python3-packaging-20.9-5.el9.noarch.r 403 kB/s | 77 kB 00:00 2024-05-07T02:13:33.675 INFO:teuthology.orchestra.run.smithi019.stdout:(12/156): cephfs-shell-19.0.0-3522.gbd4e00a3.el 472 kB/s | 27 kB 00:00 2024-05-07T02:13:33.734 INFO:teuthology.orchestra.run.smithi019.stdout:(13/156): libcephfs-devel-19.0.0-3522.gbd4e00a3 553 kB/s | 32 kB 00:00 2024-05-07T02:13:33.782 INFO:teuthology.orchestra.run.smithi027.stdout:(84/156): python3-prettytable-0.7.2-27.el9.noar 390 kB/s | 42 kB 00:00 2024-05-07T02:13:33.805 INFO:teuthology.orchestra.run.smithi064.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 976 kB/s | 195 kB 00:00 2024-05-07T02:13:33.876 INFO:teuthology.orchestra.run.smithi019.stdout:(14/156): libcephfs2-19.0.0-3522.gbd4e00a3.el9. 5.3 MB/s | 756 kB 00:00 2024-05-07T02:13:33.960 INFO:teuthology.orchestra.run.smithi019.stdout:(15/156): libcephsqlite-19.0.0-3522.gbd4e00a3.e 1.9 MB/s | 158 kB 00:00 2024-05-07T02:13:33.972 INFO:teuthology.orchestra.run.smithi064.stdout:(41/156): ceph-mgr-diskprediction-local-19.0.0- 5.1 MB/s | 7.4 MB 00:01 2024-05-07T02:13:34.043 INFO:teuthology.orchestra.run.smithi019.stdout:(16/156): librados-devel-19.0.0-3522.gbd4e00a3. 1.5 MB/s | 127 kB 00:00 2024-05-07T02:13:34.047 INFO:teuthology.orchestra.run.smithi064.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.0 MB/s | 253 kB 00:00 2024-05-07T02:13:34.130 INFO:teuthology.orchestra.run.smithi064.stdout:(43/156): libgfortran-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 803 kB 00:00 2024-05-07T02:13:34.156 INFO:teuthology.orchestra.run.smithi064.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 981 kB/s | 106 kB 00:00 2024-05-07T02:13:34.231 INFO:teuthology.orchestra.run.smithi064.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-05-07T02:13:34.249 INFO:teuthology.orchestra.run.smithi027.stdout:(85/156): python3-protobuf-3.14.0-13.el9.noarch 577 kB/s | 269 kB 00:00 2024-05-07T02:13:34.264 INFO:teuthology.orchestra.run.smithi064.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-05-07T02:13:34.348 INFO:teuthology.orchestra.run.smithi064.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 1.8 MB/s | 215 kB 00:00 2024-05-07T02:13:34.352 INFO:teuthology.orchestra.run.smithi019.stdout:(17/156): ceph-osd-19.0.0-3522.gbd4e00a3.el9.x8 5.5 MB/s | 18 MB 00:03 2024-05-07T02:13:34.394 INFO:teuthology.orchestra.run.smithi019.stdout:(18/156): libradosstriper1-19.0.0-3522.gbd4e00a 1.4 MB/s | 497 kB 00:00 2024-05-07T02:13:34.448 INFO:teuthology.orchestra.run.smithi064.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 3.7 MB/s | 689 kB 00:00 2024-05-07T02:13:34.461 INFO:teuthology.orchestra.run.smithi019.stdout:(19/156): python3-ceph-argparse-19.0.0-3522.gbd 689 kB/s | 45 kB 00:00 2024-05-07T02:13:34.490 INFO:teuthology.orchestra.run.smithi064.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-05-07T02:13:34.516 INFO:teuthology.orchestra.run.smithi027.stdout:(86/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 595 kB/s | 159 kB 00:00 2024-05-07T02:13:34.532 INFO:teuthology.orchestra.run.smithi064.stdout:(50/156): elfutils-libelf-devel-0.190-2.el9.x86 517 kB/s | 21 kB 00:00 2024-05-07T02:13:34.536 INFO:teuthology.orchestra.run.smithi019.stdout:(20/156): python3-ceph-common-19.0.0-3522.gbd4e 1.8 MB/s | 135 kB 00:00 2024-05-07T02:13:34.574 INFO:teuthology.orchestra.run.smithi064.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 4.1 MB/s | 943 kB 00:00 2024-05-07T02:13:34.599 INFO:teuthology.orchestra.run.smithi064.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 4.6 MB/s | 313 kB 00:00 2024-05-07T02:13:34.611 INFO:teuthology.orchestra.run.smithi019.stdout:(21/156): python3-cephfs-19.0.0-3522.gbd4e00a3. 2.1 MB/s | 162 kB 00:00 2024-05-07T02:13:34.617 INFO:teuthology.orchestra.run.smithi064.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 779 kB/s | 33 kB 00:00 2024-05-07T02:13:34.650 INFO:teuthology.orchestra.run.smithi064.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 550 kB/s | 18 kB 00:00 2024-05-07T02:13:34.695 INFO:teuthology.orchestra.run.smithi019.stdout:(22/156): python3-rados-19.0.0-3522.gbd4e00a3.e 3.8 MB/s | 321 kB 00:00 2024-05-07T02:13:34.717 INFO:teuthology.orchestra.run.smithi064.stdout:(55/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 26 MB/s | 3.0 MB 00:00 2024-05-07T02:13:34.734 INFO:teuthology.orchestra.run.smithi064.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 134 kB/s | 11 kB 00:00 2024-05-07T02:13:34.759 INFO:teuthology.orchestra.run.smithi064.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.8 MB/s | 160 kB 00:00 2024-05-07T02:13:34.776 INFO:teuthology.orchestra.run.smithi064.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-05-07T02:13:34.778 INFO:teuthology.orchestra.run.smithi019.stdout:(23/156): python3-rbd-19.0.0-3522.gbd4e00a3.el9 3.5 MB/s | 298 kB 00:00 2024-05-07T02:13:34.818 INFO:teuthology.orchestra.run.smithi064.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-05-07T02:13:34.852 INFO:teuthology.orchestra.run.smithi064.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 7.5 MB/s | 582 kB 00:00 2024-05-07T02:13:34.854 INFO:teuthology.orchestra.run.smithi019.stdout:(24/156): python3-rgw-19.0.0-3522.gbd4e00a3.el9 1.3 MB/s | 100 kB 00:00 2024-05-07T02:13:34.866 INFO:teuthology.orchestra.run.smithi027.stdout:(87/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.4 MB/s | 5.1 MB 00:02 2024-05-07T02:13:34.869 INFO:teuthology.orchestra.run.smithi064.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 508 kB/s | 25 kB 00:00 2024-05-07T02:13:34.886 INFO:teuthology.orchestra.run.smithi064.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-05-07T02:13:34.919 INFO:teuthology.orchestra.run.smithi064.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-05-07T02:13:34.937 INFO:teuthology.orchestra.run.smithi019.stdout:(25/156): rbd-fuse-19.0.0-3522.gbd4e00a3.el9.x8 1.0 MB/s | 85 kB 00:00 2024-05-07T02:13:34.945 INFO:teuthology.orchestra.run.smithi064.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-05-07T02:13:34.967 INFO:teuthology.orchestra.run.smithi027.stdout:(88/156): python3-pytz-2021.1-5.el9.noarch.rpm 510 kB/s | 51 kB 00:00 2024-05-07T02:13:34.970 INFO:teuthology.orchestra.run.smithi064.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 702 kB/s | 35 kB 00:00 2024-05-07T02:13:35.000 INFO:teuthology.orchestra.run.smithi027.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 576 kB/s | 279 kB 00:00 2024-05-07T02:13:35.037 INFO:teuthology.orchestra.run.smithi064.stdout:(66/156): python3-cryptography-36.0.1-4.el9.x86 1.2 MB/s | 1.2 MB 00:01 2024-05-07T02:13:35.067 INFO:teuthology.orchestra.run.smithi027.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 536 kB/s | 54 kB 00:00 2024-05-07T02:13:35.079 INFO:teuthology.orchestra.run.smithi019.stdout:(26/156): librgw2-19.0.0-3522.gbd4e00a3.el9.x86 7.1 MB/s | 5.2 MB 00:00 2024-05-07T02:13:35.112 INFO:teuthology.orchestra.run.smithi064.stdout:(67/156): openblas-openmp-0.3.21-2.el9.x86_64.r 28 MB/s | 4.7 MB 00:00 2024-05-07T02:13:35.138 INFO:teuthology.orchestra.run.smithi019.stdout:(27/156): rbd-nbd-19.0.0-3522.gbd4e00a3.el9.x86 2.8 MB/s | 170 kB 00:00 2024-05-07T02:13:35.142 INFO:teuthology.orchestra.run.smithi027.stdout:(91/156): postgresql-server-13.14-1.el9.x86_64. 1.0 MB/s | 5.8 MB 00:05 2024-05-07T02:13:35.168 INFO:teuthology.orchestra.run.smithi027.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 415 kB/s | 42 kB 00:00 2024-05-07T02:13:35.179 INFO:teuthology.orchestra.run.smithi064.stdout:(68/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 21 MB/s | 4.4 MB 00:00 2024-05-07T02:13:35.197 INFO:teuthology.orchestra.run.smithi019.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3522.g 594 kB/s | 34 kB 00:00 2024-05-07T02:13:35.221 INFO:teuthology.orchestra.run.smithi064.stdout:(69/156): postgresql-contrib-13.14-1.el9.x86_64 8.2 MB/s | 911 kB 00:00 2024-05-07T02:13:35.243 INFO:teuthology.orchestra.run.smithi027.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 441 kB/s | 44 kB 00:00 2024-05-07T02:13:35.246 INFO:teuthology.orchestra.run.smithi064.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-05-07T02:13:35.247 INFO:teuthology.orchestra.run.smithi019.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3522.gbd4e00a 2.8 MB/s | 142 kB 00:00 2024-05-07T02:13:35.277 INFO:teuthology.orchestra.run.smithi027.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.0 MB/s | 221 kB 00:00 2024-05-07T02:13:35.305 INFO:teuthology.orchestra.run.smithi064.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2024-05-07T02:13:35.344 INFO:teuthology.orchestra.run.smithi027.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 357 kB/s | 36 kB 00:00 2024-05-07T02:13:35.364 INFO:teuthology.orchestra.run.smithi019.stdout:(30/156): rbd-mirror-19.0.0-3522.gbd4e00a3.el9. 7.2 MB/s | 3.0 MB 00:00 2024-05-07T02:13:35.377 INFO:teuthology.orchestra.run.smithi027.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 474 kB/s | 47 kB 00:00 2024-05-07T02:13:35.380 INFO:teuthology.orchestra.run.smithi064.stdout:(72/156): postgresql-server-13.14-1.el9.x86_64. 36 MB/s | 5.8 MB 00:00 2024-05-07T02:13:35.405 INFO:teuthology.orchestra.run.smithi064.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 875 kB/s | 87 kB 00:00 2024-05-07T02:13:35.464 INFO:teuthology.orchestra.run.smithi064.stdout:(74/156): postgresql-13.14-1.el9.x86_64.rpm 3.7 MB/s | 1.6 MB 00:00 2024-05-07T02:13:35.478 INFO:teuthology.orchestra.run.smithi027.stdout:(97/156): uuid-1.6.2-55.el9.x86_64.rpm 577 kB/s | 58 kB 00:00 2024-05-07T02:13:35.489 INFO:teuthology.orchestra.run.smithi064.stdout:(75/156): python3-devel-3.9.18-3.el9.x86_64.rpm 2.8 MB/s | 244 kB 00:00 2024-05-07T02:13:35.515 INFO:teuthology.orchestra.run.smithi064.stdout:(76/156): python3-jinja2-2.11.3-5.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-05-07T02:13:35.540 INFO:teuthology.orchestra.run.smithi064.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-05-07T02:13:35.565 INFO:teuthology.orchestra.run.smithi064.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-05-07T02:13:35.586 INFO:teuthology.orchestra.run.smithi027.stdout:(98/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 866 kB/s | 94 kB 00:00 2024-05-07T02:13:35.687 INFO:teuthology.orchestra.run.smithi027.stdout:(99/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 637 kB/s | 64 kB 00:00 2024-05-07T02:13:35.707 INFO:teuthology.orchestra.run.smithi064.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.0 MB/s | 172 kB 00:00 2024-05-07T02:13:35.723 INFO:teuthology.orchestra.run.smithi019.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3522.gbd4e0 7.4 MB/s | 3.5 MB 00:00 2024-05-07T02:13:35.732 INFO:teuthology.orchestra.run.smithi064.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 208 kB/s | 35 kB 00:00 2024-05-07T02:13:35.774 INFO:teuthology.orchestra.run.smithi064.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.9 MB/s | 250 kB 00:00 2024-05-07T02:13:35.787 INFO:teuthology.orchestra.run.smithi027.stdout:(100/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 449 kB/s | 45 kB 00:00 2024-05-07T02:13:35.812 INFO:teuthology.orchestra.run.smithi027.stdout:(101/156): socat-1.7.4.1-5.el9.x86_64.rpm 651 kB/s | 305 kB 00:00 2024-05-07T02:13:35.858 INFO:teuthology.orchestra.run.smithi064.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 12 MB/s | 6.0 MB 00:00 2024-05-07T02:13:35.874 INFO:teuthology.orchestra.run.smithi019.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3522.gbd 1.8 MB/s | 274 kB 00:00 2024-05-07T02:13:35.879 INFO:teuthology.orchestra.run.smithi027.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 242 kB/s | 22 kB 00:00 2024-05-07T02:13:35.925 INFO:teuthology.orchestra.run.smithi064.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 23 MB/s | 5.1 MB 00:00 2024-05-07T02:13:35.931 INFO:teuthology.orchestra.run.smithi019.stdout:(33/156): ceph-mgr-rook-19.0.0-3522.gbd4e00a3.e 870 kB/s | 49 kB 00:00 2024-05-07T02:13:35.950 INFO:teuthology.orchestra.run.smithi064.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 844 kB/s | 77 kB 00:00 2024-05-07T02:13:35.976 INFO:teuthology.orchestra.run.smithi064.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 846 kB/s | 42 kB 00:00 2024-05-07T02:13:35.980 INFO:teuthology.orchestra.run.smithi027.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 5.0 MB/s | 863 kB 00:00 2024-05-07T02:13:35.990 INFO:teuthology.orchestra.run.smithi019.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3522.gb 283 kB/s | 16 kB 00:00 2024-05-07T02:13:36.001 INFO:teuthology.orchestra.run.smithi064.stdout:(86/156): python3-oauthlib-3.1.1-5.el9.noarch.r 983 kB/s | 222 kB 00:00 2024-05-07T02:13:36.026 INFO:teuthology.orchestra.run.smithi064.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-05-07T02:13:36.051 INFO:teuthology.orchestra.run.smithi064.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 269 kB 00:00 2024-05-07T02:13:36.077 INFO:teuthology.orchestra.run.smithi064.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-07T02:13:36.097 INFO:teuthology.orchestra.run.smithi019.stdout:(35/156): ceph-volume-19.0.0-3522.gbd4e00a3.el9 2.5 MB/s | 273 kB 00:00 2024-05-07T02:13:36.102 INFO:teuthology.orchestra.run.smithi064.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-05-07T02:13:36.127 INFO:teuthology.orchestra.run.smithi064.stdout:(91/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.2 MB/s | 279 kB 00:00 2024-05-07T02:13:36.153 INFO:teuthology.orchestra.run.smithi064.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 827 kB/s | 42 kB 00:00 2024-05-07T02:13:36.178 INFO:teuthology.orchestra.run.smithi064.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 882 kB/s | 44 kB 00:00 2024-05-07T02:13:36.203 INFO:teuthology.orchestra.run.smithi064.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-05-07T02:13:36.216 INFO:teuthology.orchestra.run.smithi019.stdout:(36/156): ceph-mgr-diskprediction-local-19.0.0- 8.7 MB/s | 7.4 MB 00:00 2024-05-07T02:13:36.220 INFO:teuthology.orchestra.run.smithi064.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 861 kB/s | 36 kB 00:00 2024-05-07T02:13:36.237 INFO:teuthology.orchestra.run.smithi064.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-05-07T02:13:36.262 INFO:teuthology.orchestra.run.smithi064.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 7.2 MB/s | 305 kB 00:00 2024-05-07T02:13:36.266 INFO:teuthology.orchestra.run.smithi019.stdout:(37/156): cephadm-19.0.0-3522.gbd4e00a3.el9.noa 4.4 MB/s | 763 kB 00:00 2024-05-07T02:13:36.279 INFO:teuthology.orchestra.run.smithi064.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.3 MB/s | 58 kB 00:00 2024-05-07T02:13:36.283 INFO:teuthology.orchestra.run.smithi019.stdout:(38/156): cephfs-top-19.0.0-3522.gbd4e00a3.el9. 274 kB/s | 18 kB 00:00 2024-05-07T02:13:36.304 INFO:teuthology.orchestra.run.smithi064.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 2.2 MB/s | 94 kB 00:00 2024-05-07T02:13:36.321 INFO:teuthology.orchestra.run.smithi064.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-05-07T02:13:36.338 INFO:teuthology.orchestra.run.smithi064.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-05-07T02:13:36.355 INFO:teuthology.orchestra.run.smithi064.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 665 kB/s | 22 kB 00:00 2024-05-07T02:13:36.406 INFO:teuthology.orchestra.run.smithi064.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 13 MB/s | 863 kB 00:00 2024-05-07T02:13:36.438 INFO:teuthology.orchestra.run.smithi027.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 981 kB/s | 548 kB 00:00 2024-05-07T02:13:36.463 INFO:teuthology.orchestra.run.smithi027.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 639 kB/s | 309 kB 00:00 2024-05-07T02:13:36.539 INFO:teuthology.orchestra.run.smithi027.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 194 kB/s | 19 kB 00:00 2024-05-07T02:13:36.639 INFO:teuthology.orchestra.run.smithi027.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 260 kB/s | 26 kB 00:00 2024-05-07T02:13:36.664 INFO:teuthology.orchestra.run.smithi064.stdout:(104/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-05-07T02:13:36.717 INFO:teuthology.orchestra.run.smithi019.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 178 kB/s | 80 kB 00:00 2024-05-07T02:13:36.731 INFO:teuthology.orchestra.run.smithi027.stdout:(108/156): liboath-2.6.7-2.el9.x86_64.rpm 530 kB/s | 49 kB 00:00 2024-05-07T02:13:36.831 INFO:teuthology.orchestra.run.smithi027.stdout:(109/156): libunwind-1.6.2-1.el9.x86_64.rpm 674 kB/s | 67 kB 00:00 2024-05-07T02:13:36.890 INFO:teuthology.orchestra.run.smithi027.stdout:(110/156): libarrow-9.0.0-11.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-05-07T02:13:36.932 INFO:teuthology.orchestra.run.smithi027.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-05-07T02:13:36.984 INFO:teuthology.orchestra.run.smithi019.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 735 kB/s | 195 kB 00:00 2024-05-07T02:13:36.999 INFO:teuthology.orchestra.run.smithi027.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.5 MB/s | 839 kB 00:00 2024-05-07T02:13:37.014 INFO:teuthology.orchestra.run.smithi064.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 56 kB/s | 19 kB 00:00 2024-05-07T02:13:37.032 INFO:teuthology.orchestra.run.smithi027.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 5.4 MB/s | 548 kB 00:00 2024-05-07T02:13:37.067 INFO:teuthology.orchestra.run.smithi019.stdout:(41/156): libgfortran-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 803 kB 00:00 2024-05-07T02:13:37.091 INFO:teuthology.orchestra.run.smithi027.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 472 kB/s | 43 kB 00:00 2024-05-07T02:13:37.123 INFO:teuthology.orchestra.run.smithi064.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 431 kB/s | 309 kB 00:00 2024-05-07T02:13:37.124 INFO:teuthology.orchestra.run.smithi027.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 350 kB/s | 32 kB 00:00 2024-05-07T02:13:37.183 INFO:teuthology.orchestra.run.smithi027.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 163 kB/s | 15 kB 00:00 2024-05-07T02:13:37.192 INFO:teuthology.orchestra.run.smithi019.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-05-07T02:13:37.217 INFO:teuthology.orchestra.run.smithi027.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-05-07T02:13:37.223 INFO:teuthology.orchestra.run.smithi064.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 260 kB/s | 26 kB 00:00 2024-05-07T02:13:37.284 INFO:teuthology.orchestra.run.smithi027.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-05-07T02:13:37.301 INFO:teuthology.orchestra.run.smithi019.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 991 kB/s | 106 kB 00:00 2024-05-07T02:13:37.317 INFO:teuthology.orchestra.run.smithi027.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.9 MB/s | 303 kB 00:00 2024-05-07T02:13:37.323 INFO:teuthology.orchestra.run.smithi064.stdout:(108/156): abseil-cpp-20211102.0-3.el9.x86_64.r 566 kB/s | 548 kB 00:00 2024-05-07T02:13:37.349 INFO:teuthology.orchestra.run.smithi064.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 388 kB/s | 49 kB 00:00 2024-05-07T02:13:37.360 INFO:teuthology.orchestra.run.smithi019.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 4.2 MB/s | 1.2 MB 00:00 2024-05-07T02:13:37.376 INFO:teuthology.orchestra.run.smithi027.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 384 kB/s | 35 kB 00:00 2024-05-07T02:13:37.410 INFO:teuthology.orchestra.run.smithi019.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-05-07T02:13:37.418 INFO:teuthology.orchestra.run.smithi027.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 2.0 MB/s | 206 kB 00:00 2024-05-07T02:13:37.441 INFO:teuthology.orchestra.run.smithi064.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 577 kB/s | 67 kB 00:00 2024-05-07T02:13:37.466 INFO:teuthology.orchestra.run.smithi064.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-05-07T02:13:37.469 INFO:teuthology.orchestra.run.smithi019.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-05-07T02:13:37.518 INFO:teuthology.orchestra.run.smithi027.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-05-07T02:13:37.527 INFO:teuthology.orchestra.run.smithi019.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 1.8 MB/s | 215 kB 00:00 2024-05-07T02:13:37.619 INFO:teuthology.orchestra.run.smithi027.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 107 kB/s | 11 kB 00:00 2024-05-07T02:13:37.641 INFO:teuthology.orchestra.run.smithi064.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.1 MB/s | 839 kB 00:00 2024-05-07T02:13:37.653 INFO:teuthology.orchestra.run.smithi019.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 3.7 MB/s | 689 kB 00:00 2024-05-07T02:13:37.691 INFO:teuthology.orchestra.run.smithi064.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 2.4 MB/s | 548 kB 00:00 2024-05-07T02:13:37.694 INFO:teuthology.orchestra.run.smithi019.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 2.6 MB/s | 107 kB 00:00 2024-05-07T02:13:37.711 INFO:teuthology.orchestra.run.smithi027.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 193 kB/s | 18 kB 00:00 2024-05-07T02:13:37.753 INFO:teuthology.orchestra.run.smithi019.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 4.1 MB/s | 943 kB 00:00 2024-05-07T02:13:37.779 INFO:teuthology.orchestra.run.smithi019.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 260 kB/s | 21 kB 00:00 2024-05-07T02:13:37.792 INFO:teuthology.orchestra.run.smithi064.stdout:(114/156): libarrow-9.0.0-11.el9.x86_64.rpm 5.7 MB/s | 4.4 MB 00:00 2024-05-07T02:13:37.803 INFO:teuthology.orchestra.run.smithi027.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 253 kB/s | 23 kB 00:00 2024-05-07T02:13:37.812 INFO:teuthology.orchestra.run.smithi019.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 313 kB 00:00 2024-05-07T02:13:37.817 INFO:teuthology.orchestra.run.smithi064.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 248 kB/s | 43 kB 00:00 2024-05-07T02:13:37.837 INFO:teuthology.orchestra.run.smithi019.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 567 kB/s | 33 kB 00:00 2024-05-07T02:13:37.851 INFO:teuthology.orchestra.run.smithi064.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 202 kB/s | 32 kB 00:00 2024-05-07T02:13:37.879 INFO:teuthology.orchestra.run.smithi019.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 448 kB/s | 18 kB 00:00 2024-05-07T02:13:37.895 INFO:teuthology.orchestra.run.smithi027.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 212 kB/s | 19 kB 00:00 2024-05-07T02:13:37.921 INFO:teuthology.orchestra.run.smithi019.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 274 kB/s | 11 kB 00:00 2024-05-07T02:13:37.926 INFO:teuthology.orchestra.run.smithi064.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-05-07T02:13:37.951 INFO:teuthology.orchestra.run.smithi064.stdout:(118/156): python3-certifi-2021.10.8-2.el9.noar 94 kB/s | 15 kB 00:00 2024-05-07T02:13:37.971 INFO:teuthology.orchestra.run.smithi019.stdout:(56/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.2 MB/s | 160 kB 00:00 2024-05-07T02:13:37.987 INFO:teuthology.orchestra.run.smithi027.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 215 kB/s | 20 kB 00:00 2024-05-07T02:13:38.027 INFO:teuthology.orchestra.run.smithi064.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 2.0 MB/s | 358 kB 00:00 2024-05-07T02:13:38.052 INFO:teuthology.orchestra.run.smithi064.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.4 MB/s | 303 kB 00:00 2024-05-07T02:13:38.063 INFO:teuthology.orchestra.run.smithi019.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-05-07T02:13:38.086 INFO:teuthology.orchestra.run.smithi064.stdout:(121/156): python3-colorama-0.4.4-12.el9.noarch 263 kB/s | 35 kB 00:00 2024-05-07T02:13:38.089 INFO:teuthology.orchestra.run.smithi019.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 389 kB/s | 45 kB 00:00 2024-05-07T02:13:38.095 INFO:teuthology.orchestra.run.smithi027.stdout:(128/156): python3-kubernetes-26.1.0-2.el9.noar 9.4 MB/s | 1.0 MB 00:00 2024-05-07T02:13:38.139 INFO:teuthology.orchestra.run.smithi019.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 8.7 MB/s | 662 kB 00:00 2024-05-07T02:13:38.169 INFO:teuthology.orchestra.run.smithi064.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 1.4 MB/s | 206 kB 00:00 2024-05-07T02:13:38.181 INFO:teuthology.orchestra.run.smithi019.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 6.2 MB/s | 582 kB 00:00 2024-05-07T02:13:38.187 INFO:teuthology.orchestra.run.smithi027.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 505 kB/s | 46 kB 00:00 2024-05-07T02:13:38.206 INFO:teuthology.orchestra.run.smithi019.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 384 kB/s | 25 kB 00:00 2024-05-07T02:13:38.211 INFO:teuthology.orchestra.run.smithi064.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-05-07T02:13:38.232 INFO:teuthology.orchestra.run.smithi019.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 963 kB/s | 47 kB 00:00 2024-05-07T02:13:38.253 INFO:teuthology.orchestra.run.smithi064.stdout:(124/156): python3-grpcio-1.46.7-10.el9.x86_64. 10 MB/s | 2.0 MB 00:00 2024-05-07T02:13:38.278 INFO:teuthology.orchestra.run.smithi064.stdout:(125/156): python3-jaraco-8.2.1-3.el9.noarch.rp 98 kB/s | 11 kB 00:00 2024-05-07T02:13:38.287 INFO:teuthology.orchestra.run.smithi027.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 789 kB/s | 79 kB 00:00 2024-05-07T02:13:38.290 INFO:teuthology.orchestra.run.smithi019.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.5 MB/s | 292 kB 00:00 2024-05-07T02:13:38.312 INFO:teuthology.orchestra.run.smithi064.stdout:(126/156): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-05-07T02:13:38.324 INFO:teuthology.orchestra.run.smithi019.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-05-07T02:13:38.349 INFO:teuthology.orchestra.run.smithi019.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 611 kB/s | 35 kB 00:00 2024-05-07T02:13:38.354 INFO:teuthology.orchestra.run.smithi064.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 230 kB/s | 23 kB 00:00 2024-05-07T02:13:38.379 INFO:teuthology.orchestra.run.smithi064.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 193 kB/s | 19 kB 00:00 2024-05-07T02:13:38.380 INFO:teuthology.orchestra.run.smithi027.stdout:(131/156): python3-grpcio-1.46.7-10.el9.x86_64. 2.0 MB/s | 2.0 MB 00:01 2024-05-07T02:13:38.405 INFO:teuthology.orchestra.run.smithi027.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 493 kB/s | 58 kB 00:00 2024-05-07T02:13:38.413 INFO:teuthology.orchestra.run.smithi064.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 196 kB/s | 20 kB 00:00 2024-05-07T02:13:38.480 INFO:teuthology.orchestra.run.smithi064.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 8.2 MB/s | 1.0 MB 00:00 2024-05-07T02:13:38.488 INFO:teuthology.orchestra.run.smithi027.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.5 MB/s | 272 kB 00:00 2024-05-07T02:13:38.505 INFO:teuthology.orchestra.run.smithi064.stdout:(131/156): python3-logutils-0.3.5-21.el9.noarch 369 kB/s | 46 kB 00:00 2024-05-07T02:13:38.514 INFO:teuthology.orchestra.run.smithi027.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 151 kB/s | 16 kB 00:00 2024-05-07T02:13:38.524 INFO:teuthology.orchestra.run.smithi019.stdout:(66/156): openblas-openmp-0.3.21-2.el9.x86_64.r 24 MB/s | 4.7 MB 00:00 2024-05-07T02:13:38.530 INFO:teuthology.orchestra.run.smithi064.stdout:(132/156): python3-more-itertools-8.12.0-2.el9. 672 kB/s | 79 kB 00:00 2024-05-07T02:13:38.589 INFO:teuthology.orchestra.run.smithi027.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 898 kB/s | 90 kB 00:00 2024-05-07T02:13:38.597 INFO:teuthology.orchestra.run.smithi064.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 493 kB/s | 58 kB 00:00 2024-05-07T02:13:38.614 INFO:teuthology.orchestra.run.smithi027.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 282 kB/s | 28 kB 00:00 2024-05-07T02:13:38.622 INFO:teuthology.orchestra.run.smithi064.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-05-07T02:13:38.625 INFO:teuthology.orchestra.run.smithi019.stdout:(67/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-05-07T02:13:38.648 INFO:teuthology.orchestra.run.smithi064.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 140 kB/s | 16 kB 00:00 2024-05-07T02:13:38.681 INFO:teuthology.orchestra.run.smithi027.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 334 kB/s | 31 kB 00:00 2024-05-07T02:13:38.700 INFO:teuthology.orchestra.run.smithi019.stdout:(68/156): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-05-07T02:13:38.715 INFO:teuthology.orchestra.run.smithi027.stdout:(138/156): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-05-07T02:13:38.739 INFO:teuthology.orchestra.run.smithi064.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 633 kB/s | 90 kB 00:00 2024-05-07T02:13:38.759 INFO:teuthology.orchestra.run.smithi019.stdout:(69/156): postgresql-13.14-1.el9.x86_64.rpm 6.8 MB/s | 1.6 MB 00:00 2024-05-07T02:13:38.765 INFO:teuthology.orchestra.run.smithi064.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 199 kB/s | 28 kB 00:00 2024-05-07T02:13:38.765 INFO:teuthology.orchestra.run.smithi027.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 706 kB/s | 59 kB 00:00 2024-05-07T02:13:38.790 INFO:teuthology.orchestra.run.smithi064.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 216 kB/s | 31 kB 00:00 2024-05-07T02:13:38.792 INFO:teuthology.orchestra.run.smithi019.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-05-07T02:13:38.799 INFO:teuthology.orchestra.run.smithi027.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 428 kB/s | 36 kB 00:00 2024-05-07T02:13:38.840 INFO:teuthology.orchestra.run.smithi064.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-05-07T02:13:38.849 INFO:teuthology.orchestra.run.smithi027.stdout:(141/156): python3-typing-extensions-4.4.0-2.el 610 kB/s | 51 kB 00:00 2024-05-07T02:13:38.876 INFO:teuthology.orchestra.run.smithi019.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-05-07T02:13:38.890 INFO:teuthology.orchestra.run.smithi064.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 471 kB/s | 59 kB 00:00 2024-05-07T02:13:38.899 INFO:teuthology.orchestra.run.smithi027.stdout:(142/156): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-05-07T02:13:38.916 INFO:teuthology.orchestra.run.smithi064.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 285 kB/s | 36 kB 00:00 2024-05-07T02:13:38.918 INFO:teuthology.orchestra.run.smithi019.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 2.1 MB/s | 87 kB 00:00 2024-05-07T02:13:38.941 INFO:teuthology.orchestra.run.smithi064.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 507 kB/s | 51 kB 00:00 2024-05-07T02:13:38.942 INFO:teuthology.orchestra.run.smithi027.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 976 kB/s | 90 kB 00:00 2024-05-07T02:13:39.000 INFO:teuthology.orchestra.run.smithi027.stdout:(144/156): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-05-07T02:13:39.008 INFO:teuthology.orchestra.run.smithi064.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-05-07T02:13:39.033 INFO:teuthology.orchestra.run.smithi064.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 767 kB/s | 90 kB 00:00 2024-05-07T02:13:39.034 INFO:teuthology.orchestra.run.smithi027.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 217 kB/s | 20 kB 00:00 2024-05-07T02:13:39.086 INFO:teuthology.orchestra.run.smithi064.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-05-07T02:13:39.101 INFO:teuthology.orchestra.run.smithi027.stdout:(146/156): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-05-07T02:13:39.117 INFO:teuthology.orchestra.run.smithi064.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 184 kB/s | 20 kB 00:00 2024-05-07T02:13:39.142 INFO:teuthology.orchestra.run.smithi064.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-05-07T02:13:39.143 INFO:teuthology.orchestra.run.smithi019.stdout:(73/156): postgresql-server-13.14-1.el9.x86_64. 15 MB/s | 5.8 MB 00:00 2024-05-07T02:13:39.194 INFO:teuthology.orchestra.run.smithi019.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 4.9 MB/s | 244 kB 00:00 2024-05-07T02:13:39.243 INFO:teuthology.orchestra.run.smithi064.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-05-07T02:13:39.244 INFO:teuthology.orchestra.run.smithi019.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 5.0 MB/s | 248 kB 00:00 2024-05-07T02:13:39.286 INFO:teuthology.orchestra.run.smithi019.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 632 kB/s | 26 kB 00:00 2024-05-07T02:13:39.468 INFO:teuthology.orchestra.run.smithi064.stdout:(149/156): librbd1-19.0.0-3522.gbd4e00a3.el9.x8 9.5 MB/s | 3.1 MB 00:00 2024-05-07T02:13:39.701 INFO:teuthology.orchestra.run.smithi027.stdout:(147/156): librados2-19.0.0-3522.gbd4e00a3.el9. 5.5 MB/s | 3.3 MB 00:00 2024-05-07T02:13:39.735 INFO:teuthology.orchestra.run.smithi027.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 2.3 MB/s | 1.6 MB 00:00 2024-05-07T02:13:39.744 INFO:teuthology.orchestra.run.smithi019.stdout:(77/156): ceph-test-19.0.0-3522.gbd4e00a3.el9.x 6.1 MB/s | 43 MB 00:07 2024-05-07T02:13:39.770 INFO:teuthology.orchestra.run.smithi019.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 40 kB/s | 19 kB 00:00 2024-05-07T02:13:39.894 INFO:teuthology.orchestra.run.smithi064.stdout:(150/156): librados2-19.0.0-3522.gbd4e00a3.el9. 4.3 MB/s | 3.3 MB 00:00 2024-05-07T02:13:39.902 INFO:teuthology.orchestra.run.smithi027.stdout:(149/156): librbd1-19.0.0-3522.gbd4e00a3.el9.x8 15 MB/s | 3.1 MB 00:00 2024-05-07T02:13:39.904 INFO:teuthology.orchestra.run.smithi019.stdout:(79/156): python3-babel-2.9.1-2.el9.noarch.rpm 6.1 MB/s | 6.0 MB 00:00 2024-05-07T02:13:39.929 INFO:teuthology.orchestra.run.smithi019.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 220 kB/s | 35 kB 00:00 2024-05-07T02:13:39.978 INFO:teuthology.orchestra.run.smithi019.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.0 MB/s | 250 kB 00:00 2024-05-07T02:13:40.004 INFO:teuthology.orchestra.run.smithi019.stdout:(82/156): python3-mako-1.1.4-6.el9.noarch.rpm 667 kB/s | 172 kB 00:00 2024-05-07T02:13:40.038 INFO:teuthology.orchestra.run.smithi019.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-05-07T02:13:40.063 INFO:teuthology.orchestra.run.smithi019.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-05-07T02:13:40.088 INFO:teuthology.orchestra.run.smithi019.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 856 kB/s | 42 kB 00:00 2024-05-07T02:13:40.130 INFO:teuthology.orchestra.run.smithi019.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 4.0 MB/s | 269 kB 00:00 2024-05-07T02:13:40.164 INFO:teuthology.orchestra.run.smithi019.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-05-07T02:13:40.198 INFO:teuthology.orchestra.run.smithi019.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-05-07T02:13:40.223 INFO:teuthology.orchestra.run.smithi019.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 881 kB/s | 51 kB 00:00 2024-05-07T02:13:40.244 INFO:teuthology.orchestra.run.smithi064.stdout:(151/156): NetworkManager-1.47.5-1.el9.x86_64.r 2.3 MB/s | 2.3 MB 00:01 2024-05-07T02:13:40.249 INFO:teuthology.orchestra.run.smithi019.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-05-07T02:13:40.290 INFO:teuthology.orchestra.run.smithi019.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-05-07T02:13:40.332 INFO:teuthology.orchestra.run.smithi019.stdout:(92/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.1 MB/s | 44 kB 00:00 2024-05-07T02:13:40.394 INFO:teuthology.orchestra.run.smithi064.stdout:(152/156): NetworkManager-team-1.47.5-1.el9.x86 83 kB/s | 42 kB 00:00 2024-05-07T02:13:40.419 INFO:teuthology.orchestra.run.smithi064.stdout:(153/156): NetworkManager-tui-1.47.5-1.el9.x86_ 1.4 MB/s | 248 kB 00:00 2024-05-07T02:13:40.461 INFO:teuthology.orchestra.run.smithi064.stdout:(154/156): NetworkManager-libnm-1.47.5-1.el9.x8 1.8 MB/s | 1.8 MB 00:00 2024-05-07T02:13:40.482 INFO:teuthology.orchestra.run.smithi019.stdout:(93/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 8.8 MB/s | 5.1 MB 00:00 2024-05-07T02:13:40.516 INFO:teuthology.orchestra.run.smithi019.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 1.2 MB/s | 221 kB 00:00 2024-05-07T02:13:40.541 INFO:teuthology.orchestra.run.smithi019.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 623 kB/s | 36 kB 00:00 2024-05-07T02:13:40.566 INFO:teuthology.orchestra.run.smithi019.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 960 kB/s | 47 kB 00:00 2024-05-07T02:13:40.594 INFO:teuthology.orchestra.run.smithi027.stdout:(150/156): NetworkManager-1.47.5-1.el9.x86_64.r 2.7 MB/s | 2.3 MB 00:00 2024-05-07T02:13:40.600 INFO:teuthology.orchestra.run.smithi019.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 5.2 MB/s | 305 kB 00:00 2024-05-07T02:13:40.625 INFO:teuthology.orchestra.run.smithi019.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 58 kB 00:00 2024-05-07T02:13:40.651 INFO:teuthology.orchestra.run.smithi019.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.9 MB/s | 94 kB 00:00 2024-05-07T02:13:40.676 INFO:teuthology.orchestra.run.smithi019.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2024-05-07T02:13:40.701 INFO:teuthology.orchestra.run.smithi019.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 906 kB/s | 45 kB 00:00 2024-05-07T02:13:40.702 INFO:teuthology.orchestra.run.smithi027.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 385 kB/s | 42 kB 00:00 2024-05-07T02:13:40.811 INFO:teuthology.orchestra.run.smithi027.stdout:(152/156): NetworkManager-tui-1.47.5-1.el9.x86_ 2.2 MB/s | 248 kB 00:00 2024-05-07T02:13:40.936 INFO:teuthology.orchestra.run.smithi027.stdout:(153/156): openssl-3.2.1-1.el9.x86_64.rpm 11 MB/s | 1.3 MB 00:00 2024-05-07T02:13:40.987 INFO:teuthology.orchestra.run.smithi027.stdout:(154/156): NetworkManager-libnm-1.47.5-1.el9.x8 1.7 MB/s | 1.8 MB 00:01 2024-05-07T02:13:40.993 INFO:teuthology.orchestra.run.smithi019.stdout:(102/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 21 MB/s | 16 MB 00:00 2024-05-07T02:13:41.019 INFO:teuthology.orchestra.run.smithi019.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 65 kB/s | 22 kB 00:00 2024-05-07T02:13:41.070 INFO:teuthology.orchestra.run.smithi064.stdout:(155/156): openssl-3.2.1-1.el9.x86_64.rpm 2.0 MB/s | 1.3 MB 00:00 2024-05-07T02:13:41.195 INFO:teuthology.orchestra.run.smithi064.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 3.1 MB/s | 2.4 MB 00:00 2024-05-07T02:13:41.201 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-05-07T02:13:41.201 INFO:teuthology.orchestra.run.smithi064.stdout:Total 13 MB/s | 219 MB 00:17 2024-05-07T02:13:41.436 INFO:teuthology.orchestra.run.smithi019.stdout:(104/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.1 MB/s | 863 kB 00:00 2024-05-07T02:13:41.804 INFO:teuthology.orchestra.run.smithi027.stdout:(155/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.8 MB/s | 2.4 MB 00:00 2024-05-07T02:13:41.919 INFO:teuthology.orchestra.run.smithi019.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 343 kB/s | 309 kB 00:00 2024-05-07T02:13:41.944 INFO:teuthology.orchestra.run.smithi019.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 38 kB/s | 19 kB 00:00 2024-05-07T02:13:42.053 INFO:teuthology.orchestra.run.smithi019.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 242 kB/s | 26 kB 00:00 2024-05-07T02:13:42.245 INFO:teuthology.orchestra.run.smithi019.stdout:(108/156): liboath-2.6.7-2.el9.x86_64.rpm 254 kB/s | 49 kB 00:00 2024-05-07T02:13:42.429 INFO:teuthology.orchestra.run.smithi019.stdout:(109/156): libunwind-1.6.2-1.el9.x86_64.rpm 369 kB/s | 67 kB 00:00 2024-05-07T02:13:42.466 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-05-07T02:13:42.504 INFO:teuthology.orchestra.run.smithi019.stdout:(110/156): abseil-cpp-20211102.0-3.el9.x86_64.r 363 kB/s | 548 kB 00:01 2024-05-07T02:13:42.635 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-05-07T02:13:42.635 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-05-07T02:13:42.696 INFO:teuthology.orchestra.run.smithi019.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 568 kB/s | 151 kB 00:00 2024-05-07T02:13:43.447 INFO:teuthology.orchestra.run.smithi019.stdout:(112/156): python3-asyncssh-2.13.2-5.el9.noarch 731 kB/s | 548 kB 00:00 2024-05-07T02:13:43.555 INFO:teuthology.orchestra.run.smithi019.stdout:(113/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 404 kB/s | 43 kB 00:00 2024-05-07T02:13:43.663 INFO:teuthology.orchestra.run.smithi019.stdout:(114/156): python3-cachetools-4.2.4-1.el9.noarc 299 kB/s | 32 kB 00:00 2024-05-07T02:13:43.764 INFO:teuthology.orchestra.run.smithi019.stdout:(115/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 666 kB/s | 839 kB 00:01 2024-05-07T02:13:43.789 INFO:teuthology.orchestra.run.smithi019.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 120 kB/s | 15 kB 00:00 2024-05-07T02:13:44.106 INFO:teuthology.orchestra.run.smithi019.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 503 kB/s | 172 kB 00:00 2024-05-07T02:13:44.332 INFO:teuthology.orchestra.run.smithi019.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 662 kB/s | 358 kB 00:00 2024-05-07T02:13:44.440 INFO:teuthology.orchestra.run.smithi019.stdout:(119/156): python3-colorama-0.4.4-12.el9.noarch 328 kB/s | 35 kB 00:00 2024-05-07T02:13:44.505 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-05-07T02:13:44.505 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-05-07T02:13:44.582 INFO:teuthology.orchestra.run.smithi019.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 638 kB/s | 303 kB 00:00 2024-05-07T02:13:44.816 INFO:teuthology.orchestra.run.smithi019.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 549 kB/s | 206 kB 00:00 2024-05-07T02:13:45.083 INFO:teuthology.orchestra.run.smithi019.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 543 kB/s | 144 kB 00:00 2024-05-07T02:13:45.200 INFO:teuthology.orchestra.run.smithi019.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 92 kB/s | 11 kB 00:00 2024-05-07T02:13:45.308 INFO:teuthology.orchestra.run.smithi019.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 164 kB/s | 18 kB 00:00 2024-05-07T02:13:45.417 INFO:teuthology.orchestra.run.smithi019.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 215 kB/s | 23 kB 00:00 2024-05-07T02:13:45.526 INFO:teuthology.orchestra.run.smithi019.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 181 kB/s | 19 kB 00:00 2024-05-07T02:13:45.634 INFO:teuthology.orchestra.run.smithi019.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 183 kB/s | 20 kB 00:00 2024-05-07T02:13:46.935 INFO:teuthology.orchestra.run.smithi019.stdout:(128/156): python3-kubernetes-26.1.0-2.el9.noar 805 kB/s | 1.0 MB 00:01 2024-05-07T02:13:47.051 INFO:teuthology.orchestra.run.smithi019.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 400 kB/s | 46 kB 00:00 2024-05-07T02:13:47.110 INFO:teuthology.orchestra.run.smithi019.stdout:(130/156): python3-grpcio-1.46.7-10.el9.x86_64. 826 kB/s | 2.0 MB 00:02 2024-05-07T02:13:47.235 INFO:teuthology.orchestra.run.smithi019.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 431 kB/s | 79 kB 00:00 2024-05-07T02:13:47.361 INFO:teuthology.orchestra.run.smithi019.stdout:(132/156): libarrow-9.0.0-11.el9.x86_64.rpm 831 kB/s | 4.4 MB 00:05 2024-05-07T02:13:47.386 INFO:teuthology.orchestra.run.smithi019.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 210 kB/s | 58 kB 00:00 2024-05-07T02:13:47.461 INFO:teuthology.orchestra.run.smithi019.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 165 kB/s | 16 kB 00:00 2024-05-07T02:13:47.561 INFO:teuthology.orchestra.run.smithi019.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 284 kB/s | 28 kB 00:00 2024-05-07T02:13:47.587 INFO:teuthology.orchestra.run.smithi019.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 450 kB/s | 90 kB 00:00 2024-05-07T02:13:47.662 INFO:teuthology.orchestra.run.smithi019.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 309 kB/s | 31 kB 00:00 2024-05-07T02:13:47.695 INFO:teuthology.orchestra.run.smithi019.stdout:(138/156): python3-pecan-1.4.2-3.el9.noarch.rpm 593 kB/s | 272 kB 00:00 2024-05-07T02:13:47.796 INFO:teuthology.orchestra.run.smithi019.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 360 kB/s | 36 kB 00:00 2024-05-07T02:13:47.837 INFO:teuthology.orchestra.run.smithi019.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 338 kB/s | 59 kB 00:00 2024-05-07T02:13:47.896 INFO:teuthology.orchestra.run.smithi019.stdout:(141/156): python3-typing-extensions-4.4.0-2.el 511 kB/s | 51 kB 00:00 2024-05-07T02:13:47.946 INFO:teuthology.orchestra.run.smithi019.stdout:(142/156): python3-routes-2.5.1-5.el9.noarch.rp 525 kB/s | 188 kB 00:00 2024-05-07T02:13:48.089 INFO:teuthology.orchestra.run.smithi019.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 468 kB/s | 90 kB 00:00 2024-05-07T02:13:48.197 INFO:teuthology.orchestra.run.smithi019.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 185 kB/s | 20 kB 00:00 2024-05-07T02:13:48.223 INFO:teuthology.orchestra.run.smithi019.stdout:(145/156): python3-webob-1.8.7-6.el9.noarch.rpm 598 kB/s | 230 kB 00:00 2024-05-07T02:13:48.465 INFO:teuthology.orchestra.run.smithi019.stdout:(146/156): re2-20211101-3.el9.x86_64.rpm 720 kB/s | 192 kB 00:00 2024-05-07T02:13:48.573 INFO:teuthology.orchestra.run.smithi019.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 683 kB/s | 427 kB 00:00 2024-05-07T02:13:49.098 INFO:teuthology.orchestra.run.smithi019.stdout:(148/156): librados2-19.0.0-3522.gbd4e00a3.el9. 5.3 MB/s | 3.3 MB 00:00 2024-05-07T02:13:49.182 INFO:teuthology.orchestra.run.smithi019.stdout:(149/156): librbd1-19.0.0-3522.gbd4e00a3.el9.x8 5.1 MB/s | 3.1 MB 00:00 2024-05-07T02:13:50.140 INFO:teuthology.orchestra.run.smithi019.stdout:(150/156): NetworkManager-1.47.5-1.el9.x86_64.r 2.2 MB/s | 2.3 MB 00:01 2024-05-07T02:13:50.190 INFO:teuthology.orchestra.run.smithi019.stdout:(151/156): thrift-0.15.0-2.el9.x86_64.rpm 823 kB/s | 1.6 MB 00:01 2024-05-07T02:13:50.258 INFO:teuthology.orchestra.run.smithi019.stdout:(152/156): NetworkManager-team-1.47.5-1.el9.x86 358 kB/s | 42 kB 00:00 2024-05-07T02:13:50.308 INFO:teuthology.orchestra.run.smithi019.stdout:(153/156): NetworkManager-libnm-1.47.5-1.el9.x8 1.6 MB/s | 1.8 MB 00:01 2024-05-07T02:13:50.483 INFO:teuthology.orchestra.run.smithi019.stdout:(154/156): openssl-3.2.1-1.el9.x86_64.rpm 6.0 MB/s | 1.3 MB 00:00 2024-05-07T02:13:50.817 INFO:teuthology.orchestra.run.smithi019.stdout:(155/156): NetworkManager-tui-1.47.5-1.el9.x86_ 397 kB/s | 248 kB 00:00 2024-05-07T02:13:50.908 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-05-07T02:13:51.000 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-07T02:13:51.025 INFO:teuthology.orchestra.run.smithi019.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 3.3 MB/s | 2.4 MB 00:00 2024-05-07T02:13:51.038 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-07T02:13:51.039 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-05-07T02:13:51.039 INFO:teuthology.orchestra.run.smithi019.stdout:Total 9.0 MB/s | 219 MB 00:24 2024-05-07T02:13:51.093 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-07T02:13:51.125 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-07T02:13:51.151 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-07T02:13:51.195 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-07T02:13:51.547 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-07T02:13:51.660 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : librados2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 8/164 2024-05-07T02:13:51.771 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librados2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 8/164 2024-05-07T02:13:51.803 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephfs2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 9/164 2024-05-07T02:13:51.895 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libcephfs2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 9/164 2024-05-07T02:13:51.924 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rados-2:19.0.0-3522.gbd4e00a3.el9.x86_64 10/164 2024-05-07T02:13:52.081 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-07T02:13:52.138 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-07T02:13:52.165 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephsqlite-2:19.0.0-3522.gbd4e00a3.el9.x86_64 13/164 2024-05-07T02:13:52.255 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3522.gbd4e00a3.el9.x86_64 13/164 2024-05-07T02:13:52.282 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libradosstriper1-2:19.0.0-3522.gbd4e00a3.el9.x86 14/164 2024-05-07T02:13:52.328 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3522.gbd4e00a3.el9.x86 14/164 2024-05-07T02:13:52.350 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-07T02:13:52.371 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-07T02:13:52.535 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-07T02:13:52.665 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-07T02:13:52.823 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-07T02:13:52.891 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-07T02:13:52.993 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-07T02:13:53.067 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-07T02:13:53.270 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-07T02:13:53.378 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-07T02:13:53.405 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-07T02:13:53.440 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-07T02:13:53.516 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-07T02:13:53.556 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-07T02:13:53.600 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-07T02:13:53.632 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-07T02:13:53.660 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-07T02:13:53.732 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-07T02:13:53.783 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-07T02:13:53.812 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-07T02:13:53.856 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-07T02:13:53.936 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-07T02:13:53.967 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ceph-common-2:19.0.0-3522.gbd4e00a3.el9. 32/164 2024-05-07T02:13:54.011 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ceph-argparse-2:19.0.0-3522.gbd4e00a3.el 33/164 2024-05-07T02:13:54.039 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cephfs-2:19.0.0-3522.gbd4e00a3.el9.x86_6 34/164 2024-05-07T02:13:54.113 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-07T02:13:54.162 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-07T02:13:54.175 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-05-07T02:13:54.241 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-07T02:13:54.395 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-07T02:13:54.436 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-05-07T02:13:54.436 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-05-07T02:13:54.449 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-07T02:13:55.001 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-07T02:13:55.096 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-07T02:13:55.131 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-07T02:13:55.175 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-07T02:13:55.213 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librados-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_6 44/164 2024-05-07T02:13:55.254 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-07T02:13:55.408 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-07T02:13:55.456 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-07T02:13:55.552 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-07T02:13:55.613 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-07T02:13:55.688 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-07T02:13:55.724 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-07T02:13:55.760 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-07T02:13:55.808 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-07T02:13:55.842 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-07T02:13:55.887 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-07T02:13:55.937 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-07T02:13:56.081 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-07T02:13:56.125 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-07T02:13:56.155 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-07T02:13:56.197 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-07T02:13:56.296 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-07T02:13:56.330 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-07T02:13:56.370 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-07T02:13:56.395 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-07T02:13:56.637 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-07T02:13:56.720 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-07T02:13:56.853 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-07T02:13:56.885 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-07T02:13:56.914 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-07T02:13:56.961 INFO:teuthology.orchestra.run.smithi027.stdout:(156/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 758 kB/s | 16 MB 00:21 2024-05-07T02:13:56.964 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-05-07T02:13:56.964 INFO:teuthology.orchestra.run.smithi027.stdout:Total 6.0 MB/s | 219 MB 00:36 2024-05-07T02:13:56.973 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-07T02:13:57.012 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-07T02:13:57.052 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-07T02:13:57.082 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-07T02:13:57.101 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-05-07T02:13:57.102 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-05-07T02:13:57.117 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-07T02:13:57.150 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-07T02:13:57.192 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-07T02:13:57.226 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-07T02:13:57.317 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-07T02:13:57.365 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-07T02:13:57.413 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-07T02:13:57.488 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-07T02:13:57.781 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-07T02:13:57.874 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-07T02:13:58.114 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-05-07T02:13:58.281 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-05-07T02:13:58.281 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-05-07T02:13:58.645 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-07T02:13:58.686 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-07T02:13:58.719 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-07T02:13:58.827 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-07T02:13:58.887 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-07T02:13:58.977 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-07T02:13:59.013 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-07T02:13:59.199 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-07T02:13:59.227 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-07T02:13:59.307 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-07T02:14:00.004 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-07T02:14:00.113 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-07T02:14:00.157 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-05-07T02:14:00.158 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-05-07T02:14:01.417 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-07T02:14:01.498 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-07T02:14:01.525 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-07T02:14:01.559 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-07T02:14:01.894 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-07T02:14:01.946 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : librbd1-2:19.0.0-3522.gbd4e00a3.el9.x86_64 101/164 2024-05-07T02:14:02.029 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librbd1-2:19.0.0-3522.gbd4e00a3.el9.x86_64 101/164 2024-05-07T02:14:02.059 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 102/164 2024-05-07T02:14:02.096 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-07T02:14:02.613 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-07T02:14:02.640 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librgw2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 105/164 2024-05-07T02:14:02.695 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librgw2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 105/164 2024-05-07T02:14:02.725 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 106/164 2024-05-07T02:14:03.696 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-05-07T02:14:03.782 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-07T02:14:03.852 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-07T02:14:03.976 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-07T02:14:04.040 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-07T02:14:04.076 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-07T02:14:04.175 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-07T02:14:04.619 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-07T02:14:04.849 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : librados2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 8/164 2024-05-07T02:14:05.050 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 8/164 2024-05-07T02:14:05.058 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 107/164 2024-05-07T02:14:05.088 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 107/164 2024-05-07T02:14:05.091 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephfs2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 9/164 2024-05-07T02:14:05.235 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libcephfs2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 9/164 2024-05-07T02:14:05.324 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rados-2:19.0.0-3522.gbd4e00a3.el9.x86_64 10/164 2024-05-07T02:14:05.678 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 107/164 2024-05-07T02:14:05.694 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-07T02:14:05.717 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-base-2:19.0.0-3522.gbd4e00a3.el9.x86_64 108/164 2024-05-07T02:14:05.763 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-07T02:14:05.796 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephsqlite-2:19.0.0-3522.gbd4e00a3.el9.x86_64 13/164 2024-05-07T02:14:05.798 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-base-2:19.0.0-3522.gbd4e00a3.el9.x86_64 108/164 2024-05-07T02:14:05.798 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-07T02:14:05.798 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-07T02:14:05.798 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-07T02:14:05.843 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-selinux-2:19.0.0-3522.gbd4e00a3.el9.x86_64 109/164 2024-05-07T02:14:05.946 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3522.gbd4e00a3.el9.x86_64 13/164 2024-05-07T02:14:05.979 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libradosstriper1-2:19.0.0-3522.gbd4e00a3.el9.x86 14/164 2024-05-07T02:14:06.075 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3522.gbd4e00a3.el9.x86 14/164 2024-05-07T02:14:06.106 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-07T02:14:06.180 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-07T02:14:06.272 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-05-07T02:14:06.358 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-07T02:14:06.396 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-07T02:14:06.459 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-07T02:14:06.489 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-07T02:14:06.492 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-07T02:14:06.518 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-07T02:14:06.561 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-07T02:14:06.811 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-07T02:14:06.914 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-07T02:14:06.970 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-07T02:14:07.043 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : librados2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 8/164 2024-05-07T02:14:07.058 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-07T02:14:07.172 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librados2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 8/164 2024-05-07T02:14:07.203 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephfs2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 9/164 2024-05-07T02:14:07.245 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-07T02:14:07.312 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libcephfs2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 9/164 2024-05-07T02:14:07.322 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-07T02:14:07.341 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rados-2:19.0.0-3522.gbd4e00a3.el9.x86_64 10/164 2024-05-07T02:14:07.496 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-07T02:14:07.555 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-07T02:14:07.582 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephsqlite-2:19.0.0-3522.gbd4e00a3.el9.x86_64 13/164 2024-05-07T02:14:07.655 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-07T02:14:07.672 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3522.gbd4e00a3.el9.x86_64 13/164 2024-05-07T02:14:07.700 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libradosstriper1-2:19.0.0-3522.gbd4e00a3.el9.x86 14/164 2024-05-07T02:14:07.763 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3522.gbd4e00a3.el9.x86 14/164 2024-05-07T02:14:07.776 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-07T02:14:07.797 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-07T02:14:07.868 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-07T02:14:07.904 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-07T02:14:07.959 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-07T02:14:07.985 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-07T02:14:08.081 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-07T02:14:08.112 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-07T02:14:08.134 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-07T02:14:08.202 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-07T02:14:08.263 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-07T02:14:08.282 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-07T02:14:08.303 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-07T02:14:08.351 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-07T02:14:08.426 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-07T02:14:08.453 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-07T02:14:08.499 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-07T02:14:08.537 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-07T02:14:08.552 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-07T02:14:08.594 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-07T02:14:08.708 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-07T02:14:08.752 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-07T02:14:08.766 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ceph-common-2:19.0.0-3522.gbd4e00a3.el9. 32/164 2024-05-07T02:14:08.819 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ceph-argparse-2:19.0.0-3522.gbd4e00a3.el 33/164 2024-05-07T02:14:08.856 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cephfs-2:19.0.0-3522.gbd4e00a3.el9.x86_6 34/164 2024-05-07T02:14:08.868 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-07T02:14:08.906 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-07T02:14:08.941 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-07T02:14:08.980 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-07T02:14:09.017 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-07T02:14:09.037 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-07T02:14:09.058 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-07T02:14:09.101 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-07T02:14:09.134 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-07T02:14:09.150 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-07T02:14:09.170 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-07T02:14:09.242 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-07T02:14:09.302 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-07T02:14:09.331 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-07T02:14:09.339 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-07T02:14:09.366 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-07T02:14:09.403 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-07T02:14:09.453 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-07T02:14:09.494 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ceph-common-2:19.0.0-3522.gbd4e00a3.el9. 32/164 2024-05-07T02:14:09.538 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ceph-argparse-2:19.0.0-3522.gbd4e00a3.el 33/164 2024-05-07T02:14:09.566 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cephfs-2:19.0.0-3522.gbd4e00a3.el9.x86_6 34/164 2024-05-07T02:14:09.648 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-07T02:14:09.698 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-07T02:14:09.774 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-07T02:14:09.922 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-07T02:14:09.976 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-07T02:14:10.114 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-07T02:14:10.232 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-07T02:14:10.278 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-07T02:14:10.325 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-07T02:14:10.365 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librados-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_6 44/164 2024-05-07T02:14:10.427 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-07T02:14:10.536 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-07T02:14:10.616 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-07T02:14:10.639 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-07T02:14:10.650 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-07T02:14:10.693 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-07T02:14:10.705 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-07T02:14:10.731 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librados-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_6 44/164 2024-05-07T02:14:10.764 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-07T02:14:10.827 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-07T02:14:10.901 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-07T02:14:10.919 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-07T02:14:10.984 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-07T02:14:10.995 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-07T02:14:11.031 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-07T02:14:11.079 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-07T02:14:11.141 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-07T02:14:11.178 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-07T02:14:11.214 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-07T02:14:11.250 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-07T02:14:11.252 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-07T02:14:11.288 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-07T02:14:11.298 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-07T02:14:11.345 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-07T02:14:11.365 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-07T02:14:11.387 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-07T02:14:11.427 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-07T02:14:11.431 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-07T02:14:11.474 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-07T02:14:11.619 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-07T02:14:11.638 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-07T02:14:11.662 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-07T02:14:11.692 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-07T02:14:11.694 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-07T02:14:11.730 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-07T02:14:11.734 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-07T02:14:11.785 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-07T02:14:11.849 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-07T02:14:11.892 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-07T02:14:11.914 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-07T02:14:11.940 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-07T02:14:11.965 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-07T02:14:11.990 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-07T02:14:12.057 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-07T02:14:12.069 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-07T02:14:12.120 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-07T02:14:12.156 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-07T02:14:12.188 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-07T02:14:12.198 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-07T02:14:12.231 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-07T02:14:12.267 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-07T02:14:12.302 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-07T02:14:12.342 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-07T02:14:12.381 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-07T02:14:12.397 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-07T02:14:12.411 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-07T02:14:12.445 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-07T02:14:12.479 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-07T02:14:12.521 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-07T02:14:12.554 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-07T02:14:12.576 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-07T02:14:12.618 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-07T02:14:12.646 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-07T02:14:12.655 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-07T02:14:12.690 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-07T02:14:12.702 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-07T02:14:12.733 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-07T02:14:12.751 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-07T02:14:12.781 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-07T02:14:12.804 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-07T02:14:12.854 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-07T02:14:12.893 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-07T02:14:12.902 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-07T02:14:12.946 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-07T02:14:12.951 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-07T02:14:13.028 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-07T02:14:13.091 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-07T02:14:13.240 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-07T02:14:13.304 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-07T02:14:13.347 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-07T02:14:13.419 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-07T02:14:13.570 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-07T02:14:13.644 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-07T02:14:13.709 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-07T02:14:13.756 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-07T02:14:13.790 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-07T02:14:13.893 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-07T02:14:13.950 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-07T02:14:14.041 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-07T02:14:14.075 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-07T02:14:14.252 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-07T02:14:14.273 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-07T02:14:14.353 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-07T02:14:14.625 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-07T02:14:14.676 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-07T02:14:14.709 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-07T02:14:14.836 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-07T02:14:14.905 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-07T02:14:15.015 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-07T02:14:15.041 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-07T02:14:15.060 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-07T02:14:15.168 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-07T02:14:15.305 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-07T02:14:15.341 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-07T02:14:15.448 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-07T02:14:16.341 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-07T02:14:16.474 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-07T02:14:16.513 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-07T02:14:16.554 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-07T02:14:16.582 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-07T02:14:16.615 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-07T02:14:16.950 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-07T02:14:16.995 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : librbd1-2:19.0.0-3522.gbd4e00a3.el9.x86_64 101/164 2024-05-07T02:14:17.095 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librbd1-2:19.0.0-3522.gbd4e00a3.el9.x86_64 101/164 2024-05-07T02:14:17.124 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 102/164 2024-05-07T02:14:17.161 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-07T02:14:17.677 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-07T02:14:17.706 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librgw2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 105/164 2024-05-07T02:14:17.777 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librgw2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 105/164 2024-05-07T02:14:17.807 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 106/164 2024-05-07T02:14:17.930 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3522.gbd4e00a3.el9.x86_64 109/164 2024-05-07T02:14:17.930 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /sys 2024-05-07T02:14:17.930 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /proc 2024-05-07T02:14:17.930 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /mnt 2024-05-07T02:14:17.930 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /var/tmp 2024-05-07T02:14:17.930 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /home 2024-05-07T02:14:17.931 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /root 2024-05-07T02:14:17.931 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /tmp 2024-05-07T02:14:17.931 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-07T02:14:18.108 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-volume-2:19.0.0-3522.gbd4e00a3.el9.noarch 110/164 2024-05-07T02:14:18.124 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-volume-2:19.0.0-3522.gbd4e00a3.el9.noarch 110/164 2024-05-07T02:14:18.124 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-07T02:14:18.124 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-07T02:14:18.124 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-07T02:14:18.187 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-07T02:14:18.283 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-07T02:14:18.317 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-07T02:14:18.353 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-07T02:14:18.747 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-07T02:14:18.787 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : librbd1-2:19.0.0-3522.gbd4e00a3.el9.x86_64 101/164 2024-05-07T02:14:18.910 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librbd1-2:19.0.0-3522.gbd4e00a3.el9.x86_64 101/164 2024-05-07T02:14:18.944 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 102/164 2024-05-07T02:14:19.003 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-07T02:14:19.587 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-07T02:14:19.623 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librgw2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 105/164 2024-05-07T02:14:19.740 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librgw2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 105/164 2024-05-07T02:14:19.761 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-osd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 111/164 2024-05-07T02:14:19.767 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 106/164 2024-05-07T02:14:19.788 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-osd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 111/164 2024-05-07T02:14:19.788 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-07T02:14:19.788 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-07T02:14:19.788 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-07T02:14:19.788 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-07T02:14:19.788 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-07T02:14:20.029 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mds-2:19.0.0-3522.gbd4e00a3.el9.x86_64 112/164 2024-05-07T02:14:20.052 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mds-2:19.0.0-3522.gbd4e00a3.el9.x86_64 112/164 2024-05-07T02:14:20.052 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-07T02:14:20.053 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-07T02:14:20.053 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-07T02:14:20.053 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-07T02:14:20.053 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-07T02:14:20.153 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 107/164 2024-05-07T02:14:20.188 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 107/164 2024-05-07T02:14:20.538 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mon-2:19.0.0-3522.gbd4e00a3.el9.x86_64 113/164 2024-05-07T02:14:20.561 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mon-2:19.0.0-3522.gbd4e00a3.el9.x86_64 113/164 2024-05-07T02:14:20.562 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-07T02:14:20.562 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-07T02:14:20.562 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-07T02:14:20.562 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-07T02:14:20.562 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-07T02:14:20.677 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-07T02:14:20.735 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-07T02:14:20.780 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 107/164 2024-05-07T02:14:20.798 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-07T02:14:20.816 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-base-2:19.0.0-3522.gbd4e00a3.el9.x86_64 108/164 2024-05-07T02:14:20.916 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-base-2:19.0.0-3522.gbd4e00a3.el9.x86_64 108/164 2024-05-07T02:14:20.916 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-07T02:14:20.916 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-07T02:14:20.917 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-07T02:14:20.943 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-selinux-2:19.0.0-3522.gbd4e00a3.el9.x86_64 109/164 2024-05-07T02:14:20.993 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-07T02:14:21.038 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-07T02:14:21.118 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-07T02:14:21.209 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-07T02:14:21.369 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-07T02:14:21.414 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-07T02:14:21.446 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-07T02:14:21.473 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-07T02:14:21.893 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: cephadm-2:19.0.0-3522.gbd4e00a3.el9.noarch 125/164 2024-05-07T02:14:21.921 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephadm-2:19.0.0-3522.gbd4e00a3.el9.noarch 125/164 2024-05-07T02:14:21.954 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3522.gbd4e00a3.e 126/164 2024-05-07T02:14:22.018 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3522.gbd4e00a3. 127/164 2024-05-07T02:14:22.044 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3522.gbd4e00a3.el9.noa 128/164 2024-05-07T02:14:22.517 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 107/164 2024-05-07T02:14:22.556 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 107/164 2024-05-07T02:14:23.141 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3522.gbd4e00a3.el9.noa 128/164 2024-05-07T02:14:23.188 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3522.gbd4e00a3.el9.n 129/164 2024-05-07T02:14:23.281 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 107/164 2024-05-07T02:14:23.327 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-base-2:19.0.0-3522.gbd4e00a3.el9.x86_64 108/164 2024-05-07T02:14:23.482 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-base-2:19.0.0-3522.gbd4e00a3.el9.x86_64 108/164 2024-05-07T02:14:23.482 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-07T02:14:23.482 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-07T02:14:23.483 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:14:23.527 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-selinux-2:19.0.0-3522.gbd4e00a3.el9.x86_64 109/164 2024-05-07T02:14:24.205 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3522.gbd4e00a3.el9.n 129/164 2024-05-07T02:14:24.238 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3522.gbd4 130/164 2024-05-07T02:14:24.356 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3522.gbd4 130/164 2024-05-07T02:14:24.529 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3522.gbd4e00a3.el 131/164 2024-05-07T02:14:24.557 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-2:19.0.0-3522.gbd4e00a3.el9.x86_64 132/164 2024-05-07T02:14:24.581 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3522.gbd4e00a3.el9.x86_64 132/164 2024-05-07T02:14:24.581 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-07T02:14:24.581 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-07T02:14:24.581 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-07T02:14:24.581 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-07T02:14:24.581 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-07T02:14:24.632 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-rook-2:19.0.0-3522.gbd4e00a3.el9.noarch 133/164 2024-05-07T02:14:24.644 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3522.gbd4e00a3.el9.noarch 133/164 2024-05-07T02:14:24.746 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-2:19.0.0-3522.gbd4e00a3.el9.x86_64 134/164 2024-05-07T02:14:24.799 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 135/164 2024-05-07T02:14:25.789 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-07T02:14:25.826 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-radosgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 137/164 2024-05-07T02:14:25.850 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 137/164 2024-05-07T02:14:25.850 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-07T02:14:25.850 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-07T02:14:25.850 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-07T02:14:25.850 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-07T02:14:25.851 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-07T02:14:25.893 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3522.gbd4e0 138/164 2024-05-07T02:14:25.916 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3522.gbd4e0 138/164 2024-05-07T02:14:25.916 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-07T02:14:25.916 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-07T02:14:25.917 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-07T02:14:25.969 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephfs-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 139/164 2024-05-07T02:14:25.993 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 139/164 2024-05-07T02:14:25.993 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-07T02:14:25.993 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-07T02:14:25.993 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-07T02:14:25.993 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-07T02:14:25.993 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-07T02:14:26.320 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 140/164 2024-05-07T02:14:26.344 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 140/164 2024-05-07T02:14:26.344 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-07T02:14:26.344 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-07T02:14:26.344 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-07T02:14:26.344 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-07T02:14:26.344 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-07T02:14:30.857 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-test-2:19.0.0-3522.gbd4e00a3.el9.x86_64 141/164 2024-05-07T02:14:30.899 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 142/164 2024-05-07T02:14:30.948 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-nbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 143/164 2024-05-07T02:14:30.979 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-07T02:14:31.007 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-07T02:14:31.200 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephfs-shell-2:19.0.0-3522.gbd4e00a3.el9.x86_64 146/164 2024-05-07T02:14:31.244 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-07T02:14:31.274 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephfs-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_ 148/164 2024-05-07T02:14:31.304 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-07T02:14:31.358 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-07T02:14:31.993 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-07T02:14:32.031 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-07T02:14:32.051 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-07T02:14:32.964 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephfs-top-2:19.0.0-3522.gbd4e00a3.el9.noarch 152/164 2024-05-07T02:14:32.976 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3522.gbd4e00a3.el9.x86_64 109/164 2024-05-07T02:14:32.976 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /sys 2024-05-07T02:14:32.976 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /proc 2024-05-07T02:14:32.976 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /mnt 2024-05-07T02:14:32.976 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /var/tmp 2024-05-07T02:14:32.976 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /home 2024-05-07T02:14:32.976 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /root 2024-05-07T02:14:32.976 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /tmp 2024-05-07T02:14:32.976 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-07T02:14:33.061 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-07T02:14:33.102 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-07T02:14:33.173 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-volume-2:19.0.0-3522.gbd4e00a3.el9.noarch 110/164 2024-05-07T02:14:33.186 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-07T02:14:33.192 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-volume-2:19.0.0-3522.gbd4e00a3.el9.noarch 110/164 2024-05-07T02:14:33.192 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-07T02:14:33.192 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-07T02:14:33.192 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-07T02:14:33.214 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-07T02:14:33.240 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-07T02:14:33.240 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-07T02:14:33.324 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-07T02:14:33.357 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-07T02:14:33.357 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-07T02:14:33.442 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-07T02:14:33.466 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-07T02:14:33.472 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-07T02:14:33.472 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-07T02:14:33.660 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-07T02:14:33.660 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-07T02:14:33.753 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-07T02:14:33.826 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-07T02:14:34.826 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-osd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 111/164 2024-05-07T02:14:34.852 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-osd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 111/164 2024-05-07T02:14:34.852 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-07T02:14:34.852 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-07T02:14:34.852 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-07T02:14:34.852 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-07T02:14:34.852 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-07T02:14:35.093 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mds-2:19.0.0-3522.gbd4e00a3.el9.x86_64 112/164 2024-05-07T02:14:35.116 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mds-2:19.0.0-3522.gbd4e00a3.el9.x86_64 112/164 2024-05-07T02:14:35.116 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-07T02:14:35.116 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-07T02:14:35.116 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-07T02:14:35.116 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-07T02:14:35.116 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-07T02:14:35.602 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mon-2:19.0.0-3522.gbd4e00a3.el9.x86_64 113/164 2024-05-07T02:14:35.626 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mon-2:19.0.0-3522.gbd4e00a3.el9.x86_64 113/164 2024-05-07T02:14:35.626 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-07T02:14:35.626 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-07T02:14:35.626 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-07T02:14:35.626 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-07T02:14:35.626 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-07T02:14:35.741 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-07T02:14:35.800 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-07T02:14:35.863 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-07T02:14:36.058 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-07T02:14:36.112 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-07T02:14:36.194 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-07T02:14:36.289 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-07T02:14:36.440 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-07T02:14:36.487 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-07T02:14:36.519 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-07T02:14:36.546 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-07T02:14:36.958 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: cephadm-2:19.0.0-3522.gbd4e00a3.el9.noarch 125/164 2024-05-07T02:14:36.986 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephadm-2:19.0.0-3522.gbd4e00a3.el9.noarch 125/164 2024-05-07T02:14:37.018 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3522.gbd4e00a3.e 126/164 2024-05-07T02:14:37.054 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3522.gbd4e00a3.el9.x86_64 109/164 2024-05-07T02:14:37.054 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /sys 2024-05-07T02:14:37.054 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /proc 2024-05-07T02:14:37.054 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /mnt 2024-05-07T02:14:37.054 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /var/tmp 2024-05-07T02:14:37.055 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /home 2024-05-07T02:14:37.055 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /root 2024-05-07T02:14:37.055 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /tmp 2024-05-07T02:14:37.055 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:14:37.082 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3522.gbd4e00a3. 127/164 2024-05-07T02:14:37.108 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3522.gbd4e00a3.el9.noa 128/164 2024-05-07T02:14:37.285 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-volume-2:19.0.0-3522.gbd4e00a3.el9.noarch 110/164 2024-05-07T02:14:37.331 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-volume-2:19.0.0-3522.gbd4e00a3.el9.noarch 110/164 2024-05-07T02:14:37.331 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-07T02:14:37.332 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-07T02:14:37.332 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:14:38.205 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3522.gbd4e00a3.el9.noa 128/164 2024-05-07T02:14:38.261 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3522.gbd4e00a3.el9.n 129/164 2024-05-07T02:14:39.137 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-osd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 111/164 2024-05-07T02:14:39.218 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-osd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 111/164 2024-05-07T02:14:39.219 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-07T02:14:39.219 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-07T02:14:39.219 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-07T02:14:39.219 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-07T02:14:39.219 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:14:39.276 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3522.gbd4e00a3.el9.n 129/164 2024-05-07T02:14:39.311 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3522.gbd4 130/164 2024-05-07T02:14:39.410 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-07T02:14:39.410 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-2:19.0.0-3522.gbd4e00a3.el9.x86_64 1/164 2024-05-07T02:14:39.411 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-base-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2/164 2024-05-07T02:14:39.411 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 3/164 2024-05-07T02:14:39.411 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 4/164 2024-05-07T02:14:39.411 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3522.gbd4e0 5/164 2024-05-07T02:14:39.411 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mds-2:19.0.0-3522.gbd4e00a3.el9.x86_64 6/164 2024-05-07T02:14:39.411 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-2:19.0.0-3522.gbd4e00a3.el9.x86_64 7/164 2024-05-07T02:14:39.411 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mon-2:19.0.0-3522.gbd4e00a3.el9.x86_64 8/164 2024-05-07T02:14:39.411 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-osd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 9/164 2024-05-07T02:14:39.411 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-radosgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 10/164 2024-05-07T02:14:39.412 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-selinux-2:19.0.0-3522.gbd4e00a3.el9.x86_64 11/164 2024-05-07T02:14:39.412 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-test-2:19.0.0-3522.gbd4e00a3.el9.x86_64 12/164 2024-05-07T02:14:39.412 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephfs-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 13/164 2024-05-07T02:14:39.412 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephfs-shell-2:19.0.0-3522.gbd4e00a3.el9.x86_64 14/164 2024-05-07T02:14:39.412 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephfs-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_ 15/164 2024-05-07T02:14:39.412 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephfs2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 16/164 2024-05-07T02:14:39.412 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephsqlite-2:19.0.0-3522.gbd4e00a3.el9.x86_64 17/164 2024-05-07T02:14:39.412 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_6 18/164 2024-05-07T02:14:39.412 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libradosstriper1-2:19.0.0-3522.gbd4e00a3.el9.x86 19/164 2024-05-07T02:14:39.412 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librgw2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 20/164 2024-05-07T02:14:39.412 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3522.gbd4e00a3.el 21/164 2024-05-07T02:14:39.412 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ceph-common-2:19.0.0-3522.gbd4e00a3.el9. 22/164 2024-05-07T02:14:39.412 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cephfs-2:19.0.0-3522.gbd4e00a3.el9.x86_6 23/164 2024-05-07T02:14:39.412 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rados-2:19.0.0-3522.gbd4e00a3.el9.x86_64 24/164 2024-05-07T02:14:39.413 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 25/164 2024-05-07T02:14:39.413 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 26/164 2024-05-07T02:14:39.413 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 27/164 2024-05-07T02:14:39.413 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 28/164 2024-05-07T02:14:39.413 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-nbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 29/164 2024-05-07T02:14:39.413 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3522.gbd4e00a3. 30/164 2024-05-07T02:14:39.413 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3522.gbd4e00a3.el9.noa 31/164 2024-05-07T02:14:39.413 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3522.gbd4e00a3.el9.n 32/164 2024-05-07T02:14:39.413 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3522.gbd4 33/164 2024-05-07T02:14:39.413 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3522.gbd4e00a3.el 34/164 2024-05-07T02:14:39.413 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3522.gbd4e00a3.el9.noarch 35/164 2024-05-07T02:14:39.413 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3522.gbd4e00a3.e 36/164 2024-05-07T02:14:39.413 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-volume-2:19.0.0-3522.gbd4e00a3.el9.noarch 37/164 2024-05-07T02:14:39.413 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephadm-2:19.0.0-3522.gbd4e00a3.el9.noarch 38/164 2024-05-07T02:14:39.414 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephfs-top-2:19.0.0-3522.gbd4e00a3.el9.noarch 39/164 2024-05-07T02:14:39.414 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-07T02:14:39.414 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-07T02:14:39.414 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-07T02:14:39.414 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-07T02:14:39.414 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-07T02:14:39.414 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-07T02:14:39.414 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-07T02:14:39.414 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-07T02:14:39.414 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-07T02:14:39.414 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-07T02:14:39.414 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-07T02:14:39.415 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-07T02:14:39.415 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-07T02:14:39.415 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-07T02:14:39.415 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-07T02:14:39.415 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-07T02:14:39.415 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-07T02:14:39.415 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-07T02:14:39.415 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-07T02:14:39.415 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-07T02:14:39.415 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-07T02:14:39.415 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-07T02:14:39.415 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-07T02:14:39.415 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-07T02:14:39.416 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-07T02:14:39.416 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-07T02:14:39.416 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-07T02:14:39.416 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-07T02:14:39.416 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-07T02:14:39.416 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-07T02:14:39.416 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-07T02:14:39.416 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-07T02:14:39.416 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-07T02:14:39.416 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-07T02:14:39.416 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-07T02:14:39.416 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-07T02:14:39.416 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-07T02:14:39.416 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-07T02:14:39.416 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-07T02:14:39.417 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-07T02:14:39.417 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-07T02:14:39.417 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-07T02:14:39.417 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-07T02:14:39.417 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-07T02:14:39.417 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-07T02:14:39.417 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-07T02:14:39.417 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-07T02:14:39.417 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-07T02:14:39.417 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-07T02:14:39.417 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-07T02:14:39.417 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-07T02:14:39.417 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-07T02:14:39.417 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-07T02:14:39.418 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-07T02:14:39.418 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-07T02:14:39.418 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-07T02:14:39.418 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-07T02:14:39.418 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-07T02:14:39.418 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-07T02:14:39.418 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-07T02:14:39.418 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-07T02:14:39.418 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-07T02:14:39.419 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-07T02:14:39.419 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-07T02:14:39.419 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-07T02:14:39.419 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-07T02:14:39.419 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-07T02:14:39.419 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-07T02:14:39.419 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-07T02:14:39.419 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-07T02:14:39.419 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-07T02:14:39.419 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-07T02:14:39.419 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-07T02:14:39.419 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-07T02:14:39.419 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-07T02:14:39.419 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-07T02:14:39.420 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-07T02:14:39.420 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-07T02:14:39.420 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-07T02:14:39.420 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-07T02:14:39.420 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-07T02:14:39.420 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-07T02:14:39.420 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-07T02:14:39.420 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-07T02:14:39.420 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-07T02:14:39.420 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-07T02:14:39.420 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-07T02:14:39.420 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-07T02:14:39.420 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-07T02:14:39.420 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-07T02:14:39.421 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-07T02:14:39.421 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-07T02:14:39.421 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-07T02:14:39.421 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-07T02:14:39.421 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-07T02:14:39.421 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-07T02:14:39.421 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-07T02:14:39.421 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-07T02:14:39.421 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-07T02:14:39.421 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-07T02:14:39.421 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-07T02:14:39.421 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-07T02:14:39.421 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-07T02:14:39.421 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-07T02:14:39.422 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-07T02:14:39.422 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-07T02:14:39.422 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-07T02:14:39.422 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-07T02:14:39.422 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-07T02:14:39.422 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 149/164 2024-05-07T02:14:39.422 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-07T02:14:39.422 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librbd1-2:19.0.0-3522.gbd4e00a3.el9.x86_64 151/164 2024-05-07T02:14:39.422 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-07T02:14:39.422 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-07T02:14:39.422 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-07T02:14:39.422 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-07T02:14:39.422 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-07T02:14:39.422 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-07T02:14:39.423 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-07T02:14:39.423 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-07T02:14:39.423 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-07T02:14:39.423 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-07T02:14:39.423 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-07T02:14:39.423 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-07T02:14:39.430 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3522.gbd4 130/164 2024-05-07T02:14:39.487 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mds-2:19.0.0-3522.gbd4e00a3.el9.x86_64 112/164 2024-05-07T02:14:39.570 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mds-2:19.0.0-3522.gbd4e00a3.el9.x86_64 112/164 2024-05-07T02:14:39.570 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-07T02:14:39.570 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-07T02:14:39.570 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-07T02:14:39.570 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-07T02:14:39.570 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:14:39.601 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3522.gbd4e00a3.el 131/164 2024-05-07T02:14:39.630 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-2:19.0.0-3522.gbd4e00a3.el9.x86_64 132/164 2024-05-07T02:14:39.653 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3522.gbd4e00a3.el9.x86_64 132/164 2024-05-07T02:14:39.654 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-07T02:14:39.654 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-07T02:14:39.654 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-07T02:14:39.654 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-07T02:14:39.654 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-07T02:14:39.705 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-rook-2:19.0.0-3522.gbd4e00a3.el9.noarch 133/164 2024-05-07T02:14:39.717 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3522.gbd4e00a3.el9.noarch 133/164 2024-05-07T02:14:39.819 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-2:19.0.0-3522.gbd4e00a3.el9.x86_64 134/164 2024-05-07T02:14:39.871 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 135/164 2024-05-07T02:14:40.130 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mon-2:19.0.0-3522.gbd4e00a3.el9.x86_64 113/164 2024-05-07T02:14:40.212 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mon-2:19.0.0-3522.gbd4e00a3.el9.x86_64 113/164 2024-05-07T02:14:40.212 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-07T02:14:40.212 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-07T02:14:40.212 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-07T02:14:40.212 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-07T02:14:40.212 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:14:40.366 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-07T02:14:40.430 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-07T02:14:40.506 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-07T02:14:40.768 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-07T02:14:40.837 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-07T02:14:40.858 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-07T02:14:40.891 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-radosgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 137/164 2024-05-07T02:14:40.914 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 137/164 2024-05-07T02:14:40.914 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-07T02:14:40.915 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-07T02:14:40.915 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-07T02:14:40.915 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-07T02:14:40.915 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-07T02:14:40.930 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-07T02:14:40.958 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3522.gbd4e0 138/164 2024-05-07T02:14:40.981 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3522.gbd4e0 138/164 2024-05-07T02:14:40.981 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-07T02:14:40.981 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-07T02:14:40.981 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-07T02:14:41.034 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephfs-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 139/164 2024-05-07T02:14:41.057 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 139/164 2024-05-07T02:14:41.057 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-07T02:14:41.057 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-07T02:14:41.057 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-07T02:14:41.057 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-07T02:14:41.057 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-07T02:14:41.082 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-07T02:14:41.270 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-07T02:14:41.319 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-07T02:14:41.378 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-07T02:14:41.384 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 140/164 2024-05-07T02:14:41.408 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-07T02:14:41.409 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 140/164 2024-05-07T02:14:41.409 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-07T02:14:41.409 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-07T02:14:41.409 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-07T02:14:41.409 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-07T02:14:41.409 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-07T02:14:41.847 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-07T02:14:41.847 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-07T02:14:41.847 INFO:teuthology.orchestra.run.smithi064.stdout:Upgraded: 2024-05-07T02:14:41.847 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-07T02:14:41.847 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-07T02:14:41.847 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-07T02:14:41.847 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-07T02:14:41.848 INFO:teuthology.orchestra.run.smithi064.stdout: librados2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:41.848 INFO:teuthology.orchestra.run.smithi064.stdout: librbd1-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:41.848 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-07T02:14:41.848 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-07T02:14:41.848 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-05-07T02:14:41.848 INFO:teuthology.orchestra.run.smithi064.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-07T02:14:41.848 INFO:teuthology.orchestra.run.smithi064.stdout: automake-1.16.2-8.el9.noarch 2024-05-07T02:14:41.848 INFO:teuthology.orchestra.run.smithi064.stdout: bison-3.7.4-5.el9.x86_64 2024-05-07T02:14:41.848 INFO:teuthology.orchestra.run.smithi064.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-07T02:14:41.848 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:41.848 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-base-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:41.848 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:41.848 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:41.848 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-grafana-dashboards-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-07T02:14:41.849 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-immutable-object-cache-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:41.849 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mds-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:41.849 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:41.849 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-cephadm-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-07T02:14:41.849 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-dashboard-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-07T02:14:41.849 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-07T02:14:41.849 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-modules-core-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-07T02:14:41.849 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-rook-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-07T02:14:41.849 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mon-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:41.849 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-osd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:41.849 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-prometheus-alerts-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-07T02:14:41.849 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-radosgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:41.849 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-selinux-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:41.849 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-test-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:41.850 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-volume-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-07T02:14:41.850 INFO:teuthology.orchestra.run.smithi064.stdout: cephadm-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-07T02:14:41.850 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:41.850 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-shell-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:41.850 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-top-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-07T02:14:41.850 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-07T02:14:41.850 INFO:teuthology.orchestra.run.smithi064.stdout: flex-2.6.4-9.el9.x86_64 2024-05-07T02:14:41.850 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-07T02:14:41.850 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-07T02:14:41.850 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-07T02:14:41.850 INFO:teuthology.orchestra.run.smithi064.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-07T02:14:41.850 INFO:teuthology.orchestra.run.smithi064.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-07T02:14:41.850 INFO:teuthology.orchestra.run.smithi064.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-07T02:14:41.850 INFO:teuthology.orchestra.run.smithi064.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-07T02:14:41.850 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-07T02:14:41.851 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-07T02:14:41.851 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:41.851 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:41.851 INFO:teuthology.orchestra.run.smithi064.stdout: libcephsqlite-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:41.851 INFO:teuthology.orchestra.run.smithi064.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-07T02:14:41.851 INFO:teuthology.orchestra.run.smithi064.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-07T02:14:41.851 INFO:teuthology.orchestra.run.smithi064.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-07T02:14:41.851 INFO:teuthology.orchestra.run.smithi064.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-07T02:14:41.851 INFO:teuthology.orchestra.run.smithi064.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-07T02:14:41.851 INFO:teuthology.orchestra.run.smithi064.stdout: librados-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:41.851 INFO:teuthology.orchestra.run.smithi064.stdout: libradosstriper1-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:41.851 INFO:teuthology.orchestra.run.smithi064.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-07T02:14:41.851 INFO:teuthology.orchestra.run.smithi064.stdout: librgw2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:41.851 INFO:teuthology.orchestra.run.smithi064.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-07T02:14:41.851 INFO:teuthology.orchestra.run.smithi064.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-07T02:14:41.852 INFO:teuthology.orchestra.run.smithi064.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-07T02:14:41.852 INFO:teuthology.orchestra.run.smithi064.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-07T02:14:41.852 INFO:teuthology.orchestra.run.smithi064.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-07T02:14:41.852 INFO:teuthology.orchestra.run.smithi064.stdout: lua-5.4.4-4.el9.x86_64 2024-05-07T02:14:41.852 INFO:teuthology.orchestra.run.smithi064.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-07T02:14:41.852 INFO:teuthology.orchestra.run.smithi064.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-07T02:14:41.852 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-07T02:14:41.852 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-07T02:14:41.852 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-07T02:14:41.852 INFO:teuthology.orchestra.run.smithi064.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-07T02:14:41.852 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-13.14-1.el9.x86_64 2024-05-07T02:14:41.852 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-07T02:14:41.852 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-07T02:14:41.852 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-07T02:14:41.852 INFO:teuthology.orchestra.run.smithi064.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-07T02:14:41.852 INFO:teuthology.orchestra.run.smithi064.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-07T02:14:41.853 INFO:teuthology.orchestra.run.smithi064.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-07T02:14:41.853 INFO:teuthology.orchestra.run.smithi064.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-07T02:14:41.853 INFO:teuthology.orchestra.run.smithi064.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-07T02:14:41.853 INFO:teuthology.orchestra.run.smithi064.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-07T02:14:41.853 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-07T02:14:41.853 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:41.853 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:41.853 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cephfs-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:41.853 INFO:teuthology.orchestra.run.smithi064.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-07T02:14:41.853 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-07T02:14:41.853 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-07T02:14:41.853 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-07T02:14:41.853 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-07T02:14:41.854 INFO:teuthology.orchestra.run.smithi064.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-07T02:14:41.854 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-07T02:14:41.854 INFO:teuthology.orchestra.run.smithi064.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-07T02:14:41.854 INFO:teuthology.orchestra.run.smithi064.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-07T02:14:41.854 INFO:teuthology.orchestra.run.smithi064.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-07T02:14:41.854 INFO:teuthology.orchestra.run.smithi064.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-07T02:14:41.854 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-07T02:14:41.854 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-07T02:14:41.854 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-07T02:14:41.854 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-07T02:14:41.854 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-07T02:14:41.854 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-07T02:14:41.854 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-07T02:14:41.855 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-07T02:14:41.855 INFO:teuthology.orchestra.run.smithi064.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-07T02:14:41.855 INFO:teuthology.orchestra.run.smithi064.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-07T02:14:41.855 INFO:teuthology.orchestra.run.smithi064.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-07T02:14:41.855 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-07T02:14:41.855 INFO:teuthology.orchestra.run.smithi064.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-07T02:14:41.855 INFO:teuthology.orchestra.run.smithi064.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-07T02:14:41.855 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-07T02:14:41.855 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-07T02:14:41.855 INFO:teuthology.orchestra.run.smithi064.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-07T02:14:41.855 INFO:teuthology.orchestra.run.smithi064.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-07T02:14:41.855 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-07T02:14:41.855 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ply-3.11-14.el9.noarch 2024-05-07T02:14:41.855 INFO:teuthology.orchestra.run.smithi064.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-07T02:14:41.855 INFO:teuthology.orchestra.run.smithi064.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-07T02:14:41.856 INFO:teuthology.orchestra.run.smithi064.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-07T02:14:41.856 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-07T02:14:41.856 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-07T02:14:41.856 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-07T02:14:41.856 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-07T02:14:41.856 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-07T02:14:41.856 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-07T02:14:41.856 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rados-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:41.856 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:41.856 INFO:teuthology.orchestra.run.smithi064.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-07T02:14:41.856 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-07T02:14:41.856 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-07T02:14:41.856 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:41.856 INFO:teuthology.orchestra.run.smithi064.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-07T02:14:41.856 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-07T02:14:41.857 INFO:teuthology.orchestra.run.smithi064.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-07T02:14:41.857 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-07T02:14:41.857 INFO:teuthology.orchestra.run.smithi064.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-07T02:14:41.857 INFO:teuthology.orchestra.run.smithi064.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-07T02:14:41.857 INFO:teuthology.orchestra.run.smithi064.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-07T02:14:41.857 INFO:teuthology.orchestra.run.smithi064.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-07T02:14:41.857 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-07T02:14:41.857 INFO:teuthology.orchestra.run.smithi064.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-07T02:14:41.857 INFO:teuthology.orchestra.run.smithi064.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-07T02:14:41.857 INFO:teuthology.orchestra.run.smithi064.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-07T02:14:41.857 INFO:teuthology.orchestra.run.smithi064.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-07T02:14:41.857 INFO:teuthology.orchestra.run.smithi064.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-07T02:14:41.857 INFO:teuthology.orchestra.run.smithi064.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-07T02:14:41.857 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:41.857 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:41.858 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-nbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:41.858 INFO:teuthology.orchestra.run.smithi064.stdout: re2-1:20211101-3.el9.x86_64 2024-05-07T02:14:41.858 INFO:teuthology.orchestra.run.smithi064.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-07T02:14:41.858 INFO:teuthology.orchestra.run.smithi064.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-07T02:14:41.858 INFO:teuthology.orchestra.run.smithi064.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-07T02:14:41.858 INFO:teuthology.orchestra.run.smithi064.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-07T02:14:41.858 INFO:teuthology.orchestra.run.smithi064.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-07T02:14:41.858 INFO:teuthology.orchestra.run.smithi064.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-07T02:14:41.858 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-07T02:14:41.858 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-05-07T02:14:42.075 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: cephadm-2:19.0.0-3522.gbd4e00a3.el9.noarch 125/164 2024-05-07T02:14:42.121 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephadm-2:19.0.0-3522.gbd4e00a3.el9.noarch 125/164 2024-05-07T02:14:42.181 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3522.gbd4e00a3.e 126/164 2024-05-07T02:14:42.252 DEBUG:teuthology.parallel:result is None 2024-05-07T02:14:42.277 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3522.gbd4e00a3. 127/164 2024-05-07T02:14:42.304 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3522.gbd4e00a3.el9.noa 128/164 2024-05-07T02:14:43.654 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3522.gbd4e00a3.el9.noa 128/164 2024-05-07T02:14:43.698 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3522.gbd4e00a3.el9.n 129/164 2024-05-07T02:14:44.861 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3522.gbd4e00a3.el9.n 129/164 2024-05-07T02:14:44.890 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3522.gbd4 130/164 2024-05-07T02:14:45.053 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3522.gbd4 130/164 2024-05-07T02:14:45.243 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3522.gbd4e00a3.el 131/164 2024-05-07T02:14:45.284 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-2:19.0.0-3522.gbd4e00a3.el9.x86_64 132/164 2024-05-07T02:14:45.359 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3522.gbd4e00a3.el9.x86_64 132/164 2024-05-07T02:14:45.359 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-07T02:14:45.359 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-07T02:14:45.359 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-07T02:14:45.359 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-07T02:14:45.359 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:14:45.426 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-rook-2:19.0.0-3522.gbd4e00a3.el9.noarch 133/164 2024-05-07T02:14:45.467 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3522.gbd4e00a3.el9.noarch 133/164 2024-05-07T02:14:45.583 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-2:19.0.0-3522.gbd4e00a3.el9.x86_64 134/164 2024-05-07T02:14:45.655 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 135/164 2024-05-07T02:14:45.931 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-test-2:19.0.0-3522.gbd4e00a3.el9.x86_64 141/164 2024-05-07T02:14:45.972 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 142/164 2024-05-07T02:14:46.013 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-nbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 143/164 2024-05-07T02:14:46.044 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-07T02:14:46.080 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-07T02:14:46.265 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephfs-shell-2:19.0.0-3522.gbd4e00a3.el9.x86_64 146/164 2024-05-07T02:14:46.309 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-07T02:14:46.339 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephfs-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_ 148/164 2024-05-07T02:14:46.369 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-07T02:14:46.415 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-07T02:14:46.741 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-07T02:14:46.778 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-radosgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 137/164 2024-05-07T02:14:46.854 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 137/164 2024-05-07T02:14:46.854 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-07T02:14:46.854 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-07T02:14:46.854 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-07T02:14:46.855 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-07T02:14:46.855 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:14:46.904 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3522.gbd4e0 138/164 2024-05-07T02:14:47.002 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3522.gbd4e0 138/164 2024-05-07T02:14:47.002 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-07T02:14:47.003 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-07T02:14:47.003 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:14:47.048 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-07T02:14:47.080 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephfs-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 139/164 2024-05-07T02:14:47.087 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-07T02:14:47.108 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-07T02:14:47.186 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 139/164 2024-05-07T02:14:47.186 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-07T02:14:47.186 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-07T02:14:47.186 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-07T02:14:47.186 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-07T02:14:47.186 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:14:47.614 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 140/164 2024-05-07T02:14:47.690 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 140/164 2024-05-07T02:14:47.690 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-07T02:14:47.690 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-07T02:14:47.690 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-07T02:14:47.690 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-07T02:14:47.690 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:14:48.013 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephfs-top-2:19.0.0-3522.gbd4e00a3.el9.noarch 152/164 2024-05-07T02:14:48.101 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-07T02:14:48.143 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-07T02:14:48.226 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-07T02:14:48.254 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-07T02:14:48.280 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-07T02:14:48.280 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-07T02:14:48.372 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-07T02:14:48.405 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-07T02:14:48.405 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-07T02:14:48.490 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-07T02:14:48.514 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-07T02:14:48.519 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-07T02:14:48.519 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-07T02:14:48.708 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-07T02:14:48.708 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-07T02:14:48.792 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-07T02:14:48.868 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-07T02:14:52.628 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-test-2:19.0.0-3522.gbd4e00a3.el9.x86_64 141/164 2024-05-07T02:14:52.691 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 142/164 2024-05-07T02:14:52.745 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-nbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 143/164 2024-05-07T02:14:52.786 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-07T02:14:52.824 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-07T02:14:53.146 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephfs-shell-2:19.0.0-3522.gbd4e00a3.el9.x86_64 146/164 2024-05-07T02:14:53.208 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-07T02:14:53.236 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephfs-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_ 148/164 2024-05-07T02:14:53.272 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-07T02:14:53.313 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-07T02:14:54.216 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-07T02:14:54.269 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-07T02:14:54.318 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-07T02:14:54.324 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-07T02:14:54.324 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-2:19.0.0-3522.gbd4e00a3.el9.x86_64 1/164 2024-05-07T02:14:54.324 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-base-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2/164 2024-05-07T02:14:54.324 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 3/164 2024-05-07T02:14:54.324 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 4/164 2024-05-07T02:14:54.324 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3522.gbd4e0 5/164 2024-05-07T02:14:54.324 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mds-2:19.0.0-3522.gbd4e00a3.el9.x86_64 6/164 2024-05-07T02:14:54.324 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-2:19.0.0-3522.gbd4e00a3.el9.x86_64 7/164 2024-05-07T02:14:54.324 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mon-2:19.0.0-3522.gbd4e00a3.el9.x86_64 8/164 2024-05-07T02:14:54.324 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-osd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 9/164 2024-05-07T02:14:54.324 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-radosgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 10/164 2024-05-07T02:14:54.324 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-selinux-2:19.0.0-3522.gbd4e00a3.el9.x86_64 11/164 2024-05-07T02:14:54.325 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-test-2:19.0.0-3522.gbd4e00a3.el9.x86_64 12/164 2024-05-07T02:14:54.325 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephfs-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 13/164 2024-05-07T02:14:54.325 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephfs-shell-2:19.0.0-3522.gbd4e00a3.el9.x86_64 14/164 2024-05-07T02:14:54.325 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephfs-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_ 15/164 2024-05-07T02:14:54.325 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephfs2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 16/164 2024-05-07T02:14:54.325 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephsqlite-2:19.0.0-3522.gbd4e00a3.el9.x86_64 17/164 2024-05-07T02:14:54.325 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_6 18/164 2024-05-07T02:14:54.325 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libradosstriper1-2:19.0.0-3522.gbd4e00a3.el9.x86 19/164 2024-05-07T02:14:54.325 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librgw2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 20/164 2024-05-07T02:14:54.325 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3522.gbd4e00a3.el 21/164 2024-05-07T02:14:54.325 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ceph-common-2:19.0.0-3522.gbd4e00a3.el9. 22/164 2024-05-07T02:14:54.325 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cephfs-2:19.0.0-3522.gbd4e00a3.el9.x86_6 23/164 2024-05-07T02:14:54.325 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rados-2:19.0.0-3522.gbd4e00a3.el9.x86_64 24/164 2024-05-07T02:14:54.325 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 25/164 2024-05-07T02:14:54.325 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 26/164 2024-05-07T02:14:54.325 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 27/164 2024-05-07T02:14:54.326 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 28/164 2024-05-07T02:14:54.326 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-nbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 29/164 2024-05-07T02:14:54.337 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3522.gbd4e00a3. 30/164 2024-05-07T02:14:54.337 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3522.gbd4e00a3.el9.noa 31/164 2024-05-07T02:14:54.337 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3522.gbd4e00a3.el9.n 32/164 2024-05-07T02:14:54.338 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3522.gbd4 33/164 2024-05-07T02:14:54.338 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3522.gbd4e00a3.el 34/164 2024-05-07T02:14:54.338 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3522.gbd4e00a3.el9.noarch 35/164 2024-05-07T02:14:54.338 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3522.gbd4e00a3.e 36/164 2024-05-07T02:14:54.338 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-volume-2:19.0.0-3522.gbd4e00a3.el9.noarch 37/164 2024-05-07T02:14:54.338 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephadm-2:19.0.0-3522.gbd4e00a3.el9.noarch 38/164 2024-05-07T02:14:54.338 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephfs-top-2:19.0.0-3522.gbd4e00a3.el9.noarch 39/164 2024-05-07T02:14:54.338 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-07T02:14:54.338 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-07T02:14:54.338 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-07T02:14:54.338 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-07T02:14:54.338 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-07T02:14:54.338 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-07T02:14:54.339 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-07T02:14:54.339 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-07T02:14:54.339 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-07T02:14:54.339 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-07T02:14:54.339 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-07T02:14:54.339 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-07T02:14:54.339 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-07T02:14:54.339 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-07T02:14:54.339 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-07T02:14:54.339 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-07T02:14:54.339 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-07T02:14:54.339 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-07T02:14:54.339 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-07T02:14:54.339 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-07T02:14:54.339 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-07T02:14:54.340 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-07T02:14:54.340 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-07T02:14:54.340 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-07T02:14:54.340 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-07T02:14:54.340 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-07T02:14:54.340 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-07T02:14:54.340 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-07T02:14:54.340 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-07T02:14:54.340 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-07T02:14:54.340 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-07T02:14:54.340 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-07T02:14:54.340 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-07T02:14:54.340 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-07T02:14:54.340 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-07T02:14:54.340 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-07T02:14:54.340 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-07T02:14:54.340 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-07T02:14:54.341 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-07T02:14:54.341 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-07T02:14:54.341 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-07T02:14:54.341 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-07T02:14:54.341 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-07T02:14:54.341 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-07T02:14:54.341 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-07T02:14:54.341 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-07T02:14:54.341 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-07T02:14:54.341 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-07T02:14:54.341 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-07T02:14:54.341 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-07T02:14:54.341 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-07T02:14:54.341 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-07T02:14:54.341 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-07T02:14:54.341 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-07T02:14:54.342 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-07T02:14:54.342 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-07T02:14:54.342 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-07T02:14:54.342 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-07T02:14:54.342 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-07T02:14:54.342 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-07T02:14:54.342 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-07T02:14:54.342 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-07T02:14:54.342 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-07T02:14:54.342 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-07T02:14:54.342 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-07T02:14:54.342 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-07T02:14:54.342 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-07T02:14:54.342 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-07T02:14:54.342 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-07T02:14:54.343 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-07T02:14:54.343 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-07T02:14:54.343 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-07T02:14:54.343 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-07T02:14:54.343 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-07T02:14:54.343 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-07T02:14:54.343 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-07T02:14:54.343 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-07T02:14:54.343 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-07T02:14:54.343 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-07T02:14:54.343 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-07T02:14:54.343 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-07T02:14:54.343 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-07T02:14:54.343 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-07T02:14:54.343 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-07T02:14:54.344 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-07T02:14:54.344 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-07T02:14:54.344 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-07T02:14:54.344 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-07T02:14:54.344 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-07T02:14:54.344 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-07T02:14:54.344 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-07T02:14:54.344 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-07T02:14:54.344 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-07T02:14:54.344 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-07T02:14:54.344 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-07T02:14:54.344 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-07T02:14:54.344 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-07T02:14:54.344 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-07T02:14:54.344 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-07T02:14:54.345 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-07T02:14:54.345 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-07T02:14:54.345 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-07T02:14:54.345 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-07T02:14:54.345 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-07T02:14:54.345 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-07T02:14:54.345 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-07T02:14:54.345 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-07T02:14:54.345 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-07T02:14:54.345 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-07T02:14:54.345 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 149/164 2024-05-07T02:14:54.345 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-07T02:14:54.345 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librbd1-2:19.0.0-3522.gbd4e00a3.el9.x86_64 151/164 2024-05-07T02:14:54.345 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-07T02:14:54.345 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-07T02:14:54.346 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-07T02:14:54.346 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-07T02:14:54.346 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-07T02:14:54.346 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-07T02:14:54.346 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-07T02:14:54.346 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-07T02:14:54.346 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-07T02:14:54.346 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-07T02:14:54.346 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-07T02:14:54.346 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-07T02:14:55.728 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephfs-top-2:19.0.0-3522.gbd4e00a3.el9.noarch 152/164 2024-05-07T02:14:55.834 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-07T02:14:55.898 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-07T02:14:56.057 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-07T02:14:56.094 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-07T02:14:56.132 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-07T02:14:56.132 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-07T02:14:56.286 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-07T02:14:56.332 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-07T02:14:56.332 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-07T02:14:56.469 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-07T02:14:56.507 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-07T02:14:56.542 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-07T02:14:56.542 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-07T02:14:56.836 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-07T02:14:56.837 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-07T02:14:56.961 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-07T02:14:56.961 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-07T02:14:56.961 INFO:teuthology.orchestra.run.smithi027.stdout:Upgraded: 2024-05-07T02:14:56.961 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-07T02:14:56.961 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-07T02:14:56.961 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-07T02:14:56.961 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-07T02:14:56.961 INFO:teuthology.orchestra.run.smithi027.stdout: librados2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:56.961 INFO:teuthology.orchestra.run.smithi027.stdout: librbd1-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:56.961 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-07T02:14:56.962 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-07T02:14:56.962 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-05-07T02:14:56.962 INFO:teuthology.orchestra.run.smithi027.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-07T02:14:56.962 INFO:teuthology.orchestra.run.smithi027.stdout: automake-1.16.2-8.el9.noarch 2024-05-07T02:14:56.962 INFO:teuthology.orchestra.run.smithi027.stdout: bison-3.7.4-5.el9.x86_64 2024-05-07T02:14:56.962 INFO:teuthology.orchestra.run.smithi027.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-07T02:14:56.962 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:56.962 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-base-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:56.962 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:56.962 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:56.962 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-grafana-dashboards-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-07T02:14:56.962 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-immutable-object-cache-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:56.962 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mds-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:56.962 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:56.963 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-cephadm-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-07T02:14:56.963 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-dashboard-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-07T02:14:56.963 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-07T02:14:56.963 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-modules-core-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-07T02:14:56.963 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-rook-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-07T02:14:56.963 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mon-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:56.963 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-osd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:56.963 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-prometheus-alerts-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-07T02:14:56.963 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-radosgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:56.963 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-selinux-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:56.963 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-test-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:56.963 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-volume-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-07T02:14:56.963 INFO:teuthology.orchestra.run.smithi027.stdout: cephadm-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-07T02:14:56.963 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:56.964 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-shell-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:56.964 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-top-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-07T02:14:56.964 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-07T02:14:56.964 INFO:teuthology.orchestra.run.smithi027.stdout: flex-2.6.4-9.el9.x86_64 2024-05-07T02:14:56.964 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-07T02:14:56.964 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-07T02:14:56.964 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-07T02:14:56.964 INFO:teuthology.orchestra.run.smithi027.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-07T02:14:56.964 INFO:teuthology.orchestra.run.smithi027.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-07T02:14:56.964 INFO:teuthology.orchestra.run.smithi027.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-07T02:14:56.964 INFO:teuthology.orchestra.run.smithi027.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-07T02:14:56.964 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-07T02:14:56.964 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-07T02:14:56.964 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:56.964 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:56.965 INFO:teuthology.orchestra.run.smithi027.stdout: libcephsqlite-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:56.965 INFO:teuthology.orchestra.run.smithi027.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-07T02:14:56.965 INFO:teuthology.orchestra.run.smithi027.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-07T02:14:56.965 INFO:teuthology.orchestra.run.smithi027.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-07T02:14:56.965 INFO:teuthology.orchestra.run.smithi027.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-07T02:14:56.965 INFO:teuthology.orchestra.run.smithi027.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-07T02:14:56.965 INFO:teuthology.orchestra.run.smithi027.stdout: librados-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:56.965 INFO:teuthology.orchestra.run.smithi027.stdout: libradosstriper1-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:56.965 INFO:teuthology.orchestra.run.smithi027.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-07T02:14:56.965 INFO:teuthology.orchestra.run.smithi027.stdout: librgw2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:56.965 INFO:teuthology.orchestra.run.smithi027.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-07T02:14:56.965 INFO:teuthology.orchestra.run.smithi027.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-07T02:14:56.965 INFO:teuthology.orchestra.run.smithi027.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-07T02:14:56.965 INFO:teuthology.orchestra.run.smithi027.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-07T02:14:56.966 INFO:teuthology.orchestra.run.smithi027.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-07T02:14:56.966 INFO:teuthology.orchestra.run.smithi027.stdout: lua-5.4.4-4.el9.x86_64 2024-05-07T02:14:56.966 INFO:teuthology.orchestra.run.smithi027.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-07T02:14:56.966 INFO:teuthology.orchestra.run.smithi027.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-07T02:14:56.966 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-07T02:14:56.966 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-07T02:14:56.966 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-07T02:14:56.966 INFO:teuthology.orchestra.run.smithi027.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-07T02:14:56.966 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-13.14-1.el9.x86_64 2024-05-07T02:14:56.966 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-07T02:14:56.966 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-07T02:14:56.966 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-07T02:14:56.966 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-07T02:14:56.966 INFO:teuthology.orchestra.run.smithi027.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-07T02:14:56.966 INFO:teuthology.orchestra.run.smithi027.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-07T02:14:56.967 INFO:teuthology.orchestra.run.smithi027.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-07T02:14:56.967 INFO:teuthology.orchestra.run.smithi027.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-07T02:14:56.967 INFO:teuthology.orchestra.run.smithi027.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-07T02:14:56.967 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-07T02:14:56.967 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-argparse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:56.967 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:56.967 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cephfs-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:56.967 INFO:teuthology.orchestra.run.smithi027.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-07T02:14:56.967 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-07T02:14:56.967 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-07T02:14:56.967 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-07T02:14:56.967 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-07T02:14:56.967 INFO:teuthology.orchestra.run.smithi027.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-07T02:14:56.967 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-07T02:14:56.967 INFO:teuthology.orchestra.run.smithi027.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-07T02:14:56.968 INFO:teuthology.orchestra.run.smithi027.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-07T02:14:56.968 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-07T02:14:56.968 INFO:teuthology.orchestra.run.smithi027.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-07T02:14:56.968 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-07T02:14:56.968 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-07T02:14:56.968 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-07T02:14:56.968 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-07T02:14:56.968 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-07T02:14:56.968 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-07T02:14:56.968 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-07T02:14:56.968 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-07T02:14:56.968 INFO:teuthology.orchestra.run.smithi027.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-07T02:14:56.968 INFO:teuthology.orchestra.run.smithi027.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-07T02:14:56.968 INFO:teuthology.orchestra.run.smithi027.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-07T02:14:56.968 INFO:teuthology.orchestra.run.smithi027.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-07T02:14:56.969 INFO:teuthology.orchestra.run.smithi027.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-07T02:14:56.969 INFO:teuthology.orchestra.run.smithi027.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-07T02:14:56.969 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-07T02:14:56.969 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-07T02:14:56.969 INFO:teuthology.orchestra.run.smithi027.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-07T02:14:56.969 INFO:teuthology.orchestra.run.smithi027.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-07T02:14:56.969 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-07T02:14:56.969 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ply-3.11-14.el9.noarch 2024-05-07T02:14:56.969 INFO:teuthology.orchestra.run.smithi027.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-07T02:14:56.969 INFO:teuthology.orchestra.run.smithi027.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-07T02:14:56.969 INFO:teuthology.orchestra.run.smithi027.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-07T02:14:56.969 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-07T02:14:56.969 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-07T02:14:56.969 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-07T02:14:56.969 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-07T02:14:56.970 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-07T02:14:56.970 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-07T02:14:56.970 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rados-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:56.970 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:56.970 INFO:teuthology.orchestra.run.smithi027.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-07T02:14:56.970 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-07T02:14:56.970 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-07T02:14:56.970 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:56.970 INFO:teuthology.orchestra.run.smithi027.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-07T02:14:56.970 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-07T02:14:56.970 INFO:teuthology.orchestra.run.smithi027.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-07T02:14:56.970 INFO:teuthology.orchestra.run.smithi027.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-07T02:14:56.970 INFO:teuthology.orchestra.run.smithi027.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-07T02:14:56.970 INFO:teuthology.orchestra.run.smithi027.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-07T02:14:56.970 INFO:teuthology.orchestra.run.smithi027.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-07T02:14:56.971 INFO:teuthology.orchestra.run.smithi027.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-07T02:14:56.971 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-07T02:14:56.971 INFO:teuthology.orchestra.run.smithi027.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-07T02:14:56.971 INFO:teuthology.orchestra.run.smithi027.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-07T02:14:56.971 INFO:teuthology.orchestra.run.smithi027.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-07T02:14:56.971 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-07T02:14:56.971 INFO:teuthology.orchestra.run.smithi027.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-07T02:14:56.971 INFO:teuthology.orchestra.run.smithi027.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-07T02:14:56.971 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:56.971 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:56.971 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-nbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:14:56.971 INFO:teuthology.orchestra.run.smithi027.stdout: re2-1:20211101-3.el9.x86_64 2024-05-07T02:14:56.971 INFO:teuthology.orchestra.run.smithi027.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-07T02:14:56.971 INFO:teuthology.orchestra.run.smithi027.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-07T02:14:56.971 INFO:teuthology.orchestra.run.smithi027.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-07T02:14:56.972 INFO:teuthology.orchestra.run.smithi027.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-07T02:14:56.972 INFO:teuthology.orchestra.run.smithi027.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-07T02:14:56.972 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-07T02:14:56.972 INFO:teuthology.orchestra.run.smithi027.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-07T02:14:56.972 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-05-07T02:14:56.972 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-05-07T02:14:57.101 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-07T02:14:57.361 DEBUG:teuthology.parallel:result is None 2024-05-07T02:14:57.375 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-07T02:14:57.375 INFO:teuthology.orchestra.run.smithi019.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-07T02:14:57.375 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:15:03.096 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-2:19.0.0-3522.gbd4e00a3.el9.x86_64 1/164 2024-05-07T02:15:03.096 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-base-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2/164 2024-05-07T02:15:03.096 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 3/164 2024-05-07T02:15:03.096 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 4/164 2024-05-07T02:15:03.096 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3522.gbd4e0 5/164 2024-05-07T02:15:03.096 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mds-2:19.0.0-3522.gbd4e00a3.el9.x86_64 6/164 2024-05-07T02:15:03.096 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-2:19.0.0-3522.gbd4e00a3.el9.x86_64 7/164 2024-05-07T02:15:03.097 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mon-2:19.0.0-3522.gbd4e00a3.el9.x86_64 8/164 2024-05-07T02:15:03.097 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-osd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 9/164 2024-05-07T02:15:03.097 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-radosgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 10/164 2024-05-07T02:15:03.097 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-selinux-2:19.0.0-3522.gbd4e00a3.el9.x86_64 11/164 2024-05-07T02:15:03.097 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-test-2:19.0.0-3522.gbd4e00a3.el9.x86_64 12/164 2024-05-07T02:15:03.097 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephfs-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 13/164 2024-05-07T02:15:03.097 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephfs-shell-2:19.0.0-3522.gbd4e00a3.el9.x86_64 14/164 2024-05-07T02:15:03.097 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephfs-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_ 15/164 2024-05-07T02:15:03.097 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephfs2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 16/164 2024-05-07T02:15:03.097 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephsqlite-2:19.0.0-3522.gbd4e00a3.el9.x86_64 17/164 2024-05-07T02:15:03.097 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_6 18/164 2024-05-07T02:15:03.097 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libradosstriper1-2:19.0.0-3522.gbd4e00a3.el9.x86 19/164 2024-05-07T02:15:03.097 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librgw2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 20/164 2024-05-07T02:15:03.097 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3522.gbd4e00a3.el 21/164 2024-05-07T02:15:03.097 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ceph-common-2:19.0.0-3522.gbd4e00a3.el9. 22/164 2024-05-07T02:15:03.097 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cephfs-2:19.0.0-3522.gbd4e00a3.el9.x86_6 23/164 2024-05-07T02:15:03.098 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rados-2:19.0.0-3522.gbd4e00a3.el9.x86_64 24/164 2024-05-07T02:15:03.098 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 25/164 2024-05-07T02:15:03.098 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 26/164 2024-05-07T02:15:03.098 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 27/164 2024-05-07T02:15:03.109 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 28/164 2024-05-07T02:15:03.109 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-nbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 29/164 2024-05-07T02:15:03.109 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3522.gbd4e00a3. 30/164 2024-05-07T02:15:03.109 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3522.gbd4e00a3.el9.noa 31/164 2024-05-07T02:15:03.109 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3522.gbd4e00a3.el9.n 32/164 2024-05-07T02:15:03.109 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3522.gbd4 33/164 2024-05-07T02:15:03.109 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3522.gbd4e00a3.el 34/164 2024-05-07T02:15:03.110 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3522.gbd4e00a3.el9.noarch 35/164 2024-05-07T02:15:03.110 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3522.gbd4e00a3.e 36/164 2024-05-07T02:15:03.110 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-volume-2:19.0.0-3522.gbd4e00a3.el9.noarch 37/164 2024-05-07T02:15:03.110 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephadm-2:19.0.0-3522.gbd4e00a3.el9.noarch 38/164 2024-05-07T02:15:03.110 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephfs-top-2:19.0.0-3522.gbd4e00a3.el9.noarch 39/164 2024-05-07T02:15:03.110 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-07T02:15:03.110 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-07T02:15:03.110 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-07T02:15:03.110 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-07T02:15:03.110 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-07T02:15:03.110 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-07T02:15:03.110 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-07T02:15:03.110 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-07T02:15:03.110 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-07T02:15:03.110 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-07T02:15:03.110 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-07T02:15:03.111 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-07T02:15:03.111 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-07T02:15:03.111 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-07T02:15:03.111 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-07T02:15:03.111 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-07T02:15:03.111 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-07T02:15:03.111 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-07T02:15:03.111 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-07T02:15:03.111 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-07T02:15:03.111 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-07T02:15:03.111 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-07T02:15:03.111 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-07T02:15:03.111 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-07T02:15:03.111 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-07T02:15:03.111 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-07T02:15:03.112 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-07T02:15:03.112 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-07T02:15:03.112 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-07T02:15:03.112 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-07T02:15:03.112 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-07T02:15:03.112 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-07T02:15:03.112 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-07T02:15:03.112 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-07T02:15:03.112 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-07T02:15:03.112 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-07T02:15:03.112 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-07T02:15:03.112 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-07T02:15:03.112 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-07T02:15:03.112 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-07T02:15:03.112 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-07T02:15:03.113 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-07T02:15:03.113 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-07T02:15:03.113 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-07T02:15:03.113 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-07T02:15:03.113 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-07T02:15:03.113 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-07T02:15:03.113 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-07T02:15:03.113 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-07T02:15:03.113 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-07T02:15:03.113 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-07T02:15:03.113 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-07T02:15:03.113 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-07T02:15:03.113 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-07T02:15:03.113 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-07T02:15:03.113 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-07T02:15:03.113 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-07T02:15:03.113 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-07T02:15:03.114 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-07T02:15:03.114 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-07T02:15:03.114 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-07T02:15:03.114 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-07T02:15:03.114 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-07T02:15:03.114 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-07T02:15:03.114 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-07T02:15:03.114 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-07T02:15:03.114 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-07T02:15:03.114 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-07T02:15:03.114 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-07T02:15:03.114 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-07T02:15:03.114 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-07T02:15:03.114 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-07T02:15:03.114 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-07T02:15:03.114 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-07T02:15:03.115 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-07T02:15:03.115 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-07T02:15:03.115 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-07T02:15:03.115 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-07T02:15:03.115 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-07T02:15:03.115 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-07T02:15:03.115 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-07T02:15:03.115 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-07T02:15:03.115 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-07T02:15:03.115 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-07T02:15:03.115 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-07T02:15:03.115 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-07T02:15:03.115 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-07T02:15:03.115 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-07T02:15:03.115 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-07T02:15:03.115 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-07T02:15:03.116 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-07T02:15:03.116 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-07T02:15:03.116 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-07T02:15:03.116 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-07T02:15:03.116 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-07T02:15:03.116 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-07T02:15:03.116 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-07T02:15:03.116 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-07T02:15:03.116 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-07T02:15:03.116 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-07T02:15:03.116 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-07T02:15:03.116 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-07T02:15:03.116 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-07T02:15:03.116 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-07T02:15:03.116 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-07T02:15:03.116 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-07T02:15:03.117 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-07T02:15:03.117 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-07T02:15:03.117 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-07T02:15:03.117 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 149/164 2024-05-07T02:15:03.117 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-07T02:15:03.117 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librbd1-2:19.0.0-3522.gbd4e00a3.el9.x86_64 151/164 2024-05-07T02:15:03.117 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-07T02:15:03.117 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-07T02:15:03.117 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-07T02:15:03.117 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-07T02:15:03.117 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-07T02:15:03.117 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-07T02:15:03.117 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-07T02:15:03.117 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-07T02:15:03.117 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-07T02:15:03.117 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-07T02:15:03.117 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-07T02:15:03.117 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-07T02:15:05.274 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-07T02:15:05.275 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:15:05.275 INFO:teuthology.orchestra.run.smithi019.stdout:Upgraded: 2024-05-07T02:15:05.275 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-07T02:15:05.275 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-07T02:15:05.275 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-07T02:15:05.275 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-07T02:15:05.275 INFO:teuthology.orchestra.run.smithi019.stdout: librados2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:15:05.275 INFO:teuthology.orchestra.run.smithi019.stdout: librbd1-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:15:05.275 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-07T02:15:05.275 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-07T02:15:05.275 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-05-07T02:15:05.275 INFO:teuthology.orchestra.run.smithi019.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-07T02:15:05.276 INFO:teuthology.orchestra.run.smithi019.stdout: automake-1.16.2-8.el9.noarch 2024-05-07T02:15:05.276 INFO:teuthology.orchestra.run.smithi019.stdout: bison-3.7.4-5.el9.x86_64 2024-05-07T02:15:05.276 INFO:teuthology.orchestra.run.smithi019.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-07T02:15:05.276 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:15:05.276 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-base-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:15:05.276 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:15:05.276 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:15:05.276 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-grafana-dashboards-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-07T02:15:05.276 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-immutable-object-cache-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:15:05.276 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mds-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:15:05.276 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:15:05.276 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-cephadm-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-07T02:15:05.276 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-dashboard-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-07T02:15:05.276 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-07T02:15:05.277 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-modules-core-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-07T02:15:05.277 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-rook-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-07T02:15:05.277 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mon-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:15:05.277 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-osd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:15:05.277 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-prometheus-alerts-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-07T02:15:05.277 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-radosgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:15:05.277 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-selinux-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:15:05.277 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-test-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:15:05.277 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-volume-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-07T02:15:05.277 INFO:teuthology.orchestra.run.smithi019.stdout: cephadm-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-07T02:15:05.277 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:15:05.277 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-shell-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:15:05.277 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-top-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-07T02:15:05.277 INFO:teuthology.orchestra.run.smithi019.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-07T02:15:05.278 INFO:teuthology.orchestra.run.smithi019.stdout: flex-2.6.4-9.el9.x86_64 2024-05-07T02:15:05.278 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-07T02:15:05.278 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-07T02:15:05.278 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-07T02:15:05.278 INFO:teuthology.orchestra.run.smithi019.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-07T02:15:05.278 INFO:teuthology.orchestra.run.smithi019.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-07T02:15:05.278 INFO:teuthology.orchestra.run.smithi019.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-07T02:15:05.278 INFO:teuthology.orchestra.run.smithi019.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-07T02:15:05.278 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-07T02:15:05.278 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-07T02:15:05.278 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:15:05.278 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:15:05.278 INFO:teuthology.orchestra.run.smithi019.stdout: libcephsqlite-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:15:05.278 INFO:teuthology.orchestra.run.smithi019.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-07T02:15:05.279 INFO:teuthology.orchestra.run.smithi019.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-07T02:15:05.279 INFO:teuthology.orchestra.run.smithi019.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-07T02:15:05.279 INFO:teuthology.orchestra.run.smithi019.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-07T02:15:05.279 INFO:teuthology.orchestra.run.smithi019.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-07T02:15:05.279 INFO:teuthology.orchestra.run.smithi019.stdout: librados-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:15:05.279 INFO:teuthology.orchestra.run.smithi019.stdout: libradosstriper1-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:15:05.279 INFO:teuthology.orchestra.run.smithi019.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-07T02:15:05.279 INFO:teuthology.orchestra.run.smithi019.stdout: librgw2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:15:05.279 INFO:teuthology.orchestra.run.smithi019.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-07T02:15:05.279 INFO:teuthology.orchestra.run.smithi019.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-07T02:15:05.279 INFO:teuthology.orchestra.run.smithi019.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-07T02:15:05.279 INFO:teuthology.orchestra.run.smithi019.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-07T02:15:05.279 INFO:teuthology.orchestra.run.smithi019.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-07T02:15:05.279 INFO:teuthology.orchestra.run.smithi019.stdout: lua-5.4.4-4.el9.x86_64 2024-05-07T02:15:05.279 INFO:teuthology.orchestra.run.smithi019.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-07T02:15:05.280 INFO:teuthology.orchestra.run.smithi019.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-07T02:15:05.280 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-07T02:15:05.280 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-07T02:15:05.280 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-07T02:15:05.280 INFO:teuthology.orchestra.run.smithi019.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-07T02:15:05.280 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-13.14-1.el9.x86_64 2024-05-07T02:15:05.280 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-07T02:15:05.280 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-07T02:15:05.280 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-07T02:15:05.280 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-07T02:15:05.280 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-07T02:15:05.280 INFO:teuthology.orchestra.run.smithi019.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-07T02:15:05.280 INFO:teuthology.orchestra.run.smithi019.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-07T02:15:05.280 INFO:teuthology.orchestra.run.smithi019.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-07T02:15:05.281 INFO:teuthology.orchestra.run.smithi019.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-07T02:15:05.281 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-07T02:15:05.281 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-argparse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:15:05.281 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:15:05.281 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cephfs-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:15:05.281 INFO:teuthology.orchestra.run.smithi019.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-07T02:15:05.281 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-07T02:15:05.281 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-07T02:15:05.281 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-07T02:15:05.281 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-07T02:15:05.281 INFO:teuthology.orchestra.run.smithi019.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-07T02:15:05.281 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-07T02:15:05.281 INFO:teuthology.orchestra.run.smithi019.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-07T02:15:05.281 INFO:teuthology.orchestra.run.smithi019.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-07T02:15:05.281 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-07T02:15:05.282 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-07T02:15:05.282 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-07T02:15:05.282 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-07T02:15:05.282 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-07T02:15:05.282 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-07T02:15:05.282 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-07T02:15:05.282 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-07T02:15:05.282 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-07T02:15:05.282 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-07T02:15:05.282 INFO:teuthology.orchestra.run.smithi019.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-07T02:15:05.282 INFO:teuthology.orchestra.run.smithi019.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-07T02:15:05.282 INFO:teuthology.orchestra.run.smithi019.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-07T02:15:05.282 INFO:teuthology.orchestra.run.smithi019.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-07T02:15:05.282 INFO:teuthology.orchestra.run.smithi019.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-07T02:15:05.283 INFO:teuthology.orchestra.run.smithi019.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-07T02:15:05.283 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-07T02:15:05.283 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-07T02:15:05.283 INFO:teuthology.orchestra.run.smithi019.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-07T02:15:05.283 INFO:teuthology.orchestra.run.smithi019.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-07T02:15:05.283 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-07T02:15:05.283 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ply-3.11-14.el9.noarch 2024-05-07T02:15:05.283 INFO:teuthology.orchestra.run.smithi019.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-07T02:15:05.283 INFO:teuthology.orchestra.run.smithi019.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-07T02:15:05.283 INFO:teuthology.orchestra.run.smithi019.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-07T02:15:05.283 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-07T02:15:05.283 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-07T02:15:05.283 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-07T02:15:05.283 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-07T02:15:05.283 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-07T02:15:05.284 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-07T02:15:05.284 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rados-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:15:05.284 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:15:05.284 INFO:teuthology.orchestra.run.smithi019.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-07T02:15:05.284 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-07T02:15:05.284 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-07T02:15:05.284 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:15:05.284 INFO:teuthology.orchestra.run.smithi019.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-07T02:15:05.284 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-07T02:15:05.284 INFO:teuthology.orchestra.run.smithi019.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-07T02:15:05.284 INFO:teuthology.orchestra.run.smithi019.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-07T02:15:05.284 INFO:teuthology.orchestra.run.smithi019.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-07T02:15:05.284 INFO:teuthology.orchestra.run.smithi019.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-07T02:15:05.284 INFO:teuthology.orchestra.run.smithi019.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-07T02:15:05.285 INFO:teuthology.orchestra.run.smithi019.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-07T02:15:05.285 INFO:teuthology.orchestra.run.smithi019.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-07T02:15:05.285 INFO:teuthology.orchestra.run.smithi019.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-07T02:15:05.285 INFO:teuthology.orchestra.run.smithi019.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-07T02:15:05.285 INFO:teuthology.orchestra.run.smithi019.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-07T02:15:05.285 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-07T02:15:05.285 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-07T02:15:05.285 INFO:teuthology.orchestra.run.smithi019.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-07T02:15:05.285 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:15:05.285 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:15:05.285 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-nbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-07T02:15:05.285 INFO:teuthology.orchestra.run.smithi019.stdout: re2-1:20211101-3.el9.x86_64 2024-05-07T02:15:05.285 INFO:teuthology.orchestra.run.smithi019.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-07T02:15:05.285 INFO:teuthology.orchestra.run.smithi019.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-07T02:15:05.285 INFO:teuthology.orchestra.run.smithi019.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-07T02:15:05.285 INFO:teuthology.orchestra.run.smithi019.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-07T02:15:05.286 INFO:teuthology.orchestra.run.smithi019.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-07T02:15:05.286 INFO:teuthology.orchestra.run.smithi019.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-07T02:15:05.286 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:15:05.286 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-05-07T02:15:05.898 DEBUG:teuthology.parallel:result is None 2024-05-07T02:15:05.898 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bd4e00a39d766328b458e4c953f4d722fed96a31 2024-05-07T02:15:06.049 DEBUG:teuthology.orchestra.run.smithi019:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-07T02:15:06.111 INFO:teuthology.orchestra.run.smithi019.stdout:19.0.0-3522.gbd4e00a3.el9 2024-05-07T02:15:06.112 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3522.gbd4e00a3.el9 2024-05-07T02:15:06.112 INFO:teuthology.task.install:The correct ceph version 19.0.0-3522.gbd4e00a3 is installed. 2024-05-07T02:15:06.113 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bd4e00a39d766328b458e4c953f4d722fed96a31 2024-05-07T02:15:06.331 DEBUG:teuthology.orchestra.run.smithi027:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-07T02:15:06.357 INFO:teuthology.orchestra.run.smithi027.stdout:19.0.0-3522.gbd4e00a3.el9 2024-05-07T02:15:06.358 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3522.gbd4e00a3.el9 2024-05-07T02:15:06.358 INFO:teuthology.task.install:The correct ceph version 19.0.0-3522.gbd4e00a3 is installed. 2024-05-07T02:15:06.359 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bd4e00a39d766328b458e4c953f4d722fed96a31 2024-05-07T02:15:06.572 DEBUG:teuthology.orchestra.run.smithi064:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-07T02:15:06.598 INFO:teuthology.orchestra.run.smithi064.stdout:19.0.0-3522.gbd4e00a3.el9 2024-05-07T02:15:06.598 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3522.gbd4e00a3.el9 2024-05-07T02:15:06.599 INFO:teuthology.task.install:The correct ceph version 19.0.0-3522.gbd4e00a3 is installed. 2024-05-07T02:15:06.600 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-07T02:15:06.600 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-07T02:15:06.600 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-07T02:15:06.685 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-05-07T02:15:06.685 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-07T02:15:06.718 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-07T02:15:06.718 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-07T02:15:06.750 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-07T02:15:06.751 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-07T02:15:06.751 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/daemon-helper 2024-05-07T02:15:06.822 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-07T02:15:06.924 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-05-07T02:15:06.925 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/daemon-helper 2024-05-07T02:15:06.954 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-07T02:15:07.023 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-07T02:15:07.023 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/daemon-helper 2024-05-07T02:15:07.049 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-07T02:15:07.116 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-07T02:15:07.116 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-07T02:15:07.116 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-07T02:15:07.193 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-07T02:15:07.298 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-05-07T02:15:07.298 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-07T02:15:07.328 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-07T02:15:07.399 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-07T02:15:07.399 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-07T02:15:07.425 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-07T02:15:07.490 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-07T02:15:07.490 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-07T02:15:07.490 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/stdin-killer 2024-05-07T02:15:07.559 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-07T02:15:07.663 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-05-07T02:15:07.663 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/stdin-killer 2024-05-07T02:15:07.693 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-07T02:15:07.763 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-07T02:15:07.763 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/stdin-killer 2024-05-07T02:15:07.787 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-07T02:15:07.850 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-07T02:15:07.939 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'bd4e00a39d766328b458e4c953f4d722fed96a31', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-05-07T02:15:07.940 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 2024-05-07T02:15:07.940 INFO:tasks.cephadm:Cluster fsid is 9ff84c16-0c17-11ef-bc97-c7b262605968 2024-05-07T02:15:07.940 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-07T02:15:07.940 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.19', 'mon.b': '172.21.15.27', 'mon.c': '172.21.15.64'} 2024-05-07T02:15:07.941 INFO:tasks.cephadm:First mon is mon.a on smithi019 2024-05-07T02:15:07.941 INFO:tasks.cephadm:First mgr is x 2024-05-07T02:15:07.941 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-07T02:15:07.941 DEBUG:teuthology.orchestra.run.smithi019:> sudo hostname $(hostname -s) 2024-05-07T02:15:08.007 DEBUG:teuthology.orchestra.run.smithi027:> sudo hostname $(hostname -s) 2024-05-07T02:15:08.037 DEBUG:teuthology.orchestra.run.smithi064:> sudo hostname $(hostname -s) 2024-05-07T02:15:08.061 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-07T02:15:08.061 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bd4e00a39d766328b458e4c953f4d722fed96a31 2024-05-07T02:15:08.209 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'bd4e00a39d766328b458e4c953f4d722fed96a31', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79090/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3522-gbd4e00a3', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3522.gbd4e00a3'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-03 18:03:30.054903', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240503.163550-debug', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-05-07T02:15:08.431 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-pdonnell-testing-20240503.163550-debug, sha1 bd4e00a39d766328b458e4c953f4d722fed96a31 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=bd4e00a39d766328b458e4c953f4d722fed96a31 2024-05-07T02:15:08.432 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/x86_64/flavors/default/cephadm 2024-05-07T02:15:08.432 DEBUG:teuthology.orchestra.run.smithi019:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-07T02:15:09.766 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-r--r--. 1 ubuntu ubuntu 782507 May 7 02:15 /home/ubuntu/cephtest/cephadm 2024-05-07T02:15:09.767 DEBUG:teuthology.orchestra.run.smithi027:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-07T02:15:10.768 INFO:teuthology.orchestra.run.smithi027.stdout:-rw-r--r--. 1 ubuntu ubuntu 782507 May 7 02:15 /home/ubuntu/cephtest/cephadm 2024-05-07T02:15:10.769 DEBUG:teuthology.orchestra.run.smithi064:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-07T02:15:11.738 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-r--r--. 1 ubuntu ubuntu 782507 May 7 02:15 /home/ubuntu/cephtest/cephadm 2024-05-07T02:15:11.738 DEBUG:teuthology.orchestra.run.smithi019:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-07T02:15:11.803 DEBUG:teuthology.orchestra.run.smithi027:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-07T02:15:11.823 DEBUG:teuthology.orchestra.run.smithi064:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-07T02:15:11.858 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 on all hosts... 2024-05-07T02:15:11.859 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 pull 2024-05-07T02:15:11.864 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 pull 2024-05-07T02:15:11.867 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 pull 2024-05-07T02:15:13.310 INFO:teuthology.orchestra.run.smithi064.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31... 2024-05-07T02:15:13.371 INFO:teuthology.orchestra.run.smithi027.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31... 2024-05-07T02:15:13.990 INFO:teuthology.orchestra.run.smithi019.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31... 2024-05-07T02:15:41.852 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-05-07T02:15:41.852 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_version": "ceph version 19.0.0-3522-gbd4e00a3 (bd4e00a39d766328b458e4c953f4d722fed96a31) squid (dev)", 2024-05-07T02:15:41.852 INFO:teuthology.orchestra.run.smithi027.stdout: "image_id": "afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62", 2024-05-07T02:15:41.852 INFO:teuthology.orchestra.run.smithi027.stdout: "repo_digests": [ 2024-05-07T02:15:41.852 INFO:teuthology.orchestra.run.smithi027.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2" 2024-05-07T02:15:41.852 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-05-07T02:15:41.852 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-05-07T02:15:49.421 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-07T02:15:49.422 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 19.0.0-3522-gbd4e00a3 (bd4e00a39d766328b458e4c953f4d722fed96a31) squid (dev)", 2024-05-07T02:15:49.422 INFO:teuthology.orchestra.run.smithi064.stdout: "image_id": "afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62", 2024-05-07T02:15:49.422 INFO:teuthology.orchestra.run.smithi064.stdout: "repo_digests": [ 2024-05-07T02:15:49.422 INFO:teuthology.orchestra.run.smithi064.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2" 2024-05-07T02:15:49.422 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-05-07T02:15:49.422 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-07T02:16:05.458 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:16:05.459 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_version": "ceph version 19.0.0-3522-gbd4e00a3 (bd4e00a39d766328b458e4c953f4d722fed96a31) squid (dev)", 2024-05-07T02:16:05.459 INFO:teuthology.orchestra.run.smithi019.stdout: "image_id": "afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62", 2024-05-07T02:16:05.459 INFO:teuthology.orchestra.run.smithi019.stdout: "repo_digests": [ 2024-05-07T02:16:05.459 INFO:teuthology.orchestra.run.smithi019.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2" 2024-05-07T02:16:05.459 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-05-07T02:16:05.459 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:16:05.528 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /etc/ceph 2024-05-07T02:16:05.623 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /etc/ceph 2024-05-07T02:16:05.655 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /etc/ceph 2024-05-07T02:16:05.688 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 777 /etc/ceph 2024-05-07T02:16:05.782 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 777 /etc/ceph 2024-05-07T02:16:05.811 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 777 /etc/ceph 2024-05-07T02:16:05.840 INFO:tasks.cephadm:Writing seed config... 2024-05-07T02:16:05.841 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-07T02:16:05.841 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-07T02:16:05.841 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-07T02:16:05.841 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-07T02:16:05.841 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-07T02:16:05.841 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-05-07T02:16:05.841 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-07T02:16:05.841 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-07T02:16:05.841 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-07T02:16:05.842 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-07T02:16:05.842 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-07T02:16:05.842 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-07T02:16:05.842 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-07T02:16:05.842 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-07T02:16:05.842 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-07T02:16:05.842 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-07T02:16:05.842 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-07T02:16:05.842 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-07T02:16:05.842 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-07T02:16:05.842 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-07T02:16:05.842 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-07T02:16:05.842 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-05-07T02:16:05.842 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-07T02:16:05.842 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-07T02:16:05.843 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-07T02:16:05.843 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-07T02:16:05.843 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-07T02:16:05.843 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-07T02:16:05.843 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-07T02:16:05.843 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-07T02:16:05.843 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-07T02:16:05.843 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-07T02:16:05.843 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-05-07T02:16:05.843 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-05-07T02:16:05.843 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-07T02:16:05.843 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-07T02:16:05.843 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-05-07T02:16:05.843 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-07T02:16:05.843 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-07T02:16:05.843 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-07T02:16:05.844 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-07T02:16:05.844 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-07T02:16:05.844 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-07T02:16:05.844 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-07T02:16:05.844 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-05-07T02:16:05.845 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-07T02:16:05.845 DEBUG:teuthology.orchestra.run.smithi019:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-07T02:16:05.904 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 9ff84c16-0c17-11ef-bc97-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-07T02:16:05.905 DEBUG:teuthology.orchestra.run.smithi019:mon.a> sudo journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mon.a.service 2024-05-07T02:16:05.952 DEBUG:teuthology.orchestra.run.smithi019:mgr.x> sudo journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mgr.x.service 2024-05-07T02:16:05.997 INFO:tasks.cephadm:Bootstrapping... 2024-05-07T02:16:05.997 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 -v bootstrap --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.19 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-07T02:16:06.275 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-05-07T02:16:06.276 INFO:teuthology.orchestra.run.smithi019.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31', '-v', 'bootstrap', '--fsid', '9ff84c16-0c17-11ef-bc97-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.19', '--skip-admin-label'] 2024-05-07T02:16:06.276 INFO:teuthology.orchestra.run.smithi019.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-07T02:16:06.276 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying podman|docker is present... 2024-05-07T02:16:06.337 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-07T02:16:06.338 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying lvm2 is present... 2024-05-07T02:16:06.338 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying time synchronization is in place... 2024-05-07T02:16:06.354 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-07T02:16:06.354 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-07T02:16:06.368 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-07T02:16:06.369 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-05-07T02:16:06.386 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout enabled 2024-05-07T02:16:06.402 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout active 2024-05-07T02:16:06.402 INFO:teuthology.orchestra.run.smithi019.stdout:Unit chronyd.service is enabled and running 2024-05-07T02:16:06.403 INFO:teuthology.orchestra.run.smithi019.stdout:Repeating the final host check... 2024-05-07T02:16:06.473 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-07T02:16:06.473 INFO:teuthology.orchestra.run.smithi019.stdout:podman (/bin/podman) version 4.9.4 is present 2024-05-07T02:16:06.473 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl is present 2024-05-07T02:16:06.474 INFO:teuthology.orchestra.run.smithi019.stdout:lvcreate is present 2024-05-07T02:16:06.494 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-07T02:16:06.494 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-07T02:16:06.517 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-07T02:16:06.517 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-05-07T02:16:06.541 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout enabled 2024-05-07T02:16:06.564 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout active 2024-05-07T02:16:06.565 INFO:teuthology.orchestra.run.smithi019.stdout:Unit chronyd.service is enabled and running 2024-05-07T02:16:06.565 INFO:teuthology.orchestra.run.smithi019.stdout:Host looks OK 2024-05-07T02:16:06.565 INFO:teuthology.orchestra.run.smithi019.stdout:Cluster fsid: 9ff84c16-0c17-11ef-bc97-c7b262605968 2024-05-07T02:16:06.566 INFO:teuthology.orchestra.run.smithi019.stdout:Acquiring lock 140575829454800 on /run/cephadm/9ff84c16-0c17-11ef-bc97-c7b262605968.lock 2024-05-07T02:16:06.566 INFO:teuthology.orchestra.run.smithi019.stdout:Lock 140575829454800 acquired on /run/cephadm/9ff84c16-0c17-11ef-bc97-c7b262605968.lock 2024-05-07T02:16:06.567 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying IP 172.21.15.19 port 3300 ... 2024-05-07T02:16:06.568 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying IP 172.21.15.19 port 6789 ... 2024-05-07T02:16:06.569 INFO:teuthology.orchestra.run.smithi019.stdout:Base mon IP(s) is [172.21.15.19:3300, 172.21.15.19:6789], mon addrv is [v2:172.21.15.19:3300,v1:172.21.15.19:6789] 2024-05-07T02:16:06.585 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.19 metric 100 2024-05-07T02:16:06.585 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.19 metric 100 2024-05-07T02:16:06.597 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-07T02:16:06.598 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-05-07T02:16:06.598 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-05-07T02:16:06.609 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-07T02:16:06.610 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-07T02:16:06.610 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-07T02:16:06.610 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-05-07T02:16:06.610 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-05-07T02:16:06.611 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-07T02:16:06.613 INFO:teuthology.orchestra.run.smithi019.stdout:Mon IP `172.21.15.19` is in CIDR network `172.21.0.0/20` 2024-05-07T02:16:06.613 INFO:teuthology.orchestra.run.smithi019.stdout:Mon IP `172.21.15.19` is in CIDR network `172.21.0.0/20` 2024-05-07T02:16:06.613 INFO:teuthology.orchestra.run.smithi019.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-07T02:16:06.616 INFO:teuthology.orchestra.run.smithi019.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-07T02:16:06.619 INFO:teuthology.orchestra.run.smithi019.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31... 2024-05-07T02:16:07.304 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stdout afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 2024-05-07T02:16:07.304 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31... 2024-05-07T02:16:07.304 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Getting image source signatures 2024-05-07T02:16:07.304 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Copying blob sha256:096f198f13bcfe0dfab6408b113f6513b5ae0fab440afa5e44aec6470f68ac59 2024-05-07T02:16:07.304 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Copying blob sha256:e2f2fd2adca92139da4b4b907fa90a6775d3eb4eddaa977c42ef37c66eea70bd 2024-05-07T02:16:07.305 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Copying config sha256:afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 2024-05-07T02:16:07.305 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-07T02:16:08.624 INFO:teuthology.orchestra.run.smithi019.stdout:ceph: stdout ceph version 19.0.0-3522-gbd4e00a3 (bd4e00a39d766328b458e4c953f4d722fed96a31) squid (dev) 2024-05-07T02:16:08.624 INFO:teuthology.orchestra.run.smithi019.stdout:Ceph version: ceph version 19.0.0-3522-gbd4e00a3 (bd4e00a39d766328b458e4c953f4d722fed96a31) squid (dev) 2024-05-07T02:16:08.624 INFO:teuthology.orchestra.run.smithi019.stdout:Extracting ceph user uid/gid from container image... 2024-05-07T02:16:10.639 INFO:teuthology.orchestra.run.smithi019.stdout:stat: stdout 167 167 2024-05-07T02:16:10.640 INFO:teuthology.orchestra.run.smithi019.stdout:Creating initial keys... 2024-05-07T02:16:11.846 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph-authtool: stdout AQDrjjlmtvbrCRAAfGvlsIeUwRPFzwp5NdR02A== 2024-05-07T02:16:13.142 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph-authtool: stdout AQDsjjlmV+2uFhAAWn4atXTtBIhhTVjYcAz/uw== 2024-05-07T02:16:14.324 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph-authtool: stdout AQDtjjlm/8ZuKBAANKQTkb9J5Q4ZwjHhfdo+dw== 2024-05-07T02:16:14.327 INFO:teuthology.orchestra.run.smithi019.stdout:Creating initial monmap... 2024-05-07T02:16:15.670 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-07T02:16:15.670 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-07T02:16:15.670 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 9ff84c16-0c17-11ef-bc97-c7b262605968 2024-05-07T02:16:15.670 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-07T02:16:15.670 INFO:teuthology.orchestra.run.smithi019.stdout:monmaptool for a [v2:172.21.15.19:3300,v1:172.21.15.19:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-07T02:16:15.670 INFO:teuthology.orchestra.run.smithi019.stdout:setting min_mon_release = quincy 2024-05-07T02:16:15.670 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: set fsid to 9ff84c16-0c17-11ef-bc97-c7b262605968 2024-05-07T02:16:15.670 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-07T02:16:15.670 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:16:15.670 INFO:teuthology.orchestra.run.smithi019.stdout:Creating mon... 2024-05-07T02:16:16.900 INFO:teuthology.orchestra.run.smithi019.stdout:create mon.a on 2024-05-07T02:16:17.322 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-07T02:16:17.562 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-07T02:16:17.865 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-9ff84c16-0c17-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-9ff84c16-0c17-11ef-bc97-c7b262605968.target. 2024-05-07T02:16:17.865 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-9ff84c16-0c17-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-9ff84c16-0c17-11ef-bc97-c7b262605968.target. 2024-05-07T02:16:18.248 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mon.a 2024-05-07T02:16:18.248 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Failed to reset failed state of unit ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mon.a.service: Unit ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mon.a.service not loaded. 2024-05-07T02:16:18.556 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-9ff84c16-0c17-11ef-bc97-c7b262605968.target.wants/ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mon.a.service → /etc/systemd/system/ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@.service. 2024-05-07T02:16:19.340 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:19 smithi019 podman[29385]: 2024-05-07 02:16:19.026496638 +0000 UTC m=+0.177950574 container create 9acffa6e15afbfdc64f4c5de4cb7526f1bebb1d9a6523c662a3da1fe1de4f519 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mon-a, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-07T02:16:19.340 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:19 smithi019 podman[29385]: 2024-05-07 02:16:19.338349439 +0000 UTC m=+0.489803398 container init 9acffa6e15afbfdc64f4c5de4cb7526f1bebb1d9a6523c662a3da1fe1de4f519 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=) 2024-05-07T02:16:19.485 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-07T02:16:19.485 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout disabled 2024-05-07T02:16:19.502 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-07T02:16:19.502 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-05-07T02:16:19.502 INFO:teuthology.orchestra.run.smithi019.stdout:firewalld.service is not enabled 2024-05-07T02:16:19.502 INFO:teuthology.orchestra.run.smithi019.stdout:Not possible to enable service . firewalld.service is not available 2024-05-07T02:16:19.503 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mon to start... 2024-05-07T02:16:19.503 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mon... 2024-05-07T02:16:19.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:19 smithi019 podman[29385]: 2024-05-07 02:16:19.349639954 +0000 UTC m=+0.501093896 container start 9acffa6e15afbfdc64f4c5de4cb7526f1bebb1d9a6523c662a3da1fe1de4f519 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-07T02:16:19.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:19 smithi019 bash[29385]: 9acffa6e15afbfdc64f4c5de4cb7526f1bebb1d9a6523c662a3da1fe1de4f519 2024-05-07T02:16:19.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:19 smithi019 systemd[1]: Started Ceph mon.a for 9ff84c16-0c17-11ef-bc97-c7b262605968. 2024-05-07T02:16:19.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:19 smithi019 ceph-mon[29404]: mkfs 9ff84c16-0c17-11ef-bc97-c7b262605968 2024-05-07T02:16:19.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:19 smithi019 ceph-mon[29404]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-07T02:16:20.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:20 smithi019 ceph-mon[29404]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-07T02:16:20.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:20 smithi019 ceph-mon[29404]: monmap epoch 1 2024-05-07T02:16:20.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:20 smithi019 ceph-mon[29404]: fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 2024-05-07T02:16:20.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:20 smithi019 ceph-mon[29404]: last_changed 2024-05-07T02:16:14.906770+0000 2024-05-07T02:16:20.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:20 smithi019 ceph-mon[29404]: created 2024-05-07T02:16:14.906770+0000 2024-05-07T02:16:20.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:20 smithi019 ceph-mon[29404]: min_mon_release 19 (squid) 2024-05-07T02:16:20.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:20 smithi019 ceph-mon[29404]: election_strategy: 1 2024-05-07T02:16:20.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:20 smithi019 ceph-mon[29404]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-05-07T02:16:20.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:20 smithi019 ceph-mon[29404]: fsmap 2024-05-07T02:16:20.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:20 smithi019 ceph-mon[29404]: osdmap e1: 0 total, 0 up, 0 in 2024-05-07T02:16:20.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:20 smithi019 ceph-mon[29404]: mgrmap e1: no daemons active 2024-05-07T02:16:20.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:20 smithi019 ceph-mon[29404]: from='client.? 172.21.15.19:0/2407932678' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-07T02:16:21.081 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout cluster: 2024-05-07T02:16:21.081 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout id: 9ff84c16-0c17-11ef-bc97-c7b262605968 2024-05-07T02:16:21.081 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-07T02:16:21.082 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-07T02:16:21.082 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout services: 2024-05-07T02:16:21.082 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.919597s) 2024-05-07T02:16:21.082 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-07T02:16:21.082 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-07T02:16:21.082 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-07T02:16:21.082 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout data: 2024-05-07T02:16:21.082 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-07T02:16:21.082 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-07T02:16:21.082 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-07T02:16:21.082 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout pgs: 2024-05-07T02:16:21.082 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-07T02:16:21.082 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:20.326+0000 7f7fb8400700 1 Processor -- start 2024-05-07T02:16:21.083 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:20.326+0000 7f7fb8400700 1 -- start start 2024-05-07T02:16:21.083 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:20.327+0000 7f7fb8400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fb010a590 0x7f7fb010c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:16:21.083 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:20.327+0000 7f7fb8400700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb007cdb0 con 0x7f7fb010a590 2024-05-07T02:16:21.083 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:20.328+0000 7f7fb5a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fb010a590 0x7f7fb010c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:16:21.083 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:20.328+0000 7f7fb5a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fb010a590 0x7f7fb010c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36980/0 (socket says 172.21.15.19:36980) 2024-05-07T02:16:21.083 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:20.328+0000 7f7fb5a00700 1 -- 172.21.15.19:0/1688752648 learned_addr learned my addr 172.21.15.19:0/1688752648 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:16:21.083 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:20.329+0000 7f7fb5a00700 1 -- 172.21.15.19:0/1688752648 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fb010cef0 con 0x7f7fb010a590 2024-05-07T02:16:21.083 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:20.329+0000 7f7fb5a00700 1 --2- 172.21.15.19:0/1688752648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fb010a590 0x7f7fb010c9b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7fa40098a0 tx=0x7f7fa4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba84ddf1e7fdca77 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:16:21.083 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:20.330+0000 7f7fafe00700 1 -- 172.21.15.19:0/1688752648 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fa4004160 con 0x7f7fb010a590 2024-05-07T02:16:21.083 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:20.330+0000 7f7fafe00700 1 -- 172.21.15.19:0/1688752648 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fa4004300 con 0x7f7fb010a590 2024-05-07T02:16:21.083 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:20.330+0000 7f7fafe00700 1 -- 172.21.15.19:0/1688752648 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fa40036a0 con 0x7f7fb010a590 2024-05-07T02:16:21.083 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:20.330+0000 7f7fb8400700 1 -- 172.21.15.19:0/1688752648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fb010a590 msgr2=0x7f7fb010c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:16:21.083 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:20.330+0000 7f7fb8400700 1 --2- 172.21.15.19:0/1688752648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fb010a590 0x7f7fb010c9b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7fa40098a0 tx=0x7f7fa4009bb0 comp rx=0 tx=0).stop 2024-05-07T02:16:21.083 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:20.331+0000 7f7fb8400700 1 -- 172.21.15.19:0/1688752648 shutdown_connections 2024-05-07T02:16:21.083 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:20.331+0000 7f7fb8400700 1 --2- 172.21.15.19:0/1688752648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fb010a590 0x7f7fb010c9b0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:16:21.084 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:20.331+0000 7f7fb8400700 1 -- 172.21.15.19:0/1688752648 >> 172.21.15.19:0/1688752648 conn(0x7f7fb0104050 msgr2=0x7f7fb01064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:16:21.084 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:20.331+0000 7f7fb8400700 1 -- 172.21.15.19:0/1688752648 shutdown_connections 2024-05-07T02:16:21.084 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:20.331+0000 7f7fb8400700 1 -- 172.21.15.19:0/1688752648 wait complete. 2024-05-07T02:16:21.084 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:20.332+0000 7f7fb8400700 1 Processor -- start 2024-05-07T02:16:21.084 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:20.332+0000 7f7fb8400700 1 -- start start 2024-05-07T02:16:21.084 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:20.333+0000 7f7fb8400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fb019d4d0 0x7f7fb019d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:16:21.084 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:20.333+0000 7f7fb8400700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb010d2a0 con 0x7f7fb019d4d0 2024-05-07T02:16:21.084 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:20.333+0000 7f7fb5a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fb019d4d0 0x7f7fb019d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:16:21.084 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:20.334+0000 7f7fb5a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fb019d4d0 0x7f7fb019d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36996/0 (socket says 172.21.15.19:36996) 2024-05-07T02:16:21.084 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:20.334+0000 7f7fb5a00700 1 -- 172.21.15.19:0/2407932678 learned_addr learned my addr 172.21.15.19:0/2407932678 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:16:21.084 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:20.334+0000 7f7fb5a00700 1 -- 172.21.15.19:0/2407932678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fb019de80 con 0x7f7fb019d4d0 2024-05-07T02:16:21.084 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:20.334+0000 7f7fb5a00700 1 --2- 172.21.15.19:0/2407932678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fb019d4d0 0x7f7fb019d900 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4000c00 tx=0x7f7fa4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:16:21.084 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:20.335+0000 7f7fae000700 1 -- 172.21.15.19:0/2407932678 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fa4003ca0 con 0x7f7fb019d4d0 2024-05-07T02:16:21.084 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:20.335+0000 7f7fae000700 1 -- 172.21.15.19:0/2407932678 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fa4003e40 con 0x7f7fb019d4d0 2024-05-07T02:16:21.084 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:20.335+0000 7f7fb8400700 1 -- 172.21.15.19:0/2407932678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fb019e110 con 0x7f7fb019d4d0 2024-05-07T02:16:21.085 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:20.335+0000 7f7fae000700 1 -- 172.21.15.19:0/2407932678 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fa4011780 con 0x7f7fb019d4d0 2024-05-07T02:16:21.085 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:20.335+0000 7f7fb8400700 1 -- 172.21.15.19:0/2407932678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fb01a0c40 con 0x7f7fb019d4d0 2024-05-07T02:16:21.085 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:20.335+0000 7f7fae000700 1 -- 172.21.15.19:0/2407932678 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7fa4011920 con 0x7f7fb019d4d0 2024-05-07T02:16:21.085 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:20.336+0000 7f7fae000700 1 -- 172.21.15.19:0/2407932678 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7fa4025d80 con 0x7f7fb019d4d0 2024-05-07T02:16:21.085 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:20.336+0000 7f7fb8400700 1 -- 172.21.15.19:0/2407932678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fb0064e20 con 0x7f7fb019d4d0 2024-05-07T02:16:21.085 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:20.340+0000 7f7fae000700 1 -- 172.21.15.19:0/2407932678 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f7fa4004260 con 0x7f7fb019d4d0 2024-05-07T02:16:21.085 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:20.423+0000 7f7fb8400700 1 -- 172.21.15.19:0/2407932678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f7fb002ea30 con 0x7f7fb019d4d0 2024-05-07T02:16:21.085 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:20.424+0000 7f7fae000700 1 -- 172.21.15.19:0/2407932678 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f7fb002ea30 con 0x7f7fb019d4d0 2024-05-07T02:16:21.085 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:20.425+0000 7f7fb8400700 1 -- 172.21.15.19:0/2407932678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fb019d4d0 msgr2=0x7f7fb019d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:16:21.085 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:20.425+0000 7f7fb8400700 1 --2- 172.21.15.19:0/2407932678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fb019d4d0 0x7f7fb019d900 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4000c00 tx=0x7f7fa4004750 comp rx=0 tx=0).stop 2024-05-07T02:16:21.085 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:20.425+0000 7f7fb8400700 1 -- 172.21.15.19:0/2407932678 shutdown_connections 2024-05-07T02:16:21.085 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:20.425+0000 7f7fb8400700 1 --2- 172.21.15.19:0/2407932678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fb019d4d0 0x7f7fb019d900 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:16:21.085 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:20.425+0000 7f7fb8400700 1 -- 172.21.15.19:0/2407932678 >> 172.21.15.19:0/2407932678 conn(0x7f7fb0104050 msgr2=0x7f7fb0104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:16:21.085 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:20.425+0000 7f7fb8400700 1 -- 172.21.15.19:0/2407932678 shutdown_connections 2024-05-07T02:16:21.085 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:20.426+0000 7f7fb8400700 1 -- 172.21.15.19:0/2407932678 wait complete. 2024-05-07T02:16:21.086 INFO:teuthology.orchestra.run.smithi019.stdout:mon is available 2024-05-07T02:16:21.086 INFO:teuthology.orchestra.run.smithi019.stdout:Assimilating anything we can from ceph.conf... 2024-05-07T02:16:22.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:21 smithi019 ceph-mon[29404]: from='client.? 172.21.15.19:0/1395499806' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-07T02:16:22.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:21 smithi019 ceph-mon[29404]: from='client.? 172.21.15.19:0/1395499806' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-07T02:16:22.468 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-07T02:16:22.469 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [global] 2024-05-07T02:16:22.474 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout fsid = 9ff84c16-0c17-11ef-bc97-c7b262605968 2024-05-07T02:16:22.474 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-07T02:16:22.474 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.19:3300,v1:172.21.15.19:6789] 2024-05-07T02:16:22.474 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-07T02:16:22.474 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-07T02:16:22.474 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-07T02:16:22.474 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-07T02:16:22.474 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-07T02:16:22.474 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-07T02:16:22.474 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-07T02:16:22.474 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-07T02:16:22.474 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [osd] 2024-05-07T02:16:22.475 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-07T02:16:22.475 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-07T02:16:22.475 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-07T02:16:22.475 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-07T02:16:22.475 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-07T02:16:22.475 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.773+0000 7f8442a00700 1 Processor -- start 2024-05-07T02:16:22.475 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.773+0000 7f8442a00700 1 -- start start 2024-05-07T02:16:22.475 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.774+0000 7f8442a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f843c080050 0x7f843c080460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:16:22.475 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.774+0000 7f8442a00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f843c080a70 con 0x7f843c080050 2024-05-07T02:16:22.475 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.774+0000 7f8441600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f843c080050 0x7f843c080460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:16:22.475 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.774+0000 7f8441600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f843c080050 0x7f843c080460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36998/0 (socket says 172.21.15.19:36998) 2024-05-07T02:16:22.475 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.774+0000 7f8441600700 1 -- 172.21.15.19:0/1909802055 learned_addr learned my addr 172.21.15.19:0/1909802055 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:16:22.475 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.775+0000 7f8441600700 1 -- 172.21.15.19:0/1909802055 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f843c0811d0 con 0x7f843c080050 2024-05-07T02:16:22.476 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.775+0000 7f8441600700 1 --2- 172.21.15.19:0/1909802055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f843c080050 0x7f843c080460 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f84300098a0 tx=0x7f8430009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b370507fd7ab8a40 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:16:22.476 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.775+0000 7f843be00700 1 -- 172.21.15.19:0/1909802055 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8430004160 con 0x7f843c080050 2024-05-07T02:16:22.476 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.775+0000 7f843be00700 1 -- 172.21.15.19:0/1909802055 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8430004300 con 0x7f843c080050 2024-05-07T02:16:22.476 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.775+0000 7f843be00700 1 -- 172.21.15.19:0/1909802055 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84300036a0 con 0x7f843c080050 2024-05-07T02:16:22.476 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.776+0000 7f8442a00700 1 -- 172.21.15.19:0/1909802055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f843c080050 msgr2=0x7f843c080460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:16:22.476 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.776+0000 7f8442a00700 1 --2- 172.21.15.19:0/1909802055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f843c080050 0x7f843c080460 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f84300098a0 tx=0x7f8430009bb0 comp rx=0 tx=0).stop 2024-05-07T02:16:22.476 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.776+0000 7f8442a00700 1 -- 172.21.15.19:0/1909802055 shutdown_connections 2024-05-07T02:16:22.476 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.776+0000 7f8442a00700 1 --2- 172.21.15.19:0/1909802055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f843c080050 0x7f843c080460 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:16:22.476 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.776+0000 7f8442a00700 1 -- 172.21.15.19:0/1909802055 >> 172.21.15.19:0/1909802055 conn(0x7f843c07f040 msgr2=0x7f843c109d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:16:22.476 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.776+0000 7f8442a00700 1 -- 172.21.15.19:0/1909802055 shutdown_connections 2024-05-07T02:16:22.476 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.776+0000 7f8442a00700 1 -- 172.21.15.19:0/1909802055 wait complete. 2024-05-07T02:16:22.476 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.777+0000 7f8442a00700 1 Processor -- start 2024-05-07T02:16:22.476 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.778+0000 7f8442a00700 1 -- start start 2024-05-07T02:16:22.477 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.778+0000 7f8442a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f843c080050 0x7f843c1a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:16:22.477 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.778+0000 7f8442a00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f843c081700 con 0x7f843c080050 2024-05-07T02:16:22.477 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.778+0000 7f8441600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f843c080050 0x7f843c1a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:16:22.477 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.778+0000 7f8441600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f843c080050 0x7f843c1a3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37000/0 (socket says 172.21.15.19:37000) 2024-05-07T02:16:22.477 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.778+0000 7f8441600700 1 -- 172.21.15.19:0/1395499806 learned_addr learned my addr 172.21.15.19:0/1395499806 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:16:22.477 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.779+0000 7f8441600700 1 -- 172.21.15.19:0/1395499806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f843c1a3ea0 con 0x7f843c080050 2024-05-07T02:16:22.479 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.779+0000 7f8441600700 1 --2- 172.21.15.19:0/1395499806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f843c080050 0x7f843c1a3960 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f8430000c00 tx=0x7f8430004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:16:22.479 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.779+0000 7f843a000700 1 -- 172.21.15.19:0/1395499806 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8430003db0 con 0x7f843c080050 2024-05-07T02:16:22.479 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.779+0000 7f843a000700 1 -- 172.21.15.19:0/1395499806 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8430011630 con 0x7f843c080050 2024-05-07T02:16:22.479 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.779+0000 7f8442a00700 1 -- 172.21.15.19:0/1395499806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f843c1a4130 con 0x7f843c080050 2024-05-07T02:16:22.479 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.779+0000 7f843a000700 1 -- 172.21.15.19:0/1395499806 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8430011950 con 0x7f843c080050 2024-05-07T02:16:22.480 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.780+0000 7f8442a00700 1 -- 172.21.15.19:0/1395499806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f843c1a6d40 con 0x7f843c080050 2024-05-07T02:16:22.480 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.780+0000 7f843a000700 1 -- 172.21.15.19:0/1395499806 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f843001a460 con 0x7f843c080050 2024-05-07T02:16:22.480 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.780+0000 7f843a000700 1 -- 172.21.15.19:0/1395499806 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f843001b780 con 0x7f843c080050 2024-05-07T02:16:22.480 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.781+0000 7f8442a00700 1 -- 172.21.15.19:0/1395499806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f843c064e20 con 0x7f843c080050 2024-05-07T02:16:22.480 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.784+0000 7f843a000700 1 -- 172.21.15.19:0/1395499806 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f8430004260 con 0x7f843c080050 2024-05-07T02:16:22.480 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.863+0000 7f8442a00700 1 -- 172.21.15.19:0/1395499806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f843c02e8c0 con 0x7f843c080050 2024-05-07T02:16:22.480 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.876+0000 7f843a000700 1 -- 172.21.15.19:0/1395499806 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+465 (secure 0 0 0) 0x7f843001a750 con 0x7f843c080050 2024-05-07T02:16:22.480 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.877+0000 7f843a000700 1 -- 172.21.15.19:0/1395499806 <== mon.0 v2:172.21.15.19:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f843001be10 con 0x7f843c080050 2024-05-07T02:16:22.480 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.880+0000 7f8442a00700 1 -- 172.21.15.19:0/1395499806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f843c080050 msgr2=0x7f843c1a3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:16:22.480 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.880+0000 7f8442a00700 1 --2- 172.21.15.19:0/1395499806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f843c080050 0x7f843c1a3960 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f8430000c00 tx=0x7f8430004750 comp rx=0 tx=0).stop 2024-05-07T02:16:22.480 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.880+0000 7f8442a00700 1 -- 172.21.15.19:0/1395499806 shutdown_connections 2024-05-07T02:16:22.480 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.880+0000 7f8442a00700 1 --2- 172.21.15.19:0/1395499806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f843c080050 0x7f843c1a3960 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:16:22.480 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.880+0000 7f8442a00700 1 -- 172.21.15.19:0/1395499806 >> 172.21.15.19:0/1395499806 conn(0x7f843c07f040 msgr2=0x7f843c109540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:16:22.480 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.880+0000 7f8442a00700 1 -- 172.21.15.19:0/1395499806 shutdown_connections 2024-05-07T02:16:22.481 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:21.880+0000 7f8442a00700 1 -- 172.21.15.19:0/1395499806 wait complete. 2024-05-07T02:16:22.481 INFO:teuthology.orchestra.run.smithi019.stdout:Generating new minimal ceph.conf... 2024-05-07T02:16:23.605 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:23 smithi019 ceph-mon[29404]: from='client.? 172.21.15.19:0/3114396816' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:16:23.909 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:23.161+0000 7f1019e00700 1 Processor -- start 2024-05-07T02:16:23.909 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:23.161+0000 7f1019e00700 1 -- start start 2024-05-07T02:16:23.909 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:23.162+0000 7f1019e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1014110700 0x7f1014112b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:16:23.909 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:23.162+0000 7f1019e00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1014113060 con 0x7f1014110700 2024-05-07T02:16:23.910 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:23.162+0000 7f1018a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1014110700 0x7f1014112b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:16:23.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:23.162+0000 7f1018a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1014110700 0x7f1014112b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37014/0 (socket says 172.21.15.19:37014) 2024-05-07T02:16:23.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:23.162+0000 7f1018a00700 1 -- 172.21.15.19:0/267449252 learned_addr learned my addr 172.21.15.19:0/267449252 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:16:23.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:23.163+0000 7f1018a00700 1 -- 172.21.15.19:0/267449252 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10141131e0 con 0x7f1014110700 2024-05-07T02:16:23.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:23.163+0000 7f1018a00700 1 --2- 172.21.15.19:0/267449252 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1014110700 0x7f1014112b20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f10080098a0 tx=0x7f1008009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ac4645f7c2ca20 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:16:23.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:23.163+0000 7f1013400700 1 -- 172.21.15.19:0/267449252 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1008004160 con 0x7f1014110700 2024-05-07T02:16:23.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:23.163+0000 7f1013400700 1 -- 172.21.15.19:0/267449252 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1008004300 con 0x7f1014110700 2024-05-07T02:16:23.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:23.163+0000 7f1013400700 1 -- 172.21.15.19:0/267449252 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1008011af0 con 0x7f1014110700 2024-05-07T02:16:23.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:23.164+0000 7f1019e00700 1 -- 172.21.15.19:0/267449252 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1014110700 msgr2=0x7f1014112b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:16:23.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:23.164+0000 7f1019e00700 1 --2- 172.21.15.19:0/267449252 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1014110700 0x7f1014112b20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f10080098a0 tx=0x7f1008009bb0 comp rx=0 tx=0).stop 2024-05-07T02:16:23.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:23.165+0000 7f1019e00700 1 -- 172.21.15.19:0/267449252 shutdown_connections 2024-05-07T02:16:23.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:23.165+0000 7f1019e00700 1 --2- 172.21.15.19:0/267449252 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1014110700 0x7f1014112b20 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:16:23.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:23.165+0000 7f1019e00700 1 -- 172.21.15.19:0/267449252 >> 172.21.15.19:0/267449252 conn(0x7f101407f040 msgr2=0x7f1014081460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:16:23.914 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:23.165+0000 7f1019e00700 1 -- 172.21.15.19:0/267449252 shutdown_connections 2024-05-07T02:16:23.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:23.165+0000 7f1019e00700 1 -- 172.21.15.19:0/267449252 wait complete. 2024-05-07T02:16:23.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:23.166+0000 7f1019e00700 1 Processor -- start 2024-05-07T02:16:23.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:23.166+0000 7f1019e00700 1 -- start start 2024-05-07T02:16:23.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:23.167+0000 7f1019e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1014110700 0x7f10141a4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:16:23.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:23.167+0000 7f1019e00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1014114260 con 0x7f1014110700 2024-05-07T02:16:23.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:23.167+0000 7f1018a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1014110700 0x7f10141a4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:16:23.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:23.167+0000 7f1018a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1014110700 0x7f10141a4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37022/0 (socket says 172.21.15.19:37022) 2024-05-07T02:16:23.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:23.167+0000 7f1018a00700 1 -- 172.21.15.19:0/3114396816 learned_addr learned my addr 172.21.15.19:0/3114396816 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:16:23.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:23.167+0000 7f1018a00700 1 -- 172.21.15.19:0/3114396816 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10141a4740 con 0x7f1014110700 2024-05-07T02:16:23.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:23.168+0000 7f1018a00700 1 --2- 172.21.15.19:0/3114396816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1014110700 0x7f10141a4200 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f10080117b0 tx=0x7f1008011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:16:23.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:23.168+0000 7f1011600700 1 -- 172.21.15.19:0/3114396816 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10080039c0 con 0x7f1014110700 2024-05-07T02:16:23.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:23.168+0000 7f1011600700 1 -- 172.21.15.19:0/3114396816 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1008003b60 con 0x7f1014110700 2024-05-07T02:16:23.915 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:23.168+0000 7f1019e00700 1 -- 172.21.15.19:0/3114396816 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10141a49d0 con 0x7f1014110700 2024-05-07T02:16:23.916 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:23.168+0000 7f1011600700 1 -- 172.21.15.19:0/3114396816 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10080196a0 con 0x7f1014110700 2024-05-07T02:16:23.916 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:23.168+0000 7f1019e00700 1 -- 172.21.15.19:0/3114396816 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10141a75e0 con 0x7f1014110700 2024-05-07T02:16:23.916 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:23.169+0000 7f1011600700 1 -- 172.21.15.19:0/3114396816 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1008019ca0 con 0x7f1014110700 2024-05-07T02:16:23.916 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:23.169+0000 7f1011600700 1 -- 172.21.15.19:0/3114396816 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1008024730 con 0x7f1014110700 2024-05-07T02:16:23.916 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:23.170+0000 7f1019e00700 1 -- 172.21.15.19:0/3114396816 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1014064e20 con 0x7f1014110700 2024-05-07T02:16:23.916 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:23.173+0000 7f1011600700 1 -- 172.21.15.19:0/3114396816 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f1008043ec0 con 0x7f1014110700 2024-05-07T02:16:23.916 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:23.251+0000 7f1019e00700 1 -- 172.21.15.19:0/3114396816 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f10141a7800 con 0x7f1014110700 2024-05-07T02:16:23.916 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:23.252+0000 7f1011600700 1 -- 172.21.15.19:0/3114396816 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f1008032020 con 0x7f1014110700 2024-05-07T02:16:23.916 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:23.253+0000 7f1019e00700 1 -- 172.21.15.19:0/3114396816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1014110700 msgr2=0x7f10141a4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:16:23.916 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:23.253+0000 7f1019e00700 1 --2- 172.21.15.19:0/3114396816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1014110700 0x7f10141a4200 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f10080117b0 tx=0x7f1008011b70 comp rx=0 tx=0).stop 2024-05-07T02:16:23.916 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:23.253+0000 7f1019e00700 1 -- 172.21.15.19:0/3114396816 shutdown_connections 2024-05-07T02:16:23.917 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:23.253+0000 7f1019e00700 1 --2- 172.21.15.19:0/3114396816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1014110700 0x7f10141a4200 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:16:23.917 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:23.253+0000 7f1019e00700 1 -- 172.21.15.19:0/3114396816 >> 172.21.15.19:0/3114396816 conn(0x7f101407f040 msgr2=0x7f10140808c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:16:23.917 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:23.253+0000 7f1019e00700 1 -- 172.21.15.19:0/3114396816 shutdown_connections 2024-05-07T02:16:23.917 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:23.253+0000 7f1019e00700 1 -- 172.21.15.19:0/3114396816 wait complete. 2024-05-07T02:16:23.917 INFO:teuthology.orchestra.run.smithi019.stdout:Restarting the monitor... 2024-05-07T02:16:24.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:23 smithi019 systemd[1]: Stopping Ceph mon.a for 9ff84c16-0c17-11ef-bc97-c7b262605968... 2024-05-07T02:16:24.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:24 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mon-a[29400]: 2024-05-07T02:16:24.432+0000 7f47f3c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-07T02:16:24.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:24 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mon-a[29400]: 2024-05-07T02:16:24.432+0000 7f47f3c00700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-07T02:16:24.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:24 smithi019 podman[29646]: 2024-05-07 02:16:24.485459929 +0000 UTC m=+0.231138160 container died 9acffa6e15afbfdc64f4c5de4cb7526f1bebb1d9a6523c662a3da1fe1de4f519 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mon-a, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True) 2024-05-07T02:16:25.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:25 smithi019 podman[29646]: 2024-05-07 02:16:25.258149985 +0000 UTC m=+1.003828240 container cleanup 9acffa6e15afbfdc64f4c5de4cb7526f1bebb1d9a6523c662a3da1fe1de4f519 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-07T02:16:25.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:25 smithi019 bash[29646]: ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mon-a 2024-05-07T02:16:25.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:25 smithi019 podman[29660]: 2024-05-07 02:16:25.42555348 +0000 UTC m=+0.924957568 container remove 9acffa6e15afbfdc64f4c5de4cb7526f1bebb1d9a6523c662a3da1fe1de4f519 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5) 2024-05-07T02:16:26.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:25 smithi019 systemd[1]: ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-07T02:16:26.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:25 smithi019 systemd[1]: Stopped Ceph mon.a for 9ff84c16-0c17-11ef-bc97-c7b262605968. 2024-05-07T02:16:26.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:25 smithi019 systemd[1]: ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mon.a.service: Consumed 2.075s CPU time. 2024-05-07T02:16:26.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:25 smithi019 systemd[1]: Starting Ceph mon.a for 9ff84c16-0c17-11ef-bc97-c7b262605968... 2024-05-07T02:16:26.453 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 podman[29754]: 2024-05-07 02:16:26.214459628 +0000 UTC m=+0.033888114 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 2024-05-07T02:16:26.453 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 podman[29754]: 2024-05-07 02:16:26.318245148 +0000 UTC m=+0.137673621 container create 6ca858b3ad851c88261b42f04e44b87c84c0d6606971b4d88acda280161d24c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mon-a, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-07T02:16:26.653 INFO:teuthology.orchestra.run.smithi019.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-07T02:16:26.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 podman[29754]: 2024-05-07 02:16:26.547057377 +0000 UTC m=+0.366485895 container init 6ca858b3ad851c88261b42f04e44b87c84c0d6606971b4d88acda280161d24c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a) 2024-05-07T02:16:26.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 podman[29754]: 2024-05-07 02:16:26.556840197 +0000 UTC m=+0.376268691 container start 6ca858b3ad851c88261b42f04e44b87c84c0d6606971b4d88acda280161d24c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-07T02:16:26.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 bash[29754]: 6ca858b3ad851c88261b42f04e44b87c84c0d6606971b4d88acda280161d24c4 2024-05-07T02:16:26.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 systemd[1]: Started Ceph mon.a for 9ff84c16-0c17-11ef-bc97-c7b262605968. 2024-05-07T02:16:26.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: set uid:gid to 167:167 (ceph:ceph) 2024-05-07T02:16:26.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: ceph version 19.0.0-3522-gbd4e00a3 (bd4e00a39d766328b458e4c953f4d722fed96a31) squid (dev), process ceph-mon, pid 2 2024-05-07T02:16:26.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: pidfile_write: ignore empty --pid-file 2024-05-07T02:16:26.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: load: jerasure load: lrc 2024-05-07T02:16:26.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: RocksDB version: 7.9.2 2024-05-07T02:16:26.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Git sha 0 2024-05-07T02:16:26.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Compile date 2024-05-03 18:51:23 2024-05-07T02:16:26.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: DB SUMMARY 2024-05-07T02:16:26.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: DB Session ID: KOM0C9ITPI2IWOHO8UHJ 2024-05-07T02:16:26.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: CURRENT file: CURRENT 2024-05-07T02:16:26.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: IDENTITY file: IDENTITY 2024-05-07T02:16:26.761 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-07T02:16:26.762 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-07T02:16:26.762 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 103974 ; 2024-05-07T02:16:26.762 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.error_if_exists: 0 2024-05-07T02:16:26.762 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.create_if_missing: 0 2024-05-07T02:16:26.762 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.paranoid_checks: 1 2024-05-07T02:16:26.762 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-07T02:16:26.762 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-07T02:16:26.762 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-07T02:16:26.762 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.env: 0x55c843a96500 2024-05-07T02:16:26.762 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.fs: PosixFileSystem 2024-05-07T02:16:26.764 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.info_log: 0x55c85d6e0060 2024-05-07T02:16:26.764 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.max_file_opening_threads: 16 2024-05-07T02:16:26.764 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.statistics: (nil) 2024-05-07T02:16:26.764 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.use_fsync: 0 2024-05-07T02:16:26.764 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.max_log_file_size: 0 2024-05-07T02:16:26.764 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-07T02:16:26.764 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-07T02:16:26.764 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.keep_log_file_num: 1000 2024-05-07T02:16:26.764 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.recycle_log_file_num: 0 2024-05-07T02:16:26.764 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.allow_fallocate: 1 2024-05-07T02:16:26.764 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.allow_mmap_reads: 0 2024-05-07T02:16:26.765 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.allow_mmap_writes: 0 2024-05-07T02:16:26.765 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.use_direct_reads: 0 2024-05-07T02:16:26.765 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-07T02:16:26.765 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.create_missing_column_families: 0 2024-05-07T02:16:26.765 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.db_log_dir: 2024-05-07T02:16:26.765 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.wal_dir: 2024-05-07T02:16:26.765 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-07T02:16:26.765 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-07T02:16:26.765 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-07T02:16:26.765 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-07T02:16:26.765 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-07T02:16:26.765 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-07T02:16:26.765 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.advise_random_on_open: 1 2024-05-07T02:16:26.765 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.db_write_buffer_size: 0 2024-05-07T02:16:26.765 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.write_buffer_manager: 0x55c85d5aa8c0 2024-05-07T02:16:26.766 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-07T02:16:26.766 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-07T02:16:26.766 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-07T02:16:26.766 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.rate_limiter: (nil) 2024-05-07T02:16:26.766 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-07T02:16:26.766 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.wal_recovery_mode: 2 2024-05-07T02:16:26.766 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.enable_thread_tracking: 0 2024-05-07T02:16:26.766 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.enable_pipelined_write: 0 2024-05-07T02:16:26.766 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.unordered_write: 0 2024-05-07T02:16:26.766 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-07T02:16:26.766 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-07T02:16:26.766 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-07T02:16:26.766 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-07T02:16:26.767 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.row_cache: None 2024-05-07T02:16:26.767 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.wal_filter: None 2024-05-07T02:16:26.767 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-07T02:16:26.767 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.allow_ingest_behind: 0 2024-05-07T02:16:26.767 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.two_write_queues: 0 2024-05-07T02:16:26.767 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.manual_wal_flush: 0 2024-05-07T02:16:26.767 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.wal_compression: 0 2024-05-07T02:16:26.767 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.atomic_flush: 0 2024-05-07T02:16:26.767 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-07T02:16:26.767 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-07T02:16:26.767 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-07T02:16:26.767 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.log_readahead_size: 0 2024-05-07T02:16:26.767 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-07T02:16:26.767 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.best_efforts_recovery: 0 2024-05-07T02:16:26.768 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-07T02:16:26.768 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-07T02:16:26.768 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.allow_data_in_errors: 0 2024-05-07T02:16:26.768 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.db_host_id: __hostname__ 2024-05-07T02:16:26.768 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-07T02:16:26.768 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.max_background_jobs: 2 2024-05-07T02:16:26.768 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.max_background_compactions: -1 2024-05-07T02:16:26.768 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.max_subcompactions: 1 2024-05-07T02:16:26.768 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-07T02:16:26.768 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-07T02:16:26.768 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-07T02:16:26.768 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.max_total_wal_size: 0 2024-05-07T02:16:26.768 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-07T02:16:26.768 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-07T02:16:26.768 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-07T02:16:26.769 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-07T02:16:26.769 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.max_open_files: -1 2024-05-07T02:16:26.769 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.bytes_per_sync: 0 2024-05-07T02:16:26.769 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-07T02:16:26.769 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-07T02:16:26.769 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.compaction_readahead_size: 0 2024-05-07T02:16:26.769 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.max_background_flushes: -1 2024-05-07T02:16:26.769 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Compression algorithms supported: 2024-05-07T02:16:26.769 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-07T02:16:26.769 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: kZSTD supported: 0 2024-05-07T02:16:26.769 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: kXpressCompression supported: 0 2024-05-07T02:16:26.769 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: kLZ4HCCompression supported: 1 2024-05-07T02:16:26.769 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: kZlibCompression supported: 1 2024-05-07T02:16:26.770 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: kSnappyCompression supported: 1 2024-05-07T02:16:26.770 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: kLZ4Compression supported: 1 2024-05-07T02:16:26.770 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: kBZip2Compression supported: 0 2024-05-07T02:16:26.770 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-07T02:16:26.770 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-07T02:16:26.770 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-07T02:16:26.770 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-07T02:16:26.770 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-07T02:16:26.770 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.merge_operator: 2024-05-07T02:16:26.770 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.compaction_filter: None 2024-05-07T02:16:26.770 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.compaction_filter_factory: None 2024-05-07T02:16:26.770 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.sst_partitioner_factory: None 2024-05-07T02:16:26.770 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-07T02:16:26.770 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-07T02:16:26.771 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c85d6e0220) 2024-05-07T02:16:26.771 INFO:journalctl@ceph.mon.a.smithi019.stdout: cache_index_and_filter_blocks: 1 2024-05-07T02:16:26.771 INFO:journalctl@ceph.mon.a.smithi019.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-07T02:16:26.771 INFO:journalctl@ceph.mon.a.smithi019.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-07T02:16:26.771 INFO:journalctl@ceph.mon.a.smithi019.stdout: pin_top_level_index_and_filter: 1 2024-05-07T02:16:26.771 INFO:journalctl@ceph.mon.a.smithi019.stdout: index_type: 0 2024-05-07T02:16:26.771 INFO:journalctl@ceph.mon.a.smithi019.stdout: data_block_index_type: 0 2024-05-07T02:16:26.771 INFO:journalctl@ceph.mon.a.smithi019.stdout: index_shortening: 1 2024-05-07T02:16:26.771 INFO:journalctl@ceph.mon.a.smithi019.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-07T02:16:26.771 INFO:journalctl@ceph.mon.a.smithi019.stdout: checksum: 4 2024-05-07T02:16:26.771 INFO:journalctl@ceph.mon.a.smithi019.stdout: no_block_cache: 0 2024-05-07T02:16:26.771 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_cache: 0x55c85d5dd1f0 2024-05-07T02:16:26.771 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_cache_name: BinnedLRUCache 2024-05-07T02:16:26.771 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_cache_options: 2024-05-07T02:16:26.772 INFO:journalctl@ceph.mon.a.smithi019.stdout: capacity : 536870912 2024-05-07T02:16:26.772 INFO:journalctl@ceph.mon.a.smithi019.stdout: num_shard_bits : 4 2024-05-07T02:16:26.772 INFO:journalctl@ceph.mon.a.smithi019.stdout: strict_capacity_limit : 0 2024-05-07T02:16:26.772 INFO:journalctl@ceph.mon.a.smithi019.stdout: high_pri_pool_ratio: 0.000 2024-05-07T02:16:26.772 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_cache_compressed: (nil) 2024-05-07T02:16:26.772 INFO:journalctl@ceph.mon.a.smithi019.stdout: persistent_cache: (nil) 2024-05-07T02:16:26.772 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_size: 4096 2024-05-07T02:16:26.772 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_size_deviation: 10 2024-05-07T02:16:26.772 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_restart_interval: 16 2024-05-07T02:16:26.772 INFO:journalctl@ceph.mon.a.smithi019.stdout: index_block_restart_interval: 1 2024-05-07T02:16:26.772 INFO:journalctl@ceph.mon.a.smithi019.stdout: metadata_block_size: 4096 2024-05-07T02:16:26.772 INFO:journalctl@ceph.mon.a.smithi019.stdout: partition_filters: 0 2024-05-07T02:16:26.772 INFO:journalctl@ceph.mon.a.smithi019.stdout: use_delta_encoding: 1 2024-05-07T02:16:26.772 INFO:journalctl@ceph.mon.a.smithi019.stdout: filter_policy: bloomfilter 2024-05-07T02:16:26.773 INFO:journalctl@ceph.mon.a.smithi019.stdout: whole_key_filtering: 1 2024-05-07T02:16:26.773 INFO:journalctl@ceph.mon.a.smithi019.stdout: verify_compression: 0 2024-05-07T02:16:26.773 INFO:journalctl@ceph.mon.a.smithi019.stdout: read_amp_bytes_per_bit: 0 2024-05-07T02:16:26.773 INFO:journalctl@ceph.mon.a.smithi019.stdout: format_version: 5 2024-05-07T02:16:26.773 INFO:journalctl@ceph.mon.a.smithi019.stdout: enable_index_compression: 1 2024-05-07T02:16:26.773 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_align: 0 2024-05-07T02:16:26.773 INFO:journalctl@ceph.mon.a.smithi019.stdout: max_auto_readahead_size: 262144 2024-05-07T02:16:26.773 INFO:journalctl@ceph.mon.a.smithi019.stdout: prepopulate_block_cache: 0 2024-05-07T02:16:26.773 INFO:journalctl@ceph.mon.a.smithi019.stdout: initial_auto_readahead_size: 8192 2024-05-07T02:16:26.773 INFO:journalctl@ceph.mon.a.smithi019.stdout: num_file_reads_for_auto_readahead: 2 2024-05-07T02:16:26.773 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.write_buffer_size: 33554432 2024-05-07T02:16:26.773 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.max_write_buffer_number: 2 2024-05-07T02:16:26.773 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.compression: NoCompression 2024-05-07T02:16:26.773 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.bottommost_compression: Disabled 2024-05-07T02:16:26.773 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.prefix_extractor: nullptr 2024-05-07T02:16:26.773 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-07T02:16:26.773 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.num_levels: 7 2024-05-07T02:16:26.774 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-07T02:16:26.774 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-07T02:16:26.774 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-07T02:16:26.774 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-07T02:16:26.774 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-07T02:16:26.774 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-07T02:16:26.774 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-07T02:16:26.774 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-07T02:16:26.774 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-07T02:16:26.774 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-07T02:16:26.774 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-07T02:16:26.774 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-07T02:16:26.774 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-07T02:16:26.774 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.compression_opts.level: 32767 2024-05-07T02:16:26.775 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.compression_opts.strategy: 0 2024-05-07T02:16:26.775 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-07T02:16:26.775 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-07T02:16:26.775 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-07T02:16:26.775 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-07T02:16:26.775 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.compression_opts.enabled: false 2024-05-07T02:16:26.775 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-07T02:16:26.775 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-07T02:16:26.775 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-07T02:16:26.775 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-07T02:16:26.775 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.target_file_size_base: 67108864 2024-05-07T02:16:26.775 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-07T02:16:26.775 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-07T02:16:26.775 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-07T02:16:26.775 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-07T02:16:26.776 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-07T02:16:26.776 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-07T02:16:26.776 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-07T02:16:26.776 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-07T02:16:26.776 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-07T02:16:26.776 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-07T02:16:26.776 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-07T02:16:26.776 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-07T02:16:26.776 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-07T02:16:26.776 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-07T02:16:26.776 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.arena_block_size: 1048576 2024-05-07T02:16:26.776 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-07T02:16:26.776 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-07T02:16:26.776 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.disable_auto_compactions: 0 2024-05-07T02:16:26.776 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-07T02:16:26.777 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-07T02:16:26.777 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-07T02:16:26.777 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-07T02:16:26.777 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-07T02:16:26.777 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-07T02:16:26.777 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-07T02:16:26.777 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-07T02:16:26.777 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-07T02:16:26.777 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-07T02:16:26.777 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-07T02:16:26.777 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.inplace_update_support: 0 2024-05-07T02:16:26.777 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-07T02:16:26.777 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-07T02:16:26.777 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-07T02:16:26.777 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-07T02:16:26.777 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.bloom_locality: 0 2024-05-07T02:16:26.777 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.max_successive_merges: 0 2024-05-07T02:16:26.778 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-07T02:16:26.778 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.paranoid_file_checks: 0 2024-05-07T02:16:26.778 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.force_consistency_checks: 1 2024-05-07T02:16:26.778 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.report_bg_io_stats: 0 2024-05-07T02:16:26.778 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.ttl: 2592000 2024-05-07T02:16:26.778 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-07T02:16:26.778 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-07T02:16:26.778 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-07T02:16:26.778 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.enable_blob_files: false 2024-05-07T02:16:26.778 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.min_blob_size: 0 2024-05-07T02:16:26.778 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.blob_file_size: 268435456 2024-05-07T02:16:26.778 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-07T02:16:26.778 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-07T02:16:26.778 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-07T02:16:26.779 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-07T02:16:26.779 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-07T02:16:26.779 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.blob_file_starting_level: 0 2024-05-07T02:16:26.779 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-07T02:16:26.779 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-07T02:16:26.779 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-07T02:16:26.779 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: cd923d23-e8af-4923-9b19-9f37519ab92e 2024-05-07T02:16:26.779 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715048186690413, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-07T02:16:26.779 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-07T02:16:26.779 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715048186692646, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99208, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 322, "table_properties": {"data_size": 97167, "index_size": 297, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13897, "raw_average_key_size": 49, "raw_value_size": 89562, "raw_average_value_size": 318, "num_data_blocks": 12, "num_entries": 281, "num_filter_entries": 281, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715048186, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "cd923d23-e8af-4923-9b19-9f37519ab92e", "db_session_id": "KOM0C9ITPI2IWOHO8UHJ", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-07T02:16:26.779 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715048186692813, "job": 1, "event": "recovery_finished"} 2024-05-07T02:16:26.779 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-07T02:16:26.779 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-07T02:16:26.779 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55c85d5c1c00 2024-05-07T02:16:26.780 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: DB pointer 0x55c8623b4000 2024-05-07T02:16:26.780 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-07T02:16:26.780 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-07T02:16:26.780 INFO:journalctl@ceph.mon.a.smithi019.stdout: ** DB Stats ** 2024-05-07T02:16:26.780 INFO:journalctl@ceph.mon.a.smithi019.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-07T02:16:26.780 INFO:journalctl@ceph.mon.a.smithi019.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-07T02:16:26.780 INFO:journalctl@ceph.mon.a.smithi019.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-07T02:16:26.780 INFO:journalctl@ceph.mon.a.smithi019.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-07T02:16:26.780 INFO:journalctl@ceph.mon.a.smithi019.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-07T02:16:26.780 INFO:journalctl@ceph.mon.a.smithi019.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-07T02:16:26.780 INFO:journalctl@ceph.mon.a.smithi019.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-07T02:16:26.780 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-05-07T02:16:26.780 INFO:journalctl@ceph.mon.a.smithi019.stdout: ** Compaction Stats [default] ** 2024-05-07T02:16:26.780 INFO:journalctl@ceph.mon.a.smithi019.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-07T02:16:26.780 INFO:journalctl@ceph.mon.a.smithi019.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-07T02:16:26.780 INFO:journalctl@ceph.mon.a.smithi019.stdout: L0 2/0 98.74 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 65.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-07T02:16:26.781 INFO:journalctl@ceph.mon.a.smithi019.stdout: Sum 2/0 98.74 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 65.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-07T02:16:26.781 INFO:journalctl@ceph.mon.a.smithi019.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 65.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-07T02:16:26.781 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-05-07T02:16:26.781 INFO:journalctl@ceph.mon.a.smithi019.stdout: ** Compaction Stats [default] ** 2024-05-07T02:16:26.781 INFO:journalctl@ceph.mon.a.smithi019.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-07T02:16:26.781 INFO:journalctl@ceph.mon.a.smithi019.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-07T02:16:26.781 INFO:journalctl@ceph.mon.a.smithi019.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 65.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-07T02:16:26.781 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-05-07T02:16:26.781 INFO:journalctl@ceph.mon.a.smithi019.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-07T02:16:26.781 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-05-07T02:16:26.781 INFO:journalctl@ceph.mon.a.smithi019.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-07T02:16:26.781 INFO:journalctl@ceph.mon.a.smithi019.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-07T02:16:26.781 INFO:journalctl@ceph.mon.a.smithi019.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-07T02:16:26.781 INFO:journalctl@ceph.mon.a.smithi019.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-07T02:16:26.781 INFO:journalctl@ceph.mon.a.smithi019.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-07T02:16:26.781 INFO:journalctl@ceph.mon.a.smithi019.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-07T02:16:26.782 INFO:journalctl@ceph.mon.a.smithi019.stdout: Cumulative compaction: 0.00 GB write, 11.99 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-07T02:16:26.782 INFO:journalctl@ceph.mon.a.smithi019.stdout: Interval compaction: 0.00 GB write, 11.99 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-07T02:16:26.782 INFO:journalctl@ceph.mon.a.smithi019.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-07T02:16:26.782 INFO:journalctl@ceph.mon.a.smithi019.stdout: Block cache BinnedLRUCache@0x55c85d5dd1f0#2 capacity: 512.00 MB usage: 1.41 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-05-07T02:16:26.782 INFO:journalctl@ceph.mon.a.smithi019.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.91 KB,0.000172853%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-07T02:16:26.782 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-05-07T02:16:26.782 INFO:journalctl@ceph.mon.a.smithi019.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-07T02:16:26.782 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: starting mon.a rank 0 at public addrs [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] at bind addrs [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 2024-05-07T02:16:26.782 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: mon.a@-1(???) e1 preinit fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 2024-05-07T02:16:26.782 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: mon.a@-1(???).mds e1 new map 2024-05-07T02:16:26.782 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: mon.a@-1(???).mds e1 print_map 2024-05-07T02:16:26.782 INFO:journalctl@ceph.mon.a.smithi019.stdout: e1 2024-05-07T02:16:26.782 INFO:journalctl@ceph.mon.a.smithi019.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-07T02:16:26.782 INFO:journalctl@ceph.mon.a.smithi019.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-07T02:16:26.782 INFO:journalctl@ceph.mon.a.smithi019.stdout: legacy client fscid: -1 2024-05-07T02:16:26.782 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-05-07T02:16:26.783 INFO:journalctl@ceph.mon.a.smithi019.stdout: No filesystems configured 2024-05-07T02:16:26.783 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-07T02:16:26.783 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-07T02:16:26.783 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-07T02:16:26.783 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-07T02:16:26.783 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-07T02:16:26.783 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-07T02:16:26.783 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: monmap epoch 1 2024-05-07T02:16:26.783 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 2024-05-07T02:16:26.783 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: last_changed 2024-05-07T02:16:14.906770+0000 2024-05-07T02:16:26.783 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: created 2024-05-07T02:16:14.906770+0000 2024-05-07T02:16:26.783 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: min_mon_release 19 (squid) 2024-05-07T02:16:26.783 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: election_strategy: 1 2024-05-07T02:16:26.783 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-05-07T02:16:26.783 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: fsmap 2024-05-07T02:16:26.783 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: osdmap e1: 0 total, 0 up, 0 in 2024-05-07T02:16:26.784 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:26 smithi019 ceph-mon[29781]: mgrmap e1: no daemons active 2024-05-07T02:16:28.175 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:27.419+0000 7f3d52200700 1 Processor -- start 2024-05-07T02:16:28.175 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:27.419+0000 7f3d52200700 1 -- start start 2024-05-07T02:16:28.175 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:27.420+0000 7f3d52200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d4c1089b0 0x7f3d4c108dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:16:28.175 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:27.420+0000 7f3d52200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4c1093b0 con 0x7f3d4c1089b0 2024-05-07T02:16:28.176 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:27.420+0000 7f3d4b400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d4c1089b0 0x7f3d4c108dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:16:28.177 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:27.421+0000 7f3d4b400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d4c1089b0 0x7f3d4c108dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37032/0 (socket says 172.21.15.19:37032) 2024-05-07T02:16:28.177 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:27.421+0000 7f3d4b400700 1 -- 172.21.15.19:0/2047331055 learned_addr learned my addr 172.21.15.19:0/2047331055 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:16:28.177 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:27.421+0000 7f3d4b400700 1 -- 172.21.15.19:0/2047331055 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d4c109ae0 con 0x7f3d4c1089b0 2024-05-07T02:16:28.177 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:27.421+0000 7f3d4b400700 1 --2- 172.21.15.19:0/2047331055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d4c1089b0 0x7f3d4c108dc0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c0098a0 tx=0x7f3d3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af8d13a0efd5d0cd server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:16:28.177 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:27.422+0000 7f3d4a000700 1 -- 172.21.15.19:0/2047331055 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d3c004160 con 0x7f3d4c1089b0 2024-05-07T02:16:28.177 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:27.422+0000 7f3d4a000700 1 -- 172.21.15.19:0/2047331055 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3d3c004300 con 0x7f3d4c1089b0 2024-05-07T02:16:28.177 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:27.422+0000 7f3d4a000700 1 -- 172.21.15.19:0/2047331055 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d3c011af0 con 0x7f3d4c1089b0 2024-05-07T02:16:28.178 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:27.423+0000 7f3d52200700 1 -- 172.21.15.19:0/2047331055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d4c1089b0 msgr2=0x7f3d4c108dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:16:28.178 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:27.423+0000 7f3d52200700 1 --2- 172.21.15.19:0/2047331055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d4c1089b0 0x7f3d4c108dc0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c0098a0 tx=0x7f3d3c009bb0 comp rx=0 tx=0).stop 2024-05-07T02:16:28.178 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:27.423+0000 7f3d52200700 1 -- 172.21.15.19:0/2047331055 shutdown_connections 2024-05-07T02:16:28.178 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:27.423+0000 7f3d52200700 1 --2- 172.21.15.19:0/2047331055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d4c1089b0 0x7f3d4c108dc0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:16:28.178 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:27.423+0000 7f3d52200700 1 -- 172.21.15.19:0/2047331055 >> 172.21.15.19:0/2047331055 conn(0x7f3d4c104050 msgr2=0x7f3d4c1064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:16:28.178 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:27.423+0000 7f3d52200700 1 -- 172.21.15.19:0/2047331055 shutdown_connections 2024-05-07T02:16:28.178 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:27.423+0000 7f3d52200700 1 -- 172.21.15.19:0/2047331055 wait complete. 2024-05-07T02:16:28.179 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:27.424+0000 7f3d52200700 1 Processor -- start 2024-05-07T02:16:28.179 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:27.424+0000 7f3d52200700 1 -- start start 2024-05-07T02:16:28.180 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:27.425+0000 7f3d52200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d4c1089b0 0x7f3d4c19bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:16:28.180 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:27.425+0000 7f3d52200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4c07c120 con 0x7f3d4c1089b0 2024-05-07T02:16:28.180 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:27.425+0000 7f3d4b400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d4c1089b0 0x7f3d4c19bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:16:28.180 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:27.425+0000 7f3d4b400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d4c1089b0 0x7f3d4c19bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37042/0 (socket says 172.21.15.19:37042) 2024-05-07T02:16:28.180 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:27.425+0000 7f3d4b400700 1 -- 172.21.15.19:0/2022847246 learned_addr learned my addr 172.21.15.19:0/2022847246 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:16:28.180 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:27.426+0000 7f3d4b400700 1 -- 172.21.15.19:0/2022847246 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d4c19c090 con 0x7f3d4c1089b0 2024-05-07T02:16:28.180 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:27.426+0000 7f3d4b400700 1 --2- 172.21.15.19:0/2022847246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d4c1089b0 0x7f3d4c19bb50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c011890 tx=0x7f3d3c011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:16:28.180 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:27.426+0000 7f3d43e00700 1 -- 172.21.15.19:0/2022847246 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d3c0039e0 con 0x7f3d4c1089b0 2024-05-07T02:16:28.180 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:27.426+0000 7f3d43e00700 1 -- 172.21.15.19:0/2022847246 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3d3c003b80 con 0x7f3d4c1089b0 2024-05-07T02:16:28.180 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:27.426+0000 7f3d52200700 1 -- 172.21.15.19:0/2022847246 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d4c19c320 con 0x7f3d4c1089b0 2024-05-07T02:16:28.181 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:27.426+0000 7f3d43e00700 1 -- 172.21.15.19:0/2022847246 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d3c024750 con 0x7f3d4c1089b0 2024-05-07T02:16:28.181 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:27.426+0000 7f3d52200700 1 -- 172.21.15.19:0/2022847246 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d4c19ef30 con 0x7f3d4c1089b0 2024-05-07T02:16:28.181 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:27.427+0000 7f3d43e00700 1 -- 172.21.15.19:0/2022847246 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3d3c022020 con 0x7f3d4c1089b0 2024-05-07T02:16:28.182 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:27.427+0000 7f3d43e00700 1 -- 172.21.15.19:0/2022847246 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3d3c02dd50 con 0x7f3d4c1089b0 2024-05-07T02:16:28.182 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:27.428+0000 7f3d52200700 1 -- 172.21.15.19:0/2022847246 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d4c064e20 con 0x7f3d4c1089b0 2024-05-07T02:16:28.182 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:27.431+0000 7f3d43e00700 1 -- 172.21.15.19:0/2022847246 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f3d3c01ab50 con 0x7f3d4c1089b0 2024-05-07T02:16:28.182 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:27.509+0000 7f3d52200700 1 -- 172.21.15.19:0/2022847246 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f3d4c194d60 con 0x7f3d4c1089b0 2024-05-07T02:16:28.182 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:27.514+0000 7f3d43e00700 1 -- 172.21.15.19:0/2022847246 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f3d3c02d3f0 con 0x7f3d4c1089b0 2024-05-07T02:16:28.182 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:27.515+0000 7f3d52200700 1 -- 172.21.15.19:0/2022847246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d4c1089b0 msgr2=0x7f3d4c19bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:16:28.182 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:27.515+0000 7f3d52200700 1 --2- 172.21.15.19:0/2022847246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d4c1089b0 0x7f3d4c19bb50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c011890 tx=0x7f3d3c011970 comp rx=0 tx=0).stop 2024-05-07T02:16:28.183 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:27.515+0000 7f3d52200700 1 -- 172.21.15.19:0/2022847246 shutdown_connections 2024-05-07T02:16:28.183 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:27.515+0000 7f3d52200700 1 --2- 172.21.15.19:0/2022847246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d4c1089b0 0x7f3d4c19bb50 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:16:28.183 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:27.515+0000 7f3d52200700 1 -- 172.21.15.19:0/2022847246 >> 172.21.15.19:0/2022847246 conn(0x7f3d4c104050 msgr2=0x7f3d4c191e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:16:28.183 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:27.515+0000 7f3d52200700 1 -- 172.21.15.19:0/2022847246 shutdown_connections 2024-05-07T02:16:28.183 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:27.515+0000 7f3d52200700 1 -- 172.21.15.19:0/2022847246 wait complete. 2024-05-07T02:16:28.183 INFO:teuthology.orchestra.run.smithi019.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-07T02:16:28.209 INFO:teuthology.orchestra.run.smithi019.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-07T02:16:28.209 INFO:teuthology.orchestra.run.smithi019.stdout:Creating mgr... 2024-05-07T02:16:28.211 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-07T02:16:28.212 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-07T02:16:28.643 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mgr.x 2024-05-07T02:16:28.643 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Failed to reset failed state of unit ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mgr.x.service: Unit ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mgr.x.service not loaded. 2024-05-07T02:16:28.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:28 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/2022847246' entity='client.admin' 2024-05-07T02:16:28.914 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-9ff84c16-0c17-11ef-bc97-c7b262605968.target.wants/ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@.service. 2024-05-07T02:16:29.341 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:28 smithi019 systemd[1]: Starting Ceph mgr.x for 9ff84c16-0c17-11ef-bc97-c7b262605968... 2024-05-07T02:16:29.601 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:29 smithi019 podman[29996]: 2024-05-07 02:16:29.246288407 +0000 UTC m=+0.038952517 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 2024-05-07T02:16:29.601 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:29 smithi019 podman[29996]: 2024-05-07 02:16:29.362480436 +0000 UTC m=+0.155144579 container create 4c3f8d297faab2f2313221454c39d5782efabd0930beca29d0ee3c3c00fc61fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0) 2024-05-07T02:16:29.601 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:29 smithi019 podman[29996]: 2024-05-07 02:16:29.590781191 +0000 UTC m=+0.383445337 container init 4c3f8d297faab2f2313221454c39d5782efabd0930beca29d0ee3c3c00fc61fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.license=GPLv2) 2024-05-07T02:16:29.601 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:29 smithi019 podman[29996]: 2024-05-07 02:16:29.601725179 +0000 UTC m=+0.394389345 container start 4c3f8d297faab2f2313221454c39d5782efabd0930beca29d0ee3c3c00fc61fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-07T02:16:29.719 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-07T02:16:29.719 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout disabled 2024-05-07T02:16:29.741 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-07T02:16:29.741 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-05-07T02:16:29.741 INFO:teuthology.orchestra.run.smithi019.stdout:firewalld.service is not enabled 2024-05-07T02:16:29.741 INFO:teuthology.orchestra.run.smithi019.stdout:Not possible to enable service . firewalld.service is not available 2024-05-07T02:16:29.759 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-07T02:16:29.759 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout disabled 2024-05-07T02:16:29.775 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-07T02:16:29.775 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-05-07T02:16:29.775 INFO:teuthology.orchestra.run.smithi019.stdout:firewalld.service is not enabled 2024-05-07T02:16:29.775 INFO:teuthology.orchestra.run.smithi019.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-07T02:16:29.775 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mgr to start... 2024-05-07T02:16:29.775 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mgr... 2024-05-07T02:16:29.895 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:29 smithi019 bash[29996]: 4c3f8d297faab2f2313221454c39d5782efabd0930beca29d0ee3c3c00fc61fc 2024-05-07T02:16:29.895 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:29 smithi019 systemd[1]: Started Ceph mgr.x for 9ff84c16-0c17-11ef-bc97-c7b262605968. 2024-05-07T02:16:30.202 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:29 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:29.928+0000 7fc5995a6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-07T02:16:30.202 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:30 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:30.202+0000 7fc5995a6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-07T02:16:30.491 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:30 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:30.338+0000 7fc5995a6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-07T02:16:31.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:30 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/2749819574' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-07T02:16:31.145 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:30 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:30.771+0000 7fc5995a6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-07T02:16:31.145 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:30 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:30.896+0000 7fc5995a6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-07T02:16:31.648 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-07T02:16:31.648 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-05-07T02:16:31.648 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsid": "9ff84c16-0c17-11ef-bc97-c7b262605968", 2024-05-07T02:16:31.648 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "health": { 2024-05-07T02:16:31.648 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-07T02:16:31.648 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-07T02:16:31.648 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-07T02:16:31.649 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-07T02:16:31.649 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-07T02:16:31.649 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-07T02:16:31.649 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 0 2024-05-07T02:16:31.649 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-05-07T02:16:31.649 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-07T02:16:31.649 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "a" 2024-05-07T02:16:31.649 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-05-07T02:16:31.649 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-05-07T02:16:31.649 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-07T02:16:31.649 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-07T02:16:31.649 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-07T02:16:31.650 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-07T02:16:31.650 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-07T02:16:31.650 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-07T02:16:31.650 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-07T02:16:31.650 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-07T02:16:31.650 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-07T02:16:31.650 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-07T02:16:31.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-07T02:16:31.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-07T02:16:31.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-07T02:16:31.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-07T02:16:31.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-07T02:16:31.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-07T02:16:31.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-07T02:16:31.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-07T02:16:31.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-07T02:16:31.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-07T02:16:31.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-07T02:16:31.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-07T02:16:31.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-07T02:16:31.659 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-07T02:16:31.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-07T02:16:31.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-07T02:16:31.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-07T02:16:31.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-07T02:16:31.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-07T02:16:31.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-07T02:16:31.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-07T02:16:31.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-07T02:16:31.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-07T02:16:31.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-07T02:16:31.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-07T02:16:31.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "restful" 2024-05-07T02:16:31.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-05-07T02:16:31.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-07T02:16:31.660 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-07T02:16:31.661 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-07T02:16:31.661 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-07T02:16:31.661 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modified": "2024-05-07T02:16:19.508731+0000", 2024-05-07T02:16:31.661 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-07T02:16:31.661 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-07T02:16:31.661 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-07T02:16:31.661 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-05-07T02:16:31.661 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:30.601+0000 7f971ca00700 1 Processor -- start 2024-05-07T02:16:31.661 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:30.601+0000 7f971ca00700 1 -- start start 2024-05-07T02:16:31.661 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:30.602+0000 7f971ca00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9718109ae0 0x7f9718109ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:16:31.661 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:30.602+0000 7f971ca00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f971810a4e0 con 0x7f9718109ae0 2024-05-07T02:16:31.661 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:30.602+0000 7f9715e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9718109ae0 0x7f9718109ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:16:31.661 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:30.602+0000 7f9715e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9718109ae0 0x7f9718109ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35386/0 (socket says 172.21.15.19:35386) 2024-05-07T02:16:31.661 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:30.603+0000 7f9715e00700 1 -- 172.21.15.19:0/2107557583 learned_addr learned my addr 172.21.15.19:0/2107557583 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:16:31.661 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:30.603+0000 7f9715e00700 1 -- 172.21.15.19:0/2107557583 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f971810ac10 con 0x7f9718109ae0 2024-05-07T02:16:31.661 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:30.603+0000 7f9715e00700 1 --2- 172.21.15.19:0/2107557583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9718109ae0 0x7f9718109ef0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f97080098a0 tx=0x7f9708009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e776f9cfca3d240a server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:16:31.662 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:30.604+0000 7f9714a00700 1 -- 172.21.15.19:0/2107557583 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9708004160 con 0x7f9718109ae0 2024-05-07T02:16:31.662 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:30.604+0000 7f9714a00700 1 -- 172.21.15.19:0/2107557583 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9708004300 con 0x7f9718109ae0 2024-05-07T02:16:31.662 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:30.604+0000 7f9714a00700 1 -- 172.21.15.19:0/2107557583 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9708011af0 con 0x7f9718109ae0 2024-05-07T02:16:31.662 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:30.605+0000 7f971ca00700 1 -- 172.21.15.19:0/2107557583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9718109ae0 msgr2=0x7f9718109ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:16:31.662 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:30.605+0000 7f971ca00700 1 --2- 172.21.15.19:0/2107557583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9718109ae0 0x7f9718109ef0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f97080098a0 tx=0x7f9708009bb0 comp rx=0 tx=0).stop 2024-05-07T02:16:31.662 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:30.605+0000 7f971ca00700 1 -- 172.21.15.19:0/2107557583 shutdown_connections 2024-05-07T02:16:31.662 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:30.605+0000 7f971ca00700 1 --2- 172.21.15.19:0/2107557583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9718109ae0 0x7f9718109ef0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:16:31.662 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:30.605+0000 7f971ca00700 1 -- 172.21.15.19:0/2107557583 >> 172.21.15.19:0/2107557583 conn(0x7f97181051a0 msgr2=0x7f97181075f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:16:31.662 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:30.605+0000 7f971ca00700 1 -- 172.21.15.19:0/2107557583 shutdown_connections 2024-05-07T02:16:31.662 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:30.605+0000 7f971ca00700 1 -- 172.21.15.19:0/2107557583 wait complete. 2024-05-07T02:16:31.662 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:30.606+0000 7f971ca00700 1 Processor -- start 2024-05-07T02:16:31.662 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:30.606+0000 7f971ca00700 1 -- start start 2024-05-07T02:16:31.662 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:30.606+0000 7f971ca00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9718109ae0 0x7f971807ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:16:31.662 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:30.607+0000 7f971ca00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f971810dfc0 con 0x7f9718109ae0 2024-05-07T02:16:31.662 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:30.607+0000 7f9715e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9718109ae0 0x7f971807ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:16:31.662 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:30.607+0000 7f9715e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9718109ae0 0x7f971807ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35394/0 (socket says 172.21.15.19:35394) 2024-05-07T02:16:31.663 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:30.607+0000 7f9715e00700 1 -- 172.21.15.19:0/2749819574 learned_addr learned my addr 172.21.15.19:0/2749819574 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:16:31.663 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:30.607+0000 7f9715e00700 1 -- 172.21.15.19:0/2749819574 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f971807ef80 con 0x7f9718109ae0 2024-05-07T02:16:31.663 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:30.608+0000 7f9715e00700 1 --2- 172.21.15.19:0/2749819574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9718109ae0 0x7f971807ea40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9708009fb0 tx=0x7f9708011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:16:31.663 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:30.608+0000 7f970ea00700 1 -- 172.21.15.19:0/2749819574 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97080036a0 con 0x7f9718109ae0 2024-05-07T02:16:31.663 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:30.608+0000 7f970ea00700 1 -- 172.21.15.19:0/2749819574 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9708003cd0 con 0x7f9718109ae0 2024-05-07T02:16:31.663 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:30.608+0000 7f970ea00700 1 -- 172.21.15.19:0/2749819574 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f970801a430 con 0x7f9718109ae0 2024-05-07T02:16:31.663 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:30.608+0000 7f971ca00700 1 -- 172.21.15.19:0/2749819574 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f971807d1d0 con 0x7f9718109ae0 2024-05-07T02:16:31.663 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:30.608+0000 7f971ca00700 1 -- 172.21.15.19:0/2749819574 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f971807d6b0 con 0x7f9718109ae0 2024-05-07T02:16:31.663 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:30.609+0000 7f970ea00700 1 -- 172.21.15.19:0/2749819574 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f970801a960 con 0x7f9718109ae0 2024-05-07T02:16:31.663 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:30.609+0000 7f970ea00700 1 -- 172.21.15.19:0/2749819574 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f97080193c0 con 0x7f9718109ae0 2024-05-07T02:16:31.663 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:30.609+0000 7f971ca00700 1 -- 172.21.15.19:0/2749819574 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9718064e20 con 0x7f9718109ae0 2024-05-07T02:16:31.663 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:30.612+0000 7f970ea00700 1 -- 172.21.15.19:0/2749819574 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f9708043ec0 con 0x7f9718109ae0 2024-05-07T02:16:31.663 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:30.695+0000 7f971ca00700 1 -- 172.21.15.19:0/2749819574 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f971807dd50 con 0x7f9718109ae0 2024-05-07T02:16:31.663 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:30.696+0000 7f970ea00700 1 -- 172.21.15.19:0/2749819574 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f9708032020 con 0x7f9718109ae0 2024-05-07T02:16:31.663 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:30.697+0000 7f971ca00700 1 -- 172.21.15.19:0/2749819574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9718109ae0 msgr2=0x7f971807ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:16:31.664 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:30.697+0000 7f971ca00700 1 --2- 172.21.15.19:0/2749819574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9718109ae0 0x7f971807ea40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9708009fb0 tx=0x7f9708011890 comp rx=0 tx=0).stop 2024-05-07T02:16:31.664 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:30.697+0000 7f971ca00700 1 -- 172.21.15.19:0/2749819574 shutdown_connections 2024-05-07T02:16:31.664 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:30.697+0000 7f971ca00700 1 --2- 172.21.15.19:0/2749819574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9718109ae0 0x7f971807ea40 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:16:31.664 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:30.697+0000 7f971ca00700 1 -- 172.21.15.19:0/2749819574 >> 172.21.15.19:0/2749819574 conn(0x7f97181051a0 msgr2=0x7f97181069d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:16:31.664 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:30.697+0000 7f971ca00700 1 -- 172.21.15.19:0/2749819574 shutdown_connections 2024-05-07T02:16:31.664 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:30.697+0000 7f971ca00700 1 -- 172.21.15.19:0/2749819574 wait complete. 2024-05-07T02:16:31.664 INFO:teuthology.orchestra.run.smithi019.stdout:mgr not available, waiting (1/15)... 2024-05-07T02:16:32.894 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:32 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:32.559+0000 7fc5995a6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-07T02:16:32.895 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:32 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:32.796+0000 7fc5995a6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-07T02:16:33.394 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:33 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:33.016+0000 7fc5995a6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-07T02:16:33.807 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:33 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:33.412+0000 7fc5995a6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-07T02:16:33.807 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:33 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:33.538+0000 7fc5995a6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-07T02:16:34.065 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:33 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:33.807+0000 7fc5995a6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-07T02:16:34.385 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:34 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:34.075+0000 7fc5995a6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-07T02:16:34.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:34 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/3326816469' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-07T02:16:35.036 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:34 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:34.903+0000 7fc5995a6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-07T02:16:35.047 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-07T02:16:35.047 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-05-07T02:16:35.047 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsid": "9ff84c16-0c17-11ef-bc97-c7b262605968", 2024-05-07T02:16:35.047 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "health": { 2024-05-07T02:16:35.047 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-07T02:16:35.047 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-07T02:16:35.047 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-07T02:16:35.047 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-07T02:16:35.047 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-07T02:16:35.047 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-07T02:16:35.047 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 0 2024-05-07T02:16:35.047 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-05-07T02:16:35.048 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-07T02:16:35.048 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "a" 2024-05-07T02:16:35.048 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-05-07T02:16:35.048 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-05-07T02:16:35.048 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-07T02:16:35.048 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-07T02:16:35.048 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-07T02:16:35.048 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-07T02:16:35.048 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-07T02:16:35.048 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-07T02:16:35.048 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-07T02:16:35.048 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-07T02:16:35.048 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-07T02:16:35.048 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-07T02:16:35.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-07T02:16:35.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-07T02:16:35.055 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-07T02:16:35.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-07T02:16:35.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-07T02:16:35.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-07T02:16:35.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-07T02:16:35.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-07T02:16:35.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-07T02:16:35.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-07T02:16:35.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-07T02:16:35.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-07T02:16:35.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-07T02:16:35.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-07T02:16:35.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-07T02:16:35.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-07T02:16:35.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-07T02:16:35.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-07T02:16:35.056 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-07T02:16:35.057 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-07T02:16:35.057 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-07T02:16:35.057 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-07T02:16:35.057 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-07T02:16:35.057 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-07T02:16:35.057 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-07T02:16:35.057 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "restful" 2024-05-07T02:16:35.057 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-05-07T02:16:35.057 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-07T02:16:35.057 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-07T02:16:35.057 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-07T02:16:35.057 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-07T02:16:35.057 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modified": "2024-05-07T02:16:19.508731+0000", 2024-05-07T02:16:35.057 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-07T02:16:35.057 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-07T02:16:35.058 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-07T02:16:35.058 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-05-07T02:16:35.058 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:34.293+0000 7f2c41c00700 1 Processor -- start 2024-05-07T02:16:35.058 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:34.294+0000 7f2c41c00700 1 -- start start 2024-05-07T02:16:35.058 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:34.294+0000 7f2c41c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c3c07ecf0 0x7f2c3c07d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:16:35.058 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:34.294+0000 7f2c41c00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c07d750 con 0x7f2c3c07ecf0 2024-05-07T02:16:35.058 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:34.295+0000 7f2c3be00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c3c07ecf0 0x7f2c3c07d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:16:35.058 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:34.295+0000 7f2c3be00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c3c07ecf0 0x7f2c3c07d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35402/0 (socket says 172.21.15.19:35402) 2024-05-07T02:16:35.058 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:34.295+0000 7f2c3be00700 1 -- 172.21.15.19:0/1947798223 learned_addr learned my addr 172.21.15.19:0/1947798223 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:16:35.058 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:34.295+0000 7f2c3be00700 1 -- 172.21.15.19:0/1947798223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c3c07d8d0 con 0x7f2c3c07ecf0 2024-05-07T02:16:35.058 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:34.296+0000 7f2c3be00700 1 --2- 172.21.15.19:0/1947798223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c3c07ecf0 0x7f2c3c07d210 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c0098a0 tx=0x7f2c2c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9c4d03489726c21 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:16:35.058 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:34.296+0000 7f2c3aa00700 1 -- 172.21.15.19:0/1947798223 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c2c004160 con 0x7f2c3c07ecf0 2024-05-07T02:16:35.058 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:34.296+0000 7f2c3aa00700 1 -- 172.21.15.19:0/1947798223 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2c2c004300 con 0x7f2c3c07ecf0 2024-05-07T02:16:35.058 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:34.296+0000 7f2c3aa00700 1 -- 172.21.15.19:0/1947798223 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c2c011af0 con 0x7f2c3c07ecf0 2024-05-07T02:16:35.058 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:34.297+0000 7f2c41c00700 1 -- 172.21.15.19:0/1947798223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c3c07ecf0 msgr2=0x7f2c3c07d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:16:35.059 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:34.297+0000 7f2c41c00700 1 --2- 172.21.15.19:0/1947798223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c3c07ecf0 0x7f2c3c07d210 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c0098a0 tx=0x7f2c2c009bb0 comp rx=0 tx=0).stop 2024-05-07T02:16:35.059 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:34.297+0000 7f2c41c00700 1 -- 172.21.15.19:0/1947798223 shutdown_connections 2024-05-07T02:16:35.059 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:34.297+0000 7f2c41c00700 1 --2- 172.21.15.19:0/1947798223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c3c07ecf0 0x7f2c3c07d210 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:16:35.059 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:34.297+0000 7f2c41c00700 1 -- 172.21.15.19:0/1947798223 >> 172.21.15.19:0/1947798223 conn(0x7f2c3c104f70 msgr2=0x7f2c3c107400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:16:35.059 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:34.297+0000 7f2c41c00700 1 -- 172.21.15.19:0/1947798223 shutdown_connections 2024-05-07T02:16:35.059 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:34.297+0000 7f2c41c00700 1 -- 172.21.15.19:0/1947798223 wait complete. 2024-05-07T02:16:35.059 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:34.299+0000 7f2c41c00700 1 Processor -- start 2024-05-07T02:16:35.059 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:34.299+0000 7f2c41c00700 1 -- start start 2024-05-07T02:16:35.059 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:34.299+0000 7f2c41c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c3c07ecf0 0x7f2c3c19fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:16:35.059 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:34.299+0000 7f2c41c00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c10fe30 con 0x7f2c3c07ecf0 2024-05-07T02:16:35.059 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:34.300+0000 7f2c3be00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c3c07ecf0 0x7f2c3c19fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:16:35.059 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:34.300+0000 7f2c3be00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c3c07ecf0 0x7f2c3c19fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35410/0 (socket says 172.21.15.19:35410) 2024-05-07T02:16:35.059 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:34.300+0000 7f2c3be00700 1 -- 172.21.15.19:0/3326816469 learned_addr learned my addr 172.21.15.19:0/3326816469 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:16:35.059 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:34.300+0000 7f2c3be00700 1 -- 172.21.15.19:0/3326816469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c3c1a0320 con 0x7f2c3c07ecf0 2024-05-07T02:16:35.059 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:34.301+0000 7f2c3be00700 1 --2- 172.21.15.19:0/3326816469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c3c07ecf0 0x7f2c3c19fde0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c0117b0 tx=0x7f2c2c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:16:35.059 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:34.301+0000 7f2c38c00700 1 -- 172.21.15.19:0/3326816469 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c2c0039c0 con 0x7f2c3c07ecf0 2024-05-07T02:16:35.060 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:34.301+0000 7f2c38c00700 1 -- 172.21.15.19:0/3326816469 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2c2c003b60 con 0x7f2c3c07ecf0 2024-05-07T02:16:35.060 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:34.301+0000 7f2c41c00700 1 -- 172.21.15.19:0/3326816469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c3c1a05b0 con 0x7f2c3c07ecf0 2024-05-07T02:16:35.060 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:34.301+0000 7f2c38c00700 1 -- 172.21.15.19:0/3326816469 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c2c0196a0 con 0x7f2c3c07ecf0 2024-05-07T02:16:35.060 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:34.301+0000 7f2c41c00700 1 -- 172.21.15.19:0/3326816469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c3c1a31c0 con 0x7f2c3c07ecf0 2024-05-07T02:16:35.060 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:34.301+0000 7f2c38c00700 1 -- 172.21.15.19:0/3326816469 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2c2c019ca0 con 0x7f2c3c07ecf0 2024-05-07T02:16:35.060 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:34.302+0000 7f2c38c00700 1 -- 172.21.15.19:0/3326816469 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2c2c024730 con 0x7f2c3c07ecf0 2024-05-07T02:16:35.060 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:34.303+0000 7f2c41c00700 1 -- 172.21.15.19:0/3326816469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c1c005360 con 0x7f2c3c07ecf0 2024-05-07T02:16:35.060 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:34.305+0000 7f2c38c00700 1 -- 172.21.15.19:0/3326816469 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f2c2c043ec0 con 0x7f2c3c07ecf0 2024-05-07T02:16:35.060 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:34.385+0000 7f2c41c00700 1 -- 172.21.15.19:0/3326816469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f2c1c005920 con 0x7f2c3c07ecf0 2024-05-07T02:16:35.060 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:34.386+0000 7f2c38c00700 1 -- 172.21.15.19:0/3326816469 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f2c2c032020 con 0x7f2c3c07ecf0 2024-05-07T02:16:35.060 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:34.388+0000 7f2c41c00700 1 -- 172.21.15.19:0/3326816469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c3c07ecf0 msgr2=0x7f2c3c19fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:16:35.060 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:34.388+0000 7f2c41c00700 1 --2- 172.21.15.19:0/3326816469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c3c07ecf0 0x7f2c3c19fde0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c0117b0 tx=0x7f2c2c011b70 comp rx=0 tx=0).stop 2024-05-07T02:16:35.060 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:34.388+0000 7f2c41c00700 1 -- 172.21.15.19:0/3326816469 shutdown_connections 2024-05-07T02:16:35.060 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:34.388+0000 7f2c41c00700 1 --2- 172.21.15.19:0/3326816469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c3c07ecf0 0x7f2c3c19fde0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:16:35.061 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:34.388+0000 7f2c41c00700 1 -- 172.21.15.19:0/3326816469 >> 172.21.15.19:0/3326816469 conn(0x7f2c3c104f70 msgr2=0x7f2c3c105880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:16:35.061 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:34.388+0000 7f2c41c00700 1 -- 172.21.15.19:0/3326816469 shutdown_connections 2024-05-07T02:16:35.061 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:34.388+0000 7f2c41c00700 1 -- 172.21.15.19:0/3326816469 wait complete. 2024-05-07T02:16:35.061 INFO:teuthology.orchestra.run.smithi019.stdout:mgr not available, waiting (2/15)... 2024-05-07T02:16:35.394 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:35 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:35.206+0000 7fc5995a6200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-07T02:16:36.144 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:35 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:35.780+0000 7fc5995a6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-07T02:16:36.145 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:35 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:35.912+0000 7fc5995a6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-07T02:16:36.644 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:36 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:36.387+0000 7fc5995a6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-07T02:16:37.144 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:36 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:36.855+0000 7fc5995a6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-07T02:16:37.144 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:36 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:36.987+0000 7fc5995a6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-07T02:16:37.515 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:37 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:37.335+0000 7fc5995a6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-07T02:16:37.838 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:37 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:37.531+0000 7fc5995a6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-07T02:16:38.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:37 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/3332139802' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-07T02:16:38.145 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:37 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:37.923+0000 7fc5995a6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-07T02:16:38.145 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:38 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:38.080+0000 7fc5995a6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-07T02:16:38.482 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:38 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:38.213+0000 7fc5995a6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-07T02:16:38.495 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-07T02:16:38.495 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-05-07T02:16:38.495 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsid": "9ff84c16-0c17-11ef-bc97-c7b262605968", 2024-05-07T02:16:38.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "health": { 2024-05-07T02:16:38.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-07T02:16:38.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-07T02:16:38.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-07T02:16:38.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-07T02:16:38.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-07T02:16:38.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-07T02:16:38.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 0 2024-05-07T02:16:38.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-05-07T02:16:38.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-07T02:16:38.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "a" 2024-05-07T02:16:38.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-05-07T02:16:38.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-05-07T02:16:38.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-07T02:16:38.497 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-07T02:16:38.497 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-07T02:16:38.497 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-07T02:16:38.497 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-07T02:16:38.497 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-07T02:16:38.497 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-07T02:16:38.497 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-07T02:16:38.497 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-07T02:16:38.505 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-07T02:16:38.505 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-07T02:16:38.505 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-07T02:16:38.505 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-07T02:16:38.505 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-07T02:16:38.505 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-07T02:16:38.505 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-07T02:16:38.505 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-07T02:16:38.505 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-07T02:16:38.505 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-07T02:16:38.506 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-07T02:16:38.506 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-07T02:16:38.506 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-07T02:16:38.506 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-07T02:16:38.506 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-07T02:16:38.506 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-07T02:16:38.506 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-07T02:16:38.506 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-07T02:16:38.506 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-07T02:16:38.506 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-07T02:16:38.506 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-07T02:16:38.506 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-07T02:16:38.506 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-07T02:16:38.506 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-07T02:16:38.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-07T02:16:38.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-07T02:16:38.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "restful" 2024-05-07T02:16:38.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-05-07T02:16:38.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-07T02:16:38.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-07T02:16:38.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-07T02:16:38.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-07T02:16:38.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modified": "2024-05-07T02:16:19.508731+0000", 2024-05-07T02:16:38.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-07T02:16:38.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-07T02:16:38.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-07T02:16:38.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-05-07T02:16:38.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:37.745+0000 7f0095c00700 1 Processor -- start 2024-05-07T02:16:38.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:37.746+0000 7f0095c00700 1 -- start start 2024-05-07T02:16:38.508 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:37.746+0000 7f0095c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f009007ed60 0x7f009007d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:16:38.508 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:37.746+0000 7f0095c00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f009007d770 con 0x7f009007ed60 2024-05-07T02:16:38.508 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:37.746+0000 7f008ea00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f009007ed60 0x7f009007d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:16:38.508 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:37.747+0000 7f008ea00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f009007ed60 0x7f009007d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35412/0 (socket says 172.21.15.19:35412) 2024-05-07T02:16:38.508 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:37.747+0000 7f008ea00700 1 -- 172.21.15.19:0/305386122 learned_addr learned my addr 172.21.15.19:0/305386122 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:16:38.508 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:37.747+0000 7f008ea00700 1 -- 172.21.15.19:0/305386122 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f009007d8f0 con 0x7f009007ed60 2024-05-07T02:16:38.508 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:37.747+0000 7f008ea00700 1 --2- 172.21.15.19:0/305386122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f009007ed60 0x7f009007d230 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f00800098a0 tx=0x7f0080009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5df670acc4e67ffd server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:16:38.508 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:37.748+0000 7f008d600700 1 -- 172.21.15.19:0/305386122 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0080004160 con 0x7f009007ed60 2024-05-07T02:16:38.508 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:37.748+0000 7f008d600700 1 -- 172.21.15.19:0/305386122 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0080004300 con 0x7f009007ed60 2024-05-07T02:16:38.508 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:37.748+0000 7f008d600700 1 -- 172.21.15.19:0/305386122 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0080011af0 con 0x7f009007ed60 2024-05-07T02:16:38.508 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:37.749+0000 7f0095c00700 1 -- 172.21.15.19:0/305386122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f009007ed60 msgr2=0x7f009007d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:16:38.508 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:37.749+0000 7f0095c00700 1 --2- 172.21.15.19:0/305386122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f009007ed60 0x7f009007d230 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f00800098a0 tx=0x7f0080009bb0 comp rx=0 tx=0).stop 2024-05-07T02:16:38.508 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:37.749+0000 7f0095c00700 1 -- 172.21.15.19:0/305386122 shutdown_connections 2024-05-07T02:16:38.508 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:37.749+0000 7f0095c00700 1 --2- 172.21.15.19:0/305386122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f009007ed60 0x7f009007d230 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:16:38.509 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:37.749+0000 7f0095c00700 1 -- 172.21.15.19:0/305386122 >> 172.21.15.19:0/305386122 conn(0x7f0090105150 msgr2=0x7f00901075c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:16:38.509 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:37.749+0000 7f0095c00700 1 -- 172.21.15.19:0/305386122 shutdown_connections 2024-05-07T02:16:38.509 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:37.749+0000 7f0095c00700 1 -- 172.21.15.19:0/305386122 wait complete. 2024-05-07T02:16:38.509 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:37.750+0000 7f0095c00700 1 Processor -- start 2024-05-07T02:16:38.509 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:37.750+0000 7f0095c00700 1 -- start start 2024-05-07T02:16:38.509 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:37.751+0000 7f0095c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f009007ed60 0x7f009019ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:16:38.509 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:37.751+0000 7f0095c00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0090110030 con 0x7f009007ed60 2024-05-07T02:16:38.509 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:37.751+0000 7f008ea00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f009007ed60 0x7f009019ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:16:38.509 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:37.751+0000 7f008ea00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f009007ed60 0x7f009019ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35420/0 (socket says 172.21.15.19:35420) 2024-05-07T02:16:38.509 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:37.751+0000 7f008ea00700 1 -- 172.21.15.19:0/3332139802 learned_addr learned my addr 172.21.15.19:0/3332139802 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:16:38.509 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:37.752+0000 7f008ea00700 1 -- 172.21.15.19:0/3332139802 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00901a04f0 con 0x7f009007ed60 2024-05-07T02:16:38.509 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:37.752+0000 7f008ea00700 1 --2- 172.21.15.19:0/3332139802 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f009007ed60 0x7f009019ffb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f00800117b0 tx=0x7f0080011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:16:38.509 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:37.752+0000 7f0087400700 1 -- 172.21.15.19:0/3332139802 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0080003af0 con 0x7f009007ed60 2024-05-07T02:16:38.509 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:37.752+0000 7f0087400700 1 -- 172.21.15.19:0/3332139802 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0080020070 con 0x7f009007ed60 2024-05-07T02:16:38.510 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:37.752+0000 7f0095c00700 1 -- 172.21.15.19:0/3332139802 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00901a0780 con 0x7f009007ed60 2024-05-07T02:16:38.510 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:37.752+0000 7f0087400700 1 -- 172.21.15.19:0/3332139802 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0080019630 con 0x7f009007ed60 2024-05-07T02:16:38.510 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:37.752+0000 7f0095c00700 1 -- 172.21.15.19:0/3332139802 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00901a3390 con 0x7f009007ed60 2024-05-07T02:16:38.510 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:37.753+0000 7f0087400700 1 -- 172.21.15.19:0/3332139802 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0080019c80 con 0x7f009007ed60 2024-05-07T02:16:38.510 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:37.753+0000 7f0087400700 1 -- 172.21.15.19:0/3332139802 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f008000b720 con 0x7f009007ed60 2024-05-07T02:16:38.510 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:37.754+0000 7f0095c00700 1 -- 172.21.15.19:0/3332139802 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0090064e20 con 0x7f009007ed60 2024-05-07T02:16:38.510 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:37.756+0000 7f0087400700 1 -- 172.21.15.19:0/3332139802 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f00800042a0 con 0x7f009007ed60 2024-05-07T02:16:38.510 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:37.838+0000 7f0095c00700 1 -- 172.21.15.19:0/3332139802 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f00901a3560 con 0x7f009007ed60 2024-05-07T02:16:38.510 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:37.839+0000 7f0087400700 1 -- 172.21.15.19:0/3332139802 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7f008001a9c0 con 0x7f009007ed60 2024-05-07T02:16:38.510 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:37.841+0000 7f0095c00700 1 -- 172.21.15.19:0/3332139802 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f009007ed60 msgr2=0x7f009019ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:16:38.510 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:37.841+0000 7f0095c00700 1 --2- 172.21.15.19:0/3332139802 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f009007ed60 0x7f009019ffb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f00800117b0 tx=0x7f0080011b70 comp rx=0 tx=0).stop 2024-05-07T02:16:38.510 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:37.841+0000 7f0095c00700 1 -- 172.21.15.19:0/3332139802 shutdown_connections 2024-05-07T02:16:38.510 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:37.841+0000 7f0095c00700 1 --2- 172.21.15.19:0/3332139802 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f009007ed60 0x7f009019ffb0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:16:38.511 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:37.841+0000 7f0095c00700 1 -- 172.21.15.19:0/3332139802 >> 172.21.15.19:0/3332139802 conn(0x7f0090105150 msgr2=0x7f00901059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:16:38.511 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:37.841+0000 7f0095c00700 1 -- 172.21.15.19:0/3332139802 shutdown_connections 2024-05-07T02:16:38.511 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:37.841+0000 7f0095c00700 1 -- 172.21.15.19:0/3332139802 wait complete. 2024-05-07T02:16:38.511 INFO:teuthology.orchestra.run.smithi019.stdout:mgr not available, waiting (3/15)... 2024-05-07T02:16:39.331 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:39 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:39.028+0000 7fc5995a6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-07T02:16:39.644 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:39 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:39.331+0000 7fc5995a6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-07T02:16:39.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:39 smithi019 ceph-mon[29781]: Activating manager daemon x 2024-05-07T02:16:39.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:39 smithi019 ceph-mon[29781]: mgrmap e2: x(active, starting, since 0.0145922s) 2024-05-07T02:16:39.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:39 smithi019 ceph-mon[29781]: from='mgr.14100 172.21.15.19:0/390980089' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-07T02:16:39.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:39 smithi019 ceph-mon[29781]: from='mgr.14100 172.21.15.19:0/390980089' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-07T02:16:39.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:39 smithi019 ceph-mon[29781]: from='mgr.14100 172.21.15.19:0/390980089' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-07T02:16:39.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:39 smithi019 ceph-mon[29781]: from='mgr.14100 172.21.15.19:0/390980089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-07T02:16:39.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:39 smithi019 ceph-mon[29781]: from='mgr.14100 172.21.15.19:0/390980089' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-07T02:16:39.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:39 smithi019 ceph-mon[29781]: Manager daemon x is now available 2024-05-07T02:16:39.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:39 smithi019 ceph-mon[29781]: from='mgr.14100 172.21.15.19:0/390980089' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-07T02:16:39.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:39 smithi019 ceph-mon[29781]: from='mgr.14100 172.21.15.19:0/390980089' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-07T02:16:40.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:40 smithi019 ceph-mon[29781]: from='mgr.14100 172.21.15.19:0/390980089' entity='mgr.x' 2024-05-07T02:16:40.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:40 smithi019 ceph-mon[29781]: from='mgr.14100 172.21.15.19:0/390980089' entity='mgr.x' 2024-05-07T02:16:40.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:40 smithi019 ceph-mon[29781]: from='mgr.14100 172.21.15.19:0/390980089' entity='mgr.x' 2024-05-07T02:16:40.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:40 smithi019 ceph-mon[29781]: mgrmap e3: x(active, since 1.02539s) 2024-05-07T02:16:42.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-07T02:16:42.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-05-07T02:16:42.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsid": "9ff84c16-0c17-11ef-bc97-c7b262605968", 2024-05-07T02:16:42.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "health": { 2024-05-07T02:16:42.448 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-07T02:16:42.448 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-07T02:16:42.448 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-07T02:16:42.448 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-07T02:16:42.448 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-07T02:16:42.448 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-07T02:16:42.448 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 0 2024-05-07T02:16:42.448 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-05-07T02:16:42.448 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-07T02:16:42.448 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "a" 2024-05-07T02:16:42.448 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-05-07T02:16:42.448 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-05-07T02:16:42.448 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-07T02:16:42.450 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-07T02:16:42.450 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-07T02:16:42.450 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-07T02:16:42.450 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-07T02:16:42.450 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-07T02:16:42.450 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-07T02:16:42.450 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-07T02:16:42.450 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-07T02:16:42.450 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-07T02:16:42.450 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-07T02:16:42.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-07T02:16:42.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-07T02:16:42.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-07T02:16:42.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-07T02:16:42.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-07T02:16:42.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-07T02:16:42.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-07T02:16:42.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-07T02:16:42.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-07T02:16:42.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-07T02:16:42.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-07T02:16:42.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-07T02:16:42.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-07T02:16:42.454 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-07T02:16:42.455 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-07T02:16:42.455 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-07T02:16:42.455 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-07T02:16:42.455 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-07T02:16:42.455 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-07T02:16:42.455 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-07T02:16:42.455 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-07T02:16:42.455 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-07T02:16:42.455 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-07T02:16:42.455 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-07T02:16:42.455 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "restful" 2024-05-07T02:16:42.455 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-05-07T02:16:42.455 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-07T02:16:42.455 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-07T02:16:42.455 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-07T02:16:42.456 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-07T02:16:42.456 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modified": "2024-05-07T02:16:19.508731+0000", 2024-05-07T02:16:42.456 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-07T02:16:42.461 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-05-07T02:16:42.461 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-07T02:16:42.461 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-05-07T02:16:42.462 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.180+0000 7fbd45200700 1 Processor -- start 2024-05-07T02:16:42.462 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.180+0000 7fbd45200700 1 -- start start 2024-05-07T02:16:42.462 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.181+0000 7fbd45200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd40109b30 0x7fbd40109f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:16:42.462 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.181+0000 7fbd45200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4010a530 con 0x7fbd40109b30 2024-05-07T02:16:42.462 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.181+0000 7fbd3de00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd40109b30 0x7fbd40109f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:16:42.462 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.181+0000 7fbd3de00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd40109b30 0x7fbd40109f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41874/0 (socket says 172.21.15.19:41874) 2024-05-07T02:16:42.462 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.181+0000 7fbd3de00700 1 -- 172.21.15.19:0/4245404315 learned_addr learned my addr 172.21.15.19:0/4245404315 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:16:42.462 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.181+0000 7fbd3de00700 1 -- 172.21.15.19:0/4245404315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd4010ac60 con 0x7fbd40109b30 2024-05-07T02:16:42.462 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.182+0000 7fbd3de00700 1 --2- 172.21.15.19:0/4245404315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd40109b30 0x7fbd40109f40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbd300098a0 tx=0x7fbd30009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6852f7cf1ccc4217 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:16:42.462 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.182+0000 7fbd3ca00700 1 -- 172.21.15.19:0/4245404315 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd30004160 con 0x7fbd40109b30 2024-05-07T02:16:42.462 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.182+0000 7fbd3ca00700 1 -- 172.21.15.19:0/4245404315 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbd30004300 con 0x7fbd40109b30 2024-05-07T02:16:42.462 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.182+0000 7fbd3ca00700 1 -- 172.21.15.19:0/4245404315 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd30011af0 con 0x7fbd40109b30 2024-05-07T02:16:42.462 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.183+0000 7fbd45200700 1 -- 172.21.15.19:0/4245404315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd40109b30 msgr2=0x7fbd40109f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:16:42.463 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.183+0000 7fbd45200700 1 --2- 172.21.15.19:0/4245404315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd40109b30 0x7fbd40109f40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbd300098a0 tx=0x7fbd30009bb0 comp rx=0 tx=0).stop 2024-05-07T02:16:42.463 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.183+0000 7fbd45200700 1 -- 172.21.15.19:0/4245404315 shutdown_connections 2024-05-07T02:16:42.463 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.183+0000 7fbd45200700 1 --2- 172.21.15.19:0/4245404315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd40109b30 0x7fbd40109f40 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:16:42.463 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.183+0000 7fbd45200700 1 -- 172.21.15.19:0/4245404315 >> 172.21.15.19:0/4245404315 conn(0x7fbd401051b0 msgr2=0x7fbd40107620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:16:42.463 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.183+0000 7fbd45200700 1 -- 172.21.15.19:0/4245404315 shutdown_connections 2024-05-07T02:16:42.463 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.183+0000 7fbd45200700 1 -- 172.21.15.19:0/4245404315 wait complete. 2024-05-07T02:16:42.463 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.184+0000 7fbd45200700 1 Processor -- start 2024-05-07T02:16:42.463 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.185+0000 7fbd45200700 1 -- start start 2024-05-07T02:16:42.463 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.185+0000 7fbd45200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd40109b30 0x7fbd4007f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:16:42.463 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.185+0000 7fbd45200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4010de90 con 0x7fbd40109b30 2024-05-07T02:16:42.463 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.185+0000 7fbd3de00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd40109b30 0x7fbd4007f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:16:42.464 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.185+0000 7fbd3de00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd40109b30 0x7fbd4007f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41890/0 (socket says 172.21.15.19:41890) 2024-05-07T02:16:42.464 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.185+0000 7fbd3de00700 1 -- 172.21.15.19:0/513069838 learned_addr learned my addr 172.21.15.19:0/513069838 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:16:42.464 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.186+0000 7fbd3de00700 1 -- 172.21.15.19:0/513069838 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd4007f9b0 con 0x7fbd40109b30 2024-05-07T02:16:42.464 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.186+0000 7fbd3de00700 1 --2- 172.21.15.19:0/513069838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd40109b30 0x7fbd4007f470 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbd30011670 tx=0x7fbd30004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:16:42.464 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.186+0000 7fbd36a00700 1 -- 172.21.15.19:0/513069838 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd300036a0 con 0x7fbd40109b30 2024-05-07T02:16:42.464 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.186+0000 7fbd45200700 1 -- 172.21.15.19:0/513069838 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd4007dc00 con 0x7fbd40109b30 2024-05-07T02:16:42.464 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.186+0000 7fbd36a00700 1 -- 172.21.15.19:0/513069838 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbd30003cd0 con 0x7fbd40109b30 2024-05-07T02:16:42.464 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.186+0000 7fbd36a00700 1 -- 172.21.15.19:0/513069838 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd300193c0 con 0x7fbd40109b30 2024-05-07T02:16:42.464 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.186+0000 7fbd45200700 1 -- 172.21.15.19:0/513069838 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd4007e0e0 con 0x7fbd40109b30 2024-05-07T02:16:42.464 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.187+0000 7fbd45200700 1 -- 172.21.15.19:0/513069838 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd40064e20 con 0x7fbd40109b30 2024-05-07T02:16:42.464 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.188+0000 7fbd36a00700 1 -- 172.21.15.19:0/513069838 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 3) ==== 50105+0+0 (secure 0 0 0) 0x7fbd30019680 con 0x7fbd40109b30 2024-05-07T02:16:42.464 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.188+0000 7fbd36a00700 1 --2- 172.21.15.19:0/513069838 >> [v2:172.21.15.19:6800/3614957309,v1:172.21.15.19:6801/3614957309] conn(0x7fbd2403e380 0x7fbd24040890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:16:42.464 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.188+0000 7fbd36a00700 1 -- 172.21.15.19:0/513069838 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbd300535a0 con 0x7fbd40109b30 2024-05-07T02:16:42.464 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.189+0000 7fbd3d400700 1 --2- 172.21.15.19:0/513069838 >> [v2:172.21.15.19:6800/3614957309,v1:172.21.15.19:6801/3614957309] conn(0x7fbd2403e380 0x7fbd24040890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:16:42.464 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.189+0000 7fbd3d400700 1 --2- 172.21.15.19:0/513069838 >> [v2:172.21.15.19:6800/3614957309,v1:172.21.15.19:6801/3614957309] conn(0x7fbd2403e380 0x7fbd24040890 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c006fd0 tx=0x7fbd2c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:16:42.464 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.195+0000 7fbd36a00700 1 -- 172.21.15.19:0/513069838 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbd3001a430 con 0x7fbd40109b30 2024-05-07T02:16:42.465 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.405+0000 7fbd36a00700 1 -- 172.21.15.19:0/513069838 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7fbd3001a6c0 con 0x7fbd40109b30 2024-05-07T02:16:42.465 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.528+0000 7fbd45200700 1 -- 172.21.15.19:0/513069838 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fbd4007e830 con 0x7fbd40109b30 2024-05-07T02:16:42.465 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.529+0000 7fbd36a00700 1 -- 172.21.15.19:0/513069838 <== mon.0 v2:172.21.15.19:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fbd30003840 con 0x7fbd40109b30 2024-05-07T02:16:42.465 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.534+0000 7fbd45200700 1 -- 172.21.15.19:0/513069838 >> [v2:172.21.15.19:6800/3614957309,v1:172.21.15.19:6801/3614957309] conn(0x7fbd2403e380 msgr2=0x7fbd24040890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:16:42.465 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.534+0000 7fbd45200700 1 --2- 172.21.15.19:0/513069838 >> [v2:172.21.15.19:6800/3614957309,v1:172.21.15.19:6801/3614957309] conn(0x7fbd2403e380 0x7fbd24040890 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c006fd0 tx=0x7fbd2c006e40 comp rx=0 tx=0).stop 2024-05-07T02:16:42.465 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.534+0000 7fbd45200700 1 -- 172.21.15.19:0/513069838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd40109b30 msgr2=0x7fbd4007f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:16:42.465 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.534+0000 7fbd45200700 1 --2- 172.21.15.19:0/513069838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd40109b30 0x7fbd4007f470 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbd30011670 tx=0x7fbd30004280 comp rx=0 tx=0).stop 2024-05-07T02:16:42.465 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.534+0000 7fbd45200700 1 -- 172.21.15.19:0/513069838 shutdown_connections 2024-05-07T02:16:42.465 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.534+0000 7fbd45200700 1 --2- 172.21.15.19:0/513069838 >> [v2:172.21.15.19:6800/3614957309,v1:172.21.15.19:6801/3614957309] conn(0x7fbd2403e380 0x7fbd24040890 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:16:42.465 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.534+0000 7fbd45200700 1 --2- 172.21.15.19:0/513069838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd40109b30 0x7fbd4007f470 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:16:42.465 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.534+0000 7fbd45200700 1 -- 172.21.15.19:0/513069838 >> 172.21.15.19:0/513069838 conn(0x7fbd401051b0 msgr2=0x7fbd40105a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:16:42.465 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.534+0000 7fbd45200700 1 -- 172.21.15.19:0/513069838 shutdown_connections 2024-05-07T02:16:42.466 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:41.534+0000 7fbd45200700 1 -- 172.21.15.19:0/513069838 wait complete. 2024-05-07T02:16:42.466 INFO:teuthology.orchestra.run.smithi019.stdout:mgr is available 2024-05-07T02:16:42.758 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:42 smithi019 ceph-mon[29781]: mgrmap e4: x(active, since 2s) 2024-05-07T02:16:42.758 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:42 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/513069838' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-07T02:16:43.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:43 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/1486864336' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-07T02:16:43.975 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-07T02:16:43.975 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [global] 2024-05-07T02:16:43.975 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout fsid = 9ff84c16-0c17-11ef-bc97-c7b262605968 2024-05-07T02:16:43.976 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-07T02:16:43.976 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-07T02:16:43.976 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-07T02:16:43.976 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-07T02:16:43.976 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-07T02:16:43.982 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-07T02:16:43.982 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-07T02:16:43.983 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-07T02:16:43.983 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-05-07T02:16:43.983 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [osd] 2024-05-07T02:16:43.983 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-07T02:16:43.983 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-07T02:16:43.983 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-07T02:16:43.983 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-07T02:16:43.983 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-07T02:16:43.983 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.139+0000 7fb520800700 1 Processor -- start 2024-05-07T02:16:43.983 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.139+0000 7fb520800700 1 -- start start 2024-05-07T02:16:43.983 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.140+0000 7fb520800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb518109b30 0x7fb518109f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:16:43.983 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.140+0000 7fb520800700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb51810a530 con 0x7fb518109b30 2024-05-07T02:16:43.983 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.140+0000 7fb51de00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb518109b30 0x7fb518109f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:16:43.984 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.140+0000 7fb51de00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb518109b30 0x7fb518109f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41892/0 (socket says 172.21.15.19:41892) 2024-05-07T02:16:43.984 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.140+0000 7fb51de00700 1 -- 172.21.15.19:0/3839783384 learned_addr learned my addr 172.21.15.19:0/3839783384 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:16:43.984 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.141+0000 7fb51de00700 1 -- 172.21.15.19:0/3839783384 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb51810ac60 con 0x7fb518109b30 2024-05-07T02:16:43.984 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.141+0000 7fb51de00700 1 --2- 172.21.15.19:0/3839783384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb518109b30 0x7fb518109f40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb50c0098a0 tx=0x7fb50c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cee9e3cb7f4bd07 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:16:43.984 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.141+0000 7fb51ca00700 1 -- 172.21.15.19:0/3839783384 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb50c004160 con 0x7fb518109b30 2024-05-07T02:16:43.984 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.141+0000 7fb51ca00700 1 -- 172.21.15.19:0/3839783384 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb50c004300 con 0x7fb518109b30 2024-05-07T02:16:43.984 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.141+0000 7fb51ca00700 1 -- 172.21.15.19:0/3839783384 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb50c011af0 con 0x7fb518109b30 2024-05-07T02:16:43.984 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.142+0000 7fb520800700 1 -- 172.21.15.19:0/3839783384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb518109b30 msgr2=0x7fb518109f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:16:43.984 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.142+0000 7fb520800700 1 --2- 172.21.15.19:0/3839783384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb518109b30 0x7fb518109f40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb50c0098a0 tx=0x7fb50c009bb0 comp rx=0 tx=0).stop 2024-05-07T02:16:43.984 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.142+0000 7fb520800700 1 -- 172.21.15.19:0/3839783384 shutdown_connections 2024-05-07T02:16:43.984 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.142+0000 7fb520800700 1 --2- 172.21.15.19:0/3839783384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb518109b30 0x7fb518109f40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:16:43.984 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.142+0000 7fb520800700 1 -- 172.21.15.19:0/3839783384 >> 172.21.15.19:0/3839783384 conn(0x7fb518105270 msgr2=0x7fb5181076c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:16:43.984 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.142+0000 7fb520800700 1 -- 172.21.15.19:0/3839783384 shutdown_connections 2024-05-07T02:16:43.984 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.143+0000 7fb520800700 1 -- 172.21.15.19:0/3839783384 wait complete. 2024-05-07T02:16:43.984 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.144+0000 7fb520800700 1 Processor -- start 2024-05-07T02:16:43.985 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.144+0000 7fb520800700 1 -- start start 2024-05-07T02:16:43.985 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.144+0000 7fb520800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb518109b30 0x7fb51819ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:16:43.985 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.144+0000 7fb520800700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb51810de90 con 0x7fb518109b30 2024-05-07T02:16:43.985 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.145+0000 7fb51de00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb518109b30 0x7fb51819ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:16:43.985 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.145+0000 7fb51de00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb518109b30 0x7fb51819ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41894/0 (socket says 172.21.15.19:41894) 2024-05-07T02:16:43.985 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.145+0000 7fb51de00700 1 -- 172.21.15.19:0/1486864336 learned_addr learned my addr 172.21.15.19:0/1486864336 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:16:43.985 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.145+0000 7fb51de00700 1 -- 172.21.15.19:0/1486864336 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb51819e310 con 0x7fb518109b30 2024-05-07T02:16:43.985 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.145+0000 7fb51de00700 1 --2- 172.21.15.19:0/1486864336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb518109b30 0x7fb51819ddd0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb50c011890 tx=0x7fb50c011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:16:43.985 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.146+0000 7fb512a00700 1 -- 172.21.15.19:0/1486864336 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb50c0039e0 con 0x7fb518109b30 2024-05-07T02:16:43.985 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.146+0000 7fb512a00700 1 -- 172.21.15.19:0/1486864336 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb50c003b80 con 0x7fb518109b30 2024-05-07T02:16:43.985 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.146+0000 7fb520800700 1 -- 172.21.15.19:0/1486864336 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb51819e5a0 con 0x7fb518109b30 2024-05-07T02:16:43.985 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.146+0000 7fb512a00700 1 -- 172.21.15.19:0/1486864336 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb50c0246e0 con 0x7fb518109b30 2024-05-07T02:16:43.985 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.146+0000 7fb520800700 1 -- 172.21.15.19:0/1486864336 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5181a1170 con 0x7fb518109b30 2024-05-07T02:16:43.985 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.147+0000 7fb512a00700 1 -- 172.21.15.19:0/1486864336 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7fb50c022020 con 0x7fb518109b30 2024-05-07T02:16:43.985 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.147+0000 7fb507e00700 1 -- 172.21.15.19:0/1486864336 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4f4005330 con 0x7fb518109b30 2024-05-07T02:16:43.986 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.148+0000 7fb512a00700 1 --2- 172.21.15.19:0/1486864336 >> [v2:172.21.15.19:6800/3614957309,v1:172.21.15.19:6801/3614957309] conn(0x7fb50003e4e0 0x7fb5000409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:16:43.986 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.148+0000 7fb512a00700 1 -- 172.21.15.19:0/1486864336 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb50c053640 con 0x7fb518109b30 2024-05-07T02:16:43.986 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.148+0000 7fb51d400700 1 --2- 172.21.15.19:0/1486864336 >> [v2:172.21.15.19:6800/3614957309,v1:172.21.15.19:6801/3614957309] conn(0x7fb50003e4e0 0x7fb5000409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:16:43.986 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.149+0000 7fb51d400700 1 --2- 172.21.15.19:0/1486864336 >> [v2:172.21.15.19:6800/3614957309,v1:172.21.15.19:6801/3614957309] conn(0x7fb50003e4e0 0x7fb5000409f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb508006fd0 tx=0x7fb508006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:16:43.986 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.154+0000 7fb512a00700 1 -- 172.21.15.19:0/1486864336 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb50c012040 con 0x7fb518109b30 2024-05-07T02:16:43.986 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.390+0000 7fb507e00700 1 -- 172.21.15.19:0/1486864336 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fb4f4002400 con 0x7fb518109b30 2024-05-07T02:16:43.986 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.393+0000 7fb512a00700 1 -- 172.21.15.19:0/1486864336 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7fb50c0533a0 con 0x7fb518109b30 2024-05-07T02:16:43.986 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.396+0000 7fb507e00700 1 -- 172.21.15.19:0/1486864336 >> [v2:172.21.15.19:6800/3614957309,v1:172.21.15.19:6801/3614957309] conn(0x7fb50003e4e0 msgr2=0x7fb5000409f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:16:43.986 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.396+0000 7fb507e00700 1 --2- 172.21.15.19:0/1486864336 >> [v2:172.21.15.19:6800/3614957309,v1:172.21.15.19:6801/3614957309] conn(0x7fb50003e4e0 0x7fb5000409f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb508006fd0 tx=0x7fb508006e40 comp rx=0 tx=0).stop 2024-05-07T02:16:43.986 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.396+0000 7fb507e00700 1 -- 172.21.15.19:0/1486864336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb518109b30 msgr2=0x7fb51819ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:16:43.986 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.396+0000 7fb507e00700 1 --2- 172.21.15.19:0/1486864336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb518109b30 0x7fb51819ddd0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb50c011890 tx=0x7fb50c011970 comp rx=0 tx=0).stop 2024-05-07T02:16:43.986 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.396+0000 7fb507e00700 1 -- 172.21.15.19:0/1486864336 shutdown_connections 2024-05-07T02:16:43.987 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.396+0000 7fb507e00700 1 --2- 172.21.15.19:0/1486864336 >> [v2:172.21.15.19:6800/3614957309,v1:172.21.15.19:6801/3614957309] conn(0x7fb50003e4e0 0x7fb5000409f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:16:43.987 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.396+0000 7fb507e00700 1 --2- 172.21.15.19:0/1486864336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb518109b30 0x7fb51819ddd0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:16:43.987 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.396+0000 7fb507e00700 1 -- 172.21.15.19:0/1486864336 >> 172.21.15.19:0/1486864336 conn(0x7fb518105270 msgr2=0x7fb518106b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:16:43.987 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.397+0000 7fb507e00700 1 -- 172.21.15.19:0/1486864336 shutdown_connections 2024-05-07T02:16:43.987 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:43.397+0000 7fb507e00700 1 -- 172.21.15.19:0/1486864336 wait complete. 2024-05-07T02:16:43.987 INFO:teuthology.orchestra.run.smithi019.stdout:Enabling cephadm module... 2024-05-07T02:16:45.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:45 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/2733606141' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-07T02:16:45.373 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:45 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: ignoring --setuser ceph since I am not root 2024-05-07T02:16:45.373 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:45 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: ignoring --setgroup ceph since I am not root 2024-05-07T02:16:45.644 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:45 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:45.435+0000 7f1387901200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-07T02:16:45.706 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:44.697+0000 7f9b60e00700 1 Processor -- start 2024-05-07T02:16:45.706 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:44.697+0000 7f9b60e00700 1 -- start start 2024-05-07T02:16:45.706 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:44.698+0000 7f9b60e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b5c110850 0x7f9b5c110cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:16:45.706 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:44.698+0000 7f9b60e00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5c1112d0 con 0x7f9b5c110850 2024-05-07T02:16:45.706 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:44.698+0000 7f9b59e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b5c110850 0x7f9b5c110cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:16:45.706 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:44.698+0000 7f9b59e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b5c110850 0x7f9b5c110cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41908/0 (socket says 172.21.15.19:41908) 2024-05-07T02:16:45.708 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:44.698+0000 7f9b59e00700 1 -- 172.21.15.19:0/799295062 learned_addr learned my addr 172.21.15.19:0/799295062 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:16:45.708 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:44.699+0000 7f9b59e00700 1 -- 172.21.15.19:0/799295062 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b5c1135e0 con 0x7f9b5c110850 2024-05-07T02:16:45.708 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:44.699+0000 7f9b59e00700 1 --2- 172.21.15.19:0/799295062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b5c110850 0x7f9b5c110cc0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c0098a0 tx=0x7f9b4c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bbb5264df51eb7b server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:16:45.708 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:44.700+0000 7f9b58a00700 1 -- 172.21.15.19:0/799295062 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b4c004160 con 0x7f9b5c110850 2024-05-07T02:16:45.708 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:44.700+0000 7f9b58a00700 1 -- 172.21.15.19:0/799295062 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9b4c004300 con 0x7f9b5c110850 2024-05-07T02:16:45.708 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:44.700+0000 7f9b58a00700 1 -- 172.21.15.19:0/799295062 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b4c011af0 con 0x7f9b5c110850 2024-05-07T02:16:45.709 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:44.700+0000 7f9b60e00700 1 -- 172.21.15.19:0/799295062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b5c110850 msgr2=0x7f9b5c110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:16:45.709 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:44.700+0000 7f9b60e00700 1 --2- 172.21.15.19:0/799295062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b5c110850 0x7f9b5c110cc0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c0098a0 tx=0x7f9b4c009bb0 comp rx=0 tx=0).stop 2024-05-07T02:16:45.709 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:44.701+0000 7f9b60e00700 1 -- 172.21.15.19:0/799295062 shutdown_connections 2024-05-07T02:16:45.709 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:44.701+0000 7f9b60e00700 1 --2- 172.21.15.19:0/799295062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b5c110850 0x7f9b5c110cc0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:16:45.709 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:44.701+0000 7f9b60e00700 1 -- 172.21.15.19:0/799295062 >> 172.21.15.19:0/799295062 conn(0x7f9b5c07f2a0 msgr2=0x7f9b5c0816f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:16:45.709 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:44.701+0000 7f9b60e00700 1 -- 172.21.15.19:0/799295062 shutdown_connections 2024-05-07T02:16:45.709 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:44.701+0000 7f9b60e00700 1 -- 172.21.15.19:0/799295062 wait complete. 2024-05-07T02:16:45.709 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:44.702+0000 7f9b60e00700 1 Processor -- start 2024-05-07T02:16:45.709 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:44.702+0000 7f9b60e00700 1 -- start start 2024-05-07T02:16:45.712 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:44.703+0000 7f9b60e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b5c110850 0x7f9b5c1a4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:16:45.712 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:44.703+0000 7f9b60e00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5c1144d0 con 0x7f9b5c110850 2024-05-07T02:16:45.712 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:44.703+0000 7f9b59e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b5c110850 0x7f9b5c1a4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:16:45.712 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:44.703+0000 7f9b59e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b5c110850 0x7f9b5c1a4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41922/0 (socket says 172.21.15.19:41922) 2024-05-07T02:16:45.712 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:44.703+0000 7f9b59e00700 1 -- 172.21.15.19:0/2733606141 learned_addr learned my addr 172.21.15.19:0/2733606141 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:16:45.712 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:44.703+0000 7f9b59e00700 1 -- 172.21.15.19:0/2733606141 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b5c1a49c0 con 0x7f9b5c110850 2024-05-07T02:16:45.712 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:44.704+0000 7f9b59e00700 1 --2- 172.21.15.19:0/2733606141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b5c110850 0x7f9b5c1a4480 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c0117b0 tx=0x7f9b4c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:16:45.712 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:44.704+0000 7f9b52a00700 1 -- 172.21.15.19:0/2733606141 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b4c0039c0 con 0x7f9b5c110850 2024-05-07T02:16:45.712 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:44.704+0000 7f9b52a00700 1 -- 172.21.15.19:0/2733606141 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9b4c003b60 con 0x7f9b5c110850 2024-05-07T02:16:45.712 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:44.704+0000 7f9b52a00700 1 -- 172.21.15.19:0/2733606141 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b4c0196a0 con 0x7f9b5c110850 2024-05-07T02:16:45.712 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:44.704+0000 7f9b60e00700 1 -- 172.21.15.19:0/2733606141 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b5c1a4c50 con 0x7f9b5c110850 2024-05-07T02:16:45.712 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:44.704+0000 7f9b60e00700 1 -- 172.21.15.19:0/2733606141 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b5c1a7860 con 0x7f9b5c110850 2024-05-07T02:16:45.713 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:44.705+0000 7f9b60e00700 1 -- 172.21.15.19:0/2733606141 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b5c064e20 con 0x7f9b5c110850 2024-05-07T02:16:45.713 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:44.705+0000 7f9b52a00700 1 -- 172.21.15.19:0/2733606141 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f9b4c01a430 con 0x7f9b5c110850 2024-05-07T02:16:45.713 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:44.706+0000 7f9b52a00700 1 --2- 172.21.15.19:0/2733606141 >> [v2:172.21.15.19:6800/3614957309,v1:172.21.15.19:6801/3614957309] conn(0x7f9b4003e530 0x7f9b40040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:16:45.713 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:44.706+0000 7f9b52a00700 1 -- 172.21.15.19:0/2733606141 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9b4c053b90 con 0x7f9b5c110850 2024-05-07T02:16:45.713 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:44.706+0000 7f9b59400700 1 --2- 172.21.15.19:0/2733606141 >> [v2:172.21.15.19:6800/3614957309,v1:172.21.15.19:6801/3614957309] conn(0x7f9b4003e530 0x7f9b40040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:16:45.713 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:44.707+0000 7f9b59400700 1 --2- 172.21.15.19:0/2733606141 >> [v2:172.21.15.19:6800/3614957309,v1:172.21.15.19:6801/3614957309] conn(0x7f9b4003e530 0x7f9b40040a40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9b48006fd0 tx=0x7f9b48006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:16:45.713 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:44.712+0000 7f9b52a00700 1 -- 172.21.15.19:0/2733606141 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9b4c00b3e0 con 0x7f9b5c110850 2024-05-07T02:16:45.713 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:45.009+0000 7f9b60e00700 1 -- 172.21.15.19:0/2733606141 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f9b5c1a7a30 con 0x7f9b5c110850 2024-05-07T02:16:45.713 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:45.063+0000 7f9b52a00700 1 -- 172.21.15.19:0/2733606141 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f9b4c02cdc0 con 0x7f9b5c110850 2024-05-07T02:16:45.713 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:45.066+0000 7f9b60e00700 1 -- 172.21.15.19:0/2733606141 >> [v2:172.21.15.19:6800/3614957309,v1:172.21.15.19:6801/3614957309] conn(0x7f9b4003e530 msgr2=0x7f9b40040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:16:45.714 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:45.066+0000 7f9b60e00700 1 --2- 172.21.15.19:0/2733606141 >> [v2:172.21.15.19:6800/3614957309,v1:172.21.15.19:6801/3614957309] conn(0x7f9b4003e530 0x7f9b40040a40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9b48006fd0 tx=0x7f9b48006e40 comp rx=0 tx=0).stop 2024-05-07T02:16:45.715 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:45.066+0000 7f9b60e00700 1 -- 172.21.15.19:0/2733606141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b5c110850 msgr2=0x7f9b5c1a4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:16:45.715 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:45.067+0000 7f9b52a00700 1 -- 172.21.15.19:0/2733606141 <== mon.0 v2:172.21.15.19:3300/0 8 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f9b4c01ab00 con 0x7f9b5c110850 2024-05-07T02:16:45.715 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:45.067+0000 7f9b60e00700 1 --2- 172.21.15.19:0/2733606141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b5c110850 0x7f9b5c1a4480 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c0117b0 tx=0x7f9b4c011b70 comp rx=0 tx=0).stop 2024-05-07T02:16:45.715 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:45.067+0000 7f9b60e00700 1 -- 172.21.15.19:0/2733606141 shutdown_connections 2024-05-07T02:16:45.715 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:45.067+0000 7f9b60e00700 1 --2- 172.21.15.19:0/2733606141 >> [v2:172.21.15.19:6800/3614957309,v1:172.21.15.19:6801/3614957309] conn(0x7f9b4003e530 0x7f9b40040a40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:16:45.715 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:45.067+0000 7f9b60e00700 1 --2- 172.21.15.19:0/2733606141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b5c110850 0x7f9b5c1a4480 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:16:45.715 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:45.067+0000 7f9b60e00700 1 -- 172.21.15.19:0/2733606141 >> 172.21.15.19:0/2733606141 conn(0x7f9b5c07f2a0 msgr2=0x7f9b5c080b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:16:45.715 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:45.069+0000 7f9b60e00700 1 -- 172.21.15.19:0/2733606141 shutdown_connections 2024-05-07T02:16:45.715 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:45.069+0000 7f9b60e00700 1 -- 172.21.15.19:0/2733606141 wait complete. 2024-05-07T02:16:46.010 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:45 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:45.687+0000 7f1387901200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-07T02:16:46.010 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:45 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:45.826+0000 7f1387901200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-07T02:16:46.365 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:46 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/2733606141' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-07T02:16:46.365 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:46 smithi019 ceph-mon[29781]: mgrmap e5: x(active, since 5s) 2024-05-07T02:16:46.366 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:46 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:46.245+0000 7f1387901200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-07T02:16:46.644 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:46 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:46.365+0000 7f1387901200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-07T02:16:47.269 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:47 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/2773709135' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-07T02:16:47.289 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-05-07T02:16:47.289 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-07T02:16:47.289 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-07T02:16:47.289 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-07T02:16:47.289 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-07T02:16:47.292 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-05-07T02:16:47.292 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.381+0000 7fbe18c00700 1 Processor -- start 2024-05-07T02:16:47.292 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.381+0000 7fbe18c00700 1 -- start start 2024-05-07T02:16:47.292 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.382+0000 7fbe18c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe1407ed10 0x7fbe1407d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:16:47.292 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.382+0000 7fbe18c00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1407d770 con 0x7fbe1407ed10 2024-05-07T02:16:47.292 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.382+0000 7fbe13400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe1407ed10 0x7fbe1407d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:16:47.292 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.382+0000 7fbe13400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe1407ed10 0x7fbe1407d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41950/0 (socket says 172.21.15.19:41950) 2024-05-07T02:16:47.292 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.382+0000 7fbe13400700 1 -- 172.21.15.19:0/775215038 learned_addr learned my addr 172.21.15.19:0/775215038 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:16:47.292 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.383+0000 7fbe13400700 1 -- 172.21.15.19:0/775215038 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe1407d8f0 con 0x7fbe1407ed10 2024-05-07T02:16:47.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.383+0000 7fbe13400700 1 --2- 172.21.15.19:0/775215038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe1407ed10 0x7fbe1407d230 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbe040098a0 tx=0x7fbe04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88f8ee4879e65280 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:16:47.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.383+0000 7fbe12000700 1 -- 172.21.15.19:0/775215038 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe04004160 con 0x7fbe1407ed10 2024-05-07T02:16:47.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.383+0000 7fbe12000700 1 -- 172.21.15.19:0/775215038 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbe04004300 con 0x7fbe1407ed10 2024-05-07T02:16:47.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.383+0000 7fbe12000700 1 -- 172.21.15.19:0/775215038 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe04011af0 con 0x7fbe1407ed10 2024-05-07T02:16:47.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.384+0000 7fbe18c00700 1 -- 172.21.15.19:0/775215038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe1407ed10 msgr2=0x7fbe1407d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:16:47.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.384+0000 7fbe18c00700 1 --2- 172.21.15.19:0/775215038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe1407ed10 0x7fbe1407d230 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbe040098a0 tx=0x7fbe04009bb0 comp rx=0 tx=0).stop 2024-05-07T02:16:47.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.384+0000 7fbe18c00700 1 -- 172.21.15.19:0/775215038 shutdown_connections 2024-05-07T02:16:47.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.384+0000 7fbe18c00700 1 --2- 172.21.15.19:0/775215038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe1407ed10 0x7fbe1407d230 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:16:47.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.384+0000 7fbe18c00700 1 -- 172.21.15.19:0/775215038 >> 172.21.15.19:0/775215038 conn(0x7fbe14104f70 msgr2=0x7fbe14107400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:16:47.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.385+0000 7fbe18c00700 1 -- 172.21.15.19:0/775215038 shutdown_connections 2024-05-07T02:16:47.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.385+0000 7fbe18c00700 1 -- 172.21.15.19:0/775215038 wait complete. 2024-05-07T02:16:47.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.386+0000 7fbe18c00700 1 Processor -- start 2024-05-07T02:16:47.293 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.386+0000 7fbe18c00700 1 -- start start 2024-05-07T02:16:47.294 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.386+0000 7fbe18c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe1407ed10 0x7fbe1419fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:16:47.294 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.386+0000 7fbe18c00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1410fe10 con 0x7fbe1407ed10 2024-05-07T02:16:47.294 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.387+0000 7fbe13400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe1407ed10 0x7fbe1419fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:16:47.297 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.387+0000 7fbe13400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe1407ed10 0x7fbe1419fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41966/0 (socket says 172.21.15.19:41966) 2024-05-07T02:16:47.298 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.387+0000 7fbe13400700 1 -- 172.21.15.19:0/2773709135 learned_addr learned my addr 172.21.15.19:0/2773709135 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:16:47.298 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.387+0000 7fbe13400700 1 -- 172.21.15.19:0/2773709135 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe141a0330 con 0x7fbe1407ed10 2024-05-07T02:16:47.298 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.387+0000 7fbe13400700 1 --2- 172.21.15.19:0/2773709135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe1407ed10 0x7fbe1419fdf0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbe040117b0 tx=0x7fbe04011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:16:47.298 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.388+0000 7fbe0be00700 1 -- 172.21.15.19:0/2773709135 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe040039c0 con 0x7fbe1407ed10 2024-05-07T02:16:47.298 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.388+0000 7fbe0be00700 1 -- 172.21.15.19:0/2773709135 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbe04003b60 con 0x7fbe1407ed10 2024-05-07T02:16:47.298 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.388+0000 7fbe0be00700 1 -- 172.21.15.19:0/2773709135 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe040196a0 con 0x7fbe1407ed10 2024-05-07T02:16:47.298 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.388+0000 7fbe18c00700 1 -- 172.21.15.19:0/2773709135 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe141a05c0 con 0x7fbe1407ed10 2024-05-07T02:16:47.298 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.389+0000 7fbe18c00700 1 -- 172.21.15.19:0/2773709135 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe141a31d0 con 0x7fbe1407ed10 2024-05-07T02:16:47.298 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.390+0000 7fbe18c00700 1 -- 172.21.15.19:0/2773709135 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe14064e20 con 0x7fbe1407ed10 2024-05-07T02:16:47.298 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.390+0000 7fbe0be00700 1 -- 172.21.15.19:0/2773709135 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7fbe0401ab00 con 0x7fbe1407ed10 2024-05-07T02:16:47.298 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.390+0000 7fbe0be00700 1 --2- 172.21.15.19:0/2773709135 >> [v2:172.21.15.19:6800/3614957309,v1:172.21.15.19:6801/3614957309] conn(0x7fbdf803e5e0 0x7fbdf8040af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:16:47.298 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.391+0000 7fbe0be00700 1 -- 172.21.15.19:0/2773709135 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbe04053c00 con 0x7fbe1407ed10 2024-05-07T02:16:47.298 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.391+0000 7fbe12a00700 1 -- 172.21.15.19:0/2773709135 >> [v2:172.21.15.19:6800/3614957309,v1:172.21.15.19:6801/3614957309] conn(0x7fbdf803e5e0 msgr2=0x7fbdf8040af0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/3614957309 2024-05-07T02:16:47.299 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.391+0000 7fbe12a00700 1 --2- 172.21.15.19:0/2773709135 >> [v2:172.21.15.19:6800/3614957309,v1:172.21.15.19:6801/3614957309] conn(0x7fbdf803e5e0 0x7fbdf8040af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-07T02:16:47.299 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.396+0000 7fbe0be00700 1 -- 172.21.15.19:0/2773709135 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbe04019bc0 con 0x7fbe1407ed10 2024-05-07T02:16:47.299 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.592+0000 7fbe12a00700 1 -- 172.21.15.19:0/2773709135 >> [v2:172.21.15.19:6800/3614957309,v1:172.21.15.19:6801/3614957309] conn(0x7fbdf803e5e0 msgr2=0x7fbdf8040af0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/3614957309 2024-05-07T02:16:47.299 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.592+0000 7fbe12a00700 1 --2- 172.21.15.19:0/2773709135 >> [v2:172.21.15.19:6800/3614957309,v1:172.21.15.19:6801/3614957309] conn(0x7fbdf803e5e0 0x7fbdf8040af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-07T02:16:47.299 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.682+0000 7fbe18c00700 1 -- 172.21.15.19:0/2773709135 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fbe141a3850 con 0x7fbe1407ed10 2024-05-07T02:16:47.299 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.682+0000 7fbe0be00700 1 -- 172.21.15.19:0/2773709135 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fbe0400b3e0 con 0x7fbe1407ed10 2024-05-07T02:16:47.299 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.685+0000 7fbe18c00700 1 -- 172.21.15.19:0/2773709135 >> [v2:172.21.15.19:6800/3614957309,v1:172.21.15.19:6801/3614957309] conn(0x7fbdf803e5e0 msgr2=0x7fbdf8040af0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-07T02:16:47.299 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.685+0000 7fbe18c00700 1 --2- 172.21.15.19:0/2773709135 >> [v2:172.21.15.19:6800/3614957309,v1:172.21.15.19:6801/3614957309] conn(0x7fbdf803e5e0 0x7fbdf8040af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:16:47.299 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.685+0000 7fbe18c00700 1 -- 172.21.15.19:0/2773709135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe1407ed10 msgr2=0x7fbe1419fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:16:47.299 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.685+0000 7fbe18c00700 1 --2- 172.21.15.19:0/2773709135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe1407ed10 0x7fbe1419fdf0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbe040117b0 tx=0x7fbe04011b70 comp rx=0 tx=0).stop 2024-05-07T02:16:47.299 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.685+0000 7fbe18c00700 1 -- 172.21.15.19:0/2773709135 shutdown_connections 2024-05-07T02:16:47.299 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.685+0000 7fbe18c00700 1 --2- 172.21.15.19:0/2773709135 >> [v2:172.21.15.19:6800/3614957309,v1:172.21.15.19:6801/3614957309] conn(0x7fbdf803e5e0 0x7fbdf8040af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:16:47.300 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.685+0000 7fbe18c00700 1 --2- 172.21.15.19:0/2773709135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe1407ed10 0x7fbe1419fdf0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:16:47.300 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.685+0000 7fbe18c00700 1 -- 172.21.15.19:0/2773709135 >> 172.21.15.19:0/2773709135 conn(0x7fbe14104f70 msgr2=0x7fbe14105840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:16:47.300 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.686+0000 7fbe18c00700 1 -- 172.21.15.19:0/2773709135 shutdown_connections 2024-05-07T02:16:47.300 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:46.686+0000 7fbe18c00700 1 -- 172.21.15.19:0/2773709135 wait complete. 2024-05-07T02:16:47.300 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for the mgr to restart... 2024-05-07T02:16:47.300 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mgr epoch 5... 2024-05-07T02:16:48.273 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:48 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:48.034+0000 7f1387901200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-07T02:16:48.644 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:48 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:48.272+0000 7f1387901200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-07T02:16:48.644 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:48 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:48.494+0000 7f1387901200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-07T02:16:49.144 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:48 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:48.892+0000 7f1387901200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-07T02:16:49.145 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:49 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:49.020+0000 7f1387901200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-07T02:16:49.544 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:49 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:49.280+0000 7f1387901200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-07T02:16:49.894 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:49 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:49.544+0000 7f1387901200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-07T02:16:50.644 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:50 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:50.360+0000 7f1387901200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-07T02:16:50.924 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:50 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:50.658+0000 7f1387901200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-07T02:16:51.644 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:51 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:51.235+0000 7f1387901200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-07T02:16:51.645 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:51 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:51.369+0000 7f1387901200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-07T02:16:52.144 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:51 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:51.846+0000 7f1387901200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-07T02:16:52.644 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:52 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:52.318+0000 7f1387901200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-07T02:16:52.644 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:52 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:52.446+0000 7f1387901200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-07T02:16:53.144 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:52 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:52.782+0000 7f1387901200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-07T02:16:53.144 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:52 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:52.967+0000 7f1387901200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-07T02:16:53.635 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:53 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:53.350+0000 7f1387901200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-07T02:16:53.636 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:53 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:53.507+0000 7f1387901200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-07T02:16:53.894 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:53 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:53.635+0000 7f1387901200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-07T02:16:54.728 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:54 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:54.423+0000 7f1387901200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-07T02:16:55.059 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:16:54 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:16:54.728+0000 7f1387901200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-07T02:16:55.060 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:54 smithi019 ceph-mon[29781]: Active manager daemon x restarted 2024-05-07T02:16:55.060 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:54 smithi019 ceph-mon[29781]: Activating manager daemon x 2024-05-07T02:16:55.060 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:54 smithi019 ceph-mon[29781]: osdmap e2: 0 total, 0 up, 0 in 2024-05-07T02:16:55.060 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:54 smithi019 ceph-mon[29781]: mgrmap e6: x(active, starting, since 0.0108092s) 2024-05-07T02:16:55.060 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:54 smithi019 ceph-mon[29781]: from='mgr.14120 172.21.15.19:0/3795770355' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-07T02:16:55.060 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:54 smithi019 ceph-mon[29781]: from='mgr.14120 172.21.15.19:0/3795770355' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-07T02:16:55.061 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:54 smithi019 ceph-mon[29781]: from='mgr.14120 172.21.15.19:0/3795770355' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-07T02:16:55.061 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:54 smithi019 ceph-mon[29781]: from='mgr.14120 172.21.15.19:0/3795770355' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-07T02:16:55.061 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:54 smithi019 ceph-mon[29781]: from='mgr.14120 172.21.15.19:0/3795770355' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-07T02:16:55.061 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:54 smithi019 ceph-mon[29781]: Manager daemon x is now available 2024-05-07T02:16:56.123 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:55 smithi019 ceph-mon[29781]: Found migration_current of "None". Setting to last migration. 2024-05-07T02:16:56.124 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:55 smithi019 ceph-mon[29781]: from='mgr.14120 172.21.15.19:0/3795770355' entity='mgr.x' 2024-05-07T02:16:56.124 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:55 smithi019 ceph-mon[29781]: from='mgr.14120 172.21.15.19:0/3795770355' entity='mgr.x' 2024-05-07T02:16:56.124 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:55 smithi019 ceph-mon[29781]: from='mgr.14120 172.21.15.19:0/3795770355' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:16:56.124 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:55 smithi019 ceph-mon[29781]: from='mgr.14120 172.21.15.19:0/3795770355' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:16:56.124 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:55 smithi019 ceph-mon[29781]: from='mgr.14120 172.21.15.19:0/3795770355' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-07T02:16:56.124 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:55 smithi019 ceph-mon[29781]: from='mgr.14120 172.21.15.19:0/3795770355' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-07T02:16:56.124 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:55 smithi019 ceph-mon[29781]: mgrmap e7: x(active, since 1.01888s) 2024-05-07T02:16:56.473 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-05-07T02:16:56.481 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-07T02:16:56.481 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-07T02:16:56.481 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-05-07T02:16:56.481 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:47.910+0000 7f2718c00700 1 Processor -- start 2024-05-07T02:16:56.481 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:47.910+0000 7f2718c00700 1 -- start start 2024-05-07T02:16:56.481 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:47.911+0000 7f2718c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f271407ed60 0x7f271407d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:16:56.481 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:47.911+0000 7f2718c00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f271407d770 con 0x7f271407ed60 2024-05-07T02:16:56.481 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:47.911+0000 7f2711e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f271407ed60 0x7f271407d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:16:56.481 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:47.911+0000 7f2711e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f271407ed60 0x7f271407d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41978/0 (socket says 172.21.15.19:41978) 2024-05-07T02:16:56.482 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:47.911+0000 7f2711e00700 1 -- 172.21.15.19:0/1937048070 learned_addr learned my addr 172.21.15.19:0/1937048070 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:16:56.482 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:47.912+0000 7f2711e00700 1 -- 172.21.15.19:0/1937048070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f271407d8f0 con 0x7f271407ed60 2024-05-07T02:16:56.482 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:47.912+0000 7f2711e00700 1 --2- 172.21.15.19:0/1937048070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f271407ed60 0x7f271407d230 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f27040098a0 tx=0x7f2704009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b91098ff7d19c360 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:16:56.482 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:47.912+0000 7f2710a00700 1 -- 172.21.15.19:0/1937048070 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2704004160 con 0x7f271407ed60 2024-05-07T02:16:56.482 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:47.912+0000 7f2710a00700 1 -- 172.21.15.19:0/1937048070 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2704004300 con 0x7f271407ed60 2024-05-07T02:16:56.482 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:47.912+0000 7f2710a00700 1 -- 172.21.15.19:0/1937048070 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2704011af0 con 0x7f271407ed60 2024-05-07T02:16:56.482 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:47.913+0000 7f2718c00700 1 -- 172.21.15.19:0/1937048070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f271407ed60 msgr2=0x7f271407d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:16:56.482 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:47.913+0000 7f2718c00700 1 --2- 172.21.15.19:0/1937048070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f271407ed60 0x7f271407d230 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f27040098a0 tx=0x7f2704009bb0 comp rx=0 tx=0).stop 2024-05-07T02:16:56.482 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:47.913+0000 7f2718c00700 1 -- 172.21.15.19:0/1937048070 shutdown_connections 2024-05-07T02:16:56.482 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:47.913+0000 7f2718c00700 1 --2- 172.21.15.19:0/1937048070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f271407ed60 0x7f271407d230 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:16:56.482 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:47.913+0000 7f2718c00700 1 -- 172.21.15.19:0/1937048070 >> 172.21.15.19:0/1937048070 conn(0x7f2714105150 msgr2=0x7f27141075c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:16:56.482 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:47.913+0000 7f2718c00700 1 -- 172.21.15.19:0/1937048070 shutdown_connections 2024-05-07T02:16:56.482 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:47.914+0000 7f2718c00700 1 -- 172.21.15.19:0/1937048070 wait complete. 2024-05-07T02:16:56.482 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:47.915+0000 7f2718c00700 1 Processor -- start 2024-05-07T02:16:56.482 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:47.915+0000 7f2718c00700 1 -- start start 2024-05-07T02:16:56.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:47.915+0000 7f2718c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f271407ed60 0x7f271419ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:16:56.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:47.915+0000 7f2718c00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2714110030 con 0x7f271407ed60 2024-05-07T02:16:56.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:47.915+0000 7f2711e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f271407ed60 0x7f271419ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:16:56.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:47.915+0000 7f2711e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f271407ed60 0x7f271419ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41984/0 (socket says 172.21.15.19:41984) 2024-05-07T02:16:56.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:47.915+0000 7f2711e00700 1 -- 172.21.15.19:0/3807061825 learned_addr learned my addr 172.21.15.19:0/3807061825 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:16:56.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:47.916+0000 7f2711e00700 1 -- 172.21.15.19:0/3807061825 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27141a04f0 con 0x7f271407ed60 2024-05-07T02:16:56.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:47.916+0000 7f2711e00700 1 --2- 172.21.15.19:0/3807061825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f271407ed60 0x7f271419ffb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2704009fb0 tx=0x7f2704011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:16:56.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:47.916+0000 7f270aa00700 1 -- 172.21.15.19:0/3807061825 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27040036a0 con 0x7f271407ed60 2024-05-07T02:16:56.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:47.917+0000 7f270aa00700 1 -- 172.21.15.19:0/3807061825 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f270401a430 con 0x7f271407ed60 2024-05-07T02:16:56.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:47.917+0000 7f270aa00700 1 -- 172.21.15.19:0/3807061825 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2704019550 con 0x7f271407ed60 2024-05-07T02:16:56.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:47.917+0000 7f2718c00700 1 -- 172.21.15.19:0/3807061825 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27141a0780 con 0x7f271407ed60 2024-05-07T02:16:56.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:47.917+0000 7f2718c00700 1 -- 172.21.15.19:0/3807061825 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27141a3390 con 0x7f271407ed60 2024-05-07T02:16:56.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:47.918+0000 7f270aa00700 1 -- 172.21.15.19:0/3807061825 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f2704003840 con 0x7f271407ed60 2024-05-07T02:16:56.483 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:47.918+0000 7f270aa00700 1 --2- 172.21.15.19:0/3807061825 >> [v2:172.21.15.19:6800/3614957309,v1:172.21.15.19:6801/3614957309] conn(0x7f26f803e530 0x7f26f8040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:16:56.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:47.918+0000 7f270aa00700 1 -- 172.21.15.19:0/3807061825 --> [v2:172.21.15.19:6800/3614957309,v1:172.21.15.19:6801/3614957309] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f26f8041150 con 0x7f26f803e530 2024-05-07T02:16:56.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:47.918+0000 7f270aa00700 1 -- 172.21.15.19:0/3807061825 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2704053860 con 0x7f271407ed60 2024-05-07T02:16:56.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:47.919+0000 7f2711400700 1 -- 172.21.15.19:0/3807061825 >> [v2:172.21.15.19:6800/3614957309,v1:172.21.15.19:6801/3614957309] conn(0x7f26f803e530 msgr2=0x7f26f8040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/3614957309 2024-05-07T02:16:56.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:47.919+0000 7f2711400700 1 --2- 172.21.15.19:0/3807061825 >> [v2:172.21.15.19:6800/3614957309,v1:172.21.15.19:6801/3614957309] conn(0x7f26f803e530 0x7f26f8040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-07T02:16:56.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:48.120+0000 7f2711400700 1 -- 172.21.15.19:0/3807061825 >> [v2:172.21.15.19:6800/3614957309,v1:172.21.15.19:6801/3614957309] conn(0x7f26f803e530 msgr2=0x7f26f8040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/3614957309 2024-05-07T02:16:56.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:48.120+0000 7f2711400700 1 --2- 172.21.15.19:0/3807061825 >> [v2:172.21.15.19:6800/3614957309,v1:172.21.15.19:6801/3614957309] conn(0x7f26f803e530 0x7f26f8040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-07T02:16:56.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:48.521+0000 7f2711400700 1 -- 172.21.15.19:0/3807061825 >> [v2:172.21.15.19:6800/3614957309,v1:172.21.15.19:6801/3614957309] conn(0x7f26f803e530 msgr2=0x7f26f8040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/3614957309 2024-05-07T02:16:56.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:48.521+0000 7f2711400700 1 --2- 172.21.15.19:0/3807061825 >> [v2:172.21.15.19:6800/3614957309,v1:172.21.15.19:6801/3614957309] conn(0x7f26f803e530 0x7f26f8040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-07T02:16:56.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:49.322+0000 7f2711400700 1 -- 172.21.15.19:0/3807061825 >> [v2:172.21.15.19:6800/3614957309,v1:172.21.15.19:6801/3614957309] conn(0x7f26f803e530 msgr2=0x7f26f8040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/3614957309 2024-05-07T02:16:56.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:49.322+0000 7f2711400700 1 --2- 172.21.15.19:0/3807061825 >> [v2:172.21.15.19:6800/3614957309,v1:172.21.15.19:6801/3614957309] conn(0x7f26f803e530 0x7f26f8040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-07T02:16:56.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:50.924+0000 7f2711400700 1 -- 172.21.15.19:0/3807061825 >> [v2:172.21.15.19:6800/3614957309,v1:172.21.15.19:6801/3614957309] conn(0x7f26f803e530 msgr2=0x7f26f8040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/3614957309 2024-05-07T02:16:56.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:50.924+0000 7f2711400700 1 --2- 172.21.15.19:0/3807061825 >> [v2:172.21.15.19:6800/3614957309,v1:172.21.15.19:6801/3614957309] conn(0x7f26f803e530 0x7f26f8040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-07T02:16:56.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:54.127+0000 7f2711400700 1 -- 172.21.15.19:0/3807061825 >> [v2:172.21.15.19:6800/3614957309,v1:172.21.15.19:6801/3614957309] conn(0x7f26f803e530 msgr2=0x7f26f8040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/3614957309 2024-05-07T02:16:56.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:54.128+0000 7f2711400700 1 --2- 172.21.15.19:0/3807061825 >> [v2:172.21.15.19:6800/3614957309,v1:172.21.15.19:6801/3614957309] conn(0x7f26f803e530 0x7f26f8040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-05-07T02:16:56.484 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:54.737+0000 7f270aa00700 1 -- 172.21.15.19:0/3807061825 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mgrmap(e 6) ==== 49989+0+0 (secure 0 0 0) 0x7f2704019800 con 0x7f271407ed60 2024-05-07T02:16:56.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:54.737+0000 7f270aa00700 1 -- 172.21.15.19:0/3807061825 >> [v2:172.21.15.19:6800/3614957309,v1:172.21.15.19:6801/3614957309] conn(0x7f26f803e530 msgr2=0x7f26f8040a40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-07T02:16:56.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:54.737+0000 7f270aa00700 1 --2- 172.21.15.19:0/3807061825 >> [v2:172.21.15.19:6800/3614957309,v1:172.21.15.19:6801/3614957309] conn(0x7f26f803e530 0x7f26f8040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:16:56.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:55.743+0000 7f270aa00700 1 -- 172.21.15.19:0/3807061825 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7f2704010380 con 0x7f271407ed60 2024-05-07T02:16:56.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:55.744+0000 7f270aa00700 1 --2- 172.21.15.19:0/3807061825 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f26f8041e70 0x7f26f8044290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:16:56.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:55.744+0000 7f270aa00700 1 -- 172.21.15.19:0/3807061825 --> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2704022e70 con 0x7f26f8041e70 2024-05-07T02:16:56.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:55.745+0000 7f2711400700 1 --2- 172.21.15.19:0/3807061825 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f26f8041e70 0x7f26f8044290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:16:56.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:55.748+0000 7f2711400700 1 --2- 172.21.15.19:0/3807061825 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f26f8041e70 0x7f26f8044290 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2700003a10 tx=0x7f27000092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:16:56.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:55.751+0000 7f270aa00700 1 -- 172.21.15.19:0/3807061825 <== mgr.14120 v2:172.21.15.19:6800/174304059 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f2704022e70 con 0x7f26f8041e70 2024-05-07T02:16:56.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:55.761+0000 7f2718c00700 1 -- 172.21.15.19:0/3807061825 --> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f2714064e20 con 0x7f26f8041e70 2024-05-07T02:16:56.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:55.762+0000 7f270aa00700 1 -- 172.21.15.19:0/3807061825 <== mgr.14120 v2:172.21.15.19:6800/174304059 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f2714064e20 con 0x7f26f8041e70 2024-05-07T02:16:56.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:55.762+0000 7f2718c00700 1 -- 172.21.15.19:0/3807061825 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f26f8041e70 msgr2=0x7f26f8044290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:16:56.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:55.762+0000 7f2718c00700 1 --2- 172.21.15.19:0/3807061825 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f26f8041e70 0x7f26f8044290 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2700003a10 tx=0x7f27000092b0 comp rx=0 tx=0).stop 2024-05-07T02:16:56.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:55.762+0000 7f2718c00700 1 -- 172.21.15.19:0/3807061825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f271407ed60 msgr2=0x7f271419ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:16:56.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:55.762+0000 7f2718c00700 1 --2- 172.21.15.19:0/3807061825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f271407ed60 0x7f271419ffb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2704009fb0 tx=0x7f2704011940 comp rx=0 tx=0).stop 2024-05-07T02:16:56.485 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:55.762+0000 7f2718c00700 1 -- 172.21.15.19:0/3807061825 shutdown_connections 2024-05-07T02:16:56.486 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:55.763+0000 7f2718c00700 1 --2- 172.21.15.19:0/3807061825 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f26f8041e70 0x7f26f8044290 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:16:56.486 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:55.763+0000 7f2718c00700 1 --2- 172.21.15.19:0/3807061825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f271407ed60 0x7f271419ffb0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:16:56.486 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:55.763+0000 7f2718c00700 1 --2- 172.21.15.19:0/3807061825 >> [v2:172.21.15.19:6800/3614957309,v1:172.21.15.19:6801/3614957309] conn(0x7f26f803e530 0x7f26f8040a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:16:56.486 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:55.763+0000 7f2718c00700 1 -- 172.21.15.19:0/3807061825 >> 172.21.15.19:0/3807061825 conn(0x7f2714105150 msgr2=0x7f27141059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:16:56.486 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:55.763+0000 7f2718c00700 1 -- 172.21.15.19:0/3807061825 shutdown_connections 2024-05-07T02:16:56.486 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:55.763+0000 7f2718c00700 1 -- 172.21.15.19:0/3807061825 wait complete. 2024-05-07T02:16:56.486 INFO:teuthology.orchestra.run.smithi019.stdout:mgr epoch 5 is available 2024-05-07T02:16:56.486 INFO:teuthology.orchestra.run.smithi019.stdout:Setting orchestrator backend to cephadm... 2024-05-07T02:16:57.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:56 smithi019 ceph-mon[29781]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-07T02:16:57.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:56 smithi019 ceph-mon[29781]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-07T02:16:57.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:56 smithi019 ceph-mon[29781]: from='mgr.14120 172.21.15.19:0/3795770355' entity='mgr.x' 2024-05-07T02:16:57.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:56 smithi019 ceph-mon[29781]: from='mgr.14120 172.21.15.19:0/3795770355' entity='mgr.x' 2024-05-07T02:16:57.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:56 smithi019 ceph-mon[29781]: from='mgr.14120 172.21.15.19:0/3795770355' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:16:58.072 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.100+0000 7fac65c00700 1 Processor -- start 2024-05-07T02:16:58.072 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.101+0000 7fac65c00700 1 -- start start 2024-05-07T02:16:58.072 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.101+0000 7fac65c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac6010bd30 0x7fac6010c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:16:58.072 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.101+0000 7fac65c00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fac6010c730 con 0x7fac6010bd30 2024-05-07T02:16:58.072 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.102+0000 7fac5ea00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac6010bd30 0x7fac6010c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:16:58.072 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.102+0000 7fac5ea00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac6010bd30 0x7fac6010c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41196/0 (socket says 172.21.15.19:41196) 2024-05-07T02:16:58.073 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.102+0000 7fac5ea00700 1 -- 172.21.15.19:0/1418205339 learned_addr learned my addr 172.21.15.19:0/1418205339 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:16:58.073 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.102+0000 7fac5ea00700 1 -- 172.21.15.19:0/1418205339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac6010ce60 con 0x7fac6010bd30 2024-05-07T02:16:58.073 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.102+0000 7fac5ea00700 1 --2- 172.21.15.19:0/1418205339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac6010bd30 0x7fac6010c140 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fac500098a0 tx=0x7fac50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c10d40ff161a9d39 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:16:58.073 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.103+0000 7fac5d600700 1 -- 172.21.15.19:0/1418205339 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac50004160 con 0x7fac6010bd30 2024-05-07T02:16:58.073 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.103+0000 7fac5d600700 1 -- 172.21.15.19:0/1418205339 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fac50004300 con 0x7fac6010bd30 2024-05-07T02:16:58.073 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.103+0000 7fac5d600700 1 -- 172.21.15.19:0/1418205339 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac50011af0 con 0x7fac6010bd30 2024-05-07T02:16:58.073 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.104+0000 7fac65c00700 1 -- 172.21.15.19:0/1418205339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac6010bd30 msgr2=0x7fac6010c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:16:58.073 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.104+0000 7fac65c00700 1 --2- 172.21.15.19:0/1418205339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac6010bd30 0x7fac6010c140 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fac500098a0 tx=0x7fac50009bb0 comp rx=0 tx=0).stop 2024-05-07T02:16:58.073 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.104+0000 7fac65c00700 1 -- 172.21.15.19:0/1418205339 shutdown_connections 2024-05-07T02:16:58.073 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.104+0000 7fac65c00700 1 --2- 172.21.15.19:0/1418205339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac6010bd30 0x7fac6010c140 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:16:58.073 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.104+0000 7fac65c00700 1 -- 172.21.15.19:0/1418205339 >> 172.21.15.19:0/1418205339 conn(0x7fac6007f2a0 msgr2=0x7fac60109d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:16:58.073 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.104+0000 7fac65c00700 1 -- 172.21.15.19:0/1418205339 shutdown_connections 2024-05-07T02:16:58.073 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.104+0000 7fac65c00700 1 -- 172.21.15.19:0/1418205339 wait complete. 2024-05-07T02:16:58.073 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.105+0000 7fac65c00700 1 Processor -- start 2024-05-07T02:16:58.073 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.106+0000 7fac65c00700 1 -- start start 2024-05-07T02:16:58.074 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.106+0000 7fac65c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac6010bd30 0x7fac601a0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:16:58.074 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.106+0000 7fac65c00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fac60110090 con 0x7fac6010bd30 2024-05-07T02:16:58.074 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.106+0000 7fac5ea00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac6010bd30 0x7fac601a0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:16:58.074 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.106+0000 7fac5ea00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac6010bd30 0x7fac601a0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41202/0 (socket says 172.21.15.19:41202) 2024-05-07T02:16:58.074 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.106+0000 7fac5ea00700 1 -- 172.21.15.19:0/1197776413 learned_addr learned my addr 172.21.15.19:0/1197776413 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:16:58.074 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.107+0000 7fac5ea00700 1 -- 172.21.15.19:0/1197776413 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac601a0580 con 0x7fac6010bd30 2024-05-07T02:16:58.074 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.107+0000 7fac5ea00700 1 --2- 172.21.15.19:0/1197776413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac6010bd30 0x7fac601a0040 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fac500090e0 tx=0x7fac50011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:16:58.074 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.107+0000 7fac57400700 1 -- 172.21.15.19:0/1197776413 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac500039e0 con 0x7fac6010bd30 2024-05-07T02:16:58.074 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.107+0000 7fac57400700 1 -- 172.21.15.19:0/1197776413 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fac50003b80 con 0x7fac6010bd30 2024-05-07T02:16:58.074 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.107+0000 7fac65c00700 1 -- 172.21.15.19:0/1197776413 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac601a0810 con 0x7fac6010bd30 2024-05-07T02:16:58.074 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.108+0000 7fac57400700 1 -- 172.21.15.19:0/1197776413 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac50024690 con 0x7fac6010bd30 2024-05-07T02:16:58.074 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.108+0000 7fac65c00700 1 -- 172.21.15.19:0/1197776413 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac601a3420 con 0x7fac6010bd30 2024-05-07T02:16:58.075 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.109+0000 7fac65c00700 1 -- 172.21.15.19:0/1197776413 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac60064e20 con 0x7fac6010bd30 2024-05-07T02:16:58.075 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.109+0000 7fac57400700 1 -- 172.21.15.19:0/1197776413 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fac50022020 con 0x7fac6010bd30 2024-05-07T02:16:58.075 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.110+0000 7fac57400700 1 --2- 172.21.15.19:0/1197776413 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7fac4403e580 0x7fac44040a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:16:58.075 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.110+0000 7fac57400700 1 -- 172.21.15.19:0/1197776413 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fac500537a0 con 0x7fac6010bd30 2024-05-07T02:16:58.075 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.110+0000 7fac5e000700 1 --2- 172.21.15.19:0/1197776413 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7fac4403e580 0x7fac44040a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:16:58.075 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.111+0000 7fac5e000700 1 --2- 172.21.15.19:0/1197776413 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7fac4403e580 0x7fac44040a90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fac4c006fd0 tx=0x7fac4c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:16:58.075 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.117+0000 7fac57400700 1 -- 172.21.15.19:0/1197776413 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fac50016260 con 0x7fac6010bd30 2024-05-07T02:16:58.075 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.362+0000 7fac65c00700 1 -- 172.21.15.19:0/1197776413 --> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fac60109890 con 0x7fac4403e580 2024-05-07T02:16:58.075 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.376+0000 7fac57400700 1 -- 172.21.15.19:0/1197776413 <== mgr.14120 v2:172.21.15.19:6800/174304059 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fac60109890 con 0x7fac4403e580 2024-05-07T02:16:58.075 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.379+0000 7fac65c00700 1 -- 172.21.15.19:0/1197776413 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7fac4403e580 msgr2=0x7fac44040a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:16:58.075 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.379+0000 7fac65c00700 1 --2- 172.21.15.19:0/1197776413 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7fac4403e580 0x7fac44040a90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fac4c006fd0 tx=0x7fac4c006e40 comp rx=0 tx=0).stop 2024-05-07T02:16:58.075 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.379+0000 7fac65c00700 1 -- 172.21.15.19:0/1197776413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac6010bd30 msgr2=0x7fac601a0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:16:58.075 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.379+0000 7fac65c00700 1 --2- 172.21.15.19:0/1197776413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac6010bd30 0x7fac601a0040 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fac500090e0 tx=0x7fac50011940 comp rx=0 tx=0).stop 2024-05-07T02:16:58.075 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.380+0000 7fac65c00700 1 -- 172.21.15.19:0/1197776413 shutdown_connections 2024-05-07T02:16:58.075 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.380+0000 7fac65c00700 1 --2- 172.21.15.19:0/1197776413 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7fac4403e580 0x7fac44040a90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:16:58.075 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.380+0000 7fac65c00700 1 --2- 172.21.15.19:0/1197776413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac6010bd30 0x7fac601a0040 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:16:58.076 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.380+0000 7fac65c00700 1 -- 172.21.15.19:0/1197776413 >> 172.21.15.19:0/1197776413 conn(0x7fac6007f2a0 msgr2=0x7fac60109160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:16:58.076 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.380+0000 7fac65c00700 1 -- 172.21.15.19:0/1197776413 shutdown_connections 2024-05-07T02:16:58.076 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:57.380+0000 7fac65c00700 1 -- 172.21.15.19:0/1197776413 wait complete. 2024-05-07T02:16:58.266 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:57 smithi019 ceph-mon[29781]: [07/May/2024:02:16:56] ENGINE Bus STARTING 2024-05-07T02:16:58.266 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:57 smithi019 ceph-mon[29781]: [07/May/2024:02:16:56] ENGINE Serving on http://172.21.15.19:8765 2024-05-07T02:16:58.266 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:57 smithi019 ceph-mon[29781]: [07/May/2024:02:16:56] ENGINE Serving on https://172.21.15.19:7150 2024-05-07T02:16:58.269 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:58 smithi019 ceph-mon[29781]: [07/May/2024:02:16:56] ENGINE Bus STARTED 2024-05-07T02:16:58.269 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:58 smithi019 ceph-mon[29781]: mgrmap e8: x(active, since 2s) 2024-05-07T02:16:58.269 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:58 smithi019 ceph-mon[29781]: from='mgr.14120 172.21.15.19:0/3795770355' entity='mgr.x' 2024-05-07T02:16:58.269 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:58 smithi019 ceph-mon[29781]: from='mgr.14120 172.21.15.19:0/3795770355' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:16:59.055 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:16:59 smithi019 ceph-mon[29781]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:16:59.666 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-07T02:16:59.666 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:58.798+0000 7f4c9fe00700 1 Processor -- start 2024-05-07T02:16:59.666 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:58.798+0000 7f4c9fe00700 1 -- start start 2024-05-07T02:16:59.666 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:58.799+0000 7f4c9fe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ca0109910 0x7f4ca0109d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:16:59.666 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:58.799+0000 7f4c9fe00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca010a310 con 0x7f4ca0109910 2024-05-07T02:16:59.666 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:58.799+0000 7f4c9ea00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ca0109910 0x7f4ca0109d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:16:59.667 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:58.799+0000 7f4c9ea00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ca0109910 0x7f4ca0109d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41214/0 (socket says 172.21.15.19:41214) 2024-05-07T02:16:59.667 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:58.799+0000 7f4c9ea00700 1 -- 172.21.15.19:0/2144540726 learned_addr learned my addr 172.21.15.19:0/2144540726 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:16:59.667 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:58.800+0000 7f4c9ea00700 1 -- 172.21.15.19:0/2144540726 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca010a9f0 con 0x7f4ca0109910 2024-05-07T02:16:59.667 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:58.800+0000 7f4c9ea00700 1 --2- 172.21.15.19:0/2144540726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ca0109910 0x7f4ca0109d20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4c900098a0 tx=0x7f4c90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=691eccf925cc5f0c server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:16:59.667 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:58.801+0000 7f4c9e000700 1 -- 172.21.15.19:0/2144540726 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c90004160 con 0x7f4ca0109910 2024-05-07T02:16:59.667 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:58.801+0000 7f4c9e000700 1 -- 172.21.15.19:0/2144540726 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4c90004300 con 0x7f4ca0109910 2024-05-07T02:16:59.667 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:58.801+0000 7f4c9e000700 1 -- 172.21.15.19:0/2144540726 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c90011af0 con 0x7f4ca0109910 2024-05-07T02:16:59.667 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:58.801+0000 7f4c9fe00700 1 -- 172.21.15.19:0/2144540726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ca0109910 msgr2=0x7f4ca0109d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:16:59.667 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:58.801+0000 7f4c9fe00700 1 --2- 172.21.15.19:0/2144540726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ca0109910 0x7f4ca0109d20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4c900098a0 tx=0x7f4c90009bb0 comp rx=0 tx=0).stop 2024-05-07T02:16:59.667 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:58.802+0000 7f4c9fe00700 1 -- 172.21.15.19:0/2144540726 shutdown_connections 2024-05-07T02:16:59.667 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:58.802+0000 7f4c9fe00700 1 --2- 172.21.15.19:0/2144540726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ca0109910 0x7f4ca0109d20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:16:59.667 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:58.802+0000 7f4c9fe00700 1 -- 172.21.15.19:0/2144540726 >> 172.21.15.19:0/2144540726 conn(0x7f4ca0104f70 msgr2=0x7f4ca0107400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:16:59.667 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:58.802+0000 7f4c9fe00700 1 -- 172.21.15.19:0/2144540726 shutdown_connections 2024-05-07T02:16:59.667 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:58.802+0000 7f4c9fe00700 1 -- 172.21.15.19:0/2144540726 wait complete. 2024-05-07T02:16:59.668 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:58.803+0000 7f4c9fe00700 1 Processor -- start 2024-05-07T02:16:59.668 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:58.803+0000 7f4c9fe00700 1 -- start start 2024-05-07T02:16:59.668 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:58.803+0000 7f4c9fe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ca0109910 0x7f4ca019fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:16:59.668 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:58.803+0000 7f4c9fe00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca010dda0 con 0x7f4ca0109910 2024-05-07T02:16:59.668 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:58.804+0000 7f4c9ea00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ca0109910 0x7f4ca019fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:16:59.668 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:58.804+0000 7f4c9ea00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ca0109910 0x7f4ca019fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41228/0 (socket says 172.21.15.19:41228) 2024-05-07T02:16:59.668 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:58.804+0000 7f4c9ea00700 1 -- 172.21.15.19:0/4215047633 learned_addr learned my addr 172.21.15.19:0/4215047633 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:16:59.668 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:58.804+0000 7f4c9ea00700 1 -- 172.21.15.19:0/4215047633 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca01a03a0 con 0x7f4ca0109910 2024-05-07T02:16:59.668 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:58.805+0000 7f4c9ea00700 1 --2- 172.21.15.19:0/4215047633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ca0109910 0x7f4ca019fe60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4c900090e0 tx=0x7f4c90011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:16:59.668 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:58.805+0000 7f4c97400700 1 -- 172.21.15.19:0/4215047633 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c900039e0 con 0x7f4ca0109910 2024-05-07T02:16:59.668 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:58.805+0000 7f4c97400700 1 -- 172.21.15.19:0/4215047633 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4c90003b80 con 0x7f4ca0109910 2024-05-07T02:16:59.668 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:58.805+0000 7f4c97400700 1 -- 172.21.15.19:0/4215047633 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c90024690 con 0x7f4ca0109910 2024-05-07T02:16:59.668 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:58.805+0000 7f4c9fe00700 1 -- 172.21.15.19:0/4215047633 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ca01a0630 con 0x7f4ca0109910 2024-05-07T02:16:59.668 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:58.805+0000 7f4c9fe00700 1 -- 172.21.15.19:0/4215047633 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ca01a3240 con 0x7f4ca0109910 2024-05-07T02:16:59.668 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:58.807+0000 7f4c97400700 1 -- 172.21.15.19:0/4215047633 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f4c90022020 con 0x7f4ca0109910 2024-05-07T02:16:59.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:58.807+0000 7f4c9fe00700 1 -- 172.21.15.19:0/4215047633 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ca0064e20 con 0x7f4ca0109910 2024-05-07T02:16:59.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:58.807+0000 7f4c97400700 1 --2- 172.21.15.19:0/4215047633 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f4c8403e580 0x7f4c84040a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:16:59.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:58.807+0000 7f4c97400700 1 -- 172.21.15.19:0/4215047633 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4c90053720 con 0x7f4ca0109910 2024-05-07T02:16:59.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:58.808+0000 7f4c97e00700 1 --2- 172.21.15.19:0/4215047633 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f4c8403e580 0x7f4c84040a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:16:59.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:58.808+0000 7f4c97e00700 1 --2- 172.21.15.19:0/4215047633 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f4c8403e580 0x7f4c84040a90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4c88006fd0 tx=0x7f4c88006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:16:59.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:58.813+0000 7f4c97400700 1 -- 172.21.15.19:0/4215047633 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4c90016260 con 0x7f4ca0109910 2024-05-07T02:16:59.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:59.055+0000 7f4c9fe00700 1 -- 172.21.15.19:0/4215047633 --> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f4ca007d8c0 con 0x7f4c8403e580 2024-05-07T02:16:59.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:59.057+0000 7f4c97400700 1 -- 172.21.15.19:0/4215047633 <== mgr.14120 v2:172.21.15.19:6800/174304059 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f4ca007d8c0 con 0x7f4c8403e580 2024-05-07T02:16:59.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:59.060+0000 7f4c9fe00700 1 -- 172.21.15.19:0/4215047633 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f4c8403e580 msgr2=0x7f4c84040a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:16:59.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:59.060+0000 7f4c9fe00700 1 --2- 172.21.15.19:0/4215047633 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f4c8403e580 0x7f4c84040a90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4c88006fd0 tx=0x7f4c88006e40 comp rx=0 tx=0).stop 2024-05-07T02:16:59.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:59.060+0000 7f4c9fe00700 1 -- 172.21.15.19:0/4215047633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ca0109910 msgr2=0x7f4ca019fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:16:59.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:59.060+0000 7f4c9fe00700 1 --2- 172.21.15.19:0/4215047633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ca0109910 0x7f4ca019fe60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4c900090e0 tx=0x7f4c90011940 comp rx=0 tx=0).stop 2024-05-07T02:16:59.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:59.060+0000 7f4c9fe00700 1 -- 172.21.15.19:0/4215047633 shutdown_connections 2024-05-07T02:16:59.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:59.060+0000 7f4c9fe00700 1 --2- 172.21.15.19:0/4215047633 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f4c8403e580 0x7f4c84040a90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:16:59.670 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:59.060+0000 7f4c9fe00700 1 --2- 172.21.15.19:0/4215047633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ca0109910 0x7f4ca019fe60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:16:59.670 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:59.060+0000 7f4c9fe00700 1 -- 172.21.15.19:0/4215047633 >> 172.21.15.19:0/4215047633 conn(0x7f4ca0104f70 msgr2=0x7f4ca007d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:16:59.670 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:59.060+0000 7f4c9fe00700 1 -- 172.21.15.19:0/4215047633 shutdown_connections 2024-05-07T02:16:59.670 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:16:59.060+0000 7f4c9fe00700 1 -- 172.21.15.19:0/4215047633 wait complete. 2024-05-07T02:16:59.670 INFO:teuthology.orchestra.run.smithi019.stdout:Generating ssh key... 2024-05-07T02:17:00.327 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:00 smithi019 ceph-mon[29781]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:17:01.361 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:17:00 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: Generating public/private rsa key pair. 2024-05-07T02:17:01.361 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:17:00 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: Your identification has been saved in /tmp/tmpwsx4hcb4/key. 2024-05-07T02:17:01.361 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:17:00 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: Your public key has been saved in /tmp/tmpwsx4hcb4/key.pub. 2024-05-07T02:17:01.361 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:17:00 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: The key fingerprint is: 2024-05-07T02:17:01.361 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:17:00 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: SHA256:y2Fqxp5KRCz39YBLieC6CgGKFJdVWfwmW6znXt3he+4 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968 2024-05-07T02:17:01.361 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:17:00 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: The key's randomart image is: 2024-05-07T02:17:01.361 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:17:00 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: +---[RSA 3072]----+ 2024-05-07T02:17:01.361 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:17:00 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: | ...o...+. | 2024-05-07T02:17:01.362 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:17:00 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: | .o+ . + . | 2024-05-07T02:17:01.362 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:17:00 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: |..o = + o o | 2024-05-07T02:17:01.362 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:17:00 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: |=. + o o + = | 2024-05-07T02:17:01.362 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:17:00 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: |= . o S B . | 2024-05-07T02:17:01.362 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:17:00 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: | o . . + = . o o| 2024-05-07T02:17:01.362 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:17:00 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: |o . = o o . o.| 2024-05-07T02:17:01.362 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:17:00 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: |o . + . .. o| 2024-05-07T02:17:01.362 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:17:00 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: |. ..o .. +E| 2024-05-07T02:17:01.362 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:17:00 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: +----[SHA256]-----+ 2024-05-07T02:17:01.893 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.327+0000 7f90cfa00700 1 Processor -- start 2024-05-07T02:17:01.893 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.328+0000 7f90cfa00700 1 -- start start 2024-05-07T02:17:01.893 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.328+0000 7f90cfa00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90c810bcb0 0x7f90c810c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:01.893 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.328+0000 7f90cfa00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f90c810c6b0 con 0x7f90c810bcb0 2024-05-07T02:17:01.893 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.329+0000 7f90cd000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90c810bcb0 0x7f90c810c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:01.893 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.329+0000 7f90cd000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90c810bcb0 0x7f90c810c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48762/0 (socket says 172.21.15.19:48762) 2024-05-07T02:17:01.893 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.329+0000 7f90cd000700 1 -- 172.21.15.19:0/1171046174 learned_addr learned my addr 172.21.15.19:0/1171046174 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:17:01.893 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.329+0000 7f90cd000700 1 -- 172.21.15.19:0/1171046174 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90c810cde0 con 0x7f90c810bcb0 2024-05-07T02:17:01.893 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.330+0000 7f90cd000700 1 --2- 172.21.15.19:0/1171046174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90c810bcb0 0x7f90c810c0c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f90bc0098a0 tx=0x7f90bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da951be77c6df1d server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:01.893 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.330+0000 7f90c7400700 1 -- 172.21.15.19:0/1171046174 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90bc004160 con 0x7f90c810bcb0 2024-05-07T02:17:01.894 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.330+0000 7f90c7400700 1 -- 172.21.15.19:0/1171046174 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f90bc004300 con 0x7f90c810bcb0 2024-05-07T02:17:01.894 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.330+0000 7f90c7400700 1 -- 172.21.15.19:0/1171046174 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90bc011af0 con 0x7f90c810bcb0 2024-05-07T02:17:01.894 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.331+0000 7f90cfa00700 1 -- 172.21.15.19:0/1171046174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90c810bcb0 msgr2=0x7f90c810c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:01.894 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.331+0000 7f90cfa00700 1 --2- 172.21.15.19:0/1171046174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90c810bcb0 0x7f90c810c0c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f90bc0098a0 tx=0x7f90bc009bb0 comp rx=0 tx=0).stop 2024-05-07T02:17:01.894 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.331+0000 7f90cfa00700 1 -- 172.21.15.19:0/1171046174 shutdown_connections 2024-05-07T02:17:01.894 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.331+0000 7f90cfa00700 1 --2- 172.21.15.19:0/1171046174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90c810bcb0 0x7f90c810c0c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:01.894 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.331+0000 7f90cfa00700 1 -- 172.21.15.19:0/1171046174 >> 172.21.15.19:0/1171046174 conn(0x7f90c8107490 msgr2=0x7f90c81098b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:17:01.894 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.331+0000 7f90cfa00700 1 -- 172.21.15.19:0/1171046174 shutdown_connections 2024-05-07T02:17:01.894 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.331+0000 7f90cfa00700 1 -- 172.21.15.19:0/1171046174 wait complete. 2024-05-07T02:17:01.894 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.332+0000 7f90cfa00700 1 Processor -- start 2024-05-07T02:17:01.894 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.333+0000 7f90cfa00700 1 -- start start 2024-05-07T02:17:01.894 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.333+0000 7f90cfa00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90c810bcb0 0x7f90c819ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:01.894 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.333+0000 7f90cfa00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f90c8110190 con 0x7f90c810bcb0 2024-05-07T02:17:01.895 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.333+0000 7f90cd000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90c810bcb0 0x7f90c819ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:01.895 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.333+0000 7f90cd000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90c810bcb0 0x7f90c819ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48768/0 (socket says 172.21.15.19:48768) 2024-05-07T02:17:01.895 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.333+0000 7f90cd000700 1 -- 172.21.15.19:0/3318568984 learned_addr learned my addr 172.21.15.19:0/3318568984 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:17:01.895 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.334+0000 7f90cd000700 1 -- 172.21.15.19:0/3318568984 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90c81a0520 con 0x7f90c810bcb0 2024-05-07T02:17:01.895 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.334+0000 7f90cd000700 1 --2- 172.21.15.19:0/3318568984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90c810bcb0 0x7f90c819ffe0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f90bc011890 tx=0x7f90bc011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:01.895 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.334+0000 7f90c5600700 1 -- 172.21.15.19:0/3318568984 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90bc0039e0 con 0x7f90c810bcb0 2024-05-07T02:17:01.895 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.334+0000 7f90c5600700 1 -- 172.21.15.19:0/3318568984 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f90bc003b80 con 0x7f90c810bcb0 2024-05-07T02:17:01.895 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.334+0000 7f90c5600700 1 -- 172.21.15.19:0/3318568984 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90bc0246e0 con 0x7f90c810bcb0 2024-05-07T02:17:01.895 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.334+0000 7f90cfa00700 1 -- 172.21.15.19:0/3318568984 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90c81a07b0 con 0x7f90c810bcb0 2024-05-07T02:17:01.895 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.335+0000 7f90cfa00700 1 -- 172.21.15.19:0/3318568984 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90c81a33c0 con 0x7f90c810bcb0 2024-05-07T02:17:01.895 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.336+0000 7f90cfa00700 1 -- 172.21.15.19:0/3318568984 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90c8064e20 con 0x7f90c810bcb0 2024-05-07T02:17:01.895 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.336+0000 7f90c5600700 1 -- 172.21.15.19:0/3318568984 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f90bc022020 con 0x7f90c810bcb0 2024-05-07T02:17:01.895 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.336+0000 7f90c5600700 1 --2- 172.21.15.19:0/3318568984 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f90b003e580 0x7f90b0040a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:01.895 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.336+0000 7f90c5600700 1 -- 172.21.15.19:0/3318568984 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f90bc053700 con 0x7f90c810bcb0 2024-05-07T02:17:01.896 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.337+0000 7f90c7e00700 1 --2- 172.21.15.19:0/3318568984 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f90b003e580 0x7f90b0040a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:01.896 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.337+0000 7f90c7e00700 1 --2- 172.21.15.19:0/3318568984 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f90b003e580 0x7f90b0040a90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f90b4006fd0 tx=0x7f90b4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:01.896 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.342+0000 7f90c5600700 1 -- 172.21.15.19:0/3318568984 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f90bc0212c0 con 0x7f90c810bcb0 2024-05-07T02:17:01.896 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.579+0000 7f90cfa00700 1 -- 172.21.15.19:0/3318568984 --> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f90c81092f0 con 0x7f90b003e580 2024-05-07T02:17:01.896 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.953+0000 7f90c5600700 1 -- 172.21.15.19:0/3318568984 <== mgr.14120 v2:172.21.15.19:6800/174304059 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f90c81092f0 con 0x7f90b003e580 2024-05-07T02:17:01.896 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.956+0000 7f90cfa00700 1 -- 172.21.15.19:0/3318568984 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f90b003e580 msgr2=0x7f90b0040a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:01.896 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.956+0000 7f90cfa00700 1 --2- 172.21.15.19:0/3318568984 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f90b003e580 0x7f90b0040a90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f90b4006fd0 tx=0x7f90b4006e40 comp rx=0 tx=0).stop 2024-05-07T02:17:01.896 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.956+0000 7f90cfa00700 1 -- 172.21.15.19:0/3318568984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90c810bcb0 msgr2=0x7f90c819ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:01.896 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.956+0000 7f90cfa00700 1 --2- 172.21.15.19:0/3318568984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90c810bcb0 0x7f90c819ffe0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f90bc011890 tx=0x7f90bc011970 comp rx=0 tx=0).stop 2024-05-07T02:17:01.896 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.957+0000 7f90cfa00700 1 -- 172.21.15.19:0/3318568984 shutdown_connections 2024-05-07T02:17:01.896 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.957+0000 7f90cfa00700 1 --2- 172.21.15.19:0/3318568984 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f90b003e580 0x7f90b0040a90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:01.896 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.957+0000 7f90cfa00700 1 --2- 172.21.15.19:0/3318568984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90c810bcb0 0x7f90c819ffe0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:01.896 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.957+0000 7f90cfa00700 1 -- 172.21.15.19:0/3318568984 >> 172.21.15.19:0/3318568984 conn(0x7f90c8107490 msgr2=0x7f90c8108be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:17:01.896 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.957+0000 7f90cfa00700 1 -- 172.21.15.19:0/3318568984 shutdown_connections 2024-05-07T02:17:01.897 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:00.957+0000 7f90cfa00700 1 -- 172.21.15.19:0/3318568984 wait complete. 2024-05-07T02:17:02.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:01 smithi019 ceph-mon[29781]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:17:02.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:01 smithi019 ceph-mon[29781]: Generating ssh key... 2024-05-07T02:17:02.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:01 smithi019 ceph-mon[29781]: from='mgr.14120 172.21.15.19:0/3795770355' entity='mgr.x' 2024-05-07T02:17:02.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:01 smithi019 ceph-mon[29781]: from='mgr.14120 172.21.15.19:0/3795770355' entity='mgr.x' 2024-05-07T02:17:03.500 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:03 smithi019 ceph-mon[29781]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:17:03.514 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDWaSFHRR01PItWFiaCHApyakbyvTm1iFnKo7o7gyMhFt8Iz/7y19vrY8+uZYTceXTfy5uypRm0N3v1PfKQaGosbxTtdcqdnPShx3ob3VKMkoliJmB6qUHonbR5HL0hLYnDY5U/S3GruG338biAE4zFOBgKyGkxuW/9GdqElVpDQbOFlWWxxdLRu6Ziab9zKrRba8o8a+Z/qt9PjGkl75dW0wWrMAHShNo8zhOMvjCEB6jowhzgFrHlSM6fBcBiZMKZ00PhXX6cBPiQjN7eSg5nLyamkmhGujUJt0Ddo5Wjzudm5pu0vZMww3f1+QKhDMfHroaxhEHu3radW9Z0Ts6YWFvOUq4yQVNYHbusvEiG1gYrORuiXGMD/GLXQ0oQjQy37VsswvCfGVy70iBDDSFmRzeVVhu+4rADDk5dX2mFK8MsBA1Jbz+XDydBmIP41hzuKwTjOZSVgO4eLKiwXoaPkDOqm8bxPLdil0LdUk0tyAoku0Cly6pvlo8BYcHJwA0= ceph-9ff84c16-0c17-11ef-bc97-c7b262605968 2024-05-07T02:17:03.514 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.556+0000 7f2e49400700 1 Processor -- start 2024-05-07T02:17:03.514 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.557+0000 7f2e49400700 1 -- start start 2024-05-07T02:17:03.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.557+0000 7f2e49400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e44109950 0x7f2e44109d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:03.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.557+0000 7f2e49400700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4410a350 con 0x7f2e44109950 2024-05-07T02:17:03.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.557+0000 7f2e43e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e44109950 0x7f2e44109d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:03.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.557+0000 7f2e43e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e44109950 0x7f2e44109d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48782/0 (socket says 172.21.15.19:48782) 2024-05-07T02:17:03.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.557+0000 7f2e43e00700 1 -- 172.21.15.19:0/218623699 learned_addr learned my addr 172.21.15.19:0/218623699 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:17:03.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.558+0000 7f2e43e00700 1 -- 172.21.15.19:0/218623699 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e4410aa80 con 0x7f2e44109950 2024-05-07T02:17:03.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.558+0000 7f2e43e00700 1 --2- 172.21.15.19:0/218623699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e44109950 0x7f2e44109d60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2e34009e80 tx=0x7f2e3400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ba1baa71f45c40b server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:03.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.559+0000 7f2e42a00700 1 -- 172.21.15.19:0/218623699 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e34017070 con 0x7f2e44109950 2024-05-07T02:17:03.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.559+0000 7f2e42a00700 1 -- 172.21.15.19:0/218623699 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2e3400b8c0 con 0x7f2e44109950 2024-05-07T02:17:03.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.559+0000 7f2e42a00700 1 -- 172.21.15.19:0/218623699 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e34004490 con 0x7f2e44109950 2024-05-07T02:17:03.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.559+0000 7f2e49400700 1 -- 172.21.15.19:0/218623699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e44109950 msgr2=0x7f2e44109d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:03.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.559+0000 7f2e49400700 1 --2- 172.21.15.19:0/218623699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e44109950 0x7f2e44109d60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2e34009e80 tx=0x7f2e3400b330 comp rx=0 tx=0).stop 2024-05-07T02:17:03.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.560+0000 7f2e49400700 1 -- 172.21.15.19:0/218623699 shutdown_connections 2024-05-07T02:17:03.518 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.560+0000 7f2e49400700 1 --2- 172.21.15.19:0/218623699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e44109950 0x7f2e44109d60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:03.518 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.560+0000 7f2e49400700 1 -- 172.21.15.19:0/218623699 >> 172.21.15.19:0/218623699 conn(0x7f2e44104ff0 msgr2=0x7f2e44107440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:17:03.518 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.560+0000 7f2e49400700 1 -- 172.21.15.19:0/218623699 shutdown_connections 2024-05-07T02:17:03.518 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.560+0000 7f2e49400700 1 -- 172.21.15.19:0/218623699 wait complete. 2024-05-07T02:17:03.518 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.561+0000 7f2e49400700 1 Processor -- start 2024-05-07T02:17:03.518 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.561+0000 7f2e49400700 1 -- start start 2024-05-07T02:17:03.518 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.561+0000 7f2e49400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e44109950 0x7f2e4419dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:03.518 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.561+0000 7f2e49400700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4410de30 con 0x7f2e44109950 2024-05-07T02:17:03.521 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.562+0000 7f2e43e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e44109950 0x7f2e4419dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:03.521 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.562+0000 7f2e43e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e44109950 0x7f2e4419dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48794/0 (socket says 172.21.15.19:48794) 2024-05-07T02:17:03.521 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.562+0000 7f2e43e00700 1 -- 172.21.15.19:0/2075138673 learned_addr learned my addr 172.21.15.19:0/2075138673 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:17:03.521 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.562+0000 7f2e43e00700 1 -- 172.21.15.19:0/2075138673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e4419e1c0 con 0x7f2e44109950 2024-05-07T02:17:03.522 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.562+0000 7f2e43e00700 1 --2- 172.21.15.19:0/2075138673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e44109950 0x7f2e4419dc80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2e34009fd0 tx=0x7f2e3400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:03.522 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.563+0000 7f2e40c00700 1 -- 172.21.15.19:0/2075138673 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e34020020 con 0x7f2e44109950 2024-05-07T02:17:03.522 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.563+0000 7f2e40c00700 1 -- 172.21.15.19:0/2075138673 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2e34012430 con 0x7f2e44109950 2024-05-07T02:17:03.522 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.563+0000 7f2e40c00700 1 -- 172.21.15.19:0/2075138673 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e34017040 con 0x7f2e44109950 2024-05-07T02:17:03.522 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.563+0000 7f2e49400700 1 -- 172.21.15.19:0/2075138673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e4419e450 con 0x7f2e44109950 2024-05-07T02:17:03.522 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.563+0000 7f2e49400700 1 -- 172.21.15.19:0/2075138673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e441a1060 con 0x7f2e44109950 2024-05-07T02:17:03.522 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.564+0000 7f2e49400700 1 -- 172.21.15.19:0/2075138673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e44064e20 con 0x7f2e44109950 2024-05-07T02:17:03.522 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.564+0000 7f2e40c00700 1 -- 172.21.15.19:0/2075138673 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f2e34023020 con 0x7f2e44109950 2024-05-07T02:17:03.522 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.565+0000 7f2e40c00700 1 --2- 172.21.15.19:0/2075138673 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f2e2803e580 0x7f2e28040a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:03.522 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.565+0000 7f2e40c00700 1 -- 172.21.15.19:0/2075138673 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2e3400f050 con 0x7f2e44109950 2024-05-07T02:17:03.522 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.565+0000 7f2e43400700 1 --2- 172.21.15.19:0/2075138673 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f2e2803e580 0x7f2e28040a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:03.522 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.566+0000 7f2e43400700 1 --2- 172.21.15.19:0/2075138673 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f2e2803e580 0x7f2e28040a90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2e30006fd0 tx=0x7f2e30006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:03.522 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.572+0000 7f2e40c00700 1 -- 172.21.15.19:0/2075138673 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2e3402a020 con 0x7f2e44109950 2024-05-07T02:17:03.522 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.805+0000 7f2e49400700 1 -- 172.21.15.19:0/2075138673 --> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f2e44105f10 con 0x7f2e2803e580 2024-05-07T02:17:03.523 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.807+0000 7f2e40c00700 1 -- 172.21.15.19:0/2075138673 <== mgr.14120 v2:172.21.15.19:6800/174304059 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f2e44105f10 con 0x7f2e2803e580 2024-05-07T02:17:03.523 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.810+0000 7f2e49400700 1 -- 172.21.15.19:0/2075138673 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f2e2803e580 msgr2=0x7f2e28040a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:03.523 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.810+0000 7f2e49400700 1 --2- 172.21.15.19:0/2075138673 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f2e2803e580 0x7f2e28040a90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2e30006fd0 tx=0x7f2e30006e40 comp rx=0 tx=0).stop 2024-05-07T02:17:03.523 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.810+0000 7f2e49400700 1 -- 172.21.15.19:0/2075138673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e44109950 msgr2=0x7f2e4419dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:03.523 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.810+0000 7f2e49400700 1 --2- 172.21.15.19:0/2075138673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e44109950 0x7f2e4419dc80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2e34009fd0 tx=0x7f2e3400bd50 comp rx=0 tx=0).stop 2024-05-07T02:17:03.523 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.810+0000 7f2e49400700 1 -- 172.21.15.19:0/2075138673 shutdown_connections 2024-05-07T02:17:03.523 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.810+0000 7f2e49400700 1 --2- 172.21.15.19:0/2075138673 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f2e2803e580 0x7f2e28040a90 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:03.523 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.810+0000 7f2e49400700 1 --2- 172.21.15.19:0/2075138673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e44109950 0x7f2e4419dc80 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:03.523 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.810+0000 7f2e49400700 1 -- 172.21.15.19:0/2075138673 >> 172.21.15.19:0/2075138673 conn(0x7f2e44104ff0 msgr2=0x7f2e44105800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:17:03.523 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.811+0000 7f2e49400700 1 -- 172.21.15.19:0/2075138673 shutdown_connections 2024-05-07T02:17:03.523 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:02.811+0000 7f2e49400700 1 -- 172.21.15.19:0/2075138673 wait complete. 2024-05-07T02:17:03.523 INFO:teuthology.orchestra.run.smithi019.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-07T02:17:03.523 INFO:teuthology.orchestra.run.smithi019.stdout:Adding key to root@localhost authorized_keys... 2024-05-07T02:17:03.524 INFO:teuthology.orchestra.run.smithi019.stdout:Adding host smithi019... 2024-05-07T02:17:05.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:05 smithi019 ceph-mon[29781]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi019", "addr": "172.21.15.19", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:17:06.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:06 smithi019 ceph-mon[29781]: Deploying cephadm binary to smithi019 2024-05-07T02:17:11.591 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Added host 'smithi019' with addr '172.21.15.19' 2024-05-07T02:17:11.591 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:04.147+0000 7fc425600700 1 Processor -- start 2024-05-07T02:17:11.591 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:04.147+0000 7fc425600700 1 -- start start 2024-05-07T02:17:11.591 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:04.147+0000 7fc425600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc42010bcb0 0x7fc42010c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:11.592 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:04.147+0000 7fc425600700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc42010c6b0 con 0x7fc42010bcb0 2024-05-07T02:17:11.592 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:04.148+0000 7fc41ea00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc42010bcb0 0x7fc42010c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:11.592 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:04.148+0000 7fc41ea00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc42010bcb0 0x7fc42010c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48804/0 (socket says 172.21.15.19:48804) 2024-05-07T02:17:11.592 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:04.148+0000 7fc41ea00700 1 -- 172.21.15.19:0/1525856796 learned_addr learned my addr 172.21.15.19:0/1525856796 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:17:11.592 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:04.148+0000 7fc41ea00700 1 -- 172.21.15.19:0/1525856796 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc42010cde0 con 0x7fc42010bcb0 2024-05-07T02:17:11.592 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:04.149+0000 7fc41ea00700 1 --2- 172.21.15.19:0/1525856796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc42010bcb0 0x7fc42010c0c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc4100098a0 tx=0x7fc410009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1593196902866b22 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:11.592 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:04.149+0000 7fc41e000700 1 -- 172.21.15.19:0/1525856796 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc410004160 con 0x7fc42010bcb0 2024-05-07T02:17:11.592 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:04.149+0000 7fc41e000700 1 -- 172.21.15.19:0/1525856796 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc410004300 con 0x7fc42010bcb0 2024-05-07T02:17:11.592 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:04.149+0000 7fc41e000700 1 -- 172.21.15.19:0/1525856796 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc410011af0 con 0x7fc42010bcb0 2024-05-07T02:17:11.592 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:04.150+0000 7fc425600700 1 -- 172.21.15.19:0/1525856796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc42010bcb0 msgr2=0x7fc42010c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:11.592 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:04.150+0000 7fc425600700 1 --2- 172.21.15.19:0/1525856796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc42010bcb0 0x7fc42010c0c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc4100098a0 tx=0x7fc410009bb0 comp rx=0 tx=0).stop 2024-05-07T02:17:11.592 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:04.150+0000 7fc425600700 1 -- 172.21.15.19:0/1525856796 shutdown_connections 2024-05-07T02:17:11.592 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:04.150+0000 7fc425600700 1 --2- 172.21.15.19:0/1525856796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc42010bcb0 0x7fc42010c0c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:11.592 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:04.150+0000 7fc425600700 1 -- 172.21.15.19:0/1525856796 >> 172.21.15.19:0/1525856796 conn(0x7fc420107490 msgr2=0x7fc4201098b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:17:11.593 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:04.150+0000 7fc425600700 1 -- 172.21.15.19:0/1525856796 shutdown_connections 2024-05-07T02:17:11.593 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:04.151+0000 7fc425600700 1 -- 172.21.15.19:0/1525856796 wait complete. 2024-05-07T02:17:11.593 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:04.152+0000 7fc425600700 1 Processor -- start 2024-05-07T02:17:11.593 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:04.152+0000 7fc425600700 1 -- start start 2024-05-07T02:17:11.593 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:04.152+0000 7fc425600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc42010bcb0 0x7fc42019ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:11.593 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:04.152+0000 7fc425600700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc420110190 con 0x7fc42010bcb0 2024-05-07T02:17:11.593 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:04.153+0000 7fc41ea00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc42010bcb0 0x7fc42019ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:11.593 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:04.153+0000 7fc41ea00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc42010bcb0 0x7fc42019ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48812/0 (socket says 172.21.15.19:48812) 2024-05-07T02:17:11.593 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:04.153+0000 7fc41ea00700 1 -- 172.21.15.19:0/3073970710 learned_addr learned my addr 172.21.15.19:0/3073970710 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:17:11.593 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:04.153+0000 7fc41ea00700 1 -- 172.21.15.19:0/3073970710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4201a0520 con 0x7fc42010bcb0 2024-05-07T02:17:11.593 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:04.153+0000 7fc41ea00700 1 --2- 172.21.15.19:0/3073970710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc42010bcb0 0x7fc42019ffe0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc410009fb0 tx=0x7fc410011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:11.593 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:04.154+0000 7fc417400700 1 -- 172.21.15.19:0/3073970710 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4100036a0 con 0x7fc42010bcb0 2024-05-07T02:17:11.593 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:04.154+0000 7fc417400700 1 -- 172.21.15.19:0/3073970710 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc410003cd0 con 0x7fc42010bcb0 2024-05-07T02:17:11.593 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:04.154+0000 7fc425600700 1 -- 172.21.15.19:0/3073970710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4201a07b0 con 0x7fc42010bcb0 2024-05-07T02:17:11.593 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:04.154+0000 7fc417400700 1 -- 172.21.15.19:0/3073970710 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc41001a430 con 0x7fc42010bcb0 2024-05-07T02:17:11.594 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:04.154+0000 7fc425600700 1 -- 172.21.15.19:0/3073970710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4201a33c0 con 0x7fc42010bcb0 2024-05-07T02:17:11.594 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:04.155+0000 7fc425600700 1 -- 172.21.15.19:0/3073970710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc420064e20 con 0x7fc42010bcb0 2024-05-07T02:17:11.594 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:04.156+0000 7fc417400700 1 -- 172.21.15.19:0/3073970710 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fc410003840 con 0x7fc42010bcb0 2024-05-07T02:17:11.594 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:04.156+0000 7fc417400700 1 --2- 172.21.15.19:0/3073970710 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7fc40403e4e0 0x7fc4040409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:11.594 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:04.156+0000 7fc417400700 1 -- 172.21.15.19:0/3073970710 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc410053730 con 0x7fc42010bcb0 2024-05-07T02:17:11.594 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:04.157+0000 7fc417e00700 1 --2- 172.21.15.19:0/3073970710 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7fc40403e4e0 0x7fc4040409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:11.594 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:04.157+0000 7fc417e00700 1 --2- 172.21.15.19:0/3073970710 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7fc40403e4e0 0x7fc4040409f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc408006fd0 tx=0x7fc408006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:11.594 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:04.162+0000 7fc417400700 1 -- 172.21.15.19:0/3073970710 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc410024440 con 0x7fc42010bcb0 2024-05-07T02:17:11.594 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:04.404+0000 7fc425600700 1 -- 172.21.15.19:0/3073970710 --> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi019", "addr": "172.21.15.19", "target": ["mon-mgr", ""]}) -- 0x7fc4201092f0 con 0x7fc40403e4e0 2024-05-07T02:17:11.594 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:10.734+0000 7fc417400700 1 -- 172.21.15.19:0/3073970710 <== mgr.14120 v2:172.21.15.19:6800/174304059 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fc4201092f0 con 0x7fc40403e4e0 2024-05-07T02:17:11.595 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:10.738+0000 7fc425600700 1 -- 172.21.15.19:0/3073970710 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7fc40403e4e0 msgr2=0x7fc4040409f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:11.595 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:10.738+0000 7fc425600700 1 --2- 172.21.15.19:0/3073970710 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7fc40403e4e0 0x7fc4040409f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc408006fd0 tx=0x7fc408006e40 comp rx=0 tx=0).stop 2024-05-07T02:17:11.595 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:10.738+0000 7fc425600700 1 -- 172.21.15.19:0/3073970710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc42010bcb0 msgr2=0x7fc42019ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:11.595 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:10.738+0000 7fc425600700 1 --2- 172.21.15.19:0/3073970710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc42010bcb0 0x7fc42019ffe0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc410009fb0 tx=0x7fc410011890 comp rx=0 tx=0).stop 2024-05-07T02:17:11.595 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:10.739+0000 7fc425600700 1 -- 172.21.15.19:0/3073970710 shutdown_connections 2024-05-07T02:17:11.595 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:10.739+0000 7fc425600700 1 --2- 172.21.15.19:0/3073970710 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7fc40403e4e0 0x7fc4040409f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:11.595 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:10.739+0000 7fc425600700 1 --2- 172.21.15.19:0/3073970710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc42010bcb0 0x7fc42019ffe0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:11.595 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:10.739+0000 7fc425600700 1 -- 172.21.15.19:0/3073970710 >> 172.21.15.19:0/3073970710 conn(0x7fc420107490 msgr2=0x7fc420108be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:17:11.595 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:10.739+0000 7fc425600700 1 -- 172.21.15.19:0/3073970710 shutdown_connections 2024-05-07T02:17:11.595 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:10.739+0000 7fc425600700 1 -- 172.21.15.19:0/3073970710 wait complete. 2024-05-07T02:17:11.595 INFO:teuthology.orchestra.run.smithi019.stdout:Deploying unmanaged mon service... 2024-05-07T02:17:11.867 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:11 smithi019 ceph-mon[29781]: from='mgr.14120 172.21.15.19:0/3795770355' entity='mgr.x' 2024-05-07T02:17:11.867 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:11 smithi019 ceph-mon[29781]: Added host smithi019 2024-05-07T02:17:11.868 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:11 smithi019 ceph-mon[29781]: from='mgr.14120 172.21.15.19:0/3795770355' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:17:13.567 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-07T02:17:13.567 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.341+0000 7f16bf400700 1 Processor -- start 2024-05-07T02:17:13.567 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.341+0000 7f16bf400700 1 -- start start 2024-05-07T02:17:13.567 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.341+0000 7f16bf400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16b8109b30 0x7f16b8109f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:13.567 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.341+0000 7f16bf400700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f16b810a530 con 0x7f16b8109b30 2024-05-07T02:17:13.567 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.342+0000 7f16bca00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16b8109b30 0x7f16b8109f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:13.567 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.342+0000 7f16bca00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16b8109b30 0x7f16b8109f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49912/0 (socket says 172.21.15.19:49912) 2024-05-07T02:17:13.567 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.342+0000 7f16bca00700 1 -- 172.21.15.19:0/1709106527 learned_addr learned my addr 172.21.15.19:0/1709106527 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:17:13.567 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.342+0000 7f16bca00700 1 -- 172.21.15.19:0/1709106527 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16b810ac60 con 0x7f16b8109b30 2024-05-07T02:17:13.567 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.343+0000 7f16bca00700 1 --2- 172.21.15.19:0/1709106527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16b8109b30 0x7f16b8109f40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f16ac009e80 tx=0x7f16ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7716742869f00f9b server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:13.568 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.343+0000 7f16b7400700 1 -- 172.21.15.19:0/1709106527 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16ac017070 con 0x7f16b8109b30 2024-05-07T02:17:13.568 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.343+0000 7f16b7400700 1 -- 172.21.15.19:0/1709106527 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f16ac00b8c0 con 0x7f16b8109b30 2024-05-07T02:17:13.568 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.343+0000 7f16b7400700 1 -- 172.21.15.19:0/1709106527 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16ac004490 con 0x7f16b8109b30 2024-05-07T02:17:13.568 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.344+0000 7f16bf400700 1 -- 172.21.15.19:0/1709106527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16b8109b30 msgr2=0x7f16b8109f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:13.568 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.344+0000 7f16bf400700 1 --2- 172.21.15.19:0/1709106527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16b8109b30 0x7f16b8109f40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f16ac009e80 tx=0x7f16ac00b330 comp rx=0 tx=0).stop 2024-05-07T02:17:13.568 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.344+0000 7f16bf400700 1 -- 172.21.15.19:0/1709106527 shutdown_connections 2024-05-07T02:17:13.568 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.344+0000 7f16bf400700 1 --2- 172.21.15.19:0/1709106527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16b8109b30 0x7f16b8109f40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:13.568 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.344+0000 7f16bf400700 1 -- 172.21.15.19:0/1709106527 >> 172.21.15.19:0/1709106527 conn(0x7f16b8105270 msgr2=0x7f16b81076c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:17:13.568 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.344+0000 7f16bf400700 1 -- 172.21.15.19:0/1709106527 shutdown_connections 2024-05-07T02:17:13.568 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.344+0000 7f16bf400700 1 -- 172.21.15.19:0/1709106527 wait complete. 2024-05-07T02:17:13.568 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.345+0000 7f16bf400700 1 Processor -- start 2024-05-07T02:17:13.568 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.346+0000 7f16bf400700 1 -- start start 2024-05-07T02:17:13.568 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.346+0000 7f16bf400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16b8109b30 0x7f16b819dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:13.569 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.346+0000 7f16bf400700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f16b810de90 con 0x7f16b8109b30 2024-05-07T02:17:13.569 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.346+0000 7f16bca00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16b8109b30 0x7f16b819dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:13.569 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.346+0000 7f16bca00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16b8109b30 0x7f16b819dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49926/0 (socket says 172.21.15.19:49926) 2024-05-07T02:17:13.569 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.346+0000 7f16bca00700 1 -- 172.21.15.19:0/952228687 learned_addr learned my addr 172.21.15.19:0/952228687 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:17:13.569 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.347+0000 7f16bca00700 1 -- 172.21.15.19:0/952228687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16b819e2b0 con 0x7f16b8109b30 2024-05-07T02:17:13.569 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.347+0000 7f16bca00700 1 --2- 172.21.15.19:0/952228687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16b8109b30 0x7f16b819dd70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f16ac009fd0 tx=0x7f16ac00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:13.569 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.347+0000 7f16b5600700 1 -- 172.21.15.19:0/952228687 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16ac020020 con 0x7f16b8109b30 2024-05-07T02:17:13.569 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.348+0000 7f16b5600700 1 -- 172.21.15.19:0/952228687 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f16ac012430 con 0x7f16b8109b30 2024-05-07T02:17:13.569 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.348+0000 7f16b5600700 1 -- 172.21.15.19:0/952228687 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16ac017040 con 0x7f16b8109b30 2024-05-07T02:17:13.569 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.348+0000 7f16bf400700 1 -- 172.21.15.19:0/952228687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16b819e540 con 0x7f16b8109b30 2024-05-07T02:17:13.569 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.348+0000 7f16bf400700 1 -- 172.21.15.19:0/952228687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16b81a1150 con 0x7f16b8109b30 2024-05-07T02:17:13.569 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.349+0000 7f16bf400700 1 -- 172.21.15.19:0/952228687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16b8064e20 con 0x7f16b8109b30 2024-05-07T02:17:13.569 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.349+0000 7f16b5600700 1 -- 172.21.15.19:0/952228687 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f16ac023020 con 0x7f16b8109b30 2024-05-07T02:17:13.569 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.350+0000 7f16b5600700 1 --2- 172.21.15.19:0/952228687 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f16a003e580 0x7f16a0040a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:13.570 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.350+0000 7f16b5600700 1 -- 172.21.15.19:0/952228687 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f16ac00f050 con 0x7f16b8109b30 2024-05-07T02:17:13.570 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.350+0000 7f16b7e00700 1 --2- 172.21.15.19:0/952228687 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f16a003e580 0x7f16a0040a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:13.570 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.351+0000 7f16b7e00700 1 --2- 172.21.15.19:0/952228687 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f16a003e580 0x7f16a0040a90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f16a4006fd0 tx=0x7f16a4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:13.570 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.356+0000 7f16b5600700 1 -- 172.21.15.19:0/952228687 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f16ac02a020 con 0x7f16b8109b30 2024-05-07T02:17:13.570 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.619+0000 7f16bf400700 1 -- 172.21.15.19:0/952228687 --> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f16b81071c0 con 0x7f16a003e580 2024-05-07T02:17:13.570 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.627+0000 7f16b5600700 1 -- 172.21.15.19:0/952228687 <== mgr.14120 v2:172.21.15.19:6800/174304059 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f16b81071c0 con 0x7f16a003e580 2024-05-07T02:17:13.570 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.630+0000 7f16bf400700 1 -- 172.21.15.19:0/952228687 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f16a003e580 msgr2=0x7f16a0040a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:13.570 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.630+0000 7f16bf400700 1 --2- 172.21.15.19:0/952228687 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f16a003e580 0x7f16a0040a90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f16a4006fd0 tx=0x7f16a4006e40 comp rx=0 tx=0).stop 2024-05-07T02:17:13.570 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.630+0000 7f16bf400700 1 -- 172.21.15.19:0/952228687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16b8109b30 msgr2=0x7f16b819dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:13.570 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.630+0000 7f16bf400700 1 --2- 172.21.15.19:0/952228687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16b8109b30 0x7f16b819dd70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f16ac009fd0 tx=0x7f16ac00bd50 comp rx=0 tx=0).stop 2024-05-07T02:17:13.570 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.630+0000 7f16bf400700 1 -- 172.21.15.19:0/952228687 shutdown_connections 2024-05-07T02:17:13.571 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.630+0000 7f16bf400700 1 --2- 172.21.15.19:0/952228687 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f16a003e580 0x7f16a0040a90 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:13.571 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.630+0000 7f16bf400700 1 --2- 172.21.15.19:0/952228687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16b8109b30 0x7f16b819dd70 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:13.571 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.630+0000 7f16bf400700 1 -- 172.21.15.19:0/952228687 >> 172.21.15.19:0/952228687 conn(0x7f16b8105270 msgr2=0x7f16b8106ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:17:13.571 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.630+0000 7f16bf400700 1 -- 172.21.15.19:0/952228687 shutdown_connections 2024-05-07T02:17:13.571 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:12.630+0000 7f16bf400700 1 -- 172.21.15.19:0/952228687 wait complete. 2024-05-07T02:17:13.571 INFO:teuthology.orchestra.run.smithi019.stdout:Deploying unmanaged mgr service... 2024-05-07T02:17:13.884 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:13 smithi019 ceph-mon[29781]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:17:13.885 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:13 smithi019 ceph-mon[29781]: Saving service mon spec with placement count:5 2024-05-07T02:17:13.885 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:13 smithi019 ceph-mon[29781]: from='mgr.14120 172.21.15.19:0/3795770355' entity='mgr.x' 2024-05-07T02:17:15.488 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-07T02:17:15.488 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.474+0000 7fe265a00700 1 Processor -- start 2024-05-07T02:17:15.489 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.474+0000 7fe265a00700 1 -- start start 2024-05-07T02:17:15.489 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.474+0000 7fe265a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe26007d4b0 0x7fe26010c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:15.489 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.475+0000 7fe265a00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe26010cac0 con 0x7fe26007d4b0 2024-05-07T02:17:15.489 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.475+0000 7fe25fe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe26007d4b0 0x7fe26010c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:15.489 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.475+0000 7fe25fe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe26007d4b0 0x7fe26010c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49938/0 (socket says 172.21.15.19:49938) 2024-05-07T02:17:15.491 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.475+0000 7fe25fe00700 1 -- 172.21.15.19:0/2602794091 learned_addr learned my addr 172.21.15.19:0/2602794091 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:17:15.491 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.475+0000 7fe25fe00700 1 -- 172.21.15.19:0/2602794091 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe26010cc40 con 0x7fe26007d4b0 2024-05-07T02:17:15.491 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.476+0000 7fe25fe00700 1 --2- 172.21.15.19:0/2602794091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe26007d4b0 0x7fe26010c4f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe2500098a0 tx=0x7fe250009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e62628000d95000 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:15.491 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.476+0000 7fe25ea00700 1 -- 172.21.15.19:0/2602794091 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe250004160 con 0x7fe26007d4b0 2024-05-07T02:17:15.491 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.476+0000 7fe25ea00700 1 -- 172.21.15.19:0/2602794091 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe250004300 con 0x7fe26007d4b0 2024-05-07T02:17:15.491 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.476+0000 7fe25ea00700 1 -- 172.21.15.19:0/2602794091 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe250011af0 con 0x7fe26007d4b0 2024-05-07T02:17:15.491 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.477+0000 7fe265a00700 1 -- 172.21.15.19:0/2602794091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe26007d4b0 msgr2=0x7fe26010c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:15.493 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.477+0000 7fe265a00700 1 --2- 172.21.15.19:0/2602794091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe26007d4b0 0x7fe26010c4f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe2500098a0 tx=0x7fe250009bb0 comp rx=0 tx=0).stop 2024-05-07T02:17:15.493 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.477+0000 7fe265a00700 1 -- 172.21.15.19:0/2602794091 shutdown_connections 2024-05-07T02:17:15.493 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.477+0000 7fe265a00700 1 --2- 172.21.15.19:0/2602794091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe26007d4b0 0x7fe26010c4f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:15.493 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.477+0000 7fe265a00700 1 -- 172.21.15.19:0/2602794091 >> 172.21.15.19:0/2602794091 conn(0x7fe260103ea0 msgr2=0x7fe260106330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:17:15.493 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.477+0000 7fe265a00700 1 -- 172.21.15.19:0/2602794091 shutdown_connections 2024-05-07T02:17:15.493 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.477+0000 7fe265a00700 1 -- 172.21.15.19:0/2602794091 wait complete. 2024-05-07T02:17:15.493 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.478+0000 7fe265a00700 1 Processor -- start 2024-05-07T02:17:15.493 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.479+0000 7fe265a00700 1 -- start start 2024-05-07T02:17:15.493 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.479+0000 7fe265a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe26007d4b0 0x7fe26019fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:15.493 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.479+0000 7fe265a00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe26010dcc0 con 0x7fe26007d4b0 2024-05-07T02:17:15.493 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.479+0000 7fe25fe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe26007d4b0 0x7fe26019fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:15.494 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.479+0000 7fe25fe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe26007d4b0 0x7fe26019fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49954/0 (socket says 172.21.15.19:49954) 2024-05-07T02:17:15.494 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.479+0000 7fe25fe00700 1 -- 172.21.15.19:0/3109870679 learned_addr learned my addr 172.21.15.19:0/3109870679 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:17:15.494 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.480+0000 7fe25fe00700 1 -- 172.21.15.19:0/3109870679 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2601a03a0 con 0x7fe26007d4b0 2024-05-07T02:17:15.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.480+0000 7fe25fe00700 1 --2- 172.21.15.19:0/3109870679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe26007d4b0 0x7fe26019fe60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe250009fb0 tx=0x7fe250011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:15.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.480+0000 7fe25cc00700 1 -- 172.21.15.19:0/3109870679 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe250011bc0 con 0x7fe26007d4b0 2024-05-07T02:17:15.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.480+0000 7fe25cc00700 1 -- 172.21.15.19:0/3109870679 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe250024440 con 0x7fe26007d4b0 2024-05-07T02:17:15.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.480+0000 7fe25cc00700 1 -- 172.21.15.19:0/3109870679 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe25001a5c0 con 0x7fe26007d4b0 2024-05-07T02:17:15.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.480+0000 7fe265a00700 1 -- 172.21.15.19:0/3109870679 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2601a0630 con 0x7fe26007d4b0 2024-05-07T02:17:15.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.481+0000 7fe265a00700 1 -- 172.21.15.19:0/3109870679 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2601a3240 con 0x7fe26007d4b0 2024-05-07T02:17:15.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.482+0000 7fe265a00700 1 -- 172.21.15.19:0/3109870679 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe260064e20 con 0x7fe26007d4b0 2024-05-07T02:17:15.496 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.482+0000 7fe25cc00700 1 -- 172.21.15.19:0/3109870679 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fe2500245f0 con 0x7fe26007d4b0 2024-05-07T02:17:15.497 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.483+0000 7fe25cc00700 1 --2- 172.21.15.19:0/3109870679 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7fe24403e530 0x7fe244040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:15.497 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.483+0000 7fe25cc00700 1 -- 172.21.15.19:0/3109870679 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe250053890 con 0x7fe26007d4b0 2024-05-07T02:17:15.497 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.483+0000 7fe25f400700 1 --2- 172.21.15.19:0/3109870679 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7fe24403e530 0x7fe244040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:15.497 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.484+0000 7fe25f400700 1 --2- 172.21.15.19:0/3109870679 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7fe24403e530 0x7fe244040a40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe24c006fd0 tx=0x7fe24c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:15.497 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.491+0000 7fe25cc00700 1 -- 172.21.15.19:0/3109870679 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe250017e40 con 0x7fe26007d4b0 2024-05-07T02:17:15.497 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.534+0000 7fe25cc00700 1 -- 172.21.15.19:0/3109870679 <== mon.0 v2:172.21.15.19:3300/0 7 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe250018e30 con 0x7fe26007d4b0 2024-05-07T02:17:15.497 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.765+0000 7fe265a00700 1 -- 172.21.15.19:0/3109870679 --> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fe260063c00 con 0x7fe24403e530 2024-05-07T02:17:15.497 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.773+0000 7fe25cc00700 1 -- 172.21.15.19:0/3109870679 <== mgr.14120 v2:172.21.15.19:6800/174304059 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe260063c00 con 0x7fe24403e530 2024-05-07T02:17:15.499 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.776+0000 7fe265a00700 1 -- 172.21.15.19:0/3109870679 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7fe24403e530 msgr2=0x7fe244040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:15.499 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.776+0000 7fe265a00700 1 --2- 172.21.15.19:0/3109870679 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7fe24403e530 0x7fe244040a40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe24c006fd0 tx=0x7fe24c006e40 comp rx=0 tx=0).stop 2024-05-07T02:17:15.499 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.776+0000 7fe265a00700 1 -- 172.21.15.19:0/3109870679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe26007d4b0 msgr2=0x7fe26019fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:15.499 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.776+0000 7fe265a00700 1 --2- 172.21.15.19:0/3109870679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe26007d4b0 0x7fe26019fe60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe250009fb0 tx=0x7fe250011890 comp rx=0 tx=0).stop 2024-05-07T02:17:15.499 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.777+0000 7fe265a00700 1 -- 172.21.15.19:0/3109870679 shutdown_connections 2024-05-07T02:17:15.499 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.777+0000 7fe265a00700 1 --2- 172.21.15.19:0/3109870679 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7fe24403e530 0x7fe244040a40 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:15.499 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.777+0000 7fe265a00700 1 --2- 172.21.15.19:0/3109870679 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe26007d4b0 0x7fe26019fe60 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:15.499 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.777+0000 7fe265a00700 1 -- 172.21.15.19:0/3109870679 >> 172.21.15.19:0/3109870679 conn(0x7fe260103ea0 msgr2=0x7fe260108540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:17:15.499 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.777+0000 7fe265a00700 1 -- 172.21.15.19:0/3109870679 shutdown_connections 2024-05-07T02:17:15.500 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:14.777+0000 7fe265a00700 1 -- 172.21.15.19:0/3109870679 wait complete. 2024-05-07T02:17:15.813 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:15 smithi019 ceph-mon[29781]: from='mgr.14120 172.21.15.19:0/3795770355' entity='mgr.x' 2024-05-07T02:17:15.813 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:15 smithi019 ceph-mon[29781]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:17:15.814 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:15 smithi019 ceph-mon[29781]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:17:15.814 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:15 smithi019 ceph-mon[29781]: Saving service mgr spec with placement count:2 2024-05-07T02:17:15.814 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:15 smithi019 ceph-mon[29781]: from='mgr.14120 172.21.15.19:0/3795770355' entity='mgr.x' 2024-05-07T02:17:16.822 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:16 smithi019 ceph-mon[29781]: from='mgr.14120 172.21.15.19:0/3795770355' entity='mgr.x' 2024-05-07T02:17:16.823 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:16 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/3236249213' entity='client.admin' 2024-05-07T02:17:17.511 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.179+0000 7f393ba00700 1 Processor -- start 2024-05-07T02:17:17.511 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.179+0000 7f393ba00700 1 -- start start 2024-05-07T02:17:17.511 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.180+0000 7f393ba00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f393407f5f0 0x7f393407dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:17.511 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.180+0000 7f393ba00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f393407e000 con 0x7f393407f5f0 2024-05-07T02:17:17.511 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.180+0000 7f3939000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f393407f5f0 0x7f393407dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:17.511 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.180+0000 7f3939000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f393407f5f0 0x7f393407dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49970/0 (socket says 172.21.15.19:49970) 2024-05-07T02:17:17.511 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.180+0000 7f3939000700 1 -- 172.21.15.19:0/1321891468 learned_addr learned my addr 172.21.15.19:0/1321891468 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:17:17.511 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.181+0000 7f3939000700 1 -- 172.21.15.19:0/1321891468 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f393407e180 con 0x7f393407f5f0 2024-05-07T02:17:17.511 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.181+0000 7f3939000700 1 --2- 172.21.15.19:0/1321891468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f393407f5f0 0x7f393407dac0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f39280098a0 tx=0x7f3928009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f3febb3be179fbd server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:17.511 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.181+0000 7f3933400700 1 -- 172.21.15.19:0/1321891468 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3928004160 con 0x7f393407f5f0 2024-05-07T02:17:17.516 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.181+0000 7f3933400700 1 -- 172.21.15.19:0/1321891468 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3928004300 con 0x7f393407f5f0 2024-05-07T02:17:17.516 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.181+0000 7f3933400700 1 -- 172.21.15.19:0/1321891468 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3928011a20 con 0x7f393407f5f0 2024-05-07T02:17:17.516 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.182+0000 7f393ba00700 1 -- 172.21.15.19:0/1321891468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f393407f5f0 msgr2=0x7f393407dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:17.516 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.182+0000 7f393ba00700 1 --2- 172.21.15.19:0/1321891468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f393407f5f0 0x7f393407dac0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f39280098a0 tx=0x7f3928009bb0 comp rx=0 tx=0).stop 2024-05-07T02:17:17.516 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.183+0000 7f393ba00700 1 -- 172.21.15.19:0/1321891468 shutdown_connections 2024-05-07T02:17:17.516 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.183+0000 7f393ba00700 1 --2- 172.21.15.19:0/1321891468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f393407f5f0 0x7f393407dac0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:17.516 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.183+0000 7f393ba00700 1 -- 172.21.15.19:0/1321891468 >> 172.21.15.19:0/1321891468 conn(0x7f39341051d0 msgr2=0x7f3934107620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:17:17.516 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.183+0000 7f393ba00700 1 -- 172.21.15.19:0/1321891468 shutdown_connections 2024-05-07T02:17:17.516 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.183+0000 7f393ba00700 1 -- 172.21.15.19:0/1321891468 wait complete. 2024-05-07T02:17:17.516 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.184+0000 7f393ba00700 1 Processor -- start 2024-05-07T02:17:17.516 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.184+0000 7f393ba00700 1 -- start start 2024-05-07T02:17:17.516 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.184+0000 7f393ba00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f393407f5f0 0x7f393419ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:17.516 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.185+0000 7f393ba00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3934110050 con 0x7f393407f5f0 2024-05-07T02:17:17.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.185+0000 7f3939000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f393407f5f0 0x7f393419ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:17.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.185+0000 7f3939000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f393407f5f0 0x7f393419ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49978/0 (socket says 172.21.15.19:49978) 2024-05-07T02:17:17.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.185+0000 7f3939000700 1 -- 172.21.15.19:0/3236249213 learned_addr learned my addr 172.21.15.19:0/3236249213 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:17:17.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.185+0000 7f3939000700 1 -- 172.21.15.19:0/3236249213 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39341a0460 con 0x7f393407f5f0 2024-05-07T02:17:17.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.186+0000 7f3939000700 1 --2- 172.21.15.19:0/3236249213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f393407f5f0 0x7f393419ff20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f39280119f0 tx=0x7f3928011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:17.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.186+0000 7f3931600700 1 -- 172.21.15.19:0/3236249213 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39280039e0 con 0x7f393407f5f0 2024-05-07T02:17:17.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.186+0000 7f3931600700 1 -- 172.21.15.19:0/3236249213 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f392801b8f0 con 0x7f393407f5f0 2024-05-07T02:17:17.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.186+0000 7f393ba00700 1 -- 172.21.15.19:0/3236249213 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39341a06f0 con 0x7f393407f5f0 2024-05-07T02:17:17.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.186+0000 7f393ba00700 1 -- 172.21.15.19:0/3236249213 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39341a3300 con 0x7f393407f5f0 2024-05-07T02:17:17.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.186+0000 7f3931600700 1 -- 172.21.15.19:0/3236249213 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39280237a0 con 0x7f393407f5f0 2024-05-07T02:17:17.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.188+0000 7f393ba00700 1 -- 172.21.15.19:0/3236249213 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3934064e20 con 0x7f393407f5f0 2024-05-07T02:17:17.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.188+0000 7f3931600700 1 -- 172.21.15.19:0/3236249213 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f3928023a70 con 0x7f393407f5f0 2024-05-07T02:17:17.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.189+0000 7f3931600700 1 --2- 172.21.15.19:0/3236249213 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f391c03e530 0x7f391c040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:17.517 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.189+0000 7f3931600700 1 -- 172.21.15.19:0/3236249213 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3928054150 con 0x7f393407f5f0 2024-05-07T02:17:17.518 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.189+0000 7f3933e00700 1 --2- 172.21.15.19:0/3236249213 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f391c03e530 0x7f391c040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:17.518 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.190+0000 7f3933e00700 1 --2- 172.21.15.19:0/3236249213 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f391c03e530 0x7f391c040a40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3920006fd0 tx=0x7f3920006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:17.518 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.195+0000 7f3931600700 1 -- 172.21.15.19:0/3236249213 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f392801f030 con 0x7f393407f5f0 2024-05-07T02:17:17.518 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.473+0000 7f393ba00700 1 -- 172.21.15.19:0/3236249213 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f39341a3900 con 0x7f393407f5f0 2024-05-07T02:17:17.518 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.478+0000 7f3931600700 1 -- 172.21.15.19:0/3236249213 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f392800b3e0 con 0x7f393407f5f0 2024-05-07T02:17:17.518 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.481+0000 7f393ba00700 1 -- 172.21.15.19:0/3236249213 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f391c03e530 msgr2=0x7f391c040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:17.518 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.481+0000 7f393ba00700 1 --2- 172.21.15.19:0/3236249213 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f391c03e530 0x7f391c040a40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3920006fd0 tx=0x7f3920006e40 comp rx=0 tx=0).stop 2024-05-07T02:17:17.518 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.481+0000 7f393ba00700 1 -- 172.21.15.19:0/3236249213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f393407f5f0 msgr2=0x7f393419ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:17.518 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.481+0000 7f393ba00700 1 --2- 172.21.15.19:0/3236249213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f393407f5f0 0x7f393419ff20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f39280119f0 tx=0x7f3928011ad0 comp rx=0 tx=0).stop 2024-05-07T02:17:17.518 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.481+0000 7f393ba00700 1 -- 172.21.15.19:0/3236249213 shutdown_connections 2024-05-07T02:17:17.518 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.481+0000 7f393ba00700 1 --2- 172.21.15.19:0/3236249213 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f391c03e530 0x7f391c040a40 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:17.518 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.482+0000 7f393ba00700 1 --2- 172.21.15.19:0/3236249213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f393407f5f0 0x7f393419ff20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:17.518 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.482+0000 7f393ba00700 1 -- 172.21.15.19:0/3236249213 >> 172.21.15.19:0/3236249213 conn(0x7f39341051d0 msgr2=0x7f39341069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:17:17.518 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.482+0000 7f393ba00700 1 -- 172.21.15.19:0/3236249213 shutdown_connections 2024-05-07T02:17:17.519 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:16.482+0000 7f393ba00700 1 -- 172.21.15.19:0/3236249213 wait complete. 2024-05-07T02:17:17.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:17 smithi019 ceph-mon[29781]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:17:19.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:19 smithi019 ceph-mon[29781]: from='mgr.14120 172.21.15.19:0/3795770355' entity='mgr.x' 2024-05-07T02:17:19.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:19 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/656326290' entity='client.admin' 2024-05-07T02:17:19.505 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.505+0000 7f4e49800700 1 Processor -- start 2024-05-07T02:17:19.506 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.505+0000 7f4e49800700 1 -- start start 2024-05-07T02:17:19.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.506+0000 7f4e49800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e4407cc80 0x7f4e4410c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:19.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.506+0000 7f4e49800700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4410cbf0 con 0x7f4e4407cc80 2024-05-07T02:17:19.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.506+0000 7f4e42a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e4407cc80 0x7f4e4410c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:19.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.506+0000 7f4e42a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e4407cc80 0x7f4e4410c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49988/0 (socket says 172.21.15.19:49988) 2024-05-07T02:17:19.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.506+0000 7f4e42a00700 1 -- 172.21.15.19:0/3276439731 learned_addr learned my addr 172.21.15.19:0/3276439731 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:17:19.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.507+0000 7f4e42a00700 1 -- 172.21.15.19:0/3276439731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e4410cd70 con 0x7f4e4407cc80 2024-05-07T02:17:19.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.507+0000 7f4e42a00700 1 --2- 172.21.15.19:0/3276439731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e4407cc80 0x7f4e4410c600 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4e340098a0 tx=0x7f4e34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d34f8f464b53e71f server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:19.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.507+0000 7f4e41600700 1 -- 172.21.15.19:0/3276439731 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e34004160 con 0x7f4e4407cc80 2024-05-07T02:17:19.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.507+0000 7f4e41600700 1 -- 172.21.15.19:0/3276439731 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e34004300 con 0x7f4e4407cc80 2024-05-07T02:17:19.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.507+0000 7f4e41600700 1 -- 172.21.15.19:0/3276439731 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e34011a20 con 0x7f4e4407cc80 2024-05-07T02:17:19.507 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.508+0000 7f4e49800700 1 -- 172.21.15.19:0/3276439731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e4407cc80 msgr2=0x7f4e4410c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:19.508 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.508+0000 7f4e49800700 1 --2- 172.21.15.19:0/3276439731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e4407cc80 0x7f4e4410c600 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4e340098a0 tx=0x7f4e34009bb0 comp rx=0 tx=0).stop 2024-05-07T02:17:19.512 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.508+0000 7f4e49800700 1 -- 172.21.15.19:0/3276439731 shutdown_connections 2024-05-07T02:17:19.512 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.508+0000 7f4e49800700 1 --2- 172.21.15.19:0/3276439731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e4407cc80 0x7f4e4410c600 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:19.512 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.508+0000 7f4e49800700 1 -- 172.21.15.19:0/3276439731 >> 172.21.15.19:0/3276439731 conn(0x7f4e44104050 msgr2=0x7f4e441064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:17:19.512 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.508+0000 7f4e49800700 1 -- 172.21.15.19:0/3276439731 shutdown_connections 2024-05-07T02:17:19.512 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.509+0000 7f4e49800700 1 -- 172.21.15.19:0/3276439731 wait complete. 2024-05-07T02:17:19.512 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.510+0000 7f4e49800700 1 Processor -- start 2024-05-07T02:17:19.512 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.510+0000 7f4e49800700 1 -- start start 2024-05-07T02:17:19.512 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.510+0000 7f4e49800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e4407cc80 0x7f4e44109d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:19.512 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.510+0000 7f4e49800700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4410de30 con 0x7f4e4407cc80 2024-05-07T02:17:19.513 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.511+0000 7f4e42a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e4407cc80 0x7f4e44109d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:19.513 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.511+0000 7f4e42a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e4407cc80 0x7f4e44109d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49998/0 (socket says 172.21.15.19:49998) 2024-05-07T02:17:19.513 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.511+0000 7f4e42a00700 1 -- 172.21.15.19:0/656326290 learned_addr learned my addr 172.21.15.19:0/656326290 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:17:19.513 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.511+0000 7f4e42a00700 1 -- 172.21.15.19:0/656326290 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e4410a2b0 con 0x7f4e4407cc80 2024-05-07T02:17:19.513 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.511+0000 7f4e42a00700 1 --2- 172.21.15.19:0/656326290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e4407cc80 0x7f4e44109d70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4e340119f0 tx=0x7f4e34011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:19.513 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.512+0000 7f4e3b400700 1 -- 172.21.15.19:0/656326290 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e340039e0 con 0x7f4e4407cc80 2024-05-07T02:17:19.513 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.512+0000 7f4e3b400700 1 -- 172.21.15.19:0/656326290 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e3401b8f0 con 0x7f4e4407cc80 2024-05-07T02:17:19.513 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.512+0000 7f4e49800700 1 -- 172.21.15.19:0/656326290 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e44108500 con 0x7f4e4407cc80 2024-05-07T02:17:19.513 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.512+0000 7f4e3b400700 1 -- 172.21.15.19:0/656326290 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e340237a0 con 0x7f4e4407cc80 2024-05-07T02:17:19.513 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.512+0000 7f4e49800700 1 -- 172.21.15.19:0/656326290 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e441089e0 con 0x7f4e4407cc80 2024-05-07T02:17:19.514 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.513+0000 7f4e49800700 1 -- 172.21.15.19:0/656326290 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e44064e20 con 0x7f4e4407cc80 2024-05-07T02:17:19.514 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.514+0000 7f4e3b400700 1 -- 172.21.15.19:0/656326290 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f4e3401b440 con 0x7f4e4407cc80 2024-05-07T02:17:19.514 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.515+0000 7f4e3b400700 1 --2- 172.21.15.19:0/656326290 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f4e2803e530 0x7f4e28040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:19.514 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.515+0000 7f4e3b400700 1 -- 172.21.15.19:0/656326290 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4e34053e30 con 0x7f4e4407cc80 2024-05-07T02:17:19.514 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.515+0000 7f4e42000700 1 --2- 172.21.15.19:0/656326290 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f4e2803e530 0x7f4e28040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:19.514 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.516+0000 7f4e42000700 1 --2- 172.21.15.19:0/656326290 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f4e2803e530 0x7f4e28040a40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4e30006fd0 tx=0x7f4e30006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:19.514 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.521+0000 7f4e3b400700 1 -- 172.21.15.19:0/656326290 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4e3401f030 con 0x7f4e4407cc80 2024-05-07T02:17:19.514 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.762+0000 7f4e49800700 1 -- 172.21.15.19:0/656326290 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f4e44109140 con 0x7f4e4407cc80 2024-05-07T02:17:19.514 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.766+0000 7f4e3b400700 1 -- 172.21.15.19:0/656326290 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f4e3401bb50 con 0x7f4e4407cc80 2024-05-07T02:17:19.514 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.770+0000 7f4e49800700 1 -- 172.21.15.19:0/656326290 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f4e2803e530 msgr2=0x7f4e28040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:19.514 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.770+0000 7f4e49800700 1 --2- 172.21.15.19:0/656326290 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f4e2803e530 0x7f4e28040a40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4e30006fd0 tx=0x7f4e30006e40 comp rx=0 tx=0).stop 2024-05-07T02:17:19.514 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.770+0000 7f4e49800700 1 -- 172.21.15.19:0/656326290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e4407cc80 msgr2=0x7f4e44109d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:19.514 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.770+0000 7f4e49800700 1 --2- 172.21.15.19:0/656326290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e4407cc80 0x7f4e44109d70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4e340119f0 tx=0x7f4e34011ad0 comp rx=0 tx=0).stop 2024-05-07T02:17:19.515 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.770+0000 7f4e49800700 1 -- 172.21.15.19:0/656326290 shutdown_connections 2024-05-07T02:17:19.515 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.770+0000 7f4e49800700 1 --2- 172.21.15.19:0/656326290 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f4e2803e530 0x7f4e28040a40 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:19.515 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.770+0000 7f4e49800700 1 --2- 172.21.15.19:0/656326290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e4407cc80 0x7f4e44109d70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:19.515 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.770+0000 7f4e49800700 1 -- 172.21.15.19:0/656326290 >> 172.21.15.19:0/656326290 conn(0x7f4e44104050 msgr2=0x7f4e441047d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:17:19.515 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.770+0000 7f4e49800700 1 -- 172.21.15.19:0/656326290 shutdown_connections 2024-05-07T02:17:19.515 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:18.770+0000 7f4e49800700 1 -- 172.21.15.19:0/656326290 wait complete. 2024-05-07T02:17:19.515 INFO:teuthology.orchestra.run.smithi019.stdout:Enabling the dashboard module... 2024-05-07T02:17:20.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:20 smithi019 ceph-mon[29781]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:17:21.675 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:21 smithi019 ceph-mon[29781]: from='mgr.14120 172.21.15.19:0/3795770355' entity='mgr.x' 2024-05-07T02:17:21.675 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:21 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/3401355320' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-07T02:17:21.675 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:21 smithi019 ceph-mon[29781]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:17:21.675 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:17:21 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: ignoring --setuser ceph since I am not root 2024-05-07T02:17:21.675 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:17:21 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: ignoring --setgroup ceph since I am not root 2024-05-07T02:17:21.957 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:17:21 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:17:21.849+0000 7f29d8169200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-07T02:17:22.252 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:17:22 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:17:22.114+0000 7f29d8169200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-07T02:17:22.252 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:17:22 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:17:22.251+0000 7f29d8169200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-07T02:17:22.516 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:22 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/3401355320' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-07T02:17:22.516 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:22 smithi019 ceph-mon[29781]: mgrmap e9: x(active, since 26s) 2024-05-07T02:17:22.786 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:17:22 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:17:22.667+0000 7f29d8169200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-07T02:17:22.786 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:17:22 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:17:22.786+0000 7f29d8169200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-07T02:17:23.488 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:20.194+0000 7f8300200700 1 Processor -- start 2024-05-07T02:17:23.488 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:20.194+0000 7f8300200700 1 -- start start 2024-05-07T02:17:23.489 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:20.195+0000 7f8300200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82f8109ad0 0x7f82f8109ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:23.489 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:20.195+0000 7f8300200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f82f810a4d0 con 0x7f82f8109ad0 2024-05-07T02:17:23.489 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:20.195+0000 7f82fd800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82f8109ad0 0x7f82f8109ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:23.489 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:20.195+0000 7f82fd800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82f8109ad0 0x7f82f8109ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49170/0 (socket says 172.21.15.19:49170) 2024-05-07T02:17:23.489 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:20.195+0000 7f82fd800700 1 -- 172.21.15.19:0/2675318399 learned_addr learned my addr 172.21.15.19:0/2675318399 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:17:23.489 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:20.196+0000 7f82fd800700 1 -- 172.21.15.19:0/2675318399 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f810ac00 con 0x7f82f8109ad0 2024-05-07T02:17:23.489 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:20.196+0000 7f82fd800700 1 --2- 172.21.15.19:0/2675318399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82f8109ad0 0x7f82f8109ee0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f82ec009e80 tx=0x7f82ec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d836d864909dc4e server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:23.489 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:20.197+0000 7f82f7e00700 1 -- 172.21.15.19:0/2675318399 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82ec017070 con 0x7f82f8109ad0 2024-05-07T02:17:23.489 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:20.197+0000 7f82f7e00700 1 -- 172.21.15.19:0/2675318399 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82ec00b8c0 con 0x7f82f8109ad0 2024-05-07T02:17:23.489 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:20.197+0000 7f82f7e00700 1 -- 172.21.15.19:0/2675318399 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82ec004490 con 0x7f82f8109ad0 2024-05-07T02:17:23.489 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:20.198+0000 7f8300200700 1 -- 172.21.15.19:0/2675318399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82f8109ad0 msgr2=0x7f82f8109ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:23.489 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:20.198+0000 7f8300200700 1 --2- 172.21.15.19:0/2675318399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82f8109ad0 0x7f82f8109ee0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f82ec009e80 tx=0x7f82ec00b330 comp rx=0 tx=0).stop 2024-05-07T02:17:23.489 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:20.198+0000 7f8300200700 1 -- 172.21.15.19:0/2675318399 shutdown_connections 2024-05-07T02:17:23.489 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:20.198+0000 7f8300200700 1 --2- 172.21.15.19:0/2675318399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82f8109ad0 0x7f82f8109ee0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:23.490 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:20.198+0000 7f8300200700 1 -- 172.21.15.19:0/2675318399 >> 172.21.15.19:0/2675318399 conn(0x7f82f8105150 msgr2=0x7f82f81075c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:17:23.490 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:20.198+0000 7f8300200700 1 -- 172.21.15.19:0/2675318399 shutdown_connections 2024-05-07T02:17:23.490 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:20.198+0000 7f8300200700 1 -- 172.21.15.19:0/2675318399 wait complete. 2024-05-07T02:17:23.490 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:20.199+0000 7f8300200700 1 Processor -- start 2024-05-07T02:17:23.490 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:20.199+0000 7f8300200700 1 -- start start 2024-05-07T02:17:23.490 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:20.200+0000 7f8300200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82f8109ad0 0x7f82f819fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:23.490 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:20.200+0000 7f8300200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f82f810dfb0 con 0x7f82f8109ad0 2024-05-07T02:17:23.490 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:20.200+0000 7f82fd800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82f8109ad0 0x7f82f819fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:23.490 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:20.200+0000 7f82fd800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82f8109ad0 0x7f82f819fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49176/0 (socket says 172.21.15.19:49176) 2024-05-07T02:17:23.490 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:20.200+0000 7f82fd800700 1 -- 172.21.15.19:0/3401355320 learned_addr learned my addr 172.21.15.19:0/3401355320 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:17:23.490 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:20.200+0000 7f82fd800700 1 -- 172.21.15.19:0/3401355320 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f81a0530 con 0x7f82f8109ad0 2024-05-07T02:17:23.490 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:20.201+0000 7f82fd800700 1 --2- 172.21.15.19:0/3401355320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82f8109ad0 0x7f82f819fff0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00bc80 tx=0x7f82ec00bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:23.490 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:20.201+0000 7f82f6000700 1 -- 172.21.15.19:0/3401355320 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82ec020020 con 0x7f82f8109ad0 2024-05-07T02:17:23.490 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:20.201+0000 7f82f6000700 1 -- 172.21.15.19:0/3401355320 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82ec003b50 con 0x7f82f8109ad0 2024-05-07T02:17:23.491 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:20.201+0000 7f82f6000700 1 -- 172.21.15.19:0/3401355320 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82ec017040 con 0x7f82f8109ad0 2024-05-07T02:17:23.491 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:20.201+0000 7f8300200700 1 -- 172.21.15.19:0/3401355320 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82f81a07c0 con 0x7f82f8109ad0 2024-05-07T02:17:23.491 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:20.201+0000 7f8300200700 1 -- 172.21.15.19:0/3401355320 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82f81a33d0 con 0x7f82f8109ad0 2024-05-07T02:17:23.491 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:20.203+0000 7f8300200700 1 -- 172.21.15.19:0/3401355320 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82f8064e20 con 0x7f82f8109ad0 2024-05-07T02:17:23.491 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:20.203+0000 7f82f6000700 1 -- 172.21.15.19:0/3401355320 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f82ec023020 con 0x7f82f8109ad0 2024-05-07T02:17:23.491 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:20.204+0000 7f82f6000700 1 --2- 172.21.15.19:0/3401355320 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f82e003e580 0x7f82e0040a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:23.491 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:20.204+0000 7f82f6000700 1 -- 172.21.15.19:0/3401355320 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f82ec00f050 con 0x7f82f8109ad0 2024-05-07T02:17:23.491 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:20.204+0000 7f82fce00700 1 --2- 172.21.15.19:0/3401355320 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f82e003e580 0x7f82e0040a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:23.491 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:20.205+0000 7f82fce00700 1 --2- 172.21.15.19:0/3401355320 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f82e003e580 0x7f82e0040a90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f82e4006fd0 tx=0x7f82e4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:23.491 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:20.209+0000 7f82f6000700 1 -- 172.21.15.19:0/3401355320 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f82ec02a020 con 0x7f82f8109ad0 2024-05-07T02:17:23.491 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:20.519+0000 7f8300200700 1 -- 172.21.15.19:0/3401355320 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f82f81a35a0 con 0x7f82f8109ad0 2024-05-07T02:17:23.491 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:21.426+0000 7f82f6000700 1 -- 172.21.15.19:0/3401355320 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f82ec012a00 con 0x7f82f8109ad0 2024-05-07T02:17:23.491 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:21.430+0000 7f82f6000700 1 -- 172.21.15.19:0/3401355320 <== mon.0 v2:172.21.15.19:3300/0 8 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f82ec011d60 con 0x7f82f8109ad0 2024-05-07T02:17:23.491 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:21.432+0000 7f8300200700 1 -- 172.21.15.19:0/3401355320 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f82e003e580 msgr2=0x7f82e0040a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:23.492 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:21.432+0000 7f8300200700 1 --2- 172.21.15.19:0/3401355320 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f82e003e580 0x7f82e0040a90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f82e4006fd0 tx=0x7f82e4006e40 comp rx=0 tx=0).stop 2024-05-07T02:17:23.492 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:21.432+0000 7f8300200700 1 -- 172.21.15.19:0/3401355320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82f8109ad0 msgr2=0x7f82f819fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:23.492 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:21.432+0000 7f8300200700 1 --2- 172.21.15.19:0/3401355320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82f8109ad0 0x7f82f819fff0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00bc80 tx=0x7f82ec00bd60 comp rx=0 tx=0).stop 2024-05-07T02:17:23.492 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:21.432+0000 7f8300200700 1 -- 172.21.15.19:0/3401355320 shutdown_connections 2024-05-07T02:17:23.492 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:21.432+0000 7f8300200700 1 --2- 172.21.15.19:0/3401355320 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f82e003e580 0x7f82e0040a90 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:23.492 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:21.432+0000 7f8300200700 1 --2- 172.21.15.19:0/3401355320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82f8109ad0 0x7f82f819fff0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:23.492 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:21.432+0000 7f8300200700 1 -- 172.21.15.19:0/3401355320 >> 172.21.15.19:0/3401355320 conn(0x7f82f8105150 msgr2=0x7f82f807d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:17:23.492 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:21.433+0000 7f8300200700 1 -- 172.21.15.19:0/3401355320 shutdown_connections 2024-05-07T02:17:23.492 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:21.433+0000 7f8300200700 1 -- 172.21.15.19:0/3401355320 wait complete. 2024-05-07T02:17:24.669 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:24 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/1956000478' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-07T02:17:24.669 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:17:24 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:17:24.418+0000 7f29d8169200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-07T02:17:25.125 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:17:24 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:17:24.668+0000 7f29d8169200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-07T02:17:25.125 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:17:24 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:17:24.911+0000 7f29d8169200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-07T02:17:25.133 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-05-07T02:17:25.141 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-07T02:17:25.141 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-07T02:17:25.141 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-07T02:17:25.141 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-07T02:17:25.141 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-05-07T02:17:25.141 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.142+0000 7f81d7000700 1 Processor -- start 2024-05-07T02:17:25.141 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.142+0000 7f81d7000700 1 -- start start 2024-05-07T02:17:25.141 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.143+0000 7f81d7000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81d007ed20 0x7f81d007d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:25.141 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.143+0000 7f81d7000700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f81d007d730 con 0x7f81d007ed20 2024-05-07T02:17:25.141 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.143+0000 7f81cfe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81d007ed20 0x7f81d007d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:25.141 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.143+0000 7f81cfe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81d007ed20 0x7f81d007d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49206/0 (socket says 172.21.15.19:49206) 2024-05-07T02:17:25.141 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.143+0000 7f81cfe00700 1 -- 172.21.15.19:0/3890031158 learned_addr learned my addr 172.21.15.19:0/3890031158 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:17:25.142 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.144+0000 7f81cfe00700 1 -- 172.21.15.19:0/3890031158 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81d007d8b0 con 0x7f81d007ed20 2024-05-07T02:17:25.142 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.144+0000 7f81cfe00700 1 --2- 172.21.15.19:0/3890031158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81d007ed20 0x7f81d007d1f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f81c00098a0 tx=0x7f81c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d41d68e5dd316e8d server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:25.142 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.145+0000 7f81cea00700 1 -- 172.21.15.19:0/3890031158 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81c0004160 con 0x7f81d007ed20 2024-05-07T02:17:25.142 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.145+0000 7f81cea00700 1 -- 172.21.15.19:0/3890031158 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81c0004300 con 0x7f81d007ed20 2024-05-07T02:17:25.142 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.145+0000 7f81cea00700 1 -- 172.21.15.19:0/3890031158 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81c0011a20 con 0x7f81d007ed20 2024-05-07T02:17:25.142 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.146+0000 7f81d7000700 1 -- 172.21.15.19:0/3890031158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81d007ed20 msgr2=0x7f81d007d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:25.142 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.146+0000 7f81d7000700 1 --2- 172.21.15.19:0/3890031158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81d007ed20 0x7f81d007d1f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f81c00098a0 tx=0x7f81c0009bb0 comp rx=0 tx=0).stop 2024-05-07T02:17:25.142 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.146+0000 7f81d7000700 1 -- 172.21.15.19:0/3890031158 shutdown_connections 2024-05-07T02:17:25.142 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.146+0000 7f81d7000700 1 --2- 172.21.15.19:0/3890031158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81d007ed20 0x7f81d007d1f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:25.142 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.146+0000 7f81d7000700 1 -- 172.21.15.19:0/3890031158 >> 172.21.15.19:0/3890031158 conn(0x7f81d01051a0 msgr2=0x7f81d01075f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:17:25.142 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.146+0000 7f81d7000700 1 -- 172.21.15.19:0/3890031158 shutdown_connections 2024-05-07T02:17:25.142 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.146+0000 7f81d7000700 1 -- 172.21.15.19:0/3890031158 wait complete. 2024-05-07T02:17:25.142 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.148+0000 7f81d7000700 1 Processor -- start 2024-05-07T02:17:25.143 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.148+0000 7f81d7000700 1 -- start start 2024-05-07T02:17:25.143 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.149+0000 7f81d7000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81d007ed20 0x7f81d019ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:25.143 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.149+0000 7f81d7000700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f81d0110060 con 0x7f81d007ed20 2024-05-07T02:17:25.143 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.149+0000 7f81cfe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81d007ed20 0x7f81d019ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:25.143 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.149+0000 7f81cfe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81d007ed20 0x7f81d019ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49216/0 (socket says 172.21.15.19:49216) 2024-05-07T02:17:25.143 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.149+0000 7f81cfe00700 1 -- 172.21.15.19:0/1956000478 learned_addr learned my addr 172.21.15.19:0/1956000478 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:17:25.143 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.150+0000 7f81cfe00700 1 -- 172.21.15.19:0/1956000478 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81d01a0470 con 0x7f81d007ed20 2024-05-07T02:17:25.143 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.150+0000 7f81cfe00700 1 --2- 172.21.15.19:0/1956000478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81d007ed20 0x7f81d019ff30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f81c0000c00 tx=0x7f81c0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:25.143 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.150+0000 7f81ccc00700 1 -- 172.21.15.19:0/1956000478 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81c0003aa0 con 0x7f81d007ed20 2024-05-07T02:17:25.143 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.150+0000 7f81d7000700 1 -- 172.21.15.19:0/1956000478 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81d01a0700 con 0x7f81d007ed20 2024-05-07T02:17:25.143 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.150+0000 7f81ccc00700 1 -- 172.21.15.19:0/1956000478 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81c001b8f0 con 0x7f81d007ed20 2024-05-07T02:17:25.143 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.150+0000 7f81ccc00700 1 -- 172.21.15.19:0/1956000478 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81c0023890 con 0x7f81d007ed20 2024-05-07T02:17:25.143 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.151+0000 7f81d7000700 1 -- 172.21.15.19:0/1956000478 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81d01a3310 con 0x7f81d007ed20 2024-05-07T02:17:25.143 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.152+0000 7f81d7000700 1 -- 172.21.15.19:0/1956000478 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81d0064e20 con 0x7f81d007ed20 2024-05-07T02:17:25.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.152+0000 7f81ccc00700 1 -- 172.21.15.19:0/1956000478 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f81c001b440 con 0x7f81d007ed20 2024-05-07T02:17:25.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.153+0000 7f81ccc00700 1 --2- 172.21.15.19:0/1956000478 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f81b403e630 0x7f81b4040b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:25.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.153+0000 7f81ccc00700 1 -- 172.21.15.19:0/1956000478 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f81c0053f20 con 0x7f81d007ed20 2024-05-07T02:17:25.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.153+0000 7f81cf400700 1 -- 172.21.15.19:0/1956000478 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f81b403e630 msgr2=0x7f81b4040b40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/174304059 2024-05-07T02:17:25.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.153+0000 7f81cf400700 1 --2- 172.21.15.19:0/1956000478 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f81b403e630 0x7f81b4040b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-07T02:17:25.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.159+0000 7f81ccc00700 1 -- 172.21.15.19:0/1956000478 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f81c001f030 con 0x7f81d007ed20 2024-05-07T02:17:25.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.354+0000 7f81cf400700 1 -- 172.21.15.19:0/1956000478 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f81b403e630 msgr2=0x7f81b4040b40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/174304059 2024-05-07T02:17:25.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.354+0000 7f81cf400700 1 --2- 172.21.15.19:0/1956000478 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f81b403e630 0x7f81b4040b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-07T02:17:25.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.458+0000 7f81d7000700 1 -- 172.21.15.19:0/1956000478 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f81d01a34e0 con 0x7f81d007ed20 2024-05-07T02:17:25.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.459+0000 7f81ccc00700 1 -- 172.21.15.19:0/1956000478 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f81d01a34e0 con 0x7f81d007ed20 2024-05-07T02:17:25.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.462+0000 7f81d7000700 1 -- 172.21.15.19:0/1956000478 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f81b403e630 msgr2=0x7f81b4040b40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-07T02:17:25.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.462+0000 7f81d7000700 1 --2- 172.21.15.19:0/1956000478 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f81b403e630 0x7f81b4040b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:25.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.462+0000 7f81d7000700 1 -- 172.21.15.19:0/1956000478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81d007ed20 msgr2=0x7f81d019ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:25.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.462+0000 7f81d7000700 1 --2- 172.21.15.19:0/1956000478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81d007ed20 0x7f81d019ff30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f81c0000c00 tx=0x7f81c0011aa0 comp rx=0 tx=0).stop 2024-05-07T02:17:25.145 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.462+0000 7f81d7000700 1 -- 172.21.15.19:0/1956000478 shutdown_connections 2024-05-07T02:17:25.145 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.462+0000 7f81d7000700 1 --2- 172.21.15.19:0/1956000478 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7f81b403e630 0x7f81b4040b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:25.145 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.462+0000 7f81d7000700 1 --2- 172.21.15.19:0/1956000478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81d007ed20 0x7f81d019ff30 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:25.145 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.462+0000 7f81d7000700 1 -- 172.21.15.19:0/1956000478 >> 172.21.15.19:0/1956000478 conn(0x7f81d01051a0 msgr2=0x7f81d01069b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:17:25.145 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.462+0000 7f81d7000700 1 -- 172.21.15.19:0/1956000478 shutdown_connections 2024-05-07T02:17:25.145 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:24.462+0000 7f81d7000700 1 -- 172.21.15.19:0/1956000478 wait complete. 2024-05-07T02:17:25.145 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for the mgr to restart... 2024-05-07T02:17:25.145 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mgr epoch 9... 2024-05-07T02:17:25.395 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:17:25 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:17:25.353+0000 7f29d8169200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-07T02:17:25.756 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:17:25 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:17:25.487+0000 7f29d8169200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-07T02:17:25.756 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:17:25 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:17:25.755+0000 7f29d8169200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-07T02:17:26.394 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:17:26 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:17:26.027+0000 7f29d8169200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-07T02:17:27.144 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:17:26 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:17:26.846+0000 7f29d8169200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-07T02:17:27.644 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:17:27 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:17:27.148+0000 7f29d8169200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-07T02:17:28.144 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:17:27 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:17:27.730+0000 7f29d8169200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-07T02:17:28.145 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:17:27 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:17:27.864+0000 7f29d8169200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-07T02:17:28.644 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:17:28 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:17:28.345+0000 7f29d8169200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-07T02:17:29.144 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:17:28 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:17:28.819+0000 7f29d8169200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-07T02:17:29.145 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:17:28 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:17:28.950+0000 7f29d8169200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-07T02:17:29.644 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:17:29 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:17:29.290+0000 7f29d8169200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-07T02:17:29.645 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:17:29 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:17:29.477+0000 7f29d8169200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-07T02:17:30.144 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:17:29 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:17:29.861+0000 7f29d8169200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-07T02:17:30.144 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:17:30 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:17:30.020+0000 7f29d8169200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-07T02:17:30.644 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:17:30 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:17:30.150+0000 7f29d8169200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-07T02:17:31.248 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:17:30 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:17:30.944+0000 7f29d8169200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-07T02:17:31.644 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:17:31 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:17:31.248+0000 7f29d8169200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-07T02:17:31.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:31 smithi019 ceph-mon[29781]: Active manager daemon x restarted 2024-05-07T02:17:31.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:31 smithi019 ceph-mon[29781]: Activating manager daemon x 2024-05-07T02:17:31.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:31 smithi019 ceph-mon[29781]: osdmap e3: 0 total, 0 up, 0 in 2024-05-07T02:17:31.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:31 smithi019 ceph-mon[29781]: mgrmap e10: x(active, starting, since 0.01223s) 2024-05-07T02:17:31.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:31 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-07T02:17:31.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:31 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-07T02:17:31.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:31 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-07T02:17:31.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:31 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-07T02:17:31.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:31 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-07T02:17:31.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:31 smithi019 ceph-mon[29781]: Manager daemon x is now available 2024-05-07T02:17:32.630 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:32 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:17:32.630 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:32 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-07T02:17:32.630 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:32 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-07T02:17:32.630 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:32 smithi019 ceph-mon[29781]: mgrmap e11: x(active, since 1.01707s) 2024-05-07T02:17:33.008 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-05-07T02:17:33.008 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-07T02:17:33.009 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-07T02:17:33.009 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-05-07T02:17:33.009 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:25.814+0000 7fb777000700 1 Processor -- start 2024-05-07T02:17:33.009 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:25.814+0000 7fb777000700 1 -- start start 2024-05-07T02:17:33.009 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:25.814+0000 7fb777000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb77007cc20 0x7fb77010c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:33.009 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:25.814+0000 7fb777000700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb77010cc30 con 0x7fb77007cc20 2024-05-07T02:17:33.009 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:25.815+0000 7fb76fe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb77007cc20 0x7fb77010c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:33.009 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:25.815+0000 7fb76fe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb77007cc20 0x7fb77010c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49228/0 (socket says 172.21.15.19:49228) 2024-05-07T02:17:33.009 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:25.815+0000 7fb76fe00700 1 -- 172.21.15.19:0/530439709 learned_addr learned my addr 172.21.15.19:0/530439709 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:17:33.009 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:25.815+0000 7fb76fe00700 1 -- 172.21.15.19:0/530439709 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb77010cdb0 con 0x7fb77007cc20 2024-05-07T02:17:33.009 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:25.816+0000 7fb76fe00700 1 --2- 172.21.15.19:0/530439709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb77007cc20 0x7fb77010c660 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb7600098a0 tx=0x7fb760009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c841f4b0234a28c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:33.009 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:25.816+0000 7fb76ea00700 1 -- 172.21.15.19:0/530439709 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb760004160 con 0x7fb77007cc20 2024-05-07T02:17:33.009 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:25.816+0000 7fb76ea00700 1 -- 172.21.15.19:0/530439709 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb760004300 con 0x7fb77007cc20 2024-05-07T02:17:33.010 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:25.816+0000 7fb76ea00700 1 -- 172.21.15.19:0/530439709 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb760011a20 con 0x7fb77007cc20 2024-05-07T02:17:33.010 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:25.817+0000 7fb777000700 1 -- 172.21.15.19:0/530439709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb77007cc20 msgr2=0x7fb77010c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:33.010 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:25.817+0000 7fb777000700 1 --2- 172.21.15.19:0/530439709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb77007cc20 0x7fb77010c660 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb7600098a0 tx=0x7fb760009bb0 comp rx=0 tx=0).stop 2024-05-07T02:17:33.010 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:25.817+0000 7fb777000700 1 -- 172.21.15.19:0/530439709 shutdown_connections 2024-05-07T02:17:33.010 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:25.817+0000 7fb777000700 1 --2- 172.21.15.19:0/530439709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb77007cc20 0x7fb77010c660 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:33.010 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:25.817+0000 7fb777000700 1 -- 172.21.15.19:0/530439709 >> 172.21.15.19:0/530439709 conn(0x7fb770104050 msgr2=0x7fb7701064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:17:33.010 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:25.818+0000 7fb777000700 1 -- 172.21.15.19:0/530439709 shutdown_connections 2024-05-07T02:17:33.010 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:25.818+0000 7fb777000700 1 -- 172.21.15.19:0/530439709 wait complete. 2024-05-07T02:17:33.010 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:25.819+0000 7fb777000700 1 Processor -- start 2024-05-07T02:17:33.010 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:25.819+0000 7fb777000700 1 -- start start 2024-05-07T02:17:33.010 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:25.819+0000 7fb777000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb77007cc20 0x7fb770109dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:33.010 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:25.819+0000 7fb777000700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb77010de70 con 0x7fb77007cc20 2024-05-07T02:17:33.010 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:25.820+0000 7fb76fe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb77007cc20 0x7fb770109dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:33.011 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:25.820+0000 7fb76fe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb77007cc20 0x7fb770109dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49244/0 (socket says 172.21.15.19:49244) 2024-05-07T02:17:33.011 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:25.820+0000 7fb76fe00700 1 -- 172.21.15.19:0/1386222517 learned_addr learned my addr 172.21.15.19:0/1386222517 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:17:33.011 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:25.820+0000 7fb76fe00700 1 -- 172.21.15.19:0/1386222517 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb77010a310 con 0x7fb77007cc20 2024-05-07T02:17:33.011 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:25.820+0000 7fb76fe00700 1 --2- 172.21.15.19:0/1386222517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb77007cc20 0x7fb770109dd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb7600119f0 tx=0x7fb760011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:33.011 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:25.821+0000 7fb76cc00700 1 -- 172.21.15.19:0/1386222517 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7600039e0 con 0x7fb77007cc20 2024-05-07T02:17:33.011 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:25.821+0000 7fb76cc00700 1 -- 172.21.15.19:0/1386222517 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb76001b8f0 con 0x7fb77007cc20 2024-05-07T02:17:33.011 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:25.821+0000 7fb777000700 1 -- 172.21.15.19:0/1386222517 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb770108560 con 0x7fb77007cc20 2024-05-07T02:17:33.011 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:25.821+0000 7fb76cc00700 1 -- 172.21.15.19:0/1386222517 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7600237a0 con 0x7fb77007cc20 2024-05-07T02:17:33.011 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:25.821+0000 7fb777000700 1 -- 172.21.15.19:0/1386222517 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb770108a40 con 0x7fb77007cc20 2024-05-07T02:17:33.011 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:25.823+0000 7fb76cc00700 1 -- 172.21.15.19:0/1386222517 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7fb76002c3f0 con 0x7fb77007cc20 2024-05-07T02:17:33.011 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:25.823+0000 7fb76cc00700 1 --2- 172.21.15.19:0/1386222517 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7fb75403e590 0x7fb754040aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:33.011 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:25.823+0000 7fb76cc00700 1 -- 172.21.15.19:0/1386222517 --> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb7540411b0 con 0x7fb75403e590 2024-05-07T02:17:33.011 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:25.823+0000 7fb76cc00700 1 -- 172.21.15.19:0/1386222517 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb760053f50 con 0x7fb77007cc20 2024-05-07T02:17:33.011 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:25.824+0000 7fb76f400700 1 -- 172.21.15.19:0/1386222517 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7fb75403e590 msgr2=0x7fb754040aa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/174304059 2024-05-07T02:17:33.012 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:25.824+0000 7fb76f400700 1 --2- 172.21.15.19:0/1386222517 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7fb75403e590 0x7fb754040aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-07T02:17:33.012 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:26.025+0000 7fb76f400700 1 -- 172.21.15.19:0/1386222517 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7fb75403e590 msgr2=0x7fb754040aa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/174304059 2024-05-07T02:17:33.012 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:26.025+0000 7fb76f400700 1 --2- 172.21.15.19:0/1386222517 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7fb75403e590 0x7fb754040aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-07T02:17:33.012 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:26.426+0000 7fb76f400700 1 -- 172.21.15.19:0/1386222517 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7fb75403e590 msgr2=0x7fb754040aa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/174304059 2024-05-07T02:17:33.012 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:26.426+0000 7fb76f400700 1 --2- 172.21.15.19:0/1386222517 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7fb75403e590 0x7fb754040aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-07T02:17:33.012 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:27.227+0000 7fb76f400700 1 -- 172.21.15.19:0/1386222517 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7fb75403e590 msgr2=0x7fb754040aa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/174304059 2024-05-07T02:17:33.012 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:27.227+0000 7fb76f400700 1 --2- 172.21.15.19:0/1386222517 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7fb75403e590 0x7fb754040aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-07T02:17:33.012 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:28.829+0000 7fb76f400700 1 -- 172.21.15.19:0/1386222517 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7fb75403e590 msgr2=0x7fb754040aa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/174304059 2024-05-07T02:17:33.012 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:28.829+0000 7fb76f400700 1 --2- 172.21.15.19:0/1386222517 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7fb75403e590 0x7fb754040aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-07T02:17:33.012 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:31.259+0000 7fb76cc00700 1 -- 172.21.15.19:0/1386222517 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mgrmap(e 10) ==== 50002+0+0 (secure 0 0 0) 0x7fb76002cb00 con 0x7fb77007cc20 2024-05-07T02:17:33.012 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:31.259+0000 7fb76cc00700 1 -- 172.21.15.19:0/1386222517 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7fb75403e590 msgr2=0x7fb754040aa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-07T02:17:33.012 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:31.259+0000 7fb76cc00700 1 --2- 172.21.15.19:0/1386222517 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7fb75403e590 0x7fb754040aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:33.012 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:32.265+0000 7fb76cc00700 1 -- 172.21.15.19:0/1386222517 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7fb760055ed0 con 0x7fb77007cc20 2024-05-07T02:17:33.012 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:32.265+0000 7fb76cc00700 1 --2- 172.21.15.19:0/1386222517 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fb760053ae0 0x7fb754043fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:33.013 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:32.265+0000 7fb76cc00700 1 -- 172.21.15.19:0/1386222517 --> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb76002ae50 con 0x7fb760053ae0 2024-05-07T02:17:33.013 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:32.267+0000 7fb76f400700 1 --2- 172.21.15.19:0/1386222517 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fb760053ae0 0x7fb754043fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:33.013 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:32.268+0000 7fb76f400700 1 --2- 172.21.15.19:0/1386222517 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fb760053ae0 0x7fb754043fd0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb75c003a10 tx=0x7fb75c009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:33.013 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:32.270+0000 7fb76cc00700 1 -- 172.21.15.19:0/1386222517 <== mgr.14152 v2:172.21.15.19:6800/2059564551 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fb76002ae50 con 0x7fb760053ae0 2024-05-07T02:17:33.013 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:32.283+0000 7fb777000700 1 -- 172.21.15.19:0/1386222517 --> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fb770064e20 con 0x7fb760053ae0 2024-05-07T02:17:33.013 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:32.284+0000 7fb76cc00700 1 -- 172.21.15.19:0/1386222517 <== mgr.14152 v2:172.21.15.19:6800/2059564551 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fb770064e20 con 0x7fb760053ae0 2024-05-07T02:17:33.013 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:32.285+0000 7fb777000700 1 -- 172.21.15.19:0/1386222517 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fb760053ae0 msgr2=0x7fb754043fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:33.013 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:32.285+0000 7fb777000700 1 --2- 172.21.15.19:0/1386222517 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fb760053ae0 0x7fb754043fd0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb75c003a10 tx=0x7fb75c009270 comp rx=0 tx=0).stop 2024-05-07T02:17:33.013 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:32.285+0000 7fb777000700 1 -- 172.21.15.19:0/1386222517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb77007cc20 msgr2=0x7fb770109dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:33.013 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:32.285+0000 7fb777000700 1 --2- 172.21.15.19:0/1386222517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb77007cc20 0x7fb770109dd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb7600119f0 tx=0x7fb760011ad0 comp rx=0 tx=0).stop 2024-05-07T02:17:33.013 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:32.285+0000 7fb777000700 1 -- 172.21.15.19:0/1386222517 shutdown_connections 2024-05-07T02:17:33.013 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:32.285+0000 7fb777000700 1 --2- 172.21.15.19:0/1386222517 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fb760053ae0 0x7fb754043fd0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:33.013 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:32.285+0000 7fb777000700 1 --2- 172.21.15.19:0/1386222517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb77007cc20 0x7fb770109dd0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:33.013 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:32.285+0000 7fb777000700 1 --2- 172.21.15.19:0/1386222517 >> [v2:172.21.15.19:6800/174304059,v1:172.21.15.19:6801/174304059] conn(0x7fb75403e590 0x7fb754040aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:33.014 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:32.285+0000 7fb777000700 1 -- 172.21.15.19:0/1386222517 >> 172.21.15.19:0/1386222517 conn(0x7fb770104050 msgr2=0x7fb7701047f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:17:33.014 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:32.285+0000 7fb777000700 1 -- 172.21.15.19:0/1386222517 shutdown_connections 2024-05-07T02:17:33.014 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:32.285+0000 7fb777000700 1 -- 172.21.15.19:0/1386222517 wait complete. 2024-05-07T02:17:33.014 INFO:teuthology.orchestra.run.smithi019.stdout:mgr epoch 9 is available 2024-05-07T02:17:33.014 INFO:teuthology.orchestra.run.smithi019.stdout:Generating a dashboard self-signed certificate... 2024-05-07T02:17:33.467 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:33 smithi019 ceph-mon[29781]: [07/May/2024:02:17:31] ENGINE Bus STARTING 2024-05-07T02:17:33.468 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:33 smithi019 ceph-mon[29781]: [07/May/2024:02:17:31] ENGINE Serving on http://172.21.15.19:8765 2024-05-07T02:17:33.468 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:33 smithi019 ceph-mon[29781]: [07/May/2024:02:17:32] ENGINE Serving on https://172.21.15.19:7150 2024-05-07T02:17:33.468 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:33 smithi019 ceph-mon[29781]: [07/May/2024:02:17:32] ENGINE Bus STARTED 2024-05-07T02:17:33.468 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:33 smithi019 ceph-mon[29781]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-07T02:17:33.468 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:33 smithi019 ceph-mon[29781]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-07T02:17:34.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:34 smithi019 ceph-mon[29781]: mgrmap e12: x(active, since 2s) 2024-05-07T02:17:34.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:34 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:17:35.189 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-07T02:17:35.189 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.142+0000 7f70cb000700 1 Processor -- start 2024-05-07T02:17:35.191 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.142+0000 7f70cb000700 1 -- start start 2024-05-07T02:17:35.191 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.142+0000 7f70cb000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70c407cc40 0x7f70c410c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:35.191 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.142+0000 7f70cb000700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f70c410cc10 con 0x7f70c407cc40 2024-05-07T02:17:35.191 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.143+0000 7f70c3e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70c407cc40 0x7f70c410c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:35.191 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.143+0000 7f70c3e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70c407cc40 0x7f70c410c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51158/0 (socket says 172.21.15.19:51158) 2024-05-07T02:17:35.191 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.143+0000 7f70c3e00700 1 -- 172.21.15.19:0/4045387081 learned_addr learned my addr 172.21.15.19:0/4045387081 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:17:35.191 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.144+0000 7f70c3e00700 1 -- 172.21.15.19:0/4045387081 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c410cd90 con 0x7f70c407cc40 2024-05-07T02:17:35.191 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.144+0000 7f70c3e00700 1 --2- 172.21.15.19:0/4045387081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70c407cc40 0x7f70c410c640 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f70b40098a0 tx=0x7f70b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59919f68e8ac31d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:35.191 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.144+0000 7f70c2a00700 1 -- 172.21.15.19:0/4045387081 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70b4004160 con 0x7f70c407cc40 2024-05-07T02:17:35.191 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.144+0000 7f70c2a00700 1 -- 172.21.15.19:0/4045387081 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70b4004300 con 0x7f70c407cc40 2024-05-07T02:17:35.191 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.144+0000 7f70c2a00700 1 -- 172.21.15.19:0/4045387081 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70b4011a20 con 0x7f70c407cc40 2024-05-07T02:17:35.191 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.145+0000 7f70cb000700 1 -- 172.21.15.19:0/4045387081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70c407cc40 msgr2=0x7f70c410c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:35.191 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.145+0000 7f70cb000700 1 --2- 172.21.15.19:0/4045387081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70c407cc40 0x7f70c410c640 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f70b40098a0 tx=0x7f70b4009bb0 comp rx=0 tx=0).stop 2024-05-07T02:17:35.192 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.145+0000 7f70cb000700 1 -- 172.21.15.19:0/4045387081 shutdown_connections 2024-05-07T02:17:35.195 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.145+0000 7f70cb000700 1 --2- 172.21.15.19:0/4045387081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70c407cc40 0x7f70c410c640 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:35.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.145+0000 7f70cb000700 1 -- 172.21.15.19:0/4045387081 >> 172.21.15.19:0/4045387081 conn(0x7f70c4104050 msgr2=0x7f70c41064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:17:35.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.145+0000 7f70cb000700 1 -- 172.21.15.19:0/4045387081 shutdown_connections 2024-05-07T02:17:35.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.146+0000 7f70cb000700 1 -- 172.21.15.19:0/4045387081 wait complete. 2024-05-07T02:17:35.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.147+0000 7f70cb000700 1 Processor -- start 2024-05-07T02:17:35.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.147+0000 7f70cb000700 1 -- start start 2024-05-07T02:17:35.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.148+0000 7f70cb000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70c407cc40 0x7f70c419ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:35.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.148+0000 7f70cb000700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f70c410de50 con 0x7f70c407cc40 2024-05-07T02:17:35.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.148+0000 7f70c3e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70c407cc40 0x7f70c419ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:35.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.148+0000 7f70c3e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70c407cc40 0x7f70c419ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51168/0 (socket says 172.21.15.19:51168) 2024-05-07T02:17:35.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.148+0000 7f70c3e00700 1 -- 172.21.15.19:0/877354770 learned_addr learned my addr 172.21.15.19:0/877354770 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:17:35.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.148+0000 7f70c3e00700 1 -- 172.21.15.19:0/877354770 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c41a0450 con 0x7f70c407cc40 2024-05-07T02:17:35.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.149+0000 7f70c3e00700 1 --2- 172.21.15.19:0/877354770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70c407cc40 0x7f70c419ff10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f70b4000c00 tx=0x7f70b4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:35.196 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.149+0000 7f70c0c00700 1 -- 172.21.15.19:0/877354770 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70b4003aa0 con 0x7f70c407cc40 2024-05-07T02:17:35.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.149+0000 7f70c0c00700 1 -- 172.21.15.19:0/877354770 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70b401b8f0 con 0x7f70c407cc40 2024-05-07T02:17:35.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.149+0000 7f70c0c00700 1 -- 172.21.15.19:0/877354770 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70b4023890 con 0x7f70c407cc40 2024-05-07T02:17:35.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.149+0000 7f70cb000700 1 -- 172.21.15.19:0/877354770 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70c41a06e0 con 0x7f70c407cc40 2024-05-07T02:17:35.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.149+0000 7f70cb000700 1 -- 172.21.15.19:0/877354770 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70c41a32f0 con 0x7f70c407cc40 2024-05-07T02:17:35.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.151+0000 7f70cb000700 1 -- 172.21.15.19:0/877354770 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70c4064e20 con 0x7f70c407cc40 2024-05-07T02:17:35.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.154+0000 7f70c0c00700 1 -- 172.21.15.19:0/877354770 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f70b401b440 con 0x7f70c407cc40 2024-05-07T02:17:35.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.155+0000 7f70c0c00700 1 --2- 172.21.15.19:0/877354770 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f70a803e5e0 0x7f70a8040af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:35.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.155+0000 7f70c0c00700 1 -- 172.21.15.19:0/877354770 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f70b40540c0 con 0x7f70c407cc40 2024-05-07T02:17:35.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.155+0000 7f70c3400700 1 --2- 172.21.15.19:0/877354770 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f70a803e5e0 0x7f70a8040af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:35.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.157+0000 7f70c3400700 1 --2- 172.21.15.19:0/877354770 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f70a803e5e0 0x7f70a8040af0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f70b0006fd0 tx=0x7f70b0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:35.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.162+0000 7f70c0c00700 1 -- 172.21.15.19:0/877354770 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f70b401f030 con 0x7f70c407cc40 2024-05-07T02:17:35.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.414+0000 7f70cb000700 1 -- 172.21.15.19:0/877354770 --> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f70c402e910 con 0x7f70a803e5e0 2024-05-07T02:17:35.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.591+0000 7f70c0c00700 1 -- 172.21.15.19:0/877354770 <== mgr.14152 v2:172.21.15.19:6800/2059564551 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f70c402e910 con 0x7f70a803e5e0 2024-05-07T02:17:35.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.595+0000 7f70cb000700 1 -- 172.21.15.19:0/877354770 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f70a803e5e0 msgr2=0x7f70a8040af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:35.197 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.595+0000 7f70cb000700 1 --2- 172.21.15.19:0/877354770 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f70a803e5e0 0x7f70a8040af0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f70b0006fd0 tx=0x7f70b0006e40 comp rx=0 tx=0).stop 2024-05-07T02:17:35.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.595+0000 7f70cb000700 1 -- 172.21.15.19:0/877354770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70c407cc40 msgr2=0x7f70c419ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:35.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.595+0000 7f70cb000700 1 --2- 172.21.15.19:0/877354770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70c407cc40 0x7f70c419ff10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f70b4000c00 tx=0x7f70b4011aa0 comp rx=0 tx=0).stop 2024-05-07T02:17:35.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.595+0000 7f70cb000700 1 -- 172.21.15.19:0/877354770 shutdown_connections 2024-05-07T02:17:35.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.595+0000 7f70cb000700 1 --2- 172.21.15.19:0/877354770 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f70a803e5e0 0x7f70a8040af0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:35.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.595+0000 7f70cb000700 1 --2- 172.21.15.19:0/877354770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70c407cc40 0x7f70c419ff10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:35.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.595+0000 7f70cb000700 1 -- 172.21.15.19:0/877354770 >> 172.21.15.19:0/877354770 conn(0x7f70c4104050 msgr2=0x7f70c4108610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:17:35.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.595+0000 7f70cb000700 1 -- 172.21.15.19:0/877354770 shutdown_connections 2024-05-07T02:17:35.198 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:34.596+0000 7f70cb000700 1 -- 172.21.15.19:0/877354770 wait complete. 2024-05-07T02:17:35.198 INFO:teuthology.orchestra.run.smithi019.stdout:Creating initial admin user... 2024-05-07T02:17:35.694 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:35 smithi019 ceph-mon[29781]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:17:35.694 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:35 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:17:35.694 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:35 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:17:37.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:37 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:17:37.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:37 smithi019 ceph-mon[29781]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:17:37.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:37 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:17:37.295 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$1v4onj8W6ucip9JS7Rx03uWapcbElo8Q09abO.8rLgRK3rbjswjIS", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1715048256, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-07T02:17:37.295 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:35.929+0000 7f8397200700 1 Processor -- start 2024-05-07T02:17:37.295 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:35.929+0000 7f8397200700 1 -- start start 2024-05-07T02:17:37.295 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:35.930+0000 7f8397200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f839007f7b0 0x7f839007dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:37.295 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:35.930+0000 7f8397200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f839007e1c0 con 0x7f839007f7b0 2024-05-07T02:17:37.295 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:35.930+0000 7f838fe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f839007f7b0 0x7f839007dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:37.296 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:35.930+0000 7f838fe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f839007f7b0 0x7f839007dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51176/0 (socket says 172.21.15.19:51176) 2024-05-07T02:17:37.296 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:35.930+0000 7f838fe00700 1 -- 172.21.15.19:0/1031749879 learned_addr learned my addr 172.21.15.19:0/1031749879 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:17:37.296 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:35.931+0000 7f838fe00700 1 -- 172.21.15.19:0/1031749879 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f839007e340 con 0x7f839007f7b0 2024-05-07T02:17:37.296 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:35.931+0000 7f838fe00700 1 --2- 172.21.15.19:0/1031749879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f839007f7b0 0x7f839007dc80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f83800098a0 tx=0x7f8380009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c0fdc057a2d2aff server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:37.296 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:35.931+0000 7f838ea00700 1 -- 172.21.15.19:0/1031749879 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8380004160 con 0x7f839007f7b0 2024-05-07T02:17:37.296 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:35.931+0000 7f838ea00700 1 -- 172.21.15.19:0/1031749879 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8380004300 con 0x7f839007f7b0 2024-05-07T02:17:37.296 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:35.932+0000 7f838ea00700 1 -- 172.21.15.19:0/1031749879 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8380011a20 con 0x7f839007f7b0 2024-05-07T02:17:37.296 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:35.932+0000 7f8397200700 1 -- 172.21.15.19:0/1031749879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f839007f7b0 msgr2=0x7f839007dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:37.296 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:35.932+0000 7f8397200700 1 --2- 172.21.15.19:0/1031749879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f839007f7b0 0x7f839007dc80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f83800098a0 tx=0x7f8380009bb0 comp rx=0 tx=0).stop 2024-05-07T02:17:37.296 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:35.932+0000 7f8397200700 1 -- 172.21.15.19:0/1031749879 shutdown_connections 2024-05-07T02:17:37.296 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:35.933+0000 7f8397200700 1 --2- 172.21.15.19:0/1031749879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f839007f7b0 0x7f839007dc80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:37.296 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:35.933+0000 7f8397200700 1 -- 172.21.15.19:0/1031749879 >> 172.21.15.19:0/1031749879 conn(0x7f8390105200 msgr2=0x7f8390107650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:17:37.297 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:35.933+0000 7f8397200700 1 -- 172.21.15.19:0/1031749879 shutdown_connections 2024-05-07T02:17:37.297 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:35.933+0000 7f8397200700 1 -- 172.21.15.19:0/1031749879 wait complete. 2024-05-07T02:17:37.297 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:35.934+0000 7f8397200700 1 Processor -- start 2024-05-07T02:17:37.297 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:35.934+0000 7f8397200700 1 -- start start 2024-05-07T02:17:37.297 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:35.934+0000 7f8397200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f839007f7b0 0x7f839019ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:37.297 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:35.934+0000 7f8397200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f83901100c0 con 0x7f839007f7b0 2024-05-07T02:17:37.297 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:35.935+0000 7f838fe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f839007f7b0 0x7f839019ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:37.297 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:35.935+0000 7f838fe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f839007f7b0 0x7f839019ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51186/0 (socket says 172.21.15.19:51186) 2024-05-07T02:17:37.297 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:35.935+0000 7f838fe00700 1 -- 172.21.15.19:0/4227224788 learned_addr learned my addr 172.21.15.19:0/4227224788 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:17:37.297 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:35.935+0000 7f838fe00700 1 -- 172.21.15.19:0/4227224788 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83901a04a0 con 0x7f839007f7b0 2024-05-07T02:17:37.297 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:35.936+0000 7f838fe00700 1 --2- 172.21.15.19:0/4227224788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f839007f7b0 0x7f839019ff60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8380009520 tx=0x7f8380011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:37.298 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:35.936+0000 7f838cc00700 1 -- 172.21.15.19:0/4227224788 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83800039e0 con 0x7f839007f7b0 2024-05-07T02:17:37.298 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:35.936+0000 7f838cc00700 1 -- 172.21.15.19:0/4227224788 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f838001b8f0 con 0x7f839007f7b0 2024-05-07T02:17:37.298 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:35.936+0000 7f838cc00700 1 -- 172.21.15.19:0/4227224788 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83800197d0 con 0x7f839007f7b0 2024-05-07T02:17:37.298 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:35.937+0000 7f8397200700 1 -- 172.21.15.19:0/4227224788 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83901a0730 con 0x7f839007f7b0 2024-05-07T02:17:37.298 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:35.937+0000 7f8397200700 1 -- 172.21.15.19:0/4227224788 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83901a3340 con 0x7f839007f7b0 2024-05-07T02:17:37.298 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:35.938+0000 7f8397200700 1 -- 172.21.15.19:0/4227224788 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8390064e20 con 0x7f839007f7b0 2024-05-07T02:17:37.298 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:35.938+0000 7f838cc00700 1 -- 172.21.15.19:0/4227224788 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f838001b440 con 0x7f839007f7b0 2024-05-07T02:17:37.298 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:35.939+0000 7f838cc00700 1 --2- 172.21.15.19:0/4227224788 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f837403e580 0x7f8374040a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:37.298 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:35.939+0000 7f838cc00700 1 -- 172.21.15.19:0/4227224788 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8380054110 con 0x7f839007f7b0 2024-05-07T02:17:37.298 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:35.939+0000 7f838f400700 1 --2- 172.21.15.19:0/4227224788 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f837403e580 0x7f8374040a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:37.298 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:35.940+0000 7f838f400700 1 --2- 172.21.15.19:0/4227224788 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f837403e580 0x7f8374040a90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f837c006fd0 tx=0x7f837c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:37.298 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:35.945+0000 7f838cc00700 1 -- 172.21.15.19:0/4227224788 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f838001f030 con 0x7f839007f7b0 2024-05-07T02:17:37.298 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:36.202+0000 7f8397200700 1 -- 172.21.15.19:0/4227224788 --> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f83901a35d0 con 0x7f837403e580 2024-05-07T02:17:37.299 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:36.502+0000 7f838cc00700 1 -- 172.21.15.19:0/4227224788 <== mgr.14152 v2:172.21.15.19:6800/2059564551 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f83901a35d0 con 0x7f837403e580 2024-05-07T02:17:37.299 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:36.506+0000 7f8397200700 1 -- 172.21.15.19:0/4227224788 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f837403e580 msgr2=0x7f8374040a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:37.299 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:36.506+0000 7f8397200700 1 --2- 172.21.15.19:0/4227224788 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f837403e580 0x7f8374040a90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f837c006fd0 tx=0x7f837c006e40 comp rx=0 tx=0).stop 2024-05-07T02:17:37.299 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:36.506+0000 7f8397200700 1 -- 172.21.15.19:0/4227224788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f839007f7b0 msgr2=0x7f839019ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:37.299 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:36.506+0000 7f8397200700 1 --2- 172.21.15.19:0/4227224788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f839007f7b0 0x7f839019ff60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8380009520 tx=0x7f8380011aa0 comp rx=0 tx=0).stop 2024-05-07T02:17:37.299 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:36.506+0000 7f8397200700 1 -- 172.21.15.19:0/4227224788 shutdown_connections 2024-05-07T02:17:37.299 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:36.506+0000 7f8397200700 1 --2- 172.21.15.19:0/4227224788 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f837403e580 0x7f8374040a90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:37.299 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:36.506+0000 7f8397200700 1 --2- 172.21.15.19:0/4227224788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f839007f7b0 0x7f839019ff60 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:37.299 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:36.506+0000 7f8397200700 1 -- 172.21.15.19:0/4227224788 >> 172.21.15.19:0/4227224788 conn(0x7f8390105200 msgr2=0x7f8390106a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:17:37.299 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:36.507+0000 7f8397200700 1 -- 172.21.15.19:0/4227224788 shutdown_connections 2024-05-07T02:17:37.299 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:36.507+0000 7f8397200700 1 -- 172.21.15.19:0/4227224788 wait complete. 2024-05-07T02:17:37.299 INFO:teuthology.orchestra.run.smithi019.stdout:Fetching dashboard port number... 2024-05-07T02:17:39.265 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:39 smithi019 ceph-mon[29781]: mgrmap e13: x(active, since 6s) 2024-05-07T02:17:39.266 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:39 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/1117254233' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-07T02:17:39.692 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 8443 2024-05-07T02:17:39.698 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:37.951+0000 7f9810a00700 1 Processor -- start 2024-05-07T02:17:39.698 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:37.951+0000 7f9810a00700 1 -- start start 2024-05-07T02:17:39.698 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:37.951+0000 7f9810a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f980c10bcb0 0x7f980c10c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:39.699 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:37.951+0000 7f9810a00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f980c10c6b0 con 0x7f980c10bcb0 2024-05-07T02:17:39.699 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:37.952+0000 7f9809e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f980c10bcb0 0x7f980c10c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:39.699 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:37.952+0000 7f9809e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f980c10bcb0 0x7f980c10c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51190/0 (socket says 172.21.15.19:51190) 2024-05-07T02:17:39.699 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:37.952+0000 7f9809e00700 1 -- 172.21.15.19:0/2860724691 learned_addr learned my addr 172.21.15.19:0/2860724691 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:17:39.699 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:37.952+0000 7f9809e00700 1 -- 172.21.15.19:0/2860724691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f980c10cde0 con 0x7f980c10bcb0 2024-05-07T02:17:39.699 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:37.953+0000 7f9809e00700 1 --2- 172.21.15.19:0/2860724691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f980c10bcb0 0x7f980c10c0c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f97fc0098a0 tx=0x7f97fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f853efcebbfe0fb2 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:39.699 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:37.953+0000 7f9808a00700 1 -- 172.21.15.19:0/2860724691 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97fc004160 con 0x7f980c10bcb0 2024-05-07T02:17:39.699 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:37.953+0000 7f9808a00700 1 -- 172.21.15.19:0/2860724691 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97fc004300 con 0x7f980c10bcb0 2024-05-07T02:17:39.699 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:37.953+0000 7f9808a00700 1 -- 172.21.15.19:0/2860724691 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97fc011a20 con 0x7f980c10bcb0 2024-05-07T02:17:39.699 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:37.954+0000 7f9810a00700 1 -- 172.21.15.19:0/2860724691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f980c10bcb0 msgr2=0x7f980c10c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:39.700 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:37.954+0000 7f9810a00700 1 --2- 172.21.15.19:0/2860724691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f980c10bcb0 0x7f980c10c0c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f97fc0098a0 tx=0x7f97fc009bb0 comp rx=0 tx=0).stop 2024-05-07T02:17:39.700 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:37.954+0000 7f9810a00700 1 -- 172.21.15.19:0/2860724691 shutdown_connections 2024-05-07T02:17:39.700 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:37.954+0000 7f9810a00700 1 --2- 172.21.15.19:0/2860724691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f980c10bcb0 0x7f980c10c0c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:39.700 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:37.954+0000 7f9810a00700 1 -- 172.21.15.19:0/2860724691 >> 172.21.15.19:0/2860724691 conn(0x7f980c107490 msgr2=0x7f980c1098b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:17:39.700 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:37.954+0000 7f9810a00700 1 -- 172.21.15.19:0/2860724691 shutdown_connections 2024-05-07T02:17:39.700 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:37.954+0000 7f9810a00700 1 -- 172.21.15.19:0/2860724691 wait complete. 2024-05-07T02:17:39.700 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:37.955+0000 7f9810a00700 1 Processor -- start 2024-05-07T02:17:39.700 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:37.955+0000 7f9810a00700 1 -- start start 2024-05-07T02:17:39.700 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:37.956+0000 7f9810a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f980c10bcb0 0x7f980c19ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:39.700 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:37.956+0000 7f9810a00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f980c110190 con 0x7f980c10bcb0 2024-05-07T02:17:39.700 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:37.956+0000 7f9809e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f980c10bcb0 0x7f980c19ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:39.701 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:37.956+0000 7f9809e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f980c10bcb0 0x7f980c19ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51196/0 (socket says 172.21.15.19:51196) 2024-05-07T02:17:39.701 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:37.956+0000 7f9809e00700 1 -- 172.21.15.19:0/1117254233 learned_addr learned my addr 172.21.15.19:0/1117254233 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:17:39.701 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:37.956+0000 7f9809e00700 1 -- 172.21.15.19:0/1117254233 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f980c1a04a0 con 0x7f980c10bcb0 2024-05-07T02:17:39.701 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:37.957+0000 7f9809e00700 1 --2- 172.21.15.19:0/1117254233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f980c10bcb0 0x7f980c19ff60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f97fc003e70 tx=0x7f97fc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:39.701 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:37.957+0000 7f9802a00700 1 -- 172.21.15.19:0/1117254233 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97fc003aa0 con 0x7f980c10bcb0 2024-05-07T02:17:39.701 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:37.957+0000 7f9802a00700 1 -- 172.21.15.19:0/1117254233 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97fc01a8e0 con 0x7f980c10bcb0 2024-05-07T02:17:39.701 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:37.957+0000 7f9810a00700 1 -- 172.21.15.19:0/1117254233 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f980c1a0730 con 0x7f980c10bcb0 2024-05-07T02:17:39.701 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:37.957+0000 7f9802a00700 1 -- 172.21.15.19:0/1117254233 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97fc024870 con 0x7f980c10bcb0 2024-05-07T02:17:39.701 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:37.957+0000 7f9810a00700 1 -- 172.21.15.19:0/1117254233 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f980c1a3340 con 0x7f980c10bcb0 2024-05-07T02:17:39.701 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:37.958+0000 7f9802a00700 1 -- 172.21.15.19:0/1117254233 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f97fc022020 con 0x7f980c10bcb0 2024-05-07T02:17:39.701 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:37.958+0000 7f9810a00700 1 -- 172.21.15.19:0/1117254233 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97ec005360 con 0x7f980c10bcb0 2024-05-07T02:17:39.702 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:37.959+0000 7f9802a00700 1 --2- 172.21.15.19:0/1117254233 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f97f003e580 0x7f97f0040a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:39.702 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:37.959+0000 7f9802a00700 1 -- 172.21.15.19:0/1117254233 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f97fc054990 con 0x7f980c10bcb0 2024-05-07T02:17:39.702 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:37.959+0000 7f9809400700 1 --2- 172.21.15.19:0/1117254233 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f97f003e580 0x7f97f0040a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:39.702 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:37.960+0000 7f9809400700 1 --2- 172.21.15.19:0/1117254233 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f97f003e580 0x7f97f0040a90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f97f8006fd0 tx=0x7f97f8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:39.702 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:37.965+0000 7f9802a00700 1 -- 172.21.15.19:0/1117254233 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f97fc00e260 con 0x7f980c10bcb0 2024-05-07T02:17:39.702 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:38.103+0000 7f9802a00700 1 -- 172.21.15.19:0/1117254233 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f97fc02dc90 con 0x7f980c10bcb0 2024-05-07T02:17:39.702 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:38.210+0000 7f9810a00700 1 -- 172.21.15.19:0/1117254233 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f97ec005650 con 0x7f980c10bcb0 2024-05-07T02:17:39.702 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:38.212+0000 7f9802a00700 1 -- 172.21.15.19:0/1117254233 <== mon.0 v2:172.21.15.19:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f97fc02ce20 con 0x7f980c10bcb0 2024-05-07T02:17:39.702 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:38.216+0000 7f9810a00700 1 -- 172.21.15.19:0/1117254233 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f97f003e580 msgr2=0x7f97f0040a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:39.702 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:38.216+0000 7f9810a00700 1 --2- 172.21.15.19:0/1117254233 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f97f003e580 0x7f97f0040a90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f97f8006fd0 tx=0x7f97f8006e40 comp rx=0 tx=0).stop 2024-05-07T02:17:39.703 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:38.216+0000 7f9810a00700 1 -- 172.21.15.19:0/1117254233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f980c10bcb0 msgr2=0x7f980c19ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:39.703 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:38.216+0000 7f9810a00700 1 --2- 172.21.15.19:0/1117254233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f980c10bcb0 0x7f980c19ff60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f97fc003e70 tx=0x7f97fc011aa0 comp rx=0 tx=0).stop 2024-05-07T02:17:39.703 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:38.216+0000 7f9810a00700 1 -- 172.21.15.19:0/1117254233 shutdown_connections 2024-05-07T02:17:39.703 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:38.216+0000 7f9810a00700 1 --2- 172.21.15.19:0/1117254233 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f97f003e580 0x7f97f0040a90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:39.703 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:38.216+0000 7f9810a00700 1 --2- 172.21.15.19:0/1117254233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f980c10bcb0 0x7f980c19ff60 secure :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f97fc003e70 tx=0x7f97fc011aa0 comp rx=0 tx=0).stop 2024-05-07T02:17:39.703 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:38.216+0000 7f9810a00700 1 -- 172.21.15.19:0/1117254233 >> 172.21.15.19:0/1117254233 conn(0x7f980c107490 msgr2=0x7f980c108bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:17:39.703 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:38.217+0000 7f9810a00700 1 -- 172.21.15.19:0/1117254233 shutdown_connections 2024-05-07T02:17:39.703 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:38.217+0000 7f9810a00700 1 -- 172.21.15.19:0/1117254233 wait complete. 2024-05-07T02:17:39.733 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-07T02:17:39.733 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout disabled 2024-05-07T02:17:39.762 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-07T02:17:39.763 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-05-07T02:17:39.763 INFO:teuthology.orchestra.run.smithi019.stdout:firewalld.service is not enabled 2024-05-07T02:17:39.763 INFO:teuthology.orchestra.run.smithi019.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-07T02:17:39.770 INFO:teuthology.orchestra.run.smithi019.stdout:Ceph Dashboard is now available at: 2024-05-07T02:17:39.770 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:17:39.770 INFO:teuthology.orchestra.run.smithi019.stdout: URL: https://smithi019.front.sepia.ceph.com:8443/ 2024-05-07T02:17:39.770 INFO:teuthology.orchestra.run.smithi019.stdout: User: admin 2024-05-07T02:17:39.770 INFO:teuthology.orchestra.run.smithi019.stdout: Password: 6387drh8i3 2024-05-07T02:17:39.770 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:17:39.771 INFO:teuthology.orchestra.run.smithi019.stdout:Saving cluster configuration to /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/config directory 2024-05-07T02:17:39.773 INFO:teuthology.orchestra.run.smithi019.stdout:Enabling autotune for osd_memory_target 2024-05-07T02:17:41.511 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.540+0000 7fa481a00700 1 Processor -- start 2024-05-07T02:17:41.512 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.541+0000 7fa481a00700 1 -- start start 2024-05-07T02:17:41.512 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.541+0000 7fa481a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa47c07ed60 0x7fa47c07d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:41.512 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.541+0000 7fa481a00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa47c07d770 con 0x7fa47c07ed60 2024-05-07T02:17:41.512 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.542+0000 7fa47aa00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa47c07ed60 0x7fa47c07d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:41.512 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.542+0000 7fa47aa00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa47c07ed60 0x7fa47c07d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48942/0 (socket says 172.21.15.19:48942) 2024-05-07T02:17:41.512 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.542+0000 7fa47aa00700 1 -- 172.21.15.19:0/3548031375 learned_addr learned my addr 172.21.15.19:0/3548031375 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:17:41.512 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.542+0000 7fa47aa00700 1 -- 172.21.15.19:0/3548031375 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa47c07d8f0 con 0x7fa47c07ed60 2024-05-07T02:17:41.512 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.543+0000 7fa47aa00700 1 --2- 172.21.15.19:0/3548031375 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa47c07ed60 0x7fa47c07d230 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa46c0098a0 tx=0x7fa46c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e0e5899a77cdb69 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:41.512 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.543+0000 7fa47a000700 1 -- 172.21.15.19:0/3548031375 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa46c004160 con 0x7fa47c07ed60 2024-05-07T02:17:41.512 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.543+0000 7fa47a000700 1 -- 172.21.15.19:0/3548031375 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa46c004300 con 0x7fa47c07ed60 2024-05-07T02:17:41.514 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.543+0000 7fa47a000700 1 -- 172.21.15.19:0/3548031375 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa46c011a20 con 0x7fa47c07ed60 2024-05-07T02:17:41.514 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.544+0000 7fa481a00700 1 -- 172.21.15.19:0/3548031375 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa47c07ed60 msgr2=0x7fa47c07d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:41.514 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.544+0000 7fa481a00700 1 --2- 172.21.15.19:0/3548031375 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa47c07ed60 0x7fa47c07d230 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa46c0098a0 tx=0x7fa46c009bb0 comp rx=0 tx=0).stop 2024-05-07T02:17:41.514 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.544+0000 7fa481a00700 1 -- 172.21.15.19:0/3548031375 shutdown_connections 2024-05-07T02:17:41.515 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.544+0000 7fa481a00700 1 --2- 172.21.15.19:0/3548031375 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa47c07ed60 0x7fa47c07d230 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:41.515 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.544+0000 7fa481a00700 1 -- 172.21.15.19:0/3548031375 >> 172.21.15.19:0/3548031375 conn(0x7fa47c105150 msgr2=0x7fa47c1075c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:17:41.515 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.544+0000 7fa481a00700 1 -- 172.21.15.19:0/3548031375 shutdown_connections 2024-05-07T02:17:41.515 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.544+0000 7fa481a00700 1 -- 172.21.15.19:0/3548031375 wait complete. 2024-05-07T02:17:41.515 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.545+0000 7fa481a00700 1 Processor -- start 2024-05-07T02:17:41.515 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.546+0000 7fa481a00700 1 -- start start 2024-05-07T02:17:41.515 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.546+0000 7fa481a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa47c07ed60 0x7fa47c19ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:41.515 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.546+0000 7fa481a00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa47c110070 con 0x7fa47c07ed60 2024-05-07T02:17:41.515 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.547+0000 7fa47aa00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa47c07ed60 0x7fa47c19ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:41.518 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.547+0000 7fa47aa00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa47c07ed60 0x7fa47c19ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48954/0 (socket says 172.21.15.19:48954) 2024-05-07T02:17:41.519 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.547+0000 7fa47aa00700 1 -- 172.21.15.19:0/1973412155 learned_addr learned my addr 172.21.15.19:0/1973412155 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:17:41.519 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.547+0000 7fa47aa00700 1 -- 172.21.15.19:0/1973412155 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa47c1a0460 con 0x7fa47c07ed60 2024-05-07T02:17:41.519 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.547+0000 7fa47aa00700 1 --2- 172.21.15.19:0/1973412155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa47c07ed60 0x7fa47c19ff20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa46c000c00 tx=0x7fa46c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:41.519 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.548+0000 7fa473400700 1 -- 172.21.15.19:0/1973412155 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa46c003aa0 con 0x7fa47c07ed60 2024-05-07T02:17:41.519 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.548+0000 7fa473400700 1 -- 172.21.15.19:0/1973412155 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa46c01b8f0 con 0x7fa47c07ed60 2024-05-07T02:17:41.519 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.548+0000 7fa473400700 1 -- 172.21.15.19:0/1973412155 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa46c023890 con 0x7fa47c07ed60 2024-05-07T02:17:41.519 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.548+0000 7fa481a00700 1 -- 172.21.15.19:0/1973412155 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa47c1a06f0 con 0x7fa47c07ed60 2024-05-07T02:17:41.519 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.548+0000 7fa481a00700 1 -- 172.21.15.19:0/1973412155 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa47c1a3300 con 0x7fa47c07ed60 2024-05-07T02:17:41.519 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.549+0000 7fa473400700 1 -- 172.21.15.19:0/1973412155 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fa46c02c3f0 con 0x7fa47c07ed60 2024-05-07T02:17:41.519 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.549+0000 7fa481a00700 1 -- 172.21.15.19:0/1973412155 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa47c064e20 con 0x7fa47c07ed60 2024-05-07T02:17:41.519 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.550+0000 7fa473400700 1 --2- 172.21.15.19:0/1973412155 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fa46003e680 0x7fa460040b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:41.519 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.550+0000 7fa473400700 1 -- 172.21.15.19:0/1973412155 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa46c0540c0 con 0x7fa47c07ed60 2024-05-07T02:17:41.520 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.550+0000 7fa473e00700 1 --2- 172.21.15.19:0/1973412155 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fa46003e680 0x7fa460040b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:41.520 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.551+0000 7fa473e00700 1 --2- 172.21.15.19:0/1973412155 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fa46003e680 0x7fa460040b90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa464006fd0 tx=0x7fa464006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:41.520 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.556+0000 7fa473400700 1 -- 172.21.15.19:0/1973412155 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa46c01f030 con 0x7fa47c07ed60 2024-05-07T02:17:41.520 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.792+0000 7fa481a00700 1 -- 172.21.15.19:0/1973412155 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fa47c1a34d0 con 0x7fa47c07ed60 2024-05-07T02:17:41.520 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.793+0000 7fa473400700 1 -- 172.21.15.19:0/1973412155 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fa46c02b3a0 con 0x7fa47c07ed60 2024-05-07T02:17:41.520 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.796+0000 7fa481a00700 1 -- 172.21.15.19:0/1973412155 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fa46003e680 msgr2=0x7fa460040b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:41.520 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.796+0000 7fa481a00700 1 --2- 172.21.15.19:0/1973412155 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fa46003e680 0x7fa460040b90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa464006fd0 tx=0x7fa464006e40 comp rx=0 tx=0).stop 2024-05-07T02:17:41.520 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.796+0000 7fa481a00700 1 -- 172.21.15.19:0/1973412155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa47c07ed60 msgr2=0x7fa47c19ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:41.520 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.796+0000 7fa481a00700 1 --2- 172.21.15.19:0/1973412155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa47c07ed60 0x7fa47c19ff20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa46c000c00 tx=0x7fa46c011aa0 comp rx=0 tx=0).stop 2024-05-07T02:17:41.520 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.797+0000 7fa481a00700 1 -- 172.21.15.19:0/1973412155 shutdown_connections 2024-05-07T02:17:41.520 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.797+0000 7fa481a00700 1 --2- 172.21.15.19:0/1973412155 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fa46003e680 0x7fa460040b90 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:41.520 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.797+0000 7fa481a00700 1 --2- 172.21.15.19:0/1973412155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa47c07ed60 0x7fa47c19ff20 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:41.521 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.797+0000 7fa481a00700 1 -- 172.21.15.19:0/1973412155 >> 172.21.15.19:0/1973412155 conn(0x7fa47c105150 msgr2=0x7fa47c105910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:17:41.521 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.797+0000 7fa481a00700 1 -- 172.21.15.19:0/1973412155 shutdown_connections 2024-05-07T02:17:41.521 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:40.797+0000 7fa481a00700 1 -- 172.21.15.19:0/1973412155 wait complete. 2024-05-07T02:17:43.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.363+0000 7f7147600700 1 Processor -- start 2024-05-07T02:17:43.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.363+0000 7f7147600700 1 -- start start 2024-05-07T02:17:43.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.364+0000 7f7147600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f714010bd50 0x7f714010c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:43.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.364+0000 7f7147600700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f714010c750 con 0x7f714010bd50 2024-05-07T02:17:43.390 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.364+0000 7f7144c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f714010bd50 0x7f714010c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:43.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.364+0000 7f7144c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f714010bd50 0x7f714010c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48956/0 (socket says 172.21.15.19:48956) 2024-05-07T02:17:43.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.364+0000 7f7144c00700 1 -- 172.21.15.19:0/2506973044 learned_addr learned my addr 172.21.15.19:0/2506973044 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:17:43.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.365+0000 7f7144c00700 1 -- 172.21.15.19:0/2506973044 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f714010ce80 con 0x7f714010bd50 2024-05-07T02:17:43.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.365+0000 7f7144c00700 1 --2- 172.21.15.19:0/2506973044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f714010bd50 0x7f714010c160 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f71340098a0 tx=0x7f7134009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6c585de889cea32 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:43.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.365+0000 7f713f400700 1 -- 172.21.15.19:0/2506973044 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7134004160 con 0x7f714010bd50 2024-05-07T02:17:43.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.365+0000 7f713f400700 1 -- 172.21.15.19:0/2506973044 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7134004300 con 0x7f714010bd50 2024-05-07T02:17:43.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.365+0000 7f713f400700 1 -- 172.21.15.19:0/2506973044 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7134011a20 con 0x7f714010bd50 2024-05-07T02:17:43.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.366+0000 7f7147600700 1 -- 172.21.15.19:0/2506973044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f714010bd50 msgr2=0x7f714010c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:43.394 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.366+0000 7f7147600700 1 --2- 172.21.15.19:0/2506973044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f714010bd50 0x7f714010c160 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f71340098a0 tx=0x7f7134009bb0 comp rx=0 tx=0).stop 2024-05-07T02:17:43.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.366+0000 7f7147600700 1 -- 172.21.15.19:0/2506973044 shutdown_connections 2024-05-07T02:17:43.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.366+0000 7f7147600700 1 --2- 172.21.15.19:0/2506973044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f714010bd50 0x7f714010c160 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:43.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.366+0000 7f7147600700 1 -- 172.21.15.19:0/2506973044 >> 172.21.15.19:0/2506973044 conn(0x7f714007f220 msgr2=0x7f7140109ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:17:43.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.367+0000 7f7147600700 1 -- 172.21.15.19:0/2506973044 shutdown_connections 2024-05-07T02:17:43.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.367+0000 7f7147600700 1 -- 172.21.15.19:0/2506973044 wait complete. 2024-05-07T02:17:43.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.368+0000 7f7147600700 1 Processor -- start 2024-05-07T02:17:43.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.368+0000 7f7147600700 1 -- start start 2024-05-07T02:17:43.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.368+0000 7f7147600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f714010bd50 0x7f71401a0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:43.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.368+0000 7f7147600700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7140110230 con 0x7f714010bd50 2024-05-07T02:17:43.395 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.368+0000 7f7144c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f714010bd50 0x7f71401a0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:43.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.369+0000 7f7144c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f714010bd50 0x7f71401a0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48972/0 (socket says 172.21.15.19:48972) 2024-05-07T02:17:43.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.369+0000 7f7144c00700 1 -- 172.21.15.19:0/192908436 learned_addr learned my addr 172.21.15.19:0/192908436 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:17:43.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.369+0000 7f7144c00700 1 -- 172.21.15.19:0/192908436 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71401a0560 con 0x7f714010bd50 2024-05-07T02:17:43.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.369+0000 7f7144c00700 1 --2- 172.21.15.19:0/192908436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f714010bd50 0x7f71401a0020 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7134011a20 tx=0x7f7134011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:43.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.370+0000 7f713d600700 1 -- 172.21.15.19:0/192908436 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71340039c0 con 0x7f714010bd50 2024-05-07T02:17:43.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.370+0000 7f713d600700 1 -- 172.21.15.19:0/192908436 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7134003b60 con 0x7f714010bd50 2024-05-07T02:17:43.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.370+0000 7f7147600700 1 -- 172.21.15.19:0/192908436 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71401a07f0 con 0x7f714010bd50 2024-05-07T02:17:43.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.370+0000 7f713d600700 1 -- 172.21.15.19:0/192908436 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7134019700 con 0x7f714010bd50 2024-05-07T02:17:43.396 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.370+0000 7f7147600700 1 -- 172.21.15.19:0/192908436 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71401a3400 con 0x7f714010bd50 2024-05-07T02:17:43.399 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.371+0000 7f7147600700 1 -- 172.21.15.19:0/192908436 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7140064e20 con 0x7f714010bd50 2024-05-07T02:17:43.399 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.371+0000 7f713d600700 1 -- 172.21.15.19:0/192908436 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f713401a430 con 0x7f714010bd50 2024-05-07T02:17:43.399 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.372+0000 7f713d600700 1 --2- 172.21.15.19:0/192908436 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f712803e630 0x7f7128040b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:43.399 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.372+0000 7f713d600700 1 -- 172.21.15.19:0/192908436 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7134053eb0 con 0x7f714010bd50 2024-05-07T02:17:43.399 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.372+0000 7f713fe00700 1 --2- 172.21.15.19:0/192908436 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f712803e630 0x7f7128040b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:43.399 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.373+0000 7f713fe00700 1 --2- 172.21.15.19:0/192908436 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f712803e630 0x7f7128040b40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f712c006fd0 tx=0x7f712c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:43.399 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.378+0000 7f713d600700 1 -- 172.21.15.19:0/192908436 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7134018da0 con 0x7f714010bd50 2024-05-07T02:17:43.399 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.736+0000 7f7147600700 1 -- 172.21.15.19:0/192908436 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f71401a3710 con 0x7f714010bd50 2024-05-07T02:17:43.400 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.740+0000 7f713d600700 1 -- 172.21.15.19:0/192908436 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f7134023dc0 con 0x7f714010bd50 2024-05-07T02:17:43.400 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.743+0000 7f7147600700 1 -- 172.21.15.19:0/192908436 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f712803e630 msgr2=0x7f7128040b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:43.400 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.743+0000 7f7147600700 1 --2- 172.21.15.19:0/192908436 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f712803e630 0x7f7128040b40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f712c006fd0 tx=0x7f712c006e40 comp rx=0 tx=0).stop 2024-05-07T02:17:43.400 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.743+0000 7f7147600700 1 -- 172.21.15.19:0/192908436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f714010bd50 msgr2=0x7f71401a0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:43.400 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.743+0000 7f7147600700 1 --2- 172.21.15.19:0/192908436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f714010bd50 0x7f71401a0020 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7134011a20 tx=0x7f7134011b00 comp rx=0 tx=0).stop 2024-05-07T02:17:43.400 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.743+0000 7f7147600700 1 -- 172.21.15.19:0/192908436 shutdown_connections 2024-05-07T02:17:43.400 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.743+0000 7f7147600700 1 --2- 172.21.15.19:0/192908436 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f712803e630 0x7f7128040b40 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:43.400 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.743+0000 7f7147600700 1 --2- 172.21.15.19:0/192908436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f714010bd50 0x7f71401a0020 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:43.400 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.743+0000 7f7147600700 1 -- 172.21.15.19:0/192908436 >> 172.21.15.19:0/192908436 conn(0x7f714007f220 msgr2=0x7f7140109270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:17:43.400 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.744+0000 7f7147600700 1 -- 172.21.15.19:0/192908436 shutdown_connections 2024-05-07T02:17:43.400 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-05-07T02:17:42.744+0000 7f7147600700 1 -- 172.21.15.19:0/192908436 wait complete. 2024-05-07T02:17:43.400 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-07T02:17:43.400 INFO:teuthology.orchestra.run.smithi019.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-07T02:17:43.400 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:17:43.401 INFO:teuthology.orchestra.run.smithi019.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-07T02:17:43.401 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:17:43.401 INFO:teuthology.orchestra.run.smithi019.stdout:Or, if you are only running a single cluster on this host: 2024-05-07T02:17:43.401 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:17:43.401 INFO:teuthology.orchestra.run.smithi019.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-07T02:17:43.401 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:17:43.401 INFO:teuthology.orchestra.run.smithi019.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-07T02:17:43.401 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:17:43.401 INFO:teuthology.orchestra.run.smithi019.stdout: ceph telemetry on 2024-05-07T02:17:43.401 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:17:43.401 INFO:teuthology.orchestra.run.smithi019.stdout:For more information see: 2024-05-07T02:17:43.401 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:17:43.401 INFO:teuthology.orchestra.run.smithi019.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-07T02:17:43.401 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:17:43.401 INFO:teuthology.orchestra.run.smithi019.stdout:Bootstrap complete. 2024-05-07T02:17:43.508 INFO:tasks.cephadm:Fetching config... 2024-05-07T02:17:43.508 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-07T02:17:43.508 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-07T02:17:43.564 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-07T02:17:43.564 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-07T02:17:43.564 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-07T02:17:43.608 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-07T02:17:43.608 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-07T02:17:43.608 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-07T02:17:43.726 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-07T02:17:43.726 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-07T02:17:43.726 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-07T02:17:43.826 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-07T02:17:43.826 DEBUG:teuthology.orchestra.run.smithi019:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDWaSFHRR01PItWFiaCHApyakbyvTm1iFnKo7o7gyMhFt8Iz/7y19vrY8+uZYTceXTfy5uypRm0N3v1PfKQaGosbxTtdcqdnPShx3ob3VKMkoliJmB6qUHonbR5HL0hLYnDY5U/S3GruG338biAE4zFOBgKyGkxuW/9GdqElVpDQbOFlWWxxdLRu6Ziab9zKrRba8o8a+Z/qt9PjGkl75dW0wWrMAHShNo8zhOMvjCEB6jowhzgFrHlSM6fBcBiZMKZ00PhXX6cBPiQjN7eSg5nLyamkmhGujUJt0Ddo5Wjzudm5pu0vZMww3f1+QKhDMfHroaxhEHu3radW9Z0Ts6YWFvOUq4yQVNYHbusvEiG1gYrORuiXGMD/GLXQ0oQjQy37VsswvCfGVy70iBDDSFmRzeVVhu+4rADDk5dX2mFK8MsBA1Jbz+XDydBmIP41hzuKwTjOZSVgO4eLKiwXoaPkDOqm8bxPLdil0LdUk0tyAoku0Cly6pvlo8BYcHJwA0= ceph-9ff84c16-0c17-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-07T02:17:43.928 INFO:teuthology.orchestra.run.smithi019.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDWaSFHRR01PItWFiaCHApyakbyvTm1iFnKo7o7gyMhFt8Iz/7y19vrY8+uZYTceXTfy5uypRm0N3v1PfKQaGosbxTtdcqdnPShx3ob3VKMkoliJmB6qUHonbR5HL0hLYnDY5U/S3GruG338biAE4zFOBgKyGkxuW/9GdqElVpDQbOFlWWxxdLRu6Ziab9zKrRba8o8a+Z/qt9PjGkl75dW0wWrMAHShNo8zhOMvjCEB6jowhzgFrHlSM6fBcBiZMKZ00PhXX6cBPiQjN7eSg5nLyamkmhGujUJt0Ddo5Wjzudm5pu0vZMww3f1+QKhDMfHroaxhEHu3radW9Z0Ts6YWFvOUq4yQVNYHbusvEiG1gYrORuiXGMD/GLXQ0oQjQy37VsswvCfGVy70iBDDSFmRzeVVhu+4rADDk5dX2mFK8MsBA1Jbz+XDydBmIP41hzuKwTjOZSVgO4eLKiwXoaPkDOqm8bxPLdil0LdUk0tyAoku0Cly6pvlo8BYcHJwA0= ceph-9ff84c16-0c17-11ef-bc97-c7b262605968 2024-05-07T02:17:43.958 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:43 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/192908436' entity='client.admin' 2024-05-07T02:17:43.961 DEBUG:teuthology.orchestra.run.smithi027:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDWaSFHRR01PItWFiaCHApyakbyvTm1iFnKo7o7gyMhFt8Iz/7y19vrY8+uZYTceXTfy5uypRm0N3v1PfKQaGosbxTtdcqdnPShx3ob3VKMkoliJmB6qUHonbR5HL0hLYnDY5U/S3GruG338biAE4zFOBgKyGkxuW/9GdqElVpDQbOFlWWxxdLRu6Ziab9zKrRba8o8a+Z/qt9PjGkl75dW0wWrMAHShNo8zhOMvjCEB6jowhzgFrHlSM6fBcBiZMKZ00PhXX6cBPiQjN7eSg5nLyamkmhGujUJt0Ddo5Wjzudm5pu0vZMww3f1+QKhDMfHroaxhEHu3radW9Z0Ts6YWFvOUq4yQVNYHbusvEiG1gYrORuiXGMD/GLXQ0oQjQy37VsswvCfGVy70iBDDSFmRzeVVhu+4rADDk5dX2mFK8MsBA1Jbz+XDydBmIP41hzuKwTjOZSVgO4eLKiwXoaPkDOqm8bxPLdil0LdUk0tyAoku0Cly6pvlo8BYcHJwA0= ceph-9ff84c16-0c17-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-07T02:17:44.010 INFO:teuthology.orchestra.run.smithi027.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDWaSFHRR01PItWFiaCHApyakbyvTm1iFnKo7o7gyMhFt8Iz/7y19vrY8+uZYTceXTfy5uypRm0N3v1PfKQaGosbxTtdcqdnPShx3ob3VKMkoliJmB6qUHonbR5HL0hLYnDY5U/S3GruG338biAE4zFOBgKyGkxuW/9GdqElVpDQbOFlWWxxdLRu6Ziab9zKrRba8o8a+Z/qt9PjGkl75dW0wWrMAHShNo8zhOMvjCEB6jowhzgFrHlSM6fBcBiZMKZ00PhXX6cBPiQjN7eSg5nLyamkmhGujUJt0Ddo5Wjzudm5pu0vZMww3f1+QKhDMfHroaxhEHu3radW9Z0Ts6YWFvOUq4yQVNYHbusvEiG1gYrORuiXGMD/GLXQ0oQjQy37VsswvCfGVy70iBDDSFmRzeVVhu+4rADDk5dX2mFK8MsBA1Jbz+XDydBmIP41hzuKwTjOZSVgO4eLKiwXoaPkDOqm8bxPLdil0LdUk0tyAoku0Cly6pvlo8BYcHJwA0= ceph-9ff84c16-0c17-11ef-bc97-c7b262605968 2024-05-07T02:17:44.023 DEBUG:teuthology.orchestra.run.smithi064:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDWaSFHRR01PItWFiaCHApyakbyvTm1iFnKo7o7gyMhFt8Iz/7y19vrY8+uZYTceXTfy5uypRm0N3v1PfKQaGosbxTtdcqdnPShx3ob3VKMkoliJmB6qUHonbR5HL0hLYnDY5U/S3GruG338biAE4zFOBgKyGkxuW/9GdqElVpDQbOFlWWxxdLRu6Ziab9zKrRba8o8a+Z/qt9PjGkl75dW0wWrMAHShNo8zhOMvjCEB6jowhzgFrHlSM6fBcBiZMKZ00PhXX6cBPiQjN7eSg5nLyamkmhGujUJt0Ddo5Wjzudm5pu0vZMww3f1+QKhDMfHroaxhEHu3radW9Z0Ts6YWFvOUq4yQVNYHbusvEiG1gYrORuiXGMD/GLXQ0oQjQy37VsswvCfGVy70iBDDSFmRzeVVhu+4rADDk5dX2mFK8MsBA1Jbz+XDydBmIP41hzuKwTjOZSVgO4eLKiwXoaPkDOqm8bxPLdil0LdUk0tyAoku0Cly6pvlo8BYcHJwA0= ceph-9ff84c16-0c17-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-07T02:17:44.066 INFO:teuthology.orchestra.run.smithi064.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDWaSFHRR01PItWFiaCHApyakbyvTm1iFnKo7o7gyMhFt8Iz/7y19vrY8+uZYTceXTfy5uypRm0N3v1PfKQaGosbxTtdcqdnPShx3ob3VKMkoliJmB6qUHonbR5HL0hLYnDY5U/S3GruG338biAE4zFOBgKyGkxuW/9GdqElVpDQbOFlWWxxdLRu6Ziab9zKrRba8o8a+Z/qt9PjGkl75dW0wWrMAHShNo8zhOMvjCEB6jowhzgFrHlSM6fBcBiZMKZ00PhXX6cBPiQjN7eSg5nLyamkmhGujUJt0Ddo5Wjzudm5pu0vZMww3f1+QKhDMfHroaxhEHu3radW9Z0Ts6YWFvOUq4yQVNYHbusvEiG1gYrORuiXGMD/GLXQ0oQjQy37VsswvCfGVy70iBDDSFmRzeVVhu+4rADDk5dX2mFK8MsBA1Jbz+XDydBmIP41hzuKwTjOZSVgO4eLKiwXoaPkDOqm8bxPLdil0LdUk0tyAoku0Cly6pvlo8BYcHJwA0= ceph-9ff84c16-0c17-11ef-bc97-c7b262605968 2024-05-07T02:17:44.078 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-07T02:17:44.525 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:17:46.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:46.862+0000 7fdc1f200700 1 -- 172.21.15.19:0/3401725318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc20104510 msgr2=0x7fdc20106930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:46.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:46.862+0000 7fdc1f200700 1 --2- 172.21.15.19:0/3401725318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc20104510 0x7fdc20106930 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdc10009910 tx=0x7fdc10009c20 comp rx=0 tx=0).stop 2024-05-07T02:17:46.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:46.862+0000 7fdc1f200700 1 -- 172.21.15.19:0/3401725318 shutdown_connections 2024-05-07T02:17:46.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:46.862+0000 7fdc1f200700 1 --2- 172.21.15.19:0/3401725318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc20104510 0x7fdc20106930 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:46.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:46.862+0000 7fdc1f200700 1 -- 172.21.15.19:0/3401725318 >> 172.21.15.19:0/3401725318 conn(0x7fdc200fdf70 msgr2=0x7fdc20100400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:17:46.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:46.862+0000 7fdc1f200700 1 -- 172.21.15.19:0/3401725318 shutdown_connections 2024-05-07T02:17:46.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:46.862+0000 7fdc1f200700 1 -- 172.21.15.19:0/3401725318 wait complete. 2024-05-07T02:17:46.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:46.864+0000 7fdc1f200700 1 Processor -- start 2024-05-07T02:17:46.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:46.864+0000 7fdc1f200700 1 -- start start 2024-05-07T02:17:46.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:46.865+0000 7fdc1f200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc20104510 0x7fdc20103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:46.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:46.865+0000 7fdc1f200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdc201099d0 con 0x7fdc20104510 2024-05-07T02:17:46.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:46.865+0000 7fdc1de00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc20104510 0x7fdc20103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:46.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:46.865+0000 7fdc1de00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc20104510 0x7fdc20103c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48994/0 (socket says 172.21.15.19:48994) 2024-05-07T02:17:46.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:46.865+0000 7fdc1de00700 1 -- 172.21.15.19:0/1893112681 learned_addr learned my addr 172.21.15.19:0/1893112681 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:17:46.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:46.866+0000 7fdc1de00700 1 -- 172.21.15.19:0/1893112681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc201041d0 con 0x7fdc20104510 2024-05-07T02:17:46.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:46.866+0000 7fdc1de00700 1 --2- 172.21.15.19:0/1893112681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc20104510 0x7fdc20103c90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdc100050d0 tx=0x7fdc10009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:46.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:46.866+0000 7fdc16a00700 1 -- 172.21.15.19:0/1893112681 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc10027070 con 0x7fdc20104510 2024-05-07T02:17:46.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:46.866+0000 7fdc16a00700 1 -- 172.21.15.19:0/1893112681 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc1000be10 con 0x7fdc20104510 2024-05-07T02:17:46.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:46.866+0000 7fdc1f200700 1 -- 172.21.15.19:0/1893112681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc20102420 con 0x7fdc20104510 2024-05-07T02:17:46.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:46.867+0000 7fdc16a00700 1 -- 172.21.15.19:0/1893112681 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc10022d80 con 0x7fdc20104510 2024-05-07T02:17:46.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:46.867+0000 7fdc1f200700 1 -- 172.21.15.19:0/1893112681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc20102900 con 0x7fdc20104510 2024-05-07T02:17:46.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:46.868+0000 7fdc1f200700 1 -- 172.21.15.19:0/1893112681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc2010a150 con 0x7fdc20104510 2024-05-07T02:17:46.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:46.868+0000 7fdc16a00700 1 -- 172.21.15.19:0/1893112681 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fdc1000b960 con 0x7fdc20104510 2024-05-07T02:17:46.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:46.869+0000 7fdc16a00700 1 --2- 172.21.15.19:0/1893112681 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fdc0403e570 0x7fdc04040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:46.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:46.869+0000 7fdc16a00700 1 -- 172.21.15.19:0/1893112681 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdc1001e030 con 0x7fdc20104510 2024-05-07T02:17:46.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:46.869+0000 7fdc1d400700 1 --2- 172.21.15.19:0/1893112681 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fdc0403e570 0x7fdc04040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:46.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:46.870+0000 7fdc1d400700 1 --2- 172.21.15.19:0/1893112681 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fdc0403e570 0x7fdc04040a80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c006fd0 tx=0x7fdc0c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:46.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:46.875+0000 7fdc16a00700 1 -- 172.21.15.19:0/1893112681 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdc10013070 con 0x7fdc20104510 2024-05-07T02:17:47.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:47.110+0000 7fdc1f200700 1 -- 172.21.15.19:0/1893112681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fdc20192310 con 0x7fdc20104510 2024-05-07T02:17:47.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:47.115+0000 7fdc16a00700 1 -- 172.21.15.19:0/1893112681 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fdc1000de00 con 0x7fdc20104510 2024-05-07T02:17:47.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:47.118+0000 7fdc1f200700 1 -- 172.21.15.19:0/1893112681 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fdc0403e570 msgr2=0x7fdc04040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:47.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:47.118+0000 7fdc1f200700 1 --2- 172.21.15.19:0/1893112681 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fdc0403e570 0x7fdc04040a80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c006fd0 tx=0x7fdc0c006e40 comp rx=0 tx=0).stop 2024-05-07T02:17:47.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:47.118+0000 7fdc1f200700 1 -- 172.21.15.19:0/1893112681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc20104510 msgr2=0x7fdc20103c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:47.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:47.118+0000 7fdc1f200700 1 --2- 172.21.15.19:0/1893112681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc20104510 0x7fdc20103c90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdc100050d0 tx=0x7fdc10009f80 comp rx=0 tx=0).stop 2024-05-07T02:17:47.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:47.118+0000 7fdc1f200700 1 -- 172.21.15.19:0/1893112681 shutdown_connections 2024-05-07T02:17:47.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:47.118+0000 7fdc1f200700 1 --2- 172.21.15.19:0/1893112681 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fdc0403e570 0x7fdc04040a80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:47.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:47.118+0000 7fdc1f200700 1 --2- 172.21.15.19:0/1893112681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc20104510 0x7fdc20103c90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:47.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:47.118+0000 7fdc1f200700 1 -- 172.21.15.19:0/1893112681 >> 172.21.15.19:0/1893112681 conn(0x7fdc200fdf70 msgr2=0x7fdc200fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:17:47.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:47.119+0000 7fdc1f200700 1 -- 172.21.15.19:0/1893112681 shutdown_connections 2024-05-07T02:17:47.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:47.119+0000 7fdc1f200700 1 -- 172.21.15.19:0/1893112681 wait complete. 2024-05-07T02:17:48.084 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-07T02:17:48.084 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-07T02:17:48.306 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:48 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/1893112681' entity='client.admin' 2024-05-07T02:17:48.541 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:17:51.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:51.419+0000 7f01cf800700 1 -- 172.21.15.19:0/1732922170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01c8077780 msgr2=0x7f01c8075bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:51.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:51.419+0000 7f01cf800700 1 --2- 172.21.15.19:0/1732922170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01c8077780 0x7f01c8075bb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f01bc009910 tx=0x7f01bc009c20 comp rx=0 tx=0).stop 2024-05-07T02:17:51.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:51.419+0000 7f01cf800700 1 -- 172.21.15.19:0/1732922170 shutdown_connections 2024-05-07T02:17:51.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:51.419+0000 7f01cf800700 1 --2- 172.21.15.19:0/1732922170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01c8077780 0x7f01c8075bb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:51.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:51.419+0000 7f01cf800700 1 -- 172.21.15.19:0/1732922170 >> 172.21.15.19:0/1732922170 conn(0x7f01c80ff260 msgr2=0x7f01c8101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:17:51.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:51.419+0000 7f01cf800700 1 -- 172.21.15.19:0/1732922170 shutdown_connections 2024-05-07T02:17:51.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:51.419+0000 7f01cf800700 1 -- 172.21.15.19:0/1732922170 wait complete. 2024-05-07T02:17:51.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:51.420+0000 7f01cf800700 1 Processor -- start 2024-05-07T02:17:51.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:51.420+0000 7f01cf800700 1 -- start start 2024-05-07T02:17:51.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:51.421+0000 7f01cf800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01c819fd90 0x7f01c81a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:51.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:51.421+0000 7f01cf800700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f01c8110270 con 0x7f01c819fd90 2024-05-07T02:17:51.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:51.421+0000 7f01cce00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01c819fd90 0x7f01c81a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:51.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:51.421+0000 7f01cce00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01c819fd90 0x7f01c81a01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60480/0 (socket says 172.21.15.19:60480) 2024-05-07T02:17:51.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:51.421+0000 7f01cce00700 1 -- 172.21.15.19:0/2793364827 learned_addr learned my addr 172.21.15.19:0/2793364827 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:17:51.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:51.421+0000 7f01cce00700 1 -- 172.21.15.19:0/2793364827 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c81a0740 con 0x7f01c819fd90 2024-05-07T02:17:51.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:51.422+0000 7f01cce00700 1 --2- 172.21.15.19:0/2793364827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01c819fd90 0x7f01c81a01c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f01bc009910 tx=0x7f01bc004f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:51.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:51.422+0000 7f01c5600700 1 -- 172.21.15.19:0/2793364827 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01bc027070 con 0x7f01c819fd90 2024-05-07T02:17:51.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:51.422+0000 7f01c5600700 1 -- 172.21.15.19:0/2793364827 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01bc00be10 con 0x7f01c819fd90 2024-05-07T02:17:51.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:51.422+0000 7f01cf800700 1 -- 172.21.15.19:0/2793364827 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01c81a09d0 con 0x7f01c819fd90 2024-05-07T02:17:51.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:51.422+0000 7f01c5600700 1 -- 172.21.15.19:0/2793364827 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01bc022d40 con 0x7f01c819fd90 2024-05-07T02:17:51.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:51.422+0000 7f01cf800700 1 -- 172.21.15.19:0/2793364827 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01c81a3500 con 0x7f01c819fd90 2024-05-07T02:17:51.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:51.423+0000 7f01cf800700 1 -- 172.21.15.19:0/2793364827 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01c8110950 con 0x7f01c819fd90 2024-05-07T02:17:51.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:51.424+0000 7f01c5600700 1 -- 172.21.15.19:0/2793364827 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f01bc022500 con 0x7f01c819fd90 2024-05-07T02:17:51.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:51.424+0000 7f01c5600700 1 --2- 172.21.15.19:0/2793364827 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f01b003e570 0x7f01b0040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:51.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:51.425+0000 7f01c5600700 1 -- 172.21.15.19:0/2793364827 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f01bc01e030 con 0x7f01c819fd90 2024-05-07T02:17:51.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:51.425+0000 7f01c7e00700 1 --2- 172.21.15.19:0/2793364827 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f01b003e570 0x7f01b0040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:51.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:51.426+0000 7f01c7e00700 1 --2- 172.21.15.19:0/2793364827 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f01b003e570 0x7f01b0040a80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f01b4006fd0 tx=0x7f01b4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:51.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:51.430+0000 7f01c5600700 1 -- 172.21.15.19:0/2793364827 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f01bc013070 con 0x7f01c819fd90 2024-05-07T02:17:51.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:51.673+0000 7f01cf800700 1 -- 172.21.15.19:0/2793364827 --> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f01c8063c00 con 0x7f01b003e570 2024-05-07T02:17:51.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:51.683+0000 7f01c5600700 1 -- 172.21.15.19:0/2793364827 <== mgr.14152 v2:172.21.15.19:6800/2059564551 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f01c8063c00 con 0x7f01b003e570 2024-05-07T02:17:51.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:51.686+0000 7f01cf800700 1 -- 172.21.15.19:0/2793364827 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f01b003e570 msgr2=0x7f01b0040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:51.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:51.686+0000 7f01cf800700 1 --2- 172.21.15.19:0/2793364827 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f01b003e570 0x7f01b0040a80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f01b4006fd0 tx=0x7f01b4006e40 comp rx=0 tx=0).stop 2024-05-07T02:17:51.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:51.686+0000 7f01cf800700 1 -- 172.21.15.19:0/2793364827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01c819fd90 msgr2=0x7f01c81a01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:51.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:51.686+0000 7f01cf800700 1 --2- 172.21.15.19:0/2793364827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01c819fd90 0x7f01c81a01c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f01bc009910 tx=0x7f01bc004f60 comp rx=0 tx=0).stop 2024-05-07T02:17:51.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:51.686+0000 7f01cf800700 1 -- 172.21.15.19:0/2793364827 shutdown_connections 2024-05-07T02:17:51.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:51.686+0000 7f01cf800700 1 --2- 172.21.15.19:0/2793364827 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f01b003e570 0x7f01b0040a80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:51.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:51.686+0000 7f01cf800700 1 --2- 172.21.15.19:0/2793364827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01c819fd90 0x7f01c81a01c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:51.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:51.686+0000 7f01cf800700 1 -- 172.21.15.19:0/2793364827 >> 172.21.15.19:0/2793364827 conn(0x7f01c80ff260 msgr2=0x7f01c80fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:17:51.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:51.686+0000 7f01cf800700 1 -- 172.21.15.19:0/2793364827 shutdown_connections 2024-05-07T02:17:51.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:51.687+0000 7f01cf800700 1 -- 172.21.15.19:0/2793364827 wait complete. 2024-05-07T02:17:51.991 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:51 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:17:51.992 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:51 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:17:51.992 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:51 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi019", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:17:51.992 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:51 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:17:51.992 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:51 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:17:51.992 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:51 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:17:51.992 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:51 smithi019 ceph-mon[29781]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:17:51.992 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:51 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:17:51.992 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:51 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:17:51.992 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:51 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:17:51.992 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:51 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:17:52.431 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi027 2024-05-07T02:17:52.432 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-05-07T02:17:52.432 DEBUG:teuthology.orchestra.run.smithi027:> dd of=/etc/ceph/ceph.conf 2024-05-07T02:17:52.451 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-05-07T02:17:52.451 DEBUG:teuthology.orchestra.run.smithi027:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-07T02:17:52.509 INFO:tasks.cephadm:Adding host smithi027 to orchestrator... 2024-05-07T02:17:52.510 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph orch host add smithi027 2024-05-07T02:17:52.927 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:17:53.228 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:52 smithi019 ceph-mon[29781]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:17:53.229 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:52 smithi019 ceph-mon[29781]: Updating smithi019:/etc/ceph/ceph.conf 2024-05-07T02:17:53.229 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:52 smithi019 ceph-mon[29781]: Updating smithi019:/var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-07T02:17:54.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:53 smithi019 ceph-mon[29781]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-05-07T02:17:54.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:53 smithi019 ceph-mon[29781]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:17:55.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:55.063+0000 7f670a400700 1 -- 172.21.15.19:0/3318083816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6704105fe0 msgr2=0x7f67041063f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:17:55.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:55.063+0000 7f670a400700 1 --2- 172.21.15.19:0/3318083816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6704105fe0 0x7f67041063f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f66f4009910 tx=0x7f66f4009c20 comp rx=0 tx=0).stop 2024-05-07T02:17:55.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:55.063+0000 7f670a400700 1 -- 172.21.15.19:0/3318083816 shutdown_connections 2024-05-07T02:17:55.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:55.063+0000 7f670a400700 1 --2- 172.21.15.19:0/3318083816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6704105fe0 0x7f67041063f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:17:55.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:55.064+0000 7f670a400700 1 -- 172.21.15.19:0/3318083816 >> 172.21.15.19:0/3318083816 conn(0x7f6704101890 msgr2=0x7f6704103ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:17:55.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:55.064+0000 7f670a400700 1 -- 172.21.15.19:0/3318083816 shutdown_connections 2024-05-07T02:17:55.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:55.064+0000 7f670a400700 1 -- 172.21.15.19:0/3318083816 wait complete. 2024-05-07T02:17:55.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:55.065+0000 7f670a400700 1 Processor -- start 2024-05-07T02:17:55.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:55.065+0000 7f670a400700 1 -- start start 2024-05-07T02:17:55.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:55.066+0000 7f670a400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6704105fe0 0x7f67041a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:55.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:55.066+0000 7f670a400700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f67041102d0 con 0x7f6704105fe0 2024-05-07T02:17:55.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:55.066+0000 7f6703400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6704105fe0 0x7f67041a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:55.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:55.066+0000 7f6703400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6704105fe0 0x7f67041a17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60502/0 (socket says 172.21.15.19:60502) 2024-05-07T02:17:55.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:55.066+0000 7f6703400700 1 -- 172.21.15.19:0/3274323114 learned_addr learned my addr 172.21.15.19:0/3274323114 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:17:55.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:55.066+0000 7f6703400700 1 -- 172.21.15.19:0/3274323114 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67041a1d00 con 0x7f6704105fe0 2024-05-07T02:17:55.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:55.067+0000 7f6703400700 1 --2- 172.21.15.19:0/3274323114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6704105fe0 0x7f67041a17c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f66f4006010 tx=0x7f66f4005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:55.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:55.067+0000 7f66fbe00700 1 -- 172.21.15.19:0/3274323114 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f66f4026070 con 0x7f6704105fe0 2024-05-07T02:17:55.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:55.067+0000 7f66fbe00700 1 -- 172.21.15.19:0/3274323114 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66f400bba0 con 0x7f6704105fe0 2024-05-07T02:17:55.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:55.067+0000 7f66fbe00700 1 -- 172.21.15.19:0/3274323114 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f66f4003910 con 0x7f6704105fe0 2024-05-07T02:17:55.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:55.067+0000 7f670a400700 1 -- 172.21.15.19:0/3274323114 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67041a1ff0 con 0x7f6704105fe0 2024-05-07T02:17:55.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:55.067+0000 7f670a400700 1 -- 172.21.15.19:0/3274323114 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f670419b9e0 con 0x7f6704105fe0 2024-05-07T02:17:55.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:55.069+0000 7f670a400700 1 -- 172.21.15.19:0/3274323114 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6704107490 con 0x7f6704105fe0 2024-05-07T02:17:55.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:55.069+0000 7f66fbe00700 1 -- 172.21.15.19:0/3274323114 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f66f4029070 con 0x7f6704105fe0 2024-05-07T02:17:55.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:55.070+0000 7f66fbe00700 1 --2- 172.21.15.19:0/3274323114 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f66e803e570 0x7f66e8040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:17:55.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:55.070+0000 7f66fbe00700 1 -- 172.21.15.19:0/3274323114 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f66f401e070 con 0x7f6704105fe0 2024-05-07T02:17:55.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:55.070+0000 7f6702a00700 1 --2- 172.21.15.19:0/3274323114 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f66e803e570 0x7f66e8040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:17:55.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:55.071+0000 7f6702a00700 1 --2- 172.21.15.19:0/3274323114 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f66e803e570 0x7f66e8040a80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f66f0006fd0 tx=0x7f66f0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:17:55.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:55.076+0000 7f66fbe00700 1 -- 172.21.15.19:0/3274323114 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f66f4013070 con 0x7f6704105fe0 2024-05-07T02:17:55.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:17:55.323+0000 7f670a400700 1 -- 172.21.15.19:0/3274323114 --> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi027", "target": ["mon-mgr", ""]}) -- 0x7f670419cf10 con 0x7f66e803e570 2024-05-07T02:17:55.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:55 smithi019 ceph-mon[29781]: Updating smithi019:/var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-07T02:17:55.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:55 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:17:55.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:55 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:17:55.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:55 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:17:56.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:56 smithi019 ceph-mon[29781]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:17:56.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:56 smithi019 ceph-mon[29781]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi027", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:17:57.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:57 smithi019 ceph-mon[29781]: Deploying cephadm binary to smithi027 2024-05-07T02:17:58.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:17:58 smithi019 ceph-mon[29781]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:00.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:00.171+0000 7f66fbe00700 1 -- 172.21.15.19:0/3274323114 <== mgr.14152 v2:172.21.15.19:6800/2059564551 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f670419cf10 con 0x7f66e803e570 2024-05-07T02:18:00.173 INFO:teuthology.orchestra.run.smithi019.stdout:Added host 'smithi027' with addr '172.21.15.27' 2024-05-07T02:18:00.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:00.175+0000 7f670a400700 1 -- 172.21.15.19:0/3274323114 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f66e803e570 msgr2=0x7f66e8040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:18:00.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:00.175+0000 7f670a400700 1 --2- 172.21.15.19:0/3274323114 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f66e803e570 0x7f66e8040a80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f66f0006fd0 tx=0x7f66f0006e40 comp rx=0 tx=0).stop 2024-05-07T02:18:00.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:00.175+0000 7f670a400700 1 -- 172.21.15.19:0/3274323114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6704105fe0 msgr2=0x7f67041a17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:18:00.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:00.175+0000 7f670a400700 1 --2- 172.21.15.19:0/3274323114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6704105fe0 0x7f67041a17c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f66f4006010 tx=0x7f66f4005780 comp rx=0 tx=0).stop 2024-05-07T02:18:00.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:00.176+0000 7f670a400700 1 -- 172.21.15.19:0/3274323114 shutdown_connections 2024-05-07T02:18:00.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:00.176+0000 7f670a400700 1 --2- 172.21.15.19:0/3274323114 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f66e803e570 0x7f66e8040a80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:00.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:00.176+0000 7f670a400700 1 --2- 172.21.15.19:0/3274323114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6704105fe0 0x7f67041a17c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:00.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:00.176+0000 7f670a400700 1 -- 172.21.15.19:0/3274323114 >> 172.21.15.19:0/3274323114 conn(0x7f6704101890 msgr2=0x7f670410bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:18:00.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:00.176+0000 7f670a400700 1 -- 172.21.15.19:0/3274323114 shutdown_connections 2024-05-07T02:18:00.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:00.176+0000 7f670a400700 1 -- 172.21.15.19:0/3274323114 wait complete. 2024-05-07T02:18:00.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:00 smithi019 ceph-mon[29781]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:00.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:00 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:00.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:00 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:18:01.031 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-07T02:18:01.486 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:18:01.632 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:01 smithi019 ceph-mon[29781]: Added host smithi027 2024-05-07T02:18:01.632 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:01 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:01.632 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:01 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:02.643 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:02 smithi019 ceph-mon[29781]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:03.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:03.715+0000 7efd05800700 1 -- 172.21.15.19:0/2446762321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd00103df0 msgr2=0x7efd00104200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:18:03.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:03.715+0000 7efd05800700 1 --2- 172.21.15.19:0/2446762321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd00103df0 0x7efd00104200 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efcf0009910 tx=0x7efcf0009c20 comp rx=0 tx=0).stop 2024-05-07T02:18:03.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:03.715+0000 7efd05800700 1 -- 172.21.15.19:0/2446762321 shutdown_connections 2024-05-07T02:18:03.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:03.715+0000 7efd05800700 1 --2- 172.21.15.19:0/2446762321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd00103df0 0x7efd00104200 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:03.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:03.715+0000 7efd05800700 1 -- 172.21.15.19:0/2446762321 >> 172.21.15.19:0/2446762321 conn(0x7efd000ff6a0 msgr2=0x7efd00101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:18:03.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:03.715+0000 7efd05800700 1 -- 172.21.15.19:0/2446762321 shutdown_connections 2024-05-07T02:18:03.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:03.715+0000 7efd05800700 1 -- 172.21.15.19:0/2446762321 wait complete. 2024-05-07T02:18:03.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:03.717+0000 7efd05800700 1 Processor -- start 2024-05-07T02:18:03.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:03.717+0000 7efd05800700 1 -- start start 2024-05-07T02:18:03.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:03.717+0000 7efd05800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd00103df0 0x7efd0019f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:18:03.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:03.717+0000 7efd05800700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efd0010e0e0 con 0x7efd00103df0 2024-05-07T02:18:03.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:03.718+0000 7efcfea00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd00103df0 0x7efd0019f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:18:03.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:03.718+0000 7efcfea00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd00103df0 0x7efd0019f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40312/0 (socket says 172.21.15.19:40312) 2024-05-07T02:18:03.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:03.718+0000 7efcfea00700 1 -- 172.21.15.19:0/358101002 learned_addr learned my addr 172.21.15.19:0/358101002 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:18:03.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:03.718+0000 7efcfea00700 1 -- 172.21.15.19:0/358101002 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd0019fb30 con 0x7efd00103df0 2024-05-07T02:18:03.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:03.719+0000 7efcfea00700 1 --2- 172.21.15.19:0/358101002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd00103df0 0x7efd0019f5f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efcf0006010 tx=0x7efcf0005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:18:03.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:03.719+0000 7efcf7400700 1 -- 172.21.15.19:0/358101002 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efcf0026070 con 0x7efd00103df0 2024-05-07T02:18:03.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:03.719+0000 7efd05800700 1 -- 172.21.15.19:0/358101002 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd0019fe20 con 0x7efd00103df0 2024-05-07T02:18:03.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:03.719+0000 7efcf7400700 1 -- 172.21.15.19:0/358101002 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcf000bba0 con 0x7efd00103df0 2024-05-07T02:18:03.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:03.719+0000 7efcf7400700 1 -- 172.21.15.19:0/358101002 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efcf0003910 con 0x7efd00103df0 2024-05-07T02:18:03.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:03.719+0000 7efd05800700 1 -- 172.21.15.19:0/358101002 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd00199950 con 0x7efd00103df0 2024-05-07T02:18:03.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:03.721+0000 7efd05800700 1 -- 172.21.15.19:0/358101002 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd001052a0 con 0x7efd00103df0 2024-05-07T02:18:03.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:03.721+0000 7efcf7400700 1 -- 172.21.15.19:0/358101002 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7efcf0029070 con 0x7efd00103df0 2024-05-07T02:18:03.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:03.722+0000 7efcf7400700 1 --2- 172.21.15.19:0/358101002 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7efce403e510 0x7efce4040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:18:03.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:03.722+0000 7efcf7400700 1 -- 172.21.15.19:0/358101002 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7efcf001e070 con 0x7efd00103df0 2024-05-07T02:18:03.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:03.722+0000 7efcfe000700 1 --2- 172.21.15.19:0/358101002 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7efce403e510 0x7efce4040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:18:03.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:03.723+0000 7efcfe000700 1 --2- 172.21.15.19:0/358101002 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7efce403e510 0x7efce4040a20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7efcec006fd0 tx=0x7efcec006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:18:03.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:03.728+0000 7efcf7400700 1 -- 172.21.15.19:0/358101002 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efcf0013070 con 0x7efd00103df0 2024-05-07T02:18:03.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:03.975+0000 7efd05800700 1 -- 172.21.15.19:0/358101002 --> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efd0019ad00 con 0x7efce403e510 2024-05-07T02:18:03.976 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:03 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:03.976 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:03 smithi019 ceph-mon[29781]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:03.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:03.978+0000 7efcf7400700 1 -- 172.21.15.19:0/358101002 <== mgr.14152 v2:172.21.15.19:6800/2059564551 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7efd0019ad00 con 0x7efce403e510 2024-05-07T02:18:03.978 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:18:03.978 INFO:teuthology.orchestra.run.smithi019.stdout:[{"addr": "172.21.15.19", "hostname": "smithi019", "labels": [], "status": ""}, {"addr": "172.21.15.27", "hostname": "smithi027", "labels": [], "status": ""}] 2024-05-07T02:18:03.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:03.981+0000 7efd05800700 1 -- 172.21.15.19:0/358101002 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7efce403e510 msgr2=0x7efce4040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:18:03.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:03.981+0000 7efd05800700 1 --2- 172.21.15.19:0/358101002 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7efce403e510 0x7efce4040a20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7efcec006fd0 tx=0x7efcec006e40 comp rx=0 tx=0).stop 2024-05-07T02:18:03.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:03.981+0000 7efd05800700 1 -- 172.21.15.19:0/358101002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd00103df0 msgr2=0x7efd0019f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:18:03.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:03.981+0000 7efd05800700 1 --2- 172.21.15.19:0/358101002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd00103df0 0x7efd0019f5f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efcf0006010 tx=0x7efcf0005780 comp rx=0 tx=0).stop 2024-05-07T02:18:03.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:03.982+0000 7efd05800700 1 -- 172.21.15.19:0/358101002 shutdown_connections 2024-05-07T02:18:03.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:03.982+0000 7efd05800700 1 --2- 172.21.15.19:0/358101002 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7efce403e510 0x7efce4040a20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:03.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:03.982+0000 7efd05800700 1 --2- 172.21.15.19:0/358101002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd00103df0 0x7efd0019f5f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:03.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:03.982+0000 7efd05800700 1 -- 172.21.15.19:0/358101002 >> 172.21.15.19:0/358101002 conn(0x7efd000ff6a0 msgr2=0x7efd00109920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:18:03.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:03.982+0000 7efd05800700 1 -- 172.21.15.19:0/358101002 shutdown_connections 2024-05-07T02:18:03.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:03.982+0000 7efd05800700 1 -- 172.21.15.19:0/358101002 wait complete. 2024-05-07T02:18:04.674 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi064 2024-05-07T02:18:04.674 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-07T02:18:04.674 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/etc/ceph/ceph.conf 2024-05-07T02:18:04.693 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-07T02:18:04.693 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-07T02:18:04.751 INFO:tasks.cephadm:Adding host smithi064 to orchestrator... 2024-05-07T02:18:04.751 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph orch host add smithi064 2024-05-07T02:18:05.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:04 smithi019 ceph-mon[29781]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-07T02:18:05.201 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:18:06.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:05 smithi019 ceph-mon[29781]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:07.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:07 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:07.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:07 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:07.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:07 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:07.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:07 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:07.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:07 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:18:07.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:07 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:18:07.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:07 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:18:07.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:07 smithi019 ceph-mon[29781]: Updating smithi027:/etc/ceph/ceph.conf 2024-05-07T02:18:07.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:07 smithi019 ceph-mon[29781]: Updating smithi027:/var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-07T02:18:07.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:07.526+0000 7f388b400700 1 -- 172.21.15.19:0/2363566451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3884077cf0 msgr2=0x7f3884078100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:18:07.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:07.526+0000 7f388b400700 1 --2- 172.21.15.19:0/2363566451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3884077cf0 0x7f3884078100 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3878009910 tx=0x7f3878009c20 comp rx=0 tx=0).stop 2024-05-07T02:18:07.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:07.526+0000 7f388b400700 1 -- 172.21.15.19:0/2363566451 shutdown_connections 2024-05-07T02:18:07.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:07.526+0000 7f388b400700 1 --2- 172.21.15.19:0/2363566451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3884077cf0 0x7f3884078100 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:07.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:07.526+0000 7f388b400700 1 -- 172.21.15.19:0/2363566451 >> 172.21.15.19:0/2363566451 conn(0x7f3884101850 msgr2=0x7f3884103ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:18:07.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:07.526+0000 7f388b400700 1 -- 172.21.15.19:0/2363566451 shutdown_connections 2024-05-07T02:18:07.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:07.526+0000 7f388b400700 1 -- 172.21.15.19:0/2363566451 wait complete. 2024-05-07T02:18:07.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:07.528+0000 7f388b400700 1 Processor -- start 2024-05-07T02:18:07.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:07.528+0000 7f388b400700 1 -- start start 2024-05-07T02:18:07.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:07.528+0000 7f388b400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3884077cf0 0x7f388419ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:18:07.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:07.528+0000 7f388b400700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3884114710 con 0x7f3884077cf0 2024-05-07T02:18:07.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:07.528+0000 7f3888a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3884077cf0 0x7f388419ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:18:07.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:07.528+0000 7f3888a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3884077cf0 0x7f388419ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40328/0 (socket says 172.21.15.19:40328) 2024-05-07T02:18:07.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:07.528+0000 7f3888a00700 1 -- 172.21.15.19:0/389066040 learned_addr learned my addr 172.21.15.19:0/389066040 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:18:07.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:07.529+0000 7f3888a00700 1 -- 172.21.15.19:0/389066040 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38841a04b0 con 0x7f3884077cf0 2024-05-07T02:18:07.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:07.529+0000 7f3888a00700 1 --2- 172.21.15.19:0/389066040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3884077cf0 0x7f388419ff70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3878006010 tx=0x7f3878005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:18:07.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:07.529+0000 7f3881600700 1 -- 172.21.15.19:0/389066040 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3878026070 con 0x7f3884077cf0 2024-05-07T02:18:07.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:07.529+0000 7f3881600700 1 -- 172.21.15.19:0/389066040 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f387800bc00 con 0x7f3884077cf0 2024-05-07T02:18:07.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:07.529+0000 7f3881600700 1 -- 172.21.15.19:0/389066040 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3878003910 con 0x7f3884077cf0 2024-05-07T02:18:07.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:07.530+0000 7f388b400700 1 -- 172.21.15.19:0/389066040 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38841a0740 con 0x7f3884077cf0 2024-05-07T02:18:07.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:07.530+0000 7f388b400700 1 -- 172.21.15.19:0/389066040 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38841a0ba0 con 0x7f3884077cf0 2024-05-07T02:18:07.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:07.531+0000 7f388b400700 1 -- 172.21.15.19:0/389066040 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3884079430 con 0x7f3884077cf0 2024-05-07T02:18:07.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:07.531+0000 7f3881600700 1 -- 172.21.15.19:0/389066040 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f3878029070 con 0x7f3884077cf0 2024-05-07T02:18:07.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:07.532+0000 7f3881600700 1 --2- 172.21.15.19:0/389066040 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f386c03e570 0x7f386c040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:18:07.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:07.532+0000 7f3881600700 1 -- 172.21.15.19:0/389066040 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f387801e070 con 0x7f3884077cf0 2024-05-07T02:18:07.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:07.532+0000 7f3883e00700 1 --2- 172.21.15.19:0/389066040 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f386c03e570 0x7f386c040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:18:07.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:07.533+0000 7f3883e00700 1 --2- 172.21.15.19:0/389066040 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f386c03e570 0x7f386c040a80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3870006fd0 tx=0x7f3870006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:18:07.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:07.538+0000 7f3881600700 1 -- 172.21.15.19:0/389066040 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3878013070 con 0x7f3884077cf0 2024-05-07T02:18:07.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:07.784+0000 7f388b400700 1 -- 172.21.15.19:0/389066040 --> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi064", "target": ["mon-mgr", ""]}) -- 0x7f38841a1340 con 0x7f386c03e570 2024-05-07T02:18:08.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:08 smithi019 ceph-mon[29781]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2024-05-07T02:18:08.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:08 smithi019 ceph-mon[29781]: Updating smithi027:/var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-07T02:18:08.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:08 smithi019 ceph-mon[29781]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:08.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:08 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:08.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:08 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:08.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:08 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:09.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:09 smithi019 ceph-mon[29781]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi064", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:18:09.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:09 smithi019 ceph-mon[29781]: Deploying cephadm binary to smithi064 2024-05-07T02:18:10.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:10 smithi019 ceph-mon[29781]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:12.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:12 smithi019 ceph-mon[29781]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:12.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:12.701+0000 7f3881600700 1 -- 172.21.15.19:0/389066040 <== mgr.14152 v2:172.21.15.19:6800/2059564551 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f38841a1340 con 0x7f386c03e570 2024-05-07T02:18:12.702 INFO:teuthology.orchestra.run.smithi019.stdout:Added host 'smithi064' with addr '172.21.15.64' 2024-05-07T02:18:12.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:12.705+0000 7f388b400700 1 -- 172.21.15.19:0/389066040 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f386c03e570 msgr2=0x7f386c040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:18:12.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:12.705+0000 7f388b400700 1 --2- 172.21.15.19:0/389066040 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f386c03e570 0x7f386c040a80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3870006fd0 tx=0x7f3870006e40 comp rx=0 tx=0).stop 2024-05-07T02:18:12.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:12.705+0000 7f388b400700 1 -- 172.21.15.19:0/389066040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3884077cf0 msgr2=0x7f388419ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:18:12.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:12.705+0000 7f388b400700 1 --2- 172.21.15.19:0/389066040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3884077cf0 0x7f388419ff70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3878006010 tx=0x7f3878005780 comp rx=0 tx=0).stop 2024-05-07T02:18:12.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:12.705+0000 7f388b400700 1 -- 172.21.15.19:0/389066040 shutdown_connections 2024-05-07T02:18:12.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:12.705+0000 7f388b400700 1 --2- 172.21.15.19:0/389066040 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f386c03e570 0x7f386c040a80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:12.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:12.705+0000 7f388b400700 1 --2- 172.21.15.19:0/389066040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3884077cf0 0x7f388419ff70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:12.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:12.705+0000 7f388b400700 1 -- 172.21.15.19:0/389066040 >> 172.21.15.19:0/389066040 conn(0x7f3884101850 msgr2=0x7f3884102740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:18:12.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:12.706+0000 7f388b400700 1 -- 172.21.15.19:0/389066040 shutdown_connections 2024-05-07T02:18:12.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:12.706+0000 7f388b400700 1 -- 172.21.15.19:0/389066040 wait complete. 2024-05-07T02:18:13.465 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-07T02:18:13.912 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:18:14.057 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:13 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:14.058 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:13 smithi019 ceph-mon[29781]: Added host smithi064 2024-05-07T02:18:14.058 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:13 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:18:14.058 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:13 smithi019 ceph-mon[29781]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:14.058 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:13 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:15.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:14 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:15.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:15.977+0000 7f1a73200700 1 -- 172.21.15.19:0/754545263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a741099c0 msgr2=0x7f1a74109dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:18:15.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:15.977+0000 7f1a73200700 1 --2- 172.21.15.19:0/754545263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a741099c0 0x7f1a74109dd0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1a64009910 tx=0x7f1a64009c20 comp rx=0 tx=0).stop 2024-05-07T02:18:15.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:15.977+0000 7f1a73200700 1 -- 172.21.15.19:0/754545263 shutdown_connections 2024-05-07T02:18:15.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:15.977+0000 7f1a73200700 1 --2- 172.21.15.19:0/754545263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a741099c0 0x7f1a74109dd0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:15.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:15.977+0000 7f1a73200700 1 -- 172.21.15.19:0/754545263 >> 172.21.15.19:0/754545263 conn(0x7f1a740ff4b0 msgr2=0x7f1a74101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:18:15.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:15.977+0000 7f1a73200700 1 -- 172.21.15.19:0/754545263 shutdown_connections 2024-05-07T02:18:15.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:15.977+0000 7f1a73200700 1 -- 172.21.15.19:0/754545263 wait complete. 2024-05-07T02:18:15.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:15.979+0000 7f1a73200700 1 Processor -- start 2024-05-07T02:18:15.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:15.979+0000 7f1a73200700 1 -- start start 2024-05-07T02:18:15.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:15.980+0000 7f1a73200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a741099c0 0x7f1a74199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:18:15.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:15.980+0000 7f1a73200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7410df10 con 0x7f1a741099c0 2024-05-07T02:18:15.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:15.980+0000 7f1a71e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a741099c0 0x7f1a74199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:18:15.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:15.980+0000 7f1a71e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a741099c0 0x7f1a74199790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49304/0 (socket says 172.21.15.19:49304) 2024-05-07T02:18:15.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:15.980+0000 7f1a71e00700 1 -- 172.21.15.19:0/2680473077 learned_addr learned my addr 172.21.15.19:0/2680473077 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:18:15.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:15.980+0000 7f1a71e00700 1 -- 172.21.15.19:0/2680473077 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a74199cd0 con 0x7f1a741099c0 2024-05-07T02:18:15.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:15.981+0000 7f1a71e00700 1 --2- 172.21.15.19:0/2680473077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a741099c0 0x7f1a74199790 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1a640092e0 tx=0x7f1a64005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:18:15.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:15.981+0000 7f1a6aa00700 1 -- 172.21.15.19:0/2680473077 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a64026070 con 0x7f1a741099c0 2024-05-07T02:18:15.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:15.981+0000 7f1a6aa00700 1 -- 172.21.15.19:0/2680473077 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a6400b6f0 con 0x7f1a741099c0 2024-05-07T02:18:15.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:15.981+0000 7f1a6aa00700 1 -- 172.21.15.19:0/2680473077 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a640037f0 con 0x7f1a741099c0 2024-05-07T02:18:15.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:15.981+0000 7f1a73200700 1 -- 172.21.15.19:0/2680473077 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a74199fc0 con 0x7f1a741099c0 2024-05-07T02:18:15.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:15.981+0000 7f1a73200700 1 -- 172.21.15.19:0/2680473077 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a7419dfc0 con 0x7f1a741099c0 2024-05-07T02:18:15.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:15.982+0000 7f1a73200700 1 -- 172.21.15.19:0/2680473077 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a741050d0 con 0x7f1a741099c0 2024-05-07T02:18:15.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:15.983+0000 7f1a6aa00700 1 -- 172.21.15.19:0/2680473077 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f1a64029070 con 0x7f1a741099c0 2024-05-07T02:18:15.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:15.983+0000 7f1a6aa00700 1 --2- 172.21.15.19:0/2680473077 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f1a5803e570 0x7f1a58040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:18:15.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:15.983+0000 7f1a6aa00700 1 -- 172.21.15.19:0/2680473077 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1a6401e070 con 0x7f1a741099c0 2024-05-07T02:18:15.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:15.984+0000 7f1a71400700 1 --2- 172.21.15.19:0/2680473077 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f1a5803e570 0x7f1a58040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:18:15.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:15.984+0000 7f1a71400700 1 --2- 172.21.15.19:0/2680473077 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f1a5803e570 0x7f1a58040a80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1a60006fd0 tx=0x7f1a60006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:18:15.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:15.989+0000 7f1a6aa00700 1 -- 172.21.15.19:0/2680473077 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1a64013070 con 0x7f1a741099c0 2024-05-07T02:18:16.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:16.233+0000 7f1a73200700 1 -- 172.21.15.19:0/2680473077 --> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1a7419aaf0 con 0x7f1a5803e570 2024-05-07T02:18:16.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:16.235+0000 7f1a6aa00700 1 -- 172.21.15.19:0/2680473077 <== mgr.14152 v2:172.21.15.19:6800/2059564551 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+238 (secure 0 0 0) 0x7f1a7419aaf0 con 0x7f1a5803e570 2024-05-07T02:18:16.235 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:18:16.236 INFO:teuthology.orchestra.run.smithi019.stdout:[{"addr": "172.21.15.19", "hostname": "smithi019", "labels": [], "status": ""}, {"addr": "172.21.15.27", "hostname": "smithi027", "labels": [], "status": ""}, {"addr": "172.21.15.64", "hostname": "smithi064", "labels": [], "status": ""}] 2024-05-07T02:18:16.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:16.238+0000 7f1a73200700 1 -- 172.21.15.19:0/2680473077 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f1a5803e570 msgr2=0x7f1a58040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:18:16.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:16.238+0000 7f1a73200700 1 --2- 172.21.15.19:0/2680473077 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f1a5803e570 0x7f1a58040a80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1a60006fd0 tx=0x7f1a60006e40 comp rx=0 tx=0).stop 2024-05-07T02:18:16.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:16.238+0000 7f1a73200700 1 -- 172.21.15.19:0/2680473077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a741099c0 msgr2=0x7f1a74199790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:18:16.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:16.238+0000 7f1a73200700 1 --2- 172.21.15.19:0/2680473077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a741099c0 0x7f1a74199790 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1a640092e0 tx=0x7f1a64005780 comp rx=0 tx=0).stop 2024-05-07T02:18:16.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:16.239+0000 7f1a73200700 1 -- 172.21.15.19:0/2680473077 shutdown_connections 2024-05-07T02:18:16.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:16.239+0000 7f1a73200700 1 --2- 172.21.15.19:0/2680473077 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f1a5803e570 0x7f1a58040a80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:16.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:16.239+0000 7f1a73200700 1 --2- 172.21.15.19:0/2680473077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a741099c0 0x7f1a74199790 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:16.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:16.239+0000 7f1a73200700 1 -- 172.21.15.19:0/2680473077 >> 172.21.15.19:0/2680473077 conn(0x7f1a740ff4b0 msgr2=0x7f1a741004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:18:16.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:16.239+0000 7f1a73200700 1 -- 172.21.15.19:0/2680473077 shutdown_connections 2024-05-07T02:18:16.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:16.239+0000 7f1a73200700 1 -- 172.21.15.19:0/2680473077 wait complete. 2024-05-07T02:18:16.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:16 smithi019 ceph-mon[29781]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:16.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:16 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:17.011 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-07T02:18:17.011 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph osd crush tunables default 2024-05-07T02:18:17.462 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:18:17.806 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:17 smithi019 ceph-mon[29781]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-07T02:18:18.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:18 smithi019 ceph-mon[29781]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:19.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:19.588+0000 7fceb9800700 1 -- 172.21.15.19:0/57595461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fceb4075d30 msgr2=0x7fceb4076140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:18:19.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:19.588+0000 7fceb9800700 1 --2- 172.21.15.19:0/57595461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fceb4075d30 0x7fceb4076140 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcea4009910 tx=0x7fcea4009c20 comp rx=0 tx=0).stop 2024-05-07T02:18:19.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:19.588+0000 7fceb9800700 1 -- 172.21.15.19:0/57595461 shutdown_connections 2024-05-07T02:18:19.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:19.588+0000 7fceb9800700 1 --2- 172.21.15.19:0/57595461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fceb4075d30 0x7fceb4076140 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:19.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:19.588+0000 7fceb9800700 1 -- 172.21.15.19:0/57595461 >> 172.21.15.19:0/57595461 conn(0x7fceb40ff6a0 msgr2=0x7fceb4101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:18:19.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:19.589+0000 7fceb9800700 1 -- 172.21.15.19:0/57595461 shutdown_connections 2024-05-07T02:18:19.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:19.589+0000 7fceb9800700 1 -- 172.21.15.19:0/57595461 wait complete. 2024-05-07T02:18:19.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:19.590+0000 7fceb9800700 1 Processor -- start 2024-05-07T02:18:19.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:19.590+0000 7fceb9800700 1 -- start start 2024-05-07T02:18:19.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:19.591+0000 7fceb9800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fceb4075d30 0x7fceb41a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:18:19.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:19.591+0000 7fceb9800700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fceb4110340 con 0x7fceb4075d30 2024-05-07T02:18:19.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:19.591+0000 7fceb2a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fceb4075d30 0x7fceb41a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:18:19.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:19.591+0000 7fceb2a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fceb4075d30 0x7fceb41a17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45190/0 (socket says 172.21.15.19:45190) 2024-05-07T02:18:19.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:19.591+0000 7fceb2a00700 1 -- 172.21.15.19:0/188656138 learned_addr learned my addr 172.21.15.19:0/188656138 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:18:19.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:19.592+0000 7fceb2a00700 1 -- 172.21.15.19:0/188656138 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb41a1d00 con 0x7fceb4075d30 2024-05-07T02:18:19.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:19.592+0000 7fceb2a00700 1 --2- 172.21.15.19:0/188656138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fceb4075d30 0x7fceb41a17c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcea4006010 tx=0x7fcea4005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:18:19.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:19.592+0000 7fceabe00700 1 -- 172.21.15.19:0/188656138 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcea4026070 con 0x7fceb4075d30 2024-05-07T02:18:19.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:19.592+0000 7fceabe00700 1 -- 172.21.15.19:0/188656138 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcea400bc00 con 0x7fceb4075d30 2024-05-07T02:18:19.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:19.592+0000 7fceb9800700 1 -- 172.21.15.19:0/188656138 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceb41a1f90 con 0x7fceb4075d30 2024-05-07T02:18:19.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:19.592+0000 7fceabe00700 1 -- 172.21.15.19:0/188656138 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcea4003910 con 0x7fceb4075d30 2024-05-07T02:18:19.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:19.592+0000 7fceb9800700 1 -- 172.21.15.19:0/188656138 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceb419ba30 con 0x7fceb4075d30 2024-05-07T02:18:19.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:19.594+0000 7fceabe00700 1 -- 172.21.15.19:0/188656138 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fcea4029070 con 0x7fceb4075d30 2024-05-07T02:18:19.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:19.594+0000 7fceb9800700 1 -- 172.21.15.19:0/188656138 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fceb40772a0 con 0x7fceb4075d30 2024-05-07T02:18:19.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:19.594+0000 7fceabe00700 1 --2- 172.21.15.19:0/188656138 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fce9803e570 0x7fce98040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:18:19.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:19.595+0000 7fceabe00700 1 -- 172.21.15.19:0/188656138 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcea401e070 con 0x7fceb4075d30 2024-05-07T02:18:19.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:19.595+0000 7fceaa000700 1 --2- 172.21.15.19:0/188656138 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fce9803e570 0x7fce98040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:18:19.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:19.596+0000 7fceaa000700 1 --2- 172.21.15.19:0/188656138 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fce9803e570 0x7fce98040a80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fce9c006fd0 tx=0x7fce9c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:18:19.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:19.601+0000 7fceabe00700 1 -- 172.21.15.19:0/188656138 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcea4013070 con 0x7fceb4075d30 2024-05-07T02:18:19.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:19.848+0000 7fceb9800700 1 -- 172.21.15.19:0/188656138 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fceb4064e20 con 0x7fceb4075d30 2024-05-07T02:18:19.848 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:19 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:19.848 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:19 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:19.848 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:19 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:19.849 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:19 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:19.849 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:19 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi064", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:18:19.849 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:19 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:18:19.849 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:19 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:18:19.849 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:19 smithi019 ceph-mon[29781]: Updating smithi064:/etc/ceph/ceph.conf 2024-05-07T02:18:19.849 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:19 smithi019 ceph-mon[29781]: Updating smithi064:/var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-07T02:18:19.849 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:19 smithi019 ceph-mon[29781]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:19.849 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:19 smithi019 ceph-mon[29781]: Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2024-05-07T02:18:20.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:20.813+0000 7fceabe00700 1 -- 172.21.15.19:0/188656138 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fcea402ada0 con 0x7fceb4075d30 2024-05-07T02:18:20.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:20.817+0000 7fceb9800700 1 -- 172.21.15.19:0/188656138 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fce9803e570 msgr2=0x7fce98040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:18:20.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:20.817+0000 7fceb9800700 1 --2- 172.21.15.19:0/188656138 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fce9803e570 0x7fce98040a80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fce9c006fd0 tx=0x7fce9c006e40 comp rx=0 tx=0).stop 2024-05-07T02:18:20.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:20.817+0000 7fceb9800700 1 -- 172.21.15.19:0/188656138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fceb4075d30 msgr2=0x7fceb41a17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:18:20.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:20.817+0000 7fceb9800700 1 --2- 172.21.15.19:0/188656138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fceb4075d30 0x7fceb41a17c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcea4006010 tx=0x7fcea4005780 comp rx=0 tx=0).stop 2024-05-07T02:18:20.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:20.817+0000 7fceb9800700 1 -- 172.21.15.19:0/188656138 shutdown_connections 2024-05-07T02:18:20.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:20.817+0000 7fceb9800700 1 --2- 172.21.15.19:0/188656138 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fce9803e570 0x7fce98040a80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:20.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:20.817+0000 7fceb9800700 1 --2- 172.21.15.19:0/188656138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fceb4075d30 0x7fceb41a17c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:20.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:20.817+0000 7fceb9800700 1 -- 172.21.15.19:0/188656138 >> 172.21.15.19:0/188656138 conn(0x7fceb40ff6a0 msgr2=0x7fceb410c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:18:20.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:20.818+0000 7fceb9800700 1 -- 172.21.15.19:0/188656138 shutdown_connections 2024-05-07T02:18:20.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:20.818+0000 7fceb9800700 1 -- 172.21.15.19:0/188656138 wait complete. 2024-05-07T02:18:20.819 INFO:teuthology.orchestra.run.smithi019.stderr:adjusted tunables profile to default 2024-05-07T02:18:21.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:20 smithi019 ceph-mon[29781]: Updating smithi064:/var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-07T02:18:21.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:20 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/188656138' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-07T02:18:21.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:20 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:21.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:20 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:21.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:20 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:21.535 INFO:tasks.cephadm:Adding mon.a on smithi019 2024-05-07T02:18:21.535 INFO:tasks.cephadm:Adding mon.b on smithi027 2024-05-07T02:18:21.535 INFO:tasks.cephadm:Adding mon.c on smithi064 2024-05-07T02:18:21.535 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph orch apply mon '3;smithi019:172.21.15.19=a;smithi027:172.21.15.27=b;smithi064:172.21.15.64=c' 2024-05-07T02:18:21.784 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-07T02:18:22.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:21 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/188656138' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-07T02:18:22.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:21 smithi019 ceph-mon[29781]: osdmap e4: 0 total, 0 up, 0 in 2024-05-07T02:18:22.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:21 smithi019 ceph-mon[29781]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:23.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:23.256+0000 7fa9b4505700 1 -- 172.21.15.64:0/3411626782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9ac104d50 msgr2=0x7fa9ac105160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:18:23.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:23.256+0000 7fa9b4505700 1 --2- 172.21.15.64:0/3411626782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9ac104d50 0x7fa9ac105160 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa9a400b0a0 tx=0x7fa9a400b3b0 comp rx=0 tx=0).stop 2024-05-07T02:18:23.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:23.256+0000 7fa9b4505700 1 -- 172.21.15.64:0/3411626782 shutdown_connections 2024-05-07T02:18:23.256 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:23.256+0000 7fa9b4505700 1 --2- 172.21.15.64:0/3411626782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9ac104d50 0x7fa9ac105160 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:23.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:23.256+0000 7fa9b4505700 1 -- 172.21.15.64:0/3411626782 >> 172.21.15.64:0/3411626782 conn(0x7fa9ac0785b0 msgr2=0x7fa9ac078a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:18:23.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:23.256+0000 7fa9b4505700 1 -- 172.21.15.64:0/3411626782 shutdown_connections 2024-05-07T02:18:23.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:23.256+0000 7fa9b4505700 1 -- 172.21.15.64:0/3411626782 wait complete. 2024-05-07T02:18:23.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:23.257+0000 7fa9b4505700 1 Processor -- start 2024-05-07T02:18:23.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:23.257+0000 7fa9b4505700 1 -- start start 2024-05-07T02:18:23.257 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:23.257+0000 7fa9b4505700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9ac104d50 0x7fa9ac19f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:18:23.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:23.257+0000 7fa9b4505700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ac10e140 con 0x7fa9ac104d50 2024-05-07T02:18:23.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:23.258+0000 7fa9b22a1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9ac104d50 0x7fa9ac19f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:18:23.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:23.258+0000 7fa9b22a1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9ac104d50 0x7fa9ac19f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.64:45572/0 (socket says 172.21.15.64:45572) 2024-05-07T02:18:23.258 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:23.258+0000 7fa9b22a1700 1 -- 172.21.15.64:0/3195959376 learned_addr learned my addr 172.21.15.64:0/3195959376 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-07T02:18:23.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:23.258+0000 7fa9b22a1700 1 -- 172.21.15.64:0/3195959376 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9ac19fb00 con 0x7fa9ac104d50 2024-05-07T02:18:23.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:23.259+0000 7fa9b22a1700 1 --2- 172.21.15.64:0/3195959376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9ac104d50 0x7fa9ac19f5c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4015040 tx=0x7fa9a4009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:18:23.259 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:23.259+0000 7fa9ab7fe700 1 -- 172.21.15.64:0/3195959376 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9a4021020 con 0x7fa9ac104d50 2024-05-07T02:18:23.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:23.259+0000 7fa9ab7fe700 1 -- 172.21.15.64:0/3195959376 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9a4004000 con 0x7fa9ac104d50 2024-05-07T02:18:23.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:23.259+0000 7fa9b4505700 1 -- 172.21.15.64:0/3195959376 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9ac19fd90 con 0x7fa9ac104d50 2024-05-07T02:18:23.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:23.259+0000 7fa9ab7fe700 1 -- 172.21.15.64:0/3195959376 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9a4017040 con 0x7fa9ac104d50 2024-05-07T02:18:23.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:23.259+0000 7fa9b4505700 1 -- 172.21.15.64:0/3195959376 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9ac199750 con 0x7fa9ac104d50 2024-05-07T02:18:23.260 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:23.260+0000 7fa9b4505700 1 -- 172.21.15.64:0/3195959376 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9ac06b6d0 con 0x7fa9ac104d50 2024-05-07T02:18:23.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:23.261+0000 7fa9ab7fe700 1 -- 172.21.15.64:0/3195959376 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fa9a4024020 con 0x7fa9ac104d50 2024-05-07T02:18:23.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:23.261+0000 7fa9ab7fe700 1 --2- 172.21.15.64:0/3195959376 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fa99403e510 0x7fa994040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:18:23.261 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:23.261+0000 7fa9ab7fe700 1 -- 172.21.15.64:0/3195959376 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa9a400f050 con 0x7fa9ac104d50 2024-05-07T02:18:23.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:23.262+0000 7fa9b1aa0700 1 --2- 172.21.15.64:0/3195959376 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fa99403e510 0x7fa994040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:18:23.262 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:23.263+0000 7fa9b1aa0700 1 --2- 172.21.15.64:0/3195959376 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fa99403e510 0x7fa994040a20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa9a0006fd0 tx=0x7fa9a0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:18:23.278 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:23.278+0000 7fa9ab7fe700 1 -- 172.21.15.64:0/3195959376 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa9a402b020 con 0x7fa9ac104d50 2024-05-07T02:18:23.504 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:23.504+0000 7fa9b4505700 1 -- 172.21.15.64:0/3195959376 --> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi019:172.21.15.19=a;smithi027:172.21.15.27=b;smithi064:172.21.15.64=c", "target": ["mon-mgr", ""]}) -- 0x7fa9ac19ac60 con 0x7fa99403e510 2024-05-07T02:18:23.522 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:23.523+0000 7fa9ab7fe700 1 -- 172.21.15.64:0/3195959376 <== mgr.14152 v2:172.21.15.19:6800/2059564551 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fa9ac19ac60 con 0x7fa99403e510 2024-05-07T02:18:23.523 INFO:teuthology.orchestra.run.smithi064.stdout:Scheduled mon update... 2024-05-07T02:18:23.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:23.524+0000 7fa9b4505700 1 -- 172.21.15.64:0/3195959376 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fa99403e510 msgr2=0x7fa994040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:18:23.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:23.524+0000 7fa9b4505700 1 --2- 172.21.15.64:0/3195959376 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fa99403e510 0x7fa994040a20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa9a0006fd0 tx=0x7fa9a0006e40 comp rx=0 tx=0).stop 2024-05-07T02:18:23.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:23.524+0000 7fa9b4505700 1 -- 172.21.15.64:0/3195959376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9ac104d50 msgr2=0x7fa9ac19f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:18:23.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:23.524+0000 7fa9b4505700 1 --2- 172.21.15.64:0/3195959376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9ac104d50 0x7fa9ac19f5c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4015040 tx=0x7fa9a4009800 comp rx=0 tx=0).stop 2024-05-07T02:18:23.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:23.524+0000 7fa9b4505700 1 -- 172.21.15.64:0/3195959376 shutdown_connections 2024-05-07T02:18:23.524 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:23.524+0000 7fa9b4505700 1 --2- 172.21.15.64:0/3195959376 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fa99403e510 0x7fa994040a20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:23.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:23.524+0000 7fa9b4505700 1 --2- 172.21.15.64:0/3195959376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9ac104d50 0x7fa9ac19f5c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:23.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:23.524+0000 7fa9b4505700 1 -- 172.21.15.64:0/3195959376 >> 172.21.15.64:0/3195959376 conn(0x7fa9ac0785b0 msgr2=0x7fa9ac105ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:18:23.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:23.524+0000 7fa9b4505700 1 -- 172.21.15.64:0/3195959376 shutdown_connections 2024-05-07T02:18:23.525 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:23.524+0000 7fa9b4505700 1 -- 172.21.15.64:0/3195959376 wait complete. 2024-05-07T02:18:24.089 DEBUG:teuthology.orchestra.run.smithi027:mon.b> sudo journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mon.b.service 2024-05-07T02:18:24.091 DEBUG:teuthology.orchestra.run.smithi064:mon.c> sudo journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mon.c.service 2024-05-07T02:18:24.094 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-07T02:18:24.094 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-07T02:18:24.850 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:24 smithi019 ceph-mon[29781]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:24.851 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:24 smithi019 ceph-mon[29781]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi019:172.21.15.19=a;smithi027:172.21.15.27=b;smithi064:172.21.15.64=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:18:24.851 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:24 smithi019 ceph-mon[29781]: Saving service mon spec with placement smithi019:172.21.15.19=a;smithi027:172.21.15.27=b;smithi064:172.21.15.64=c;count:3 2024-05-07T02:18:24.851 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:24 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:24.851 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:24 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:18:24.851 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:24 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:18:24.851 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:24 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:18:24.851 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:24 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:24.851 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:24 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-07T02:18:24.851 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:24 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:18:24.851 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:24 smithi019 ceph-mon[29781]: Deploying daemon mon.c on smithi064 2024-05-07T02:18:24.953 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-07T02:18:26.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:26 smithi019 ceph-mon[29781]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:27.517 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:26.911+0000 7f7ce5cdc700 1 -- 172.21.15.64:0/2458743258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ce0102440 msgr2=0x7f7ce0102850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:18:27.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:26.911+0000 7f7ce5cdc700 1 --2- 172.21.15.64:0/2458743258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ce0102440 0x7f7ce0102850 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7cd0009910 tx=0x7f7cd0009c20 comp rx=0 tx=0).stop 2024-05-07T02:18:27.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:26.911+0000 7f7ce5cdc700 1 -- 172.21.15.64:0/2458743258 shutdown_connections 2024-05-07T02:18:27.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:26.911+0000 7f7ce5cdc700 1 --2- 172.21.15.64:0/2458743258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ce0102440 0x7f7ce0102850 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:27.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:26.911+0000 7f7ce5cdc700 1 -- 172.21.15.64:0/2458743258 >> 172.21.15.64:0/2458743258 conn(0x7f7ce00fde30 msgr2=0x7f7ce0100250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:18:27.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:26.911+0000 7f7ce5cdc700 1 -- 172.21.15.64:0/2458743258 shutdown_connections 2024-05-07T02:18:27.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:26.912+0000 7f7ce5cdc700 1 -- 172.21.15.64:0/2458743258 wait complete. 2024-05-07T02:18:27.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:26.913+0000 7f7ce5cdc700 1 Processor -- start 2024-05-07T02:18:27.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:26.913+0000 7f7ce5cdc700 1 -- start start 2024-05-07T02:18:27.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:26.914+0000 7f7ce5cdc700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ce0102440 0x7f7ce019e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:18:27.518 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:26.914+0000 7f7ce5cdc700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce010c730 con 0x7f7ce0102440 2024-05-07T02:18:27.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:26.914+0000 7f7cdf7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ce0102440 0x7f7ce019e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:18:27.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:26.914+0000 7f7cdf7fe700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ce0102440 0x7f7ce019e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.64:38596/0 (socket says 172.21.15.64:38596) 2024-05-07T02:18:27.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:26.914+0000 7f7cdf7fe700 1 -- 172.21.15.64:0/2317022933 learned_addr learned my addr 172.21.15.64:0/2317022933 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-07T02:18:27.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:26.914+0000 7f7cdf7fe700 1 -- 172.21.15.64:0/2317022933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ce019eaf0 con 0x7f7ce0102440 2024-05-07T02:18:27.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:26.915+0000 7f7cdf7fe700 1 --2- 172.21.15.64:0/2317022933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ce0102440 0x7f7ce019e5b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7cd00056d0 tx=0x7f7cd00057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:18:27.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:26.915+0000 7f7cdcff9700 1 -- 172.21.15.64:0/2317022933 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7cd0026070 con 0x7f7ce0102440 2024-05-07T02:18:27.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:26.915+0000 7f7ce5cdc700 1 -- 172.21.15.64:0/2317022933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ce019ede0 con 0x7f7ce0102440 2024-05-07T02:18:27.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:26.915+0000 7f7cdcff9700 1 -- 172.21.15.64:0/2317022933 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cd000bba0 con 0x7f7ce0102440 2024-05-07T02:18:27.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:26.915+0000 7f7cdcff9700 1 -- 172.21.15.64:0/2317022933 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7cd0003920 con 0x7f7ce0102440 2024-05-07T02:18:27.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:26.915+0000 7f7ce5cdc700 1 -- 172.21.15.64:0/2317022933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ce0198910 con 0x7f7ce0102440 2024-05-07T02:18:27.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:26.916+0000 7f7ce5cdc700 1 -- 172.21.15.64:0/2317022933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ce01038f0 con 0x7f7ce0102440 2024-05-07T02:18:27.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:26.916+0000 7f7cdcff9700 1 -- 172.21.15.64:0/2317022933 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f7cd0029070 con 0x7f7ce0102440 2024-05-07T02:18:27.519 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:26.916+0000 7f7cdcff9700 1 --2- 172.21.15.64:0/2317022933 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f7cc803e510 0x7f7cc8040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:18:27.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:26.917+0000 7f7cdcff9700 1 -- 172.21.15.64:0/2317022933 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7cd001e070 con 0x7f7ce0102440 2024-05-07T02:18:27.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:26.917+0000 7f7cdeffd700 1 --2- 172.21.15.64:0/2317022933 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f7cc803e510 0x7f7cc8040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:18:27.520 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:26.918+0000 7f7cdeffd700 1 --2- 172.21.15.64:0/2317022933 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f7cc803e510 0x7f7cc8040a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4006fd0 tx=0x7f7cd4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:18:27.520 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-07T02:18:27.520 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":1,"fsid":"9ff84c16-0c17-11ef-bc97-c7b262605968","modified":"2024-05-07T02:16:14.906770Z","created":"2024-05-07T02:16:14.906770Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-07T02:18:27.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:26.925+0000 7f7cdcff9700 1 -- 172.21.15.64:0/2317022933 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7cd0013070 con 0x7f7ce0102440 2024-05-07T02:18:27.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:27.231+0000 7f7ce5cdc700 1 -- 172.21.15.64:0/2317022933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f7ce019f010 con 0x7f7ce0102440 2024-05-07T02:18:27.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:27.232+0000 7f7cdcff9700 1 -- 172.21.15.64:0/2317022933 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f7cd0030020 con 0x7f7ce0102440 2024-05-07T02:18:27.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:27.234+0000 7f7ce5cdc700 1 -- 172.21.15.64:0/2317022933 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f7cc803e510 msgr2=0x7f7cc8040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:18:27.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:27.234+0000 7f7ce5cdc700 1 --2- 172.21.15.64:0/2317022933 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f7cc803e510 0x7f7cc8040a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4006fd0 tx=0x7f7cd4006e40 comp rx=0 tx=0).stop 2024-05-07T02:18:27.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:27.234+0000 7f7ce5cdc700 1 -- 172.21.15.64:0/2317022933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ce0102440 msgr2=0x7f7ce019e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:18:27.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:27.234+0000 7f7ce5cdc700 1 --2- 172.21.15.64:0/2317022933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ce0102440 0x7f7ce019e5b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7cd00056d0 tx=0x7f7cd00057b0 comp rx=0 tx=0).stop 2024-05-07T02:18:27.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:27.234+0000 7f7ce5cdc700 1 -- 172.21.15.64:0/2317022933 shutdown_connections 2024-05-07T02:18:27.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:27.234+0000 7f7ce5cdc700 1 --2- 172.21.15.64:0/2317022933 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f7cc803e510 0x7f7cc8040a20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:27.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:27.234+0000 7f7ce5cdc700 1 --2- 172.21.15.64:0/2317022933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ce0102440 0x7f7ce019e5b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:27.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:27.234+0000 7f7ce5cdc700 1 -- 172.21.15.64:0/2317022933 >> 172.21.15.64:0/2317022933 conn(0x7f7ce00fde30 msgr2=0x7f7ce010a570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:18:27.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:27.235+0000 7f7ce5cdc700 1 -- 172.21.15.64:0/2317022933 shutdown_connections 2024-05-07T02:18:27.521 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:27.235+0000 7f7ce5cdc700 1 -- 172.21.15.64:0/2317022933 wait complete. 2024-05-07T02:18:27.521 INFO:teuthology.orchestra.run.smithi064.stderr:dumped monmap epoch 1 2024-05-07T02:18:27.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:27 smithi019 ceph-mon[29781]: from='client.? 172.21.15.64:0/2317022933' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-07T02:18:28.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:28 smithi019 ceph-mon[29781]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:29.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:29 smithi064 ceph-mon[34699]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-07T02:18:29.359 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-07T02:18:29.359 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-07T02:18:29.600 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.c/config 2024-05-07T02:18:34.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.540+0000 7f8437df1700 1 -- 172.21.15.64:0/2267784480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8430102dd0 msgr2=0x7f841c005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:18:34.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.540+0000 7f8437df1700 1 --2- 172.21.15.64:0/2267784480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8430102dd0 0x7f841c005610 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8424005fa0 tx=0x7f842400f4a0 comp rx=0 tx=0).stop 2024-05-07T02:18:34.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.541+0000 7f8437df1700 1 -- 172.21.15.64:0/2267784480 shutdown_connections 2024-05-07T02:18:34.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.541+0000 7f8437df1700 1 --2- 172.21.15.64:0/2267784480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8430102dd0 0x7f841c005610 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:34.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.541+0000 7f8437df1700 1 -- 172.21.15.64:0/2267784480 >> 172.21.15.64:0/2267784480 conn(0x7f84300797e0 msgr2=0x7f8430079c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:18:34.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.541+0000 7f8437df1700 1 -- 172.21.15.64:0/2267784480 shutdown_connections 2024-05-07T02:18:34.542 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.541+0000 7f8437df1700 1 -- 172.21.15.64:0/2267784480 wait complete. 2024-05-07T02:18:34.542 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:34 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-07T02:18:34.542 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:34 smithi064 ceph-mon[34699]: mon.a calling monitor election 2024-05-07T02:18:34.542 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:34 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-07T02:18:34.542 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:34 smithi064 ceph-mon[34699]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:34.542 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:34 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-07T02:18:34.542 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:34 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-07T02:18:34.542 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:34 smithi064 ceph-mon[34699]: mon.c calling monitor election 2024-05-07T02:18:34.542 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:34 smithi064 ceph-mon[34699]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:34.543 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:34 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-07T02:18:34.543 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:34 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-07T02:18:34.543 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:34 smithi064 ceph-mon[34699]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:34.543 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:34 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-07T02:18:34.543 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:34 smithi064 ceph-mon[34699]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-07T02:18:34.543 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:34 smithi064 ceph-mon[34699]: monmap epoch 2 2024-05-07T02:18:34.543 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:34 smithi064 ceph-mon[34699]: fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 2024-05-07T02:18:34.543 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:34 smithi064 ceph-mon[34699]: last_changed 2024-05-07T02:18:29.166142+0000 2024-05-07T02:18:34.543 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:34 smithi064 ceph-mon[34699]: created 2024-05-07T02:16:14.906770+0000 2024-05-07T02:18:34.543 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:34 smithi064 ceph-mon[34699]: min_mon_release 19 (squid) 2024-05-07T02:18:34.543 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:34 smithi064 ceph-mon[34699]: election_strategy: 1 2024-05-07T02:18:34.543 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:34 smithi064 ceph-mon[34699]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-05-07T02:18:34.543 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:34 smithi064 ceph-mon[34699]: 1: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.c 2024-05-07T02:18:34.543 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:34 smithi064 ceph-mon[34699]: fsmap 2024-05-07T02:18:34.543 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:34 smithi064 ceph-mon[34699]: osdmap e4: 0 total, 0 up, 0 in 2024-05-07T02:18:34.544 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:34 smithi064 ceph-mon[34699]: mgrmap e13: x(active, since 62s) 2024-05-07T02:18:34.544 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:34 smithi064 ceph-mon[34699]: overall HEALTH_OK 2024-05-07T02:18:34.544 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:34 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:34.544 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:34 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:34.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.542+0000 7f8437df1700 1 Processor -- start 2024-05-07T02:18:34.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.542+0000 7f8437df1700 1 -- start start 2024-05-07T02:18:34.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.542+0000 7f8437df1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8430102dd0 0x7f843019f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:18:34.545 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.542+0000 7f8437df1700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f843019fc00 0x7f84301998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:18:34.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.542+0000 7f8437df1700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f843010ab10 con 0x7f843019fc00 2024-05-07T02:18:34.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.543+0000 7f8437df1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f843010a990 con 0x7f8430102dd0 2024-05-07T02:18:34.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.543+0000 7f8435b8d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8430102dd0 0x7f843019f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:18:34.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.543+0000 7f8435b8d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8430102dd0 0x7f843019f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44868/0 (socket says 172.21.15.64:44868) 2024-05-07T02:18:34.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.543+0000 7f8435b8d700 1 -- 172.21.15.64:0/2888829481 learned_addr learned my addr 172.21.15.64:0/2888829481 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-07T02:18:34.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.543+0000 7f8435b8d700 1 -- 172.21.15.64:0/2888829481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8430102dd0 msgr2=0x7f843019f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-07T02:18:34.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.543+0000 7f8435b8d700 1 -- 172.21.15.64:0/2888829481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8430102dd0 msgr2=0x7f843019f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-07T02:18:34.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.543+0000 7f8435b8d700 1 --2- 172.21.15.64:0/2888829481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8430102dd0 0x7f843019f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-07T02:18:34.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.543+0000 7f8435b8d700 1 --2- 172.21.15.64:0/2888829481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8430102dd0 0x7f843019f6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-07T02:18:34.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.543+0000 7f843538c700 1 --2- 172.21.15.64:0/2888829481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f843019fc00 0x7f84301998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:18:34.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.544+0000 7f843538c700 1 -- 172.21.15.64:0/2888829481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8430102dd0 msgr2=0x7f843019f6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-07T02:18:34.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.544+0000 7f843538c700 1 --2- 172.21.15.64:0/2888829481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8430102dd0 0x7f843019f6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:34.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.544+0000 7f843538c700 1 -- 172.21.15.64:0/2888829481 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84301a01d0 con 0x7f843019fc00 2024-05-07T02:18:34.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.544+0000 7f843538c700 1 --2- 172.21.15.64:0/2888829481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f843019fc00 0x7f84301998e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f842c00b4d0 tx=0x7f842c00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:18:34.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.545+0000 7f8422ffd700 1 -- 172.21.15.64:0/2888829481 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f842c011630 con 0x7f843019fc00 2024-05-07T02:18:34.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.545+0000 7f8437df1700 1 -- 172.21.15.64:0/2888829481 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8430199e80 con 0x7f843019fc00 2024-05-07T02:18:34.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.545+0000 7f8437df1700 1 -- 172.21.15.64:0/2888829481 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f843019a410 con 0x7f843019fc00 2024-05-07T02:18:34.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.545+0000 7f8422ffd700 1 -- 172.21.15.64:0/2888829481 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f842c011c80 con 0x7f843019fc00 2024-05-07T02:18:34.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.545+0000 7f8422ffd700 1 -- 172.21.15.64:0/2888829481 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f842c01adc0 con 0x7f843019fc00 2024-05-07T02:18:34.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.546+0000 7f8437df1700 1 -- 172.21.15.64:0/2888829481 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f843019a720 con 0x7f843019fc00 2024-05-07T02:18:34.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.547+0000 7f8422ffd700 1 -- 172.21.15.64:0/2888829481 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f842c018020 con 0x7f843019fc00 2024-05-07T02:18:34.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.547+0000 7f8422ffd700 1 --2- 172.21.15.64:0/2888829481 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f841403e2b0 0x7f84140407c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:18:34.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.547+0000 7f8422ffd700 1 -- 172.21.15.64:0/2888829481 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f842c0521b0 con 0x7f843019fc00 2024-05-07T02:18:34.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.547+0000 7f8435b8d700 1 --2- 172.21.15.64:0/2888829481 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f841403e2b0 0x7f84140407c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:18:34.548 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.549+0000 7f8435b8d700 1 --2- 172.21.15.64:0/2888829481 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f841403e2b0 0x7f84140407c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8424003800 tx=0x7f8424021040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:18:34.564 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.565+0000 7f8422ffd700 1 -- 172.21.15.64:0/2888829481 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f842c01f020 con 0x7f843019fc00 2024-05-07T02:18:34.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:34 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-07T02:18:34.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:34 smithi019 ceph-mon[29781]: mon.a calling monitor election 2024-05-07T02:18:34.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:34 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-07T02:18:34.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:34 smithi019 ceph-mon[29781]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:34.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:34 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-07T02:18:34.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:34 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-07T02:18:34.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:34 smithi019 ceph-mon[29781]: mon.c calling monitor election 2024-05-07T02:18:34.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:34 smithi019 ceph-mon[29781]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:34.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:34 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-07T02:18:34.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:34 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-07T02:18:34.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:34 smithi019 ceph-mon[29781]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:34.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:34 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-07T02:18:34.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:34 smithi019 ceph-mon[29781]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-07T02:18:34.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:34 smithi019 ceph-mon[29781]: monmap epoch 2 2024-05-07T02:18:34.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:34 smithi019 ceph-mon[29781]: fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 2024-05-07T02:18:34.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:34 smithi019 ceph-mon[29781]: last_changed 2024-05-07T02:18:29.166142+0000 2024-05-07T02:18:34.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:34 smithi019 ceph-mon[29781]: created 2024-05-07T02:16:14.906770+0000 2024-05-07T02:18:34.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:34 smithi019 ceph-mon[29781]: min_mon_release 19 (squid) 2024-05-07T02:18:34.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:34 smithi019 ceph-mon[29781]: election_strategy: 1 2024-05-07T02:18:34.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:34 smithi019 ceph-mon[29781]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-05-07T02:18:34.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:34 smithi019 ceph-mon[29781]: 1: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.c 2024-05-07T02:18:34.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:34 smithi019 ceph-mon[29781]: fsmap 2024-05-07T02:18:34.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:34 smithi019 ceph-mon[29781]: osdmap e4: 0 total, 0 up, 0 in 2024-05-07T02:18:34.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:34 smithi019 ceph-mon[29781]: mgrmap e13: x(active, since 62s) 2024-05-07T02:18:34.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:34 smithi019 ceph-mon[29781]: overall HEALTH_OK 2024-05-07T02:18:34.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:34 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:34.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:34 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:34.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.870+0000 7f8437df1700 1 -- 172.21.15.64:0/2888829481 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f843019ae10 con 0x7f843019fc00 2024-05-07T02:18:34.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.871+0000 7f8422ffd700 1 -- 172.21.15.64:0/2888829481 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7f842c016220 con 0x7f843019fc00 2024-05-07T02:18:34.871 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-07T02:18:34.871 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":2,"fsid":"9ff84c16-0c17-11ef-bc97-c7b262605968","modified":"2024-05-07T02:18:29.166142Z","created":"2024-05-07T02:16:14.906770Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-07T02:18:34.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.873+0000 7f8437df1700 1 -- 172.21.15.64:0/2888829481 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f841403e2b0 msgr2=0x7f84140407c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:18:34.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.873+0000 7f8437df1700 1 --2- 172.21.15.64:0/2888829481 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f841403e2b0 0x7f84140407c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8424003800 tx=0x7f8424021040 comp rx=0 tx=0).stop 2024-05-07T02:18:34.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.873+0000 7f8437df1700 1 -- 172.21.15.64:0/2888829481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f843019fc00 msgr2=0x7f84301998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:18:34.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.873+0000 7f8437df1700 1 --2- 172.21.15.64:0/2888829481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f843019fc00 0x7f84301998e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f842c00b4d0 tx=0x7f842c00b890 comp rx=0 tx=0).stop 2024-05-07T02:18:34.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.873+0000 7f8437df1700 1 -- 172.21.15.64:0/2888829481 shutdown_connections 2024-05-07T02:18:34.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.873+0000 7f8437df1700 1 --2- 172.21.15.64:0/2888829481 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f841403e2b0 0x7f84140407c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:34.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.873+0000 7f8437df1700 1 --2- 172.21.15.64:0/2888829481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8430102dd0 0x7f843019f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:34.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.873+0000 7f8437df1700 1 --2- 172.21.15.64:0/2888829481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f843019fc00 0x7f84301998e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:34.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.873+0000 7f8437df1700 1 -- 172.21.15.64:0/2888829481 >> 172.21.15.64:0/2888829481 conn(0x7f84300797e0 msgr2=0x7f843006c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:18:34.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.874+0000 7f8437df1700 1 -- 172.21.15.64:0/2888829481 shutdown_connections 2024-05-07T02:18:34.874 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:34.874+0000 7f8437df1700 1 -- 172.21.15.64:0/2888829481 wait complete. 2024-05-07T02:18:34.875 INFO:teuthology.orchestra.run.smithi064.stderr:dumped monmap epoch 2 2024-05-07T02:18:35.437 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:35 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:35.438 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:35 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-07T02:18:35.438 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:35 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:18:35.438 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:35 smithi064 ceph-mon[34699]: Deploying daemon mon.b on smithi027 2024-05-07T02:18:35.438 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:35 smithi064 ceph-mon[34699]: from='client.? 172.21.15.64:0/2888829481' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-07T02:18:35.438 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:35 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-07T02:18:35.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:35 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:35.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:35 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-07T02:18:35.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:35 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:18:35.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:35 smithi019 ceph-mon[29781]: Deploying daemon mon.b on smithi027 2024-05-07T02:18:35.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:35 smithi019 ceph-mon[29781]: from='client.? 172.21.15.64:0/2888829481' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-07T02:18:35.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:35 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-07T02:18:36.467 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-07T02:18:36.468 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-07T02:18:36.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:36 smithi064 ceph-mon[34699]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:36.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:36 smithi019 ceph-mon[29781]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:36.644 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:18:36 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:18:36.166+0000 7f299ce00700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-07T02:18:36.721 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.c/config 2024-05-07T02:18:38.091 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: load: jerasure load: lrc 2024-05-07T02:18:38.111 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:38.111+0000 7ff9b78b0700 1 -- 172.21.15.64:0/3577649021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9b0102b70 msgr2=0x7ff9b0102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:18:38.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:38.111+0000 7ff9b78b0700 1 --2- 172.21.15.64:0/3577649021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9b0102b70 0x7ff9b0102f80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac009910 tx=0x7ff9ac009c20 comp rx=0 tx=0).stop 2024-05-07T02:18:38.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:38.111+0000 7ff9b78b0700 1 -- 172.21.15.64:0/3577649021 shutdown_connections 2024-05-07T02:18:38.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:38.111+0000 7ff9b78b0700 1 --2- 172.21.15.64:0/3577649021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9b0102b70 0x7ff9b0102f80 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:38.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:38.112+0000 7ff9b78b0700 1 -- 172.21.15.64:0/3577649021 >> 172.21.15.64:0/3577649021 conn(0x7ff9b00fe6b0 msgr2=0x7ff9b0100b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:18:38.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:38.112+0000 7ff9b78b0700 1 -- 172.21.15.64:0/3577649021 shutdown_connections 2024-05-07T02:18:38.112 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:38.112+0000 7ff9b78b0700 1 -- 172.21.15.64:0/3577649021 wait complete. 2024-05-07T02:18:38.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:38.112+0000 7ff9b78b0700 1 Processor -- start 2024-05-07T02:18:38.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:38.112+0000 7ff9b78b0700 1 -- start start 2024-05-07T02:18:38.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:38.113+0000 7ff9b78b0700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9b0102b70 0x7ff9b019be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:18:38.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:38.113+0000 7ff9b78b0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9b019c380 0x7ff9b01a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:18:38.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:38.113+0000 7ff9b78b0700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b0110430 con 0x7ff9b0102b70 2024-05-07T02:18:38.113 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:38.113+0000 7ff9b78b0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b01102b0 con 0x7ff9b019c380 2024-05-07T02:18:38.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:38.113+0000 7ff9b4e4b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9b019c380 0x7ff9b01a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:18:38.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:38.113+0000 7ff9b4e4b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9b019c380 0x7ff9b01a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:45478/0 (socket says 172.21.15.64:45478) 2024-05-07T02:18:38.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:38.113+0000 7ff9b4e4b700 1 -- 172.21.15.64:0/4132215832 learned_addr learned my addr 172.21.15.64:0/4132215832 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-07T02:18:38.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:38.113+0000 7ff9b4e4b700 1 -- 172.21.15.64:0/4132215832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9b019c380 msgr2=0x7ff9b01a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-07T02:18:38.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:38.113+0000 7ff9b4e4b700 1 -- 172.21.15.64:0/4132215832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9b019c380 msgr2=0x7ff9b01a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-07T02:18:38.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:38.113+0000 7ff9b4e4b700 1 --2- 172.21.15.64:0/4132215832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9b019c380 0x7ff9b01a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-07T02:18:38.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:38.113+0000 7ff9b4e4b700 1 --2- 172.21.15.64:0/4132215832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9b019c380 0x7ff9b01a0670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-07T02:18:38.114 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:38.113+0000 7ff9b564c700 1 --2- 172.21.15.64:0/4132215832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9b0102b70 0x7ff9b019be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:18:38.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:38.114+0000 7ff9b564c700 1 -- 172.21.15.64:0/4132215832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9b019c380 msgr2=0x7ff9b01a0670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-07T02:18:38.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:38.114+0000 7ff9b564c700 1 --2- 172.21.15.64:0/4132215832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9b019c380 0x7ff9b01a0670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:38.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:38.114+0000 7ff9b564c700 1 -- 172.21.15.64:0/4132215832 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b019c980 con 0x7ff9b0102b70 2024-05-07T02:18:38.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:38.114+0000 7ff9b564c700 1 --2- 172.21.15.64:0/4132215832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9b0102b70 0x7ff9b019be40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac014000 tx=0x7ff9ac005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:18:38.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:38.115+0000 7ff9a67fc700 1 -- 172.21.15.64:0/4132215832 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7ff9ac005dc0 con 0x7ff9b0102b70 2024-05-07T02:18:38.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:38.115+0000 7ff9b78b0700 1 -- 172.21.15.64:0/4132215832 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9b01a0bb0 con 0x7ff9b0102b70 2024-05-07T02:18:38.115 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:38.115+0000 7ff9a67fc700 1 -- 172.21.15.64:0/4132215832 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9ac00b750 con 0x7ff9b0102b70 2024-05-07T02:18:38.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:38.115+0000 7ff9a67fc700 1 -- 172.21.15.64:0/4132215832 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7ff9ac0039a0 con 0x7ff9b0102b70 2024-05-07T02:18:38.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:38.115+0000 7ff9b78b0700 1 -- 172.21.15.64:0/4132215832 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9b01a10e0 con 0x7ff9b0102b70 2024-05-07T02:18:38.116 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:38.116+0000 7ff9b78b0700 1 -- 172.21.15.64:0/4132215832 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9b006c5c0 con 0x7ff9b0102b70 2024-05-07T02:18:38.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:38.117+0000 7ff9a67fc700 1 -- 172.21.15.64:0/4132215832 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7ff9ac029070 con 0x7ff9b0102b70 2024-05-07T02:18:38.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:38.117+0000 7ff9a67fc700 1 --2- 172.21.15.64:0/4132215832 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7ff99c03e2b0 0x7ff99c0407c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:18:38.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:38.117+0000 7ff9a67fc700 1 -- 172.21.15.64:0/4132215832 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff9ac01e070 con 0x7ff9b0102b70 2024-05-07T02:18:38.117 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:38.118+0000 7ff9b4e4b700 1 --2- 172.21.15.64:0/4132215832 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7ff99c03e2b0 0x7ff99c0407c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:18:38.118 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:38.119+0000 7ff9b4e4b700 1 --2- 172.21.15.64:0/4132215832 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7ff99c03e2b0 0x7ff99c0407c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0109450 tx=0x7ff9a0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:18:38.135 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:38.136+0000 7ff9a67fc700 1 -- 172.21.15.64:0/4132215832 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff9ac030080 con 0x7ff9b0102b70 2024-05-07T02:18:38.137 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:38.137+0000 7ff9a67fc700 1 -- 172.21.15.64:0/4132215832 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac00e7c0 con 0x7ff9b0102b70 2024-05-07T02:18:38.446 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:38.446+0000 7ff9b78b0700 1 -- 172.21.15.64:0/4132215832 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7ff9b00008d0 con 0x7ff9b0102b70 2024-05-07T02:18:38.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: RocksDB version: 7.9.2 2024-05-07T02:18:38.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Git sha 0 2024-05-07T02:18:38.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Compile date 2024-05-03 18:51:23 2024-05-07T02:18:38.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: DB SUMMARY 2024-05-07T02:18:38.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: DB Session ID: 5381M9877PJZ3OQMK6RW 2024-05-07T02:18:38.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: CURRENT file: CURRENT 2024-05-07T02:18:38.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: IDENTITY file: IDENTITY 2024-05-07T02:18:38.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-07T02:18:38.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-05-07T02:18:38.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-05-07T02:18:38.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.error_if_exists: 0 2024-05-07T02:18:38.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.create_if_missing: 0 2024-05-07T02:18:38.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.paranoid_checks: 1 2024-05-07T02:18:38.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-07T02:18:38.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-07T02:18:38.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-07T02:18:38.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.env: 0x55e31f566500 2024-05-07T02:18:38.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.fs: PosixFileSystem 2024-05-07T02:18:38.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.info_log: 0x55e3216be020 2024-05-07T02:18:38.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.max_file_opening_threads: 16 2024-05-07T02:18:38.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.statistics: (nil) 2024-05-07T02:18:38.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.use_fsync: 0 2024-05-07T02:18:38.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.max_log_file_size: 0 2024-05-07T02:18:38.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-07T02:18:38.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-07T02:18:38.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.keep_log_file_num: 1000 2024-05-07T02:18:38.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.recycle_log_file_num: 0 2024-05-07T02:18:38.459 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.allow_fallocate: 1 2024-05-07T02:18:38.459 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.allow_mmap_reads: 0 2024-05-07T02:18:38.459 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.allow_mmap_writes: 0 2024-05-07T02:18:38.459 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.use_direct_reads: 0 2024-05-07T02:18:38.459 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-07T02:18:38.459 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.create_missing_column_families: 0 2024-05-07T02:18:38.459 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.db_log_dir: 2024-05-07T02:18:38.459 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.wal_dir: 2024-05-07T02:18:38.459 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-07T02:18:38.459 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-07T02:18:38.459 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-07T02:18:38.459 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-07T02:18:38.461 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-07T02:18:38.461 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-07T02:18:38.461 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.advise_random_on_open: 1 2024-05-07T02:18:38.461 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.db_write_buffer_size: 0 2024-05-07T02:18:38.461 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.write_buffer_manager: 0x55e3215888c0 2024-05-07T02:18:38.461 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-07T02:18:38.461 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-07T02:18:38.462 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-07T02:18:38.462 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.rate_limiter: (nil) 2024-05-07T02:18:38.462 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-07T02:18:38.462 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.wal_recovery_mode: 2 2024-05-07T02:18:38.462 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.enable_thread_tracking: 0 2024-05-07T02:18:38.462 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.enable_pipelined_write: 0 2024-05-07T02:18:38.462 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.unordered_write: 0 2024-05-07T02:18:38.462 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-07T02:18:38.462 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-07T02:18:38.462 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-07T02:18:38.462 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-07T02:18:38.462 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.row_cache: None 2024-05-07T02:18:38.462 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.wal_filter: None 2024-05-07T02:18:38.462 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-07T02:18:38.462 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.allow_ingest_behind: 0 2024-05-07T02:18:38.463 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.two_write_queues: 0 2024-05-07T02:18:38.463 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.manual_wal_flush: 0 2024-05-07T02:18:38.463 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.wal_compression: 0 2024-05-07T02:18:38.463 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.atomic_flush: 0 2024-05-07T02:18:38.463 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-07T02:18:38.463 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-07T02:18:38.463 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-07T02:18:38.463 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.log_readahead_size: 0 2024-05-07T02:18:38.463 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-07T02:18:38.463 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.best_efforts_recovery: 0 2024-05-07T02:18:38.463 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-07T02:18:38.463 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-07T02:18:38.463 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.allow_data_in_errors: 0 2024-05-07T02:18:38.463 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.db_host_id: __hostname__ 2024-05-07T02:18:38.463 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-07T02:18:38.464 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.max_background_jobs: 2 2024-05-07T02:18:38.464 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.max_background_compactions: -1 2024-05-07T02:18:38.464 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.max_subcompactions: 1 2024-05-07T02:18:38.464 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-07T02:18:38.464 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-07T02:18:38.464 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-07T02:18:38.464 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.max_total_wal_size: 0 2024-05-07T02:18:38.464 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-07T02:18:38.464 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-07T02:18:38.464 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-07T02:18:38.464 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-07T02:18:38.464 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.max_open_files: -1 2024-05-07T02:18:38.464 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.bytes_per_sync: 0 2024-05-07T02:18:38.464 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-07T02:18:38.464 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-07T02:18:38.464 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.compaction_readahead_size: 0 2024-05-07T02:18:38.465 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.max_background_flushes: -1 2024-05-07T02:18:38.465 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Compression algorithms supported: 2024-05-07T02:18:38.465 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-07T02:18:38.465 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: kZSTD supported: 0 2024-05-07T02:18:38.465 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: kXpressCompression supported: 0 2024-05-07T02:18:38.465 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: kLZ4HCCompression supported: 1 2024-05-07T02:18:38.465 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: kZlibCompression supported: 1 2024-05-07T02:18:38.465 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: kSnappyCompression supported: 1 2024-05-07T02:18:38.465 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: kLZ4Compression supported: 1 2024-05-07T02:18:38.465 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: kBZip2Compression supported: 0 2024-05-07T02:18:38.465 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-07T02:18:38.465 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-07T02:18:38.465 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-05-07T02:18:38.465 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-07T02:18:38.465 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-07T02:18:38.465 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.merge_operator: 2024-05-07T02:18:38.465 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.compaction_filter: None 2024-05-07T02:18:38.466 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.compaction_filter_factory: None 2024-05-07T02:18:38.466 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.sst_partitioner_factory: None 2024-05-07T02:18:38.466 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-07T02:18:38.466 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-07T02:18:38.466 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e3216be1e0) 2024-05-07T02:18:38.466 INFO:journalctl@ceph.mon.b.smithi027.stdout: cache_index_and_filter_blocks: 1 2024-05-07T02:18:38.466 INFO:journalctl@ceph.mon.b.smithi027.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-07T02:18:38.466 INFO:journalctl@ceph.mon.b.smithi027.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-07T02:18:38.466 INFO:journalctl@ceph.mon.b.smithi027.stdout: pin_top_level_index_and_filter: 1 2024-05-07T02:18:38.466 INFO:journalctl@ceph.mon.b.smithi027.stdout: index_type: 0 2024-05-07T02:18:38.466 INFO:journalctl@ceph.mon.b.smithi027.stdout: data_block_index_type: 0 2024-05-07T02:18:38.466 INFO:journalctl@ceph.mon.b.smithi027.stdout: index_shortening: 1 2024-05-07T02:18:38.466 INFO:journalctl@ceph.mon.b.smithi027.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-07T02:18:38.466 INFO:journalctl@ceph.mon.b.smithi027.stdout: checksum: 4 2024-05-07T02:18:38.467 INFO:journalctl@ceph.mon.b.smithi027.stdout: no_block_cache: 0 2024-05-07T02:18:38.467 INFO:journalctl@ceph.mon.b.smithi027.stdout: block_cache: 0x55e3215bb1f0 2024-05-07T02:18:38.467 INFO:journalctl@ceph.mon.b.smithi027.stdout: block_cache_name: BinnedLRUCache 2024-05-07T02:18:38.467 INFO:journalctl@ceph.mon.b.smithi027.stdout: block_cache_options: 2024-05-07T02:18:38.467 INFO:journalctl@ceph.mon.b.smithi027.stdout: capacity : 536870912 2024-05-07T02:18:38.467 INFO:journalctl@ceph.mon.b.smithi027.stdout: num_shard_bits : 4 2024-05-07T02:18:38.467 INFO:journalctl@ceph.mon.b.smithi027.stdout: strict_capacity_limit : 0 2024-05-07T02:18:38.467 INFO:journalctl@ceph.mon.b.smithi027.stdout: high_pri_pool_ratio: 0.000 2024-05-07T02:18:38.467 INFO:journalctl@ceph.mon.b.smithi027.stdout: block_cache_compressed: (nil) 2024-05-07T02:18:38.467 INFO:journalctl@ceph.mon.b.smithi027.stdout: persistent_cache: (nil) 2024-05-07T02:18:38.467 INFO:journalctl@ceph.mon.b.smithi027.stdout: block_size: 4096 2024-05-07T02:18:38.467 INFO:journalctl@ceph.mon.b.smithi027.stdout: block_size_deviation: 10 2024-05-07T02:18:38.467 INFO:journalctl@ceph.mon.b.smithi027.stdout: block_restart_interval: 16 2024-05-07T02:18:38.467 INFO:journalctl@ceph.mon.b.smithi027.stdout: index_block_restart_interval: 1 2024-05-07T02:18:38.467 INFO:journalctl@ceph.mon.b.smithi027.stdout: metadata_block_size: 4096 2024-05-07T02:18:38.468 INFO:journalctl@ceph.mon.b.smithi027.stdout: partition_filters: 0 2024-05-07T02:18:38.468 INFO:journalctl@ceph.mon.b.smithi027.stdout: use_delta_encoding: 1 2024-05-07T02:18:38.468 INFO:journalctl@ceph.mon.b.smithi027.stdout: filter_policy: bloomfilter 2024-05-07T02:18:38.468 INFO:journalctl@ceph.mon.b.smithi027.stdout: whole_key_filtering: 1 2024-05-07T02:18:38.468 INFO:journalctl@ceph.mon.b.smithi027.stdout: verify_compression: 0 2024-05-07T02:18:38.468 INFO:journalctl@ceph.mon.b.smithi027.stdout: read_amp_bytes_per_bit: 0 2024-05-07T02:18:38.468 INFO:journalctl@ceph.mon.b.smithi027.stdout: format_version: 5 2024-05-07T02:18:38.468 INFO:journalctl@ceph.mon.b.smithi027.stdout: enable_index_compression: 1 2024-05-07T02:18:38.468 INFO:journalctl@ceph.mon.b.smithi027.stdout: block_align: 0 2024-05-07T02:18:38.468 INFO:journalctl@ceph.mon.b.smithi027.stdout: max_auto_readahead_size: 262144 2024-05-07T02:18:38.468 INFO:journalctl@ceph.mon.b.smithi027.stdout: prepopulate_block_cache: 0 2024-05-07T02:18:38.468 INFO:journalctl@ceph.mon.b.smithi027.stdout: initial_auto_readahead_size: 8192 2024-05-07T02:18:38.468 INFO:journalctl@ceph.mon.b.smithi027.stdout: num_file_reads_for_auto_readahead: 2 2024-05-07T02:18:38.468 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.write_buffer_size: 33554432 2024-05-07T02:18:38.468 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.max_write_buffer_number: 2 2024-05-07T02:18:38.468 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.compression: NoCompression 2024-05-07T02:18:38.469 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.bottommost_compression: Disabled 2024-05-07T02:18:38.469 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.prefix_extractor: nullptr 2024-05-07T02:18:38.469 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-07T02:18:38.469 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.num_levels: 7 2024-05-07T02:18:38.469 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-07T02:18:38.469 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-07T02:18:38.469 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-07T02:18:38.469 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-07T02:18:38.469 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-07T02:18:38.469 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-07T02:18:38.469 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-07T02:18:38.469 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-07T02:18:38.469 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-07T02:18:38.469 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-07T02:18:38.469 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-07T02:18:38.469 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-07T02:18:38.470 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-07T02:18:38.470 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.compression_opts.level: 32767 2024-05-07T02:18:38.470 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.compression_opts.strategy: 0 2024-05-07T02:18:38.470 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-07T02:18:38.470 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-07T02:18:38.470 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-07T02:18:38.470 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-07T02:18:38.470 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.compression_opts.enabled: false 2024-05-07T02:18:38.470 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-07T02:18:38.470 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-07T02:18:38.470 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-07T02:18:38.470 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-07T02:18:38.470 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.target_file_size_base: 67108864 2024-05-07T02:18:38.471 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-07T02:18:38.471 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-07T02:18:38.471 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-07T02:18:38.471 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-07T02:18:38.471 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-07T02:18:38.471 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-07T02:18:38.471 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-07T02:18:38.471 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-07T02:18:38.471 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-07T02:18:38.471 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-07T02:18:38.471 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-07T02:18:38.471 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-07T02:18:38.471 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-07T02:18:38.471 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-07T02:18:38.471 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.arena_block_size: 1048576 2024-05-07T02:18:38.472 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-07T02:18:38.472 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-07T02:18:38.472 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.disable_auto_compactions: 0 2024-05-07T02:18:38.472 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-07T02:18:38.472 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-07T02:18:38.472 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-07T02:18:38.472 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-07T02:18:38.472 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-07T02:18:38.472 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-07T02:18:38.472 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-07T02:18:38.472 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-07T02:18:38.472 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-07T02:18:38.472 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-07T02:18:38.472 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-07T02:18:38.472 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.inplace_update_support: 0 2024-05-07T02:18:38.472 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-07T02:18:38.473 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-07T02:18:38.473 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-07T02:18:38.473 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-07T02:18:38.473 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.bloom_locality: 0 2024-05-07T02:18:38.473 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.max_successive_merges: 0 2024-05-07T02:18:38.473 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-07T02:18:38.473 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.paranoid_file_checks: 0 2024-05-07T02:18:38.473 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.force_consistency_checks: 1 2024-05-07T02:18:38.473 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.report_bg_io_stats: 0 2024-05-07T02:18:38.473 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.ttl: 2592000 2024-05-07T02:18:38.473 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-07T02:18:38.473 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-07T02:18:38.473 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-07T02:18:38.473 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.enable_blob_files: false 2024-05-07T02:18:38.473 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.min_blob_size: 0 2024-05-07T02:18:38.474 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.blob_file_size: 268435456 2024-05-07T02:18:38.474 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-07T02:18:38.474 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-07T02:18:38.474 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-07T02:18:38.474 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-07T02:18:38.474 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-07T02:18:38.474 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.blob_file_starting_level: 0 2024-05-07T02:18:38.474 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-07T02:18:38.474 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-07T02:18:38.474 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-07T02:18:38.474 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: a53b7c0c-00c3-4972-888f-af702eead855 2024-05-07T02:18:38.474 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715048318093913, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-07T02:18:38.474 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-07T02:18:38.474 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715048318094349, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715048318, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a53b7c0c-00c3-4972-888f-af702eead855", "db_session_id": "5381M9877PJZ3OQMK6RW", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-07T02:18:38.475 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715048318094432, "job": 1, "event": "recovery_finished"} 2024-05-07T02:18:38.475 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-07T02:18:38.475 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-07T02:18:38.475 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55e32159fc00 2024-05-07T02:18:38.475 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: DB pointer 0x55e326392000 2024-05-07T02:18:38.475 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-07T02:18:38.475 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-07T02:18:38.475 INFO:journalctl@ceph.mon.b.smithi027.stdout: ** DB Stats ** 2024-05-07T02:18:38.475 INFO:journalctl@ceph.mon.b.smithi027.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-07T02:18:38.475 INFO:journalctl@ceph.mon.b.smithi027.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-07T02:18:38.475 INFO:journalctl@ceph.mon.b.smithi027.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-07T02:18:38.475 INFO:journalctl@ceph.mon.b.smithi027.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-07T02:18:38.475 INFO:journalctl@ceph.mon.b.smithi027.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-07T02:18:38.475 INFO:journalctl@ceph.mon.b.smithi027.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-07T02:18:38.475 INFO:journalctl@ceph.mon.b.smithi027.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-07T02:18:38.476 INFO:journalctl@ceph.mon.b.smithi027.stdout: 2024-05-07T02:18:38.476 INFO:journalctl@ceph.mon.b.smithi027.stdout: ** Compaction Stats [default] ** 2024-05-07T02:18:38.476 INFO:journalctl@ceph.mon.b.smithi027.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-07T02:18:38.476 INFO:journalctl@ceph.mon.b.smithi027.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-07T02:18:38.476 INFO:journalctl@ceph.mon.b.smithi027.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-07T02:18:38.476 INFO:journalctl@ceph.mon.b.smithi027.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-07T02:18:38.476 INFO:journalctl@ceph.mon.b.smithi027.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-07T02:18:38.476 INFO:journalctl@ceph.mon.b.smithi027.stdout: 2024-05-07T02:18:38.476 INFO:journalctl@ceph.mon.b.smithi027.stdout: ** Compaction Stats [default] ** 2024-05-07T02:18:38.476 INFO:journalctl@ceph.mon.b.smithi027.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-07T02:18:38.476 INFO:journalctl@ceph.mon.b.smithi027.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-07T02:18:38.476 INFO:journalctl@ceph.mon.b.smithi027.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-07T02:18:38.476 INFO:journalctl@ceph.mon.b.smithi027.stdout: 2024-05-07T02:18:38.476 INFO:journalctl@ceph.mon.b.smithi027.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-07T02:18:38.476 INFO:journalctl@ceph.mon.b.smithi027.stdout: 2024-05-07T02:18:38.477 INFO:journalctl@ceph.mon.b.smithi027.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-07T02:18:38.477 INFO:journalctl@ceph.mon.b.smithi027.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-07T02:18:38.477 INFO:journalctl@ceph.mon.b.smithi027.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-07T02:18:38.477 INFO:journalctl@ceph.mon.b.smithi027.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-07T02:18:38.477 INFO:journalctl@ceph.mon.b.smithi027.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-07T02:18:38.477 INFO:journalctl@ceph.mon.b.smithi027.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-07T02:18:38.477 INFO:journalctl@ceph.mon.b.smithi027.stdout: Cumulative compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-07T02:18:38.477 INFO:journalctl@ceph.mon.b.smithi027.stdout: Interval compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-07T02:18:38.477 INFO:journalctl@ceph.mon.b.smithi027.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-07T02:18:38.477 INFO:journalctl@ceph.mon.b.smithi027.stdout: Block cache BinnedLRUCache@0x55e3215bb1f0#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-05-07T02:18:38.477 INFO:journalctl@ceph.mon.b.smithi027.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-07T02:18:38.477 INFO:journalctl@ceph.mon.b.smithi027.stdout: 2024-05-07T02:18:38.477 INFO:journalctl@ceph.mon.b.smithi027.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-07T02:18:38.477 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-05-07T02:18:38.477 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: using public_addr v2:172.21.15.27:0/0 -> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] 2024-05-07T02:18:38.477 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: starting mon.b rank -1 at public addrs [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] at bind addrs [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 2024-05-07T02:18:38.477 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: mon.b@-1(???) e0 preinit fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 2024-05-07T02:18:38.478 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: mon.b@-1(synchronizing).mds e1 new map 2024-05-07T02:18:38.478 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: mon.b@-1(synchronizing).mds e1 print_map 2024-05-07T02:18:38.478 INFO:journalctl@ceph.mon.b.smithi027.stdout: e1 2024-05-07T02:18:38.478 INFO:journalctl@ceph.mon.b.smithi027.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-07T02:18:38.478 INFO:journalctl@ceph.mon.b.smithi027.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-07T02:18:38.478 INFO:journalctl@ceph.mon.b.smithi027.stdout: legacy client fscid: -1 2024-05-07T02:18:38.478 INFO:journalctl@ceph.mon.b.smithi027.stdout: 2024-05-07T02:18:38.478 INFO:journalctl@ceph.mon.b.smithi027.stdout: No filesystems configured 2024-05-07T02:18:38.478 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-07T02:18:38.478 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-07T02:18:38.478 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-07T02:18:38.478 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-07T02:18:38.478 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-07T02:18:38.478 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-07T02:18:38.478 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-07T02:18:38.479 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-07T02:18:38.479 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-07T02:18:38.479 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-07T02:18:38.479 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: Deploying cephadm binary to smithi027 2024-05-07T02:18:38.479 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:38.479 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:38.479 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:38.479 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:18:38.479 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: Added host smithi027 2024-05-07T02:18:38.479 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:38.479 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:38.479 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:38.479 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:38.479 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:38.479 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-07T02:18:38.480 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:38.480 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:38.480 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:38.480 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:38.480 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:38.480 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:18:38.480 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:18:38.480 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:18:38.480 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: Updating smithi027:/etc/ceph/ceph.conf 2024-05-07T02:18:38.480 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: Updating smithi027:/var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-07T02:18:38.480 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2024-05-07T02:18:38.480 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: Updating smithi027:/var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-07T02:18:38.480 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:38.480 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:38.480 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:38.481 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:38.481 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi064", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:18:38.481 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: Deploying cephadm binary to smithi064 2024-05-07T02:18:38.481 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:38.481 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:38.481 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:38.481 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: Added host smithi064 2024-05-07T02:18:38.481 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:18:38.481 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:38.481 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:38.481 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:38.481 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:38.481 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:38.481 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-07T02:18:38.481 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:38.482 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:38.482 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:38.482 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:38.482 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:38.482 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi064", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:18:38.482 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:18:38.482 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:18:38.482 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: Updating smithi064:/etc/ceph/ceph.conf 2024-05-07T02:18:38.482 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: Updating smithi064:/var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-07T02:18:38.482 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:38.482 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2024-05-07T02:18:38.482 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: Updating smithi064:/var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-07T02:18:38.482 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/188656138' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-07T02:18:38.482 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:38.483 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:38.483 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:38.483 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/188656138' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-07T02:18:38.483 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: osdmap e4: 0 total, 0 up, 0 in 2024-05-07T02:18:38.483 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:38.483 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:38.483 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi019:172.21.15.19=a;smithi027:172.21.15.27=b;smithi064:172.21.15.64=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:18:38.483 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: Saving service mon spec with placement smithi019:172.21.15.19=a;smithi027:172.21.15.27=b;smithi064:172.21.15.64=c;count:3 2024-05-07T02:18:38.483 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:38.483 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:18:38.483 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:18:38.483 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:18:38.483 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:38.483 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-07T02:18:38.483 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:18:38.484 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: Deploying daemon mon.c on smithi064 2024-05-07T02:18:38.484 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:38.484 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='client.? 172.21.15.64:0/2317022933' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-07T02:18:38.484 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:38.484 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-07T02:18:38.484 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: mon.a calling monitor election 2024-05-07T02:18:38.484 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-07T02:18:38.484 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:38.484 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-07T02:18:38.484 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-07T02:18:38.484 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: mon.c calling monitor election 2024-05-07T02:18:38.484 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:38.484 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-07T02:18:38.484 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-07T02:18:38.484 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:38.485 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-07T02:18:38.485 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-07T02:18:38.485 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: monmap epoch 2 2024-05-07T02:18:38.485 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 2024-05-07T02:18:38.485 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: last_changed 2024-05-07T02:18:29.166142+0000 2024-05-07T02:18:38.485 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: created 2024-05-07T02:16:14.906770+0000 2024-05-07T02:18:38.485 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: min_mon_release 19 (squid) 2024-05-07T02:18:38.485 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: election_strategy: 1 2024-05-07T02:18:38.485 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-05-07T02:18:38.485 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: 1: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.c 2024-05-07T02:18:38.485 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: fsmap 2024-05-07T02:18:38.485 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: osdmap e4: 0 total, 0 up, 0 in 2024-05-07T02:18:38.485 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: mgrmap e13: x(active, since 62s) 2024-05-07T02:18:38.485 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: overall HEALTH_OK 2024-05-07T02:18:38.485 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:38.486 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:38.486 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:38.486 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-07T02:18:38.486 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:18:38.486 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: Deploying daemon mon.b on smithi027 2024-05-07T02:18:38.486 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='client.? 172.21.15.64:0/2888829481' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-07T02:18:38.486 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-07T02:18:38.486 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:38.486 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 ceph-mon[34398]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-07T02:18:38.486 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 bash[34378]: 8527eb176b97e0c657a1d7b8d84c9aa482539a672a167da8ce3dda282aa64082 2024-05-07T02:18:38.486 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:38 smithi027 systemd[1]: Started Ceph mon.b for 9ff84c16-0c17-11ef-bc97-c7b262605968. 2024-05-07T02:18:43.160 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:43.161+0000 7ff9a67fc700 1 -- 172.21.15.64:0/4132215832 <== mon.0 v2:172.21.15.19:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1275 (secure 0 0 0) 0x7ff9ac038510 con 0x7ff9b0102b70 2024-05-07T02:18:43.160 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-07T02:18:43.161 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":3,"fsid":"9ff84c16-0c17-11ef-bc97-c7b262605968","modified":"2024-05-07T02:18:38.135491Z","created":"2024-05-07T02:16:14.906770Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:3300","nonce":0},{"type":"v1","addr":"172.21.15.27:6789","nonce":0}]},"addr":"172.21.15.27:6789/0","public_addr":"172.21.15.27:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-07T02:18:43.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:43.164+0000 7ff9b78b0700 1 -- 172.21.15.64:0/4132215832 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7ff99c03e2b0 msgr2=0x7ff99c0407c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:18:43.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:43.164+0000 7ff9b78b0700 1 --2- 172.21.15.64:0/4132215832 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7ff99c03e2b0 0x7ff99c0407c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0109450 tx=0x7ff9a0008040 comp rx=0 tx=0).stop 2024-05-07T02:18:43.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:43.164+0000 7ff9b78b0700 1 -- 172.21.15.64:0/4132215832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9b0102b70 msgr2=0x7ff9b019be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:18:43.163 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:43.164+0000 7ff9b78b0700 1 --2- 172.21.15.64:0/4132215832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9b0102b70 0x7ff9b019be40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac014000 tx=0x7ff9ac005030 comp rx=0 tx=0).stop 2024-05-07T02:18:43.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:43.164+0000 7ff9b78b0700 1 -- 172.21.15.64:0/4132215832 shutdown_connections 2024-05-07T02:18:43.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:43.164+0000 7ff9b78b0700 1 --2- 172.21.15.64:0/4132215832 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7ff99c03e2b0 0x7ff99c0407c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:43.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:43.164+0000 7ff9b78b0700 1 --2- 172.21.15.64:0/4132215832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9b0102b70 0x7ff9b019be40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:43.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:43.164+0000 7ff9b78b0700 1 --2- 172.21.15.64:0/4132215832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9b019c380 0x7ff9b01a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:43.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:43.164+0000 7ff9b78b0700 1 -- 172.21.15.64:0/4132215832 >> 172.21.15.64:0/4132215832 conn(0x7ff9b00fe6b0 msgr2=0x7ff9b010c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:18:43.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:43.164+0000 7ff9b78b0700 1 -- 172.21.15.64:0/4132215832 shutdown_connections 2024-05-07T02:18:43.164 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:43.164+0000 7ff9b78b0700 1 -- 172.21.15.64:0/4132215832 wait complete. 2024-05-07T02:18:43.164 INFO:teuthology.orchestra.run.smithi064.stderr:dumped monmap epoch 3 2024-05-07T02:18:43.465 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:43 smithi064 ceph-mon[34699]: mon.c calling monitor election 2024-05-07T02:18:43.466 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:43 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-07T02:18:43.466 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:43 smithi064 ceph-mon[34699]: mon.a calling monitor election 2024-05-07T02:18:43.466 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:43 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-07T02:18:43.466 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:43 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-07T02:18:43.466 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:43 smithi064 ceph-mon[34699]: from='client.? 172.21.15.64:0/4132215832' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-07T02:18:43.466 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:43 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-07T02:18:43.466 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:43 smithi064 ceph-mon[34699]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:43.466 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:43 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-07T02:18:43.466 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:43 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-07T02:18:43.466 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:43 smithi064 ceph-mon[34699]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:43.466 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:43 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-07T02:18:43.466 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:43 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-07T02:18:43.467 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:43 smithi064 ceph-mon[34699]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-07T02:18:43.467 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:43 smithi064 ceph-mon[34699]: monmap epoch 3 2024-05-07T02:18:43.467 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:43 smithi064 ceph-mon[34699]: fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 2024-05-07T02:18:43.467 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:43 smithi064 ceph-mon[34699]: last_changed 2024-05-07T02:18:38.135491+0000 2024-05-07T02:18:43.467 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:43 smithi064 ceph-mon[34699]: created 2024-05-07T02:16:14.906770+0000 2024-05-07T02:18:43.467 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:43 smithi064 ceph-mon[34699]: min_mon_release 19 (squid) 2024-05-07T02:18:43.467 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:43 smithi064 ceph-mon[34699]: election_strategy: 1 2024-05-07T02:18:43.467 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:43 smithi064 ceph-mon[34699]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-05-07T02:18:43.467 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:43 smithi064 ceph-mon[34699]: 1: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.c 2024-05-07T02:18:43.467 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:43 smithi064 ceph-mon[34699]: 2: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.b 2024-05-07T02:18:43.467 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:43 smithi064 ceph-mon[34699]: fsmap 2024-05-07T02:18:43.467 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:43 smithi064 ceph-mon[34699]: osdmap e4: 0 total, 0 up, 0 in 2024-05-07T02:18:43.467 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:43 smithi064 ceph-mon[34699]: mgrmap e13: x(active, since 71s) 2024-05-07T02:18:43.467 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:43 smithi064 ceph-mon[34699]: overall HEALTH_OK 2024-05-07T02:18:43.468 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:43 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:43.468 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:43 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:43.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:43 smithi019 ceph-mon[29781]: mon.c calling monitor election 2024-05-07T02:18:43.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:43 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-07T02:18:43.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:43 smithi019 ceph-mon[29781]: mon.a calling monitor election 2024-05-07T02:18:43.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:43 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-07T02:18:43.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:43 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-07T02:18:43.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:43 smithi019 ceph-mon[29781]: from='client.? 172.21.15.64:0/4132215832' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-07T02:18:43.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:43 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-07T02:18:43.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:43 smithi019 ceph-mon[29781]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:43.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:43 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-07T02:18:43.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:43 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-07T02:18:43.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:43 smithi019 ceph-mon[29781]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:43.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:43 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-07T02:18:43.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:43 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-07T02:18:43.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:43 smithi019 ceph-mon[29781]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-07T02:18:43.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:43 smithi019 ceph-mon[29781]: monmap epoch 3 2024-05-07T02:18:43.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:43 smithi019 ceph-mon[29781]: fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 2024-05-07T02:18:43.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:43 smithi019 ceph-mon[29781]: last_changed 2024-05-07T02:18:38.135491+0000 2024-05-07T02:18:43.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:43 smithi019 ceph-mon[29781]: created 2024-05-07T02:16:14.906770+0000 2024-05-07T02:18:43.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:43 smithi019 ceph-mon[29781]: min_mon_release 19 (squid) 2024-05-07T02:18:43.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:43 smithi019 ceph-mon[29781]: election_strategy: 1 2024-05-07T02:18:43.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:43 smithi019 ceph-mon[29781]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-05-07T02:18:43.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:43 smithi019 ceph-mon[29781]: 1: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.c 2024-05-07T02:18:43.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:43 smithi019 ceph-mon[29781]: 2: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.b 2024-05-07T02:18:43.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:43 smithi019 ceph-mon[29781]: fsmap 2024-05-07T02:18:43.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:43 smithi019 ceph-mon[29781]: osdmap e4: 0 total, 0 up, 0 in 2024-05-07T02:18:43.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:43 smithi019 ceph-mon[29781]: mgrmap e13: x(active, since 71s) 2024-05-07T02:18:43.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:43 smithi019 ceph-mon[29781]: overall HEALTH_OK 2024-05-07T02:18:43.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:43 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:43.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:43 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:43.746 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-07T02:18:43.746 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph config generate-minimal-conf 2024-05-07T02:18:44.178 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:18:44.385 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:44 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:44.385 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:44 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:44.385 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:44 smithi064 ceph-mon[34699]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:44.385 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:44 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:18:44.386 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:44 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-07T02:18:44.501 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:44 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:44.501 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:44 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:44.501 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:44 smithi019 ceph-mon[29781]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:44.501 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:44 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:18:44.501 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:44 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-07T02:18:45.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: mon.c calling monitor election 2024-05-07T02:18:45.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-07T02:18:45.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: mon.a calling monitor election 2024-05-07T02:18:45.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-07T02:18:45.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-07T02:18:45.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: from='client.? 172.21.15.64:0/4132215832' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-07T02:18:45.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-07T02:18:45.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:45.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-07T02:18:45.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-07T02:18:45.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:45.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-07T02:18:45.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-07T02:18:45.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-07T02:18:45.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: monmap epoch 3 2024-05-07T02:18:45.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 2024-05-07T02:18:45.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: last_changed 2024-05-07T02:18:38.135491+0000 2024-05-07T02:18:45.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: created 2024-05-07T02:16:14.906770+0000 2024-05-07T02:18:45.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: min_mon_release 19 (squid) 2024-05-07T02:18:45.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: election_strategy: 1 2024-05-07T02:18:45.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-05-07T02:18:45.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: 1: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.c 2024-05-07T02:18:45.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: 2: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.b 2024-05-07T02:18:45.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: fsmap 2024-05-07T02:18:45.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: osdmap e4: 0 total, 0 up, 0 in 2024-05-07T02:18:45.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: mgrmap e13: x(active, since 71s) 2024-05-07T02:18:45.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: overall HEALTH_OK 2024-05-07T02:18:45.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:45.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:45.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:45.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:45.459 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:45.459 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:18:45.459 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-07T02:18:46.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.192+0000 7f72be600700 1 -- 172.21.15.19:0/2037020012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72b8075b60 msgr2=0x7f72b8075f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:18:46.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.192+0000 7f72be600700 1 --2- 172.21.15.19:0/2037020012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72b8075b60 0x7f72b8075f70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f72a8009910 tx=0x7f72a8009c20 comp rx=0 tx=0).stop 2024-05-07T02:18:46.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.193+0000 7f72be600700 1 -- 172.21.15.19:0/2037020012 shutdown_connections 2024-05-07T02:18:46.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.193+0000 7f72be600700 1 --2- 172.21.15.19:0/2037020012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72b8075b60 0x7f72b8075f70 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:46.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.193+0000 7f72be600700 1 -- 172.21.15.19:0/2037020012 >> 172.21.15.19:0/2037020012 conn(0x7f72b80ff6a0 msgr2=0x7f72b8101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:18:46.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.193+0000 7f72be600700 1 -- 172.21.15.19:0/2037020012 shutdown_connections 2024-05-07T02:18:46.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.193+0000 7f72be600700 1 -- 172.21.15.19:0/2037020012 wait complete. 2024-05-07T02:18:46.301 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:45 smithi064 ceph-mon[34699]: mon.b calling monitor election 2024-05-07T02:18:46.302 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:45 smithi064 ceph-mon[34699]: Updating smithi019:/etc/ceph/ceph.conf 2024-05-07T02:18:46.302 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:45 smithi064 ceph-mon[34699]: Updating smithi027:/etc/ceph/ceph.conf 2024-05-07T02:18:46.302 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:45 smithi064 ceph-mon[34699]: Updating smithi064:/etc/ceph/ceph.conf 2024-05-07T02:18:46.302 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:45 smithi064 ceph-mon[34699]: mon.b calling monitor election 2024-05-07T02:18:46.302 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:45 smithi064 ceph-mon[34699]: mon.c calling monitor election 2024-05-07T02:18:46.302 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:45 smithi064 ceph-mon[34699]: mon.a calling monitor election 2024-05-07T02:18:46.302 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:45 smithi064 ceph-mon[34699]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-07T02:18:46.302 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:45 smithi064 ceph-mon[34699]: monmap epoch 3 2024-05-07T02:18:46.302 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:45 smithi064 ceph-mon[34699]: fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 2024-05-07T02:18:46.302 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:45 smithi064 ceph-mon[34699]: last_changed 2024-05-07T02:18:38.135491+0000 2024-05-07T02:18:46.302 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:45 smithi064 ceph-mon[34699]: created 2024-05-07T02:16:14.906770+0000 2024-05-07T02:18:46.302 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:45 smithi064 ceph-mon[34699]: min_mon_release 19 (squid) 2024-05-07T02:18:46.302 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:45 smithi064 ceph-mon[34699]: election_strategy: 1 2024-05-07T02:18:46.303 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:45 smithi064 ceph-mon[34699]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-05-07T02:18:46.303 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:45 smithi064 ceph-mon[34699]: 1: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.c 2024-05-07T02:18:46.303 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:45 smithi064 ceph-mon[34699]: 2: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.b 2024-05-07T02:18:46.303 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:45 smithi064 ceph-mon[34699]: fsmap 2024-05-07T02:18:46.303 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:45 smithi064 ceph-mon[34699]: osdmap e4: 0 total, 0 up, 0 in 2024-05-07T02:18:46.303 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:45 smithi064 ceph-mon[34699]: mgrmap e13: x(active, since 73s) 2024-05-07T02:18:46.303 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:45 smithi064 ceph-mon[34699]: overall HEALTH_OK 2024-05-07T02:18:46.303 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:45 smithi064 ceph-mon[34699]: Updating smithi027:/var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-07T02:18:46.303 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:45 smithi064 ceph-mon[34699]: Updating smithi064:/var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-07T02:18:46.303 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:45 smithi064 ceph-mon[34699]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:46.303 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:45 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:46.303 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:45 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:46.303 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:45 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:46.303 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:45 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:46.303 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:45 smithi064 ceph-mon[34699]: Updating smithi019:/var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-07T02:18:46.304 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: mon.b calling monitor election 2024-05-07T02:18:46.304 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: Updating smithi019:/etc/ceph/ceph.conf 2024-05-07T02:18:46.304 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: Updating smithi027:/etc/ceph/ceph.conf 2024-05-07T02:18:46.304 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: Updating smithi064:/etc/ceph/ceph.conf 2024-05-07T02:18:46.304 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: mon.b calling monitor election 2024-05-07T02:18:46.304 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: mon.c calling monitor election 2024-05-07T02:18:46.304 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: mon.a calling monitor election 2024-05-07T02:18:46.304 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-07T02:18:46.304 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: monmap epoch 3 2024-05-07T02:18:46.304 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 2024-05-07T02:18:46.304 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: last_changed 2024-05-07T02:18:38.135491+0000 2024-05-07T02:18:46.305 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: created 2024-05-07T02:16:14.906770+0000 2024-05-07T02:18:46.305 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: min_mon_release 19 (squid) 2024-05-07T02:18:46.305 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: election_strategy: 1 2024-05-07T02:18:46.305 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-05-07T02:18:46.305 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: 1: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.c 2024-05-07T02:18:46.305 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: 2: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.b 2024-05-07T02:18:46.305 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: fsmap 2024-05-07T02:18:46.305 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: osdmap e4: 0 total, 0 up, 0 in 2024-05-07T02:18:46.305 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: mgrmap e13: x(active, since 73s) 2024-05-07T02:18:46.305 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: overall HEALTH_OK 2024-05-07T02:18:46.305 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: Updating smithi027:/var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-07T02:18:46.305 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: Updating smithi064:/var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-07T02:18:46.305 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:46.305 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:46.306 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:46.306 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:46.306 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:46.306 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:45 smithi027 ceph-mon[34398]: Updating smithi019:/var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-07T02:18:46.306 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:45 smithi019 ceph-mon[29781]: mon.b calling monitor election 2024-05-07T02:18:46.306 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:45 smithi019 ceph-mon[29781]: Updating smithi019:/etc/ceph/ceph.conf 2024-05-07T02:18:46.306 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:45 smithi019 ceph-mon[29781]: Updating smithi027:/etc/ceph/ceph.conf 2024-05-07T02:18:46.306 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:45 smithi019 ceph-mon[29781]: Updating smithi064:/etc/ceph/ceph.conf 2024-05-07T02:18:46.306 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:45 smithi019 ceph-mon[29781]: mon.b calling monitor election 2024-05-07T02:18:46.306 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:45 smithi019 ceph-mon[29781]: mon.c calling monitor election 2024-05-07T02:18:46.306 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:45 smithi019 ceph-mon[29781]: mon.a calling monitor election 2024-05-07T02:18:46.306 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:45 smithi019 ceph-mon[29781]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-07T02:18:46.306 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:45 smithi019 ceph-mon[29781]: monmap epoch 3 2024-05-07T02:18:46.306 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:45 smithi019 ceph-mon[29781]: fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 2024-05-07T02:18:46.307 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:45 smithi019 ceph-mon[29781]: last_changed 2024-05-07T02:18:38.135491+0000 2024-05-07T02:18:46.307 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:45 smithi019 ceph-mon[29781]: created 2024-05-07T02:16:14.906770+0000 2024-05-07T02:18:46.307 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:45 smithi019 ceph-mon[29781]: min_mon_release 19 (squid) 2024-05-07T02:18:46.307 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:45 smithi019 ceph-mon[29781]: election_strategy: 1 2024-05-07T02:18:46.307 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:45 smithi019 ceph-mon[29781]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-05-07T02:18:46.307 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:45 smithi019 ceph-mon[29781]: 1: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.c 2024-05-07T02:18:46.307 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:45 smithi019 ceph-mon[29781]: 2: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.b 2024-05-07T02:18:46.307 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:45 smithi019 ceph-mon[29781]: fsmap 2024-05-07T02:18:46.307 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:45 smithi019 ceph-mon[29781]: osdmap e4: 0 total, 0 up, 0 in 2024-05-07T02:18:46.307 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:45 smithi019 ceph-mon[29781]: mgrmap e13: x(active, since 73s) 2024-05-07T02:18:46.307 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:45 smithi019 ceph-mon[29781]: overall HEALTH_OK 2024-05-07T02:18:46.307 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:45 smithi019 ceph-mon[29781]: Updating smithi027:/var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-07T02:18:46.307 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:45 smithi019 ceph-mon[29781]: Updating smithi064:/var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-07T02:18:46.307 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:45 smithi019 ceph-mon[29781]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:46.307 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:45 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:46.307 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:45 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:46.308 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:45 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:46.308 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:45 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:46.308 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:45 smithi019 ceph-mon[29781]: Updating smithi019:/var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-07T02:18:46.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.195+0000 7f72be600700 1 Processor -- start 2024-05-07T02:18:46.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.195+0000 7f72be600700 1 -- start start 2024-05-07T02:18:46.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.195+0000 7f72be600700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72b8075b60 0x7f72b81a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:18:46.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.195+0000 7f72be600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72b81a07e0 0x7f72b81a4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:18:46.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.196+0000 7f72be600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72b81a5010 0x7f72b81a5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:18:46.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.196+0000 7f72be600700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f72b8114750 con 0x7f72b81a07e0 2024-05-07T02:18:46.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.196+0000 7f72be600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f72b81145d0 con 0x7f72b81a5010 2024-05-07T02:18:46.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.196+0000 7f72be600700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f72b81148d0 con 0x7f72b8075b60 2024-05-07T02:18:46.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.196+0000 7f72b7400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72b8075b60 0x7f72b81a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:18:46.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.196+0000 7f72b7400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72b8075b60 0x7f72b81a02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:51026/0 (socket says 172.21.15.19:51026) 2024-05-07T02:18:46.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.196+0000 7f72b7400700 1 -- 172.21.15.19:0/3317076421 learned_addr learned my addr 172.21.15.19:0/3317076421 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:18:46.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.196+0000 7f72b7e00700 1 --2- 172.21.15.19:0/3317076421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72b81a5010 0x7f72b81a5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:18:46.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.196+0000 7f72b6a00700 1 --2- 172.21.15.19:0/3317076421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72b81a07e0 0x7f72b81a4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:18:46.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.196+0000 7f72b7400700 1 -- 172.21.15.19:0/3317076421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72b81a5010 msgr2=0x7f72b81a5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:18:46.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.196+0000 7f72b7400700 1 --2- 172.21.15.19:0/3317076421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72b81a5010 0x7f72b81a5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:46.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.196+0000 7f72b7400700 1 -- 172.21.15.19:0/3317076421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72b81a07e0 msgr2=0x7f72b81a4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:18:46.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.196+0000 7f72b7400700 1 --2- 172.21.15.19:0/3317076421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72b81a07e0 0x7f72b81a4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:46.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.197+0000 7f72b7400700 1 -- 172.21.15.19:0/3317076421 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b81acd80 con 0x7f72b8075b60 2024-05-07T02:18:46.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.197+0000 7f72b7e00700 1 --2- 172.21.15.19:0/3317076421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72b81a5010 0x7f72b81a5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:18:46.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.197+0000 7f72b7400700 1 --2- 172.21.15.19:0/3317076421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72b8075b60 0x7f72b81a02a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f72a8004f80 tx=0x7f72a8005060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:18:46.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.197+0000 7f72afe00700 1 -- 172.21.15.19:0/3317076421 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a8027070 con 0x7f72b8075b60 2024-05-07T02:18:46.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.197+0000 7f72afe00700 1 -- 172.21.15.19:0/3317076421 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72a800be10 con 0x7f72b8075b60 2024-05-07T02:18:46.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.197+0000 7f72afe00700 1 -- 172.21.15.19:0/3317076421 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a800f460 con 0x7f72b8075b60 2024-05-07T02:18:46.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.197+0000 7f72be600700 1 -- 172.21.15.19:0/3317076421 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72b81acf50 con 0x7f72b8075b60 2024-05-07T02:18:46.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.198+0000 7f72be600700 1 -- 172.21.15.19:0/3317076421 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72b81ad2f0 con 0x7f72b8075b60 2024-05-07T02:18:46.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.199+0000 7f72afe00700 1 -- 172.21.15.19:0/3317076421 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f72a802a070 con 0x7f72b8075b60 2024-05-07T02:18:46.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.199+0000 7f72afe00700 1 --2- 172.21.15.19:0/3317076421 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f729c03e9a0 0x7f729c040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:18:46.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.199+0000 7f72afe00700 1 -- 172.21.15.19:0/3317076421 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f72a8053e50 con 0x7f72b8075b60 2024-05-07T02:18:46.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.199+0000 7f72be600700 1 -- 172.21.15.19:0/3317076421 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72b8064e20 con 0x7f72b8075b60 2024-05-07T02:18:46.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.200+0000 7f72b6a00700 1 --2- 172.21.15.19:0/3317076421 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f729c03e9a0 0x7f729c040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:18:46.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.201+0000 7f72b6a00700 1 --2- 172.21.15.19:0/3317076421 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f729c03e9a0 0x7f729c040eb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f72b8100d00 tx=0x7f72a4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:18:46.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.205+0000 7f72afe00700 1 -- 172.21.15.19:0/3317076421 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f72a8031020 con 0x7f72b8075b60 2024-05-07T02:18:46.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.461+0000 7f72be600700 1 -- 172.21.15.19:0/3317076421 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f72b81ad5f0 con 0x7f72b8075b60 2024-05-07T02:18:46.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.461+0000 7f72afe00700 1 -- 172.21.15.19:0/3317076421 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+271 (secure 0 0 0) 0x7f72a8013070 con 0x7f72b8075b60 2024-05-07T02:18:46.461 INFO:teuthology.orchestra.run.smithi019.stdout:# minimal ceph.conf for 9ff84c16-0c17-11ef-bc97-c7b262605968 2024-05-07T02:18:46.461 INFO:teuthology.orchestra.run.smithi019.stdout:[global] 2024-05-07T02:18:46.462 INFO:teuthology.orchestra.run.smithi019.stdout: fsid = 9ff84c16-0c17-11ef-bc97-c7b262605968 2024-05-07T02:18:46.462 INFO:teuthology.orchestra.run.smithi019.stdout: mon_host = [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] 2024-05-07T02:18:46.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.464+0000 7f72be600700 1 -- 172.21.15.19:0/3317076421 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f729c03e9a0 msgr2=0x7f729c040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:18:46.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.464+0000 7f72be600700 1 --2- 172.21.15.19:0/3317076421 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f729c03e9a0 0x7f729c040eb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f72b8100d00 tx=0x7f72a4008040 comp rx=0 tx=0).stop 2024-05-07T02:18:46.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.464+0000 7f72be600700 1 -- 172.21.15.19:0/3317076421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72b8075b60 msgr2=0x7f72b81a02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:18:46.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.464+0000 7f72be600700 1 --2- 172.21.15.19:0/3317076421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72b8075b60 0x7f72b81a02a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f72a8004f80 tx=0x7f72a8005060 comp rx=0 tx=0).stop 2024-05-07T02:18:46.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.464+0000 7f72be600700 1 -- 172.21.15.19:0/3317076421 shutdown_connections 2024-05-07T02:18:46.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.464+0000 7f72be600700 1 --2- 172.21.15.19:0/3317076421 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f729c03e9a0 0x7f729c040eb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:46.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.464+0000 7f72be600700 1 --2- 172.21.15.19:0/3317076421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72b81a5010 0x7f72b81a5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:46.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.464+0000 7f72be600700 1 --2- 172.21.15.19:0/3317076421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72b8075b60 0x7f72b81a02a0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:46.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.464+0000 7f72be600700 1 --2- 172.21.15.19:0/3317076421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72b81a07e0 0x7f72b81a4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:46.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.464+0000 7f72be600700 1 -- 172.21.15.19:0/3317076421 >> 172.21.15.19:0/3317076421 conn(0x7f72b80ff6a0 msgr2=0x7f72b8102190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:18:46.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.465+0000 7f72be600700 1 -- 172.21.15.19:0/3317076421 shutdown_connections 2024-05-07T02:18:46.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:46.465+0000 7f72be600700 1 -- 172.21.15.19:0/3317076421 wait complete. 2024-05-07T02:18:47.142 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:46 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-07T02:18:47.142 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:46 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:47.142 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:46 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:47.142 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:46 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:47.143 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:46 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:47.143 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:46 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/3317076421' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:18:47.143 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:46 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:47.143 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:46 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:47.143 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:46 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:47.143 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:46 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:47.143 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:46 smithi019 ceph-mon[29781]: Reconfiguring mon.a (unknown last config time)... 2024-05-07T02:18:47.143 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:46 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-07T02:18:47.143 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:46 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-07T02:18:47.143 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:46 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:18:47.143 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:46 smithi019 ceph-mon[29781]: Reconfiguring daemon mon.a on smithi019 2024-05-07T02:18:47.193 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-07T02:18:47.194 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-07T02:18:47.194 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/ceph/ceph.conf 2024-05-07T02:18:47.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:46 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-07T02:18:47.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:46 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:47.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:46 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:47.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:46 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:47.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:46 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:47.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:46 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/3317076421' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:18:47.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:46 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:47.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:46 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:47.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:46 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:47.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:46 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:47.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:46 smithi027 ceph-mon[34398]: Reconfiguring mon.a (unknown last config time)... 2024-05-07T02:18:47.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:46 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-07T02:18:47.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:46 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-07T02:18:47.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:46 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:18:47.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:46 smithi027 ceph-mon[34398]: Reconfiguring daemon mon.a on smithi019 2024-05-07T02:18:47.264 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-07T02:18:47.264 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-07T02:18:47.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:46 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-07T02:18:47.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:46 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:47.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:46 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:47.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:46 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:47.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:46 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:47.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:46 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/3317076421' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:18:47.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:46 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:47.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:46 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:47.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:46 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:47.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:46 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:47.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:46 smithi064 ceph-mon[34699]: Reconfiguring mon.a (unknown last config time)... 2024-05-07T02:18:47.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:46 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-07T02:18:47.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:46 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-07T02:18:47.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:46 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:18:47.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:46 smithi064 ceph-mon[34699]: Reconfiguring daemon mon.a on smithi019 2024-05-07T02:18:47.367 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-05-07T02:18:47.367 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/ceph/ceph.conf 2024-05-07T02:18:47.400 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-05-07T02:18:47.400 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-07T02:18:47.468 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-07T02:18:47.469 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.conf 2024-05-07T02:18:47.501 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-07T02:18:47.501 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-07T02:18:47.569 INFO:tasks.cephadm:Adding mgr.x on smithi019 2024-05-07T02:18:47.569 INFO:tasks.cephadm:Adding mgr.y on smithi027 2024-05-07T02:18:47.569 INFO:tasks.cephadm:Adding mgr.z on smithi064 2024-05-07T02:18:47.569 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph orch apply mgr '3;smithi019=x;smithi027=y;smithi064=z' 2024-05-07T02:18:47.853 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.c/config 2024-05-07T02:18:48.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:47 smithi019 ceph-mon[29781]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:48.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:47 smithi027 ceph-mon[34398]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:48.238 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:47 smithi064 ceph-mon[34699]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:49.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.289+0000 7fb5c403a700 1 -- 172.21.15.64:0/3213572346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5bc102c10 msgr2=0x7fb5bc103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:18:49.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.289+0000 7fb5c403a700 1 --2- 172.21.15.64:0/3213572346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5bc102c10 0x7fb5bc103020 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8009910 tx=0x7fb5b8009c20 comp rx=0 tx=0).stop 2024-05-07T02:18:49.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.289+0000 7fb5c403a700 1 -- 172.21.15.64:0/3213572346 shutdown_connections 2024-05-07T02:18:49.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.289+0000 7fb5c403a700 1 --2- 172.21.15.64:0/3213572346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5bc102c10 0x7fb5bc103020 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:49.291 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.289+0000 7fb5c403a700 1 -- 172.21.15.64:0/3213572346 >> 172.21.15.64:0/3213572346 conn(0x7fb5bc0fe250 msgr2=0x7fb5bc100700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:18:49.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.289+0000 7fb5c403a700 1 -- 172.21.15.64:0/3213572346 shutdown_connections 2024-05-07T02:18:49.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.289+0000 7fb5c403a700 1 -- 172.21.15.64:0/3213572346 wait complete. 2024-05-07T02:18:49.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.290+0000 7fb5c403a700 1 Processor -- start 2024-05-07T02:18:49.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.290+0000 7fb5c403a700 1 -- start start 2024-05-07T02:18:49.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.290+0000 7fb5c403a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5bc102c10 0x7fb5bc199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:18:49.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.290+0000 7fb5c403a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5bc199ec0 0x7fb5bc19ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:18:49.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.290+0000 7fb5c403a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5bc19f390 0x7fb5bc1a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:18:49.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.290+0000 7fb5c403a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc108bd0 con 0x7fb5bc102c10 2024-05-07T02:18:49.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.290+0000 7fb5c403a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc108a50 con 0x7fb5bc19f390 2024-05-07T02:18:49.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.290+0000 7fb5c403a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc108d50 con 0x7fb5bc199ec0 2024-05-07T02:18:49.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.290+0000 7fb5c15d5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5bc199ec0 0x7fb5bc19ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:18:49.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.290+0000 7fb5c15d5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5bc199ec0 0x7fb5bc19ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35824/0 (socket says 172.21.15.64:35824) 2024-05-07T02:18:49.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.290+0000 7fb5c15d5700 1 -- 172.21.15.64:0/3355011494 learned_addr learned my addr 172.21.15.64:0/3355011494 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-07T02:18:49.292 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.290+0000 7fb5c25d7700 1 --2- 172.21.15.64:0/3355011494 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5bc19f390 0x7fb5bc1a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:18:49.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.290+0000 7fb5c1dd6700 1 --2- 172.21.15.64:0/3355011494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5bc102c10 0x7fb5bc199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:18:49.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.291+0000 7fb5c15d5700 1 -- 172.21.15.64:0/3355011494 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5bc19f390 msgr2=0x7fb5bc1a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:18:49.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.291+0000 7fb5c15d5700 1 --2- 172.21.15.64:0/3355011494 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5bc19f390 0x7fb5bc1a17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:49.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.291+0000 7fb5c15d5700 1 -- 172.21.15.64:0/3355011494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5bc102c10 msgr2=0x7fb5bc199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:18:49.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.291+0000 7fb5c15d5700 1 --2- 172.21.15.64:0/3355011494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5bc102c10 0x7fb5bc199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:49.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.291+0000 7fb5c15d5700 1 -- 172.21.15.64:0/3355011494 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5bc1a1d20 con 0x7fb5bc199ec0 2024-05-07T02:18:49.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.291+0000 7fb5c15d5700 1 --2- 172.21.15.64:0/3355011494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5bc199ec0 0x7fb5bc19ee50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac00b780 tx=0x7fb5ac00ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:18:49.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.291+0000 7fb5b2ffd700 1 -- 172.21.15.64:0/3355011494 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac004230 con 0x7fb5bc199ec0 2024-05-07T02:18:49.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.291+0000 7fb5b2ffd700 1 -- 172.21.15.64:0/3355011494 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5ac0043d0 con 0x7fb5bc199ec0 2024-05-07T02:18:49.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.291+0000 7fb5c403a700 1 -- 172.21.15.64:0/3355011494 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5bc0fe750 con 0x7fb5bc199ec0 2024-05-07T02:18:49.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.291+0000 7fb5b2ffd700 1 -- 172.21.15.64:0/3355011494 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac011b20 con 0x7fb5bc199ec0 2024-05-07T02:18:49.293 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.291+0000 7fb5c403a700 1 -- 172.21.15.64:0/3355011494 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5bc0fed10 con 0x7fb5bc199ec0 2024-05-07T02:18:49.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.292+0000 7fb5b2ffd700 1 -- 172.21.15.64:0/3355011494 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fb5ac011cc0 con 0x7fb5bc199ec0 2024-05-07T02:18:49.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.292+0000 7fb5c403a700 1 -- 172.21.15.64:0/3355011494 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5bc064e20 con 0x7fb5bc199ec0 2024-05-07T02:18:49.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.292+0000 7fb5b2ffd700 1 --2- 172.21.15.64:0/3355011494 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fb5a803e9a0 0x7fb5a8040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:18:49.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.292+0000 7fb5b2ffd700 1 -- 172.21.15.64:0/3355011494 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb5ac052310 con 0x7fb5bc199ec0 2024-05-07T02:18:49.294 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.293+0000 7fb5c1dd6700 1 --2- 172.21.15.64:0/3355011494 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fb5a803e9a0 0x7fb5a8040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:18:49.295 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.294+0000 7fb5c1dd6700 1 --2- 172.21.15.64:0/3355011494 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fb5a803e9a0 0x7fb5a8040eb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8006010 tx=0x7fb5b801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:18:49.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.298+0000 7fb5b2ffd700 1 -- 172.21.15.64:0/3355011494 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb5ac020740 con 0x7fb5bc199ec0 2024-05-07T02:18:49.528 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.528+0000 7fb5c403a700 1 -- 172.21.15.64:0/3355011494 --> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi019=x;smithi027=y;smithi064=z", "target": ["mon-mgr", ""]}) -- 0x7fb5bc063c90 con 0x7fb5a803e9a0 2024-05-07T02:18:49.544 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.544+0000 7fb5b2ffd700 1 -- 172.21.15.64:0/3355011494 <== mgr.14152 v2:172.21.15.19:6800/2059564551 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fb5bc063c90 con 0x7fb5a803e9a0 2024-05-07T02:18:49.544 INFO:teuthology.orchestra.run.smithi064.stdout:Scheduled mgr update... 2024-05-07T02:18:49.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.546+0000 7fb5c403a700 1 -- 172.21.15.64:0/3355011494 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fb5a803e9a0 msgr2=0x7fb5a8040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:18:49.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.546+0000 7fb5c403a700 1 --2- 172.21.15.64:0/3355011494 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fb5a803e9a0 0x7fb5a8040eb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8006010 tx=0x7fb5b801a040 comp rx=0 tx=0).stop 2024-05-07T02:18:49.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.546+0000 7fb5c403a700 1 -- 172.21.15.64:0/3355011494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5bc199ec0 msgr2=0x7fb5bc19ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:18:49.546 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.546+0000 7fb5c403a700 1 --2- 172.21.15.64:0/3355011494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5bc199ec0 0x7fb5bc19ee50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac00b780 tx=0x7fb5ac00ba90 comp rx=0 tx=0).stop 2024-05-07T02:18:49.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.546+0000 7fb5c403a700 1 -- 172.21.15.64:0/3355011494 shutdown_connections 2024-05-07T02:18:49.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.546+0000 7fb5c403a700 1 --2- 172.21.15.64:0/3355011494 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fb5a803e9a0 0x7fb5a8040eb0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:49.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.546+0000 7fb5c403a700 1 --2- 172.21.15.64:0/3355011494 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5bc19f390 0x7fb5bc1a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:49.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.546+0000 7fb5c403a700 1 --2- 172.21.15.64:0/3355011494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5bc102c10 0x7fb5bc199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:49.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.546+0000 7fb5c403a700 1 --2- 172.21.15.64:0/3355011494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb5bc199ec0 0x7fb5bc19ee50 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:49.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.546+0000 7fb5c403a700 1 -- 172.21.15.64:0/3355011494 >> 172.21.15.64:0/3355011494 conn(0x7fb5bc0fe250 msgr2=0x7fb5bc06c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:18:49.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.546+0000 7fb5c403a700 1 -- 172.21.15.64:0/3355011494 shutdown_connections 2024-05-07T02:18:49.547 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:18:49.546+0000 7fb5c403a700 1 -- 172.21.15.64:0/3355011494 wait complete. 2024-05-07T02:18:49.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:49 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:49.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:49 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:49.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:49 smithi064 ceph-mon[34699]: Reconfiguring mon.b (monmap changed)... 2024-05-07T02:18:49.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:49 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-07T02:18:49.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:49 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-07T02:18:49.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:49 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:18:49.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:49 smithi064 ceph-mon[34699]: Reconfiguring daemon mon.b on smithi027 2024-05-07T02:18:49.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:49 smithi064 ceph-mon[34699]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:49.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:49 smithi064 ceph-mon[34699]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi019=x;smithi027=y;smithi064=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:18:49.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:49 smithi064 ceph-mon[34699]: Saving service mgr spec with placement smithi019=x;smithi027=y;smithi064=z;count:3 2024-05-07T02:18:49.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:49 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:49.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:49 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:49.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:49 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:49.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:49 smithi027 ceph-mon[34398]: Reconfiguring mon.b (monmap changed)... 2024-05-07T02:18:49.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:49 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-07T02:18:49.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:49 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-07T02:18:49.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:49 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:18:49.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:49 smithi027 ceph-mon[34398]: Reconfiguring daemon mon.b on smithi027 2024-05-07T02:18:49.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:49 smithi027 ceph-mon[34398]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:49.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:49 smithi027 ceph-mon[34398]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi019=x;smithi027=y;smithi064=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:18:49.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:49 smithi027 ceph-mon[34398]: Saving service mgr spec with placement smithi019=x;smithi027=y;smithi064=z;count:3 2024-05-07T02:18:49.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:49 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:50.099 DEBUG:teuthology.orchestra.run.smithi027:mgr.y> sudo journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mgr.y.service 2024-05-07T02:18:50.102 DEBUG:teuthology.orchestra.run.smithi064:mgr.z> sudo journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mgr.z.service 2024-05-07T02:18:50.104 INFO:tasks.cephadm:Deploying OSDs... 2024-05-07T02:18:50.104 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-07T02:18:50.104 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/scratch_devs of=/dev/stdout 2024-05-07T02:18:50.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:49 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:50.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:49 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:50.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:49 smithi019 ceph-mon[29781]: Reconfiguring mon.b (monmap changed)... 2024-05-07T02:18:50.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:49 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-07T02:18:50.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:49 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-07T02:18:50.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:49 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:18:50.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:49 smithi019 ceph-mon[29781]: Reconfiguring daemon mon.b on smithi027 2024-05-07T02:18:50.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:49 smithi019 ceph-mon[29781]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:50.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:49 smithi019 ceph-mon[29781]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi019=x;smithi027=y;smithi064=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:18:50.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:49 smithi019 ceph-mon[29781]: Saving service mgr spec with placement smithi019=x;smithi027=y;smithi064=z;count:3 2024-05-07T02:18:50.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:49 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:50.164 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-07T02:18:50.164 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_1 2024-05-07T02:18:50.266 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-07T02:18:50.266 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-07T02:18:50.266 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 6h/6d Inode: 632 Links: 1 2024-05-07T02:18:50.266 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-07T02:18:50.267 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-05-07T02:18:50.267 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-05-07 02:17:49.271872491 +0000 2024-05-07T02:18:50.267 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-05-07 02:12:13.553362869 +0000 2024-05-07T02:18:50.267 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-05-07 02:12:13.553362869 +0000 2024-05-07T02:18:50.267 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-05-07 02:12:13.553362869 +0000 2024-05-07T02:18:50.267 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-07T02:18:50.356 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-05-07T02:18:50.357 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-05-07T02:18:50.357 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000723814 s, 707 kB/s 2024-05-07T02:18:50.361 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-07T02:18:50.418 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_2 2024-05-07T02:18:50.467 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-07T02:18:50.467 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-07T02:18:50.467 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-05-07T02:18:50.467 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-07T02:18:50.467 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-05-07T02:18:50.467 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-05-07 02:17:49.374871376 +0000 2024-05-07T02:18:50.467 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-05-07 02:12:14.172356694 +0000 2024-05-07T02:18:50.468 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-05-07 02:12:14.172356694 +0000 2024-05-07T02:18:50.468 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-05-07 02:12:14.172356694 +0000 2024-05-07T02:18:50.468 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-07T02:18:50.537 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-05-07T02:18:50.537 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-05-07T02:18:50.537 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000390997 s, 1.3 MB/s 2024-05-07T02:18:50.540 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-07T02:18:50.597 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_3 2024-05-07T02:18:50.649 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-07T02:18:50.649 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-07T02:18:50.649 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 6h/6d Inode: 672 Links: 1 2024-05-07T02:18:50.649 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-07T02:18:50.650 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-05-07T02:18:50.650 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-05-07 02:17:49.439870672 +0000 2024-05-07T02:18:50.650 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-05-07 02:12:14.804350389 +0000 2024-05-07T02:18:50.650 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-05-07 02:12:14.804350389 +0000 2024-05-07T02:18:50.650 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-05-07 02:12:14.804350389 +0000 2024-05-07T02:18:50.650 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-07T02:18:50.731 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-05-07T02:18:50.731 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-05-07T02:18:50.731 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000365332 s, 1.4 MB/s 2024-05-07T02:18:50.735 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-07T02:18:50.791 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_4 2024-05-07T02:18:50.844 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-07T02:18:50.844 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-07T02:18:50.844 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-05-07T02:18:50.844 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-07T02:18:50.844 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-05-07T02:18:50.844 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-05-07 02:17:49.513869871 +0000 2024-05-07T02:18:50.844 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-05-07 02:12:15.388344564 +0000 2024-05-07T02:18:50.844 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-05-07 02:12:15.388344564 +0000 2024-05-07T02:18:50.844 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-05-07 02:12:15.388344564 +0000 2024-05-07T02:18:50.845 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-07T02:18:50.926 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-05-07T02:18:50.926 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-05-07T02:18:50.926 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000401985 s, 1.3 MB/s 2024-05-07T02:18:50.930 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-07T02:18:50.988 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-05-07T02:18:50.988 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/scratch_devs of=/dev/stdout 2024-05-07T02:18:51.007 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-07T02:18:51.007 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_1 2024-05-07T02:18:51.066 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-07T02:18:51.066 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-07T02:18:51.067 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-05-07T02:18:51.067 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-07T02:18:51.067 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-05-07T02:18:51.067 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-05-07 02:18:37.531060639 +0000 2024-05-07T02:18:51.067 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-05-07 02:11:15.682536935 +0000 2024-05-07T02:18:51.067 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-05-07 02:11:15.682536935 +0000 2024-05-07T02:18:51.068 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-05-07 02:11:15.682536935 +0000 2024-05-07T02:18:51.068 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-07T02:18:51.136 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-05-07T02:18:51.136 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-05-07T02:18:51.137 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000288933 s, 1.8 MB/s 2024-05-07T02:18:51.138 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-07T02:18:51.198 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_2 2024-05-07T02:18:51.257 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-07T02:18:51.257 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-07T02:18:51.257 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-05-07T02:18:51.257 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-07T02:18:51.258 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-05-07T02:18:51.258 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-05-07 02:18:37.531060639 +0000 2024-05-07T02:18:51.258 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-05-07 02:11:16.032528062 +0000 2024-05-07T02:18:51.258 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-05-07 02:11:16.032528062 +0000 2024-05-07T02:18:51.258 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-05-07 02:11:16.032528062 +0000 2024-05-07T02:18:51.258 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-07T02:18:51.326 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-05-07T02:18:51.326 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-05-07T02:18:51.326 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000194604 s, 2.6 MB/s 2024-05-07T02:18:51.328 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-07T02:18:51.390 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_3 2024-05-07T02:18:51.451 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-07T02:18:51.451 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-07T02:18:51.451 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-07T02:18:51.451 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-07T02:18:51.451 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-05-07T02:18:51.451 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-05-07 02:18:37.532060613 +0000 2024-05-07T02:18:51.451 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-05-07 02:11:16.391518962 +0000 2024-05-07T02:18:51.451 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-05-07 02:11:16.391518962 +0000 2024-05-07T02:18:51.452 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-05-07 02:11:16.391518962 +0000 2024-05-07T02:18:51.452 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-07T02:18:51.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:51 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:51.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:51 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:51.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:51 smithi027 ceph-mon[34398]: Reconfiguring mon.c (monmap changed)... 2024-05-07T02:18:51.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:51 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-07T02:18:51.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:51 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-07T02:18:51.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:51 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:18:51.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:51 smithi027 ceph-mon[34398]: Reconfiguring daemon mon.c on smithi064 2024-05-07T02:18:51.482 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-05-07T02:18:51.483 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-05-07T02:18:51.483 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000223187 s, 2.3 MB/s 2024-05-07T02:18:51.484 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-07T02:18:51.543 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_4 2024-05-07T02:18:51.587 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:51 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:51.587 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:51 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:51.587 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:51 smithi064 ceph-mon[34699]: Reconfiguring mon.c (monmap changed)... 2024-05-07T02:18:51.587 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:51 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-07T02:18:51.587 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:51 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-07T02:18:51.587 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:51 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:18:51.587 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:51 smithi064 ceph-mon[34699]: Reconfiguring daemon mon.c on smithi064 2024-05-07T02:18:51.601 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-07T02:18:51.601 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-07T02:18:51.601 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-05-07T02:18:51.601 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-07T02:18:51.601 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-05-07T02:18:51.601 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-05-07 02:18:37.532060613 +0000 2024-05-07T02:18:51.601 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-05-07 02:11:16.738510166 +0000 2024-05-07T02:18:51.602 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-05-07 02:11:16.738510166 +0000 2024-05-07T02:18:51.602 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-05-07 02:11:16.738510166 +0000 2024-05-07T02:18:51.602 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-07T02:18:51.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:51 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:51.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:51 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:51.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:51 smithi019 ceph-mon[29781]: Reconfiguring mon.c (monmap changed)... 2024-05-07T02:18:51.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:51 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-07T02:18:51.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:51 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-07T02:18:51.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:51 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:18:51.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:51 smithi019 ceph-mon[29781]: Reconfiguring daemon mon.c on smithi064 2024-05-07T02:18:51.668 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-05-07T02:18:51.669 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-05-07T02:18:51.669 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000213513 s, 2.4 MB/s 2024-05-07T02:18:51.670 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-07T02:18:51.730 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-07T02:18:51.731 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/scratch_devs of=/dev/stdout 2024-05-07T02:18:51.748 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-07T02:18:51.748 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_1 2024-05-07T02:18:51.806 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-07T02:18:51.806 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-07T02:18:51.806 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-05-07T02:18:51.806 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-07T02:18:51.806 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-05-07T02:18:51.806 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-05-07 02:18:28.584294374 +0000 2024-05-07T02:18:51.806 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-05-07 02:11:14.112651195 +0000 2024-05-07T02:18:51.806 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-05-07 02:11:14.112651195 +0000 2024-05-07T02:18:51.806 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-05-07 02:11:14.112651195 +0000 2024-05-07T02:18:51.807 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-07T02:18:51.874 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-05-07T02:18:51.874 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-05-07T02:18:51.875 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000218789 s, 2.3 MB/s 2024-05-07T02:18:51.876 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-07T02:18:51.933 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_2 2024-05-07T02:18:51.991 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-07T02:18:51.992 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-07T02:18:51.992 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-05-07T02:18:51.992 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-07T02:18:51.992 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-05-07T02:18:51.992 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-05-07 02:18:28.585294354 +0000 2024-05-07T02:18:51.992 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-05-07 02:11:14.477644129 +0000 2024-05-07T02:18:51.992 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-05-07 02:11:14.477644129 +0000 2024-05-07T02:18:51.992 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-05-07 02:11:14.477644129 +0000 2024-05-07T02:18:51.992 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-07T02:18:52.060 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-05-07T02:18:52.061 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-05-07T02:18:52.061 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000180168 s, 2.8 MB/s 2024-05-07T02:18:52.062 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-07T02:18:52.124 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_3 2024-05-07T02:18:52.188 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-07T02:18:52.188 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-07T02:18:52.188 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-05-07T02:18:52.189 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-07T02:18:52.189 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-05-07T02:18:52.189 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-05-07 02:18:28.586294335 +0000 2024-05-07T02:18:52.189 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-05-07 02:11:14.825637392 +0000 2024-05-07T02:18:52.189 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-05-07 02:11:14.825637392 +0000 2024-05-07T02:18:52.189 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-05-07 02:11:14.825637392 +0000 2024-05-07T02:18:52.189 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-07T02:18:52.266 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-05-07T02:18:52.266 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-05-07T02:18:52.266 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000190687 s, 2.7 MB/s 2024-05-07T02:18:52.266 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-07T02:18:52.323 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_4 2024-05-07T02:18:52.379 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-07T02:18:52.379 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-07T02:18:52.379 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-05-07T02:18:52.379 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-07T02:18:52.379 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-05-07T02:18:52.379 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-05-07 02:18:28.586294335 +0000 2024-05-07T02:18:52.379 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-05-07 02:11:15.171630694 +0000 2024-05-07T02:18:52.380 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-05-07 02:11:15.171630694 +0000 2024-05-07T02:18:52.380 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-05-07 02:11:15.171630694 +0000 2024-05-07T02:18:52.380 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-07T02:18:52.443 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-05-07T02:18:52.443 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-05-07T02:18:52.443 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000208964 s, 2.5 MB/s 2024-05-07T02:18:52.444 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-07T02:18:52.503 INFO:tasks.cephadm:Deploying osd.0 on smithi019 with /dev/vg_nvme/lv_4... 2024-05-07T02:18:52.503 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-07T02:18:52.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:52 smithi064 ceph-mon[34699]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:52.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:52 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:52.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:52 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:52.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:52 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:18:52.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:52 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:18:52.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:52 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:18:52.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:52 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:52.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:52 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-07T02:18:52.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:52 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-07T02:18:52.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:52 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-07T02:18:52.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:52 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:18:52.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:52 smithi019 ceph-mon[29781]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:52.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:52 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:52.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:52 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:52.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:52 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:18:52.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:52 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:18:52.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:52 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:18:52.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:52 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:52.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:52 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-07T02:18:52.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:52 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-07T02:18:52.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:52 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-07T02:18:52.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:52 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:18:52.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:52 smithi027 ceph-mon[34398]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:52.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:52 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:52.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:52 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:52.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:52 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:18:52.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:52 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:18:52.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:52 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:18:52.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:52 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:52.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:52 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-07T02:18:52.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:52 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-07T02:18:52.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:52 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-07T02:18:52.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:52 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:18:52.954 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:18:53.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:53 smithi019 ceph-mon[29781]: Deploying daemon mgr.z on smithi064 2024-05-07T02:18:53.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:53 smithi027 ceph-mon[34398]: Deploying daemon mgr.z on smithi064 2024-05-07T02:18:53.745 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:53 smithi064 ceph-mon[34699]: Deploying daemon mgr.z on smithi064 2024-05-07T02:18:54.108 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:18:53 smithi064 systemd[1]: Starting Ceph mgr.z for 9ff84c16-0c17-11ef-bc97-c7b262605968... 2024-05-07T02:18:54.440 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:54 smithi064 ceph-mon[34699]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:54.440 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:18:54 smithi064 podman[36557]: 2024-05-07 02:18:54.092503689 +0000 UTC m=+0.018428503 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-07T02:18:54.440 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:18:54 smithi064 podman[36557]: 2024-05-07 02:18:54.222213025 +0000 UTC m=+0.148137845 container create d79d6701e1613fcaaea532bd15d03b769a00e2d42f2746a53a8e3a98e76a713a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430) 2024-05-07T02:18:54.440 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:18:54 smithi064 podman[36557]: 2024-05-07 02:18:54.441221153 +0000 UTC m=+0.367145942 container init d79d6701e1613fcaaea532bd15d03b769a00e2d42f2746a53a8e3a98e76a713a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z, ceph=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-07T02:18:54.641 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:54 smithi019 ceph-mon[29781]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:54.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:54 smithi027 ceph-mon[34398]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:54.812 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:18:54 smithi064 podman[36557]: 2024-05-07 02:18:54.444520592 +0000 UTC m=+0.370445381 container start d79d6701e1613fcaaea532bd15d03b769a00e2d42f2746a53a8e3a98e76a713a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z, org.label-schema.build-date=20240430, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True) 2024-05-07T02:18:54.813 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:18:54 smithi064 ceph-mgr[36576]: -- 172.21.15.64:0/3649970722 <== mon.1 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5586a52ae5a0 con 0x5586aa038480 2024-05-07T02:18:54.813 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:18:54 smithi064 bash[36557]: d79d6701e1613fcaaea532bd15d03b769a00e2d42f2746a53a8e3a98e76a713a 2024-05-07T02:18:54.813 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:18:54 smithi064 systemd[1]: Started Ceph mgr.z for 9ff84c16-0c17-11ef-bc97-c7b262605968. 2024-05-07T02:18:54.813 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:18:54 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:18:54.621+0000 7fdb09227200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-07T02:18:54.813 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:18:54 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:18:54.812+0000 7fdb09227200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-07T02:18:55.108 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:18:54 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:18:54.919+0000 7fdb09227200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-07T02:18:55.580 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:18:55 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:18:55.241+0000 7fdb09227200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-07T02:18:55.580 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:18:55 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:18:55.329+0000 7fdb09227200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-07T02:18:55.706 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-07T02:18:55.706 INFO:teuthology.orchestra.run.smithi019.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-07T02:18:55.706 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10+0 records in 2024-05-07T02:18:55.706 INFO:teuthology.orchestra.run.smithi019.stderr:10+0 records out 2024-05-07T02:18:55.706 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0390748 s, 268 MB/s 2024-05-07T02:18:55.706 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping successful for: 2024-05-07T02:18:55.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:55 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:55.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:55 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:55.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:55 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:55.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:55 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-07T02:18:55.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:55 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-07T02:18:55.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:55 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-07T02:18:55.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:55 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:18:55.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:55 smithi064 ceph-mon[34699]: Deploying daemon mgr.y on smithi027 2024-05-07T02:18:55.889 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:55 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:55.889 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:55 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:55.889 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:55 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:55.889 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:55 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-07T02:18:55.889 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:55 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-07T02:18:55.889 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:55 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-07T02:18:55.889 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:55 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:18:55.889 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:55 smithi027 ceph-mon[34398]: Deploying daemon mgr.y on smithi027 2024-05-07T02:18:55.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:55 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:55.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:55 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:55.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:55 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:55.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:55 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-07T02:18:55.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:55 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-07T02:18:55.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:55 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-07T02:18:55.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:55 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:18:55.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:55 smithi019 ceph-mon[29781]: Deploying daemon mgr.y on smithi027 2024-05-07T02:18:56.479 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi019:vg_nvme/lv_4 2024-05-07T02:18:56.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:56 smithi064 ceph-mon[34699]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:56.858 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:18:56 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:18:56.590+0000 7fdb09227200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-07T02:18:56.859 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:18:56 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:18:56.766+0000 7fdb09227200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-07T02:18:56.886 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:56 smithi027 ceph-mon[34398]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:56.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:56 smithi019 ceph-mon[29781]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:56.943 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:18:57.241 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:18:56 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:18:56.942+0000 7fdb09227200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-07T02:18:57.241 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:18:57 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:18:57.241+0000 7fdb09227200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-07T02:18:57.371 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:18:57 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:18:57.179+0000 7f62d9122200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-07T02:18:57.372 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:18:57 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:18:57.371+0000 7f62d9122200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-07T02:18:57.546 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:18:57 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:18:57.336+0000 7fdb09227200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-07T02:18:57.546 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:18:57 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:18:57.546+0000 7fdb09227200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-07T02:18:57.706 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:18:57 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:18:57.478+0000 7f62d9122200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-07T02:18:57.858 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:18:57 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:18:57.749+0000 7fdb09227200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-07T02:18:58.160 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:18:57 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:18:57.806+0000 7f62d9122200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-07T02:18:58.161 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:18:57 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:18:57.895+0000 7f62d9122200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-07T02:18:58.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:58 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:58.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:58 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:58.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:58 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:58.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:58 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:58.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:58 smithi019 ceph-mon[29781]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:58.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:58 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:18:58.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:58 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:58.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:58 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:58.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:58 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:58.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:58 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:58.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:58 smithi027 ceph-mon[34398]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:58.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:58 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:18:58.479 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:18:58 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:18:58.404+0000 7fdb09227200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-07T02:18:58.479 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:58 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:58.479 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:58 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:58.479 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:58 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:58.479 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:58 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:18:58.479 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:58 smithi064 ceph-mon[34699]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:18:58.479 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:58 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:18:58.858 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:18:58 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:18:58.646+0000 7fdb09227200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-07T02:18:59.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:59.074+0000 7f2a88c00700 1 -- 172.21.15.19:0/4179476445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a84105480 msgr2=0x7f2a84105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:18:59.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:59.074+0000 7f2a88c00700 1 --2- 172.21.15.19:0/4179476445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a84105480 0x7f2a84105890 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c009990 tx=0x7f2a6c009ca0 comp rx=0 tx=0).stop 2024-05-07T02:18:59.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:59.074+0000 7f2a88c00700 1 -- 172.21.15.19:0/4179476445 shutdown_connections 2024-05-07T02:18:59.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:59.074+0000 7f2a88c00700 1 --2- 172.21.15.19:0/4179476445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a84105e20 0x7f2a8410c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:59.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:59.074+0000 7f2a88c00700 1 --2- 172.21.15.19:0/4179476445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a84104350 0x7f2a84104760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:59.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:59.074+0000 7f2a88c00700 1 --2- 172.21.15.19:0/4179476445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a84105480 0x7f2a84105890 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:59.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:59.074+0000 7f2a88c00700 1 -- 172.21.15.19:0/4179476445 >> 172.21.15.19:0/4179476445 conn(0x7f2a840ff990 msgr2=0x7f2a84101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:18:59.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:59.074+0000 7f2a88c00700 1 -- 172.21.15.19:0/4179476445 shutdown_connections 2024-05-07T02:18:59.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:59.074+0000 7f2a88c00700 1 -- 172.21.15.19:0/4179476445 wait complete. 2024-05-07T02:18:59.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:59.076+0000 7f2a88c00700 1 Processor -- start 2024-05-07T02:18:59.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:59.077+0000 7f2a88c00700 1 -- start start 2024-05-07T02:18:59.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:59.077+0000 7f2a88c00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a84104350 0x7f2a8419cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:18:59.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:59.077+0000 7f2a88c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a84105480 0x7f2a8419d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:18:59.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:59.077+0000 7f2a88c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a84105e20 0x7f2a841a4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:18:59.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:59.077+0000 7f2a88c00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8410fe00 con 0x7f2a84105480 2024-05-07T02:18:59.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:59.077+0000 7f2a88c00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8410fc80 con 0x7f2a84105e20 2024-05-07T02:18:59.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:59.077+0000 7f2a88c00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8410ff80 con 0x7f2a84104350 2024-05-07T02:18:59.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:59.077+0000 7f2a81e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a84104350 0x7f2a8419cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:18:59.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:59.077+0000 7f2a81e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a84104350 0x7f2a8419cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:48276/0 (socket says 172.21.15.19:48276) 2024-05-07T02:18:59.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:59.077+0000 7f2a81e00700 1 -- 172.21.15.19:0/1114798553 learned_addr learned my addr 172.21.15.19:0/1114798553 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:18:59.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:59.078+0000 7f2a7be00700 1 --2- 172.21.15.19:0/1114798553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a84105480 0x7f2a8419d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:18:59.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:59.078+0000 7f2a82800700 1 --2- 172.21.15.19:0/1114798553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a84105e20 0x7f2a841a4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:18:59.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:59.078+0000 7f2a81e00700 1 -- 172.21.15.19:0/1114798553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a84105e20 msgr2=0x7f2a841a4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:18:59.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:59.078+0000 7f2a81e00700 1 --2- 172.21.15.19:0/1114798553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a84105e20 0x7f2a841a4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:59.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:59.078+0000 7f2a81e00700 1 -- 172.21.15.19:0/1114798553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a84105480 msgr2=0x7f2a8419d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:18:59.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:59.078+0000 7f2a81e00700 1 --2- 172.21.15.19:0/1114798553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a84105480 0x7f2a8419d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:18:59.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:59.078+0000 7f2a81e00700 1 -- 172.21.15.19:0/1114798553 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a841a46b0 con 0x7f2a84104350 2024-05-07T02:18:59.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:59.078+0000 7f2a81e00700 1 --2- 172.21.15.19:0/1114798553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a84104350 0x7f2a8419cb50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400e940 tx=0x7f2a7400ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:18:59.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:59.078+0000 7f2a7aa00700 1 -- 172.21.15.19:0/1114798553 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a74004220 con 0x7f2a84104350 2024-05-07T02:18:59.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:59.079+0000 7f2a7aa00700 1 -- 172.21.15.19:0/1114798553 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a740043c0 con 0x7f2a84104350 2024-05-07T02:18:59.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:59.079+0000 7f2a7aa00700 1 -- 172.21.15.19:0/1114798553 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a74003c60 con 0x7f2a84104350 2024-05-07T02:18:59.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:59.079+0000 7f2a88c00700 1 -- 172.21.15.19:0/1114798553 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a841a49a0 con 0x7f2a84104350 2024-05-07T02:18:59.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:59.079+0000 7f2a88c00700 1 -- 172.21.15.19:0/1114798553 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a841a4e90 con 0x7f2a84104350 2024-05-07T02:18:59.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:59.080+0000 7f2a88c00700 1 -- 172.21.15.19:0/1114798553 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a840512c0 con 0x7f2a84104350 2024-05-07T02:18:59.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:59.080+0000 7f2a7aa00700 1 -- 172.21.15.19:0/1114798553 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f2a7400c7e0 con 0x7f2a84104350 2024-05-07T02:18:59.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:59.081+0000 7f2a7aa00700 1 --2- 172.21.15.19:0/1114798553 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f2a6803e750 0x7f2a68040c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:18:59.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:59.081+0000 7f2a7aa00700 1 -- 172.21.15.19:0/1114798553 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2a74052ac0 con 0x7f2a84104350 2024-05-07T02:18:59.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:59.081+0000 7f2a7be00700 1 --2- 172.21.15.19:0/1114798553 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f2a6803e750 0x7f2a68040c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:18:59.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:59.082+0000 7f2a7be00700 1 --2- 172.21.15.19:0/1114798553 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f2a6803e750 0x7f2a68040c60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2a8419dc00 tx=0x7f2a6c00b720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:18:59.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:59.086+0000 7f2a7aa00700 1 -- 172.21.15.19:0/1114798553 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2a74054e20 con 0x7f2a84104350 2024-05-07T02:18:59.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:18:59.337+0000 7f2a88c00700 1 -- 172.21.15.19:0/1114798553 --> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f2a84077110 con 0x7f2a6803e750 2024-05-07T02:18:59.358 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:18:59 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:18:59.097+0000 7fdb09227200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-07T02:18:59.358 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:18:59 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:18:59.198+0000 7fdb09227200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-07T02:18:59.456 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:18:59 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:18:59.199+0000 7f62d9122200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-07T02:18:59.456 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:18:59 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:18:59.374+0000 7f62d9122200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-07T02:18:59.847 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:18:59 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:18:59.550+0000 7f62d9122200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-07T02:18:59.853 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:18:59 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:18:59.590+0000 7fdb09227200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-07T02:19:00.108 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:18:59 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:18:59.963+0000 7fdb09227200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-07T02:19:00.108 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:19:00 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:19:00.058+0000 7fdb09227200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-07T02:19:00.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:59 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:00.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:59 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:00.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:59 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:00.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:59 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:00.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:59 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:19:00.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:59 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:19:00.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:59 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:00.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:59 smithi064 ceph-mon[34699]: Reconfiguring mgr.x (unknown last config time)... 2024-05-07T02:19:00.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:59 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-07T02:19:00.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:59 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-07T02:19:00.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:59 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:19:00.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:59 smithi064 ceph-mon[34699]: Reconfiguring daemon mgr.x on smithi019 2024-05-07T02:19:00.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:59 smithi064 ceph-mon[34699]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:00.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:59 smithi064 ceph-mon[34699]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:19:00.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:59 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-07T02:19:00.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:59 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-07T02:19:00.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:18:59 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:19:00.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:59 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:00.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:59 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:00.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:59 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:00.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:59 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:00.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:59 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:19:00.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:59 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:19:00.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:59 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:00.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:59 smithi019 ceph-mon[29781]: Reconfiguring mgr.x (unknown last config time)... 2024-05-07T02:19:00.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:59 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-07T02:19:00.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:59 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-07T02:19:00.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:59 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:19:00.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:59 smithi019 ceph-mon[29781]: Reconfiguring daemon mgr.x on smithi019 2024-05-07T02:19:00.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:59 smithi019 ceph-mon[29781]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:00.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:59 smithi019 ceph-mon[29781]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:19:00.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:59 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-07T02:19:00.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:59 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-07T02:19:00.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:18:59 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:19:00.152 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:18:59 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:18:59.847+0000 7f62d9122200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-07T02:19:00.153 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:18:59 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:18:59.943+0000 7f62d9122200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-07T02:19:00.153 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:19:00 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:19:00.153+0000 7f62d9122200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-07T02:19:00.153 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:59 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:00.153 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:59 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:00.153 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:59 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:00.153 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:59 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:00.153 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:59 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:19:00.154 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:59 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:19:00.154 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:59 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:00.154 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:59 smithi027 ceph-mon[34398]: Reconfiguring mgr.x (unknown last config time)... 2024-05-07T02:19:00.154 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:59 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-07T02:19:00.154 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:59 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-07T02:19:00.154 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:59 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:19:00.154 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:59 smithi027 ceph-mon[34398]: Reconfiguring daemon mgr.x on smithi019 2024-05-07T02:19:00.154 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:59 smithi027 ceph-mon[34398]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:00.154 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:59 smithi027 ceph-mon[34398]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:19:00.154 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:59 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-07T02:19:00.154 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:59 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-07T02:19:00.155 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:18:59 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:19:00.456 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:19:00 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:19:00.350+0000 7f62d9122200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-07T02:19:00.608 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:19:00 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:19:00.309+0000 7fdb09227200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-07T02:19:00.608 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:19:00 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:19:00.447+0000 7fdb09227200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-07T02:19:01.108 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:19:00 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:19:00.761+0000 7fdb09227200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-07T02:19:01.108 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:19:00 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:19:00.880+0000 7fdb09227200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-07T02:19:01.108 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:19:00 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:19:00.974+0000 7fdb09227200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-07T02:19:01.456 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:19:00 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:19:00.974+0000 7f62d9122200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-07T02:19:01.456 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:19:01 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:19:01.202+0000 7f62d9122200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-07T02:19:01.858 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:19:01 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:19:01.603+0000 7fdb09227200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-07T02:19:01.858 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:19:01 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:19:01.828+0000 7fdb09227200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-07T02:19:01.956 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:19:01 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:19:01.641+0000 7f62d9122200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-07T02:19:01.956 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:19:01 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:19:01.740+0000 7f62d9122200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-07T02:19:02.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:19:02.362+0000 7f2a7aa00700 1 -- 172.21.15.19:0/1114798553 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 14) ==== 99937+0+0 (secure 0 0 0) 0x7f2a7401dd20 con 0x7f2a84104350 2024-05-07T02:19:02.384 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:02 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:02.384 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:02 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:02.384 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:02 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:19:02.384 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:02 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:19:02.384 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:02 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:19:02.384 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:02 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:02.384 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:02 smithi019 ceph-mon[29781]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:02.384 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:02 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:02.384 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:02 smithi019 ceph-mon[29781]: Standby manager daemon z started 2024-05-07T02:19:02.384 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:02 smithi019 ceph-mon[29781]: from='mgr.? 172.21.15.64:0/664781271' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-07T02:19:02.384 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:02 smithi019 ceph-mon[29781]: from='mgr.? 172.21.15.64:0/664781271' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-07T02:19:02.384 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:02 smithi019 ceph-mon[29781]: from='mgr.? 172.21.15.64:0/664781271' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-07T02:19:02.385 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:02 smithi019 ceph-mon[29781]: from='mgr.? 172.21.15.64:0/664781271' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-07T02:19:02.456 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:19:02 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:19:02.121+0000 7f62d9122200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-07T02:19:02.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:02 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:02.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:02 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:02.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:02 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:19:02.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:02 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:19:02.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:02 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:19:02.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:02 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:02.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:02 smithi027 ceph-mon[34398]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:02.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:02 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:02.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:02 smithi027 ceph-mon[34398]: Standby manager daemon z started 2024-05-07T02:19:02.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:02 smithi027 ceph-mon[34398]: from='mgr.? 172.21.15.64:0/664781271' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-07T02:19:02.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:02 smithi027 ceph-mon[34398]: from='mgr.? 172.21.15.64:0/664781271' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-07T02:19:02.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:02 smithi027 ceph-mon[34398]: from='mgr.? 172.21.15.64:0/664781271' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-07T02:19:02.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:02 smithi027 ceph-mon[34398]: from='mgr.? 172.21.15.64:0/664781271' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-07T02:19:02.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:02 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:02.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:02 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:02.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:02 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:19:02.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:02 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:19:02.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:02 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:19:02.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:02 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:02.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:02 smithi064 ceph-mon[34699]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:02.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:02 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:02.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:02 smithi064 ceph-mon[34699]: Standby manager daemon z started 2024-05-07T02:19:02.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:02 smithi064 ceph-mon[34699]: from='mgr.? 172.21.15.64:0/664781271' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-07T02:19:02.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:02 smithi064 ceph-mon[34699]: from='mgr.? 172.21.15.64:0/664781271' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-07T02:19:02.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:02 smithi064 ceph-mon[34699]: from='mgr.? 172.21.15.64:0/664781271' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-07T02:19:02.610 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:02 smithi064 ceph-mon[34699]: from='mgr.? 172.21.15.64:0/664781271' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-07T02:19:02.837 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:19:02 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:19:02.493+0000 7f62d9122200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-07T02:19:02.838 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:19:02 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:19:02.587+0000 7f62d9122200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-07T02:19:03.206 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:19:02 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:19:02.837+0000 7f62d9122200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-07T02:19:03.206 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:19:02 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:19:02.974+0000 7f62d9122200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-07T02:19:03.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:03 smithi019 ceph-mon[29781]: mgrmap e14: x(active, since 91s), standbys: z 2024-05-07T02:19:03.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:03 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-07T02:19:03.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:03 smithi027 ceph-mon[34398]: mgrmap e14: x(active, since 91s), standbys: z 2024-05-07T02:19:03.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:03 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-07T02:19:03.706 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:19:03 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:19:03.289+0000 7f62d9122200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-07T02:19:03.706 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:19:03 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:19:03.408+0000 7f62d9122200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-07T02:19:03.707 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:19:03 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:19:03.502+0000 7f62d9122200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-07T02:19:03.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:03 smithi064 ceph-mon[34699]: mgrmap e14: x(active, since 91s), standbys: z 2024-05-07T02:19:03.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:03 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-07T02:19:04.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:19:04.394+0000 7f2a7aa00700 1 -- 172.21.15.19:0/1114798553 <== mon.1 v2:172.21.15.64:3300/0 8 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f2a7400cc20 con 0x7f2a84104350 2024-05-07T02:19:04.456 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:19:04 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:19:04.125+0000 7f62d9122200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-07T02:19:04.456 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:19:04 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:19:04.347+0000 7f62d9122200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-07T02:19:04.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:04 smithi027 ceph-mon[34398]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:04.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:04 smithi027 ceph-mon[34398]: from='mgr.? 172.21.15.27:0/734569482' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-07T02:19:04.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:04 smithi027 ceph-mon[34398]: Standby manager daemon y started 2024-05-07T02:19:04.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:04 smithi027 ceph-mon[34398]: from='mgr.? 172.21.15.27:0/734569482' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-07T02:19:04.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:04 smithi027 ceph-mon[34398]: from='mgr.? 172.21.15.27:0/734569482' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-07T02:19:04.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:04 smithi027 ceph-mon[34398]: from='mgr.? 172.21.15.27:0/734569482' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-07T02:19:04.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:04 smithi019 ceph-mon[29781]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:04.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:04 smithi019 ceph-mon[29781]: from='mgr.? 172.21.15.27:0/734569482' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-07T02:19:04.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:04 smithi019 ceph-mon[29781]: Standby manager daemon y started 2024-05-07T02:19:04.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:04 smithi019 ceph-mon[29781]: from='mgr.? 172.21.15.27:0/734569482' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-07T02:19:04.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:04 smithi019 ceph-mon[29781]: from='mgr.? 172.21.15.27:0/734569482' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-07T02:19:04.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:04 smithi019 ceph-mon[29781]: from='mgr.? 172.21.15.27:0/734569482' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-07T02:19:04.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:04 smithi064 ceph-mon[34699]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:04.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:04 smithi064 ceph-mon[34699]: from='mgr.? 172.21.15.27:0/734569482' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-07T02:19:04.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:04 smithi064 ceph-mon[34699]: Standby manager daemon y started 2024-05-07T02:19:04.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:04 smithi064 ceph-mon[34699]: from='mgr.? 172.21.15.27:0/734569482' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-07T02:19:04.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:04 smithi064 ceph-mon[34699]: from='mgr.? 172.21.15.27:0/734569482' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-07T02:19:04.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:04 smithi064 ceph-mon[34699]: from='mgr.? 172.21.15.27:0/734569482' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-07T02:19:05.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:05 smithi019 ceph-mon[29781]: mgrmap e15: x(active, since 93s), standbys: z, y 2024-05-07T02:19:05.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:05 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-07T02:19:05.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:05 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/2474759772' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b437cf22-ec8e-48f0-86a2-2413a0ab40dd"}]: dispatch 2024-05-07T02:19:05.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:05 smithi019 ceph-mon[29781]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b437cf22-ec8e-48f0-86a2-2413a0ab40dd"}]: dispatch 2024-05-07T02:19:05.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:05 smithi019 ceph-mon[29781]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b437cf22-ec8e-48f0-86a2-2413a0ab40dd"}]': finished 2024-05-07T02:19:05.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:05 smithi019 ceph-mon[29781]: osdmap e5: 1 total, 0 up, 1 in 2024-05-07T02:19:05.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:05 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-07T02:19:05.674 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:05 smithi064 ceph-mon[34699]: mgrmap e15: x(active, since 93s), standbys: z, y 2024-05-07T02:19:05.674 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:05 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-07T02:19:05.675 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:05 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/2474759772' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b437cf22-ec8e-48f0-86a2-2413a0ab40dd"}]: dispatch 2024-05-07T02:19:05.675 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:05 smithi064 ceph-mon[34699]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b437cf22-ec8e-48f0-86a2-2413a0ab40dd"}]: dispatch 2024-05-07T02:19:05.675 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:05 smithi064 ceph-mon[34699]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b437cf22-ec8e-48f0-86a2-2413a0ab40dd"}]': finished 2024-05-07T02:19:05.675 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:05 smithi064 ceph-mon[34699]: osdmap e5: 1 total, 0 up, 1 in 2024-05-07T02:19:05.675 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:05 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-07T02:19:05.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:05 smithi027 ceph-mon[34398]: mgrmap e15: x(active, since 93s), standbys: z, y 2024-05-07T02:19:05.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:05 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-07T02:19:05.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:05 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/2474759772' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b437cf22-ec8e-48f0-86a2-2413a0ab40dd"}]: dispatch 2024-05-07T02:19:05.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:05 smithi027 ceph-mon[34398]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b437cf22-ec8e-48f0-86a2-2413a0ab40dd"}]: dispatch 2024-05-07T02:19:05.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:05 smithi027 ceph-mon[34398]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b437cf22-ec8e-48f0-86a2-2413a0ab40dd"}]': finished 2024-05-07T02:19:05.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:05 smithi027 ceph-mon[34398]: osdmap e5: 1 total, 0 up, 1 in 2024-05-07T02:19:05.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:05 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-07T02:19:06.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:06 smithi019 ceph-mon[29781]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:06.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:06 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/3209014834' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-07T02:19:06.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:06 smithi027 ceph-mon[34398]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:06.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:06 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/3209014834' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-07T02:19:06.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:06 smithi064 ceph-mon[34699]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:06.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:06 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/3209014834' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-07T02:19:08.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:08 smithi027 ceph-mon[34398]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:08.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:08 smithi064 ceph-mon[34699]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:08.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:08 smithi019 ceph-mon[29781]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:10.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:10 smithi027 ceph-mon[34398]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:10.854 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:10 smithi019 ceph-mon[29781]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:10.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:10 smithi064 ceph-mon[34699]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:12.560 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:12 smithi019 ceph-mon[29781]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:12.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:12 smithi027 ceph-mon[34398]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:12.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:12 smithi064 ceph-mon[34699]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:14.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:14 smithi027 ceph-mon[34398]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:14.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:14 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-07T02:19:14.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:14 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:19:14.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:14 smithi064 ceph-mon[34699]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:14.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:14 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-07T02:19:14.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:14 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:19:14.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:14 smithi019 ceph-mon[29781]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:14.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:14 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-07T02:19:14.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:14 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:19:15.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:15 smithi019 ceph-mon[29781]: Deploying daemon osd.0 on smithi019 2024-05-07T02:19:15.676 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:15 smithi064 ceph-mon[34699]: Deploying daemon osd.0 on smithi019 2024-05-07T02:19:15.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:15 smithi027 ceph-mon[34398]: Deploying daemon osd.0 on smithi019 2024-05-07T02:19:16.680 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:16 smithi019 ceph-mon[29781]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:16.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:16 smithi027 ceph-mon[34398]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:16.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:16 smithi064 ceph-mon[34699]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:18.580 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:18 smithi019 ceph-mon[29781]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:18.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:18 smithi027 ceph-mon[34398]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:18.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:18 smithi064 ceph-mon[34699]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:20.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:20 smithi027 ceph-mon[34398]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:20.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:20 smithi064 ceph-mon[34699]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:20.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:20 smithi019 ceph-mon[29781]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:22.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:22 smithi019 ceph-mon[29781]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:22.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:22 smithi027 ceph-mon[34398]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:22.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:22 smithi064 ceph-mon[34699]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:24.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:23 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:24.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:23 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:24.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:23 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:19:24.072 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:23 smithi019 ceph-mon[29781]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:24.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:23 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:24.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:23 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:24.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:23 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:19:24.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:23 smithi064 ceph-mon[34699]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:24.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:23 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:24.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:23 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:24.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:23 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:19:24.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:23 smithi027 ceph-mon[34398]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:26.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:26 smithi064 ceph-mon[34699]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:26.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:26 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:26.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:26 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:26.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:26 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:19:26.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:26 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:19:26.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:26 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:26.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:26 smithi019 ceph-mon[29781]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:26.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:26 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:26.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:26 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:26.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:26 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:19:26.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:26 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:19:26.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:26 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:26.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:26 smithi027 ceph-mon[34398]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:26.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:26 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:26.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:26 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:26.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:26 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:19:26.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:26 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:19:26.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:26 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:27.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:27 smithi064 ceph-mon[34699]: from='osd.0 [v2:172.21.15.19:6802/2390539683,v1:172.21.15.19:6803/2390539683]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-07T02:19:27.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:27 smithi064 ceph-mon[34699]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-07T02:19:27.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:27 smithi019 ceph-mon[29781]: from='osd.0 [v2:172.21.15.19:6802/2390539683,v1:172.21.15.19:6803/2390539683]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-07T02:19:27.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:27 smithi019 ceph-mon[29781]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-07T02:19:27.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:27 smithi027 ceph-mon[34398]: from='osd.0 [v2:172.21.15.19:6802/2390539683,v1:172.21.15.19:6803/2390539683]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-07T02:19:27.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:27 smithi027 ceph-mon[34398]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-07T02:19:28.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:19:28.287+0000 7f2a7aa00700 1 -- 172.21.15.19:0/1114798553 <== mgr.14152 v2:172.21.15.19:6800/2059564551 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2a84077110 con 0x7f2a6803e750 2024-05-07T02:19:28.289 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 0 on host 'smithi019' 2024-05-07T02:19:28.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:19:28.291+0000 7f2a88c00700 1 -- 172.21.15.19:0/1114798553 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f2a6803e750 msgr2=0x7f2a68040c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:19:28.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:19:28.291+0000 7f2a88c00700 1 --2- 172.21.15.19:0/1114798553 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f2a6803e750 0x7f2a68040c60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2a8419dc00 tx=0x7f2a6c00b720 comp rx=0 tx=0).stop 2024-05-07T02:19:28.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:19:28.291+0000 7f2a88c00700 1 -- 172.21.15.19:0/1114798553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a84104350 msgr2=0x7f2a8419cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:19:28.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:19:28.291+0000 7f2a88c00700 1 --2- 172.21.15.19:0/1114798553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a84104350 0x7f2a8419cb50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400e940 tx=0x7f2a7400ec50 comp rx=0 tx=0).stop 2024-05-07T02:19:28.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:19:28.291+0000 7f2a88c00700 1 -- 172.21.15.19:0/1114798553 shutdown_connections 2024-05-07T02:19:28.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:19:28.291+0000 7f2a88c00700 1 --2- 172.21.15.19:0/1114798553 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f2a6803e750 0x7f2a68040c60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:19:28.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:19:28.291+0000 7f2a88c00700 1 --2- 172.21.15.19:0/1114798553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a84105e20 0x7f2a841a4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:19:28.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:19:28.291+0000 7f2a88c00700 1 --2- 172.21.15.19:0/1114798553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a84104350 0x7f2a8419cb50 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:19:28.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:19:28.291+0000 7f2a88c00700 1 --2- 172.21.15.19:0/1114798553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a84105480 0x7f2a8419d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:19:28.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:19:28.291+0000 7f2a88c00700 1 -- 172.21.15.19:0/1114798553 >> 172.21.15.19:0/1114798553 conn(0x7f2a840ff990 msgr2=0x7f2a840759f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:19:28.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:19:28.291+0000 7f2a88c00700 1 -- 172.21.15.19:0/1114798553 shutdown_connections 2024-05-07T02:19:28.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:19:28.291+0000 7f2a88c00700 1 -- 172.21.15.19:0/1114798553 wait complete. 2024-05-07T02:19:28.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:28 smithi019 ceph-mon[29781]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:28.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:28 smithi019 ceph-mon[29781]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-07T02:19:28.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:28 smithi019 ceph-mon[29781]: from='osd.0 [v2:172.21.15.19:6802/2390539683,v1:172.21.15.19:6803/2390539683]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-07T02:19:28.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:28 smithi019 ceph-mon[29781]: osdmap e6: 1 total, 0 up, 1 in 2024-05-07T02:19:28.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:28 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-07T02:19:28.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:28 smithi019 ceph-mon[29781]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-07T02:19:28.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:28 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:28.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:28 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:28.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:28 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:19:28.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:28 smithi027 ceph-mon[34398]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:28.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:28 smithi027 ceph-mon[34398]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-07T02:19:28.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:28 smithi027 ceph-mon[34398]: from='osd.0 [v2:172.21.15.19:6802/2390539683,v1:172.21.15.19:6803/2390539683]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-07T02:19:28.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:28 smithi027 ceph-mon[34398]: osdmap e6: 1 total, 0 up, 1 in 2024-05-07T02:19:28.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:28 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-07T02:19:28.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:28 smithi027 ceph-mon[34398]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-07T02:19:28.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:28 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:28.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:28 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:28.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:28 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:19:28.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:28 smithi064 ceph-mon[34699]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:28.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:28 smithi064 ceph-mon[34699]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-07T02:19:28.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:28 smithi064 ceph-mon[34699]: from='osd.0 [v2:172.21.15.19:6802/2390539683,v1:172.21.15.19:6803/2390539683]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-07T02:19:28.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:28 smithi064 ceph-mon[34699]: osdmap e6: 1 total, 0 up, 1 in 2024-05-07T02:19:28.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:28 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-07T02:19:28.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:28 smithi064 ceph-mon[34699]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-07T02:19:28.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:28 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:28.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:28 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:28.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:28 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:19:29.098 DEBUG:teuthology.orchestra.run.smithi019:osd.0> sudo journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@osd.0.service 2024-05-07T02:19:29.103 INFO:tasks.cephadm:Deploying osd.1 on smithi027 with /dev/vg_nvme/lv_4... 2024-05-07T02:19:29.103 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-07T02:19:29.357 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 07 02:19:29 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-0[44288]: 2024-05-07T02:19:29.238+0000 7fa549600700 -1 osd.0 0 waiting for initial osdmap 2024-05-07T02:19:29.357 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 07 02:19:29 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-0[44288]: 2024-05-07T02:19:29.256+0000 7fa53f400700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-07T02:19:29.359 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.b/config 2024-05-07T02:19:29.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:29 smithi027 ceph-mon[34398]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-07T02:19:29.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:29 smithi027 ceph-mon[34398]: osdmap e7: 1 total, 0 up, 1 in 2024-05-07T02:19:29.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:29 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-07T02:19:29.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:29 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-07T02:19:29.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:29 smithi027 ceph-mon[34398]: from='osd.0 ' entity='osd.0' 2024-05-07T02:19:29.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:29 smithi019 ceph-mon[29781]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-07T02:19:29.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:29 smithi019 ceph-mon[29781]: osdmap e7: 1 total, 0 up, 1 in 2024-05-07T02:19:29.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:29 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-07T02:19:29.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:29 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-07T02:19:29.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:29 smithi019 ceph-mon[29781]: from='osd.0 ' entity='osd.0' 2024-05-07T02:19:29.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:29 smithi064 ceph-mon[34699]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-07T02:19:29.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:29 smithi064 ceph-mon[34699]: osdmap e7: 1 total, 0 up, 1 in 2024-05-07T02:19:29.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:29 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-07T02:19:29.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:29 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-07T02:19:29.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:29 smithi064 ceph-mon[34699]: from='osd.0 ' entity='osd.0' 2024-05-07T02:19:30.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:30 smithi027 ceph-mon[34398]: purged_snaps scrub starts 2024-05-07T02:19:30.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:30 smithi027 ceph-mon[34398]: purged_snaps scrub ok 2024-05-07T02:19:30.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:30 smithi027 ceph-mon[34398]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:30.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:30 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-07T02:19:30.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:30 smithi027 ceph-mon[34398]: osd.0 [v2:172.21.15.19:6802/2390539683,v1:172.21.15.19:6803/2390539683] boot 2024-05-07T02:19:30.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:30 smithi027 ceph-mon[34398]: osdmap e8: 1 total, 1 up, 1 in 2024-05-07T02:19:30.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:30 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-07T02:19:30.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:30 smithi019 ceph-mon[29781]: purged_snaps scrub starts 2024-05-07T02:19:30.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:30 smithi019 ceph-mon[29781]: purged_snaps scrub ok 2024-05-07T02:19:30.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:30 smithi019 ceph-mon[29781]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:30.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:30 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-07T02:19:30.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:30 smithi019 ceph-mon[29781]: osd.0 [v2:172.21.15.19:6802/2390539683,v1:172.21.15.19:6803/2390539683] boot 2024-05-07T02:19:30.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:30 smithi019 ceph-mon[29781]: osdmap e8: 1 total, 1 up, 1 in 2024-05-07T02:19:30.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:30 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-07T02:19:30.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:30 smithi064 ceph-mon[34699]: purged_snaps scrub starts 2024-05-07T02:19:30.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:30 smithi064 ceph-mon[34699]: purged_snaps scrub ok 2024-05-07T02:19:30.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:30 smithi064 ceph-mon[34699]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-07T02:19:30.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:30 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-07T02:19:30.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:30 smithi064 ceph-mon[34699]: osd.0 [v2:172.21.15.19:6802/2390539683,v1:172.21.15.19:6803/2390539683] boot 2024-05-07T02:19:30.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:30 smithi064 ceph-mon[34699]: osdmap e8: 1 total, 1 up, 1 in 2024-05-07T02:19:30.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:30 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-07T02:19:31.302 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-07T02:19:31.303 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-07T02:19:31.303 INFO:teuthology.orchestra.run.smithi027.stderr: stderr: 10+0 records in 2024-05-07T02:19:31.303 INFO:teuthology.orchestra.run.smithi027.stderr:10+0 records out 2024-05-07T02:19:31.303 INFO:teuthology.orchestra.run.smithi027.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111313 s, 942 MB/s 2024-05-07T02:19:31.303 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping successful for: 2024-05-07T02:19:31.901 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi027:vg_nvme/lv_4 2024-05-07T02:19:32.144 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.b/config 2024-05-07T02:19:32.520 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:32 smithi027 ceph-mon[34398]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-07T02:19:32.520 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:32 smithi027 ceph-mon[34398]: osdmap e9: 1 total, 1 up, 1 in 2024-05-07T02:19:32.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:32 smithi019 ceph-mon[29781]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-07T02:19:32.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:32 smithi019 ceph-mon[29781]: osdmap e9: 1 total, 1 up, 1 in 2024-05-07T02:19:32.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:32 smithi064 ceph-mon[34699]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-07T02:19:32.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:32 smithi064 ceph-mon[34699]: osdmap e9: 1 total, 1 up, 1 in 2024-05-07T02:19:33.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:33.775+0000 7fe69e471700 1 -- 172.21.15.27:0/2173511192 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe69810c210 msgr2=0x7fe69810e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:19:33.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:33.775+0000 7fe69e471700 1 --2- 172.21.15.27:0/2173511192 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe69810c210 0x7fe69810e720 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe69000b0a0 tx=0x7fe69000b3b0 comp rx=0 tx=0).stop 2024-05-07T02:19:33.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:33.775+0000 7fe69e471700 1 -- 172.21.15.27:0/2173511192 shutdown_connections 2024-05-07T02:19:33.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:33.775+0000 7fe69e471700 1 --2- 172.21.15.27:0/2173511192 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe69810c210 0x7fe69810e720 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:19:33.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:33.775+0000 7fe69e471700 1 --2- 172.21.15.27:0/2173511192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe698076690 0x7fe6981092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:19:33.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:33.775+0000 7fe69e471700 1 --2- 172.21.15.27:0/2173511192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe698109820 0x7fe69810bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:19:33.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:33.775+0000 7fe69e471700 1 -- 172.21.15.27:0/2173511192 >> 172.21.15.27:0/2173511192 conn(0x7fe6980fe890 msgr2=0x7fe698100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:19:33.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:33.775+0000 7fe69e471700 1 -- 172.21.15.27:0/2173511192 shutdown_connections 2024-05-07T02:19:33.775 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:33.775+0000 7fe69e471700 1 -- 172.21.15.27:0/2173511192 wait complete. 2024-05-07T02:19:33.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:33.776+0000 7fe69e471700 1 Processor -- start 2024-05-07T02:19:33.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:33.776+0000 7fe69e471700 1 -- start start 2024-05-07T02:19:33.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:33.776+0000 7fe69e471700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe698076690 0x7fe69819cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:19:33.777 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:33.776+0000 7fe69e471700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe698109820 0x7fe69819d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:19:33.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:33.776+0000 7fe69e471700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe69810c210 0x7fe6981a4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:19:33.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:33.776+0000 7fe69e471700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe698112070 con 0x7fe698109820 2024-05-07T02:19:33.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:33.776+0000 7fe69e471700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe698111ef0 con 0x7fe69810c210 2024-05-07T02:19:33.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:33.776+0000 7fe69e471700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe6981121f0 con 0x7fe698076690 2024-05-07T02:19:33.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:33.776+0000 7fe69ca0e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe69810c210 0x7fe6981a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:19:33.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:33.777+0000 7fe69ca0e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe69810c210 0x7fe6981a4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40640/0 (socket says 172.21.15.27:40640) 2024-05-07T02:19:33.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:33.777+0000 7fe69ca0e700 1 -- 172.21.15.27:0/2082237737 learned_addr learned my addr 172.21.15.27:0/2082237737 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-07T02:19:33.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:33.777+0000 7fe697fff700 1 --2- 172.21.15.27:0/2082237737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe698076690 0x7fe69819cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:19:33.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:33.777+0000 7fe69ca0e700 1 -- 172.21.15.27:0/2082237737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe698076690 msgr2=0x7fe69819cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:19:33.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:33.777+0000 7fe69ca0e700 1 --2- 172.21.15.27:0/2082237737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe698076690 0x7fe69819cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:19:33.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:33.777+0000 7fe69ca0e700 1 -- 172.21.15.27:0/2082237737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe698109820 msgr2=0x7fe69819d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:19:33.778 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:33.777+0000 7fe69ca0e700 1 --2- 172.21.15.27:0/2082237737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe698109820 0x7fe69819d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:19:33.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:33.777+0000 7fe69ca0e700 1 -- 172.21.15.27:0/2082237737 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6981a46c0 con 0x7fe69810c210 2024-05-07T02:19:33.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:33.777+0000 7fe69ca0e700 1 --2- 172.21.15.27:0/2082237737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe69810c210 0x7fe6981a4040 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe690009fd0 tx=0x7fe690004e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:19:33.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:33.777+0000 7fe6957fa700 1 -- 172.21.15.27:0/2082237737 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe690021070 con 0x7fe69810c210 2024-05-07T02:19:33.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:33.777+0000 7fe6957fa700 1 -- 172.21.15.27:0/2082237737 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe690007d00 con 0x7fe69810c210 2024-05-07T02:19:33.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:33.777+0000 7fe69e471700 1 -- 172.21.15.27:0/2082237737 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6981a4950 con 0x7fe69810c210 2024-05-07T02:19:33.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:33.777+0000 7fe69e471700 1 -- 172.21.15.27:0/2082237737 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6981a4f10 con 0x7fe69810c210 2024-05-07T02:19:33.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:33.777+0000 7fe6957fa700 1 -- 172.21.15.27:0/2082237737 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe690005420 con 0x7fe69810c210 2024-05-07T02:19:33.779 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:33.779+0000 7fe69e471700 1 -- 172.21.15.27:0/2082237737 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6980512c0 con 0x7fe69810c210 2024-05-07T02:19:33.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:33.780+0000 7fe6957fa700 1 -- 172.21.15.27:0/2082237737 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7fe690024070 con 0x7fe69810c210 2024-05-07T02:19:33.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:33.781+0000 7fe6957fa700 1 --2- 172.21.15.27:0/2082237737 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fe6800b27f0 0x7fe6800b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:19:33.780 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:33.781+0000 7fe6957fa700 1 -- 172.21.15.27:0/2082237737 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fe69000f020 con 0x7fe69810c210 2024-05-07T02:19:33.781 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:33.781+0000 7fe697fff700 1 --2- 172.21.15.27:0/2082237737 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fe6800b27f0 0x7fe6800b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:19:33.782 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:33.782+0000 7fe697fff700 1 --2- 172.21.15.27:0/2082237737 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fe6800b27f0 0x7fe6800b4d00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe688005950 tx=0x7fe6880058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:19:33.786 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:33.786+0000 7fe6957fa700 1 -- 172.21.15.27:0/2082237737 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe69002b030 con 0x7fe69810c210 2024-05-07T02:19:34.010 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:34.010+0000 7fe69e471700 1 -- 172.21.15.27:0/2082237737 --> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fe698063c00 con 0x7fe6800b27f0 2024-05-07T02:19:34.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:34 smithi019 ceph-mon[29781]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-07T02:19:34.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:34 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:34.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:34 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:34.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:34 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:19:34.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:34 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:34.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:34 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:19:34.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:34 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:19:34.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:34 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:34.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:34 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-07T02:19:34.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:34 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-07T02:19:34.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:34 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:19:34.665 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:34 smithi027 ceph-mon[34398]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-07T02:19:34.665 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:34 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:34.665 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:34 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:34.665 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:34 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:19:34.665 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:34 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:34.665 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:34 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:19:34.666 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:34 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:19:34.666 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:34 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:34.666 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:34 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-07T02:19:34.666 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:34 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-07T02:19:34.666 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:34 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:19:34.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:34 smithi064 ceph-mon[34699]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-07T02:19:34.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:34 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:34.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:34 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:34.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:34 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:19:34.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:34 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:34.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:34 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:19:34.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:34 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:19:34.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:34 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:34.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:34 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-07T02:19:34.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:34 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-07T02:19:34.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:34 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:19:35.668 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:35 smithi027 ceph-mon[34398]: Detected new or changed devices on smithi019 2024-05-07T02:19:35.669 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:35 smithi027 ceph-mon[34398]: Adjusting osd_memory_target on smithi019 to 17094M 2024-05-07T02:19:35.669 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:35 smithi027 ceph-mon[34398]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:19:35.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:35 smithi064 ceph-mon[34699]: Detected new or changed devices on smithi019 2024-05-07T02:19:35.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:35 smithi064 ceph-mon[34699]: Adjusting osd_memory_target on smithi019 to 17094M 2024-05-07T02:19:35.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:35 smithi064 ceph-mon[34699]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:19:35.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:35 smithi019 ceph-mon[29781]: Detected new or changed devices on smithi019 2024-05-07T02:19:35.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:35 smithi019 ceph-mon[29781]: Adjusting osd_memory_target on smithi019 to 17094M 2024-05-07T02:19:35.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:35 smithi019 ceph-mon[29781]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:19:36.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:36 smithi027 ceph-mon[34398]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-07T02:19:36.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:36 smithi064 ceph-mon[34699]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-07T02:19:36.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:36 smithi019 ceph-mon[29781]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-07T02:19:37.415 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:37 smithi027 ceph-mon[34398]: from='client.? 172.21.15.27:0/2179847784' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc753e16-634d-4c14-abd2-935281e1499e"}]: dispatch 2024-05-07T02:19:37.416 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:37 smithi027 ceph-mon[34398]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc753e16-634d-4c14-abd2-935281e1499e"}]: dispatch 2024-05-07T02:19:37.416 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:37 smithi027 ceph-mon[34398]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dc753e16-634d-4c14-abd2-935281e1499e"}]': finished 2024-05-07T02:19:37.416 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:37 smithi027 ceph-mon[34398]: osdmap e10: 2 total, 1 up, 2 in 2024-05-07T02:19:37.416 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:37 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-07T02:19:37.416 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:37 smithi027 ceph-mon[34398]: from='client.? 172.21.15.27:0/3203014560' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-07T02:19:37.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:37 smithi064 ceph-mon[34699]: from='client.? 172.21.15.27:0/2179847784' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc753e16-634d-4c14-abd2-935281e1499e"}]: dispatch 2024-05-07T02:19:37.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:37 smithi064 ceph-mon[34699]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc753e16-634d-4c14-abd2-935281e1499e"}]: dispatch 2024-05-07T02:19:37.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:37 smithi064 ceph-mon[34699]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dc753e16-634d-4c14-abd2-935281e1499e"}]': finished 2024-05-07T02:19:37.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:37 smithi064 ceph-mon[34699]: osdmap e10: 2 total, 1 up, 2 in 2024-05-07T02:19:37.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:37 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-07T02:19:37.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:37 smithi064 ceph-mon[34699]: from='client.? 172.21.15.27:0/3203014560' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-07T02:19:37.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:37 smithi019 ceph-mon[29781]: from='client.? 172.21.15.27:0/2179847784' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc753e16-634d-4c14-abd2-935281e1499e"}]: dispatch 2024-05-07T02:19:37.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:37 smithi019 ceph-mon[29781]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc753e16-634d-4c14-abd2-935281e1499e"}]: dispatch 2024-05-07T02:19:37.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:37 smithi019 ceph-mon[29781]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dc753e16-634d-4c14-abd2-935281e1499e"}]': finished 2024-05-07T02:19:37.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:37 smithi019 ceph-mon[29781]: osdmap e10: 2 total, 1 up, 2 in 2024-05-07T02:19:37.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:37 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-07T02:19:37.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:37 smithi019 ceph-mon[29781]: from='client.? 172.21.15.27:0/3203014560' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-07T02:19:38.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:38 smithi027 ceph-mon[34398]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-07T02:19:39.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:38 smithi064 ceph-mon[34699]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-07T02:19:39.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:38 smithi019 ceph-mon[29781]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-07T02:19:40.936 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:40 smithi027 ceph-mon[34398]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-07T02:19:41.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:40 smithi064 ceph-mon[34699]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-07T02:19:41.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:40 smithi019 ceph-mon[29781]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-07T02:19:42.773 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:42 smithi027 ceph-mon[34398]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-07T02:19:43.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:42 smithi064 ceph-mon[34699]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-07T02:19:43.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:42 smithi019 ceph-mon[29781]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-07T02:19:44.826 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:44 smithi027 ceph-mon[34398]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-07T02:19:44.827 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:44 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-07T02:19:44.827 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:44 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:19:44.924 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:44 smithi064 ceph-mon[34699]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-07T02:19:44.924 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:44 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-07T02:19:44.924 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:44 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:19:45.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:44 smithi019 ceph-mon[29781]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-07T02:19:45.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:44 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-07T02:19:45.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:44 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:19:45.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:45 smithi027 ceph-mon[34398]: Deploying daemon osd.1 on smithi027 2024-05-07T02:19:46.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:45 smithi064 ceph-mon[34699]: Deploying daemon osd.1 on smithi027 2024-05-07T02:19:46.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:45 smithi019 ceph-mon[29781]: Deploying daemon osd.1 on smithi027 2024-05-07T02:19:46.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:46 smithi027 ceph-mon[34398]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-07T02:19:47.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:46 smithi064 ceph-mon[34699]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-07T02:19:47.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:46 smithi019 ceph-mon[29781]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-07T02:19:48.844 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:48 smithi027 ceph-mon[34398]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-07T02:19:48.944 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:48 smithi064 ceph-mon[34699]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-07T02:19:49.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:48 smithi019 ceph-mon[29781]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-07T02:19:50.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:50 smithi019 ceph-mon[29781]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-07T02:19:50.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:50 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:50.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:50 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:50.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:50 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:19:50.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:50 smithi027 ceph-mon[34398]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-07T02:19:50.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:50 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:50.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:50 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:50.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:50 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:19:50.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:50 smithi064 ceph-mon[34699]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-07T02:19:50.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:50 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:50.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:50 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:50.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:50 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:19:52.621 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:52 smithi027 ceph-mon[34398]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-07T02:19:52.621 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:52 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:52.621 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:52 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:52.621 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:52 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:19:52.621 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:52 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:19:52.621 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:52 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:52.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:52 smithi019 ceph-mon[29781]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-07T02:19:52.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:52 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:52.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:52 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:52.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:52 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:19:52.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:52 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:19:52.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:52 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:52.700 INFO:teuthology.orchestra.run.smithi027.stdout:Created osd(s) 1 on host 'smithi027' 2024-05-07T02:19:52.701 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:52.700+0000 7fe6957fa700 1 -- 172.21.15.27:0/2082237737 <== mgr.14152 v2:172.21.15.19:6800/2059564551 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe698063c00 con 0x7fe6800b27f0 2024-05-07T02:19:52.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:52.703+0000 7fe69e471700 1 -- 172.21.15.27:0/2082237737 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fe6800b27f0 msgr2=0x7fe6800b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:19:52.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:52.703+0000 7fe69e471700 1 --2- 172.21.15.27:0/2082237737 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fe6800b27f0 0x7fe6800b4d00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe688005950 tx=0x7fe6880058e0 comp rx=0 tx=0).stop 2024-05-07T02:19:52.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:52.703+0000 7fe69e471700 1 -- 172.21.15.27:0/2082237737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe69810c210 msgr2=0x7fe6981a4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:19:52.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:52.703+0000 7fe69e471700 1 --2- 172.21.15.27:0/2082237737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe69810c210 0x7fe6981a4040 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe690009fd0 tx=0x7fe690004e50 comp rx=0 tx=0).stop 2024-05-07T02:19:52.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:52.703+0000 7fe69e471700 1 -- 172.21.15.27:0/2082237737 shutdown_connections 2024-05-07T02:19:52.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:52.703+0000 7fe69e471700 1 --2- 172.21.15.27:0/2082237737 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fe6800b27f0 0x7fe6800b4d00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:19:52.703 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:52.703+0000 7fe69e471700 1 --2- 172.21.15.27:0/2082237737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe69810c210 0x7fe6981a4040 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:19:52.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:52.703+0000 7fe69e471700 1 --2- 172.21.15.27:0/2082237737 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe698076690 0x7fe69819cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:19:52.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:52.703+0000 7fe69e471700 1 --2- 172.21.15.27:0/2082237737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe698109820 0x7fe69819d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:19:52.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:52.703+0000 7fe69e471700 1 -- 172.21.15.27:0/2082237737 >> 172.21.15.27:0/2082237737 conn(0x7fe6980fe890 msgr2=0x7fe69810a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:19:52.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:52.703+0000 7fe69e471700 1 -- 172.21.15.27:0/2082237737 shutdown_connections 2024-05-07T02:19:52.704 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:19:52.703+0000 7fe69e471700 1 -- 172.21.15.27:0/2082237737 wait complete. 2024-05-07T02:19:52.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:52 smithi064 ceph-mon[34699]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-07T02:19:52.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:52 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:52.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:52 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:52.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:52 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:19:52.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:52 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:19:52.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:52 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:53.295 DEBUG:teuthology.orchestra.run.smithi027:osd.1> sudo journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@osd.1.service 2024-05-07T02:19:53.303 INFO:tasks.cephadm:Deploying osd.2 on smithi064 with /dev/vg_nvme/lv_4... 2024-05-07T02:19:53.303 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-07T02:19:53.552 INFO:journalctl@ceph.osd.1.smithi027.stdout:May 07 02:19:53 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-1[39018]: 2024-05-07T02:19:53.339+0000 7f8184b657c0 -1 osd.1 0 log_to_monitors true 2024-05-07T02:19:53.559 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.c/config 2024-05-07T02:19:53.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:53 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:53.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:53 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:53.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:53 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:19:53.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:53 smithi027 ceph-mon[34398]: from='osd.1 [v2:172.21.15.27:6800/4292470615,v1:172.21.15.27:6801/4292470615]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-07T02:19:53.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:53 smithi027 ceph-mon[34398]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-07T02:19:53.985 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:53 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:53.985 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:53 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:53.985 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:53 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:19:53.985 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:53 smithi064 ceph-mon[34699]: from='osd.1 [v2:172.21.15.27:6800/4292470615,v1:172.21.15.27:6801/4292470615]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-07T02:19:53.985 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:53 smithi064 ceph-mon[34699]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-07T02:19:54.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:53 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:54.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:53 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:54.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:53 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:19:54.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:53 smithi019 ceph-mon[29781]: from='osd.1 [v2:172.21.15.27:6800/4292470615,v1:172.21.15.27:6801/4292470615]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-07T02:19:54.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:53 smithi019 ceph-mon[29781]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-07T02:19:54.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:54 smithi027 ceph-mon[34398]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-07T02:19:54.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:54 smithi027 ceph-mon[34398]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-07T02:19:54.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:54 smithi027 ceph-mon[34398]: from='osd.1 [v2:172.21.15.27:6800/4292470615,v1:172.21.15.27:6801/4292470615]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-05-07T02:19:54.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:54 smithi027 ceph-mon[34398]: osdmap e11: 2 total, 1 up, 2 in 2024-05-07T02:19:54.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:54 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-07T02:19:54.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:54 smithi027 ceph-mon[34398]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-05-07T02:19:54.957 INFO:journalctl@ceph.osd.1.smithi027.stdout:May 07 02:19:54 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-1[39018]: 2024-05-07T02:19:54.692+0000 7f817a1c9700 -1 osd.1 0 waiting for initial osdmap 2024-05-07T02:19:54.957 INFO:journalctl@ceph.osd.1.smithi027.stdout:May 07 02:19:54 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-1[39018]: 2024-05-07T02:19:54.698+0000 7f8175f7e700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-07T02:19:54.999 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:54 smithi064 ceph-mon[34699]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-07T02:19:54.999 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:54 smithi064 ceph-mon[34699]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-07T02:19:54.999 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:54 smithi064 ceph-mon[34699]: from='osd.1 [v2:172.21.15.27:6800/4292470615,v1:172.21.15.27:6801/4292470615]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-05-07T02:19:54.999 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:54 smithi064 ceph-mon[34699]: osdmap e11: 2 total, 1 up, 2 in 2024-05-07T02:19:54.999 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:54 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-07T02:19:54.999 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:54 smithi064 ceph-mon[34699]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-05-07T02:19:55.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:54 smithi019 ceph-mon[29781]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-07T02:19:55.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:54 smithi019 ceph-mon[29781]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-07T02:19:55.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:54 smithi019 ceph-mon[29781]: from='osd.1 [v2:172.21.15.27:6800/4292470615,v1:172.21.15.27:6801/4292470615]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-05-07T02:19:55.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:54 smithi019 ceph-mon[29781]: osdmap e11: 2 total, 1 up, 2 in 2024-05-07T02:19:55.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:54 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-07T02:19:55.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:54 smithi019 ceph-mon[29781]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-05-07T02:19:55.464 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-07T02:19:55.464 INFO:teuthology.orchestra.run.smithi064.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-07T02:19:55.465 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10+0 records in 2024-05-07T02:19:55.465 INFO:teuthology.orchestra.run.smithi064.stderr:10+0 records out 2024-05-07T02:19:55.465 INFO:teuthology.orchestra.run.smithi064.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0128381 s, 817 MB/s 2024-05-07T02:19:55.465 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping successful for: 2024-05-07T02:19:55.780 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:55 smithi027 ceph-mon[34398]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-05-07T02:19:55.780 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:55 smithi027 ceph-mon[34398]: osdmap e12: 2 total, 1 up, 2 in 2024-05-07T02:19:55.780 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:55 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-07T02:19:55.780 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:55 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-07T02:19:55.780 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:55 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-07T02:19:55.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:55 smithi064 ceph-mon[34699]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-05-07T02:19:55.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:55 smithi064 ceph-mon[34699]: osdmap e12: 2 total, 1 up, 2 in 2024-05-07T02:19:55.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:55 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-07T02:19:55.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:55 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-07T02:19:55.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:55 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-07T02:19:55.859 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:55 smithi019 ceph-mon[29781]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-05-07T02:19:55.859 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:55 smithi019 ceph-mon[29781]: osdmap e12: 2 total, 1 up, 2 in 2024-05-07T02:19:55.859 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:55 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-07T02:19:55.859 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:55 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-07T02:19:55.859 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:55 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-07T02:19:56.097 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi064:vg_nvme/lv_4 2024-05-07T02:19:56.339 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.c/config 2024-05-07T02:19:56.772 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:56 smithi064 ceph-mon[34699]: purged_snaps scrub starts 2024-05-07T02:19:56.772 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:56 smithi064 ceph-mon[34699]: purged_snaps scrub ok 2024-05-07T02:19:56.772 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:56 smithi064 ceph-mon[34699]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-07T02:19:56.772 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:56 smithi064 ceph-mon[34699]: osd.1 [v2:172.21.15.27:6800/4292470615,v1:172.21.15.27:6801/4292470615] boot 2024-05-07T02:19:56.772 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:56 smithi064 ceph-mon[34699]: osdmap e13: 2 total, 2 up, 2 in 2024-05-07T02:19:56.772 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:56 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-07T02:19:56.772 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:56 smithi064 ceph-mon[34699]: Detected new or changed devices on smithi027 2024-05-07T02:19:56.772 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:56 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:56.773 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:56 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:56.773 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:56 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:19:56.773 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:56 smithi064 ceph-mon[34699]: Adjusting osd_memory_target on smithi027 to 17088M 2024-05-07T02:19:56.773 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:56 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:56.773 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:56 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:19:56.773 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:56 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:19:56.773 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:56 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:57.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:56 smithi019 ceph-mon[29781]: purged_snaps scrub starts 2024-05-07T02:19:57.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:56 smithi019 ceph-mon[29781]: purged_snaps scrub ok 2024-05-07T02:19:57.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:56 smithi019 ceph-mon[29781]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-07T02:19:57.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:56 smithi019 ceph-mon[29781]: osd.1 [v2:172.21.15.27:6800/4292470615,v1:172.21.15.27:6801/4292470615] boot 2024-05-07T02:19:57.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:56 smithi019 ceph-mon[29781]: osdmap e13: 2 total, 2 up, 2 in 2024-05-07T02:19:57.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:56 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-07T02:19:57.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:56 smithi019 ceph-mon[29781]: Detected new or changed devices on smithi027 2024-05-07T02:19:57.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:56 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:57.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:56 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:57.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:56 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:19:57.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:56 smithi019 ceph-mon[29781]: Adjusting osd_memory_target on smithi027 to 17088M 2024-05-07T02:19:57.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:56 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:57.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:56 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:19:57.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:56 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:19:57.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:56 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:57.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:56 smithi027 ceph-mon[34398]: purged_snaps scrub starts 2024-05-07T02:19:57.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:56 smithi027 ceph-mon[34398]: purged_snaps scrub ok 2024-05-07T02:19:57.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:56 smithi027 ceph-mon[34398]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-07T02:19:57.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:56 smithi027 ceph-mon[34398]: osd.1 [v2:172.21.15.27:6800/4292470615,v1:172.21.15.27:6801/4292470615] boot 2024-05-07T02:19:57.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:56 smithi027 ceph-mon[34398]: osdmap e13: 2 total, 2 up, 2 in 2024-05-07T02:19:57.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:56 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-07T02:19:57.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:56 smithi027 ceph-mon[34398]: Detected new or changed devices on smithi027 2024-05-07T02:19:57.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:56 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:57.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:56 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:57.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:56 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:19:57.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:56 smithi027 ceph-mon[34398]: Adjusting osd_memory_target on smithi027 to 17088M 2024-05-07T02:19:57.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:56 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:57.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:56 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:19:57.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:56 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:19:57.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:56 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:19:57.865 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:19:57.863+0000 7f5a90558700 1 -- 172.21.15.64:0/23922690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a88104d10 msgr2=0x7f5a8810b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:19:57.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:19:57.863+0000 7f5a90558700 1 --2- 172.21.15.64:0/23922690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a88104d10 0x7f5a8810b5c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5a8000b0a0 tx=0x7f5a8000b3b0 comp rx=0 tx=0).stop 2024-05-07T02:19:57.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:19:57.864+0000 7f5a90558700 1 -- 172.21.15.64:0/23922690 shutdown_connections 2024-05-07T02:19:57.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:19:57.864+0000 7f5a90558700 1 --2- 172.21.15.64:0/23922690 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a88104d10 0x7f5a8810b5c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:19:57.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:19:57.864+0000 7f5a90558700 1 --2- 172.21.15.64:0/23922690 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a88103240 0x7f5a88103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:19:57.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:19:57.864+0000 7f5a90558700 1 --2- 172.21.15.64:0/23922690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a88104370 0x7f5a88104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:19:57.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:19:57.864+0000 7f5a90558700 1 -- 172.21.15.64:0/23922690 >> 172.21.15.64:0/23922690 conn(0x7f5a880fe880 msgr2=0x7f5a88100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:19:57.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:19:57.864+0000 7f5a90558700 1 -- 172.21.15.64:0/23922690 shutdown_connections 2024-05-07T02:19:57.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:19:57.864+0000 7f5a90558700 1 -- 172.21.15.64:0/23922690 wait complete. 2024-05-07T02:19:57.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:19:57.864+0000 7f5a90558700 1 Processor -- start 2024-05-07T02:19:57.866 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:19:57.864+0000 7f5a90558700 1 -- start start 2024-05-07T02:19:57.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:19:57.865+0000 7f5a90558700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a88103240 0x7f5a8819a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:19:57.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:19:57.865+0000 7f5a90558700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a88104370 0x7f5a8819ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:19:57.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:19:57.865+0000 7f5a90558700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a88104d10 0x7f5a881a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:19:57.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:19:57.865+0000 7f5a90558700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8810fe70 con 0x7f5a88104370 2024-05-07T02:19:57.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:19:57.865+0000 7f5a90558700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8810fcf0 con 0x7f5a88104d10 2024-05-07T02:19:57.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:19:57.865+0000 7f5a90558700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8810fff0 con 0x7f5a88103240 2024-05-07T02:19:57.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:19:57.865+0000 7f5a8e2f4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a88103240 0x7f5a8819a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:19:57.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:19:57.865+0000 7f5a8e2f4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a88103240 0x7f5a8819a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:38754/0 (socket says 172.21.15.64:38754) 2024-05-07T02:19:57.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:19:57.865+0000 7f5a8e2f4700 1 -- 172.21.15.64:0/278349485 learned_addr learned my addr 172.21.15.64:0/278349485 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-07T02:19:57.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:19:57.865+0000 7f5a8eaf5700 1 --2- 172.21.15.64:0/278349485 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a88104d10 0x7f5a881a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:19:57.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:19:57.865+0000 7f5a8e2f4700 1 -- 172.21.15.64:0/278349485 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a88104d10 msgr2=0x7f5a881a1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:19:57.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:19:57.865+0000 7f5a8e2f4700 1 --2- 172.21.15.64:0/278349485 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a88104d10 0x7f5a881a1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:19:57.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:19:57.865+0000 7f5a8e2f4700 1 -- 172.21.15.64:0/278349485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a88104370 msgr2=0x7f5a8819ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:19:57.867 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:19:57.865+0000 7f5a8daf3700 1 --2- 172.21.15.64:0/278349485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a88104370 0x7f5a8819ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:19:57.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:19:57.865+0000 7f5a8e2f4700 1 --2- 172.21.15.64:0/278349485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a88104370 0x7f5a8819ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:19:57.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:19:57.865+0000 7f5a8e2f4700 1 -- 172.21.15.64:0/278349485 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a881a2490 con 0x7f5a88103240 2024-05-07T02:19:57.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:19:57.865+0000 7f5a8e2f4700 1 --2- 172.21.15.64:0/278349485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a88103240 0x7f5a8819a930 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5a8400c970 tx=0x7f5a8400cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:19:57.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:19:57.865+0000 7f5a7f7fe700 1 -- 172.21.15.64:0/278349485 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8400ce60 con 0x7f5a88103240 2024-05-07T02:19:57.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:19:57.865+0000 7f5a90558700 1 -- 172.21.15.64:0/278349485 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a881a2780 con 0x7f5a88103240 2024-05-07T02:19:57.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:19:57.865+0000 7f5a7f7fe700 1 -- 172.21.15.64:0/278349485 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a84007900 con 0x7f5a88103240 2024-05-07T02:19:57.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:19:57.866+0000 7f5a7f7fe700 1 -- 172.21.15.64:0/278349485 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a84003c70 con 0x7f5a88103240 2024-05-07T02:19:57.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:19:57.866+0000 7f5a90558700 1 -- 172.21.15.64:0/278349485 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a881a2d90 con 0x7f5a88103240 2024-05-07T02:19:57.868 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:19:57.867+0000 7f5a90558700 1 -- 172.21.15.64:0/278349485 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a880512c0 con 0x7f5a88103240 2024-05-07T02:19:57.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:19:57.869+0000 7f5a7f7fe700 1 -- 172.21.15.64:0/278349485 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f5a84004220 con 0x7f5a88103240 2024-05-07T02:19:57.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:19:57.870+0000 7f5a7f7fe700 1 --2- 172.21.15.64:0/278349485 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f5a740b27f0 0x7f5a740b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:19:57.869 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:19:57.870+0000 7f5a7f7fe700 1 -- 172.21.15.64:0/278349485 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f5a840e08b0 con 0x7f5a88103240 2024-05-07T02:19:57.870 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:19:57.870+0000 7f5a8daf3700 1 --2- 172.21.15.64:0/278349485 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f5a740b27f0 0x7f5a740b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:19:57.871 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:19:57.871+0000 7f5a8daf3700 1 --2- 172.21.15.64:0/278349485 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f5a740b27f0 0x7f5a740b4d00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5a8819b9e0 tx=0x7f5a78008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:19:57.873 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:19:57.873+0000 7f5a7f7fe700 1 -- 172.21.15.64:0/278349485 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5a84014030 con 0x7f5a88103240 2024-05-07T02:19:58.096 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:19:58.096+0000 7f5a90558700 1 -- 172.21.15.64:0/278349485 --> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f5a88063c00 con 0x7f5a740b27f0 2024-05-07T02:19:58.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:58 smithi064 ceph-mon[34699]: osdmap e14: 2 total, 2 up, 2 in 2024-05-07T02:19:58.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:58 smithi064 ceph-mon[34699]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-07T02:19:58.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:58 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-07T02:19:58.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:58 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-07T02:19:58.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:58 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:19:58.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:58 smithi019 ceph-mon[29781]: osdmap e14: 2 total, 2 up, 2 in 2024-05-07T02:19:58.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:58 smithi019 ceph-mon[29781]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-07T02:19:58.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:58 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-07T02:19:58.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:58 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-07T02:19:58.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:58 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:19:58.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:58 smithi027 ceph-mon[34398]: osdmap e14: 2 total, 2 up, 2 in 2024-05-07T02:19:58.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:58 smithi027 ceph-mon[34398]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-07T02:19:58.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:58 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-07T02:19:58.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:58 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-07T02:19:58.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:58 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:19:59.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:19:59 smithi064 ceph-mon[34699]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:19:59.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:19:59 smithi019 ceph-mon[29781]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:19:59.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:19:59 smithi027 ceph-mon[34398]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:20:00.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:00 smithi064 ceph-mon[34699]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-07T02:20:00.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:00 smithi064 ceph-mon[34699]: overall HEALTH_OK 2024-05-07T02:20:00.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:00 smithi019 ceph-mon[29781]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-07T02:20:00.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:00 smithi019 ceph-mon[29781]: overall HEALTH_OK 2024-05-07T02:20:00.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:00 smithi027 ceph-mon[34398]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-07T02:20:00.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:00 smithi027 ceph-mon[34398]: overall HEALTH_OK 2024-05-07T02:20:01.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:01 smithi064 ceph-mon[34699]: from='client.? 172.21.15.64:0/3819931771' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40570071-646e-4e66-9a16-c55ab743ac86"}]: dispatch 2024-05-07T02:20:01.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:01 smithi064 ceph-mon[34699]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40570071-646e-4e66-9a16-c55ab743ac86"}]: dispatch 2024-05-07T02:20:01.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:01 smithi064 ceph-mon[34699]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "40570071-646e-4e66-9a16-c55ab743ac86"}]': finished 2024-05-07T02:20:01.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:01 smithi064 ceph-mon[34699]: osdmap e15: 3 total, 2 up, 3 in 2024-05-07T02:20:01.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:01 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-07T02:20:01.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:01 smithi019 ceph-mon[29781]: from='client.? 172.21.15.64:0/3819931771' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40570071-646e-4e66-9a16-c55ab743ac86"}]: dispatch 2024-05-07T02:20:01.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:01 smithi019 ceph-mon[29781]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40570071-646e-4e66-9a16-c55ab743ac86"}]: dispatch 2024-05-07T02:20:01.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:01 smithi019 ceph-mon[29781]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "40570071-646e-4e66-9a16-c55ab743ac86"}]': finished 2024-05-07T02:20:01.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:01 smithi019 ceph-mon[29781]: osdmap e15: 3 total, 2 up, 3 in 2024-05-07T02:20:01.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:01 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-07T02:20:01.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:01 smithi027 ceph-mon[34398]: from='client.? 172.21.15.64:0/3819931771' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40570071-646e-4e66-9a16-c55ab743ac86"}]: dispatch 2024-05-07T02:20:01.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:01 smithi027 ceph-mon[34398]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40570071-646e-4e66-9a16-c55ab743ac86"}]: dispatch 2024-05-07T02:20:01.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:01 smithi027 ceph-mon[34398]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "40570071-646e-4e66-9a16-c55ab743ac86"}]': finished 2024-05-07T02:20:01.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:01 smithi027 ceph-mon[34398]: osdmap e15: 3 total, 2 up, 3 in 2024-05-07T02:20:01.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:01 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-07T02:20:02.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:02 smithi064 ceph-mon[34699]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-07T02:20:02.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:02 smithi064 ceph-mon[34699]: from='client.? 172.21.15.64:0/442955303' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-07T02:20:02.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:02 smithi019 ceph-mon[29781]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-07T02:20:02.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:02 smithi019 ceph-mon[29781]: from='client.? 172.21.15.64:0/442955303' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-07T02:20:02.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:02 smithi027 ceph-mon[34398]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-07T02:20:02.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:02 smithi027 ceph-mon[34398]: from='client.? 172.21.15.64:0/442955303' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-07T02:20:04.611 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:04 smithi064 ceph-mon[34699]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-07T02:20:04.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:04 smithi019 ceph-mon[29781]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-07T02:20:04.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:04 smithi027 ceph-mon[34398]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-07T02:20:06.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:06 smithi064 ceph-mon[34699]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-07T02:20:06.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:06 smithi019 ceph-mon[29781]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-07T02:20:06.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:06 smithi027 ceph-mon[34398]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-07T02:20:08.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:08 smithi019 ceph-mon[29781]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-07T02:20:08.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:08 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-07T02:20:08.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:08 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:20:08.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:08 smithi027 ceph-mon[34398]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-07T02:20:08.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:08 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-07T02:20:08.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:08 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:20:08.814 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:08 smithi064 ceph-mon[34699]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-07T02:20:08.814 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:08 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-07T02:20:08.814 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:08 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:20:09.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:09 smithi019 ceph-mon[29781]: Deploying daemon osd.2 on smithi064 2024-05-07T02:20:09.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:09 smithi027 ceph-mon[34398]: Deploying daemon osd.2 on smithi064 2024-05-07T02:20:09.744 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:09 smithi064 ceph-mon[34699]: Deploying daemon osd.2 on smithi064 2024-05-07T02:20:10.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:10 smithi019 ceph-mon[29781]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-07T02:20:10.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:10 smithi027 ceph-mon[34398]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-07T02:20:10.774 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:10 smithi064 ceph-mon[34699]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-07T02:20:12.640 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:12 smithi064 ceph-mon[34699]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-07T02:20:12.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:12 smithi019 ceph-mon[29781]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-07T02:20:12.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:12 smithi027 ceph-mon[34398]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-07T02:20:14.682 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:14 smithi064 ceph-mon[34699]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-07T02:20:14.682 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:14 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:14.683 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:14 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:20:14.683 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:14 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:14.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:14 smithi027 ceph-mon[34398]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-07T02:20:14.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:14 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:14.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:14 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:20:14.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:14 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:14.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:14 smithi019 ceph-mon[29781]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-07T02:20:14.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:14 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:14.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:14 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:20:14.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:14 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:16.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:16 smithi064 ceph-mon[34699]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-07T02:20:16.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:16 smithi027 ceph-mon[34398]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-07T02:20:16.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:16 smithi019 ceph-mon[29781]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-07T02:20:17.538 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:20:17.539+0000 7f5a7f7fe700 1 -- 172.21.15.64:0/278349485 <== mgr.14152 v2:172.21.15.19:6800/2059564551 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5a88063c00 con 0x7f5a740b27f0 2024-05-07T02:20:17.538 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 2 on host 'smithi064' 2024-05-07T02:20:17.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:20:17.541+0000 7f5a90558700 1 -- 172.21.15.64:0/278349485 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f5a740b27f0 msgr2=0x7f5a740b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:20:17.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:20:17.541+0000 7f5a90558700 1 --2- 172.21.15.64:0/278349485 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f5a740b27f0 0x7f5a740b4d00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5a8819b9e0 tx=0x7f5a78008040 comp rx=0 tx=0).stop 2024-05-07T02:20:17.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:20:17.541+0000 7f5a90558700 1 -- 172.21.15.64:0/278349485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a88103240 msgr2=0x7f5a8819a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:20:17.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:20:17.541+0000 7f5a90558700 1 --2- 172.21.15.64:0/278349485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a88103240 0x7f5a8819a930 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5a8400c970 tx=0x7f5a8400cc80 comp rx=0 tx=0).stop 2024-05-07T02:20:17.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:20:17.541+0000 7f5a90558700 1 -- 172.21.15.64:0/278349485 shutdown_connections 2024-05-07T02:20:17.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:20:17.541+0000 7f5a90558700 1 --2- 172.21.15.64:0/278349485 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f5a740b27f0 0x7f5a740b4d00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:20:17.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:20:17.541+0000 7f5a90558700 1 --2- 172.21.15.64:0/278349485 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a88104d10 0x7f5a881a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:20:17.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:20:17.541+0000 7f5a90558700 1 --2- 172.21.15.64:0/278349485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a88103240 0x7f5a8819a930 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:20:17.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:20:17.541+0000 7f5a90558700 1 --2- 172.21.15.64:0/278349485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a88104370 0x7f5a8819ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:20:17.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:20:17.541+0000 7f5a90558700 1 -- 172.21.15.64:0/278349485 >> 172.21.15.64:0/278349485 conn(0x7f5a880fe880 msgr2=0x7f5a881075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:20:17.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:20:17.541+0000 7f5a90558700 1 -- 172.21.15.64:0/278349485 shutdown_connections 2024-05-07T02:20:17.541 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:20:17.541+0000 7f5a90558700 1 -- 172.21.15.64:0/278349485 wait complete. 2024-05-07T02:20:17.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:17 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:17.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:17 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:17.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:17 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:20:17.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:17 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:20:17.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:17 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:17.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:17 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:17.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:17 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:17.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:17 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:17.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:17 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:20:17.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:17 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:20:17.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:17 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:17.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:17 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:17.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:17 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:17.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:17 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:17.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:17 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:20:17.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:17 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:20:17.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:17 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:17.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:17 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:18.234 DEBUG:teuthology.orchestra.run.smithi064:osd.2> sudo journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@osd.2.service 2024-05-07T02:20:18.236 INFO:tasks.cephadm:Deploying osd.3 on smithi019 with /dev/vg_nvme/lv_3... 2024-05-07T02:20:18.237 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-07T02:20:18.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:18 smithi019 ceph-mon[29781]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-07T02:20:18.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:18 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:18.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:18 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:20:18.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:18 smithi019 ceph-mon[29781]: from='osd.2 [v2:172.21.15.64:6800/1180552939,v1:172.21.15.64:6801/1180552939]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-07T02:20:18.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:18 smithi019 ceph-mon[29781]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-07T02:20:18.692 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:20:18.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:18 smithi064 ceph-mon[34699]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-07T02:20:18.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:18 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:18.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:18 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:20:18.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:18 smithi064 ceph-mon[34699]: from='osd.2 [v2:172.21.15.64:6800/1180552939,v1:172.21.15.64:6801/1180552939]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-07T02:20:18.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:18 smithi064 ceph-mon[34699]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-07T02:20:18.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:18 smithi027 ceph-mon[34398]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-07T02:20:18.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:18 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:18.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:18 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:20:18.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:18 smithi027 ceph-mon[34398]: from='osd.2 [v2:172.21.15.64:6800/1180552939,v1:172.21.15.64:6801/1180552939]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-07T02:20:18.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:18 smithi027 ceph-mon[34398]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-07T02:20:19.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:19 smithi064 ceph-mon[34699]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-07T02:20:19.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:19 smithi064 ceph-mon[34699]: osdmap e16: 3 total, 2 up, 3 in 2024-05-07T02:20:19.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:19 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-07T02:20:19.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:19 smithi064 ceph-mon[34699]: from='osd.2 [v2:172.21.15.64:6800/1180552939,v1:172.21.15.64:6801/1180552939]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-07T02:20:19.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:19 smithi064 ceph-mon[34699]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-07T02:20:19.859 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 07 02:20:19 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-2[39873]: 2024-05-07T02:20:19.563+0000 7fbd599a0700 -1 osd.2 0 waiting for initial osdmap 2024-05-07T02:20:19.859 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 07 02:20:19 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-2[39873]: 2024-05-07T02:20:19.569+0000 7fbd55755700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-07T02:20:19.870 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:19 smithi019 ceph-mon[29781]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-07T02:20:19.870 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:19 smithi019 ceph-mon[29781]: osdmap e16: 3 total, 2 up, 3 in 2024-05-07T02:20:19.870 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:19 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-07T02:20:19.870 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:19 smithi019 ceph-mon[29781]: from='osd.2 [v2:172.21.15.64:6800/1180552939,v1:172.21.15.64:6801/1180552939]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-07T02:20:19.870 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:19 smithi019 ceph-mon[29781]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-07T02:20:19.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:19 smithi027 ceph-mon[34398]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-07T02:20:19.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:19 smithi027 ceph-mon[34398]: osdmap e16: 3 total, 2 up, 3 in 2024-05-07T02:20:19.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:19 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-07T02:20:19.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:19 smithi027 ceph-mon[34398]: from='osd.2 [v2:172.21.15.64:6800/1180552939,v1:172.21.15.64:6801/1180552939]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-07T02:20:19.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:19 smithi027 ceph-mon[34398]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-07T02:20:20.836 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:20 smithi019 ceph-mon[29781]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-07T02:20:20.836 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:20 smithi019 ceph-mon[29781]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-07T02:20:20.836 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:20 smithi019 ceph-mon[29781]: osdmap e17: 3 total, 2 up, 3 in 2024-05-07T02:20:20.836 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:20 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-07T02:20:20.836 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:20 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-07T02:20:20.846 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:20 smithi064 ceph-mon[34699]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-07T02:20:20.846 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:20 smithi064 ceph-mon[34699]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-07T02:20:20.846 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:20 smithi064 ceph-mon[34699]: osdmap e17: 3 total, 2 up, 3 in 2024-05-07T02:20:20.846 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:20 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-07T02:20:20.846 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:20 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-07T02:20:20.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:20 smithi027 ceph-mon[34398]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-07T02:20:20.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:20 smithi027 ceph-mon[34398]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-07T02:20:20.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:20 smithi027 ceph-mon[34398]: osdmap e17: 3 total, 2 up, 3 in 2024-05-07T02:20:20.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:20 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-07T02:20:20.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:20 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-07T02:20:21.759 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-07T02:20:21.759 INFO:teuthology.orchestra.run.smithi019.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-07T02:20:21.759 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10+0 records in 2024-05-07T02:20:21.759 INFO:teuthology.orchestra.run.smithi019.stderr:10+0 records out 2024-05-07T02:20:21.759 INFO:teuthology.orchestra.run.smithi019.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0357901 s, 293 MB/s 2024-05-07T02:20:21.759 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 2024-05-07T02:20:21.759 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping successful for: 2024-05-07T02:20:21.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:21 smithi064 ceph-mon[34699]: purged_snaps scrub starts 2024-05-07T02:20:21.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:21 smithi064 ceph-mon[34699]: purged_snaps scrub ok 2024-05-07T02:20:21.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:21 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-07T02:20:21.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:21 smithi064 ceph-mon[34699]: osd.2 [v2:172.21.15.64:6800/1180552939,v1:172.21.15.64:6801/1180552939] boot 2024-05-07T02:20:21.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:21 smithi064 ceph-mon[34699]: osdmap e18: 3 total, 3 up, 3 in 2024-05-07T02:20:21.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:21 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-07T02:20:21.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:21 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:21.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:21 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:21.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:21 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:20:21.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:21 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:21.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:21 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:20:21.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:21 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:20:21.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:21 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:21.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:21 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-07T02:20:21.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:21 smithi019 ceph-mon[29781]: purged_snaps scrub starts 2024-05-07T02:20:21.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:21 smithi019 ceph-mon[29781]: purged_snaps scrub ok 2024-05-07T02:20:21.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:21 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-07T02:20:21.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:21 smithi019 ceph-mon[29781]: osd.2 [v2:172.21.15.64:6800/1180552939,v1:172.21.15.64:6801/1180552939] boot 2024-05-07T02:20:21.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:21 smithi019 ceph-mon[29781]: osdmap e18: 3 total, 3 up, 3 in 2024-05-07T02:20:21.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:21 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-07T02:20:21.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:21 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:21.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:21 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:21.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:21 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:20:21.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:21 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:21.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:21 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:20:21.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:21 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:20:21.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:21 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:21.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:21 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-07T02:20:21.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:21 smithi027 ceph-mon[34398]: purged_snaps scrub starts 2024-05-07T02:20:21.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:21 smithi027 ceph-mon[34398]: purged_snaps scrub ok 2024-05-07T02:20:21.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:21 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-07T02:20:21.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:21 smithi027 ceph-mon[34398]: osd.2 [v2:172.21.15.64:6800/1180552939,v1:172.21.15.64:6801/1180552939] boot 2024-05-07T02:20:21.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:21 smithi027 ceph-mon[34398]: osdmap e18: 3 total, 3 up, 3 in 2024-05-07T02:20:21.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:21 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-07T02:20:21.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:21 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:21.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:21 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:21.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:21 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:20:21.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:21 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:21.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:21 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:20:21.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:21 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:20:21.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:21 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:21.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:21 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-07T02:20:22.545 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi019:vg_nvme/lv_3 2024-05-07T02:20:22.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:22 smithi064 ceph-mon[34699]: Detected new or changed devices on smithi064 2024-05-07T02:20:22.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:22 smithi064 ceph-mon[34699]: Adjusting osd_memory_target on smithi064 to 17088M 2024-05-07T02:20:22.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:22 smithi064 ceph-mon[34699]: pgmap v93: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-07T02:20:22.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:22 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-07T02:20:22.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:22 smithi064 ceph-mon[34699]: osdmap e19: 3 total, 3 up, 3 in 2024-05-07T02:20:22.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:22 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-07T02:20:22.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:22 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-07T02:20:22.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:22 smithi064 ceph-mon[34699]: osdmap e20: 3 total, 3 up, 3 in 2024-05-07T02:20:22.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:22 smithi019 ceph-mon[29781]: Detected new or changed devices on smithi064 2024-05-07T02:20:22.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:22 smithi019 ceph-mon[29781]: Adjusting osd_memory_target on smithi064 to 17088M 2024-05-07T02:20:22.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:22 smithi019 ceph-mon[29781]: pgmap v93: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-07T02:20:22.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:22 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-07T02:20:22.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:22 smithi019 ceph-mon[29781]: osdmap e19: 3 total, 3 up, 3 in 2024-05-07T02:20:22.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:22 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-07T02:20:22.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:22 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-07T02:20:22.874 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:22 smithi019 ceph-mon[29781]: osdmap e20: 3 total, 3 up, 3 in 2024-05-07T02:20:22.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:22 smithi027 ceph-mon[34398]: Detected new or changed devices on smithi064 2024-05-07T02:20:22.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:22 smithi027 ceph-mon[34398]: Adjusting osd_memory_target on smithi064 to 17088M 2024-05-07T02:20:22.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:22 smithi027 ceph-mon[34398]: pgmap v93: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-07T02:20:22.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:22 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-07T02:20:22.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:22 smithi027 ceph-mon[34398]: osdmap e19: 3 total, 3 up, 3 in 2024-05-07T02:20:22.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:22 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-07T02:20:22.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:22 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-07T02:20:22.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:22 smithi027 ceph-mon[34398]: osdmap e20: 3 total, 3 up, 3 in 2024-05-07T02:20:22.981 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:20:23.130 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 07 02:20:22 smithi019 sudo[48836]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-07T02:20:23.130 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 07 02:20:22 smithi019 sudo[48836]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-07T02:20:23.130 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 07 02:20:22 smithi019 sudo[48836]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-07T02:20:23.130 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 07 02:20:22 smithi019 sudo[48836]: pam_unix(sudo:session): session closed for user root 2024-05-07T02:20:23.130 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 07 02:20:22 smithi019 sudo[48854]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-07T02:20:23.130 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 07 02:20:22 smithi019 sudo[48854]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-07T02:20:23.130 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 07 02:20:22 smithi019 sudo[48854]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-07T02:20:23.130 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 07 02:20:22 smithi019 sudo[48854]: pam_unix(sudo:session): session closed for user root 2024-05-07T02:20:23.361 INFO:journalctl@ceph.osd.1.smithi027.stdout:May 07 02:20:23 smithi027 sudo[43012]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-07T02:20:23.361 INFO:journalctl@ceph.osd.1.smithi027.stdout:May 07 02:20:23 smithi027 sudo[43012]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-07T02:20:23.361 INFO:journalctl@ceph.osd.1.smithi027.stdout:May 07 02:20:23 smithi027 sudo[43012]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-07T02:20:23.361 INFO:journalctl@ceph.osd.1.smithi027.stdout:May 07 02:20:23 smithi027 sudo[43012]: pam_unix(sudo:session): session closed for user root 2024-05-07T02:20:23.362 INFO:journalctl@ceph.osd.1.smithi027.stdout:May 07 02:20:23 smithi027 sudo[43015]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-07T02:20:23.362 INFO:journalctl@ceph.osd.1.smithi027.stdout:May 07 02:20:23 smithi027 sudo[43015]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-07T02:20:23.362 INFO:journalctl@ceph.osd.1.smithi027.stdout:May 07 02:20:23 smithi027 sudo[43015]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-07T02:20:23.362 INFO:journalctl@ceph.osd.1.smithi027.stdout:May 07 02:20:23 smithi027 sudo[43015]: pam_unix(sudo:session): session closed for user root 2024-05-07T02:20:23.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:23 smithi019 sudo[48873]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-07T02:20:23.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:23 smithi019 sudo[48873]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-07T02:20:23.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:23 smithi019 sudo[48873]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-07T02:20:23.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:23 smithi019 sudo[48873]: pam_unix(sudo:session): session closed for user root 2024-05-07T02:20:23.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:23 smithi019 sudo[48876]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-07T02:20:23.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:23 smithi019 sudo[48876]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-07T02:20:23.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:23 smithi019 sudo[48876]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-07T02:20:23.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:23 smithi019 sudo[48876]: pam_unix(sudo:session): session closed for user root 2024-05-07T02:20:23.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:23 smithi064 sudo[43949]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-07T02:20:23.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:23 smithi064 sudo[43949]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-07T02:20:23.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:23 smithi064 sudo[43949]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-07T02:20:23.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:23 smithi064 sudo[43949]: pam_unix(sudo:session): session closed for user root 2024-05-07T02:20:23.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:23 smithi064 sudo[43952]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-07T02:20:23.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:23 smithi064 sudo[43952]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-07T02:20:23.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:23 smithi064 sudo[43952]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-07T02:20:23.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:23 smithi064 sudo[43952]: pam_unix(sudo:session): session closed for user root 2024-05-07T02:20:23.609 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 07 02:20:23 smithi064 sudo[43943]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-07T02:20:23.609 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 07 02:20:23 smithi064 sudo[43943]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-07T02:20:23.609 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 07 02:20:23 smithi064 sudo[43943]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-07T02:20:23.609 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 07 02:20:23 smithi064 sudo[43943]: pam_unix(sudo:session): session closed for user root 2024-05-07T02:20:23.609 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 07 02:20:23 smithi064 sudo[43946]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-07T02:20:23.609 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 07 02:20:23 smithi064 sudo[43946]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-07T02:20:23.609 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 07 02:20:23 smithi064 sudo[43946]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-07T02:20:23.609 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 07 02:20:23 smithi064 sudo[43946]: pam_unix(sudo:session): session closed for user root 2024-05-07T02:20:23.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:23 smithi027 sudo[43018]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-07T02:20:23.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:23 smithi027 sudo[43018]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-07T02:20:23.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:23 smithi027 sudo[43018]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-07T02:20:23.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:23 smithi027 sudo[43018]: pam_unix(sudo:session): session closed for user root 2024-05-07T02:20:23.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:23 smithi027 sudo[43021]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-07T02:20:23.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:23 smithi027 sudo[43021]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-07T02:20:23.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:23 smithi027 sudo[43021]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-07T02:20:23.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:23 smithi027 sudo[43021]: pam_unix(sudo:session): session closed for user root 2024-05-07T02:20:24.204 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:24 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:24.204 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:24 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:24.205 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:24 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:24.205 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:24 smithi019 ceph-mon[29781]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-07T02:20:24.205 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:24 smithi019 ceph-mon[29781]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-07T02:20:24.205 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:24 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-07T02:20:24.205 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:24 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-07T02:20:24.205 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:24 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-07T02:20:24.205 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:24 smithi019 ceph-mon[29781]: pgmap v96: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:24.205 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:24 smithi019 ceph-mon[29781]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-07T02:20:24.205 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:24 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-07T02:20:24.205 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:24 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-07T02:20:24.205 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:24 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-07T02:20:24.205 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:24 smithi019 ceph-mon[29781]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-07T02:20:24.205 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:24 smithi019 ceph-mon[29781]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-07T02:20:24.205 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:24 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-07T02:20:24.206 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:24 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-07T02:20:24.206 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:24 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-07T02:20:24.206 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:24 smithi019 ceph-mon[29781]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-07T02:20:24.206 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:24 smithi019 ceph-mon[29781]: osdmap e21: 3 total, 3 up, 3 in 2024-05-07T02:20:24.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:24 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:24.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:24 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:24.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:24 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:24.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:24 smithi064 ceph-mon[34699]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-07T02:20:24.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:24 smithi064 ceph-mon[34699]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-07T02:20:24.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:24 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-07T02:20:24.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:24 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-07T02:20:24.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:24 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-07T02:20:24.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:24 smithi064 ceph-mon[34699]: pgmap v96: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:24.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:24 smithi064 ceph-mon[34699]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-07T02:20:24.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:24 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-07T02:20:24.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:24 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-07T02:20:24.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:24 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-07T02:20:24.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:24 smithi064 ceph-mon[34699]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-07T02:20:24.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:24 smithi064 ceph-mon[34699]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-07T02:20:24.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:24 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-07T02:20:24.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:24 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-07T02:20:24.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:24 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-07T02:20:24.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:24 smithi064 ceph-mon[34699]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-07T02:20:24.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:24 smithi064 ceph-mon[34699]: osdmap e21: 3 total, 3 up, 3 in 2024-05-07T02:20:24.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:24 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:24.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:24 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:24.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:24 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:24.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:24 smithi027 ceph-mon[34398]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-07T02:20:24.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:24 smithi027 ceph-mon[34398]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-07T02:20:24.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:24 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-07T02:20:24.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:24 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-07T02:20:24.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:24 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-07T02:20:24.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:24 smithi027 ceph-mon[34398]: pgmap v96: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:24.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:24 smithi027 ceph-mon[34398]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-07T02:20:24.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:24 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-07T02:20:24.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:24 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-07T02:20:24.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:24 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-07T02:20:24.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:24 smithi027 ceph-mon[34398]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-07T02:20:24.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:24 smithi027 ceph-mon[34398]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-07T02:20:24.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:24 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-07T02:20:24.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:24 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-07T02:20:24.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:24 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-07T02:20:24.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:24 smithi027 ceph-mon[34398]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-07T02:20:24.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:24 smithi027 ceph-mon[34398]: osdmap e21: 3 total, 3 up, 3 in 2024-05-07T02:20:25.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:25.221+0000 7f3639600700 1 -- 172.21.15.19:0/3377170934 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f363410a110 msgr2=0x7f363410c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:20:25.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:25.221+0000 7f3639600700 1 --2- 172.21.15.19:0/3377170934 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f363410a110 0x7f363410c530 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f362c00b0a0 tx=0x7f362c00b3b0 comp rx=0 tx=0).stop 2024-05-07T02:20:25.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:25.221+0000 7f3639600700 1 -- 172.21.15.19:0/3377170934 shutdown_connections 2024-05-07T02:20:25.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:25.221+0000 7f3639600700 1 --2- 172.21.15.19:0/3377170934 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f363410a110 0x7f363410c530 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:20:25.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:25.221+0000 7f3639600700 1 --2- 172.21.15.19:0/3377170934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3634104e50 0x7f3634107270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:20:25.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:25.221+0000 7f3639600700 1 --2- 172.21.15.19:0/3377170934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f36341077b0 0x7f3634109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:20:25.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:25.221+0000 7f3639600700 1 -- 172.21.15.19:0/3377170934 >> 172.21.15.19:0/3377170934 conn(0x7f36340fe890 msgr2=0x7f3634100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:20:25.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:25.222+0000 7f3639600700 1 -- 172.21.15.19:0/3377170934 shutdown_connections 2024-05-07T02:20:25.223 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:25 smithi019 ceph-mon[29781]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-07T02:20:25.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:25.222+0000 7f3639600700 1 -- 172.21.15.19:0/3377170934 wait complete. 2024-05-07T02:20:25.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:25.224+0000 7f3639600700 1 Processor -- start 2024-05-07T02:20:25.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:25.224+0000 7f3639600700 1 -- start start 2024-05-07T02:20:25.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:25.225+0000 7f3639600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3634104e50 0x7f363419a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:20:25.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:25.225+0000 7f3639600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36341077b0 0x7f363419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:20:25.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:25.225+0000 7f3639600700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f363410a110 0x7f36341a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:20:25.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:25.225+0000 7f3639600700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f363410fdf0 con 0x7f36341077b0 2024-05-07T02:20:25.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:25.225+0000 7f3639600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f363410fc70 con 0x7f3634104e50 2024-05-07T02:20:25.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:25.225+0000 7f3639600700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f363410ff70 con 0x7f363410a110 2024-05-07T02:20:25.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:25.225+0000 7f3632a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3634104e50 0x7f363419a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:20:25.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:25.225+0000 7f3632a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3634104e50 0x7f363419a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:52000/0 (socket says 172.21.15.19:52000) 2024-05-07T02:20:25.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:25.225+0000 7f3632a00700 1 -- 172.21.15.19:0/3442601279 learned_addr learned my addr 172.21.15.19:0/3442601279 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:20:25.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:25.225+0000 7f3632000700 1 --2- 172.21.15.19:0/3442601279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36341077b0 0x7f363419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:20:25.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:25.225+0000 7f3633400700 1 --2- 172.21.15.19:0/3442601279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f363410a110 0x7f36341a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:20:25.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:25.226+0000 7f3632a00700 1 -- 172.21.15.19:0/3442601279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f363410a110 msgr2=0x7f36341a1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:20:25.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:25.226+0000 7f3632a00700 1 --2- 172.21.15.19:0/3442601279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f363410a110 0x7f36341a1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:20:25.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:25.226+0000 7f3632a00700 1 -- 172.21.15.19:0/3442601279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36341077b0 msgr2=0x7f363419adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:20:25.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:25.226+0000 7f3632a00700 1 --2- 172.21.15.19:0/3442601279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36341077b0 0x7f363419adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:20:25.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:25.226+0000 7f3632a00700 1 -- 172.21.15.19:0/3442601279 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36341a2410 con 0x7f3634104e50 2024-05-07T02:20:25.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:25.226+0000 7f3633400700 1 --2- 172.21.15.19:0/3442601279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f363410a110 0x7f36341a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:20:25.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:25.226+0000 7f3632a00700 1 --2- 172.21.15.19:0/3442601279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3634104e50 0x7f363419a8b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f362400ea30 tx=0x7f362400ed40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:20:25.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:25.226+0000 7f362b400700 1 -- 172.21.15.19:0/3442601279 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362400cbd0 con 0x7f3634104e50 2024-05-07T02:20:25.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:25.226+0000 7f362b400700 1 -- 172.21.15.19:0/3442601279 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36240044d0 con 0x7f3634104e50 2024-05-07T02:20:25.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:25.226+0000 7f362b400700 1 -- 172.21.15.19:0/3442601279 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3624010430 con 0x7f3634104e50 2024-05-07T02:20:25.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:25.226+0000 7f3639600700 1 -- 172.21.15.19:0/3442601279 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36341a2700 con 0x7f3634104e50 2024-05-07T02:20:25.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:25.227+0000 7f3639600700 1 -- 172.21.15.19:0/3442601279 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36341a2c40 con 0x7f3634104e50 2024-05-07T02:20:25.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:25.228+0000 7f3639600700 1 -- 172.21.15.19:0/3442601279 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36340512c0 con 0x7f3634104e50 2024-05-07T02:20:25.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:25.230+0000 7f362b400700 1 -- 172.21.15.19:0/3442601279 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f3624003ea0 con 0x7f3634104e50 2024-05-07T02:20:25.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:25.231+0000 7f362b400700 1 --2- 172.21.15.19:0/3442601279 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f36180b2880 0x7f36180b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:20:25.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:25.232+0000 7f362b400700 1 -- 172.21.15.19:0/3442601279 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 3237+0+0 (secure 0 0 0) 0x7f36240e10f0 con 0x7f3634104e50 2024-05-07T02:20:25.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:25.232+0000 7f3632000700 1 --2- 172.21.15.19:0/3442601279 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f36180b2880 0x7f36180b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:20:25.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:25.233+0000 7f3632000700 1 --2- 172.21.15.19:0/3442601279 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f36180b2880 0x7f36180b4d90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f363419b960 tx=0x7f3620009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:20:25.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:25.236+0000 7f362b400700 1 -- 172.21.15.19:0/3442601279 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f36240aa140 con 0x7f3634104e50 2024-05-07T02:20:25.472 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:25 smithi027 ceph-mon[34398]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-07T02:20:25.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:25.484+0000 7f3639600700 1 -- 172.21.15.19:0/3442601279 --> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f3634063c00 con 0x7f36180b2880 2024-05-07T02:20:25.487 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:25 smithi064 ceph-mon[34699]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-07T02:20:26.475 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:26 smithi019 ceph-mon[29781]: pgmap v98: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:26.475 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:26 smithi019 ceph-mon[29781]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:20:26.475 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:26 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-07T02:20:26.475 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:26 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-07T02:20:26.475 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:26 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:20:26.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:26 smithi064 ceph-mon[34699]: pgmap v98: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:26.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:26 smithi064 ceph-mon[34699]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:20:26.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:26 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-07T02:20:26.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:26 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-07T02:20:26.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:26 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:20:26.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:26 smithi027 ceph-mon[34398]: pgmap v98: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:26.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:26 smithi027 ceph-mon[34398]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:20:26.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:26 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-07T02:20:26.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:26 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-07T02:20:26.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:26 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:20:28.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:28 smithi019 ceph-mon[29781]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:28.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:28 smithi027 ceph-mon[34398]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:28.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:28 smithi064 ceph-mon[34699]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:30.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:30 smithi019 ceph-mon[29781]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:30.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:30 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/1575542462' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ceaa4252-4dfa-427a-9b1a-76adf3b8e609"}]: dispatch 2024-05-07T02:20:30.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:30 smithi019 ceph-mon[29781]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ceaa4252-4dfa-427a-9b1a-76adf3b8e609"}]: dispatch 2024-05-07T02:20:30.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:30 smithi019 ceph-mon[29781]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ceaa4252-4dfa-427a-9b1a-76adf3b8e609"}]': finished 2024-05-07T02:20:30.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:30 smithi019 ceph-mon[29781]: osdmap e22: 4 total, 3 up, 4 in 2024-05-07T02:20:30.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:30 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-07T02:20:30.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:30 smithi027 ceph-mon[34398]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:30.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:30 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/1575542462' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ceaa4252-4dfa-427a-9b1a-76adf3b8e609"}]: dispatch 2024-05-07T02:20:30.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:30 smithi027 ceph-mon[34398]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ceaa4252-4dfa-427a-9b1a-76adf3b8e609"}]: dispatch 2024-05-07T02:20:30.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:30 smithi027 ceph-mon[34398]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ceaa4252-4dfa-427a-9b1a-76adf3b8e609"}]': finished 2024-05-07T02:20:30.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:30 smithi027 ceph-mon[34398]: osdmap e22: 4 total, 3 up, 4 in 2024-05-07T02:20:30.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:30 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-07T02:20:30.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:30 smithi064 ceph-mon[34699]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:30.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:30 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/1575542462' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ceaa4252-4dfa-427a-9b1a-76adf3b8e609"}]: dispatch 2024-05-07T02:20:30.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:30 smithi064 ceph-mon[34699]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ceaa4252-4dfa-427a-9b1a-76adf3b8e609"}]: dispatch 2024-05-07T02:20:30.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:30 smithi064 ceph-mon[34699]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ceaa4252-4dfa-427a-9b1a-76adf3b8e609"}]': finished 2024-05-07T02:20:30.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:30 smithi064 ceph-mon[34699]: osdmap e22: 4 total, 3 up, 4 in 2024-05-07T02:20:30.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:30 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-07T02:20:31.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:31 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/38037824' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-07T02:20:31.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:31 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/38037824' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-07T02:20:31.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:31 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/38037824' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-07T02:20:32.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:32 smithi019 ceph-mon[29781]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:32.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:32 smithi027 ceph-mon[34398]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:32.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:32 smithi064 ceph-mon[34699]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:34.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:34 smithi019 ceph-mon[29781]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:34.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:34 smithi027 ceph-mon[34398]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:34.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:34 smithi064 ceph-mon[34699]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:36.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:36 smithi027 ceph-mon[34398]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:36.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:36 smithi019 ceph-mon[29781]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:36.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:36 smithi064 ceph-mon[34699]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:38.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:38 smithi019 ceph-mon[29781]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:38.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:38 smithi027 ceph-mon[34398]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:38.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:38 smithi064 ceph-mon[34699]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:39.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:39 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-07T02:20:39.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:39 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:20:39.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:39 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-07T02:20:39.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:39 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:20:39.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:39 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-07T02:20:39.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:39 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:20:40.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:40 smithi027 ceph-mon[34398]: Deploying daemon osd.3 on smithi019 2024-05-07T02:20:40.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:40 smithi027 ceph-mon[34398]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:40.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:40 smithi064 ceph-mon[34699]: Deploying daemon osd.3 on smithi019 2024-05-07T02:20:40.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:40 smithi064 ceph-mon[34699]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:40.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:40 smithi019 ceph-mon[29781]: Deploying daemon osd.3 on smithi019 2024-05-07T02:20:40.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:40 smithi019 ceph-mon[29781]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:42.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:42 smithi027 ceph-mon[34398]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:42.760 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:42 smithi019 ceph-mon[29781]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:42.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:42 smithi064 ceph-mon[34699]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:44.680 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:44 smithi019 ceph-mon[29781]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:44.684 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:44 smithi064 ceph-mon[34699]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:44.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:44 smithi027 ceph-mon[34398]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:46.701 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:46 smithi019 ceph-mon[29781]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:46.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:46 smithi027 ceph-mon[34398]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:46.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:46 smithi064 ceph-mon[34699]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:48.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:48 smithi027 ceph-mon[34398]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:48.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:48 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:48.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:48 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:20:48.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:48 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:48.729 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:48 smithi019 ceph-mon[29781]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:48.730 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:48 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:48.730 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:48 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:20:48.730 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:48 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:48.860 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:48 smithi064 ceph-mon[34699]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:48.860 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:48 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:48.860 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:48 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:20:48.860 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:48 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:50.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:50 smithi027 ceph-mon[34398]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:50.720 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:50 smithi019 ceph-mon[29781]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:50.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:50 smithi064 ceph-mon[34699]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:52.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:52 smithi027 ceph-mon[34398]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:52.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:52 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:52.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:52 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:52.823 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:52 smithi019 ceph-mon[29781]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:52.824 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:52 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:52.824 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:52 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:52.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:52 smithi064 ceph-mon[34699]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:52.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:52 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:52.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:52 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:53.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:53 smithi027 ceph-mon[34398]: from='osd.3 [v2:172.21.15.19:6810/387118141,v1:172.21.15.19:6811/387118141]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-07T02:20:53.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:53 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:20:53.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:53 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:20:53.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:53 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:53.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:53 smithi064 ceph-mon[34699]: from='osd.3 [v2:172.21.15.19:6810/387118141,v1:172.21.15.19:6811/387118141]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-07T02:20:53.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:53 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:20:53.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:53 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:20:53.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:53 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:53.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:53 smithi019 ceph-mon[29781]: from='osd.3 [v2:172.21.15.19:6810/387118141,v1:172.21.15.19:6811/387118141]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-07T02:20:53.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:53 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:20:53.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:53 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:20:53.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:53 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:54.054 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 3 on host 'smithi019' 2024-05-07T02:20:54.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:54.052+0000 7f362b400700 1 -- 172.21.15.19:0/3442601279 <== mgr.14152 v2:172.21.15.19:6800/2059564551 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3634063c00 con 0x7f36180b2880 2024-05-07T02:20:54.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:54.056+0000 7f3639600700 1 -- 172.21.15.19:0/3442601279 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f36180b2880 msgr2=0x7f36180b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:20:54.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:54.056+0000 7f3639600700 1 --2- 172.21.15.19:0/3442601279 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f36180b2880 0x7f36180b4d90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f363419b960 tx=0x7f3620009260 comp rx=0 tx=0).stop 2024-05-07T02:20:54.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:54.057+0000 7f3639600700 1 -- 172.21.15.19:0/3442601279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3634104e50 msgr2=0x7f363419a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:20:54.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:54.057+0000 7f3639600700 1 --2- 172.21.15.19:0/3442601279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3634104e50 0x7f363419a8b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f362400ea30 tx=0x7f362400ed40 comp rx=0 tx=0).stop 2024-05-07T02:20:54.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:54.057+0000 7f3639600700 1 -- 172.21.15.19:0/3442601279 shutdown_connections 2024-05-07T02:20:54.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:54.057+0000 7f3639600700 1 --2- 172.21.15.19:0/3442601279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f363410a110 0x7f36341a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:20:54.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:54.057+0000 7f3639600700 1 --2- 172.21.15.19:0/3442601279 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f36180b2880 0x7f36180b4d90 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:20:54.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:54.057+0000 7f3639600700 1 --2- 172.21.15.19:0/3442601279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3634104e50 0x7f363419a8b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:20:54.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:54.057+0000 7f3639600700 1 --2- 172.21.15.19:0/3442601279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36341077b0 0x7f363419adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:20:54.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:54.057+0000 7f3639600700 1 -- 172.21.15.19:0/3442601279 >> 172.21.15.19:0/3442601279 conn(0x7f36340fe890 msgr2=0x7f3634105920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:20:54.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:54.057+0000 7f3639600700 1 -- 172.21.15.19:0/3442601279 shutdown_connections 2024-05-07T02:20:54.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:20:54.057+0000 7f3639600700 1 -- 172.21.15.19:0/3442601279 wait complete. 2024-05-07T02:20:54.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:54 smithi064 ceph-mon[34699]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:54.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:54 smithi064 ceph-mon[34699]: from='osd.3 [v2:172.21.15.19:6810/387118141,v1:172.21.15.19:6811/387118141]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-07T02:20:54.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:54 smithi064 ceph-mon[34699]: osdmap e23: 4 total, 3 up, 4 in 2024-05-07T02:20:54.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:54 smithi064 ceph-mon[34699]: from='osd.3 [v2:172.21.15.19:6810/387118141,v1:172.21.15.19:6811/387118141]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-07T02:20:54.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:54 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-07T02:20:54.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:54 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:54.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:54 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:54.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:54 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:20:54.882 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:54 smithi019 ceph-mon[29781]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:54.882 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:54 smithi019 ceph-mon[29781]: from='osd.3 [v2:172.21.15.19:6810/387118141,v1:172.21.15.19:6811/387118141]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-07T02:20:54.882 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:54 smithi019 ceph-mon[29781]: osdmap e23: 4 total, 3 up, 4 in 2024-05-07T02:20:54.883 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:54 smithi019 ceph-mon[29781]: from='osd.3 [v2:172.21.15.19:6810/387118141,v1:172.21.15.19:6811/387118141]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-07T02:20:54.883 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:54 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-07T02:20:54.883 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:54 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:54.883 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:54 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:54.883 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:54 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:20:54.902 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:54 smithi027 ceph-mon[34398]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:54.902 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:54 smithi027 ceph-mon[34398]: from='osd.3 [v2:172.21.15.19:6810/387118141,v1:172.21.15.19:6811/387118141]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-07T02:20:54.902 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:54 smithi027 ceph-mon[34398]: osdmap e23: 4 total, 3 up, 4 in 2024-05-07T02:20:54.902 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:54 smithi027 ceph-mon[34398]: from='osd.3 [v2:172.21.15.19:6810/387118141,v1:172.21.15.19:6811/387118141]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-07T02:20:54.902 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:54 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-07T02:20:54.902 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:54 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:54.902 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:54 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:20:54.902 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:54 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:20:54.945 DEBUG:teuthology.orchestra.run.smithi019:osd.3> sudo journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@osd.3.service 2024-05-07T02:20:54.969 INFO:tasks.cephadm:Deploying osd.4 on smithi027 with /dev/vg_nvme/lv_3... 2024-05-07T02:20:54.969 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-07T02:20:55.217 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.b/config 2024-05-07T02:20:55.456 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 07 02:20:55 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-3[51201]: 2024-05-07T02:20:55.342+0000 7f589a800700 -1 osd.3 0 waiting for initial osdmap 2024-05-07T02:20:55.456 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 07 02:20:55 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-3[51201]: 2024-05-07T02:20:55.359+0000 7f588fc00700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-07T02:20:55.598 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:55 smithi027 ceph-mon[34398]: from='osd.3 [v2:172.21.15.19:6810/387118141,v1:172.21.15.19:6811/387118141]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-07T02:20:55.598 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:55 smithi027 ceph-mon[34398]: osdmap e24: 4 total, 3 up, 4 in 2024-05-07T02:20:55.598 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:55 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-07T02:20:55.599 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:55 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-07T02:20:55.599 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:55 smithi027 ceph-mon[34398]: from='osd.3 [v2:172.21.15.19:6810/387118141,v1:172.21.15.19:6811/387118141]' entity='osd.3' 2024-05-07T02:20:55.599 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:55 smithi027 ceph-mon[34398]: osd.3 [v2:172.21.15.19:6810/387118141,v1:172.21.15.19:6811/387118141] boot 2024-05-07T02:20:55.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:55 smithi064 ceph-mon[34699]: from='osd.3 [v2:172.21.15.19:6810/387118141,v1:172.21.15.19:6811/387118141]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-07T02:20:55.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:55 smithi064 ceph-mon[34699]: osdmap e24: 4 total, 3 up, 4 in 2024-05-07T02:20:55.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:55 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-07T02:20:55.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:55 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-07T02:20:55.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:55 smithi064 ceph-mon[34699]: from='osd.3 [v2:172.21.15.19:6810/387118141,v1:172.21.15.19:6811/387118141]' entity='osd.3' 2024-05-07T02:20:55.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:55 smithi064 ceph-mon[34699]: osd.3 [v2:172.21.15.19:6810/387118141,v1:172.21.15.19:6811/387118141] boot 2024-05-07T02:20:55.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:55 smithi019 ceph-mon[29781]: from='osd.3 [v2:172.21.15.19:6810/387118141,v1:172.21.15.19:6811/387118141]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-07T02:20:55.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:55 smithi019 ceph-mon[29781]: osdmap e24: 4 total, 3 up, 4 in 2024-05-07T02:20:55.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:55 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-07T02:20:55.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:55 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-07T02:20:55.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:55 smithi019 ceph-mon[29781]: from='osd.3 [v2:172.21.15.19:6810/387118141,v1:172.21.15.19:6811/387118141]' entity='osd.3' 2024-05-07T02:20:55.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:55 smithi019 ceph-mon[29781]: osd.3 [v2:172.21.15.19:6810/387118141,v1:172.21.15.19:6811/387118141] boot 2024-05-07T02:20:56.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:56 smithi019 ceph-mon[29781]: purged_snaps scrub starts 2024-05-07T02:20:56.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:56 smithi019 ceph-mon[29781]: purged_snaps scrub ok 2024-05-07T02:20:56.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:56 smithi019 ceph-mon[29781]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:56.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:56 smithi019 ceph-mon[29781]: osdmap e25: 4 total, 4 up, 4 in 2024-05-07T02:20:56.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:56 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-07T02:20:56.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:56 smithi019 ceph-mon[29781]: osdmap e26: 4 total, 4 up, 4 in 2024-05-07T02:20:56.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:56 smithi027 ceph-mon[34398]: purged_snaps scrub starts 2024-05-07T02:20:56.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:56 smithi027 ceph-mon[34398]: purged_snaps scrub ok 2024-05-07T02:20:56.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:56 smithi027 ceph-mon[34398]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:56.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:56 smithi027 ceph-mon[34398]: osdmap e25: 4 total, 4 up, 4 in 2024-05-07T02:20:56.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:56 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-07T02:20:56.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:56 smithi027 ceph-mon[34398]: osdmap e26: 4 total, 4 up, 4 in 2024-05-07T02:20:56.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:56 smithi064 ceph-mon[34699]: purged_snaps scrub starts 2024-05-07T02:20:56.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:56 smithi064 ceph-mon[34699]: purged_snaps scrub ok 2024-05-07T02:20:56.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:56 smithi064 ceph-mon[34699]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-07T02:20:56.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:56 smithi064 ceph-mon[34699]: osdmap e25: 4 total, 4 up, 4 in 2024-05-07T02:20:56.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:56 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-07T02:20:56.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:56 smithi064 ceph-mon[34699]: osdmap e26: 4 total, 4 up, 4 in 2024-05-07T02:20:57.316 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-07T02:20:57.316 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-07T02:20:57.316 INFO:teuthology.orchestra.run.smithi027.stderr: stderr: 10+0 records in 2024-05-07T02:20:57.316 INFO:teuthology.orchestra.run.smithi027.stderr:10+0 records out 2024-05-07T02:20:57.316 INFO:teuthology.orchestra.run.smithi027.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0162525 s, 645 MB/s 2024-05-07T02:20:57.316 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping successful for: 2024-05-07T02:20:57.998 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi027:vg_nvme/lv_3 2024-05-07T02:20:58.244 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.b/config 2024-05-07T02:20:58.679 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:58 smithi027 ceph-mon[34398]: pgmap v119: 1 pgs: 1 peering; 577 KiB data, 280 MiB used, 357 GiB / 358 GiB avail 2024-05-07T02:20:58.679 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:20:58 smithi027 ceph-mon[34398]: osdmap e27: 4 total, 4 up, 4 in 2024-05-07T02:20:58.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:58 smithi064 ceph-mon[34699]: pgmap v119: 1 pgs: 1 peering; 577 KiB data, 280 MiB used, 357 GiB / 358 GiB avail 2024-05-07T02:20:58.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:20:58 smithi064 ceph-mon[34699]: osdmap e27: 4 total, 4 up, 4 in 2024-05-07T02:20:58.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:58 smithi019 ceph-mon[29781]: pgmap v119: 1 pgs: 1 peering; 577 KiB data, 280 MiB used, 357 GiB / 358 GiB avail 2024-05-07T02:20:58.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:20:58 smithi019 ceph-mon[29781]: osdmap e27: 4 total, 4 up, 4 in 2024-05-07T02:20:59.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:20:59.834+0000 7fa2c2ff4700 1 -- 172.21.15.27:0/1803934751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2bc105e40 msgr2=0x7fa2bc10c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:20:59.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:20:59.834+0000 7fa2c2ff4700 1 --2- 172.21.15.27:0/1803934751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2bc105e40 0x7fa2bc10c6f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa2b400b0a0 tx=0x7fa2b400b3b0 comp rx=0 tx=0).stop 2024-05-07T02:20:59.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:20:59.834+0000 7fa2c2ff4700 1 -- 172.21.15.27:0/1803934751 shutdown_connections 2024-05-07T02:20:59.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:20:59.834+0000 7fa2c2ff4700 1 --2- 172.21.15.27:0/1803934751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2bc105e40 0x7fa2bc10c6f0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:20:59.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:20:59.834+0000 7fa2c2ff4700 1 --2- 172.21.15.27:0/1803934751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2bc104370 0x7fa2bc104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:20:59.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:20:59.834+0000 7fa2c2ff4700 1 --2- 172.21.15.27:0/1803934751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2bc1054a0 0x7fa2bc1058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:20:59.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:20:59.834+0000 7fa2c2ff4700 1 -- 172.21.15.27:0/1803934751 >> 172.21.15.27:0/1803934751 conn(0x7fa2bc0ff9d0 msgr2=0x7fa2bc101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:20:59.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:20:59.834+0000 7fa2c2ff4700 1 -- 172.21.15.27:0/1803934751 shutdown_connections 2024-05-07T02:20:59.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:20:59.834+0000 7fa2c2ff4700 1 -- 172.21.15.27:0/1803934751 wait complete. 2024-05-07T02:20:59.836 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:20:59.835+0000 7fa2c2ff4700 1 Processor -- start 2024-05-07T02:20:59.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:20:59.835+0000 7fa2c2ff4700 1 -- start start 2024-05-07T02:20:59.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:20:59.835+0000 7fa2c2ff4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2bc104370 0x7fa2bc078570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:20:59.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:20:59.835+0000 7fa2c2ff4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2bc1054a0 0x7fa2bc076c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:20:59.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:20:59.835+0000 7fa2c2ff4700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2bc105e40 0x7fa2bc077160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:20:59.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:20:59.835+0000 7fa2c2ff4700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa2bc10fe50 con 0x7fa2bc105e40 2024-05-07T02:20:59.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:20:59.835+0000 7fa2c2ff4700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa2bc10fcd0 con 0x7fa2bc104370 2024-05-07T02:20:59.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:20:59.835+0000 7fa2c2ff4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa2bc10ffd0 con 0x7fa2bc1054a0 2024-05-07T02:20:59.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:20:59.835+0000 7fa2bbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2bc1054a0 0x7fa2bc076c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:20:59.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:20:59.835+0000 7fa2c0d90700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2bc104370 0x7fa2bc078570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:20:59.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:20:59.835+0000 7fa2bbfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2bc1054a0 0x7fa2bc076c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.27:54174/0 (socket says 172.21.15.27:54174) 2024-05-07T02:20:59.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:20:59.835+0000 7fa2bbfff700 1 -- 172.21.15.27:0/508870728 learned_addr learned my addr 172.21.15.27:0/508870728 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-07T02:20:59.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:20:59.835+0000 7fa2c0d90700 1 -- 172.21.15.27:0/508870728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2bc1054a0 msgr2=0x7fa2bc076c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:20:59.837 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:20:59.835+0000 7fa2c0d90700 1 --2- 172.21.15.27:0/508870728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2bc1054a0 0x7fa2bc076c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:20:59.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:20:59.835+0000 7fa2c0d90700 1 -- 172.21.15.27:0/508870728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2bc105e40 msgr2=0x7fa2bc077160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:20:59.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:20:59.835+0000 7fa2c0d90700 1 --2- 172.21.15.27:0/508870728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2bc105e40 0x7fa2bc077160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:20:59.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:20:59.835+0000 7fa2c0d90700 1 -- 172.21.15.27:0/508870728 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2bc0778c0 con 0x7fa2bc104370 2024-05-07T02:20:59.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:20:59.836+0000 7fa2c0d90700 1 --2- 172.21.15.27:0/508870728 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2bc104370 0x7fa2bc078570 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc104070 tx=0x7fa2ac00da20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:20:59.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:20:59.836+0000 7fa2b9ffb700 1 -- 172.21.15.27:0/508870728 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac017070 con 0x7fa2bc104370 2024-05-07T02:20:59.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:20:59.836+0000 7fa2c2ff4700 1 -- 172.21.15.27:0/508870728 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2bc1a49a0 con 0x7fa2bc104370 2024-05-07T02:20:59.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:20:59.836+0000 7fa2b9ffb700 1 -- 172.21.15.27:0/508870728 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2ac00f690 con 0x7fa2bc104370 2024-05-07T02:20:59.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:20:59.836+0000 7fa2b9ffb700 1 -- 172.21.15.27:0/508870728 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac003a80 con 0x7fa2bc104370 2024-05-07T02:20:59.838 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:20:59.836+0000 7fa2c2ff4700 1 -- 172.21.15.27:0/508870728 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2bc1a4f00 con 0x7fa2bc104370 2024-05-07T02:20:59.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:20:59.837+0000 7fa2c2ff4700 1 -- 172.21.15.27:0/508870728 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2bc0512c0 con 0x7fa2bc104370 2024-05-07T02:20:59.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:20:59.838+0000 7fa2b9ffb700 1 -- 172.21.15.27:0/508870728 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fa2ac00fb40 con 0x7fa2bc104370 2024-05-07T02:20:59.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:20:59.839+0000 7fa2b9ffb700 1 --2- 172.21.15.27:0/508870728 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fa2a40b2880 0x7fa2a40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:20:59.839 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:20:59.839+0000 7fa2b9ffb700 1 -- 172.21.15.27:0/508870728 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3669+0+0 (secure 0 0 0) 0x7fa2ac0e0860 con 0x7fa2bc104370 2024-05-07T02:20:59.840 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:20:59.840+0000 7fa2bbfff700 1 --2- 172.21.15.27:0/508870728 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fa2a40b2880 0x7fa2a40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:20:59.841 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:20:59.841+0000 7fa2bbfff700 1 --2- 172.21.15.27:0/508870728 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fa2a40b2880 0x7fa2a40b4d90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc077f20 tx=0x7fa2b0009290 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:20:59.846 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:20:59.846+0000 7fa2b9ffb700 1 -- 172.21.15.27:0/508870728 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa2ac0a9740 con 0x7fa2bc104370 2024-05-07T02:21:00.070 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:21:00.071+0000 7fa2c2ff4700 1 -- 172.21.15.27:0/508870728 --> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fa2bc063cc0 con 0x7fa2a40b2880 2024-05-07T02:21:00.736 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:00 smithi027 ceph-mon[34398]: pgmap v121: 1 pgs: 1 peering; 577 KiB data, 280 MiB used, 357 GiB / 358 GiB avail 2024-05-07T02:21:00.737 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:00 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:00.737 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:00 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:00.737 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:00 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:21:00.737 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:00 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:21:00.737 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:00 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:00.737 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:00 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:21:00.737 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:00 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:21:00.737 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:00 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:00.737 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:00 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-07T02:21:00.737 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:00 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-07T02:21:00.737 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:00 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:21:00.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:00 smithi064 ceph-mon[34699]: pgmap v121: 1 pgs: 1 peering; 577 KiB data, 280 MiB used, 357 GiB / 358 GiB avail 2024-05-07T02:21:00.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:00 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:00.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:00 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:00.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:00 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:21:00.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:00 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:21:00.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:00 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:00.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:00 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:21:00.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:00 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:21:00.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:00 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:00.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:00 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-07T02:21:00.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:00 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-07T02:21:00.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:00 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:21:00.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:00 smithi019 ceph-mon[29781]: pgmap v121: 1 pgs: 1 peering; 577 KiB data, 280 MiB used, 357 GiB / 358 GiB avail 2024-05-07T02:21:00.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:00 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:00.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:00 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:00.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:00 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:21:00.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:00 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:21:00.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:00 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:00.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:00 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:21:00.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:00 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:21:00.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:00 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:00.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:00 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-07T02:21:00.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:00 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-07T02:21:00.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:00 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:21:01.740 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:01 smithi027 ceph-mon[34398]: Detected new or changed devices on smithi019 2024-05-07T02:21:01.740 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:01 smithi027 ceph-mon[34398]: Adjusting osd_memory_target on smithi019 to 8547M 2024-05-07T02:21:01.740 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:01 smithi027 ceph-mon[34398]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:21:01.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:01 smithi064 ceph-mon[34699]: Detected new or changed devices on smithi019 2024-05-07T02:21:01.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:01 smithi064 ceph-mon[34699]: Adjusting osd_memory_target on smithi019 to 8547M 2024-05-07T02:21:01.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:01 smithi064 ceph-mon[34699]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:21:01.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:01 smithi019 ceph-mon[29781]: Detected new or changed devices on smithi019 2024-05-07T02:21:01.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:01 smithi019 ceph-mon[29781]: Adjusting osd_memory_target on smithi019 to 8547M 2024-05-07T02:21:01.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:01 smithi019 ceph-mon[29781]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:21:02.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:02 smithi064 ceph-mon[34699]: pgmap v122: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-07T02:21:02.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:02 smithi019 ceph-mon[29781]: pgmap v122: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-07T02:21:02.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:02 smithi027 ceph-mon[34398]: pgmap v122: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-07T02:21:03.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:03 smithi064 ceph-mon[34699]: from='client.? 172.21.15.27:0/2601749202' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a6d1ebc-62ca-444e-8f29-1980418d0a8b"}]: dispatch 2024-05-07T02:21:03.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:03 smithi064 ceph-mon[34699]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a6d1ebc-62ca-444e-8f29-1980418d0a8b"}]: dispatch 2024-05-07T02:21:03.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:03 smithi064 ceph-mon[34699]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a6d1ebc-62ca-444e-8f29-1980418d0a8b"}]': finished 2024-05-07T02:21:03.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:03 smithi064 ceph-mon[34699]: osdmap e28: 5 total, 4 up, 5 in 2024-05-07T02:21:03.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:03 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-07T02:21:03.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:03 smithi019 ceph-mon[29781]: from='client.? 172.21.15.27:0/2601749202' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a6d1ebc-62ca-444e-8f29-1980418d0a8b"}]: dispatch 2024-05-07T02:21:03.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:03 smithi019 ceph-mon[29781]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a6d1ebc-62ca-444e-8f29-1980418d0a8b"}]: dispatch 2024-05-07T02:21:03.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:03 smithi019 ceph-mon[29781]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a6d1ebc-62ca-444e-8f29-1980418d0a8b"}]': finished 2024-05-07T02:21:03.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:03 smithi019 ceph-mon[29781]: osdmap e28: 5 total, 4 up, 5 in 2024-05-07T02:21:03.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:03 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-07T02:21:03.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:03 smithi027 ceph-mon[34398]: from='client.? 172.21.15.27:0/2601749202' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a6d1ebc-62ca-444e-8f29-1980418d0a8b"}]: dispatch 2024-05-07T02:21:03.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:03 smithi027 ceph-mon[34398]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a6d1ebc-62ca-444e-8f29-1980418d0a8b"}]: dispatch 2024-05-07T02:21:03.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:03 smithi027 ceph-mon[34398]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a6d1ebc-62ca-444e-8f29-1980418d0a8b"}]': finished 2024-05-07T02:21:03.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:03 smithi027 ceph-mon[34398]: osdmap e28: 5 total, 4 up, 5 in 2024-05-07T02:21:03.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:03 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-07T02:21:04.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:04 smithi064 ceph-mon[34699]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-05-07T02:21:04.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:04 smithi064 ceph-mon[34699]: from='client.? 172.21.15.27:0/491692866' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-07T02:21:04.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:04 smithi019 ceph-mon[29781]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-05-07T02:21:04.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:04 smithi019 ceph-mon[29781]: from='client.? 172.21.15.27:0/491692866' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-07T02:21:04.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:04 smithi027 ceph-mon[34398]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-05-07T02:21:04.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:04 smithi027 ceph-mon[34398]: from='client.? 172.21.15.27:0/491692866' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-07T02:21:06.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:06 smithi064 ceph-mon[34699]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-07T02:21:06.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:06 smithi019 ceph-mon[29781]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-07T02:21:06.933 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:06 smithi027 ceph-mon[34398]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-07T02:21:08.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:08 smithi064 ceph-mon[34699]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-07T02:21:08.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:08 smithi019 ceph-mon[29781]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-07T02:21:08.898 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:08 smithi027 ceph-mon[34398]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-07T02:21:10.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:10 smithi027 ceph-mon[34398]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-07T02:21:10.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:10 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-07T02:21:10.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:10 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:21:10.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:10 smithi064 ceph-mon[34699]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-07T02:21:10.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:10 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-07T02:21:10.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:10 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:21:10.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:10 smithi019 ceph-mon[29781]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-07T02:21:10.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:10 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-07T02:21:10.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:10 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:21:11.786 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:11 smithi027 ceph-mon[34398]: Deploying daemon osd.4 on smithi027 2024-05-07T02:21:11.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:11 smithi064 ceph-mon[34699]: Deploying daemon osd.4 on smithi027 2024-05-07T02:21:11.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:11 smithi019 ceph-mon[29781]: Deploying daemon osd.4 on smithi027 2024-05-07T02:21:12.664 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:12 smithi027 ceph-mon[34398]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-07T02:21:12.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:12 smithi064 ceph-mon[34699]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-07T02:21:12.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:12 smithi019 ceph-mon[29781]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-07T02:21:14.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:14 smithi027 ceph-mon[34398]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-07T02:21:14.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:14 smithi064 ceph-mon[34699]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-07T02:21:14.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:14 smithi019 ceph-mon[29781]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-07T02:21:16.855 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:16 smithi027 ceph-mon[34398]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-07T02:21:16.855 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:16 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:16.855 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:16 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:16.855 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:16 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:21:16.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:16 smithi064 ceph-mon[34699]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-07T02:21:16.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:16 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:16.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:16 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:16.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:16 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:21:16.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:16 smithi019 ceph-mon[29781]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-07T02:21:16.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:16 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:16.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:16 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:16.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:16 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:21:18.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:18 smithi027 ceph-mon[34398]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-07T02:21:18.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:18 smithi064 ceph-mon[34699]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-07T02:21:18.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:18 smithi019 ceph-mon[29781]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-07T02:21:19.610 INFO:teuthology.orchestra.run.smithi027.stdout:Created osd(s) 4 on host 'smithi027' 2024-05-07T02:21:19.610 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:21:19.609+0000 7fa2b9ffb700 1 -- 172.21.15.27:0/508870728 <== mgr.14152 v2:172.21.15.19:6800/2059564551 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa2bc063cc0 con 0x7fa2a40b2880 2024-05-07T02:21:19.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:21:19.612+0000 7fa2c2ff4700 1 -- 172.21.15.27:0/508870728 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fa2a40b2880 msgr2=0x7fa2a40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:21:19.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:21:19.612+0000 7fa2c2ff4700 1 --2- 172.21.15.27:0/508870728 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fa2a40b2880 0x7fa2a40b4d90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc077f20 tx=0x7fa2b0009290 comp rx=0 tx=0).stop 2024-05-07T02:21:19.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:21:19.612+0000 7fa2c2ff4700 1 -- 172.21.15.27:0/508870728 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2bc104370 msgr2=0x7fa2bc078570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:21:19.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:21:19.612+0000 7fa2c2ff4700 1 --2- 172.21.15.27:0/508870728 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2bc104370 0x7fa2bc078570 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc104070 tx=0x7fa2ac00da20 comp rx=0 tx=0).stop 2024-05-07T02:21:19.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:21:19.612+0000 7fa2c2ff4700 1 -- 172.21.15.27:0/508870728 shutdown_connections 2024-05-07T02:21:19.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:21:19.612+0000 7fa2c2ff4700 1 --2- 172.21.15.27:0/508870728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2bc105e40 0x7fa2bc077160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:21:19.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:21:19.612+0000 7fa2c2ff4700 1 --2- 172.21.15.27:0/508870728 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fa2a40b2880 0x7fa2a40b4d90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:21:19.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:21:19.612+0000 7fa2c2ff4700 1 --2- 172.21.15.27:0/508870728 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2bc104370 0x7fa2bc078570 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:21:19.612 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:21:19.612+0000 7fa2c2ff4700 1 --2- 172.21.15.27:0/508870728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2bc1054a0 0x7fa2bc076c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:21:19.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:21:19.612+0000 7fa2c2ff4700 1 -- 172.21.15.27:0/508870728 >> 172.21.15.27:0/508870728 conn(0x7fa2bc0ff9d0 msgr2=0x7fa2bc1086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:21:19.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:21:19.612+0000 7fa2c2ff4700 1 -- 172.21.15.27:0/508870728 shutdown_connections 2024-05-07T02:21:19.613 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:21:19.612+0000 7fa2c2ff4700 1 -- 172.21.15.27:0/508870728 wait complete. 2024-05-07T02:21:20.276 DEBUG:teuthology.orchestra.run.smithi027:osd.4> sudo journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@osd.4.service 2024-05-07T02:21:20.278 INFO:tasks.cephadm:Deploying osd.5 on smithi064 with /dev/vg_nvme/lv_3... 2024-05-07T02:21:20.279 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-07T02:21:20.534 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.c/config 2024-05-07T02:21:20.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:20 smithi064 ceph-mon[34699]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-07T02:21:20.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:20 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:20.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:20 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:20.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:20 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:20.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:20 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:20.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:20 smithi064 ceph-mon[34699]: from='osd.4 [v2:172.21.15.27:6808/3662186068,v1:172.21.15.27:6809/3662186068]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-07T02:21:20.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:20 smithi064 ceph-mon[34699]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-07T02:21:20.829 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:20 smithi027 ceph-mon[34398]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-07T02:21:20.830 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:20 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:20.830 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:20 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:20.830 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:20 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:20.830 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:20 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:20.830 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:20 smithi027 ceph-mon[34398]: from='osd.4 [v2:172.21.15.27:6808/3662186068,v1:172.21.15.27:6809/3662186068]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-07T02:21:20.830 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:20 smithi027 ceph-mon[34398]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-07T02:21:20.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:20 smithi019 ceph-mon[29781]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-07T02:21:20.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:20 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:20.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:20 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:20.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:20 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:20.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:20 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:20.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:20 smithi019 ceph-mon[29781]: from='osd.4 [v2:172.21.15.27:6808/3662186068,v1:172.21.15.27:6809/3662186068]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-07T02:21:20.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:20 smithi019 ceph-mon[29781]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-07T02:21:21.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:21 smithi027 ceph-mon[34398]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-07T02:21:21.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:21 smithi027 ceph-mon[34398]: from='osd.4 [v2:172.21.15.27:6808/3662186068,v1:172.21.15.27:6809/3662186068]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-05-07T02:21:21.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:21 smithi027 ceph-mon[34398]: osdmap e29: 5 total, 4 up, 5 in 2024-05-07T02:21:21.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:21 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-07T02:21:21.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:21 smithi027 ceph-mon[34398]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-05-07T02:21:21.970 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:21 smithi064 ceph-mon[34699]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-07T02:21:21.970 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:21 smithi064 ceph-mon[34699]: from='osd.4 [v2:172.21.15.27:6808/3662186068,v1:172.21.15.27:6809/3662186068]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-05-07T02:21:21.970 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:21 smithi064 ceph-mon[34699]: osdmap e29: 5 total, 4 up, 5 in 2024-05-07T02:21:21.971 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:21 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-07T02:21:21.971 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:21 smithi064 ceph-mon[34699]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-05-07T02:21:22.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:21 smithi019 ceph-mon[29781]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-07T02:21:22.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:21 smithi019 ceph-mon[29781]: from='osd.4 [v2:172.21.15.27:6808/3662186068,v1:172.21.15.27:6809/3662186068]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-05-07T02:21:22.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:21 smithi019 ceph-mon[29781]: osdmap e29: 5 total, 4 up, 5 in 2024-05-07T02:21:22.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:21 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-07T02:21:22.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:21 smithi019 ceph-mon[29781]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-05-07T02:21:22.898 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:22 smithi064 ceph-mon[34699]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-07T02:21:22.898 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:22 smithi064 ceph-mon[34699]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-05-07T02:21:22.898 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:22 smithi064 ceph-mon[34699]: osdmap e30: 5 total, 4 up, 5 in 2024-05-07T02:21:22.898 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:22 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-07T02:21:22.898 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:22 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-07T02:21:22.898 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:22 smithi064 ceph-mon[34699]: from='osd.4 ' entity='osd.4' 2024-05-07T02:21:22.899 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-07T02:21:22.899 INFO:teuthology.orchestra.run.smithi064.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-07T02:21:22.899 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10+0 records in 2024-05-07T02:21:22.899 INFO:teuthology.orchestra.run.smithi064.stderr:10+0 records out 2024-05-07T02:21:22.899 INFO:teuthology.orchestra.run.smithi064.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0126681 s, 828 MB/s 2024-05-07T02:21:22.899 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 2024-05-07T02:21:22.899 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping successful for: 2024-05-07T02:21:22.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:22 smithi027 ceph-mon[34398]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-07T02:21:22.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:22 smithi027 ceph-mon[34398]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-05-07T02:21:22.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:22 smithi027 ceph-mon[34398]: osdmap e30: 5 total, 4 up, 5 in 2024-05-07T02:21:22.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:22 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-07T02:21:22.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:22 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-07T02:21:22.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:22 smithi027 ceph-mon[34398]: from='osd.4 ' entity='osd.4' 2024-05-07T02:21:22.957 INFO:journalctl@ceph.osd.4.smithi027.stdout:May 07 02:21:22 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-4[45836]: 2024-05-07T02:21:22.570+0000 7f8655054700 -1 osd.4 0 waiting for initial osdmap 2024-05-07T02:21:22.957 INFO:journalctl@ceph.osd.4.smithi027.stdout:May 07 02:21:22 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-4[45836]: 2024-05-07T02:21:22.578+0000 7f864ce01700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-07T02:21:23.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:22 smithi019 ceph-mon[29781]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-07T02:21:23.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:22 smithi019 ceph-mon[29781]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-05-07T02:21:23.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:22 smithi019 ceph-mon[29781]: osdmap e30: 5 total, 4 up, 5 in 2024-05-07T02:21:23.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:22 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-07T02:21:23.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:22 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-07T02:21:23.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:22 smithi019 ceph-mon[29781]: from='osd.4 ' entity='osd.4' 2024-05-07T02:21:23.588 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi064:vg_nvme/lv_3 2024-05-07T02:21:23.834 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.c/config 2024-05-07T02:21:23.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:23 smithi064 ceph-mon[34699]: purged_snaps scrub starts 2024-05-07T02:21:23.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:23 smithi064 ceph-mon[34699]: purged_snaps scrub ok 2024-05-07T02:21:23.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:23 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-07T02:21:23.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:23 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:23.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:23 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:23.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:23 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:21:23.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:23 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:21:23.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:23 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:23.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:23 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:21:23.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:23 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:21:23.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:23 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:23.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:23 smithi064 ceph-mon[34699]: osd.4 [v2:172.21.15.27:6808/3662186068,v1:172.21.15.27:6809/3662186068] boot 2024-05-07T02:21:23.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:23 smithi064 ceph-mon[34699]: osdmap e31: 5 total, 5 up, 5 in 2024-05-07T02:21:23.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:23 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-07T02:21:24.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:23 smithi019 ceph-mon[29781]: purged_snaps scrub starts 2024-05-07T02:21:24.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:23 smithi019 ceph-mon[29781]: purged_snaps scrub ok 2024-05-07T02:21:24.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:23 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-07T02:21:24.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:23 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:24.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:23 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:24.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:23 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:21:24.047 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:23 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:21:24.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:23 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:24.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:23 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:21:24.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:23 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:21:24.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:23 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:24.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:23 smithi019 ceph-mon[29781]: osd.4 [v2:172.21.15.27:6808/3662186068,v1:172.21.15.27:6809/3662186068] boot 2024-05-07T02:21:24.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:23 smithi019 ceph-mon[29781]: osdmap e31: 5 total, 5 up, 5 in 2024-05-07T02:21:24.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:23 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-07T02:21:24.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:23 smithi027 ceph-mon[34398]: purged_snaps scrub starts 2024-05-07T02:21:24.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:23 smithi027 ceph-mon[34398]: purged_snaps scrub ok 2024-05-07T02:21:24.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:23 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-07T02:21:24.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:23 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:24.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:23 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:24.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:23 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:21:24.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:23 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:21:24.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:23 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:24.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:23 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:21:24.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:23 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:21:24.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:23 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:24.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:23 smithi027 ceph-mon[34398]: osd.4 [v2:172.21.15.27:6808/3662186068,v1:172.21.15.27:6809/3662186068] boot 2024-05-07T02:21:24.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:23 smithi027 ceph-mon[34398]: osdmap e31: 5 total, 5 up, 5 in 2024-05-07T02:21:24.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:23 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-07T02:21:24.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:24 smithi027 ceph-mon[34398]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-07T02:21:24.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:24 smithi027 ceph-mon[34398]: Detected new or changed devices on smithi027 2024-05-07T02:21:24.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:24 smithi027 ceph-mon[34398]: Adjusting osd_memory_target on smithi027 to 8544M 2024-05-07T02:21:24.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:24 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:21:24.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:24 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:21:24.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:24 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:21:24.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:24 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:24.997 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:24 smithi064 ceph-mon[34699]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-07T02:21:24.998 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:24 smithi064 ceph-mon[34699]: Detected new or changed devices on smithi027 2024-05-07T02:21:24.998 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:24 smithi064 ceph-mon[34699]: Adjusting osd_memory_target on smithi027 to 8544M 2024-05-07T02:21:24.998 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:24 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:21:24.998 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:24 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:21:24.998 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:24 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:21:24.998 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:24 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:25.069 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:24 smithi019 ceph-mon[29781]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-07T02:21:25.069 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:24 smithi019 ceph-mon[29781]: Detected new or changed devices on smithi027 2024-05-07T02:21:25.069 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:24 smithi019 ceph-mon[29781]: Adjusting osd_memory_target on smithi027 to 8544M 2024-05-07T02:21:25.069 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:24 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:21:25.069 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:24 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:21:25.069 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:24 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:21:25.069 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:24 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:25.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:25.547+0000 7f7158852700 1 -- 172.21.15.64:0/1597476337 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7154103230 msgr2=0x7f7154103640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:21:25.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:25.547+0000 7f7158852700 1 --2- 172.21.15.64:0/1597476337 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7154103230 0x7f7154103640 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7144009990 tx=0x7f7144009ca0 comp rx=0 tx=0).stop 2024-05-07T02:21:25.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:25.547+0000 7f7158852700 1 -- 172.21.15.64:0/1597476337 shutdown_connections 2024-05-07T02:21:25.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:25.547+0000 7f7158852700 1 --2- 172.21.15.64:0/1597476337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7154104d00 0x7f715410b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:21:25.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:25.547+0000 7f7158852700 1 --2- 172.21.15.64:0/1597476337 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7154103230 0x7f7154103640 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:21:25.551 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:25.547+0000 7f7158852700 1 --2- 172.21.15.64:0/1597476337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7154104360 0x7f7154104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:21:25.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:25.547+0000 7f7158852700 1 -- 172.21.15.64:0/1597476337 >> 172.21.15.64:0/1597476337 conn(0x7f71540fe870 msgr2=0x7f7154100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:21:25.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:25.547+0000 7f7158852700 1 -- 172.21.15.64:0/1597476337 shutdown_connections 2024-05-07T02:21:25.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:25.547+0000 7f7158852700 1 -- 172.21.15.64:0/1597476337 wait complete. 2024-05-07T02:21:25.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:25.548+0000 7f7158852700 1 Processor -- start 2024-05-07T02:21:25.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:25.548+0000 7f7158852700 1 -- start start 2024-05-07T02:21:25.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:25.548+0000 7f7158852700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7154103230 0x7f715419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:21:25.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:25.548+0000 7f7158852700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7154104360 0x7f715419aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:21:25.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:25.549+0000 7f7158852700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7154104d00 0x7f71541a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:21:25.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:25.549+0000 7f7158852700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f715410ec80 con 0x7f7154103230 2024-05-07T02:21:25.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:25.549+0000 7f7158852700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f715410eb00 con 0x7f7154104d00 2024-05-07T02:21:25.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:25.549+0000 7f7158852700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f715410ee00 con 0x7f7154104360 2024-05-07T02:21:25.552 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:25.549+0000 7f7151d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7154104360 0x7f715419aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:21:25.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:25.549+0000 7f7151d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7154104360 0x7f715419aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47210/0 (socket says 172.21.15.64:47210) 2024-05-07T02:21:25.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:25.549+0000 7f7151d9b700 1 -- 172.21.15.64:0/3783379272 learned_addr learned my addr 172.21.15.64:0/3783379272 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-07T02:21:25.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:25.549+0000 7f7152d9d700 1 --2- 172.21.15.64:0/3783379272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7154104d00 0x7f71541a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:21:25.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:25.549+0000 7f715259c700 1 --2- 172.21.15.64:0/3783379272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7154103230 0x7f715419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:21:25.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:25.549+0000 7f7151d9b700 1 -- 172.21.15.64:0/3783379272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7154104d00 msgr2=0x7f71541a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:21:25.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:25.549+0000 7f7151d9b700 1 --2- 172.21.15.64:0/3783379272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7154104d00 0x7f71541a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:21:25.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:25.549+0000 7f7151d9b700 1 -- 172.21.15.64:0/3783379272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7154103230 msgr2=0x7f715419a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:21:25.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:25.549+0000 7f7151d9b700 1 --2- 172.21.15.64:0/3783379272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7154103230 0x7f715419a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:21:25.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:25.549+0000 7f7151d9b700 1 -- 172.21.15.64:0/3783379272 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71541a2500 con 0x7f7154104360 2024-05-07T02:21:25.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:25.549+0000 7f7151d9b700 1 --2- 172.21.15.64:0/3783379272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7154104360 0x7f715419aee0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f714800c920 tx=0x7f714800cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:21:25.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:25.549+0000 7f71437fe700 1 -- 172.21.15.64:0/3783379272 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7148004a30 con 0x7f7154104360 2024-05-07T02:21:25.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:25.550+0000 7f71437fe700 1 -- 172.21.15.64:0/3783379272 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71480054d0 con 0x7f7154104360 2024-05-07T02:21:25.553 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:25.550+0000 7f7158852700 1 -- 172.21.15.64:0/3783379272 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71541a27f0 con 0x7f7154104360 2024-05-07T02:21:25.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:25.550+0000 7f71437fe700 1 -- 172.21.15.64:0/3783379272 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7148007c70 con 0x7f7154104360 2024-05-07T02:21:25.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:25.550+0000 7f7158852700 1 -- 172.21.15.64:0/3783379272 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71541a2d80 con 0x7f7154104360 2024-05-07T02:21:25.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:25.551+0000 7f7158852700 1 -- 172.21.15.64:0/3783379272 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71540512c0 con 0x7f7154104360 2024-05-07T02:21:25.554 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:25.552+0000 7f71437fe700 1 -- 172.21.15.64:0/3783379272 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7148004c60 con 0x7f7154104360 2024-05-07T02:21:25.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:25.553+0000 7f71437fe700 1 --2- 172.21.15.64:0/3783379272 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f713c0b2900 0x7f713c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:21:25.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:25.553+0000 7f71437fe700 1 -- 172.21.15.64:0/3783379272 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4101+0+0 (secure 0 0 0) 0x7f71480e0fd0 con 0x7f7154104360 2024-05-07T02:21:25.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:25.553+0000 7f715259c700 1 --2- 172.21.15.64:0/3783379272 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f713c0b2900 0x7f713c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:21:25.555 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:25.554+0000 7f715259c700 1 --2- 172.21.15.64:0/3783379272 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f713c0b2900 0x7f713c0b4e10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f71440038a0 tx=0x7f714401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:21:25.558 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:25.558+0000 7f71437fe700 1 -- 172.21.15.64:0/3783379272 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7148014030 con 0x7f7154104360 2024-05-07T02:21:25.778 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:25.779+0000 7f7158852700 1 -- 172.21.15.64:0/3783379272 --> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f7154100960 con 0x7f713c0b2900 2024-05-07T02:21:26.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:25 smithi019 ceph-mon[29781]: osdmap e32: 5 total, 5 up, 5 in 2024-05-07T02:21:26.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:25 smithi019 ceph-mon[29781]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-07T02:21:26.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:25 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-07T02:21:26.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:25 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-07T02:21:26.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:25 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:21:26.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:25 smithi027 ceph-mon[34398]: osdmap e32: 5 total, 5 up, 5 in 2024-05-07T02:21:26.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:25 smithi027 ceph-mon[34398]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-07T02:21:26.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:25 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-07T02:21:26.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:25 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-07T02:21:26.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:25 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:21:26.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:25 smithi064 ceph-mon[34699]: osdmap e32: 5 total, 5 up, 5 in 2024-05-07T02:21:26.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:25 smithi064 ceph-mon[34699]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-07T02:21:26.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:25 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-07T02:21:26.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:25 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-07T02:21:26.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:25 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:21:27.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:26 smithi064 ceph-mon[34699]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:21:27.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:26 smithi019 ceph-mon[29781]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:21:27.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:26 smithi027 ceph-mon[34398]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:21:28.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:27 smithi064 ceph-mon[34699]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-07T02:21:28.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:27 smithi019 ceph-mon[29781]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-07T02:21:28.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:27 smithi027 ceph-mon[34398]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-07T02:21:29.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:28 smithi027 ceph-mon[34398]: from='client.? 172.21.15.64:0/3452295163' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60c207e9-55e8-4bed-87c3-f6a06a2a1fd1"}]: dispatch 2024-05-07T02:21:29.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:28 smithi027 ceph-mon[34398]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60c207e9-55e8-4bed-87c3-f6a06a2a1fd1"}]: dispatch 2024-05-07T02:21:29.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:28 smithi027 ceph-mon[34398]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "60c207e9-55e8-4bed-87c3-f6a06a2a1fd1"}]': finished 2024-05-07T02:21:29.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:28 smithi027 ceph-mon[34398]: osdmap e33: 6 total, 5 up, 6 in 2024-05-07T02:21:29.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:28 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-07T02:21:29.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:28 smithi064 ceph-mon[34699]: from='client.? 172.21.15.64:0/3452295163' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60c207e9-55e8-4bed-87c3-f6a06a2a1fd1"}]: dispatch 2024-05-07T02:21:29.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:28 smithi064 ceph-mon[34699]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60c207e9-55e8-4bed-87c3-f6a06a2a1fd1"}]: dispatch 2024-05-07T02:21:29.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:28 smithi064 ceph-mon[34699]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "60c207e9-55e8-4bed-87c3-f6a06a2a1fd1"}]': finished 2024-05-07T02:21:29.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:28 smithi064 ceph-mon[34699]: osdmap e33: 6 total, 5 up, 6 in 2024-05-07T02:21:29.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:28 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-07T02:21:29.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:28 smithi019 ceph-mon[29781]: from='client.? 172.21.15.64:0/3452295163' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60c207e9-55e8-4bed-87c3-f6a06a2a1fd1"}]: dispatch 2024-05-07T02:21:29.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:28 smithi019 ceph-mon[29781]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60c207e9-55e8-4bed-87c3-f6a06a2a1fd1"}]: dispatch 2024-05-07T02:21:29.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:28 smithi019 ceph-mon[29781]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "60c207e9-55e8-4bed-87c3-f6a06a2a1fd1"}]': finished 2024-05-07T02:21:29.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:28 smithi019 ceph-mon[29781]: osdmap e33: 6 total, 5 up, 6 in 2024-05-07T02:21:29.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:28 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-07T02:21:30.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:29 smithi027 ceph-mon[34398]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-07T02:21:30.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:29 smithi027 ceph-mon[34398]: from='client.? 172.21.15.64:0/3743692119' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-07T02:21:30.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:29 smithi064 ceph-mon[34699]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-07T02:21:30.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:29 smithi064 ceph-mon[34699]: from='client.? 172.21.15.64:0/3743692119' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-07T02:21:30.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:29 smithi019 ceph-mon[29781]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-07T02:21:30.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:29 smithi019 ceph-mon[29781]: from='client.? 172.21.15.64:0/3743692119' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-07T02:21:32.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:32 smithi019 ceph-mon[29781]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-07T02:21:32.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:32 smithi027 ceph-mon[34398]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-07T02:21:32.816 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:32 smithi064 ceph-mon[34699]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-07T02:21:34.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:34 smithi064 ceph-mon[34699]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-07T02:21:34.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:34 smithi019 ceph-mon[29781]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-07T02:21:34.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:34 smithi027 ceph-mon[34398]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-07T02:21:36.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:36 smithi064 ceph-mon[34699]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-07T02:21:36.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:36 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-07T02:21:36.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:36 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:21:36.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:36 smithi027 ceph-mon[34398]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-07T02:21:36.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:36 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-07T02:21:36.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:36 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:21:36.858 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:36 smithi019 ceph-mon[29781]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-07T02:21:36.858 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:36 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-07T02:21:36.858 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:36 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:21:37.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:37 smithi064 ceph-mon[34699]: Deploying daemon osd.5 on smithi064 2024-05-07T02:21:37.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:37 smithi027 ceph-mon[34398]: Deploying daemon osd.5 on smithi064 2024-05-07T02:21:37.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:37 smithi019 ceph-mon[29781]: Deploying daemon osd.5 on smithi064 2024-05-07T02:21:38.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:38 smithi064 ceph-mon[34699]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-07T02:21:38.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:38 smithi027 ceph-mon[34398]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-07T02:21:38.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:38 smithi019 ceph-mon[29781]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-07T02:21:40.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:40 smithi027 ceph-mon[34398]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-07T02:21:40.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:40 smithi064 ceph-mon[34699]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-07T02:21:40.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:40 smithi019 ceph-mon[29781]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-07T02:21:42.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:42 smithi027 ceph-mon[34398]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-07T02:21:42.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:42 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:42.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:42 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:42.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:42 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:21:42.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:42 smithi064 ceph-mon[34699]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-07T02:21:42.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:42 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:42.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:42 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:42.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:42 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:21:42.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:42 smithi019 ceph-mon[29781]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-07T02:21:42.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:42 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:42.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:42 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:42.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:42 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:21:44.580 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:44 smithi064 ceph-mon[34699]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-07T02:21:44.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:44 smithi027 ceph-mon[34398]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-07T02:21:44.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:44 smithi019 ceph-mon[29781]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-07T02:21:46.069 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 5 on host 'smithi064' 2024-05-07T02:21:46.070 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:46.069+0000 7f71437fe700 1 -- 172.21.15.64:0/3783379272 <== mgr.14152 v2:172.21.15.19:6800/2059564551 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7154100960 con 0x7f713c0b2900 2024-05-07T02:21:46.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:46.072+0000 7f7158852700 1 -- 172.21.15.64:0/3783379272 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f713c0b2900 msgr2=0x7f713c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:21:46.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:46.072+0000 7f7158852700 1 --2- 172.21.15.64:0/3783379272 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f713c0b2900 0x7f713c0b4e10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f71440038a0 tx=0x7f714401a040 comp rx=0 tx=0).stop 2024-05-07T02:21:46.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:46.072+0000 7f7158852700 1 -- 172.21.15.64:0/3783379272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7154104360 msgr2=0x7f715419aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:21:46.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:46.072+0000 7f7158852700 1 --2- 172.21.15.64:0/3783379272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7154104360 0x7f715419aee0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f714800c920 tx=0x7f714800cc30 comp rx=0 tx=0).stop 2024-05-07T02:21:46.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:46.072+0000 7f7158852700 1 -- 172.21.15.64:0/3783379272 shutdown_connections 2024-05-07T02:21:46.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:46.072+0000 7f7158852700 1 --2- 172.21.15.64:0/3783379272 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f713c0b2900 0x7f713c0b4e10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:21:46.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:46.072+0000 7f7158852700 1 --2- 172.21.15.64:0/3783379272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7154104d00 0x7f71541a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:21:46.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:46.072+0000 7f7158852700 1 --2- 172.21.15.64:0/3783379272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7154103230 0x7f715419a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:21:46.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:46.072+0000 7f7158852700 1 --2- 172.21.15.64:0/3783379272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7154104360 0x7f715419aee0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:21:46.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:46.072+0000 7f7158852700 1 -- 172.21.15.64:0/3783379272 >> 172.21.15.64:0/3783379272 conn(0x7f71540fe870 msgr2=0x7f7154100250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:21:46.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:46.072+0000 7f7158852700 1 -- 172.21.15.64:0/3783379272 shutdown_connections 2024-05-07T02:21:46.072 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:21:46.072+0000 7f7158852700 1 -- 172.21.15.64:0/3783379272 wait complete. 2024-05-07T02:21:46.629 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:46 smithi064 ceph-mon[34699]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-07T02:21:46.629 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:46 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:46.629 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:46 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:46.629 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:46 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:46.629 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:46 smithi064 ceph-mon[34699]: from='osd.5 [v2:172.21.15.64:6808/1264591849,v1:172.21.15.64:6809/1264591849]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-07T02:21:46.629 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:46 smithi064 ceph-mon[34699]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-07T02:21:46.629 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:46 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:46.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:46 smithi027 ceph-mon[34398]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-07T02:21:46.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:46 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:46.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:46 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:46.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:46 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:46.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:46 smithi027 ceph-mon[34398]: from='osd.5 [v2:172.21.15.64:6808/1264591849,v1:172.21.15.64:6809/1264591849]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-07T02:21:46.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:46 smithi027 ceph-mon[34398]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-07T02:21:46.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:46 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:46.710 DEBUG:teuthology.orchestra.run.smithi064:osd.5> sudo journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@osd.5.service 2024-05-07T02:21:46.712 INFO:tasks.cephadm:Deploying osd.6 on smithi019 with /dev/vg_nvme/lv_2... 2024-05-07T02:21:46.712 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-07T02:21:46.799 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:46 smithi019 ceph-mon[29781]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-07T02:21:46.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:46 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:46.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:46 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:46.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:46 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:46.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:46 smithi019 ceph-mon[29781]: from='osd.5 [v2:172.21.15.64:6808/1264591849,v1:172.21.15.64:6809/1264591849]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-07T02:21:46.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:46 smithi019 ceph-mon[29781]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-07T02:21:46.800 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:46 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:47.174 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:21:48.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:48 smithi019 ceph-mon[29781]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-07T02:21:48.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:48 smithi019 ceph-mon[29781]: from='osd.5 [v2:172.21.15.64:6808/1264591849,v1:172.21.15.64:6809/1264591849]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-07T02:21:48.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:48 smithi019 ceph-mon[29781]: osdmap e34: 6 total, 5 up, 6 in 2024-05-07T02:21:48.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:48 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-07T02:21:48.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:48 smithi019 ceph-mon[29781]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-07T02:21:48.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:48 smithi019 ceph-mon[29781]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-07T02:21:48.295 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:48 smithi064 ceph-mon[34699]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-07T02:21:48.295 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:48 smithi064 ceph-mon[34699]: from='osd.5 [v2:172.21.15.64:6808/1264591849,v1:172.21.15.64:6809/1264591849]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-07T02:21:48.295 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:48 smithi064 ceph-mon[34699]: osdmap e34: 6 total, 5 up, 6 in 2024-05-07T02:21:48.295 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:48 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-07T02:21:48.295 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:48 smithi064 ceph-mon[34699]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-07T02:21:48.295 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:48 smithi064 ceph-mon[34699]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-07T02:21:48.296 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:21:48 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5[46831]: 2024-05-07T02:21:48.094+0000 7f317b607700 -1 osd.5 0 waiting for initial osdmap 2024-05-07T02:21:48.296 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:21:48 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5[46831]: 2024-05-07T02:21:48.103+0000 7f31773bc700 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-07T02:21:48.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:48 smithi027 ceph-mon[34398]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-07T02:21:48.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:48 smithi027 ceph-mon[34398]: from='osd.5 [v2:172.21.15.64:6808/1264591849,v1:172.21.15.64:6809/1264591849]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-07T02:21:48.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:48 smithi027 ceph-mon[34398]: osdmap e34: 6 total, 5 up, 6 in 2024-05-07T02:21:48.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:48 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-07T02:21:48.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:48 smithi027 ceph-mon[34398]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-07T02:21:48.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:48 smithi027 ceph-mon[34398]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-07T02:21:49.231 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:49 smithi064 ceph-mon[34699]: purged_snaps scrub starts 2024-05-07T02:21:49.231 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:49 smithi064 ceph-mon[34699]: purged_snaps scrub ok 2024-05-07T02:21:49.231 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:49 smithi064 ceph-mon[34699]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-07T02:21:49.232 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:49 smithi064 ceph-mon[34699]: osdmap e35: 6 total, 5 up, 6 in 2024-05-07T02:21:49.232 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:49 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-07T02:21:49.232 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:49 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-07T02:21:49.339 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:49 smithi019 ceph-mon[29781]: purged_snaps scrub starts 2024-05-07T02:21:49.340 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:49 smithi019 ceph-mon[29781]: purged_snaps scrub ok 2024-05-07T02:21:49.340 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:49 smithi019 ceph-mon[29781]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-07T02:21:49.340 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:49 smithi019 ceph-mon[29781]: osdmap e35: 6 total, 5 up, 6 in 2024-05-07T02:21:49.340 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:49 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-07T02:21:49.340 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:49 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-07T02:21:49.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:49 smithi027 ceph-mon[34398]: purged_snaps scrub starts 2024-05-07T02:21:49.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:49 smithi027 ceph-mon[34398]: purged_snaps scrub ok 2024-05-07T02:21:49.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:49 smithi027 ceph-mon[34398]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-07T02:21:49.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:49 smithi027 ceph-mon[34398]: osdmap e35: 6 total, 5 up, 6 in 2024-05-07T02:21:49.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:49 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-07T02:21:49.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:49 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-07T02:21:50.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:50 smithi019 ceph-mon[29781]: osd.5 [v2:172.21.15.64:6808/1264591849,v1:172.21.15.64:6809/1264591849] boot 2024-05-07T02:21:50.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:50 smithi019 ceph-mon[29781]: osdmap e36: 6 total, 6 up, 6 in 2024-05-07T02:21:50.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:50 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-07T02:21:50.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:50 smithi019 ceph-mon[29781]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-07T02:21:50.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:50 smithi019 ceph-mon[29781]: Detected new or changed devices on smithi064 2024-05-07T02:21:50.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:50 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:50.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:50 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:50.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:50 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:21:50.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:50 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:21:50.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:50 smithi019 ceph-mon[29781]: Adjusting osd_memory_target on smithi064 to 8544M 2024-05-07T02:21:50.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:50 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:50.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:50 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:21:50.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:50 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:21:50.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:50 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:50.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:50 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:21:50.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:50 smithi019 ceph-mon[29781]: osdmap e37: 6 total, 6 up, 6 in 2024-05-07T02:21:50.410 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-07T02:21:50.411 INFO:teuthology.orchestra.run.smithi019.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-07T02:21:50.411 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10+0 records in 2024-05-07T02:21:50.411 INFO:teuthology.orchestra.run.smithi019.stderr:10+0 records out 2024-05-07T02:21:50.411 INFO:teuthology.orchestra.run.smithi019.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0358169 s, 293 MB/s 2024-05-07T02:21:50.411 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping successful for: 2024-05-07T02:21:50.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:50 smithi027 ceph-mon[34398]: osd.5 [v2:172.21.15.64:6808/1264591849,v1:172.21.15.64:6809/1264591849] boot 2024-05-07T02:21:50.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:50 smithi027 ceph-mon[34398]: osdmap e36: 6 total, 6 up, 6 in 2024-05-07T02:21:50.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:50 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-07T02:21:50.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:50 smithi027 ceph-mon[34398]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-07T02:21:50.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:50 smithi027 ceph-mon[34398]: Detected new or changed devices on smithi064 2024-05-07T02:21:50.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:50 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:50.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:50 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:50.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:50 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:21:50.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:50 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:21:50.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:50 smithi027 ceph-mon[34398]: Adjusting osd_memory_target on smithi064 to 8544M 2024-05-07T02:21:50.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:50 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:50.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:50 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:21:50.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:50 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:21:50.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:50 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:50.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:50 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:21:50.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:50 smithi027 ceph-mon[34398]: osdmap e37: 6 total, 6 up, 6 in 2024-05-07T02:21:50.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:50 smithi064 ceph-mon[34699]: osd.5 [v2:172.21.15.64:6808/1264591849,v1:172.21.15.64:6809/1264591849] boot 2024-05-07T02:21:50.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:50 smithi064 ceph-mon[34699]: osdmap e36: 6 total, 6 up, 6 in 2024-05-07T02:21:50.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:50 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-07T02:21:50.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:50 smithi064 ceph-mon[34699]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-07T02:21:50.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:50 smithi064 ceph-mon[34699]: Detected new or changed devices on smithi064 2024-05-07T02:21:50.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:50 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:50.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:50 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:50.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:50 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:21:50.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:50 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:21:50.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:50 smithi064 ceph-mon[34699]: Adjusting osd_memory_target on smithi064 to 8544M 2024-05-07T02:21:50.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:50 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:50.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:50 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:21:50.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:50 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:21:50.610 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:50 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:50.610 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:50 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:21:50.610 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:50 smithi064 ceph-mon[34699]: osdmap e37: 6 total, 6 up, 6 in 2024-05-07T02:21:51.243 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi019:vg_nvme/lv_2 2024-05-07T02:21:51.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:51 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:21:51.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:51 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:21:51.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:51 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:51.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:51 smithi019 ceph-mon[29781]: osdmap e38: 6 total, 6 up, 6 in 2024-05-07T02:21:51.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:51 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:21:51.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:51 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:21:51.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:51 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:51.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:51 smithi027 ceph-mon[34398]: osdmap e38: 6 total, 6 up, 6 in 2024-05-07T02:21:51.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:51 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:21:51.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:51 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:21:51.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:51 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:21:51.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:51 smithi064 ceph-mon[34699]: osdmap e38: 6 total, 6 up, 6 in 2024-05-07T02:21:51.693 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:21:52.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:52 smithi019 ceph-mon[29781]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-07T02:21:52.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:52 smithi027 ceph-mon[34398]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-07T02:21:52.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:52 smithi064 ceph-mon[34699]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-07T02:21:54.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:21:54.012+0000 7f34fb200700 1 -- 172.21.15.19:0/1526021495 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34fc1075b0 msgr2=0x7f34fc1099d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:21:54.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:21:54.012+0000 7f34fb200700 1 --2- 172.21.15.19:0/1526021495 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34fc1075b0 0x7f34fc1099d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f34e80098f0 tx=0x7f34e8009c00 comp rx=0 tx=0).stop 2024-05-07T02:21:54.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:21:54.012+0000 7f34fb200700 1 -- 172.21.15.19:0/1526021495 shutdown_connections 2024-05-07T02:21:54.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:21:54.012+0000 7f34fb200700 1 --2- 172.21.15.19:0/1526021495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34fc109f10 0x7f34fc10c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:21:54.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:21:54.012+0000 7f34fb200700 1 --2- 172.21.15.19:0/1526021495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34fc104c50 0x7f34fc107070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:21:54.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:21:54.012+0000 7f34fb200700 1 --2- 172.21.15.19:0/1526021495 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34fc1075b0 0x7f34fc1099d0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:21:54.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:21:54.012+0000 7f34fb200700 1 -- 172.21.15.19:0/1526021495 >> 172.21.15.19:0/1526021495 conn(0x7f34fc0fe690 msgr2=0x7f34fc100b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:21:54.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:21:54.012+0000 7f34fb200700 1 -- 172.21.15.19:0/1526021495 shutdown_connections 2024-05-07T02:21:54.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:21:54.012+0000 7f34fb200700 1 -- 172.21.15.19:0/1526021495 wait complete. 2024-05-07T02:21:54.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:21:54.014+0000 7f34fb200700 1 Processor -- start 2024-05-07T02:21:54.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:21:54.014+0000 7f34fb200700 1 -- start start 2024-05-07T02:21:54.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:21:54.015+0000 7f34fb200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34fc104c50 0x7f34fc19c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:21:54.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:21:54.015+0000 7f34fb200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34fc1075b0 0x7f34fc19cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:21:54.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:21:54.015+0000 7f34fb200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34fc109f10 0x7f34fc1a3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:21:54.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:21:54.015+0000 7f34fb200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f34fc10fc10 con 0x7f34fc109f10 2024-05-07T02:21:54.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:21:54.015+0000 7f34fb200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f34fc10fa90 con 0x7f34fc1075b0 2024-05-07T02:21:54.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:21:54.015+0000 7f34fb200700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f34fc10fd90 con 0x7f34fc104c50 2024-05-07T02:21:54.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:21:54.015+0000 7f34fa800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34fc109f10 0x7f34fc1a3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:21:54.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:21:54.015+0000 7f34f9e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34fc104c50 0x7f34fc19c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:21:54.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:21:54.015+0000 7f34fa800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34fc109f10 0x7f34fc1a3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37696/0 (socket says 172.21.15.19:37696) 2024-05-07T02:21:54.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:21:54.015+0000 7f34fa800700 1 -- 172.21.15.19:0/2456570576 learned_addr learned my addr 172.21.15.19:0/2456570576 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:21:54.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:21:54.015+0000 7f34f9400700 1 --2- 172.21.15.19:0/2456570576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34fc1075b0 0x7f34fc19cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:21:54.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:21:54.015+0000 7f34f9e00700 1 -- 172.21.15.19:0/2456570576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34fc1075b0 msgr2=0x7f34fc19cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:21:54.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:21:54.015+0000 7f34f9e00700 1 --2- 172.21.15.19:0/2456570576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34fc1075b0 0x7f34fc19cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:21:54.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:21:54.016+0000 7f34f9e00700 1 -- 172.21.15.19:0/2456570576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34fc109f10 msgr2=0x7f34fc1a3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:21:54.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:21:54.016+0000 7f34f9e00700 1 --2- 172.21.15.19:0/2456570576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34fc109f10 0x7f34fc1a3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:21:54.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:21:54.016+0000 7f34f9e00700 1 -- 172.21.15.19:0/2456570576 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34fc1a4550 con 0x7f34fc104c50 2024-05-07T02:21:54.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:21:54.016+0000 7f34f9400700 1 --2- 172.21.15.19:0/2456570576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34fc1075b0 0x7f34fc19cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:21:54.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:21:54.016+0000 7f34f9e00700 1 --2- 172.21.15.19:0/2456570576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34fc104c50 0x7f34fc19c9f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f34ec00e9c0 tx=0x7f34ec00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:21:54.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:21:54.016+0000 7f34f2a00700 1 -- 172.21.15.19:0/2456570576 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ec00c9c0 con 0x7f34fc104c50 2024-05-07T02:21:54.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:21:54.016+0000 7f34f2a00700 1 -- 172.21.15.19:0/2456570576 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34ec00cb60 con 0x7f34fc104c50 2024-05-07T02:21:54.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:21:54.016+0000 7f34fb200700 1 -- 172.21.15.19:0/2456570576 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34fc1a4840 con 0x7f34fc104c50 2024-05-07T02:21:54.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:21:54.016+0000 7f34f2a00700 1 -- 172.21.15.19:0/2456570576 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ec018680 con 0x7f34fc104c50 2024-05-07T02:21:54.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:21:54.016+0000 7f34fb200700 1 -- 172.21.15.19:0/2456570576 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34fc1a4dd0 con 0x7f34fc104c50 2024-05-07T02:21:54.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:21:54.017+0000 7f34fb200700 1 -- 172.21.15.19:0/2456570576 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34fc0512c0 con 0x7f34fc104c50 2024-05-07T02:21:54.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:21:54.020+0000 7f34f2a00700 1 -- 172.21.15.19:0/2456570576 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f34ec04aab0 con 0x7f34fc104c50 2024-05-07T02:21:54.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:21:54.022+0000 7f34f2a00700 1 --2- 172.21.15.19:0/2456570576 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f34e00b2900 0x7f34e00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:21:54.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:21:54.022+0000 7f34f2a00700 1 -- 172.21.15.19:0/2456570576 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4533+0+0 (secure 0 0 0) 0x7f34ec0e1650 con 0x7f34fc104c50 2024-05-07T02:21:54.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:21:54.022+0000 7f34f9400700 1 --2- 172.21.15.19:0/2456570576 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f34e00b2900 0x7f34e00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:21:54.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:21:54.023+0000 7f34f9400700 1 --2- 172.21.15.19:0/2456570576 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f34e00b2900 0x7f34e00b4e10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f34fc19daa0 tx=0x7f34e8019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:21:54.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:21:54.024+0000 7f34f2a00700 1 -- 172.21.15.19:0/2456570576 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f34ec0aa190 con 0x7f34fc104c50 2024-05-07T02:21:54.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:21:54.271+0000 7f34fb200700 1 -- 172.21.15.19:0/2456570576 --> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f34fc1a5060 con 0x7f34e00b2900 2024-05-07T02:21:54.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:54 smithi019 ceph-mon[29781]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-05-07T02:21:54.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:54 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-07T02:21:54.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:54 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-07T02:21:54.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:54 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:21:54.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:54 smithi027 ceph-mon[34398]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-05-07T02:21:54.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:54 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-07T02:21:54.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:54 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-07T02:21:54.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:54 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:21:54.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:54 smithi064 ceph-mon[34699]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-05-07T02:21:54.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:54 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-07T02:21:54.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:54 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-07T02:21:54.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:54 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:21:55.560 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:55 smithi019 ceph-mon[29781]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:21:55.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:55 smithi027 ceph-mon[34398]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:21:55.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:55 smithi064 ceph-mon[34699]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:21:56.700 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:56 smithi064 ceph-mon[34699]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-05-07T02:21:56.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:56 smithi027 ceph-mon[34398]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-05-07T02:21:56.860 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:56 smithi019 ceph-mon[29781]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-05-07T02:21:58.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:58 smithi027 ceph-mon[34398]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-07T02:21:58.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:58 smithi064 ceph-mon[34699]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-07T02:21:58.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:58 smithi019 ceph-mon[29781]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-07T02:21:59.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:59 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/2932167090' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3e038e0-4026-45f1-a923-c07cbb307f64"}]: dispatch 2024-05-07T02:21:59.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:59 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/2932167090' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f3e038e0-4026-45f1-a923-c07cbb307f64"}]': finished 2024-05-07T02:21:59.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:59 smithi019 ceph-mon[29781]: osdmap e39: 7 total, 6 up, 7 in 2024-05-07T02:21:59.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:21:59 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-07T02:21:59.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:59 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/2932167090' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3e038e0-4026-45f1-a923-c07cbb307f64"}]: dispatch 2024-05-07T02:21:59.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:59 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/2932167090' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f3e038e0-4026-45f1-a923-c07cbb307f64"}]': finished 2024-05-07T02:21:59.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:59 smithi027 ceph-mon[34398]: osdmap e39: 7 total, 6 up, 7 in 2024-05-07T02:21:59.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:21:59 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-07T02:21:59.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:59 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/2932167090' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3e038e0-4026-45f1-a923-c07cbb307f64"}]: dispatch 2024-05-07T02:21:59.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:59 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/2932167090' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f3e038e0-4026-45f1-a923-c07cbb307f64"}]': finished 2024-05-07T02:21:59.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:59 smithi064 ceph-mon[34699]: osdmap e39: 7 total, 6 up, 7 in 2024-05-07T02:21:59.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:21:59 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-07T02:22:00.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:00 smithi027 ceph-mon[34398]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-07T02:22:00.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:00 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/4210454201' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-07T02:22:00.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:00 smithi064 ceph-mon[34699]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-07T02:22:00.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:00 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/4210454201' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-07T02:22:00.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:00 smithi019 ceph-mon[29781]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-07T02:22:00.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:00 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/4210454201' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-07T02:22:02.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:02 smithi027 ceph-mon[34398]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-07T02:22:02.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:02 smithi064 ceph-mon[34699]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-07T02:22:02.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:02 smithi019 ceph-mon[29781]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-07T02:22:04.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:04 smithi027 ceph-mon[34398]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-07T02:22:04.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:04 smithi064 ceph-mon[34699]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-07T02:22:04.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:04 smithi019 ceph-mon[29781]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-07T02:22:06.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:06 smithi019 ceph-mon[29781]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-07T02:22:06.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:06 smithi027 ceph-mon[34398]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-07T02:22:06.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:06 smithi064 ceph-mon[34699]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-07T02:22:08.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:08 smithi019 ceph-mon[29781]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-07T02:22:08.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:08 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-07T02:22:08.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:08 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:22:08.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:08 smithi027 ceph-mon[34398]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-07T02:22:08.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:08 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-07T02:22:08.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:08 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:22:08.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:08 smithi064 ceph-mon[34699]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-07T02:22:08.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:08 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-07T02:22:08.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:08 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:22:09.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:09 smithi027 ceph-mon[34398]: Deploying daemon osd.6 on smithi019 2024-05-07T02:22:09.827 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:09 smithi019 ceph-mon[29781]: Deploying daemon osd.6 on smithi019 2024-05-07T02:22:09.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:09 smithi064 ceph-mon[34699]: Deploying daemon osd.6 on smithi019 2024-05-07T02:22:10.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:10 smithi019 ceph-mon[29781]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-07T02:22:10.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:10 smithi064 ceph-mon[34699]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-07T02:22:10.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:10 smithi027 ceph-mon[34398]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-07T02:22:12.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:12 smithi019 ceph-mon[29781]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-07T02:22:12.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:12 smithi064 ceph-mon[34699]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-07T02:22:12.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:12 smithi027 ceph-mon[34398]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-07T02:22:14.595 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:14 smithi019 ceph-mon[29781]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-07T02:22:14.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:14 smithi064 ceph-mon[34699]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-07T02:22:14.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:14 smithi027 ceph-mon[34398]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-07T02:22:16.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:16 smithi064 ceph-mon[34699]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-07T02:22:16.884 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:16 smithi019 ceph-mon[29781]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-07T02:22:16.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:16 smithi027 ceph-mon[34398]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-07T02:22:18.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:18 smithi064 ceph-mon[34699]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-07T02:22:18.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:18 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:18.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:18 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:18.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:18 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:22:18.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:18 smithi019 ceph-mon[29781]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-07T02:22:18.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:18 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:18.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:18 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:18.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:18 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:22:18.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:18 smithi027 ceph-mon[34398]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-07T02:22:18.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:18 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:18.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:18 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:18.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:18 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:22:20.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:20 smithi019 ceph-mon[29781]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-07T02:22:20.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:20 smithi064 ceph-mon[34699]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-07T02:22:20.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:20 smithi027 ceph-mon[34398]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-07T02:22:22.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:22 smithi019 ceph-mon[29781]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-07T02:22:22.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:22 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:22.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:22 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:22.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:22 smithi019 ceph-mon[29781]: from='osd.6 [v2:172.21.15.19:6818/2158875543,v1:172.21.15.19:6819/2158875543]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-07T02:22:22.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:22 smithi019 ceph-mon[29781]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-07T02:22:22.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:22 smithi064 ceph-mon[34699]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-07T02:22:22.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:22 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:22.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:22 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:22.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:22 smithi064 ceph-mon[34699]: from='osd.6 [v2:172.21.15.19:6818/2158875543,v1:172.21.15.19:6819/2158875543]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-07T02:22:22.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:22 smithi064 ceph-mon[34699]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-07T02:22:22.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:22 smithi027 ceph-mon[34398]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-07T02:22:22.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:22 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:22.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:22 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:22.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:22 smithi027 ceph-mon[34398]: from='osd.6 [v2:172.21.15.19:6818/2158875543,v1:172.21.15.19:6819/2158875543]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-07T02:22:22.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:22 smithi027 ceph-mon[34398]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-07T02:22:23.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:22:23.926+0000 7f34f2a00700 1 -- 172.21.15.19:0/2456570576 <== mgr.14152 v2:172.21.15.19:6800/2059564551 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f34fc1a5060 con 0x7f34e00b2900 2024-05-07T02:22:23.928 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 6 on host 'smithi019' 2024-05-07T02:22:23.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:22:23.931+0000 7f34fb200700 1 -- 172.21.15.19:0/2456570576 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f34e00b2900 msgr2=0x7f34e00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:22:23.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:22:23.931+0000 7f34fb200700 1 --2- 172.21.15.19:0/2456570576 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f34e00b2900 0x7f34e00b4e10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f34fc19daa0 tx=0x7f34e8019040 comp rx=0 tx=0).stop 2024-05-07T02:22:23.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:22:23.931+0000 7f34fb200700 1 -- 172.21.15.19:0/2456570576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34fc104c50 msgr2=0x7f34fc19c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:22:23.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:22:23.931+0000 7f34fb200700 1 --2- 172.21.15.19:0/2456570576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34fc104c50 0x7f34fc19c9f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f34ec00e9c0 tx=0x7f34ec00ecd0 comp rx=0 tx=0).stop 2024-05-07T02:22:23.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:22:23.931+0000 7f34fb200700 1 -- 172.21.15.19:0/2456570576 shutdown_connections 2024-05-07T02:22:23.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:22:23.931+0000 7f34fb200700 1 --2- 172.21.15.19:0/2456570576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34fc109f10 0x7f34fc1a3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:22:23.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:22:23.931+0000 7f34fb200700 1 --2- 172.21.15.19:0/2456570576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f34fc104c50 0x7f34fc19c9f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:22:23.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:22:23.931+0000 7f34fb200700 1 --2- 172.21.15.19:0/2456570576 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f34e00b2900 0x7f34e00b4e10 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:22:23.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:22:23.931+0000 7f34fb200700 1 --2- 172.21.15.19:0/2456570576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34fc1075b0 0x7f34fc19cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:22:23.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:22:23.932+0000 7f34fb200700 1 -- 172.21.15.19:0/2456570576 >> 172.21.15.19:0/2456570576 conn(0x7f34fc0fe690 msgr2=0x7f34fc102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:22:23.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:22:23.932+0000 7f34fb200700 1 -- 172.21.15.19:0/2456570576 shutdown_connections 2024-05-07T02:22:23.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:22:23.932+0000 7f34fb200700 1 -- 172.21.15.19:0/2456570576 wait complete. 2024-05-07T02:22:24.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:24 smithi019 ceph-mon[29781]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-07T02:22:24.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:24 smithi019 ceph-mon[29781]: from='osd.6 [v2:172.21.15.19:6818/2158875543,v1:172.21.15.19:6819/2158875543]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-07T02:22:24.381 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:24 smithi019 ceph-mon[29781]: osdmap e40: 7 total, 6 up, 7 in 2024-05-07T02:22:24.381 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:24 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-07T02:22:24.381 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:24 smithi019 ceph-mon[29781]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-07T02:22:24.381 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:24 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:22:24.381 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:24 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:22:24.381 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:24 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:24.381 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:24 smithi019 ceph-mon[29781]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-07T02:22:24.381 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:24 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:24.381 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:24 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:22:24.381 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:24 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:24.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:24 smithi027 ceph-mon[34398]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-07T02:22:24.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:24 smithi027 ceph-mon[34398]: from='osd.6 [v2:172.21.15.19:6818/2158875543,v1:172.21.15.19:6819/2158875543]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-07T02:22:24.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:24 smithi027 ceph-mon[34398]: osdmap e40: 7 total, 6 up, 7 in 2024-05-07T02:22:24.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:24 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-07T02:22:24.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:24 smithi027 ceph-mon[34398]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-07T02:22:24.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:24 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:22:24.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:24 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:22:24.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:24 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:24.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:24 smithi027 ceph-mon[34398]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-07T02:22:24.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:24 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:24.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:24 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:22:24.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:24 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:24.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:24 smithi064 ceph-mon[34699]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-07T02:22:24.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:24 smithi064 ceph-mon[34699]: from='osd.6 [v2:172.21.15.19:6818/2158875543,v1:172.21.15.19:6819/2158875543]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-07T02:22:24.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:24 smithi064 ceph-mon[34699]: osdmap e40: 7 total, 6 up, 7 in 2024-05-07T02:22:24.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:24 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-07T02:22:24.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:24 smithi064 ceph-mon[34699]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-07T02:22:24.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:24 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:22:24.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:24 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:22:24.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:24 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:24.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:24 smithi064 ceph-mon[34699]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-07T02:22:24.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:24 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:24.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:24 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:22:24.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:24 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:24.872 DEBUG:teuthology.orchestra.run.smithi019:osd.6> sudo journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@osd.6.service 2024-05-07T02:22:24.877 INFO:tasks.cephadm:Deploying osd.7 on smithi027 with /dev/vg_nvme/lv_2... 2024-05-07T02:22:24.877 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-07T02:22:25.135 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.b/config 2024-05-07T02:22:25.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:25 smithi027 ceph-mon[34398]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-07T02:22:25.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:25 smithi027 ceph-mon[34398]: osdmap e41: 7 total, 6 up, 7 in 2024-05-07T02:22:25.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:25 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-07T02:22:25.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:25 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-07T02:22:25.388 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:25 smithi019 ceph-mon[29781]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-07T02:22:25.388 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:25 smithi019 ceph-mon[29781]: osdmap e41: 7 total, 6 up, 7 in 2024-05-07T02:22:25.388 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:25 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-07T02:22:25.389 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:25 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-07T02:22:25.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:25 smithi064 ceph-mon[34699]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-07T02:22:25.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:25 smithi064 ceph-mon[34699]: osdmap e41: 7 total, 6 up, 7 in 2024-05-07T02:22:25.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:25 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-07T02:22:25.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:25 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-07T02:22:26.278 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:26 smithi019 ceph-mon[29781]: purged_snaps scrub starts 2024-05-07T02:22:26.279 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:26 smithi019 ceph-mon[29781]: purged_snaps scrub ok 2024-05-07T02:22:26.279 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:26 smithi019 ceph-mon[29781]: osd.6 [v2:172.21.15.19:6818/2158875543,v1:172.21.15.19:6819/2158875543] boot 2024-05-07T02:22:26.279 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:26 smithi019 ceph-mon[29781]: osdmap e42: 7 total, 7 up, 7 in 2024-05-07T02:22:26.279 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:26 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-07T02:22:26.279 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:26 smithi019 ceph-mon[29781]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-07T02:22:26.405 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:26 smithi064 ceph-mon[34699]: purged_snaps scrub starts 2024-05-07T02:22:26.405 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:26 smithi064 ceph-mon[34699]: purged_snaps scrub ok 2024-05-07T02:22:26.405 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:26 smithi064 ceph-mon[34699]: osd.6 [v2:172.21.15.19:6818/2158875543,v1:172.21.15.19:6819/2158875543] boot 2024-05-07T02:22:26.405 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:26 smithi064 ceph-mon[34699]: osdmap e42: 7 total, 7 up, 7 in 2024-05-07T02:22:26.405 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:26 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-07T02:22:26.405 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:26 smithi064 ceph-mon[34699]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-07T02:22:26.408 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:26 smithi027 ceph-mon[34398]: purged_snaps scrub starts 2024-05-07T02:22:26.408 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:26 smithi027 ceph-mon[34398]: purged_snaps scrub ok 2024-05-07T02:22:26.408 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:26 smithi027 ceph-mon[34398]: osd.6 [v2:172.21.15.19:6818/2158875543,v1:172.21.15.19:6819/2158875543] boot 2024-05-07T02:22:26.409 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:26 smithi027 ceph-mon[34398]: osdmap e42: 7 total, 7 up, 7 in 2024-05-07T02:22:26.409 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:26 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-07T02:22:26.409 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:26 smithi027 ceph-mon[34398]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-05-07T02:22:27.145 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:27 smithi027 ceph-mon[34398]: osdmap e43: 7 total, 7 up, 7 in 2024-05-07T02:22:27.202 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-07T02:22:27.202 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-07T02:22:27.202 INFO:teuthology.orchestra.run.smithi027.stderr: stderr: 10+0 records in 2024-05-07T02:22:27.202 INFO:teuthology.orchestra.run.smithi027.stderr:10+0 records out 2024-05-07T02:22:27.202 INFO:teuthology.orchestra.run.smithi027.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112583 s, 931 MB/s 2024-05-07T02:22:27.202 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping successful for: 2024-05-07T02:22:27.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:27 smithi064 ceph-mon[34699]: osdmap e43: 7 total, 7 up, 7 in 2024-05-07T02:22:27.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:27 smithi019 ceph-mon[29781]: osdmap e43: 7 total, 7 up, 7 in 2024-05-07T02:22:27.904 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi027:vg_nvme/lv_2 2024-05-07T02:22:28.150 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.b/config 2024-05-07T02:22:28.411 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:28 smithi027 ceph-mon[34398]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-07T02:22:28.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:28 smithi064 ceph-mon[34699]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-07T02:22:28.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:28 smithi019 ceph-mon[29781]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-07T02:22:29.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:29.881+0000 7f55c2f0e700 1 -- 172.21.15.27:0/2828370539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55bc104370 msgr2=0x7f55bc104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:22:29.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:29.881+0000 7f55c2f0e700 1 --2- 172.21.15.27:0/2828370539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55bc104370 0x7f55bc104780 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f55b0009990 tx=0x7f55b0009ca0 comp rx=0 tx=0).stop 2024-05-07T02:22:29.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:29.881+0000 7f55c2f0e700 1 -- 172.21.15.27:0/2828370539 shutdown_connections 2024-05-07T02:22:29.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:29.881+0000 7f55c2f0e700 1 --2- 172.21.15.27:0/2828370539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55bc105e40 0x7f55bc10c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:22:29.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:29.881+0000 7f55c2f0e700 1 --2- 172.21.15.27:0/2828370539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55bc104370 0x7f55bc104780 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:22:29.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:29.881+0000 7f55c2f0e700 1 --2- 172.21.15.27:0/2828370539 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55bc1054a0 0x7f55bc1058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:22:29.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:29.881+0000 7f55c2f0e700 1 -- 172.21.15.27:0/2828370539 >> 172.21.15.27:0/2828370539 conn(0x7f55bc0ff9d0 msgr2=0x7f55bc101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:22:29.883 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:29.881+0000 7f55c2f0e700 1 -- 172.21.15.27:0/2828370539 shutdown_connections 2024-05-07T02:22:29.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:29.881+0000 7f55c2f0e700 1 -- 172.21.15.27:0/2828370539 wait complete. 2024-05-07T02:22:29.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:29.882+0000 7f55c2f0e700 1 Processor -- start 2024-05-07T02:22:29.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:29.882+0000 7f55c2f0e700 1 -- start start 2024-05-07T02:22:29.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:29.882+0000 7f55c2f0e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55bc104370 0x7f55bc19a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:22:29.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:29.882+0000 7f55c2f0e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55bc1054a0 0x7f55bc19aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:22:29.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:29.882+0000 7f55c2f0e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55bc105e40 0x7f55bc1a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:22:29.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:29.882+0000 7f55c2f0e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc10fe30 con 0x7f55bc104370 2024-05-07T02:22:29.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:29.882+0000 7f55c2f0e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc10fcb0 con 0x7f55bc105e40 2024-05-07T02:22:29.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:29.882+0000 7f55c2f0e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc10ffb0 con 0x7f55bc1054a0 2024-05-07T02:22:29.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:29.882+0000 7f55c14ab700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55bc105e40 0x7f55bc1a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:22:29.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:29.882+0000 7f55c14ab700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55bc105e40 0x7f55bc1a1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50702/0 (socket says 172.21.15.27:50702) 2024-05-07T02:22:29.884 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:29.882+0000 7f55c14ab700 1 -- 172.21.15.27:0/1877765854 learned_addr learned my addr 172.21.15.27:0/1877765854 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-07T02:22:29.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:29.882+0000 7f55bbfff700 1 --2- 172.21.15.27:0/1877765854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55bc1054a0 0x7f55bc19aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:22:29.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:29.882+0000 7f55c14ab700 1 -- 172.21.15.27:0/1877765854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55bc1054a0 msgr2=0x7f55bc19aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:22:29.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:29.882+0000 7f55c14ab700 1 --2- 172.21.15.27:0/1877765854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55bc1054a0 0x7f55bc19aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:22:29.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:29.882+0000 7f55c14ab700 1 -- 172.21.15.27:0/1877765854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55bc104370 msgr2=0x7f55bc19a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:22:29.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:29.882+0000 7f55c14ab700 1 --2- 172.21.15.27:0/1877765854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55bc104370 0x7f55bc19a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:22:29.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:29.882+0000 7f55c14ab700 1 -- 172.21.15.27:0/1877765854 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55bc1a24d0 con 0x7f55bc105e40 2024-05-07T02:22:29.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:29.882+0000 7f55bbfff700 1 --2- 172.21.15.27:0/1877765854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55bc1054a0 0x7f55bc19aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-07T02:22:29.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:29.883+0000 7f55c14ab700 1 --2- 172.21.15.27:0/1877765854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55bc105e40 0x7f55bc1a1e50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f55b400b970 tx=0x7f55b400bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:22:29.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:29.883+0000 7f55b9ffb700 1 -- 172.21.15.27:0/1877765854 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b4019070 con 0x7f55bc105e40 2024-05-07T02:22:29.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:29.883+0000 7f55c2f0e700 1 -- 172.21.15.27:0/1877765854 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55bc1a27c0 con 0x7f55bc105e40 2024-05-07T02:22:29.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:29.883+0000 7f55b9ffb700 1 -- 172.21.15.27:0/1877765854 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55b4014440 con 0x7f55bc105e40 2024-05-07T02:22:29.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:29.883+0000 7f55b9ffb700 1 -- 172.21.15.27:0/1877765854 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b400ce60 con 0x7f55bc105e40 2024-05-07T02:22:29.885 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:29.883+0000 7f55c2f0e700 1 -- 172.21.15.27:0/1877765854 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55bc1a2d30 con 0x7f55bc105e40 2024-05-07T02:22:29.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:29.884+0000 7f55c2f0e700 1 -- 172.21.15.27:0/1877765854 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55bc0512c0 con 0x7f55bc105e40 2024-05-07T02:22:29.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:29.886+0000 7f55b9ffb700 1 -- 172.21.15.27:0/1877765854 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f55b401c070 con 0x7f55bc105e40 2024-05-07T02:22:29.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:29.886+0000 7f55b9ffb700 1 --2- 172.21.15.27:0/1877765854 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f55ac0b2880 0x7f55ac0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:22:29.886 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:29.886+0000 7f55b9ffb700 1 -- 172.21.15.27:0/1877765854 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4965+0+0 (secure 0 0 0) 0x7f55b40e0e20 con 0x7f55bc105e40 2024-05-07T02:22:29.887 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:29.887+0000 7f55c0caa700 1 --2- 172.21.15.27:0/1877765854 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f55ac0b2880 0x7f55ac0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:22:29.888 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:29.888+0000 7f55c0caa700 1 --2- 172.21.15.27:0/1877765854 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f55ac0b2880 0x7f55ac0b4d90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f55b00098e0 tx=0x7f55b000b680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:22:29.890 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:29.890+0000 7f55b9ffb700 1 -- 172.21.15.27:0/1877765854 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f55b40a9850 con 0x7f55bc105e40 2024-05-07T02:22:30.118 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:30.118+0000 7f55c2f0e700 1 -- 172.21.15.27:0/1877765854 --> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f55bc063c00 con 0x7f55ac0b2880 2024-05-07T02:22:30.913 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:30 smithi027 ceph-mon[34398]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-07T02:22:30.913 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:30 smithi027 ceph-mon[34398]: Detected new or changed devices on smithi019 2024-05-07T02:22:30.913 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:30 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:30.913 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:30 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:30.913 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:30 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:22:30.913 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:30 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:22:30.914 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:30 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:22:30.914 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:30 smithi027 ceph-mon[34398]: Adjusting osd_memory_target on smithi019 to 5698M 2024-05-07T02:22:30.914 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:30 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:30.914 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:30 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:22:30.914 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:30 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:22:30.914 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:30 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:30.914 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:30 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-07T02:22:30.914 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:30 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-07T02:22:30.914 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:30 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:22:31.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:30 smithi064 ceph-mon[34699]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-07T02:22:31.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:30 smithi064 ceph-mon[34699]: Detected new or changed devices on smithi019 2024-05-07T02:22:31.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:30 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:31.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:30 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:31.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:30 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:22:31.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:30 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:22:31.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:30 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:22:31.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:30 smithi064 ceph-mon[34699]: Adjusting osd_memory_target on smithi019 to 5698M 2024-05-07T02:22:31.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:30 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:31.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:30 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:22:31.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:30 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:22:31.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:30 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:31.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:30 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-07T02:22:31.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:30 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-07T02:22:31.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:30 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:22:31.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:30 smithi019 ceph-mon[29781]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-07T02:22:31.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:30 smithi019 ceph-mon[29781]: Detected new or changed devices on smithi019 2024-05-07T02:22:31.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:30 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:31.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:30 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:31.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:30 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:22:31.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:30 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:22:31.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:30 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:22:31.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:30 smithi019 ceph-mon[29781]: Adjusting osd_memory_target on smithi019 to 5698M 2024-05-07T02:22:31.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:30 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:31.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:30 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:22:31.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:30 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:22:31.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:30 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:31.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:30 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-07T02:22:31.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:30 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-07T02:22:31.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:30 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:22:31.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:31 smithi027 ceph-mon[34398]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:22:32.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:31 smithi064 ceph-mon[34699]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:22:32.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:31 smithi019 ceph-mon[29781]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:22:32.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:32 smithi027 ceph-mon[34398]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-07T02:22:33.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:32 smithi064 ceph-mon[34699]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-07T02:22:33.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:32 smithi019 ceph-mon[29781]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-07T02:22:33.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:33 smithi027 ceph-mon[34398]: from='client.? 172.21.15.27:0/3598336884' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d69e34d4-2028-43b8-8ada-ff4a9b579620"}]: dispatch 2024-05-07T02:22:33.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:33 smithi027 ceph-mon[34398]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d69e34d4-2028-43b8-8ada-ff4a9b579620"}]: dispatch 2024-05-07T02:22:33.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:33 smithi027 ceph-mon[34398]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d69e34d4-2028-43b8-8ada-ff4a9b579620"}]': finished 2024-05-07T02:22:33.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:33 smithi027 ceph-mon[34398]: osdmap e44: 8 total, 7 up, 8 in 2024-05-07T02:22:33.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:33 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-07T02:22:34.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:33 smithi064 ceph-mon[34699]: from='client.? 172.21.15.27:0/3598336884' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d69e34d4-2028-43b8-8ada-ff4a9b579620"}]: dispatch 2024-05-07T02:22:34.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:33 smithi064 ceph-mon[34699]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d69e34d4-2028-43b8-8ada-ff4a9b579620"}]: dispatch 2024-05-07T02:22:34.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:33 smithi064 ceph-mon[34699]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d69e34d4-2028-43b8-8ada-ff4a9b579620"}]': finished 2024-05-07T02:22:34.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:33 smithi064 ceph-mon[34699]: osdmap e44: 8 total, 7 up, 8 in 2024-05-07T02:22:34.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:33 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-07T02:22:34.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:33 smithi019 ceph-mon[29781]: from='client.? 172.21.15.27:0/3598336884' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d69e34d4-2028-43b8-8ada-ff4a9b579620"}]: dispatch 2024-05-07T02:22:34.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:33 smithi019 ceph-mon[29781]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d69e34d4-2028-43b8-8ada-ff4a9b579620"}]: dispatch 2024-05-07T02:22:34.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:33 smithi019 ceph-mon[29781]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d69e34d4-2028-43b8-8ada-ff4a9b579620"}]': finished 2024-05-07T02:22:34.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:33 smithi019 ceph-mon[29781]: osdmap e44: 8 total, 7 up, 8 in 2024-05-07T02:22:34.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:33 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-07T02:22:34.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:34 smithi027 ceph-mon[34398]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-07T02:22:34.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:34 smithi027 ceph-mon[34398]: from='client.? 172.21.15.27:0/3992619884' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-07T02:22:35.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:34 smithi064 ceph-mon[34699]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-07T02:22:35.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:34 smithi064 ceph-mon[34699]: from='client.? 172.21.15.27:0/3992619884' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-07T02:22:35.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:34 smithi019 ceph-mon[29781]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-07T02:22:35.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:34 smithi019 ceph-mon[29781]: from='client.? 172.21.15.27:0/3992619884' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-07T02:22:36.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:36 smithi027 ceph-mon[34398]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-07T02:22:37.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:36 smithi064 ceph-mon[34699]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-07T02:22:37.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:36 smithi019 ceph-mon[29781]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-07T02:22:38.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:38 smithi027 ceph-mon[34398]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-07T02:22:39.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:38 smithi064 ceph-mon[34699]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-07T02:22:39.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:38 smithi019 ceph-mon[29781]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-07T02:22:40.894 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:40 smithi027 ceph-mon[34398]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-07T02:22:41.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:40 smithi064 ceph-mon[34699]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-07T02:22:41.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:40 smithi019 ceph-mon[29781]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-07T02:22:41.860 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:41 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-07T02:22:41.860 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:41 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:22:42.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:41 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-07T02:22:42.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:41 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:22:42.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:41 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-07T02:22:42.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:41 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:22:42.955 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:42 smithi027 ceph-mon[34398]: Deploying daemon osd.7 on smithi027 2024-05-07T02:22:42.955 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:42 smithi027 ceph-mon[34398]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-07T02:22:43.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:42 smithi064 ceph-mon[34699]: Deploying daemon osd.7 on smithi027 2024-05-07T02:22:43.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:42 smithi064 ceph-mon[34699]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-07T02:22:43.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:42 smithi019 ceph-mon[29781]: Deploying daemon osd.7 on smithi027 2024-05-07T02:22:43.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:42 smithi019 ceph-mon[29781]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-07T02:22:44.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:44 smithi027 ceph-mon[34398]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-07T02:22:45.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:44 smithi064 ceph-mon[34699]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-07T02:22:45.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:44 smithi019 ceph-mon[29781]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-07T02:22:47.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:46 smithi064 ceph-mon[34699]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-07T02:22:47.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:46 smithi019 ceph-mon[29781]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-07T02:22:47.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:46 smithi027 ceph-mon[34398]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-07T02:22:48.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:47 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:48.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:47 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:48.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:47 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:22:48.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:47 smithi027 ceph-mon[34398]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-07T02:22:48.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:47 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:48.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:47 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:48.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:47 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:22:48.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:47 smithi064 ceph-mon[34699]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-07T02:22:48.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:47 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:48.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:47 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:48.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:47 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:22:48.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:47 smithi019 ceph-mon[29781]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-07T02:22:50.606 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:50 smithi027 ceph-mon[34398]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-07T02:22:50.714 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:50.714+0000 7f55b9ffb700 1 -- 172.21.15.27:0/1877765854 <== mgr.14152 v2:172.21.15.19:6800/2059564551 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f55bc063c00 con 0x7f55ac0b2880 2024-05-07T02:22:50.714 INFO:teuthology.orchestra.run.smithi027.stdout:Created osd(s) 7 on host 'smithi027' 2024-05-07T02:22:50.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:50.716+0000 7f55c2f0e700 1 -- 172.21.15.27:0/1877765854 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f55ac0b2880 msgr2=0x7f55ac0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:22:50.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:50.716+0000 7f55c2f0e700 1 --2- 172.21.15.27:0/1877765854 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f55ac0b2880 0x7f55ac0b4d90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f55b00098e0 tx=0x7f55b000b680 comp rx=0 tx=0).stop 2024-05-07T02:22:50.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:50.716+0000 7f55c2f0e700 1 -- 172.21.15.27:0/1877765854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55bc105e40 msgr2=0x7f55bc1a1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:22:50.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:50.716+0000 7f55c2f0e700 1 --2- 172.21.15.27:0/1877765854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55bc105e40 0x7f55bc1a1e50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f55b400b970 tx=0x7f55b400bd30 comp rx=0 tx=0).stop 2024-05-07T02:22:50.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:50.716+0000 7f55c2f0e700 1 -- 172.21.15.27:0/1877765854 shutdown_connections 2024-05-07T02:22:50.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:50.716+0000 7f55c2f0e700 1 --2- 172.21.15.27:0/1877765854 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f55ac0b2880 0x7f55ac0b4d90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:22:50.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:50.716+0000 7f55c2f0e700 1 --2- 172.21.15.27:0/1877765854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55bc105e40 0x7f55bc1a1e50 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:22:50.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:50.716+0000 7f55c2f0e700 1 --2- 172.21.15.27:0/1877765854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f55bc104370 0x7f55bc19a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:22:50.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:50.716+0000 7f55c2f0e700 1 --2- 172.21.15.27:0/1877765854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55bc1054a0 0x7f55bc19aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:22:50.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:50.716+0000 7f55c2f0e700 1 -- 172.21.15.27:0/1877765854 >> 172.21.15.27:0/1877765854 conn(0x7f55bc0ff9d0 msgr2=0x7f55bc1086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:22:50.716 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:50.716+0000 7f55c2f0e700 1 -- 172.21.15.27:0/1877765854 shutdown_connections 2024-05-07T02:22:50.717 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:22:50.716+0000 7f55c2f0e700 1 -- 172.21.15.27:0/1877765854 wait complete. 2024-05-07T02:22:50.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:50 smithi064 ceph-mon[34699]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-07T02:22:50.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:50 smithi019 ceph-mon[29781]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-07T02:22:51.437 DEBUG:teuthology.orchestra.run.smithi027:osd.7> sudo journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@osd.7.service 2024-05-07T02:22:51.439 INFO:tasks.cephadm:Deploying osd.8 on smithi064 with /dev/vg_nvme/lv_2... 2024-05-07T02:22:51.439 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-07T02:22:51.698 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.c/config 2024-05-07T02:22:51.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:51 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:51.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:51 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:51.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:51 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:51.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:51 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:51.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:51 smithi027 ceph-mon[34398]: from='osd.7 [v2:172.21.15.27:6816/2756612091,v1:172.21.15.27:6817/2756612091]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-07T02:22:51.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:51 smithi027 ceph-mon[34398]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-07T02:22:51.821 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:51 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:51.821 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:51 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:51.821 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:51 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:51.822 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:51 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:51.822 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:51 smithi064 ceph-mon[34699]: from='osd.7 [v2:172.21.15.27:6816/2756612091,v1:172.21.15.27:6817/2756612091]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-07T02:22:51.822 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:51 smithi064 ceph-mon[34699]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-07T02:22:52.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:51 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:52.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:51 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:52.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:51 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:52.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:51 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:52.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:51 smithi019 ceph-mon[29781]: from='osd.7 [v2:172.21.15.27:6816/2756612091,v1:172.21.15.27:6817/2756612091]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-07T02:22:52.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:51 smithi019 ceph-mon[29781]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-07T02:22:52.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:52 smithi027 ceph-mon[34398]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-07T02:22:52.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:52 smithi027 ceph-mon[34398]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-07T02:22:52.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:52 smithi027 ceph-mon[34398]: from='osd.7 [v2:172.21.15.27:6816/2756612091,v1:172.21.15.27:6817/2756612091]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-05-07T02:22:52.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:52 smithi027 ceph-mon[34398]: osdmap e45: 8 total, 7 up, 8 in 2024-05-07T02:22:52.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:52 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-07T02:22:52.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:52 smithi027 ceph-mon[34398]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-05-07T02:22:52.957 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:22:52 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:22:52.813+0000 7fb27624c700 -1 osd.7 0 waiting for initial osdmap 2024-05-07T02:22:52.957 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:22:52 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:22:52.822+0000 7fb272001700 -1 osd.7 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-07T02:22:53.065 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:52 smithi064 ceph-mon[34699]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-07T02:22:53.065 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:52 smithi064 ceph-mon[34699]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-07T02:22:53.065 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:52 smithi064 ceph-mon[34699]: from='osd.7 [v2:172.21.15.27:6816/2756612091,v1:172.21.15.27:6817/2756612091]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-05-07T02:22:53.065 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:52 smithi064 ceph-mon[34699]: osdmap e45: 8 total, 7 up, 8 in 2024-05-07T02:22:53.065 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:52 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-07T02:22:53.066 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:52 smithi064 ceph-mon[34699]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-05-07T02:22:53.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:52 smithi019 ceph-mon[29781]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-07T02:22:53.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:52 smithi019 ceph-mon[29781]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-07T02:22:53.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:52 smithi019 ceph-mon[29781]: from='osd.7 [v2:172.21.15.27:6816/2756612091,v1:172.21.15.27:6817/2756612091]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-05-07T02:22:53.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:52 smithi019 ceph-mon[29781]: osdmap e45: 8 total, 7 up, 8 in 2024-05-07T02:22:53.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:52 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-07T02:22:53.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:52 smithi019 ceph-mon[29781]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-05-07T02:22:53.811 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-07T02:22:53.811 INFO:teuthology.orchestra.run.smithi064.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-07T02:22:53.811 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10+0 records in 2024-05-07T02:22:53.812 INFO:teuthology.orchestra.run.smithi064.stderr:10+0 records out 2024-05-07T02:22:53.812 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129041 s, 813 MB/s 2024-05-07T02:22:53.812 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping successful for: 2024-05-07T02:22:54.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:53 smithi064 ceph-mon[34699]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-05-07T02:22:54.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:53 smithi064 ceph-mon[34699]: osdmap e46: 8 total, 7 up, 8 in 2024-05-07T02:22:54.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:53 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-07T02:22:54.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:53 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-07T02:22:54.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:53 smithi064 ceph-mon[34699]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-07T02:22:54.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:53 smithi019 ceph-mon[29781]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-05-07T02:22:54.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:53 smithi019 ceph-mon[29781]: osdmap e46: 8 total, 7 up, 8 in 2024-05-07T02:22:54.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:53 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-07T02:22:54.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:53 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-07T02:22:54.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:53 smithi019 ceph-mon[29781]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-07T02:22:54.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:53 smithi027 ceph-mon[34398]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-05-07T02:22:54.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:53 smithi027 ceph-mon[34398]: osdmap e46: 8 total, 7 up, 8 in 2024-05-07T02:22:54.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:53 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-07T02:22:54.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:53 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-07T02:22:54.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:53 smithi027 ceph-mon[34398]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-07T02:22:54.555 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi064:vg_nvme/lv_2 2024-05-07T02:22:54.798 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.c/config 2024-05-07T02:22:54.921 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:54 smithi027 ceph-mon[34398]: purged_snaps scrub starts 2024-05-07T02:22:54.921 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:54 smithi027 ceph-mon[34398]: purged_snaps scrub ok 2024-05-07T02:22:54.922 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:54 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-07T02:22:54.922 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:54 smithi027 ceph-mon[34398]: osd.7 [v2:172.21.15.27:6816/2756612091,v1:172.21.15.27:6817/2756612091] boot 2024-05-07T02:22:54.922 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:54 smithi027 ceph-mon[34398]: osdmap e47: 8 total, 8 up, 8 in 2024-05-07T02:22:54.922 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:54 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-07T02:22:55.088 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:54 smithi064 ceph-mon[34699]: purged_snaps scrub starts 2024-05-07T02:22:55.088 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:54 smithi064 ceph-mon[34699]: purged_snaps scrub ok 2024-05-07T02:22:55.088 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:54 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-07T02:22:55.088 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:54 smithi064 ceph-mon[34699]: osd.7 [v2:172.21.15.27:6816/2756612091,v1:172.21.15.27:6817/2756612091] boot 2024-05-07T02:22:55.088 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:54 smithi064 ceph-mon[34699]: osdmap e47: 8 total, 8 up, 8 in 2024-05-07T02:22:55.088 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:54 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-07T02:22:55.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:54 smithi019 ceph-mon[29781]: purged_snaps scrub starts 2024-05-07T02:22:55.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:54 smithi019 ceph-mon[29781]: purged_snaps scrub ok 2024-05-07T02:22:55.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:54 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-07T02:22:55.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:54 smithi019 ceph-mon[29781]: osd.7 [v2:172.21.15.27:6816/2756612091,v1:172.21.15.27:6817/2756612091] boot 2024-05-07T02:22:55.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:54 smithi019 ceph-mon[29781]: osdmap e47: 8 total, 8 up, 8 in 2024-05-07T02:22:55.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:54 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-07T02:22:55.976 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:55 smithi064 ceph-mon[34699]: osdmap e48: 8 total, 8 up, 8 in 2024-05-07T02:22:55.976 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:55 smithi064 ceph-mon[34699]: Detected new or changed devices on smithi027 2024-05-07T02:22:55.976 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:55 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:55.976 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:55 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:55.976 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:55 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:22:55.977 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:55 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:22:55.977 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:55 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:22:55.977 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:55 smithi064 ceph-mon[34699]: Adjusting osd_memory_target on smithi027 to 5696M 2024-05-07T02:22:55.977 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:55 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:55.977 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:55 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:22:55.977 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:55 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:22:55.977 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:55 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:55.977 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:55 smithi064 ceph-mon[34699]: pgmap v200: 1 pgs: 1 remapped; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-07T02:22:55.977 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:55 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:22:56.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:55 smithi019 ceph-mon[29781]: osdmap e48: 8 total, 8 up, 8 in 2024-05-07T02:22:56.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:55 smithi019 ceph-mon[29781]: Detected new or changed devices on smithi027 2024-05-07T02:22:56.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:55 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:56.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:55 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:56.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:55 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:22:56.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:55 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:22:56.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:55 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:22:56.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:55 smithi019 ceph-mon[29781]: Adjusting osd_memory_target on smithi027 to 5696M 2024-05-07T02:22:56.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:55 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:56.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:55 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:22:56.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:55 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:22:56.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:55 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:56.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:55 smithi019 ceph-mon[29781]: pgmap v200: 1 pgs: 1 remapped; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-07T02:22:56.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:55 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:22:56.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:55 smithi027 ceph-mon[34398]: osdmap e48: 8 total, 8 up, 8 in 2024-05-07T02:22:56.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:55 smithi027 ceph-mon[34398]: Detected new or changed devices on smithi027 2024-05-07T02:22:56.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:55 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:56.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:55 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:56.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:55 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:22:56.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:55 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:22:56.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:55 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:22:56.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:55 smithi027 ceph-mon[34398]: Adjusting osd_memory_target on smithi027 to 5696M 2024-05-07T02:22:56.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:55 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:56.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:55 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:22:56.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:55 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:22:56.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:55 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:56.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:55 smithi027 ceph-mon[34398]: pgmap v200: 1 pgs: 1 remapped; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-07T02:22:56.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:55 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:22:56.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:22:56.666+0000 7f38497d7700 1 -- 172.21.15.64:0/1144151431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3844107820 msgr2=0x7f3844109c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:22:56.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:22:56.666+0000 7f38497d7700 1 --2- 172.21.15.64:0/1144151431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3844107820 0x7f3844109c40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3838009990 tx=0x7f3838009ca0 comp rx=0 tx=0).stop 2024-05-07T02:22:56.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:22:56.666+0000 7f38497d7700 1 -- 172.21.15.64:0/1144151431 shutdown_connections 2024-05-07T02:22:56.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:22:56.666+0000 7f38497d7700 1 --2- 172.21.15.64:0/1144151431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f384410a180 0x7f384410c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:22:56.668 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:22:56.666+0000 7f38497d7700 1 --2- 172.21.15.64:0/1144151431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38441026a0 0x7f3844102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:22:56.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:22:56.666+0000 7f38497d7700 1 --2- 172.21.15.64:0/1144151431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3844107820 0x7f3844109c40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:22:56.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:22:56.666+0000 7f38497d7700 1 -- 172.21.15.64:0/1144151431 >> 172.21.15.64:0/1144151431 conn(0x7f38440fe020 msgr2=0x7f38441004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:22:56.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:22:56.666+0000 7f38497d7700 1 -- 172.21.15.64:0/1144151431 shutdown_connections 2024-05-07T02:22:56.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:22:56.666+0000 7f38497d7700 1 -- 172.21.15.64:0/1144151431 wait complete. 2024-05-07T02:22:56.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:22:56.666+0000 7f38497d7700 1 Processor -- start 2024-05-07T02:22:56.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:22:56.667+0000 7f38497d7700 1 -- start start 2024-05-07T02:22:56.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:22:56.667+0000 7f38497d7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38441026a0 0x7f384419a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:22:56.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:22:56.667+0000 7f38497d7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3844107820 0x7f384419aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:22:56.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:22:56.667+0000 7f38497d7700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f384410a180 0x7f38441a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:22:56.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:22:56.667+0000 7f38497d7700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f384410fe30 con 0x7f384410a180 2024-05-07T02:22:56.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:22:56.667+0000 7f38497d7700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f384410fcb0 con 0x7f3844107820 2024-05-07T02:22:56.669 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:22:56.667+0000 7f38497d7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f384410ffb0 con 0x7f38441026a0 2024-05-07T02:22:56.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:22:56.667+0000 7f3842ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38441026a0 0x7f384419a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:22:56.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:22:56.667+0000 7f3842ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38441026a0 0x7f384419a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:43118/0 (socket says 172.21.15.64:43118) 2024-05-07T02:22:56.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:22:56.667+0000 7f3842ffd700 1 -- 172.21.15.64:0/272797845 learned_addr learned my addr 172.21.15.64:0/272797845 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-07T02:22:56.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:22:56.667+0000 7f38427fc700 1 --2- 172.21.15.64:0/272797845 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3844107820 0x7f384419aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:22:56.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:22:56.667+0000 7f38437fe700 1 --2- 172.21.15.64:0/272797845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f384410a180 0x7f38441a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:22:56.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:22:56.667+0000 7f3842ffd700 1 -- 172.21.15.64:0/272797845 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3844107820 msgr2=0x7f384419aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:22:56.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:22:56.667+0000 7f3842ffd700 1 --2- 172.21.15.64:0/272797845 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3844107820 0x7f384419aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:22:56.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:22:56.667+0000 7f3842ffd700 1 -- 172.21.15.64:0/272797845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f384410a180 msgr2=0x7f38441a1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:22:56.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:22:56.667+0000 7f3842ffd700 1 --2- 172.21.15.64:0/272797845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f384410a180 0x7f38441a1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:22:56.670 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:22:56.667+0000 7f3842ffd700 1 -- 172.21.15.64:0/272797845 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38441a2480 con 0x7f38441026a0 2024-05-07T02:22:56.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:22:56.667+0000 7f38427fc700 1 --2- 172.21.15.64:0/272797845 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3844107820 0x7f384419aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:22:56.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:22:56.668+0000 7f3842ffd700 1 --2- 172.21.15.64:0/272797845 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38441026a0 0x7f384419a970 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f383400e970 tx=0x7f383400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:22:56.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:22:56.668+0000 7f382bfff700 1 -- 172.21.15.64:0/272797845 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383400cbd0 con 0x7f38441026a0 2024-05-07T02:22:56.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:22:56.668+0000 7f382bfff700 1 -- 172.21.15.64:0/272797845 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f383400cd70 con 0x7f38441026a0 2024-05-07T02:22:56.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:22:56.668+0000 7f38497d7700 1 -- 172.21.15.64:0/272797845 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38441a2770 con 0x7f38441026a0 2024-05-07T02:22:56.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:22:56.668+0000 7f382bfff700 1 -- 172.21.15.64:0/272797845 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3834005680 con 0x7f38441026a0 2024-05-07T02:22:56.671 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:22:56.668+0000 7f38497d7700 1 -- 172.21.15.64:0/272797845 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38441a2d30 con 0x7f38441026a0 2024-05-07T02:22:56.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:22:56.669+0000 7f38497d7700 1 -- 172.21.15.64:0/272797845 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38440512c0 con 0x7f38441026a0 2024-05-07T02:22:56.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:22:56.671+0000 7f382bfff700 1 -- 172.21.15.64:0/272797845 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f3834004750 con 0x7f38441026a0 2024-05-07T02:22:56.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:22:56.672+0000 7f382bfff700 1 --2- 172.21.15.64:0/272797845 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f382c0b2880 0x7f382c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:22:56.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:22:56.672+0000 7f382bfff700 1 -- 172.21.15.64:0/272797845 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5397+0+0 (secure 0 0 0) 0x7f38340e1b20 con 0x7f38441026a0 2024-05-07T02:22:56.672 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:22:56.672+0000 7f38427fc700 1 --2- 172.21.15.64:0/272797845 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f382c0b2880 0x7f382c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:22:56.673 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:22:56.673+0000 7f38427fc700 1 --2- 172.21.15.64:0/272797845 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f382c0b2880 0x7f382c0b4d90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f384419b9d0 tx=0x7f383800b720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:22:56.675 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:22:56.676+0000 7f382bfff700 1 -- 172.21.15.64:0/272797845 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3834054df0 con 0x7f38441026a0 2024-05-07T02:22:56.901 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:22:56.901+0000 7f38497d7700 1 -- 172.21.15.64:0/272797845 --> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f3844063c00 con 0x7f382c0b2880 2024-05-07T02:22:57.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:56 smithi064 ceph-mon[34699]: osdmap e49: 8 total, 8 up, 8 in 2024-05-07T02:22:57.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:56 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:22:57.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:56 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:22:57.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:56 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:57.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:56 smithi064 ceph-mon[34699]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-05-07T02:22:57.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:56 smithi019 ceph-mon[29781]: osdmap e49: 8 total, 8 up, 8 in 2024-05-07T02:22:57.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:56 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:22:57.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:56 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:22:57.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:56 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:57.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:56 smithi019 ceph-mon[29781]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-05-07T02:22:57.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:56 smithi027 ceph-mon[34398]: osdmap e49: 8 total, 8 up, 8 in 2024-05-07T02:22:57.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:56 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:22:57.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:56 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:22:57.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:56 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:22:57.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:56 smithi027 ceph-mon[34398]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-05-07T02:22:58.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:57 smithi064 ceph-mon[34699]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:22:58.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:57 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-07T02:22:58.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:57 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-07T02:22:58.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:57 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:22:58.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:22:57 smithi064 ceph-mon[34699]: pgmap v202: 1 pgs: 1 remapped; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-07T02:22:58.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:57 smithi019 ceph-mon[29781]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:22:58.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:57 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-07T02:22:58.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:57 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-07T02:22:58.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:57 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:22:58.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:22:57 smithi019 ceph-mon[29781]: pgmap v202: 1 pgs: 1 remapped; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-07T02:22:58.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:57 smithi027 ceph-mon[34398]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:22:58.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:57 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-07T02:22:58.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:57 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-07T02:22:58.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:57 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:22:58.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:22:57 smithi027 ceph-mon[34398]: pgmap v202: 1 pgs: 1 remapped; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-07T02:23:00.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:00 smithi064 ceph-mon[34699]: pgmap v203: 1 pgs: 1 remapped; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-07T02:23:00.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:00 smithi064 ceph-mon[34699]: from='client.? 172.21.15.64:0/3527104345' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c1a4179-75b6-42ff-ac29-378801e6f8f0"}]: dispatch 2024-05-07T02:23:00.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:00 smithi064 ceph-mon[34699]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c1a4179-75b6-42ff-ac29-378801e6f8f0"}]: dispatch 2024-05-07T02:23:00.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:00 smithi064 ceph-mon[34699]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6c1a4179-75b6-42ff-ac29-378801e6f8f0"}]': finished 2024-05-07T02:23:00.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:00 smithi064 ceph-mon[34699]: osdmap e50: 9 total, 8 up, 9 in 2024-05-07T02:23:00.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:00 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-07T02:23:00.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:00 smithi027 ceph-mon[34398]: pgmap v203: 1 pgs: 1 remapped; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-07T02:23:00.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:00 smithi027 ceph-mon[34398]: from='client.? 172.21.15.64:0/3527104345' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c1a4179-75b6-42ff-ac29-378801e6f8f0"}]: dispatch 2024-05-07T02:23:00.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:00 smithi027 ceph-mon[34398]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c1a4179-75b6-42ff-ac29-378801e6f8f0"}]: dispatch 2024-05-07T02:23:00.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:00 smithi027 ceph-mon[34398]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6c1a4179-75b6-42ff-ac29-378801e6f8f0"}]': finished 2024-05-07T02:23:00.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:00 smithi027 ceph-mon[34398]: osdmap e50: 9 total, 8 up, 9 in 2024-05-07T02:23:00.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:00 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-07T02:23:00.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:00 smithi019 ceph-mon[29781]: pgmap v203: 1 pgs: 1 remapped; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-05-07T02:23:00.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:00 smithi019 ceph-mon[29781]: from='client.? 172.21.15.64:0/3527104345' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c1a4179-75b6-42ff-ac29-378801e6f8f0"}]: dispatch 2024-05-07T02:23:00.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:00 smithi019 ceph-mon[29781]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6c1a4179-75b6-42ff-ac29-378801e6f8f0"}]: dispatch 2024-05-07T02:23:00.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:00 smithi019 ceph-mon[29781]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6c1a4179-75b6-42ff-ac29-378801e6f8f0"}]': finished 2024-05-07T02:23:00.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:00 smithi019 ceph-mon[29781]: osdmap e50: 9 total, 8 up, 9 in 2024-05-07T02:23:00.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:00 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-07T02:23:01.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:01 smithi027 ceph-mon[34398]: from='client.? 172.21.15.64:0/3906873172' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-07T02:23:01.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:01 smithi064 ceph-mon[34699]: from='client.? 172.21.15.64:0/3906873172' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-07T02:23:01.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:01 smithi019 ceph-mon[29781]: from='client.? 172.21.15.64:0/3906873172' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-07T02:23:02.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:02 smithi027 ceph-mon[34398]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 176 KiB/s, 0 objects/s recovering 2024-05-07T02:23:02.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:02 smithi027 ceph-mon[34398]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-05-07T02:23:02.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:02 smithi027 ceph-mon[34398]: Cluster is now healthy 2024-05-07T02:23:02.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:02 smithi064 ceph-mon[34699]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 176 KiB/s, 0 objects/s recovering 2024-05-07T02:23:02.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:02 smithi064 ceph-mon[34699]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-05-07T02:23:02.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:02 smithi064 ceph-mon[34699]: Cluster is now healthy 2024-05-07T02:23:02.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:02 smithi019 ceph-mon[29781]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 176 KiB/s, 0 objects/s recovering 2024-05-07T02:23:02.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:02 smithi019 ceph-mon[29781]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-05-07T02:23:02.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:02 smithi019 ceph-mon[29781]: Cluster is now healthy 2024-05-07T02:23:04.633 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:04 smithi064 ceph-mon[34699]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-07T02:23:04.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:04 smithi027 ceph-mon[34398]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-07T02:23:04.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:04 smithi019 ceph-mon[29781]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-07T02:23:06.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:06 smithi019 ceph-mon[29781]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-05-07T02:23:06.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:06 smithi027 ceph-mon[34398]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-05-07T02:23:06.843 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:06 smithi064 ceph-mon[34699]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-05-07T02:23:08.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:08 smithi027 ceph-mon[34398]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-07T02:23:08.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:08 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-07T02:23:08.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:08 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:23:08.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:08 smithi027 ceph-mon[34398]: Deploying daemon osd.8 on smithi064 2024-05-07T02:23:08.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:08 smithi064 ceph-mon[34699]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-07T02:23:08.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:08 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-07T02:23:08.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:08 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:23:08.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:08 smithi064 ceph-mon[34699]: Deploying daemon osd.8 on smithi064 2024-05-07T02:23:08.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:08 smithi019 ceph-mon[29781]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-07T02:23:08.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:08 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-07T02:23:08.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:08 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:23:08.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:08 smithi019 ceph-mon[29781]: Deploying daemon osd.8 on smithi064 2024-05-07T02:23:10.661 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:10 smithi064 ceph-mon[34699]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-07T02:23:10.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:10 smithi027 ceph-mon[34398]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-07T02:23:10.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:10 smithi019 ceph-mon[29781]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-07T02:23:12.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:12 smithi064 ceph-mon[34699]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-07T02:23:12.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:12 smithi019 ceph-mon[29781]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-07T02:23:12.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:12 smithi027 ceph-mon[34398]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-07T02:23:14.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:14 smithi027 ceph-mon[34398]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-07T02:23:14.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:14 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:14.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:14 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:14.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:14 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:23:14.977 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:14 smithi064 ceph-mon[34699]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-07T02:23:14.977 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:14 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:14.977 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:14 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:14.977 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:14 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:23:15.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:14 smithi019 ceph-mon[29781]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-07T02:23:15.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:14 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:15.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:14 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:15.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:14 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:23:16.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:16 smithi064 ceph-mon[34699]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-07T02:23:16.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:16 smithi027 ceph-mon[34398]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-07T02:23:17.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:16 smithi019 ceph-mon[29781]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-07T02:23:17.894 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 8 on host 'smithi064' 2024-05-07T02:23:17.894 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:23:17.894+0000 7f382bfff700 1 -- 172.21.15.64:0/272797845 <== mgr.14152 v2:172.21.15.19:6800/2059564551 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3844063c00 con 0x7f382c0b2880 2024-05-07T02:23:17.896 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:23:17.896+0000 7f38497d7700 1 -- 172.21.15.64:0/272797845 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f382c0b2880 msgr2=0x7f382c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:23:17.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:23:17.896+0000 7f38497d7700 1 --2- 172.21.15.64:0/272797845 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f382c0b2880 0x7f382c0b4d90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f384419b9d0 tx=0x7f383800b720 comp rx=0 tx=0).stop 2024-05-07T02:23:17.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:23:17.896+0000 7f38497d7700 1 -- 172.21.15.64:0/272797845 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38441026a0 msgr2=0x7f384419a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:23:17.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:23:17.896+0000 7f38497d7700 1 --2- 172.21.15.64:0/272797845 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38441026a0 0x7f384419a970 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f383400e970 tx=0x7f383400ec80 comp rx=0 tx=0).stop 2024-05-07T02:23:17.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:23:17.896+0000 7f38497d7700 1 -- 172.21.15.64:0/272797845 shutdown_connections 2024-05-07T02:23:17.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:23:17.896+0000 7f38497d7700 1 --2- 172.21.15.64:0/272797845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f384410a180 0x7f38441a1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:23:17.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:23:17.896+0000 7f38497d7700 1 --2- 172.21.15.64:0/272797845 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38441026a0 0x7f384419a970 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:23:17.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:23:17.896+0000 7f38497d7700 1 --2- 172.21.15.64:0/272797845 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f382c0b2880 0x7f382c0b4d90 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:23:17.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:23:17.896+0000 7f38497d7700 1 --2- 172.21.15.64:0/272797845 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3844107820 0x7f384419aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:23:17.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:23:17.896+0000 7f38497d7700 1 -- 172.21.15.64:0/272797845 >> 172.21.15.64:0/272797845 conn(0x7f38440fe020 msgr2=0x7f3844108720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:23:17.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:23:17.897+0000 7f38497d7700 1 -- 172.21.15.64:0/272797845 shutdown_connections 2024-05-07T02:23:17.897 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:23:17.897+0000 7f38497d7700 1 -- 172.21.15.64:0/272797845 wait complete. 2024-05-07T02:23:18.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:17 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:18.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:17 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:18.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:17 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:23:18.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:17 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:23:18.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:17 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:18.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:17 smithi064 ceph-mon[34699]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-07T02:23:18.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:17 smithi064 ceph-mon[34699]: from='osd.8 [v2:172.21.15.64:6816/1987024836,v1:172.21.15.64:6817/1987024836]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-07T02:23:18.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:17 smithi064 ceph-mon[34699]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-07T02:23:18.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:17 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:18.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:17 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:18.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:17 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:23:18.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:17 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:23:18.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:17 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:18.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:17 smithi019 ceph-mon[29781]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-07T02:23:18.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:17 smithi019 ceph-mon[29781]: from='osd.8 [v2:172.21.15.64:6816/1987024836,v1:172.21.15.64:6817/1987024836]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-07T02:23:18.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:17 smithi019 ceph-mon[29781]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-07T02:23:18.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:17 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:18.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:17 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:18.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:17 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:23:18.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:17 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:23:18.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:17 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:18.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:17 smithi027 ceph-mon[34398]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-07T02:23:18.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:17 smithi027 ceph-mon[34398]: from='osd.8 [v2:172.21.15.64:6816/1987024836,v1:172.21.15.64:6817/1987024836]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-07T02:23:18.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:17 smithi027 ceph-mon[34398]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-07T02:23:18.717 DEBUG:teuthology.orchestra.run.smithi064:osd.8> sudo journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@osd.8.service 2024-05-07T02:23:18.721 INFO:tasks.cephadm:Deploying osd.9 on smithi019 with /dev/vg_nvme/lv_1... 2024-05-07T02:23:18.722 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-07T02:23:18.961 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:18 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:18.961 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:18 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:18.961 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:18 smithi064 ceph-mon[34699]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-07T02:23:18.961 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:18 smithi064 ceph-mon[34699]: from='osd.8 [v2:172.21.15.64:6816/1987024836,v1:172.21.15.64:6817/1987024836]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-07T02:23:18.961 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:18 smithi064 ceph-mon[34699]: osdmap e51: 9 total, 8 up, 9 in 2024-05-07T02:23:18.961 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:18 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-07T02:23:18.962 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:18 smithi064 ceph-mon[34699]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-07T02:23:18.962 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:18 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:23:19.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:18 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:19.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:18 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:19.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:18 smithi019 ceph-mon[29781]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-07T02:23:19.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:18 smithi019 ceph-mon[29781]: from='osd.8 [v2:172.21.15.64:6816/1987024836,v1:172.21.15.64:6817/1987024836]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-07T02:23:19.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:18 smithi019 ceph-mon[29781]: osdmap e51: 9 total, 8 up, 9 in 2024-05-07T02:23:19.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:18 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-07T02:23:19.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:18 smithi019 ceph-mon[29781]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-07T02:23:19.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:18 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:23:19.204 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:23:19.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:18 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:19.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:18 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:19.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:18 smithi027 ceph-mon[34398]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-07T02:23:19.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:18 smithi027 ceph-mon[34398]: from='osd.8 [v2:172.21.15.64:6816/1987024836,v1:172.21.15.64:6817/1987024836]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-07T02:23:19.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:18 smithi027 ceph-mon[34398]: osdmap e51: 9 total, 8 up, 9 in 2024-05-07T02:23:19.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:18 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-07T02:23:19.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:18 smithi027 ceph-mon[34398]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-07T02:23:19.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:18 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:23:19.873 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:23:19 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:23:19.618+0000 7f27601dc700 -1 osd.8 0 waiting for initial osdmap 2024-05-07T02:23:19.873 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:23:19 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:23:19.632+0000 7f2757f89700 -1 osd.8 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-07T02:23:19.948 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:19 smithi019 ceph-mon[29781]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-07T02:23:19.948 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:19 smithi019 ceph-mon[29781]: osdmap e52: 9 total, 8 up, 9 in 2024-05-07T02:23:19.948 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:19 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-07T02:23:19.948 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:19 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-07T02:23:19.948 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:19 smithi019 ceph-mon[29781]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-07T02:23:19.948 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:19 smithi019 ceph-mon[29781]: from='osd.8 ' entity='osd.8' 2024-05-07T02:23:20.159 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:19 smithi064 ceph-mon[34699]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-07T02:23:20.159 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:19 smithi064 ceph-mon[34699]: osdmap e52: 9 total, 8 up, 9 in 2024-05-07T02:23:20.159 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:19 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-07T02:23:20.159 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:19 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-07T02:23:20.159 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:19 smithi064 ceph-mon[34699]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-07T02:23:20.159 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:19 smithi064 ceph-mon[34699]: from='osd.8 ' entity='osd.8' 2024-05-07T02:23:20.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:19 smithi027 ceph-mon[34398]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-07T02:23:20.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:19 smithi027 ceph-mon[34398]: osdmap e52: 9 total, 8 up, 9 in 2024-05-07T02:23:20.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:19 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-07T02:23:20.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:19 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-07T02:23:20.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:19 smithi027 ceph-mon[34398]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-07T02:23:20.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:19 smithi027 ceph-mon[34398]: from='osd.8 ' entity='osd.8' 2024-05-07T02:23:21.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:20 smithi064 ceph-mon[34699]: purged_snaps scrub starts 2024-05-07T02:23:21.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:20 smithi064 ceph-mon[34699]: purged_snaps scrub ok 2024-05-07T02:23:21.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:20 smithi064 ceph-mon[34699]: osd.8 [v2:172.21.15.64:6816/1987024836,v1:172.21.15.64:6817/1987024836] boot 2024-05-07T02:23:21.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:20 smithi064 ceph-mon[34699]: osdmap e53: 9 total, 9 up, 9 in 2024-05-07T02:23:21.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:20 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-07T02:23:21.134 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:20 smithi019 ceph-mon[29781]: purged_snaps scrub starts 2024-05-07T02:23:21.134 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:20 smithi019 ceph-mon[29781]: purged_snaps scrub ok 2024-05-07T02:23:21.134 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:20 smithi019 ceph-mon[29781]: osd.8 [v2:172.21.15.64:6816/1987024836,v1:172.21.15.64:6817/1987024836] boot 2024-05-07T02:23:21.134 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:20 smithi019 ceph-mon[29781]: osdmap e53: 9 total, 9 up, 9 in 2024-05-07T02:23:21.134 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:20 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-07T02:23:21.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:20 smithi027 ceph-mon[34398]: purged_snaps scrub starts 2024-05-07T02:23:21.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:20 smithi027 ceph-mon[34398]: purged_snaps scrub ok 2024-05-07T02:23:21.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:20 smithi027 ceph-mon[34398]: osd.8 [v2:172.21.15.64:6816/1987024836,v1:172.21.15.64:6817/1987024836] boot 2024-05-07T02:23:21.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:20 smithi027 ceph-mon[34398]: osdmap e53: 9 total, 9 up, 9 in 2024-05-07T02:23:21.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:20 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-07T02:23:22.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:21 smithi064 ceph-mon[34699]: osdmap e54: 9 total, 9 up, 9 in 2024-05-07T02:23:22.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:21 smithi064 ceph-mon[34699]: pgmap v219: 1 pgs: 1 peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-07T02:23:22.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:21 smithi019 ceph-mon[29781]: osdmap e54: 9 total, 9 up, 9 in 2024-05-07T02:23:22.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:21 smithi019 ceph-mon[29781]: pgmap v219: 1 pgs: 1 peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-07T02:23:22.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:21 smithi027 ceph-mon[34398]: osdmap e54: 9 total, 9 up, 9 in 2024-05-07T02:23:22.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:21 smithi027 ceph-mon[34398]: pgmap v219: 1 pgs: 1 peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-07T02:23:22.500 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-07T02:23:22.500 INFO:teuthology.orchestra.run.smithi019.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-07T02:23:22.500 INFO:teuthology.orchestra.run.smithi019.stderr: stderr: 10+0 records in 2024-05-07T02:23:22.500 INFO:teuthology.orchestra.run.smithi019.stderr:10+0 records out 2024-05-07T02:23:22.500 INFO:teuthology.orchestra.run.smithi019.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0329271 s, 318 MB/s 2024-05-07T02:23:22.500 INFO:teuthology.orchestra.run.smithi019.stderr:--> Zapping successful for: 2024-05-07T02:23:23.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:22 smithi027 ceph-mon[34398]: osdmap e55: 9 total, 9 up, 9 in 2024-05-07T02:23:23.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:22 smithi027 ceph-mon[34398]: Detected new or changed devices on smithi064 2024-05-07T02:23:23.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:22 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:23.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:22 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:23.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:22 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:23:23.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:22 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:23:23.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:22 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:23:23.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:22 smithi027 ceph-mon[34398]: Adjusting osd_memory_target on smithi064 to 5696M 2024-05-07T02:23:23.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:22 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:23.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:22 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:23:23.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:22 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:23:23.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:22 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:23.226 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:22 smithi019 ceph-mon[29781]: osdmap e55: 9 total, 9 up, 9 in 2024-05-07T02:23:23.227 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:22 smithi019 ceph-mon[29781]: Detected new or changed devices on smithi064 2024-05-07T02:23:23.227 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:22 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:23.227 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:22 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:23.227 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:22 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:23:23.227 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:22 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:23:23.227 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:22 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:23:23.227 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:22 smithi019 ceph-mon[29781]: Adjusting osd_memory_target on smithi064 to 5696M 2024-05-07T02:23:23.227 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:22 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:23.227 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:22 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:23:23.227 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:22 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:23:23.227 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:22 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:23.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:22 smithi064 ceph-mon[34699]: osdmap e55: 9 total, 9 up, 9 in 2024-05-07T02:23:23.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:22 smithi064 ceph-mon[34699]: Detected new or changed devices on smithi064 2024-05-07T02:23:23.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:22 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:23.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:22 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:23.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:22 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:23:23.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:22 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:23:23.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:22 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:23:23.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:22 smithi064 ceph-mon[34699]: Adjusting osd_memory_target on smithi064 to 5696M 2024-05-07T02:23:23.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:22 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:23.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:22 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:23:23.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:22 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:23:23.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:22 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:23.413 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi019:vg_nvme/lv_1 2024-05-07T02:23:23.841 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:23:24.155 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:23 smithi019 ceph-mon[29781]: pgmap v221: 1 pgs: 1 peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-07T02:23:24.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:23 smithi027 ceph-mon[34398]: pgmap v221: 1 pgs: 1 peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-07T02:23:24.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:23 smithi064 ceph-mon[34699]: pgmap v221: 1 pgs: 1 peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-07T02:23:26.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:26.221+0000 7f0b5c800700 1 -- 172.21.15.19:0/44392689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b54103f60 msgr2=0x7f0b54104370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:23:26.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:26.221+0000 7f0b5c800700 1 --2- 172.21.15.19:0/44392689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b54103f60 0x7f0b54104370 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0b44009990 tx=0x7f0b44009ca0 comp rx=0 tx=0).stop 2024-05-07T02:23:26.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:26.222+0000 7f0b5c800700 1 -- 172.21.15.19:0/44392689 shutdown_connections 2024-05-07T02:23:26.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:26.222+0000 7f0b5c800700 1 --2- 172.21.15.19:0/44392689 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b541049e0 0x7f0b5410c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:23:26.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:26.222+0000 7f0b5c800700 1 --2- 172.21.15.19:0/44392689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b541059b0 0x7f0b54105dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:23:26.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:26.222+0000 7f0b5c800700 1 --2- 172.21.15.19:0/44392689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b54103f60 0x7f0b54104370 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:23:26.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:26.222+0000 7f0b5c800700 1 -- 172.21.15.19:0/44392689 >> 172.21.15.19:0/44392689 conn(0x7f0b540ff9d0 msgr2=0x7f0b54101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:23:26.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:26.222+0000 7f0b5c800700 1 -- 172.21.15.19:0/44392689 shutdown_connections 2024-05-07T02:23:26.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:26.222+0000 7f0b5c800700 1 -- 172.21.15.19:0/44392689 wait complete. 2024-05-07T02:23:26.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:26.223+0000 7f0b5c800700 1 Processor -- start 2024-05-07T02:23:26.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:26.223+0000 7f0b5c800700 1 -- start start 2024-05-07T02:23:26.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:26.224+0000 7f0b5c800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b54103f60 0x7f0b54078570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:23:26.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:26.224+0000 7f0b5c800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b541049e0 0x7f0b54076c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:23:26.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:26.224+0000 7f0b5c800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b541059b0 0x7f0b54077160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:23:26.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:26.224+0000 7f0b5c800700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0b5410ff40 con 0x7f0b541059b0 2024-05-07T02:23:26.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:26.224+0000 7f0b5c800700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0b5410fdc0 con 0x7f0b541049e0 2024-05-07T02:23:26.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:26.224+0000 7f0b5c800700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0b541100c0 con 0x7f0b54103f60 2024-05-07T02:23:26.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:26.224+0000 7f0b59e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b54103f60 0x7f0b54078570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:23:26.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:26.224+0000 7f0b59e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b54103f60 0x7f0b54078570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:35904/0 (socket says 172.21.15.19:35904) 2024-05-07T02:23:26.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:26.224+0000 7f0b59e00700 1 -- 172.21.15.19:0/2736970086 learned_addr learned my addr 172.21.15.19:0/2736970086 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:23:26.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:26.224+0000 7f0b5a800700 1 --2- 172.21.15.19:0/2736970086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b541059b0 0x7f0b54077160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:23:26.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:26.224+0000 7f0b59400700 1 --2- 172.21.15.19:0/2736970086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b541049e0 0x7f0b54076c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:23:26.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:26.225+0000 7f0b59e00700 1 -- 172.21.15.19:0/2736970086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b541049e0 msgr2=0x7f0b54076c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:23:26.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:26.225+0000 7f0b59e00700 1 --2- 172.21.15.19:0/2736970086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b541049e0 0x7f0b54076c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:23:26.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:26.225+0000 7f0b59e00700 1 -- 172.21.15.19:0/2736970086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b541059b0 msgr2=0x7f0b54077160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:23:26.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:26.225+0000 7f0b59e00700 1 --2- 172.21.15.19:0/2736970086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b541059b0 0x7f0b54077160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:23:26.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:26.225+0000 7f0b59e00700 1 -- 172.21.15.19:0/2736970086 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b540778c0 con 0x7f0b54103f60 2024-05-07T02:23:26.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:26.225+0000 7f0b5a800700 1 --2- 172.21.15.19:0/2736970086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b541059b0 0x7f0b54077160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:23:26.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:26.225+0000 7f0b59e00700 1 --2- 172.21.15.19:0/2736970086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b54103f60 0x7f0b54078570 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0b4800e940 tx=0x7f0b4800ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:23:26.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:26.225+0000 7f0b4ea00700 1 -- 172.21.15.19:0/2736970086 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b48004220 con 0x7f0b54103f60 2024-05-07T02:23:26.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:26.225+0000 7f0b4ea00700 1 -- 172.21.15.19:0/2736970086 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b480043c0 con 0x7f0b54103f60 2024-05-07T02:23:26.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:26.225+0000 7f0b4ea00700 1 -- 172.21.15.19:0/2736970086 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b48003c60 con 0x7f0b54103f60 2024-05-07T02:23:26.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:26.225+0000 7f0b5c800700 1 -- 172.21.15.19:0/2736970086 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b541a4b20 con 0x7f0b54103f60 2024-05-07T02:23:26.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:26.226+0000 7f0b5c800700 1 -- 172.21.15.19:0/2736970086 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b541a5050 con 0x7f0b54103f60 2024-05-07T02:23:26.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:26.227+0000 7f0b5c800700 1 -- 172.21.15.19:0/2736970086 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b540512c0 con 0x7f0b54103f60 2024-05-07T02:23:26.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:26.229+0000 7f0b4ea00700 1 -- 172.21.15.19:0/2736970086 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f0b4800c7e0 con 0x7f0b54103f60 2024-05-07T02:23:26.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:26.231+0000 7f0b4ea00700 1 --2- 172.21.15.19:0/2736970086 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f0b3c0b2900 0x7f0b3c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:23:26.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:26.231+0000 7f0b4ea00700 1 -- 172.21.15.19:0/2736970086 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5829+0+0 (secure 0 0 0) 0x7f0b480e1ad0 con 0x7f0b54103f60 2024-05-07T02:23:26.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:26.231+0000 7f0b59400700 1 --2- 172.21.15.19:0/2736970086 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f0b3c0b2900 0x7f0b3c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:23:26.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:26.232+0000 7f0b59400700 1 --2- 172.21.15.19:0/2736970086 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f0b3c0b2900 0x7f0b3c0b4e10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0b54077f20 tx=0x7f0b4400b720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:23:26.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:26.233+0000 7f0b4ea00700 1 -- 172.21.15.19:0/2736970086 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0b480aa190 con 0x7f0b54103f60 2024-05-07T02:23:26.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:26.485+0000 7f0b5c800700 1 -- 172.21.15.19:0/2736970086 --> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f0b54063c00 con 0x7f0b3c0b2900 2024-05-07T02:23:26.485 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:26 smithi019 ceph-mon[29781]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-07T02:23:26.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:26 smithi027 ceph-mon[34398]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-07T02:23:26.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:26 smithi064 ceph-mon[34699]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-07T02:23:27.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:27 smithi027 ceph-mon[34398]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:23:27.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:27 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-07T02:23:27.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:27 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-07T02:23:27.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:27 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:23:27.712 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:27 smithi019 ceph-mon[29781]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:23:27.712 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:27 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-07T02:23:27.712 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:27 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-07T02:23:27.712 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:27 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:23:27.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:27 smithi064 ceph-mon[34699]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:23:27.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:27 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-07T02:23:27.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:27 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-07T02:23:27.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:27 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:23:28.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:28 smithi027 ceph-mon[34398]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-05-07T02:23:28.723 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:28 smithi019 ceph-mon[29781]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-05-07T02:23:28.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:28 smithi064 ceph-mon[34699]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-05-07T02:23:30.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:30 smithi027 ceph-mon[34398]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-07T02:23:30.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:30 smithi064 ceph-mon[34699]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-07T02:23:30.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:30 smithi019 ceph-mon[29781]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-07T02:23:31.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:31 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/3067763470' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9df7ff66-d39f-4709-8324-74cf9e0e052d"}]: dispatch 2024-05-07T02:23:31.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:31 smithi027 ceph-mon[34398]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9df7ff66-d39f-4709-8324-74cf9e0e052d"}]: dispatch 2024-05-07T02:23:31.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:31 smithi027 ceph-mon[34398]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9df7ff66-d39f-4709-8324-74cf9e0e052d"}]': finished 2024-05-07T02:23:31.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:31 smithi027 ceph-mon[34398]: osdmap e56: 10 total, 9 up, 10 in 2024-05-07T02:23:31.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:31 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-07T02:23:31.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:31 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/3067763470' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9df7ff66-d39f-4709-8324-74cf9e0e052d"}]: dispatch 2024-05-07T02:23:31.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:31 smithi064 ceph-mon[34699]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9df7ff66-d39f-4709-8324-74cf9e0e052d"}]: dispatch 2024-05-07T02:23:31.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:31 smithi064 ceph-mon[34699]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9df7ff66-d39f-4709-8324-74cf9e0e052d"}]': finished 2024-05-07T02:23:31.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:31 smithi064 ceph-mon[34699]: osdmap e56: 10 total, 9 up, 10 in 2024-05-07T02:23:31.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:31 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-07T02:23:31.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:31 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/3067763470' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9df7ff66-d39f-4709-8324-74cf9e0e052d"}]: dispatch 2024-05-07T02:23:31.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:31 smithi019 ceph-mon[29781]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9df7ff66-d39f-4709-8324-74cf9e0e052d"}]: dispatch 2024-05-07T02:23:31.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:31 smithi019 ceph-mon[29781]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9df7ff66-d39f-4709-8324-74cf9e0e052d"}]': finished 2024-05-07T02:23:31.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:31 smithi019 ceph-mon[29781]: osdmap e56: 10 total, 9 up, 10 in 2024-05-07T02:23:31.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:31 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-07T02:23:32.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:32 smithi027 ceph-mon[34398]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-07T02:23:32.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:32 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/586361939' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-07T02:23:32.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:32 smithi064 ceph-mon[34699]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-07T02:23:32.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:32 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/586361939' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-07T02:23:32.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:32 smithi019 ceph-mon[29781]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-07T02:23:32.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:32 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/586361939' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-07T02:23:34.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:34 smithi019 ceph-mon[29781]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-07T02:23:34.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:34 smithi027 ceph-mon[34398]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-07T02:23:34.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:34 smithi064 ceph-mon[34699]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-07T02:23:36.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:36 smithi064 ceph-mon[34699]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-07T02:23:36.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:36 smithi019 ceph-mon[29781]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-07T02:23:36.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:36 smithi027 ceph-mon[34398]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-07T02:23:38.757 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:38 smithi019 ceph-mon[29781]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-07T02:23:38.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:38 smithi064 ceph-mon[34699]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-07T02:23:38.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:38 smithi027 ceph-mon[34398]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-07T02:23:40.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:40 smithi019 ceph-mon[29781]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-07T02:23:40.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:40 smithi064 ceph-mon[34699]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-07T02:23:40.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:40 smithi027 ceph-mon[34398]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-07T02:23:41.829 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:41 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-07T02:23:41.829 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:41 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:23:41.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:41 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-07T02:23:41.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:41 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:23:41.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:41 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-07T02:23:41.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:41 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:23:42.746 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:42 smithi019 ceph-mon[29781]: Deploying daemon osd.9 on smithi019 2024-05-07T02:23:42.746 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:42 smithi019 ceph-mon[29781]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-07T02:23:42.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:42 smithi064 ceph-mon[34699]: Deploying daemon osd.9 on smithi019 2024-05-07T02:23:42.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:42 smithi064 ceph-mon[34699]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-07T02:23:42.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:42 smithi027 ceph-mon[34398]: Deploying daemon osd.9 on smithi019 2024-05-07T02:23:42.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:42 smithi027 ceph-mon[34398]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-07T02:23:44.638 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:44 smithi019 ceph-mon[29781]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-07T02:23:44.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:44 smithi064 ceph-mon[34699]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-07T02:23:44.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:44 smithi027 ceph-mon[34398]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-07T02:23:46.858 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:46 smithi019 ceph-mon[29781]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-07T02:23:46.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:46 smithi064 ceph-mon[34699]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-07T02:23:46.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:46 smithi027 ceph-mon[34398]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-07T02:23:48.848 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:48 smithi019 ceph-mon[29781]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-07T02:23:48.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:48 smithi064 ceph-mon[34699]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-07T02:23:48.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:48 smithi027 ceph-mon[34398]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-07T02:23:50.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:50 smithi064 ceph-mon[34699]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-07T02:23:50.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:50 smithi019 ceph-mon[29781]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-07T02:23:50.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:50 smithi027 ceph-mon[34398]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-07T02:23:52.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:51 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:52.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:51 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:52.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:51 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:23:52.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:51 smithi064 ceph-mon[34699]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-07T02:23:52.113 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:51 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:52.113 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:51 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:52.113 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:51 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:23:52.113 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:51 smithi019 ceph-mon[29781]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-07T02:23:52.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:51 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:52.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:51 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:52.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:51 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:23:52.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:51 smithi027 ceph-mon[34398]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-07T02:23:54.608 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:54 smithi019 ceph-mon[29781]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-07T02:23:54.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:54 smithi027 ceph-mon[34398]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-07T02:23:54.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:54 smithi064 ceph-mon[34699]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-07T02:23:55.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:55 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:55.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:55 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:55.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:55 smithi064 ceph-mon[34699]: from='osd.9 [v2:172.21.15.19:6826/4124921646,v1:172.21.15.19:6827/4124921646]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-07T02:23:55.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:55 smithi064 ceph-mon[34699]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-07T02:23:55.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:55 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:55.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:55 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:55.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:55 smithi019 ceph-mon[29781]: from='osd.9 [v2:172.21.15.19:6826/4124921646,v1:172.21.15.19:6827/4124921646]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-07T02:23:55.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:55 smithi019 ceph-mon[29781]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-07T02:23:55.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:55 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:55.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:55 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:55.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:55 smithi027 ceph-mon[34398]: from='osd.9 [v2:172.21.15.19:6826/4124921646,v1:172.21.15.19:6827/4124921646]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-07T02:23:55.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:55 smithi027 ceph-mon[34398]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-07T02:23:56.741 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 9 on host 'smithi019' 2024-05-07T02:23:56.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:56.739+0000 7f0b4ea00700 1 -- 172.21.15.19:0/2736970086 <== mgr.14152 v2:172.21.15.19:6800/2059564551 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0b54063c00 con 0x7f0b3c0b2900 2024-05-07T02:23:56.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:56.743+0000 7f0b5c800700 1 -- 172.21.15.19:0/2736970086 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f0b3c0b2900 msgr2=0x7f0b3c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:23:56.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:56.743+0000 7f0b5c800700 1 --2- 172.21.15.19:0/2736970086 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f0b3c0b2900 0x7f0b3c0b4e10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0b54077f20 tx=0x7f0b4400b720 comp rx=0 tx=0).stop 2024-05-07T02:23:56.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:56.743+0000 7f0b5c800700 1 -- 172.21.15.19:0/2736970086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b54103f60 msgr2=0x7f0b54078570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:23:56.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:56.743+0000 7f0b5c800700 1 --2- 172.21.15.19:0/2736970086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b54103f60 0x7f0b54078570 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0b4800e940 tx=0x7f0b4800ec50 comp rx=0 tx=0).stop 2024-05-07T02:23:56.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:56.743+0000 7f0b5c800700 1 -- 172.21.15.19:0/2736970086 shutdown_connections 2024-05-07T02:23:56.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:56.743+0000 7f0b5c800700 1 --2- 172.21.15.19:0/2736970086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b541059b0 0x7f0b54077160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:23:56.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:56.743+0000 7f0b5c800700 1 --2- 172.21.15.19:0/2736970086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b54103f60 0x7f0b54078570 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:23:56.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:56.743+0000 7f0b5c800700 1 --2- 172.21.15.19:0/2736970086 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f0b3c0b2900 0x7f0b3c0b4e10 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:23:56.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:56.743+0000 7f0b5c800700 1 --2- 172.21.15.19:0/2736970086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b541049e0 0x7f0b54076c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:23:56.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:56.743+0000 7f0b5c800700 1 -- 172.21.15.19:0/2736970086 >> 172.21.15.19:0/2736970086 conn(0x7f0b540ff9d0 msgr2=0x7f0b541086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:23:56.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:56.744+0000 7f0b5c800700 1 -- 172.21.15.19:0/2736970086 shutdown_connections 2024-05-07T02:23:56.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:23:56.744+0000 7f0b5c800700 1 -- 172.21.15.19:0/2736970086 wait complete. 2024-05-07T02:23:56.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:56 smithi019 ceph-mon[29781]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-07T02:23:56.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:56 smithi019 ceph-mon[29781]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-07T02:23:56.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:56 smithi019 ceph-mon[29781]: osdmap e57: 10 total, 9 up, 10 in 2024-05-07T02:23:56.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:56 smithi019 ceph-mon[29781]: from='osd.9 [v2:172.21.15.19:6826/4124921646,v1:172.21.15.19:6827/4124921646]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-07T02:23:56.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:56 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-07T02:23:56.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:56 smithi019 ceph-mon[29781]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-07T02:23:56.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:56 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:23:56.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:56 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:23:56.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:56 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:56.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:56 smithi027 ceph-mon[34398]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-07T02:23:56.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:56 smithi027 ceph-mon[34398]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-07T02:23:56.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:56 smithi027 ceph-mon[34398]: osdmap e57: 10 total, 9 up, 10 in 2024-05-07T02:23:56.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:56 smithi027 ceph-mon[34398]: from='osd.9 [v2:172.21.15.19:6826/4124921646,v1:172.21.15.19:6827/4124921646]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-07T02:23:56.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:56 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-07T02:23:56.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:56 smithi027 ceph-mon[34398]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-07T02:23:56.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:56 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:23:56.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:56 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:23:56.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:56 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:57.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:56 smithi064 ceph-mon[34699]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-07T02:23:57.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:56 smithi064 ceph-mon[34699]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-07T02:23:57.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:56 smithi064 ceph-mon[34699]: osdmap e57: 10 total, 9 up, 10 in 2024-05-07T02:23:57.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:56 smithi064 ceph-mon[34699]: from='osd.9 [v2:172.21.15.19:6826/4124921646,v1:172.21.15.19:6827/4124921646]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-07T02:23:57.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:56 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-07T02:23:57.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:56 smithi064 ceph-mon[34699]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-05-07T02:23:57.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:56 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:23:57.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:56 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:23:57.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:56 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:57.705 DEBUG:teuthology.orchestra.run.smithi019:osd.9> sudo journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@osd.9.service 2024-05-07T02:23:57.716 INFO:tasks.cephadm:Deploying osd.10 on smithi027 with /dev/vg_nvme/lv_1... 2024-05-07T02:23:57.717 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-07T02:23:57.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:57 smithi019 ceph-mon[29781]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-07T02:23:57.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:57 smithi019 ceph-mon[29781]: osdmap e58: 10 total, 9 up, 10 in 2024-05-07T02:23:57.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:57 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-07T02:23:57.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:57 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-07T02:23:57.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:57 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:23:57.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:57 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:57.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:57 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:57.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:57 smithi027 ceph-mon[34398]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-07T02:23:57.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:57 smithi027 ceph-mon[34398]: osdmap e58: 10 total, 9 up, 10 in 2024-05-07T02:23:57.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:57 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-07T02:23:57.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:57 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-07T02:23:57.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:57 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:23:57.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:57 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:57.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:57 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:57.962 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.b/config 2024-05-07T02:23:58.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:57 smithi064 ceph-mon[34699]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-05-07T02:23:58.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:57 smithi064 ceph-mon[34699]: osdmap e58: 10 total, 9 up, 10 in 2024-05-07T02:23:58.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:57 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-07T02:23:58.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:57 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-07T02:23:58.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:57 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:23:58.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:57 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:58.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:57 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:23:58.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:58 smithi027 ceph-mon[34398]: purged_snaps scrub starts 2024-05-07T02:23:58.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:58 smithi027 ceph-mon[34398]: purged_snaps scrub ok 2024-05-07T02:23:58.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:58 smithi027 ceph-mon[34398]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-07T02:23:58.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:58 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-07T02:23:58.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:58 smithi027 ceph-mon[34398]: osd.9 [v2:172.21.15.19:6826/4124921646,v1:172.21.15.19:6827/4124921646] boot 2024-05-07T02:23:58.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:58 smithi027 ceph-mon[34398]: osdmap e59: 10 total, 10 up, 10 in 2024-05-07T02:23:58.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:58 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-07T02:23:59.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:58 smithi064 ceph-mon[34699]: purged_snaps scrub starts 2024-05-07T02:23:59.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:58 smithi064 ceph-mon[34699]: purged_snaps scrub ok 2024-05-07T02:23:59.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:58 smithi064 ceph-mon[34699]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-07T02:23:59.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:58 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-07T02:23:59.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:58 smithi064 ceph-mon[34699]: osd.9 [v2:172.21.15.19:6826/4124921646,v1:172.21.15.19:6827/4124921646] boot 2024-05-07T02:23:59.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:58 smithi064 ceph-mon[34699]: osdmap e59: 10 total, 10 up, 10 in 2024-05-07T02:23:59.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:58 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-07T02:23:59.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:58 smithi019 ceph-mon[29781]: purged_snaps scrub starts 2024-05-07T02:23:59.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:58 smithi019 ceph-mon[29781]: purged_snaps scrub ok 2024-05-07T02:23:59.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:58 smithi019 ceph-mon[29781]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-05-07T02:23:59.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:58 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-07T02:23:59.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:58 smithi019 ceph-mon[29781]: osd.9 [v2:172.21.15.19:6826/4124921646,v1:172.21.15.19:6827/4124921646] boot 2024-05-07T02:23:59.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:58 smithi019 ceph-mon[29781]: osdmap e59: 10 total, 10 up, 10 in 2024-05-07T02:23:59.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:58 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-07T02:23:59.774 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:23:59 smithi019 ceph-mon[29781]: osdmap e60: 10 total, 10 up, 10 in 2024-05-07T02:23:59.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:23:59 smithi027 ceph-mon[34398]: osdmap e60: 10 total, 10 up, 10 in 2024-05-07T02:24:00.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:23:59 smithi064 ceph-mon[34699]: osdmap e60: 10 total, 10 up, 10 in 2024-05-07T02:24:00.139 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-07T02:24:00.139 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-07T02:24:00.139 INFO:teuthology.orchestra.run.smithi027.stderr: stderr: 10+0 records in 2024-05-07T02:24:00.139 INFO:teuthology.orchestra.run.smithi027.stderr:10+0 records out 2024-05-07T02:24:00.140 INFO:teuthology.orchestra.run.smithi027.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111029 s, 944 MB/s 2024-05-07T02:24:00.140 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping successful for: 2024-05-07T02:24:00.843 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi027:vg_nvme/lv_1 2024-05-07T02:24:01.095 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.b/config 2024-05-07T02:24:01.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:00 smithi064 ceph-mon[34699]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-05-07T02:24:01.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:00 smithi019 ceph-mon[29781]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-05-07T02:24:01.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:00 smithi027 ceph-mon[34398]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-05-07T02:24:02.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:02.854+0000 7fdec1e8a700 1 -- 172.21.15.27:0/608434108 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdebc104d20 msgr2=0x7fdebc10b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:24:02.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:02.854+0000 7fdec1e8a700 1 --2- 172.21.15.27:0/608434108 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdebc104d20 0x7fdebc10b5d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdeb400b0a0 tx=0x7fdeb400b3b0 comp rx=0 tx=0).stop 2024-05-07T02:24:02.854 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:02.854+0000 7fdec1e8a700 1 -- 172.21.15.27:0/608434108 shutdown_connections 2024-05-07T02:24:02.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:02.854+0000 7fdec1e8a700 1 --2- 172.21.15.27:0/608434108 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdebc104d20 0x7fdebc10b5d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:24:02.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:02.854+0000 7fdec1e8a700 1 --2- 172.21.15.27:0/608434108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdebc103250 0x7fdebc103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:24:02.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:02.854+0000 7fdec1e8a700 1 --2- 172.21.15.27:0/608434108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdebc104380 0x7fdebc104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:24:02.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:02.854+0000 7fdec1e8a700 1 -- 172.21.15.27:0/608434108 >> 172.21.15.27:0/608434108 conn(0x7fdebc0fe890 msgr2=0x7fdebc100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:24:02.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:02.854+0000 7fdec1e8a700 1 -- 172.21.15.27:0/608434108 shutdown_connections 2024-05-07T02:24:02.855 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:02.854+0000 7fdec1e8a700 1 -- 172.21.15.27:0/608434108 wait complete. 2024-05-07T02:24:02.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:02.855+0000 7fdec1e8a700 1 Processor -- start 2024-05-07T02:24:02.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:02.855+0000 7fdec1e8a700 1 -- start start 2024-05-07T02:24:02.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:02.855+0000 7fdec1e8a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdebc103250 0x7fdebc19a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:24:02.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:02.855+0000 7fdec1e8a700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdebc104380 0x7fdebc19adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:24:02.857 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:02.855+0000 7fdec1e8a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdebc104d20 0x7fdebc1a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:24:02.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:02.855+0000 7fdec1e8a700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc10ecb0 con 0x7fdebc104380 2024-05-07T02:24:02.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:02.855+0000 7fdec1e8a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc10eb30 con 0x7fdebc103250 2024-05-07T02:24:02.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:02.855+0000 7fdec1e8a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc10ee30 con 0x7fdebc104d20 2024-05-07T02:24:02.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:02.856+0000 7fdebb7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdebc103250 0x7fdebc19a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:24:02.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:02.856+0000 7fdebb7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdebc103250 0x7fdebc19a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34792/0 (socket says 172.21.15.27:34792) 2024-05-07T02:24:02.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:02.856+0000 7fdebb7fe700 1 -- 172.21.15.27:0/1157117956 learned_addr learned my addr 172.21.15.27:0/1157117956 (peer_addr_for_me v2:172.21.15.27:0/0) 2024-05-07T02:24:02.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:02.856+0000 7fdebbfff700 1 --2- 172.21.15.27:0/1157117956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdebc104d20 0x7fdebc1a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:24:02.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:02.856+0000 7fdebaffd700 1 --2- 172.21.15.27:0/1157117956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdebc104380 0x7fdebc19adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:24:02.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:02.856+0000 7fdebb7fe700 1 -- 172.21.15.27:0/1157117956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdebc104d20 msgr2=0x7fdebc1a1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:24:02.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:02.856+0000 7fdebb7fe700 1 --2- 172.21.15.27:0/1157117956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdebc104d20 0x7fdebc1a1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:24:02.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:02.856+0000 7fdebb7fe700 1 -- 172.21.15.27:0/1157117956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdebc104380 msgr2=0x7fdebc19adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:24:02.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:02.856+0000 7fdebb7fe700 1 --2- 172.21.15.27:0/1157117956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdebc104380 0x7fdebc19adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:24:02.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:02.856+0000 7fdebb7fe700 1 -- 172.21.15.27:0/1157117956 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdebc1a2410 con 0x7fdebc103250 2024-05-07T02:24:02.858 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:02.856+0000 7fdebb7fe700 1 --2- 172.21.15.27:0/1157117956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdebc103250 0x7fdebc19a8b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdeb000c970 tx=0x7fdeb000cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:24:02.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:02.856+0000 7fdeb8ff9700 1 -- 172.21.15.27:0/1157117956 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb000ce60 con 0x7fdebc103250 2024-05-07T02:24:02.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:02.856+0000 7fdec1e8a700 1 -- 172.21.15.27:0/1157117956 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdebc1a2700 con 0x7fdebc103250 2024-05-07T02:24:02.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:02.856+0000 7fdeb8ff9700 1 -- 172.21.15.27:0/1157117956 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdeb0007900 con 0x7fdebc103250 2024-05-07T02:24:02.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:02.856+0000 7fdeb8ff9700 1 -- 172.21.15.27:0/1157117956 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb0003c70 con 0x7fdebc103250 2024-05-07T02:24:02.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:02.856+0000 7fdec1e8a700 1 -- 172.21.15.27:0/1157117956 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdebc1a2c40 con 0x7fdebc103250 2024-05-07T02:24:02.859 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:02.857+0000 7fdec1e8a700 1 -- 172.21.15.27:0/1157117956 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdebc0512c0 con 0x7fdebc103250 2024-05-07T02:24:02.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:02.859+0000 7fdeb8ff9700 1 -- 172.21.15.27:0/1157117956 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fdeb0004220 con 0x7fdebc103250 2024-05-07T02:24:02.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:02.860+0000 7fdeb8ff9700 1 --2- 172.21.15.27:0/1157117956 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fdea40b2880 0x7fdea40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:24:02.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:02.860+0000 7fdeb8ff9700 1 -- 172.21.15.27:0/1157117956 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6261+0+0 (secure 0 0 0) 0x7fdeb00e1920 con 0x7fdebc103250 2024-05-07T02:24:02.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:02.860+0000 7fdebaffd700 1 --2- 172.21.15.27:0/1157117956 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fdea40b2880 0x7fdea40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:24:02.860 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:02.861+0000 7fdebaffd700 1 --2- 172.21.15.27:0/1157117956 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fdea40b2880 0x7fdea40b4d90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdeac005950 tx=0x7fdeac0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:24:02.863 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:02.863+0000 7fdeb8ff9700 1 -- 172.21.15.27:0/1157117956 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdeb0014030 con 0x7fdebc103250 2024-05-07T02:24:02.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:02 smithi019 ceph-mon[29781]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-07T02:24:02.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:02 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:02.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:02 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:02.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:02 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:24:02.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:02 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:24:02.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:02 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:24:02.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:02 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:24:02.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:02 smithi027 ceph-mon[34398]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-07T02:24:02.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:02 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:02.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:02 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:02.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:02 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:24:02.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:02 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:24:02.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:02 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:24:02.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:02 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:24:03.089 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:03.089+0000 7fdec1e8a700 1 -- 172.21.15.27:0/1157117956 --> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fdebc1a2f00 con 0x7fdea40b2880 2024-05-07T02:24:03.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:02 smithi064 ceph-mon[34699]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-07T02:24:03.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:02 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:03.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:02 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:03.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:02 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:24:03.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:02 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:24:03.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:02 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:24:03.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:02 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:24:04.029 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:03 smithi027 ceph-mon[34398]: Detected new or changed devices on smithi019 2024-05-07T02:24:04.029 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:03 smithi027 ceph-mon[34398]: Adjusting osd_memory_target on smithi019 to 4273M 2024-05-07T02:24:04.029 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:03 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:04.029 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:03 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:24:04.029 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:03 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:24:04.029 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:03 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:04.030 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:03 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-07T02:24:04.030 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:03 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-07T02:24:04.030 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:03 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:24:04.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:03 smithi064 ceph-mon[34699]: Detected new or changed devices on smithi019 2024-05-07T02:24:04.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:03 smithi064 ceph-mon[34699]: Adjusting osd_memory_target on smithi019 to 4273M 2024-05-07T02:24:04.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:03 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:04.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:03 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:24:04.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:03 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:24:04.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:03 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:04.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:03 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-07T02:24:04.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:03 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-07T02:24:04.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:03 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:24:04.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:03 smithi019 ceph-mon[29781]: Detected new or changed devices on smithi019 2024-05-07T02:24:04.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:03 smithi019 ceph-mon[29781]: Adjusting osd_memory_target on smithi019 to 4273M 2024-05-07T02:24:04.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:03 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:04.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:03 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:24:04.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:03 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:24:04.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:03 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:04.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:03 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-07T02:24:04.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:03 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-07T02:24:04.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:03 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:24:05.099 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:04 smithi027 ceph-mon[34398]: from='client.24377 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:24:05.099 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:04 smithi027 ceph-mon[34398]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-07T02:24:05.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:04 smithi064 ceph-mon[34699]: from='client.24377 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:24:05.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:04 smithi064 ceph-mon[34699]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-07T02:24:05.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:04 smithi019 ceph-mon[29781]: from='client.24377 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:24:05.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:04 smithi019 ceph-mon[29781]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-07T02:24:07.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:06 smithi064 ceph-mon[34699]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-07T02:24:07.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:06 smithi064 ceph-mon[34699]: from='client.? 172.21.15.27:0/2287973361' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71eaa9a7-667d-4cb7-935c-42177d20a279"}]: dispatch 2024-05-07T02:24:07.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:06 smithi064 ceph-mon[34699]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71eaa9a7-667d-4cb7-935c-42177d20a279"}]: dispatch 2024-05-07T02:24:07.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:06 smithi064 ceph-mon[34699]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "71eaa9a7-667d-4cb7-935c-42177d20a279"}]': finished 2024-05-07T02:24:07.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:06 smithi064 ceph-mon[34699]: osdmap e61: 11 total, 10 up, 11 in 2024-05-07T02:24:07.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:06 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-07T02:24:07.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:06 smithi064 ceph-mon[34699]: from='client.? 172.21.15.27:0/3177587966' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-07T02:24:07.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:06 smithi019 ceph-mon[29781]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-07T02:24:07.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:06 smithi019 ceph-mon[29781]: from='client.? 172.21.15.27:0/2287973361' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71eaa9a7-667d-4cb7-935c-42177d20a279"}]: dispatch 2024-05-07T02:24:07.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:06 smithi019 ceph-mon[29781]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71eaa9a7-667d-4cb7-935c-42177d20a279"}]: dispatch 2024-05-07T02:24:07.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:06 smithi019 ceph-mon[29781]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "71eaa9a7-667d-4cb7-935c-42177d20a279"}]': finished 2024-05-07T02:24:07.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:06 smithi019 ceph-mon[29781]: osdmap e61: 11 total, 10 up, 11 in 2024-05-07T02:24:07.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:06 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-07T02:24:07.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:06 smithi019 ceph-mon[29781]: from='client.? 172.21.15.27:0/3177587966' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-07T02:24:07.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:06 smithi027 ceph-mon[34398]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-07T02:24:07.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:06 smithi027 ceph-mon[34398]: from='client.? 172.21.15.27:0/2287973361' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71eaa9a7-667d-4cb7-935c-42177d20a279"}]: dispatch 2024-05-07T02:24:07.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:06 smithi027 ceph-mon[34398]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71eaa9a7-667d-4cb7-935c-42177d20a279"}]: dispatch 2024-05-07T02:24:07.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:06 smithi027 ceph-mon[34398]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "71eaa9a7-667d-4cb7-935c-42177d20a279"}]': finished 2024-05-07T02:24:07.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:06 smithi027 ceph-mon[34398]: osdmap e61: 11 total, 10 up, 11 in 2024-05-07T02:24:07.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:06 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-07T02:24:07.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:06 smithi027 ceph-mon[34398]: from='client.? 172.21.15.27:0/3177587966' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-07T02:24:08.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:07 smithi064 ceph-mon[34699]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-07T02:24:08.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:07 smithi019 ceph-mon[29781]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-07T02:24:08.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:07 smithi027 ceph-mon[34398]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-07T02:24:10.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:10 smithi027 ceph-mon[34398]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-07T02:24:10.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:10 smithi064 ceph-mon[34699]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-07T02:24:10.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:10 smithi019 ceph-mon[29781]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-07T02:24:12.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:12 smithi027 ceph-mon[34398]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-07T02:24:12.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:12 smithi064 ceph-mon[34699]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-07T02:24:12.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:12 smithi019 ceph-mon[29781]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-07T02:24:14.471 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:14 smithi027 ceph-mon[34398]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-07T02:24:14.471 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:14 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-07T02:24:14.471 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:14 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:24:14.471 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:14 smithi027 ceph-mon[34398]: Deploying daemon osd.10 on smithi027 2024-05-07T02:24:14.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:14 smithi064 ceph-mon[34699]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-07T02:24:14.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:14 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-07T02:24:14.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:14 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:24:14.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:14 smithi064 ceph-mon[34699]: Deploying daemon osd.10 on smithi027 2024-05-07T02:24:14.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:14 smithi019 ceph-mon[29781]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-07T02:24:14.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:14 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-07T02:24:14.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:14 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:24:14.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:14 smithi019 ceph-mon[29781]: Deploying daemon osd.10 on smithi027 2024-05-07T02:24:16.563 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:16 smithi027 ceph-mon[34398]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-07T02:24:16.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:16 smithi064 ceph-mon[34699]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-07T02:24:16.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:16 smithi019 ceph-mon[29781]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-07T02:24:18.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:18 smithi064 ceph-mon[34699]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-07T02:24:18.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:18 smithi019 ceph-mon[29781]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-07T02:24:18.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:18 smithi027 ceph-mon[34398]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-07T02:24:20.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:20 smithi064 ceph-mon[34699]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-07T02:24:20.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:20 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:20.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:20 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:24:20.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:20 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:20.875 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:20 smithi027 ceph-mon[34398]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-07T02:24:20.875 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:20 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:20.876 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:20 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:24:20.876 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:20 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:20.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:20 smithi019 ceph-mon[29781]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-07T02:24:20.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:20 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:20.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:20 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:24:20.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:20 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:22.831 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:22 smithi027 ceph-mon[34398]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-07T02:24:22.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:22 smithi064 ceph-mon[34699]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-07T02:24:22.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:22 smithi019 ceph-mon[29781]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-07T02:24:24.233 INFO:teuthology.orchestra.run.smithi027.stdout:Created osd(s) 10 on host 'smithi027' 2024-05-07T02:24:24.233 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:24.232+0000 7fdeb8ff9700 1 -- 172.21.15.27:0/1157117956 <== mgr.14152 v2:172.21.15.19:6800/2059564551 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fdebc1a2f00 con 0x7fdea40b2880 2024-05-07T02:24:24.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:24.235+0000 7fdec1e8a700 1 -- 172.21.15.27:0/1157117956 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fdea40b2880 msgr2=0x7fdea40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:24:24.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:24.235+0000 7fdec1e8a700 1 --2- 172.21.15.27:0/1157117956 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fdea40b2880 0x7fdea40b4d90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdeac005950 tx=0x7fdeac0058e0 comp rx=0 tx=0).stop 2024-05-07T02:24:24.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:24.235+0000 7fdec1e8a700 1 -- 172.21.15.27:0/1157117956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdebc103250 msgr2=0x7fdebc19a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:24:24.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:24.235+0000 7fdec1e8a700 1 --2- 172.21.15.27:0/1157117956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdebc103250 0x7fdebc19a8b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdeb000c970 tx=0x7fdeb000cc80 comp rx=0 tx=0).stop 2024-05-07T02:24:24.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:24.235+0000 7fdec1e8a700 1 -- 172.21.15.27:0/1157117956 shutdown_connections 2024-05-07T02:24:24.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:24.235+0000 7fdec1e8a700 1 --2- 172.21.15.27:0/1157117956 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdebc104d20 0x7fdebc1a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:24:24.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:24.235+0000 7fdec1e8a700 1 --2- 172.21.15.27:0/1157117956 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fdea40b2880 0x7fdea40b4d90 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:24:24.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:24.235+0000 7fdec1e8a700 1 --2- 172.21.15.27:0/1157117956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdebc103250 0x7fdebc19a8b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:24:24.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:24.235+0000 7fdec1e8a700 1 --2- 172.21.15.27:0/1157117956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdebc104380 0x7fdebc19adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:24:24.235 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:24.235+0000 7fdec1e8a700 1 -- 172.21.15.27:0/1157117956 >> 172.21.15.27:0/1157117956 conn(0x7fdebc0fe890 msgr2=0x7fdebc1001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:24:24.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:24.235+0000 7fdec1e8a700 1 -- 172.21.15.27:0/1157117956 shutdown_connections 2024-05-07T02:24:24.236 INFO:teuthology.orchestra.run.smithi027.stderr:2024-05-07T02:24:24.235+0000 7fdec1e8a700 1 -- 172.21.15.27:0/1157117956 wait complete. 2024-05-07T02:24:24.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:24 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:24.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:24 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:24.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:24 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:24:24.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:24 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:24:24.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:24 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:24.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:24 smithi027 ceph-mon[34398]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-07T02:24:24.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:24 smithi027 ceph-mon[34398]: from='osd.10 [v2:172.21.15.27:6824/1428931817,v1:172.21.15.27:6825/1428931817]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-07T02:24:24.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:24 smithi027 ceph-mon[34398]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-07T02:24:24.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:24 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:24.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:24 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:24:24.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:24 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:24.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:24 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:24.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:24 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:24.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:24 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:24:24.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:24 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:24:24.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:24 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:24.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:24 smithi064 ceph-mon[34699]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-07T02:24:24.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:24 smithi064 ceph-mon[34699]: from='osd.10 [v2:172.21.15.27:6824/1428931817,v1:172.21.15.27:6825/1428931817]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-07T02:24:24.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:24 smithi064 ceph-mon[34699]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-07T02:24:24.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:24 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:24.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:24 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:24:24.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:24 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:24.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:24 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:24.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:24 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:24.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:24 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:24:24.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:24 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:24:24.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:24 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:24.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:24 smithi019 ceph-mon[29781]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-07T02:24:24.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:24 smithi019 ceph-mon[29781]: from='osd.10 [v2:172.21.15.27:6824/1428931817,v1:172.21.15.27:6825/1428931817]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-07T02:24:24.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:24 smithi019 ceph-mon[29781]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-07T02:24:24.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:24 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:24.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:24 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:24:24.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:24 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:24.980 DEBUG:teuthology.orchestra.run.smithi027:osd.10> sudo journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@osd.10.service 2024-05-07T02:24:24.982 INFO:tasks.cephadm:Deploying osd.11 on smithi064 with /dev/vg_nvme/lv_1... 2024-05-07T02:24:24.983 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-07T02:24:25.244 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.c/config 2024-05-07T02:24:25.469 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:25 smithi027 ceph-mon[34398]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-07T02:24:25.470 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:25 smithi027 ceph-mon[34398]: from='osd.10 [v2:172.21.15.27:6824/1428931817,v1:172.21.15.27:6825/1428931817]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-05-07T02:24:25.470 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:25 smithi027 ceph-mon[34398]: osdmap e62: 11 total, 10 up, 11 in 2024-05-07T02:24:25.470 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:25 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-07T02:24:25.470 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:25 smithi027 ceph-mon[34398]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-05-07T02:24:25.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:25 smithi019 ceph-mon[29781]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-07T02:24:25.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:25 smithi019 ceph-mon[29781]: from='osd.10 [v2:172.21.15.27:6824/1428931817,v1:172.21.15.27:6825/1428931817]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-05-07T02:24:25.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:25 smithi019 ceph-mon[29781]: osdmap e62: 11 total, 10 up, 11 in 2024-05-07T02:24:25.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:25 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-07T02:24:25.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:25 smithi019 ceph-mon[29781]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-05-07T02:24:25.708 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:25 smithi064 ceph-mon[34699]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-07T02:24:25.708 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:25 smithi064 ceph-mon[34699]: from='osd.10 [v2:172.21.15.27:6824/1428931817,v1:172.21.15.27:6825/1428931817]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-05-07T02:24:25.708 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:25 smithi064 ceph-mon[34699]: osdmap e62: 11 total, 10 up, 11 in 2024-05-07T02:24:25.708 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:25 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-07T02:24:25.708 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:25 smithi064 ceph-mon[34699]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-05-07T02:24:26.207 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:24:26 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:24:26.117+0000 7f53724ee700 -1 osd.10 0 waiting for initial osdmap 2024-05-07T02:24:26.207 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:24:26 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:24:26.136+0000 7f536a29b700 -1 osd.10 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-07T02:24:26.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:26 smithi019 ceph-mon[29781]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-07T02:24:26.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:26 smithi019 ceph-mon[29781]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-05-07T02:24:26.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:26 smithi019 ceph-mon[29781]: osdmap e63: 11 total, 10 up, 11 in 2024-05-07T02:24:26.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:26 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-07T02:24:26.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:26 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-07T02:24:26.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:26 smithi019 ceph-mon[29781]: from='osd.10 ' entity='osd.10' 2024-05-07T02:24:26.671 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:26 smithi027 ceph-mon[34398]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-07T02:24:26.671 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:26 smithi027 ceph-mon[34398]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-05-07T02:24:26.671 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:26 smithi027 ceph-mon[34398]: osdmap e63: 11 total, 10 up, 11 in 2024-05-07T02:24:26.671 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:26 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-07T02:24:26.671 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:26 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-07T02:24:26.671 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:26 smithi027 ceph-mon[34398]: from='osd.10 ' entity='osd.10' 2024-05-07T02:24:26.800 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:26 smithi064 ceph-mon[34699]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-07T02:24:26.800 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:26 smithi064 ceph-mon[34699]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-05-07T02:24:26.800 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:26 smithi064 ceph-mon[34699]: osdmap e63: 11 total, 10 up, 11 in 2024-05-07T02:24:26.800 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:26 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-07T02:24:26.800 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:26 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-07T02:24:26.801 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:26 smithi064 ceph-mon[34699]: from='osd.10 ' entity='osd.10' 2024-05-07T02:24:27.624 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-07T02:24:27.624 INFO:teuthology.orchestra.run.smithi064.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-07T02:24:27.624 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10+0 records in 2024-05-07T02:24:27.624 INFO:teuthology.orchestra.run.smithi064.stderr:10+0 records out 2024-05-07T02:24:27.625 INFO:teuthology.orchestra.run.smithi064.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113817 s, 921 MB/s 2024-05-07T02:24:27.625 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping successful for: 2024-05-07T02:24:27.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:27 smithi019 ceph-mon[29781]: purged_snaps scrub starts 2024-05-07T02:24:27.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:27 smithi019 ceph-mon[29781]: purged_snaps scrub ok 2024-05-07T02:24:27.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:27 smithi019 ceph-mon[29781]: osd.10 [v2:172.21.15.27:6824/1428931817,v1:172.21.15.27:6825/1428931817] boot 2024-05-07T02:24:27.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:27 smithi019 ceph-mon[29781]: osdmap e64: 11 total, 11 up, 11 in 2024-05-07T02:24:27.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:27 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-07T02:24:27.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:27 smithi019 ceph-mon[29781]: osdmap e65: 11 total, 11 up, 11 in 2024-05-07T02:24:27.652 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:27 smithi064 ceph-mon[34699]: purged_snaps scrub starts 2024-05-07T02:24:27.653 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:27 smithi064 ceph-mon[34699]: purged_snaps scrub ok 2024-05-07T02:24:27.653 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:27 smithi064 ceph-mon[34699]: osd.10 [v2:172.21.15.27:6824/1428931817,v1:172.21.15.27:6825/1428931817] boot 2024-05-07T02:24:27.653 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:27 smithi064 ceph-mon[34699]: osdmap e64: 11 total, 11 up, 11 in 2024-05-07T02:24:27.653 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:27 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-07T02:24:27.653 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:27 smithi064 ceph-mon[34699]: osdmap e65: 11 total, 11 up, 11 in 2024-05-07T02:24:27.668 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:27 smithi027 ceph-mon[34398]: purged_snaps scrub starts 2024-05-07T02:24:27.668 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:27 smithi027 ceph-mon[34398]: purged_snaps scrub ok 2024-05-07T02:24:27.668 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:27 smithi027 ceph-mon[34398]: osd.10 [v2:172.21.15.27:6824/1428931817,v1:172.21.15.27:6825/1428931817] boot 2024-05-07T02:24:27.668 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:27 smithi027 ceph-mon[34398]: osdmap e64: 11 total, 11 up, 11 in 2024-05-07T02:24:27.668 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:27 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-07T02:24:27.668 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:27 smithi027 ceph-mon[34398]: osdmap e65: 11 total, 11 up, 11 in 2024-05-07T02:24:28.481 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi064:vg_nvme/lv_1 2024-05-07T02:24:28.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:28 smithi027 ceph-mon[34398]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 722 MiB used, 983 GiB / 983 GiB avail 2024-05-07T02:24:28.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:28 smithi027 ceph-mon[34398]: osdmap e66: 11 total, 11 up, 11 in 2024-05-07T02:24:28.727 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.c/config 2024-05-07T02:24:28.849 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:28 smithi064 ceph-mon[34699]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 722 MiB used, 983 GiB / 983 GiB avail 2024-05-07T02:24:28.849 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:28 smithi064 ceph-mon[34699]: osdmap e66: 11 total, 11 up, 11 in 2024-05-07T02:24:28.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:28 smithi019 ceph-mon[29781]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 722 MiB used, 983 GiB / 983 GiB avail 2024-05-07T02:24:28.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:28 smithi019 ceph-mon[29781]: osdmap e66: 11 total, 11 up, 11 in 2024-05-07T02:24:29.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:29 smithi064 ceph-mon[34699]: Detected new or changed devices on smithi027 2024-05-07T02:24:29.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:29 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:29.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:29 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:29.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:29 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:24:29.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:29 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:24:29.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:29 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:24:29.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:29 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:24:29.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:29 smithi064 ceph-mon[34699]: Adjusting osd_memory_target on smithi027 to 4272M 2024-05-07T02:24:29.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:29 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:29.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:29 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:24:29.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:29 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:24:29.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:29 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:29.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:29 smithi019 ceph-mon[29781]: Detected new or changed devices on smithi027 2024-05-07T02:24:29.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:29 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:29.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:29 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:29.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:29 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:24:29.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:29 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:24:29.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:29 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:24:29.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:29 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:24:29.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:29 smithi019 ceph-mon[29781]: Adjusting osd_memory_target on smithi027 to 4272M 2024-05-07T02:24:29.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:29 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:29.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:29 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:24:29.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:29 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:24:29.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:29 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:29.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:29 smithi027 ceph-mon[34398]: Detected new or changed devices on smithi027 2024-05-07T02:24:29.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:29 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:29.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:29 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:29.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:29 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:24:29.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:29 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:24:29.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:29 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:24:29.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:29 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:24:29.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:29 smithi027 ceph-mon[34398]: Adjusting osd_memory_target on smithi027 to 4272M 2024-05-07T02:24:29.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:29 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:29.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:29 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:24:29.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:29 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:24:29.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:29 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:30.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:30.639+0000 7fcbb78cb700 1 -- 172.21.15.64:0/1625795334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbb0103230 msgr2=0x7fcbb0103640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:24:30.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:30.639+0000 7fcbb78cb700 1 --2- 172.21.15.64:0/1625795334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbb0103230 0x7fcbb0103640 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcbac009990 tx=0x7fcbac009ca0 comp rx=0 tx=0).stop 2024-05-07T02:24:30.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:30.640+0000 7fcbb78cb700 1 -- 172.21.15.64:0/1625795334 shutdown_connections 2024-05-07T02:24:30.641 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:30.640+0000 7fcbb78cb700 1 --2- 172.21.15.64:0/1625795334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbb0104d00 0x7fcbb010b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:24:30.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:30.640+0000 7fcbb78cb700 1 --2- 172.21.15.64:0/1625795334 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbb0103230 0x7fcbb0103640 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:24:30.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:30.640+0000 7fcbb78cb700 1 --2- 172.21.15.64:0/1625795334 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbb0104360 0x7fcbb0104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:24:30.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:30.640+0000 7fcbb78cb700 1 -- 172.21.15.64:0/1625795334 >> 172.21.15.64:0/1625795334 conn(0x7fcbb00fe870 msgr2=0x7fcbb0100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:24:30.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:30.640+0000 7fcbb78cb700 1 -- 172.21.15.64:0/1625795334 shutdown_connections 2024-05-07T02:24:30.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:30.640+0000 7fcbb78cb700 1 -- 172.21.15.64:0/1625795334 wait complete. 2024-05-07T02:24:30.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:30.640+0000 7fcbb78cb700 1 Processor -- start 2024-05-07T02:24:30.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:30.640+0000 7fcbb78cb700 1 -- start start 2024-05-07T02:24:30.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:30.641+0000 7fcbb78cb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbb0103230 0x7fcbb019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:24:30.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:30.641+0000 7fcbb78cb700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbb0104360 0x7fcbb019af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:24:30.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:30.641+0000 7fcbb78cb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbb0104d00 0x7fcbb01a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:24:30.642 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:30.641+0000 7fcbb78cb700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb010ed50 con 0x7fcbb0104360 2024-05-07T02:24:30.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:30.641+0000 7fcbb78cb700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb010ebd0 con 0x7fcbb0104d00 2024-05-07T02:24:30.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:30.641+0000 7fcbb78cb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb010eed0 con 0x7fcbb0103230 2024-05-07T02:24:30.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:30.641+0000 7fcbb5667700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbb0103230 0x7fcbb019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:24:30.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:30.641+0000 7fcbb5667700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbb0103230 0x7fcbb019a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35172/0 (socket says 172.21.15.64:35172) 2024-05-07T02:24:30.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:30.641+0000 7fcbb5667700 1 -- 172.21.15.64:0/3991298322 learned_addr learned my addr 172.21.15.64:0/3991298322 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-07T02:24:30.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:30.641+0000 7fcbb5e68700 1 --2- 172.21.15.64:0/3991298322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbb0104d00 0x7fcbb01a1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:24:30.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:30.641+0000 7fcbb5667700 1 -- 172.21.15.64:0/3991298322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbb0104d00 msgr2=0x7fcbb01a1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:24:30.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:30.641+0000 7fcbb5667700 1 --2- 172.21.15.64:0/3991298322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbb0104d00 0x7fcbb01a1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:24:30.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:30.641+0000 7fcbb5667700 1 -- 172.21.15.64:0/3991298322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbb0104360 msgr2=0x7fcbb019af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:24:30.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:30.641+0000 7fcbb5667700 1 --2- 172.21.15.64:0/3991298322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbb0104360 0x7fcbb019af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:24:30.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:30.641+0000 7fcbb5667700 1 -- 172.21.15.64:0/3991298322 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb01a2520 con 0x7fcbb0103230 2024-05-07T02:24:30.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:30.641+0000 7fcbb5e68700 1 --2- 172.21.15.64:0/3991298322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbb0104d00 0x7fcbb01a1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:24:30.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:30.641+0000 7fcbb5667700 1 --2- 172.21.15.64:0/3991298322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbb0103230 0x7fcbb019a9c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcbac0098e0 tx=0x7fcbac0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:24:30.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:30.641+0000 7fcba27fc700 1 -- 172.21.15.64:0/3991298322 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbac027070 con 0x7fcbb0103230 2024-05-07T02:24:30.643 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:30.641+0000 7fcba27fc700 1 -- 172.21.15.64:0/3991298322 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbac022470 con 0x7fcbb0103230 2024-05-07T02:24:30.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:30.641+0000 7fcbb78cb700 1 -- 172.21.15.64:0/3991298322 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbb01a27b0 con 0x7fcbb0103230 2024-05-07T02:24:30.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:30.642+0000 7fcba27fc700 1 -- 172.21.15.64:0/3991298322 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbac00f6c0 con 0x7fcbb0103230 2024-05-07T02:24:30.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:30.642+0000 7fcbb78cb700 1 -- 172.21.15.64:0/3991298322 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbb01a2c80 con 0x7fcbb0103230 2024-05-07T02:24:30.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:30.642+0000 7fcbb78cb700 1 -- 172.21.15.64:0/3991298322 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbb00512c0 con 0x7fcbb0103230 2024-05-07T02:24:30.644 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:30.644+0000 7fcba27fc700 1 -- 172.21.15.64:0/3991298322 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fcbac02a070 con 0x7fcbb0103230 2024-05-07T02:24:30.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:30.645+0000 7fcba27fc700 1 --2- 172.21.15.64:0/3991298322 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fcb9c0b2880 0x7fcb9c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:24:30.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:30.645+0000 7fcba27fc700 1 -- 172.21.15.64:0/3991298322 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6694+0+0 (secure 0 0 0) 0x7fcbac0e3130 con 0x7fcbb0103230 2024-05-07T02:24:30.645 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:30.645+0000 7fcbb4e66700 1 --2- 172.21.15.64:0/3991298322 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fcb9c0b2880 0x7fcb9c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:24:30.646 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:30.646+0000 7fcbb4e66700 1 --2- 172.21.15.64:0/3991298322 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fcb9c0b2880 0x7fcb9c0b4d90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcbb019ba70 tx=0x7fcba4005c80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:24:30.648 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:30.648+0000 7fcba27fc700 1 -- 172.21.15.64:0/3991298322 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcbac031020 con 0x7fcbb0103230 2024-05-07T02:24:30.872 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:30.872+0000 7fcbb78cb700 1 -- 172.21.15.64:0/3991298322 --> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fcbb0100910 con 0x7fcb9c0b2880 2024-05-07T02:24:30.872 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:30 smithi064 ceph-mon[34699]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 723 MiB used, 983 GiB / 983 GiB avail 2024-05-07T02:24:30.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:30 smithi019 ceph-mon[29781]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 723 MiB used, 983 GiB / 983 GiB avail 2024-05-07T02:24:30.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:30 smithi027 ceph-mon[34398]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 723 MiB used, 983 GiB / 983 GiB avail 2024-05-07T02:24:31.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:31 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-07T02:24:31.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:31 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-07T02:24:31.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:31 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:24:31.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:31 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-07T02:24:31.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:31 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-07T02:24:31.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:31 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:24:31.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:31 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-07T02:24:31.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:31 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-07T02:24:31.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:31 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:24:32.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:32 smithi064 ceph-mon[34699]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:24:32.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:32 smithi064 ceph-mon[34699]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 722 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-07T02:24:32.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:32 smithi019 ceph-mon[29781]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:24:32.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:32 smithi019 ceph-mon[29781]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 722 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-07T02:24:32.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:32 smithi027 ceph-mon[34398]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:24:32.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:32 smithi027 ceph-mon[34398]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 722 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-07T02:24:34.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:34 smithi019 ceph-mon[29781]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-07T02:24:34.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:34 smithi019 ceph-mon[29781]: from='client.? 172.21.15.64:0/1345763388' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3330627f-ab5d-4e6e-83cf-fc8b74c54ee3"}]: dispatch 2024-05-07T02:24:34.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:34 smithi019 ceph-mon[29781]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3330627f-ab5d-4e6e-83cf-fc8b74c54ee3"}]: dispatch 2024-05-07T02:24:34.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:34 smithi019 ceph-mon[29781]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3330627f-ab5d-4e6e-83cf-fc8b74c54ee3"}]': finished 2024-05-07T02:24:34.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:34 smithi019 ceph-mon[29781]: osdmap e67: 12 total, 11 up, 12 in 2024-05-07T02:24:34.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:34 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-07T02:24:34.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:34 smithi019 ceph-mon[29781]: from='client.? 172.21.15.64:0/1317885595' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-07T02:24:34.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:34 smithi027 ceph-mon[34398]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-07T02:24:34.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:34 smithi027 ceph-mon[34398]: from='client.? 172.21.15.64:0/1345763388' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3330627f-ab5d-4e6e-83cf-fc8b74c54ee3"}]: dispatch 2024-05-07T02:24:34.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:34 smithi027 ceph-mon[34398]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3330627f-ab5d-4e6e-83cf-fc8b74c54ee3"}]: dispatch 2024-05-07T02:24:34.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:34 smithi027 ceph-mon[34398]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3330627f-ab5d-4e6e-83cf-fc8b74c54ee3"}]': finished 2024-05-07T02:24:34.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:34 smithi027 ceph-mon[34398]: osdmap e67: 12 total, 11 up, 12 in 2024-05-07T02:24:34.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:34 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-07T02:24:34.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:34 smithi027 ceph-mon[34398]: from='client.? 172.21.15.64:0/1317885595' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-07T02:24:35.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:34 smithi064 ceph-mon[34699]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-07T02:24:35.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:34 smithi064 ceph-mon[34699]: from='client.? 172.21.15.64:0/1345763388' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3330627f-ab5d-4e6e-83cf-fc8b74c54ee3"}]: dispatch 2024-05-07T02:24:35.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:34 smithi064 ceph-mon[34699]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3330627f-ab5d-4e6e-83cf-fc8b74c54ee3"}]: dispatch 2024-05-07T02:24:35.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:34 smithi064 ceph-mon[34699]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3330627f-ab5d-4e6e-83cf-fc8b74c54ee3"}]': finished 2024-05-07T02:24:35.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:34 smithi064 ceph-mon[34699]: osdmap e67: 12 total, 11 up, 12 in 2024-05-07T02:24:35.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:34 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-07T02:24:35.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:34 smithi064 ceph-mon[34699]: from='client.? 172.21.15.64:0/1317885595' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-07T02:24:36.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:36 smithi019 ceph-mon[29781]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-07T02:24:36.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:36 smithi027 ceph-mon[34398]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-07T02:24:37.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:36 smithi064 ceph-mon[34699]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-07T02:24:38.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:38 smithi027 ceph-mon[34398]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-07T02:24:39.006 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:38 smithi064 ceph-mon[34699]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-07T02:24:39.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:38 smithi019 ceph-mon[29781]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-07T02:24:40.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:40 smithi027 ceph-mon[34398]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-07T02:24:41.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:40 smithi064 ceph-mon[34699]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-07T02:24:41.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:40 smithi019 ceph-mon[29781]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-07T02:24:41.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:41 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-07T02:24:41.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:41 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:24:41.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:41 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-07T02:24:41.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:41 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:24:42.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:41 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-07T02:24:42.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:41 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:24:42.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:42 smithi027 ceph-mon[34398]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-07T02:24:42.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:42 smithi027 ceph-mon[34398]: Deploying daemon osd.11 on smithi064 2024-05-07T02:24:43.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:42 smithi064 ceph-mon[34699]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-07T02:24:43.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:42 smithi064 ceph-mon[34699]: Deploying daemon osd.11 on smithi064 2024-05-07T02:24:43.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:42 smithi019 ceph-mon[29781]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-07T02:24:43.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:42 smithi019 ceph-mon[29781]: Deploying daemon osd.11 on smithi064 2024-05-07T02:24:44.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:44 smithi027 ceph-mon[34398]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-07T02:24:44.971 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:44 smithi064 ceph-mon[34699]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-07T02:24:45.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:44 smithi019 ceph-mon[29781]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-07T02:24:46.774 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:46 smithi064 ceph-mon[34699]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-07T02:24:46.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:46 smithi027 ceph-mon[34398]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-07T02:24:47.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:46 smithi019 ceph-mon[29781]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-07T02:24:48.853 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:48 smithi064 ceph-mon[34699]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-07T02:24:48.854 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:48 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:48.854 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:48 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:24:48.854 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:48 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:48.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:48 smithi027 ceph-mon[34398]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-07T02:24:48.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:48 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:48.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:48 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:24:48.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:48 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:49.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:48 smithi019 ceph-mon[29781]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-07T02:24:49.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:48 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:49.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:48 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:24:49.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:48 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:50.761 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:50 smithi064 ceph-mon[34699]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-07T02:24:50.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:50 smithi027 ceph-mon[34398]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-07T02:24:51.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:50 smithi019 ceph-mon[29781]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-07T02:24:52.123 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 11 on host 'smithi064' 2024-05-07T02:24:52.123 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:52.123+0000 7fcba27fc700 1 -- 172.21.15.64:0/3991298322 <== mgr.14152 v2:172.21.15.19:6800/2059564551 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fcbb0100910 con 0x7fcb9c0b2880 2024-05-07T02:24:52.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:52.125+0000 7fcbb78cb700 1 -- 172.21.15.64:0/3991298322 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fcb9c0b2880 msgr2=0x7fcb9c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:24:52.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:52.125+0000 7fcbb78cb700 1 --2- 172.21.15.64:0/3991298322 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fcb9c0b2880 0x7fcb9c0b4d90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcbb019ba70 tx=0x7fcba4005c80 comp rx=0 tx=0).stop 2024-05-07T02:24:52.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:52.125+0000 7fcbb78cb700 1 -- 172.21.15.64:0/3991298322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbb0103230 msgr2=0x7fcbb019a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:24:52.125 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:52.125+0000 7fcbb78cb700 1 --2- 172.21.15.64:0/3991298322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbb0103230 0x7fcbb019a9c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcbac0098e0 tx=0x7fcbac0047b0 comp rx=0 tx=0).stop 2024-05-07T02:24:52.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:52.125+0000 7fcbb78cb700 1 -- 172.21.15.64:0/3991298322 shutdown_connections 2024-05-07T02:24:52.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:52.125+0000 7fcbb78cb700 1 --2- 172.21.15.64:0/3991298322 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fcb9c0b2880 0x7fcb9c0b4d90 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:24:52.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:52.125+0000 7fcbb78cb700 1 --2- 172.21.15.64:0/3991298322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbb0104d00 0x7fcbb01a1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:24:52.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:52.125+0000 7fcbb78cb700 1 --2- 172.21.15.64:0/3991298322 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbb0103230 0x7fcbb019a9c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:24:52.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:52.125+0000 7fcbb78cb700 1 --2- 172.21.15.64:0/3991298322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbb0104360 0x7fcbb019af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:24:52.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:52.125+0000 7fcbb78cb700 1 -- 172.21.15.64:0/3991298322 >> 172.21.15.64:0/3991298322 conn(0x7fcbb00fe870 msgr2=0x7fcbb0100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:24:52.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:52.126+0000 7fcbb78cb700 1 -- 172.21.15.64:0/3991298322 shutdown_connections 2024-05-07T02:24:52.126 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:24:52.126+0000 7fcbb78cb700 1 -- 172.21.15.64:0/3991298322 wait complete. 2024-05-07T02:24:52.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:52 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:52.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:52 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:52.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:52 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:24:52.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:52 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:24:52.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:52 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:52.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:52 smithi064 ceph-mon[34699]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-07T02:24:52.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:52 smithi064 ceph-mon[34699]: from='osd.11 [v2:172.21.15.64:6824/4045070679,v1:172.21.15.64:6825/4045070679]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-07T02:24:52.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:52 smithi064 ceph-mon[34699]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-07T02:24:52.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:52 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:52.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:52 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:52.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:52 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:24:52.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:52 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:52.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:52 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:52.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:52 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:24:52.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:52 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:24:52.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:52 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:52.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:52 smithi027 ceph-mon[34398]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-07T02:24:52.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:52 smithi027 ceph-mon[34398]: from='osd.11 [v2:172.21.15.64:6824/4045070679,v1:172.21.15.64:6825/4045070679]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-07T02:24:52.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:52 smithi027 ceph-mon[34398]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-07T02:24:52.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:52 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:52.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:52 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:52.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:52 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:24:52.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:52 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:52.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:52 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:52.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:52 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:24:52.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:52 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:24:52.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:52 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:52.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:52 smithi019 ceph-mon[29781]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-07T02:24:52.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:52 smithi019 ceph-mon[29781]: from='osd.11 [v2:172.21.15.64:6824/4045070679,v1:172.21.15.64:6825/4045070679]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-07T02:24:52.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:52 smithi019 ceph-mon[29781]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-07T02:24:52.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:52 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:52.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:52 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:52.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:52 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:24:52.850 DEBUG:teuthology.orchestra.run.smithi064:osd.11> sudo journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@osd.11.service 2024-05-07T02:24:52.852 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-07T02:24:52.853 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph osd stat -f json 2024-05-07T02:24:53.324 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:24:53.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:53 smithi064 ceph-mon[34699]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-07T02:24:53.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:53 smithi064 ceph-mon[34699]: from='osd.11 [v2:172.21.15.64:6824/4045070679,v1:172.21.15.64:6825/4045070679]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-07T02:24:53.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:53 smithi064 ceph-mon[34699]: osdmap e68: 12 total, 11 up, 12 in 2024-05-07T02:24:53.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:53 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-07T02:24:53.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:53 smithi064 ceph-mon[34699]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-07T02:24:53.602 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:53 smithi019 ceph-mon[29781]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-07T02:24:53.602 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:53 smithi019 ceph-mon[29781]: from='osd.11 [v2:172.21.15.64:6824/4045070679,v1:172.21.15.64:6825/4045070679]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-07T02:24:53.602 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:53 smithi019 ceph-mon[29781]: osdmap e68: 12 total, 11 up, 12 in 2024-05-07T02:24:53.602 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:53 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-07T02:24:53.602 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:53 smithi019 ceph-mon[29781]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-07T02:24:53.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:53 smithi027 ceph-mon[34398]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-07T02:24:53.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:53 smithi027 ceph-mon[34398]: from='osd.11 [v2:172.21.15.64:6824/4045070679,v1:172.21.15.64:6825/4045070679]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-07T02:24:53.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:53 smithi027 ceph-mon[34398]: osdmap e68: 12 total, 11 up, 12 in 2024-05-07T02:24:53.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:53 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-07T02:24:53.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:53 smithi027 ceph-mon[34398]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-07T02:24:54.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:54 smithi064 ceph-mon[34699]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-07T02:24:54.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:54 smithi064 ceph-mon[34699]: osdmap e69: 12 total, 11 up, 12 in 2024-05-07T02:24:54.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:54 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-07T02:24:54.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:54 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-07T02:24:54.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:54 smithi064 ceph-mon[34699]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-07T02:24:54.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:54 smithi064 ceph-mon[34699]: from='osd.11 ' entity='osd.11' 2024-05-07T02:24:54.359 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:24:54 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:24:54.019+0000 7f74258a6700 -1 osd.11 0 waiting for initial osdmap 2024-05-07T02:24:54.359 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:24:54 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:24:54.035+0000 7f741d653700 -1 osd.11 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-07T02:24:54.534 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:54 smithi019 ceph-mon[29781]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-07T02:24:54.534 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:54 smithi019 ceph-mon[29781]: osdmap e69: 12 total, 11 up, 12 in 2024-05-07T02:24:54.534 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:54 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-07T02:24:54.534 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:54 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-07T02:24:54.534 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:54 smithi019 ceph-mon[29781]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-07T02:24:54.534 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:54 smithi019 ceph-mon[29781]: from='osd.11 ' entity='osd.11' 2024-05-07T02:24:54.608 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:54 smithi027 ceph-mon[34398]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-07T02:24:54.608 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:54 smithi027 ceph-mon[34398]: osdmap e69: 12 total, 11 up, 12 in 2024-05-07T02:24:54.608 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:54 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-07T02:24:54.608 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:54 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-07T02:24:54.608 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:54 smithi027 ceph-mon[34398]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-07T02:24:54.608 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:54 smithi027 ceph-mon[34398]: from='osd.11 ' entity='osd.11' 2024-05-07T02:24:55.532 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:55 smithi019 ceph-mon[29781]: purged_snaps scrub starts 2024-05-07T02:24:55.532 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:55 smithi019 ceph-mon[29781]: purged_snaps scrub ok 2024-05-07T02:24:55.532 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:55 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-07T02:24:55.532 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:55 smithi019 ceph-mon[29781]: osd.11 [v2:172.21.15.64:6824/4045070679,v1:172.21.15.64:6825/4045070679] boot 2024-05-07T02:24:55.532 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:55 smithi019 ceph-mon[29781]: osdmap e70: 12 total, 12 up, 12 in 2024-05-07T02:24:55.532 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:55 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-07T02:24:55.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:55 smithi064 ceph-mon[34699]: purged_snaps scrub starts 2024-05-07T02:24:55.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:55 smithi064 ceph-mon[34699]: purged_snaps scrub ok 2024-05-07T02:24:55.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:55 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-07T02:24:55.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:55 smithi064 ceph-mon[34699]: osd.11 [v2:172.21.15.64:6824/4045070679,v1:172.21.15.64:6825/4045070679] boot 2024-05-07T02:24:55.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:55 smithi064 ceph-mon[34699]: osdmap e70: 12 total, 12 up, 12 in 2024-05-07T02:24:55.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:55 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-07T02:24:55.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:55 smithi027 ceph-mon[34398]: purged_snaps scrub starts 2024-05-07T02:24:55.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:55 smithi027 ceph-mon[34398]: purged_snaps scrub ok 2024-05-07T02:24:55.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:55 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-07T02:24:55.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:55 smithi027 ceph-mon[34398]: osd.11 [v2:172.21.15.64:6824/4045070679,v1:172.21.15.64:6825/4045070679] boot 2024-05-07T02:24:55.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:55 smithi027 ceph-mon[34398]: osdmap e70: 12 total, 12 up, 12 in 2024-05-07T02:24:55.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:55 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-07T02:24:55.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:55.900+0000 7fa97a200700 1 -- 172.21.15.19:0/2996378907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9741077a0 msgr2=0x7fa974109bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:24:55.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:55.900+0000 7fa97a200700 1 --2- 172.21.15.19:0/2996378907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9741077a0 0x7fa974109bc0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa960009990 tx=0x7fa960009ca0 comp rx=0 tx=0).stop 2024-05-07T02:24:55.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:55.901+0000 7fa97a200700 1 -- 172.21.15.19:0/2996378907 shutdown_connections 2024-05-07T02:24:55.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:55.901+0000 7fa97a200700 1 --2- 172.21.15.19:0/2996378907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa97410a100 0x7fa97410c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:24:55.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:55.901+0000 7fa97a200700 1 --2- 172.21.15.19:0/2996378907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa974104e40 0x7fa974107260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:24:55.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:55.901+0000 7fa97a200700 1 --2- 172.21.15.19:0/2996378907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa9741077a0 0x7fa974109bc0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:24:55.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:55.901+0000 7fa97a200700 1 -- 172.21.15.19:0/2996378907 >> 172.21.15.19:0/2996378907 conn(0x7fa9740fe880 msgr2=0x7fa974100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:24:55.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:55.901+0000 7fa97a200700 1 -- 172.21.15.19:0/2996378907 shutdown_connections 2024-05-07T02:24:55.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:55.901+0000 7fa97a200700 1 -- 172.21.15.19:0/2996378907 wait complete. 2024-05-07T02:24:55.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:55.902+0000 7fa97a200700 1 Processor -- start 2024-05-07T02:24:55.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:55.902+0000 7fa97a200700 1 -- start start 2024-05-07T02:24:55.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:55.903+0000 7fa97a200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa974104e40 0x7fa97419a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:24:55.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:55.903+0000 7fa97a200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9741077a0 0x7fa97419ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:24:55.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:55.903+0000 7fa97a200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa97410a100 0x7fa9741a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:24:55.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:55.903+0000 7fa97a200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa97410fda0 con 0x7fa9741077a0 2024-05-07T02:24:55.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:55.903+0000 7fa97a200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa97410fc20 con 0x7fa974104e40 2024-05-07T02:24:55.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:55.903+0000 7fa97a200700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa97410ff20 con 0x7fa97410a100 2024-05-07T02:24:55.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:55.903+0000 7fa973400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa974104e40 0x7fa97419a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:24:55.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:55.903+0000 7fa973400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa974104e40 0x7fa97419a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:59520/0 (socket says 172.21.15.19:59520) 2024-05-07T02:24:55.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:55.903+0000 7fa973400700 1 -- 172.21.15.19:0/2952516505 learned_addr learned my addr 172.21.15.19:0/2952516505 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:24:55.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:55.903+0000 7fa973e00700 1 --2- 172.21.15.19:0/2952516505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa97410a100 0x7fa9741a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:24:55.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:55.903+0000 7fa972a00700 1 --2- 172.21.15.19:0/2952516505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9741077a0 0x7fa97419ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:24:55.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:55.904+0000 7fa973e00700 1 -- 172.21.15.19:0/2952516505 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa974104e40 msgr2=0x7fa97419a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:24:55.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:55.904+0000 7fa973e00700 1 --2- 172.21.15.19:0/2952516505 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa974104e40 0x7fa97419a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:24:55.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:55.904+0000 7fa973e00700 1 -- 172.21.15.19:0/2952516505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9741077a0 msgr2=0x7fa97419ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:24:55.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:55.904+0000 7fa973e00700 1 --2- 172.21.15.19:0/2952516505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9741077a0 0x7fa97419ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:24:55.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:55.904+0000 7fa973e00700 1 -- 172.21.15.19:0/2952516505 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9741a2440 con 0x7fa97410a100 2024-05-07T02:24:55.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:55.904+0000 7fa973e00700 1 --2- 172.21.15.19:0/2952516505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa97410a100 0x7fa9741a1dc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa96c00efc0 tx=0x7fa96c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:24:55.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:55.904+0000 7fa96be00700 1 -- 172.21.15.19:0/2952516505 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c0092e0 con 0x7fa97410a100 2024-05-07T02:24:55.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:55.905+0000 7fa96be00700 1 -- 172.21.15.19:0/2952516505 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa96c004bb0 con 0x7fa97410a100 2024-05-07T02:24:55.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:55.905+0000 7fa97a200700 1 -- 172.21.15.19:0/2952516505 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9741a26d0 con 0x7fa97410a100 2024-05-07T02:24:55.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:55.905+0000 7fa96be00700 1 -- 172.21.15.19:0/2952516505 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c004020 con 0x7fa97410a100 2024-05-07T02:24:55.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:55.905+0000 7fa97a200700 1 -- 172.21.15.19:0/2952516505 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9741a2bc0 con 0x7fa97410a100 2024-05-07T02:24:55.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:55.906+0000 7fa97a200700 1 -- 172.21.15.19:0/2952516505 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9740512c0 con 0x7fa97410a100 2024-05-07T02:24:55.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:55.908+0000 7fa96be00700 1 -- 172.21.15.19:0/2952516505 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fa96c01b030 con 0x7fa97410a100 2024-05-07T02:24:55.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:55.910+0000 7fa96be00700 1 --2- 172.21.15.19:0/2952516505 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fa9580b2880 0x7fa9580b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:24:55.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:55.910+0000 7fa96be00700 1 -- 172.21.15.19:0/2952516505 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7fa96c0e18f0 con 0x7fa97410a100 2024-05-07T02:24:55.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:55.910+0000 7fa973400700 1 --2- 172.21.15.19:0/2952516505 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fa9580b2880 0x7fa9580b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:24:55.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:55.911+0000 7fa973400700 1 --2- 172.21.15.19:0/2952516505 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fa9580b2880 0x7fa9580b4d90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa964005950 tx=0x7fa964009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:24:55.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:55.914+0000 7fa96be00700 1 -- 172.21.15.19:0/2952516505 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa96c022020 con 0x7fa97410a100 2024-05-07T02:24:56.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:56.154+0000 7fa97a200700 1 -- 172.21.15.19:0/2952516505 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fa9740643d0 con 0x7fa97410a100 2024-05-07T02:24:56.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:56.155+0000 7fa96be00700 1 -- 172.21.15.19:0/2952516505 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v70) ==== 74+0+133 (secure 0 0 0) 0x7fa96c010040 con 0x7fa97410a100 2024-05-07T02:24:56.155 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:24:56.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:56.158+0000 7fa97a200700 1 -- 172.21.15.19:0/2952516505 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fa9580b2880 msgr2=0x7fa9580b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:24:56.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:56.158+0000 7fa97a200700 1 --2- 172.21.15.19:0/2952516505 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fa9580b2880 0x7fa9580b4d90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa964005950 tx=0x7fa964009260 comp rx=0 tx=0).stop 2024-05-07T02:24:56.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:56.158+0000 7fa97a200700 1 -- 172.21.15.19:0/2952516505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa97410a100 msgr2=0x7fa9741a1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:24:56.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:56.158+0000 7fa97a200700 1 --2- 172.21.15.19:0/2952516505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa97410a100 0x7fa9741a1dc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa96c00efc0 tx=0x7fa96c00c3a0 comp rx=0 tx=0).stop 2024-05-07T02:24:56.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:56.159+0000 7fa97a200700 1 -- 172.21.15.19:0/2952516505 shutdown_connections 2024-05-07T02:24:56.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:56.159+0000 7fa97a200700 1 --2- 172.21.15.19:0/2952516505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa97410a100 0x7fa9741a1dc0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:24:56.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:56.159+0000 7fa97a200700 1 --2- 172.21.15.19:0/2952516505 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fa9580b2880 0x7fa9580b4d90 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:24:56.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:56.159+0000 7fa97a200700 1 --2- 172.21.15.19:0/2952516505 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa974104e40 0x7fa97419a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:24:56.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:56.159+0000 7fa97a200700 1 --2- 172.21.15.19:0/2952516505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9741077a0 0x7fa97419ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:24:56.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:56.159+0000 7fa97a200700 1 -- 172.21.15.19:0/2952516505 >> 172.21.15.19:0/2952516505 conn(0x7fa9740fe880 msgr2=0x7fa974105910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:24:56.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:56.159+0000 7fa97a200700 1 -- 172.21.15.19:0/2952516505 shutdown_connections 2024-05-07T02:24:56.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:56.159+0000 7fa97a200700 1 -- 172.21.15.19:0/2952516505 wait complete. 2024-05-07T02:24:56.606 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:56 smithi019 ceph-mon[29781]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:24:56.606 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:56 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/2952516505' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-07T02:24:56.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:56 smithi064 ceph-mon[34699]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:24:56.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:56 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/2952516505' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-07T02:24:56.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:56 smithi027 ceph-mon[34398]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:24:56.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:56 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/2952516505' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-07T02:24:56.988 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":70,"num_osds":12,"num_up_osds":12,"osd_up_since":1715048695,"num_in_osds":12,"osd_in_since":1715048673,"num_remapped_pgs":0} 2024-05-07T02:24:56.989 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-07T02:24:57.443 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:24:57.598 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:57 smithi019 ceph-mon[29781]: osdmap e71: 12 total, 12 up, 12 in 2024-05-07T02:24:57.598 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:57 smithi019 ceph-mon[29781]: Detected new or changed devices on smithi064 2024-05-07T02:24:57.598 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:57 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:57.598 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:57 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:57.598 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:57 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:24:57.599 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:57 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:24:57.599 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:57 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:24:57.599 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:57 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:24:57.599 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:57 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:57.599 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:57 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:24:57.599 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:57 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:24:57.599 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:57 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:57.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:57 smithi064 ceph-mon[34699]: osdmap e71: 12 total, 12 up, 12 in 2024-05-07T02:24:57.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:57 smithi064 ceph-mon[34699]: Detected new or changed devices on smithi064 2024-05-07T02:24:57.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:57 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:57.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:57 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:57.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:57 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:24:57.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:57 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:24:57.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:57 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:24:57.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:57 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:24:57.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:57 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:57.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:57 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:24:57.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:57 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:24:57.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:57 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:57.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:57 smithi027 ceph-mon[34398]: osdmap e71: 12 total, 12 up, 12 in 2024-05-07T02:24:57.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:57 smithi027 ceph-mon[34398]: Detected new or changed devices on smithi064 2024-05-07T02:24:57.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:57 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:57.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:57 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:57.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:57 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:24:57.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:57 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:24:57.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:57 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:24:57.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:57 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:24:57.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:57 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:57.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:57 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:24:57.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:57 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:24:57.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:57 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:24:58.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:58 smithi064 ceph-mon[34699]: Adjusting osd_memory_target on smithi064 to 4272M 2024-05-07T02:24:58.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:24:58 smithi064 ceph-mon[34699]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:24:58.616 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:58 smithi019 ceph-mon[29781]: Adjusting osd_memory_target on smithi064 to 4272M 2024-05-07T02:24:58.616 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:24:58 smithi019 ceph-mon[29781]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:24:58.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:58 smithi027 ceph-mon[34398]: Adjusting osd_memory_target on smithi064 to 4272M 2024-05-07T02:24:58.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:24:58 smithi027 ceph-mon[34398]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:24:59.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:59.766+0000 7f9827200700 1 -- 172.21.15.19:0/517363558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f982010a7c0 msgr2=0x7f982010abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:24:59.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:59.766+0000 7f9827200700 1 --2- 172.21.15.19:0/517363558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f982010a7c0 0x7f982010abd0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9810009990 tx=0x7f9810009ca0 comp rx=0 tx=0).stop 2024-05-07T02:24:59.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:59.766+0000 7f9827200700 1 -- 172.21.15.19:0/517363558 shutdown_connections 2024-05-07T02:24:59.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:59.766+0000 7f9827200700 1 --2- 172.21.15.19:0/517363558 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f982010b110 0x7f9820117620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:24:59.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:59.766+0000 7f9827200700 1 --2- 172.21.15.19:0/517363558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f982010f6c0 0x7f982010fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:24:59.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:59.766+0000 7f9827200700 1 --2- 172.21.15.19:0/517363558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f982010a7c0 0x7f982010abd0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:24:59.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:59.766+0000 7f9827200700 1 -- 172.21.15.19:0/517363558 >> 172.21.15.19:0/517363558 conn(0x7f9820076b00 msgr2=0x7f9820076f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:24:59.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:59.767+0000 7f9827200700 1 -- 172.21.15.19:0/517363558 shutdown_connections 2024-05-07T02:24:59.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:59.767+0000 7f9827200700 1 -- 172.21.15.19:0/517363558 wait complete. 2024-05-07T02:24:59.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:59.768+0000 7f9827200700 1 Processor -- start 2024-05-07T02:24:59.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:59.768+0000 7f9827200700 1 -- start start 2024-05-07T02:24:59.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:59.769+0000 7f9827200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f982010a7c0 0x7f98201a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:24:59.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:59.769+0000 7f9827200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f982010b110 0x7f98201a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:24:59.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:59.769+0000 7f9827200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f982010f6c0 0x7f98201a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:24:59.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:59.769+0000 7f9827200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f982011a900 con 0x7f982010f6c0 2024-05-07T02:24:59.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:59.769+0000 7f9827200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f982011a780 con 0x7f982010a7c0 2024-05-07T02:24:59.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:59.769+0000 7f9827200700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f982011aa80 con 0x7f982010b110 2024-05-07T02:24:59.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:59.769+0000 7f9825e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f982010a7c0 0x7f98201a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:24:59.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:59.769+0000 7f9825e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f982010a7c0 0x7f98201a1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:34590/0 (socket says 172.21.15.19:34590) 2024-05-07T02:24:59.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:59.769+0000 7f9825e00700 1 -- 172.21.15.19:0/3437002331 learned_addr learned my addr 172.21.15.19:0/3437002331 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:24:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:59.769+0000 7f9825400700 1 --2- 172.21.15.19:0/3437002331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f982010b110 0x7f98201a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:24:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:59.769+0000 7f9826800700 1 --2- 172.21.15.19:0/3437002331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f982010f6c0 0x7f98201a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:24:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:59.770+0000 7f9825e00700 1 -- 172.21.15.19:0/3437002331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f982010b110 msgr2=0x7f98201a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:24:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:59.770+0000 7f9825e00700 1 --2- 172.21.15.19:0/3437002331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f982010b110 0x7f98201a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:24:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:59.770+0000 7f9825e00700 1 -- 172.21.15.19:0/3437002331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f982010f6c0 msgr2=0x7f98201a5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:24:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:59.770+0000 7f9825e00700 1 --2- 172.21.15.19:0/3437002331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f982010f6c0 0x7f98201a5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:24:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:59.770+0000 7f9825e00700 1 -- 172.21.15.19:0/3437002331 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98201a6140 con 0x7f982010a7c0 2024-05-07T02:24:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:59.770+0000 7f9826800700 1 --2- 172.21.15.19:0/3437002331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f982010f6c0 0x7f98201a5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:24:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:59.770+0000 7f9825e00700 1 --2- 172.21.15.19:0/3437002331 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f982010a7c0 0x7f98201a1270 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f981400e9a0 tx=0x7f981400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:24:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:59.770+0000 7f981aa00700 1 -- 172.21.15.19:0/3437002331 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981400cbd0 con 0x7f982010a7c0 2024-05-07T02:24:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:59.771+0000 7f981aa00700 1 -- 172.21.15.19:0/3437002331 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f981400cd70 con 0x7f982010a7c0 2024-05-07T02:24:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:59.771+0000 7f9827200700 1 -- 172.21.15.19:0/3437002331 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98201a6430 con 0x7f982010a7c0 2024-05-07T02:24:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:59.771+0000 7f981aa00700 1 -- 172.21.15.19:0/3437002331 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9814018680 con 0x7f982010a7c0 2024-05-07T02:24:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:59.771+0000 7f9827200700 1 -- 172.21.15.19:0/3437002331 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98201add50 con 0x7f982010a7c0 2024-05-07T02:24:59.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:59.772+0000 7f9827200700 1 -- 172.21.15.19:0/3437002331 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98200512c0 con 0x7f982010a7c0 2024-05-07T02:24:59.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:59.774+0000 7f981aa00700 1 -- 172.21.15.19:0/3437002331 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f981404aab0 con 0x7f982010a7c0 2024-05-07T02:24:59.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:59.776+0000 7f981aa00700 1 --2- 172.21.15.19:0/3437002331 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f98080b2900 0x7f98080b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:24:59.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:59.776+0000 7f981aa00700 1 -- 172.21.15.19:0/3437002331 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f98140e1fe0 con 0x7f982010a7c0 2024-05-07T02:24:59.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:59.776+0000 7f9825400700 1 --2- 172.21.15.19:0/3437002331 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f98080b2900 0x7f98080b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:24:59.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:59.777+0000 7f9825400700 1 --2- 172.21.15.19:0/3437002331 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f98080b2900 0x7f98080b4e10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9820077f90 tx=0x7f981000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:24:59.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:24:59.778+0000 7f981aa00700 1 -- 172.21.15.19:0/3437002331 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f98140aa120 con 0x7f982010a7c0 2024-05-07T02:25:00.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:00.016+0000 7f9827200700 1 -- 172.21.15.19:0/3437002331 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f982002e5f0 con 0x7f982010a7c0 2024-05-07T02:25:00.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:00.018+0000 7f981aa00700 1 -- 172.21.15.19:0/3437002331 <== mon.2 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+18383 (secure 0 0 0) 0x7f98140b21a0 con 0x7f982010a7c0 2024-05-07T02:25:00.018 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:25:00.019 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":71,"fsid":"9ff84c16-0c17-11ef-bc97-c7b262605968","created":"2024-05-07T02:16:19.507657+0000","modified":"2024-05-07T02:24:56.259396+0000","last_up_change":"2024-05-07T02:24:55.025124+0000","last_in_change":"2024-05-07T02:24:33.842422+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-07T02:20:21.439697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b437cf22-ec8e-48f0-86a2-2413a0ab40dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6803","nonce":2390539683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6805","nonce":2390539683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6809","nonce":2390539683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6807","nonce":2390539683}]},"public_addr":"172.21.15.19:6803/2390539683","cluster_addr":"172.21.15.19:6805/2390539683","heartbeat_back_addr":"172.21.15.19:6809/2390539683","heartbeat_front_addr":"172.21.15.19:6807/2390539683","state":["exists","up"]},{"osd":1,"uuid":"dc753e16-634d-4c14-abd2-935281e1499e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6801","nonce":4292470615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6803","nonce":4292470615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6807","nonce":4292470615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6805","nonce":4292470615}]},"public_addr":"172.21.15.27:6801/4292470615","cluster_addr":"172.21.15.27:6803/4292470615","heartbeat_back_addr":"172.21.15.27:6807/4292470615","heartbeat_front_addr":"172.21.15.27:6805/4292470615","state":["exists","up"]},{"osd":2,"uuid":"40570071-646e-4e66-9a16-c55ab743ac86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6801","nonce":1180552939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6803","nonce":1180552939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6807","nonce":1180552939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6805","nonce":1180552939}]},"public_addr":"172.21.15.64:6801/1180552939","cluster_addr":"172.21.15.64:6803/1180552939","heartbeat_back_addr":"172.21.15.64:6807/1180552939","heartbeat_front_addr":"172.21.15.64:6805/1180552939","state":["exists","up"]},{"osd":3,"uuid":"ceaa4252-4dfa-427a-9b1a-76adf3b8e609","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6811","nonce":387118141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6813","nonce":387118141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6817","nonce":387118141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6815","nonce":387118141}]},"public_addr":"172.21.15.19:6811/387118141","cluster_addr":"172.21.15.19:6813/387118141","heartbeat_back_addr":"172.21.15.19:6817/387118141","heartbeat_front_addr":"172.21.15.19:6815/387118141","state":["exists","up"]},{"osd":4,"uuid":"5a6d1ebc-62ca-444e-8f29-1980418d0a8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6809","nonce":3662186068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6811","nonce":3662186068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6815","nonce":3662186068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6813","nonce":3662186068}]},"public_addr":"172.21.15.27:6809/3662186068","cluster_addr":"172.21.15.27:6811/3662186068","heartbeat_back_addr":"172.21.15.27:6815/3662186068","heartbeat_front_addr":"172.21.15.27:6813/3662186068","state":["exists","up"]},{"osd":5,"uuid":"60c207e9-55e8-4bed-87c3-f6a06a2a1fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6809","nonce":1264591849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6811","nonce":1264591849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6815","nonce":1264591849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6813","nonce":1264591849}]},"public_addr":"172.21.15.64:6809/1264591849","cluster_addr":"172.21.15.64:6811/1264591849","heartbeat_back_addr":"172.21.15.64:6815/1264591849","heartbeat_front_addr":"172.21.15.64:6813/1264591849","state":["exists","up"]},{"osd":6,"uuid":"f3e038e0-4026-45f1-a923-c07cbb307f64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6819","nonce":2158875543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6821","nonce":2158875543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6825","nonce":2158875543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6823","nonce":2158875543}]},"public_addr":"172.21.15.19:6819/2158875543","cluster_addr":"172.21.15.19:6821/2158875543","heartbeat_back_addr":"172.21.15.19:6825/2158875543","heartbeat_front_addr":"172.21.15.19:6823/2158875543","state":["exists","up"]},{"osd":7,"uuid":"d69e34d4-2028-43b8-8ada-ff4a9b579620","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6817","nonce":2756612091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6819","nonce":2756612091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6823","nonce":2756612091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6821","nonce":2756612091}]},"public_addr":"172.21.15.27:6817/2756612091","cluster_addr":"172.21.15.27:6819/2756612091","heartbeat_back_addr":"172.21.15.27:6823/2756612091","heartbeat_front_addr":"172.21.15.27:6821/2756612091","state":["exists","up"]},{"osd":8,"uuid":"6c1a4179-75b6-42ff-ac29-378801e6f8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6817","nonce":1987024836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6819","nonce":1987024836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6823","nonce":1987024836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6821","nonce":1987024836}]},"public_addr":"172.21.15.64:6817/1987024836","cluster_addr":"172.21.15.64:6819/1987024836","heartbeat_back_addr":"172.21.15.64:6823/1987024836","heartbeat_front_addr":"172.21.15.64:6821/1987024836","state":["exists","up"]},{"osd":9,"uuid":"9df7ff66-d39f-4709-8324-74cf9e0e052d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6827","nonce":4124921646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6829","nonce":4124921646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6833","nonce":4124921646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6831","nonce":4124921646}]},"public_addr":"172.21.15.19:6827/4124921646","cluster_addr":"172.21.15.19:6829/4124921646","heartbeat_back_addr":"172.21.15.19:6833/4124921646","heartbeat_front_addr":"172.21.15.19:6831/4124921646","state":["exists","up"]},{"osd":10,"uuid":"71eaa9a7-667d-4cb7-935c-42177d20a279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6825","nonce":1428931817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6827","nonce":1428931817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6831","nonce":1428931817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6829","nonce":1428931817}]},"public_addr":"172.21.15.27:6825/1428931817","cluster_addr":"172.21.15.27:6827/1428931817","heartbeat_back_addr":"172.21.15.27:6831/1428931817","heartbeat_front_addr":"172.21.15.27:6829/1428931817","state":["exists","up"]},{"osd":11,"uuid":"3330627f-ab5d-4e6e-83cf-fc8b74c54ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6825","nonce":4045070679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6827","nonce":4045070679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6831","nonce":4045070679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6829","nonce":4045070679}]},"public_addr":"172.21.15.64:6825/4045070679","cluster_addr":"172.21.15.64:6827/4045070679","heartbeat_back_addr":"172.21.15.64:6831/4045070679","heartbeat_front_addr":"172.21.15.64:6829/4045070679","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:19:27.902633+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:19:54.305685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:20:18.727053+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:20:53.579199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:21:21.058018+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:21:47.091494+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:22:23.269018+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:22:51.863628+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:23:18.537774+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:23:55.763993+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:24:24.956178+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:24:52.681201+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/1956516258":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/1816772181":"2024-05-08T02:17:31.252868+0000","172.21.15.19:6801/174304059":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/1514051750":"2024-05-08T02:16:54.732592+0000","172.21.15.19:6800/174304059":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/3930045871":"2024-05-08T02:16:54.732592+0000","172.21.15.19:6800/3614957309":"2024-05-08T02:16:54.732592+0000","172.21.15.19:0/1328365399":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/139593344":"2024-05-08T02:16:54.732592+0000","172.21.15.19:6801/3614957309":"2024-05-08T02:16:54.732592+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-07T02:25:00.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:00.021+0000 7f9827200700 1 -- 172.21.15.19:0/3437002331 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f98080b2900 msgr2=0x7f98080b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:00.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:00.021+0000 7f9827200700 1 --2- 172.21.15.19:0/3437002331 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f98080b2900 0x7f98080b4e10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9820077f90 tx=0x7f981000b260 comp rx=0 tx=0).stop 2024-05-07T02:25:00.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:00.021+0000 7f9827200700 1 -- 172.21.15.19:0/3437002331 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f982010a7c0 msgr2=0x7f98201a1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:00.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:00.021+0000 7f9827200700 1 --2- 172.21.15.19:0/3437002331 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f982010a7c0 0x7f98201a1270 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f981400e9a0 tx=0x7f981400ecb0 comp rx=0 tx=0).stop 2024-05-07T02:25:00.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:00.021+0000 7f9827200700 1 -- 172.21.15.19:0/3437002331 shutdown_connections 2024-05-07T02:25:00.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:00.021+0000 7f9827200700 1 --2- 172.21.15.19:0/3437002331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f982010f6c0 0x7f98201a5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:00.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:00.021+0000 7f9827200700 1 --2- 172.21.15.19:0/3437002331 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f98080b2900 0x7f98080b4e10 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:00.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:00.021+0000 7f9827200700 1 --2- 172.21.15.19:0/3437002331 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f982010a7c0 0x7f98201a1270 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:00.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:00.021+0000 7f9827200700 1 --2- 172.21.15.19:0/3437002331 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f982010b110 0x7f98201a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:00.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:00.021+0000 7f9827200700 1 -- 172.21.15.19:0/3437002331 >> 172.21.15.19:0/3437002331 conn(0x7f9820076b00 msgr2=0x7f98201088c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:25:00.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:00.022+0000 7f9827200700 1 -- 172.21.15.19:0/3437002331 shutdown_connections 2024-05-07T02:25:00.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:00.022+0000 7f9827200700 1 -- 172.21.15.19:0/3437002331 wait complete. 2024-05-07T02:25:00.745 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:00 smithi019 ceph-mon[29781]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:00.745 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:00 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/3437002331' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-07T02:25:00.821 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-07T02:20:21.439697+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-07T02:25:00.822 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-07T02:25:00.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:00 smithi064 ceph-mon[34699]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:00.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:00 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/3437002331' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-07T02:25:00.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:00 smithi027 ceph-mon[34398]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:00.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:00 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/3437002331' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-07T02:25:01.328 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:25:02.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:02 smithi019 ceph-mon[29781]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:02.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:02 smithi064 ceph-mon[34699]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:02.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:02 smithi027 ceph-mon[34398]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:03.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.571+0000 7fdd23c00700 1 -- 172.21.15.19:0/2810009372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd1c10b2e0 msgr2=0x7fdd1c117810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:03.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.571+0000 7fdd23c00700 1 --2- 172.21.15.19:0/2810009372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd1c10b2e0 0x7fdd1c117810 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdd1400b0a0 tx=0x7fdd1400b3b0 comp rx=0 tx=0).stop 2024-05-07T02:25:03.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.571+0000 7fdd23c00700 1 -- 172.21.15.19:0/2810009372 shutdown_connections 2024-05-07T02:25:03.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.571+0000 7fdd23c00700 1 --2- 172.21.15.19:0/2810009372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd1c10b2e0 0x7fdd1c117810 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:03.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.571+0000 7fdd23c00700 1 --2- 172.21.15.19:0/2810009372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd1c110970 0x7fdd1c110d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:03.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.571+0000 7fdd23c00700 1 --2- 172.21.15.19:0/2810009372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd1c10a990 0x7fdd1c10ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:03.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.571+0000 7fdd23c00700 1 -- 172.21.15.19:0/2810009372 >> 172.21.15.19:0/2810009372 conn(0x7fdd1c077c80 msgr2=0x7fdd1c0780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:25:03.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.572+0000 7fdd23c00700 1 -- 172.21.15.19:0/2810009372 shutdown_connections 2024-05-07T02:25:03.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.572+0000 7fdd23c00700 1 -- 172.21.15.19:0/2810009372 wait complete. 2024-05-07T02:25:03.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.573+0000 7fdd23c00700 1 Processor -- start 2024-05-07T02:25:03.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.573+0000 7fdd23c00700 1 -- start start 2024-05-07T02:25:03.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.574+0000 7fdd23c00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd1c10a990 0x7fdd1c1a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:25:03.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.574+0000 7fdd23c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd1c10b2e0 0x7fdd1c1a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:25:03.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.574+0000 7fdd23c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd1c110970 0x7fdd1c1a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:25:03.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.574+0000 7fdd23c00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1c11aa90 con 0x7fdd1c10b2e0 2024-05-07T02:25:03.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.574+0000 7fdd23c00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1c11a910 con 0x7fdd1c110970 2024-05-07T02:25:03.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.574+0000 7fdd23c00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1c11ac10 con 0x7fdd1c10a990 2024-05-07T02:25:03.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.574+0000 7fdd1be00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd1c10b2e0 0x7fdd1c1a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:25:03.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.574+0000 7fdd21c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd1c110970 0x7fdd1c1a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:25:03.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.574+0000 7fdd21c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd1c110970 0x7fdd1c1a5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:34598/0 (socket says 172.21.15.19:34598) 2024-05-07T02:25:03.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.574+0000 7fdd21c00700 1 -- 172.21.15.19:0/3413348887 learned_addr learned my addr 172.21.15.19:0/3413348887 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:25:03.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.574+0000 7fdd21200700 1 --2- 172.21.15.19:0/3413348887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd1c10a990 0x7fdd1c1a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:25:03.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.574+0000 7fdd21c00700 1 -- 172.21.15.19:0/3413348887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd1c10a990 msgr2=0x7fdd1c1a13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:03.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.574+0000 7fdd21c00700 1 --2- 172.21.15.19:0/3413348887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd1c10a990 0x7fdd1c1a13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:03.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.574+0000 7fdd21c00700 1 -- 172.21.15.19:0/3413348887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd1c10b2e0 msgr2=0x7fdd1c1a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:03.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.574+0000 7fdd21c00700 1 --2- 172.21.15.19:0/3413348887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd1c10b2e0 0x7fdd1c1a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:03.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.574+0000 7fdd21c00700 1 -- 172.21.15.19:0/3413348887 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd1c1a6290 con 0x7fdd1c110970 2024-05-07T02:25:03.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.575+0000 7fdd21c00700 1 --2- 172.21.15.19:0/3413348887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd1c110970 0x7fdd1c1a5bf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdd1400b780 tx=0x7fdd14004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:25:03.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.575+0000 7fdd19600700 1 -- 172.21.15.19:0/3413348887 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd14021070 con 0x7fdd1c110970 2024-05-07T02:25:03.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.575+0000 7fdd23c00700 1 -- 172.21.15.19:0/3413348887 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd1c1a6520 con 0x7fdd1c110970 2024-05-07T02:25:03.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.575+0000 7fdd19600700 1 -- 172.21.15.19:0/3413348887 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd14007e30 con 0x7fdd1c110970 2024-05-07T02:25:03.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.575+0000 7fdd19600700 1 -- 172.21.15.19:0/3413348887 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd14004700 con 0x7fdd1c110970 2024-05-07T02:25:03.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.575+0000 7fdd23c00700 1 -- 172.21.15.19:0/3413348887 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd1c1adea0 con 0x7fdd1c110970 2024-05-07T02:25:03.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.576+0000 7fdd23c00700 1 -- 172.21.15.19:0/3413348887 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd1c10c080 con 0x7fdd1c110970 2024-05-07T02:25:03.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.579+0000 7fdd19600700 1 -- 172.21.15.19:0/3413348887 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fdd14007980 con 0x7fdd1c110970 2024-05-07T02:25:03.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.580+0000 7fdd19600700 1 --2- 172.21.15.19:0/3413348887 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fdd040b2880 0x7fdd040b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:25:03.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.580+0000 7fdd19600700 1 -- 172.21.15.19:0/3413348887 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fdd1400f020 con 0x7fdd1c110970 2024-05-07T02:25:03.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.580+0000 7fdd21200700 1 --2- 172.21.15.19:0/3413348887 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fdd040b2880 0x7fdd040b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:25:03.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.581+0000 7fdd21200700 1 --2- 172.21.15.19:0/3413348887 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fdd040b2880 0x7fdd040b4d90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c07d0f0 tx=0x7fdd10009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:25:03.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.585+0000 7fdd19600700 1 -- 172.21.15.19:0/3413348887 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdd1402a020 con 0x7fdd1c110970 2024-05-07T02:25:03.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.821+0000 7fdd23c00700 1 -- 172.21.15.19:0/3413348887 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdd1c1a2720 con 0x7fdd1c110970 2024-05-07T02:25:03.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.821+0000 7fdd19600700 1 -- 172.21.15.19:0/3413348887 <== mon.2 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7fdd140ab350 con 0x7fdd1c110970 2024-05-07T02:25:03.822 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-05-07T02:25:03.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.825+0000 7fdd23c00700 1 -- 172.21.15.19:0/3413348887 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fdd040b2880 msgr2=0x7fdd040b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:03.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.825+0000 7fdd23c00700 1 --2- 172.21.15.19:0/3413348887 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fdd040b2880 0x7fdd040b4d90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c07d0f0 tx=0x7fdd10009260 comp rx=0 tx=0).stop 2024-05-07T02:25:03.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.825+0000 7fdd23c00700 1 -- 172.21.15.19:0/3413348887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd1c110970 msgr2=0x7fdd1c1a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:03.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.825+0000 7fdd23c00700 1 --2- 172.21.15.19:0/3413348887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd1c110970 0x7fdd1c1a5bf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdd1400b780 tx=0x7fdd14004d00 comp rx=0 tx=0).stop 2024-05-07T02:25:03.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.825+0000 7fdd23c00700 1 -- 172.21.15.19:0/3413348887 shutdown_connections 2024-05-07T02:25:03.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.825+0000 7fdd23c00700 1 --2- 172.21.15.19:0/3413348887 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fdd040b2880 0x7fdd040b4d90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:03.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.825+0000 7fdd23c00700 1 --2- 172.21.15.19:0/3413348887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd1c110970 0x7fdd1c1a5bf0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:03.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.825+0000 7fdd23c00700 1 --2- 172.21.15.19:0/3413348887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdd1c10a990 0x7fdd1c1a13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:03.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.825+0000 7fdd23c00700 1 --2- 172.21.15.19:0/3413348887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd1c10b2e0 0x7fdd1c1a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:03.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.825+0000 7fdd23c00700 1 -- 172.21.15.19:0/3413348887 >> 172.21.15.19:0/3413348887 conn(0x7fdd1c077c80 msgr2=0x7fdd1c07b230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:25:03.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.825+0000 7fdd23c00700 1 -- 172.21.15.19:0/3413348887 shutdown_connections 2024-05-07T02:25:03.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:03.825+0000 7fdd23c00700 1 -- 172.21.15.19:0/3413348887 wait complete. 2024-05-07T02:25:04.603 INFO:tasks.cephadm:Adding mds.a on smithi019 2024-05-07T02:25:04.603 INFO:tasks.cephadm:Adding mds.d on smithi019 2024-05-07T02:25:04.603 INFO:tasks.cephadm:Adding mds.g on smithi019 2024-05-07T02:25:04.603 INFO:tasks.cephadm:Adding mds.j on smithi019 2024-05-07T02:25:04.604 INFO:tasks.cephadm:Adding mds.b on smithi027 2024-05-07T02:25:04.604 INFO:tasks.cephadm:Adding mds.e on smithi027 2024-05-07T02:25:04.604 INFO:tasks.cephadm:Adding mds.h on smithi027 2024-05-07T02:25:04.604 INFO:tasks.cephadm:Adding mds.k on smithi027 2024-05-07T02:25:04.604 INFO:tasks.cephadm:Adding mds.c on smithi064 2024-05-07T02:25:04.604 INFO:tasks.cephadm:Adding mds.f on smithi064 2024-05-07T02:25:04.604 INFO:tasks.cephadm:Adding mds.i on smithi064 2024-05-07T02:25:04.604 INFO:tasks.cephadm:Adding mds.l on smithi064 2024-05-07T02:25:04.604 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph orch apply mds all '12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi027=b;smithi027=e;smithi027=h;smithi027=k;smithi064=c;smithi064=f;smithi064=i;smithi064=l' 2024-05-07T02:25:04.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:04 smithi064 ceph-mon[34699]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:04.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:04 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/3413348887' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-07T02:25:04.864 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.c/config 2024-05-07T02:25:04.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:04 smithi019 ceph-mon[29781]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:04.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:04 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/3413348887' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-07T02:25:04.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:04 smithi027 ceph-mon[34398]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:04.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:04 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/3413348887' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-07T02:25:06.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:06 smithi064 ceph-mon[34699]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:06.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.737+0000 7fcd3186e700 1 -- 172.21.15.64:0/2306027667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd2c1054a0 msgr2=0x7fcd2c1058b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:06.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.737+0000 7fcd3186e700 1 --2- 172.21.15.64:0/2306027667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd2c1054a0 0x7fcd2c1058b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcd20009990 tx=0x7fcd20009ca0 comp rx=0 tx=0).stop 2024-05-07T02:25:06.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.737+0000 7fcd3186e700 1 -- 172.21.15.64:0/2306027667 shutdown_connections 2024-05-07T02:25:06.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.737+0000 7fcd3186e700 1 --2- 172.21.15.64:0/2306027667 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd2c105e40 0x7fcd2c10c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:06.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.737+0000 7fcd3186e700 1 --2- 172.21.15.64:0/2306027667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd2c104370 0x7fcd2c104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:06.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.737+0000 7fcd3186e700 1 --2- 172.21.15.64:0/2306027667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd2c1054a0 0x7fcd2c1058b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:06.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.737+0000 7fcd3186e700 1 -- 172.21.15.64:0/2306027667 >> 172.21.15.64:0/2306027667 conn(0x7fcd2c0ff9d0 msgr2=0x7fcd2c101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:25:06.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.737+0000 7fcd3186e700 1 -- 172.21.15.64:0/2306027667 shutdown_connections 2024-05-07T02:25:06.739 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.737+0000 7fcd3186e700 1 -- 172.21.15.64:0/2306027667 wait complete. 2024-05-07T02:25:06.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.738+0000 7fcd3186e700 1 Processor -- start 2024-05-07T02:25:06.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.738+0000 7fcd3186e700 1 -- start start 2024-05-07T02:25:06.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.738+0000 7fcd3186e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd2c104370 0x7fcd2c19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:25:06.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.738+0000 7fcd3186e700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd2c1054a0 0x7fcd2c19aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:25:06.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.738+0000 7fcd3186e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd2c105e40 0x7fcd2c1a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:25:06.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.738+0000 7fcd3186e700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2c10fe50 con 0x7fcd2c1054a0 2024-05-07T02:25:06.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.738+0000 7fcd3186e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2c10fcd0 con 0x7fcd2c104370 2024-05-07T02:25:06.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.738+0000 7fcd3186e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2c10ffd0 con 0x7fcd2c105e40 2024-05-07T02:25:06.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.738+0000 7fcd2b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd2c105e40 0x7fcd2c1a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:25:06.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.738+0000 7fcd2b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd2c105e40 0x7fcd2c1a1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36136/0 (socket says 172.21.15.64:36136) 2024-05-07T02:25:06.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.738+0000 7fcd2b7fe700 1 -- 172.21.15.64:0/2249231049 learned_addr learned my addr 172.21.15.64:0/2249231049 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-07T02:25:06.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.738+0000 7fcd2affd700 1 --2- 172.21.15.64:0/2249231049 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd2c104370 0x7fcd2c19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:25:06.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.738+0000 7fcd2b7fe700 1 -- 172.21.15.64:0/2249231049 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd2c104370 msgr2=0x7fcd2c19a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:06.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.738+0000 7fcd2b7fe700 1 --2- 172.21.15.64:0/2249231049 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd2c104370 0x7fcd2c19a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:06.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.738+0000 7fcd2a7fc700 1 --2- 172.21.15.64:0/2249231049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd2c1054a0 0x7fcd2c19aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:25:06.740 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.738+0000 7fcd2b7fe700 1 -- 172.21.15.64:0/2249231049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd2c1054a0 msgr2=0x7fcd2c19aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:06.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.739+0000 7fcd2b7fe700 1 --2- 172.21.15.64:0/2249231049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd2c1054a0 0x7fcd2c19aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:06.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.739+0000 7fcd2b7fe700 1 -- 172.21.15.64:0/2249231049 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd2c1a2500 con 0x7fcd2c105e40 2024-05-07T02:25:06.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.739+0000 7fcd2b7fe700 1 --2- 172.21.15.64:0/2249231049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd2c105e40 0x7fcd2c1a1e80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcd2400ada0 tx=0x7fcd2400c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:25:06.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.739+0000 7fcd3086c700 1 -- 172.21.15.64:0/2249231049 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd24017020 con 0x7fcd2c105e40 2024-05-07T02:25:06.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.739+0000 7fcd3086c700 1 -- 172.21.15.64:0/2249231049 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd2400ce30 con 0x7fcd2c105e40 2024-05-07T02:25:06.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.739+0000 7fcd3086c700 1 -- 172.21.15.64:0/2249231049 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd24005350 con 0x7fcd2c105e40 2024-05-07T02:25:06.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.739+0000 7fcd3186e700 1 -- 172.21.15.64:0/2249231049 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd2c1a27f0 con 0x7fcd2c105e40 2024-05-07T02:25:06.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.739+0000 7fcd3186e700 1 -- 172.21.15.64:0/2249231049 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd2c1a2db0 con 0x7fcd2c105e40 2024-05-07T02:25:06.741 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.740+0000 7fcd3186e700 1 -- 172.21.15.64:0/2249231049 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd2c0512c0 con 0x7fcd2c105e40 2024-05-07T02:25:06.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.742+0000 7fcd3086c700 1 -- 172.21.15.64:0/2249231049 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fcd2400c980 con 0x7fcd2c105e40 2024-05-07T02:25:06.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.742+0000 7fcd3086c700 1 --2- 172.21.15.64:0/2249231049 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fcd140b2900 0x7fcd140b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:25:06.742 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.743+0000 7fcd3086c700 1 -- 172.21.15.64:0/2249231049 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fcd240e1a10 con 0x7fcd2c105e40 2024-05-07T02:25:06.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.743+0000 7fcd2affd700 1 --2- 172.21.15.64:0/2249231049 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fcd140b2900 0x7fcd140b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:25:06.743 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.744+0000 7fcd2affd700 1 --2- 172.21.15.64:0/2249231049 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fcd140b2900 0x7fcd140b4e10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c1053a0 tx=0x7fcd1c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:25:06.746 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.746+0000 7fcd3086c700 1 -- 172.21.15.64:0/2249231049 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcd24021020 con 0x7fcd2c105e40 2024-05-07T02:25:06.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:06 smithi019 ceph-mon[29781]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:06.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:06 smithi027 ceph-mon[34398]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:06.971 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.971+0000 7fcd3186e700 1 -- 172.21.15.64:0/2249231049 --> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi027=b;smithi027=e;smithi027=h;smithi027=k;smithi064=c;smithi064=f;smithi064=i;smithi064=l", "target": ["mon-mgr", ""]}) -- 0x7fcd2c063ec0 con 0x7fcd140b2900 2024-05-07T02:25:06.990 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.990+0000 7fcd3086c700 1 -- 172.21.15.64:0/2249231049 <== mgr.14152 v2:172.21.15.19:6800/2059564551 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fcd2c063ec0 con 0x7fcd140b2900 2024-05-07T02:25:06.990 INFO:teuthology.orchestra.run.smithi064.stdout:Scheduled mds.all update... 2024-05-07T02:25:06.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.992+0000 7fcd3186e700 1 -- 172.21.15.64:0/2249231049 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fcd140b2900 msgr2=0x7fcd140b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:06.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.992+0000 7fcd3186e700 1 --2- 172.21.15.64:0/2249231049 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fcd140b2900 0x7fcd140b4e10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c1053a0 tx=0x7fcd1c009190 comp rx=0 tx=0).stop 2024-05-07T02:25:06.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.992+0000 7fcd3186e700 1 -- 172.21.15.64:0/2249231049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd2c105e40 msgr2=0x7fcd2c1a1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:06.992 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.992+0000 7fcd3186e700 1 --2- 172.21.15.64:0/2249231049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd2c105e40 0x7fcd2c1a1e80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcd2400ada0 tx=0x7fcd2400c330 comp rx=0 tx=0).stop 2024-05-07T02:25:06.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.992+0000 7fcd3186e700 1 -- 172.21.15.64:0/2249231049 shutdown_connections 2024-05-07T02:25:06.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.992+0000 7fcd3186e700 1 --2- 172.21.15.64:0/2249231049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd2c105e40 0x7fcd2c1a1e80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:06.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.992+0000 7fcd3186e700 1 --2- 172.21.15.64:0/2249231049 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fcd140b2900 0x7fcd140b4e10 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:06.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.992+0000 7fcd3186e700 1 --2- 172.21.15.64:0/2249231049 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd2c104370 0x7fcd2c19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:06.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.992+0000 7fcd3186e700 1 --2- 172.21.15.64:0/2249231049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd2c1054a0 0x7fcd2c19aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:06.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.992+0000 7fcd3186e700 1 -- 172.21.15.64:0/2249231049 >> 172.21.15.64:0/2249231049 conn(0x7fcd2c0ff9d0 msgr2=0x7fcd2c1086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:25:06.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.993+0000 7fcd3186e700 1 -- 172.21.15.64:0/2249231049 shutdown_connections 2024-05-07T02:25:06.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-07T02:25:06.993+0000 7fcd3186e700 1 -- 172.21.15.64:0/2249231049 wait complete. 2024-05-07T02:25:07.825 DEBUG:teuthology.orchestra.run.smithi019:mds.a> sudo journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mds.a.service 2024-05-07T02:25:07.832 DEBUG:teuthology.orchestra.run.smithi019:mds.d> sudo journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mds.d.service 2024-05-07T02:25:07.879 DEBUG:teuthology.orchestra.run.smithi019:mds.g> sudo journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mds.g.service 2024-05-07T02:25:07.925 DEBUG:teuthology.orchestra.run.smithi019:mds.j> sudo journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mds.j.service 2024-05-07T02:25:07.970 DEBUG:teuthology.orchestra.run.smithi027:mds.b> sudo journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mds.b.service 2024-05-07T02:25:07.973 DEBUG:teuthology.orchestra.run.smithi027:mds.e> sudo journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mds.e.service 2024-05-07T02:25:08.016 DEBUG:teuthology.orchestra.run.smithi027:mds.h> sudo journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mds.h.service 2024-05-07T02:25:08.058 DEBUG:teuthology.orchestra.run.smithi027:mds.k> sudo journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mds.k.service 2024-05-07T02:25:08.101 DEBUG:teuthology.orchestra.run.smithi064:mds.c> sudo journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mds.c.service 2024-05-07T02:25:08.104 DEBUG:teuthology.orchestra.run.smithi064:mds.f> sudo journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mds.f.service 2024-05-07T02:25:08.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:07 smithi064 ceph-mon[34699]: from='client.24442 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi027=b;smithi027=e;smithi027=h;smithi027=k;smithi064=c;smithi064=f;smithi064=i;smithi064=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:25:08.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:07 smithi064 ceph-mon[34699]: Saving service mds.all spec with placement smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi027=b;smithi027=e;smithi027=h;smithi027=k;smithi064=c;smithi064=f;smithi064=i;smithi064=l;count:12 2024-05-07T02:25:08.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:07 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:08.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:07 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:25:08.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:07 smithi064 ceph-mon[34699]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:08.109 DEBUG:teuthology.orchestra.run.smithi064:mds.i> sudo journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mds.i.service 2024-05-07T02:25:08.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:07 smithi019 ceph-mon[29781]: from='client.24442 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi027=b;smithi027=e;smithi027=h;smithi027=k;smithi064=c;smithi064=f;smithi064=i;smithi064=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:25:08.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:07 smithi019 ceph-mon[29781]: Saving service mds.all spec with placement smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi027=b;smithi027=e;smithi027=h;smithi027=k;smithi064=c;smithi064=f;smithi064=i;smithi064=l;count:12 2024-05-07T02:25:08.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:07 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:08.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:07 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:25:08.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:07 smithi019 ceph-mon[29781]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:08.151 DEBUG:teuthology.orchestra.run.smithi064:mds.l> sudo journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mds.l.service 2024-05-07T02:25:08.194 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-07T02:25:08.195 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-07T02:25:08.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:07 smithi027 ceph-mon[34398]: from='client.24442 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi027=b;smithi027=e;smithi027=h;smithi027=k;smithi064=c;smithi064=f;smithi064=i;smithi064=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:25:08.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:07 smithi027 ceph-mon[34398]: Saving service mds.all spec with placement smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi027=b;smithi027=e;smithi027=h;smithi027=k;smithi064=c;smithi064=f;smithi064=i;smithi064=l;count:12 2024-05-07T02:25:08.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:07 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:08.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:07 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:25:08.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:07 smithi027 ceph-mon[34398]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:08.655 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:25:09.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:08 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:25:09.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:08 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:25:09.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:08 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:09.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:08 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-07T02:25:09.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:08 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-07T02:25:09.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:08 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:25:09.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:08 smithi019 ceph-mon[29781]: Deploying daemon mds.l on smithi064 2024-05-07T02:25:09.201 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:08 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:25:09.201 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:08 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:25:09.201 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:08 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:09.201 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:08 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-07T02:25:09.201 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:08 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-07T02:25:09.201 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:08 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:25:09.201 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:08 smithi064 ceph-mon[34699]: Deploying daemon mds.l on smithi064 2024-05-07T02:25:09.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:08 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:25:09.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:08 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:25:09.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:08 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:09.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:08 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-07T02:25:09.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:08 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-07T02:25:09.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:08 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:25:09.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:08 smithi027 ceph-mon[34398]: Deploying daemon mds.l on smithi064 2024-05-07T02:25:10.268 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:10 smithi064 ceph-mon[34699]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:10.306 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:10 smithi019 ceph-mon[29781]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:10.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:10 smithi027 ceph-mon[34398]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:10.608 INFO:journalctl@ceph.mds.l.smithi064.stdout:May 07 02:25:10 smithi064 systemd[1]: Starting Ceph mds.l for 9ff84c16-0c17-11ef-bc97-c7b262605968... 2024-05-07T02:25:10.928 INFO:journalctl@ceph.mds.l.smithi064.stdout:May 07 02:25:10 smithi064 podman[64870]: 2024-05-07 02:25:10.575872063 +0000 UTC m=+0.018007404 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-07T02:25:10.928 INFO:journalctl@ceph.mds.l.smithi064.stdout:May 07 02:25:10 smithi064 podman[64870]: 2024-05-07 02:25:10.700898473 +0000 UTC m=+0.143033871 container create 62bcd6ca49bcc856ab4b8e82e1b96fec0dd6ae9d194a565d37423fd8c51d768e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-07T02:25:10.928 INFO:journalctl@ceph.mds.l.smithi064.stdout:May 07 02:25:10 smithi064 podman[64870]: 2024-05-07 02:25:10.928776703 +0000 UTC m=+0.370912043 container init 62bcd6ca49bcc856ab4b8e82e1b96fec0dd6ae9d194a565d37423fd8c51d768e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-07T02:25:10.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:10.943+0000 7f2f87400700 1 -- 172.21.15.19:0/3333544038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f80106580 msgr2=0x7f2f80106990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:10.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:10.943+0000 7f2f87400700 1 --2- 172.21.15.19:0/3333544038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f80106580 0x7f2f80106990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c009990 tx=0x7f2f6c009ca0 comp rx=0 tx=0).stop 2024-05-07T02:25:10.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:10.944+0000 7f2f87400700 1 -- 172.21.15.19:0/3333544038 shutdown_connections 2024-05-07T02:25:10.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:10.944+0000 7f2f87400700 1 --2- 172.21.15.19:0/3333544038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f80106ed0 0x7f2f80113360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:10.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:10.944+0000 7f2f87400700 1 --2- 172.21.15.19:0/3333544038 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f8010c5c0 0x7f2f8010c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:10.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:10.944+0000 7f2f87400700 1 --2- 172.21.15.19:0/3333544038 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f80106580 0x7f2f80106990 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:10.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:10.944+0000 7f2f87400700 1 -- 172.21.15.19:0/3333544038 >> 172.21.15.19:0/3333544038 conn(0x7f2f80102080 msgr2=0x7f2f801044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:25:10.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:10.944+0000 7f2f87400700 1 -- 172.21.15.19:0/3333544038 shutdown_connections 2024-05-07T02:25:10.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:10.944+0000 7f2f87400700 1 -- 172.21.15.19:0/3333544038 wait complete. 2024-05-07T02:25:10.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:10.945+0000 7f2f87400700 1 Processor -- start 2024-05-07T02:25:10.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:10.946+0000 7f2f87400700 1 -- start start 2024-05-07T02:25:10.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:10.946+0000 7f2f87400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f80106580 0x7f2f8019d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:25:10.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:10.946+0000 7f2f87400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f80106ed0 0x7f2f8019d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:25:10.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:10.946+0000 7f2f87400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f8010c5c0 0x7f2f801a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:25:10.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:10.946+0000 7f2f87400700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2f801166f0 con 0x7f2f80106ed0 2024-05-07T02:25:10.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:10.946+0000 7f2f87400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2f80116570 con 0x7f2f8010c5c0 2024-05-07T02:25:10.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:10.946+0000 7f2f87400700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2f80116870 con 0x7f2f80106580 2024-05-07T02:25:10.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:10.947+0000 7f2f84a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f80106580 0x7f2f8019d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:25:10.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:10.947+0000 7f2f84a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f80106580 0x7f2f8019d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:40428/0 (socket says 172.21.15.19:40428) 2024-05-07T02:25:10.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:10.947+0000 7f2f84a00700 1 -- 172.21.15.19:0/2990996890 learned_addr learned my addr 172.21.15.19:0/2990996890 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:25:10.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:10.947+0000 7f2f7fe00700 1 --2- 172.21.15.19:0/2990996890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f80106ed0 0x7f2f8019d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:25:10.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:10.947+0000 7f2f85400700 1 --2- 172.21.15.19:0/2990996890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f8010c5c0 0x7f2f801a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:25:10.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:10.947+0000 7f2f84a00700 1 -- 172.21.15.19:0/2990996890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f8010c5c0 msgr2=0x7f2f801a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:10.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:10.947+0000 7f2f84a00700 1 --2- 172.21.15.19:0/2990996890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f8010c5c0 0x7f2f801a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:10.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:10.947+0000 7f2f84a00700 1 -- 172.21.15.19:0/2990996890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f80106ed0 msgr2=0x7f2f8019d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:10.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:10.947+0000 7f2f84a00700 1 --2- 172.21.15.19:0/2990996890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f80106ed0 0x7f2f8019d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:10.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:10.947+0000 7f2f84a00700 1 -- 172.21.15.19:0/2990996890 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f801a1f70 con 0x7f2f80106580 2024-05-07T02:25:10.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:10.948+0000 7f2f84a00700 1 --2- 172.21.15.19:0/2990996890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f80106580 0x7f2f8019d0a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2f7400efc0 tx=0x7f2f7400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:25:10.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:10.948+0000 7f2f7d600700 1 -- 172.21.15.19:0/2990996890 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7400c9c0 con 0x7f2f80106580 2024-05-07T02:25:10.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:10.948+0000 7f2f7d600700 1 -- 172.21.15.19:0/2990996890 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f74019070 con 0x7f2f80106580 2024-05-07T02:25:10.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:10.948+0000 7f2f87400700 1 -- 172.21.15.19:0/2990996890 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f801a2200 con 0x7f2f80106580 2024-05-07T02:25:10.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:10.948+0000 7f2f7d600700 1 -- 172.21.15.19:0/2990996890 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f74014480 con 0x7f2f80106580 2024-05-07T02:25:10.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:10.948+0000 7f2f87400700 1 -- 172.21.15.19:0/2990996890 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f801a2710 con 0x7f2f80106580 2024-05-07T02:25:10.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:10.950+0000 7f2f87400700 1 -- 172.21.15.19:0/2990996890 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f800512c0 con 0x7f2f80106580 2024-05-07T02:25:10.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:10.952+0000 7f2f7d600700 1 -- 172.21.15.19:0/2990996890 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f2f74012050 con 0x7f2f80106580 2024-05-07T02:25:10.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:10.953+0000 7f2f7d600700 1 --2- 172.21.15.19:0/2990996890 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f2f680b2880 0x7f2f680b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:25:10.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:10.954+0000 7f2f7d600700 1 -- 172.21.15.19:0/2990996890 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f2f740e1d30 con 0x7f2f80106580 2024-05-07T02:25:10.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:10.954+0000 7f2f7fe00700 1 --2- 172.21.15.19:0/2990996890 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f2f680b2880 0x7f2f680b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:25:10.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:10.954+0000 7f2f7fe00700 1 --2- 172.21.15.19:0/2990996890 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f2f680b2880 0x7f2f680b4d90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c006010 tx=0x7f2f6c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:25:10.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:10.956+0000 7f2f7d600700 1 -- 172.21.15.19:0/2990996890 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2f74010040 con 0x7f2f80106580 2024-05-07T02:25:11.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:11.253+0000 7f2f87400700 1 -- 172.21.15.19:0/2990996890 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f8002e5f0 con 0x7f2f80106580 2024-05-07T02:25:11.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:11.254+0000 7f2f7d600700 1 -- 172.21.15.19:0/2990996890 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1390 (secure 0 0 0) 0x7f2f740a9e50 con 0x7f2f80106580 2024-05-07T02:25:11.254 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:25:11.254 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/477529985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":477529985},{"type":"v1","addr":"172.21.15.64:6833","nonce":477529985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-07T02:25:11.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:11.257+0000 7f2f87400700 1 -- 172.21.15.19:0/2990996890 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f2f680b2880 msgr2=0x7f2f680b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:11.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:11.257+0000 7f2f87400700 1 --2- 172.21.15.19:0/2990996890 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f2f680b2880 0x7f2f680b4d90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c006010 tx=0x7f2f6c00b260 comp rx=0 tx=0).stop 2024-05-07T02:25:11.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:11.257+0000 7f2f87400700 1 -- 172.21.15.19:0/2990996890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f80106580 msgr2=0x7f2f8019d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:11.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:11.257+0000 7f2f87400700 1 --2- 172.21.15.19:0/2990996890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f80106580 0x7f2f8019d0a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2f7400efc0 tx=0x7f2f7400c3a0 comp rx=0 tx=0).stop 2024-05-07T02:25:11.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:11.257+0000 7f2f87400700 1 -- 172.21.15.19:0/2990996890 shutdown_connections 2024-05-07T02:25:11.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:11.257+0000 7f2f87400700 1 --2- 172.21.15.19:0/2990996890 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f2f680b2880 0x7f2f680b4d90 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:11.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:11.257+0000 7f2f87400700 1 --2- 172.21.15.19:0/2990996890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f8010c5c0 0x7f2f801a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:11.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:11.257+0000 7f2f87400700 1 --2- 172.21.15.19:0/2990996890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2f80106580 0x7f2f8019d0a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:11.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:11.257+0000 7f2f87400700 1 --2- 172.21.15.19:0/2990996890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f80106ed0 0x7f2f8019d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:11.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:11.258+0000 7f2f87400700 1 -- 172.21.15.19:0/2990996890 >> 172.21.15.19:0/2990996890 conn(0x7f2f80102080 msgr2=0x7f2f80104820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:25:11.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:11.258+0000 7f2f87400700 1 -- 172.21.15.19:0/2990996890 shutdown_connections 2024-05-07T02:25:11.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:11.258+0000 7f2f87400700 1 -- 172.21.15.19:0/2990996890 wait complete. 2024-05-07T02:25:11.259 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 2 2024-05-07T02:25:11.358 INFO:journalctl@ceph.mds.l.smithi064.stdout:May 07 02:25:10 smithi064 podman[64870]: 2024-05-07 02:25:10.931652729 +0000 UTC m=+0.373788077 container start 62bcd6ca49bcc856ab4b8e82e1b96fec0dd6ae9d194a565d37423fd8c51d768e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-l, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , ceph=True) 2024-05-07T02:25:11.358 INFO:journalctl@ceph.mds.l.smithi064.stdout:May 07 02:25:10 smithi064 ceph-mds[64888]: -- 172.21.15.64:0/905821245 <== mon.1 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5611c3469a40 con 0x5611c34b1a80 2024-05-07T02:25:11.358 INFO:journalctl@ceph.mds.l.smithi064.stdout:May 07 02:25:10 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-l[64884]: starting mds.l at 2024-05-07T02:25:11.358 INFO:journalctl@ceph.mds.l.smithi064.stdout:May 07 02:25:11 smithi064 bash[64870]: 62bcd6ca49bcc856ab4b8e82e1b96fec0dd6ae9d194a565d37423fd8c51d768e 2024-05-07T02:25:11.358 INFO:journalctl@ceph.mds.l.smithi064.stdout:May 07 02:25:11 smithi064 systemd[1]: Started Ceph mds.l for 9ff84c16-0c17-11ef-bc97-c7b262605968. 2024-05-07T02:25:12.098 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:11 smithi027 ceph-mon[34398]: mds.? [v2:172.21.15.64:6832/477529985,v1:172.21.15.64:6833/477529985] up:boot 2024-05-07T02:25:12.098 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:11 smithi027 ceph-mon[34398]: fsmap 1 up:standby 2024-05-07T02:25:12.098 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:11 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-07T02:25:12.098 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:11 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-07T02:25:12.098 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:11 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:12.098 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:11 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:12.098 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:11 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:12.098 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:11 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-07T02:25:12.098 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:11 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-07T02:25:12.099 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:11 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:25:12.099 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:11 smithi027 ceph-mon[34398]: Deploying daemon mds.k on smithi027 2024-05-07T02:25:12.099 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:11 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/2990996890' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:25:12.099 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:11 smithi027 ceph-mon[34398]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:12.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:11 smithi064 ceph-mon[34699]: mds.? [v2:172.21.15.64:6832/477529985,v1:172.21.15.64:6833/477529985] up:boot 2024-05-07T02:25:12.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:11 smithi064 ceph-mon[34699]: fsmap 1 up:standby 2024-05-07T02:25:12.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:11 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-07T02:25:12.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:11 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-07T02:25:12.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:11 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:12.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:11 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:12.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:11 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:12.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:11 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-07T02:25:12.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:11 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-07T02:25:12.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:11 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:25:12.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:11 smithi064 ceph-mon[34699]: Deploying daemon mds.k on smithi027 2024-05-07T02:25:12.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:11 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/2990996890' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:25:12.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:11 smithi064 ceph-mon[34699]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:12.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:11 smithi019 ceph-mon[29781]: mds.? [v2:172.21.15.64:6832/477529985,v1:172.21.15.64:6833/477529985] up:boot 2024-05-07T02:25:12.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:11 smithi019 ceph-mon[29781]: fsmap 1 up:standby 2024-05-07T02:25:12.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:11 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-07T02:25:12.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:11 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-07T02:25:12.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:11 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:12.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:11 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:12.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:11 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:12.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:11 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-07T02:25:12.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:11 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-07T02:25:12.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:11 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:25:12.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:11 smithi019 ceph-mon[29781]: Deploying daemon mds.k on smithi027 2024-05-07T02:25:12.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:11 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/2990996890' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:25:12.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:11 smithi019 ceph-mon[29781]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:13.684 INFO:journalctl@ceph.mds.k.smithi027.stdout:May 07 02:25:13 smithi027 systemd[1]: Starting Ceph mds.k for 9ff84c16-0c17-11ef-bc97-c7b262605968... 2024-05-07T02:25:13.956 INFO:journalctl@ceph.mds.k.smithi027.stdout:May 07 02:25:13 smithi027 podman[64208]: 2024-05-07 02:25:13.589368155 +0000 UTC m=+0.018897733 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-07T02:25:13.957 INFO:journalctl@ceph.mds.k.smithi027.stdout:May 07 02:25:13 smithi027 podman[64208]: 2024-05-07 02:25:13.714923758 +0000 UTC m=+0.144453323 container create 56b71961f80a3ff9337269477f56d400a3fcc51ff27f6e4e4ca47a567fece2d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a) 2024-05-07T02:25:14.055 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-07T02:25:14.456 INFO:journalctl@ceph.mds.k.smithi027.stdout:May 07 02:25:13 smithi027 podman[64208]: 2024-05-07 02:25:13.984382944 +0000 UTC m=+0.413912503 container init 56b71961f80a3ff9337269477f56d400a3fcc51ff27f6e4e4ca47a567fece2d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-k, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-07T02:25:14.456 INFO:journalctl@ceph.mds.k.smithi027.stdout:May 07 02:25:13 smithi027 podman[64208]: 2024-05-07 02:25:13.987087302 +0000 UTC m=+0.416616861 container start 56b71961f80a3ff9337269477f56d400a3fcc51ff27f6e4e4ca47a567fece2d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-k, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-07T02:25:14.457 INFO:journalctl@ceph.mds.k.smithi027.stdout:May 07 02:25:14 smithi027 ceph-mds[64227]: -- 172.21.15.27:0/3493702967 <== mon.2 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c2db065a40 con 0x55c2db0ad600 2024-05-07T02:25:14.457 INFO:journalctl@ceph.mds.k.smithi027.stdout:May 07 02:25:14 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-k[64223]: starting mds.k at 2024-05-07T02:25:14.457 INFO:journalctl@ceph.mds.k.smithi027.stdout:May 07 02:25:14 smithi027 bash[64208]: 56b71961f80a3ff9337269477f56d400a3fcc51ff27f6e4e4ca47a567fece2d2 2024-05-07T02:25:14.457 INFO:journalctl@ceph.mds.k.smithi027.stdout:May 07 02:25:14 smithi027 systemd[1]: Started Ceph mds.k for 9ff84c16-0c17-11ef-bc97-c7b262605968. 2024-05-07T02:25:14.526 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:25:14.734 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:14 smithi019 ceph-mon[29781]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:14.734 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:14 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:14.734 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:14 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:14.735 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:14 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:14.735 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:14 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-07T02:25:14.735 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:14 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-07T02:25:14.735 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:14 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:25:14.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:14 smithi064 ceph-mon[34699]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:14.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:14 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:14.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:14 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:14.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:14 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:14.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:14 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-07T02:25:14.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:14 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-07T02:25:14.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:14 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:25:14.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:14 smithi027 ceph-mon[34398]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:14.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:14 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:14.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:14 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:14.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:14 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:14.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:14 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-07T02:25:14.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:14 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-07T02:25:14.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:14 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:25:15.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:15 smithi027 ceph-mon[34398]: Deploying daemon mds.j on smithi019 2024-05-07T02:25:15.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:15 smithi027 ceph-mon[34398]: mds.? [v2:172.21.15.27:6832/2171873162,v1:172.21.15.27:6833/2171873162] up:boot 2024-05-07T02:25:15.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:15 smithi027 ceph-mon[34398]: fsmap 2 up:standby 2024-05-07T02:25:15.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:15 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-07T02:25:15.742 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:15 smithi019 ceph-mon[29781]: Deploying daemon mds.j on smithi019 2024-05-07T02:25:15.743 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:15 smithi019 ceph-mon[29781]: mds.? [v2:172.21.15.27:6832/2171873162,v1:172.21.15.27:6833/2171873162] up:boot 2024-05-07T02:25:15.743 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:15 smithi019 ceph-mon[29781]: fsmap 2 up:standby 2024-05-07T02:25:15.743 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:15 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-07T02:25:15.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:15 smithi064 ceph-mon[34699]: Deploying daemon mds.j on smithi019 2024-05-07T02:25:15.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:15 smithi064 ceph-mon[34699]: mds.? [v2:172.21.15.27:6832/2171873162,v1:172.21.15.27:6833/2171873162] up:boot 2024-05-07T02:25:15.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:15 smithi064 ceph-mon[34699]: fsmap 2 up:standby 2024-05-07T02:25:15.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:15 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-07T02:25:16.561 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:16 smithi019 ceph-mon[29781]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:16.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:16 smithi027 ceph-mon[34398]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:16.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:16 smithi064 ceph-mon[34699]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:17.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:16.969+0000 7f4e23800700 1 -- 172.21.15.19:0/2657688335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e1c110b80 msgr2=0x7f4e1c112fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:17.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:16.969+0000 7f4e23800700 1 --2- 172.21.15.19:0/2657688335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e1c110b80 0x7f4e1c112fe0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4e1400b0a0 tx=0x7f4e1400b3b0 comp rx=0 tx=0).stop 2024-05-07T02:25:17.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:16.969+0000 7f4e23800700 1 -- 172.21.15.19:0/2657688335 shutdown_connections 2024-05-07T02:25:17.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:16.969+0000 7f4e23800700 1 --2- 172.21.15.19:0/2657688335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e1c110b80 0x7f4e1c112fe0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:17.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:16.969+0000 7f4e23800700 1 --2- 172.21.15.19:0/2657688335 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e1c076e40 0x7f4e1c077250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:17.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:16.969+0000 7f4e23800700 1 --2- 172.21.15.19:0/2657688335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e1c077790 0x7f4e1c110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:17.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:16.969+0000 7f4e23800700 1 -- 172.21.15.19:0/2657688335 >> 172.21.15.19:0/2657688335 conn(0x7f4e1c0ffea0 msgr2=0x7f4e1c1022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:25:17.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:16.969+0000 7f4e23800700 1 -- 172.21.15.19:0/2657688335 shutdown_connections 2024-05-07T02:25:17.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:16.970+0000 7f4e23800700 1 -- 172.21.15.19:0/2657688335 wait complete. 2024-05-07T02:25:17.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:16.971+0000 7f4e23800700 1 Processor -- start 2024-05-07T02:25:17.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:16.971+0000 7f4e23800700 1 -- start start 2024-05-07T02:25:17.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:16.972+0000 7f4e23800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e1c076e40 0x7f4e1c19d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:25:17.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:16.972+0000 7f4e23800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e1c077790 0x7f4e1c19d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:25:17.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:16.972+0000 7f4e23800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e1c110b80 0x7f4e1c1a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:25:17.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:16.972+0000 7f4e23800700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4e1c1166b0 con 0x7f4e1c077790 2024-05-07T02:25:17.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:16.972+0000 7f4e23800700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4e1c116530 con 0x7f4e1c076e40 2024-05-07T02:25:17.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:16.972+0000 7f4e23800700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4e1c116830 con 0x7f4e1c110b80 2024-05-07T02:25:17.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:16.972+0000 7f4e1be00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e1c077790 0x7f4e1c19d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:25:17.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:16.972+0000 7f4e1be00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e1c077790 0x7f4e1c19d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48592/0 (socket says 172.21.15.19:48592) 2024-05-07T02:25:17.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:16.972+0000 7f4e21800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e1c110b80 0x7f4e1c1a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:25:17.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:16.972+0000 7f4e20e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e1c076e40 0x7f4e1c19d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:25:17.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:16.972+0000 7f4e1be00700 1 -- 172.21.15.19:0/601420220 learned_addr learned my addr 172.21.15.19:0/601420220 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:25:17.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:16.973+0000 7f4e1be00700 1 -- 172.21.15.19:0/601420220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e1c110b80 msgr2=0x7f4e1c1a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:17.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:16.973+0000 7f4e1be00700 1 --2- 172.21.15.19:0/601420220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e1c110b80 0x7f4e1c1a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:17.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:16.973+0000 7f4e1be00700 1 -- 172.21.15.19:0/601420220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e1c076e40 msgr2=0x7f4e1c19d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:17.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:16.973+0000 7f4e1be00700 1 --2- 172.21.15.19:0/601420220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e1c076e40 0x7f4e1c19d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:17.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:16.973+0000 7f4e1be00700 1 -- 172.21.15.19:0/601420220 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e1c1a1ed0 con 0x7f4e1c077790 2024-05-07T02:25:17.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:16.973+0000 7f4e21800700 1 --2- 172.21.15.19:0/601420220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e1c110b80 0x7f4e1c1a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:25:17.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:16.973+0000 7f4e1be00700 1 --2- 172.21.15.19:0/601420220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e1c077790 0x7f4e1c19d5a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4e0800e9a0 tx=0x7f4e0800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:25:17.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:16.973+0000 7f4e19600700 1 -- 172.21.15.19:0/601420220 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e0800ca10 con 0x7f4e1c077790 2024-05-07T02:25:17.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:16.973+0000 7f4e19600700 1 -- 172.21.15.19:0/601420220 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e0800cbb0 con 0x7f4e1c077790 2024-05-07T02:25:17.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:16.973+0000 7f4e23800700 1 -- 172.21.15.19:0/601420220 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e1c1a21c0 con 0x7f4e1c077790 2024-05-07T02:25:17.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:16.973+0000 7f4e19600700 1 -- 172.21.15.19:0/601420220 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e08018680 con 0x7f4e1c077790 2024-05-07T02:25:17.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:16.974+0000 7f4e23800700 1 -- 172.21.15.19:0/601420220 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e1c1a9ae0 con 0x7f4e1c077790 2024-05-07T02:25:17.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:16.975+0000 7f4e23800700 1 -- 172.21.15.19:0/601420220 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e1c0512c0 con 0x7f4e1c077790 2024-05-07T02:25:17.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:16.978+0000 7f4e19600700 1 -- 172.21.15.19:0/601420220 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f4e08010430 con 0x7f4e1c077790 2024-05-07T02:25:17.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:16.979+0000 7f4e19600700 1 --2- 172.21.15.19:0/601420220 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f4e040b2880 0x7f4e040b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:25:17.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:16.979+0000 7f4e19600700 1 -- 172.21.15.19:0/601420220 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f4e080e2220 con 0x7f4e1c077790 2024-05-07T02:25:17.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:16.979+0000 7f4e20e00700 1 --2- 172.21.15.19:0/601420220 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f4e040b2880 0x7f4e040b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:25:17.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:16.980+0000 7f4e20e00700 1 --2- 172.21.15.19:0/601420220 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f4e040b2880 0x7f4e040b4d90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4e10009c60 tx=0x7f4e10009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:25:17.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:16.982+0000 7f4e19600700 1 -- 172.21.15.19:0/601420220 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4e080aa340 con 0x7f4e1c077790 2024-05-07T02:25:17.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:17.280+0000 7f4e23800700 1 -- 172.21.15.19:0/601420220 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4e1c1a9cb0 con 0x7f4e1c077790 2024-05-07T02:25:17.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:17.282+0000 7f4e19600700 1 -- 172.21.15.19:0/601420220 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2215 (secure 0 0 0) 0x7f4e080b23c0 con 0x7f4e1c077790 2024-05-07T02:25:17.282 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:25:17.283 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/477529985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":477529985},{"type":"v1","addr":"172.21.15.64:6833","nonce":477529985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/2171873162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2171873162},{"type":"v1","addr":"172.21.15.27:6833","nonce":2171873162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-07T02:25:17.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:17.285+0000 7f4e23800700 1 -- 172.21.15.19:0/601420220 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f4e040b2880 msgr2=0x7f4e040b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:17.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:17.285+0000 7f4e23800700 1 --2- 172.21.15.19:0/601420220 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f4e040b2880 0x7f4e040b4d90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4e10009c60 tx=0x7f4e10009260 comp rx=0 tx=0).stop 2024-05-07T02:25:17.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:17.285+0000 7f4e23800700 1 -- 172.21.15.19:0/601420220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e1c077790 msgr2=0x7f4e1c19d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:17.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:17.285+0000 7f4e23800700 1 --2- 172.21.15.19:0/601420220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e1c077790 0x7f4e1c19d5a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4e0800e9a0 tx=0x7f4e0800ecb0 comp rx=0 tx=0).stop 2024-05-07T02:25:17.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:17.286+0000 7f4e23800700 1 -- 172.21.15.19:0/601420220 shutdown_connections 2024-05-07T02:25:17.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:17.286+0000 7f4e23800700 1 --2- 172.21.15.19:0/601420220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4e1c110b80 0x7f4e1c1a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:17.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:17.286+0000 7f4e23800700 1 --2- 172.21.15.19:0/601420220 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f4e040b2880 0x7f4e040b4d90 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:17.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:17.286+0000 7f4e23800700 1 --2- 172.21.15.19:0/601420220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e1c076e40 0x7f4e1c19d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:17.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:17.286+0000 7f4e23800700 1 --2- 172.21.15.19:0/601420220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e1c077790 0x7f4e1c19d5a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:17.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:17.286+0000 7f4e23800700 1 -- 172.21.15.19:0/601420220 >> 172.21.15.19:0/601420220 conn(0x7f4e1c0ffea0 msgr2=0x7f4e1c1002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:25:17.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:17.286+0000 7f4e23800700 1 -- 172.21.15.19:0/601420220 shutdown_connections 2024-05-07T02:25:17.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:17.286+0000 7f4e23800700 1 -- 172.21.15.19:0/601420220 wait complete. 2024-05-07T02:25:17.287 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 3 2024-05-07T02:25:17.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:17 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/601420220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:25:17.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:17 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/601420220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:25:17.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:17 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/601420220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:25:18.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:18 smithi019 ceph-mon[29781]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:18.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:18 smithi064 ceph-mon[34699]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:18.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:18 smithi027 ceph-mon[34398]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:19.560 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 07 02:25:19 smithi019 systemd[1]: Starting Ceph mds.j for 9ff84c16-0c17-11ef-bc97-c7b262605968... 2024-05-07T02:25:19.881 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 07 02:25:19 smithi019 podman[70344]: 2024-05-07 02:25:19.467723839 +0000 UTC m=+0.041260393 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-07T02:25:19.881 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 07 02:25:19 smithi019 podman[70344]: 2024-05-07 02:25:19.601918789 +0000 UTC m=+0.175455353 container create 3b7491ac7c35b9bb21cf4e34ecccb775a5f15b6b74392f2989456df2fc9bb8a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430) 2024-05-07T02:25:19.882 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 07 02:25:19 smithi019 podman[70344]: 2024-05-07 02:25:19.881019567 +0000 UTC m=+0.454556151 container init 3b7491ac7c35b9bb21cf4e34ecccb775a5f15b6b74392f2989456df2fc9bb8a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-j, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-07T02:25:20.077 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-07T02:25:20.145 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 07 02:25:19 smithi019 podman[70344]: 2024-05-07 02:25:19.891485626 +0000 UTC m=+0.465022160 container start 3b7491ac7c35b9bb21cf4e34ecccb775a5f15b6b74392f2989456df2fc9bb8a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-j, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux ) 2024-05-07T02:25:20.145 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 07 02:25:19 smithi019 bash[70344]: 3b7491ac7c35b9bb21cf4e34ecccb775a5f15b6b74392f2989456df2fc9bb8a2 2024-05-07T02:25:20.145 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 07 02:25:19 smithi019 systemd[1]: Started Ceph mds.j for 9ff84c16-0c17-11ef-bc97-c7b262605968. 2024-05-07T02:25:20.145 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 07 02:25:20 smithi019 ceph-mds[70362]: -- 172.21.15.19:0/1494722809 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5585ce945a40 con 0x5585ce98da80 2024-05-07T02:25:20.145 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 07 02:25:20 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-j[70358]: starting mds.j at 2024-05-07T02:25:20.537 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:25:20.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:20 smithi064 ceph-mon[34699]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:20.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:20 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:20.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:20 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:20.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:20 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:20.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:20 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-07T02:25:20.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:20 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-07T02:25:20.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:20 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:25:20.867 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:20 smithi019 ceph-mon[29781]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:20.867 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:20 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:20.867 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:20 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:20.867 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:20 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:20.867 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:20 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-07T02:25:20.867 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:20 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-07T02:25:20.867 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:20 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:25:20.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:20 smithi027 ceph-mon[34398]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:20.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:20 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:20.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:20 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:20.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:20 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:20.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:20 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-07T02:25:20.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:20 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-07T02:25:20.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:20 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:25:21.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:21 smithi064 ceph-mon[34699]: Deploying daemon mds.d on smithi019 2024-05-07T02:25:21.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:21 smithi064 ceph-mon[34699]: mds.? [v2:172.21.15.19:6834/2379917839,v1:172.21.15.19:6835/2379917839] up:boot 2024-05-07T02:25:21.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:21 smithi064 ceph-mon[34699]: fsmap 3 up:standby 2024-05-07T02:25:21.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:21 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-07T02:25:21.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:21 smithi019 ceph-mon[29781]: Deploying daemon mds.d on smithi019 2024-05-07T02:25:21.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:21 smithi019 ceph-mon[29781]: mds.? [v2:172.21.15.19:6834/2379917839,v1:172.21.15.19:6835/2379917839] up:boot 2024-05-07T02:25:21.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:21 smithi019 ceph-mon[29781]: fsmap 3 up:standby 2024-05-07T02:25:21.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:21 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-07T02:25:21.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:21 smithi027 ceph-mon[34398]: Deploying daemon mds.d on smithi019 2024-05-07T02:25:21.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:21 smithi027 ceph-mon[34398]: mds.? [v2:172.21.15.19:6834/2379917839,v1:172.21.15.19:6835/2379917839] up:boot 2024-05-07T02:25:21.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:21 smithi027 ceph-mon[34398]: fsmap 3 up:standby 2024-05-07T02:25:21.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:21 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-07T02:25:22.830 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:22 smithi019 ceph-mon[29781]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:22.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:22 smithi064 ceph-mon[34699]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:22.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:22 smithi027 ceph-mon[34398]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:23.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:22.976+0000 7f3751000700 1 -- 172.21.15.19:0/153928203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f374c10a990 msgr2=0x7f374c10ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:23.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:22.976+0000 7f3751000700 1 --2- 172.21.15.19:0/153928203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f374c10a990 0x7f374c10ada0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3738009990 tx=0x7f3738009ca0 comp rx=0 tx=0).stop 2024-05-07T02:25:23.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:22.976+0000 7f3751000700 1 -- 172.21.15.19:0/153928203 shutdown_connections 2024-05-07T02:25:23.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:22.976+0000 7f3751000700 1 --2- 172.21.15.19:0/153928203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f374c10b2e0 0x7f374c117810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:23.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:22.976+0000 7f3751000700 1 --2- 172.21.15.19:0/153928203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f374c110970 0x7f374c110d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:23.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:22.976+0000 7f3751000700 1 --2- 172.21.15.19:0/153928203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f374c10a990 0x7f374c10ada0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:23.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:22.977+0000 7f3751000700 1 -- 172.21.15.19:0/153928203 >> 172.21.15.19:0/153928203 conn(0x7f374c077c80 msgr2=0x7f374c0780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:25:23.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:22.977+0000 7f3751000700 1 -- 172.21.15.19:0/153928203 shutdown_connections 2024-05-07T02:25:23.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:22.977+0000 7f3751000700 1 -- 172.21.15.19:0/153928203 wait complete. 2024-05-07T02:25:23.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:22.978+0000 7f3751000700 1 Processor -- start 2024-05-07T02:25:23.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:22.978+0000 7f3751000700 1 -- start start 2024-05-07T02:25:23.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:22.978+0000 7f3751000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f374c10a990 0x7f374c1a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:25:23.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:22.978+0000 7f3751000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f374c10b2e0 0x7f374c1a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:25:23.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:22.978+0000 7f3751000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f374c110970 0x7f374c1a5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:25:23.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:22.978+0000 7f3751000700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f374c11aae0 con 0x7f374c10a990 2024-05-07T02:25:23.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:22.978+0000 7f3751000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f374c11a960 con 0x7f374c10b2e0 2024-05-07T02:25:23.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:22.978+0000 7f3751000700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f374c11ac60 con 0x7f374c110970 2024-05-07T02:25:23.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:22.979+0000 7f3749400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f374c10b2e0 0x7f374c1a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:25:23.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:22.979+0000 7f3749400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f374c10b2e0 0x7f374c1a1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:37118/0 (socket says 172.21.15.19:37118) 2024-05-07T02:25:23.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:22.979+0000 7f3749e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f374c10a990 0x7f374c1a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:25:23.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:22.979+0000 7f3749400700 1 -- 172.21.15.19:0/4056780332 learned_addr learned my addr 172.21.15.19:0/4056780332 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:25:23.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:22.979+0000 7f374a800700 1 --2- 172.21.15.19:0/4056780332 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f374c110970 0x7f374c1a5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:25:23.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:22.979+0000 7f3749400700 1 -- 172.21.15.19:0/4056780332 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f374c110970 msgr2=0x7f374c1a5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:23.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:22.979+0000 7f3749400700 1 --2- 172.21.15.19:0/4056780332 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f374c110970 0x7f374c1a5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:23.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:22.979+0000 7f3749400700 1 -- 172.21.15.19:0/4056780332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f374c10a990 msgr2=0x7f374c1a14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:23.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:22.979+0000 7f3749400700 1 --2- 172.21.15.19:0/4056780332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f374c10a990 0x7f374c1a14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:23.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:22.979+0000 7f3749400700 1 -- 172.21.15.19:0/4056780332 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f374c1a63a0 con 0x7f374c10b2e0 2024-05-07T02:25:23.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:22.980+0000 7f3749400700 1 --2- 172.21.15.19:0/4056780332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f374c10b2e0 0x7f374c1a1a70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3738000c00 tx=0x7f37380047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:25:23.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:22.980+0000 7f3742a00700 1 -- 172.21.15.19:0/4056780332 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3738026070 con 0x7f374c10b2e0 2024-05-07T02:25:23.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:22.980+0000 7f3742a00700 1 -- 172.21.15.19:0/4056780332 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3738021410 con 0x7f374c10b2e0 2024-05-07T02:25:23.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:22.980+0000 7f3751000700 1 -- 172.21.15.19:0/4056780332 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f374c1a6630 con 0x7f374c10b2e0 2024-05-07T02:25:23.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:22.980+0000 7f3742a00700 1 -- 172.21.15.19:0/4056780332 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3738017650 con 0x7f374c10b2e0 2024-05-07T02:25:23.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:22.980+0000 7f3751000700 1 -- 172.21.15.19:0/4056780332 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f374c1a6b10 con 0x7f374c10b2e0 2024-05-07T02:25:23.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:22.981+0000 7f3751000700 1 -- 172.21.15.19:0/4056780332 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f374c0512c0 con 0x7f374c10b2e0 2024-05-07T02:25:23.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:22.984+0000 7f3742a00700 1 -- 172.21.15.19:0/4056780332 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f3738029070 con 0x7f374c10b2e0 2024-05-07T02:25:23.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:22.986+0000 7f3742a00700 1 --2- 172.21.15.19:0/4056780332 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f37300b2880 0x7f37300b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:25:23.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:22.986+0000 7f3742a00700 1 -- 172.21.15.19:0/4056780332 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f37380e2980 con 0x7f374c10b2e0 2024-05-07T02:25:23.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:22.986+0000 7f3749e00700 1 --2- 172.21.15.19:0/4056780332 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f37300b2880 0x7f37300b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:25:23.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:22.987+0000 7f3749e00700 1 --2- 172.21.15.19:0/4056780332 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f37300b2880 0x7f37300b4d90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f374c07d0f0 tx=0x7f373c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:25:23.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:22.991+0000 7f3742a00700 1 -- 172.21.15.19:0/4056780332 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3738013070 con 0x7f374c10b2e0 2024-05-07T02:25:23.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:23.285+0000 7f3751000700 1 -- 172.21.15.19:0/4056780332 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f374c112ef0 con 0x7f374c10b2e0 2024-05-07T02:25:23.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:23.286+0000 7f3742a00700 1 -- 172.21.15.19:0/4056780332 <== mon.2 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3040 (secure 0 0 0) 0x7f37380aab30 con 0x7f374c10b2e0 2024-05-07T02:25:23.286 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:25:23.286 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/477529985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":477529985},{"type":"v1","addr":"172.21.15.64:6833","nonce":477529985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/2171873162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2171873162},{"type":"v1","addr":"172.21.15.27:6833","nonce":2171873162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24469,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2379917839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2379917839},{"type":"v1","addr":"172.21.15.19:6835","nonce":2379917839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[]} 2024-05-07T02:25:23.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:23.289+0000 7f3751000700 1 -- 172.21.15.19:0/4056780332 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f37300b2880 msgr2=0x7f37300b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:23.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:23.289+0000 7f3751000700 1 --2- 172.21.15.19:0/4056780332 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f37300b2880 0x7f37300b4d90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f374c07d0f0 tx=0x7f373c009260 comp rx=0 tx=0).stop 2024-05-07T02:25:23.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:23.289+0000 7f3751000700 1 -- 172.21.15.19:0/4056780332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f374c10b2e0 msgr2=0x7f374c1a1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:23.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:23.289+0000 7f3751000700 1 --2- 172.21.15.19:0/4056780332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f374c10b2e0 0x7f374c1a1a70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3738000c00 tx=0x7f37380047b0 comp rx=0 tx=0).stop 2024-05-07T02:25:23.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:23.289+0000 7f3751000700 1 -- 172.21.15.19:0/4056780332 shutdown_connections 2024-05-07T02:25:23.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:23.289+0000 7f3751000700 1 --2- 172.21.15.19:0/4056780332 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f374c110970 0x7f374c1a5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:23.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:23.289+0000 7f3751000700 1 --2- 172.21.15.19:0/4056780332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f374c10a990 0x7f374c1a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:23.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:23.289+0000 7f3751000700 1 --2- 172.21.15.19:0/4056780332 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f37300b2880 0x7f37300b4d90 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:23.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:23.289+0000 7f3751000700 1 --2- 172.21.15.19:0/4056780332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f374c10b2e0 0x7f374c1a1a70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:23.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:23.289+0000 7f3751000700 1 -- 172.21.15.19:0/4056780332 >> 172.21.15.19:0/4056780332 conn(0x7f374c077c80 msgr2=0x7f374c07b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:25:23.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:23.289+0000 7f3751000700 1 -- 172.21.15.19:0/4056780332 shutdown_connections 2024-05-07T02:25:23.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:23.289+0000 7f3751000700 1 -- 172.21.15.19:0/4056780332 wait complete. 2024-05-07T02:25:23.290 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 4 2024-05-07T02:25:23.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:23 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/4056780332' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:25:23.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:23 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/4056780332' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:25:23.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:23 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/4056780332' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:25:24.670 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:24 smithi019 ceph-mon[29781]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:24.858 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:24 smithi027 ceph-mon[34398]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:24.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:24 smithi064 ceph-mon[34699]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:25.145 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 07 02:25:24 smithi019 systemd[1]: Starting Ceph mds.d for 9ff84c16-0c17-11ef-bc97-c7b262605968... 2024-05-07T02:25:25.619 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 07 02:25:25 smithi019 podman[70799]: 2024-05-07 02:25:25.173571457 +0000 UTC m=+0.041943873 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-07T02:25:25.619 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 07 02:25:25 smithi019 podman[70799]: 2024-05-07 02:25:25.310243488 +0000 UTC m=+0.178615919 container create 0533f004eb397c67c5abe93ccee3219aa0eeebc954cf5e6b4ddb6919dcb5c251 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-d, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430) 2024-05-07T02:25:25.619 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 07 02:25:25 smithi019 podman[70799]: 2024-05-07 02:25:25.618322615 +0000 UTC m=+0.486695050 container init 0533f004eb397c67c5abe93ccee3219aa0eeebc954cf5e6b4ddb6919dcb5c251 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-07T02:25:25.895 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 07 02:25:25 smithi019 podman[70799]: 2024-05-07 02:25:25.628208996 +0000 UTC m=+0.496581397 container start 0533f004eb397c67c5abe93ccee3219aa0eeebc954cf5e6b4ddb6919dcb5c251 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-d, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True) 2024-05-07T02:25:25.895 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 07 02:25:25 smithi019 bash[70799]: 0533f004eb397c67c5abe93ccee3219aa0eeebc954cf5e6b4ddb6919dcb5c251 2024-05-07T02:25:25.896 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 07 02:25:25 smithi019 systemd[1]: Started Ceph mds.d for 9ff84c16-0c17-11ef-bc97-c7b262605968. 2024-05-07T02:25:25.896 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 07 02:25:25 smithi019 ceph-mds[70819]: -- 172.21.15.19:0/774860146 <== mon.2 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e9b67d7a40 con 0x55e9b681f180 2024-05-07T02:25:25.896 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 07 02:25:25 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-d[70814]: starting mds.d at 2024-05-07T02:25:26.675 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-07T02:25:26.853 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:26 smithi027 ceph-mon[34398]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:26.853 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:26 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:26.853 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:26 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:26.853 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:26 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:26.853 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:26 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-07T02:25:26.854 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:26 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-07T02:25:26.854 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:26 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:25:26.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:26 smithi064 ceph-mon[34699]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:26.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:26 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:26.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:26 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:26.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:26 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:26.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:26 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-07T02:25:26.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:26 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-07T02:25:26.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:26 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:25:26.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:26 smithi019 ceph-mon[29781]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:26.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:26 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:26.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:26 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:26.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:26 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:26.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:26 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-07T02:25:26.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:26 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-07T02:25:26.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:26 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:25:27.127 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:25:27.809 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:27 smithi019 ceph-mon[29781]: Deploying daemon mds.e on smithi027 2024-05-07T02:25:27.809 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:27 smithi019 ceph-mon[29781]: mds.? [v2:172.21.15.19:6836/2067515538,v1:172.21.15.19:6837/2067515538] up:boot 2024-05-07T02:25:27.809 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:27 smithi019 ceph-mon[29781]: fsmap 4 up:standby 2024-05-07T02:25:27.809 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:27 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-07T02:25:27.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:27 smithi064 ceph-mon[34699]: Deploying daemon mds.e on smithi027 2024-05-07T02:25:27.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:27 smithi064 ceph-mon[34699]: mds.? [v2:172.21.15.19:6836/2067515538,v1:172.21.15.19:6837/2067515538] up:boot 2024-05-07T02:25:27.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:27 smithi064 ceph-mon[34699]: fsmap 4 up:standby 2024-05-07T02:25:27.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:27 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-07T02:25:27.926 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:27 smithi027 ceph-mon[34398]: Deploying daemon mds.e on smithi027 2024-05-07T02:25:27.926 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:27 smithi027 ceph-mon[34398]: mds.? [v2:172.21.15.19:6836/2067515538,v1:172.21.15.19:6837/2067515538] up:boot 2024-05-07T02:25:27.926 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:27 smithi027 ceph-mon[34398]: fsmap 4 up:standby 2024-05-07T02:25:27.926 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:27 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-07T02:25:28.206 INFO:journalctl@ceph.mds.e.smithi027.stdout:May 07 02:25:28 smithi027 systemd[1]: Starting Ceph mds.e for 9ff84c16-0c17-11ef-bc97-c7b262605968... 2024-05-07T02:25:28.610 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:28 smithi027 ceph-mon[34398]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:28.610 INFO:journalctl@ceph.mds.e.smithi027.stdout:May 07 02:25:28 smithi027 podman[64532]: 2024-05-07 02:25:28.23722708 +0000 UTC m=+0.018570552 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-07T02:25:28.610 INFO:journalctl@ceph.mds.e.smithi027.stdout:May 07 02:25:28 smithi027 podman[64532]: 2024-05-07 02:25:28.354857771 +0000 UTC m=+0.136201268 container create cbc890c768f0de80270e5d8364181b7005237b99c16be67d6915db34d7307b0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-e, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-07T02:25:28.610 INFO:journalctl@ceph.mds.e.smithi027.stdout:May 07 02:25:28 smithi027 podman[64532]: 2024-05-07 02:25:28.574181163 +0000 UTC m=+0.355524621 container init cbc890c768f0de80270e5d8364181b7005237b99c16be67d6915db34d7307b0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-e, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a) 2024-05-07T02:25:28.610 INFO:journalctl@ceph.mds.e.smithi027.stdout:May 07 02:25:28 smithi027 podman[64532]: 2024-05-07 02:25:28.577078471 +0000 UTC m=+0.358421926 container start cbc890c768f0de80270e5d8364181b7005237b99c16be67d6915db34d7307b0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-e, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-07T02:25:28.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:28 smithi064 ceph-mon[34699]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:28.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:28 smithi019 ceph-mon[29781]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:28.956 INFO:journalctl@ceph.mds.e.smithi027.stdout:May 07 02:25:28 smithi027 ceph-mds[64551]: -- 172.21.15.27:0/2883774728 <== mon.2 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5587e4091a40 con 0x5587e40d9a80 2024-05-07T02:25:28.956 INFO:journalctl@ceph.mds.e.smithi027.stdout:May 07 02:25:28 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-e[64547]: starting mds.e at 2024-05-07T02:25:28.956 INFO:journalctl@ceph.mds.e.smithi027.stdout:May 07 02:25:28 smithi027 bash[64532]: cbc890c768f0de80270e5d8364181b7005237b99c16be67d6915db34d7307b0a 2024-05-07T02:25:28.956 INFO:journalctl@ceph.mds.e.smithi027.stdout:May 07 02:25:28 smithi027 systemd[1]: Started Ceph mds.e for 9ff84c16-0c17-11ef-bc97-c7b262605968. 2024-05-07T02:25:29.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.578+0000 7f185e400700 1 -- 172.21.15.19:0/207590777 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1858108560 msgr2=0x7f185810fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:29.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.578+0000 7f185e400700 1 --2- 172.21.15.19:0/207590777 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1858108560 0x7f185810fec0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f185000b0a0 tx=0x7f185000b3b0 comp rx=0 tx=0).stop 2024-05-07T02:25:29.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.579+0000 7f185e400700 1 -- 172.21.15.19:0/207590777 shutdown_connections 2024-05-07T02:25:29.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.579+0000 7f185e400700 1 --2- 172.21.15.19:0/207590777 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1858108560 0x7f185810fec0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:29.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.579+0000 7f185e400700 1 --2- 172.21.15.19:0/207590777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18581032e0 0x7f18581036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:29.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.579+0000 7f185e400700 1 --2- 172.21.15.19:0/207590777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1858107c10 0x7f1858108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:29.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.579+0000 7f185e400700 1 -- 172.21.15.19:0/207590777 >> 172.21.15.19:0/207590777 conn(0x7f1858078ac0 msgr2=0x7f1858078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:25:29.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.579+0000 7f185e400700 1 -- 172.21.15.19:0/207590777 shutdown_connections 2024-05-07T02:25:29.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.579+0000 7f185e400700 1 -- 172.21.15.19:0/207590777 wait complete. 2024-05-07T02:25:29.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.580+0000 7f185e400700 1 Processor -- start 2024-05-07T02:25:29.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.580+0000 7f185e400700 1 -- start start 2024-05-07T02:25:29.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.581+0000 7f185e400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18581032e0 0x7f185819ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:25:29.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.581+0000 7f185e400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1858107c10 0x7f185819d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:25:29.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.581+0000 7f185e400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1858108560 0x7f18581a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:25:29.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.581+0000 7f185e400700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f18581120c0 con 0x7f1858108560 2024-05-07T02:25:29.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.581+0000 7f185e400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1858111f40 con 0x7f1858107c10 2024-05-07T02:25:29.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.581+0000 7f185e400700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1858112240 con 0x7f18581032e0 2024-05-07T02:25:29.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.581+0000 7f185d000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18581032e0 0x7f185819ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:25:29.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.581+0000 7f185d000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18581032e0 0x7f185819ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:35210/0 (socket says 172.21.15.19:35210) 2024-05-07T02:25:29.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.581+0000 7f185d000700 1 -- 172.21.15.19:0/299261287 learned_addr learned my addr 172.21.15.19:0/299261287 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:25:29.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.581+0000 7f1857e00700 1 --2- 172.21.15.19:0/299261287 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1858107c10 0x7f185819d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:25:29.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.581+0000 7f185da00700 1 --2- 172.21.15.19:0/299261287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1858108560 0x7f18581a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:25:29.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.582+0000 7f185d000700 1 -- 172.21.15.19:0/299261287 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1858107c10 msgr2=0x7f185819d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:29.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.582+0000 7f185d000700 1 --2- 172.21.15.19:0/299261287 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1858107c10 0x7f185819d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:29.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.582+0000 7f185d000700 1 -- 172.21.15.19:0/299261287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1858108560 msgr2=0x7f18581a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:29.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.582+0000 7f185d000700 1 --2- 172.21.15.19:0/299261287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1858108560 0x7f18581a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:29.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.582+0000 7f185d000700 1 -- 172.21.15.19:0/299261287 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18581a1d20 con 0x7f18581032e0 2024-05-07T02:25:29.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.582+0000 7f185d000700 1 --2- 172.21.15.19:0/299261287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18581032e0 0x7f185819ce50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f184c00e9a0 tx=0x7f184c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:25:29.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.582+0000 7f1855600700 1 -- 172.21.15.19:0/299261287 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184c00cbd0 con 0x7f18581032e0 2024-05-07T02:25:29.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.582+0000 7f1855600700 1 -- 172.21.15.19:0/299261287 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f184c00cd70 con 0x7f18581032e0 2024-05-07T02:25:29.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.582+0000 7f185e400700 1 -- 172.21.15.19:0/299261287 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18581a2010 con 0x7f18581032e0 2024-05-07T02:25:29.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.582+0000 7f1855600700 1 -- 172.21.15.19:0/299261287 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184c018680 con 0x7f18581032e0 2024-05-07T02:25:29.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.583+0000 7f185e400700 1 -- 172.21.15.19:0/299261287 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18581a9930 con 0x7f18581032e0 2024-05-07T02:25:29.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.584+0000 7f185e400700 1 -- 172.21.15.19:0/299261287 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18580512c0 con 0x7f18581032e0 2024-05-07T02:25:29.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.586+0000 7f1855600700 1 -- 172.21.15.19:0/299261287 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f184c010430 con 0x7f18581032e0 2024-05-07T02:25:29.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.587+0000 7f1855600700 1 --2- 172.21.15.19:0/299261287 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f18400b2880 0x7f18400b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:25:29.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.588+0000 7f1855600700 1 -- 172.21.15.19:0/299261287 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f184c0e2090 con 0x7f18581032e0 2024-05-07T02:25:29.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.588+0000 7f1857e00700 1 --2- 172.21.15.19:0/299261287 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f18400b2880 0x7f18400b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:25:29.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.588+0000 7f1857e00700 1 --2- 172.21.15.19:0/299261287 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f18400b2880 0x7f18400b4d90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1858108060 tx=0x7f1844009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:25:29.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.592+0000 7f1855600700 1 -- 172.21.15.19:0/299261287 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f184c0aa1b0 con 0x7f18581032e0 2024-05-07T02:25:29.806 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:29 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:29.807 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:29 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:29.807 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:29 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:29.807 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:29 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-07T02:25:29.807 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:29 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-07T02:25:29.807 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:29 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:25:29.807 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:29 smithi064 ceph-mon[34699]: mds.? [v2:172.21.15.27:6834/1000540600,v1:172.21.15.27:6835/1000540600] up:boot 2024-05-07T02:25:29.807 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:29 smithi064 ceph-mon[34699]: fsmap 5 up:standby 2024-05-07T02:25:29.807 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:29 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-07T02:25:29.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.880+0000 7f185e400700 1 -- 172.21.15.19:0/299261287 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18581a9b00 con 0x7f18581032e0 2024-05-07T02:25:29.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.881+0000 7f1855600700 1 -- 172.21.15.19:0/299261287 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4690 (secure 0 0 0) 0x7f184c0a9ab0 con 0x7f18581032e0 2024-05-07T02:25:29.881 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:25:29.881 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/477529985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":477529985},{"type":"v1","addr":"172.21.15.64:6833","nonce":477529985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/2171873162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2171873162},{"type":"v1","addr":"172.21.15.27:6833","nonce":2171873162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24469,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2379917839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2379917839},{"type":"v1","addr":"172.21.15.19:6835","nonce":2379917839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2067515538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2067515538},{"type":"v1","addr":"172.21.15.19:6837","nonce":2067515538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1000540600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1000540600},{"type":"v1","addr":"172.21.15.27:6835","nonce":1000540600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-07T02:25:29.882 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:29 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:29.882 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:29 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:29.882 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:29 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:29.882 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:29 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-07T02:25:29.882 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:29 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-07T02:25:29.882 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:29 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:25:29.882 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:29 smithi019 ceph-mon[29781]: mds.? [v2:172.21.15.27:6834/1000540600,v1:172.21.15.27:6835/1000540600] up:boot 2024-05-07T02:25:29.882 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:29 smithi019 ceph-mon[29781]: fsmap 5 up:standby 2024-05-07T02:25:29.882 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:29 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-07T02:25:29.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.885+0000 7f185e400700 1 -- 172.21.15.19:0/299261287 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f18400b2880 msgr2=0x7f18400b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:29.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.885+0000 7f185e400700 1 --2- 172.21.15.19:0/299261287 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f18400b2880 0x7f18400b4d90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1858108060 tx=0x7f1844009260 comp rx=0 tx=0).stop 2024-05-07T02:25:29.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.885+0000 7f185e400700 1 -- 172.21.15.19:0/299261287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18581032e0 msgr2=0x7f185819ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:29.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.885+0000 7f185e400700 1 --2- 172.21.15.19:0/299261287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18581032e0 0x7f185819ce50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f184c00e9a0 tx=0x7f184c00ecb0 comp rx=0 tx=0).stop 2024-05-07T02:25:29.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.885+0000 7f185e400700 1 -- 172.21.15.19:0/299261287 shutdown_connections 2024-05-07T02:25:29.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.885+0000 7f185e400700 1 --2- 172.21.15.19:0/299261287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1858108560 0x7f18581a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:29.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.885+0000 7f185e400700 1 --2- 172.21.15.19:0/299261287 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f18581032e0 0x7f185819ce50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:29.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.885+0000 7f185e400700 1 --2- 172.21.15.19:0/299261287 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f18400b2880 0x7f18400b4d90 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:29.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.885+0000 7f185e400700 1 --2- 172.21.15.19:0/299261287 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1858107c10 0x7f185819d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:29.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.885+0000 7f185e400700 1 -- 172.21.15.19:0/299261287 >> 172.21.15.19:0/299261287 conn(0x7f1858078ac0 msgr2=0x7f1858100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:25:29.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.886+0000 7f185e400700 1 -- 172.21.15.19:0/299261287 shutdown_connections 2024-05-07T02:25:29.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:29.886+0000 7f185e400700 1 -- 172.21.15.19:0/299261287 wait complete. 2024-05-07T02:25:29.887 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 6 2024-05-07T02:25:30.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:29 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:30.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:29 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:30.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:29 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:30.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:29 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-07T02:25:30.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:29 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-07T02:25:30.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:29 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:25:30.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:29 smithi027 ceph-mon[34398]: mds.? [v2:172.21.15.27:6834/1000540600,v1:172.21.15.27:6835/1000540600] up:boot 2024-05-07T02:25:30.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:29 smithi027 ceph-mon[34398]: fsmap 5 up:standby 2024-05-07T02:25:30.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:29 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-07T02:25:30.777 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:30 smithi064 ceph-mon[34699]: Deploying daemon mds.f on smithi064 2024-05-07T02:25:30.777 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:30 smithi064 ceph-mon[34699]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:30.777 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:30 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/299261287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:25:30.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:30 smithi019 ceph-mon[29781]: Deploying daemon mds.f on smithi064 2024-05-07T02:25:30.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:30 smithi019 ceph-mon[29781]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:30.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:30 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/299261287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:25:31.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:30 smithi027 ceph-mon[34398]: Deploying daemon mds.f on smithi064 2024-05-07T02:25:31.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:30 smithi027 ceph-mon[34398]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:31.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:30 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/299261287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:25:32.108 INFO:journalctl@ceph.mds.f.smithi064.stdout:May 07 02:25:31 smithi064 systemd[1]: Started Ceph mds.f for 9ff84c16-0c17-11ef-bc97-c7b262605968. 2024-05-07T02:25:32.682 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-07T02:25:32.853 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:32 smithi064 ceph-mon[34699]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:32.853 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:32 smithi064 ceph-mon[34699]: mds.? [v2:172.21.15.64:6834/1997318872,v1:172.21.15.64:6835/1997318872] up:boot 2024-05-07T02:25:32.853 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:32 smithi064 ceph-mon[34699]: fsmap 6 up:standby 2024-05-07T02:25:32.854 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:32 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-07T02:25:32.854 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:32 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:32.854 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:32 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:32.854 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:32 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:32.854 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:32 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-07T02:25:32.854 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:32 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-07T02:25:32.854 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:32 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:25:33.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:32 smithi019 ceph-mon[29781]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:33.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:32 smithi019 ceph-mon[29781]: mds.? [v2:172.21.15.64:6834/1997318872,v1:172.21.15.64:6835/1997318872] up:boot 2024-05-07T02:25:33.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:32 smithi019 ceph-mon[29781]: fsmap 6 up:standby 2024-05-07T02:25:33.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:32 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-07T02:25:33.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:32 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:33.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:32 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:33.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:32 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:33.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:32 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-07T02:25:33.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:32 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-07T02:25:33.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:32 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:25:33.151 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:25:33.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:32 smithi027 ceph-mon[34398]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:33.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:32 smithi027 ceph-mon[34398]: mds.? [v2:172.21.15.64:6834/1997318872,v1:172.21.15.64:6835/1997318872] up:boot 2024-05-07T02:25:33.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:32 smithi027 ceph-mon[34398]: fsmap 6 up:standby 2024-05-07T02:25:33.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:32 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-07T02:25:33.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:32 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:33.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:32 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:33.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:32 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:33.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:32 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-07T02:25:33.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:32 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-07T02:25:33.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:32 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:25:33.809 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:33 smithi064 ceph-mon[34699]: Deploying daemon mds.c on smithi064 2024-05-07T02:25:34.109 INFO:journalctl@ceph.mds.c.smithi064.stdout:May 07 02:25:34 smithi064 systemd[1]: Starting Ceph mds.c for 9ff84c16-0c17-11ef-bc97-c7b262605968... 2024-05-07T02:25:34.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:33 smithi019 ceph-mon[29781]: Deploying daemon mds.c on smithi064 2024-05-07T02:25:34.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:33 smithi027 ceph-mon[34398]: Deploying daemon mds.c on smithi064 2024-05-07T02:25:34.521 INFO:journalctl@ceph.mds.c.smithi064.stdout:May 07 02:25:34 smithi064 podman[65485]: 2024-05-07 02:25:34.143131847 +0000 UTC m=+0.018113458 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-07T02:25:34.522 INFO:journalctl@ceph.mds.c.smithi064.stdout:May 07 02:25:34 smithi064 podman[65485]: 2024-05-07 02:25:34.261302556 +0000 UTC m=+0.136284138 container create a0ff8ef5c27e2d6a9aa5bb8ff64a4a70f27287736b67facc8cd17fdb5e500fef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-c, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-07T02:25:34.522 INFO:journalctl@ceph.mds.c.smithi064.stdout:May 07 02:25:34 smithi064 podman[65485]: 2024-05-07 02:25:34.522466924 +0000 UTC m=+0.397448502 container init a0ff8ef5c27e2d6a9aa5bb8ff64a4a70f27287736b67facc8cd17fdb5e500fef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-07T02:25:34.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:34 smithi064 ceph-mon[34699]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:34.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:34 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:34.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:34 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:34.859 INFO:journalctl@ceph.mds.c.smithi064.stdout:May 07 02:25:34 smithi064 podman[65485]: 2024-05-07 02:25:34.52540759 +0000 UTC m=+0.400389177 container start a0ff8ef5c27e2d6a9aa5bb8ff64a4a70f27287736b67facc8cd17fdb5e500fef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-c, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True) 2024-05-07T02:25:34.859 INFO:journalctl@ceph.mds.c.smithi064.stdout:May 07 02:25:34 smithi064 ceph-mds[65504]: -- 172.21.15.64:0/3907945964 <== mon.1 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e3de405a40 con 0x55e3de44d180 2024-05-07T02:25:34.859 INFO:journalctl@ceph.mds.c.smithi064.stdout:May 07 02:25:34 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-c[65500]: starting mds.c at 2024-05-07T02:25:34.859 INFO:journalctl@ceph.mds.c.smithi064.stdout:May 07 02:25:34 smithi064 bash[65485]: a0ff8ef5c27e2d6a9aa5bb8ff64a4a70f27287736b67facc8cd17fdb5e500fef 2024-05-07T02:25:34.859 INFO:journalctl@ceph.mds.c.smithi064.stdout:May 07 02:25:34 smithi064 systemd[1]: Started Ceph mds.c for 9ff84c16-0c17-11ef-bc97-c7b262605968. 2024-05-07T02:25:34.978 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:34 smithi019 ceph-mon[29781]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:34.978 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:34 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:34.978 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:34 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:35.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:34 smithi027 ceph-mon[34398]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:35.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:34 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:35.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:34 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:35.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.644+0000 7f5c09400700 1 -- 172.21.15.19:0/2323695262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c04106ed0 msgr2=0x7f5c04113360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:35.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.644+0000 7f5c09400700 1 --2- 172.21.15.19:0/2323695262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c04106ed0 0x7f5c04113360 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc00b0a0 tx=0x7f5bfc00b3b0 comp rx=0 tx=0).stop 2024-05-07T02:25:35.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.644+0000 7f5c09400700 1 -- 172.21.15.19:0/2323695262 shutdown_connections 2024-05-07T02:25:35.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.644+0000 7f5c09400700 1 --2- 172.21.15.19:0/2323695262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c04106ed0 0x7f5c04113360 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:35.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.644+0000 7f5c09400700 1 --2- 172.21.15.19:0/2323695262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c0410c5c0 0x7f5c0410c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:35.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.644+0000 7f5c09400700 1 --2- 172.21.15.19:0/2323695262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c04106580 0x7f5c04106990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:35.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.644+0000 7f5c09400700 1 -- 172.21.15.19:0/2323695262 >> 172.21.15.19:0/2323695262 conn(0x7f5c04102080 msgr2=0x7f5c041044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:25:35.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.644+0000 7f5c09400700 1 -- 172.21.15.19:0/2323695262 shutdown_connections 2024-05-07T02:25:35.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.644+0000 7f5c09400700 1 -- 172.21.15.19:0/2323695262 wait complete. 2024-05-07T02:25:35.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.645+0000 7f5c09400700 1 Processor -- start 2024-05-07T02:25:35.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.646+0000 7f5c09400700 1 -- start start 2024-05-07T02:25:35.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.646+0000 7f5c09400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c04106580 0x7f5c0419d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:25:35.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.646+0000 7f5c09400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c04106ed0 0x7f5c0419d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:25:35.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.646+0000 7f5c09400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c0410c5c0 0x7f5c041a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:25:35.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.646+0000 7f5c09400700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5c04116740 con 0x7f5c04106ed0 2024-05-07T02:25:35.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.646+0000 7f5c09400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5c041165c0 con 0x7f5c04106580 2024-05-07T02:25:35.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.646+0000 7f5c09400700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5c041168c0 con 0x7f5c0410c5c0 2024-05-07T02:25:35.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.647+0000 7f5c02800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c04106580 0x7f5c0419d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:25:35.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.647+0000 7f5c02800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c04106580 0x7f5c0419d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:46962/0 (socket says 172.21.15.19:46962) 2024-05-07T02:25:35.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.647+0000 7f5c02800700 1 -- 172.21.15.19:0/1177052202 learned_addr learned my addr 172.21.15.19:0/1177052202 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:25:35.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.647+0000 7f5bf9e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c04106ed0 0x7f5c0419d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:25:35.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.647+0000 7f5c03200700 1 --2- 172.21.15.19:0/1177052202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c0410c5c0 0x7f5c041a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:25:35.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.647+0000 7f5c03200700 1 -- 172.21.15.19:0/1177052202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c04106580 msgr2=0x7f5c0419d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:35.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.647+0000 7f5c03200700 1 --2- 172.21.15.19:0/1177052202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c04106580 0x7f5c0419d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:35.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.647+0000 7f5c03200700 1 -- 172.21.15.19:0/1177052202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c04106ed0 msgr2=0x7f5c0419d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:35.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.647+0000 7f5c03200700 1 --2- 172.21.15.19:0/1177052202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c04106ed0 0x7f5c0419d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:35.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.647+0000 7f5c03200700 1 -- 172.21.15.19:0/1177052202 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c041a1ee0 con 0x7f5c0410c5c0 2024-05-07T02:25:35.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.647+0000 7f5bf9e00700 1 --2- 172.21.15.19:0/1177052202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c04106ed0 0x7f5c0419d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:25:35.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.648+0000 7f5c03200700 1 --2- 172.21.15.19:0/1177052202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c0410c5c0 0x7f5c041a1840 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc00b070 tx=0x7f5bfc004c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:25:35.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.648+0000 7f5bfbe00700 1 -- 172.21.15.19:0/1177052202 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc021070 con 0x7f5c0410c5c0 2024-05-07T02:25:35.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.648+0000 7f5c09400700 1 -- 172.21.15.19:0/1177052202 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c041a2170 con 0x7f5c0410c5c0 2024-05-07T02:25:35.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.648+0000 7f5bfbe00700 1 -- 172.21.15.19:0/1177052202 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bfc007e30 con 0x7f5c0410c5c0 2024-05-07T02:25:35.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.648+0000 7f5bfbe00700 1 -- 172.21.15.19:0/1177052202 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc0045f0 con 0x7f5c0410c5c0 2024-05-07T02:25:35.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.648+0000 7f5c09400700 1 -- 172.21.15.19:0/1177052202 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c041a9af0 con 0x7f5c0410c5c0 2024-05-07T02:25:35.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.650+0000 7f5c09400700 1 -- 172.21.15.19:0/1177052202 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c0410e980 con 0x7f5c0410c5c0 2024-05-07T02:25:35.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.652+0000 7f5bfbe00700 1 -- 172.21.15.19:0/1177052202 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f5bfc007980 con 0x7f5c0410c5c0 2024-05-07T02:25:35.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.654+0000 7f5bfbe00700 1 --2- 172.21.15.19:0/1177052202 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f5be80b2900 0x7f5be80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:25:35.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.654+0000 7f5bfbe00700 1 -- 172.21.15.19:0/1177052202 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f5bfc00f020 con 0x7f5c0410c5c0 2024-05-07T02:25:35.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.654+0000 7f5c02800700 1 --2- 172.21.15.19:0/1177052202 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f5be80b2900 0x7f5be80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:25:35.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.655+0000 7f5c02800700 1 --2- 172.21.15.19:0/1177052202 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f5be80b2900 0x7f5be80b4e10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4005950 tx=0x7f5bf4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:25:35.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.659+0000 7f5bfbe00700 1 -- 172.21.15.19:0/1177052202 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5bfc02a020 con 0x7f5c0410c5c0 2024-05-07T02:25:35.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.949+0000 7f5c09400700 1 -- 172.21.15.19:0/1177052202 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c0419e370 con 0x7f5c0410c5c0 2024-05-07T02:25:35.950 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:35 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:35.951 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:35 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-07T02:25:35.951 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:35 smithi019 ceph-mon[29781]: mds.? [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] up:boot 2024-05-07T02:25:35.951 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:35 smithi019 ceph-mon[29781]: fsmap 7 up:standby 2024-05-07T02:25:35.951 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:35 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-07T02:25:35.951 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:35 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-07T02:25:35.951 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:35 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:25:35.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.950+0000 7f5bfbe00700 1 -- 172.21.15.19:0/1177052202 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6340 (secure 0 0 0) 0x7f5bfc0ab280 con 0x7f5c0410c5c0 2024-05-07T02:25:35.951 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:25:35.952 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/477529985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":477529985},{"type":"v1","addr":"172.21.15.64:6833","nonce":477529985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/2171873162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2171873162},{"type":"v1","addr":"172.21.15.27:6833","nonce":2171873162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24469,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2379917839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2379917839},{"type":"v1","addr":"172.21.15.19:6835","nonce":2379917839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2067515538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2067515538},{"type":"v1","addr":"172.21.15.19:6837","nonce":2067515538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1000540600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1000540600},{"type":"v1","addr":"172.21.15.27:6835","nonce":1000540600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1997318872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1997318872},{"type":"v1","addr":"172.21.15.64:6835","nonce":1997318872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2585907777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2585907777},{"type":"v1","addr":"172.21.15.64:6837","nonce":2585907777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-05-07T02:25:35.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.953+0000 7f5c09400700 1 -- 172.21.15.19:0/1177052202 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f5be80b2900 msgr2=0x7f5be80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:35.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.953+0000 7f5c09400700 1 --2- 172.21.15.19:0/1177052202 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f5be80b2900 0x7f5be80b4e10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4005950 tx=0x7f5bf4009260 comp rx=0 tx=0).stop 2024-05-07T02:25:35.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.953+0000 7f5c09400700 1 -- 172.21.15.19:0/1177052202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c0410c5c0 msgr2=0x7f5c041a1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:35.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.953+0000 7f5c09400700 1 --2- 172.21.15.19:0/1177052202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c0410c5c0 0x7f5c041a1840 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc00b070 tx=0x7f5bfc004c80 comp rx=0 tx=0).stop 2024-05-07T02:25:35.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.953+0000 7f5c09400700 1 -- 172.21.15.19:0/1177052202 shutdown_connections 2024-05-07T02:25:35.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.953+0000 7f5c09400700 1 --2- 172.21.15.19:0/1177052202 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5c0410c5c0 0x7f5c041a1840 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:35.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.953+0000 7f5c09400700 1 --2- 172.21.15.19:0/1177052202 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f5be80b2900 0x7f5be80b4e10 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:35.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.953+0000 7f5c09400700 1 --2- 172.21.15.19:0/1177052202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c04106580 0x7f5c0419d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:35.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.953+0000 7f5c09400700 1 --2- 172.21.15.19:0/1177052202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c04106ed0 0x7f5c0419d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:35.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.953+0000 7f5c09400700 1 -- 172.21.15.19:0/1177052202 >> 172.21.15.19:0/1177052202 conn(0x7f5c04102080 msgr2=0x7f5c041045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:25:35.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.953+0000 7f5c09400700 1 -- 172.21.15.19:0/1177052202 shutdown_connections 2024-05-07T02:25:35.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:35.954+0000 7f5c09400700 1 -- 172.21.15.19:0/1177052202 wait complete. 2024-05-07T02:25:35.955 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 8 2024-05-07T02:25:36.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:35 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:36.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:35 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-07T02:25:36.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:35 smithi064 ceph-mon[34699]: mds.? [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] up:boot 2024-05-07T02:25:36.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:35 smithi064 ceph-mon[34699]: fsmap 7 up:standby 2024-05-07T02:25:36.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:35 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-07T02:25:36.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:35 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-07T02:25:36.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:35 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:25:36.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:35 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:36.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:35 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-07T02:25:36.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:35 smithi027 ceph-mon[34398]: mds.? [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] up:boot 2024-05-07T02:25:36.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:35 smithi027 ceph-mon[34398]: fsmap 7 up:standby 2024-05-07T02:25:36.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:35 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-07T02:25:36.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:35 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-07T02:25:36.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:35 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:25:36.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:36 smithi019 ceph-mon[29781]: Deploying daemon mds.h on smithi027 2024-05-07T02:25:36.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:36 smithi019 ceph-mon[29781]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:36.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:36 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/1177052202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:25:37.057 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:36 smithi027 ceph-mon[34398]: Deploying daemon mds.h on smithi027 2024-05-07T02:25:37.057 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:36 smithi027 ceph-mon[34398]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:37.057 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:36 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/1177052202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:25:37.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:36 smithi064 ceph-mon[34699]: Deploying daemon mds.h on smithi027 2024-05-07T02:25:37.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:36 smithi064 ceph-mon[34699]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:37.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:36 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/1177052202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:25:37.314 INFO:journalctl@ceph.mds.h.smithi027.stdout:May 07 02:25:37 smithi027 podman[64834]: 2024-05-07 02:25:37.188212296 +0000 UTC m=+0.018529676 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-07T02:25:37.314 INFO:journalctl@ceph.mds.h.smithi027.stdout:May 07 02:25:37 smithi027 podman[64834]: 2024-05-07 02:25:37.315113746 +0000 UTC m=+0.145431113 container create 6d86f8fb2e15f285adf1d9c48733a5de5b1cf227baecb429e3ad239471ca6429 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-h, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, io.buildah.version=1.33.5) 2024-05-07T02:25:37.956 INFO:journalctl@ceph.mds.h.smithi027.stdout:May 07 02:25:37 smithi027 podman[64834]: 2024-05-07 02:25:37.584724236 +0000 UTC m=+0.415041602 container init 6d86f8fb2e15f285adf1d9c48733a5de5b1cf227baecb429e3ad239471ca6429 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-h, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-07T02:25:37.956 INFO:journalctl@ceph.mds.h.smithi027.stdout:May 07 02:25:37 smithi027 podman[64834]: 2024-05-07 02:25:37.587408113 +0000 UTC m=+0.417725482 container start 6d86f8fb2e15f285adf1d9c48733a5de5b1cf227baecb429e3ad239471ca6429 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-h, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_CLEAN=True, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-07T02:25:37.956 INFO:journalctl@ceph.mds.h.smithi027.stdout:May 07 02:25:37 smithi027 ceph-mds[64853]: -- 172.21.15.27:0/2894280722 <== mon.2 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562bb2ea3a40 con 0x562bb2eeb600 2024-05-07T02:25:37.957 INFO:journalctl@ceph.mds.h.smithi027.stdout:May 07 02:25:37 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-h[64849]: starting mds.h at 2024-05-07T02:25:37.957 INFO:journalctl@ceph.mds.h.smithi027.stdout:May 07 02:25:37 smithi027 bash[64834]: 6d86f8fb2e15f285adf1d9c48733a5de5b1cf227baecb429e3ad239471ca6429 2024-05-07T02:25:37.957 INFO:journalctl@ceph.mds.h.smithi027.stdout:May 07 02:25:37 smithi027 systemd[1]: Started Ceph mds.h for 9ff84c16-0c17-11ef-bc97-c7b262605968. 2024-05-07T02:25:38.917 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-07T02:25:39.006 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:38 smithi019 ceph-mon[29781]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:39.006 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:38 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:39.006 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:38 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:39.006 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:38 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:39.006 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:38 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-07T02:25:39.006 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:38 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-07T02:25:39.006 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:38 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:25:39.006 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:38 smithi019 ceph-mon[29781]: mds.? [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] up:boot 2024-05-07T02:25:39.006 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:38 smithi019 ceph-mon[29781]: fsmap 8 up:standby 2024-05-07T02:25:39.006 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:38 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-07T02:25:39.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:38 smithi064 ceph-mon[34699]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:39.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:38 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:39.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:38 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:39.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:38 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:39.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:38 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-07T02:25:39.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:38 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-07T02:25:39.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:38 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:25:39.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:38 smithi064 ceph-mon[34699]: mds.? [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] up:boot 2024-05-07T02:25:39.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:38 smithi064 ceph-mon[34699]: fsmap 8 up:standby 2024-05-07T02:25:39.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:38 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-07T02:25:39.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:38 smithi027 ceph-mon[34398]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:39.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:39.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:39.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:39.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-07T02:25:39.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-07T02:25:39.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:25:39.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:38 smithi027 ceph-mon[34398]: mds.? [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] up:boot 2024-05-07T02:25:39.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:38 smithi027 ceph-mon[34398]: fsmap 8 up:standby 2024-05-07T02:25:39.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:38 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-07T02:25:39.393 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:25:39.830 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:39 smithi019 ceph-mon[29781]: Deploying daemon mds.i on smithi064 2024-05-07T02:25:40.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:39 smithi064 ceph-mon[34699]: Deploying daemon mds.i on smithi064 2024-05-07T02:25:40.109 INFO:journalctl@ceph.mds.i.smithi064.stdout:May 07 02:25:39 smithi064 systemd[1]: Starting Ceph mds.i for 9ff84c16-0c17-11ef-bc97-c7b262605968... 2024-05-07T02:25:40.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:39 smithi027 ceph-mon[34398]: Deploying daemon mds.i on smithi064 2024-05-07T02:25:40.456 INFO:journalctl@ceph.mds.i.smithi064.stdout:May 07 02:25:40 smithi064 podman[65794]: 2024-05-07 02:25:40.191896051 +0000 UTC m=+0.115081065 container create 7d7e4fe2e2ccc47c9f87d42e2af9ed8f811a67db8bb7235f417eeec78c7e823d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux ) 2024-05-07T02:25:40.456 INFO:journalctl@ceph.mds.i.smithi064.stdout:May 07 02:25:40 smithi064 podman[65794]: 2024-05-07 02:25:40.094911747 +0000 UTC m=+0.018096744 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-07T02:25:40.456 INFO:journalctl@ceph.mds.i.smithi064.stdout:May 07 02:25:40 smithi064 podman[65794]: 2024-05-07 02:25:40.419609518 +0000 UTC m=+0.342794493 container init 7d7e4fe2e2ccc47c9f87d42e2af9ed8f811a67db8bb7235f417eeec78c7e823d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-i, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-07T02:25:40.456 INFO:journalctl@ceph.mds.i.smithi064.stdout:May 07 02:25:40 smithi064 podman[65794]: 2024-05-07 02:25:40.422310382 +0000 UTC m=+0.345495362 container start 7d7e4fe2e2ccc47c9f87d42e2af9ed8f811a67db8bb7235f417eeec78c7e823d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-i, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-07T02:25:40.456 INFO:journalctl@ceph.mds.i.smithi064.stdout:May 07 02:25:40 smithi064 ceph-mds[65813]: -- 172.21.15.64:0/379384974 <== mon.1 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555a6c9dda40 con 0x555a6ca25a80 2024-05-07T02:25:40.755 INFO:journalctl@ceph.mds.i.smithi064.stdout:May 07 02:25:40 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-i[65809]: starting mds.i at 2024-05-07T02:25:40.755 INFO:journalctl@ceph.mds.i.smithi064.stdout:May 07 02:25:40 smithi064 bash[65794]: 7d7e4fe2e2ccc47c9f87d42e2af9ed8f811a67db8bb7235f417eeec78c7e823d 2024-05-07T02:25:40.756 INFO:journalctl@ceph.mds.i.smithi064.stdout:May 07 02:25:40 smithi064 systemd[1]: Started Ceph mds.i for 9ff84c16-0c17-11ef-bc97-c7b262605968. 2024-05-07T02:25:40.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:40 smithi019 ceph-mon[29781]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:40.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:40 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:40.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:40 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:40.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:40 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:40.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:40 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-07T02:25:40.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:40 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-07T02:25:40.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:40 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:25:41.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:40 smithi064 ceph-mon[34699]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:41.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:40 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:41.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:40 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:41.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:40 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:41.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:40 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-07T02:25:41.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:40 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-07T02:25:41.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:40 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:25:41.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:40 smithi027 ceph-mon[34398]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:41.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:40 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:41.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:40 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:41.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:40 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:41.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:40 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-07T02:25:41.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:40 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-07T02:25:41.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:40 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:25:41.850 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:41 smithi064 ceph-mon[34699]: Deploying daemon mds.g on smithi019 2024-05-07T02:25:41.851 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:41 smithi064 ceph-mon[34699]: mds.? [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] up:boot 2024-05-07T02:25:41.851 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:41 smithi064 ceph-mon[34699]: fsmap 9 up:standby 2024-05-07T02:25:41.851 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:41 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-07T02:25:41.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:41.894+0000 7f0415000700 1 -- 172.21.15.19:0/961000266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04100764b0 msgr2=0x7f0410114e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:41.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:41.894+0000 7f0415000700 1 --2- 172.21.15.19:0/961000266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04100764b0 0x7f0410114e30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f03fc009990 tx=0x7f03fc009ca0 comp rx=0 tx=0).stop 2024-05-07T02:25:41.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:41.894+0000 7f0415000700 1 -- 172.21.15.19:0/961000266 shutdown_connections 2024-05-07T02:25:41.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:41.894+0000 7f0415000700 1 --2- 172.21.15.19:0/961000266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0410115370 0x7f0410117790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:41.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:41.894+0000 7f0415000700 1 --2- 172.21.15.19:0/961000266 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0410075b60 0x7f0410075f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:41.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:41.894+0000 7f0415000700 1 --2- 172.21.15.19:0/961000266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04100764b0 0x7f0410114e30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:41.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:41.894+0000 7f0415000700 1 -- 172.21.15.19:0/961000266 >> 172.21.15.19:0/961000266 conn(0x7f0410100040 msgr2=0x7f0410102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:25:41.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:41.894+0000 7f0415000700 1 -- 172.21.15.19:0/961000266 shutdown_connections 2024-05-07T02:25:41.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:41.894+0000 7f0415000700 1 -- 172.21.15.19:0/961000266 wait complete. 2024-05-07T02:25:41.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:41.896+0000 7f0415000700 1 Processor -- start 2024-05-07T02:25:41.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:41.896+0000 7f0415000700 1 -- start start 2024-05-07T02:25:41.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:41.896+0000 7f0415000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0410075b60 0x7f04101a1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:25:41.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:41.896+0000 7f0415000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04100764b0 0x7f04101a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:25:41.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:41.896+0000 7f0415000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0410115370 0x7f04101a5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:25:41.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:41.896+0000 7f0415000700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f041011aac0 con 0x7f0410115370 2024-05-07T02:25:41.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:41.896+0000 7f0415000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f041011a940 con 0x7f0410075b60 2024-05-07T02:25:41.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:41.896+0000 7f0415000700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f041011ac40 con 0x7f04100764b0 2024-05-07T02:25:41.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:41.897+0000 7f040d400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04100764b0 0x7f04101a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:25:41.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:41.897+0000 7f040de00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0410075b60 0x7f04101a1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:25:41.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:41.897+0000 7f040d400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04100764b0 0x7f04101a1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:54564/0 (socket says 172.21.15.19:54564) 2024-05-07T02:25:41.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:41.897+0000 7f040d400700 1 -- 172.21.15.19:0/3908408935 learned_addr learned my addr 172.21.15.19:0/3908408935 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:25:41.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:41.897+0000 7f040e800700 1 --2- 172.21.15.19:0/3908408935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0410115370 0x7f04101a5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:25:41.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:41.897+0000 7f040d400700 1 -- 172.21.15.19:0/3908408935 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0410075b60 msgr2=0x7f04101a1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:41.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:41.897+0000 7f040d400700 1 --2- 172.21.15.19:0/3908408935 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0410075b60 0x7f04101a1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:41.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:41.897+0000 7f040d400700 1 -- 172.21.15.19:0/3908408935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0410115370 msgr2=0x7f04101a5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:41.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:41.897+0000 7f040d400700 1 --2- 172.21.15.19:0/3908408935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0410115370 0x7f04101a5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:41.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:41.897+0000 7f040d400700 1 -- 172.21.15.19:0/3908408935 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04101a6360 con 0x7f04100764b0 2024-05-07T02:25:41.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:41.897+0000 7f040d400700 1 --2- 172.21.15.19:0/3908408935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04100764b0 0x7f04101a1a30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f03fc005910 tx=0x7f03fc00b670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:25:41.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:41.898+0000 7f0406a00700 1 -- 172.21.15.19:0/3908408935 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03fc027070 con 0x7f04100764b0 2024-05-07T02:25:41.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:41.898+0000 7f0415000700 1 -- 172.21.15.19:0/3908408935 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04101a65f0 con 0x7f04100764b0 2024-05-07T02:25:41.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:41.898+0000 7f0406a00700 1 -- 172.21.15.19:0/3908408935 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03fc022470 con 0x7f04100764b0 2024-05-07T02:25:41.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:41.898+0000 7f0406a00700 1 -- 172.21.15.19:0/3908408935 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03fc017650 con 0x7f04100764b0 2024-05-07T02:25:41.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:41.898+0000 7f0415000700 1 -- 172.21.15.19:0/3908408935 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04101a6ad0 con 0x7f04100764b0 2024-05-07T02:25:41.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:41.899+0000 7f0415000700 1 -- 172.21.15.19:0/3908408935 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04100512c0 con 0x7f04100764b0 2024-05-07T02:25:41.901 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:41 smithi019 ceph-mon[29781]: Deploying daemon mds.g on smithi019 2024-05-07T02:25:41.901 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:41 smithi019 ceph-mon[29781]: mds.? [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] up:boot 2024-05-07T02:25:41.901 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:41 smithi019 ceph-mon[29781]: fsmap 9 up:standby 2024-05-07T02:25:41.901 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:41 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-07T02:25:41.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:41.901+0000 7f0406a00700 1 -- 172.21.15.19:0/3908408935 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f03fc02a070 con 0x7f04100764b0 2024-05-07T02:25:41.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:41.903+0000 7f0406a00700 1 --2- 172.21.15.19:0/3908408935 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f03f40b2880 0x7f03f40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:25:41.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:41.903+0000 7f0406a00700 1 -- 172.21.15.19:0/3908408935 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f03fc0e2b60 con 0x7f04100764b0 2024-05-07T02:25:41.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:41.903+0000 7f040de00700 1 --2- 172.21.15.19:0/3908408935 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f03f40b2880 0x7f03f40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:25:41.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:41.904+0000 7f040de00700 1 --2- 172.21.15.19:0/3908408935 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f03f40b2880 0x7f03f40b4d90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0400009c60 tx=0x7f0400009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:25:41.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:41.908+0000 7f0406a00700 1 -- 172.21.15.19:0/3908408935 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f03fc013070 con 0x7f04100764b0 2024-05-07T02:25:42.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:41 smithi027 ceph-mon[34398]: Deploying daemon mds.g on smithi019 2024-05-07T02:25:42.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:41 smithi027 ceph-mon[34398]: mds.? [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] up:boot 2024-05-07T02:25:42.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:41 smithi027 ceph-mon[34398]: fsmap 9 up:standby 2024-05-07T02:25:42.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:41 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-07T02:25:42.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:42.206+0000 7f0415000700 1 -- 172.21.15.19:0/3908408935 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f041002e5f0 con 0x7f04100764b0 2024-05-07T02:25:42.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:42.207+0000 7f0406a00700 1 -- 172.21.15.19:0/3908408935 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+7992 (secure 0 0 0) 0x7f03fc0aad10 con 0x7f04100764b0 2024-05-07T02:25:42.208 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:25:42.208 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/477529985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":477529985},{"type":"v1","addr":"172.21.15.64:6833","nonce":477529985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/2171873162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2171873162},{"type":"v1","addr":"172.21.15.27:6833","nonce":2171873162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24469,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2379917839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2379917839},{"type":"v1","addr":"172.21.15.19:6835","nonce":2379917839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2067515538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2067515538},{"type":"v1","addr":"172.21.15.19:6837","nonce":2067515538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1000540600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1000540600},{"type":"v1","addr":"172.21.15.27:6835","nonce":1000540600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1997318872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1997318872},{"type":"v1","addr":"172.21.15.64:6835","nonce":1997318872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2585907777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2585907777},{"type":"v1","addr":"172.21.15.64:6837","nonce":2585907777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24515,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2583138143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2583138143},{"type":"v1","addr":"172.21.15.27:6837","nonce":2583138143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/1583120004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":1583120004},{"type":"v1","addr":"172.21.15.64:6839","nonce":1583120004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-07T02:25:42.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:42.210+0000 7f0415000700 1 -- 172.21.15.19:0/3908408935 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f03f40b2880 msgr2=0x7f03f40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:42.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:42.210+0000 7f0415000700 1 --2- 172.21.15.19:0/3908408935 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f03f40b2880 0x7f03f40b4d90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0400009c60 tx=0x7f0400009260 comp rx=0 tx=0).stop 2024-05-07T02:25:42.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:42.210+0000 7f0415000700 1 -- 172.21.15.19:0/3908408935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04100764b0 msgr2=0x7f04101a1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:42.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:42.210+0000 7f0415000700 1 --2- 172.21.15.19:0/3908408935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04100764b0 0x7f04101a1a30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f03fc005910 tx=0x7f03fc00b670 comp rx=0 tx=0).stop 2024-05-07T02:25:42.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:42.211+0000 7f0415000700 1 -- 172.21.15.19:0/3908408935 shutdown_connections 2024-05-07T02:25:42.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:42.211+0000 7f0415000700 1 --2- 172.21.15.19:0/3908408935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0410115370 0x7f04101a5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:42.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:42.211+0000 7f0415000700 1 --2- 172.21.15.19:0/3908408935 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f03f40b2880 0x7f03f40b4d90 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:42.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:42.211+0000 7f0415000700 1 --2- 172.21.15.19:0/3908408935 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0410075b60 0x7f04101a1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:42.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:42.211+0000 7f0415000700 1 --2- 172.21.15.19:0/3908408935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04100764b0 0x7f04101a1a30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:42.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:42.211+0000 7f0415000700 1 -- 172.21.15.19:0/3908408935 >> 172.21.15.19:0/3908408935 conn(0x7f0410100040 msgr2=0x7f04101006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:25:42.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:42.211+0000 7f0415000700 1 -- 172.21.15.19:0/3908408935 shutdown_connections 2024-05-07T02:25:42.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:42.211+0000 7f0415000700 1 -- 172.21.15.19:0/3908408935 wait complete. 2024-05-07T02:25:42.213 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 10 2024-05-07T02:25:43.102 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:42 smithi019 ceph-mon[29781]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:43.102 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:42 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/3908408935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:25:43.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:42 smithi064 ceph-mon[34699]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:43.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:42 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/3908408935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:25:43.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:42 smithi027 ceph-mon[34398]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:43.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:42 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/3908408935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:25:44.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:44 smithi019 ceph-mon[29781]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:45.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:44 smithi064 ceph-mon[34699]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:45.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:44 smithi027 ceph-mon[34398]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:45.326 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 07 02:25:44 smithi019 podman[71578]: 2024-05-07 02:25:44.995957952 +0000 UTC m=+0.163729209 container create c5b8a079a2c9c679c2edd12c2d8cafe85454bb587b44e96714343d65c5607e02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-g, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-07T02:25:45.326 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 07 02:25:45 smithi019 podman[71578]: 2024-05-07 02:25:45.324998408 +0000 UTC m=+0.492769672 container init c5b8a079a2c9c679c2edd12c2d8cafe85454bb587b44e96714343d65c5607e02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-07T02:25:45.645 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 07 02:25:45 smithi019 podman[71578]: 2024-05-07 02:25:45.335633945 +0000 UTC m=+0.503405177 container start c5b8a079a2c9c679c2edd12c2d8cafe85454bb587b44e96714343d65c5607e02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-g, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=) 2024-05-07T02:25:45.645 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 07 02:25:45 smithi019 bash[71578]: c5b8a079a2c9c679c2edd12c2d8cafe85454bb587b44e96714343d65c5607e02 2024-05-07T02:25:45.645 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 07 02:25:45 smithi019 systemd[1]: Started Ceph mds.g for 9ff84c16-0c17-11ef-bc97-c7b262605968. 2024-05-07T02:25:45.645 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 07 02:25:45 smithi019 ceph-mds[71597]: -- 172.21.15.19:0/1635895497 <== mon.1 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d65ba61a40 con 0x55d65baa9180 2024-05-07T02:25:45.645 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 07 02:25:45 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-g[71593]: starting mds.g at 2024-05-07T02:25:46.167 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-07T02:25:46.824 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:46 smithi019 ceph-mon[29781]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:46.824 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:46 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:46.824 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:46 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:46.824 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:46 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:46.824 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:46 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-07T02:25:46.824 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:46 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-07T02:25:46.824 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:46 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:25:46.824 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:46 smithi019 ceph-mon[29781]: Deploying daemon mds.a on smithi019 2024-05-07T02:25:46.824 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:46 smithi019 ceph-mon[29781]: mds.? [v2:172.21.15.19:6838/1763670446,v1:172.21.15.19:6839/1763670446] up:boot 2024-05-07T02:25:46.825 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:46 smithi019 ceph-mon[29781]: fsmap 10 up:standby 2024-05-07T02:25:46.825 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:46 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-07T02:25:46.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:46 smithi064 ceph-mon[34699]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:46.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:46 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:46.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:46 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:46.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:46 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:46.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:46 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-07T02:25:46.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:46 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-07T02:25:46.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:46 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:25:46.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:46 smithi064 ceph-mon[34699]: Deploying daemon mds.a on smithi019 2024-05-07T02:25:46.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:46 smithi064 ceph-mon[34699]: mds.? [v2:172.21.15.19:6838/1763670446,v1:172.21.15.19:6839/1763670446] up:boot 2024-05-07T02:25:46.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:46 smithi064 ceph-mon[34699]: fsmap 10 up:standby 2024-05-07T02:25:46.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:46 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-07T02:25:46.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:46 smithi027 ceph-mon[34398]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:46.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:46 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:46.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:46 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:46.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:46 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:46.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:46 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-07T02:25:46.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:46 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-07T02:25:46.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:46 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:25:46.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:46 smithi027 ceph-mon[34398]: Deploying daemon mds.a on smithi019 2024-05-07T02:25:46.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:46 smithi027 ceph-mon[34398]: mds.? [v2:172.21.15.19:6838/1763670446,v1:172.21.15.19:6839/1763670446] up:boot 2024-05-07T02:25:46.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:46 smithi027 ceph-mon[34398]: fsmap 10 up:standby 2024-05-07T02:25:46.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:46 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-07T02:25:47.337 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:25:48.738 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:48 smithi019 ceph-mon[29781]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:48.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:48 smithi064 ceph-mon[34699]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:48.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:48 smithi027 ceph-mon[34398]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:49.146 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 07 02:25:48 smithi019 systemd[1]: Starting Ceph mds.a for 9ff84c16-0c17-11ef-bc97-c7b262605968... 2024-05-07T02:25:50.089 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 07 02:25:49 smithi019 podman[71975]: 2024-05-07 02:25:49.736357046 +0000 UTC m=+0.036085855 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-07T02:25:50.089 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 07 02:25:49 smithi019 podman[71975]: 2024-05-07 02:25:49.869642386 +0000 UTC m=+0.169371235 container create 34df7d493449bf07acead413e6c1d11e86f9a50a258a548b206839fb75c65669 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-a, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD) 2024-05-07T02:25:50.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:50 smithi019 ceph-mon[29781]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:50.645 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 07 02:25:50 smithi019 podman[71975]: 2024-05-07 02:25:50.358989167 +0000 UTC m=+0.658718007 container init 34df7d493449bf07acead413e6c1d11e86f9a50a258a548b206839fb75c65669 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-a, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True) 2024-05-07T02:25:50.645 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 07 02:25:50 smithi019 podman[71975]: 2024-05-07 02:25:50.369327453 +0000 UTC m=+0.669056293 container start 34df7d493449bf07acead413e6c1d11e86f9a50a258a548b206839fb75c65669 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-a, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-07T02:25:50.645 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 07 02:25:50 smithi019 ceph-mds[72017]: -- 172.21.15.19:0/2126794579 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55778849fa40 con 0x5577884e7180 2024-05-07T02:25:50.645 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 07 02:25:50 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-a[72008]: starting mds.a at 2024-05-07T02:25:50.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:50.719+0000 7fded1c00700 1 -- 172.21.15.19:0/1955168675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdecc109850 msgr2=0x7fdecc1111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:50.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:50.719+0000 7fded1c00700 1 --2- 172.21.15.19:0/1955168675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdecc109850 0x7fdecc1111b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdec400b0d0 tx=0x7fdec400b3e0 comp rx=0 tx=0).stop 2024-05-07T02:25:50.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:50.719+0000 7fded1c00700 1 -- 172.21.15.19:0/1955168675 shutdown_connections 2024-05-07T02:25:50.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:50.719+0000 7fded1c00700 1 --2- 172.21.15.19:0/1955168675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdecc109850 0x7fdecc1111b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:50.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:50.719+0000 7fded1c00700 1 --2- 172.21.15.19:0/1955168675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdecc1045d0 0x7fdecc1049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:50.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:50.719+0000 7fded1c00700 1 --2- 172.21.15.19:0/1955168675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdecc108f00 0x7fdecc109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:50.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:50.719+0000 7fded1c00700 1 -- 172.21.15.19:0/1955168675 >> 172.21.15.19:0/1955168675 conn(0x7fdecc0ffea0 msgr2=0x7fdecc1022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:25:50.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:50.719+0000 7fded1c00700 1 -- 172.21.15.19:0/1955168675 shutdown_connections 2024-05-07T02:25:50.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:50.719+0000 7fded1c00700 1 -- 172.21.15.19:0/1955168675 wait complete. 2024-05-07T02:25:50.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:50.721+0000 7fded1c00700 1 Processor -- start 2024-05-07T02:25:50.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:50.721+0000 7fded1c00700 1 -- start start 2024-05-07T02:25:50.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:50.721+0000 7fded1c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdecc1045d0 0x7fdecc076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:25:50.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:50.721+0000 7fded1c00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdecc108f00 0x7fdecc077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:25:50.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:50.721+0000 7fded1c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdecc109850 0x7fdecc077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:25:50.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:50.721+0000 7fded1c00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdecc114480 con 0x7fdecc1045d0 2024-05-07T02:25:50.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:50.721+0000 7fded1c00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdecc114300 con 0x7fdecc109850 2024-05-07T02:25:50.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:50.721+0000 7fded1c00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdecc114600 con 0x7fdecc108f00 2024-05-07T02:25:50.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:50.722+0000 7fdecaa00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdecc1045d0 0x7fdecc076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:25:50.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:50.722+0000 7fdec3e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdecc108f00 0x7fdecc077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:25:50.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:50.722+0000 7fdecaa00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdecc1045d0 0x7fdecc076bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50072/0 (socket says 172.21.15.19:50072) 2024-05-07T02:25:50.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:50.722+0000 7fdecaa00700 1 -- 172.21.15.19:0/611885466 learned_addr learned my addr 172.21.15.19:0/611885466 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:25:50.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:50.722+0000 7fdecb400700 1 --2- 172.21.15.19:0/611885466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdecc109850 0x7fdecc077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:25:50.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:50.722+0000 7fdecb400700 1 -- 172.21.15.19:0/611885466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdecc108f00 msgr2=0x7fdecc077110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:50.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:50.722+0000 7fdecb400700 1 --2- 172.21.15.19:0/611885466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdecc108f00 0x7fdecc077110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:50.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:50.722+0000 7fdecb400700 1 -- 172.21.15.19:0/611885466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdecc1045d0 msgr2=0x7fdecc076bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:50.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:50.722+0000 7fdecb400700 1 --2- 172.21.15.19:0/611885466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdecc1045d0 0x7fdecc076bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:50.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:50.722+0000 7fdecb400700 1 -- 172.21.15.19:0/611885466 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdecc1a9110 con 0x7fdecc109850 2024-05-07T02:25:50.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:50.722+0000 7fdec3e00700 1 --2- 172.21.15.19:0/611885466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdecc108f00 0x7fdecc077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:25:50.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:50.722+0000 7fdecaa00700 1 --2- 172.21.15.19:0/611885466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdecc1045d0 0x7fdecc076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:25:50.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:50.723+0000 7fdecb400700 1 --2- 172.21.15.19:0/611885466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdecc109850 0x7fdecc077650 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdec400b0d0 tx=0x7fdec4007a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:25:50.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:50.723+0000 7fdec3400700 1 -- 172.21.15.19:0/611885466 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec401f070 con 0x7fdecc109850 2024-05-07T02:25:50.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:50.723+0000 7fdec3400700 1 -- 172.21.15.19:0/611885466 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdec4004c50 con 0x7fdecc109850 2024-05-07T02:25:50.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:50.723+0000 7fdec3400700 1 -- 172.21.15.19:0/611885466 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec401b430 con 0x7fdecc109850 2024-05-07T02:25:50.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:50.723+0000 7fded1c00700 1 -- 172.21.15.19:0/611885466 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdecc1a93a0 con 0x7fdecc109850 2024-05-07T02:25:50.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:50.723+0000 7fded1c00700 1 -- 172.21.15.19:0/611885466 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdecc1a97f0 con 0x7fdecc109850 2024-05-07T02:25:50.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:50.724+0000 7fded1c00700 1 -- 172.21.15.19:0/611885466 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdecc0512c0 con 0x7fdecc109850 2024-05-07T02:25:50.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:50.727+0000 7fdec3400700 1 -- 172.21.15.19:0/611885466 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fdec4022070 con 0x7fdecc109850 2024-05-07T02:25:50.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:50.728+0000 7fdec3400700 1 --2- 172.21.15.19:0/611885466 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fdeb00b2880 0x7fdeb00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:25:50.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:50.729+0000 7fdec3400700 1 -- 172.21.15.19:0/611885466 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fdec400f020 con 0x7fdecc109850 2024-05-07T02:25:50.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:50.729+0000 7fdecaa00700 1 --2- 172.21.15.19:0/611885466 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fdeb00b2880 0x7fdeb00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:25:50.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:50.729+0000 7fdecaa00700 1 --2- 172.21.15.19:0/611885466 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fdeb00b2880 0x7fdeb00b4d90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdebc011550 tx=0x7fdebc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:25:50.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:50.733+0000 7fdec3400700 1 -- 172.21.15.19:0/611885466 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdec4029020 con 0x7fdecc109850 2024-05-07T02:25:50.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:50 smithi064 ceph-mon[34699]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:50.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:50 smithi027 ceph-mon[34398]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:51.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:51.027+0000 7fded1c00700 1 -- 172.21.15.19:0/611885466 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdecc02e690 con 0x7fdecc109850 2024-05-07T02:25:51.029 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 07 02:25:50 smithi019 bash[71975]: 34df7d493449bf07acead413e6c1d11e86f9a50a258a548b206839fb75c65669 2024-05-07T02:25:51.029 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 07 02:25:50 smithi019 systemd[1]: Started Ceph mds.a for 9ff84c16-0c17-11ef-bc97-c7b262605968. 2024-05-07T02:25:51.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:51.029+0000 7fdec3400700 1 -- 172.21.15.19:0/611885466 <== mon.2 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9644 (secure 0 0 0) 0x7fdec40aae70 con 0x7fdecc109850 2024-05-07T02:25:51.030 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:25:51.030 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1763670446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1763670446},{"type":"v1","addr":"172.21.15.19:6839","nonce":1763670446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/477529985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":477529985},{"type":"v1","addr":"172.21.15.64:6833","nonce":477529985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/2171873162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2171873162},{"type":"v1","addr":"172.21.15.27:6833","nonce":2171873162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24469,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2379917839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2379917839},{"type":"v1","addr":"172.21.15.19:6835","nonce":2379917839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2067515538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2067515538},{"type":"v1","addr":"172.21.15.19:6837","nonce":2067515538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1000540600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1000540600},{"type":"v1","addr":"172.21.15.27:6835","nonce":1000540600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1997318872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1997318872},{"type":"v1","addr":"172.21.15.64:6835","nonce":1997318872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2585907777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2585907777},{"type":"v1","addr":"172.21.15.64:6837","nonce":2585907777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24515,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2583138143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2583138143},{"type":"v1","addr":"172.21.15.27:6837","nonce":2583138143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/1583120004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":1583120004},{"type":"v1","addr":"172.21.15.64:6839","nonce":1583120004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24539,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/3905645246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3905645246},{"type":"v1","addr":"172.21.15.19:6841","nonce":3905645246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[]} 2024-05-07T02:25:51.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:51.031+0000 7fded1c00700 1 -- 172.21.15.19:0/611885466 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fdeb00b2880 msgr2=0x7fdeb00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:51.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:51.031+0000 7fded1c00700 1 --2- 172.21.15.19:0/611885466 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fdeb00b2880 0x7fdeb00b4d90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdebc011550 tx=0x7fdebc009260 comp rx=0 tx=0).stop 2024-05-07T02:25:51.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:51.032+0000 7fded1c00700 1 -- 172.21.15.19:0/611885466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdecc109850 msgr2=0x7fdecc077650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:51.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:51.032+0000 7fded1c00700 1 --2- 172.21.15.19:0/611885466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdecc109850 0x7fdecc077650 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdec400b0d0 tx=0x7fdec4007a00 comp rx=0 tx=0).stop 2024-05-07T02:25:51.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:51.032+0000 7fded1c00700 1 -- 172.21.15.19:0/611885466 shutdown_connections 2024-05-07T02:25:51.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:51.032+0000 7fded1c00700 1 --2- 172.21.15.19:0/611885466 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fdeb00b2880 0x7fdeb00b4d90 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:51.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:51.032+0000 7fded1c00700 1 --2- 172.21.15.19:0/611885466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdecc109850 0x7fdecc077650 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:51.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:51.032+0000 7fded1c00700 1 --2- 172.21.15.19:0/611885466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdecc1045d0 0x7fdecc076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:51.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:51.032+0000 7fded1c00700 1 --2- 172.21.15.19:0/611885466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdecc108f00 0x7fdecc077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:51.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:51.032+0000 7fded1c00700 1 -- 172.21.15.19:0/611885466 >> 172.21.15.19:0/611885466 conn(0x7fdecc0ffea0 msgr2=0x7fdecc108670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:25:51.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:51.032+0000 7fded1c00700 1 -- 172.21.15.19:0/611885466 shutdown_connections 2024-05-07T02:25:51.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:51.032+0000 7fded1c00700 1 -- 172.21.15.19:0/611885466 wait complete. 2024-05-07T02:25:51.034 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 12 2024-05-07T02:25:51.804 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:51 smithi027 ceph-mon[34398]: mds.? [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] up:boot 2024-05-07T02:25:51.804 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:51 smithi027 ceph-mon[34398]: fsmap 11 up:standby 2024-05-07T02:25:51.804 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:51 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-07T02:25:51.804 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:51 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:51.804 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:51 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:51.804 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:51 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:51.804 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:51 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-07T02:25:51.804 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:51 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-07T02:25:51.804 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:51 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:25:51.805 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:51 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/611885466' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:25:51.833 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:51 smithi019 ceph-mon[29781]: mds.? [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] up:boot 2024-05-07T02:25:51.833 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:51 smithi019 ceph-mon[29781]: fsmap 11 up:standby 2024-05-07T02:25:51.833 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:51 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-07T02:25:51.833 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:51 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:51.833 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:51 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:51.833 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:51 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:51.833 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:51 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-07T02:25:51.833 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:51 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-07T02:25:51.833 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:51 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:25:51.834 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:51 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/611885466' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:25:51.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:51 smithi064 ceph-mon[34699]: mds.? [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] up:boot 2024-05-07T02:25:51.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:51 smithi064 ceph-mon[34699]: fsmap 11 up:standby 2024-05-07T02:25:51.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:51 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-07T02:25:51.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:51 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:51.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:51 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:51.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:51 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:51.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:51 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-07T02:25:51.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:51 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-07T02:25:51.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:51 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:25:51.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:51 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/611885466' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:25:52.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:52 smithi064 ceph-mon[34699]: Deploying daemon mds.b on smithi027 2024-05-07T02:25:52.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:52 smithi064 ceph-mon[34699]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:52.833 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:52 smithi027 ceph-mon[34398]: Deploying daemon mds.b on smithi027 2024-05-07T02:25:52.833 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:52 smithi027 ceph-mon[34398]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:52.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:52 smithi019 ceph-mon[29781]: Deploying daemon mds.b on smithi027 2024-05-07T02:25:52.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:52 smithi019 ceph-mon[29781]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:53.207 INFO:journalctl@ceph.mds.b.smithi027.stdout:May 07 02:25:53 smithi027 systemd[1]: Starting Ceph mds.b for 9ff84c16-0c17-11ef-bc97-c7b262605968... 2024-05-07T02:25:53.581 INFO:journalctl@ceph.mds.b.smithi027.stdout:May 07 02:25:53 smithi027 podman[65144]: 2024-05-07 02:25:53.150262567 +0000 UTC m=+0.018710562 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-07T02:25:53.582 INFO:journalctl@ceph.mds.b.smithi027.stdout:May 07 02:25:53 smithi027 podman[65144]: 2024-05-07 02:25:53.337753385 +0000 UTC m=+0.206201367 container create 92b7019a59cef8ce29cac6e03a37e63b48c1b781fc9a143bcdfe79df5cce0c22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, ceph=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-07T02:25:53.582 INFO:journalctl@ceph.mds.b.smithi027.stdout:May 07 02:25:53 smithi027 podman[65144]: 2024-05-07 02:25:53.582741168 +0000 UTC m=+0.451189154 container init 92b7019a59cef8ce29cac6e03a37e63b48c1b781fc9a143bcdfe79df5cce0c22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430) 2024-05-07T02:25:53.900 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-07T02:25:53.956 INFO:journalctl@ceph.mds.b.smithi027.stdout:May 07 02:25:53 smithi027 podman[65144]: 2024-05-07 02:25:53.585778479 +0000 UTC m=+0.454226473 container start 92b7019a59cef8ce29cac6e03a37e63b48c1b781fc9a143bcdfe79df5cce0c22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-b, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-07T02:25:53.956 INFO:journalctl@ceph.mds.b.smithi027.stdout:May 07 02:25:53 smithi027 ceph-mds[65163]: -- 172.21.15.27:0/3154132585 <== mon.2 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562b57619a40 con 0x562b57661180 2024-05-07T02:25:53.956 INFO:journalctl@ceph.mds.b.smithi027.stdout:May 07 02:25:53 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-b[65159]: starting mds.b at 2024-05-07T02:25:53.957 INFO:journalctl@ceph.mds.b.smithi027.stdout:May 07 02:25:53 smithi027 bash[65144]: 92b7019a59cef8ce29cac6e03a37e63b48c1b781fc9a143bcdfe79df5cce0c22 2024-05-07T02:25:53.957 INFO:journalctl@ceph.mds.b.smithi027.stdout:May 07 02:25:53 smithi027 systemd[1]: Started Ceph mds.b for 9ff84c16-0c17-11ef-bc97-c7b262605968. 2024-05-07T02:25:54.345 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:25:55.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:54 smithi019 ceph-mon[29781]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:55.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:54 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:55.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:54 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:55.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:54 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:55.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:54 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:55.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:54 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:55.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:54 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:25:55.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:54 smithi019 ceph-mon[29781]: mds.? [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] up:boot 2024-05-07T02:25:55.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:54 smithi019 ceph-mon[29781]: fsmap 12 up:standby 2024-05-07T02:25:55.011 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:54 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-07T02:25:55.062 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:54 smithi027 ceph-mon[34398]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:55.062 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:54 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:55.062 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:54 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:55.062 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:54 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:55.063 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:54 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:55.063 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:54 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:55.063 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:54 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:25:55.063 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:54 smithi027 ceph-mon[34398]: mds.? [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] up:boot 2024-05-07T02:25:55.063 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:54 smithi027 ceph-mon[34398]: fsmap 12 up:standby 2024-05-07T02:25:55.063 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:54 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-07T02:25:55.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:54 smithi064 ceph-mon[34699]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:55.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:54 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:55.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:54 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:55.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:54 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:55.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:54 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:55.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:54 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:55.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:54 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:25:55.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:54 smithi064 ceph-mon[34699]: mds.? [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] up:boot 2024-05-07T02:25:55.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:54 smithi064 ceph-mon[34699]: fsmap 12 up:standby 2024-05-07T02:25:55.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:54 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-07T02:25:56.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:56.942+0000 7f3be1a00700 1 -- 172.21.15.19:0/3376459398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bdc105be0 msgr2=0x7f3bdc10e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:56.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:56.942+0000 7f3be1a00700 1 --2- 172.21.15.19:0/3376459398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bdc105be0 0x7f3bdc10e130 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4009990 tx=0x7f3bc4009ca0 comp rx=0 tx=0).stop 2024-05-07T02:25:56.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:56.942+0000 7f3be1a00700 1 -- 172.21.15.19:0/3376459398 shutdown_connections 2024-05-07T02:25:56.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:56.942+0000 7f3be1a00700 1 --2- 172.21.15.19:0/3376459398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bdc10e700 0x7f3bdc110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:56.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:56.942+0000 7f3be1a00700 1 --2- 172.21.15.19:0/3376459398 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bdc105290 0x7f3bdc1056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:56.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:56.942+0000 7f3be1a00700 1 --2- 172.21.15.19:0/3376459398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bdc105be0 0x7f3bdc10e130 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:56.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:56.942+0000 7f3be1a00700 1 -- 172.21.15.19:0/3376459398 >> 172.21.15.19:0/3376459398 conn(0x7f3bdc078ac0 msgr2=0x7f3bdc078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:25:56.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:56.943+0000 7f3be1a00700 1 -- 172.21.15.19:0/3376459398 shutdown_connections 2024-05-07T02:25:56.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:56.943+0000 7f3be1a00700 1 -- 172.21.15.19:0/3376459398 wait complete. 2024-05-07T02:25:56.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:56.944+0000 7f3be1a00700 1 Processor -- start 2024-05-07T02:25:56.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:56.944+0000 7f3be1a00700 1 -- start start 2024-05-07T02:25:56.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:56.945+0000 7f3be1a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bdc105290 0x7f3bdc102f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:25:56.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:56.945+0000 7f3be1a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bdc105be0 0x7f3bdc1034c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:25:56.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:56.945+0000 7f3be1a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bdc10e700 0x7f3bdc103a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:25:56.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:56.945+0000 7f3be1a00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3bdc1142b0 con 0x7f3bdc10e700 2024-05-07T02:25:56.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:56.945+0000 7f3be1a00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3bdc114130 con 0x7f3bdc105be0 2024-05-07T02:25:56.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:56.945+0000 7f3be1a00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3bdc114430 con 0x7f3bdc105290 2024-05-07T02:25:56.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:56.945+0000 7f3bdbe00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bdc105290 0x7f3bdc102f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:25:56.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:56.945+0000 7f3be1000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bdc10e700 0x7f3bdc103a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:25:56.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:56.945+0000 7f3bdbe00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bdc105290 0x7f3bdc102f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:49278/0 (socket says 172.21.15.19:49278) 2024-05-07T02:25:56.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:56.945+0000 7f3bdbe00700 1 -- 172.21.15.19:0/2083398661 learned_addr learned my addr 172.21.15.19:0/2083398661 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:25:56.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:56.945+0000 7f3bd3400700 1 --2- 172.21.15.19:0/2083398661 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bdc105be0 0x7f3bdc1034c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:25:56.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:56.946+0000 7f3bd3400700 1 -- 172.21.15.19:0/2083398661 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bdc105290 msgr2=0x7f3bdc102f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:56.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:56.946+0000 7f3bd3400700 1 --2- 172.21.15.19:0/2083398661 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bdc105290 0x7f3bdc102f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:56.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:56.946+0000 7f3bd3400700 1 -- 172.21.15.19:0/2083398661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bdc10e700 msgr2=0x7f3bdc103a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:56.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:56.946+0000 7f3bd3400700 1 --2- 172.21.15.19:0/2083398661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bdc10e700 0x7f3bdc103a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:56.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:56.946+0000 7f3bd3400700 1 -- 172.21.15.19:0/2083398661 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bdc1a9080 con 0x7f3bdc105be0 2024-05-07T02:25:56.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:56.946+0000 7f3be1000700 1 --2- 172.21.15.19:0/2083398661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bdc10e700 0x7f3bdc103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:25:56.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:56.946+0000 7f3bdbe00700 1 --2- 172.21.15.19:0/2083398661 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bdc105290 0x7f3bdc102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:25:56.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:56.946+0000 7f3bd3400700 1 --2- 172.21.15.19:0/2083398661 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bdc105be0 0x7f3bdc1034c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4000c00 tx=0x7f3bc40047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:25:56.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:56.947+0000 7f3bd9600700 1 -- 172.21.15.19:0/2083398661 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc4026070 con 0x7f3bdc105be0 2024-05-07T02:25:56.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:56.947+0000 7f3bd9600700 1 -- 172.21.15.19:0/2083398661 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bc4021410 con 0x7f3bdc105be0 2024-05-07T02:25:56.947 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:56 smithi019 ceph-mon[29781]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:56.947 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:56 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:56.947 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:56 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:56.948 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:56 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:56.948 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:56 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:56.948 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:56 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:56.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:56.947+0000 7f3be1a00700 1 -- 172.21.15.19:0/2083398661 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bdc1a9310 con 0x7f3bdc105be0 2024-05-07T02:25:56.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:56.947+0000 7f3bd9600700 1 -- 172.21.15.19:0/2083398661 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc4017650 con 0x7f3bdc105be0 2024-05-07T02:25:56.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:56.947+0000 7f3be1a00700 1 -- 172.21.15.19:0/2083398661 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bdc1a9840 con 0x7f3bdc105be0 2024-05-07T02:25:56.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:56.948+0000 7f3be1a00700 1 -- 172.21.15.19:0/2083398661 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bdc0512c0 con 0x7f3bdc105be0 2024-05-07T02:25:56.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:56.950+0000 7f3bd9600700 1 -- 172.21.15.19:0/2083398661 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f3bc4029070 con 0x7f3bdc105be0 2024-05-07T02:25:56.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:56.952+0000 7f3bd9600700 1 --2- 172.21.15.19:0/2083398661 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f3bc00b2900 0x7f3bc00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:25:56.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:56.952+0000 7f3bd9600700 1 -- 172.21.15.19:0/2083398661 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f3bc40e2a50 con 0x7f3bdc105be0 2024-05-07T02:25:56.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:56.952+0000 7f3bdbe00700 1 --2- 172.21.15.19:0/2083398661 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f3bc00b2900 0x7f3bc00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:25:56.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:56.953+0000 7f3bdbe00700 1 --2- 172.21.15.19:0/2083398661 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f3bc00b2900 0x7f3bc00b4e10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc011550 tx=0x7f3bcc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:25:56.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:56.957+0000 7f3bd9600700 1 -- 172.21.15.19:0/2083398661 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3bc4013070 con 0x7f3bdc105be0 2024-05-07T02:25:57.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:56 smithi064 ceph-mon[34699]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:57.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:56 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:57.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:56 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:57.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:56 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:57.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:56 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:57.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:56 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:57.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:56 smithi027 ceph-mon[34398]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:57.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:56 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:57.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:56 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:57.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:56 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:57.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:56 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:57.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:56 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:25:57.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:57.245+0000 7f3be1a00700 1 -- 172.21.15.19:0/2083398661 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3bdc02e5f0 con 0x7f3bdc105be0 2024-05-07T02:25:57.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:57.246+0000 7f3bd9600700 1 -- 172.21.15.19:0/2083398661 <== mon.2 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10470 (secure 0 0 0) 0x7f3bc40aac00 con 0x7f3bdc105be0 2024-05-07T02:25:57.247 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:25:57.247 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1763670446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1763670446},{"type":"v1","addr":"172.21.15.19:6839","nonce":1763670446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/477529985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":477529985},{"type":"v1","addr":"172.21.15.64:6833","nonce":477529985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/2171873162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2171873162},{"type":"v1","addr":"172.21.15.27:6833","nonce":2171873162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24469,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2379917839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2379917839},{"type":"v1","addr":"172.21.15.19:6835","nonce":2379917839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2067515538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2067515538},{"type":"v1","addr":"172.21.15.19:6837","nonce":2067515538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1000540600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1000540600},{"type":"v1","addr":"172.21.15.27:6835","nonce":1000540600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1997318872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1997318872},{"type":"v1","addr":"172.21.15.64:6835","nonce":1997318872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2585907777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2585907777},{"type":"v1","addr":"172.21.15.64:6837","nonce":2585907777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24515,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2583138143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2583138143},{"type":"v1","addr":"172.21.15.27:6837","nonce":2583138143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/1583120004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":1583120004},{"type":"v1","addr":"172.21.15.64:6839","nonce":1583120004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24539,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/3905645246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3905645246},{"type":"v1","addr":"172.21.15.19:6841","nonce":3905645246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24551,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6839/3963268634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3963268634},{"type":"v1","addr":"172.21.15.27:6839","nonce":3963268634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-05-07T02:25:57.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:57.249+0000 7f3be1a00700 1 -- 172.21.15.19:0/2083398661 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f3bc00b2900 msgr2=0x7f3bc00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:57.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:57.249+0000 7f3be1a00700 1 --2- 172.21.15.19:0/2083398661 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f3bc00b2900 0x7f3bc00b4e10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc011550 tx=0x7f3bcc009260 comp rx=0 tx=0).stop 2024-05-07T02:25:57.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:57.249+0000 7f3be1a00700 1 -- 172.21.15.19:0/2083398661 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bdc105be0 msgr2=0x7f3bdc1034c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:25:57.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:57.249+0000 7f3be1a00700 1 --2- 172.21.15.19:0/2083398661 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bdc105be0 0x7f3bdc1034c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4000c00 tx=0x7f3bc40047b0 comp rx=0 tx=0).stop 2024-05-07T02:25:57.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:57.250+0000 7f3be1a00700 1 -- 172.21.15.19:0/2083398661 shutdown_connections 2024-05-07T02:25:57.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:57.250+0000 7f3be1a00700 1 --2- 172.21.15.19:0/2083398661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bdc10e700 0x7f3bdc103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:57.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:57.250+0000 7f3be1a00700 1 --2- 172.21.15.19:0/2083398661 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3bdc105290 0x7f3bdc102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:57.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:57.250+0000 7f3be1a00700 1 --2- 172.21.15.19:0/2083398661 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f3bc00b2900 0x7f3bc00b4e10 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:57.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:57.250+0000 7f3be1a00700 1 --2- 172.21.15.19:0/2083398661 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bdc105be0 0x7f3bdc1034c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:25:57.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:57.250+0000 7f3be1a00700 1 -- 172.21.15.19:0/2083398661 >> 172.21.15.19:0/2083398661 conn(0x7f3bdc078ac0 msgr2=0x7f3bdc100fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:25:57.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:57.250+0000 7f3be1a00700 1 -- 172.21.15.19:0/2083398661 shutdown_connections 2024-05-07T02:25:57.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:25:57.250+0000 7f3be1a00700 1 -- 172.21.15.19:0/2083398661 wait complete. 2024-05-07T02:25:57.251 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 13 2024-05-07T02:25:57.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:57 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/2083398661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:25:58.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:57 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/2083398661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:25:58.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:57 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/2083398661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:25:58.430 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-05-07T02:25:58.885 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:25:59.032 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:25:58 smithi019 ceph-mon[29781]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:59.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:25:58 smithi064 ceph-mon[34699]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:25:59.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:25:58 smithi027 ceph-mon[34398]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:01.105 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:00 smithi019 ceph-mon[29781]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:01.105 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:00 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:26:01.106 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:00 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:26:01.106 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:00 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:26:01.106 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:00 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:26:01.106 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:00 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:26:01.106 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:00 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:26:01.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:00 smithi064 ceph-mon[34699]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:01.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:00 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:26:01.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:00 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:26:01.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:00 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:26:01.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:00 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:26:01.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:00 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:26:01.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:00 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:26:01.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:00 smithi027 ceph-mon[34398]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:01.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:00 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:26:01.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:00 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:26:01.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:00 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:26:01.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:00 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:26:01.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:00 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:26:01.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:00 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:26:01.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.352+0000 7fad3ee00700 1 -- 172.21.15.19:0/2410871686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad380764b0 msgr2=0x7fad38114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:01.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.352+0000 7fad3ee00700 1 --2- 172.21.15.19:0/2410871686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad380764b0 0x7fad38114e20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fad24009990 tx=0x7fad24009ca0 comp rx=0 tx=0).stop 2024-05-07T02:26:01.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.353+0000 7fad3ee00700 1 -- 172.21.15.19:0/2410871686 shutdown_connections 2024-05-07T02:26:01.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.353+0000 7fad3ee00700 1 --2- 172.21.15.19:0/2410871686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad38115360 0x7fad38117780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:01.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.353+0000 7fad3ee00700 1 --2- 172.21.15.19:0/2410871686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad38075b60 0x7fad38075f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:01.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.353+0000 7fad3ee00700 1 --2- 172.21.15.19:0/2410871686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad380764b0 0x7fad38114e20 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:01.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.353+0000 7fad3ee00700 1 -- 172.21.15.19:0/2410871686 >> 172.21.15.19:0/2410871686 conn(0x7fad380ffe70 msgr2=0x7fad381022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:26:01.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.353+0000 7fad3ee00700 1 -- 172.21.15.19:0/2410871686 shutdown_connections 2024-05-07T02:26:01.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.353+0000 7fad3ee00700 1 -- 172.21.15.19:0/2410871686 wait complete. 2024-05-07T02:26:01.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.354+0000 7fad3ee00700 1 Processor -- start 2024-05-07T02:26:01.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.354+0000 7fad3ee00700 1 -- start start 2024-05-07T02:26:01.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.355+0000 7fad3ee00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad38075b60 0x7fad381a1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:01.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.355+0000 7fad3ee00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad380764b0 0x7fad381a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:01.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.355+0000 7fad3ee00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad38115360 0x7fad381a5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:01.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.355+0000 7fad3ee00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fad3811aab0 con 0x7fad38115360 2024-05-07T02:26:01.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.355+0000 7fad3ee00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fad3811a930 con 0x7fad38075b60 2024-05-07T02:26:01.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.355+0000 7fad3ee00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fad3811ac30 con 0x7fad380764b0 2024-05-07T02:26:01.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.355+0000 7fad37400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad380764b0 0x7fad381a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:01.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.355+0000 7fad37400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad380764b0 0x7fad381a19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:44792/0 (socket says 172.21.15.19:44792) 2024-05-07T02:26:01.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.355+0000 7fad37400700 1 -- 172.21.15.19:0/1678635837 learned_addr learned my addr 172.21.15.19:0/1678635837 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:26:01.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.355+0000 7fad37e00700 1 --2- 172.21.15.19:0/1678635837 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad38075b60 0x7fad381a1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:01.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.356+0000 7fad37400700 1 -- 172.21.15.19:0/1678635837 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad38075b60 msgr2=0x7fad381a1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:01.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.356+0000 7fad37400700 1 --2- 172.21.15.19:0/1678635837 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad38075b60 0x7fad381a1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:01.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.356+0000 7fad37400700 1 -- 172.21.15.19:0/1678635837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad38115360 msgr2=0x7fad381a5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:01.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.356+0000 7fad37400700 1 --2- 172.21.15.19:0/1678635837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad38115360 0x7fad381a5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:01.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.356+0000 7fad37400700 1 -- 172.21.15.19:0/1678635837 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad381a62f0 con 0x7fad380764b0 2024-05-07T02:26:01.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.356+0000 7fad37400700 1 --2- 172.21.15.19:0/1678635837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad380764b0 0x7fad381a19c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fad24000c00 tx=0x7fad240057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:26:01.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.356+0000 7fad34c00700 1 -- 172.21.15.19:0/1678635837 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad24027070 con 0x7fad380764b0 2024-05-07T02:26:01.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.356+0000 7fad34c00700 1 -- 172.21.15.19:0/1678635837 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad24022470 con 0x7fad380764b0 2024-05-07T02:26:01.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.357+0000 7fad3ee00700 1 -- 172.21.15.19:0/1678635837 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad381a6580 con 0x7fad380764b0 2024-05-07T02:26:01.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.357+0000 7fad34c00700 1 -- 172.21.15.19:0/1678635837 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad24017650 con 0x7fad380764b0 2024-05-07T02:26:01.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.357+0000 7fad3ee00700 1 -- 172.21.15.19:0/1678635837 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad381adf00 con 0x7fad380764b0 2024-05-07T02:26:01.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.359+0000 7fad3ee00700 1 -- 172.21.15.19:0/1678635837 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad18005330 con 0x7fad380764b0 2024-05-07T02:26:01.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.361+0000 7fad34c00700 1 -- 172.21.15.19:0/1678635837 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fad2402a070 con 0x7fad380764b0 2024-05-07T02:26:01.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.362+0000 7fad34c00700 1 --2- 172.21.15.19:0/1678635837 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fad1c0b2900 0x7fad1c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:01.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.362+0000 7fad34c00700 1 -- 172.21.15.19:0/1678635837 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fad240e2a10 con 0x7fad380764b0 2024-05-07T02:26:01.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.362+0000 7fad37e00700 1 --2- 172.21.15.19:0/1678635837 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fad1c0b2900 0x7fad1c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:01.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.363+0000 7fad37e00700 1 --2- 172.21.15.19:0/1678635837 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fad1c0b2900 0x7fad1c0b4e10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fad28005950 tx=0x7fad2800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:26:01.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.366+0000 7fad34c00700 1 -- 172.21.15.19:0/1678635837 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fad24013070 con 0x7fad380764b0 2024-05-07T02:26:01.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.615+0000 7fad3ee00700 1 -- 172.21.15.19:0/1678635837 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fad18005620 con 0x7fad380764b0 2024-05-07T02:26:01.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.616+0000 7fad34c00700 1 -- 172.21.15.19:0/1678635837 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v71) ==== 84+0+65 (secure 0 0 0) 0x7fad240aabc0 con 0x7fad380764b0 2024-05-07T02:26:01.616 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:26:01.616 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-05-07T02:26:01.616 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:26:01.617 INFO:teuthology.orchestra.run.smithi019.stdout: "poolnum": 1, 2024-05-07T02:26:01.617 INFO:teuthology.orchestra.run.smithi019.stdout: "poolname": ".mgr" 2024-05-07T02:26:01.617 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:26:01.617 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-05-07T02:26:01.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.619+0000 7fad3ee00700 1 -- 172.21.15.19:0/1678635837 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fad1c0b2900 msgr2=0x7fad1c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:01.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.619+0000 7fad3ee00700 1 --2- 172.21.15.19:0/1678635837 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fad1c0b2900 0x7fad1c0b4e10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fad28005950 tx=0x7fad2800b410 comp rx=0 tx=0).stop 2024-05-07T02:26:01.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.619+0000 7fad3ee00700 1 -- 172.21.15.19:0/1678635837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad380764b0 msgr2=0x7fad381a19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:01.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.619+0000 7fad3ee00700 1 --2- 172.21.15.19:0/1678635837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad380764b0 0x7fad381a19c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fad24000c00 tx=0x7fad240057c0 comp rx=0 tx=0).stop 2024-05-07T02:26:01.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.619+0000 7fad3ee00700 1 -- 172.21.15.19:0/1678635837 shutdown_connections 2024-05-07T02:26:01.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.619+0000 7fad3ee00700 1 --2- 172.21.15.19:0/1678635837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad38115360 0x7fad381a5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:01.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.619+0000 7fad3ee00700 1 --2- 172.21.15.19:0/1678635837 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fad1c0b2900 0x7fad1c0b4e10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:01.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.619+0000 7fad3ee00700 1 --2- 172.21.15.19:0/1678635837 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fad38075b60 0x7fad381a1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:01.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.619+0000 7fad3ee00700 1 --2- 172.21.15.19:0/1678635837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad380764b0 0x7fad381a19c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:01.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.619+0000 7fad3ee00700 1 -- 172.21.15.19:0/1678635837 >> 172.21.15.19:0/1678635837 conn(0x7fad380ffe70 msgr2=0x7fad38101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:26:01.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.619+0000 7fad3ee00700 1 -- 172.21.15.19:0/1678635837 shutdown_connections 2024-05-07T02:26:01.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:01.620+0000 7fad3ee00700 1 -- 172.21.15.19:0/1678635837 wait complete. 2024-05-07T02:26:01.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:01 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/1678635837' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-07T02:26:02.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:01 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/1678635837' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-07T02:26:02.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:01 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/1678635837' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-07T02:26:03.031 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-07T02:26:03.031 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-07T02:26:03.092 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:02 smithi019 ceph-mon[29781]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:03.092 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:02 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:26:03.092 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:02 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:26:03.092 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:02 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:26:03.092 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:02 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:26:03.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:02 smithi064 ceph-mon[34699]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:03.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:02 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:26:03.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:02 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:26:03.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:02 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:26:03.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:02 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:26:03.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:02 smithi027 ceph-mon[34398]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:03.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:02 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:26:03.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:02 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:26:03.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:02 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:26:03.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:02 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:26:03.468 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:26:03.877 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:03 smithi019 ceph-mon[29781]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:04.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:03 smithi064 ceph-mon[34699]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:04.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:03 smithi027 ceph-mon[34398]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:05.818 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:05 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:26:05.819 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:05 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:26:05.819 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:05 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:26:05.819 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:05 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:26:05.819 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:05 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:26:05.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:05.902+0000 7f3cd7200700 1 -- 172.21.15.19:0/521490461 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cd0076e40 msgr2=0x7f3cd0077250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:05.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:05.902+0000 7f3cd7200700 1 --2- 172.21.15.19:0/521490461 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cd0076e40 0x7f3cd0077250 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0009990 tx=0x7f3cc0009ca0 comp rx=0 tx=0).stop 2024-05-07T02:26:05.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:05.903+0000 7f3cd7200700 1 -- 172.21.15.19:0/521490461 shutdown_connections 2024-05-07T02:26:05.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:05.903+0000 7f3cd7200700 1 --2- 172.21.15.19:0/521490461 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cd0110b90 0x7f3cd0112ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:05.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:05.903+0000 7f3cd7200700 1 --2- 172.21.15.19:0/521490461 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cd0076e40 0x7f3cd0077250 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:05.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:05.903+0000 7f3cd7200700 1 --2- 172.21.15.19:0/521490461 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3cd0077790 0x7f3cd0110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:05.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:05.903+0000 7f3cd7200700 1 -- 172.21.15.19:0/521490461 >> 172.21.15.19:0/521490461 conn(0x7f3cd00ffef0 msgr2=0x7f3cd0102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:26:05.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:05.903+0000 7f3cd7200700 1 -- 172.21.15.19:0/521490461 shutdown_connections 2024-05-07T02:26:05.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:05.903+0000 7f3cd7200700 1 -- 172.21.15.19:0/521490461 wait complete. 2024-05-07T02:26:05.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:05.904+0000 7f3cd7200700 1 Processor -- start 2024-05-07T02:26:05.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:05.904+0000 7f3cd7200700 1 -- start start 2024-05-07T02:26:05.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:05.904+0000 7f3cd7200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cd0076e40 0x7f3cd019d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:05.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:05.904+0000 7f3cd7200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cd0077790 0x7f3cd019d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:05.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:05.904+0000 7f3cd7200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3cd0110b90 0x7f3cd01a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:05.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:05.904+0000 7f3cd7200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd01166d0 con 0x7f3cd0110b90 2024-05-07T02:26:05.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:05.904+0000 7f3cd7200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd0116550 con 0x7f3cd0077790 2024-05-07T02:26:05.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:05.904+0000 7f3cd7200700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd0116850 con 0x7f3cd0076e40 2024-05-07T02:26:05.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:05.905+0000 7f3cc7400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cd0077790 0x7f3cd019d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:05.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:05.905+0000 7f3cd5200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3cd0110b90 0x7f3cd01a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:05.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:05.905+0000 7f3cc7400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cd0077790 0x7f3cd019d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:45530/0 (socket says 172.21.15.19:45530) 2024-05-07T02:26:05.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:05.905+0000 7f3cc7400700 1 -- 172.21.15.19:0/2110272527 learned_addr learned my addr 172.21.15.19:0/2110272527 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:26:05.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:05.905+0000 7f3ccfe00700 1 --2- 172.21.15.19:0/2110272527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cd0076e40 0x7f3cd019d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:05.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:05.905+0000 7f3cc7400700 1 -- 172.21.15.19:0/2110272527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cd0076e40 msgr2=0x7f3cd019d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:05.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:05.905+0000 7f3cc7400700 1 --2- 172.21.15.19:0/2110272527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cd0076e40 0x7f3cd019d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:05.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:05.905+0000 7f3cc7400700 1 -- 172.21.15.19:0/2110272527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3cd0110b90 msgr2=0x7f3cd01a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:05.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:05.905+0000 7f3cc7400700 1 --2- 172.21.15.19:0/2110272527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3cd0110b90 0x7f3cd01a1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:05.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:05.905+0000 7f3cc7400700 1 -- 172.21.15.19:0/2110272527 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cd01a1f30 con 0x7f3cd0077790 2024-05-07T02:26:05.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:05.905+0000 7f3ccfe00700 1 --2- 172.21.15.19:0/2110272527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cd0076e40 0x7f3cd019d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:26:05.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:05.905+0000 7f3cc7400700 1 --2- 172.21.15.19:0/2110272527 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cd0077790 0x7f3cd019d600 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3cb800e9a0 tx=0x7f3cb800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:26:05.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:05.906+0000 7f3ccd600700 1 -- 172.21.15.19:0/2110272527 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb800cbd0 con 0x7f3cd0077790 2024-05-07T02:26:05.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:05.906+0000 7f3ccd600700 1 -- 172.21.15.19:0/2110272527 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cb800cd70 con 0x7f3cd0077790 2024-05-07T02:26:05.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:05.906+0000 7f3cd7200700 1 -- 172.21.15.19:0/2110272527 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cd01a2220 con 0x7f3cd0077790 2024-05-07T02:26:05.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:05.906+0000 7f3ccd600700 1 -- 172.21.15.19:0/2110272527 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8018680 con 0x7f3cd0077790 2024-05-07T02:26:05.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:05.906+0000 7f3cd7200700 1 -- 172.21.15.19:0/2110272527 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cd01a9b40 con 0x7f3cd0077790 2024-05-07T02:26:05.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:05.907+0000 7f3cd7200700 1 -- 172.21.15.19:0/2110272527 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cd00512c0 con 0x7f3cd0077790 2024-05-07T02:26:05.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:05.909+0000 7f3ccd600700 1 -- 172.21.15.19:0/2110272527 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f3cb8010430 con 0x7f3cd0077790 2024-05-07T02:26:05.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:05.911+0000 7f3ccd600700 1 --2- 172.21.15.19:0/2110272527 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f3cb40b2880 0x7f3cb40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:05.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:05.911+0000 7f3ccd600700 1 -- 172.21.15.19:0/2110272527 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f3cb80e2090 con 0x7f3cd0077790 2024-05-07T02:26:05.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:05.911+0000 7f3ccfe00700 1 --2- 172.21.15.19:0/2110272527 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f3cb40b2880 0x7f3cb40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:05.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:05.912+0000 7f3ccfe00700 1 --2- 172.21.15.19:0/2110272527 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f3cb40b2880 0x7f3cb40b4d90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3cc000b2e0 tx=0x7f3cc0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:26:05.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:05.916+0000 7f3ccd600700 1 -- 172.21.15.19:0/2110272527 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3cb80aa1b0 con 0x7f3cd0077790 2024-05-07T02:26:05.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:05 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:26:05.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:05 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:26:05.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:05 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:26:05.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:05 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:26:05.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:05 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:26:06.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:05 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:26:06.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:05 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:26:06.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:05 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:26:06.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:05 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:26:06.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:05 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:26:06.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:06.156+0000 7f3cd7200700 1 -- 172.21.15.19:0/2110272527 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f3cd002e5f0 con 0x7f3cd0077790 2024-05-07T02:26:06.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:06.764+0000 7f3ccd600700 1 -- 172.21.15.19:0/2110272527 <== mon.2 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v72) ==== 138+0+0 (secure 0 0 0) 0x7f3cb80a9ab0 con 0x7f3cd0077790 2024-05-07T02:26:06.767 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:06 smithi019 ceph-mon[29781]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:06.767 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:06 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/2110272527' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-07T02:26:06.767 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:06 smithi019 ceph-mon[29781]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-07T02:26:06.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:06.772+0000 7f3cd7200700 1 -- 172.21.15.19:0/2110272527 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f3cb40b2880 msgr2=0x7f3cb40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:06.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:06.772+0000 7f3cd7200700 1 --2- 172.21.15.19:0/2110272527 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f3cb40b2880 0x7f3cb40b4d90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3cc000b2e0 tx=0x7f3cc0005fd0 comp rx=0 tx=0).stop 2024-05-07T02:26:06.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:06.772+0000 7f3cd7200700 1 -- 172.21.15.19:0/2110272527 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cd0077790 msgr2=0x7f3cd019d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:06.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:06.772+0000 7f3cd7200700 1 --2- 172.21.15.19:0/2110272527 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cd0077790 0x7f3cd019d600 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3cb800e9a0 tx=0x7f3cb800ecb0 comp rx=0 tx=0).stop 2024-05-07T02:26:06.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:06.772+0000 7f3cd7200700 1 -- 172.21.15.19:0/2110272527 shutdown_connections 2024-05-07T02:26:06.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:06.772+0000 7f3cd7200700 1 --2- 172.21.15.19:0/2110272527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3cd0110b90 0x7f3cd01a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:06.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:06.772+0000 7f3cd7200700 1 --2- 172.21.15.19:0/2110272527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3cd0076e40 0x7f3cd019d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:06.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:06.772+0000 7f3cd7200700 1 --2- 172.21.15.19:0/2110272527 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f3cb40b2880 0x7f3cb40b4d90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:06.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:06.773+0000 7f3cd7200700 1 --2- 172.21.15.19:0/2110272527 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cd0077790 0x7f3cd019d600 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:06.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:06.773+0000 7f3cd7200700 1 -- 172.21.15.19:0/2110272527 >> 172.21.15.19:0/2110272527 conn(0x7f3cd00ffef0 msgr2=0x7f3cd01004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:26:06.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:06.773+0000 7f3cd7200700 1 -- 172.21.15.19:0/2110272527 shutdown_connections 2024-05-07T02:26:06.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:06.773+0000 7f3cd7200700 1 -- 172.21.15.19:0/2110272527 wait complete. 2024-05-07T02:26:06.777 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' created 2024-05-07T02:26:07.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:06 smithi064 ceph-mon[34699]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:07.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:06 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/2110272527' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-07T02:26:07.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:06 smithi064 ceph-mon[34699]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-07T02:26:07.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:06 smithi027 ceph-mon[34398]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:07.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:06 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/2110272527' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-07T02:26:07.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:06 smithi027 ceph-mon[34398]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-07T02:26:07.623 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-05-07T02:26:07.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:07 smithi019 ceph-mon[29781]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-07T02:26:07.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:07 smithi019 ceph-mon[29781]: osdmap e72: 12 total, 12 up, 12 in 2024-05-07T02:26:08.101 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:26:08.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:07 smithi064 ceph-mon[34699]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-07T02:26:08.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:07 smithi064 ceph-mon[34699]: osdmap e72: 12 total, 12 up, 12 in 2024-05-07T02:26:08.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:07 smithi027 ceph-mon[34398]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-07T02:26:08.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:07 smithi027 ceph-mon[34398]: osdmap e72: 12 total, 12 up, 12 in 2024-05-07T02:26:09.099 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:08 smithi019 ceph-mon[29781]: pgmap v320: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:09.099 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:08 smithi019 ceph-mon[29781]: osdmap e73: 12 total, 12 up, 12 in 2024-05-07T02:26:09.099 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:08 smithi019 ceph-mon[29781]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-07T02:26:09.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:08 smithi064 ceph-mon[34699]: pgmap v320: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:09.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:08 smithi064 ceph-mon[34699]: osdmap e73: 12 total, 12 up, 12 in 2024-05-07T02:26:09.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:08 smithi064 ceph-mon[34699]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-07T02:26:09.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:08 smithi027 ceph-mon[34398]: pgmap v320: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:09.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:08 smithi027 ceph-mon[34398]: osdmap e73: 12 total, 12 up, 12 in 2024-05-07T02:26:09.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:08 smithi027 ceph-mon[34398]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-07T02:26:10.027 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:09 smithi019 ceph-mon[29781]: osdmap e74: 12 total, 12 up, 12 in 2024-05-07T02:26:10.027 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:09 smithi019 ceph-mon[29781]: pgmap v323: 33 pgs: 21 unknown, 12 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:10.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:09 smithi064 ceph-mon[34699]: osdmap e74: 12 total, 12 up, 12 in 2024-05-07T02:26:10.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:09 smithi064 ceph-mon[34699]: pgmap v323: 33 pgs: 21 unknown, 12 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:10.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:09 smithi027 ceph-mon[34398]: osdmap e74: 12 total, 12 up, 12 in 2024-05-07T02:26:10.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:09 smithi027 ceph-mon[34398]: pgmap v323: 33 pgs: 21 unknown, 12 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:10.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.435+0000 7f4127200700 1 -- 172.21.15.19:0/3002335154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f412810af50 msgr2=0x7f412810b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:10.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.435+0000 7f4127200700 1 --2- 172.21.15.19:0/3002335154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f412810af50 0x7f412810b360 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f41140099c0 tx=0x7f4114009cd0 comp rx=0 tx=0).stop 2024-05-07T02:26:10.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.435+0000 7f4127200700 1 -- 172.21.15.19:0/3002335154 shutdown_connections 2024-05-07T02:26:10.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.435+0000 7f4127200700 1 --2- 172.21.15.19:0/3002335154 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f412810b8a0 0x7f4128113200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:10.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.435+0000 7f4127200700 1 --2- 172.21.15.19:0/3002335154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4128106620 0x7f4128106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:10.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.435+0000 7f4127200700 1 --2- 172.21.15.19:0/3002335154 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f412810af50 0x7f412810b360 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:10.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.435+0000 7f4127200700 1 -- 172.21.15.19:0/3002335154 >> 172.21.15.19:0/3002335154 conn(0x7f4128102030 msgr2=0x7f4128104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:26:10.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.435+0000 7f4127200700 1 -- 172.21.15.19:0/3002335154 shutdown_connections 2024-05-07T02:26:10.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.435+0000 7f4127200700 1 -- 172.21.15.19:0/3002335154 wait complete. 2024-05-07T02:26:10.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.436+0000 7f4127200700 1 Processor -- start 2024-05-07T02:26:10.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.437+0000 7f4127200700 1 -- start start 2024-05-07T02:26:10.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.437+0000 7f4127200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4128106620 0x7f412819cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:10.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.437+0000 7f4127200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f412810af50 0x7f412819d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:10.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.437+0000 7f4127200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f412810b8a0 0x7f41281a15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:10.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.437+0000 7f4127200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f41281164b0 con 0x7f4128106620 2024-05-07T02:26:10.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.437+0000 7f4127200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4128116330 con 0x7f412810b8a0 2024-05-07T02:26:10.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.437+0000 7f4127200700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4128116630 con 0x7f412810af50 2024-05-07T02:26:10.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.437+0000 7f4125e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4128106620 0x7f412819cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:10.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.438+0000 7f4125e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4128106620 0x7f412819cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37316/0 (socket says 172.21.15.19:37316) 2024-05-07T02:26:10.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.438+0000 7f4125e00700 1 -- 172.21.15.19:0/1760059316 learned_addr learned my addr 172.21.15.19:0/1760059316 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:26:10.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.438+0000 7f4126800700 1 --2- 172.21.15.19:0/1760059316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f412810b8a0 0x7f41281a15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:10.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.438+0000 7f4125400700 1 --2- 172.21.15.19:0/1760059316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f412810af50 0x7f412819d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:10.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.438+0000 7f4125e00700 1 -- 172.21.15.19:0/1760059316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f412810af50 msgr2=0x7f412819d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:10.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.438+0000 7f4125e00700 1 --2- 172.21.15.19:0/1760059316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f412810af50 0x7f412819d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:10.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.438+0000 7f4125e00700 1 -- 172.21.15.19:0/1760059316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f412810b8a0 msgr2=0x7f41281a15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:10.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.438+0000 7f4125e00700 1 --2- 172.21.15.19:0/1760059316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f412810b8a0 0x7f41281a15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:10.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.438+0000 7f4125e00700 1 -- 172.21.15.19:0/1760059316 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41281a1c90 con 0x7f4128106620 2024-05-07T02:26:10.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.439+0000 7f4125e00700 1 --2- 172.21.15.19:0/1760059316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4128106620 0x7f412819cdc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f411800e9a0 tx=0x7f411800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:26:10.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.439+0000 7f411ea00700 1 -- 172.21.15.19:0/1760059316 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411800ca10 con 0x7f4128106620 2024-05-07T02:26:10.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.439+0000 7f411ea00700 1 -- 172.21.15.19:0/1760059316 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f411800cbb0 con 0x7f4128106620 2024-05-07T02:26:10.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.439+0000 7f4127200700 1 -- 172.21.15.19:0/1760059316 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41281a1f80 con 0x7f4128106620 2024-05-07T02:26:10.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.439+0000 7f411ea00700 1 -- 172.21.15.19:0/1760059316 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4118018680 con 0x7f4128106620 2024-05-07T02:26:10.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.439+0000 7f4127200700 1 -- 172.21.15.19:0/1760059316 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41281a98a0 con 0x7f4128106620 2024-05-07T02:26:10.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.441+0000 7f4127200700 1 -- 172.21.15.19:0/1760059316 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f412806a620 con 0x7f4128106620 2024-05-07T02:26:10.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.443+0000 7f411ea00700 1 -- 172.21.15.19:0/1760059316 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f4118010430 con 0x7f4128106620 2024-05-07T02:26:10.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.445+0000 7f411ea00700 1 --2- 172.21.15.19:0/1760059316 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f410c0b2880 0x7f410c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:10.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.445+0000 7f411ea00700 1 -- 172.21.15.19:0/1760059316 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7497+0+0 (secure 0 0 0) 0x7f41180e2370 con 0x7f4128106620 2024-05-07T02:26:10.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.445+0000 7f4125400700 1 --2- 172.21.15.19:0/1760059316 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f410c0b2880 0x7f410c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:10.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.446+0000 7f4125400700 1 --2- 172.21.15.19:0/1760059316 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f410c0b2880 0x7f410c0b4d90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4114000c00 tx=0x7f411401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:26:10.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.448+0000 7f411ea00700 1 -- 172.21.15.19:0/1760059316 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f41180aa3c0 con 0x7f4128106620 2024-05-07T02:26:10.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.688+0000 7f4127200700 1 -- 172.21.15.19:0/1760059316 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f412802e5f0 con 0x7f4128106620 2024-05-07T02:26:10.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.830+0000 7f411ea00700 1 -- 172.21.15.19:0/1760059316 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v75) ==== 171+0+0 (secure 0 0 0) 0x7f41180a9cc0 con 0x7f4128106620 2024-05-07T02:26:10.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.835+0000 7f4127200700 1 -- 172.21.15.19:0/1760059316 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f410c0b2880 msgr2=0x7f410c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:10.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.836+0000 7f4127200700 1 --2- 172.21.15.19:0/1760059316 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f410c0b2880 0x7f410c0b4d90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4114000c00 tx=0x7f411401a040 comp rx=0 tx=0).stop 2024-05-07T02:26:10.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.836+0000 7f4127200700 1 -- 172.21.15.19:0/1760059316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4128106620 msgr2=0x7f412819cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:10.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.836+0000 7f4127200700 1 --2- 172.21.15.19:0/1760059316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4128106620 0x7f412819cdc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f411800e9a0 tx=0x7f411800ecb0 comp rx=0 tx=0).stop 2024-05-07T02:26:10.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.836+0000 7f4127200700 1 -- 172.21.15.19:0/1760059316 shutdown_connections 2024-05-07T02:26:10.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.836+0000 7f4127200700 1 --2- 172.21.15.19:0/1760059316 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f410c0b2880 0x7f410c0b4d90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:10.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.836+0000 7f4127200700 1 --2- 172.21.15.19:0/1760059316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f412810b8a0 0x7f41281a15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:10.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.836+0000 7f4127200700 1 --2- 172.21.15.19:0/1760059316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4128106620 0x7f412819cdc0 secure :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f411800e9a0 tx=0x7f411800ecb0 comp rx=0 tx=0).stop 2024-05-07T02:26:10.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.836+0000 7f4127200700 1 --2- 172.21.15.19:0/1760059316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f412810af50 0x7f412819d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:10.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.836+0000 7f4127200700 1 -- 172.21.15.19:0/1760059316 >> 172.21.15.19:0/1760059316 conn(0x7f4128102030 msgr2=0x7f412810a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:26:10.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.837+0000 7f4127200700 1 -- 172.21.15.19:0/1760059316 shutdown_connections 2024-05-07T02:26:10.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:10.837+0000 7f4127200700 1 -- 172.21.15.19:0/1760059316 wait complete. 2024-05-07T02:26:10.843 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' created 2024-05-07T02:26:11.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:10 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/1760059316' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-07T02:26:11.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:10 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/1760059316' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-07T02:26:11.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:10 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/1760059316' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-07T02:26:11.713 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-07T02:26:11.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:11 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/1760059316' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-07T02:26:11.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:11 smithi019 ceph-mon[29781]: osdmap e75: 12 total, 12 up, 12 in 2024-05-07T02:26:11.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:11 smithi019 ceph-mon[29781]: pgmap v325: 97 pgs: 10 creating+peering, 26 active+clean, 61 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:12.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:11 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/1760059316' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-07T02:26:12.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:11 smithi064 ceph-mon[34699]: osdmap e75: 12 total, 12 up, 12 in 2024-05-07T02:26:12.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:11 smithi064 ceph-mon[34699]: pgmap v325: 97 pgs: 10 creating+peering, 26 active+clean, 61 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:12.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:11 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/1760059316' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-07T02:26:12.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:11 smithi027 ceph-mon[34398]: osdmap e75: 12 total, 12 up, 12 in 2024-05-07T02:26:12.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:11 smithi027 ceph-mon[34398]: pgmap v325: 97 pgs: 10 creating+peering, 26 active+clean, 61 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:12.208 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:26:12.859 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:12 smithi019 ceph-mon[29781]: osdmap e76: 12 total, 12 up, 12 in 2024-05-07T02:26:12.860 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:12 smithi019 ceph-mon[29781]: osdmap e77: 12 total, 12 up, 12 in 2024-05-07T02:26:13.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:12 smithi027 ceph-mon[34398]: osdmap e76: 12 total, 12 up, 12 in 2024-05-07T02:26:13.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:12 smithi027 ceph-mon[34398]: osdmap e77: 12 total, 12 up, 12 in 2024-05-07T02:26:13.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:12 smithi064 ceph-mon[34699]: osdmap e76: 12 total, 12 up, 12 in 2024-05-07T02:26:13.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:12 smithi064 ceph-mon[34699]: osdmap e77: 12 total, 12 up, 12 in 2024-05-07T02:26:14.024 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:13 smithi019 ceph-mon[29781]: pgmap v328: 97 pgs: 24 creating+peering, 33 active+clean, 40 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:14.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:13 smithi027 ceph-mon[34398]: pgmap v328: 97 pgs: 24 creating+peering, 33 active+clean, 40 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:14.260 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:13 smithi064 ceph-mon[34699]: pgmap v328: 97 pgs: 24 creating+peering, 33 active+clean, 40 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:14.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:14.653+0000 7efc5f400700 1 -- 172.21.15.19:0/4033364577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc58076bf0 msgr2=0x7efc58077000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:14.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:14.653+0000 7efc5f400700 1 --2- 172.21.15.19:0/4033364577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc58076bf0 0x7efc58077000 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efc4c009990 tx=0x7efc4c009ca0 comp rx=0 tx=0).stop 2024-05-07T02:26:14.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:14.654+0000 7efc5f400700 1 -- 172.21.15.19:0/4033364577 shutdown_connections 2024-05-07T02:26:14.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:14.654+0000 7efc5f400700 1 --2- 172.21.15.19:0/4033364577 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc58110c00 0x7efc58113020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:14.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:14.654+0000 7efc5f400700 1 --2- 172.21.15.19:0/4033364577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc58076bf0 0x7efc58077000 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:14.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:14.654+0000 7efc5f400700 1 --2- 172.21.15.19:0/4033364577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc58077540 0x7efc581106c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:14.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:14.654+0000 7efc5f400700 1 -- 172.21.15.19:0/4033364577 >> 172.21.15.19:0/4033364577 conn(0x7efc580ffea0 msgr2=0x7efc581022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:26:14.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:14.654+0000 7efc5f400700 1 -- 172.21.15.19:0/4033364577 shutdown_connections 2024-05-07T02:26:14.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:14.654+0000 7efc5f400700 1 -- 172.21.15.19:0/4033364577 wait complete. 2024-05-07T02:26:14.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:14.655+0000 7efc5f400700 1 Processor -- start 2024-05-07T02:26:14.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:14.655+0000 7efc5f400700 1 -- start start 2024-05-07T02:26:14.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:14.656+0000 7efc5f400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc58076bf0 0x7efc5819d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:14.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:14.656+0000 7efc5f400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc58077540 0x7efc5819d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:14.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:14.656+0000 7efc5f400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc58110c00 0x7efc581a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:14.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:14.656+0000 7efc5f400700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efc581166d0 con 0x7efc58110c00 2024-05-07T02:26:14.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:14.656+0000 7efc5f400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efc58116550 con 0x7efc58077540 2024-05-07T02:26:14.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:14.656+0000 7efc5f400700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7efc58116850 con 0x7efc58076bf0 2024-05-07T02:26:14.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:14.656+0000 7efc57e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc58077540 0x7efc5819d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:14.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:14.656+0000 7efc57e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc58077540 0x7efc5819d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:44100/0 (socket says 172.21.15.19:44100) 2024-05-07T02:26:14.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:14.656+0000 7efc57e00700 1 -- 172.21.15.19:0/3531394595 learned_addr learned my addr 172.21.15.19:0/3531394595 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:26:14.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:14.656+0000 7efc5ca00700 1 --2- 172.21.15.19:0/3531394595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc58076bf0 0x7efc5819d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:14.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:14.656+0000 7efc5d400700 1 --2- 172.21.15.19:0/3531394595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc58110c00 0x7efc581a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:14.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:14.657+0000 7efc5ca00700 1 -- 172.21.15.19:0/3531394595 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc58077540 msgr2=0x7efc5819d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:14.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:14.657+0000 7efc5ca00700 1 --2- 172.21.15.19:0/3531394595 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc58077540 0x7efc5819d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:14.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:14.657+0000 7efc5ca00700 1 -- 172.21.15.19:0/3531394595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc58110c00 msgr2=0x7efc581a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:14.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:14.657+0000 7efc5ca00700 1 --2- 172.21.15.19:0/3531394595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc58110c00 0x7efc581a1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:14.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:14.657+0000 7efc5ca00700 1 -- 172.21.15.19:0/3531394595 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc581a1f10 con 0x7efc58076bf0 2024-05-07T02:26:14.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:14.657+0000 7efc5d400700 1 --2- 172.21.15.19:0/3531394595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc58110c00 0x7efc581a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:26:14.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:14.657+0000 7efc5ca00700 1 --2- 172.21.15.19:0/3531394595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc58076bf0 0x7efc5819d040 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efc4c0098e0 tx=0x7efc4c0057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:26:14.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:14.657+0000 7efc55600700 1 -- 172.21.15.19:0/3531394595 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4c027070 con 0x7efc58076bf0 2024-05-07T02:26:14.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:14.657+0000 7efc55600700 1 -- 172.21.15.19:0/3531394595 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc4c022470 con 0x7efc58076bf0 2024-05-07T02:26:14.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:14.657+0000 7efc5f400700 1 -- 172.21.15.19:0/3531394595 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc581a21a0 con 0x7efc58076bf0 2024-05-07T02:26:14.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:14.658+0000 7efc55600700 1 -- 172.21.15.19:0/3531394595 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4c017650 con 0x7efc58076bf0 2024-05-07T02:26:14.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:14.658+0000 7efc5f400700 1 -- 172.21.15.19:0/3531394595 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc581a2680 con 0x7efc58076bf0 2024-05-07T02:26:14.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:14.659+0000 7efc5f400700 1 -- 172.21.15.19:0/3531394595 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc580512c0 con 0x7efc58076bf0 2024-05-07T02:26:14.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:14.662+0000 7efc55600700 1 -- 172.21.15.19:0/3531394595 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7efc4c02a070 con 0x7efc58076bf0 2024-05-07T02:26:14.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:14.663+0000 7efc55600700 1 --2- 172.21.15.19:0/3531394595 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7efc400b2880 0x7efc400b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:14.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:14.663+0000 7efc55600700 1 -- 172.21.15.19:0/3531394595 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7879+0+0 (secure 0 0 0) 0x7efc4c0e30e0 con 0x7efc58076bf0 2024-05-07T02:26:14.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:14.663+0000 7efc57e00700 1 --2- 172.21.15.19:0/3531394595 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7efc400b2880 0x7efc400b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:14.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:14.664+0000 7efc57e00700 1 --2- 172.21.15.19:0/3531394595 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7efc400b2880 0x7efc400b4d90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efc581033f0 tx=0x7efc44009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:26:14.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:14.665+0000 7efc55600700 1 -- 172.21.15.19:0/3531394595 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efc4c05fdf0 con 0x7efc58076bf0 2024-05-07T02:26:14.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:14.958+0000 7efc5f400700 1 -- 172.21.15.19:0/3531394595 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7efc5802e5f0 con 0x7efc58076bf0 2024-05-07T02:26:15.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:15.020+0000 7efc55600700 1 -- 172.21.15.19:0/3531394595 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7efc4c013070 con 0x7efc58076bf0 2024-05-07T02:26:15.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:15.025+0000 7efc5f400700 1 -- 172.21.15.19:0/3531394595 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7efc400b2880 msgr2=0x7efc400b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:15.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:15.025+0000 7efc5f400700 1 --2- 172.21.15.19:0/3531394595 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7efc400b2880 0x7efc400b4d90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efc581033f0 tx=0x7efc44009260 comp rx=0 tx=0).stop 2024-05-07T02:26:15.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:15.025+0000 7efc5f400700 1 -- 172.21.15.19:0/3531394595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc58076bf0 msgr2=0x7efc5819d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:15.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:15.025+0000 7efc5f400700 1 --2- 172.21.15.19:0/3531394595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc58076bf0 0x7efc5819d040 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efc4c0098e0 tx=0x7efc4c0057c0 comp rx=0 tx=0).stop 2024-05-07T02:26:15.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:15.025+0000 7efc5f400700 1 -- 172.21.15.19:0/3531394595 shutdown_connections 2024-05-07T02:26:15.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:15.025+0000 7efc5f400700 1 --2- 172.21.15.19:0/3531394595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc58110c00 0x7efc581a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:15.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:15.025+0000 7efc5f400700 1 --2- 172.21.15.19:0/3531394595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc58076bf0 0x7efc5819d040 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:15.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:15.025+0000 7efc5f400700 1 --2- 172.21.15.19:0/3531394595 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7efc400b2880 0x7efc400b4d90 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:15.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:15.025+0000 7efc5f400700 1 --2- 172.21.15.19:0/3531394595 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc58077540 0x7efc5819d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:15.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:15.025+0000 7efc5f400700 1 -- 172.21.15.19:0/3531394595 >> 172.21.15.19:0/3531394595 conn(0x7efc580ffea0 msgr2=0x7efc58101cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:26:15.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:15.027+0000 7efc5f400700 1 -- 172.21.15.19:0/3531394595 shutdown_connections 2024-05-07T02:26:15.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:15.027+0000 7efc5f400700 1 -- 172.21.15.19:0/3531394595 wait complete. 2024-05-07T02:26:15.031 INFO:teuthology.orchestra.run.smithi019.stderr:new fs with metadata pool 2 and data pool 3 2024-05-07T02:26:15.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:15 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/3531394595' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-07T02:26:15.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:15 smithi064 ceph-mon[34699]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-07T02:26:15.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:15 smithi064 ceph-mon[34699]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-07T02:26:15.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:15 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/3531394595' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-07T02:26:15.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:15 smithi019 ceph-mon[29781]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-07T02:26:15.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:15 smithi019 ceph-mon[29781]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-07T02:26:15.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:15 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/3531394595' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-07T02:26:15.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:15 smithi027 ceph-mon[34398]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-07T02:26:15.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:15 smithi027 ceph-mon[34398]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-07T02:26:15.916 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-05-07T02:26:15.916 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-05-07T02:26:16.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:16 smithi019 ceph-mon[29781]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-07T02:26:16.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:16 smithi019 ceph-mon[29781]: osdmap e78: 12 total, 12 up, 12 in 2024-05-07T02:26:16.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:16 smithi019 ceph-mon[29781]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-07T02:26:16.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:16 smithi019 ceph-mon[29781]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-07T02:26:16.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:16 smithi019 ceph-mon[29781]: pgmap v330: 97 pgs: 24 creating+peering, 57 active+clean, 16 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:16.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:16 smithi064 ceph-mon[34699]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-07T02:26:16.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:16 smithi064 ceph-mon[34699]: osdmap e78: 12 total, 12 up, 12 in 2024-05-07T02:26:16.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:16 smithi064 ceph-mon[34699]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-07T02:26:16.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:16 smithi064 ceph-mon[34699]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-07T02:26:16.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:16 smithi064 ceph-mon[34699]: pgmap v330: 97 pgs: 24 creating+peering, 57 active+clean, 16 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:16.406 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:26:16.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:16 smithi027 ceph-mon[34398]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-07T02:26:16.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:16 smithi027 ceph-mon[34398]: osdmap e78: 12 total, 12 up, 12 in 2024-05-07T02:26:16.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:16 smithi027 ceph-mon[34398]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-07T02:26:16.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:16 smithi027 ceph-mon[34398]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-07T02:26:16.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:16 smithi027 ceph-mon[34398]: pgmap v330: 97 pgs: 24 creating+peering, 57 active+clean, 16 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:17.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:17 smithi019 ceph-mon[29781]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-07T02:26:17.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:17 smithi019 ceph-mon[29781]: Cluster is now healthy 2024-05-07T02:26:17.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:17 smithi019 ceph-mon[29781]: mds.? [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] up:active 2024-05-07T02:26:17.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:17 smithi019 ceph-mon[29781]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-07T02:26:17.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:17 smithi019 ceph-mon[29781]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-07T02:26:17.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:17 smithi064 ceph-mon[34699]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-07T02:26:17.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:17 smithi064 ceph-mon[34699]: Cluster is now healthy 2024-05-07T02:26:17.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:17 smithi064 ceph-mon[34699]: mds.? [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] up:active 2024-05-07T02:26:17.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:17 smithi064 ceph-mon[34699]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-07T02:26:17.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:17 smithi064 ceph-mon[34699]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-07T02:26:17.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:17 smithi027 ceph-mon[34398]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-07T02:26:17.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:17 smithi027 ceph-mon[34398]: Cluster is now healthy 2024-05-07T02:26:17.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:17 smithi027 ceph-mon[34398]: mds.? [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] up:active 2024-05-07T02:26:17.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:17 smithi027 ceph-mon[34398]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-07T02:26:17.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:17 smithi027 ceph-mon[34398]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-07T02:26:18.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:18 smithi019 ceph-mon[29781]: pgmap v331: 97 pgs: 97 active+clean; 578 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-05-07T02:26:18.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:18 smithi064 ceph-mon[34699]: pgmap v331: 97 pgs: 97 active+clean; 578 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-05-07T02:26:18.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:18 smithi027 ceph-mon[34398]: pgmap v331: 97 pgs: 97 active+clean; 578 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-05-07T02:26:18.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:18.903+0000 7f77b0000700 1 -- 172.21.15.19:0/1737353380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77a8103bc0 msgr2=0x7f77a81080f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:18.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:18.903+0000 7f77b0000700 1 --2- 172.21.15.19:0/1737353380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77a8103bc0 0x7f77a81080f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7794009990 tx=0x7f7794009ca0 comp rx=0 tx=0).stop 2024-05-07T02:26:18.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:18.903+0000 7f77b0000700 1 -- 172.21.15.19:0/1737353380 shutdown_connections 2024-05-07T02:26:18.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:18.903+0000 7f77b0000700 1 --2- 172.21.15.19:0/1737353380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77a81086c0 0x7f77a8108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:18.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:18.903+0000 7f77b0000700 1 --2- 172.21.15.19:0/1737353380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77a8103270 0x7f77a8103680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:18.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:18.903+0000 7f77b0000700 1 --2- 172.21.15.19:0/1737353380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77a8103bc0 0x7f77a81080f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:18.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:18.903+0000 7f77b0000700 1 -- 172.21.15.19:0/1737353380 >> 172.21.15.19:0/1737353380 conn(0x7f77a80fef30 msgr2=0x7f77a8101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:26:18.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:18.903+0000 7f77b0000700 1 -- 172.21.15.19:0/1737353380 shutdown_connections 2024-05-07T02:26:18.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:18.903+0000 7f77b0000700 1 -- 172.21.15.19:0/1737353380 wait complete. 2024-05-07T02:26:18.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:18.904+0000 7f77b0000700 1 Processor -- start 2024-05-07T02:26:18.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:18.905+0000 7f77b0000700 1 -- start start 2024-05-07T02:26:18.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:18.905+0000 7f77b0000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77a8103270 0x7f77a81a1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:18.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:18.905+0000 7f77b0000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77a8103bc0 0x7f77a81a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:18.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:18.905+0000 7f77b0000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77a81086c0 0x7f77a81a5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:18.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:18.905+0000 7f77b0000700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f77a811aa90 con 0x7f77a8103bc0 2024-05-07T02:26:18.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:18.905+0000 7f77b0000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f77a811a910 con 0x7f77a8103270 2024-05-07T02:26:18.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:18.905+0000 7f77b0000700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f77a811ac10 con 0x7f77a81086c0 2024-05-07T02:26:18.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:18.905+0000 7f77ad600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77a8103270 0x7f77a81a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:18.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:18.906+0000 7f77ad600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77a8103270 0x7f77a81a1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:60198/0 (socket says 172.21.15.19:60198) 2024-05-07T02:26:18.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:18.906+0000 7f77ad600700 1 -- 172.21.15.19:0/1300860076 learned_addr learned my addr 172.21.15.19:0/1300860076 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:26:18.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:18.906+0000 7f77acc00700 1 --2- 172.21.15.19:0/1300860076 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77a8103bc0 0x7f77a81a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:18.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:18.906+0000 7f77ae000700 1 --2- 172.21.15.19:0/1300860076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77a81086c0 0x7f77a81a5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:18.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:18.906+0000 7f77ad600700 1 -- 172.21.15.19:0/1300860076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77a81086c0 msgr2=0x7f77a81a5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:18.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:18.906+0000 7f77ad600700 1 --2- 172.21.15.19:0/1300860076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77a81086c0 0x7f77a81a5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:18.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:18.906+0000 7f77ad600700 1 -- 172.21.15.19:0/1300860076 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77a8103bc0 msgr2=0x7f77a81a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:18.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:18.906+0000 7f77ad600700 1 --2- 172.21.15.19:0/1300860076 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77a8103bc0 0x7f77a81a1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:18.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:18.906+0000 7f77ad600700 1 -- 172.21.15.19:0/1300860076 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a81a6350 con 0x7f77a8103270 2024-05-07T02:26:18.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:18.906+0000 7f77ae000700 1 --2- 172.21.15.19:0/1300860076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77a81086c0 0x7f77a81a5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:26:18.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:18.906+0000 7f77ad600700 1 --2- 172.21.15.19:0/1300860076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77a8103270 0x7f77a81a1480 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f779c00ef10 tx=0x7f779c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:26:18.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:18.907+0000 7f77a6000700 1 -- 172.21.15.19:0/1300860076 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c00ca70 con 0x7f77a8103270 2024-05-07T02:26:18.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:18.907+0000 7f77b0000700 1 -- 172.21.15.19:0/1300860076 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77a81a65e0 con 0x7f77a8103270 2024-05-07T02:26:18.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:18.907+0000 7f77a6000700 1 -- 172.21.15.19:0/1300860076 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f779c019070 con 0x7f77a8103270 2024-05-07T02:26:18.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:18.907+0000 7f77a6000700 1 -- 172.21.15.19:0/1300860076 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c014480 con 0x7f77a8103270 2024-05-07T02:26:18.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:18.907+0000 7f77b0000700 1 -- 172.21.15.19:0/1300860076 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77a81adf60 con 0x7f77a8103270 2024-05-07T02:26:18.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:18.908+0000 7f77b0000700 1 -- 172.21.15.19:0/1300860076 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77a806c920 con 0x7f77a8103270 2024-05-07T02:26:18.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:18.911+0000 7f77a6000700 1 -- 172.21.15.19:0/1300860076 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f779c012050 con 0x7f77a8103270 2024-05-07T02:26:18.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:18.912+0000 7f77a6000700 1 --2- 172.21.15.19:0/1300860076 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f77900b2880 0x7f77900b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:18.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:18.912+0000 7f77a6000700 1 -- 172.21.15.19:0/1300860076 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f779c0e2800 con 0x7f77a8103270 2024-05-07T02:26:18.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:18.913+0000 7f77acc00700 1 --2- 172.21.15.19:0/1300860076 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f77900b2880 0x7f77900b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:18.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:18.913+0000 7f77acc00700 1 --2- 172.21.15.19:0/1300860076 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f77900b2880 0x7f77900b4d90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f77a8103a10 tx=0x7f779400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:26:18.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:18.914+0000 7f77a6000700 1 -- 172.21.15.19:0/1300860076 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f779c010040 con 0x7f77a8103270 2024-05-07T02:26:19.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:19.168+0000 7f77b0000700 1 -- 172.21.15.19:0/1300860076 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f77a80512c0 con 0x7f77a8103270 2024-05-07T02:26:19.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:19.228+0000 7f77a6000700 1 -- 172.21.15.19:0/1300860076 <== mon.2 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v79) ==== 157+0+0 (secure 0 0 0) 0x7f779c0aa5d0 con 0x7f77a8103270 2024-05-07T02:26:19.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:19.234+0000 7f77b0000700 1 -- 172.21.15.19:0/1300860076 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f77900b2880 msgr2=0x7f77900b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:19.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:19.234+0000 7f77b0000700 1 --2- 172.21.15.19:0/1300860076 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f77900b2880 0x7f77900b4d90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f77a8103a10 tx=0x7f779400b260 comp rx=0 tx=0).stop 2024-05-07T02:26:19.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:19.234+0000 7f77b0000700 1 -- 172.21.15.19:0/1300860076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77a8103270 msgr2=0x7f77a81a1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:19.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:19.234+0000 7f77b0000700 1 --2- 172.21.15.19:0/1300860076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77a8103270 0x7f77a81a1480 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f779c00ef10 tx=0x7f779c00c3a0 comp rx=0 tx=0).stop 2024-05-07T02:26:19.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:19.234+0000 7f77b0000700 1 -- 172.21.15.19:0/1300860076 shutdown_connections 2024-05-07T02:26:19.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:19.234+0000 7f77b0000700 1 --2- 172.21.15.19:0/1300860076 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77a81086c0 0x7f77a81a5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:19.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:19.234+0000 7f77b0000700 1 --2- 172.21.15.19:0/1300860076 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f77900b2880 0x7f77900b4d90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:19.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:19.234+0000 7f77b0000700 1 --2- 172.21.15.19:0/1300860076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77a8103270 0x7f77a81a1480 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:19.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:19.234+0000 7f77b0000700 1 --2- 172.21.15.19:0/1300860076 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77a8103bc0 0x7f77a81a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:19.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:19.234+0000 7f77b0000700 1 -- 172.21.15.19:0/1300860076 >> 172.21.15.19:0/1300860076 conn(0x7f77a80fef30 msgr2=0x7f77a81060c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:26:19.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:19.235+0000 7f77b0000700 1 -- 172.21.15.19:0/1300860076 shutdown_connections 2024-05-07T02:26:19.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:19.235+0000 7f77b0000700 1 -- 172.21.15.19:0/1300860076 wait complete. 2024-05-07T02:26:19.564 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:19 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/1300860076' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-07T02:26:19.564 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:19 smithi027 ceph-mon[34398]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-07T02:26:19.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:19 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/1300860076' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-07T02:26:19.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:19 smithi064 ceph-mon[34699]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-07T02:26:19.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:19 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/1300860076' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-07T02:26:19.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:19 smithi019 ceph-mon[29781]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-07T02:26:19.956 INFO:journalctl@ceph.mds.b.smithi027.stdout:May 07 02:26:19 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-b[65159]: 2024-05-07T02:26:19.564+0000 7f022f5f6700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-07T02:26:20.127 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-05-07T02:26:20.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:20 smithi019 ceph-mon[29781]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-07T02:26:20.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:20 smithi019 ceph-mon[29781]: osdmap e79: 12 total, 12 up, 12 in 2024-05-07T02:26:20.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:20 smithi019 ceph-mon[29781]: pgmap v333: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-05-07T02:26:20.598 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:26:20.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:20 smithi064 ceph-mon[34699]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-07T02:26:20.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:20 smithi064 ceph-mon[34699]: osdmap e79: 12 total, 12 up, 12 in 2024-05-07T02:26:20.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:20 smithi064 ceph-mon[34699]: pgmap v333: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-05-07T02:26:20.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:20 smithi027 ceph-mon[34398]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-07T02:26:20.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:20 smithi027 ceph-mon[34398]: osdmap e79: 12 total, 12 up, 12 in 2024-05-07T02:26:20.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:20 smithi027 ceph-mon[34398]: pgmap v333: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-05-07T02:26:22.719 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:22 smithi019 ceph-mon[29781]: pgmap v334: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-07T02:26:22.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:22 smithi064 ceph-mon[34699]: pgmap v334: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-07T02:26:22.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:22 smithi027 ceph-mon[34398]: pgmap v334: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-07T02:26:23.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:23.089+0000 7f9a76800700 1 -- 172.21.15.19:0/1378315324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a70109870 msgr2=0x7f9a701111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:23.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:23.089+0000 7f9a76800700 1 --2- 172.21.15.19:0/1378315324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a70109870 0x7f9a701111d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9a68009fc0 tx=0x7f9a6800b330 comp rx=0 tx=0).stop 2024-05-07T02:26:23.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:23.089+0000 7f9a76800700 1 -- 172.21.15.19:0/1378315324 shutdown_connections 2024-05-07T02:26:23.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:23.089+0000 7f9a76800700 1 --2- 172.21.15.19:0/1378315324 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a70109870 0x7f9a701111d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:23.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:23.089+0000 7f9a76800700 1 --2- 172.21.15.19:0/1378315324 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a701045f0 0x7f9a70104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:23.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:23.089+0000 7f9a76800700 1 --2- 172.21.15.19:0/1378315324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a70108f20 0x7f9a70109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:23.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:23.089+0000 7f9a76800700 1 -- 172.21.15.19:0/1378315324 >> 172.21.15.19:0/1378315324 conn(0x7f9a700ffec0 msgr2=0x7f9a70102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:26:23.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:23.089+0000 7f9a76800700 1 -- 172.21.15.19:0/1378315324 shutdown_connections 2024-05-07T02:26:23.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:23.089+0000 7f9a76800700 1 -- 172.21.15.19:0/1378315324 wait complete. 2024-05-07T02:26:23.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:23.090+0000 7f9a76800700 1 Processor -- start 2024-05-07T02:26:23.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:23.090+0000 7f9a76800700 1 -- start start 2024-05-07T02:26:23.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:23.091+0000 7f9a76800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a701045f0 0x7f9a7019d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:23.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:23.091+0000 7f9a76800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a70108f20 0x7f9a7019d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:23.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:23.091+0000 7f9a76800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a70109870 0x7f9a701a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:23.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:23.091+0000 7f9a76800700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9a701144f0 con 0x7f9a701045f0 2024-05-07T02:26:23.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:23.091+0000 7f9a76800700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9a70114370 con 0x7f9a70108f20 2024-05-07T02:26:23.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:23.091+0000 7f9a76800700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9a70114670 con 0x7f9a70109870 2024-05-07T02:26:23.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:23.091+0000 7f9a6f400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a701045f0 0x7f9a7019d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:23.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:23.091+0000 7f9a6ea00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a70108f20 0x7f9a7019d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:23.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:23.091+0000 7f9a6ea00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a70108f20 0x7f9a7019d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:60214/0 (socket says 172.21.15.19:60214) 2024-05-07T02:26:23.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:23.091+0000 7f9a6ea00700 1 -- 172.21.15.19:0/2385452305 learned_addr learned my addr 172.21.15.19:0/2385452305 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:26:23.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:23.091+0000 7f9a6f400700 1 --2- 172.21.15.19:0/2385452305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a701045f0 0x7f9a7019d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49828/0 (socket says 172.21.15.19:49828) 2024-05-07T02:26:23.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:23.091+0000 7f9a6fe00700 1 --2- 172.21.15.19:0/2385452305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a70109870 0x7f9a701a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:23.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:23.092+0000 7f9a6ea00700 1 -- 172.21.15.19:0/2385452305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a70109870 msgr2=0x7f9a701a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:23.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:23.092+0000 7f9a6ea00700 1 --2- 172.21.15.19:0/2385452305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a70109870 0x7f9a701a1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:23.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:23.092+0000 7f9a6ea00700 1 -- 172.21.15.19:0/2385452305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a701045f0 msgr2=0x7f9a7019d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:23.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:23.092+0000 7f9a6ea00700 1 --2- 172.21.15.19:0/2385452305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a701045f0 0x7f9a7019d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:23.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:23.092+0000 7f9a6ea00700 1 -- 172.21.15.19:0/2385452305 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a701a1f10 con 0x7f9a70108f20 2024-05-07T02:26:23.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:23.092+0000 7f9a6ea00700 1 --2- 172.21.15.19:0/2385452305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a70108f20 0x7f9a7019d5e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c00ec70 tx=0x7f9a5c00ef80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:26:23.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:23.093+0000 7f9a67e00700 1 -- 172.21.15.19:0/2385452305 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c00cc90 con 0x7f9a70108f20 2024-05-07T02:26:23.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:23.093+0000 7f9a76800700 1 -- 172.21.15.19:0/2385452305 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a701a2200 con 0x7f9a70108f20 2024-05-07T02:26:23.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:23.093+0000 7f9a67e00700 1 -- 172.21.15.19:0/2385452305 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a5c00ce30 con 0x7f9a70108f20 2024-05-07T02:26:23.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:23.093+0000 7f9a67e00700 1 -- 172.21.15.19:0/2385452305 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c010430 con 0x7f9a70108f20 2024-05-07T02:26:23.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:23.093+0000 7f9a76800700 1 -- 172.21.15.19:0/2385452305 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a701a9b20 con 0x7f9a70108f20 2024-05-07T02:26:23.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:23.094+0000 7f9a76800700 1 -- 172.21.15.19:0/2385452305 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a700512c0 con 0x7f9a70108f20 2024-05-07T02:26:23.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:23.097+0000 7f9a67e00700 1 -- 172.21.15.19:0/2385452305 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f9a5c003680 con 0x7f9a70108f20 2024-05-07T02:26:23.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:23.099+0000 7f9a67e00700 1 --2- 172.21.15.19:0/2385452305 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f9a540b2880 0x7f9a540b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:23.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:23.099+0000 7f9a67e00700 1 -- 172.21.15.19:0/2385452305 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8277+0+0 (secure 0 0 0) 0x7f9a5c0e24e0 con 0x7f9a70108f20 2024-05-07T02:26:23.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:23.099+0000 7f9a6f400700 1 --2- 172.21.15.19:0/2385452305 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f9a540b2880 0x7f9a540b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:23.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:23.100+0000 7f9a6f400700 1 --2- 172.21.15.19:0/2385452305 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f9a540b2880 0x7f9a540b4d90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9a60009c60 tx=0x7f9a60009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:26:23.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:23.104+0000 7f9a67e00700 1 -- 172.21.15.19:0/2385452305 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9a5c0aa170 con 0x7f9a70108f20 2024-05-07T02:26:23.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:23.340+0000 7f9a76800700 1 -- 172.21.15.19:0/2385452305 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f9a701a9cf0 con 0x7f9a70108f20 2024-05-07T02:26:23.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:23 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/2385452305' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-07T02:26:23.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:23 smithi019 ceph-mon[29781]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-07T02:26:23.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:23 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/2385452305' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-07T02:26:23.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:23 smithi064 ceph-mon[34699]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-07T02:26:23.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:23 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/2385452305' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-07T02:26:23.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:23 smithi027 ceph-mon[34398]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-07T02:26:24.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:24.529+0000 7f9a67e00700 1 -- 172.21.15.19:0/2385452305 <== mon.2 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v81) ==== 228+0+0 (secure 0 0 0) 0x7f9a5c0a9a70 con 0x7f9a70108f20 2024-05-07T02:26:24.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:24.540+0000 7f9a76800700 1 -- 172.21.15.19:0/2385452305 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f9a540b2880 msgr2=0x7f9a540b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:24.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:24.540+0000 7f9a76800700 1 --2- 172.21.15.19:0/2385452305 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f9a540b2880 0x7f9a540b4d90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9a60009c60 tx=0x7f9a60009260 comp rx=0 tx=0).stop 2024-05-07T02:26:24.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:24.540+0000 7f9a76800700 1 -- 172.21.15.19:0/2385452305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a70108f20 msgr2=0x7f9a7019d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:24.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:24.540+0000 7f9a76800700 1 --2- 172.21.15.19:0/2385452305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a70108f20 0x7f9a7019d5e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c00ec70 tx=0x7f9a5c00ef80 comp rx=0 tx=0).stop 2024-05-07T02:26:24.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:24.541+0000 7f9a76800700 1 -- 172.21.15.19:0/2385452305 shutdown_connections 2024-05-07T02:26:24.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:24.541+0000 7f9a76800700 1 --2- 172.21.15.19:0/2385452305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a70109870 0x7f9a701a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:24.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:24.541+0000 7f9a76800700 1 --2- 172.21.15.19:0/2385452305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a701045f0 0x7f9a7019d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:24.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:24.541+0000 7f9a76800700 1 --2- 172.21.15.19:0/2385452305 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f9a540b2880 0x7f9a540b4d90 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:24.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:24.541+0000 7f9a76800700 1 --2- 172.21.15.19:0/2385452305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a70108f20 0x7f9a7019d5e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:24.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:24.541+0000 7f9a76800700 1 -- 172.21.15.19:0/2385452305 >> 172.21.15.19:0/2385452305 conn(0x7f9a700ffec0 msgr2=0x7f9a7010a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:26:24.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:24.541+0000 7f9a76800700 1 -- 172.21.15.19:0/2385452305 shutdown_connections 2024-05-07T02:26:24.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:24.542+0000 7f9a76800700 1 -- 172.21.15.19:0/2385452305 wait complete. 2024-05-07T02:26:24.547 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data_ec' created 2024-05-07T02:26:24.858 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:24 smithi027 ceph-mon[34398]: pgmap v335: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-07T02:26:24.858 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:24 smithi027 ceph-mon[34398]: osdmap e80: 12 total, 12 up, 12 in 2024-05-07T02:26:24.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:24 smithi064 ceph-mon[34699]: pgmap v335: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-07T02:26:24.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:24 smithi064 ceph-mon[34699]: osdmap e80: 12 total, 12 up, 12 in 2024-05-07T02:26:24.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:24 smithi019 ceph-mon[29781]: pgmap v335: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-07T02:26:24.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:24 smithi019 ceph-mon[29781]: osdmap e80: 12 total, 12 up, 12 in 2024-05-07T02:26:25.415 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-05-07T02:26:25.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:25 smithi064 ceph-mon[34699]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-07T02:26:25.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:25 smithi064 ceph-mon[34699]: osdmap e81: 12 total, 12 up, 12 in 2024-05-07T02:26:25.873 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:25 smithi027 ceph-mon[34398]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-07T02:26:25.873 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:25 smithi027 ceph-mon[34398]: osdmap e81: 12 total, 12 up, 12 in 2024-05-07T02:26:25.875 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:26:25.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:25 smithi019 ceph-mon[29781]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-07T02:26:25.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:25 smithi019 ceph-mon[29781]: osdmap e81: 12 total, 12 up, 12 in 2024-05-07T02:26:26.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:26 smithi064 ceph-mon[34699]: pgmap v338: 129 pgs: 4 creating+peering, 28 unknown, 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:26.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:26 smithi064 ceph-mon[34699]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-07T02:26:26.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:26 smithi064 ceph-mon[34699]: osdmap e82: 12 total, 12 up, 12 in 2024-05-07T02:26:26.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:26 smithi019 ceph-mon[29781]: pgmap v338: 129 pgs: 4 creating+peering, 28 unknown, 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:26.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:26 smithi019 ceph-mon[29781]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-07T02:26:26.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:26 smithi019 ceph-mon[29781]: osdmap e82: 12 total, 12 up, 12 in 2024-05-07T02:26:26.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:26 smithi027 ceph-mon[34398]: pgmap v338: 129 pgs: 4 creating+peering, 28 unknown, 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:26.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:26 smithi027 ceph-mon[34398]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-07T02:26:26.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:26 smithi027 ceph-mon[34398]: osdmap e82: 12 total, 12 up, 12 in 2024-05-07T02:26:27.820 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:27 smithi019 ceph-mon[29781]: osdmap e83: 12 total, 12 up, 12 in 2024-05-07T02:26:27.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:27 smithi064 ceph-mon[34699]: osdmap e83: 12 total, 12 up, 12 in 2024-05-07T02:26:27.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:27 smithi027 ceph-mon[34398]: osdmap e83: 12 total, 12 up, 12 in 2024-05-07T02:26:28.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.247+0000 7f8a2d000700 1 -- 172.21.15.19:0/3556354261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a28115200 msgr2=0x7f8a28117620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:28.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.247+0000 7f8a2d000700 1 --2- 172.21.15.19:0/3556354261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a28115200 0x7f8a28117620 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8a20009f90 tx=0x7f8a2000b330 comp rx=0 tx=0).stop 2024-05-07T02:26:28.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.247+0000 7f8a2d000700 1 -- 172.21.15.19:0/3556354261 shutdown_connections 2024-05-07T02:26:28.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.247+0000 7f8a2d000700 1 --2- 172.21.15.19:0/3556354261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a28115200 0x7f8a28117620 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:28.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.247+0000 7f8a2d000700 1 --2- 172.21.15.19:0/3556354261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a28077cf0 0x7f8a28078100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:28.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.247+0000 7f8a2d000700 1 --2- 172.21.15.19:0/3556354261 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a28078640 0x7f8a28114cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:28.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.247+0000 7f8a2d000700 1 -- 172.21.15.19:0/3556354261 >> 172.21.15.19:0/3556354261 conn(0x7f8a281020c0 msgr2=0x7f8a28104510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:26:28.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.247+0000 7f8a2d000700 1 -- 172.21.15.19:0/3556354261 shutdown_connections 2024-05-07T02:26:28.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.248+0000 7f8a2d000700 1 -- 172.21.15.19:0/3556354261 wait complete. 2024-05-07T02:26:28.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.249+0000 7f8a2d000700 1 Processor -- start 2024-05-07T02:26:28.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.249+0000 7f8a2d000700 1 -- start start 2024-05-07T02:26:28.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.249+0000 7f8a2d000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a28077cf0 0x7f8a281a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:28.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.249+0000 7f8a2d000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a28078640 0x7f8a281a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:28.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.249+0000 7f8a2d000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a28115200 0x7f8a281a5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:28.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.249+0000 7f8a2d000700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2811aae0 con 0x7f8a28077cf0 2024-05-07T02:26:28.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.249+0000 7f8a2d000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2811a960 con 0x7f8a28115200 2024-05-07T02:26:28.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.249+0000 7f8a2d000700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2811ac60 con 0x7f8a28078640 2024-05-07T02:26:28.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.250+0000 7f8a25400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a28078640 0x7f8a281a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:28.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.250+0000 7f8a25e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a28077cf0 0x7f8a281a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:28.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.250+0000 7f8a25400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a28078640 0x7f8a281a1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:44024/0 (socket says 172.21.15.19:44024) 2024-05-07T02:26:28.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.250+0000 7f8a25400700 1 -- 172.21.15.19:0/4103681806 learned_addr learned my addr 172.21.15.19:0/4103681806 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:26:28.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.250+0000 7f8a26800700 1 --2- 172.21.15.19:0/4103681806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a28115200 0x7f8a281a5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:28.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.250+0000 7f8a25400700 1 -- 172.21.15.19:0/4103681806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a28115200 msgr2=0x7f8a281a5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:28.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.250+0000 7f8a25400700 1 --2- 172.21.15.19:0/4103681806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a28115200 0x7f8a281a5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:28.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.250+0000 7f8a25400700 1 -- 172.21.15.19:0/4103681806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a28077cf0 msgr2=0x7f8a281a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:28.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.250+0000 7f8a25400700 1 --2- 172.21.15.19:0/4103681806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a28077cf0 0x7f8a281a13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:28.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.250+0000 7f8a25400700 1 -- 172.21.15.19:0/4103681806 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a281a62a0 con 0x7f8a28078640 2024-05-07T02:26:28.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.251+0000 7f8a25400700 1 --2- 172.21.15.19:0/4103681806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a28078640 0x7f8a281a1970 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400e970 tx=0x7f8a1400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:26:28.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.251+0000 7f8a1ea00700 1 -- 172.21.15.19:0/4103681806 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a14004a30 con 0x7f8a28078640 2024-05-07T02:26:28.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.251+0000 7f8a1ea00700 1 -- 172.21.15.19:0/4103681806 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a1400cc70 con 0x7f8a28078640 2024-05-07T02:26:28.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.251+0000 7f8a1ea00700 1 -- 172.21.15.19:0/4103681806 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a14005580 con 0x7f8a28078640 2024-05-07T02:26:28.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.251+0000 7f8a2d000700 1 -- 172.21.15.19:0/4103681806 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a281a6590 con 0x7f8a28078640 2024-05-07T02:26:28.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.251+0000 7f8a2d000700 1 -- 172.21.15.19:0/4103681806 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a281adeb0 con 0x7f8a28078640 2024-05-07T02:26:28.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.252+0000 7f8a2d000700 1 -- 172.21.15.19:0/4103681806 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a280512c0 con 0x7f8a28078640 2024-05-07T02:26:28.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.255+0000 7f8a1ea00700 1 -- 172.21.15.19:0/4103681806 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f8a1400c7c0 con 0x7f8a28078640 2024-05-07T02:26:28.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.256+0000 7f8a1ea00700 1 --2- 172.21.15.19:0/4103681806 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f8a0c0b2900 0x7f8a0c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:28.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.257+0000 7f8a1ea00700 1 -- 172.21.15.19:0/4103681806 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8770+0+0 (secure 0 0 0) 0x7f8a140e2730 con 0x7f8a28078640 2024-05-07T02:26:28.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.257+0000 7f8a25e00700 1 --2- 172.21.15.19:0/4103681806 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f8a0c0b2900 0x7f8a0c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:28.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.258+0000 7f8a25e00700 1 --2- 172.21.15.19:0/4103681806 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f8a0c0b2900 0x7f8a0c0b4e10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8a18005fd0 tx=0x7f8a18009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:26:28.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.259+0000 7f8a1ea00700 1 -- 172.21.15.19:0/4103681806 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8a14054df0 con 0x7f8a28078640 2024-05-07T02:26:28.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.497+0000 7f8a2d000700 1 -- 172.21.15.19:0/4103681806 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f8a2802e5f0 con 0x7f8a28078640 2024-05-07T02:26:28.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.567+0000 7f8a1ea00700 1 -- 172.21.15.19:0/4103681806 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v84) ==== 169+0+0 (secure 0 0 0) 0x7f8a140aa1d0 con 0x7f8a28078640 2024-05-07T02:26:28.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.572+0000 7f8a2d000700 1 -- 172.21.15.19:0/4103681806 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f8a0c0b2900 msgr2=0x7f8a0c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:28.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.572+0000 7f8a2d000700 1 --2- 172.21.15.19:0/4103681806 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f8a0c0b2900 0x7f8a0c0b4e10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8a18005fd0 tx=0x7f8a18009310 comp rx=0 tx=0).stop 2024-05-07T02:26:28.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.572+0000 7f8a2d000700 1 -- 172.21.15.19:0/4103681806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a28078640 msgr2=0x7f8a281a1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:28.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.572+0000 7f8a2d000700 1 --2- 172.21.15.19:0/4103681806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a28078640 0x7f8a281a1970 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400e970 tx=0x7f8a1400ec80 comp rx=0 tx=0).stop 2024-05-07T02:26:28.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.572+0000 7f8a2d000700 1 -- 172.21.15.19:0/4103681806 shutdown_connections 2024-05-07T02:26:28.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.572+0000 7f8a2d000700 1 --2- 172.21.15.19:0/4103681806 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f8a0c0b2900 0x7f8a0c0b4e10 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:28.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.572+0000 7f8a2d000700 1 --2- 172.21.15.19:0/4103681806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a28115200 0x7f8a281a5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:28.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.572+0000 7f8a2d000700 1 --2- 172.21.15.19:0/4103681806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a28077cf0 0x7f8a281a13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:28.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.572+0000 7f8a2d000700 1 --2- 172.21.15.19:0/4103681806 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a28078640 0x7f8a281a1970 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:28.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.572+0000 7f8a2d000700 1 -- 172.21.15.19:0/4103681806 >> 172.21.15.19:0/4103681806 conn(0x7f8a281020c0 msgr2=0x7f8a2807aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:26:28.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.572+0000 7f8a2d000700 1 -- 172.21.15.19:0/4103681806 shutdown_connections 2024-05-07T02:26:28.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:28.572+0000 7f8a2d000700 1 -- 172.21.15.19:0/4103681806 wait complete. 2024-05-07T02:26:28.574 INFO:teuthology.orchestra.run.smithi019.stderr:set pool 4 allow_ec_overwrites to true 2024-05-07T02:26:28.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:28 smithi064 ceph-mon[34699]: pgmap v341: 129 pgs: 8 creating+peering, 15 unknown, 106 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:28.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:28 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/4103681806' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-07T02:26:28.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:28 smithi064 ceph-mon[34699]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-07T02:26:28.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:28 smithi019 ceph-mon[29781]: pgmap v341: 129 pgs: 8 creating+peering, 15 unknown, 106 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:28.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:28 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/4103681806' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-07T02:26:28.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:28 smithi019 ceph-mon[29781]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-07T02:26:28.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:28 smithi027 ceph-mon[34398]: pgmap v341: 129 pgs: 8 creating+peering, 15 unknown, 106 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:28.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:28 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/4103681806' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-07T02:26:28.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:28 smithi027 ceph-mon[34398]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-07T02:26:29.480 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-05-07T02:26:29.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:29 smithi064 ceph-mon[34699]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-07T02:26:29.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:29 smithi064 ceph-mon[34699]: osdmap e84: 12 total, 12 up, 12 in 2024-05-07T02:26:29.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:29 smithi019 ceph-mon[29781]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-07T02:26:29.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:29 smithi019 ceph-mon[29781]: osdmap e84: 12 total, 12 up, 12 in 2024-05-07T02:26:29.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:29 smithi027 ceph-mon[34398]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-07T02:26:29.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:29 smithi027 ceph-mon[34398]: osdmap e84: 12 total, 12 up, 12 in 2024-05-07T02:26:29.966 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:26:30.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:30 smithi064 ceph-mon[34699]: pgmap v343: 129 pgs: 8 creating+peering, 121 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:30.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:30 smithi019 ceph-mon[29781]: pgmap v343: 129 pgs: 8 creating+peering, 121 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:30.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:30 smithi027 ceph-mon[34398]: pgmap v343: 129 pgs: 8 creating+peering, 121 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:32.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:32.384+0000 7f7907800700 1 -- 172.21.15.19:0/3350145022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7900110ba0 msgr2=0x7f7900113000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:32.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:32.384+0000 7f7907800700 1 --2- 172.21.15.19:0/3350145022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7900110ba0 0x7f7900113000 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f78f800b0a0 tx=0x7f78f800b3b0 comp rx=0 tx=0).stop 2024-05-07T02:26:32.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:32.384+0000 7f7907800700 1 -- 172.21.15.19:0/3350145022 shutdown_connections 2024-05-07T02:26:32.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:32.384+0000 7f7907800700 1 --2- 172.21.15.19:0/3350145022 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7900110ba0 0x7f7900113000 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:32.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:32.384+0000 7f7907800700 1 --2- 172.21.15.19:0/3350145022 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7900075d30 0x7f7900076140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:32.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:32.384+0000 7f7907800700 1 --2- 172.21.15.19:0/3350145022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7900076680 0x7f7900110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:32.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:32.384+0000 7f7907800700 1 -- 172.21.15.19:0/3350145022 >> 172.21.15.19:0/3350145022 conn(0x7f7900100040 msgr2=0x7f7900102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:26:32.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:32.384+0000 7f7907800700 1 -- 172.21.15.19:0/3350145022 shutdown_connections 2024-05-07T02:26:32.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:32.384+0000 7f7907800700 1 -- 172.21.15.19:0/3350145022 wait complete. 2024-05-07T02:26:32.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:32.386+0000 7f7907800700 1 Processor -- start 2024-05-07T02:26:32.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:32.386+0000 7f7907800700 1 -- start start 2024-05-07T02:26:32.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:32.386+0000 7f7907800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7900075d30 0x7f790019cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:32.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:32.386+0000 7f7907800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7900076680 0x7f790019d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:32.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:32.386+0000 7f7907800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7900110ba0 0x7f79001a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:32.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:32.386+0000 7f7907800700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7900116680 con 0x7f7900076680 2024-05-07T02:26:32.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:32.386+0000 7f7907800700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7900116500 con 0x7f7900110ba0 2024-05-07T02:26:32.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:32.386+0000 7f7907800700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7900116800 con 0x7f7900075d30 2024-05-07T02:26:32.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:32.387+0000 7f78f7e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7900076680 0x7f790019d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:32.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:32.387+0000 7f7904e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7900075d30 0x7f790019cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:32.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:32.387+0000 7f7904e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7900075d30 0x7f790019cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:42810/0 (socket says 172.21.15.19:42810) 2024-05-07T02:26:32.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:32.387+0000 7f78f7e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7900076680 0x7f790019d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49236/0 (socket says 172.21.15.19:49236) 2024-05-07T02:26:32.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:32.387+0000 7f7904e00700 1 -- 172.21.15.19:0/3037222309 learned_addr learned my addr 172.21.15.19:0/3037222309 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:26:32.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:32.387+0000 7f7905800700 1 --2- 172.21.15.19:0/3037222309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7900110ba0 0x7f79001a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:32.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:32.387+0000 7f7904e00700 1 -- 172.21.15.19:0/3037222309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7900110ba0 msgr2=0x7f79001a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:32.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:32.387+0000 7f7904e00700 1 --2- 172.21.15.19:0/3037222309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7900110ba0 0x7f79001a1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:32.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:32.387+0000 7f7904e00700 1 -- 172.21.15.19:0/3037222309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7900076680 msgr2=0x7f790019d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:32.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:32.387+0000 7f7904e00700 1 --2- 172.21.15.19:0/3037222309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7900076680 0x7f790019d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:32.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:32.387+0000 7f7904e00700 1 -- 172.21.15.19:0/3037222309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79001a1ea0 con 0x7f7900075d30 2024-05-07T02:26:32.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:32.388+0000 7f7904e00700 1 --2- 172.21.15.19:0/3037222309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7900075d30 0x7f790019cfd0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f78f000eb30 tx=0x7f78f000ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:26:32.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:32.388+0000 7f78fe000700 1 -- 172.21.15.19:0/3037222309 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f000cbd0 con 0x7f7900075d30 2024-05-07T02:26:32.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:32.388+0000 7f7907800700 1 -- 172.21.15.19:0/3037222309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79001a2190 con 0x7f7900075d30 2024-05-07T02:26:32.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:32.388+0000 7f78fe000700 1 -- 172.21.15.19:0/3037222309 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78f00044d0 con 0x7f7900075d30 2024-05-07T02:26:32.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:32.388+0000 7f78fe000700 1 -- 172.21.15.19:0/3037222309 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f0003e30 con 0x7f7900075d30 2024-05-07T02:26:32.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:32.388+0000 7f7907800700 1 -- 172.21.15.19:0/3037222309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79001a9ab0 con 0x7f7900075d30 2024-05-07T02:26:32.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:32.390+0000 7f7907800700 1 -- 172.21.15.19:0/3037222309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79000512c0 con 0x7f7900075d30 2024-05-07T02:26:32.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:32.392+0000 7f78fe000700 1 -- 172.21.15.19:0/3037222309 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f78f0004000 con 0x7f7900075d30 2024-05-07T02:26:32.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:32.393+0000 7f78fe000700 1 --2- 172.21.15.19:0/3037222309 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f78e40b2880 0x7f78e40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:32.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:32.394+0000 7f78fe000700 1 -- 172.21.15.19:0/3037222309 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8770+0+0 (secure 0 0 0) 0x7f78f00e2870 con 0x7f7900075d30 2024-05-07T02:26:32.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:32.394+0000 7f78f7e00700 1 --2- 172.21.15.19:0/3037222309 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f78e40b2880 0x7f78e40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:32.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:32.394+0000 7f78f7e00700 1 --2- 172.21.15.19:0/3037222309 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f78e40b2880 0x7f78e40b4d90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f79001034b0 tx=0x7f78e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:26:32.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:32.398+0000 7f78fe000700 1 -- 172.21.15.19:0/3037222309 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f78f00042f0 con 0x7f7900075d30 2024-05-07T02:26:32.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:32 smithi019 ceph-mon[29781]: pgmap v344: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:32.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:32.690+0000 7f7907800700 1 -- 172.21.15.19:0/3037222309 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f790002e5f0 con 0x7f7900075d30 2024-05-07T02:26:32.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:32 smithi064 ceph-mon[34699]: pgmap v344: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:32.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:32 smithi027 ceph-mon[34398]: pgmap v344: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:33.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:33.618+0000 7f78fe000700 1 -- 172.21.15.19:0/3037222309 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f78f00aa310 con 0x7f7900075d30 2024-05-07T02:26:33.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:33.622+0000 7f7907800700 1 -- 172.21.15.19:0/3037222309 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f78e40b2880 msgr2=0x7f78e40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:33.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:33.622+0000 7f7907800700 1 --2- 172.21.15.19:0/3037222309 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f78e40b2880 0x7f78e40b4d90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f79001034b0 tx=0x7f78e8009260 comp rx=0 tx=0).stop 2024-05-07T02:26:33.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:33.622+0000 7f7907800700 1 -- 172.21.15.19:0/3037222309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7900075d30 msgr2=0x7f790019cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:33.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:33.622+0000 7f7907800700 1 --2- 172.21.15.19:0/3037222309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7900075d30 0x7f790019cfd0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f78f000eb30 tx=0x7f78f000ee40 comp rx=0 tx=0).stop 2024-05-07T02:26:33.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:33.622+0000 7f7907800700 1 -- 172.21.15.19:0/3037222309 shutdown_connections 2024-05-07T02:26:33.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:33.622+0000 7f7907800700 1 --2- 172.21.15.19:0/3037222309 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f78e40b2880 0x7f78e40b4d90 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:33.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:33.622+0000 7f7907800700 1 --2- 172.21.15.19:0/3037222309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7900110ba0 0x7f79001a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:33.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:33.622+0000 7f7907800700 1 --2- 172.21.15.19:0/3037222309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7900075d30 0x7f790019cfd0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:33.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:33.622+0000 7f7907800700 1 --2- 172.21.15.19:0/3037222309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7900076680 0x7f790019d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:33.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:33.622+0000 7f7907800700 1 -- 172.21.15.19:0/3037222309 >> 172.21.15.19:0/3037222309 conn(0x7f7900100040 msgr2=0x7f7900101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:26:33.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:33.623+0000 7f7907800700 1 -- 172.21.15.19:0/3037222309 shutdown_connections 2024-05-07T02:26:33.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:33.623+0000 7f7907800700 1 -- 172.21.15.19:0/3037222309 wait complete. 2024-05-07T02:26:33.627 INFO:teuthology.orchestra.run.smithi019.stderr:added data pool 4 to fsmap 2024-05-07T02:26:33.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:33 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/3037222309' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-07T02:26:33.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:33 smithi064 ceph-mon[34699]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-07T02:26:33.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:33 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/3037222309' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-07T02:26:33.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:33 smithi019 ceph-mon[29781]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-07T02:26:33.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:33 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/3037222309' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-07T02:26:33.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:33 smithi027 ceph-mon[34398]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-07T02:26:34.623 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-07T02:26:34.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:34 smithi064 ceph-mon[34699]: pgmap v345: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:34.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:34 smithi064 ceph-mon[34699]: osdmap e85: 12 total, 12 up, 12 in 2024-05-07T02:26:34.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:34 smithi064 ceph-mon[34699]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-07T02:26:34.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:34 smithi064 ceph-mon[34699]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-07T02:26:34.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:34 smithi019 ceph-mon[29781]: pgmap v345: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:34.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:34 smithi019 ceph-mon[29781]: osdmap e85: 12 total, 12 up, 12 in 2024-05-07T02:26:34.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:34 smithi019 ceph-mon[29781]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-07T02:26:34.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:34 smithi019 ceph-mon[29781]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-07T02:26:34.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:34 smithi027 ceph-mon[34398]: pgmap v345: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:34.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:34 smithi027 ceph-mon[34398]: osdmap e85: 12 total, 12 up, 12 in 2024-05-07T02:26:34.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:34 smithi027 ceph-mon[34398]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-07T02:26:34.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:34 smithi027 ceph-mon[34398]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-07T02:26:35.097 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:26:35.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:35 smithi019 ceph-mon[29781]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-07T02:26:35.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:35 smithi019 ceph-mon[29781]: Cluster is now healthy 2024-05-07T02:26:35.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:35 smithi027 ceph-mon[34398]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-07T02:26:35.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:35 smithi027 ceph-mon[34398]: Cluster is now healthy 2024-05-07T02:26:36.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:35 smithi064 ceph-mon[34699]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-07T02:26:36.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:35 smithi064 ceph-mon[34699]: Cluster is now healthy 2024-05-07T02:26:36.889 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:36 smithi019 ceph-mon[29781]: pgmap v347: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:36.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:36 smithi027 ceph-mon[34398]: pgmap v347: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:37.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:36 smithi064 ceph-mon[34699]: pgmap v347: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:37.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.501+0000 7f7a4f000700 1 -- 172.21.15.19:0/1593433589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a4810b110 msgr2=0x7f7a48117640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:37.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.501+0000 7f7a4f000700 1 --2- 172.21.15.19:0/1593433589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a4810b110 0x7f7a48117640 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7a40009f90 tx=0x7f7a4000b330 comp rx=0 tx=0).stop 2024-05-07T02:26:37.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.502+0000 7f7a4f000700 1 -- 172.21.15.19:0/1593433589 shutdown_connections 2024-05-07T02:26:37.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.502+0000 7f7a4f000700 1 --2- 172.21.15.19:0/1593433589 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a4810b110 0x7f7a48117640 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:37.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.502+0000 7f7a4f000700 1 --2- 172.21.15.19:0/1593433589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a481107a0 0x7f7a48110bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:37.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.502+0000 7f7a4f000700 1 --2- 172.21.15.19:0/1593433589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a4810a7c0 0x7f7a4810abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:37.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.502+0000 7f7a4f000700 1 -- 172.21.15.19:0/1593433589 >> 172.21.15.19:0/1593433589 conn(0x7f7a48076b00 msgr2=0x7f7a48076f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:26:37.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.502+0000 7f7a4f000700 1 -- 172.21.15.19:0/1593433589 shutdown_connections 2024-05-07T02:26:37.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.502+0000 7f7a4f000700 1 -- 172.21.15.19:0/1593433589 wait complete. 2024-05-07T02:26:37.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.503+0000 7f7a4f000700 1 Processor -- start 2024-05-07T02:26:37.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.503+0000 7f7a4f000700 1 -- start start 2024-05-07T02:26:37.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.504+0000 7f7a4f000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a4810a7c0 0x7f7a481a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:37.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.504+0000 7f7a4f000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a4810b110 0x7f7a481a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:37.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.504+0000 7f7a4f000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a481107a0 0x7f7a481a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:37.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.504+0000 7f7a4f000700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4811a900 con 0x7f7a481107a0 2024-05-07T02:26:37.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.504+0000 7f7a4f000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4811a780 con 0x7f7a4810a7c0 2024-05-07T02:26:37.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.504+0000 7f7a4f000700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4811aa80 con 0x7f7a4810b110 2024-05-07T02:26:37.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.504+0000 7f7a4dc00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a4810a7c0 0x7f7a481a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:37.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.504+0000 7f7a4d200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a4810b110 0x7f7a481a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:37.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.504+0000 7f7a4dc00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a4810a7c0 0x7f7a481a1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:33862/0 (socket says 172.21.15.19:33862) 2024-05-07T02:26:37.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.504+0000 7f7a4dc00700 1 -- 172.21.15.19:0/3987189802 learned_addr learned my addr 172.21.15.19:0/3987189802 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:26:37.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.504+0000 7f7a4e600700 1 --2- 172.21.15.19:0/3987189802 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a481107a0 0x7f7a481a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:37.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.505+0000 7f7a4dc00700 1 -- 172.21.15.19:0/3987189802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a4810b110 msgr2=0x7f7a481a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:37.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.505+0000 7f7a4dc00700 1 --2- 172.21.15.19:0/3987189802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a4810b110 0x7f7a481a17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:37.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.505+0000 7f7a4dc00700 1 -- 172.21.15.19:0/3987189802 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a481107a0 msgr2=0x7f7a481a5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:37.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.505+0000 7f7a4dc00700 1 --2- 172.21.15.19:0/3987189802 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a481107a0 0x7f7a481a5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:37.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.505+0000 7f7a4dc00700 1 -- 172.21.15.19:0/3987189802 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a481a6120 con 0x7f7a4810a7c0 2024-05-07T02:26:37.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.505+0000 7f7a4d200700 1 --2- 172.21.15.19:0/3987189802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a4810b110 0x7f7a481a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:26:37.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.505+0000 7f7a4dc00700 1 --2- 172.21.15.19:0/3987189802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a4810a7c0 0x7f7a481a1250 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c00e9c0 tx=0x7f7a3c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:26:37.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.505+0000 7f7a46000700 1 -- 172.21.15.19:0/3987189802 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c004a30 con 0x7f7a4810a7c0 2024-05-07T02:26:37.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.505+0000 7f7a46000700 1 -- 172.21.15.19:0/3987189802 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a3c00cc70 con 0x7f7a4810a7c0 2024-05-07T02:26:37.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.505+0000 7f7a4f000700 1 -- 172.21.15.19:0/3987189802 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a481a6410 con 0x7f7a4810a7c0 2024-05-07T02:26:37.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.505+0000 7f7a46000700 1 -- 172.21.15.19:0/3987189802 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c005490 con 0x7f7a4810a7c0 2024-05-07T02:26:37.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.506+0000 7f7a4f000700 1 -- 172.21.15.19:0/3987189802 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a481add30 con 0x7f7a4810a7c0 2024-05-07T02:26:37.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.507+0000 7f7a4f000700 1 -- 172.21.15.19:0/3987189802 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a480512c0 con 0x7f7a4810a7c0 2024-05-07T02:26:37.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.509+0000 7f7a46000700 1 -- 172.21.15.19:0/3987189802 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7a3c004bd0 con 0x7f7a4810a7c0 2024-05-07T02:26:37.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.511+0000 7f7a46000700 1 --2- 172.21.15.19:0/3987189802 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f7a300b2880 0x7f7a300b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:37.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.511+0000 7f7a46000700 1 -- 172.21.15.19:0/3987189802 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f7a3c0e2750 con 0x7f7a4810a7c0 2024-05-07T02:26:37.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.511+0000 7f7a4d200700 1 --2- 172.21.15.19:0/3987189802 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f7a300b2880 0x7f7a300b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:37.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.512+0000 7f7a4d200700 1 --2- 172.21.15.19:0/3987189802 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f7a300b2880 0x7f7a300b4d90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7a48077de0 tx=0x7f7a34009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:26:37.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.513+0000 7f7a46000700 1 -- 172.21.15.19:0/3987189802 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7a3c054df0 con 0x7f7a4810a7c0 2024-05-07T02:26:37.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.808+0000 7f7a4f000700 1 -- 172.21.15.19:0/3987189802 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7a4802e5f0 con 0x7f7a4810a7c0 2024-05-07T02:26:37.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.809+0000 7f7a46000700 1 -- 172.21.15.19:0/3987189802 <== mon.2 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11788 (secure 0 0 0) 0x7f7a3c0aa1d0 con 0x7f7a4810a7c0 2024-05-07T02:26:37.810 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:26:37.810 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1763670446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1763670446},{"type":"v1","addr":"172.21.15.19:6839","nonce":1763670446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/477529985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":477529985},{"type":"v1","addr":"172.21.15.64:6833","nonce":477529985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/2171873162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2171873162},{"type":"v1","addr":"172.21.15.27:6833","nonce":2171873162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24469,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2379917839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2379917839},{"type":"v1","addr":"172.21.15.19:6835","nonce":2379917839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2067515538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2067515538},{"type":"v1","addr":"172.21.15.19:6837","nonce":2067515538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1000540600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1000540600},{"type":"v1","addr":"172.21.15.27:6835","nonce":1000540600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1997318872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1997318872},{"type":"v1","addr":"172.21.15.64:6835","nonce":1997318872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2585907777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2585907777},{"type":"v1","addr":"172.21.15.64:6837","nonce":2585907777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24515,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2583138143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2583138143},{"type":"v1","addr":"172.21.15.27:6837","nonce":2583138143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/1583120004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":1583120004},{"type":"v1","addr":"172.21.15.64:6839","nonce":1583120004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24539,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/3905645246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3905645246},{"type":"v1","addr":"172.21.15.19:6841","nonce":3905645246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-07T02:26:14.960465+0000","modified":"2024-05-07T02:26:32.693402+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24551},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6839/3963268634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3963268634},{"type":"v1","addr":"172.21.15.27:6839","nonce":3963268634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24551,"qdb_cluster":[24551]},"id":1}]} 2024-05-07T02:26:37.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.813+0000 7f7a4f000700 1 -- 172.21.15.19:0/3987189802 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f7a300b2880 msgr2=0x7f7a300b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:37.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.813+0000 7f7a4f000700 1 --2- 172.21.15.19:0/3987189802 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f7a300b2880 0x7f7a300b4d90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7a48077de0 tx=0x7f7a34009260 comp rx=0 tx=0).stop 2024-05-07T02:26:37.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.813+0000 7f7a4f000700 1 -- 172.21.15.19:0/3987189802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a4810a7c0 msgr2=0x7f7a481a1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:37.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.813+0000 7f7a4f000700 1 --2- 172.21.15.19:0/3987189802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a4810a7c0 0x7f7a481a1250 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c00e9c0 tx=0x7f7a3c00ecd0 comp rx=0 tx=0).stop 2024-05-07T02:26:37.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.813+0000 7f7a4f000700 1 -- 172.21.15.19:0/3987189802 shutdown_connections 2024-05-07T02:26:37.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.813+0000 7f7a4f000700 1 --2- 172.21.15.19:0/3987189802 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a481107a0 0x7f7a481a5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:37.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.813+0000 7f7a4f000700 1 --2- 172.21.15.19:0/3987189802 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f7a300b2880 0x7f7a300b4d90 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:37.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.813+0000 7f7a4f000700 1 --2- 172.21.15.19:0/3987189802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a4810a7c0 0x7f7a481a1250 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:37.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.813+0000 7f7a4f000700 1 --2- 172.21.15.19:0/3987189802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a4810b110 0x7f7a481a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:37.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.813+0000 7f7a4f000700 1 -- 172.21.15.19:0/3987189802 >> 172.21.15.19:0/3987189802 conn(0x7f7a48076b00 msgr2=0x7f7a48108880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:26:37.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.813+0000 7f7a4f000700 1 -- 172.21.15.19:0/3987189802 shutdown_connections 2024-05-07T02:26:37.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:37.813+0000 7f7a4f000700 1 -- 172.21.15.19:0/3987189802 wait complete. 2024-05-07T02:26:37.814 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 17 2024-05-07T02:26:38.694 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-07T02:26:38.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:38 smithi019 ceph-mon[29781]: pgmap v348: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:38.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:38 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/3987189802' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:26:38.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:38 smithi027 ceph-mon[34398]: pgmap v348: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:38.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:38 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/3987189802' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:26:39.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:38 smithi064 ceph-mon[34699]: pgmap v348: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:39.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:38 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/3987189802' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:26:39.173 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:26:40.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:40 smithi019 ceph-mon[29781]: pgmap v349: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:40.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:40 smithi027 ceph-mon[34398]: pgmap v349: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:41.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:40 smithi064 ceph-mon[34699]: pgmap v349: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:41.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.592+0000 7f9eec000700 1 -- 172.21.15.19:0/1199332192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ee410e8f0 msgr2=0x7f9ee4110dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:41.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.592+0000 7f9eec000700 1 --2- 172.21.15.19:0/1199332192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ee410e8f0 0x7f9ee4110dd0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9edc009990 tx=0x7f9edc009ca0 comp rx=0 tx=0).stop 2024-05-07T02:26:41.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.592+0000 7f9eec000700 1 -- 172.21.15.19:0/1199332192 shutdown_connections 2024-05-07T02:26:41.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.592+0000 7f9eec000700 1 --2- 172.21.15.19:0/1199332192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ee410e8f0 0x7f9ee4110dd0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:41.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.592+0000 7f9eec000700 1 --2- 172.21.15.19:0/1199332192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ee41054c0 0x7f9ee41058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:41.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.592+0000 7f9eec000700 1 --2- 172.21.15.19:0/1199332192 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ee4105e10 0x7f9ee410e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:41.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.592+0000 7f9eec000700 1 -- 172.21.15.19:0/1199332192 >> 172.21.15.19:0/1199332192 conn(0x7f9ee40fedc0 msgr2=0x7f9ee41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:26:41.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.592+0000 7f9eec000700 1 -- 172.21.15.19:0/1199332192 shutdown_connections 2024-05-07T02:26:41.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.592+0000 7f9eec000700 1 -- 172.21.15.19:0/1199332192 wait complete. 2024-05-07T02:26:41.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.593+0000 7f9eec000700 1 Processor -- start 2024-05-07T02:26:41.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.594+0000 7f9eec000700 1 -- start start 2024-05-07T02:26:41.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.594+0000 7f9eec000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ee41054c0 0x7f9ee419cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:41.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.594+0000 7f9eec000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ee4105e10 0x7f9ee419d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:41.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.594+0000 7f9eec000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ee410e8f0 0x7f9ee41a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:41.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.594+0000 7f9eec000700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee4114470 con 0x7f9ee410e8f0 2024-05-07T02:26:41.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.594+0000 7f9eec000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41142f0 con 0x7f9ee4105e10 2024-05-07T02:26:41.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.594+0000 7f9eec000700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee41145f0 con 0x7f9ee41054c0 2024-05-07T02:26:41.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.595+0000 7f9eea000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ee410e8f0 0x7f9ee41a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:41.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.595+0000 7f9eea000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ee410e8f0 0x7f9ee41a1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59238/0 (socket says 172.21.15.19:59238) 2024-05-07T02:26:41.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.595+0000 7f9ee8c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ee4105e10 0x7f9ee419d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:41.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.595+0000 7f9eea000700 1 -- 172.21.15.19:0/45001850 learned_addr learned my addr 172.21.15.19:0/45001850 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:26:41.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.595+0000 7f9ee9600700 1 --2- 172.21.15.19:0/45001850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ee41054c0 0x7f9ee419cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.595+0000 7f9eea000700 1 -- 172.21.15.19:0/45001850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ee41054c0 msgr2=0x7f9ee419cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.595+0000 7f9eea000700 1 --2- 172.21.15.19:0/45001850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ee41054c0 0x7f9ee419cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.595+0000 7f9eea000700 1 -- 172.21.15.19:0/45001850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ee4105e10 msgr2=0x7f9ee419d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.595+0000 7f9eea000700 1 --2- 172.21.15.19:0/45001850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ee4105e10 0x7f9ee419d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.595+0000 7f9eea000700 1 -- 172.21.15.19:0/45001850 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ee41a1e30 con 0x7f9ee410e8f0 2024-05-07T02:26:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.595+0000 7f9ee9600700 1 --2- 172.21.15.19:0/45001850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ee41054c0 0x7f9ee419cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:26:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.596+0000 7f9eea000700 1 --2- 172.21.15.19:0/45001850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ee410e8f0 0x7f9ee41a1790 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9edc0057c0 tx=0x7f9edc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:26:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.596+0000 7f9ee2000700 1 -- 172.21.15.19:0/45001850 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc027070 con 0x7f9ee410e8f0 2024-05-07T02:26:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.596+0000 7f9ee2000700 1 -- 172.21.15.19:0/45001850 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9edc022470 con 0x7f9ee410e8f0 2024-05-07T02:26:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.596+0000 7f9eec000700 1 -- 172.21.15.19:0/45001850 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ee41a20c0 con 0x7f9ee410e8f0 2024-05-07T02:26:41.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.596+0000 7f9ee2000700 1 -- 172.21.15.19:0/45001850 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc017650 con 0x7f9ee410e8f0 2024-05-07T02:26:41.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.596+0000 7f9eec000700 1 -- 172.21.15.19:0/45001850 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ee41a25a0 con 0x7f9ee410e8f0 2024-05-07T02:26:41.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.597+0000 7f9eec000700 1 -- 172.21.15.19:0/45001850 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ee40512c0 con 0x7f9ee410e8f0 2024-05-07T02:26:41.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.600+0000 7f9ee2000700 1 -- 172.21.15.19:0/45001850 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f9edc02a070 con 0x7f9ee410e8f0 2024-05-07T02:26:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.601+0000 7f9ee2000700 1 --2- 172.21.15.19:0/45001850 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f9ecc0b2880 0x7f9ecc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.602+0000 7f9ee2000700 1 -- 172.21.15.19:0/45001850 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f9edc0e33a0 con 0x7f9ee410e8f0 2024-05-07T02:26:41.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.602+0000 7f9ee9600700 1 --2- 172.21.15.19:0/45001850 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f9ecc0b2880 0x7f9ecc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:41.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.603+0000 7f9ee9600700 1 --2- 172.21.15.19:0/45001850 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f9ecc0b2880 0x7f9ecc0b4d90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8003e30 tx=0x7f9ed800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:26:41.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.605+0000 7f9ee2000700 1 -- 172.21.15.19:0/45001850 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9edc013070 con 0x7f9ee410e8f0 2024-05-07T02:26:41.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.849+0000 7f9eec000700 1 -- 172.21.15.19:0/45001850 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9ee41a9b60 con 0x7f9ee410e8f0 2024-05-07T02:26:41.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.851+0000 7f9ee2000700 1 -- 172.21.15.19:0/45001850 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24055 (secure 0 0 0) 0x7f9edc0aae80 con 0x7f9ee410e8f0 2024-05-07T02:26:41.851 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:26:41.852 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":85,"fsid":"9ff84c16-0c17-11ef-bc97-c7b262605968","created":"2024-05-07T02:16:19.507657+0000","modified":"2024-05-07T02:26:32.693063+0000","last_up_change":"2024-05-07T02:24:55.025124+0000","last_in_change":"2024-05-07T02:24:33.842422+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-07T02:20:21.439697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-07T02:26:06.178814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-07T02:26:10.705693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-07T02:26:23.534133+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b437cf22-ec8e-48f0-86a2-2413a0ab40dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6803","nonce":2390539683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6805","nonce":2390539683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6809","nonce":2390539683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6807","nonce":2390539683}]},"public_addr":"172.21.15.19:6803/2390539683","cluster_addr":"172.21.15.19:6805/2390539683","heartbeat_back_addr":"172.21.15.19:6809/2390539683","heartbeat_front_addr":"172.21.15.19:6807/2390539683","state":["exists","up"]},{"osd":1,"uuid":"dc753e16-634d-4c14-abd2-935281e1499e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6801","nonce":4292470615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6803","nonce":4292470615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6807","nonce":4292470615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6805","nonce":4292470615}]},"public_addr":"172.21.15.27:6801/4292470615","cluster_addr":"172.21.15.27:6803/4292470615","heartbeat_back_addr":"172.21.15.27:6807/4292470615","heartbeat_front_addr":"172.21.15.27:6805/4292470615","state":["exists","up"]},{"osd":2,"uuid":"40570071-646e-4e66-9a16-c55ab743ac86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6801","nonce":1180552939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6803","nonce":1180552939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6807","nonce":1180552939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6805","nonce":1180552939}]},"public_addr":"172.21.15.64:6801/1180552939","cluster_addr":"172.21.15.64:6803/1180552939","heartbeat_back_addr":"172.21.15.64:6807/1180552939","heartbeat_front_addr":"172.21.15.64:6805/1180552939","state":["exists","up"]},{"osd":3,"uuid":"ceaa4252-4dfa-427a-9b1a-76adf3b8e609","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6811","nonce":387118141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6813","nonce":387118141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6817","nonce":387118141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6815","nonce":387118141}]},"public_addr":"172.21.15.19:6811/387118141","cluster_addr":"172.21.15.19:6813/387118141","heartbeat_back_addr":"172.21.15.19:6817/387118141","heartbeat_front_addr":"172.21.15.19:6815/387118141","state":["exists","up"]},{"osd":4,"uuid":"5a6d1ebc-62ca-444e-8f29-1980418d0a8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6809","nonce":3662186068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6811","nonce":3662186068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6815","nonce":3662186068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6813","nonce":3662186068}]},"public_addr":"172.21.15.27:6809/3662186068","cluster_addr":"172.21.15.27:6811/3662186068","heartbeat_back_addr":"172.21.15.27:6815/3662186068","heartbeat_front_addr":"172.21.15.27:6813/3662186068","state":["exists","up"]},{"osd":5,"uuid":"60c207e9-55e8-4bed-87c3-f6a06a2a1fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6809","nonce":1264591849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6811","nonce":1264591849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6815","nonce":1264591849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6813","nonce":1264591849}]},"public_addr":"172.21.15.64:6809/1264591849","cluster_addr":"172.21.15.64:6811/1264591849","heartbeat_back_addr":"172.21.15.64:6815/1264591849","heartbeat_front_addr":"172.21.15.64:6813/1264591849","state":["exists","up"]},{"osd":6,"uuid":"f3e038e0-4026-45f1-a923-c07cbb307f64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6819","nonce":2158875543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6821","nonce":2158875543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6825","nonce":2158875543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6823","nonce":2158875543}]},"public_addr":"172.21.15.19:6819/2158875543","cluster_addr":"172.21.15.19:6821/2158875543","heartbeat_back_addr":"172.21.15.19:6825/2158875543","heartbeat_front_addr":"172.21.15.19:6823/2158875543","state":["exists","up"]},{"osd":7,"uuid":"d69e34d4-2028-43b8-8ada-ff4a9b579620","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6817","nonce":2756612091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6819","nonce":2756612091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6823","nonce":2756612091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6821","nonce":2756612091}]},"public_addr":"172.21.15.27:6817/2756612091","cluster_addr":"172.21.15.27:6819/2756612091","heartbeat_back_addr":"172.21.15.27:6823/2756612091","heartbeat_front_addr":"172.21.15.27:6821/2756612091","state":["exists","up"]},{"osd":8,"uuid":"6c1a4179-75b6-42ff-ac29-378801e6f8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6817","nonce":1987024836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6819","nonce":1987024836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6823","nonce":1987024836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6821","nonce":1987024836}]},"public_addr":"172.21.15.64:6817/1987024836","cluster_addr":"172.21.15.64:6819/1987024836","heartbeat_back_addr":"172.21.15.64:6823/1987024836","heartbeat_front_addr":"172.21.15.64:6821/1987024836","state":["exists","up"]},{"osd":9,"uuid":"9df7ff66-d39f-4709-8324-74cf9e0e052d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6827","nonce":4124921646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6829","nonce":4124921646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6833","nonce":4124921646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6831","nonce":4124921646}]},"public_addr":"172.21.15.19:6827/4124921646","cluster_addr":"172.21.15.19:6829/4124921646","heartbeat_back_addr":"172.21.15.19:6833/4124921646","heartbeat_front_addr":"172.21.15.19:6831/4124921646","state":["exists","up"]},{"osd":10,"uuid":"71eaa9a7-667d-4cb7-935c-42177d20a279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6825","nonce":1428931817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6827","nonce":1428931817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6831","nonce":1428931817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6829","nonce":1428931817}]},"public_addr":"172.21.15.27:6825/1428931817","cluster_addr":"172.21.15.27:6827/1428931817","heartbeat_back_addr":"172.21.15.27:6831/1428931817","heartbeat_front_addr":"172.21.15.27:6829/1428931817","state":["exists","up"]},{"osd":11,"uuid":"3330627f-ab5d-4e6e-83cf-fc8b74c54ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6825","nonce":4045070679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6827","nonce":4045070679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6831","nonce":4045070679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6829","nonce":4045070679}]},"public_addr":"172.21.15.64:6825/4045070679","cluster_addr":"172.21.15.64:6827/4045070679","heartbeat_back_addr":"172.21.15.64:6831/4045070679","heartbeat_front_addr":"172.21.15.64:6829/4045070679","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:19:27.902633+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:19:54.305685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:20:18.727053+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:20:53.579199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:21:21.058018+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:21:47.091494+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:22:23.269018+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:22:51.863628+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:23:18.537774+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:23:55.763993+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:24:24.956178+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:24:52.681201+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/1956516258":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/1816772181":"2024-05-08T02:17:31.252868+0000","172.21.15.19:6801/174304059":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/1514051750":"2024-05-08T02:16:54.732592+0000","172.21.15.19:6800/174304059":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/3930045871":"2024-05-08T02:16:54.732592+0000","172.21.15.19:6800/3614957309":"2024-05-08T02:16:54.732592+0000","172.21.15.19:0/1328365399":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/139593344":"2024-05-08T02:16:54.732592+0000","172.21.15.19:6801/3614957309":"2024-05-08T02:16:54.732592+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-07T02:26:41.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.855+0000 7f9eec000700 1 -- 172.21.15.19:0/45001850 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f9ecc0b2880 msgr2=0x7f9ecc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:41.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.855+0000 7f9eec000700 1 --2- 172.21.15.19:0/45001850 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f9ecc0b2880 0x7f9ecc0b4d90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8003e30 tx=0x7f9ed800b040 comp rx=0 tx=0).stop 2024-05-07T02:26:41.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.855+0000 7f9eec000700 1 -- 172.21.15.19:0/45001850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ee410e8f0 msgr2=0x7f9ee41a1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:41.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.855+0000 7f9eec000700 1 --2- 172.21.15.19:0/45001850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ee410e8f0 0x7f9ee41a1790 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9edc0057c0 tx=0x7f9edc0057f0 comp rx=0 tx=0).stop 2024-05-07T02:26:41.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.855+0000 7f9eec000700 1 -- 172.21.15.19:0/45001850 shutdown_connections 2024-05-07T02:26:41.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.855+0000 7f9eec000700 1 --2- 172.21.15.19:0/45001850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ee410e8f0 0x7f9ee41a1790 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:41.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.855+0000 7f9eec000700 1 --2- 172.21.15.19:0/45001850 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9ee41054c0 0x7f9ee419cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:41.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.855+0000 7f9eec000700 1 --2- 172.21.15.19:0/45001850 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f9ecc0b2880 0x7f9ecc0b4d90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:41.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.855+0000 7f9eec000700 1 --2- 172.21.15.19:0/45001850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ee4105e10 0x7f9ee419d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:41.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.855+0000 7f9eec000700 1 -- 172.21.15.19:0/45001850 >> 172.21.15.19:0/45001850 conn(0x7f9ee40fedc0 msgr2=0x7f9ee4103140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:26:41.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.855+0000 7f9eec000700 1 -- 172.21.15.19:0/45001850 shutdown_connections 2024-05-07T02:26:41.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:41.855+0000 7f9eec000700 1 -- 172.21.15.19:0/45001850 wait complete. 2024-05-07T02:26:42.711 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-07T02:26:42.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:42 smithi019 ceph-mon[29781]: pgmap v350: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:42.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:42 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/45001850' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-07T02:26:42.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:42 smithi027 ceph-mon[34398]: pgmap v350: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:42.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:42 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/45001850' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-07T02:26:43.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:42 smithi064 ceph-mon[34699]: pgmap v350: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:43.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:42 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/45001850' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-07T02:26:43.172 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:26:44.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:44 smithi019 ceph-mon[29781]: pgmap v351: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:44.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:44 smithi027 ceph-mon[34398]: pgmap v351: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:45.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:44 smithi064 ceph-mon[34699]: pgmap v351: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:45.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.579+0000 7f342b200700 1 -- 172.21.15.19:0/1259513104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f342c109680 msgr2=0x7f342c110fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:45.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.579+0000 7f342b200700 1 --2- 172.21.15.19:0/1259513104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f342c109680 0x7f342c110fe0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f342400b0d0 tx=0x7f342400b3e0 comp rx=0 tx=0).stop 2024-05-07T02:26:45.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.579+0000 7f342b200700 1 -- 172.21.15.19:0/1259513104 shutdown_connections 2024-05-07T02:26:45.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.579+0000 7f342b200700 1 --2- 172.21.15.19:0/1259513104 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f342c109680 0x7f342c110fe0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:45.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.579+0000 7f342b200700 1 --2- 172.21.15.19:0/1259513104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f342c104400 0x7f342c104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:45.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.579+0000 7f342b200700 1 --2- 172.21.15.19:0/1259513104 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f342c108d30 0x7f342c109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:45.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.579+0000 7f342b200700 1 -- 172.21.15.19:0/1259513104 >> 172.21.15.19:0/1259513104 conn(0x7f342c079d10 msgr2=0x7f342c07a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:26:45.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.580+0000 7f342b200700 1 -- 172.21.15.19:0/1259513104 shutdown_connections 2024-05-07T02:26:45.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.580+0000 7f342b200700 1 -- 172.21.15.19:0/1259513104 wait complete. 2024-05-07T02:26:45.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.581+0000 7f342b200700 1 Processor -- start 2024-05-07T02:26:45.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.582+0000 7f342b200700 1 -- start start 2024-05-07T02:26:45.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.582+0000 7f342b200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f342c104400 0x7f342c19cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:45.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.582+0000 7f342b200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f342c108d30 0x7f342c19d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:45.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.582+0000 7f342b200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f342c109680 0x7f342c1a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:45.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.582+0000 7f342b200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f342c114300 con 0x7f342c104400 2024-05-07T02:26:45.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.582+0000 7f342b200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f342c114180 con 0x7f342c108d30 2024-05-07T02:26:45.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.582+0000 7f342b200700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f342c114480 con 0x7f342c109680 2024-05-07T02:26:45.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.582+0000 7f3429e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f342c104400 0x7f342c19cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:45.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.583+0000 7f3429e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f342c104400 0x7f342c19cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59258/0 (socket says 172.21.15.19:59258) 2024-05-07T02:26:45.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.583+0000 7f3429e00700 1 -- 172.21.15.19:0/527369915 learned_addr learned my addr 172.21.15.19:0/527369915 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:26:45.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.583+0000 7f3429400700 1 --2- 172.21.15.19:0/527369915 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f342c108d30 0x7f342c19d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:45.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.583+0000 7f342a800700 1 --2- 172.21.15.19:0/527369915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f342c109680 0x7f342c1a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:45.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.583+0000 7f3429400700 1 -- 172.21.15.19:0/527369915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f342c109680 msgr2=0x7f342c1a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:45.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.583+0000 7f3429400700 1 --2- 172.21.15.19:0/527369915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f342c109680 0x7f342c1a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:45.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.583+0000 7f3429400700 1 -- 172.21.15.19:0/527369915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f342c104400 msgr2=0x7f342c19cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:45.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.583+0000 7f3429400700 1 --2- 172.21.15.19:0/527369915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f342c104400 0x7f342c19cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:45.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.583+0000 7f3429400700 1 -- 172.21.15.19:0/527369915 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f342c1a1e20 con 0x7f342c108d30 2024-05-07T02:26:45.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.583+0000 7f3429400700 1 --2- 172.21.15.19:0/527369915 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f342c108d30 0x7f342c19d4f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f341800e9c0 tx=0x7f341800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:26:45.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.584+0000 7f3422a00700 1 -- 172.21.15.19:0/527369915 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341800c9c0 con 0x7f342c108d30 2024-05-07T02:26:45.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.584+0000 7f342b200700 1 -- 172.21.15.19:0/527369915 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f342c1a2110 con 0x7f342c108d30 2024-05-07T02:26:45.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.584+0000 7f3422a00700 1 -- 172.21.15.19:0/527369915 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f341800cb60 con 0x7f342c108d30 2024-05-07T02:26:45.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.584+0000 7f3422a00700 1 -- 172.21.15.19:0/527369915 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3418018680 con 0x7f342c108d30 2024-05-07T02:26:45.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.584+0000 7f342b200700 1 -- 172.21.15.19:0/527369915 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f342c1a9a30 con 0x7f342c108d30 2024-05-07T02:26:45.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.585+0000 7f342b200700 1 -- 172.21.15.19:0/527369915 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f342c1058b0 con 0x7f342c108d30 2024-05-07T02:26:45.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.588+0000 7f3422a00700 1 -- 172.21.15.19:0/527369915 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f3418010430 con 0x7f342c108d30 2024-05-07T02:26:45.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.590+0000 7f3422a00700 1 --2- 172.21.15.19:0/527369915 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f34100b2880 0x7f34100b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:45.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.590+0000 7f3422a00700 1 -- 172.21.15.19:0/527369915 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f34180e25b0 con 0x7f342c108d30 2024-05-07T02:26:45.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.590+0000 7f3429e00700 1 --2- 172.21.15.19:0/527369915 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f34100b2880 0x7f34100b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:45.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.591+0000 7f3429e00700 1 --2- 172.21.15.19:0/527369915 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f34100b2880 0x7f34100b4d90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f341c009d20 tx=0x7f341c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:26:45.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.595+0000 7f3422a00700 1 -- 172.21.15.19:0/527369915 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f34180aa030 con 0x7f342c108d30 2024-05-07T02:26:45.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.839+0000 7f342b200700 1 -- 172.21.15.19:0/527369915 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f342c1a9c00 con 0x7f342c108d30 2024-05-07T02:26:45.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.841+0000 7f3422a00700 1 -- 172.21.15.19:0/527369915 <== mon.2 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24055 (secure 0 0 0) 0x7f34180b20b0 con 0x7f342c108d30 2024-05-07T02:26:45.841 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:26:45.842 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":85,"fsid":"9ff84c16-0c17-11ef-bc97-c7b262605968","created":"2024-05-07T02:16:19.507657+0000","modified":"2024-05-07T02:26:32.693063+0000","last_up_change":"2024-05-07T02:24:55.025124+0000","last_in_change":"2024-05-07T02:24:33.842422+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-07T02:20:21.439697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-07T02:26:06.178814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-07T02:26:10.705693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-07T02:26:23.534133+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b437cf22-ec8e-48f0-86a2-2413a0ab40dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6803","nonce":2390539683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6805","nonce":2390539683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6809","nonce":2390539683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6807","nonce":2390539683}]},"public_addr":"172.21.15.19:6803/2390539683","cluster_addr":"172.21.15.19:6805/2390539683","heartbeat_back_addr":"172.21.15.19:6809/2390539683","heartbeat_front_addr":"172.21.15.19:6807/2390539683","state":["exists","up"]},{"osd":1,"uuid":"dc753e16-634d-4c14-abd2-935281e1499e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6801","nonce":4292470615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6803","nonce":4292470615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6807","nonce":4292470615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6805","nonce":4292470615}]},"public_addr":"172.21.15.27:6801/4292470615","cluster_addr":"172.21.15.27:6803/4292470615","heartbeat_back_addr":"172.21.15.27:6807/4292470615","heartbeat_front_addr":"172.21.15.27:6805/4292470615","state":["exists","up"]},{"osd":2,"uuid":"40570071-646e-4e66-9a16-c55ab743ac86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6801","nonce":1180552939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6803","nonce":1180552939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6807","nonce":1180552939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6805","nonce":1180552939}]},"public_addr":"172.21.15.64:6801/1180552939","cluster_addr":"172.21.15.64:6803/1180552939","heartbeat_back_addr":"172.21.15.64:6807/1180552939","heartbeat_front_addr":"172.21.15.64:6805/1180552939","state":["exists","up"]},{"osd":3,"uuid":"ceaa4252-4dfa-427a-9b1a-76adf3b8e609","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6811","nonce":387118141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6813","nonce":387118141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6817","nonce":387118141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6815","nonce":387118141}]},"public_addr":"172.21.15.19:6811/387118141","cluster_addr":"172.21.15.19:6813/387118141","heartbeat_back_addr":"172.21.15.19:6817/387118141","heartbeat_front_addr":"172.21.15.19:6815/387118141","state":["exists","up"]},{"osd":4,"uuid":"5a6d1ebc-62ca-444e-8f29-1980418d0a8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6809","nonce":3662186068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6811","nonce":3662186068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6815","nonce":3662186068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6813","nonce":3662186068}]},"public_addr":"172.21.15.27:6809/3662186068","cluster_addr":"172.21.15.27:6811/3662186068","heartbeat_back_addr":"172.21.15.27:6815/3662186068","heartbeat_front_addr":"172.21.15.27:6813/3662186068","state":["exists","up"]},{"osd":5,"uuid":"60c207e9-55e8-4bed-87c3-f6a06a2a1fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6809","nonce":1264591849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6811","nonce":1264591849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6815","nonce":1264591849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6813","nonce":1264591849}]},"public_addr":"172.21.15.64:6809/1264591849","cluster_addr":"172.21.15.64:6811/1264591849","heartbeat_back_addr":"172.21.15.64:6815/1264591849","heartbeat_front_addr":"172.21.15.64:6813/1264591849","state":["exists","up"]},{"osd":6,"uuid":"f3e038e0-4026-45f1-a923-c07cbb307f64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6819","nonce":2158875543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6821","nonce":2158875543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6825","nonce":2158875543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6823","nonce":2158875543}]},"public_addr":"172.21.15.19:6819/2158875543","cluster_addr":"172.21.15.19:6821/2158875543","heartbeat_back_addr":"172.21.15.19:6825/2158875543","heartbeat_front_addr":"172.21.15.19:6823/2158875543","state":["exists","up"]},{"osd":7,"uuid":"d69e34d4-2028-43b8-8ada-ff4a9b579620","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6817","nonce":2756612091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6819","nonce":2756612091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6823","nonce":2756612091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6821","nonce":2756612091}]},"public_addr":"172.21.15.27:6817/2756612091","cluster_addr":"172.21.15.27:6819/2756612091","heartbeat_back_addr":"172.21.15.27:6823/2756612091","heartbeat_front_addr":"172.21.15.27:6821/2756612091","state":["exists","up"]},{"osd":8,"uuid":"6c1a4179-75b6-42ff-ac29-378801e6f8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6817","nonce":1987024836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6819","nonce":1987024836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6823","nonce":1987024836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6821","nonce":1987024836}]},"public_addr":"172.21.15.64:6817/1987024836","cluster_addr":"172.21.15.64:6819/1987024836","heartbeat_back_addr":"172.21.15.64:6823/1987024836","heartbeat_front_addr":"172.21.15.64:6821/1987024836","state":["exists","up"]},{"osd":9,"uuid":"9df7ff66-d39f-4709-8324-74cf9e0e052d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6827","nonce":4124921646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6829","nonce":4124921646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6833","nonce":4124921646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6831","nonce":4124921646}]},"public_addr":"172.21.15.19:6827/4124921646","cluster_addr":"172.21.15.19:6829/4124921646","heartbeat_back_addr":"172.21.15.19:6833/4124921646","heartbeat_front_addr":"172.21.15.19:6831/4124921646","state":["exists","up"]},{"osd":10,"uuid":"71eaa9a7-667d-4cb7-935c-42177d20a279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6825","nonce":1428931817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6827","nonce":1428931817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6831","nonce":1428931817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6829","nonce":1428931817}]},"public_addr":"172.21.15.27:6825/1428931817","cluster_addr":"172.21.15.27:6827/1428931817","heartbeat_back_addr":"172.21.15.27:6831/1428931817","heartbeat_front_addr":"172.21.15.27:6829/1428931817","state":["exists","up"]},{"osd":11,"uuid":"3330627f-ab5d-4e6e-83cf-fc8b74c54ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6825","nonce":4045070679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6827","nonce":4045070679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6831","nonce":4045070679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6829","nonce":4045070679}]},"public_addr":"172.21.15.64:6825/4045070679","cluster_addr":"172.21.15.64:6827/4045070679","heartbeat_back_addr":"172.21.15.64:6831/4045070679","heartbeat_front_addr":"172.21.15.64:6829/4045070679","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:19:27.902633+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:19:54.305685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:20:18.727053+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:20:53.579199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:21:21.058018+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:21:47.091494+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:22:23.269018+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:22:51.863628+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:23:18.537774+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:23:55.763993+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:24:24.956178+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:24:52.681201+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/1956516258":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/1816772181":"2024-05-08T02:17:31.252868+0000","172.21.15.19:6801/174304059":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/1514051750":"2024-05-08T02:16:54.732592+0000","172.21.15.19:6800/174304059":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/3930045871":"2024-05-08T02:16:54.732592+0000","172.21.15.19:6800/3614957309":"2024-05-08T02:16:54.732592+0000","172.21.15.19:0/1328365399":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/139593344":"2024-05-08T02:16:54.732592+0000","172.21.15.19:6801/3614957309":"2024-05-08T02:16:54.732592+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-07T02:26:45.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.844+0000 7f342b200700 1 -- 172.21.15.19:0/527369915 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f34100b2880 msgr2=0x7f34100b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:45.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.844+0000 7f342b200700 1 --2- 172.21.15.19:0/527369915 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f34100b2880 0x7f34100b4d90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f341c009d20 tx=0x7f341c009260 comp rx=0 tx=0).stop 2024-05-07T02:26:45.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.844+0000 7f342b200700 1 -- 172.21.15.19:0/527369915 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f342c108d30 msgr2=0x7f342c19d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:45.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.844+0000 7f342b200700 1 --2- 172.21.15.19:0/527369915 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f342c108d30 0x7f342c19d4f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f341800e9c0 tx=0x7f341800ecd0 comp rx=0 tx=0).stop 2024-05-07T02:26:45.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.844+0000 7f342b200700 1 -- 172.21.15.19:0/527369915 shutdown_connections 2024-05-07T02:26:45.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.844+0000 7f342b200700 1 --2- 172.21.15.19:0/527369915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f342c109680 0x7f342c1a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:45.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.844+0000 7f342b200700 1 --2- 172.21.15.19:0/527369915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f342c104400 0x7f342c19cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:45.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.844+0000 7f342b200700 1 --2- 172.21.15.19:0/527369915 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f34100b2880 0x7f34100b4d90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:45.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.844+0000 7f342b200700 1 --2- 172.21.15.19:0/527369915 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f342c108d30 0x7f342c19d4f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:45.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.844+0000 7f342b200700 1 -- 172.21.15.19:0/527369915 >> 172.21.15.19:0/527369915 conn(0x7f342c079d10 msgr2=0x7f342c10f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:26:45.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.845+0000 7f342b200700 1 -- 172.21.15.19:0/527369915 shutdown_connections 2024-05-07T02:26:45.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:45.845+0000 7f342b200700 1 -- 172.21.15.19:0/527369915 wait complete. 2024-05-07T02:26:46.779 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-07T02:26:46.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:46 smithi027 ceph-mon[34398]: pgmap v352: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:46.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:46 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/527369915' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-07T02:26:47.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:46 smithi064 ceph-mon[34699]: pgmap v352: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:47.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:46 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/527369915' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-07T02:26:47.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:46 smithi019 ceph-mon[29781]: pgmap v352: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:47.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:46 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/527369915' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-07T02:26:47.238 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:26:48.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:48 smithi027 ceph-mon[34398]: pgmap v353: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:49.104 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:48 smithi019 ceph-mon[29781]: pgmap v353: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:49.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:48 smithi064 ceph-mon[34699]: pgmap v353: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:49.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:49.773+0000 7faaf9600700 1 -- 172.21.15.19:0/2938816913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faaf4106580 msgr2=0x7faaf4106990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:49.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:49.773+0000 7faaf9600700 1 --2- 172.21.15.19:0/2938816913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faaf4106580 0x7faaf4106990 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7faae0009990 tx=0x7faae0009ca0 comp rx=0 tx=0).stop 2024-05-07T02:26:49.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:49.773+0000 7faaf9600700 1 -- 172.21.15.19:0/2938816913 shutdown_connections 2024-05-07T02:26:49.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:49.773+0000 7faaf9600700 1 --2- 172.21.15.19:0/2938816913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faaf4106ed0 0x7faaf4113360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:49.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:49.773+0000 7faaf9600700 1 --2- 172.21.15.19:0/2938816913 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faaf410c5c0 0x7faaf410c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:49.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:49.773+0000 7faaf9600700 1 --2- 172.21.15.19:0/2938816913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faaf4106580 0x7faaf4106990 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:49.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:49.773+0000 7faaf9600700 1 -- 172.21.15.19:0/2938816913 >> 172.21.15.19:0/2938816913 conn(0x7faaf4102080 msgr2=0x7faaf41044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:26:49.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:49.773+0000 7faaf9600700 1 -- 172.21.15.19:0/2938816913 shutdown_connections 2024-05-07T02:26:49.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:49.774+0000 7faaf9600700 1 -- 172.21.15.19:0/2938816913 wait complete. 2024-05-07T02:26:49.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:49.775+0000 7faaf9600700 1 Processor -- start 2024-05-07T02:26:49.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:49.775+0000 7faaf9600700 1 -- start start 2024-05-07T02:26:49.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:49.775+0000 7faaf9600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faaf4106580 0x7faaf419d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:49.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:49.775+0000 7faaf9600700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faaf4106ed0 0x7faaf419d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:49.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:49.775+0000 7faaf9600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faaf410c5c0 0x7faaf41a1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:49.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:49.775+0000 7faaf9600700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faaf41166f0 con 0x7faaf410c5c0 2024-05-07T02:26:49.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:49.775+0000 7faaf9600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7faaf4116570 con 0x7faaf4106580 2024-05-07T02:26:49.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:49.775+0000 7faaf9600700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7faaf4116870 con 0x7faaf4106ed0 2024-05-07T02:26:49.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:49.776+0000 7faaf3400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faaf410c5c0 0x7faaf41a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:49.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:49.776+0000 7faaf2000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faaf4106ed0 0x7faaf419d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:49.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:49.776+0000 7faaf2a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faaf4106580 0x7faaf419d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:49.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:49.776+0000 7faaf3400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faaf410c5c0 0x7faaf41a1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56450/0 (socket says 172.21.15.19:56450) 2024-05-07T02:26:49.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:49.776+0000 7faaf3400700 1 -- 172.21.15.19:0/279904483 learned_addr learned my addr 172.21.15.19:0/279904483 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:26:49.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:49.776+0000 7faaf2000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faaf4106ed0 0x7faaf419d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:59590/0 (socket says 172.21.15.19:59590) 2024-05-07T02:26:49.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:49.776+0000 7faaf2a00700 1 -- 172.21.15.19:0/279904483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faaf4106ed0 msgr2=0x7faaf419d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:49.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:49.776+0000 7faaf2a00700 1 --2- 172.21.15.19:0/279904483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faaf4106ed0 0x7faaf419d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:49.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:49.776+0000 7faaf2a00700 1 -- 172.21.15.19:0/279904483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faaf410c5c0 msgr2=0x7faaf41a1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:49.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:49.776+0000 7faaf2a00700 1 --2- 172.21.15.19:0/279904483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faaf410c5c0 0x7faaf41a1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:49.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:49.776+0000 7faaf2a00700 1 -- 172.21.15.19:0/279904483 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf41a1ff0 con 0x7faaf4106580 2024-05-07T02:26:49.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:49.776+0000 7faaf2000700 1 --2- 172.21.15.19:0/279904483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faaf4106ed0 0x7faaf419d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:26:49.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:49.777+0000 7faaf2a00700 1 --2- 172.21.15.19:0/279904483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faaf4106580 0x7faaf419d120 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faae400e9c0 tx=0x7faae400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:26:49.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:49.777+0000 7faaeb400700 1 -- 172.21.15.19:0/279904483 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae400c9c0 con 0x7faaf4106580 2024-05-07T02:26:49.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:49.777+0000 7faaeb400700 1 -- 172.21.15.19:0/279904483 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faae400cb60 con 0x7faaf4106580 2024-05-07T02:26:49.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:49.777+0000 7faaf9600700 1 -- 172.21.15.19:0/279904483 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaf41a22e0 con 0x7faaf4106580 2024-05-07T02:26:49.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:49.777+0000 7faaeb400700 1 -- 172.21.15.19:0/279904483 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae4018680 con 0x7faaf4106580 2024-05-07T02:26:49.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:49.777+0000 7faaf9600700 1 -- 172.21.15.19:0/279904483 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaf41a9c00 con 0x7faaf4106580 2024-05-07T02:26:49.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:49.778+0000 7faaf9600700 1 -- 172.21.15.19:0/279904483 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaf410e980 con 0x7faaf4106580 2024-05-07T02:26:49.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:49.781+0000 7faaeb400700 1 -- 172.21.15.19:0/279904483 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7faae4010430 con 0x7faaf4106580 2024-05-07T02:26:49.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:49.782+0000 7faaeb400700 1 --2- 172.21.15.19:0/279904483 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7faad80b2880 0x7faad80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:49.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:49.782+0000 7faaeb400700 1 -- 172.21.15.19:0/279904483 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7faae40e25b0 con 0x7faaf4106580 2024-05-07T02:26:49.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:49.782+0000 7faaf2000700 1 --2- 172.21.15.19:0/279904483 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7faad80b2880 0x7faad80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:49.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:49.783+0000 7faaf2000700 1 --2- 172.21.15.19:0/279904483 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7faad80b2880 0x7faad80b4d90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7faae0000c00 tx=0x7faae000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:26:49.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:49.787+0000 7faaeb400700 1 -- 172.21.15.19:0/279904483 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faae40aa030 con 0x7faaf4106580 2024-05-07T02:26:50.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:50.075+0000 7faaf9600700 1 -- 172.21.15.19:0/279904483 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faaf40512c0 con 0x7faaf4106580 2024-05-07T02:26:50.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:50.076+0000 7faaeb400700 1 -- 172.21.15.19:0/279904483 <== mon.2 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11788 (secure 0 0 0) 0x7faae40b20b0 con 0x7faaf4106580 2024-05-07T02:26:50.077 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:26:50.077 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1763670446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1763670446},{"type":"v1","addr":"172.21.15.19:6839","nonce":1763670446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/477529985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":477529985},{"type":"v1","addr":"172.21.15.64:6833","nonce":477529985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/2171873162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2171873162},{"type":"v1","addr":"172.21.15.27:6833","nonce":2171873162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24469,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2379917839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2379917839},{"type":"v1","addr":"172.21.15.19:6835","nonce":2379917839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2067515538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2067515538},{"type":"v1","addr":"172.21.15.19:6837","nonce":2067515538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1000540600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1000540600},{"type":"v1","addr":"172.21.15.27:6835","nonce":1000540600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24499,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/1997318872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1997318872},{"type":"v1","addr":"172.21.15.64:6835","nonce":1997318872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2585907777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2585907777},{"type":"v1","addr":"172.21.15.64:6837","nonce":2585907777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24515,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6837/2583138143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2583138143},{"type":"v1","addr":"172.21.15.27:6837","nonce":2583138143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/1583120004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":1583120004},{"type":"v1","addr":"172.21.15.64:6839","nonce":1583120004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24539,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/3905645246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3905645246},{"type":"v1","addr":"172.21.15.19:6841","nonce":3905645246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-07T02:26:14.960465+0000","modified":"2024-05-07T02:26:32.693402+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24551},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6839/3963268634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3963268634},{"type":"v1","addr":"172.21.15.27:6839","nonce":3963268634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24551,"qdb_cluster":[24551]},"id":1}]} 2024-05-07T02:26:50.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:50.081+0000 7faaf9600700 1 -- 172.21.15.19:0/279904483 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7faad80b2880 msgr2=0x7faad80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:50.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:50.081+0000 7faaf9600700 1 --2- 172.21.15.19:0/279904483 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7faad80b2880 0x7faad80b4d90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7faae0000c00 tx=0x7faae000b260 comp rx=0 tx=0).stop 2024-05-07T02:26:50.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:50.081+0000 7faaf9600700 1 -- 172.21.15.19:0/279904483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faaf4106580 msgr2=0x7faaf419d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:50.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:50.081+0000 7faaf9600700 1 --2- 172.21.15.19:0/279904483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faaf4106580 0x7faaf419d120 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faae400e9c0 tx=0x7faae400ecd0 comp rx=0 tx=0).stop 2024-05-07T02:26:50.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:50.081+0000 7faaf9600700 1 -- 172.21.15.19:0/279904483 shutdown_connections 2024-05-07T02:26:50.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:50.081+0000 7faaf9600700 1 --2- 172.21.15.19:0/279904483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faaf410c5c0 0x7faaf41a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:50.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:50.081+0000 7faaf9600700 1 --2- 172.21.15.19:0/279904483 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7faad80b2880 0x7faad80b4d90 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:50.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:50.081+0000 7faaf9600700 1 --2- 172.21.15.19:0/279904483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faaf4106580 0x7faaf419d120 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:50.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:50.081+0000 7faaf9600700 1 --2- 172.21.15.19:0/279904483 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faaf4106ed0 0x7faaf419d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:50.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:50.081+0000 7faaf9600700 1 -- 172.21.15.19:0/279904483 >> 172.21.15.19:0/279904483 conn(0x7faaf4102080 msgr2=0x7faaf41048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:26:50.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:50.082+0000 7faaf9600700 1 -- 172.21.15.19:0/279904483 shutdown_connections 2024-05-07T02:26:50.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:50.082+0000 7faaf9600700 1 -- 172.21.15.19:0/279904483 wait complete. 2024-05-07T02:26:50.083 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 17 2024-05-07T02:26:50.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:50 smithi019 ceph-mon[29781]: pgmap v354: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:50.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:50 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/279904483' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:26:50.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:50 smithi027 ceph-mon[34398]: pgmap v354: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:50.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:50 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/279904483' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:26:51.093 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-07T02:26:51.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:50 smithi064 ceph-mon[34699]: pgmap v354: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:51.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:50 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/279904483' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:26:51.561 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:26:52.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:52 smithi027 ceph-mon[34398]: pgmap v355: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:53.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:52 smithi064 ceph-mon[34699]: pgmap v355: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:53.110 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:52 smithi019 ceph-mon[29781]: pgmap v355: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:54.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.061+0000 7f7920e00700 1 -- 172.21.15.19:0/2059495647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f791c075b60 msgr2=0x7f791c075f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:54.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.061+0000 7f7920e00700 1 --2- 172.21.15.19:0/2059495647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f791c075b60 0x7f791c075f70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f790c009990 tx=0x7f790c009ca0 comp rx=0 tx=0).stop 2024-05-07T02:26:54.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.061+0000 7f7920e00700 1 -- 172.21.15.19:0/2059495647 shutdown_connections 2024-05-07T02:26:54.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.061+0000 7f7920e00700 1 --2- 172.21.15.19:0/2059495647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f791c115180 0x7f791c1175a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:54.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.061+0000 7f7920e00700 1 --2- 172.21.15.19:0/2059495647 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f791c075b60 0x7f791c075f70 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:54.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.061+0000 7f7920e00700 1 --2- 172.21.15.19:0/2059495647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f791c0764b0 0x7f791c114c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:54.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.061+0000 7f7920e00700 1 -- 172.21.15.19:0/2059495647 >> 172.21.15.19:0/2059495647 conn(0x7f791c079d10 msgr2=0x7f791c07a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:26:54.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.061+0000 7f7920e00700 1 -- 172.21.15.19:0/2059495647 shutdown_connections 2024-05-07T02:26:54.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.061+0000 7f7920e00700 1 -- 172.21.15.19:0/2059495647 wait complete. 2024-05-07T02:26:54.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.063+0000 7f7920e00700 1 Processor -- start 2024-05-07T02:26:54.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.063+0000 7f7920e00700 1 -- start start 2024-05-07T02:26:54.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.064+0000 7f7920e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f791c075b60 0x7f791c1a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:54.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.064+0000 7f7920e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f791c0764b0 0x7f791c1a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:54.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.064+0000 7f7920e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f791c115180 0x7f791c1a5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:54.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.064+0000 7f7920e00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f791c11a8e0 con 0x7f791c075b60 2024-05-07T02:26:54.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.064+0000 7f7920e00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f791c11a760 con 0x7f791c0764b0 2024-05-07T02:26:54.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.064+0000 7f7920e00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f791c11aa60 con 0x7f791c115180 2024-05-07T02:26:54.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.064+0000 7f791aa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f791c0764b0 0x7f791c1a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:54.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.064+0000 7f791b400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f791c075b60 0x7f791c1a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:54.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.064+0000 7f791aa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f791c0764b0 0x7f791c1a17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:42418/0 (socket says 172.21.15.19:42418) 2024-05-07T02:26:54.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.064+0000 7f791aa00700 1 -- 172.21.15.19:0/1334588425 learned_addr learned my addr 172.21.15.19:0/1334588425 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:26:54.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.064+0000 7f791be00700 1 --2- 172.21.15.19:0/1334588425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f791c115180 0x7f791c1a5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:54.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.065+0000 7f791aa00700 1 -- 172.21.15.19:0/1334588425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f791c115180 msgr2=0x7f791c1a5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:54.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.065+0000 7f791aa00700 1 --2- 172.21.15.19:0/1334588425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f791c115180 0x7f791c1a5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:54.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.065+0000 7f791aa00700 1 -- 172.21.15.19:0/1334588425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f791c075b60 msgr2=0x7f791c1a1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:54.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.065+0000 7f791aa00700 1 --2- 172.21.15.19:0/1334588425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f791c075b60 0x7f791c1a1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:54.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.065+0000 7f791aa00700 1 -- 172.21.15.19:0/1334588425 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f791c1a6100 con 0x7f791c0764b0 2024-05-07T02:26:54.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.065+0000 7f791b400700 1 --2- 172.21.15.19:0/1334588425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f791c075b60 0x7f791c1a1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:26:54.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.065+0000 7f791aa00700 1 --2- 172.21.15.19:0/1334588425 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f791c0764b0 0x7f791c1a17d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f790800e9a0 tx=0x7f790800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:26:54.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.065+0000 7f7913e00700 1 -- 172.21.15.19:0/1334588425 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790800ca10 con 0x7f791c0764b0 2024-05-07T02:26:54.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.065+0000 7f7913e00700 1 -- 172.21.15.19:0/1334588425 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f790800cbb0 con 0x7f791c0764b0 2024-05-07T02:26:54.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.065+0000 7f7920e00700 1 -- 172.21.15.19:0/1334588425 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f791c1a63f0 con 0x7f791c0764b0 2024-05-07T02:26:54.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.065+0000 7f7913e00700 1 -- 172.21.15.19:0/1334588425 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7908018680 con 0x7f791c0764b0 2024-05-07T02:26:54.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.066+0000 7f7920e00700 1 -- 172.21.15.19:0/1334588425 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f791c1add10 con 0x7f791c0764b0 2024-05-07T02:26:54.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.067+0000 7f7920e00700 1 -- 172.21.15.19:0/1334588425 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f791c0512c0 con 0x7f791c0764b0 2024-05-07T02:26:54.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.069+0000 7f7913e00700 1 -- 172.21.15.19:0/1334588425 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7908049a20 con 0x7f791c0764b0 2024-05-07T02:26:54.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.071+0000 7f7913e00700 1 --2- 172.21.15.19:0/1334588425 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f79000b2900 0x7f79000b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:54.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.071+0000 7f7913e00700 1 -- 172.21.15.19:0/1334588425 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f79080e2680 con 0x7f791c0764b0 2024-05-07T02:26:54.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.071+0000 7f791b400700 1 --2- 172.21.15.19:0/1334588425 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f79000b2900 0x7f79000b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:54.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.072+0000 7f791b400700 1 --2- 172.21.15.19:0/1334588425 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f79000b2900 0x7f79000b4e10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f790c009750 tx=0x7f790c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:26:54.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.073+0000 7f7913e00700 1 -- 172.21.15.19:0/1334588425 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f79080aa100 con 0x7f791c0764b0 2024-05-07T02:26:54.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.314+0000 7f7920e00700 1 -- 172.21.15.19:0/1334588425 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f791c1adee0 con 0x7f791c0764b0 2024-05-07T02:26:54.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.316+0000 7f7913e00700 1 -- 172.21.15.19:0/1334588425 <== mon.2 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24055 (secure 0 0 0) 0x7f79080b2180 con 0x7f791c0764b0 2024-05-07T02:26:54.316 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:26:54.317 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":85,"fsid":"9ff84c16-0c17-11ef-bc97-c7b262605968","created":"2024-05-07T02:16:19.507657+0000","modified":"2024-05-07T02:26:32.693063+0000","last_up_change":"2024-05-07T02:24:55.025124+0000","last_in_change":"2024-05-07T02:24:33.842422+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-07T02:20:21.439697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-07T02:26:06.178814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-07T02:26:10.705693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-07T02:26:23.534133+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b437cf22-ec8e-48f0-86a2-2413a0ab40dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6803","nonce":2390539683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6805","nonce":2390539683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6809","nonce":2390539683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6807","nonce":2390539683}]},"public_addr":"172.21.15.19:6803/2390539683","cluster_addr":"172.21.15.19:6805/2390539683","heartbeat_back_addr":"172.21.15.19:6809/2390539683","heartbeat_front_addr":"172.21.15.19:6807/2390539683","state":["exists","up"]},{"osd":1,"uuid":"dc753e16-634d-4c14-abd2-935281e1499e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6801","nonce":4292470615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6803","nonce":4292470615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6807","nonce":4292470615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6805","nonce":4292470615}]},"public_addr":"172.21.15.27:6801/4292470615","cluster_addr":"172.21.15.27:6803/4292470615","heartbeat_back_addr":"172.21.15.27:6807/4292470615","heartbeat_front_addr":"172.21.15.27:6805/4292470615","state":["exists","up"]},{"osd":2,"uuid":"40570071-646e-4e66-9a16-c55ab743ac86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6801","nonce":1180552939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6803","nonce":1180552939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6807","nonce":1180552939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6805","nonce":1180552939}]},"public_addr":"172.21.15.64:6801/1180552939","cluster_addr":"172.21.15.64:6803/1180552939","heartbeat_back_addr":"172.21.15.64:6807/1180552939","heartbeat_front_addr":"172.21.15.64:6805/1180552939","state":["exists","up"]},{"osd":3,"uuid":"ceaa4252-4dfa-427a-9b1a-76adf3b8e609","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6811","nonce":387118141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6813","nonce":387118141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6817","nonce":387118141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6815","nonce":387118141}]},"public_addr":"172.21.15.19:6811/387118141","cluster_addr":"172.21.15.19:6813/387118141","heartbeat_back_addr":"172.21.15.19:6817/387118141","heartbeat_front_addr":"172.21.15.19:6815/387118141","state":["exists","up"]},{"osd":4,"uuid":"5a6d1ebc-62ca-444e-8f29-1980418d0a8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6809","nonce":3662186068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6811","nonce":3662186068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6815","nonce":3662186068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6813","nonce":3662186068}]},"public_addr":"172.21.15.27:6809/3662186068","cluster_addr":"172.21.15.27:6811/3662186068","heartbeat_back_addr":"172.21.15.27:6815/3662186068","heartbeat_front_addr":"172.21.15.27:6813/3662186068","state":["exists","up"]},{"osd":5,"uuid":"60c207e9-55e8-4bed-87c3-f6a06a2a1fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6809","nonce":1264591849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6811","nonce":1264591849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6815","nonce":1264591849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6813","nonce":1264591849}]},"public_addr":"172.21.15.64:6809/1264591849","cluster_addr":"172.21.15.64:6811/1264591849","heartbeat_back_addr":"172.21.15.64:6815/1264591849","heartbeat_front_addr":"172.21.15.64:6813/1264591849","state":["exists","up"]},{"osd":6,"uuid":"f3e038e0-4026-45f1-a923-c07cbb307f64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6819","nonce":2158875543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6821","nonce":2158875543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6825","nonce":2158875543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6823","nonce":2158875543}]},"public_addr":"172.21.15.19:6819/2158875543","cluster_addr":"172.21.15.19:6821/2158875543","heartbeat_back_addr":"172.21.15.19:6825/2158875543","heartbeat_front_addr":"172.21.15.19:6823/2158875543","state":["exists","up"]},{"osd":7,"uuid":"d69e34d4-2028-43b8-8ada-ff4a9b579620","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6817","nonce":2756612091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6819","nonce":2756612091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6823","nonce":2756612091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6821","nonce":2756612091}]},"public_addr":"172.21.15.27:6817/2756612091","cluster_addr":"172.21.15.27:6819/2756612091","heartbeat_back_addr":"172.21.15.27:6823/2756612091","heartbeat_front_addr":"172.21.15.27:6821/2756612091","state":["exists","up"]},{"osd":8,"uuid":"6c1a4179-75b6-42ff-ac29-378801e6f8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6817","nonce":1987024836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6819","nonce":1987024836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6823","nonce":1987024836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6821","nonce":1987024836}]},"public_addr":"172.21.15.64:6817/1987024836","cluster_addr":"172.21.15.64:6819/1987024836","heartbeat_back_addr":"172.21.15.64:6823/1987024836","heartbeat_front_addr":"172.21.15.64:6821/1987024836","state":["exists","up"]},{"osd":9,"uuid":"9df7ff66-d39f-4709-8324-74cf9e0e052d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6827","nonce":4124921646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6829","nonce":4124921646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6833","nonce":4124921646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6831","nonce":4124921646}]},"public_addr":"172.21.15.19:6827/4124921646","cluster_addr":"172.21.15.19:6829/4124921646","heartbeat_back_addr":"172.21.15.19:6833/4124921646","heartbeat_front_addr":"172.21.15.19:6831/4124921646","state":["exists","up"]},{"osd":10,"uuid":"71eaa9a7-667d-4cb7-935c-42177d20a279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6825","nonce":1428931817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6827","nonce":1428931817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6831","nonce":1428931817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6829","nonce":1428931817}]},"public_addr":"172.21.15.27:6825/1428931817","cluster_addr":"172.21.15.27:6827/1428931817","heartbeat_back_addr":"172.21.15.27:6831/1428931817","heartbeat_front_addr":"172.21.15.27:6829/1428931817","state":["exists","up"]},{"osd":11,"uuid":"3330627f-ab5d-4e6e-83cf-fc8b74c54ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6825","nonce":4045070679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6827","nonce":4045070679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6831","nonce":4045070679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6829","nonce":4045070679}]},"public_addr":"172.21.15.64:6825/4045070679","cluster_addr":"172.21.15.64:6827/4045070679","heartbeat_back_addr":"172.21.15.64:6831/4045070679","heartbeat_front_addr":"172.21.15.64:6829/4045070679","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:19:27.902633+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:19:54.305685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:20:18.727053+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:20:53.579199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:21:21.058018+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:21:47.091494+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:22:23.269018+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:22:51.863628+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:23:18.537774+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:23:55.763993+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:24:24.956178+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:24:52.681201+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/1956516258":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/1816772181":"2024-05-08T02:17:31.252868+0000","172.21.15.19:6801/174304059":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/1514051750":"2024-05-08T02:16:54.732592+0000","172.21.15.19:6800/174304059":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/3930045871":"2024-05-08T02:16:54.732592+0000","172.21.15.19:6800/3614957309":"2024-05-08T02:16:54.732592+0000","172.21.15.19:0/1328365399":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/139593344":"2024-05-08T02:16:54.732592+0000","172.21.15.19:6801/3614957309":"2024-05-08T02:16:54.732592+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-07T02:26:54.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.319+0000 7f7920e00700 1 -- 172.21.15.19:0/1334588425 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f79000b2900 msgr2=0x7f79000b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:54.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.319+0000 7f7920e00700 1 --2- 172.21.15.19:0/1334588425 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f79000b2900 0x7f79000b4e10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f790c009750 tx=0x7f790c00b260 comp rx=0 tx=0).stop 2024-05-07T02:26:54.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.319+0000 7f7920e00700 1 -- 172.21.15.19:0/1334588425 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f791c0764b0 msgr2=0x7f791c1a17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:54.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.319+0000 7f7920e00700 1 --2- 172.21.15.19:0/1334588425 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f791c0764b0 0x7f791c1a17d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f790800e9a0 tx=0x7f790800ecb0 comp rx=0 tx=0).stop 2024-05-07T02:26:54.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.320+0000 7f7920e00700 1 -- 172.21.15.19:0/1334588425 shutdown_connections 2024-05-07T02:26:54.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.320+0000 7f7920e00700 1 --2- 172.21.15.19:0/1334588425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f791c115180 0x7f791c1a5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:54.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.320+0000 7f7920e00700 1 --2- 172.21.15.19:0/1334588425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f791c075b60 0x7f791c1a1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:54.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.320+0000 7f7920e00700 1 --2- 172.21.15.19:0/1334588425 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f79000b2900 0x7f79000b4e10 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:54.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.320+0000 7f7920e00700 1 --2- 172.21.15.19:0/1334588425 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f791c0764b0 0x7f791c1a17d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:54.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.320+0000 7f7920e00700 1 -- 172.21.15.19:0/1334588425 >> 172.21.15.19:0/1334588425 conn(0x7f791c079d10 msgr2=0x7f791c100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:26:54.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.320+0000 7f7920e00700 1 -- 172.21.15.19:0/1334588425 shutdown_connections 2024-05-07T02:26:54.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:54.321+0000 7f7920e00700 1 -- 172.21.15.19:0/1334588425 wait complete. 2024-05-07T02:26:54.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:54 smithi027 ceph-mon[34398]: pgmap v356: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:54.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:54 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/1334588425' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-07T02:26:55.086 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:54 smithi019 ceph-mon[29781]: pgmap v356: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:55.087 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:54 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/1334588425' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-07T02:26:55.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:54 smithi064 ceph-mon[34699]: pgmap v356: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:55.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:54 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/1334588425' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-07T02:26:55.157 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-07T02:26:14.960465+0000', 'modified': '2024-05-07T02:26:32.693402+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24551}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24551': {'gid': 24551, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.27:6839/3963268634', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6838', 'nonce': 3963268634}, {'type': 'v1', 'addr': '172.21.15.27:6839', 'nonce': 3963268634}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24551, 'qdb_cluster': [24551]} 2024-05-07T02:26:55.157 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-07T02:26:55.157 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.b status 2024-05-07T02:26:55.608 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:26:57.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:56 smithi064 ceph-mon[34699]: pgmap v357: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:57.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:56 smithi019 ceph-mon[29781]: pgmap v357: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:57.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:56 smithi027 ceph-mon[34398]: pgmap v357: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:58.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.226+0000 7f54df000700 1 -- 172.21.15.19:0/3185617147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54d810b110 msgr2=0x7f54d8117640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:58.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.226+0000 7f54df000700 1 --2- 172.21.15.19:0/3185617147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54d810b110 0x7f54d8117640 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f54d000b0a0 tx=0x7f54d000b3b0 comp rx=0 tx=0).stop 2024-05-07T02:26:58.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.227+0000 7f54df000700 1 -- 172.21.15.19:0/3185617147 shutdown_connections 2024-05-07T02:26:58.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.227+0000 7f54df000700 1 --2- 172.21.15.19:0/3185617147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54d810b110 0x7f54d8117640 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:58.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.227+0000 7f54df000700 1 --2- 172.21.15.19:0/3185617147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54d81107a0 0x7f54d8110bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:58.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.227+0000 7f54df000700 1 --2- 172.21.15.19:0/3185617147 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54d810a7c0 0x7f54d810abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:58.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.227+0000 7f54df000700 1 -- 172.21.15.19:0/3185617147 >> 172.21.15.19:0/3185617147 conn(0x7f54d8076b00 msgr2=0x7f54d8076f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:26:58.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.227+0000 7f54df000700 1 -- 172.21.15.19:0/3185617147 shutdown_connections 2024-05-07T02:26:58.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.227+0000 7f54df000700 1 -- 172.21.15.19:0/3185617147 wait complete. 2024-05-07T02:26:58.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.228+0000 7f54df000700 1 Processor -- start 2024-05-07T02:26:58.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.229+0000 7f54df000700 1 -- start start 2024-05-07T02:26:58.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.229+0000 7f54df000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54d810a7c0 0x7f54d81a1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:58.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.229+0000 7f54df000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54d810b110 0x7f54d81a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:58.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.229+0000 7f54df000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54d81107a0 0x7f54d81a5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:58.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.229+0000 7f54df000700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f54d811a8b0 con 0x7f54d810a7c0 2024-05-07T02:26:58.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.229+0000 7f54df000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f54d811a730 con 0x7f54d810b110 2024-05-07T02:26:58.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.229+0000 7f54df000700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f54d811aa30 con 0x7f54d81107a0 2024-05-07T02:26:58.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.230+0000 7f54ddc00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54d810a7c0 0x7f54d81a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:58.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.230+0000 7f54ddc00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54d810a7c0 0x7f54d81a1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56490/0 (socket says 172.21.15.19:56490) 2024-05-07T02:26:58.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.230+0000 7f54ddc00700 1 -- 172.21.15.19:0/2146595803 learned_addr learned my addr 172.21.15.19:0/2146595803 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:26:58.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.230+0000 7f54dd200700 1 --2- 172.21.15.19:0/2146595803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54d810b110 0x7f54d81a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:58.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.230+0000 7f54de600700 1 --2- 172.21.15.19:0/2146595803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54d81107a0 0x7f54d81a5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:58.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.230+0000 7f54ddc00700 1 -- 172.21.15.19:0/2146595803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54d81107a0 msgr2=0x7f54d81a5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:58.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.230+0000 7f54ddc00700 1 --2- 172.21.15.19:0/2146595803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54d81107a0 0x7f54d81a5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:58.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.230+0000 7f54ddc00700 1 -- 172.21.15.19:0/2146595803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54d810b110 msgr2=0x7f54d81a17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:58.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.230+0000 7f54ddc00700 1 --2- 172.21.15.19:0/2146595803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54d810b110 0x7f54d81a17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:58.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.230+0000 7f54ddc00700 1 -- 172.21.15.19:0/2146595803 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d81a60f0 con 0x7f54d810a7c0 2024-05-07T02:26:58.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.230+0000 7f54dd200700 1 --2- 172.21.15.19:0/2146595803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54d810b110 0x7f54d81a17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:26:58.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.231+0000 7f54ddc00700 1 --2- 172.21.15.19:0/2146595803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54d810a7c0 0x7f54d81a1220 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f54cc00e9a0 tx=0x7f54cc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:26:58.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.231+0000 7f54d6000700 1 -- 172.21.15.19:0/2146595803 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc00ca10 con 0x7f54d810a7c0 2024-05-07T02:26:58.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.231+0000 7f54d6000700 1 -- 172.21.15.19:0/2146595803 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54cc00cbb0 con 0x7f54d810a7c0 2024-05-07T02:26:58.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.231+0000 7f54df000700 1 -- 172.21.15.19:0/2146595803 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54d81a63e0 con 0x7f54d810a7c0 2024-05-07T02:26:58.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.231+0000 7f54d6000700 1 -- 172.21.15.19:0/2146595803 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc018680 con 0x7f54d810a7c0 2024-05-07T02:26:58.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.231+0000 7f54df000700 1 -- 172.21.15.19:0/2146595803 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54d81add00 con 0x7f54d810a7c0 2024-05-07T02:26:58.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.235+0000 7f54d6000700 1 -- 172.21.15.19:0/2146595803 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f54cc010430 con 0x7f54d810a7c0 2024-05-07T02:26:58.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.236+0000 7f54ef0edb80 1 Processor -- start 2024-05-07T02:26:58.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.237+0000 7f54ef0edb80 1 -- start start 2024-05-07T02:26:58.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.237+0000 7f54ef0edb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf5ef3a40 0x55faf5ef3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:58.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.237+0000 7f54ef0edb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55faf5ef4390 0x55faf5efd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:58.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.237+0000 7f54ef0edb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faf5efdb10 0x55faf5efff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:58.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.237+0000 7f54ef0edb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55faf5e159d0 con 0x55faf5efdb10 2024-05-07T02:26:58.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.237+0000 7f54ef0edb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55faf5db3710 con 0x55faf5ef4390 2024-05-07T02:26:58.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.237+0000 7f54ef0edb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55faf5e11990 con 0x55faf5ef3a40 2024-05-07T02:26:58.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.237+0000 7f54de600700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf5ef3a40 0x55faf5ef3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:58.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.237+0000 7f54ddc00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55faf5ef4390 0x55faf5efd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:58.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.238+0000 7f54de600700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf5ef3a40 0x55faf5ef3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:59638/0 (socket says 172.21.15.19:59638) 2024-05-07T02:26:58.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.238+0000 7f54dd200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faf5efdb10 0x55faf5efff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:58.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.238+0000 7f54de600700 1 -- 172.21.15.19:0/2427041600 learned_addr learned my addr 172.21.15.19:0/2427041600 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:26:58.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.238+0000 7f54d6000700 1 --2- 172.21.15.19:0/2146595803 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f54c00b2980 0x7f54c00b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:58.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.238+0000 7f54d6000700 1 -- 172.21.15.19:0/2146595803 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f54cc0e2530 con 0x7f54d810a7c0 2024-05-07T02:26:58.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.238+0000 7f54de600700 1 -- 172.21.15.19:0/2427041600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55faf5ef4390 msgr2=0x55faf5efd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:26:58.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.238+0000 7f54de600700 1 --2- 172.21.15.19:0/2427041600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55faf5ef4390 0x55faf5efd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:58.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.238+0000 7f54de600700 1 -- 172.21.15.19:0/2427041600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faf5efdb10 msgr2=0x55faf5efff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:26:58.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.238+0000 7f54de600700 1 --2- 172.21.15.19:0/2427041600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faf5efdb10 0x55faf5efff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:58.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.238+0000 7f54de600700 1 -- 172.21.15.19:0/2427041600 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faf5d62880 con 0x55faf5ef3a40 2024-05-07T02:26:58.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.238+0000 7f54dd200700 1 --2- 172.21.15.19:0/2427041600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faf5efdb10 0x55faf5efff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:26:58.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.238+0000 7f54de600700 1 --2- 172.21.15.19:0/2146595803 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f54c00b2980 0x7f54c00b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:58.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.239+0000 7f54de600700 1 --2- 172.21.15.19:0/2427041600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf5ef3a40 0x55faf5ef3e50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f54d0015000 tx=0x7f54d00092e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=857063a8957e5ca0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:26:58.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.239+0000 7f54caa00700 1 -- 172.21.15.19:0/2427041600 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d002e070 con 0x55faf5ef3a40 2024-05-07T02:26:58.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.239+0000 7f54de600700 1 --2- 172.21.15.19:0/2146595803 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f54c00b2980 0x7f54c00b4e90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f54d002b000 tx=0x7f54d002c000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:26:58.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.239+0000 7f54caa00700 1 -- 172.21.15.19:0/2427041600 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54d0024070 con 0x55faf5ef3a40 2024-05-07T02:26:58.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.239+0000 7f54caa00700 1 -- 172.21.15.19:0/2427041600 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d00052a0 con 0x55faf5ef3a40 2024-05-07T02:26:58.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.240+0000 7f54ef0edb80 1 -- 172.21.15.19:0/2427041600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf5ef3a40 msgr2=0x55faf5ef3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:58.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.240+0000 7f54ef0edb80 1 --2- 172.21.15.19:0/2427041600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf5ef3a40 0x55faf5ef3e50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f54d0015000 tx=0x7f54d00092e0 comp rx=0 tx=0).stop 2024-05-07T02:26:58.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.241+0000 7f54ef0edb80 1 -- 172.21.15.19:0/2427041600 shutdown_connections 2024-05-07T02:26:58.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.241+0000 7f54ef0edb80 1 --2- 172.21.15.19:0/2427041600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faf5efdb10 0x55faf5efff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:58.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.241+0000 7f54ef0edb80 1 --2- 172.21.15.19:0/2427041600 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf5ef3a40 0x55faf5ef3e50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:58.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.241+0000 7f54ef0edb80 1 --2- 172.21.15.19:0/2427041600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55faf5ef4390 0x55faf5efd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:58.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.241+0000 7f54ef0edb80 1 -- 172.21.15.19:0/2427041600 >> 172.21.15.19:0/2427041600 conn(0x55faf5d75df0 msgr2=0x55faf5eee830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:26:58.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.241+0000 7f54ef0edb80 1 -- 172.21.15.19:0/2427041600 shutdown_connections 2024-05-07T02:26:58.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.241+0000 7f54ef0edb80 1 -- 172.21.15.19:0/2427041600 wait complete. 2024-05-07T02:26:58.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.241+0000 7f54ef0edb80 1 -- start start 2024-05-07T02:26:58.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.242+0000 7f54ef0edb80 1 Processor -- start 2024-05-07T02:26:58.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.242+0000 7f54ef0edb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faf5ef3a40 0x55faf5efaa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:58.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.242+0000 7f54ef0edb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55faf5ef4390 0x55faf5efafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:58.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.242+0000 7f54ef0edb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf5efdb10 0x55faf5efb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:58.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.242+0000 7f54ef0edb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55faf5f008b0 con 0x55faf5ef3a40 2024-05-07T02:26:58.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.242+0000 7f54ef0edb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55faf5d7bad0 con 0x55faf5ef4390 2024-05-07T02:26:58.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.242+0000 7f54ef0edb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55faf5f00a30 con 0x55faf5efdb10 2024-05-07T02:26:58.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.242+0000 7f54ef0edb80 10 client.0 _ll_register_callbacks cb 0x55faf5dbcc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:26:58.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.243+0000 7f54dd200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55faf5ef4390 0x55faf5efafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:58.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.243+0000 7f54dd200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55faf5ef4390 0x55faf5efafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:42448/0 (socket says 172.21.15.19:42448) 2024-05-07T02:26:58.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.243+0000 7f54dd200700 1 -- 172.21.15.19:0/1893936677 learned_addr learned my addr 172.21.15.19:0/1893936677 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:26:58.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.243+0000 7f54de600700 1 --2- 172.21.15.19:0/1893936677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf5efdb10 0x55faf5efb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:58.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.243+0000 7f54ddc00700 1 --2- 172.21.15.19:0/1893936677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faf5ef3a40 0x55faf5efaa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:58.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.243+0000 7f54dd200700 1 -- 172.21.15.19:0/1893936677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf5efdb10 msgr2=0x55faf5efb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:26:58.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.243+0000 7f54dd200700 1 --2- 172.21.15.19:0/1893936677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf5efdb10 0x55faf5efb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:58.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.243+0000 7f54dd200700 1 -- 172.21.15.19:0/1893936677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faf5ef3a40 msgr2=0x55faf5efaa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:26:58.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.243+0000 7f54dd200700 1 --2- 172.21.15.19:0/1893936677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faf5ef3a40 0x55faf5efaa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:58.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.243+0000 7f54dd200700 1 -- 172.21.15.19:0/1893936677 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55faf5ddb3d0 con 0x55faf5ef4390 2024-05-07T02:26:58.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.243+0000 7f54dd200700 1 --2- 172.21.15.19:0/1893936677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55faf5ef4390 0x55faf5efafa0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f54c4011d10 tx=0x7f54c400f6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57775b87dc586da1 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:26:58.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.243+0000 7f54bf400700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:26:58.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.244+0000 7f54bf400700 1 -- 172.21.15.19:0/1893936677 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c401f060 con 0x55faf5ef4390 2024-05-07T02:26:58.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.244+0000 7f54ef0edb80 1 -- 172.21.15.19:0/1893936677 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54d0024070 con 0x55faf5ef4390 2024-05-07T02:26:58.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.244+0000 7f54bf400700 1 -- 172.21.15.19:0/1893936677 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f54c4004030 con 0x55faf5ef4390 2024-05-07T02:26:58.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.244+0000 7f54bf400700 1 -- 172.21.15.19:0/1893936677 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f54c4026050 con 0x55faf5ef4390 2024-05-07T02:26:58.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.244+0000 7f54ef0edb80 10 client.24647 fetch_fsmap learned FSMap version 17 2024-05-07T02:26:58.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.244+0000 7f54ef0edb80 1 -- 172.21.15.19:0/1893936677 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55faf5f0b540 con 0x55faf5ef4390 2024-05-07T02:26:58.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.244+0000 7f54bf400700 1 -- 172.21.15.19:0/1893936677 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54d0024070 con 0x55faf5ef4390 2024-05-07T02:26:58.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.245+0000 7f54bf400700 1 -- 172.21.15.19:0/1893936677 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f54c4019070 con 0x55faf5ef4390 2024-05-07T02:26:58.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.245+0000 7f54ef0edb80 10 client.24647 fetch_fsmap finished waiting for FSMap version 17 2024-05-07T02:26:58.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.245+0000 7f54ef0edb80 10 client.24647 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-07T02:26:58.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.245+0000 7f54ef0edb80 20 client.24647 populate_metadata read hostname 'smithi019' 2024-05-07T02:26:58.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.245+0000 7f54ef0edb80 1 --2- 172.21.15.19:0/1893936677 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55faf5f0daf0 0x55faf5f0ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:58.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.245+0000 7f54ef0edb80 4 client.24647 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:26:58.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.245+0000 7f54ef0edb80 1 -- 172.21.15.19:0/1893936677 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55faf5ec0290 con 0x55faf5f0daf0 2024-05-07T02:26:58.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.245+0000 7f54ddc00700 1 --2- 172.21.15.19:0/1893936677 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55faf5f0daf0 0x55faf5f0ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:58.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.246+0000 7f54ddc00700 1 --2- 172.21.15.19:0/1893936677 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55faf5f0daf0 0x55faf5f0ef60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d246b418f6c33d0 server_cookie=e0fe39faa12baad5 in_seq=0 out_seq=0 2024-05-07T02:26:58.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.246+0000 7f54bf400700 10 client.24647 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:26:58.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.247+0000 7f54bf400700 1 -- 172.21.15.19:0/1893936677 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55faf5ec0290 con 0x55faf5f0daf0 2024-05-07T02:26:58.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.247+0000 7f54bf400700 10 client.24647 handle_command_reply: tid=0 2024-05-07T02:26:58.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.247+0000 7f54ef0edb80 1 client.24647 shutdown 2024-05-07T02:26:58.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.247+0000 7f54ef0edb80 1 -- 172.21.15.19:0/1893936677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55faf5ef4390 msgr2=0x55faf5efafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:58.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.247+0000 7f54ef0edb80 1 --2- 172.21.15.19:0/1893936677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55faf5ef4390 0x55faf5efafa0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f54c4011d10 tx=0x7f54c400f6a0 comp rx=0 tx=0).stop 2024-05-07T02:26:58.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.247+0000 7f54ef0edb80 1 -- 172.21.15.19:0/1893936677 shutdown_connections 2024-05-07T02:26:58.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.247+0000 7f54ef0edb80 1 --2- 172.21.15.19:0/1893936677 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55faf5f0daf0 0x55faf5f0ef60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:58.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.247+0000 7f54ef0edb80 1 --2- 172.21.15.19:0/1893936677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf5efdb10 0x55faf5efb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:58.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.247+0000 7f54ef0edb80 1 --2- 172.21.15.19:0/1893936677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faf5ef3a40 0x55faf5efaa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:58.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.247+0000 7f54bf400700 0 client.24647 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:26:58.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.247+0000 7f54ef0edb80 1 --2- 172.21.15.19:0/1893936677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55faf5ef4390 0x55faf5efafa0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:58.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.247+0000 7f54ef0edb80 1 -- 172.21.15.19:0/1893936677 >> 172.21.15.19:0/1893936677 conn(0x55faf5d75df0 msgr2=0x55faf5ef0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:26:58.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.247+0000 7f54ef0edb80 1 -- 172.21.15.19:0/1893936677 shutdown_connections 2024-05-07T02:26:58.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.247+0000 7f54ef0edb80 1 -- 172.21.15.19:0/1893936677 wait complete. 2024-05-07T02:26:58.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.247+0000 7f54ef0edb80 20 client.24647 trim_cache size 0 max 16384 2024-05-07T02:26:58.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.270+0000 7f54ef0edb80 1 Processor -- start 2024-05-07T02:26:58.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.271+0000 7f54ef0edb80 1 -- start start 2024-05-07T02:26:58.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.271+0000 7f54ef0edb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faf5ef3a40 0x55faf5f09d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:58.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.271+0000 7f54ef0edb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55faf5ef4390 0x55faf5efb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:58.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.271+0000 7f54ef0edb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf5efdb10 0x55faf5ef2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:58.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.271+0000 7f54ef0edb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55faf5f0d2a0 con 0x55faf5ef3a40 2024-05-07T02:26:58.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.271+0000 7f54ef0edb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55faf5f0d970 con 0x55faf5ef4390 2024-05-07T02:26:58.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.271+0000 7f54ef0edb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55faf5f0e040 con 0x55faf5efdb10 2024-05-07T02:26:58.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.271+0000 7f54ddc00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faf5ef3a40 0x55faf5f09d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:58.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.271+0000 7f54ddc00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faf5ef3a40 0x55faf5f09d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56532/0 (socket says 172.21.15.19:56532) 2024-05-07T02:26:58.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.271+0000 7f54ddc00700 1 -- 172.21.15.19:0/2713450751 learned_addr learned my addr 172.21.15.19:0/2713450751 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:26:58.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.271+0000 7f54de600700 1 --2- 172.21.15.19:0/2713450751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf5efdb10 0x55faf5ef2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:58.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.272+0000 7f54dd200700 1 --2- 172.21.15.19:0/2713450751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55faf5ef4390 0x55faf5efb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:58.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.272+0000 7f54ddc00700 1 -- 172.21.15.19:0/2713450751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf5efdb10 msgr2=0x55faf5ef2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:26:58.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.272+0000 7f54ddc00700 1 --2- 172.21.15.19:0/2713450751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf5efdb10 0x55faf5ef2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:58.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.272+0000 7f54ddc00700 1 -- 172.21.15.19:0/2713450751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55faf5ef4390 msgr2=0x55faf5efb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:26:58.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.272+0000 7f54ddc00700 1 --2- 172.21.15.19:0/2713450751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55faf5ef4390 0x55faf5efb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:58.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.272+0000 7f54ddc00700 1 -- 172.21.15.19:0/2713450751 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faf5efd300 con 0x55faf5ef3a40 2024-05-07T02:26:58.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.272+0000 7f54dd200700 1 --2- 172.21.15.19:0/2713450751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55faf5ef4390 0x55faf5efb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:26:58.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.272+0000 7f54ddc00700 1 --2- 172.21.15.19:0/2713450751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faf5ef3a40 0x55faf5f09d00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f54cc0a5930 tx=0x7f54cc0a5040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7d65f09769cb2b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:26:58.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.273+0000 7f54ca000700 1 -- 172.21.15.19:0/2713450751 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc0a9020 con 0x55faf5ef3a40 2024-05-07T02:26:58.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.273+0000 7f54ca000700 1 -- 172.21.15.19:0/2713450751 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54cc0a93c0 con 0x55faf5ef3a40 2024-05-07T02:26:58.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.273+0000 7f54ca000700 1 -- 172.21.15.19:0/2713450751 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc0e2330 con 0x55faf5ef3a40 2024-05-07T02:26:58.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.273+0000 7f54ef0edb80 1 -- 172.21.15.19:0/2713450751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faf5ef3a40 msgr2=0x55faf5f09d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:58.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.273+0000 7f54ef0edb80 1 --2- 172.21.15.19:0/2713450751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faf5ef3a40 0x55faf5f09d00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f54cc0a5930 tx=0x7f54cc0a5040 comp rx=0 tx=0).stop 2024-05-07T02:26:58.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.273+0000 7f54ef0edb80 1 -- 172.21.15.19:0/2713450751 shutdown_connections 2024-05-07T02:26:58.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.274+0000 7f54ef0edb80 1 --2- 172.21.15.19:0/2713450751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf5efdb10 0x55faf5ef2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:58.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.274+0000 7f54ef0edb80 1 --2- 172.21.15.19:0/2713450751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faf5ef3a40 0x55faf5f09d00 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:58.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.274+0000 7f54ef0edb80 1 --2- 172.21.15.19:0/2713450751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55faf5ef4390 0x55faf5efb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:58.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.274+0000 7f54ef0edb80 1 -- 172.21.15.19:0/2713450751 >> 172.21.15.19:0/2713450751 conn(0x55faf5d75df0 msgr2=0x55faf5ef9350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:26:58.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.274+0000 7f54ef0edb80 1 -- 172.21.15.19:0/2713450751 shutdown_connections 2024-05-07T02:26:58.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.274+0000 7f54ef0edb80 1 -- 172.21.15.19:0/2713450751 wait complete. 2024-05-07T02:26:58.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.274+0000 7f54ef0edb80 1 -- start start 2024-05-07T02:26:58.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.274+0000 7f54ef0edb80 1 Processor -- start 2024-05-07T02:26:58.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.275+0000 7f54ef0edb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faf5ef3a40 0x55faf5f18920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:58.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.275+0000 7f54ef0edb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55faf5ef4390 0x55faf5f11c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:58.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.275+0000 7f54ef0edb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf5efdb10 0x55faf5f121e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:58.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.275+0000 7f54ef0edb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55faf5f0ede0 con 0x55faf5ef3a40 2024-05-07T02:26:58.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.275+0000 7f54ef0edb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55faf5f0e710 con 0x55faf5ef4390 2024-05-07T02:26:58.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.275+0000 7f54ef0edb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55faf5f0f4b0 con 0x55faf5efdb10 2024-05-07T02:26:58.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.275+0000 7f54ef0edb80 10 client.0 _ll_register_callbacks cb 0x55faf5dbcc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:26:58.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.275+0000 7f54de600700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf5efdb10 0x55faf5f121e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:58.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.275+0000 7f54de600700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf5efdb10 0x55faf5f121e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:59670/0 (socket says 172.21.15.19:59670) 2024-05-07T02:26:58.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.275+0000 7f54de600700 1 -- 172.21.15.19:0/3560638141 learned_addr learned my addr 172.21.15.19:0/3560638141 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:26:58.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.275+0000 7f54ddc00700 1 --2- 172.21.15.19:0/3560638141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faf5ef3a40 0x55faf5f18920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:58.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.276+0000 7f54dd200700 1 --2- 172.21.15.19:0/3560638141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55faf5ef4390 0x55faf5f11c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:58.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.276+0000 7f54de600700 1 -- 172.21.15.19:0/3560638141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55faf5ef4390 msgr2=0x55faf5f11c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:26:58.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.276+0000 7f54de600700 1 --2- 172.21.15.19:0/3560638141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55faf5ef4390 0x55faf5f11c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:58.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.276+0000 7f54de600700 1 -- 172.21.15.19:0/3560638141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faf5ef3a40 msgr2=0x55faf5f18920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:26:58.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.276+0000 7f54de600700 1 --2- 172.21.15.19:0/3560638141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faf5ef3a40 0x55faf5f18920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:58.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.276+0000 7f54de600700 1 -- 172.21.15.19:0/3560638141 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55faf5f12980 con 0x55faf5efdb10 2024-05-07T02:26:58.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.276+0000 7f54ddc00700 1 --2- 172.21.15.19:0/3560638141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faf5ef3a40 0x55faf5f18920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:26:58.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.276+0000 7f54de600700 1 --2- 172.21.15.19:0/3560638141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf5efdb10 0x55faf5f121e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f54d000d010 tx=0x7f54d0006060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83005c7c0d7081c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:26:58.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.276+0000 7f54bf400700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:26:58.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.277+0000 7f54bf400700 1 -- 172.21.15.19:0/3560638141 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d002e040 con 0x55faf5efdb10 2024-05-07T02:26:58.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.277+0000 7f54ef0edb80 1 -- 172.21.15.19:0/3560638141 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54cc0a93c0 con 0x55faf5efdb10 2024-05-07T02:26:58.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.277+0000 7f54bf400700 1 -- 172.21.15.19:0/3560638141 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f54d00075c0 con 0x55faf5efdb10 2024-05-07T02:26:58.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.277+0000 7f54bf400700 1 -- 172.21.15.19:0/3560638141 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f54d0044050 con 0x55faf5efdb10 2024-05-07T02:26:58.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.277+0000 7f54ef0edb80 10 client.24652 fetch_fsmap learned FSMap version 17 2024-05-07T02:26:58.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.277+0000 7f54ef0edb80 1 -- 172.21.15.19:0/3560638141 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55faf5f78f60 con 0x55faf5efdb10 2024-05-07T02:26:58.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.277+0000 7f54bf400700 1 -- 172.21.15.19:0/3560638141 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54cc0a93c0 con 0x55faf5efdb10 2024-05-07T02:26:58.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.278+0000 7f54bf400700 1 -- 172.21.15.19:0/3560638141 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f54d0011070 con 0x55faf5efdb10 2024-05-07T02:26:58.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.278+0000 7f54ef0edb80 10 client.24652 fetch_fsmap finished waiting for FSMap version 17 2024-05-07T02:26:58.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.278+0000 7f54ef0edb80 10 client.24652 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-07T02:26:58.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.278+0000 7f54ef0edb80 20 client.24652 populate_metadata read hostname 'smithi019' 2024-05-07T02:26:58.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.278+0000 7f54ef0edb80 1 --2- 172.21.15.19:0/3560638141 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55faf5f7ae30 0x55faf5f7c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:26:58.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.278+0000 7f54ef0edb80 4 client.24652 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-07T02:26:58.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.278+0000 7f54ef0edb80 1 -- 172.21.15.19:0/3560638141 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "status"}) -- 0x55faf5e233b0 con 0x55faf5f7ae30 2024-05-07T02:26:58.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.278+0000 7f54ddc00700 1 --2- 172.21.15.19:0/3560638141 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55faf5f7ae30 0x55faf5f7c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:26:58.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.279+0000 7f54ddc00700 1 --2- 172.21.15.19:0/3560638141 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55faf5f7ae30 0x55faf5f7c2a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e88534b5e07e1f41 server_cookie=2678b0ca615b2326 in_seq=0 out_seq=0 2024-05-07T02:26:58.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.279+0000 7f54bf400700 10 client.24652 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:26:58.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.279+0000 7f54bf400700 1 -- 172.21.15.19:0/3560638141 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55faf5e233b0 con 0x55faf5f7ae30 2024-05-07T02:26:58.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.279+0000 7f54bf400700 10 client.24652 handle_command_reply: tid=0 2024-05-07T02:26:58.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.279+0000 7f54ef0edb80 1 client.24652 shutdown 2024-05-07T02:26:58.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.279+0000 7f54ef0edb80 1 -- 172.21.15.19:0/3560638141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf5efdb10 msgr2=0x55faf5f121e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:58.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.279+0000 7f54ef0edb80 1 --2- 172.21.15.19:0/3560638141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf5efdb10 0x55faf5f121e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f54d000d010 tx=0x7f54d0006060 comp rx=0 tx=0).stop 2024-05-07T02:26:58.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.279+0000 7f54ef0edb80 1 -- 172.21.15.19:0/3560638141 shutdown_connections 2024-05-07T02:26:58.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.279+0000 7f54ef0edb80 1 --2- 172.21.15.19:0/3560638141 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55faf5f7ae30 0x55faf5f7c2a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:58.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.279+0000 7f54ef0edb80 1 --2- 172.21.15.19:0/3560638141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55faf5efdb10 0x55faf5f121e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:58.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.279+0000 7f54ef0edb80 1 --2- 172.21.15.19:0/3560638141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faf5ef3a40 0x55faf5f18920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:58.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.279+0000 7f54ef0edb80 1 --2- 172.21.15.19:0/3560638141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55faf5ef4390 0x55faf5f11c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:58.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.279+0000 7f54ef0edb80 1 -- 172.21.15.19:0/3560638141 >> 172.21.15.19:0/3560638141 conn(0x55faf5d75df0 msgr2=0x55faf5ef8f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:26:58.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.279+0000 7f54bf400700 0 client.24652 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:26:58.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.280+0000 7f54ef0edb80 1 -- 172.21.15.19:0/3560638141 shutdown_connections 2024-05-07T02:26:58.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.280+0000 7f54ef0edb80 1 -- 172.21.15.19:0/3560638141 wait complete. 2024-05-07T02:26:58.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.280+0000 7f54ef0edb80 20 client.24652 trim_cache size 0 max 16384 2024-05-07T02:26:58.282 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:26:58.282 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "9ff84c16-0c17-11ef-bc97-c7b262605968", 2024-05-07T02:26:58.282 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-05-07T02:26:58.282 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24551, 2024-05-07T02:26:58.282 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-05-07T02:26:58.282 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-05-07T02:26:58.283 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-05-07T02:26:58.283 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 43.263162485999999, 2024-05-07T02:26:58.283 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 17, 2024-05-07T02:26:58.283 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 85, 2024-05-07T02:26:58.283 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 85, 2024-05-07T02:26:58.283 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 64.657536164000007 2024-05-07T02:26:58.283 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:26:58.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.280+0000 7f54df000700 1 -- 172.21.15.19:0/2146595803 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f54c00b2980 msgr2=0x7f54c00b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:58.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.280+0000 7f54df000700 1 --2- 172.21.15.19:0/2146595803 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f54c00b2980 0x7f54c00b4e90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f54d002b000 tx=0x7f54d002c000 comp rx=0 tx=0).stop 2024-05-07T02:26:58.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.280+0000 7f54df000700 1 -- 172.21.15.19:0/2146595803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54d810a7c0 msgr2=0x7f54d81a1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:26:58.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.280+0000 7f54df000700 1 --2- 172.21.15.19:0/2146595803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54d810a7c0 0x7f54d81a1220 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f54cc00e9a0 tx=0x7f54cc00ecb0 comp rx=0 tx=0).stop 2024-05-07T02:26:58.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.281+0000 7f54df000700 1 -- 172.21.15.19:0/2146595803 shutdown_connections 2024-05-07T02:26:58.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.281+0000 7f54df000700 1 --2- 172.21.15.19:0/2146595803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54d81107a0 0x7f54d81a5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:58.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.281+0000 7f54df000700 1 --2- 172.21.15.19:0/2146595803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54d810a7c0 0x7f54d81a1220 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:58.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.281+0000 7f54df000700 1 --2- 172.21.15.19:0/2146595803 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f54c00b2980 0x7f54c00b4e90 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:58.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.281+0000 7f54df000700 1 --2- 172.21.15.19:0/2146595803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54d810b110 0x7f54d81a17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:26:58.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.281+0000 7f54df000700 1 -- 172.21.15.19:0/2146595803 >> 172.21.15.19:0/2146595803 conn(0x7f54d8076b00 msgr2=0x7f54d81088c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:26:58.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.281+0000 7f54df000700 1 -- 172.21.15.19:0/2146595803 shutdown_connections 2024-05-07T02:26:58.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:26:58.281+0000 7f54df000700 1 -- 172.21.15.19:0/2146595803 wait complete. 2024-05-07T02:26:59.089 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:26:58 smithi019 ceph-mon[29781]: pgmap v358: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:59.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:26:58 smithi064 ceph-mon[34699]: pgmap v358: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:59.172 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-07T02:26:59.172 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-07T02:26:59.173 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph fs ls 2024-05-07T02:26:59.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:26:58 smithi027 ceph-mon[34398]: pgmap v358: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:26:59.633 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:27:01.057 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:00 smithi019 ceph-mon[29781]: pgmap v359: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:27:01.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:00 smithi064 ceph-mon[34699]: pgmap v359: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:27:01.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:00 smithi027 ceph-mon[34398]: pgmap v359: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:27:02.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.046+0000 7f5e12e00700 1 -- 172.21.15.19:0/4265584755 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e0c10e700 msgr2=0x7f5e0c110be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:02.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.046+0000 7f5e12e00700 1 --2- 172.21.15.19:0/4265584755 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e0c10e700 0x7f5e0c110be0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5e04009f90 tx=0x7f5e0400b330 comp rx=0 tx=0).stop 2024-05-07T02:27:02.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.046+0000 7f5e12e00700 1 -- 172.21.15.19:0/4265584755 shutdown_connections 2024-05-07T02:27:02.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.046+0000 7f5e12e00700 1 --2- 172.21.15.19:0/4265584755 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e0c10e700 0x7f5e0c110be0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:02.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.046+0000 7f5e12e00700 1 --2- 172.21.15.19:0/4265584755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e0c105290 0x7f5e0c1056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:02.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.046+0000 7f5e12e00700 1 --2- 172.21.15.19:0/4265584755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e0c105be0 0x7f5e0c10e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:02.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.046+0000 7f5e12e00700 1 -- 172.21.15.19:0/4265584755 >> 172.21.15.19:0/4265584755 conn(0x7f5e0c078ac0 msgr2=0x7f5e0c078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:27:02.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.047+0000 7f5e12e00700 1 -- 172.21.15.19:0/4265584755 shutdown_connections 2024-05-07T02:27:02.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.047+0000 7f5e12e00700 1 -- 172.21.15.19:0/4265584755 wait complete. 2024-05-07T02:27:02.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.049+0000 7f5e12e00700 1 Processor -- start 2024-05-07T02:27:02.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.049+0000 7f5e12e00700 1 -- start start 2024-05-07T02:27:02.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.049+0000 7f5e12e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e0c105290 0x7f5e0c19ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:27:02.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.049+0000 7f5e12e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e0c105be0 0x7f5e0c19d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:27:02.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.049+0000 7f5e12e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e0c10e700 0x7f5e0c1a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:27:02.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.049+0000 7f5e12e00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c114260 con 0x7f5e0c105290 2024-05-07T02:27:02.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.049+0000 7f5e12e00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c1140e0 con 0x7f5e0c10e700 2024-05-07T02:27:02.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.049+0000 7f5e12e00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c1143e0 con 0x7f5e0c105be0 2024-05-07T02:27:02.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.050+0000 7f5e11a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e0c105290 0x7f5e0c19ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:27:02.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.050+0000 7f5e11a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e0c105290 0x7f5e0c19ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35984/0 (socket says 172.21.15.19:35984) 2024-05-07T02:27:02.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.050+0000 7f5e11a00700 1 -- 172.21.15.19:0/230112105 learned_addr learned my addr 172.21.15.19:0/230112105 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:27:02.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.050+0000 7f5e11000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e0c105be0 0x7f5e0c19d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:27:02.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.050+0000 7f5e12400700 1 --2- 172.21.15.19:0/230112105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e0c10e700 0x7f5e0c1a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:27:02.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.050+0000 7f5e11a00700 1 -- 172.21.15.19:0/230112105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e0c105be0 msgr2=0x7f5e0c19d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:02.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.050+0000 7f5e11a00700 1 --2- 172.21.15.19:0/230112105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e0c105be0 0x7f5e0c19d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:02.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.050+0000 7f5e11a00700 1 -- 172.21.15.19:0/230112105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e0c10e700 msgr2=0x7f5e0c1a1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:02.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.050+0000 7f5e11a00700 1 --2- 172.21.15.19:0/230112105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e0c10e700 0x7f5e0c1a1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:02.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.050+0000 7f5e11a00700 1 -- 172.21.15.19:0/230112105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e0c1a1d00 con 0x7f5e0c105290 2024-05-07T02:27:02.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.050+0000 7f5e12400700 1 --2- 172.21.15.19:0/230112105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e0c10e700 0x7f5e0c1a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:27:02.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.051+0000 7f5e11a00700 1 --2- 172.21.15.19:0/230112105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e0c105290 0x7f5e0c19ce30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5e0000e9a0 tx=0x7f5e0000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:27:02.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.051+0000 7f5e0a000700 1 -- 172.21.15.19:0/230112105 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e0000ca10 con 0x7f5e0c105290 2024-05-07T02:27:02.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.051+0000 7f5e0a000700 1 -- 172.21.15.19:0/230112105 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e0000cbb0 con 0x7f5e0c105290 2024-05-07T02:27:02.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.051+0000 7f5e12e00700 1 -- 172.21.15.19:0/230112105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e0c1a1ff0 con 0x7f5e0c105290 2024-05-07T02:27:02.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.051+0000 7f5e0a000700 1 -- 172.21.15.19:0/230112105 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e00018680 con 0x7f5e0c105290 2024-05-07T02:27:02.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.051+0000 7f5e12e00700 1 -- 172.21.15.19:0/230112105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e0c1a9910 con 0x7f5e0c105290 2024-05-07T02:27:02.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.052+0000 7f5e12e00700 1 -- 172.21.15.19:0/230112105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e0c0512c0 con 0x7f5e0c105290 2024-05-07T02:27:02.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.055+0000 7f5e0a000700 1 -- 172.21.15.19:0/230112105 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f5e00010430 con 0x7f5e0c105290 2024-05-07T02:27:02.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.057+0000 7f5e0a000700 1 --2- 172.21.15.19:0/230112105 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f5df40b2880 0x7f5df40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:27:02.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.057+0000 7f5e0a000700 1 -- 172.21.15.19:0/230112105 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f5e000e2960 con 0x7f5e0c105290 2024-05-07T02:27:02.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.057+0000 7f5e11000700 1 --2- 172.21.15.19:0/230112105 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f5df40b2880 0x7f5df40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:27:02.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.058+0000 7f5e11000700 1 --2- 172.21.15.19:0/230112105 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f5df40b2880 0x7f5df40b4d90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009750 tx=0x7f5df8009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:27:02.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.061+0000 7f5e0a000700 1 -- 172.21.15.19:0/230112105 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5e000aa3e0 con 0x7f5e0c105290 2024-05-07T02:27:02.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.350+0000 7f5e12e00700 1 -- 172.21.15.19:0/230112105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5e0c1a9ae0 con 0x7f5e0c105290 2024-05-07T02:27:02.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.351+0000 7f5e0a000700 1 -- 172.21.15.19:0/230112105 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f5e000a9ce0 con 0x7f5e0c105290 2024-05-07T02:27:02.351 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-07T02:27:02.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.355+0000 7f5e12e00700 1 -- 172.21.15.19:0/230112105 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f5df40b2880 msgr2=0x7f5df40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:02.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.355+0000 7f5e12e00700 1 --2- 172.21.15.19:0/230112105 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f5df40b2880 0x7f5df40b4d90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5df8009750 tx=0x7f5df8009310 comp rx=0 tx=0).stop 2024-05-07T02:27:02.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.355+0000 7f5e12e00700 1 -- 172.21.15.19:0/230112105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e0c105290 msgr2=0x7f5e0c19ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:02.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.355+0000 7f5e12e00700 1 --2- 172.21.15.19:0/230112105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e0c105290 0x7f5e0c19ce30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5e0000e9a0 tx=0x7f5e0000ecb0 comp rx=0 tx=0).stop 2024-05-07T02:27:02.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.356+0000 7f5e12e00700 1 -- 172.21.15.19:0/230112105 shutdown_connections 2024-05-07T02:27:02.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.356+0000 7f5e12e00700 1 --2- 172.21.15.19:0/230112105 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f5df40b2880 0x7f5df40b4d90 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:02.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.356+0000 7f5e12e00700 1 --2- 172.21.15.19:0/230112105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e0c10e700 0x7f5e0c1a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:02.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.356+0000 7f5e12e00700 1 --2- 172.21.15.19:0/230112105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e0c105290 0x7f5e0c19ce30 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:02.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.356+0000 7f5e12e00700 1 --2- 172.21.15.19:0/230112105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5e0c105be0 0x7f5e0c19d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:02.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.356+0000 7f5e12e00700 1 -- 172.21.15.19:0/230112105 >> 172.21.15.19:0/230112105 conn(0x7f5e0c078ac0 msgr2=0x7f5e0c102f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:27:02.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.356+0000 7f5e12e00700 1 -- 172.21.15.19:0/230112105 shutdown_connections 2024-05-07T02:27:02.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:02.356+0000 7f5e12e00700 1 -- 172.21.15.19:0/230112105 wait complete. 2024-05-07T02:27:03.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:02 smithi064 ceph-mon[34699]: pgmap v360: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:27:03.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:02 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/230112105' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-07T02:27:03.117 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:02 smithi019 ceph-mon[29781]: pgmap v360: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:27:03.117 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:02 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/230112105' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-07T02:27:03.199 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-07T02:27:03.199 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-07T02:27:03.199 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-05-07T02:27:03.199 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-05-07T02:27:03.199 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-07T02:27:03.199 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-07T02:27:03.199 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-07T02:27:03.199 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-07T02:27:03.199 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-05-07T02:27:03.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-07T02:27:03.200 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-05-07T02:27:03.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:02 smithi027 ceph-mon[34398]: pgmap v360: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:27:03.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:02 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/230112105' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-07T02:27:03.261 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-07T02:27:03.262 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-07T02:27:03.262 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-05-07T02:27:03.262 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-05-07T02:27:03.262 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-05-07T02:27:03.262 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-05-07T02:27:03.262 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-07T02:27:03.262 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-05-07T02:27:03.262 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-05-07T02:27:03.262 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-07T02:27:03.262 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-05-07T02:27:03.262 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-05-07T02:27:03.262 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-07T02:27:03.263 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-05-07T02:27:03.263 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-05-07T02:27:03.263 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-05-07T02:27:03.263 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 41991sec preferred_lft 41991sec 2024-05-07T02:27:03.263 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-05-07T02:27:03.263 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-05-07T02:27:03.263 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-07T02:27:03.263 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-05-07T02:27:03.263 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-05-07T02:27:03.264 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-07T02:27:03.264 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-07T02:27:03.264 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-05-07T02:27:03.264 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link add name ceph-brx type bridge 2024-05-07T02:27:03.264 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip addr flush dev ceph-brx 2024-05-07T02:27:03.265 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set ceph-brx up 2024-05-07T02:27:03.265 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-07T02:27:03.265 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-05-07T02:27:03.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-07T02:27:03.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-07T02:27:03.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-07T02:27:03.781 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-07T02:27:03.858 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-05-07T02:27:03.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-07T02:27:03.863 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-05-07T02:27:03.934 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-05-07T02:27:03.934 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-07T02:27:03.934 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-07T02:27:03.934 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-07T02:27:03.934 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-07T02:27:03.935 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-07T02:27:03.935 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-05-07T02:27:03.935 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-07T02:27:03.935 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-07T02:27:03.935 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-07T02:27:03.935 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-05-07T02:27:04.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-07T02:27:04.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-07T02:27:04.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-07T02:27:04.677 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-05-07T02:27:04.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-07T02:27:04.735 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list-id 2024-05-07T02:27:04.792 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-07T02:27:04.792 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-05-07T02:27:04.792 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-07T02:27:04.792 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-05-07T02:27:04.792 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-05-07T02:27:04.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-07T02:27:04.930 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:04 smithi019 ceph-mon[29781]: pgmap v361: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:27:04.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-07T02:27:04.992 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-05-07T02:27:04.992 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-07T02:27:04.992 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-05-07T02:27:04.992 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-05-07T02:27:04.992 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-07T02:27:04.992 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-05-07T02:27:04.992 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-05-07T02:27:04.992 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-05-07T02:27:04.993 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-05-07T02:27:05.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-07T02:27:05.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:04 smithi064 ceph-mon[34699]: pgmap v361: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:27:05.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:04 smithi027 ceph-mon[34398]: pgmap v361: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:27:05.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-07T02:27:05.303 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-07T02:27:05.303 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-05-07T02:27:05.303 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-05-07T02:27:05.303 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set dev brx.0 master ceph-brx 2024-05-07T02:27:05.303 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-05-07T02:27:05.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-07T02:27:05.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-07T02:27:05.512 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-05-07T02:27:05.512 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-07T02:27:05.512 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-05-07T02:27:05.556 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-05-07T02:27:05.556 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-07T02:27:05.556 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-05-07T02:27:05.646 DEBUG:teuthology.orchestra.run.smithi019:> sudo modprobe fuse 2024-05-07T02:27:05.777 DEBUG:teuthology.orchestra.run.smithi019:> cat /proc/self/mounts | awk '{print $2}' 2024-05-07T02:27:05.820 INFO:teuthology.orchestra.run.smithi019.stdout:/proc 2024-05-07T02:27:05.821 INFO:teuthology.orchestra.run.smithi019.stdout:/sys 2024-05-07T02:27:05.821 INFO:teuthology.orchestra.run.smithi019.stdout:/dev 2024-05-07T02:27:05.821 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/security 2024-05-07T02:27:05.821 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/shm 2024-05-07T02:27:05.821 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/pts 2024-05-07T02:27:05.821 INFO:teuthology.orchestra.run.smithi019.stdout:/run 2024-05-07T02:27:05.821 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/cgroup 2024-05-07T02:27:05.821 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/pstore 2024-05-07T02:27:05.821 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/bpf 2024-05-07T02:27:05.821 INFO:teuthology.orchestra.run.smithi019.stdout:/ 2024-05-07T02:27:05.821 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/selinux 2024-05-07T02:27:05.821 INFO:teuthology.orchestra.run.smithi019.stdout:/proc/sys/fs/binfmt_misc 2024-05-07T02:27:05.821 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/mqueue 2024-05-07T02:27:05.821 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/hugepages 2024-05-07T02:27:05.821 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/debug 2024-05-07T02:27:05.822 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/tracing 2024-05-07T02:27:05.822 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/fuse/connections 2024-05-07T02:27:05.822 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/config 2024-05-07T02:27:05.822 INFO:teuthology.orchestra.run.smithi019.stdout:/run/credentials/systemd-sysctl.service 2024-05-07T02:27:05.822 INFO:teuthology.orchestra.run.smithi019.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-07T02:27:05.822 INFO:teuthology.orchestra.run.smithi019.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-07T02:27:05.822 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/nfs/rpc_pipefs 2024-05-07T02:27:05.822 INFO:teuthology.orchestra.run.smithi019.stdout:/run/user/1000 2024-05-07T02:27:05.822 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/ceph 2024-05-07T02:27:05.822 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay 2024-05-07T02:27:05.822 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/8f369e9e68edcf6de5802b6c9c4c6ce13d883d5c3ff5e8f5240f0fc5777aefeb/merged 2024-05-07T02:27:05.822 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/dab43a7583b0e22f90f5d590b3a0894db9ae0b5fd60be62fb7dcbd18d9ba7ba6/merged 2024-05-07T02:27:05.822 INFO:teuthology.orchestra.run.smithi019.stdout:/run/user/0 2024-05-07T02:27:05.822 INFO:teuthology.orchestra.run.smithi019.stdout:/proc/sys/fs/binfmt_misc 2024-05-07T02:27:05.823 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/d201a72cc8ff2f9154239869890ed3af6b1941b7fb3b8b6e7e7ca33890f06ef5/merged 2024-05-07T02:27:05.823 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/6b386764f0c59b2b9c4eac182544f5973c2998d34f72ec8abefcaff1e924edec/merged 2024-05-07T02:27:05.823 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/25d0de45dbc5512b6403937195a0bd6990663d8a2624965f1316c39ad61e696a/merged 2024-05-07T02:27:05.823 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/81cbe839f65c2fa63e40b00c713d7e491fa1e22c78eea409a62f5bbb25ee9844/merged 2024-05-07T02:27:05.823 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/49a7176001ddbb7c1c02c8ec0627f0c349f17f35c112c2de2cbbc528be267a97/merged 2024-05-07T02:27:05.823 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/b03bb2817a716bf6b70ac37d55e59c95f4e4ae41aa47d23fc29608eb0c892d52/merged 2024-05-07T02:27:05.823 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/e0fd1498c50f9a7adbe2ddc43c6fea732b1b42ee1d3883e90a7b0c09757ee585/merged 2024-05-07T02:27:05.823 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/128897dc1e473a16b86814c44994ff97dfdb59c983fcb4bc8753bc4f3d9ec22e/merged 2024-05-07T02:27:05.823 INFO:teuthology.orchestra.run.smithi019.stdout:/run/netns 2024-05-07T02:27:05.823 INFO:teuthology.orchestra.run.smithi019.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-07T02:27:05.823 INFO:teuthology.orchestra.run.smithi019.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-07T02:27:05.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-07T02:27:05.824 DEBUG:teuthology.orchestra.run.smithi019:> ls /sys/fs/fuse/connections 2024-05-07T02:27:05.904 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-07T02:27:05.904 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-05-07T02:27:05.950 DEBUG:teuthology.orchestra.run.smithi019:> sudo modprobe fuse 2024-05-07T02:27:06.001 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:05 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:27:06.021 DEBUG:teuthology.orchestra.run.smithi019:> cat /proc/self/mounts | awk '{print $2}' 2024-05-07T02:27:06.054 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:05 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:27:06.063 INFO:teuthology.orchestra.run.smithi019.stdout:/proc 2024-05-07T02:27:06.063 INFO:teuthology.orchestra.run.smithi019.stdout:/sys 2024-05-07T02:27:06.063 INFO:teuthology.orchestra.run.smithi019.stdout:/dev 2024-05-07T02:27:06.063 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/security 2024-05-07T02:27:06.063 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/shm 2024-05-07T02:27:06.063 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/pts 2024-05-07T02:27:06.063 INFO:teuthology.orchestra.run.smithi019.stdout:/run 2024-05-07T02:27:06.064 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/cgroup 2024-05-07T02:27:06.064 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/pstore 2024-05-07T02:27:06.064 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/bpf 2024-05-07T02:27:06.064 INFO:teuthology.orchestra.run.smithi019.stdout:/ 2024-05-07T02:27:06.064 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/selinux 2024-05-07T02:27:06.064 INFO:teuthology.orchestra.run.smithi019.stdout:/proc/sys/fs/binfmt_misc 2024-05-07T02:27:06.064 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/mqueue 2024-05-07T02:27:06.064 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/hugepages 2024-05-07T02:27:06.064 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/debug 2024-05-07T02:27:06.064 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/tracing 2024-05-07T02:27:06.064 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/fuse/connections 2024-05-07T02:27:06.064 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/config 2024-05-07T02:27:06.064 INFO:teuthology.orchestra.run.smithi019.stdout:/run/credentials/systemd-sysctl.service 2024-05-07T02:27:06.064 INFO:teuthology.orchestra.run.smithi019.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-07T02:27:06.064 INFO:teuthology.orchestra.run.smithi019.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-07T02:27:06.064 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/nfs/rpc_pipefs 2024-05-07T02:27:06.065 INFO:teuthology.orchestra.run.smithi019.stdout:/run/user/1000 2024-05-07T02:27:06.065 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/ceph 2024-05-07T02:27:06.065 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay 2024-05-07T02:27:06.065 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/8f369e9e68edcf6de5802b6c9c4c6ce13d883d5c3ff5e8f5240f0fc5777aefeb/merged 2024-05-07T02:27:06.065 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/dab43a7583b0e22f90f5d590b3a0894db9ae0b5fd60be62fb7dcbd18d9ba7ba6/merged 2024-05-07T02:27:06.065 INFO:teuthology.orchestra.run.smithi019.stdout:/run/user/0 2024-05-07T02:27:06.065 INFO:teuthology.orchestra.run.smithi019.stdout:/proc/sys/fs/binfmt_misc 2024-05-07T02:27:06.065 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/d201a72cc8ff2f9154239869890ed3af6b1941b7fb3b8b6e7e7ca33890f06ef5/merged 2024-05-07T02:27:06.065 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/6b386764f0c59b2b9c4eac182544f5973c2998d34f72ec8abefcaff1e924edec/merged 2024-05-07T02:27:06.065 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/25d0de45dbc5512b6403937195a0bd6990663d8a2624965f1316c39ad61e696a/merged 2024-05-07T02:27:06.065 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/81cbe839f65c2fa63e40b00c713d7e491fa1e22c78eea409a62f5bbb25ee9844/merged 2024-05-07T02:27:06.065 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/49a7176001ddbb7c1c02c8ec0627f0c349f17f35c112c2de2cbbc528be267a97/merged 2024-05-07T02:27:06.065 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/b03bb2817a716bf6b70ac37d55e59c95f4e4ae41aa47d23fc29608eb0c892d52/merged 2024-05-07T02:27:06.065 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/e0fd1498c50f9a7adbe2ddc43c6fea732b1b42ee1d3883e90a7b0c09757ee585/merged 2024-05-07T02:27:06.065 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/128897dc1e473a16b86814c44994ff97dfdb59c983fcb4bc8753bc4f3d9ec22e/merged 2024-05-07T02:27:06.065 INFO:teuthology.orchestra.run.smithi019.stdout:/run/netns 2024-05-07T02:27:06.066 INFO:teuthology.orchestra.run.smithi019.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-07T02:27:06.066 INFO:teuthology.orchestra.run.smithi019.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-07T02:27:06.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-07T02:27:06.066 DEBUG:teuthology.orchestra.run.smithi019:> ls /sys/fs/fuse/connections 2024-05-07T02:27:06.095 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:05 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:27:06.249 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi019.stderr:2024-05-07T02:27:06.250+0000 7fceadf79480 -1 init, newargv = 0x7fce900046a0 newargc=15 2024-05-07T02:27:06.250 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi019.stderr:ceph-fuse[75951]: starting ceph client 2024-05-07T02:27:06.260 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi019.stderr:ceph-fuse[75951]: starting fuse 2024-05-07T02:27:07.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:06 smithi064 ceph-mon[34699]: pgmap v362: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:27:07.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:06 smithi019 ceph-mon[29781]: pgmap v362: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:27:07.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:06 smithi027 ceph-mon[34398]: pgmap v362: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:27:07.255 DEBUG:teuthology.orchestra.run.smithi019:> sudo modprobe fuse 2024-05-07T02:27:07.332 DEBUG:teuthology.orchestra.run.smithi019:> cat /proc/self/mounts | awk '{print $2}' 2024-05-07T02:27:07.381 INFO:teuthology.orchestra.run.smithi019.stdout:/proc 2024-05-07T02:27:07.381 INFO:teuthology.orchestra.run.smithi019.stdout:/sys 2024-05-07T02:27:07.381 INFO:teuthology.orchestra.run.smithi019.stdout:/dev 2024-05-07T02:27:07.381 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/security 2024-05-07T02:27:07.381 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/shm 2024-05-07T02:27:07.381 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/pts 2024-05-07T02:27:07.381 INFO:teuthology.orchestra.run.smithi019.stdout:/run 2024-05-07T02:27:07.381 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/cgroup 2024-05-07T02:27:07.381 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/pstore 2024-05-07T02:27:07.381 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/bpf 2024-05-07T02:27:07.381 INFO:teuthology.orchestra.run.smithi019.stdout:/ 2024-05-07T02:27:07.382 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/selinux 2024-05-07T02:27:07.382 INFO:teuthology.orchestra.run.smithi019.stdout:/proc/sys/fs/binfmt_misc 2024-05-07T02:27:07.382 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/mqueue 2024-05-07T02:27:07.382 INFO:teuthology.orchestra.run.smithi019.stdout:/dev/hugepages 2024-05-07T02:27:07.382 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/debug 2024-05-07T02:27:07.382 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/tracing 2024-05-07T02:27:07.382 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/fs/fuse/connections 2024-05-07T02:27:07.382 INFO:teuthology.orchestra.run.smithi019.stdout:/sys/kernel/config 2024-05-07T02:27:07.382 INFO:teuthology.orchestra.run.smithi019.stdout:/run/credentials/systemd-sysctl.service 2024-05-07T02:27:07.382 INFO:teuthology.orchestra.run.smithi019.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-07T02:27:07.382 INFO:teuthology.orchestra.run.smithi019.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-07T02:27:07.382 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/nfs/rpc_pipefs 2024-05-07T02:27:07.382 INFO:teuthology.orchestra.run.smithi019.stdout:/run/user/1000 2024-05-07T02:27:07.382 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/ceph 2024-05-07T02:27:07.383 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay 2024-05-07T02:27:07.383 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/8f369e9e68edcf6de5802b6c9c4c6ce13d883d5c3ff5e8f5240f0fc5777aefeb/merged 2024-05-07T02:27:07.383 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/dab43a7583b0e22f90f5d590b3a0894db9ae0b5fd60be62fb7dcbd18d9ba7ba6/merged 2024-05-07T02:27:07.383 INFO:teuthology.orchestra.run.smithi019.stdout:/run/user/0 2024-05-07T02:27:07.383 INFO:teuthology.orchestra.run.smithi019.stdout:/proc/sys/fs/binfmt_misc 2024-05-07T02:27:07.383 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/d201a72cc8ff2f9154239869890ed3af6b1941b7fb3b8b6e7e7ca33890f06ef5/merged 2024-05-07T02:27:07.383 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/6b386764f0c59b2b9c4eac182544f5973c2998d34f72ec8abefcaff1e924edec/merged 2024-05-07T02:27:07.383 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/25d0de45dbc5512b6403937195a0bd6990663d8a2624965f1316c39ad61e696a/merged 2024-05-07T02:27:07.383 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/81cbe839f65c2fa63e40b00c713d7e491fa1e22c78eea409a62f5bbb25ee9844/merged 2024-05-07T02:27:07.383 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/49a7176001ddbb7c1c02c8ec0627f0c349f17f35c112c2de2cbbc528be267a97/merged 2024-05-07T02:27:07.383 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/b03bb2817a716bf6b70ac37d55e59c95f4e4ae41aa47d23fc29608eb0c892d52/merged 2024-05-07T02:27:07.383 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/e0fd1498c50f9a7adbe2ddc43c6fea732b1b42ee1d3883e90a7b0c09757ee585/merged 2024-05-07T02:27:07.383 INFO:teuthology.orchestra.run.smithi019.stdout:/var/lib/containers/storage/overlay/128897dc1e473a16b86814c44994ff97dfdb59c983fcb4bc8753bc4f3d9ec22e/merged 2024-05-07T02:27:07.383 INFO:teuthology.orchestra.run.smithi019.stdout:/run/netns 2024-05-07T02:27:07.383 INFO:teuthology.orchestra.run.smithi019.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-07T02:27:07.383 INFO:teuthology.orchestra.run.smithi019.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-07T02:27:07.384 INFO:teuthology.orchestra.run.smithi019.stdout:/home/ubuntu/cephtest/mnt.admin 2024-05-07T02:27:07.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-07T02:27:07.384 DEBUG:teuthology.orchestra.run.smithi019:> ls /sys/fs/fuse/connections 2024-05-07T02:27:07.427 INFO:teuthology.orchestra.run.smithi019.stdout:114 2024-05-07T02:27:07.428 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [114] 2024-05-07T02:27:07.428 DEBUG:teuthology.orchestra.run.smithi019:> sudo stdin-killer -- python3 -c ' 2024-05-07T02:27:07.428 DEBUG:teuthology.orchestra.run.smithi019:> import glob 2024-05-07T02:27:07.428 DEBUG:teuthology.orchestra.run.smithi019:> import re 2024-05-07T02:27:07.428 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-05-07T02:27:07.428 DEBUG:teuthology.orchestra.run.smithi019:> import subprocess 2024-05-07T02:27:07.428 DEBUG:teuthology.orchestra.run.smithi019:> 2024-05-07T02:27:07.428 DEBUG:teuthology.orchestra.run.smithi019:> def _find_admin_socket(client_name): 2024-05-07T02:27:07.428 DEBUG:teuthology.orchestra.run.smithi019:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-05-07T02:27:07.428 DEBUG:teuthology.orchestra.run.smithi019:> files = glob.glob(asok_path) 2024-05-07T02:27:07.428 DEBUG:teuthology.orchestra.run.smithi019:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-05-07T02:27:07.428 DEBUG:teuthology.orchestra.run.smithi019:> 2024-05-07T02:27:07.428 DEBUG:teuthology.orchestra.run.smithi019:> # Given a non-glob path, it better be there 2024-05-07T02:27:07.428 DEBUG:teuthology.orchestra.run.smithi019:> if "*" not in asok_path: 2024-05-07T02:27:07.429 DEBUG:teuthology.orchestra.run.smithi019:> assert(len(files) == 1) 2024-05-07T02:27:07.429 DEBUG:teuthology.orchestra.run.smithi019:> return files[0] 2024-05-07T02:27:07.429 DEBUG:teuthology.orchestra.run.smithi019:> 2024-05-07T02:27:07.429 DEBUG:teuthology.orchestra.run.smithi019:> for f in files: 2024-05-07T02:27:07.429 DEBUG:teuthology.orchestra.run.smithi019:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-07T02:27:07.429 DEBUG:teuthology.orchestra.run.smithi019:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-07T02:27:07.429 DEBUG:teuthology.orchestra.run.smithi019:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-07T02:27:07.429 DEBUG:teuthology.orchestra.run.smithi019:> contents = proc_f.read() 2024-05-07T02:27:07.429 DEBUG:teuthology.orchestra.run.smithi019:> if mountpoint in contents: 2024-05-07T02:27:07.429 DEBUG:teuthology.orchestra.run.smithi019:> return f 2024-05-07T02:27:07.429 DEBUG:teuthology.orchestra.run.smithi019:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-07T02:27:07.429 DEBUG:teuthology.orchestra.run.smithi019:> 2024-05-07T02:27:07.429 DEBUG:teuthology.orchestra.run.smithi019:> print(_find_admin_socket("client.admin")) 2024-05-07T02:27:07.429 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-05-07T02:27:07.631 INFO:teuthology.orchestra.run.smithi019.stdout:/var/run/ceph/ceph-client.admin.75951.asok 2024-05-07T02:27:07.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-07T02:27:07.649 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.75951.asok 2024-05-07T02:27:07.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-07T02:27:07.649 DEBUG:teuthology.orchestra.run.smithi019:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.75951.asok status 2024-05-07T02:27:07.956 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:27:07.956 INFO:teuthology.orchestra.run.smithi019.stdout: "metadata": { 2024-05-07T02:27:07.957 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_sha1": "bd4e00a39d766328b458e4c953f4d722fed96a31", 2024-05-07T02:27:07.957 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_version": "ceph version 19.0.0-3522-gbd4e00a3 (bd4e00a39d766328b458e4c953f4d722fed96a31) squid (dev)", 2024-05-07T02:27:07.957 INFO:teuthology.orchestra.run.smithi019.stdout: "entity_id": "admin", 2024-05-07T02:27:07.957 INFO:teuthology.orchestra.run.smithi019.stdout: "hostname": "smithi019", 2024-05-07T02:27:07.957 INFO:teuthology.orchestra.run.smithi019.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-05-07T02:27:07.957 INFO:teuthology.orchestra.run.smithi019.stdout: "pid": "75951", 2024-05-07T02:27:07.957 INFO:teuthology.orchestra.run.smithi019.stdout: "root": "/" 2024-05-07T02:27:07.957 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:27:07.957 INFO:teuthology.orchestra.run.smithi019.stdout: "dentry_count": 0, 2024-05-07T02:27:07.957 INFO:teuthology.orchestra.run.smithi019.stdout: "dentry_pinned_count": 0, 2024-05-07T02:27:07.957 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24664, 2024-05-07T02:27:07.957 INFO:teuthology.orchestra.run.smithi019.stdout: "inst": { 2024-05-07T02:27:07.957 INFO:teuthology.orchestra.run.smithi019.stdout: "name": { 2024-05-07T02:27:07.957 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "client", 2024-05-07T02:27:07.957 INFO:teuthology.orchestra.run.smithi019.stdout: "num": 24664 2024-05-07T02:27:07.958 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:27:07.958 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": { 2024-05-07T02:27:07.958 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-05-07T02:27:07.958 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "192.168.0.1:0", 2024-05-07T02:27:07.958 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1640522314 2024-05-07T02:27:07.958 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:27:07.958 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:27:07.958 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": { 2024-05-07T02:27:07.958 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-05-07T02:27:07.958 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "192.168.0.1:0", 2024-05-07T02:27:07.958 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1640522314 2024-05-07T02:27:07.958 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:27:07.958 INFO:teuthology.orchestra.run.smithi019.stdout: "inst_str": "client.24664 192.168.0.1:0/1640522314", 2024-05-07T02:27:07.958 INFO:teuthology.orchestra.run.smithi019.stdout: "addr_str": "192.168.0.1:0/1640522314", 2024-05-07T02:27:07.958 INFO:teuthology.orchestra.run.smithi019.stdout: "inode_count": 1, 2024-05-07T02:27:07.958 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_epoch": 17, 2024-05-07T02:27:07.959 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_epoch": 85, 2024-05-07T02:27:07.959 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_epoch_barrier": 0, 2024-05-07T02:27:07.959 INFO:teuthology.orchestra.run.smithi019.stdout: "blocklisted": false, 2024-05-07T02:27:07.959 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs" 2024-05-07T02:27:07.959 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:27:07.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-07T02:27:07.980 DEBUG:teuthology.orchestra.run.smithi019:> stat --file-system '--printf=%T 2024-05-07T02:27:07.980 DEBUG:teuthology.orchestra.run.smithi019:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-05-07T02:27:08.040 INFO:teuthology.orchestra.run.smithi019.stdout:fuseblk 2024-05-07T02:27:08.040 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-05-07T02:27:08.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-07T02:27:08.040 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-05-07T02:27:08.111 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-05-07T02:27:08.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-07T02:27:08.233 INFO:teuthology.orchestra.run.smithi019.stdout:# file: . 2024-05-07T02:27:08.233 INFO:teuthology.orchestra.run.smithi019.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-05-07T02:27:08.233 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:27:08.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-07T02:27:08.243 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-07T02:27:08.243 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-07T02:27:08.294 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph osd blocklist ls 2024-05-07T02:27:08.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:07 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:27:08.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:07 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:27:08.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:07 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:27:08.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:07 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:27:08.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:07 smithi064 ceph-mon[34699]: pgmap v363: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-07T02:27:08.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:07 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:27:08.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:07 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:27:08.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:07 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:27:08.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:07 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:27:08.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:07 smithi019 ceph-mon[29781]: pgmap v363: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-07T02:27:08.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:07 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:27:08.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:07 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:27:08.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:07 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:27:08.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:07 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:27:08.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:07 smithi027 ceph-mon[34398]: pgmap v363: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-07T02:27:08.788 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:27:09.193 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:09 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:27:09.194 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:09 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:27:09.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:09 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:27:09.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:09 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:27:09.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:09 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:27:09.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:09 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:27:10.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:10 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:27:10.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:10 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:27:10.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:10 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:27:10.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:10 smithi019 ceph-mon[29781]: pgmap v364: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-07T02:27:10.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:10 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:27:10.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:10 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:27:10.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:10 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:27:10.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:10 smithi064 ceph-mon[34699]: pgmap v364: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-07T02:27:10.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:10 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:27:10.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:10 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:27:10.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:10 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:27:10.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:10 smithi027 ceph-mon[34398]: pgmap v364: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-07T02:27:11.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.330+0000 7f9357e00700 1 -- 172.21.15.19:0/1820890032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f935810af50 msgr2=0x7f935810b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:11.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.330+0000 7f9357e00700 1 --2- 172.21.15.19:0/1820890032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f935810af50 0x7f935810b360 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9344009990 tx=0x7f9344009ca0 comp rx=0 tx=0).stop 2024-05-07T02:27:11.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.330+0000 7f9357e00700 1 -- 172.21.15.19:0/1820890032 shutdown_connections 2024-05-07T02:27:11.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.330+0000 7f9357e00700 1 --2- 172.21.15.19:0/1820890032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f935810b8a0 0x7f9358113200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:11.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.330+0000 7f9357e00700 1 --2- 172.21.15.19:0/1820890032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9358106620 0x7f9358106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:11.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.330+0000 7f9357e00700 1 --2- 172.21.15.19:0/1820890032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f935810af50 0x7f935810b360 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:11.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.330+0000 7f9357e00700 1 -- 172.21.15.19:0/1820890032 >> 172.21.15.19:0/1820890032 conn(0x7f9358102030 msgr2=0x7f9358104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:27:11.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.330+0000 7f9357e00700 1 -- 172.21.15.19:0/1820890032 shutdown_connections 2024-05-07T02:27:11.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.330+0000 7f9357e00700 1 -- 172.21.15.19:0/1820890032 wait complete. 2024-05-07T02:27:11.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.331+0000 7f9357e00700 1 Processor -- start 2024-05-07T02:27:11.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.331+0000 7f9357e00700 1 -- start start 2024-05-07T02:27:11.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.332+0000 7f9357e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9358106620 0x7f935819ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:27:11.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.332+0000 7f9357e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f935810af50 0x7f935819d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:27:11.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.332+0000 7f9357e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f935810b8a0 0x7f93581a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:27:11.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.332+0000 7f9357e00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9358116500 con 0x7f935810af50 2024-05-07T02:27:11.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.332+0000 7f9357e00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9358116380 con 0x7f9358106620 2024-05-07T02:27:11.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.332+0000 7f9357e00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9358116680 con 0x7f935810b8a0 2024-05-07T02:27:11.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.332+0000 7f9356a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9358106620 0x7f935819ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:27:11.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.332+0000 7f9356000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f935810af50 0x7f935819d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:27:11.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.332+0000 7f9356a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9358106620 0x7f935819ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:34478/0 (socket says 172.21.15.19:34478) 2024-05-07T02:27:11.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.332+0000 7f9356a00700 1 -- 172.21.15.19:0/3817050031 learned_addr learned my addr 172.21.15.19:0/3817050031 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:27:11.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.333+0000 7f9356a00700 1 -- 172.21.15.19:0/3817050031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f935810b8a0 msgr2=0x7f93581a16c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-07T02:27:11.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.333+0000 7f9356a00700 1 --2- 172.21.15.19:0/3817050031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f935810b8a0 0x7f93581a16c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:11.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.333+0000 7f9356a00700 1 -- 172.21.15.19:0/3817050031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f935810af50 msgr2=0x7f935819d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:11.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.333+0000 7f9356a00700 1 --2- 172.21.15.19:0/3817050031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f935810af50 0x7f935819d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:11.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.333+0000 7f9356a00700 1 -- 172.21.15.19:0/3817050031 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93581a1d60 con 0x7f9358106620 2024-05-07T02:27:11.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.333+0000 7f9356a00700 1 --2- 172.21.15.19:0/3817050031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9358106620 0x7f935819ce90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f934800e9a0 tx=0x7f934800ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:27:11.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.333+0000 7f935ca00700 1 -- 172.21.15.19:0/3817050031 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934800cb50 con 0x7f9358106620 2024-05-07T02:27:11.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.333+0000 7f935ca00700 1 -- 172.21.15.19:0/3817050031 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f934800ccf0 con 0x7f9358106620 2024-05-07T02:27:11.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.333+0000 7f9357e00700 1 -- 172.21.15.19:0/3817050031 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93581a2050 con 0x7f9358106620 2024-05-07T02:27:11.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.334+0000 7f935ca00700 1 -- 172.21.15.19:0/3817050031 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9348018680 con 0x7f9358106620 2024-05-07T02:27:11.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.334+0000 7f9357e00700 1 -- 172.21.15.19:0/3817050031 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93581a9970 con 0x7f9358106620 2024-05-07T02:27:11.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.335+0000 7f9357e00700 1 -- 172.21.15.19:0/3817050031 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93580512c0 con 0x7f9358106620 2024-05-07T02:27:11.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.337+0000 7f935ca00700 1 -- 172.21.15.19:0/3817050031 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f9348010430 con 0x7f9358106620 2024-05-07T02:27:11.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.339+0000 7f935ca00700 1 --2- 172.21.15.19:0/3817050031 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f933c0b2900 0x7f933c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:27:11.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.339+0000 7f935ca00700 1 -- 172.21.15.19:0/3817050031 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f93480e2be0 con 0x7f9358106620 2024-05-07T02:27:11.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.339+0000 7f9356000700 1 --2- 172.21.15.19:0/3817050031 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f933c0b2900 0x7f933c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:27:11.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.340+0000 7f9356000700 1 --2- 172.21.15.19:0/3817050031 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f933c0b2900 0x7f933c0b4e10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9344006010 tx=0x7f934400b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:27:11.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.341+0000 7f935ca00700 1 -- 172.21.15.19:0/3817050031 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f93480aa660 con 0x7f9358106620 2024-05-07T02:27:11.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.596+0000 7f9357e00700 1 -- 172.21.15.19:0/3817050031 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f935802e5f0 con 0x7f9358106620 2024-05-07T02:27:11.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.597+0000 7f935ca00700 1 -- 172.21.15.19:0/3817050031 <== mon.2 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v85) ==== 81+0+589 (secure 0 0 0) 0x7f93480a9f60 con 0x7f9358106620 2024-05-07T02:27:11.599 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/1956516258 2024-05-08T02:17:31.252868+0000 2024-05-07T02:27:11.599 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/1816772181 2024-05-08T02:17:31.252868+0000 2024-05-07T02:27:11.599 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/174304059 2024-05-08T02:17:31.252868+0000 2024-05-07T02:27:11.599 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/1514051750 2024-05-08T02:16:54.732592+0000 2024-05-07T02:27:11.599 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/174304059 2024-05-08T02:17:31.252868+0000 2024-05-07T02:27:11.599 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/3930045871 2024-05-08T02:16:54.732592+0000 2024-05-07T02:27:11.599 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/3614957309 2024-05-08T02:16:54.732592+0000 2024-05-07T02:27:11.599 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/1328365399 2024-05-08T02:17:31.252868+0000 2024-05-07T02:27:11.599 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/139593344 2024-05-08T02:16:54.732592+0000 2024-05-07T02:27:11.599 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/3614957309 2024-05-08T02:16:54.732592+0000 2024-05-07T02:27:11.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.601+0000 7f9357e00700 1 -- 172.21.15.19:0/3817050031 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f933c0b2900 msgr2=0x7f933c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:11.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.601+0000 7f9357e00700 1 --2- 172.21.15.19:0/3817050031 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f933c0b2900 0x7f933c0b4e10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9344006010 tx=0x7f934400b610 comp rx=0 tx=0).stop 2024-05-07T02:27:11.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.601+0000 7f9357e00700 1 -- 172.21.15.19:0/3817050031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9358106620 msgr2=0x7f935819ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:11.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.601+0000 7f9357e00700 1 --2- 172.21.15.19:0/3817050031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9358106620 0x7f935819ce90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f934800e9a0 tx=0x7f934800ed60 comp rx=0 tx=0).stop 2024-05-07T02:27:11.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.601+0000 7f9357e00700 1 -- 172.21.15.19:0/3817050031 shutdown_connections 2024-05-07T02:27:11.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.601+0000 7f9357e00700 1 --2- 172.21.15.19:0/3817050031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f935810b8a0 0x7f93581a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:11.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.601+0000 7f9357e00700 1 --2- 172.21.15.19:0/3817050031 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f933c0b2900 0x7f933c0b4e10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:11.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.601+0000 7f9357e00700 1 --2- 172.21.15.19:0/3817050031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9358106620 0x7f935819ce90 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:11.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.601+0000 7f9357e00700 1 --2- 172.21.15.19:0/3817050031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f935810af50 0x7f935819d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:11.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.601+0000 7f9357e00700 1 -- 172.21.15.19:0/3817050031 >> 172.21.15.19:0/3817050031 conn(0x7f9358102030 msgr2=0x7f935810a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:27:11.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.601+0000 7f9357e00700 1 -- 172.21.15.19:0/3817050031 shutdown_connections 2024-05-07T02:27:11.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:11.601+0000 7f9357e00700 1 -- 172.21.15.19:0/3817050031 wait complete. 2024-05-07T02:27:11.606 INFO:teuthology.orchestra.run.smithi019.stderr:listed 10 entries 2024-05-07T02:27:12.583 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-07T02:27:12.583 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-07T02:27:12.642 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph osd blocklist ls 2024-05-07T02:27:12.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:12 smithi064 ceph-mon[34699]: pgmap v365: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-07T02:27:12.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:12 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/3817050031' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-07T02:27:12.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:12 smithi019 ceph-mon[29781]: pgmap v365: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-07T02:27:12.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:12 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/3817050031' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-07T02:27:12.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:12 smithi027 ceph-mon[34398]: pgmap v365: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-07T02:27:12.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:12 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/3817050031' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-07T02:27:13.111 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:27:14.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:14 smithi064 ceph-mon[34699]: pgmap v366: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-07T02:27:14.884 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:14 smithi019 ceph-mon[29781]: pgmap v366: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-07T02:27:14.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:14 smithi027 ceph-mon[34398]: pgmap v366: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-07T02:27:15.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.500+0000 7f21c7200700 1 -- 172.21.15.19:0/752489823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21c0108d50 msgr2=0x7f21c0109160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:15.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.500+0000 7f21c7200700 1 --2- 172.21.15.19:0/752489823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21c0108d50 0x7f21c0109160 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f21ac0099c0 tx=0x7f21ac009cd0 comp rx=0 tx=0).stop 2024-05-07T02:27:15.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.500+0000 7f21c7200700 1 -- 172.21.15.19:0/752489823 shutdown_connections 2024-05-07T02:27:15.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.500+0000 7f21c7200700 1 --2- 172.21.15.19:0/752489823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c01096a0 0x7f21c0111000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:15.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.500+0000 7f21c7200700 1 --2- 172.21.15.19:0/752489823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21c0104420 0x7f21c0104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:15.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.500+0000 7f21c7200700 1 --2- 172.21.15.19:0/752489823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21c0108d50 0x7f21c0109160 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:15.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.500+0000 7f21c7200700 1 -- 172.21.15.19:0/752489823 >> 172.21.15.19:0/752489823 conn(0x7f21c007acb0 msgr2=0x7f21c007b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:27:15.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.500+0000 7f21c7200700 1 -- 172.21.15.19:0/752489823 shutdown_connections 2024-05-07T02:27:15.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.501+0000 7f21c7200700 1 -- 172.21.15.19:0/752489823 wait complete. 2024-05-07T02:27:15.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.502+0000 7f21c7200700 1 Processor -- start 2024-05-07T02:27:15.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.502+0000 7f21c7200700 1 -- start start 2024-05-07T02:27:15.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.503+0000 7f21c7200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c0104420 0x7f21c019cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:27:15.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.503+0000 7f21c7200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21c0108d50 0x7f21c019d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:27:15.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.503+0000 7f21c7200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21c01096a0 0x7f21c01a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:27:15.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.503+0000 7f21c7200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f21c01142d0 con 0x7f21c0108d50 2024-05-07T02:27:15.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.503+0000 7f21c7200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f21c0114150 con 0x7f21c0104420 2024-05-07T02:27:15.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.503+0000 7f21c7200700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f21c0114450 con 0x7f21c01096a0 2024-05-07T02:27:15.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.503+0000 7f21c5400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21c0108d50 0x7f21c019d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:27:15.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.503+0000 7f21c5e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c0104420 0x7f21c019cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:27:15.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.503+0000 7f21c5400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21c0108d50 0x7f21c019d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35386/0 (socket says 172.21.15.19:35386) 2024-05-07T02:27:15.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.503+0000 7f21c5400700 1 -- 172.21.15.19:0/764398156 learned_addr learned my addr 172.21.15.19:0/764398156 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:27:15.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.503+0000 7f21c6800700 1 --2- 172.21.15.19:0/764398156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21c01096a0 0x7f21c01a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:27:15.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.504+0000 7f21c5e00700 1 -- 172.21.15.19:0/764398156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21c01096a0 msgr2=0x7f21c01a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:15.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.504+0000 7f21c5e00700 1 --2- 172.21.15.19:0/764398156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21c01096a0 0x7f21c01a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:15.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.504+0000 7f21c5e00700 1 -- 172.21.15.19:0/764398156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21c0108d50 msgr2=0x7f21c019d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:15.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.504+0000 7f21c5e00700 1 --2- 172.21.15.19:0/764398156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21c0108d50 0x7f21c019d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:15.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.504+0000 7f21c5e00700 1 -- 172.21.15.19:0/764398156 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21c01a1d70 con 0x7f21c0104420 2024-05-07T02:27:15.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.504+0000 7f21c5400700 1 --2- 172.21.15.19:0/764398156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21c0108d50 0x7f21c019d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:27:15.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.504+0000 7f21c5e00700 1 --2- 172.21.15.19:0/764398156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c0104420 0x7f21c019cea0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f21b400d750 tx=0x7f21b400da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:27:15.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.504+0000 7f21baa00700 1 -- 172.21.15.19:0/764398156 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b4016070 con 0x7f21c0104420 2024-05-07T02:27:15.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.504+0000 7f21baa00700 1 -- 172.21.15.19:0/764398156 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21b4005020 con 0x7f21c0104420 2024-05-07T02:27:15.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.504+0000 7f21c7200700 1 -- 172.21.15.19:0/764398156 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21c01a2060 con 0x7f21c0104420 2024-05-07T02:27:15.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.504+0000 7f21baa00700 1 -- 172.21.15.19:0/764398156 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b400fa10 con 0x7f21c0104420 2024-05-07T02:27:15.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.504+0000 7f21c7200700 1 -- 172.21.15.19:0/764398156 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21c01a9980 con 0x7f21c0104420 2024-05-07T02:27:15.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.505+0000 7f21c7200700 1 -- 172.21.15.19:0/764398156 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21c00512c0 con 0x7f21c0104420 2024-05-07T02:27:15.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.508+0000 7f21baa00700 1 -- 172.21.15.19:0/764398156 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f21b4004a30 con 0x7f21c0104420 2024-05-07T02:27:15.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.510+0000 7f21baa00700 1 --2- 172.21.15.19:0/764398156 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f21a80b2880 0x7f21a80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:27:15.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.510+0000 7f21baa00700 1 -- 172.21.15.19:0/764398156 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f21b40e23c0 con 0x7f21c0104420 2024-05-07T02:27:15.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.510+0000 7f21c5400700 1 --2- 172.21.15.19:0/764398156 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f21a80b2880 0x7f21a80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:27:15.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.511+0000 7f21c5400700 1 --2- 172.21.15.19:0/764398156 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f21a80b2880 0x7f21a80b4d90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f21c01100d0 tx=0x7f21ac005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:27:15.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.512+0000 7f21baa00700 1 -- 172.21.15.19:0/764398156 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f21b40a9e40 con 0x7f21c0104420 2024-05-07T02:27:15.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.772+0000 7f21c7200700 1 -- 172.21.15.19:0/764398156 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f21c01a9b50 con 0x7f21c0104420 2024-05-07T02:27:15.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.773+0000 7f21baa00700 1 -- 172.21.15.19:0/764398156 <== mon.2 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v85) ==== 81+0+589 (secure 0 0 0) 0x7f21b40a9740 con 0x7f21c0104420 2024-05-07T02:27:15.774 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/1956516258 2024-05-08T02:17:31.252868+0000 2024-05-07T02:27:15.774 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/1816772181 2024-05-08T02:17:31.252868+0000 2024-05-07T02:27:15.774 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/174304059 2024-05-08T02:17:31.252868+0000 2024-05-07T02:27:15.774 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/1514051750 2024-05-08T02:16:54.732592+0000 2024-05-07T02:27:15.775 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/174304059 2024-05-08T02:17:31.252868+0000 2024-05-07T02:27:15.775 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/3930045871 2024-05-08T02:16:54.732592+0000 2024-05-07T02:27:15.775 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/3614957309 2024-05-08T02:16:54.732592+0000 2024-05-07T02:27:15.775 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/1328365399 2024-05-08T02:17:31.252868+0000 2024-05-07T02:27:15.775 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/139593344 2024-05-08T02:16:54.732592+0000 2024-05-07T02:27:15.775 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/3614957309 2024-05-08T02:16:54.732592+0000 2024-05-07T02:27:15.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.776+0000 7f21c7200700 1 -- 172.21.15.19:0/764398156 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f21a80b2880 msgr2=0x7f21a80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:15.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.776+0000 7f21c7200700 1 --2- 172.21.15.19:0/764398156 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f21a80b2880 0x7f21a80b4d90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f21c01100d0 tx=0x7f21ac005fb0 comp rx=0 tx=0).stop 2024-05-07T02:27:15.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.776+0000 7f21c7200700 1 -- 172.21.15.19:0/764398156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c0104420 msgr2=0x7f21c019cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:15.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.776+0000 7f21c7200700 1 --2- 172.21.15.19:0/764398156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c0104420 0x7f21c019cea0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f21b400d750 tx=0x7f21b400da60 comp rx=0 tx=0).stop 2024-05-07T02:27:15.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.777+0000 7f21c7200700 1 -- 172.21.15.19:0/764398156 shutdown_connections 2024-05-07T02:27:15.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.777+0000 7f21c7200700 1 --2- 172.21.15.19:0/764398156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21c01096a0 0x7f21c01a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:15.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.777+0000 7f21c7200700 1 --2- 172.21.15.19:0/764398156 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f21a80b2880 0x7f21a80b4d90 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:15.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.777+0000 7f21c7200700 1 --2- 172.21.15.19:0/764398156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c0104420 0x7f21c019cea0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:15.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.777+0000 7f21c7200700 1 --2- 172.21.15.19:0/764398156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21c0108d50 0x7f21c019d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:15.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.777+0000 7f21c7200700 1 -- 172.21.15.19:0/764398156 >> 172.21.15.19:0/764398156 conn(0x7f21c007acb0 msgr2=0x7f21c0110cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:27:15.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.778+0000 7f21c7200700 1 -- 172.21.15.19:0/764398156 shutdown_connections 2024-05-07T02:27:15.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:15.778+0000 7f21c7200700 1 -- 172.21.15.19:0/764398156 wait complete. 2024-05-07T02:27:15.780 INFO:teuthology.orchestra.run.smithi019.stderr:listed 10 entries 2024-05-07T02:27:16.761 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi019.front.sepia.ceph.com... 2024-05-07T02:27:16.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-07T02:27:16.761 DEBUG:teuthology.orchestra.run.smithi019:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-05-07T02:27:16.844 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:16 smithi019 ceph-mon[29781]: pgmap v367: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-07T02:27:16.844 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:16 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/764398156' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-07T02:27:16.855 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi019.stderr:ceph-fuse[75951]: fuse finished with error 0 and tester_r 0 2024-05-07T02:27:16.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:16 smithi064 ceph-mon[34699]: pgmap v367: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-07T02:27:16.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:16 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/764398156' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-07T02:27:16.861 INFO:teuthology.orchestra.run:waiting for 300 2024-05-07T02:27:16.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:16 smithi027 ceph-mon[34398]: pgmap v367: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-07T02:27:16.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:16 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/764398156' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-07T02:27:18.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:18 smithi064 ceph-mon[34699]: pgmap v368: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-07T02:27:18.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:18 smithi019 ceph-mon[29781]: pgmap v368: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-07T02:27:18.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:18 smithi027 ceph-mon[34398]: pgmap v368: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-07T02:27:20.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:20 smithi064 ceph-mon[34699]: pgmap v369: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-07T02:27:20.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:20 smithi019 ceph-mon[29781]: pgmap v369: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-07T02:27:20.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:20 smithi027 ceph-mon[34398]: pgmap v369: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-05-07T02:27:22.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:22 smithi064 ceph-mon[34699]: pgmap v370: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-07T02:27:22.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:22 smithi019 ceph-mon[29781]: pgmap v370: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-07T02:27:22.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:22 smithi027 ceph-mon[34398]: pgmap v370: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-05-07T02:27:22.962 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-05-07T02:27:22.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-07T02:27:22.962 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-05-07T02:27:23.034 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-07T02:27:23.487 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:27:24.675 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:24 smithi019 ceph-mon[29781]: pgmap v371: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-07T02:27:24.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:24 smithi064 ceph-mon[34699]: pgmap v371: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-07T02:27:24.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:24 smithi027 ceph-mon[34398]: pgmap v371: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-07T02:27:26.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.052+0000 7fc602200700 1 -- 172.21.15.19:0/446406112 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5fc105ea0 msgr2=0x7fc5fc10e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:26.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.052+0000 7fc602200700 1 --2- 172.21.15.19:0/446406112 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5fc105ea0 0x7fc5fc10e3f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4009990 tx=0x7fc5e4009ca0 comp rx=0 tx=0).stop 2024-05-07T02:27:26.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.052+0000 7fc602200700 1 -- 172.21.15.19:0/446406112 shutdown_connections 2024-05-07T02:27:26.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.052+0000 7fc602200700 1 --2- 172.21.15.19:0/446406112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5fc10e930 0x7fc5fc110e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:26.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.052+0000 7fc602200700 1 --2- 172.21.15.19:0/446406112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5fc105550 0x7fc5fc105960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:26.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.052+0000 7fc602200700 1 --2- 172.21.15.19:0/446406112 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5fc105ea0 0x7fc5fc10e3f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:26.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.052+0000 7fc602200700 1 -- 172.21.15.19:0/446406112 >> 172.21.15.19:0/446406112 conn(0x7fc5fc078e00 msgr2=0x7fc5fc079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:27:26.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.052+0000 7fc602200700 1 -- 172.21.15.19:0/446406112 shutdown_connections 2024-05-07T02:27:26.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.053+0000 7fc602200700 1 -- 172.21.15.19:0/446406112 wait complete. 2024-05-07T02:27:26.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.055+0000 7fc602200700 1 Processor -- start 2024-05-07T02:27:26.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.055+0000 7fc602200700 1 -- start start 2024-05-07T02:27:26.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.056+0000 7fc602200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5fc105550 0x7fc5fc19ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:27:26.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.056+0000 7fc602200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5fc105ea0 0x7fc5fc19b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:27:26.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.056+0000 7fc602200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5fc10e930 0x7fc5fc19f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:27:26.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.056+0000 7fc602200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc5fc1144d0 con 0x7fc5fc105ea0 2024-05-07T02:27:26.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.056+0000 7fc602200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc5fc114350 con 0x7fc5fc105550 2024-05-07T02:27:26.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.056+0000 7fc602200700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc5fc114650 con 0x7fc5fc10e930 2024-05-07T02:27:26.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.056+0000 7fc5f3e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5fc105ea0 0x7fc5fc19b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:27:26.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.056+0000 7fc5fb400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5fc105550 0x7fc5fc19ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:27:26.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.056+0000 7fc5f3e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5fc105ea0 0x7fc5fc19b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51976/0 (socket says 172.21.15.19:51976) 2024-05-07T02:27:26.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.056+0000 7fc5f3e00700 1 -- 172.21.15.19:0/1237769731 learned_addr learned my addr 172.21.15.19:0/1237769731 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:27:26.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.056+0000 7fc5fbe00700 1 --2- 172.21.15.19:0/1237769731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5fc10e930 0x7fc5fc19f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:27:26.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.057+0000 7fc5fb400700 1 -- 172.21.15.19:0/1237769731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5fc10e930 msgr2=0x7fc5fc19f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:26.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.057+0000 7fc5fb400700 1 --2- 172.21.15.19:0/1237769731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5fc10e930 0x7fc5fc19f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:26.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.057+0000 7fc5fb400700 1 -- 172.21.15.19:0/1237769731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5fc105ea0 msgr2=0x7fc5fc19b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:26.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.057+0000 7fc5fb400700 1 --2- 172.21.15.19:0/1237769731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5fc105ea0 0x7fc5fc19b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:26.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.057+0000 7fc5fb400700 1 -- 172.21.15.19:0/1237769731 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5fc19fd10 con 0x7fc5fc105550 2024-05-07T02:27:26.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.057+0000 7fc5fb400700 1 --2- 172.21.15.19:0/1237769731 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5fc105550 0x7fc5fc19ae40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec00d950 tx=0x7fc5ec00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:27:26.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.057+0000 7fc5f8c00700 1 -- 172.21.15.19:0/1237769731 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec017070 con 0x7fc5fc105550 2024-05-07T02:27:26.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.058+0000 7fc5f8c00700 1 -- 172.21.15.19:0/1237769731 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5ec009de0 con 0x7fc5fc105550 2024-05-07T02:27:26.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.058+0000 7fc602200700 1 -- 172.21.15.19:0/1237769731 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5fc1a0000 con 0x7fc5fc105550 2024-05-07T02:27:26.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.058+0000 7fc5f8c00700 1 -- 172.21.15.19:0/1237769731 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec010e20 con 0x7fc5fc105550 2024-05-07T02:27:26.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.058+0000 7fc602200700 1 -- 172.21.15.19:0/1237769731 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5fc1a7920 con 0x7fc5fc105550 2024-05-07T02:27:26.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.059+0000 7fc602200700 1 -- 172.21.15.19:0/1237769731 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5fc0512c0 con 0x7fc5fc105550 2024-05-07T02:27:26.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.061+0000 7fc5f8c00700 1 -- 172.21.15.19:0/1237769731 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc5ec01a070 con 0x7fc5fc105550 2024-05-07T02:27:26.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.063+0000 7fc5f8c00700 1 --2- 172.21.15.19:0/1237769731 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fc5e00b2900 0x7fc5e00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:27:26.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.063+0000 7fc5f8c00700 1 -- 172.21.15.19:0/1237769731 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7fc5ec0e1db0 con 0x7fc5fc105550 2024-05-07T02:27:26.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.063+0000 7fc5f3e00700 1 --2- 172.21.15.19:0/1237769731 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fc5e00b2900 0x7fc5e00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:27:26.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.064+0000 7fc5f3e00700 1 --2- 172.21.15.19:0/1237769731 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fc5e00b2900 0x7fc5e00b4e10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc0798e0 tx=0x7fc5e400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:27:26.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.065+0000 7fc5f8c00700 1 -- 172.21.15.19:0/1237769731 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc5ec0a98b0 con 0x7fc5fc105550 2024-05-07T02:27:26.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.304+0000 7fc602200700 1 -- 172.21.15.19:0/1237769731 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc5fc02e5f0 con 0x7fc5fc105550 2024-05-07T02:27:26.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.306+0000 7fc5f8c00700 1 -- 172.21.15.19:0/1237769731 <== mon.2 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24055 (secure 0 0 0) 0x7fc5ec0a91b0 con 0x7fc5fc105550 2024-05-07T02:27:26.308 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:27:26.308 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":85,"fsid":"9ff84c16-0c17-11ef-bc97-c7b262605968","created":"2024-05-07T02:16:19.507657+0000","modified":"2024-05-07T02:26:32.693063+0000","last_up_change":"2024-05-07T02:24:55.025124+0000","last_in_change":"2024-05-07T02:24:33.842422+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-07T02:20:21.439697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-07T02:26:06.178814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-07T02:26:10.705693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-07T02:26:23.534133+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b437cf22-ec8e-48f0-86a2-2413a0ab40dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6803","nonce":2390539683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6805","nonce":2390539683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6809","nonce":2390539683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6807","nonce":2390539683}]},"public_addr":"172.21.15.19:6803/2390539683","cluster_addr":"172.21.15.19:6805/2390539683","heartbeat_back_addr":"172.21.15.19:6809/2390539683","heartbeat_front_addr":"172.21.15.19:6807/2390539683","state":["exists","up"]},{"osd":1,"uuid":"dc753e16-634d-4c14-abd2-935281e1499e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6801","nonce":4292470615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6803","nonce":4292470615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6807","nonce":4292470615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6805","nonce":4292470615}]},"public_addr":"172.21.15.27:6801/4292470615","cluster_addr":"172.21.15.27:6803/4292470615","heartbeat_back_addr":"172.21.15.27:6807/4292470615","heartbeat_front_addr":"172.21.15.27:6805/4292470615","state":["exists","up"]},{"osd":2,"uuid":"40570071-646e-4e66-9a16-c55ab743ac86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6801","nonce":1180552939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6803","nonce":1180552939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6807","nonce":1180552939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6805","nonce":1180552939}]},"public_addr":"172.21.15.64:6801/1180552939","cluster_addr":"172.21.15.64:6803/1180552939","heartbeat_back_addr":"172.21.15.64:6807/1180552939","heartbeat_front_addr":"172.21.15.64:6805/1180552939","state":["exists","up"]},{"osd":3,"uuid":"ceaa4252-4dfa-427a-9b1a-76adf3b8e609","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6811","nonce":387118141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6813","nonce":387118141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6817","nonce":387118141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6815","nonce":387118141}]},"public_addr":"172.21.15.19:6811/387118141","cluster_addr":"172.21.15.19:6813/387118141","heartbeat_back_addr":"172.21.15.19:6817/387118141","heartbeat_front_addr":"172.21.15.19:6815/387118141","state":["exists","up"]},{"osd":4,"uuid":"5a6d1ebc-62ca-444e-8f29-1980418d0a8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6809","nonce":3662186068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6811","nonce":3662186068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6815","nonce":3662186068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6813","nonce":3662186068}]},"public_addr":"172.21.15.27:6809/3662186068","cluster_addr":"172.21.15.27:6811/3662186068","heartbeat_back_addr":"172.21.15.27:6815/3662186068","heartbeat_front_addr":"172.21.15.27:6813/3662186068","state":["exists","up"]},{"osd":5,"uuid":"60c207e9-55e8-4bed-87c3-f6a06a2a1fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6809","nonce":1264591849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6811","nonce":1264591849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6815","nonce":1264591849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6813","nonce":1264591849}]},"public_addr":"172.21.15.64:6809/1264591849","cluster_addr":"172.21.15.64:6811/1264591849","heartbeat_back_addr":"172.21.15.64:6815/1264591849","heartbeat_front_addr":"172.21.15.64:6813/1264591849","state":["exists","up"]},{"osd":6,"uuid":"f3e038e0-4026-45f1-a923-c07cbb307f64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6819","nonce":2158875543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6821","nonce":2158875543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6825","nonce":2158875543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6823","nonce":2158875543}]},"public_addr":"172.21.15.19:6819/2158875543","cluster_addr":"172.21.15.19:6821/2158875543","heartbeat_back_addr":"172.21.15.19:6825/2158875543","heartbeat_front_addr":"172.21.15.19:6823/2158875543","state":["exists","up"]},{"osd":7,"uuid":"d69e34d4-2028-43b8-8ada-ff4a9b579620","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6817","nonce":2756612091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6819","nonce":2756612091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6823","nonce":2756612091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6821","nonce":2756612091}]},"public_addr":"172.21.15.27:6817/2756612091","cluster_addr":"172.21.15.27:6819/2756612091","heartbeat_back_addr":"172.21.15.27:6823/2756612091","heartbeat_front_addr":"172.21.15.27:6821/2756612091","state":["exists","up"]},{"osd":8,"uuid":"6c1a4179-75b6-42ff-ac29-378801e6f8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6817","nonce":1987024836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6819","nonce":1987024836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6823","nonce":1987024836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6821","nonce":1987024836}]},"public_addr":"172.21.15.64:6817/1987024836","cluster_addr":"172.21.15.64:6819/1987024836","heartbeat_back_addr":"172.21.15.64:6823/1987024836","heartbeat_front_addr":"172.21.15.64:6821/1987024836","state":["exists","up"]},{"osd":9,"uuid":"9df7ff66-d39f-4709-8324-74cf9e0e052d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6827","nonce":4124921646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6829","nonce":4124921646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6833","nonce":4124921646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6831","nonce":4124921646}]},"public_addr":"172.21.15.19:6827/4124921646","cluster_addr":"172.21.15.19:6829/4124921646","heartbeat_back_addr":"172.21.15.19:6833/4124921646","heartbeat_front_addr":"172.21.15.19:6831/4124921646","state":["exists","up"]},{"osd":10,"uuid":"71eaa9a7-667d-4cb7-935c-42177d20a279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6825","nonce":1428931817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6827","nonce":1428931817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6831","nonce":1428931817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6829","nonce":1428931817}]},"public_addr":"172.21.15.27:6825/1428931817","cluster_addr":"172.21.15.27:6827/1428931817","heartbeat_back_addr":"172.21.15.27:6831/1428931817","heartbeat_front_addr":"172.21.15.27:6829/1428931817","state":["exists","up"]},{"osd":11,"uuid":"3330627f-ab5d-4e6e-83cf-fc8b74c54ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6825","nonce":4045070679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6827","nonce":4045070679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6831","nonce":4045070679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6829","nonce":4045070679}]},"public_addr":"172.21.15.64:6825/4045070679","cluster_addr":"172.21.15.64:6827/4045070679","heartbeat_back_addr":"172.21.15.64:6831/4045070679","heartbeat_front_addr":"172.21.15.64:6829/4045070679","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:19:27.902633+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:19:54.305685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:20:18.727053+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:20:53.579199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:21:21.058018+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:21:47.091494+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:22:23.269018+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:22:51.863628+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:23:18.537774+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:23:55.763993+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:24:24.956178+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:24:52.681201+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/1956516258":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/1816772181":"2024-05-08T02:17:31.252868+0000","172.21.15.19:6801/174304059":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/1514051750":"2024-05-08T02:16:54.732592+0000","172.21.15.19:6800/174304059":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/3930045871":"2024-05-08T02:16:54.732592+0000","172.21.15.19:6800/3614957309":"2024-05-08T02:16:54.732592+0000","172.21.15.19:0/1328365399":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/139593344":"2024-05-08T02:16:54.732592+0000","172.21.15.19:6801/3614957309":"2024-05-08T02:16:54.732592+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-07T02:27:26.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.311+0000 7fc602200700 1 -- 172.21.15.19:0/1237769731 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fc5e00b2900 msgr2=0x7fc5e00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:26.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.311+0000 7fc602200700 1 --2- 172.21.15.19:0/1237769731 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fc5e00b2900 0x7fc5e00b4e10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc0798e0 tx=0x7fc5e400b260 comp rx=0 tx=0).stop 2024-05-07T02:27:26.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.311+0000 7fc602200700 1 -- 172.21.15.19:0/1237769731 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5fc105550 msgr2=0x7fc5fc19ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:26.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.311+0000 7fc602200700 1 --2- 172.21.15.19:0/1237769731 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5fc105550 0x7fc5fc19ae40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec00d950 tx=0x7fc5ec00dc60 comp rx=0 tx=0).stop 2024-05-07T02:27:26.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.311+0000 7fc602200700 1 -- 172.21.15.19:0/1237769731 shutdown_connections 2024-05-07T02:27:26.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.311+0000 7fc602200700 1 --2- 172.21.15.19:0/1237769731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5fc10e930 0x7fc5fc19f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:26.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.311+0000 7fc602200700 1 --2- 172.21.15.19:0/1237769731 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fc5e00b2900 0x7fc5e00b4e10 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:26.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.311+0000 7fc602200700 1 --2- 172.21.15.19:0/1237769731 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5fc105550 0x7fc5fc19ae40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:26.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.311+0000 7fc602200700 1 --2- 172.21.15.19:0/1237769731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5fc105ea0 0x7fc5fc19b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:26.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.311+0000 7fc602200700 1 -- 172.21.15.19:0/1237769731 >> 172.21.15.19:0/1237769731 conn(0x7fc5fc078e00 msgr2=0x7fc5fc1000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:27:26.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.311+0000 7fc602200700 1 -- 172.21.15.19:0/1237769731 shutdown_connections 2024-05-07T02:27:26.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:26.311+0000 7fc602200700 1 -- 172.21.15.19:0/1237769731 wait complete. 2024-05-07T02:27:26.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:26 smithi019 ceph-mon[29781]: pgmap v372: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-07T02:27:26.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:26 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/1237769731' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-07T02:27:26.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:26 smithi064 ceph-mon[34699]: pgmap v372: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-07T02:27:26.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:26 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/1237769731' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-07T02:27:26.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:26 smithi027 ceph-mon[34398]: pgmap v372: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-07T02:27:26.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:26 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/1237769731' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-07T02:27:27.314 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-07T02:27:27.760 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:27:28.836 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:28 smithi019 ceph-mon[29781]: pgmap v373: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-07T02:27:28.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:28 smithi064 ceph-mon[34699]: pgmap v373: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-07T02:27:28.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:28 smithi027 ceph-mon[34398]: pgmap v373: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-07T02:27:30.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.327+0000 7fda14a00700 1 -- 172.21.15.19:0/3101996447 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda10109680 msgr2=0x7fda10110fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:30.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.327+0000 7fda14a00700 1 --2- 172.21.15.19:0/3101996447 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda10109680 0x7fda10110fe0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fda0800b0a0 tx=0x7fda0800b3b0 comp rx=0 tx=0).stop 2024-05-07T02:27:30.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.327+0000 7fda14a00700 1 -- 172.21.15.19:0/3101996447 shutdown_connections 2024-05-07T02:27:30.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.327+0000 7fda14a00700 1 --2- 172.21.15.19:0/3101996447 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda10109680 0x7fda10110fe0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:30.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.327+0000 7fda14a00700 1 --2- 172.21.15.19:0/3101996447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda10104400 0x7fda10104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:30.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.327+0000 7fda14a00700 1 --2- 172.21.15.19:0/3101996447 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda10108d30 0x7fda10109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:30.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.327+0000 7fda14a00700 1 -- 172.21.15.19:0/3101996447 >> 172.21.15.19:0/3101996447 conn(0x7fda10079d10 msgr2=0x7fda1007a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:27:30.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.327+0000 7fda14a00700 1 -- 172.21.15.19:0/3101996447 shutdown_connections 2024-05-07T02:27:30.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.327+0000 7fda14a00700 1 -- 172.21.15.19:0/3101996447 wait complete. 2024-05-07T02:27:30.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.328+0000 7fda14a00700 1 Processor -- start 2024-05-07T02:27:30.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.328+0000 7fda14a00700 1 -- start start 2024-05-07T02:27:30.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.329+0000 7fda14a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda10104400 0x7fda1019ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:27:30.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.329+0000 7fda14a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda10108d30 0x7fda1019d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:27:30.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.329+0000 7fda14a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda10109680 0x7fda101a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:27:30.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.329+0000 7fda14a00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fda10114300 con 0x7fda10104400 2024-05-07T02:27:30.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.329+0000 7fda14a00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fda10114180 con 0x7fda10109680 2024-05-07T02:27:30.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.329+0000 7fda14a00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fda10114480 con 0x7fda10108d30 2024-05-07T02:27:30.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.329+0000 7fda0f400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda10104400 0x7fda1019ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:27:30.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.329+0000 7fda0ea00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda10108d30 0x7fda1019d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:27:30.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.329+0000 7fda0ea00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda10108d30 0x7fda1019d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:33858/0 (socket says 172.21.15.19:33858) 2024-05-07T02:27:30.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.329+0000 7fda0ea00700 1 -- 172.21.15.19:0/1633938754 learned_addr learned my addr 172.21.15.19:0/1633938754 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:27:30.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.330+0000 7fda0fe00700 1 --2- 172.21.15.19:0/1633938754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda10109680 0x7fda101a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:27:30.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.330+0000 7fda0ea00700 1 -- 172.21.15.19:0/1633938754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda10109680 msgr2=0x7fda101a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:30.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.330+0000 7fda0ea00700 1 --2- 172.21.15.19:0/1633938754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda10109680 0x7fda101a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:30.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.330+0000 7fda0ea00700 1 -- 172.21.15.19:0/1633938754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda10104400 msgr2=0x7fda1019ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:30.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.330+0000 7fda0ea00700 1 --2- 172.21.15.19:0/1633938754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda10104400 0x7fda1019ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:30.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.330+0000 7fda0ea00700 1 -- 172.21.15.19:0/1633938754 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda101a1d20 con 0x7fda10108d30 2024-05-07T02:27:30.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.330+0000 7fda0ea00700 1 --2- 172.21.15.19:0/1633938754 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda10108d30 0x7fda1019d3f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc00ea30 tx=0x7fd9fc00ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:27:30.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.331+0000 7fda07e00700 1 -- 172.21.15.19:0/1633938754 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc00cbd0 con 0x7fda10108d30 2024-05-07T02:27:30.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.331+0000 7fda14a00700 1 -- 172.21.15.19:0/1633938754 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda101a2010 con 0x7fda10108d30 2024-05-07T02:27:30.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.331+0000 7fda07e00700 1 -- 172.21.15.19:0/1633938754 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9fc0044d0 con 0x7fda10108d30 2024-05-07T02:27:30.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.331+0000 7fda14a00700 1 -- 172.21.15.19:0/1633938754 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda101a9930 con 0x7fda10108d30 2024-05-07T02:27:30.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.331+0000 7fda07e00700 1 -- 172.21.15.19:0/1633938754 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc003dc0 con 0x7fda10108d30 2024-05-07T02:27:30.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.332+0000 7fda14a00700 1 -- 172.21.15.19:0/1633938754 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda101058b0 con 0x7fda10108d30 2024-05-07T02:27:30.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.336+0000 7fda07e00700 1 -- 172.21.15.19:0/1633938754 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fd9fc004020 con 0x7fda10108d30 2024-05-07T02:27:30.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.337+0000 7fda07e00700 1 --2- 172.21.15.19:0/1633938754 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fd9f40b2880 0x7fd9f40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:27:30.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.337+0000 7fda07e00700 1 -- 172.21.15.19:0/1633938754 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7fd9fc0e25e0 con 0x7fda10108d30 2024-05-07T02:27:30.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.337+0000 7fda0f400700 1 --2- 172.21.15.19:0/1633938754 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fd9f40b2880 0x7fd9f40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:27:30.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.338+0000 7fda0f400700 1 --2- 172.21.15.19:0/1633938754 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fd9f40b2880 0x7fd9f40b4d90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fda00005950 tx=0x7fda0000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:27:30.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.338+0000 7fda07e00700 1 -- 172.21.15.19:0/1633938754 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd9fc0aa0f0 con 0x7fda10108d30 2024-05-07T02:27:30.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.577+0000 7fda14a00700 1 -- 172.21.15.19:0/1633938754 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fda101a9b00 con 0x7fda10108d30 2024-05-07T02:27:30.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.580+0000 7fda07e00700 1 -- 172.21.15.19:0/1633938754 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24055 (secure 0 0 0) 0x7fd9fc0a99f0 con 0x7fda10108d30 2024-05-07T02:27:30.580 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:27:30.581 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":85,"fsid":"9ff84c16-0c17-11ef-bc97-c7b262605968","created":"2024-05-07T02:16:19.507657+0000","modified":"2024-05-07T02:26:32.693063+0000","last_up_change":"2024-05-07T02:24:55.025124+0000","last_in_change":"2024-05-07T02:24:33.842422+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-07T02:20:21.439697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-07T02:26:06.178814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-07T02:26:10.705693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-07T02:26:23.534133+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b437cf22-ec8e-48f0-86a2-2413a0ab40dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6803","nonce":2390539683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6805","nonce":2390539683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6809","nonce":2390539683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6807","nonce":2390539683}]},"public_addr":"172.21.15.19:6803/2390539683","cluster_addr":"172.21.15.19:6805/2390539683","heartbeat_back_addr":"172.21.15.19:6809/2390539683","heartbeat_front_addr":"172.21.15.19:6807/2390539683","state":["exists","up"]},{"osd":1,"uuid":"dc753e16-634d-4c14-abd2-935281e1499e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6801","nonce":4292470615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6803","nonce":4292470615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6807","nonce":4292470615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6805","nonce":4292470615}]},"public_addr":"172.21.15.27:6801/4292470615","cluster_addr":"172.21.15.27:6803/4292470615","heartbeat_back_addr":"172.21.15.27:6807/4292470615","heartbeat_front_addr":"172.21.15.27:6805/4292470615","state":["exists","up"]},{"osd":2,"uuid":"40570071-646e-4e66-9a16-c55ab743ac86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6801","nonce":1180552939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6803","nonce":1180552939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6807","nonce":1180552939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6805","nonce":1180552939}]},"public_addr":"172.21.15.64:6801/1180552939","cluster_addr":"172.21.15.64:6803/1180552939","heartbeat_back_addr":"172.21.15.64:6807/1180552939","heartbeat_front_addr":"172.21.15.64:6805/1180552939","state":["exists","up"]},{"osd":3,"uuid":"ceaa4252-4dfa-427a-9b1a-76adf3b8e609","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6811","nonce":387118141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6813","nonce":387118141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6817","nonce":387118141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6815","nonce":387118141}]},"public_addr":"172.21.15.19:6811/387118141","cluster_addr":"172.21.15.19:6813/387118141","heartbeat_back_addr":"172.21.15.19:6817/387118141","heartbeat_front_addr":"172.21.15.19:6815/387118141","state":["exists","up"]},{"osd":4,"uuid":"5a6d1ebc-62ca-444e-8f29-1980418d0a8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6809","nonce":3662186068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6811","nonce":3662186068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6815","nonce":3662186068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6813","nonce":3662186068}]},"public_addr":"172.21.15.27:6809/3662186068","cluster_addr":"172.21.15.27:6811/3662186068","heartbeat_back_addr":"172.21.15.27:6815/3662186068","heartbeat_front_addr":"172.21.15.27:6813/3662186068","state":["exists","up"]},{"osd":5,"uuid":"60c207e9-55e8-4bed-87c3-f6a06a2a1fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6809","nonce":1264591849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6811","nonce":1264591849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6815","nonce":1264591849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6813","nonce":1264591849}]},"public_addr":"172.21.15.64:6809/1264591849","cluster_addr":"172.21.15.64:6811/1264591849","heartbeat_back_addr":"172.21.15.64:6815/1264591849","heartbeat_front_addr":"172.21.15.64:6813/1264591849","state":["exists","up"]},{"osd":6,"uuid":"f3e038e0-4026-45f1-a923-c07cbb307f64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6819","nonce":2158875543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6821","nonce":2158875543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6825","nonce":2158875543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6823","nonce":2158875543}]},"public_addr":"172.21.15.19:6819/2158875543","cluster_addr":"172.21.15.19:6821/2158875543","heartbeat_back_addr":"172.21.15.19:6825/2158875543","heartbeat_front_addr":"172.21.15.19:6823/2158875543","state":["exists","up"]},{"osd":7,"uuid":"d69e34d4-2028-43b8-8ada-ff4a9b579620","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6817","nonce":2756612091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6819","nonce":2756612091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6823","nonce":2756612091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6821","nonce":2756612091}]},"public_addr":"172.21.15.27:6817/2756612091","cluster_addr":"172.21.15.27:6819/2756612091","heartbeat_back_addr":"172.21.15.27:6823/2756612091","heartbeat_front_addr":"172.21.15.27:6821/2756612091","state":["exists","up"]},{"osd":8,"uuid":"6c1a4179-75b6-42ff-ac29-378801e6f8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6817","nonce":1987024836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6819","nonce":1987024836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6823","nonce":1987024836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6821","nonce":1987024836}]},"public_addr":"172.21.15.64:6817/1987024836","cluster_addr":"172.21.15.64:6819/1987024836","heartbeat_back_addr":"172.21.15.64:6823/1987024836","heartbeat_front_addr":"172.21.15.64:6821/1987024836","state":["exists","up"]},{"osd":9,"uuid":"9df7ff66-d39f-4709-8324-74cf9e0e052d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6827","nonce":4124921646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6829","nonce":4124921646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6833","nonce":4124921646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6831","nonce":4124921646}]},"public_addr":"172.21.15.19:6827/4124921646","cluster_addr":"172.21.15.19:6829/4124921646","heartbeat_back_addr":"172.21.15.19:6833/4124921646","heartbeat_front_addr":"172.21.15.19:6831/4124921646","state":["exists","up"]},{"osd":10,"uuid":"71eaa9a7-667d-4cb7-935c-42177d20a279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6825","nonce":1428931817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6827","nonce":1428931817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6831","nonce":1428931817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6829","nonce":1428931817}]},"public_addr":"172.21.15.27:6825/1428931817","cluster_addr":"172.21.15.27:6827/1428931817","heartbeat_back_addr":"172.21.15.27:6831/1428931817","heartbeat_front_addr":"172.21.15.27:6829/1428931817","state":["exists","up"]},{"osd":11,"uuid":"3330627f-ab5d-4e6e-83cf-fc8b74c54ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6825","nonce":4045070679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6827","nonce":4045070679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6831","nonce":4045070679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6829","nonce":4045070679}]},"public_addr":"172.21.15.64:6825/4045070679","cluster_addr":"172.21.15.64:6827/4045070679","heartbeat_back_addr":"172.21.15.64:6831/4045070679","heartbeat_front_addr":"172.21.15.64:6829/4045070679","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:19:27.902633+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:19:54.305685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:20:18.727053+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:20:53.579199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:21:21.058018+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:21:47.091494+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:22:23.269018+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:22:51.863628+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:23:18.537774+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:23:55.763993+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:24:24.956178+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:24:52.681201+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/1956516258":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/1816772181":"2024-05-08T02:17:31.252868+0000","172.21.15.19:6801/174304059":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/1514051750":"2024-05-08T02:16:54.732592+0000","172.21.15.19:6800/174304059":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/3930045871":"2024-05-08T02:16:54.732592+0000","172.21.15.19:6800/3614957309":"2024-05-08T02:16:54.732592+0000","172.21.15.19:0/1328365399":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/139593344":"2024-05-08T02:16:54.732592+0000","172.21.15.19:6801/3614957309":"2024-05-08T02:16:54.732592+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-07T02:27:30.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.582+0000 7fda14a00700 1 -- 172.21.15.19:0/1633938754 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fd9f40b2880 msgr2=0x7fd9f40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:30.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.582+0000 7fda14a00700 1 --2- 172.21.15.19:0/1633938754 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fd9f40b2880 0x7fd9f40b4d90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fda00005950 tx=0x7fda0000b410 comp rx=0 tx=0).stop 2024-05-07T02:27:30.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.582+0000 7fda14a00700 1 -- 172.21.15.19:0/1633938754 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda10108d30 msgr2=0x7fda1019d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:30.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.582+0000 7fda14a00700 1 --2- 172.21.15.19:0/1633938754 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda10108d30 0x7fda1019d3f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc00ea30 tx=0x7fd9fc00ed40 comp rx=0 tx=0).stop 2024-05-07T02:27:30.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.582+0000 7fda14a00700 1 -- 172.21.15.19:0/1633938754 shutdown_connections 2024-05-07T02:27:30.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.582+0000 7fda14a00700 1 --2- 172.21.15.19:0/1633938754 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fd9f40b2880 0x7fd9f40b4d90 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:30.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.582+0000 7fda14a00700 1 --2- 172.21.15.19:0/1633938754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda10109680 0x7fda101a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:30.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.582+0000 7fda14a00700 1 --2- 172.21.15.19:0/1633938754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda10104400 0x7fda1019ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:30.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.582+0000 7fda14a00700 1 --2- 172.21.15.19:0/1633938754 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda10108d30 0x7fda1019d3f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:30.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.582+0000 7fda14a00700 1 -- 172.21.15.19:0/1633938754 >> 172.21.15.19:0/1633938754 conn(0x7fda10079d10 msgr2=0x7fda1010f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:27:30.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.583+0000 7fda14a00700 1 -- 172.21.15.19:0/1633938754 shutdown_connections 2024-05-07T02:27:30.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:30.583+0000 7fda14a00700 1 -- 172.21.15.19:0/1633938754 wait complete. 2024-05-07T02:27:30.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:30 smithi064 ceph-mon[34699]: pgmap v374: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-07T02:27:30.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:30 smithi019 ceph-mon[29781]: pgmap v374: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-07T02:27:30.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:30 smithi027 ceph-mon[34398]: pgmap v374: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-07T02:27:31.504 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-05-07T02:27:31.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:31 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/1633938754' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-07T02:27:31.860 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:31 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-05-07T02:27:31.860 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:31 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-05-07T02:27:31.860 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:31 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-05-07T02:27:31.860 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:31 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-05-07T02:27:31.860 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:31 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-05-07T02:27:31.860 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:31 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-05-07T02:27:31.860 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:31 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-05-07T02:27:31.860 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:31 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-05-07T02:27:31.860 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:31 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [5, 10]}]: dispatch 2024-05-07T02:27:31.860 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:31 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-05-07T02:27:31.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:31 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/1633938754' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-07T02:27:31.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:31 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-05-07T02:27:31.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:31 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-05-07T02:27:31.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:31 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-05-07T02:27:31.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:31 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-05-07T02:27:31.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:31 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-05-07T02:27:31.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:31 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-05-07T02:27:31.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:31 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-05-07T02:27:31.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:31 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-05-07T02:27:31.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:31 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [5, 10]}]: dispatch 2024-05-07T02:27:31.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:31 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-05-07T02:27:31.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:31 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/1633938754' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-07T02:27:31.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:31 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-05-07T02:27:31.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:31 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-05-07T02:27:31.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:31 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-05-07T02:27:31.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:31 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-05-07T02:27:31.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:31 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-05-07T02:27:31.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:31 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-05-07T02:27:31.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:31 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-05-07T02:27:31.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:31 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-05-07T02:27:31.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:31 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [5, 10]}]: dispatch 2024-05-07T02:27:31.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:31 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-05-07T02:27:31.980 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:27:32.861 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:32 smithi064 ceph-mon[34699]: pgmap v375: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-07T02:27:32.861 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:32 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-05-07T02:27:32.861 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:32 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-05-07T02:27:32.861 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:32 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-05-07T02:27:32.861 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:32 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-05-07T02:27:32.861 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:32 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-05-07T02:27:32.861 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:32 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-05-07T02:27:32.861 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:32 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-05-07T02:27:32.861 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:32 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-05-07T02:27:32.862 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:32 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [5, 10]}]': finished 2024-05-07T02:27:32.862 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:32 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-05-07T02:27:32.862 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:32 smithi064 ceph-mon[34699]: osdmap e86: 12 total, 12 up, 12 in 2024-05-07T02:27:32.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:32 smithi019 ceph-mon[29781]: pgmap v375: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-07T02:27:32.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:32 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-05-07T02:27:32.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:32 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-05-07T02:27:32.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:32 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-05-07T02:27:32.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:32 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-05-07T02:27:32.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:32 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-05-07T02:27:32.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:32 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-05-07T02:27:32.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:32 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-05-07T02:27:32.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:32 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-05-07T02:27:32.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:32 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [5, 10]}]': finished 2024-05-07T02:27:32.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:32 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-05-07T02:27:32.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:32 smithi019 ceph-mon[29781]: osdmap e86: 12 total, 12 up, 12 in 2024-05-07T02:27:32.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:32 smithi027 ceph-mon[34398]: pgmap v375: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-07T02:27:32.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:32 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-05-07T02:27:32.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:32 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-05-07T02:27:32.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:32 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-05-07T02:27:32.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:32 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-05-07T02:27:32.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:32 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-05-07T02:27:32.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:32 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-05-07T02:27:32.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:32 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-05-07T02:27:32.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:32 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-05-07T02:27:32.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:32 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [5, 10]}]': finished 2024-05-07T02:27:32.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:32 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-05-07T02:27:32.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:32 smithi027 ceph-mon[34398]: osdmap e86: 12 total, 12 up, 12 in 2024-05-07T02:27:33.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:33 smithi019 ceph-mon[29781]: osdmap e87: 12 total, 12 up, 12 in 2024-05-07T02:27:33.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:33 smithi064 ceph-mon[34699]: osdmap e87: 12 total, 12 up, 12 in 2024-05-07T02:27:33.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:33 smithi027 ceph-mon[34398]: osdmap e87: 12 total, 12 up, 12 in 2024-05-07T02:27:34.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:34.385+0000 7f28dca00700 1 -- 172.21.15.19:0/42450546 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28d810b8a0 msgr2=0x7f28d8113200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:34.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:34.385+0000 7f28dca00700 1 --2- 172.21.15.19:0/42450546 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28d810b8a0 0x7f28d8113200 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f28d000b0a0 tx=0x7f28d000b3b0 comp rx=0 tx=0).stop 2024-05-07T02:27:34.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:34.385+0000 7f28dca00700 1 -- 172.21.15.19:0/42450546 shutdown_connections 2024-05-07T02:27:34.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:34.385+0000 7f28dca00700 1 --2- 172.21.15.19:0/42450546 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28d810b8a0 0x7f28d8113200 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:34.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:34.385+0000 7f28dca00700 1 --2- 172.21.15.19:0/42450546 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28d8106620 0x7f28d8106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:34.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:34.385+0000 7f28dca00700 1 --2- 172.21.15.19:0/42450546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28d810af50 0x7f28d810b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:34.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:34.385+0000 7f28dca00700 1 -- 172.21.15.19:0/42450546 >> 172.21.15.19:0/42450546 conn(0x7f28d8102030 msgr2=0x7f28d8104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:27:34.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:34.385+0000 7f28dca00700 1 -- 172.21.15.19:0/42450546 shutdown_connections 2024-05-07T02:27:34.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:34.385+0000 7f28dca00700 1 -- 172.21.15.19:0/42450546 wait complete. 2024-05-07T02:27:34.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:34.387+0000 7f28dca00700 1 Processor -- start 2024-05-07T02:27:34.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:34.387+0000 7f28dca00700 1 -- start start 2024-05-07T02:27:34.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:34.388+0000 7f28dca00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28d8106620 0x7f28d819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:27:34.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:34.388+0000 7f28dca00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28d810af50 0x7f28d819d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:27:34.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:34.388+0000 7f28dca00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28d810b8a0 0x7f28d81a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:27:34.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:34.388+0000 7f28dca00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f28d8116520 con 0x7f28d810af50 2024-05-07T02:27:34.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:34.388+0000 7f28dca00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f28d81163a0 con 0x7f28d810b8a0 2024-05-07T02:27:34.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:34.388+0000 7f28dca00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f28d81166a0 con 0x7f28d8106620 2024-05-07T02:27:34.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:34.388+0000 7f28d6800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28d8106620 0x7f28d819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:27:34.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:34.388+0000 7f28d5e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28d810af50 0x7f28d819d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:27:34.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:34.388+0000 7f28d6800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28d8106620 0x7f28d819cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:33882/0 (socket says 172.21.15.19:33882) 2024-05-07T02:27:34.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:34.388+0000 7f28d6800700 1 -- 172.21.15.19:0/1523979958 learned_addr learned my addr 172.21.15.19:0/1523979958 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:27:34.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:34.388+0000 7f28d7200700 1 --2- 172.21.15.19:0/1523979958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28d810b8a0 0x7f28d81a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:27:34.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:34.389+0000 7f28d6800700 1 -- 172.21.15.19:0/1523979958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28d810b8a0 msgr2=0x7f28d81a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:34.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:34.389+0000 7f28d6800700 1 --2- 172.21.15.19:0/1523979958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28d810b8a0 0x7f28d81a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:34.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:34.389+0000 7f28d6800700 1 -- 172.21.15.19:0/1523979958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28d810af50 msgr2=0x7f28d819d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:34.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:34.389+0000 7f28d6800700 1 --2- 172.21.15.19:0/1523979958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28d810af50 0x7f28d819d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:34.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:34.389+0000 7f28d6800700 1 -- 172.21.15.19:0/1523979958 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d81a1c60 con 0x7f28d8106620 2024-05-07T02:27:34.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:34.389+0000 7f28d6800700 1 --2- 172.21.15.19:0/1523979958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28d8106620 0x7f28d819cd90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f28c800e9a0 tx=0x7f28c800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:27:34.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:34.389+0000 7f28cf400700 1 -- 172.21.15.19:0/1523979958 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c800ca10 con 0x7f28d8106620 2024-05-07T02:27:34.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:34.390+0000 7f28dca00700 1 -- 172.21.15.19:0/1523979958 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28d81a1f50 con 0x7f28d8106620 2024-05-07T02:27:34.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:34.390+0000 7f28cf400700 1 -- 172.21.15.19:0/1523979958 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28c800cbb0 con 0x7f28d8106620 2024-05-07T02:27:34.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:34.390+0000 7f28cf400700 1 -- 172.21.15.19:0/1523979958 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c8018680 con 0x7f28d8106620 2024-05-07T02:27:34.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:34.390+0000 7f28dca00700 1 -- 172.21.15.19:0/1523979958 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28d81a9870 con 0x7f28d8106620 2024-05-07T02:27:34.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:34.391+0000 7f28dca00700 1 -- 172.21.15.19:0/1523979958 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28d80512c0 con 0x7f28d8106620 2024-05-07T02:27:34.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:34.395+0000 7f28cf400700 1 -- 172.21.15.19:0/1523979958 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f28c8010430 con 0x7f28d8106620 2024-05-07T02:27:34.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:34.397+0000 7f28cf400700 1 --2- 172.21.15.19:0/1523979958 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f28bc0b2880 0x7f28bc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:27:34.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:34.397+0000 7f28cf400700 1 -- 172.21.15.19:0/1523979958 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f28c80e2ab0 con 0x7f28d8106620 2024-05-07T02:27:34.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:34.397+0000 7f28d5e00700 1 --2- 172.21.15.19:0/1523979958 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f28bc0b2880 0x7f28bc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:27:34.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:34.398+0000 7f28cf400700 1 -- 172.21.15.19:0/1523979958 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f28c80aa410 con 0x7f28d8106620 2024-05-07T02:27:34.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:34.398+0000 7f28d5e00700 1 --2- 172.21.15.19:0/1523979958 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f28bc0b2880 0x7f28bc0b4d90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f28d8109d70 tx=0x7f28c40073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:27:34.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:34 smithi019 ceph-mon[29781]: pgmap v378: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:27:34.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:34.694+0000 7f28dca00700 1 -- 172.21.15.19:0/1523979958 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f28d819e0f0 con 0x7f28d8106620 2024-05-07T02:27:34.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:34 smithi064 ceph-mon[34699]: pgmap v378: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:27:34.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:34 smithi027 ceph-mon[34398]: pgmap v378: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:27:35.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:35.621+0000 7f28cf400700 1 -- 172.21.15.19:0/1523979958 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f28c80a9d10 con 0x7f28d8106620 2024-05-07T02:27:35.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:35.626+0000 7f28dca00700 1 -- 172.21.15.19:0/1523979958 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f28bc0b2880 msgr2=0x7f28bc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:35.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:35.626+0000 7f28dca00700 1 --2- 172.21.15.19:0/1523979958 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f28bc0b2880 0x7f28bc0b4d90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f28d8109d70 tx=0x7f28c40073d0 comp rx=0 tx=0).stop 2024-05-07T02:27:35.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:35.626+0000 7f28dca00700 1 -- 172.21.15.19:0/1523979958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28d8106620 msgr2=0x7f28d819cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:35.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:35.626+0000 7f28dca00700 1 --2- 172.21.15.19:0/1523979958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28d8106620 0x7f28d819cd90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f28c800e9a0 tx=0x7f28c800ecb0 comp rx=0 tx=0).stop 2024-05-07T02:27:35.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:35.626+0000 7f28dca00700 1 -- 172.21.15.19:0/1523979958 shutdown_connections 2024-05-07T02:27:35.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:35.626+0000 7f28dca00700 1 --2- 172.21.15.19:0/1523979958 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f28bc0b2880 0x7f28bc0b4d90 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:35.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:35.626+0000 7f28dca00700 1 --2- 172.21.15.19:0/1523979958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28d810b8a0 0x7f28d81a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:35.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:35.626+0000 7f28dca00700 1 --2- 172.21.15.19:0/1523979958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28d8106620 0x7f28d819cd90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:35.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:35.626+0000 7f28dca00700 1 --2- 172.21.15.19:0/1523979958 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28d810af50 0x7f28d819d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:35.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:35.626+0000 7f28dca00700 1 -- 172.21.15.19:0/1523979958 >> 172.21.15.19:0/1523979958 conn(0x7f28d8102030 msgr2=0x7f28d810a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:27:35.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:35.626+0000 7f28dca00700 1 -- 172.21.15.19:0/1523979958 shutdown_connections 2024-05-07T02:27:35.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:35.626+0000 7f28dca00700 1 -- 172.21.15.19:0/1523979958 wait complete. 2024-05-07T02:27:35.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:35 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/1523979958' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-07T02:27:35.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:35 smithi064 ceph-mon[34699]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-07T02:27:35.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:35 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/1523979958' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-07T02:27:35.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:35 smithi019 ceph-mon[29781]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-07T02:27:35.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:35 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/1523979958' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-07T02:27:35.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:35 smithi027 ceph-mon[34398]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-07T02:27:36.528 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-05-07T02:27:36.528 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-05-07T02:27:36.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:36 smithi064 ceph-mon[34699]: pgmap v379: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-07T02:27:36.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:36 smithi064 ceph-mon[34699]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-05-07T02:27:36.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:36 smithi064 ceph-mon[34699]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-07T02:27:36.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:36 smithi064 ceph-mon[34699]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-07T02:27:36.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:36 smithi019 ceph-mon[29781]: pgmap v379: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-07T02:27:36.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:36 smithi019 ceph-mon[29781]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-05-07T02:27:36.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:36 smithi019 ceph-mon[29781]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-07T02:27:36.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:36 smithi019 ceph-mon[29781]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-07T02:27:36.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:36 smithi027 ceph-mon[34398]: pgmap v379: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-07T02:27:36.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:36 smithi027 ceph-mon[34398]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-05-07T02:27:36.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:36 smithi027 ceph-mon[34398]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-07T02:27:36.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:36 smithi027 ceph-mon[34398]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-07T02:27:37.007 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:27:37.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:37 smithi019 ceph-mon[29781]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-05-07T02:27:37.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:37 smithi019 ceph-mon[29781]: Cluster is now healthy 2024-05-07T02:27:37.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:37 smithi027 ceph-mon[34398]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-05-07T02:27:37.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:37 smithi027 ceph-mon[34398]: Cluster is now healthy 2024-05-07T02:27:38.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:37 smithi064 ceph-mon[34699]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-05-07T02:27:38.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:37 smithi064 ceph-mon[34699]: Cluster is now healthy 2024-05-07T02:27:38.940 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:38 smithi019 ceph-mon[29781]: pgmap v380: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-07T02:27:38.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:38 smithi027 ceph-mon[34398]: pgmap v380: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-07T02:27:39.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:38 smithi064 ceph-mon[34699]: pgmap v380: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-07T02:27:39.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.327+0000 7fd084c00700 1 -- 172.21.15.19:0/1095454020 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd080105be0 msgr2=0x7fd08010e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:39.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.327+0000 7fd084c00700 1 --2- 172.21.15.19:0/1095454020 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd080105be0 0x7fd08010e130 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd06c009990 tx=0x7fd06c009ca0 comp rx=0 tx=0).stop 2024-05-07T02:27:39.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.327+0000 7fd084c00700 1 -- 172.21.15.19:0/1095454020 shutdown_connections 2024-05-07T02:27:39.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.327+0000 7fd084c00700 1 --2- 172.21.15.19:0/1095454020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd08010e700 0x7fd080110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:39.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.327+0000 7fd084c00700 1 --2- 172.21.15.19:0/1095454020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd080105290 0x7fd0801056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:39.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.327+0000 7fd084c00700 1 --2- 172.21.15.19:0/1095454020 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd080105be0 0x7fd08010e130 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:39.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.327+0000 7fd084c00700 1 -- 172.21.15.19:0/1095454020 >> 172.21.15.19:0/1095454020 conn(0x7fd080078b50 msgr2=0x7fd080078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:27:39.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.327+0000 7fd084c00700 1 -- 172.21.15.19:0/1095454020 shutdown_connections 2024-05-07T02:27:39.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.327+0000 7fd084c00700 1 -- 172.21.15.19:0/1095454020 wait complete. 2024-05-07T02:27:39.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.329+0000 7fd084c00700 1 Processor -- start 2024-05-07T02:27:39.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.329+0000 7fd084c00700 1 -- start start 2024-05-07T02:27:39.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.330+0000 7fd084c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd080105290 0x7fd0801a0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:27:39.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.330+0000 7fd084c00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd080105be0 0x7fd0801a0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:27:39.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.330+0000 7fd084c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd08010e700 0x7fd08019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:27:39.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.330+0000 7fd084c00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd0801142b0 con 0x7fd080105290 2024-05-07T02:27:39.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.330+0000 7fd084c00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd080114130 con 0x7fd08010e700 2024-05-07T02:27:39.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.330+0000 7fd084c00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd080114430 con 0x7fd080105be0 2024-05-07T02:27:39.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.330+0000 7fd07f400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd080105290 0x7fd0801a0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:27:39.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.331+0000 7fd07ea00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd080105be0 0x7fd0801a0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:27:39.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.331+0000 7fd07f400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd080105290 0x7fd0801a0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37900/0 (socket says 172.21.15.19:37900) 2024-05-07T02:27:39.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.331+0000 7fd07f400700 1 -- 172.21.15.19:0/4244155804 learned_addr learned my addr 172.21.15.19:0/4244155804 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:27:39.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.331+0000 7fd07fe00700 1 --2- 172.21.15.19:0/4244155804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd08010e700 0x7fd08019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:27:39.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.331+0000 7fd07ea00700 1 -- 172.21.15.19:0/4244155804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd08010e700 msgr2=0x7fd08019aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:39.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.331+0000 7fd07ea00700 1 --2- 172.21.15.19:0/4244155804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd08010e700 0x7fd08019aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:39.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.331+0000 7fd07ea00700 1 -- 172.21.15.19:0/4244155804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd080105290 msgr2=0x7fd0801a0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:39.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.331+0000 7fd07ea00700 1 --2- 172.21.15.19:0/4244155804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd080105290 0x7fd0801a0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:39.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.331+0000 7fd07ea00700 1 -- 172.21.15.19:0/4244155804 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd08019b140 con 0x7fd080105be0 2024-05-07T02:27:39.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.331+0000 7fd07f400700 1 --2- 172.21.15.19:0/4244155804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd080105290 0x7fd0801a0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:27:39.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.332+0000 7fd07ea00700 1 --2- 172.21.15.19:0/4244155804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd080105be0 0x7fd0801a0db0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd06c000c00 tx=0x7fd06c0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:27:39.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.332+0000 7fd077e00700 1 -- 172.21.15.19:0/4244155804 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06c026070 con 0x7fd080105be0 2024-05-07T02:27:39.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.332+0000 7fd084c00700 1 -- 172.21.15.19:0/4244155804 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd08019b3d0 con 0x7fd080105be0 2024-05-07T02:27:39.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.332+0000 7fd084c00700 1 -- 172.21.15.19:0/4244155804 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0801a7700 con 0x7fd080105be0 2024-05-07T02:27:39.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.332+0000 7fd077e00700 1 -- 172.21.15.19:0/4244155804 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd06c021410 con 0x7fd080105be0 2024-05-07T02:27:39.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.332+0000 7fd077e00700 1 -- 172.21.15.19:0/4244155804 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06c017650 con 0x7fd080105be0 2024-05-07T02:27:39.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.333+0000 7fd084c00700 1 -- 172.21.15.19:0/4244155804 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd060005330 con 0x7fd080105be0 2024-05-07T02:27:39.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.336+0000 7fd077e00700 1 -- 172.21.15.19:0/4244155804 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fd06c029070 con 0x7fd080105be0 2024-05-07T02:27:39.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.338+0000 7fd077e00700 1 --2- 172.21.15.19:0/4244155804 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fd0640b2880 0x7fd0640b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:27:39.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.338+0000 7fd077e00700 1 -- 172.21.15.19:0/4244155804 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fd06c0e3500 con 0x7fd080105be0 2024-05-07T02:27:39.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.338+0000 7fd07f400700 1 --2- 172.21.15.19:0/4244155804 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fd0640b2880 0x7fd0640b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:27:39.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.339+0000 7fd07f400700 1 --2- 172.21.15.19:0/4244155804 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fd0640b2880 0x7fd0640b4d90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd08019bbd0 tx=0x7fd070009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:27:39.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.340+0000 7fd077e00700 1 -- 172.21.15.19:0/4244155804 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd06c013070 con 0x7fd080105be0 2024-05-07T02:27:39.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.627+0000 7fd084c00700 1 -- 172.21.15.19:0/4244155804 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7fd0600058f0 con 0x7fd080105be0 2024-05-07T02:27:39.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.679+0000 7fd077e00700 1 -- 172.21.15.19:0/4244155804 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7fd06c0aaef0 con 0x7fd080105be0 2024-05-07T02:27:39.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.682+0000 7fd084c00700 1 -- 172.21.15.19:0/4244155804 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fd0640b2880 msgr2=0x7fd0640b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:39.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.683+0000 7fd084c00700 1 --2- 172.21.15.19:0/4244155804 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fd0640b2880 0x7fd0640b4d90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd08019bbd0 tx=0x7fd070009260 comp rx=0 tx=0).stop 2024-05-07T02:27:39.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.683+0000 7fd084c00700 1 -- 172.21.15.19:0/4244155804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd080105be0 msgr2=0x7fd0801a0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:39.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.683+0000 7fd084c00700 1 --2- 172.21.15.19:0/4244155804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd080105be0 0x7fd0801a0db0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd06c000c00 tx=0x7fd06c0047b0 comp rx=0 tx=0).stop 2024-05-07T02:27:39.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.683+0000 7fd084c00700 1 -- 172.21.15.19:0/4244155804 shutdown_connections 2024-05-07T02:27:39.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.683+0000 7fd084c00700 1 --2- 172.21.15.19:0/4244155804 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fd0640b2880 0x7fd0640b4d90 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:39.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.683+0000 7fd084c00700 1 --2- 172.21.15.19:0/4244155804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd08010e700 0x7fd08019aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:39.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.683+0000 7fd084c00700 1 --2- 172.21.15.19:0/4244155804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd080105290 0x7fd0801a0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:39.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.683+0000 7fd084c00700 1 --2- 172.21.15.19:0/4244155804 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd080105be0 0x7fd0801a0db0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:39.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.683+0000 7fd084c00700 1 -- 172.21.15.19:0/4244155804 >> 172.21.15.19:0/4244155804 conn(0x7fd080078b50 msgr2=0x7fd0800fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:27:39.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.683+0000 7fd084c00700 1 -- 172.21.15.19:0/4244155804 shutdown_connections 2024-05-07T02:27:39.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:39.683+0000 7fd084c00700 1 -- 172.21.15.19:0/4244155804 wait complete. 2024-05-07T02:27:39.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:39 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/4244155804' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-07T02:27:39.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:39 smithi019 ceph-mon[29781]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-07T02:27:39.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:39 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/4244155804' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-07T02:27:39.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:39 smithi027 ceph-mon[34398]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-07T02:27:40.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:39 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/4244155804' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-07T02:27:40.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:39 smithi064 ceph-mon[34699]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-07T02:27:40.568 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-05-07T02:27:40.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:40 smithi019 ceph-mon[29781]: pgmap v381: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-07T02:27:40.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:40 smithi019 ceph-mon[29781]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-07T02:27:40.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:40 smithi019 ceph-mon[29781]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-07T02:27:40.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:40 smithi019 ceph-mon[29781]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-07T02:27:40.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:40 smithi019 ceph-mon[29781]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-07T02:27:40.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:40 smithi019 ceph-mon[29781]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-05-07T02:27:40.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:40 smithi019 ceph-mon[29781]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-05-07T02:27:40.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:40 smithi027 ceph-mon[34398]: pgmap v381: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-07T02:27:40.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:40 smithi027 ceph-mon[34398]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-07T02:27:40.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:40 smithi027 ceph-mon[34398]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-07T02:27:40.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:40 smithi027 ceph-mon[34398]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-07T02:27:40.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:40 smithi027 ceph-mon[34398]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-07T02:27:40.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:40 smithi027 ceph-mon[34398]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-05-07T02:27:40.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:40 smithi027 ceph-mon[34398]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-05-07T02:27:41.063 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:27:41.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:40 smithi064 ceph-mon[34699]: pgmap v381: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-07T02:27:41.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:40 smithi064 ceph-mon[34699]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-07T02:27:41.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:40 smithi064 ceph-mon[34699]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-07T02:27:41.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:40 smithi064 ceph-mon[34699]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-07T02:27:41.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:40 smithi064 ceph-mon[34699]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-07T02:27:41.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:40 smithi064 ceph-mon[34699]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-05-07T02:27:41.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:40 smithi064 ceph-mon[34699]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-05-07T02:27:41.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:41 smithi019 ceph-mon[29781]: mds.? [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] up:active 2024-05-07T02:27:41.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:41 smithi019 ceph-mon[29781]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-07T02:27:41.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:41 smithi019 ceph-mon[29781]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-05-07T02:27:41.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:41 smithi019 ceph-mon[29781]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-05-07T02:27:41.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:41 smithi019 ceph-mon[29781]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-05-07T02:27:41.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:41 smithi027 ceph-mon[34398]: mds.? [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] up:active 2024-05-07T02:27:41.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:41 smithi027 ceph-mon[34398]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-07T02:27:41.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:41 smithi027 ceph-mon[34398]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-05-07T02:27:41.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:41 smithi027 ceph-mon[34398]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-05-07T02:27:41.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:41 smithi027 ceph-mon[34398]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-05-07T02:27:42.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:41 smithi064 ceph-mon[34699]: mds.? [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] up:active 2024-05-07T02:27:42.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:41 smithi064 ceph-mon[34699]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-07T02:27:42.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:41 smithi064 ceph-mon[34699]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-05-07T02:27:42.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:41 smithi064 ceph-mon[34699]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-05-07T02:27:42.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:41 smithi064 ceph-mon[34699]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-05-07T02:27:42.839 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:42 smithi019 ceph-mon[29781]: pgmap v382: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 413 B/s wr, 3 op/s; 0 B/s, 0 objects/s recovering 2024-05-07T02:27:42.839 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:42 smithi019 ceph-mon[29781]: mds.? [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] up:active 2024-05-07T02:27:42.839 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:42 smithi019 ceph-mon[29781]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-07T02:27:42.839 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:42 smithi019 ceph-mon[29781]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-05-07T02:27:42.839 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:42 smithi019 ceph-mon[29781]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=h=up:creating} 8 up:standby 2024-05-07T02:27:42.839 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:42 smithi019 ceph-mon[29781]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-05-07T02:27:43.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:42 smithi064 ceph-mon[34699]: pgmap v382: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 413 B/s wr, 3 op/s; 0 B/s, 0 objects/s recovering 2024-05-07T02:27:43.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:42 smithi064 ceph-mon[34699]: mds.? [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] up:active 2024-05-07T02:27:43.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:42 smithi064 ceph-mon[34699]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-07T02:27:43.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:42 smithi064 ceph-mon[34699]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-05-07T02:27:43.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:42 smithi064 ceph-mon[34699]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=h=up:creating} 8 up:standby 2024-05-07T02:27:43.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:42 smithi064 ceph-mon[34699]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-05-07T02:27:43.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:42 smithi027 ceph-mon[34398]: pgmap v382: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 413 B/s wr, 3 op/s; 0 B/s, 0 objects/s recovering 2024-05-07T02:27:43.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:42 smithi027 ceph-mon[34398]: mds.? [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] up:active 2024-05-07T02:27:43.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:42 smithi027 ceph-mon[34398]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-07T02:27:43.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:42 smithi027 ceph-mon[34398]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-05-07T02:27:43.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:42 smithi027 ceph-mon[34398]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=h=up:creating} 8 up:standby 2024-05-07T02:27:43.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:42 smithi027 ceph-mon[34398]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-05-07T02:27:43.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:43.487+0000 7fc4b9200700 1 -- 172.21.15.19:0/662196581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4b40764b0 msgr2=0x7fc4b4114e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:43.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:43.487+0000 7fc4b9200700 1 --2- 172.21.15.19:0/662196581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4b40764b0 0x7fc4b4114e30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0009990 tx=0x7fc4a0009ca0 comp rx=0 tx=0).stop 2024-05-07T02:27:43.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:43.487+0000 7fc4b9200700 1 -- 172.21.15.19:0/662196581 shutdown_connections 2024-05-07T02:27:43.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:43.487+0000 7fc4b9200700 1 --2- 172.21.15.19:0/662196581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4b4115370 0x7fc4b4117790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:43.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:43.487+0000 7fc4b9200700 1 --2- 172.21.15.19:0/662196581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4b4075b60 0x7fc4b4075f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:43.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:43.487+0000 7fc4b9200700 1 --2- 172.21.15.19:0/662196581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4b40764b0 0x7fc4b4114e30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:43.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:43.487+0000 7fc4b9200700 1 -- 172.21.15.19:0/662196581 >> 172.21.15.19:0/662196581 conn(0x7fc4b40fffe0 msgr2=0x7fc4b4102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:27:43.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:43.487+0000 7fc4b9200700 1 -- 172.21.15.19:0/662196581 shutdown_connections 2024-05-07T02:27:43.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:43.487+0000 7fc4b9200700 1 -- 172.21.15.19:0/662196581 wait complete. 2024-05-07T02:27:43.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:43.488+0000 7fc4b9200700 1 Processor -- start 2024-05-07T02:27:43.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:43.488+0000 7fc4b9200700 1 -- start start 2024-05-07T02:27:43.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:43.489+0000 7fc4b9200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4b4075b60 0x7fc4b41a1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:27:43.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:43.489+0000 7fc4b9200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4b40764b0 0x7fc4b41a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:27:43.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:43.489+0000 7fc4b9200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4b4115370 0x7fc4b41a5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:27:43.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:43.489+0000 7fc4b9200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b411aac0 con 0x7fc4b4115370 2024-05-07T02:27:43.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:43.489+0000 7fc4b9200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b411a940 con 0x7fc4b40764b0 2024-05-07T02:27:43.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:43.489+0000 7fc4b9200700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b411ac40 con 0x7fc4b4075b60 2024-05-07T02:27:43.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:43.489+0000 7fc4b1400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4b40764b0 0x7fc4b41a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:27:43.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:43.489+0000 7fc4b1400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4b40764b0 0x7fc4b41a1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:44140/0 (socket says 172.21.15.19:44140) 2024-05-07T02:27:43.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:43.489+0000 7fc4b1400700 1 -- 172.21.15.19:0/478274967 learned_addr learned my addr 172.21.15.19:0/478274967 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:27:43.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:43.489+0000 7fc4b2800700 1 --2- 172.21.15.19:0/478274967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4b4115370 0x7fc4b41a5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:27:43.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:43.490+0000 7fc4b1e00700 1 --2- 172.21.15.19:0/478274967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4b4075b60 0x7fc4b41a1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:27:43.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:43.490+0000 7fc4b1400700 1 -- 172.21.15.19:0/478274967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4b4075b60 msgr2=0x7fc4b41a1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:43.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:43.490+0000 7fc4b1400700 1 --2- 172.21.15.19:0/478274967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4b4075b60 0x7fc4b41a1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:43.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:43.490+0000 7fc4b1400700 1 -- 172.21.15.19:0/478274967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4b4115370 msgr2=0x7fc4b41a5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:43.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:43.490+0000 7fc4b1400700 1 --2- 172.21.15.19:0/478274967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4b4115370 0x7fc4b41a5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:43.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:43.490+0000 7fc4b1400700 1 -- 172.21.15.19:0/478274967 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b41a6360 con 0x7fc4b40764b0 2024-05-07T02:27:43.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:43.490+0000 7fc4b2800700 1 --2- 172.21.15.19:0/478274967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4b4115370 0x7fc4b41a5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:27:43.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:43.490+0000 7fc4b1400700 1 --2- 172.21.15.19:0/478274967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4b40764b0 0x7fc4b41a1a30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0000c00 tx=0x7fc4a0009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:27:43.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:43.490+0000 7fc4aaa00700 1 -- 172.21.15.19:0/478274967 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a0027070 con 0x7fc4b40764b0 2024-05-07T02:27:43.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:43.490+0000 7fc4aaa00700 1 -- 172.21.15.19:0/478274967 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4a0022470 con 0x7fc4b40764b0 2024-05-07T02:27:43.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:43.490+0000 7fc4b9200700 1 -- 172.21.15.19:0/478274967 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4b41a65f0 con 0x7fc4b40764b0 2024-05-07T02:27:43.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:43.491+0000 7fc4aaa00700 1 -- 172.21.15.19:0/478274967 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a0017650 con 0x7fc4b40764b0 2024-05-07T02:27:43.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:43.491+0000 7fc4b9200700 1 -- 172.21.15.19:0/478274967 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4b41adf70 con 0x7fc4b40764b0 2024-05-07T02:27:43.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:43.492+0000 7fc4b9200700 1 -- 172.21.15.19:0/478274967 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4b40512c0 con 0x7fc4b40764b0 2024-05-07T02:27:43.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:43.495+0000 7fc4aaa00700 1 -- 172.21.15.19:0/478274967 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc4a002a070 con 0x7fc4b40764b0 2024-05-07T02:27:43.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:43.497+0000 7fc4aaa00700 1 --2- 172.21.15.19:0/478274967 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fc4980b2900 0x7fc4980b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:27:43.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:43.497+0000 7fc4aaa00700 1 -- 172.21.15.19:0/478274967 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fc4a00e3570 con 0x7fc4b40764b0 2024-05-07T02:27:43.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:43.497+0000 7fc4b1e00700 1 --2- 172.21.15.19:0/478274967 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fc4980b2900 0x7fc4980b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:27:43.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:43.498+0000 7fc4b1e00700 1 --2- 172.21.15.19:0/478274967 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fc4980b2900 0x7fc4980b4e10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4009bf0 tx=0x7fc4a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:27:43.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:43.501+0000 7fc4aaa00700 1 -- 172.21.15.19:0/478274967 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc4a005fdf0 con 0x7fc4b40764b0 2024-05-07T02:27:43.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:43.796+0000 7fc4b9200700 1 -- 172.21.15.19:0/478274967 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fc4b402e5f0 con 0x7fc4b40764b0 2024-05-07T02:27:44.010 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:43 smithi064 ceph-mon[34699]: mds.? [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] up:active 2024-05-07T02:27:44.010 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:43 smithi064 ceph-mon[34699]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-07T02:27:44.010 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:43 smithi064 ceph-mon[34699]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-07T02:27:44.010 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:43 smithi064 ceph-mon[34699]: Cluster is now healthy 2024-05-07T02:27:44.010 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:43 smithi064 ceph-mon[34699]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=h=up:active} 8 up:standby 2024-05-07T02:27:44.010 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:43 smithi064 ceph-mon[34699]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-07T02:27:44.010 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:43 smithi064 ceph-mon[34699]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-05-07T02:27:44.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:43 smithi019 ceph-mon[29781]: mds.? [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] up:active 2024-05-07T02:27:44.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:43 smithi019 ceph-mon[29781]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-07T02:27:44.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:43 smithi019 ceph-mon[29781]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-07T02:27:44.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:43 smithi019 ceph-mon[29781]: Cluster is now healthy 2024-05-07T02:27:44.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:43 smithi019 ceph-mon[29781]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=h=up:active} 8 up:standby 2024-05-07T02:27:44.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:43 smithi019 ceph-mon[29781]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-07T02:27:44.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:43 smithi019 ceph-mon[29781]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-05-07T02:27:44.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:43 smithi027 ceph-mon[34398]: mds.? [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] up:active 2024-05-07T02:27:44.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:43 smithi027 ceph-mon[34398]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-07T02:27:44.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:43 smithi027 ceph-mon[34398]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-07T02:27:44.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:43 smithi027 ceph-mon[34398]: Cluster is now healthy 2024-05-07T02:27:44.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:43 smithi027 ceph-mon[34398]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=i=up:active,3=h=up:active} 8 up:standby 2024-05-07T02:27:44.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:43 smithi027 ceph-mon[34398]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-07T02:27:44.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:43 smithi027 ceph-mon[34398]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-05-07T02:27:44.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:44.744+0000 7fc4aaa00700 1 -- 172.21.15.19:0/478274967 <== mon.2 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v28) ==== 121+0+0 (secure 0 0 0) 0x7fc4a0013070 con 0x7fc4b40764b0 2024-05-07T02:27:44.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:44.748+0000 7fc4b9200700 1 -- 172.21.15.19:0/478274967 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fc4980b2900 msgr2=0x7fc4980b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:44.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:44.748+0000 7fc4b9200700 1 --2- 172.21.15.19:0/478274967 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fc4980b2900 0x7fc4980b4e10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4009bf0 tx=0x7fc4a4009260 comp rx=0 tx=0).stop 2024-05-07T02:27:44.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:44.748+0000 7fc4b9200700 1 -- 172.21.15.19:0/478274967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4b40764b0 msgr2=0x7fc4b41a1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:44.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:44.748+0000 7fc4b9200700 1 --2- 172.21.15.19:0/478274967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4b40764b0 0x7fc4b41a1a30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0000c00 tx=0x7fc4a0009510 comp rx=0 tx=0).stop 2024-05-07T02:27:44.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:44.748+0000 7fc4b9200700 1 -- 172.21.15.19:0/478274967 shutdown_connections 2024-05-07T02:27:44.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:44.748+0000 7fc4b9200700 1 --2- 172.21.15.19:0/478274967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc4b4115370 0x7fc4b41a5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:44.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:44.748+0000 7fc4b9200700 1 --2- 172.21.15.19:0/478274967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4b4075b60 0x7fc4b41a1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:44.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:44.748+0000 7fc4b9200700 1 --2- 172.21.15.19:0/478274967 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fc4980b2900 0x7fc4980b4e10 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:44.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:44.748+0000 7fc4b9200700 1 --2- 172.21.15.19:0/478274967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4b40764b0 0x7fc4b41a1a30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:44.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:44.749+0000 7fc4b9200700 1 -- 172.21.15.19:0/478274967 >> 172.21.15.19:0/478274967 conn(0x7fc4b40fffe0 msgr2=0x7fc4b4100700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:27:44.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:44.749+0000 7fc4b9200700 1 -- 172.21.15.19:0/478274967 shutdown_connections 2024-05-07T02:27:44.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:44.749+0000 7fc4b9200700 1 -- 172.21.15.19:0/478274967 wait complete. 2024-05-07T02:27:44.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:44 smithi027 ceph-mon[34398]: pgmap v383: 129 pgs: 129 active+clean; 582 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 469 B/s wr, 4 op/s; 0 B/s, 0 objects/s recovering 2024-05-07T02:27:44.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:44 smithi027 ceph-mon[34398]: mds.? [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] up:active 2024-05-07T02:27:44.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:44 smithi027 ceph-mon[34398]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-07T02:27:44.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:44 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/478274967' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-07T02:27:44.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:44 smithi027 ceph-mon[34398]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-07T02:27:44.956 INFO:journalctl@ceph.mds.b.smithi027.stdout:May 07 02:27:44 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-b[65159]: 2024-05-07T02:27:44.570+0000 7f022f5f6700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-05-07T02:27:44.957 INFO:journalctl@ceph.mds.b.smithi027.stdout:May 07 02:27:44 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-b[65159]: 2024-05-07T02:27:44.570+0000 7f022f5f6700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-05-07T02:27:44.957 INFO:journalctl@ceph.mds.b.smithi027.stdout:May 07 02:27:44 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-b[65159]: 2024-05-07T02:27:44.570+0000 7f022f5f6700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-05-07T02:27:45.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:44 smithi064 ceph-mon[34699]: pgmap v383: 129 pgs: 129 active+clean; 582 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 469 B/s wr, 4 op/s; 0 B/s, 0 objects/s recovering 2024-05-07T02:27:45.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:44 smithi064 ceph-mon[34699]: mds.? [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] up:active 2024-05-07T02:27:45.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:44 smithi064 ceph-mon[34699]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-07T02:27:45.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:44 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/478274967' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-07T02:27:45.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:44 smithi064 ceph-mon[34699]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-07T02:27:45.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:44 smithi019 ceph-mon[29781]: pgmap v383: 129 pgs: 129 active+clean; 582 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 469 B/s wr, 4 op/s; 0 B/s, 0 objects/s recovering 2024-05-07T02:27:45.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:44 smithi019 ceph-mon[29781]: mds.? [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] up:active 2024-05-07T02:27:45.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:44 smithi019 ceph-mon[29781]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-07T02:27:45.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:44 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/478274967' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-07T02:27:45.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:44 smithi019 ceph-mon[29781]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-07T02:27:45.735 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-05-07T02:27:46.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:45 smithi064 ceph-mon[34699]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-07T02:27:46.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:45 smithi064 ceph-mon[34699]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-07T02:27:46.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:45 smithi064 ceph-mon[34699]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:27:46.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:45 smithi019 ceph-mon[29781]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-07T02:27:46.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:45 smithi019 ceph-mon[29781]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-07T02:27:46.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:45 smithi019 ceph-mon[29781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:27:46.198 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:27:46.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:45 smithi027 ceph-mon[34398]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-07T02:27:46.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:45 smithi027 ceph-mon[34398]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-07T02:27:46.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:45 smithi027 ceph-mon[34398]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:27:47.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:46 smithi064 ceph-mon[34699]: pgmap v384: 129 pgs: 129 active+clean; 585 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s rd, 3.0 KiB/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-05-07T02:27:47.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:46 smithi019 ceph-mon[29781]: pgmap v384: 129 pgs: 129 active+clean; 585 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s rd, 3.0 KiB/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-05-07T02:27:47.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:46 smithi027 ceph-mon[34398]: pgmap v384: 129 pgs: 129 active+clean; 585 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s rd, 3.0 KiB/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2024-05-07T02:27:48.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:48.602+0000 7f9a50200700 1 -- 172.21.15.19:0/3604319703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a48110b80 msgr2=0x7f9a48112fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:48.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:48.602+0000 7f9a50200700 1 --2- 172.21.15.19:0/3604319703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a48110b80 0x7f9a48112fe0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9a4000b0a0 tx=0x7f9a4000b3b0 comp rx=0 tx=0).stop 2024-05-07T02:27:48.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:48.602+0000 7f9a50200700 1 -- 172.21.15.19:0/3604319703 shutdown_connections 2024-05-07T02:27:48.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:48.602+0000 7f9a50200700 1 --2- 172.21.15.19:0/3604319703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a48110b80 0x7f9a48112fe0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:48.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:48.602+0000 7f9a50200700 1 --2- 172.21.15.19:0/3604319703 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a48076e40 0x7f9a48077250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:48.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:48.602+0000 7f9a50200700 1 --2- 172.21.15.19:0/3604319703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a48077790 0x7f9a48110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:48.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:48.602+0000 7f9a50200700 1 -- 172.21.15.19:0/3604319703 >> 172.21.15.19:0/3604319703 conn(0x7f9a480ffea0 msgr2=0x7f9a481022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:27:48.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:48.602+0000 7f9a50200700 1 -- 172.21.15.19:0/3604319703 shutdown_connections 2024-05-07T02:27:48.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:48.602+0000 7f9a50200700 1 -- 172.21.15.19:0/3604319703 wait complete. 2024-05-07T02:27:48.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:48.604+0000 7f9a50200700 1 Processor -- start 2024-05-07T02:27:48.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:48.604+0000 7f9a50200700 1 -- start start 2024-05-07T02:27:48.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:48.605+0000 7f9a50200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a48076e40 0x7f9a4819d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:27:48.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:48.605+0000 7f9a50200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a48077790 0x7f9a4819d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:27:48.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:48.605+0000 7f9a50200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a48110b80 0x7f9a481a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:27:48.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:48.605+0000 7f9a50200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9a481166b0 con 0x7f9a48076e40 2024-05-07T02:27:48.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:48.605+0000 7f9a50200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9a48116530 con 0x7f9a48110b80 2024-05-07T02:27:48.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:48.605+0000 7f9a50200700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9a48116830 con 0x7f9a48077790 2024-05-07T02:27:48.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:48.605+0000 7f9a4d800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a48076e40 0x7f9a4819d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:27:48.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:48.605+0000 7f9a4d800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a48076e40 0x7f9a4819d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37934/0 (socket says 172.21.15.19:37934) 2024-05-07T02:27:48.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:48.605+0000 7f9a4d800700 1 -- 172.21.15.19:0/311689304 learned_addr learned my addr 172.21.15.19:0/311689304 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:27:48.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:48.605+0000 7f9a4e200700 1 --2- 172.21.15.19:0/311689304 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a48110b80 0x7f9a481a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:27:48.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:48.605+0000 7f9a4ce00700 1 --2- 172.21.15.19:0/311689304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a48077790 0x7f9a4819d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:27:48.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:48.606+0000 7f9a4ce00700 1 -- 172.21.15.19:0/311689304 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a48110b80 msgr2=0x7f9a481a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:48.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:48.606+0000 7f9a4ce00700 1 --2- 172.21.15.19:0/311689304 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a48110b80 0x7f9a481a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:48.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:48.606+0000 7f9a4ce00700 1 -- 172.21.15.19:0/311689304 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a48076e40 msgr2=0x7f9a4819d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:48.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:48.606+0000 7f9a4ce00700 1 --2- 172.21.15.19:0/311689304 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a48076e40 0x7f9a4819d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:48.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:48.606+0000 7f9a4ce00700 1 -- 172.21.15.19:0/311689304 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a481a1ed0 con 0x7f9a48077790 2024-05-07T02:27:48.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:48.606+0000 7f9a4d800700 1 --2- 172.21.15.19:0/311689304 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a48076e40 0x7f9a4819d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:27:48.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:48.606+0000 7f9a4ce00700 1 --2- 172.21.15.19:0/311689304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a48077790 0x7f9a4819d5a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9a3400e9f0 tx=0x7f9a3400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:27:48.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:48.606+0000 7f9a46000700 1 -- 172.21.15.19:0/311689304 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3400c9c0 con 0x7f9a48077790 2024-05-07T02:27:48.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:48.606+0000 7f9a46000700 1 -- 172.21.15.19:0/311689304 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a3400cb60 con 0x7f9a48077790 2024-05-07T02:27:48.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:48.606+0000 7f9a50200700 1 -- 172.21.15.19:0/311689304 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a481a21c0 con 0x7f9a48077790 2024-05-07T02:27:48.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:48.606+0000 7f9a46000700 1 -- 172.21.15.19:0/311689304 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a34018680 con 0x7f9a48077790 2024-05-07T02:27:48.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:48.607+0000 7f9a50200700 1 -- 172.21.15.19:0/311689304 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a481a9ae0 con 0x7f9a48077790 2024-05-07T02:27:48.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:48.608+0000 7f9a50200700 1 -- 172.21.15.19:0/311689304 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a480512c0 con 0x7f9a48077790 2024-05-07T02:27:48.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:48.610+0000 7f9a46000700 1 -- 172.21.15.19:0/311689304 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f9a34010430 con 0x7f9a48077790 2024-05-07T02:27:48.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:48.612+0000 7f9a46000700 1 --2- 172.21.15.19:0/311689304 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f9a300b2900 0x7f9a300b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:27:48.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:48.612+0000 7f9a46000700 1 -- 172.21.15.19:0/311689304 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f9a340e2610 con 0x7f9a48077790 2024-05-07T02:27:48.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:48.612+0000 7f9a4d800700 1 --2- 172.21.15.19:0/311689304 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f9a300b2900 0x7f9a300b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:27:48.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:48.613+0000 7f9a4d800700 1 --2- 172.21.15.19:0/311689304 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f9a300b2900 0x7f9a300b4e10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c011630 tx=0x7f9a3c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:27:48.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:48.617+0000 7f9a46000700 1 -- 172.21.15.19:0/311689304 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9a340a9f70 con 0x7f9a48077790 2024-05-07T02:27:48.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:48.911+0000 7f9a50200700 1 -- 172.21.15.19:0/311689304 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f9a481a9cb0 con 0x7f9a48077790 2024-05-07T02:27:49.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:48 smithi064 ceph-mon[34699]: pgmap v385: 129 pgs: 129 active+clean; 587 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4.0 KiB/s wr, 15 op/s; 0 B/s, 0 objects/s recovering 2024-05-07T02:27:49.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:48 smithi019 ceph-mon[29781]: pgmap v385: 129 pgs: 129 active+clean; 587 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4.0 KiB/s wr, 15 op/s; 0 B/s, 0 objects/s recovering 2024-05-07T02:27:49.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:48 smithi027 ceph-mon[34398]: pgmap v385: 129 pgs: 129 active+clean; 587 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4.0 KiB/s wr, 15 op/s; 0 B/s, 0 objects/s recovering 2024-05-07T02:27:49.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:49.787+0000 7f9a46000700 1 -- 172.21.15.19:0/311689304 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v30) ==== 115+0+0 (secure 0 0 0) 0x7f9a340a9870 con 0x7f9a48077790 2024-05-07T02:27:49.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:49.792+0000 7f9a50200700 1 -- 172.21.15.19:0/311689304 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f9a300b2900 msgr2=0x7f9a300b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:49.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:49.792+0000 7f9a50200700 1 --2- 172.21.15.19:0/311689304 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f9a300b2900 0x7f9a300b4e10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c011630 tx=0x7f9a3c009260 comp rx=0 tx=0).stop 2024-05-07T02:27:49.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:49.792+0000 7f9a50200700 1 -- 172.21.15.19:0/311689304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a48077790 msgr2=0x7f9a4819d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:49.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:49.792+0000 7f9a50200700 1 --2- 172.21.15.19:0/311689304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a48077790 0x7f9a4819d5a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9a3400e9f0 tx=0x7f9a3400ed00 comp rx=0 tx=0).stop 2024-05-07T02:27:49.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:49.792+0000 7f9a50200700 1 -- 172.21.15.19:0/311689304 shutdown_connections 2024-05-07T02:27:49.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:49.792+0000 7f9a50200700 1 --2- 172.21.15.19:0/311689304 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f9a300b2900 0x7f9a300b4e10 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:49.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:49.792+0000 7f9a50200700 1 --2- 172.21.15.19:0/311689304 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a48110b80 0x7f9a481a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:49.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:49.792+0000 7f9a50200700 1 --2- 172.21.15.19:0/311689304 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a48076e40 0x7f9a4819d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:49.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:49.792+0000 7f9a50200700 1 --2- 172.21.15.19:0/311689304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9a48077790 0x7f9a4819d5a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:49.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:49.792+0000 7f9a50200700 1 -- 172.21.15.19:0/311689304 >> 172.21.15.19:0/311689304 conn(0x7f9a480ffea0 msgr2=0x7f9a481002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:27:49.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:49.792+0000 7f9a50200700 1 -- 172.21.15.19:0/311689304 shutdown_connections 2024-05-07T02:27:49.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:49.792+0000 7f9a50200700 1 -- 172.21.15.19:0/311689304 wait complete. 2024-05-07T02:27:50.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:49 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/311689304' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-07T02:27:50.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:49 smithi064 ceph-mon[34699]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-07T02:27:50.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:49 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/311689304' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-07T02:27:50.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:49 smithi019 ceph-mon[29781]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-07T02:27:50.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:49 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/311689304' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-07T02:27:50.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:49 smithi027 ceph-mon[34398]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-07T02:27:50.642 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-05-07T02:27:50.643 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-05-07T02:27:50.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:50 smithi019 ceph-mon[29781]: pgmap v386: 129 pgs: 129 active+clean; 587 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-05-07T02:27:50.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:50 smithi019 ceph-mon[29781]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-07T02:27:50.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:50 smithi019 ceph-mon[29781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:27:51.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:50 smithi064 ceph-mon[34699]: pgmap v386: 129 pgs: 129 active+clean; 587 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-05-07T02:27:51.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:50 smithi064 ceph-mon[34699]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-07T02:27:51.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:50 smithi064 ceph-mon[34699]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:27:51.109 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:27:51.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:50 smithi027 ceph-mon[34398]: pgmap v386: 129 pgs: 129 active+clean; 587 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-05-07T02:27:51.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:50 smithi027 ceph-mon[34398]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-07T02:27:51.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:50 smithi027 ceph-mon[34398]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:27:53.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:52 smithi064 ceph-mon[34699]: pgmap v387: 129 pgs: 129 active+clean; 587 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.0 KiB/s wr, 20 op/s 2024-05-07T02:27:53.129 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:52 smithi019 ceph-mon[29781]: pgmap v387: 129 pgs: 129 active+clean; 587 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.0 KiB/s wr, 20 op/s 2024-05-07T02:27:53.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:52 smithi027 ceph-mon[34398]: pgmap v387: 129 pgs: 129 active+clean; 587 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.0 KiB/s wr, 20 op/s 2024-05-07T02:27:53.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.383+0000 7f97c1600700 1 -- 172.21.15.19:0/3981586558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97bc108f20 msgr2=0x7f97bc109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:53.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.383+0000 7f97c1600700 1 --2- 172.21.15.19:0/3981586558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97bc108f20 0x7f97bc109330 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f97a8009920 tx=0x7f97a8009c30 comp rx=0 tx=0).stop 2024-05-07T02:27:53.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.384+0000 7f97c1600700 1 -- 172.21.15.19:0/3981586558 shutdown_connections 2024-05-07T02:27:53.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.384+0000 7f97c1600700 1 --2- 172.21.15.19:0/3981586558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97bc109870 0x7f97bc1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:53.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.384+0000 7f97c1600700 1 --2- 172.21.15.19:0/3981586558 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97bc1045f0 0x7f97bc104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:53.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.384+0000 7f97c1600700 1 --2- 172.21.15.19:0/3981586558 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97bc108f20 0x7f97bc109330 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:53.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.384+0000 7f97c1600700 1 -- 172.21.15.19:0/3981586558 >> 172.21.15.19:0/3981586558 conn(0x7f97bc100040 msgr2=0x7f97bc102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:27:53.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.384+0000 7f97c1600700 1 -- 172.21.15.19:0/3981586558 shutdown_connections 2024-05-07T02:27:53.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.384+0000 7f97c1600700 1 -- 172.21.15.19:0/3981586558 wait complete. 2024-05-07T02:27:53.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.385+0000 7f97c1600700 1 Processor -- start 2024-05-07T02:27:53.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.386+0000 7f97c1600700 1 -- start start 2024-05-07T02:27:53.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.386+0000 7f97c1600700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97bc1045f0 0x7f97bc19d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:27:53.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.386+0000 7f97c1600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97bc108f20 0x7f97bc19d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:27:53.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.386+0000 7f97c1600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97bc109870 0x7f97bc1a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:27:53.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.386+0000 7f97c1600700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc1144d0 con 0x7f97bc109870 2024-05-07T02:27:53.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.386+0000 7f97c1600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc114350 con 0x7f97bc108f20 2024-05-07T02:27:53.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.386+0000 7f97c1600700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc114650 con 0x7f97bc1045f0 2024-05-07T02:27:53.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.387+0000 7f97bb400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97bc109870 0x7f97bc1a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:27:53.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.387+0000 7f97baa00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97bc1045f0 0x7f97bc19d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:27:53.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.387+0000 7f97bb400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97bc109870 0x7f97bc1a1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42236/0 (socket says 172.21.15.19:42236) 2024-05-07T02:27:53.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.387+0000 7f97bb400700 1 -- 172.21.15.19:0/3684579637 learned_addr learned my addr 172.21.15.19:0/3684579637 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:27:53.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.387+0000 7f97baa00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97bc1045f0 0x7f97bc19d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:48576/0 (socket says 172.21.15.19:48576) 2024-05-07T02:27:53.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.387+0000 7f97ba000700 1 --2- 172.21.15.19:0/3684579637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97bc108f20 0x7f97bc19d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:27:53.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.387+0000 7f97baa00700 1 -- 172.21.15.19:0/3684579637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97bc108f20 msgr2=0x7f97bc19d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:53.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.387+0000 7f97baa00700 1 --2- 172.21.15.19:0/3684579637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97bc108f20 0x7f97bc19d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:53.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.387+0000 7f97baa00700 1 -- 172.21.15.19:0/3684579637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97bc109870 msgr2=0x7f97bc1a1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:53.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.387+0000 7f97baa00700 1 --2- 172.21.15.19:0/3684579637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97bc109870 0x7f97bc1a1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:53.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.387+0000 7f97baa00700 1 -- 172.21.15.19:0/3684579637 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97bc1a2110 con 0x7f97bc1045f0 2024-05-07T02:27:53.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.387+0000 7f97ba000700 1 --2- 172.21.15.19:0/3684579637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97bc108f20 0x7f97bc19d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:27:53.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.388+0000 7f97baa00700 1 --2- 172.21.15.19:0/3684579637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97bc1045f0 0x7f97bc19d240 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f97ac00eb90 tx=0x7f97ac00eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:27:53.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.388+0000 7f97b3400700 1 -- 172.21.15.19:0/3684579637 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ac00cc90 con 0x7f97bc1045f0 2024-05-07T02:27:53.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.388+0000 7f97b3400700 1 -- 172.21.15.19:0/3684579637 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97ac00ce30 con 0x7f97bc1045f0 2024-05-07T02:27:53.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.388+0000 7f97c1600700 1 -- 172.21.15.19:0/3684579637 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97bc1a2400 con 0x7f97bc1045f0 2024-05-07T02:27:53.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.388+0000 7f97b3400700 1 -- 172.21.15.19:0/3684579637 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ac010430 con 0x7f97bc1045f0 2024-05-07T02:27:53.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.388+0000 7f97c1600700 1 -- 172.21.15.19:0/3684579637 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97bc1a9d20 con 0x7f97bc1045f0 2024-05-07T02:27:53.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.389+0000 7f97c1600700 1 -- 172.21.15.19:0/3684579637 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97bc0512c0 con 0x7f97bc1045f0 2024-05-07T02:27:53.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.392+0000 7f97b3400700 1 -- 172.21.15.19:0/3684579637 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f97ac003ef0 con 0x7f97bc1045f0 2024-05-07T02:27:53.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.394+0000 7f97b3400700 1 --2- 172.21.15.19:0/3684579637 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f97a00b2900 0x7f97a00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:27:53.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.394+0000 7f97b3400700 1 -- 172.21.15.19:0/3684579637 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f97ac0e2850 con 0x7f97bc1045f0 2024-05-07T02:27:53.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.394+0000 7f97ba000700 1 --2- 172.21.15.19:0/3684579637 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f97a00b2900 0x7f97a00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:27:53.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.395+0000 7f97ba000700 1 --2- 172.21.15.19:0/3684579637 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f97a00b2900 0x7f97a00b4e10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f97a80098f0 tx=0x7f97a8019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:27:53.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.398+0000 7f97b3400700 1 -- 172.21.15.19:0/3684579637 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f97ac0aa1b0 con 0x7f97bc1045f0 2024-05-07T02:27:53.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.685+0000 7f97c1600700 1 -- 172.21.15.19:0/3684579637 --> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f97bc076a00 con 0x7f97a00b2900 2024-05-07T02:27:53.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.721+0000 7f97b3400700 1 -- 172.21.15.19:0/3684579637 <== mgr.14152 v2:172.21.15.19:6800/2059564551 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f97bc076a00 con 0x7f97a00b2900 2024-05-07T02:27:53.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.725+0000 7f97c1600700 1 -- 172.21.15.19:0/3684579637 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f97a00b2900 msgr2=0x7f97a00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:53.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.725+0000 7f97c1600700 1 --2- 172.21.15.19:0/3684579637 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f97a00b2900 0x7f97a00b4e10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f97a80098f0 tx=0x7f97a8019040 comp rx=0 tx=0).stop 2024-05-07T02:27:53.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.725+0000 7f97c1600700 1 -- 172.21.15.19:0/3684579637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97bc1045f0 msgr2=0x7f97bc19d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:53.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.725+0000 7f97c1600700 1 --2- 172.21.15.19:0/3684579637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97bc1045f0 0x7f97bc19d240 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f97ac00eb90 tx=0x7f97ac00eea0 comp rx=0 tx=0).stop 2024-05-07T02:27:53.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.725+0000 7f97c1600700 1 -- 172.21.15.19:0/3684579637 shutdown_connections 2024-05-07T02:27:53.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.725+0000 7f97c1600700 1 --2- 172.21.15.19:0/3684579637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97bc109870 0x7f97bc1a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:53.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.725+0000 7f97c1600700 1 --2- 172.21.15.19:0/3684579637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f97bc1045f0 0x7f97bc19d240 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:53.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.725+0000 7f97c1600700 1 --2- 172.21.15.19:0/3684579637 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f97a00b2900 0x7f97a00b4e10 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:53.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.725+0000 7f97c1600700 1 --2- 172.21.15.19:0/3684579637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97bc108f20 0x7f97bc19d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:53.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.725+0000 7f97c1600700 1 -- 172.21.15.19:0/3684579637 >> 172.21.15.19:0/3684579637 conn(0x7f97bc100040 msgr2=0x7f97bc10a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:27:53.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.725+0000 7f97c1600700 1 -- 172.21.15.19:0/3684579637 shutdown_connections 2024-05-07T02:27:53.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:53.726+0000 7f97c1600700 1 -- 172.21.15.19:0/3684579637 wait complete. 2024-05-07T02:27:54.120 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:27:53 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:27:53.694+0000 7f292c600700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-07T02:27:54.121 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:27:53 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:27:53.694+0000 7f292c600700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-07T02:27:54.121 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:27:53 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:27:53.694+0000 7f292c600700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-07T02:27:54.121 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:27:53 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:27:53.694+0000 7f292c600700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-07T02:27:54.121 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:27:53 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:27:53.694+0000 7f292c600700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-07T02:27:54.593 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-07T02:27:55.082 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:27:55.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:54 smithi064 ceph-mon[34699]: pgmap v388: 129 pgs: 129 active+clean; 587 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.7 KiB/s wr, 18 op/s 2024-05-07T02:27:55.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:54 smithi064 ceph-mon[34699]: from='client.24715 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:27:55.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:54 smithi019 ceph-mon[29781]: pgmap v388: 129 pgs: 129 active+clean; 587 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.7 KiB/s wr, 18 op/s 2024-05-07T02:27:55.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:54 smithi019 ceph-mon[29781]: from='client.24715 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:27:55.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:54 smithi027 ceph-mon[34398]: pgmap v388: 129 pgs: 129 active+clean; 587 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.7 KiB/s wr, 18 op/s 2024-05-07T02:27:55.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:54 smithi027 ceph-mon[34398]: from='client.24715 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:27:56.889 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:56 smithi019 ceph-mon[29781]: pgmap v389: 129 pgs: 129 active+clean; 587 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.7 KiB/s wr, 19 op/s 2024-05-07T02:27:56.889 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:56 smithi019 ceph-mon[29781]: mgrmap e17: x(active, since 10m), standbys: z, y 2024-05-07T02:27:57.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:56 smithi064 ceph-mon[34699]: pgmap v389: 129 pgs: 129 active+clean; 587 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.7 KiB/s wr, 19 op/s 2024-05-07T02:27:57.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:56 smithi064 ceph-mon[34699]: mgrmap e17: x(active, since 10m), standbys: z, y 2024-05-07T02:27:57.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:56 smithi027 ceph-mon[34398]: pgmap v389: 129 pgs: 129 active+clean; 587 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.7 KiB/s wr, 19 op/s 2024-05-07T02:27:57.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:56 smithi027 ceph-mon[34398]: mgrmap e17: x(active, since 10m), standbys: z, y 2024-05-07T02:27:57.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.524+0000 7f0104a00700 1 -- 172.21.15.19:0/2443455246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f010010b110 msgr2=0x7f0100117620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:57.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.524+0000 7f0104a00700 1 --2- 172.21.15.19:0/2443455246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f010010b110 0x7f0100117620 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f00f800b0a0 tx=0x7f00f800b3b0 comp rx=0 tx=0).stop 2024-05-07T02:27:57.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.524+0000 7f0104a00700 1 -- 172.21.15.19:0/2443455246 shutdown_connections 2024-05-07T02:27:57.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.524+0000 7f0104a00700 1 --2- 172.21.15.19:0/2443455246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f010010b110 0x7f0100117620 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:57.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.524+0000 7f0104a00700 1 --2- 172.21.15.19:0/2443455246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f010010f6c0 0x7f010010fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:57.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.524+0000 7f0104a00700 1 --2- 172.21.15.19:0/2443455246 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f010010a7c0 0x7f010010abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:57.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.524+0000 7f0104a00700 1 -- 172.21.15.19:0/2443455246 >> 172.21.15.19:0/2443455246 conn(0x7f0100076b00 msgr2=0x7f0100076f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:27:57.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.524+0000 7f0104a00700 1 -- 172.21.15.19:0/2443455246 shutdown_connections 2024-05-07T02:27:57.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.524+0000 7f0104a00700 1 -- 172.21.15.19:0/2443455246 wait complete. 2024-05-07T02:27:57.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.525+0000 7f0104a00700 1 Processor -- start 2024-05-07T02:27:57.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.525+0000 7f0104a00700 1 -- start start 2024-05-07T02:27:57.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.526+0000 7f0104a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f010010a7c0 0x7f01001a11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:27:57.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.526+0000 7f0104a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f010010b110 0x7f01001a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:27:57.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.526+0000 7f0104a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f010010f6c0 0x7f01001a59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:27:57.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.526+0000 7f0104a00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f010011a8f0 con 0x7f010010f6c0 2024-05-07T02:27:57.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.526+0000 7f0104a00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f010011a770 con 0x7f010010b110 2024-05-07T02:27:57.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.526+0000 7f0104a00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f010011aa70 con 0x7f010010a7c0 2024-05-07T02:27:57.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.526+0000 7f00ff400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f010010a7c0 0x7f01001a11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:27:57.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.526+0000 7f00ff400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f010010a7c0 0x7f01001a11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:48626/0 (socket says 172.21.15.19:48626) 2024-05-07T02:27:57.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.526+0000 7f00ff400700 1 -- 172.21.15.19:0/2911365726 learned_addr learned my addr 172.21.15.19:0/2911365726 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:27:57.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.526+0000 7f00ffe00700 1 --2- 172.21.15.19:0/2911365726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f010010f6c0 0x7f01001a59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:27:57.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.526+0000 7f00fea00700 1 --2- 172.21.15.19:0/2911365726 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f010010b110 0x7f01001a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:27:57.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.527+0000 7f00ff400700 1 -- 172.21.15.19:0/2911365726 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f010010b110 msgr2=0x7f01001a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:57.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.527+0000 7f00ff400700 1 --2- 172.21.15.19:0/2911365726 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f010010b110 0x7f01001a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:57.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.527+0000 7f00ff400700 1 -- 172.21.15.19:0/2911365726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f010010f6c0 msgr2=0x7f01001a59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:57.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.527+0000 7f00ff400700 1 --2- 172.21.15.19:0/2911365726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f010010f6c0 0x7f01001a59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:57.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.527+0000 7f00ff400700 1 -- 172.21.15.19:0/2911365726 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01001a6070 con 0x7f010010a7c0 2024-05-07T02:27:57.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.527+0000 7f00ffe00700 1 --2- 172.21.15.19:0/2911365726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f010010f6c0 0x7f01001a59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:27:57.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.527+0000 7f00ff400700 1 --2- 172.21.15.19:0/2911365726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f010010a7c0 0x7f01001a11a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f00f000e9f0 tx=0x7f00f000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:27:57.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.527+0000 7f00f7e00700 1 -- 172.21.15.19:0/2911365726 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f0004a30 con 0x7f010010a7c0 2024-05-07T02:27:57.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.527+0000 7f00f7e00700 1 -- 172.21.15.19:0/2911365726 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00f00054d0 con 0x7f010010a7c0 2024-05-07T02:27:57.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.527+0000 7f00f7e00700 1 -- 172.21.15.19:0/2911365726 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f000cd00 con 0x7f010010a7c0 2024-05-07T02:27:57.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.527+0000 7f0104a00700 1 -- 172.21.15.19:0/2911365726 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01001a6360 con 0x7f010010a7c0 2024-05-07T02:27:57.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.527+0000 7f0104a00700 1 -- 172.21.15.19:0/2911365726 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01001adc80 con 0x7f010010a7c0 2024-05-07T02:27:57.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.528+0000 7f0104a00700 1 -- 172.21.15.19:0/2911365726 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01000512c0 con 0x7f010010a7c0 2024-05-07T02:27:57.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.531+0000 7f00f7e00700 1 -- 172.21.15.19:0/2911365726 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f00f0005000 con 0x7f010010a7c0 2024-05-07T02:27:57.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.533+0000 7f00f7e00700 1 --2- 172.21.15.19:0/2911365726 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f00e40b2910 0x7f00e40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:27:57.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.533+0000 7f00f7e00700 1 -- 172.21.15.19:0/2911365726 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f00f00e2b00 con 0x7f010010a7c0 2024-05-07T02:27:57.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.533+0000 7f00fea00700 1 --2- 172.21.15.19:0/2911365726 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f00e40b2910 0x7f00e40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:27:57.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.534+0000 7f00fea00700 1 --2- 172.21.15.19:0/2911365726 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f00e40b2910 0x7f00e40b4e20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0100077e50 tx=0x7f00ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:27:57.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.535+0000 7f00f7e00700 1 -- 172.21.15.19:0/2911365726 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f00f0054df0 con 0x7f010010a7c0 2024-05-07T02:27:57.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.794+0000 7f0104a00700 1 -- 172.21.15.19:0/2911365726 --> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f0100063c00 con 0x7f00e40b2910 2024-05-07T02:27:57.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.834+0000 7f00f7e00700 1 -- 172.21.15.19:0/2911365726 <== mgr.14152 v2:172.21.15.19:6800/2059564551 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0100063c00 con 0x7f00e40b2910 2024-05-07T02:27:57.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.836+0000 7f0104a00700 1 -- 172.21.15.19:0/2911365726 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f00e40b2910 msgr2=0x7f00e40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:57.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.836+0000 7f0104a00700 1 --2- 172.21.15.19:0/2911365726 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f00e40b2910 0x7f00e40b4e20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0100077e50 tx=0x7f00ec009260 comp rx=0 tx=0).stop 2024-05-07T02:27:57.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.836+0000 7f0104a00700 1 -- 172.21.15.19:0/2911365726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f010010a7c0 msgr2=0x7f01001a11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:27:57.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.836+0000 7f0104a00700 1 --2- 172.21.15.19:0/2911365726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f010010a7c0 0x7f01001a11a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f00f000e9f0 tx=0x7f00f000ed00 comp rx=0 tx=0).stop 2024-05-07T02:27:57.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.837+0000 7f0104a00700 1 -- 172.21.15.19:0/2911365726 shutdown_connections 2024-05-07T02:27:57.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.837+0000 7f0104a00700 1 --2- 172.21.15.19:0/2911365726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f010010f6c0 0x7f01001a59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:57.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.837+0000 7f0104a00700 1 --2- 172.21.15.19:0/2911365726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f010010a7c0 0x7f01001a11a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:57.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.837+0000 7f0104a00700 1 --2- 172.21.15.19:0/2911365726 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f00e40b2910 0x7f00e40b4e20 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:57.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.837+0000 7f0104a00700 1 --2- 172.21.15.19:0/2911365726 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f010010b110 0x7f01001a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:27:57.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.837+0000 7f0104a00700 1 -- 172.21.15.19:0/2911365726 >> 172.21.15.19:0/2911365726 conn(0x7f0100076b00 msgr2=0x7f01001087c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:27:57.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.837+0000 7f0104a00700 1 -- 172.21.15.19:0/2911365726 shutdown_connections 2024-05-07T02:27:57.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:27:57.837+0000 7f0104a00700 1 -- 172.21.15.19:0/2911365726 wait complete. 2024-05-07T02:27:58.675 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-07T02:27:58.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:58 smithi019 ceph-mon[29781]: pgmap v390: 129 pgs: 129 active+clean; 593 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 1.8 KiB/s wr, 15 op/s 2024-05-07T02:27:58.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:58 smithi019 ceph-mon[29781]: from='client.24727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:27:59.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:58 smithi064 ceph-mon[34699]: pgmap v390: 129 pgs: 129 active+clean; 593 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 1.8 KiB/s wr, 15 op/s 2024-05-07T02:27:59.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:58 smithi064 ceph-mon[34699]: from='client.24727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:27:59.151 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:27:59.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:58 smithi027 ceph-mon[34398]: pgmap v390: 129 pgs: 129 active+clean; 593 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 1.8 KiB/s wr, 15 op/s 2024-05-07T02:27:59.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:58 smithi027 ceph-mon[34398]: from='client.24727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:28:00.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:27:59 smithi019 ceph-mon[29781]: pgmap v391: 129 pgs: 129 active+clean; 593 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1023 B/s wr, 11 op/s 2024-05-07T02:28:00.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:27:59 smithi027 ceph-mon[34398]: pgmap v391: 129 pgs: 129 active+clean; 593 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1023 B/s wr, 11 op/s 2024-05-07T02:28:00.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:27:59 smithi064 ceph-mon[34699]: pgmap v391: 129 pgs: 129 active+clean; 593 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1023 B/s wr, 11 op/s 2024-05-07T02:28:01.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.617+0000 7f564c000700 1 -- 172.21.15.19:0/1298528350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f564410b2b0 msgr2=0x7f5644117830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:01.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.617+0000 7f564c000700 1 --2- 172.21.15.19:0/1298528350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f564410b2b0 0x7f5644117830 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f563c009f90 tx=0x7f563c00b330 comp rx=0 tx=0).stop 2024-05-07T02:28:01.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.618+0000 7f564c000700 1 -- 172.21.15.19:0/1298528350 shutdown_connections 2024-05-07T02:28:01.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.618+0000 7f564c000700 1 --2- 172.21.15.19:0/1298528350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f564410b2b0 0x7f5644117830 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:01.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.618+0000 7f564c000700 1 --2- 172.21.15.19:0/1298528350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5644110990 0x7f5644110da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:01.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.618+0000 7f564c000700 1 --2- 172.21.15.19:0/1298528350 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f564410a960 0x7f564410ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:01.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.618+0000 7f564c000700 1 -- 172.21.15.19:0/1298528350 >> 172.21.15.19:0/1298528350 conn(0x7f5644076d30 msgr2=0x7f5644077140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:28:01.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.618+0000 7f564c000700 1 -- 172.21.15.19:0/1298528350 shutdown_connections 2024-05-07T02:28:01.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.618+0000 7f564c000700 1 -- 172.21.15.19:0/1298528350 wait complete. 2024-05-07T02:28:01.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.620+0000 7f564c000700 1 Processor -- start 2024-05-07T02:28:01.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.620+0000 7f564c000700 1 -- start start 2024-05-07T02:28:01.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.620+0000 7f564c000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f564410a960 0x7f56441a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:01.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.620+0000 7f564c000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f564410b2b0 0x7f56441a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:01.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.620+0000 7f564c000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5644110990 0x7f56441a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:01.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.620+0000 7f564c000700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f564411ab10 con 0x7f5644110990 2024-05-07T02:28:01.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.620+0000 7f564c000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f564411a990 con 0x7f564410b2b0 2024-05-07T02:28:01.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.620+0000 7f564c000700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f564411ac90 con 0x7f564410a960 2024-05-07T02:28:01.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.621+0000 7f564a000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5644110990 0x7f56441a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:01.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.621+0000 7f5648c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f564410b2b0 0x7f56441a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:01.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.621+0000 7f564a000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5644110990 0x7f56441a5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54758/0 (socket says 172.21.15.19:54758) 2024-05-07T02:28:01.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.621+0000 7f564a000700 1 -- 172.21.15.19:0/2130083928 learned_addr learned my addr 172.21.15.19:0/2130083928 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:28:01.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.621+0000 7f5649600700 1 --2- 172.21.15.19:0/2130083928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f564410a960 0x7f56441a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:01.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.621+0000 7f5648c00700 1 -- 172.21.15.19:0/2130083928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f564410a960 msgr2=0x7f56441a1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:01.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.621+0000 7f5648c00700 1 --2- 172.21.15.19:0/2130083928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f564410a960 0x7f56441a1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:01.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.621+0000 7f5648c00700 1 -- 172.21.15.19:0/2130083928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5644110990 msgr2=0x7f56441a5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:01.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.621+0000 7f5648c00700 1 --2- 172.21.15.19:0/2130083928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5644110990 0x7f56441a5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:01.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.621+0000 7f5648c00700 1 -- 172.21.15.19:0/2130083928 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56441a6330 con 0x7f564410b2b0 2024-05-07T02:28:01.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.622+0000 7f5648c00700 1 --2- 172.21.15.19:0/2130083928 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f564410b2b0 0x7f56441a1a00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f563000e9c0 tx=0x7f563000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:28:01.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.622+0000 7f5642000700 1 -- 172.21.15.19:0/2130083928 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f563000c9c0 con 0x7f564410b2b0 2024-05-07T02:28:01.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.622+0000 7f5642000700 1 -- 172.21.15.19:0/2130083928 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f563000cb60 con 0x7f564410b2b0 2024-05-07T02:28:01.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.622+0000 7f5642000700 1 -- 172.21.15.19:0/2130083928 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5630018680 con 0x7f564410b2b0 2024-05-07T02:28:01.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.622+0000 7f564c000700 1 -- 172.21.15.19:0/2130083928 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56441a6620 con 0x7f564410b2b0 2024-05-07T02:28:01.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.622+0000 7f564c000700 1 -- 172.21.15.19:0/2130083928 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56441adf40 con 0x7f564410b2b0 2024-05-07T02:28:01.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.624+0000 7f564c000700 1 -- 172.21.15.19:0/2130083928 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f564410c0a0 con 0x7f564410b2b0 2024-05-07T02:28:01.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.626+0000 7f5642000700 1 -- 172.21.15.19:0/2130083928 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f563004aae0 con 0x7f564410b2b0 2024-05-07T02:28:01.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.627+0000 7f5642000700 1 --2- 172.21.15.19:0/2130083928 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f562c0b2910 0x7f562c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:01.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.628+0000 7f5642000700 1 -- 172.21.15.19:0/2130083928 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f56300e2a00 con 0x7f564410b2b0 2024-05-07T02:28:01.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.628+0000 7f5649600700 1 --2- 172.21.15.19:0/2130083928 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f562c0b2910 0x7f562c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:01.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.629+0000 7f5649600700 1 --2- 172.21.15.19:0/2130083928 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f562c0b2910 0x7f562c0b4e20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5638009750 tx=0x7f5638009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:28:01.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.630+0000 7f5642000700 1 -- 172.21.15.19:0/2130083928 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f56300aa200 con 0x7f564410b2b0 2024-05-07T02:28:01.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.911+0000 7f564c000700 1 -- 172.21.15.19:0/2130083928 --> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f5644063c00 con 0x7f562c0b2910 2024-05-07T02:28:01.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.923+0000 7f5642000700 1 -- 172.21.15.19:0/2130083928 <== mgr.14152 v2:172.21.15.19:6800/2059564551 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f5644063c00 con 0x7f562c0b2910 2024-05-07T02:28:01.923 INFO:teuthology.orchestra.run.smithi019.stdout:/volumes/qa/sv_0/07cfa20b-e5ed-44d6-87e2-d73a6193674f 2024-05-07T02:28:01.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.926+0000 7f564c000700 1 -- 172.21.15.19:0/2130083928 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f562c0b2910 msgr2=0x7f562c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:01.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.926+0000 7f564c000700 1 --2- 172.21.15.19:0/2130083928 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f562c0b2910 0x7f562c0b4e20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5638009750 tx=0x7f5638009310 comp rx=0 tx=0).stop 2024-05-07T02:28:01.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.926+0000 7f564c000700 1 -- 172.21.15.19:0/2130083928 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f564410b2b0 msgr2=0x7f56441a1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:01.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.926+0000 7f564c000700 1 --2- 172.21.15.19:0/2130083928 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f564410b2b0 0x7f56441a1a00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f563000e9c0 tx=0x7f563000ecd0 comp rx=0 tx=0).stop 2024-05-07T02:28:01.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.926+0000 7f564c000700 1 -- 172.21.15.19:0/2130083928 shutdown_connections 2024-05-07T02:28:01.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.926+0000 7f564c000700 1 --2- 172.21.15.19:0/2130083928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5644110990 0x7f56441a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:01.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.926+0000 7f564c000700 1 --2- 172.21.15.19:0/2130083928 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f564410a960 0x7f56441a1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:01.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.926+0000 7f564c000700 1 --2- 172.21.15.19:0/2130083928 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f562c0b2910 0x7f562c0b4e20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:01.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.926+0000 7f564c000700 1 --2- 172.21.15.19:0/2130083928 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f564410b2b0 0x7f56441a1a00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:01.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.926+0000 7f564c000700 1 -- 172.21.15.19:0/2130083928 >> 172.21.15.19:0/2130083928 conn(0x7f5644076d30 msgr2=0x7f56441088c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:28:01.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.927+0000 7f564c000700 1 -- 172.21.15.19:0/2130083928 shutdown_connections 2024-05-07T02:28:01.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:01.927+0000 7f564c000700 1 -- 172.21.15.19:0/2130083928 wait complete. 2024-05-07T02:28:02.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:02 smithi019 ceph-mon[29781]: pgmap v392: 129 pgs: 129 active+clean; 612 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-05-07T02:28:02.791 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-07T02:28:02.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:02 smithi064 ceph-mon[34699]: pgmap v392: 129 pgs: 129 active+clean; 612 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-05-07T02:28:02.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:02 smithi027 ceph-mon[34398]: pgmap v392: 129 pgs: 129 active+clean; 612 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-05-07T02:28:03.271 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:28:03.697 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:03 smithi019 ceph-mon[29781]: from='client.24737 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:28:03.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:03 smithi064 ceph-mon[34699]: from='client.24737 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:28:03.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:03 smithi027 ceph-mon[34398]: from='client.24737 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:28:04.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:04 smithi019 ceph-mon[29781]: pgmap v393: 129 pgs: 129 active+clean; 612 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-05-07T02:28:04.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:04 smithi064 ceph-mon[34699]: pgmap v393: 129 pgs: 129 active+clean; 612 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-05-07T02:28:04.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:04 smithi027 ceph-mon[34398]: pgmap v393: 129 pgs: 129 active+clean; 612 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-05-07T02:28:05.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:05.782+0000 7fcb43200700 1 -- 172.21.15.19:0/3220352443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb4410b8a0 msgr2=0x7fcb44113200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:05.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:05.782+0000 7fcb43200700 1 --2- 172.21.15.19:0/3220352443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb4410b8a0 0x7fcb44113200 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c009fc0 tx=0x7fcb3c00b330 comp rx=0 tx=0).stop 2024-05-07T02:28:05.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:05.782+0000 7fcb43200700 1 -- 172.21.15.19:0/3220352443 shutdown_connections 2024-05-07T02:28:05.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:05.782+0000 7fcb43200700 1 --2- 172.21.15.19:0/3220352443 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb4410b8a0 0x7fcb44113200 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:05.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:05.782+0000 7fcb43200700 1 --2- 172.21.15.19:0/3220352443 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb44106620 0x7fcb44106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:05.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:05.782+0000 7fcb43200700 1 --2- 172.21.15.19:0/3220352443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb4410af50 0x7fcb4410b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:05.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:05.782+0000 7fcb43200700 1 -- 172.21.15.19:0/3220352443 >> 172.21.15.19:0/3220352443 conn(0x7fcb44102030 msgr2=0x7fcb44104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:28:05.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:05.783+0000 7fcb43200700 1 -- 172.21.15.19:0/3220352443 shutdown_connections 2024-05-07T02:28:05.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:05.783+0000 7fcb43200700 1 -- 172.21.15.19:0/3220352443 wait complete. 2024-05-07T02:28:05.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:05.784+0000 7fcb43200700 1 Processor -- start 2024-05-07T02:28:05.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:05.785+0000 7fcb43200700 1 -- start start 2024-05-07T02:28:05.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:05.785+0000 7fcb43200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb44106620 0x7fcb4419ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:05.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:05.785+0000 7fcb43200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb4410af50 0x7fcb4419d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:05.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:05.785+0000 7fcb43200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb4410b8a0 0x7fcb441a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:05.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:05.785+0000 7fcb43200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcb44116520 con 0x7fcb44106620 2024-05-07T02:28:05.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:05.785+0000 7fcb43200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcb441163a0 con 0x7fcb4410af50 2024-05-07T02:28:05.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:05.785+0000 7fcb43200700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcb441166a0 con 0x7fcb4410b8a0 2024-05-07T02:28:05.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:05.786+0000 7fcb41e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb44106620 0x7fcb4419ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:05.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:05.786+0000 7fcb41400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb4410af50 0x7fcb4419d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:05.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:05.786+0000 7fcb41e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb44106620 0x7fcb4419ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54770/0 (socket says 172.21.15.19:54770) 2024-05-07T02:28:05.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:05.786+0000 7fcb41e00700 1 -- 172.21.15.19:0/4167186743 learned_addr learned my addr 172.21.15.19:0/4167186743 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:28:05.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:05.786+0000 7fcb42800700 1 --2- 172.21.15.19:0/4167186743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb4410b8a0 0x7fcb441a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:05.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:05.786+0000 7fcb41e00700 1 -- 172.21.15.19:0/4167186743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb4410b8a0 msgr2=0x7fcb441a1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:05.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:05.786+0000 7fcb41e00700 1 --2- 172.21.15.19:0/4167186743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb4410b8a0 0x7fcb441a1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:05.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:05.786+0000 7fcb41e00700 1 -- 172.21.15.19:0/4167186743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb4410af50 msgr2=0x7fcb4419d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:05.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:05.786+0000 7fcb41e00700 1 --2- 172.21.15.19:0/4167186743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb4410af50 0x7fcb4419d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:05.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:05.786+0000 7fcb41e00700 1 -- 172.21.15.19:0/4167186743 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb441a1ce0 con 0x7fcb44106620 2024-05-07T02:28:05.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:05.786+0000 7fcb41400700 1 --2- 172.21.15.19:0/4167186743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb4410af50 0x7fcb4419d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:28:05.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:05.787+0000 7fcb41e00700 1 --2- 172.21.15.19:0/4167186743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb44106620 0x7fcb4419ceb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcb3400ec10 tx=0x7fcb3400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:28:05.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:05.787+0000 7fcb3aa00700 1 -- 172.21.15.19:0/4167186743 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3400cc90 con 0x7fcb44106620 2024-05-07T02:28:05.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:05.787+0000 7fcb3aa00700 1 -- 172.21.15.19:0/4167186743 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb3400ce30 con 0x7fcb44106620 2024-05-07T02:28:05.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:05.787+0000 7fcb43200700 1 -- 172.21.15.19:0/4167186743 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb441a1fd0 con 0x7fcb44106620 2024-05-07T02:28:05.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:05.787+0000 7fcb3aa00700 1 -- 172.21.15.19:0/4167186743 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb34010430 con 0x7fcb44106620 2024-05-07T02:28:05.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:05.787+0000 7fcb43200700 1 -- 172.21.15.19:0/4167186743 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb441a98f0 con 0x7fcb44106620 2024-05-07T02:28:05.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:05.789+0000 7fcb43200700 1 -- 172.21.15.19:0/4167186743 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb440512c0 con 0x7fcb44106620 2024-05-07T02:28:05.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:05.791+0000 7fcb3aa00700 1 -- 172.21.15.19:0/4167186743 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fcb34004000 con 0x7fcb44106620 2024-05-07T02:28:05.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:05.793+0000 7fcb3aa00700 1 --2- 172.21.15.19:0/4167186743 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fcb280b2910 0x7fcb280b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:05.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:05.793+0000 7fcb3aa00700 1 -- 172.21.15.19:0/4167186743 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fcb340e2a00 con 0x7fcb44106620 2024-05-07T02:28:05.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:05.793+0000 7fcb41400700 1 --2- 172.21.15.19:0/4167186743 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fcb280b2910 0x7fcb280b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:05.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:05.794+0000 7fcb41400700 1 --2- 172.21.15.19:0/4167186743 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fcb280b2910 0x7fcb280b4e20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcb30009680 tx=0x7fcb30009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:28:05.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:05.798+0000 7fcb3aa00700 1 -- 172.21.15.19:0/4167186743 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcb340aa200 con 0x7fcb44106620 2024-05-07T02:28:06.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:06.072+0000 7fcb43200700 1 -- 172.21.15.19:0/4167186743 --> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fcb44063c00 con 0x7fcb280b2910 2024-05-07T02:28:06.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:06.114+0000 7fcb3aa00700 1 -- 172.21.15.19:0/4167186743 <== mgr.14152 v2:172.21.15.19:6800/2059564551 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcb44063c00 con 0x7fcb280b2910 2024-05-07T02:28:06.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:06.118+0000 7fcb43200700 1 -- 172.21.15.19:0/4167186743 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fcb280b2910 msgr2=0x7fcb280b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:06.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:06.118+0000 7fcb43200700 1 --2- 172.21.15.19:0/4167186743 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fcb280b2910 0x7fcb280b4e20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcb30009680 tx=0x7fcb30009260 comp rx=0 tx=0).stop 2024-05-07T02:28:06.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:06.118+0000 7fcb43200700 1 -- 172.21.15.19:0/4167186743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb44106620 msgr2=0x7fcb4419ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:06.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:06.118+0000 7fcb43200700 1 --2- 172.21.15.19:0/4167186743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb44106620 0x7fcb4419ceb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcb3400ec10 tx=0x7fcb3400ef20 comp rx=0 tx=0).stop 2024-05-07T02:28:06.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:06.118+0000 7fcb43200700 1 -- 172.21.15.19:0/4167186743 shutdown_connections 2024-05-07T02:28:06.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:06.118+0000 7fcb43200700 1 --2- 172.21.15.19:0/4167186743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcb4410b8a0 0x7fcb441a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:06.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:06.118+0000 7fcb43200700 1 --2- 172.21.15.19:0/4167186743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb44106620 0x7fcb4419ceb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:06.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:06.118+0000 7fcb43200700 1 --2- 172.21.15.19:0/4167186743 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fcb280b2910 0x7fcb280b4e20 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:06.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:06.118+0000 7fcb43200700 1 --2- 172.21.15.19:0/4167186743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb4410af50 0x7fcb4419d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:06.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:06.118+0000 7fcb43200700 1 -- 172.21.15.19:0/4167186743 >> 172.21.15.19:0/4167186743 conn(0x7fcb44102030 msgr2=0x7fcb44109030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:28:06.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:06.118+0000 7fcb43200700 1 -- 172.21.15.19:0/4167186743 shutdown_connections 2024-05-07T02:28:06.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:06.118+0000 7fcb43200700 1 -- 172.21.15.19:0/4167186743 wait complete. 2024-05-07T02:28:06.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:06 smithi064 ceph-mon[34699]: pgmap v394: 129 pgs: 129 active+clean; 612 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-05-07T02:28:06.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:06 smithi019 ceph-mon[29781]: pgmap v394: 129 pgs: 129 active+clean; 612 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-05-07T02:28:06.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:06 smithi027 ceph-mon[34398]: pgmap v394: 129 pgs: 129 active+clean; 612 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-05-07T02:28:07.012 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-07T02:28:07.489 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:28:07.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:07 smithi064 ceph-mon[34699]: from='client.14604 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:28:07.870 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:07 smithi019 ceph-mon[29781]: from='client.14604 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:28:07.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:07 smithi027 ceph-mon[34398]: from='client.14604 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:28:08.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:08 smithi064 ceph-mon[34699]: pgmap v395: 129 pgs: 129 active+clean; 631 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.6 KiB/s wr, 12 op/s 2024-05-07T02:28:08.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:08 smithi019 ceph-mon[29781]: pgmap v395: 129 pgs: 129 active+clean; 631 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.6 KiB/s wr, 12 op/s 2024-05-07T02:28:08.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:08 smithi027 ceph-mon[34398]: pgmap v395: 129 pgs: 129 active+clean; 631 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.6 KiB/s wr, 12 op/s 2024-05-07T02:28:09.591 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:09 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:28:09.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:09 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:28:09.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:09 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:28:10.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.000+0000 7ff2d9e00700 1 -- 172.21.15.19:0/478814735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2d4108f20 msgr2=0x7ff2d4109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:10.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.000+0000 7ff2d9e00700 1 --2- 172.21.15.19:0/478814735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2d4108f20 0x7ff2d4109330 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0009990 tx=0x7ff2c0009ca0 comp rx=0 tx=0).stop 2024-05-07T02:28:10.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.001+0000 7ff2d9e00700 1 -- 172.21.15.19:0/478814735 shutdown_connections 2024-05-07T02:28:10.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.001+0000 7ff2d9e00700 1 --2- 172.21.15.19:0/478814735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2d4109870 0x7ff2d41111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:10.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.001+0000 7ff2d9e00700 1 --2- 172.21.15.19:0/478814735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2d41045f0 0x7ff2d4104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:10.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.001+0000 7ff2d9e00700 1 --2- 172.21.15.19:0/478814735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2d4108f20 0x7ff2d4109330 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:10.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.001+0000 7ff2d9e00700 1 -- 172.21.15.19:0/478814735 >> 172.21.15.19:0/478814735 conn(0x7ff2d4100040 msgr2=0x7ff2d4102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:28:10.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.001+0000 7ff2d9e00700 1 -- 172.21.15.19:0/478814735 shutdown_connections 2024-05-07T02:28:10.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.001+0000 7ff2d9e00700 1 -- 172.21.15.19:0/478814735 wait complete. 2024-05-07T02:28:10.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.002+0000 7ff2d9e00700 1 Processor -- start 2024-05-07T02:28:10.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.002+0000 7ff2d9e00700 1 -- start start 2024-05-07T02:28:10.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.002+0000 7ff2d9e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2d41045f0 0x7ff2d4075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:10.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.002+0000 7ff2d9e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2d4108f20 0x7ff2d4076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:10.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.002+0000 7ff2d9e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2d4109870 0x7ff2d4076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:10.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.003+0000 7ff2d9e00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d41144d0 con 0x7ff2d41045f0 2024-05-07T02:28:10.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.003+0000 7ff2d9e00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d4114350 con 0x7ff2d4108f20 2024-05-07T02:28:10.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.003+0000 7ff2d9e00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d4114650 con 0x7ff2d4109870 2024-05-07T02:28:10.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.003+0000 7ff2d2000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2d4108f20 0x7ff2d4076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:10.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.003+0000 7ff2d2a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2d41045f0 0x7ff2d4075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:10.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.003+0000 7ff2d3400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2d4109870 0x7ff2d4076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:10.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.003+0000 7ff2d2000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2d4108f20 0x7ff2d4076000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:55214/0 (socket says 172.21.15.19:55214) 2024-05-07T02:28:10.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.003+0000 7ff2d2a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2d41045f0 0x7ff2d4075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43004/0 (socket says 172.21.15.19:43004) 2024-05-07T02:28:10.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.003+0000 7ff2d2000700 1 -- 172.21.15.19:0/1192607688 learned_addr learned my addr 172.21.15.19:0/1192607688 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:28:10.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.003+0000 7ff2d3400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2d4109870 0x7ff2d4076540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:54714/0 (socket says 172.21.15.19:54714) 2024-05-07T02:28:10.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.003+0000 7ff2d2000700 1 -- 172.21.15.19:0/1192607688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2d4109870 msgr2=0x7ff2d4076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:10.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.003+0000 7ff2d2000700 1 --2- 172.21.15.19:0/1192607688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2d4109870 0x7ff2d4076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:10.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.003+0000 7ff2d2000700 1 -- 172.21.15.19:0/1192607688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2d41045f0 msgr2=0x7ff2d4075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:10.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.003+0000 7ff2d2000700 1 --2- 172.21.15.19:0/1192607688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2d41045f0 0x7ff2d4075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:10.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.003+0000 7ff2d2000700 1 -- 172.21.15.19:0/1192607688 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2d41a9220 con 0x7ff2d4108f20 2024-05-07T02:28:10.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.003+0000 7ff2d3400700 1 --2- 172.21.15.19:0/1192607688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2d4109870 0x7ff2d4076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:28:10.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.004+0000 7ff2d2000700 1 --2- 172.21.15.19:0/1192607688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2d4108f20 0x7ff2d4076000 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0000c00 tx=0x7ff2c00057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:28:10.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.004+0000 7ff2d8a00700 1 -- 172.21.15.19:0/1192607688 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0027070 con 0x7ff2d4108f20 2024-05-07T02:28:10.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.004+0000 7ff2d8a00700 1 -- 172.21.15.19:0/1192607688 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2c000b610 con 0x7ff2d4108f20 2024-05-07T02:28:10.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.004+0000 7ff2d9e00700 1 -- 172.21.15.19:0/1192607688 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2d41a94b0 con 0x7ff2d4108f20 2024-05-07T02:28:10.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.004+0000 7ff2d8a00700 1 -- 172.21.15.19:0/1192607688 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c00173f0 con 0x7ff2d4108f20 2024-05-07T02:28:10.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.005+0000 7ff2d9e00700 1 -- 172.21.15.19:0/1192607688 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2d41a9960 con 0x7ff2d4108f20 2024-05-07T02:28:10.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.006+0000 7ff2d9e00700 1 -- 172.21.15.19:0/1192607688 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2d40512c0 con 0x7ff2d4108f20 2024-05-07T02:28:10.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.008+0000 7ff2d8a00700 1 -- 172.21.15.19:0/1192607688 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ff2c002a070 con 0x7ff2d4108f20 2024-05-07T02:28:10.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.009+0000 7ff2d8a00700 1 --2- 172.21.15.19:0/1192607688 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7ff2b80b2990 0x7ff2b80b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:10.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.010+0000 7ff2d8a00700 1 -- 172.21.15.19:0/1192607688 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7ff2c00e36a0 con 0x7ff2d4108f20 2024-05-07T02:28:10.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.010+0000 7ff2d2a00700 1 --2- 172.21.15.19:0/1192607688 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7ff2b80b2990 0x7ff2b80b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:10.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.011+0000 7ff2d2a00700 1 --2- 172.21.15.19:0/1192607688 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7ff2b80b2990 0x7ff2b80b4ea0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4009c60 tx=0x7ff2c4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:28:10.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.014+0000 7ff2d8a00700 1 -- 172.21.15.19:0/1192607688 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff2c0013070 con 0x7ff2d4108f20 2024-05-07T02:28:10.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.309+0000 7ff2d9e00700 1 -- 172.21.15.19:0/1192607688 --> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7ff2d4076ff0 con 0x7ff2b80b2990 2024-05-07T02:28:10.318 INFO:teuthology.orchestra.run.smithi019.stdout:/volumes/qa/sv_1/1d507078-2e76-4466-9990-b1c17f741a27 2024-05-07T02:28:10.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.319+0000 7ff2d8a00700 1 -- 172.21.15.19:0/1192607688 <== mgr.14152 v2:172.21.15.19:6800/2059564551 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7ff2d4076ff0 con 0x7ff2b80b2990 2024-05-07T02:28:10.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.322+0000 7ff2d9e00700 1 -- 172.21.15.19:0/1192607688 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7ff2b80b2990 msgr2=0x7ff2b80b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:10.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.322+0000 7ff2d9e00700 1 --2- 172.21.15.19:0/1192607688 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7ff2b80b2990 0x7ff2b80b4ea0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4009c60 tx=0x7ff2c4009260 comp rx=0 tx=0).stop 2024-05-07T02:28:10.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.322+0000 7ff2d9e00700 1 -- 172.21.15.19:0/1192607688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2d4108f20 msgr2=0x7ff2d4076000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:10.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.322+0000 7ff2d9e00700 1 --2- 172.21.15.19:0/1192607688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2d4108f20 0x7ff2d4076000 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0000c00 tx=0x7ff2c00057c0 comp rx=0 tx=0).stop 2024-05-07T02:28:10.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.322+0000 7ff2d9e00700 1 -- 172.21.15.19:0/1192607688 shutdown_connections 2024-05-07T02:28:10.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.322+0000 7ff2d9e00700 1 --2- 172.21.15.19:0/1192607688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2d4109870 0x7ff2d4076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:10.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.322+0000 7ff2d9e00700 1 --2- 172.21.15.19:0/1192607688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2d41045f0 0x7ff2d4075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:10.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.322+0000 7ff2d9e00700 1 --2- 172.21.15.19:0/1192607688 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7ff2b80b2990 0x7ff2b80b4ea0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:10.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.322+0000 7ff2d9e00700 1 --2- 172.21.15.19:0/1192607688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2d4108f20 0x7ff2d4076000 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:10.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.322+0000 7ff2d9e00700 1 -- 172.21.15.19:0/1192607688 >> 172.21.15.19:0/1192607688 conn(0x7ff2d4100040 msgr2=0x7ff2d4108660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:28:10.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.322+0000 7ff2d9e00700 1 -- 172.21.15.19:0/1192607688 shutdown_connections 2024-05-07T02:28:10.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:10.322+0000 7ff2d9e00700 1 -- 172.21.15.19:0/1192607688 wait complete. 2024-05-07T02:28:10.698 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:10 smithi064 ceph-mon[34699]: pgmap v396: 129 pgs: 129 active+clean; 631 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.0 KiB/s wr, 11 op/s 2024-05-07T02:28:10.698 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:10 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:28:10.698 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:10 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:28:10.698 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:10 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:28:10.698 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:10 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:28:10.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:10 smithi019 ceph-mon[29781]: pgmap v396: 129 pgs: 129 active+clean; 631 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.0 KiB/s wr, 11 op/s 2024-05-07T02:28:10.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:10 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:28:10.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:10 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:28:10.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:10 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:28:10.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:10 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:28:10.917 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:10 smithi027 ceph-mon[34398]: pgmap v396: 129 pgs: 129 active+clean; 631 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.0 KiB/s wr, 11 op/s 2024-05-07T02:28:10.917 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:10 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:28:10.918 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:10 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:28:10.918 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:10 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:28:10.918 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:10 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:28:11.204 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-07T02:28:11.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:11 smithi064 ceph-mon[34699]: from='client.24749 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:28:11.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:11 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:28:11.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:11 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:28:11.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:11 smithi019 ceph-mon[29781]: from='client.24749 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:28:11.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:11 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:28:11.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:11 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:28:11.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:11 smithi027 ceph-mon[34398]: from='client.24749 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:28:11.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:11 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:28:11.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:11 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:28:11.975 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:28:12.829 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:12 smithi019 ceph-mon[29781]: pgmap v397: 129 pgs: 129 active+clean; 640 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 4.6 KiB/s wr, 13 op/s 2024-05-07T02:28:12.830 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:12 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:28:12.830 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:12 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:28:12.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:12 smithi064 ceph-mon[34699]: pgmap v397: 129 pgs: 129 active+clean; 640 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 4.6 KiB/s wr, 13 op/s 2024-05-07T02:28:12.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:12 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:28:12.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:12 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:28:12.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:12 smithi027 ceph-mon[34398]: pgmap v397: 129 pgs: 129 active+clean; 640 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 4.6 KiB/s wr, 13 op/s 2024-05-07T02:28:12.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:12 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:28:12.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:12 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:28:14.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:14.665+0000 7fb803200700 1 -- 172.21.15.19:0/4067834410 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb80410e700 msgr2=0x7fb804110be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:14.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:14.665+0000 7fb803200700 1 --2- 172.21.15.19:0/4067834410 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb80410e700 0x7fb804110be0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc009f90 tx=0x7fb7fc00b330 comp rx=0 tx=0).stop 2024-05-07T02:28:14.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:14.665+0000 7fb803200700 1 -- 172.21.15.19:0/4067834410 shutdown_connections 2024-05-07T02:28:14.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:14.665+0000 7fb803200700 1 --2- 172.21.15.19:0/4067834410 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb80410e700 0x7fb804110be0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:14.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:14.665+0000 7fb803200700 1 --2- 172.21.15.19:0/4067834410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb804105290 0x7fb8041056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:14.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:14.665+0000 7fb803200700 1 --2- 172.21.15.19:0/4067834410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb804105be0 0x7fb80410e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:14.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:14.665+0000 7fb803200700 1 -- 172.21.15.19:0/4067834410 >> 172.21.15.19:0/4067834410 conn(0x7fb804078b50 msgr2=0x7fb804078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:28:14.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:14.665+0000 7fb803200700 1 -- 172.21.15.19:0/4067834410 shutdown_connections 2024-05-07T02:28:14.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:14.665+0000 7fb803200700 1 -- 172.21.15.19:0/4067834410 wait complete. 2024-05-07T02:28:14.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:14.667+0000 7fb803200700 1 Processor -- start 2024-05-07T02:28:14.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:14.667+0000 7fb803200700 1 -- start start 2024-05-07T02:28:14.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:14.668+0000 7fb803200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb804105290 0x7fb80419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:14.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:14.668+0000 7fb803200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb804105be0 0x7fb80419d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:14.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:14.668+0000 7fb803200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb80410e700 0x7fb8041a1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:14.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:14.668+0000 7fb803200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb804114260 con 0x7fb804105be0 2024-05-07T02:28:14.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:14.668+0000 7fb803200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb8041140e0 con 0x7fb80410e700 2024-05-07T02:28:14.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:14.668+0000 7fb803200700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb8041143e0 con 0x7fb804105290 2024-05-07T02:28:14.668 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:14 smithi019 ceph-mon[29781]: pgmap v398: 129 pgs: 129 active+clean; 640 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-05-07T02:28:14.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:14.669+0000 7fb801e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb804105290 0x7fb80419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:14.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:14.669+0000 7fb801e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb804105290 0x7fb80419cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:54726/0 (socket says 172.21.15.19:54726) 2024-05-07T02:28:14.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:14.669+0000 7fb801e00700 1 -- 172.21.15.19:0/4207458027 learned_addr learned my addr 172.21.15.19:0/4207458027 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:28:14.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:14.669+0000 7fb801400700 1 --2- 172.21.15.19:0/4207458027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb804105be0 0x7fb80419d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:14.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:14.669+0000 7fb802800700 1 --2- 172.21.15.19:0/4207458027 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb80410e700 0x7fb8041a1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:14.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:14.669+0000 7fb802800700 1 -- 172.21.15.19:0/4207458027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb804105290 msgr2=0x7fb80419cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:14.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:14.669+0000 7fb802800700 1 --2- 172.21.15.19:0/4207458027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb804105290 0x7fb80419cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:14.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:14.669+0000 7fb802800700 1 -- 172.21.15.19:0/4207458027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb804105be0 msgr2=0x7fb80419d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:14.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:14.669+0000 7fb802800700 1 --2- 172.21.15.19:0/4207458027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb804105be0 0x7fb80419d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:14.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:14.669+0000 7fb802800700 1 -- 172.21.15.19:0/4207458027 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8041a1ca0 con 0x7fb80410e700 2024-05-07T02:28:14.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:14.670+0000 7fb802800700 1 --2- 172.21.15.19:0/4207458027 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb80410e700 0x7fb8041a1600 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc009790 tx=0x7fb7fc00ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:28:14.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:14.670+0000 7fb7faa00700 1 -- 172.21.15.19:0/4207458027 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc021070 con 0x7fb80410e700 2024-05-07T02:28:14.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:14.670+0000 7fb803200700 1 -- 172.21.15.19:0/4207458027 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8041a1f30 con 0x7fb80410e700 2024-05-07T02:28:14.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:14.670+0000 7fb7faa00700 1 -- 172.21.15.19:0/4207458027 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7fc007ac0 con 0x7fb80410e700 2024-05-07T02:28:14.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:14.671+0000 7fb803200700 1 -- 172.21.15.19:0/4207458027 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8041a98b0 con 0x7fb80410e700 2024-05-07T02:28:14.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:14.671+0000 7fb7faa00700 1 -- 172.21.15.19:0/4207458027 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc005040 con 0x7fb80410e700 2024-05-07T02:28:14.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:14.673+0000 7fb803200700 1 -- 172.21.15.19:0/4207458027 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7e4005330 con 0x7fb80410e700 2024-05-07T02:28:14.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:14.674+0000 7fb7faa00700 1 -- 172.21.15.19:0/4207458027 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fb7fc024070 con 0x7fb80410e700 2024-05-07T02:28:14.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:14.676+0000 7fb7faa00700 1 --2- 172.21.15.19:0/4207458027 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fb7e80b2910 0x7fb7e80b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:14.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:14.676+0000 7fb7faa00700 1 -- 172.21.15.19:0/4207458027 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fb7fc00f020 con 0x7fb80410e700 2024-05-07T02:28:14.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:14.676+0000 7fb801e00700 1 --2- 172.21.15.19:0/4207458027 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fb7e80b2910 0x7fb7e80b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:14.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:14.677+0000 7fb801e00700 1 --2- 172.21.15.19:0/4207458027 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fb7e80b2910 0x7fb7e80b4e20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4005d10 tx=0x7fb7f4009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:28:14.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:14.681+0000 7fb7faa00700 1 -- 172.21.15.19:0/4207458027 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb7fc02b020 con 0x7fb80410e700 2024-05-07T02:28:14.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:14 smithi064 ceph-mon[34699]: pgmap v398: 129 pgs: 129 active+clean; 640 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-05-07T02:28:14.958 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:14 smithi027 ceph-mon[34398]: pgmap v398: 129 pgs: 129 active+clean; 640 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-05-07T02:28:15.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:15.004+0000 7fb803200700 1 -- 172.21.15.19:0/4207458027 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7e40058f0 con 0x7fb80410e700 2024-05-07T02:28:15.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:15.006+0000 7fb7faa00700 1 -- 172.21.15.19:0/4207458027 <== mon.2 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11924 (secure 0 0 0) 0x7fb7fc0ab220 con 0x7fb80410e700 2024-05-07T02:28:15.006 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:28:15.006 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1763670446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1763670446},{"type":"v1","addr":"172.21.15.19:6839","nonce":1763670446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/477529985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":477529985},{"type":"v1","addr":"172.21.15.64:6833","nonce":477529985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-07T02:26:14.960465+0000","modified":"2024-05-07T02:27:48.914804+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24539,"mds_2":24520,"mds_3":24515,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6833/2171873162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2171873162},{"type":"v1","addr":"172.21.15.27:6833","nonce":2171873162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/2379917839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2379917839},{"type":"v1","addr":"172.21.15.19:6835","nonce":2379917839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6837/2067515538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2067515538},{"type":"v1","addr":"172.21.15.19:6837","nonce":2067515538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6835/1000540600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1000540600},{"type":"v1","addr":"172.21.15.27:6835","nonce":1000540600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/1997318872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1997318872},{"type":"v1","addr":"172.21.15.64:6835","nonce":1997318872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":34,"addr":"172.21.15.64:6837/2585907777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2585907777},{"type":"v1","addr":"172.21.15.64:6837","nonce":2585907777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":33,"addr":"172.21.15.27:6837/2583138143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2583138143},{"type":"v1","addr":"172.21.15.27:6837","nonce":2583138143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":32,"addr":"172.21.15.64:6839/1583120004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":1583120004},{"type":"v1","addr":"172.21.15.64:6839","nonce":1583120004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":29,"addr":"172.21.15.19:6841/3905645246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3905645246},{"type":"v1","addr":"172.21.15.19:6841","nonce":3905645246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6839/3963268634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3963268634},{"type":"v1","addr":"172.21.15.27:6839","nonce":3963268634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24551,24539,24520,24505,24515]},"id":1}]} 2024-05-07T02:28:15.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:15.009+0000 7fb803200700 1 -- 172.21.15.19:0/4207458027 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fb7e80b2910 msgr2=0x7fb7e80b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:15.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:15.009+0000 7fb803200700 1 --2- 172.21.15.19:0/4207458027 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fb7e80b2910 0x7fb7e80b4e20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4005d10 tx=0x7fb7f4009310 comp rx=0 tx=0).stop 2024-05-07T02:28:15.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:15.009+0000 7fb803200700 1 -- 172.21.15.19:0/4207458027 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb80410e700 msgr2=0x7fb8041a1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:15.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:15.009+0000 7fb803200700 1 --2- 172.21.15.19:0/4207458027 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb80410e700 0x7fb8041a1600 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc009790 tx=0x7fb7fc00ccb0 comp rx=0 tx=0).stop 2024-05-07T02:28:15.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:15.009+0000 7fb803200700 1 -- 172.21.15.19:0/4207458027 shutdown_connections 2024-05-07T02:28:15.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:15.009+0000 7fb803200700 1 --2- 172.21.15.19:0/4207458027 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fb7e80b2910 0x7fb7e80b4e20 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:15.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:15.009+0000 7fb803200700 1 --2- 172.21.15.19:0/4207458027 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb80410e700 0x7fb8041a1600 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:15.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:15.009+0000 7fb803200700 1 --2- 172.21.15.19:0/4207458027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb804105290 0x7fb80419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:15.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:15.009+0000 7fb803200700 1 --2- 172.21.15.19:0/4207458027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb804105be0 0x7fb80419d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:15.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:15.009+0000 7fb803200700 1 -- 172.21.15.19:0/4207458027 >> 172.21.15.19:0/4207458027 conn(0x7fb804078b50 msgr2=0x7fb804102eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:28:15.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:15.010+0000 7fb803200700 1 -- 172.21.15.19:0/4207458027 shutdown_connections 2024-05-07T02:28:15.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:15.010+0000 7fb803200700 1 -- 172.21.15.19:0/4207458027 wait complete. 2024-05-07T02:28:15.011 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 30 2024-05-07T02:28:15.882 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-07T02:28:16.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:15 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:28:16.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:15 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:28:16.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:15 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/4207458027' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:28:16.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:15 smithi019 ceph-mon[29781]: pgmap v399: 129 pgs: 129 active+clean; 640 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-05-07T02:28:16.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:15 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:28:16.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:15 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:28:16.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:15 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/4207458027' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:28:16.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:15 smithi027 ceph-mon[34398]: pgmap v399: 129 pgs: 129 active+clean; 640 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-05-07T02:28:16.347 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:28:16.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:15 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:28:16.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:15 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:28:16.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:15 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/4207458027' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:28:16.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:15 smithi064 ceph-mon[34699]: pgmap v399: 129 pgs: 129 active+clean; 640 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-05-07T02:28:17.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:16 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:28:17.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:16 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:28:17.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:16 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:28:17.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:16 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:28:17.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:16 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:28:17.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:16 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:28:17.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:16 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:28:17.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:16 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:28:17.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:16 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:28:18.069 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:17 smithi019 ceph-mon[29781]: pgmap v400: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.5 KiB/s wr, 13 op/s 2024-05-07T02:28:18.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:17 smithi027 ceph-mon[34398]: pgmap v400: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.5 KiB/s wr, 13 op/s 2024-05-07T02:28:18.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:17 smithi064 ceph-mon[34699]: pgmap v400: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.5 KiB/s wr, 13 op/s 2024-05-07T02:28:18.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.688+0000 7f9c51000700 1 -- 172.21.15.19:0/474734961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c4c075d30 msgr2=0x7f9c4c076140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:18.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.688+0000 7f9c51000700 1 --2- 172.21.15.19:0/474734961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c4c075d30 0x7f9c4c076140 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c009990 tx=0x7f9c3c009ca0 comp rx=0 tx=0).stop 2024-05-07T02:28:18.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.688+0000 7f9c51000700 1 -- 172.21.15.19:0/474734961 shutdown_connections 2024-05-07T02:28:18.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.688+0000 7f9c51000700 1 --2- 172.21.15.19:0/474734961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c4c110ba0 0x7f9c4c113000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:18.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.688+0000 7f9c51000700 1 --2- 172.21.15.19:0/474734961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c4c075d30 0x7f9c4c076140 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:18.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.688+0000 7f9c51000700 1 --2- 172.21.15.19:0/474734961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c4c076680 0x7f9c4c110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:18.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.688+0000 7f9c51000700 1 -- 172.21.15.19:0/474734961 >> 172.21.15.19:0/474734961 conn(0x7f9c4c100040 msgr2=0x7f9c4c102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:28:18.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.688+0000 7f9c51000700 1 -- 172.21.15.19:0/474734961 shutdown_connections 2024-05-07T02:28:18.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.689+0000 7f9c51000700 1 -- 172.21.15.19:0/474734961 wait complete. 2024-05-07T02:28:18.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.690+0000 7f9c51000700 1 Processor -- start 2024-05-07T02:28:18.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.690+0000 7f9c51000700 1 -- start start 2024-05-07T02:28:18.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.691+0000 7f9c51000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c4c075d30 0x7f9c4c19d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:18.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.691+0000 7f9c51000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c4c076680 0x7f9c4c19d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:18.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.691+0000 7f9c51000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c4c110ba0 0x7f9c4c1a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:18.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.691+0000 7f9c51000700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4c1166e0 con 0x7f9c4c075d30 2024-05-07T02:28:18.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.691+0000 7f9c51000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4c116560 con 0x7f9c4c076680 2024-05-07T02:28:18.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.691+0000 7f9c51000700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4c116860 con 0x7f9c4c110ba0 2024-05-07T02:28:18.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.691+0000 7f9c49e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c4c075d30 0x7f9c4c19d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:18.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.691+0000 7f9c49e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c4c075d30 0x7f9c4c19d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43038/0 (socket says 172.21.15.19:43038) 2024-05-07T02:28:18.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.691+0000 7f9c49e00700 1 -- 172.21.15.19:0/2927457514 learned_addr learned my addr 172.21.15.19:0/2927457514 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:28:18.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.691+0000 7f9c49400700 1 --2- 172.21.15.19:0/2927457514 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c4c076680 0x7f9c4c19d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:18.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.691+0000 7f9c4a800700 1 --2- 172.21.15.19:0/2927457514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c4c110ba0 0x7f9c4c1a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:18.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.692+0000 7f9c49400700 1 -- 172.21.15.19:0/2927457514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c4c110ba0 msgr2=0x7f9c4c1a18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:18.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.692+0000 7f9c49400700 1 --2- 172.21.15.19:0/2927457514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c4c110ba0 0x7f9c4c1a18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:18.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.692+0000 7f9c49400700 1 -- 172.21.15.19:0/2927457514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c4c075d30 msgr2=0x7f9c4c19d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:18.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.692+0000 7f9c49400700 1 --2- 172.21.15.19:0/2927457514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c4c075d30 0x7f9c4c19d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:18.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.692+0000 7f9c49400700 1 -- 172.21.15.19:0/2927457514 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c4c1a1f60 con 0x7f9c4c076680 2024-05-07T02:28:18.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.692+0000 7f9c49e00700 1 --2- 172.21.15.19:0/2927457514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c4c075d30 0x7f9c4c19d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:28:18.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.692+0000 7f9c49400700 1 --2- 172.21.15.19:0/2927457514 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c4c076680 0x7f9c4c19d630 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9c3800e9f0 tx=0x7f9c3800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:28:18.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.692+0000 7f9c42a00700 1 -- 172.21.15.19:0/2927457514 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3800c9c0 con 0x7f9c4c076680 2024-05-07T02:28:18.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.693+0000 7f9c42a00700 1 -- 172.21.15.19:0/2927457514 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c3800cb60 con 0x7f9c4c076680 2024-05-07T02:28:18.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.693+0000 7f9c42a00700 1 -- 172.21.15.19:0/2927457514 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c38018680 con 0x7f9c4c076680 2024-05-07T02:28:18.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.693+0000 7f9c51000700 1 -- 172.21.15.19:0/2927457514 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c4c1a2250 con 0x7f9c4c076680 2024-05-07T02:28:18.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.693+0000 7f9c51000700 1 -- 172.21.15.19:0/2927457514 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c4c1a9b70 con 0x7f9c4c076680 2024-05-07T02:28:18.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.694+0000 7f9c51000700 1 -- 172.21.15.19:0/2927457514 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c4c0512c0 con 0x7f9c4c076680 2024-05-07T02:28:18.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.697+0000 7f9c42a00700 1 -- 172.21.15.19:0/2927457514 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f9c38010430 con 0x7f9c4c076680 2024-05-07T02:28:18.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.698+0000 7f9c42a00700 1 --2- 172.21.15.19:0/2927457514 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f9c300b2990 0x7f9c300b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:18.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.698+0000 7f9c42a00700 1 -- 172.21.15.19:0/2927457514 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f9c380e2bb0 con 0x7f9c4c076680 2024-05-07T02:28:18.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.699+0000 7f9c49e00700 1 --2- 172.21.15.19:0/2927457514 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f9c300b2990 0x7f9c300b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:18.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.699+0000 7f9c49e00700 1 --2- 172.21.15.19:0/2927457514 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f9c300b2990 0x7f9c300b4ea0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c006010 tx=0x7f9c3c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:28:18.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.700+0000 7f9c42a00700 1 -- 172.21.15.19:0/2927457514 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9c380aa3b0 con 0x7f9c4c076680 2024-05-07T02:28:18.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.939+0000 7f9c51000700 1 -- 172.21.15.19:0/2927457514 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9c4c02e5f0 con 0x7f9c4c076680 2024-05-07T02:28:18.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.942+0000 7f9c42a00700 1 -- 172.21.15.19:0/2927457514 <== mon.2 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24583 (secure 0 0 0) 0x7f9c380b2430 con 0x7f9c4c076680 2024-05-07T02:28:18.943 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:28:18.943 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":87,"fsid":"9ff84c16-0c17-11ef-bc97-c7b262605968","created":"2024-05-07T02:16:19.507657+0000","modified":"2024-05-07T02:27:32.570889+0000","last_up_change":"2024-05-07T02:24:55.025124+0000","last_in_change":"2024-05-07T02:24:33.842422+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-07T02:20:21.439697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-07T02:26:06.178814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-07T02:26:10.705693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-07T02:26:23.534133+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b437cf22-ec8e-48f0-86a2-2413a0ab40dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6803","nonce":2390539683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6805","nonce":2390539683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6809","nonce":2390539683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6807","nonce":2390539683}]},"public_addr":"172.21.15.19:6803/2390539683","cluster_addr":"172.21.15.19:6805/2390539683","heartbeat_back_addr":"172.21.15.19:6809/2390539683","heartbeat_front_addr":"172.21.15.19:6807/2390539683","state":["exists","up"]},{"osd":1,"uuid":"dc753e16-634d-4c14-abd2-935281e1499e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6801","nonce":4292470615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6803","nonce":4292470615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6807","nonce":4292470615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6805","nonce":4292470615}]},"public_addr":"172.21.15.27:6801/4292470615","cluster_addr":"172.21.15.27:6803/4292470615","heartbeat_back_addr":"172.21.15.27:6807/4292470615","heartbeat_front_addr":"172.21.15.27:6805/4292470615","state":["exists","up"]},{"osd":2,"uuid":"40570071-646e-4e66-9a16-c55ab743ac86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6801","nonce":1180552939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6803","nonce":1180552939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6807","nonce":1180552939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6805","nonce":1180552939}]},"public_addr":"172.21.15.64:6801/1180552939","cluster_addr":"172.21.15.64:6803/1180552939","heartbeat_back_addr":"172.21.15.64:6807/1180552939","heartbeat_front_addr":"172.21.15.64:6805/1180552939","state":["exists","up"]},{"osd":3,"uuid":"ceaa4252-4dfa-427a-9b1a-76adf3b8e609","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6811","nonce":387118141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6813","nonce":387118141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6817","nonce":387118141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6815","nonce":387118141}]},"public_addr":"172.21.15.19:6811/387118141","cluster_addr":"172.21.15.19:6813/387118141","heartbeat_back_addr":"172.21.15.19:6817/387118141","heartbeat_front_addr":"172.21.15.19:6815/387118141","state":["exists","up"]},{"osd":4,"uuid":"5a6d1ebc-62ca-444e-8f29-1980418d0a8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6809","nonce":3662186068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6811","nonce":3662186068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6815","nonce":3662186068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6813","nonce":3662186068}]},"public_addr":"172.21.15.27:6809/3662186068","cluster_addr":"172.21.15.27:6811/3662186068","heartbeat_back_addr":"172.21.15.27:6815/3662186068","heartbeat_front_addr":"172.21.15.27:6813/3662186068","state":["exists","up"]},{"osd":5,"uuid":"60c207e9-55e8-4bed-87c3-f6a06a2a1fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6809","nonce":1264591849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6811","nonce":1264591849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6815","nonce":1264591849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6813","nonce":1264591849}]},"public_addr":"172.21.15.64:6809/1264591849","cluster_addr":"172.21.15.64:6811/1264591849","heartbeat_back_addr":"172.21.15.64:6815/1264591849","heartbeat_front_addr":"172.21.15.64:6813/1264591849","state":["exists","up"]},{"osd":6,"uuid":"f3e038e0-4026-45f1-a923-c07cbb307f64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6819","nonce":2158875543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6821","nonce":2158875543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6825","nonce":2158875543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6823","nonce":2158875543}]},"public_addr":"172.21.15.19:6819/2158875543","cluster_addr":"172.21.15.19:6821/2158875543","heartbeat_back_addr":"172.21.15.19:6825/2158875543","heartbeat_front_addr":"172.21.15.19:6823/2158875543","state":["exists","up"]},{"osd":7,"uuid":"d69e34d4-2028-43b8-8ada-ff4a9b579620","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6817","nonce":2756612091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6819","nonce":2756612091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6823","nonce":2756612091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6821","nonce":2756612091}]},"public_addr":"172.21.15.27:6817/2756612091","cluster_addr":"172.21.15.27:6819/2756612091","heartbeat_back_addr":"172.21.15.27:6823/2756612091","heartbeat_front_addr":"172.21.15.27:6821/2756612091","state":["exists","up"]},{"osd":8,"uuid":"6c1a4179-75b6-42ff-ac29-378801e6f8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6817","nonce":1987024836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6819","nonce":1987024836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6823","nonce":1987024836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6821","nonce":1987024836}]},"public_addr":"172.21.15.64:6817/1987024836","cluster_addr":"172.21.15.64:6819/1987024836","heartbeat_back_addr":"172.21.15.64:6823/1987024836","heartbeat_front_addr":"172.21.15.64:6821/1987024836","state":["exists","up"]},{"osd":9,"uuid":"9df7ff66-d39f-4709-8324-74cf9e0e052d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6827","nonce":4124921646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6829","nonce":4124921646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6833","nonce":4124921646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6831","nonce":4124921646}]},"public_addr":"172.21.15.19:6827/4124921646","cluster_addr":"172.21.15.19:6829/4124921646","heartbeat_back_addr":"172.21.15.19:6833/4124921646","heartbeat_front_addr":"172.21.15.19:6831/4124921646","state":["exists","up"]},{"osd":10,"uuid":"71eaa9a7-667d-4cb7-935c-42177d20a279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6825","nonce":1428931817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6827","nonce":1428931817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6831","nonce":1428931817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6829","nonce":1428931817}]},"public_addr":"172.21.15.27:6825/1428931817","cluster_addr":"172.21.15.27:6827/1428931817","heartbeat_back_addr":"172.21.15.27:6831/1428931817","heartbeat_front_addr":"172.21.15.27:6829/1428931817","state":["exists","up"]},{"osd":11,"uuid":"3330627f-ab5d-4e6e-83cf-fc8b74c54ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6825","nonce":4045070679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6827","nonce":4045070679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6831","nonce":4045070679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6829","nonce":4045070679}]},"public_addr":"172.21.15.64:6825/4045070679","cluster_addr":"172.21.15.64:6827/4045070679","heartbeat_back_addr":"172.21.15.64:6831/4045070679","heartbeat_front_addr":"172.21.15.64:6829/4045070679","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:19:27.902633+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:19:54.305685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:20:18.727053+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:20:53.579199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:21:21.058018+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:21:47.091494+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:22:23.269018+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:22:51.863628+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:23:18.537774+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:23:55.763993+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:24:24.956178+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:24:52.681201+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/1956516258":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/1816772181":"2024-05-08T02:17:31.252868+0000","172.21.15.19:6801/174304059":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/1514051750":"2024-05-08T02:16:54.732592+0000","172.21.15.19:6800/174304059":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/3930045871":"2024-05-08T02:16:54.732592+0000","172.21.15.19:6800/3614957309":"2024-05-08T02:16:54.732592+0000","172.21.15.19:0/1328365399":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/139593344":"2024-05-08T02:16:54.732592+0000","172.21.15.19:6801/3614957309":"2024-05-08T02:16:54.732592+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-07T02:28:18.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.947+0000 7f9c51000700 1 -- 172.21.15.19:0/2927457514 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f9c300b2990 msgr2=0x7f9c300b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:18.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.947+0000 7f9c51000700 1 --2- 172.21.15.19:0/2927457514 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f9c300b2990 0x7f9c300b4ea0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c006010 tx=0x7f9c3c00b260 comp rx=0 tx=0).stop 2024-05-07T02:28:18.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.947+0000 7f9c51000700 1 -- 172.21.15.19:0/2927457514 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c4c076680 msgr2=0x7f9c4c19d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:18.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.947+0000 7f9c51000700 1 --2- 172.21.15.19:0/2927457514 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c4c076680 0x7f9c4c19d630 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9c3800e9f0 tx=0x7f9c3800ed00 comp rx=0 tx=0).stop 2024-05-07T02:28:18.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.947+0000 7f9c51000700 1 -- 172.21.15.19:0/2927457514 shutdown_connections 2024-05-07T02:28:18.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.947+0000 7f9c51000700 1 --2- 172.21.15.19:0/2927457514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c4c110ba0 0x7f9c4c1a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:18.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.947+0000 7f9c51000700 1 --2- 172.21.15.19:0/2927457514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c4c075d30 0x7f9c4c19d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:18.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.947+0000 7f9c51000700 1 --2- 172.21.15.19:0/2927457514 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f9c300b2990 0x7f9c300b4ea0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:18.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.947+0000 7f9c51000700 1 --2- 172.21.15.19:0/2927457514 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c4c076680 0x7f9c4c19d630 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:18.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.947+0000 7f9c51000700 1 -- 172.21.15.19:0/2927457514 >> 172.21.15.19:0/2927457514 conn(0x7f9c4c100040 msgr2=0x7f9c4c1006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:28:18.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.947+0000 7f9c51000700 1 -- 172.21.15.19:0/2927457514 shutdown_connections 2024-05-07T02:28:18.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:18.947+0000 7f9c51000700 1 -- 172.21.15.19:0/2927457514 wait complete. 2024-05-07T02:28:19.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:18 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/2927457514' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-07T02:28:19.372 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:18 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/2927457514' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-07T02:28:19.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:18 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/2927457514' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-07T02:28:19.934 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-07T02:28:19.934 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-07T02:28:20.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:20 smithi019 ceph-mon[29781]: pgmap v401: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-05-07T02:28:20.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:20 smithi064 ceph-mon[34699]: pgmap v401: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-05-07T02:28:20.399 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:28:20.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:20 smithi027 ceph-mon[34398]: pgmap v401: 129 pgs: 129 active+clean; 645 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-05-07T02:28:22.701 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:22 smithi019 ceph-mon[29781]: pgmap v402: 129 pgs: 129 active+clean; 649 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-05-07T02:28:22.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:22.833+0000 7feba0a00700 1 -- 172.21.15.19:0/523378935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb9c10b870 msgr2=0x7feb9c1131d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:22.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:22.833+0000 7feba0a00700 1 --2- 172.21.15.19:0/523378935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb9c10b870 0x7feb9c1131d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7feb9400b0d0 tx=0x7feb9400b3e0 comp rx=0 tx=0).stop 2024-05-07T02:28:22.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:22.833+0000 7feba0a00700 1 -- 172.21.15.19:0/523378935 shutdown_connections 2024-05-07T02:28:22.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:22.834+0000 7feba0a00700 1 --2- 172.21.15.19:0/523378935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb9c10b870 0x7feb9c1131d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:22.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:22.834+0000 7feba0a00700 1 --2- 172.21.15.19:0/523378935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb9c106640 0x7feb9c106a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:22.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:22.834+0000 7feba0a00700 1 --2- 172.21.15.19:0/523378935 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb9c10af20 0x7feb9c10b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:22.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:22.834+0000 7feba0a00700 1 -- 172.21.15.19:0/523378935 >> 172.21.15.19:0/523378935 conn(0x7feb9c102030 msgr2=0x7feb9c104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:28:22.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:22.834+0000 7feba0a00700 1 -- 172.21.15.19:0/523378935 shutdown_connections 2024-05-07T02:28:22.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:22.834+0000 7feba0a00700 1 -- 172.21.15.19:0/523378935 wait complete. 2024-05-07T02:28:22.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:22.835+0000 7feba0a00700 1 Processor -- start 2024-05-07T02:28:22.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:22.835+0000 7feba0a00700 1 -- start start 2024-05-07T02:28:22.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:22.836+0000 7feba0a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb9c106640 0x7feb9c19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:22.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:22.836+0000 7feba0a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb9c10af20 0x7feb9c19d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:22.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:22.836+0000 7feba0a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb9c10b870 0x7feb9c1a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:22.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:22.836+0000 7feba0a00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c1164f0 con 0x7feb9c10b870 2024-05-07T02:28:22.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:22.836+0000 7feba0a00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c116370 con 0x7feb9c10af20 2024-05-07T02:28:22.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:22.836+0000 7feba0a00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c116670 con 0x7feb9c106640 2024-05-07T02:28:22.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:22.836+0000 7feb9b400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb9c106640 0x7feb9c19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:22.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:22.836+0000 7feb9b400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb9c106640 0x7feb9c19cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:50396/0 (socket says 172.21.15.19:50396) 2024-05-07T02:28:22.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:22.836+0000 7feb9b400700 1 -- 172.21.15.19:0/22163013 learned_addr learned my addr 172.21.15.19:0/22163013 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:28:22.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:22.836+0000 7feb9aa00700 1 --2- 172.21.15.19:0/22163013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb9c10af20 0x7feb9c19d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:22.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:22.836+0000 7feb9be00700 1 --2- 172.21.15.19:0/22163013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb9c10b870 0x7feb9c1a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:22.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:22.836+0000 7feb9b400700 1 -- 172.21.15.19:0/22163013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb9c10af20 msgr2=0x7feb9c19d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:22.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:22.836+0000 7feb9b400700 1 --2- 172.21.15.19:0/22163013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb9c10af20 0x7feb9c19d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:22.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:22.836+0000 7feb9b400700 1 -- 172.21.15.19:0/22163013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb9c10b870 msgr2=0x7feb9c1a1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:22.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:22.836+0000 7feb9b400700 1 --2- 172.21.15.19:0/22163013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb9c10b870 0x7feb9c1a1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:22.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:22.836+0000 7feb9b400700 1 -- 172.21.15.19:0/22163013 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb9c1a1cb0 con 0x7feb9c106640 2024-05-07T02:28:22.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:22.837+0000 7feb9be00700 1 --2- 172.21.15.19:0/22163013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb9c10b870 0x7feb9c1a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:28:22.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:22.837+0000 7feb9b400700 1 --2- 172.21.15.19:0/22163013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb9c106640 0x7feb9c19cde0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7feb8c00e9a0 tx=0x7feb8c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:28:22.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:22.837+0000 7feb93e00700 1 -- 172.21.15.19:0/22163013 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c00cbd0 con 0x7feb9c106640 2024-05-07T02:28:22.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:22.837+0000 7feb93e00700 1 -- 172.21.15.19:0/22163013 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb8c00cd70 con 0x7feb9c106640 2024-05-07T02:28:22.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:22.837+0000 7feba0a00700 1 -- 172.21.15.19:0/22163013 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb9c1a1fa0 con 0x7feb9c106640 2024-05-07T02:28:22.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:22.837+0000 7feb93e00700 1 -- 172.21.15.19:0/22163013 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c018680 con 0x7feb9c106640 2024-05-07T02:28:22.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:22.837+0000 7feba0a00700 1 -- 172.21.15.19:0/22163013 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb9c1a98c0 con 0x7feb9c106640 2024-05-07T02:28:22.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:22.838+0000 7feba0a00700 1 -- 172.21.15.19:0/22163013 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb9c0512c0 con 0x7feb9c106640 2024-05-07T02:28:22.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:22.841+0000 7feb93e00700 1 -- 172.21.15.19:0/22163013 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7feb8c010430 con 0x7feb9c106640 2024-05-07T02:28:22.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:22.842+0000 7feb93e00700 1 --2- 172.21.15.19:0/22163013 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7feb800b2910 0x7feb800b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:22.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:22.843+0000 7feb93e00700 1 -- 172.21.15.19:0/22163013 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7feb8c0e29e0 con 0x7feb9c106640 2024-05-07T02:28:22.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:22.843+0000 7feb9aa00700 1 --2- 172.21.15.19:0/22163013 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7feb800b2910 0x7feb800b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:22.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:22.844+0000 7feb9aa00700 1 --2- 172.21.15.19:0/22163013 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7feb800b2910 0x7feb800b4e20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7feb9c109c80 tx=0x7feb88009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:28:22.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:22.847+0000 7feb93e00700 1 -- 172.21.15.19:0/22163013 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7feb8c0aa1e0 con 0x7feb9c106640 2024-05-07T02:28:22.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:22 smithi064 ceph-mon[34699]: pgmap v402: 129 pgs: 129 active+clean; 649 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-05-07T02:28:22.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:22 smithi027 ceph-mon[34398]: pgmap v402: 129 pgs: 129 active+clean; 649 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-05-07T02:28:23.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:23.079+0000 7feba0a00700 1 -- 172.21.15.19:0/22163013 --> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7feb9c063c00 con 0x7feb800b2910 2024-05-07T02:28:23.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:23.095+0000 7feb93e00700 1 -- 172.21.15.19:0/22163013 <== mgr.14152 v2:172.21.15.19:6800/2059564551 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340389 (secure 0 0 0) 0x7feb9c063c00 con 0x7feb800b2910 2024-05-07T02:28:23.097 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:28:23.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:23.103+0000 7feba0a00700 1 -- 172.21.15.19:0/22163013 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7feb800b2910 msgr2=0x7feb800b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:23.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:23.103+0000 7feba0a00700 1 --2- 172.21.15.19:0/22163013 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7feb800b2910 0x7feb800b4e20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7feb9c109c80 tx=0x7feb88009260 comp rx=0 tx=0).stop 2024-05-07T02:28:23.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:23.103+0000 7feba0a00700 1 -- 172.21.15.19:0/22163013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb9c106640 msgr2=0x7feb9c19cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:23.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:23.103+0000 7feba0a00700 1 --2- 172.21.15.19:0/22163013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb9c106640 0x7feb9c19cde0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7feb8c00e9a0 tx=0x7feb8c00ecb0 comp rx=0 tx=0).stop 2024-05-07T02:28:23.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:23.104+0000 7feba0a00700 1 -- 172.21.15.19:0/22163013 shutdown_connections 2024-05-07T02:28:23.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:23.104+0000 7feba0a00700 1 --2- 172.21.15.19:0/22163013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb9c10b870 0x7feb9c1a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:23.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:23.104+0000 7feba0a00700 1 --2- 172.21.15.19:0/22163013 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb9c106640 0x7feb9c19cde0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:23.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:23.104+0000 7feba0a00700 1 --2- 172.21.15.19:0/22163013 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7feb800b2910 0x7feb800b4e20 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:23.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:23.104+0000 7feba0a00700 1 --2- 172.21.15.19:0/22163013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb9c10af20 0x7feb9c19d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:23.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:23.104+0000 7feba0a00700 1 -- 172.21.15.19:0/22163013 >> 172.21.15.19:0/22163013 conn(0x7feb9c102030 msgr2=0x7feb9c10a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:28:23.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:23.104+0000 7feba0a00700 1 -- 172.21.15.19:0/22163013 shutdown_connections 2024-05-07T02:28:23.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:23.104+0000 7feba0a00700 1 -- 172.21.15.19:0/22163013 wait complete. 2024-05-07T02:28:23.107 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-05-07T02:28:24.230 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":402,"stamp":"2024-05-07T02:28:21.493338+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664507,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":477,"num_read_kb":353,"num_write":375,"num_write_kb":2298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":309,"ondisk_log_size":309,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375492,"kb_used_data":13860,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566332,"statfs":{"total":1151940427776,"available":1151555923968,"internally_reserved":0,"allocated":14192640,"data_stored":7915801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18448,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":134,"num_read_kb":104,"num_write":12,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006555"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.580529+0000","last_change":"2024-05-07T02:26:11.876828+0000","last_active":"2024-05-07T02:27:32.580529+0000","last_peered":"2024-05-07T02:27:32.580529+0000","last_clean":"2024-05-07T02:27:32.580529+0000","last_became_active":"2024-05-07T02:26:11.876319+0000","last_became_peered":"2024-05-07T02:26:11.876319+0000","last_unstale":"2024-05-07T02:27:32.580529+0000","last_undegraded":"2024-05-07T02:27:32.580529+0000","last_fullsized":"2024-05-07T02:27:32.580529+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:07:33.181373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.583281+0000","last_change":"2024-05-07T02:26:11.873388+0000","last_active":"2024-05-07T02:27:32.583281+0000","last_peered":"2024-05-07T02:27:32.583281+0000","last_clean":"2024-05-07T02:27:32.583281+0000","last_became_active":"2024-05-07T02:26:11.869447+0000","last_became_peered":"2024-05-07T02:26:11.869447+0000","last_unstale":"2024-05-07T02:27:32.583281+0000","last_undegraded":"2024-05-07T02:27:32.583281+0000","last_fullsized":"2024-05-07T02:27:32.583281+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:09:52.876547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.582674+0000","last_change":"2024-05-07T02:26:11.870797+0000","last_active":"2024-05-07T02:27:32.582674+0000","last_peered":"2024-05-07T02:27:32.582674+0000","last_clean":"2024-05-07T02:27:32.582674+0000","last_became_active":"2024-05-07T02:26:11.870411+0000","last_became_peered":"2024-05-07T02:26:11.870411+0000","last_unstale":"2024-05-07T02:27:32.582674+0000","last_undegraded":"2024-05-07T02:27:32.582674+0000","last_fullsized":"2024-05-07T02:27:32.582674+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:34:12.017508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.577729+0000","last_change":"2024-05-07T02:26:11.867327+0000","last_active":"2024-05-07T02:27:32.577729+0000","last_peered":"2024-05-07T02:27:32.577729+0000","last_clean":"2024-05-07T02:27:32.577729+0000","last_became_active":"2024-05-07T02:26:11.866912+0000","last_became_peered":"2024-05-07T02:26:11.866912+0000","last_unstale":"2024-05-07T02:27:32.577729+0000","last_undegraded":"2024-05-07T02:27:32.577729+0000","last_fullsized":"2024-05-07T02:27:32.577729+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:36:36.616706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.577542+0000","last_change":"2024-05-07T02:26:11.878614+0000","last_active":"2024-05-07T02:27:32.577542+0000","last_peered":"2024-05-07T02:27:32.577542+0000","last_clean":"2024-05-07T02:27:32.577542+0000","last_became_active":"2024-05-07T02:26:11.878069+0000","last_became_peered":"2024-05-07T02:26:11.878069+0000","last_unstale":"2024-05-07T02:27:32.577542+0000","last_undegraded":"2024-05-07T02:27:32.577542+0000","last_fullsized":"2024-05-07T02:27:32.577542+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:40:15.991433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.625099+0000","last_change":"2024-05-07T02:26:11.877755+0000","last_active":"2024-05-07T02:27:32.625099+0000","last_peered":"2024-05-07T02:27:32.625099+0000","last_clean":"2024-05-07T02:27:32.625099+0000","last_became_active":"2024-05-07T02:26:11.877373+0000","last_became_peered":"2024-05-07T02:26:11.877373+0000","last_unstale":"2024-05-07T02:27:32.625099+0000","last_undegraded":"2024-05-07T02:27:32.625099+0000","last_fullsized":"2024-05-07T02:27:32.625099+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:49:55.984807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.582015+0000","last_change":"2024-05-07T02:26:11.882194+0000","last_active":"2024-05-07T02:27:32.582015+0000","last_peered":"2024-05-07T02:27:32.582015+0000","last_clean":"2024-05-07T02:27:32.582015+0000","last_became_active":"2024-05-07T02:26:11.881873+0000","last_became_peered":"2024-05-07T02:26:11.881873+0000","last_unstale":"2024-05-07T02:27:32.582015+0000","last_undegraded":"2024-05-07T02:27:32.582015+0000","last_fullsized":"2024-05-07T02:27:32.582015+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:47:31.000357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.577719+0000","last_change":"2024-05-07T02:26:11.857366+0000","last_active":"2024-05-07T02:27:32.577719+0000","last_peered":"2024-05-07T02:27:32.577719+0000","last_clean":"2024-05-07T02:27:32.577719+0000","last_became_active":"2024-05-07T02:26:11.854054+0000","last_became_peered":"2024-05-07T02:26:11.854054+0000","last_unstale":"2024-05-07T02:27:32.577719+0000","last_undegraded":"2024-05-07T02:27:32.577719+0000","last_fullsized":"2024-05-07T02:27:32.577719+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:07:04.834022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.623944+0000","last_change":"2024-05-07T02:26:11.867105+0000","last_active":"2024-05-07T02:27:32.623944+0000","last_peered":"2024-05-07T02:27:32.623944+0000","last_clean":"2024-05-07T02:27:32.623944+0000","last_became_active":"2024-05-07T02:26:11.865986+0000","last_became_peered":"2024-05-07T02:26:11.865986+0000","last_unstale":"2024-05-07T02:27:32.623944+0000","last_undegraded":"2024-05-07T02:27:32.623944+0000","last_fullsized":"2024-05-07T02:27:32.623944+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:13:01.295354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.624609+0000","last_change":"2024-05-07T02:26:11.868039+0000","last_active":"2024-05-07T02:27:32.624609+0000","last_peered":"2024-05-07T02:27:32.624609+0000","last_clean":"2024-05-07T02:27:32.624609+0000","last_became_active":"2024-05-07T02:26:11.867588+0000","last_became_peered":"2024-05-07T02:26:11.867588+0000","last_unstale":"2024-05-07T02:27:32.624609+0000","last_undegraded":"2024-05-07T02:27:32.624609+0000","last_fullsized":"2024-05-07T02:27:32.624609+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:41:45.149620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.582476+0000","last_change":"2024-05-07T02:26:11.859579+0000","last_active":"2024-05-07T02:27:32.582476+0000","last_peered":"2024-05-07T02:27:32.582476+0000","last_clean":"2024-05-07T02:27:32.582476+0000","last_became_active":"2024-05-07T02:26:11.859154+0000","last_became_peered":"2024-05-07T02:26:11.859154+0000","last_unstale":"2024-05-07T02:27:32.582476+0000","last_undegraded":"2024-05-07T02:27:32.582476+0000","last_fullsized":"2024-05-07T02:27:32.582476+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:26:40.616956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.577529+0000","last_change":"2024-05-07T02:26:11.858220+0000","last_active":"2024-05-07T02:27:32.577529+0000","last_peered":"2024-05-07T02:27:32.577529+0000","last_clean":"2024-05-07T02:27:32.577529+0000","last_became_active":"2024-05-07T02:26:11.857826+0000","last_became_peered":"2024-05-07T02:26:11.857826+0000","last_unstale":"2024-05-07T02:27:32.577529+0000","last_undegraded":"2024-05-07T02:27:32.577529+0000","last_fullsized":"2024-05-07T02:27:32.577529+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:21:36.031057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.590412+0000","last_change":"2024-05-07T02:26:11.880348+0000","last_active":"2024-05-07T02:27:32.590412+0000","last_peered":"2024-05-07T02:27:32.590412+0000","last_clean":"2024-05-07T02:27:32.590412+0000","last_became_active":"2024-05-07T02:26:11.879633+0000","last_became_peered":"2024-05-07T02:26:11.879633+0000","last_unstale":"2024-05-07T02:27:32.590412+0000","last_undegraded":"2024-05-07T02:27:32.590412+0000","last_fullsized":"2024-05-07T02:27:32.590412+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:59:34.567973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.582077+0000","last_change":"2024-05-07T02:26:11.865687+0000","last_active":"2024-05-07T02:27:32.582077+0000","last_peered":"2024-05-07T02:27:32.582077+0000","last_clean":"2024-05-07T02:27:32.582077+0000","last_became_active":"2024-05-07T02:26:11.865330+0000","last_became_peered":"2024-05-07T02:26:11.865330+0000","last_unstale":"2024-05-07T02:27:32.582077+0000","last_undegraded":"2024-05-07T02:27:32.582077+0000","last_fullsized":"2024-05-07T02:27:32.582077+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:36:10.610314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.582525+0000","last_change":"2024-05-07T02:26:11.870627+0000","last_active":"2024-05-07T02:27:32.582525+0000","last_peered":"2024-05-07T02:27:32.582525+0000","last_clean":"2024-05-07T02:27:32.582525+0000","last_became_active":"2024-05-07T02:26:11.870229+0000","last_became_peered":"2024-05-07T02:26:11.870229+0000","last_unstale":"2024-05-07T02:27:32.582525+0000","last_undegraded":"2024-05-07T02:27:32.582525+0000","last_fullsized":"2024-05-07T02:27:32.582525+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:06:46.008651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.623191+0000","last_change":"2024-05-07T02:26:11.868698+0000","last_active":"2024-05-07T02:27:32.623191+0000","last_peered":"2024-05-07T02:27:32.623191+0000","last_clean":"2024-05-07T02:27:32.623191+0000","last_became_active":"2024-05-07T02:26:11.868294+0000","last_became_peered":"2024-05-07T02:26:11.868294+0000","last_unstale":"2024-05-07T02:27:32.623191+0000","last_undegraded":"2024-05-07T02:27:32.623191+0000","last_fullsized":"2024-05-07T02:27:32.623191+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:16:22.856200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.580185+0000","last_change":"2024-05-07T02:26:11.877116+0000","last_active":"2024-05-07T02:27:32.580185+0000","last_peered":"2024-05-07T02:27:32.580185+0000","last_clean":"2024-05-07T02:27:32.580185+0000","last_became_active":"2024-05-07T02:26:11.876305+0000","last_became_peered":"2024-05-07T02:26:11.876305+0000","last_unstale":"2024-05-07T02:27:32.580185+0000","last_undegraded":"2024-05-07T02:27:32.580185+0000","last_fullsized":"2024-05-07T02:27:32.580185+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:25:46.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.586264+0000","last_change":"2024-05-07T02:26:11.877488+0000","last_active":"2024-05-07T02:27:32.586264+0000","last_peered":"2024-05-07T02:27:32.586264+0000","last_clean":"2024-05-07T02:27:32.586264+0000","last_became_active":"2024-05-07T02:26:11.874500+0000","last_became_peered":"2024-05-07T02:26:11.874500+0000","last_unstale":"2024-05-07T02:27:32.586264+0000","last_undegraded":"2024-05-07T02:27:32.586264+0000","last_fullsized":"2024-05-07T02:27:32.586264+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:53:50.046342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.583431+0000","last_change":"2024-05-07T02:26:11.876500+0000","last_active":"2024-05-07T02:27:32.583431+0000","last_peered":"2024-05-07T02:27:32.583431+0000","last_clean":"2024-05-07T02:27:32.583431+0000","last_became_active":"2024-05-07T02:26:11.876084+0000","last_became_peered":"2024-05-07T02:26:11.876084+0000","last_unstale":"2024-05-07T02:27:32.583431+0000","last_undegraded":"2024-05-07T02:27:32.583431+0000","last_fullsized":"2024-05-07T02:27:32.583431+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:05:05.681488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.582344+0000","last_change":"2024-05-07T02:26:25.579209+0000","last_active":"2024-05-07T02:27:32.582344+0000","last_peered":"2024-05-07T02:27:32.582344+0000","last_clean":"2024-05-07T02:27:32.582344+0000","last_became_active":"2024-05-07T02:26:25.578665+0000","last_became_peered":"2024-05-07T02:26:25.578665+0000","last_unstale":"2024-05-07T02:27:32.582344+0000","last_undegraded":"2024-05-07T02:27:32.582344+0000","last_fullsized":"2024-05-07T02:27:32.582344+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:06:29.784016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":89,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:28:16.820916+0000","last_change":"2024-05-07T02:26:07.792993+0000","last_active":"2024-05-07T02:28:16.820916+0000","last_peered":"2024-05-07T02:28:16.820916+0000","last_clean":"2024-05-07T02:28:16.820916+0000","last_became_active":"2024-05-07T02:26:07.792402+0000","last_became_peered":"2024-05-07T02:26:07.792402+0000","last_unstale":"2024-05-07T02:28:16.820916+0000","last_undegraded":"2024-05-07T02:28:16.820916+0000","last_fullsized":"2024-05-07T02:28:16.820916+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:49:02.288255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.578807+0000","last_change":"2024-05-07T02:26:11.880654+0000","last_active":"2024-05-07T02:27:32.578807+0000","last_peered":"2024-05-07T02:27:32.578807+0000","last_clean":"2024-05-07T02:27:32.578807+0000","last_became_active":"2024-05-07T02:26:11.878759+0000","last_became_peered":"2024-05-07T02:26:11.878759+0000","last_unstale":"2024-05-07T02:27:32.578807+0000","last_undegraded":"2024-05-07T02:27:32.578807+0000","last_fullsized":"2024-05-07T02:27:32.578807+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:03:49.187502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.576828+0000","last_change":"2024-05-07T02:26:25.579835+0000","last_active":"2024-05-07T02:27:32.576828+0000","last_peered":"2024-05-07T02:27:32.576828+0000","last_clean":"2024-05-07T02:27:32.576828+0000","last_became_active":"2024-05-07T02:26:25.579105+0000","last_became_peered":"2024-05-07T02:26:25.579105+0000","last_unstale":"2024-05-07T02:27:32.576828+0000","last_undegraded":"2024-05-07T02:27:32.576828+0000","last_fullsized":"2024-05-07T02:27:32.576828+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:32:37.745967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:40.718679+0000","last_change":"2024-05-07T02:26:08.001879+0000","last_active":"2024-05-07T02:27:40.718679+0000","last_peered":"2024-05-07T02:27:40.718679+0000","last_clean":"2024-05-07T02:27:40.718679+0000","last_became_active":"2024-05-07T02:26:08.000244+0000","last_became_peered":"2024-05-07T02:26:08.000244+0000","last_unstale":"2024-05-07T02:27:40.718679+0000","last_undegraded":"2024-05-07T02:27:40.718679+0000","last_fullsized":"2024-05-07T02:27:40.718679+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:56:42.591923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.583035+0000","last_change":"2024-05-07T02:26:11.882351+0000","last_active":"2024-05-07T02:27:32.583035+0000","last_peered":"2024-05-07T02:27:32.583035+0000","last_clean":"2024-05-07T02:27:32.583035+0000","last_became_active":"2024-05-07T02:26:11.881922+0000","last_became_peered":"2024-05-07T02:26:11.881922+0000","last_unstale":"2024-05-07T02:27:32.583035+0000","last_undegraded":"2024-05-07T02:27:32.583035+0000","last_fullsized":"2024-05-07T02:27:32.583035+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:58:31.993859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.576995+0000","last_change":"2024-05-07T02:26:25.580419+0000","last_active":"2024-05-07T02:27:32.576995+0000","last_peered":"2024-05-07T02:27:32.576995+0000","last_clean":"2024-05-07T02:27:32.576995+0000","last_became_active":"2024-05-07T02:26:25.579411+0000","last_became_peered":"2024-05-07T02:26:25.579411+0000","last_unstale":"2024-05-07T02:27:32.576995+0000","last_undegraded":"2024-05-07T02:27:32.576995+0000","last_fullsized":"2024-05-07T02:27:32.576995+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:43:03.552210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:44.728261+0000","last_change":"2024-05-07T02:27:32.588255+0000","last_active":"2024-05-07T02:27:44.728261+0000","last_peered":"2024-05-07T02:27:44.728261+0000","last_clean":"2024-05-07T02:27:44.728261+0000","last_became_active":"2024-05-07T02:27:32.584405+0000","last_became_peered":"2024-05-07T02:27:32.584405+0000","last_unstale":"2024-05-07T02:27:44.728261+0000","last_undegraded":"2024-05-07T02:27:44.728261+0000","last_fullsized":"2024-05-07T02:27:44.728261+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:07:22.980041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.623629+0000","last_change":"2024-05-07T02:26:11.866949+0000","last_active":"2024-05-07T02:27:32.623629+0000","last_peered":"2024-05-07T02:27:32.623629+0000","last_clean":"2024-05-07T02:27:32.623629+0000","last_became_active":"2024-05-07T02:26:11.866453+0000","last_became_peered":"2024-05-07T02:26:11.866453+0000","last_unstale":"2024-05-07T02:27:32.623629+0000","last_undegraded":"2024-05-07T02:27:32.623629+0000","last_fullsized":"2024-05-07T02:27:32.623629+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T14:22:47.291470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.576766+0000","last_change":"2024-05-07T02:26:25.570555+0000","last_active":"2024-05-07T02:27:32.576766+0000","last_peered":"2024-05-07T02:27:32.576766+0000","last_clean":"2024-05-07T02:27:32.576766+0000","last_became_active":"2024-05-07T02:26:25.570044+0000","last_became_peered":"2024-05-07T02:26:25.570044+0000","last_unstale":"2024-05-07T02:27:32.576766+0000","last_undegraded":"2024-05-07T02:27:32.576766+0000","last_fullsized":"2024-05-07T02:27:32.576766+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:48:15.701335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:42.734914+0000","last_change":"2024-05-07T02:26:07.785938+0000","last_active":"2024-05-07T02:27:42.734914+0000","last_peered":"2024-05-07T02:27:42.734914+0000","last_clean":"2024-05-07T02:27:42.734914+0000","last_became_active":"2024-05-07T02:26:07.785573+0000","last_became_peered":"2024-05-07T02:26:07.785573+0000","last_unstale":"2024-05-07T02:27:42.734914+0000","last_undegraded":"2024-05-07T02:27:42.734914+0000","last_fullsized":"2024-05-07T02:27:42.734914+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:33:24.042227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.623876+0000","last_change":"2024-05-07T02:26:11.858732+0000","last_active":"2024-05-07T02:27:32.623876+0000","last_peered":"2024-05-07T02:27:32.623876+0000","last_clean":"2024-05-07T02:27:32.623876+0000","last_became_active":"2024-05-07T02:26:11.858270+0000","last_became_peered":"2024-05-07T02:26:11.858270+0000","last_unstale":"2024-05-07T02:27:32.623876+0000","last_undegraded":"2024-05-07T02:27:32.623876+0000","last_fullsized":"2024-05-07T02:27:32.623876+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:03:53.662296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.582877+0000","last_change":"2024-05-07T02:26:25.559025+0000","last_active":"2024-05-07T02:27:32.582877+0000","last_peered":"2024-05-07T02:27:32.582877+0000","last_clean":"2024-05-07T02:27:32.582877+0000","last_became_active":"2024-05-07T02:26:25.558502+0000","last_became_peered":"2024-05-07T02:26:25.558502+0000","last_unstale":"2024-05-07T02:27:32.582877+0000","last_undegraded":"2024-05-07T02:27:32.582877+0000","last_fullsized":"2024-05-07T02:27:32.582877+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:26:59.846341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":82,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:28:17.810156+0000","last_change":"2024-05-07T02:26:07.783386+0000","last_active":"2024-05-07T02:28:17.810156+0000","last_peered":"2024-05-07T02:28:17.810156+0000","last_clean":"2024-05-07T02:28:17.810156+0000","last_became_active":"2024-05-07T02:26:07.782823+0000","last_became_peered":"2024-05-07T02:26:07.782823+0000","last_unstale":"2024-05-07T02:28:17.810156+0000","last_undegraded":"2024-05-07T02:28:17.810156+0000","last_fullsized":"2024-05-07T02:28:17.810156+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:40:29.350809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.577398+0000","last_change":"2024-05-07T02:26:11.877426+0000","last_active":"2024-05-07T02:27:32.577398+0000","last_peered":"2024-05-07T02:27:32.577398+0000","last_clean":"2024-05-07T02:27:32.577398+0000","last_became_active":"2024-05-07T02:26:11.876976+0000","last_became_peered":"2024-05-07T02:26:11.876976+0000","last_unstale":"2024-05-07T02:27:32.577398+0000","last_undegraded":"2024-05-07T02:27:32.577398+0000","last_fullsized":"2024-05-07T02:27:32.577398+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:18:56.077064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.625394+0000","last_change":"2024-05-07T02:27:32.625394+0000","last_active":"2024-05-07T02:27:32.625394+0000","last_peered":"2024-05-07T02:27:32.625394+0000","last_clean":"2024-05-07T02:27:32.625394+0000","last_became_active":"2024-05-07T02:27:32.624826+0000","last_became_peered":"2024-05-07T02:27:32.624826+0000","last_unstale":"2024-05-07T02:27:32.625394+0000","last_undegraded":"2024-05-07T02:27:32.625394+0000","last_fullsized":"2024-05-07T02:27:32.625394+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:16:49.694003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":47,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:41.732327+0000","last_change":"2024-05-07T02:26:07.786388+0000","last_active":"2024-05-07T02:27:41.732327+0000","last_peered":"2024-05-07T02:27:41.732327+0000","last_clean":"2024-05-07T02:27:41.732327+0000","last_became_active":"2024-05-07T02:26:07.784130+0000","last_became_peered":"2024-05-07T02:26:07.784130+0000","last_unstale":"2024-05-07T02:27:41.732327+0000","last_undegraded":"2024-05-07T02:27:41.732327+0000","last_fullsized":"2024-05-07T02:27:41.732327+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:44:55.229815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.623681+0000","last_change":"2024-05-07T02:26:11.858798+0000","last_active":"2024-05-07T02:27:32.623681+0000","last_peered":"2024-05-07T02:27:32.623681+0000","last_clean":"2024-05-07T02:27:32.623681+0000","last_became_active":"2024-05-07T02:26:11.858375+0000","last_became_peered":"2024-05-07T02:26:11.858375+0000","last_unstale":"2024-05-07T02:27:32.623681+0000","last_undegraded":"2024-05-07T02:27:32.623681+0000","last_fullsized":"2024-05-07T02:27:32.623681+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:42:24.604727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.579909+0000","last_change":"2024-05-07T02:26:25.584396+0000","last_active":"2024-05-07T02:27:32.579909+0000","last_peered":"2024-05-07T02:27:32.579909+0000","last_clean":"2024-05-07T02:27:32.579909+0000","last_became_active":"2024-05-07T02:26:25.583917+0000","last_became_peered":"2024-05-07T02:26:25.583917+0000","last_unstale":"2024-05-07T02:27:32.579909+0000","last_undegraded":"2024-05-07T02:27:32.579909+0000","last_fullsized":"2024-05-07T02:27:32.579909+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:56:34.495133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:40.720542+0000","last_change":"2024-05-07T02:26:07.795300+0000","last_active":"2024-05-07T02:27:40.720542+0000","last_peered":"2024-05-07T02:27:40.720542+0000","last_clean":"2024-05-07T02:27:40.720542+0000","last_became_active":"2024-05-07T02:26:07.794633+0000","last_became_peered":"2024-05-07T02:26:07.794633+0000","last_unstale":"2024-05-07T02:27:40.720542+0000","last_undegraded":"2024-05-07T02:27:40.720542+0000","last_fullsized":"2024-05-07T02:27:40.720542+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:36:44.252118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.578205+0000","last_change":"2024-05-07T02:26:11.877717+0000","last_active":"2024-05-07T02:27:32.578205+0000","last_peered":"2024-05-07T02:27:32.578205+0000","last_clean":"2024-05-07T02:27:32.578205+0000","last_became_active":"2024-05-07T02:26:11.876644+0000","last_became_peered":"2024-05-07T02:26:11.876644+0000","last_unstale":"2024-05-07T02:27:32.578205+0000","last_undegraded":"2024-05-07T02:27:32.578205+0000","last_fullsized":"2024-05-07T02:27:32.578205+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T05:20:36.176776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.583025+0000","last_change":"2024-05-07T02:26:25.572120+0000","last_active":"2024-05-07T02:27:32.583025+0000","last_peered":"2024-05-07T02:27:32.583025+0000","last_clean":"2024-05-07T02:27:32.583025+0000","last_became_active":"2024-05-07T02:26:25.571637+0000","last_became_peered":"2024-05-07T02:26:25.571637+0000","last_unstale":"2024-05-07T02:27:32.583025+0000","last_undegraded":"2024-05-07T02:27:32.583025+0000","last_fullsized":"2024-05-07T02:27:32.583025+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:03:54.574061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:42.734993+0000","last_change":"2024-05-07T02:26:07.784160+0000","last_active":"2024-05-07T02:27:42.734993+0000","last_peered":"2024-05-07T02:27:42.734993+0000","last_clean":"2024-05-07T02:27:42.734993+0000","last_became_active":"2024-05-07T02:26:07.783646+0000","last_became_peered":"2024-05-07T02:26:07.783646+0000","last_unstale":"2024-05-07T02:27:42.734993+0000","last_undegraded":"2024-05-07T02:27:42.734993+0000","last_fullsized":"2024-05-07T02:27:42.734993+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T14:25:37.906587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.585546+0000","last_change":"2024-05-07T02:26:11.874886+0000","last_active":"2024-05-07T02:27:32.585546+0000","last_peered":"2024-05-07T02:27:32.585546+0000","last_clean":"2024-05-07T02:27:32.585546+0000","last_became_active":"2024-05-07T02:26:11.874465+0000","last_became_peered":"2024-05-07T02:26:11.874465+0000","last_unstale":"2024-05-07T02:27:32.585546+0000","last_undegraded":"2024-05-07T02:27:32.585546+0000","last_fullsized":"2024-05-07T02:27:32.585546+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:57:19.530747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.577707+0000","last_change":"2024-05-07T02:26:25.558087+0000","last_active":"2024-05-07T02:27:32.577707+0000","last_peered":"2024-05-07T02:27:32.577707+0000","last_clean":"2024-05-07T02:27:32.577707+0000","last_became_active":"2024-05-07T02:26:25.557624+0000","last_became_peered":"2024-05-07T02:26:25.557624+0000","last_unstale":"2024-05-07T02:27:32.577707+0000","last_undegraded":"2024-05-07T02:27:32.577707+0000","last_fullsized":"2024-05-07T02:27:32.577707+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:03:20.104547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":118,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:28:20.814644+0000","last_change":"2024-05-07T02:26:07.788244+0000","last_active":"2024-05-07T02:28:20.814644+0000","last_peered":"2024-05-07T02:28:20.814644+0000","last_clean":"2024-05-07T02:28:20.814644+0000","last_became_active":"2024-05-07T02:26:07.787789+0000","last_became_peered":"2024-05-07T02:26:07.787789+0000","last_unstale":"2024-05-07T02:28:20.814644+0000","last_undegraded":"2024-05-07T02:28:20.814644+0000","last_fullsized":"2024-05-07T02:28:20.814644+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:12:22.943444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":74,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.624097+0000","last_change":"2024-05-07T02:26:11.879821+0000","last_active":"2024-05-07T02:27:32.624097+0000","last_peered":"2024-05-07T02:27:32.624097+0000","last_clean":"2024-05-07T02:27:32.624097+0000","last_became_active":"2024-05-07T02:26:11.879372+0000","last_became_peered":"2024-05-07T02:26:11.879372+0000","last_unstale":"2024-05-07T02:27:32.624097+0000","last_undegraded":"2024-05-07T02:27:32.624097+0000","last_fullsized":"2024-05-07T02:27:32.624097+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:01:03.754890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.585228+0000","last_change":"2024-05-07T02:26:25.576663+0000","last_active":"2024-05-07T02:27:32.585228+0000","last_peered":"2024-05-07T02:27:32.585228+0000","last_clean":"2024-05-07T02:27:32.585228+0000","last_became_active":"2024-05-07T02:26:25.575729+0000","last_became_peered":"2024-05-07T02:26:25.575729+0000","last_unstale":"2024-05-07T02:27:32.585228+0000","last_undegraded":"2024-05-07T02:27:32.585228+0000","last_fullsized":"2024-05-07T02:27:32.585228+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:03:48.061385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":50,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:42.734377+0000","last_change":"2024-05-07T02:27:32.625336+0000","last_active":"2024-05-07T02:27:42.734377+0000","last_peered":"2024-05-07T02:27:42.734377+0000","last_clean":"2024-05-07T02:27:42.734377+0000","last_became_active":"2024-05-07T02:27:32.624805+0000","last_became_peered":"2024-05-07T02:27:32.624805+0000","last_unstale":"2024-05-07T02:27:42.734377+0000","last_undegraded":"2024-05-07T02:27:42.734377+0000","last_fullsized":"2024-05-07T02:27:42.734377+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:10:04.928297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.626374+0000","last_change":"2024-05-07T02:27:32.626374+0000","last_active":"2024-05-07T02:27:32.626374+0000","last_peered":"2024-05-07T02:27:32.626374+0000","last_clean":"2024-05-07T02:27:32.626374+0000","last_became_active":"2024-05-07T02:27:32.625489+0000","last_became_peered":"2024-05-07T02:27:32.625489+0000","last_unstale":"2024-05-07T02:27:32.626374+0000","last_undegraded":"2024-05-07T02:27:32.626374+0000","last_fullsized":"2024-05-07T02:27:32.626374+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T05:56:16.626041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.577141+0000","last_change":"2024-05-07T02:26:25.580811+0000","last_active":"2024-05-07T02:27:32.577141+0000","last_peered":"2024-05-07T02:27:32.577141+0000","last_clean":"2024-05-07T02:27:32.577141+0000","last_became_active":"2024-05-07T02:26:25.579344+0000","last_became_peered":"2024-05-07T02:26:25.579344+0000","last_unstale":"2024-05-07T02:27:32.577141+0000","last_undegraded":"2024-05-07T02:27:32.577141+0000","last_fullsized":"2024-05-07T02:27:32.577141+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:25:36.186899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:41.730235+0000","last_change":"2024-05-07T02:26:08.000010+0000","last_active":"2024-05-07T02:27:41.730235+0000","last_peered":"2024-05-07T02:27:41.730235+0000","last_clean":"2024-05-07T02:27:41.730235+0000","last_became_active":"2024-05-07T02:26:07.999576+0000","last_became_peered":"2024-05-07T02:26:07.999576+0000","last_unstale":"2024-05-07T02:27:41.730235+0000","last_undegraded":"2024-05-07T02:27:41.730235+0000","last_fullsized":"2024-05-07T02:27:41.730235+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:23:38.644205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.623974+0000","last_change":"2024-05-07T02:26:11.867098+0000","last_active":"2024-05-07T02:27:32.623974+0000","last_peered":"2024-05-07T02:27:32.623974+0000","last_clean":"2024-05-07T02:27:32.623974+0000","last_became_active":"2024-05-07T02:26:11.866036+0000","last_became_peered":"2024-05-07T02:26:11.866036+0000","last_unstale":"2024-05-07T02:27:32.623974+0000","last_undegraded":"2024-05-07T02:27:32.623974+0000","last_fullsized":"2024-05-07T02:27:32.623974+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:47:22.412323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.628243+0000","last_change":"2024-05-07T02:27:32.628243+0000","last_active":"2024-05-07T02:27:32.628243+0000","last_peered":"2024-05-07T02:27:32.628243+0000","last_clean":"2024-05-07T02:27:32.628243+0000","last_became_active":"2024-05-07T02:27:32.627048+0000","last_became_peered":"2024-05-07T02:27:32.627048+0000","last_unstale":"2024-05-07T02:27:32.628243+0000","last_undegraded":"2024-05-07T02:27:32.628243+0000","last_fullsized":"2024-05-07T02:27:32.628243+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:33:49.802873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,10],"acting":[9,8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'36","reported_seq":159,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:28:20.824114+0000","last_change":"2024-05-07T02:26:07.786841+0000","last_active":"2024-05-07T02:28:20.824114+0000","last_peered":"2024-05-07T02:28:20.824114+0000","last_clean":"2024-05-07T02:28:20.824114+0000","last_became_active":"2024-05-07T02:26:07.783836+0000","last_became_peered":"2024-05-07T02:26:07.783836+0000","last_unstale":"2024-05-07T02:28:20.824114+0000","last_undegraded":"2024-05-07T02:28:20.824114+0000","last_fullsized":"2024-05-07T02:28:20.824114+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:28:53.390009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67092,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":83,"num_read_kb":108,"num_write":38,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.578206+0000","last_change":"2024-05-07T02:26:11.857292+0000","last_active":"2024-05-07T02:27:32.578206+0000","last_peered":"2024-05-07T02:27:32.578206+0000","last_clean":"2024-05-07T02:27:32.578206+0000","last_became_active":"2024-05-07T02:26:11.856899+0000","last_became_peered":"2024-05-07T02:26:11.856899+0000","last_unstale":"2024-05-07T02:27:32.578206+0000","last_undegraded":"2024-05-07T02:27:32.578206+0000","last_fullsized":"2024-05-07T02:27:32.578206+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:45:42.279848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.624410+0000","last_change":"2024-05-07T02:26:25.570714+0000","last_active":"2024-05-07T02:27:32.624410+0000","last_peered":"2024-05-07T02:27:32.624410+0000","last_clean":"2024-05-07T02:27:32.624410+0000","last_became_active":"2024-05-07T02:26:25.570163+0000","last_became_peered":"2024-05-07T02:26:25.570163+0000","last_unstale":"2024-05-07T02:27:32.624410+0000","last_undegraded":"2024-05-07T02:27:32.624410+0000","last_fullsized":"2024-05-07T02:27:32.624410+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:55:51.903968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":51,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:44.758130+0000","last_change":"2024-05-07T02:26:07.785904+0000","last_active":"2024-05-07T02:27:44.758130+0000","last_peered":"2024-05-07T02:27:44.758130+0000","last_clean":"2024-05-07T02:27:44.758130+0000","last_became_active":"2024-05-07T02:26:07.785347+0000","last_became_peered":"2024-05-07T02:26:07.785347+0000","last_unstale":"2024-05-07T02:27:44.758130+0000","last_undegraded":"2024-05-07T02:27:44.758130+0000","last_fullsized":"2024-05-07T02:27:44.758130+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:42:05.604266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.589792+0000","last_change":"2024-05-07T02:26:11.878511+0000","last_active":"2024-05-07T02:27:32.589792+0000","last_peered":"2024-05-07T02:27:32.589792+0000","last_clean":"2024-05-07T02:27:32.589792+0000","last_became_active":"2024-05-07T02:26:11.878166+0000","last_became_peered":"2024-05-07T02:26:11.878166+0000","last_unstale":"2024-05-07T02:27:32.589792+0000","last_undegraded":"2024-05-07T02:27:32.589792+0000","last_fullsized":"2024-05-07T02:27:32.589792+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T08:14:40.436244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.580269+0000","last_change":"2024-05-07T02:26:25.579387+0000","last_active":"2024-05-07T02:27:32.580269+0000","last_peered":"2024-05-07T02:27:32.580269+0000","last_clean":"2024-05-07T02:27:32.580269+0000","last_became_active":"2024-05-07T02:26:25.578757+0000","last_became_peered":"2024-05-07T02:26:25.578757+0000","last_unstale":"2024-05-07T02:27:32.580269+0000","last_undegraded":"2024-05-07T02:27:32.580269+0000","last_fullsized":"2024-05-07T02:27:32.580269+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:19:19.650941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:42.733435+0000","last_change":"2024-05-07T02:26:07.785154+0000","last_active":"2024-05-07T02:27:42.733435+0000","last_peered":"2024-05-07T02:27:42.733435+0000","last_clean":"2024-05-07T02:27:42.733435+0000","last_became_active":"2024-05-07T02:26:07.784589+0000","last_became_peered":"2024-05-07T02:26:07.784589+0000","last_unstale":"2024-05-07T02:27:42.733435+0000","last_undegraded":"2024-05-07T02:27:42.733435+0000","last_fullsized":"2024-05-07T02:27:42.733435+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:41:41.851182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.582412+0000","last_change":"2024-05-07T02:26:11.878162+0000","last_active":"2024-05-07T02:27:32.582412+0000","last_peered":"2024-05-07T02:27:32.582412+0000","last_clean":"2024-05-07T02:27:32.582412+0000","last_became_active":"2024-05-07T02:26:11.877770+0000","last_became_peered":"2024-05-07T02:26:11.877770+0000","last_unstale":"2024-05-07T02:27:32.582412+0000","last_undegraded":"2024-05-07T02:27:32.582412+0000","last_fullsized":"2024-05-07T02:27:32.582412+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:19:08.209398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.584993+0000","last_change":"2024-05-07T02:26:25.579465+0000","last_active":"2024-05-07T02:27:32.584993+0000","last_peered":"2024-05-07T02:27:32.584993+0000","last_clean":"2024-05-07T02:27:32.584993+0000","last_became_active":"2024-05-07T02:26:25.579003+0000","last_became_peered":"2024-05-07T02:26:25.579003+0000","last_unstale":"2024-05-07T02:27:32.584993+0000","last_undegraded":"2024-05-07T02:27:32.584993+0000","last_fullsized":"2024-05-07T02:27:32.584993+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:13:24.356366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":81,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:28:16.839556+0000","last_change":"2024-05-07T02:26:07.780827+0000","last_active":"2024-05-07T02:28:16.839556+0000","last_peered":"2024-05-07T02:28:16.839556+0000","last_clean":"2024-05-07T02:28:16.839556+0000","last_became_active":"2024-05-07T02:26:07.780353+0000","last_became_peered":"2024-05-07T02:26:07.780353+0000","last_unstale":"2024-05-07T02:28:16.839556+0000","last_undegraded":"2024-05-07T02:28:16.839556+0000","last_fullsized":"2024-05-07T02:28:16.839556+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:52:40.554135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.582289+0000","last_change":"2024-05-07T02:26:11.852516+0000","last_active":"2024-05-07T02:27:32.582289+0000","last_peered":"2024-05-07T02:27:32.582289+0000","last_clean":"2024-05-07T02:27:32.582289+0000","last_became_active":"2024-05-07T02:26:11.852112+0000","last_became_peered":"2024-05-07T02:26:11.852112+0000","last_unstale":"2024-05-07T02:27:32.582289+0000","last_undegraded":"2024-05-07T02:27:32.582289+0000","last_fullsized":"2024-05-07T02:27:32.582289+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:58:46.989250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.591465+0000","last_change":"2024-05-07T02:26:25.586726+0000","last_active":"2024-05-07T02:27:32.591465+0000","last_peered":"2024-05-07T02:27:32.591465+0000","last_clean":"2024-05-07T02:27:32.591465+0000","last_became_active":"2024-05-07T02:26:25.586267+0000","last_became_peered":"2024-05-07T02:26:25.586267+0000","last_unstale":"2024-05-07T02:27:32.591465+0000","last_undegraded":"2024-05-07T02:27:32.591465+0000","last_fullsized":"2024-05-07T02:27:32.591465+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:50:00.864463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:42.733944+0000","last_change":"2024-05-07T02:27:32.599511+0000","last_active":"2024-05-07T02:27:42.733944+0000","last_peered":"2024-05-07T02:27:42.733944+0000","last_clean":"2024-05-07T02:27:42.733944+0000","last_became_active":"2024-05-07T02:27:32.590197+0000","last_became_peered":"2024-05-07T02:27:32.590197+0000","last_unstale":"2024-05-07T02:27:42.733944+0000","last_undegraded":"2024-05-07T02:27:42.733944+0000","last_fullsized":"2024-05-07T02:27:42.733944+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:23:07.401251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.623382+0000","last_change":"2024-05-07T02:26:11.857671+0000","last_active":"2024-05-07T02:27:32.623382+0000","last_peered":"2024-05-07T02:27:32.623382+0000","last_clean":"2024-05-07T02:27:32.623382+0000","last_became_active":"2024-05-07T02:26:11.857293+0000","last_became_peered":"2024-05-07T02:26:11.857293+0000","last_unstale":"2024-05-07T02:27:32.623382+0000","last_undegraded":"2024-05-07T02:27:32.623382+0000","last_fullsized":"2024-05-07T02:27:32.623382+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:48:42.267887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.623285+0000","last_change":"2024-05-07T02:26:25.563274+0000","last_active":"2024-05-07T02:27:32.623285+0000","last_peered":"2024-05-07T02:27:32.623285+0000","last_clean":"2024-05-07T02:27:32.623285+0000","last_became_active":"2024-05-07T02:26:25.562868+0000","last_became_peered":"2024-05-07T02:26:25.562868+0000","last_unstale":"2024-05-07T02:27:32.623285+0000","last_undegraded":"2024-05-07T02:27:32.623285+0000","last_fullsized":"2024-05-07T02:27:32.623285+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T14:20:03.210034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:41.710469+0000","last_change":"2024-05-07T02:26:07.784888+0000","last_active":"2024-05-07T02:27:41.710469+0000","last_peered":"2024-05-07T02:27:41.710469+0000","last_clean":"2024-05-07T02:27:41.710469+0000","last_became_active":"2024-05-07T02:26:07.784493+0000","last_became_peered":"2024-05-07T02:26:07.784493+0000","last_unstale":"2024-05-07T02:27:41.710469+0000","last_undegraded":"2024-05-07T02:27:41.710469+0000","last_fullsized":"2024-05-07T02:27:41.710469+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:21:14.054786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.579802+0000","last_change":"2024-05-07T02:26:11.858887+0000","last_active":"2024-05-07T02:27:32.579802+0000","last_peered":"2024-05-07T02:27:32.579802+0000","last_clean":"2024-05-07T02:27:32.579802+0000","last_became_active":"2024-05-07T02:26:11.858504+0000","last_became_peered":"2024-05-07T02:26:11.858504+0000","last_unstale":"2024-05-07T02:27:32.579802+0000","last_undegraded":"2024-05-07T02:27:32.579802+0000","last_fullsized":"2024-05-07T02:27:32.579802+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:53:20.290039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.627310+0000","last_change":"2024-05-07T02:27:32.627310+0000","last_active":"2024-05-07T02:27:32.627310+0000","last_peered":"2024-05-07T02:27:32.627310+0000","last_clean":"2024-05-07T02:27:32.627310+0000","last_became_active":"2024-05-07T02:27:32.626794+0000","last_became_peered":"2024-05-07T02:27:32.626794+0000","last_unstale":"2024-05-07T02:27:32.627310+0000","last_undegraded":"2024-05-07T02:27:32.627310+0000","last_fullsized":"2024-05-07T02:27:32.627310+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:06:30.466291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:41.732005+0000","last_change":"2024-05-07T02:26:08.000604+0000","last_active":"2024-05-07T02:27:41.732005+0000","last_peered":"2024-05-07T02:27:41.732005+0000","last_clean":"2024-05-07T02:27:41.732005+0000","last_became_active":"2024-05-07T02:26:08.000107+0000","last_became_peered":"2024-05-07T02:26:08.000107+0000","last_unstale":"2024-05-07T02:27:41.732005+0000","last_undegraded":"2024-05-07T02:27:41.732005+0000","last_fullsized":"2024-05-07T02:27:41.732005+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:58:06.716456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.624563+0000","last_change":"2024-05-07T02:26:11.876555+0000","last_active":"2024-05-07T02:27:32.624563+0000","last_peered":"2024-05-07T02:27:32.624563+0000","last_clean":"2024-05-07T02:27:32.624563+0000","last_became_active":"2024-05-07T02:26:11.876109+0000","last_became_peered":"2024-05-07T02:26:11.876109+0000","last_unstale":"2024-05-07T02:27:32.624563+0000","last_undegraded":"2024-05-07T02:27:32.624563+0000","last_fullsized":"2024-05-07T02:27:32.624563+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:10:24.995786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.588850+0000","last_change":"2024-05-07T02:26:25.567145+0000","last_active":"2024-05-07T02:27:32.588850+0000","last_peered":"2024-05-07T02:27:32.588850+0000","last_clean":"2024-05-07T02:27:32.588850+0000","last_became_active":"2024-05-07T02:26:25.566767+0000","last_became_peered":"2024-05-07T02:26:25.566767+0000","last_unstale":"2024-05-07T02:27:32.588850+0000","last_undegraded":"2024-05-07T02:27:32.588850+0000","last_fullsized":"2024-05-07T02:27:32.588850+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T05:10:59.333367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'6","reported_seq":85,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:28:18.825596+0000","last_change":"2024-05-07T02:26:07.789566+0000","last_active":"2024-05-07T02:28:18.825596+0000","last_peered":"2024-05-07T02:28:18.825596+0000","last_clean":"2024-05-07T02:28:18.825596+0000","last_became_active":"2024-05-07T02:26:07.789106+0000","last_became_peered":"2024-05-07T02:26:07.789106+0000","last_unstale":"2024-05-07T02:28:18.825596+0000","last_undegraded":"2024-05-07T02:28:18.825596+0000","last_fullsized":"2024-05-07T02:28:18.825596+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:45:13.620183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":38,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.624308+0000","last_change":"2024-05-07T02:26:11.869080+0000","last_active":"2024-05-07T02:27:32.624308+0000","last_peered":"2024-05-07T02:27:32.624308+0000","last_clean":"2024-05-07T02:27:32.624308+0000","last_became_active":"2024-05-07T02:26:11.868702+0000","last_became_peered":"2024-05-07T02:26:11.868702+0000","last_unstale":"2024-05-07T02:27:32.624308+0000","last_undegraded":"2024-05-07T02:27:32.624308+0000","last_fullsized":"2024-05-07T02:27:32.624308+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:05:05.497071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.588995+0000","last_change":"2024-05-07T02:26:25.867393+0000","last_active":"2024-05-07T02:27:32.588995+0000","last_peered":"2024-05-07T02:27:32.588995+0000","last_clean":"2024-05-07T02:27:32.588995+0000","last_became_active":"2024-05-07T02:26:25.866956+0000","last_became_peered":"2024-05-07T02:26:25.866956+0000","last_unstale":"2024-05-07T02:27:32.588995+0000","last_undegraded":"2024-05-07T02:27:32.588995+0000","last_fullsized":"2024-05-07T02:27:32.588995+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:23:48.821668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.623481+0000","last_change":"2024-05-07T02:26:11.867507+0000","last_active":"2024-05-07T02:27:32.623481+0000","last_peered":"2024-05-07T02:27:32.623481+0000","last_clean":"2024-05-07T02:27:32.623481+0000","last_became_active":"2024-05-07T02:26:11.867121+0000","last_became_peered":"2024-05-07T02:26:11.867121+0000","last_unstale":"2024-05-07T02:27:32.623481+0000","last_undegraded":"2024-05-07T02:27:32.623481+0000","last_fullsized":"2024-05-07T02:27:32.623481+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:44:55.442725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":84,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:28:19.846326+0000","last_change":"2024-05-07T02:26:07.999784+0000","last_active":"2024-05-07T02:28:19.846326+0000","last_peered":"2024-05-07T02:28:19.846326+0000","last_clean":"2024-05-07T02:28:19.846326+0000","last_became_active":"2024-05-07T02:26:07.999339+0000","last_became_peered":"2024-05-07T02:26:07.999339+0000","last_unstale":"2024-05-07T02:28:19.846326+0000","last_undegraded":"2024-05-07T02:28:19.846326+0000","last_fullsized":"2024-05-07T02:28:19.846326+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:50:41.622486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":38,"num_read_kb":38,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.582491+0000","last_change":"2024-05-07T02:26:25.587032+0000","last_active":"2024-05-07T02:27:32.582491+0000","last_peered":"2024-05-07T02:27:32.582491+0000","last_clean":"2024-05-07T02:27:32.582491+0000","last_became_active":"2024-05-07T02:26:25.586530+0000","last_became_peered":"2024-05-07T02:26:25.586530+0000","last_unstale":"2024-05-07T02:27:32.582491+0000","last_undegraded":"2024-05-07T02:27:32.582491+0000","last_fullsized":"2024-05-07T02:27:32.582491+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:13:13.335832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.579901+0000","last_change":"2024-05-07T02:26:11.878593+0000","last_active":"2024-05-07T02:27:32.579901+0000","last_peered":"2024-05-07T02:27:32.579901+0000","last_clean":"2024-05-07T02:27:32.579901+0000","last_became_active":"2024-05-07T02:26:11.873286+0000","last_became_peered":"2024-05-07T02:26:11.873286+0000","last_unstale":"2024-05-07T02:27:32.579901+0000","last_undegraded":"2024-05-07T02:27:32.579901+0000","last_fullsized":"2024-05-07T02:27:32.579901+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:22:33.769539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":50,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:44.758541+0000","last_change":"2024-05-07T02:26:07.790087+0000","last_active":"2024-05-07T02:27:44.758541+0000","last_peered":"2024-05-07T02:27:44.758541+0000","last_clean":"2024-05-07T02:27:44.758541+0000","last_became_active":"2024-05-07T02:26:07.789592+0000","last_became_peered":"2024-05-07T02:26:07.789592+0000","last_unstale":"2024-05-07T02:27:44.758541+0000","last_undegraded":"2024-05-07T02:27:44.758541+0000","last_fullsized":"2024-05-07T02:27:44.758541+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:44:52.569055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":114,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.581868+0000","last_change":"2024-05-07T02:24:27.877820+0000","last_active":"2024-05-07T02:27:32.581868+0000","last_peered":"2024-05-07T02:27:32.581868+0000","last_clean":"2024-05-07T02:27:32.581868+0000","last_became_active":"2024-05-07T02:24:27.873221+0000","last_became_peered":"2024-05-07T02:24:27.873221+0000","last_unstale":"2024-05-07T02:27:32.581868+0000","last_undegraded":"2024-05-07T02:27:32.581868+0000","last_fullsized":"2024-05-07T02:27:32.581868+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:20:21.582322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:20:21.582322+0000","last_clean_scrub_stamp":"2024-05-07T02:20:21.582322+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T14:16:32.772438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.624144+0000","last_change":"2024-05-07T02:26:25.868394+0000","last_active":"2024-05-07T02:27:32.624144+0000","last_peered":"2024-05-07T02:27:32.624144+0000","last_clean":"2024-05-07T02:27:32.624144+0000","last_became_active":"2024-05-07T02:26:25.867583+0000","last_became_peered":"2024-05-07T02:26:25.867583+0000","last_unstale":"2024-05-07T02:27:32.624144+0000","last_undegraded":"2024-05-07T02:27:32.624144+0000","last_fullsized":"2024-05-07T02:27:32.624144+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:55:44.705608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.589643+0000","last_change":"2024-05-07T02:26:11.868371+0000","last_active":"2024-05-07T02:27:32.589643+0000","last_peered":"2024-05-07T02:27:32.589643+0000","last_clean":"2024-05-07T02:27:32.589643+0000","last_became_active":"2024-05-07T02:26:11.867981+0000","last_became_peered":"2024-05-07T02:26:11.867981+0000","last_unstale":"2024-05-07T02:27:32.589643+0000","last_undegraded":"2024-05-07T02:27:32.589643+0000","last_fullsized":"2024-05-07T02:27:32.589643+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:08:54.687327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:42.734542+0000","last_change":"2024-05-07T02:26:07.785344+0000","last_active":"2024-05-07T02:27:42.734542+0000","last_peered":"2024-05-07T02:27:42.734542+0000","last_clean":"2024-05-07T02:27:42.734542+0000","last_became_active":"2024-05-07T02:26:07.784707+0000","last_became_peered":"2024-05-07T02:26:07.784707+0000","last_unstale":"2024-05-07T02:27:42.734542+0000","last_undegraded":"2024-05-07T02:27:42.734542+0000","last_fullsized":"2024-05-07T02:27:42.734542+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T05:00:38.754313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.579438+0000","last_change":"2024-05-07T02:26:25.571989+0000","last_active":"2024-05-07T02:27:32.579438+0000","last_peered":"2024-05-07T02:27:32.579438+0000","last_clean":"2024-05-07T02:27:32.579438+0000","last_became_active":"2024-05-07T02:26:25.571533+0000","last_became_peered":"2024-05-07T02:26:25.571533+0000","last_unstale":"2024-05-07T02:27:32.579438+0000","last_undegraded":"2024-05-07T02:27:32.579438+0000","last_fullsized":"2024-05-07T02:27:32.579438+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:54:38.669278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:44.757141+0000","last_change":"2024-05-07T02:26:07.999636+0000","last_active":"2024-05-07T02:27:44.757141+0000","last_peered":"2024-05-07T02:27:44.757141+0000","last_clean":"2024-05-07T02:27:44.757141+0000","last_became_active":"2024-05-07T02:26:07.999100+0000","last_became_peered":"2024-05-07T02:26:07.999100+0000","last_unstale":"2024-05-07T02:27:44.757141+0000","last_undegraded":"2024-05-07T02:27:44.757141+0000","last_fullsized":"2024-05-07T02:27:44.757141+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:10:24.730988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.591156+0000","last_change":"2024-05-07T02:26:11.888279+0000","last_active":"2024-05-07T02:27:32.591156+0000","last_peered":"2024-05-07T02:27:32.591156+0000","last_clean":"2024-05-07T02:27:32.591156+0000","last_became_active":"2024-05-07T02:26:11.887480+0000","last_became_peered":"2024-05-07T02:26:11.887480+0000","last_unstale":"2024-05-07T02:27:32.591156+0000","last_undegraded":"2024-05-07T02:27:32.591156+0000","last_fullsized":"2024-05-07T02:27:32.591156+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:12:12.368368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.578146+0000","last_change":"2024-05-07T02:26:25.867444+0000","last_active":"2024-05-07T02:27:32.578146+0000","last_peered":"2024-05-07T02:27:32.578146+0000","last_clean":"2024-05-07T02:27:32.578146+0000","last_became_active":"2024-05-07T02:26:25.866392+0000","last_became_peered":"2024-05-07T02:26:25.866392+0000","last_unstale":"2024-05-07T02:27:32.578146+0000","last_undegraded":"2024-05-07T02:27:32.578146+0000","last_fullsized":"2024-05-07T02:27:32.578146+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:54:36.112427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'16","reported_seq":57,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:28:16.042816+0000","last_change":"2024-05-07T02:26:07.798091+0000","last_active":"2024-05-07T02:28:16.042816+0000","last_peered":"2024-05-07T02:28:16.042816+0000","last_clean":"2024-05-07T02:28:16.042816+0000","last_became_active":"2024-05-07T02:26:07.797716+0000","last_became_peered":"2024-05-07T02:26:07.797716+0000","last_unstale":"2024-05-07T02:28:16.042816+0000","last_undegraded":"2024-05-07T02:28:16.042816+0000","last_fullsized":"2024-05-07T02:28:16.042816+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:04:54.570916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.579499+0000","last_change":"2024-05-07T02:26:11.888638+0000","last_active":"2024-05-07T02:27:32.579499+0000","last_peered":"2024-05-07T02:27:32.579499+0000","last_clean":"2024-05-07T02:27:32.579499+0000","last_became_active":"2024-05-07T02:26:11.888269+0000","last_became_peered":"2024-05-07T02:26:11.888269+0000","last_unstale":"2024-05-07T02:27:32.579499+0000","last_undegraded":"2024-05-07T02:27:32.579499+0000","last_fullsized":"2024-05-07T02:27:32.579499+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:09:38.965362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.582165+0000","last_change":"2024-05-07T02:26:25.558975+0000","last_active":"2024-05-07T02:27:32.582165+0000","last_peered":"2024-05-07T02:27:32.582165+0000","last_clean":"2024-05-07T02:27:32.582165+0000","last_became_active":"2024-05-07T02:26:25.557947+0000","last_became_peered":"2024-05-07T02:26:25.557947+0000","last_unstale":"2024-05-07T02:27:32.582165+0000","last_undegraded":"2024-05-07T02:27:32.582165+0000","last_fullsized":"2024-05-07T02:27:32.582165+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:29:32.071625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":49,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:44.759365+0000","last_change":"2024-05-07T02:26:07.796643+0000","last_active":"2024-05-07T02:27:44.759365+0000","last_peered":"2024-05-07T02:27:44.759365+0000","last_clean":"2024-05-07T02:27:44.759365+0000","last_became_active":"2024-05-07T02:26:07.796264+0000","last_became_peered":"2024-05-07T02:26:07.796264+0000","last_unstale":"2024-05-07T02:27:44.759365+0000","last_undegraded":"2024-05-07T02:27:44.759365+0000","last_fullsized":"2024-05-07T02:27:44.759365+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:29:32.454816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.583370+0000","last_change":"2024-05-07T02:26:11.873670+0000","last_active":"2024-05-07T02:27:32.583370+0000","last_peered":"2024-05-07T02:27:32.583370+0000","last_clean":"2024-05-07T02:27:32.583370+0000","last_became_active":"2024-05-07T02:26:11.871219+0000","last_became_peered":"2024-05-07T02:26:11.871219+0000","last_unstale":"2024-05-07T02:27:32.583370+0000","last_undegraded":"2024-05-07T02:27:32.583370+0000","last_fullsized":"2024-05-07T02:27:32.583370+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:49:48.048127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.577488+0000","last_change":"2024-05-07T02:26:25.868011+0000","last_active":"2024-05-07T02:27:32.577488+0000","last_peered":"2024-05-07T02:27:32.577488+0000","last_clean":"2024-05-07T02:27:32.577488+0000","last_became_active":"2024-05-07T02:26:25.867465+0000","last_became_peered":"2024-05-07T02:26:25.867465+0000","last_unstale":"2024-05-07T02:27:32.577488+0000","last_undegraded":"2024-05-07T02:27:32.577488+0000","last_fullsized":"2024-05-07T02:27:32.577488+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T05:14:40.001874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:39.698681+0000","last_change":"2024-05-07T02:26:07.999233+0000","last_active":"2024-05-07T02:27:39.698681+0000","last_peered":"2024-05-07T02:27:39.698681+0000","last_clean":"2024-05-07T02:27:39.698681+0000","last_became_active":"2024-05-07T02:26:07.998920+0000","last_became_peered":"2024-05-07T02:26:07.998920+0000","last_unstale":"2024-05-07T02:27:39.698681+0000","last_undegraded":"2024-05-07T02:27:39.698681+0000","last_fullsized":"2024-05-07T02:27:39.698681+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:33:33.878984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.577379+0000","last_change":"2024-05-07T02:26:11.856832+0000","last_active":"2024-05-07T02:27:32.577379+0000","last_peered":"2024-05-07T02:27:32.577379+0000","last_clean":"2024-05-07T02:27:32.577379+0000","last_became_active":"2024-05-07T02:26:11.856424+0000","last_became_peered":"2024-05-07T02:26:11.856424+0000","last_unstale":"2024-05-07T02:27:32.577379+0000","last_undegraded":"2024-05-07T02:27:32.577379+0000","last_fullsized":"2024-05-07T02:27:32.577379+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:00:46.839542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.627025+0000","last_change":"2024-05-07T02:27:32.626086+0000","last_active":"2024-05-07T02:27:32.627025+0000","last_peered":"2024-05-07T02:27:32.627025+0000","last_clean":"2024-05-07T02:27:32.627025+0000","last_became_active":"2024-05-07T02:27:32.625468+0000","last_became_peered":"2024-05-07T02:27:32.625468+0000","last_unstale":"2024-05-07T02:27:32.627025+0000","last_undegraded":"2024-05-07T02:27:32.627025+0000","last_fullsized":"2024-05-07T02:27:32.627025+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:17:57.245648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":48,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:42.736370+0000","last_change":"2024-05-07T02:26:07.793793+0000","last_active":"2024-05-07T02:27:42.736370+0000","last_peered":"2024-05-07T02:27:42.736370+0000","last_clean":"2024-05-07T02:27:42.736370+0000","last_became_active":"2024-05-07T02:26:07.790291+0000","last_became_peered":"2024-05-07T02:26:07.790291+0000","last_unstale":"2024-05-07T02:27:42.736370+0000","last_undegraded":"2024-05-07T02:27:42.736370+0000","last_fullsized":"2024-05-07T02:27:42.736370+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:50:14.591968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.580720+0000","last_change":"2024-05-07T02:26:11.869184+0000","last_active":"2024-05-07T02:27:32.580720+0000","last_peered":"2024-05-07T02:27:32.580720+0000","last_clean":"2024-05-07T02:27:32.580720+0000","last_became_active":"2024-05-07T02:26:11.868773+0000","last_became_peered":"2024-05-07T02:26:11.868773+0000","last_unstale":"2024-05-07T02:27:32.580720+0000","last_undegraded":"2024-05-07T02:27:32.580720+0000","last_fullsized":"2024-05-07T02:27:32.580720+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:24:04.986359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.577452+0000","last_change":"2024-05-07T02:26:25.580095+0000","last_active":"2024-05-07T02:27:32.577452+0000","last_peered":"2024-05-07T02:27:32.577452+0000","last_clean":"2024-05-07T02:27:32.577452+0000","last_became_active":"2024-05-07T02:26:25.579615+0000","last_became_peered":"2024-05-07T02:26:25.579615+0000","last_unstale":"2024-05-07T02:27:32.577452+0000","last_undegraded":"2024-05-07T02:27:32.577452+0000","last_fullsized":"2024-05-07T02:27:32.577452+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:18:05.714552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:41.729736+0000","last_change":"2024-05-07T02:26:07.784709+0000","last_active":"2024-05-07T02:27:41.729736+0000","last_peered":"2024-05-07T02:27:41.729736+0000","last_clean":"2024-05-07T02:27:41.729736+0000","last_became_active":"2024-05-07T02:26:07.784282+0000","last_became_peered":"2024-05-07T02:26:07.784282+0000","last_unstale":"2024-05-07T02:27:41.729736+0000","last_undegraded":"2024-05-07T02:27:41.729736+0000","last_fullsized":"2024-05-07T02:27:41.729736+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:01:47.023496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.579363+0000","last_change":"2024-05-07T02:26:11.879152+0000","last_active":"2024-05-07T02:27:32.579363+0000","last_peered":"2024-05-07T02:27:32.579363+0000","last_clean":"2024-05-07T02:27:32.579363+0000","last_became_active":"2024-05-07T02:26:11.878571+0000","last_became_peered":"2024-05-07T02:26:11.878571+0000","last_unstale":"2024-05-07T02:27:32.579363+0000","last_undegraded":"2024-05-07T02:27:32.579363+0000","last_fullsized":"2024-05-07T02:27:32.579363+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:04:12.637807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.577504+0000","last_change":"2024-05-07T02:26:25.867199+0000","last_active":"2024-05-07T02:27:32.577504+0000","last_peered":"2024-05-07T02:27:32.577504+0000","last_clean":"2024-05-07T02:27:32.577504+0000","last_became_active":"2024-05-07T02:26:25.866624+0000","last_became_peered":"2024-05-07T02:26:25.866624+0000","last_unstale":"2024-05-07T02:27:32.577504+0000","last_undegraded":"2024-05-07T02:27:32.577504+0000","last_fullsized":"2024-05-07T02:27:32.577504+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:14:49.513313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:39.700515+0000","last_change":"2024-05-07T02:26:07.784155+0000","last_active":"2024-05-07T02:27:39.700515+0000","last_peered":"2024-05-07T02:27:39.700515+0000","last_clean":"2024-05-07T02:27:39.700515+0000","last_became_active":"2024-05-07T02:26:07.783579+0000","last_became_peered":"2024-05-07T02:26:07.783579+0000","last_unstale":"2024-05-07T02:27:39.700515+0000","last_undegraded":"2024-05-07T02:27:39.700515+0000","last_fullsized":"2024-05-07T02:27:39.700515+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T08:51:32.037959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.582399+0000","last_change":"2024-05-07T02:26:11.868454+0000","last_active":"2024-05-07T02:27:32.582399+0000","last_peered":"2024-05-07T02:27:32.582399+0000","last_clean":"2024-05-07T02:27:32.582399+0000","last_became_active":"2024-05-07T02:26:11.868117+0000","last_became_peered":"2024-05-07T02:26:11.868117+0000","last_unstale":"2024-05-07T02:27:32.582399+0000","last_undegraded":"2024-05-07T02:27:32.582399+0000","last_fullsized":"2024-05-07T02:27:32.582399+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:24:36.703684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.623728+0000","last_change":"2024-05-07T02:26:25.573834+0000","last_active":"2024-05-07T02:27:32.623728+0000","last_peered":"2024-05-07T02:27:32.623728+0000","last_clean":"2024-05-07T02:27:32.623728+0000","last_became_active":"2024-05-07T02:26:25.573372+0000","last_became_peered":"2024-05-07T02:26:25.573372+0000","last_unstale":"2024-05-07T02:27:32.623728+0000","last_undegraded":"2024-05-07T02:27:32.623728+0000","last_fullsized":"2024-05-07T02:27:32.623728+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:03:36.916915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":79,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:28:14.825457+0000","last_change":"2024-05-07T02:26:08.000245+0000","last_active":"2024-05-07T02:28:14.825457+0000","last_peered":"2024-05-07T02:28:14.825457+0000","last_clean":"2024-05-07T02:28:14.825457+0000","last_became_active":"2024-05-07T02:26:07.999587+0000","last_became_peered":"2024-05-07T02:26:07.999587+0000","last_unstale":"2024-05-07T02:28:14.825457+0000","last_undegraded":"2024-05-07T02:28:14.825457+0000","last_fullsized":"2024-05-07T02:28:14.825457+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:44:41.754799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":34,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.582645+0000","last_change":"2024-05-07T02:26:11.858308+0000","last_active":"2024-05-07T02:27:32.582645+0000","last_peered":"2024-05-07T02:27:32.582645+0000","last_clean":"2024-05-07T02:27:32.582645+0000","last_became_active":"2024-05-07T02:26:11.857855+0000","last_became_peered":"2024-05-07T02:26:11.857855+0000","last_unstale":"2024-05-07T02:27:32.582645+0000","last_undegraded":"2024-05-07T02:27:32.582645+0000","last_fullsized":"2024-05-07T02:27:32.582645+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:11:50.021178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.583699+0000","last_change":"2024-05-07T02:26:25.584090+0000","last_active":"2024-05-07T02:27:32.583699+0000","last_peered":"2024-05-07T02:27:32.583699+0000","last_clean":"2024-05-07T02:27:32.583699+0000","last_became_active":"2024-05-07T02:26:25.580108+0000","last_became_peered":"2024-05-07T02:26:25.580108+0000","last_unstale":"2024-05-07T02:27:32.583699+0000","last_undegraded":"2024-05-07T02:27:32.583699+0000","last_fullsized":"2024-05-07T02:27:32.583699+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:56:56.964047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:42.735224+0000","last_change":"2024-05-07T02:26:07.788188+0000","last_active":"2024-05-07T02:27:42.735224+0000","last_peered":"2024-05-07T02:27:42.735224+0000","last_clean":"2024-05-07T02:27:42.735224+0000","last_became_active":"2024-05-07T02:26:07.787826+0000","last_became_peered":"2024-05-07T02:26:07.787826+0000","last_unstale":"2024-05-07T02:27:42.735224+0000","last_undegraded":"2024-05-07T02:27:42.735224+0000","last_fullsized":"2024-05-07T02:27:42.735224+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:06:14.588079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.582250+0000","last_change":"2024-05-07T02:26:11.862925+0000","last_active":"2024-05-07T02:27:32.582250+0000","last_peered":"2024-05-07T02:27:32.582250+0000","last_clean":"2024-05-07T02:27:32.582250+0000","last_became_active":"2024-05-07T02:26:11.861332+0000","last_became_peered":"2024-05-07T02:26:11.861332+0000","last_unstale":"2024-05-07T02:27:32.582250+0000","last_undegraded":"2024-05-07T02:27:32.582250+0000","last_fullsized":"2024-05-07T02:27:32.582250+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:51:11.133504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.622917+0000","last_change":"2024-05-07T02:26:25.557151+0000","last_active":"2024-05-07T02:27:32.622917+0000","last_peered":"2024-05-07T02:27:32.622917+0000","last_clean":"2024-05-07T02:27:32.622917+0000","last_became_active":"2024-05-07T02:26:25.556615+0000","last_became_peered":"2024-05-07T02:26:25.556615+0000","last_unstale":"2024-05-07T02:27:32.622917+0000","last_undegraded":"2024-05-07T02:27:32.622917+0000","last_fullsized":"2024-05-07T02:27:32.622917+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:33:20.446543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":55,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:42.736928+0000","last_change":"2024-05-07T02:27:32.598094+0000","last_active":"2024-05-07T02:27:42.736928+0000","last_peered":"2024-05-07T02:27:42.736928+0000","last_clean":"2024-05-07T02:27:42.736928+0000","last_became_active":"2024-05-07T02:27:32.594099+0000","last_became_peered":"2024-05-07T02:27:32.594099+0000","last_unstale":"2024-05-07T02:27:42.736928+0000","last_undegraded":"2024-05-07T02:27:42.736928+0000","last_fullsized":"2024-05-07T02:27:42.736928+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:44:50.979927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.582788+0000","last_change":"2024-05-07T02:26:11.860507+0000","last_active":"2024-05-07T02:27:32.582788+0000","last_peered":"2024-05-07T02:27:32.582788+0000","last_clean":"2024-05-07T02:27:32.582788+0000","last_became_active":"2024-05-07T02:26:11.860116+0000","last_became_peered":"2024-05-07T02:26:11.860116+0000","last_unstale":"2024-05-07T02:27:32.582788+0000","last_undegraded":"2024-05-07T02:27:32.582788+0000","last_fullsized":"2024-05-07T02:27:32.582788+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T05:09:38.963040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.580920+0000","last_change":"2024-05-07T02:26:11.867932+0000","last_active":"2024-05-07T02:27:32.580920+0000","last_peered":"2024-05-07T02:27:32.580920+0000","last_clean":"2024-05-07T02:27:32.580920+0000","last_became_active":"2024-05-07T02:26:11.867537+0000","last_became_peered":"2024-05-07T02:26:11.867537+0000","last_unstale":"2024-05-07T02:27:32.580920+0000","last_undegraded":"2024-05-07T02:27:32.580920+0000","last_fullsized":"2024-05-07T02:27:32.580920+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:31:35.843401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.582628+0000","last_change":"2024-05-07T02:26:11.866074+0000","last_active":"2024-05-07T02:27:32.582628+0000","last_peered":"2024-05-07T02:27:32.582628+0000","last_clean":"2024-05-07T02:27:32.582628+0000","last_became_active":"2024-05-07T02:26:11.865669+0000","last_became_peered":"2024-05-07T02:26:11.865669+0000","last_unstale":"2024-05-07T02:27:32.582628+0000","last_undegraded":"2024-05-07T02:27:32.582628+0000","last_fullsized":"2024-05-07T02:27:32.582628+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:08:51.323706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.582027+0000","last_change":"2024-05-07T02:26:11.876428+0000","last_active":"2024-05-07T02:27:32.582027+0000","last_peered":"2024-05-07T02:27:32.582027+0000","last_clean":"2024-05-07T02:27:32.582027+0000","last_became_active":"2024-05-07T02:26:11.876012+0000","last_became_peered":"2024-05-07T02:26:11.876012+0000","last_unstale":"2024-05-07T02:27:32.582027+0000","last_undegraded":"2024-05-07T02:27:32.582027+0000","last_fullsized":"2024-05-07T02:27:32.582027+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:24:58.088695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.587786+0000","last_change":"2024-05-07T02:26:11.880682+0000","last_active":"2024-05-07T02:27:32.587786+0000","last_peered":"2024-05-07T02:27:32.587786+0000","last_clean":"2024-05-07T02:27:32.587786+0000","last_became_active":"2024-05-07T02:26:11.880308+0000","last_became_peered":"2024-05-07T02:26:11.880308+0000","last_unstale":"2024-05-07T02:27:32.587786+0000","last_undegraded":"2024-05-07T02:27:32.587786+0000","last_fullsized":"2024-05-07T02:27:32.587786+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:52:45.083964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.589129+0000","last_change":"2024-05-07T02:26:11.876932+0000","last_active":"2024-05-07T02:27:32.589129+0000","last_peered":"2024-05-07T02:27:32.589129+0000","last_clean":"2024-05-07T02:27:32.589129+0000","last_became_active":"2024-05-07T02:26:11.876082+0000","last_became_peered":"2024-05-07T02:26:11.876082+0000","last_unstale":"2024-05-07T02:27:32.589129+0000","last_undegraded":"2024-05-07T02:27:32.589129+0000","last_fullsized":"2024-05-07T02:27:32.589129+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:45:21.435500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.580599+0000","last_change":"2024-05-07T02:26:11.861339+0000","last_active":"2024-05-07T02:27:32.580599+0000","last_peered":"2024-05-07T02:27:32.580599+0000","last_clean":"2024-05-07T02:27:32.580599+0000","last_became_active":"2024-05-07T02:26:11.860912+0000","last_became_peered":"2024-05-07T02:26:11.860912+0000","last_unstale":"2024-05-07T02:27:32.580599+0000","last_undegraded":"2024-05-07T02:27:32.580599+0000","last_fullsized":"2024-05-07T02:27:32.580599+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:12:57.944986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.583626+0000","last_change":"2024-05-07T02:26:11.867258+0000","last_active":"2024-05-07T02:27:32.583626+0000","last_peered":"2024-05-07T02:27:32.583626+0000","last_clean":"2024-05-07T02:27:32.583626+0000","last_became_active":"2024-05-07T02:26:11.866837+0000","last_became_peered":"2024-05-07T02:26:11.866837+0000","last_unstale":"2024-05-07T02:27:32.583626+0000","last_undegraded":"2024-05-07T02:27:32.583626+0000","last_fullsized":"2024-05-07T02:27:32.583626+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:07:58.589457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.579722+0000","last_change":"2024-05-07T02:26:11.888434+0000","last_active":"2024-05-07T02:27:32.579722+0000","last_peered":"2024-05-07T02:27:32.579722+0000","last_clean":"2024-05-07T02:27:32.579722+0000","last_became_active":"2024-05-07T02:26:11.887681+0000","last_became_peered":"2024-05-07T02:26:11.887681+0000","last_unstale":"2024-05-07T02:27:32.579722+0000","last_undegraded":"2024-05-07T02:27:32.579722+0000","last_fullsized":"2024-05-07T02:27:32.579722+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:38:58.896781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.623214+0000","last_change":"2024-05-07T02:26:11.875304+0000","last_active":"2024-05-07T02:27:32.623214+0000","last_peered":"2024-05-07T02:27:32.623214+0000","last_clean":"2024-05-07T02:27:32.623214+0000","last_became_active":"2024-05-07T02:26:11.874946+0000","last_became_peered":"2024-05-07T02:26:11.874946+0000","last_unstale":"2024-05-07T02:27:32.623214+0000","last_undegraded":"2024-05-07T02:27:32.623214+0000","last_fullsized":"2024-05-07T02:27:32.623214+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:56:04.093791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.578243+0000","last_change":"2024-05-07T02:26:11.879311+0000","last_active":"2024-05-07T02:27:32.578243+0000","last_peered":"2024-05-07T02:27:32.578243+0000","last_clean":"2024-05-07T02:27:32.578243+0000","last_became_active":"2024-05-07T02:26:11.877620+0000","last_became_peered":"2024-05-07T02:26:11.877620+0000","last_unstale":"2024-05-07T02:27:32.578243+0000","last_undegraded":"2024-05-07T02:27:32.578243+0000","last_fullsized":"2024-05-07T02:27:32.578243+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T08:41:40.525708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.623872+0000","last_change":"2024-05-07T02:26:11.876726+0000","last_active":"2024-05-07T02:27:32.623872+0000","last_peered":"2024-05-07T02:27:32.623872+0000","last_clean":"2024-05-07T02:27:32.623872+0000","last_became_active":"2024-05-07T02:26:11.876234+0000","last_became_peered":"2024-05-07T02:26:11.876234+0000","last_unstale":"2024-05-07T02:27:32.623872+0000","last_undegraded":"2024-05-07T02:27:32.623872+0000","last_fullsized":"2024-05-07T02:27:32.623872+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:26:34.422258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.590155+0000","last_change":"2024-05-07T02:26:11.879159+0000","last_active":"2024-05-07T02:27:32.590155+0000","last_peered":"2024-05-07T02:27:32.590155+0000","last_clean":"2024-05-07T02:27:32.590155+0000","last_became_active":"2024-05-07T02:26:11.878769+0000","last_became_peered":"2024-05-07T02:26:11.878769+0000","last_unstale":"2024-05-07T02:27:32.590155+0000","last_undegraded":"2024-05-07T02:27:32.590155+0000","last_fullsized":"2024-05-07T02:27:32.590155+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:05:27.089475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.590188+0000","last_change":"2024-05-07T02:27:32.590188+0000","last_active":"2024-05-07T02:27:32.590188+0000","last_peered":"2024-05-07T02:27:32.590188+0000","last_clean":"2024-05-07T02:27:32.590188+0000","last_became_active":"2024-05-07T02:27:32.589784+0000","last_became_peered":"2024-05-07T02:27:32.589784+0000","last_unstale":"2024-05-07T02:27:32.590188+0000","last_undegraded":"2024-05-07T02:27:32.590188+0000","last_fullsized":"2024-05-07T02:27:32.590188+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:32:40.488143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73905,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":396,"num_read_kb":284,"num_write":250,"num_write_kb":146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":581632,"data_stored":213185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":215,"ondisk_log_size":215,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710762,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":1007616,"data_stored":496962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906993,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32836,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712316,"statfs":{"total":95995035648,"available":95961411584,"internally_reserved":0,"allocated":1699840,"data_stored":1150758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070518,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":500876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266750,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1011712,"data_stored":497046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462979,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28596,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716556,"statfs":{"total":95995035648,"available":95965753344,"internally_reserved":0,"allocated":1617920,"data_stored":1093655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626504,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":492675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":556532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738476,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":493740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411425,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":556361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182490,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1011712,"data_stored":491992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986262,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":1007616,"data_stored":496358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822735,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28604,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716548,"statfs":{"total":95995035648,"available":95965745152,"internally_reserved":0,"allocated":1626112,"data_stored":1088846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":68828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-07T02:28:24.236 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-07T02:28:24.741 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:28:24.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:24 smithi064 ceph-mon[34699]: from='client.24763 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-07T02:28:24.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:24 smithi064 ceph-mon[34699]: pgmap v403: 129 pgs: 129 active+clean; 649 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-05-07T02:28:24.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:24 smithi019 ceph-mon[29781]: from='client.24763 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-07T02:28:24.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:24 smithi019 ceph-mon[29781]: pgmap v403: 129 pgs: 129 active+clean; 649 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-05-07T02:28:24.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:24 smithi027 ceph-mon[34398]: from='client.24763 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-07T02:28:24.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:24 smithi027 ceph-mon[34398]: pgmap v403: 129 pgs: 129 active+clean; 649 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-05-07T02:28:26.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:26 smithi019 ceph-mon[29781]: pgmap v404: 129 pgs: 129 active+clean; 649 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-05-07T02:28:26.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:26 smithi064 ceph-mon[34699]: pgmap v404: 129 pgs: 129 active+clean; 649 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-05-07T02:28:26.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:26 smithi027 ceph-mon[34398]: pgmap v404: 129 pgs: 129 active+clean; 649 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-05-07T02:28:27.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.315+0000 7fbe2d800700 1 -- 172.21.15.19:0/322464722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe2810e750 msgr2=0x7fbe28110bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:27.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.315+0000 7fbe2d800700 1 --2- 172.21.15.19:0/322464722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe2810e750 0x7fbe28110bf0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbe2000b0a0 tx=0x7fbe2000b3b0 comp rx=0 tx=0).stop 2024-05-07T02:28:27.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.315+0000 7fbe2d800700 1 -- 172.21.15.19:0/322464722 shutdown_connections 2024-05-07T02:28:27.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.315+0000 7fbe2d800700 1 --2- 172.21.15.19:0/322464722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe2810e750 0x7fbe28110bf0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:27.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.315+0000 7fbe2d800700 1 --2- 172.21.15.19:0/322464722 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe28105370 0x7fbe28105780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:27.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.315+0000 7fbe2d800700 1 --2- 172.21.15.19:0/322464722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe28105cc0 0x7fbe2810e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:27.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.315+0000 7fbe2d800700 1 -- 172.21.15.19:0/322464722 >> 172.21.15.19:0/322464722 conn(0x7fbe28078ac0 msgr2=0x7fbe28078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:28:27.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.315+0000 7fbe2d800700 1 -- 172.21.15.19:0/322464722 shutdown_connections 2024-05-07T02:28:27.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.315+0000 7fbe2d800700 1 -- 172.21.15.19:0/322464722 wait complete. 2024-05-07T02:28:27.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.316+0000 7fbe2d800700 1 Processor -- start 2024-05-07T02:28:27.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.316+0000 7fbe2d800700 1 -- start start 2024-05-07T02:28:27.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.317+0000 7fbe2d800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe28105370 0x7fbe281a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:27.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.317+0000 7fbe2d800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe28105cc0 0x7fbe281a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:27.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.317+0000 7fbe2d800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe2810e750 0x7fbe2819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:27.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.317+0000 7fbe2d800700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbe281142c0 con 0x7fbe28105370 2024-05-07T02:28:27.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.317+0000 7fbe2d800700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbe28114140 con 0x7fbe2810e750 2024-05-07T02:28:27.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.317+0000 7fbe2d800700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbe28114440 con 0x7fbe28105cc0 2024-05-07T02:28:27.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.317+0000 7fbe27e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe28105370 0x7fbe281a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:27.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.317+0000 7fbe27e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe28105370 0x7fbe281a07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53860/0 (socket says 172.21.15.19:53860) 2024-05-07T02:28:27.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.317+0000 7fbe27e00700 1 -- 172.21.15.19:0/2974941767 learned_addr learned my addr 172.21.15.19:0/2974941767 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:28:27.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.317+0000 7fbe27400700 1 --2- 172.21.15.19:0/2974941767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe28105cc0 0x7fbe281a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:27.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.317+0000 7fbe2ce00700 1 --2- 172.21.15.19:0/2974941767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe2810e750 0x7fbe2819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:27.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.317+0000 7fbe27e00700 1 -- 172.21.15.19:0/2974941767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe28105cc0 msgr2=0x7fbe281a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:27.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.317+0000 7fbe27e00700 1 --2- 172.21.15.19:0/2974941767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe28105cc0 0x7fbe281a0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:27.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.317+0000 7fbe27e00700 1 -- 172.21.15.19:0/2974941767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe2810e750 msgr2=0x7fbe2819a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:27.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.317+0000 7fbe27e00700 1 --2- 172.21.15.19:0/2974941767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe2810e750 0x7fbe2819a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:27.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.317+0000 7fbe27e00700 1 -- 172.21.15.19:0/2974941767 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe2819b090 con 0x7fbe28105370 2024-05-07T02:28:27.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.317+0000 7fbe27400700 1 --2- 172.21.15.19:0/2974941767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe28105cc0 0x7fbe281a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:28:27.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.318+0000 7fbe27e00700 1 --2- 172.21.15.19:0/2974941767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe28105370 0x7fbe281a07c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbe1800e9a0 tx=0x7fbe1800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:28:27.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.318+0000 7fbe24c00700 1 -- 172.21.15.19:0/2974941767 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1800ca10 con 0x7fbe28105370 2024-05-07T02:28:27.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.318+0000 7fbe24c00700 1 -- 172.21.15.19:0/2974941767 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe1800cbb0 con 0x7fbe28105370 2024-05-07T02:28:27.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.318+0000 7fbe2d800700 1 -- 172.21.15.19:0/2974941767 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe2819b380 con 0x7fbe28105370 2024-05-07T02:28:27.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.318+0000 7fbe24c00700 1 -- 172.21.15.19:0/2974941767 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe18018680 con 0x7fbe28105370 2024-05-07T02:28:27.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.318+0000 7fbe2d800700 1 -- 172.21.15.19:0/2974941767 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe281a7650 con 0x7fbe28105370 2024-05-07T02:28:27.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.319+0000 7fbe2d800700 1 -- 172.21.15.19:0/2974941767 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe280512c0 con 0x7fbe28105370 2024-05-07T02:28:27.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.324+0000 7fbe24c00700 1 -- 172.21.15.19:0/2974941767 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fbe18010430 con 0x7fbe28105370 2024-05-07T02:28:27.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.325+0000 7fbe24c00700 1 --2- 172.21.15.19:0/2974941767 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fbe0c0b2910 0x7fbe0c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:27.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.325+0000 7fbe24c00700 1 -- 172.21.15.19:0/2974941767 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fbe180e2c10 con 0x7fbe28105370 2024-05-07T02:28:27.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.325+0000 7fbe27400700 1 --2- 172.21.15.19:0/2974941767 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fbe0c0b2910 0x7fbe0c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:27.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.326+0000 7fbe27400700 1 --2- 172.21.15.19:0/2974941767 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fbe0c0b2910 0x7fbe0c0b4e20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbe28102460 tx=0x7fbe14009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:28:27.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.328+0000 7fbe24c00700 1 -- 172.21.15.19:0/2974941767 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbe180aa410 con 0x7fbe28105370 2024-05-07T02:28:27.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.561+0000 7fbe2d800700 1 -- 172.21.15.19:0/2974941767 --> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbe28101500 con 0x7fbe0c0b2910 2024-05-07T02:28:27.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.581+0000 7fbe24c00700 1 -- 172.21.15.19:0/2974941767 <== mgr.14152 v2:172.21.15.19:6800/2059564551 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340387 (secure 0 0 0) 0x7fbe28101500 con 0x7fbe0c0b2910 2024-05-07T02:28:27.587 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:28:27.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.588+0000 7fbe2d800700 1 -- 172.21.15.19:0/2974941767 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fbe0c0b2910 msgr2=0x7fbe0c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:27.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.588+0000 7fbe2d800700 1 --2- 172.21.15.19:0/2974941767 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fbe0c0b2910 0x7fbe0c0b4e20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbe28102460 tx=0x7fbe14009260 comp rx=0 tx=0).stop 2024-05-07T02:28:27.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.588+0000 7fbe2d800700 1 -- 172.21.15.19:0/2974941767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe28105370 msgr2=0x7fbe281a07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:27.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.588+0000 7fbe2d800700 1 --2- 172.21.15.19:0/2974941767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe28105370 0x7fbe281a07c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbe1800e9a0 tx=0x7fbe1800ecb0 comp rx=0 tx=0).stop 2024-05-07T02:28:27.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.588+0000 7fbe2d800700 1 -- 172.21.15.19:0/2974941767 shutdown_connections 2024-05-07T02:28:27.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.588+0000 7fbe2d800700 1 --2- 172.21.15.19:0/2974941767 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fbe0c0b2910 0x7fbe0c0b4e20 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:27.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.588+0000 7fbe2d800700 1 --2- 172.21.15.19:0/2974941767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe2810e750 0x7fbe2819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:27.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.588+0000 7fbe2d800700 1 --2- 172.21.15.19:0/2974941767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe28105370 0x7fbe281a07c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:27.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.588+0000 7fbe2d800700 1 --2- 172.21.15.19:0/2974941767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbe28105cc0 0x7fbe281a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:27.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.588+0000 7fbe2d800700 1 -- 172.21.15.19:0/2974941767 >> 172.21.15.19:0/2974941767 conn(0x7fbe28078ac0 msgr2=0x7fbe280ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:28:27.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.589+0000 7fbe2d800700 1 -- 172.21.15.19:0/2974941767 shutdown_connections 2024-05-07T02:28:27.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:27.589+0000 7fbe2d800700 1 -- 172.21.15.19:0/2974941767 wait complete. 2024-05-07T02:28:27.590 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-05-07T02:28:28.630 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":405,"stamp":"2024-05-07T02:28:27.497151+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665645,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":533,"num_read_kb":380,"num_write":376,"num_write_kb":2300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":310,"ondisk_log_size":310,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375500,"kb_used_data":13868,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566324,"statfs":{"total":1151940427776,"available":1151555915776,"internally_reserved":0,"allocated":14200832,"data_stored":7927745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10706,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":133,"num_read_kb":83,"num_write":8,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007620"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.580529+0000","last_change":"2024-05-07T02:26:11.876828+0000","last_active":"2024-05-07T02:27:32.580529+0000","last_peered":"2024-05-07T02:27:32.580529+0000","last_clean":"2024-05-07T02:27:32.580529+0000","last_became_active":"2024-05-07T02:26:11.876319+0000","last_became_peered":"2024-05-07T02:26:11.876319+0000","last_unstale":"2024-05-07T02:27:32.580529+0000","last_undegraded":"2024-05-07T02:27:32.580529+0000","last_fullsized":"2024-05-07T02:27:32.580529+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:07:33.181373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.583281+0000","last_change":"2024-05-07T02:26:11.873388+0000","last_active":"2024-05-07T02:27:32.583281+0000","last_peered":"2024-05-07T02:27:32.583281+0000","last_clean":"2024-05-07T02:27:32.583281+0000","last_became_active":"2024-05-07T02:26:11.869447+0000","last_became_peered":"2024-05-07T02:26:11.869447+0000","last_unstale":"2024-05-07T02:27:32.583281+0000","last_undegraded":"2024-05-07T02:27:32.583281+0000","last_fullsized":"2024-05-07T02:27:32.583281+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:09:52.876547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.582674+0000","last_change":"2024-05-07T02:26:11.870797+0000","last_active":"2024-05-07T02:27:32.582674+0000","last_peered":"2024-05-07T02:27:32.582674+0000","last_clean":"2024-05-07T02:27:32.582674+0000","last_became_active":"2024-05-07T02:26:11.870411+0000","last_became_peered":"2024-05-07T02:26:11.870411+0000","last_unstale":"2024-05-07T02:27:32.582674+0000","last_undegraded":"2024-05-07T02:27:32.582674+0000","last_fullsized":"2024-05-07T02:27:32.582674+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:34:12.017508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.577729+0000","last_change":"2024-05-07T02:26:11.867327+0000","last_active":"2024-05-07T02:27:32.577729+0000","last_peered":"2024-05-07T02:27:32.577729+0000","last_clean":"2024-05-07T02:27:32.577729+0000","last_became_active":"2024-05-07T02:26:11.866912+0000","last_became_peered":"2024-05-07T02:26:11.866912+0000","last_unstale":"2024-05-07T02:27:32.577729+0000","last_undegraded":"2024-05-07T02:27:32.577729+0000","last_fullsized":"2024-05-07T02:27:32.577729+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:36:36.616706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.577542+0000","last_change":"2024-05-07T02:26:11.878614+0000","last_active":"2024-05-07T02:27:32.577542+0000","last_peered":"2024-05-07T02:27:32.577542+0000","last_clean":"2024-05-07T02:27:32.577542+0000","last_became_active":"2024-05-07T02:26:11.878069+0000","last_became_peered":"2024-05-07T02:26:11.878069+0000","last_unstale":"2024-05-07T02:27:32.577542+0000","last_undegraded":"2024-05-07T02:27:32.577542+0000","last_fullsized":"2024-05-07T02:27:32.577542+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:40:15.991433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.625099+0000","last_change":"2024-05-07T02:26:11.877755+0000","last_active":"2024-05-07T02:27:32.625099+0000","last_peered":"2024-05-07T02:27:32.625099+0000","last_clean":"2024-05-07T02:27:32.625099+0000","last_became_active":"2024-05-07T02:26:11.877373+0000","last_became_peered":"2024-05-07T02:26:11.877373+0000","last_unstale":"2024-05-07T02:27:32.625099+0000","last_undegraded":"2024-05-07T02:27:32.625099+0000","last_fullsized":"2024-05-07T02:27:32.625099+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:49:55.984807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.582015+0000","last_change":"2024-05-07T02:26:11.882194+0000","last_active":"2024-05-07T02:27:32.582015+0000","last_peered":"2024-05-07T02:27:32.582015+0000","last_clean":"2024-05-07T02:27:32.582015+0000","last_became_active":"2024-05-07T02:26:11.881873+0000","last_became_peered":"2024-05-07T02:26:11.881873+0000","last_unstale":"2024-05-07T02:27:32.582015+0000","last_undegraded":"2024-05-07T02:27:32.582015+0000","last_fullsized":"2024-05-07T02:27:32.582015+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:47:31.000357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.577719+0000","last_change":"2024-05-07T02:26:11.857366+0000","last_active":"2024-05-07T02:27:32.577719+0000","last_peered":"2024-05-07T02:27:32.577719+0000","last_clean":"2024-05-07T02:27:32.577719+0000","last_became_active":"2024-05-07T02:26:11.854054+0000","last_became_peered":"2024-05-07T02:26:11.854054+0000","last_unstale":"2024-05-07T02:27:32.577719+0000","last_undegraded":"2024-05-07T02:27:32.577719+0000","last_fullsized":"2024-05-07T02:27:32.577719+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:07:04.834022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.623944+0000","last_change":"2024-05-07T02:26:11.867105+0000","last_active":"2024-05-07T02:27:32.623944+0000","last_peered":"2024-05-07T02:27:32.623944+0000","last_clean":"2024-05-07T02:27:32.623944+0000","last_became_active":"2024-05-07T02:26:11.865986+0000","last_became_peered":"2024-05-07T02:26:11.865986+0000","last_unstale":"2024-05-07T02:27:32.623944+0000","last_undegraded":"2024-05-07T02:27:32.623944+0000","last_fullsized":"2024-05-07T02:27:32.623944+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:13:01.295354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.624609+0000","last_change":"2024-05-07T02:26:11.868039+0000","last_active":"2024-05-07T02:27:32.624609+0000","last_peered":"2024-05-07T02:27:32.624609+0000","last_clean":"2024-05-07T02:27:32.624609+0000","last_became_active":"2024-05-07T02:26:11.867588+0000","last_became_peered":"2024-05-07T02:26:11.867588+0000","last_unstale":"2024-05-07T02:27:32.624609+0000","last_undegraded":"2024-05-07T02:27:32.624609+0000","last_fullsized":"2024-05-07T02:27:32.624609+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:41:45.149620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.582476+0000","last_change":"2024-05-07T02:26:11.859579+0000","last_active":"2024-05-07T02:27:32.582476+0000","last_peered":"2024-05-07T02:27:32.582476+0000","last_clean":"2024-05-07T02:27:32.582476+0000","last_became_active":"2024-05-07T02:26:11.859154+0000","last_became_peered":"2024-05-07T02:26:11.859154+0000","last_unstale":"2024-05-07T02:27:32.582476+0000","last_undegraded":"2024-05-07T02:27:32.582476+0000","last_fullsized":"2024-05-07T02:27:32.582476+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:26:40.616956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.577529+0000","last_change":"2024-05-07T02:26:11.858220+0000","last_active":"2024-05-07T02:27:32.577529+0000","last_peered":"2024-05-07T02:27:32.577529+0000","last_clean":"2024-05-07T02:27:32.577529+0000","last_became_active":"2024-05-07T02:26:11.857826+0000","last_became_peered":"2024-05-07T02:26:11.857826+0000","last_unstale":"2024-05-07T02:27:32.577529+0000","last_undegraded":"2024-05-07T02:27:32.577529+0000","last_fullsized":"2024-05-07T02:27:32.577529+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:21:36.031057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.590412+0000","last_change":"2024-05-07T02:26:11.880348+0000","last_active":"2024-05-07T02:27:32.590412+0000","last_peered":"2024-05-07T02:27:32.590412+0000","last_clean":"2024-05-07T02:27:32.590412+0000","last_became_active":"2024-05-07T02:26:11.879633+0000","last_became_peered":"2024-05-07T02:26:11.879633+0000","last_unstale":"2024-05-07T02:27:32.590412+0000","last_undegraded":"2024-05-07T02:27:32.590412+0000","last_fullsized":"2024-05-07T02:27:32.590412+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:59:34.567973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.582077+0000","last_change":"2024-05-07T02:26:11.865687+0000","last_active":"2024-05-07T02:27:32.582077+0000","last_peered":"2024-05-07T02:27:32.582077+0000","last_clean":"2024-05-07T02:27:32.582077+0000","last_became_active":"2024-05-07T02:26:11.865330+0000","last_became_peered":"2024-05-07T02:26:11.865330+0000","last_unstale":"2024-05-07T02:27:32.582077+0000","last_undegraded":"2024-05-07T02:27:32.582077+0000","last_fullsized":"2024-05-07T02:27:32.582077+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:36:10.610314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.582525+0000","last_change":"2024-05-07T02:26:11.870627+0000","last_active":"2024-05-07T02:27:32.582525+0000","last_peered":"2024-05-07T02:27:32.582525+0000","last_clean":"2024-05-07T02:27:32.582525+0000","last_became_active":"2024-05-07T02:26:11.870229+0000","last_became_peered":"2024-05-07T02:26:11.870229+0000","last_unstale":"2024-05-07T02:27:32.582525+0000","last_undegraded":"2024-05-07T02:27:32.582525+0000","last_fullsized":"2024-05-07T02:27:32.582525+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:06:46.008651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.623191+0000","last_change":"2024-05-07T02:26:11.868698+0000","last_active":"2024-05-07T02:27:32.623191+0000","last_peered":"2024-05-07T02:27:32.623191+0000","last_clean":"2024-05-07T02:27:32.623191+0000","last_became_active":"2024-05-07T02:26:11.868294+0000","last_became_peered":"2024-05-07T02:26:11.868294+0000","last_unstale":"2024-05-07T02:27:32.623191+0000","last_undegraded":"2024-05-07T02:27:32.623191+0000","last_fullsized":"2024-05-07T02:27:32.623191+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:16:22.856200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.580185+0000","last_change":"2024-05-07T02:26:11.877116+0000","last_active":"2024-05-07T02:27:32.580185+0000","last_peered":"2024-05-07T02:27:32.580185+0000","last_clean":"2024-05-07T02:27:32.580185+0000","last_became_active":"2024-05-07T02:26:11.876305+0000","last_became_peered":"2024-05-07T02:26:11.876305+0000","last_unstale":"2024-05-07T02:27:32.580185+0000","last_undegraded":"2024-05-07T02:27:32.580185+0000","last_fullsized":"2024-05-07T02:27:32.580185+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:25:46.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.586264+0000","last_change":"2024-05-07T02:26:11.877488+0000","last_active":"2024-05-07T02:27:32.586264+0000","last_peered":"2024-05-07T02:27:32.586264+0000","last_clean":"2024-05-07T02:27:32.586264+0000","last_became_active":"2024-05-07T02:26:11.874500+0000","last_became_peered":"2024-05-07T02:26:11.874500+0000","last_unstale":"2024-05-07T02:27:32.586264+0000","last_undegraded":"2024-05-07T02:27:32.586264+0000","last_fullsized":"2024-05-07T02:27:32.586264+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:53:50.046342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.583493+0000","last_change":"2024-05-07T02:26:11.876500+0000","last_active":"2024-05-07T02:27:32.583493+0000","last_peered":"2024-05-07T02:27:32.583493+0000","last_clean":"2024-05-07T02:27:32.583493+0000","last_became_active":"2024-05-07T02:26:11.876084+0000","last_became_peered":"2024-05-07T02:26:11.876084+0000","last_unstale":"2024-05-07T02:27:32.583493+0000","last_undegraded":"2024-05-07T02:27:32.583493+0000","last_fullsized":"2024-05-07T02:27:32.583493+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:05:05.681488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.582344+0000","last_change":"2024-05-07T02:26:25.579209+0000","last_active":"2024-05-07T02:27:32.582344+0000","last_peered":"2024-05-07T02:27:32.582344+0000","last_clean":"2024-05-07T02:27:32.582344+0000","last_became_active":"2024-05-07T02:26:25.578665+0000","last_became_peered":"2024-05-07T02:26:25.578665+0000","last_unstale":"2024-05-07T02:27:32.582344+0000","last_undegraded":"2024-05-07T02:27:32.582344+0000","last_fullsized":"2024-05-07T02:27:32.582344+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:06:29.784016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":94,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:28:21.830933+0000","last_change":"2024-05-07T02:26:07.792993+0000","last_active":"2024-05-07T02:28:21.830933+0000","last_peered":"2024-05-07T02:28:21.830933+0000","last_clean":"2024-05-07T02:28:21.830933+0000","last_became_active":"2024-05-07T02:26:07.792402+0000","last_became_peered":"2024-05-07T02:26:07.792402+0000","last_unstale":"2024-05-07T02:28:21.830933+0000","last_undegraded":"2024-05-07T02:28:21.830933+0000","last_fullsized":"2024-05-07T02:28:21.830933+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:49:02.288255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.578807+0000","last_change":"2024-05-07T02:26:11.880654+0000","last_active":"2024-05-07T02:27:32.578807+0000","last_peered":"2024-05-07T02:27:32.578807+0000","last_clean":"2024-05-07T02:27:32.578807+0000","last_became_active":"2024-05-07T02:26:11.878759+0000","last_became_peered":"2024-05-07T02:26:11.878759+0000","last_unstale":"2024-05-07T02:27:32.578807+0000","last_undegraded":"2024-05-07T02:27:32.578807+0000","last_fullsized":"2024-05-07T02:27:32.578807+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:03:49.187502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.576828+0000","last_change":"2024-05-07T02:26:25.579835+0000","last_active":"2024-05-07T02:27:32.576828+0000","last_peered":"2024-05-07T02:27:32.576828+0000","last_clean":"2024-05-07T02:27:32.576828+0000","last_became_active":"2024-05-07T02:26:25.579105+0000","last_became_peered":"2024-05-07T02:26:25.579105+0000","last_unstale":"2024-05-07T02:27:32.576828+0000","last_undegraded":"2024-05-07T02:27:32.576828+0000","last_fullsized":"2024-05-07T02:27:32.576828+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:32:37.745967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:40.718679+0000","last_change":"2024-05-07T02:26:08.001879+0000","last_active":"2024-05-07T02:27:40.718679+0000","last_peered":"2024-05-07T02:27:40.718679+0000","last_clean":"2024-05-07T02:27:40.718679+0000","last_became_active":"2024-05-07T02:26:08.000244+0000","last_became_peered":"2024-05-07T02:26:08.000244+0000","last_unstale":"2024-05-07T02:27:40.718679+0000","last_undegraded":"2024-05-07T02:27:40.718679+0000","last_fullsized":"2024-05-07T02:27:40.718679+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:56:42.591923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.583035+0000","last_change":"2024-05-07T02:26:11.882351+0000","last_active":"2024-05-07T02:27:32.583035+0000","last_peered":"2024-05-07T02:27:32.583035+0000","last_clean":"2024-05-07T02:27:32.583035+0000","last_became_active":"2024-05-07T02:26:11.881922+0000","last_became_peered":"2024-05-07T02:26:11.881922+0000","last_unstale":"2024-05-07T02:27:32.583035+0000","last_undegraded":"2024-05-07T02:27:32.583035+0000","last_fullsized":"2024-05-07T02:27:32.583035+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:58:31.993859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.576995+0000","last_change":"2024-05-07T02:26:25.580419+0000","last_active":"2024-05-07T02:27:32.576995+0000","last_peered":"2024-05-07T02:27:32.576995+0000","last_clean":"2024-05-07T02:27:32.576995+0000","last_became_active":"2024-05-07T02:26:25.579411+0000","last_became_peered":"2024-05-07T02:26:25.579411+0000","last_unstale":"2024-05-07T02:27:32.576995+0000","last_undegraded":"2024-05-07T02:27:32.576995+0000","last_fullsized":"2024-05-07T02:27:32.576995+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:43:03.552210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:44.728261+0000","last_change":"2024-05-07T02:27:32.588255+0000","last_active":"2024-05-07T02:27:44.728261+0000","last_peered":"2024-05-07T02:27:44.728261+0000","last_clean":"2024-05-07T02:27:44.728261+0000","last_became_active":"2024-05-07T02:27:32.584405+0000","last_became_peered":"2024-05-07T02:27:32.584405+0000","last_unstale":"2024-05-07T02:27:44.728261+0000","last_undegraded":"2024-05-07T02:27:44.728261+0000","last_fullsized":"2024-05-07T02:27:44.728261+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:07:22.980041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.623629+0000","last_change":"2024-05-07T02:26:11.866949+0000","last_active":"2024-05-07T02:27:32.623629+0000","last_peered":"2024-05-07T02:27:32.623629+0000","last_clean":"2024-05-07T02:27:32.623629+0000","last_became_active":"2024-05-07T02:26:11.866453+0000","last_became_peered":"2024-05-07T02:26:11.866453+0000","last_unstale":"2024-05-07T02:27:32.623629+0000","last_undegraded":"2024-05-07T02:27:32.623629+0000","last_fullsized":"2024-05-07T02:27:32.623629+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T14:22:47.291470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.576766+0000","last_change":"2024-05-07T02:26:25.570555+0000","last_active":"2024-05-07T02:27:32.576766+0000","last_peered":"2024-05-07T02:27:32.576766+0000","last_clean":"2024-05-07T02:27:32.576766+0000","last_became_active":"2024-05-07T02:26:25.570044+0000","last_became_peered":"2024-05-07T02:26:25.570044+0000","last_unstale":"2024-05-07T02:27:32.576766+0000","last_undegraded":"2024-05-07T02:27:32.576766+0000","last_fullsized":"2024-05-07T02:27:32.576766+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:48:15.701335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:42.734914+0000","last_change":"2024-05-07T02:26:07.785938+0000","last_active":"2024-05-07T02:27:42.734914+0000","last_peered":"2024-05-07T02:27:42.734914+0000","last_clean":"2024-05-07T02:27:42.734914+0000","last_became_active":"2024-05-07T02:26:07.785573+0000","last_became_peered":"2024-05-07T02:26:07.785573+0000","last_unstale":"2024-05-07T02:27:42.734914+0000","last_undegraded":"2024-05-07T02:27:42.734914+0000","last_fullsized":"2024-05-07T02:27:42.734914+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:33:24.042227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.623876+0000","last_change":"2024-05-07T02:26:11.858732+0000","last_active":"2024-05-07T02:27:32.623876+0000","last_peered":"2024-05-07T02:27:32.623876+0000","last_clean":"2024-05-07T02:27:32.623876+0000","last_became_active":"2024-05-07T02:26:11.858270+0000","last_became_peered":"2024-05-07T02:26:11.858270+0000","last_unstale":"2024-05-07T02:27:32.623876+0000","last_undegraded":"2024-05-07T02:27:32.623876+0000","last_fullsized":"2024-05-07T02:27:32.623876+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:03:53.662296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.582877+0000","last_change":"2024-05-07T02:26:25.559025+0000","last_active":"2024-05-07T02:27:32.582877+0000","last_peered":"2024-05-07T02:27:32.582877+0000","last_clean":"2024-05-07T02:27:32.582877+0000","last_became_active":"2024-05-07T02:26:25.558502+0000","last_became_peered":"2024-05-07T02:26:25.558502+0000","last_unstale":"2024-05-07T02:27:32.582877+0000","last_undegraded":"2024-05-07T02:27:32.582877+0000","last_fullsized":"2024-05-07T02:27:32.582877+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:26:59.846341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":87,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:28:22.818890+0000","last_change":"2024-05-07T02:26:07.783386+0000","last_active":"2024-05-07T02:28:22.818890+0000","last_peered":"2024-05-07T02:28:22.818890+0000","last_clean":"2024-05-07T02:28:22.818890+0000","last_became_active":"2024-05-07T02:26:07.782823+0000","last_became_peered":"2024-05-07T02:26:07.782823+0000","last_unstale":"2024-05-07T02:28:22.818890+0000","last_undegraded":"2024-05-07T02:28:22.818890+0000","last_fullsized":"2024-05-07T02:28:22.818890+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:40:29.350809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":42,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.577398+0000","last_change":"2024-05-07T02:26:11.877426+0000","last_active":"2024-05-07T02:27:32.577398+0000","last_peered":"2024-05-07T02:27:32.577398+0000","last_clean":"2024-05-07T02:27:32.577398+0000","last_became_active":"2024-05-07T02:26:11.876976+0000","last_became_peered":"2024-05-07T02:26:11.876976+0000","last_unstale":"2024-05-07T02:27:32.577398+0000","last_undegraded":"2024-05-07T02:27:32.577398+0000","last_fullsized":"2024-05-07T02:27:32.577398+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:18:56.077064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.625394+0000","last_change":"2024-05-07T02:27:32.625394+0000","last_active":"2024-05-07T02:27:32.625394+0000","last_peered":"2024-05-07T02:27:32.625394+0000","last_clean":"2024-05-07T02:27:32.625394+0000","last_became_active":"2024-05-07T02:27:32.624826+0000","last_became_peered":"2024-05-07T02:27:32.624826+0000","last_unstale":"2024-05-07T02:27:32.625394+0000","last_undegraded":"2024-05-07T02:27:32.625394+0000","last_fullsized":"2024-05-07T02:27:32.625394+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:16:49.694003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":47,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:41.732327+0000","last_change":"2024-05-07T02:26:07.786388+0000","last_active":"2024-05-07T02:27:41.732327+0000","last_peered":"2024-05-07T02:27:41.732327+0000","last_clean":"2024-05-07T02:27:41.732327+0000","last_became_active":"2024-05-07T02:26:07.784130+0000","last_became_peered":"2024-05-07T02:26:07.784130+0000","last_unstale":"2024-05-07T02:27:41.732327+0000","last_undegraded":"2024-05-07T02:27:41.732327+0000","last_fullsized":"2024-05-07T02:27:41.732327+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:44:55.229815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.623681+0000","last_change":"2024-05-07T02:26:11.858798+0000","last_active":"2024-05-07T02:27:32.623681+0000","last_peered":"2024-05-07T02:27:32.623681+0000","last_clean":"2024-05-07T02:27:32.623681+0000","last_became_active":"2024-05-07T02:26:11.858375+0000","last_became_peered":"2024-05-07T02:26:11.858375+0000","last_unstale":"2024-05-07T02:27:32.623681+0000","last_undegraded":"2024-05-07T02:27:32.623681+0000","last_fullsized":"2024-05-07T02:27:32.623681+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:42:24.604727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.579909+0000","last_change":"2024-05-07T02:26:25.584396+0000","last_active":"2024-05-07T02:27:32.579909+0000","last_peered":"2024-05-07T02:27:32.579909+0000","last_clean":"2024-05-07T02:27:32.579909+0000","last_became_active":"2024-05-07T02:26:25.583917+0000","last_became_peered":"2024-05-07T02:26:25.583917+0000","last_unstale":"2024-05-07T02:27:32.579909+0000","last_undegraded":"2024-05-07T02:27:32.579909+0000","last_fullsized":"2024-05-07T02:27:32.579909+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:56:34.495133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:40.720542+0000","last_change":"2024-05-07T02:26:07.795300+0000","last_active":"2024-05-07T02:27:40.720542+0000","last_peered":"2024-05-07T02:27:40.720542+0000","last_clean":"2024-05-07T02:27:40.720542+0000","last_became_active":"2024-05-07T02:26:07.794633+0000","last_became_peered":"2024-05-07T02:26:07.794633+0000","last_unstale":"2024-05-07T02:27:40.720542+0000","last_undegraded":"2024-05-07T02:27:40.720542+0000","last_fullsized":"2024-05-07T02:27:40.720542+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:36:44.252118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.578205+0000","last_change":"2024-05-07T02:26:11.877717+0000","last_active":"2024-05-07T02:27:32.578205+0000","last_peered":"2024-05-07T02:27:32.578205+0000","last_clean":"2024-05-07T02:27:32.578205+0000","last_became_active":"2024-05-07T02:26:11.876644+0000","last_became_peered":"2024-05-07T02:26:11.876644+0000","last_unstale":"2024-05-07T02:27:32.578205+0000","last_undegraded":"2024-05-07T02:27:32.578205+0000","last_fullsized":"2024-05-07T02:27:32.578205+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T05:20:36.176776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.583109+0000","last_change":"2024-05-07T02:26:25.572120+0000","last_active":"2024-05-07T02:27:32.583109+0000","last_peered":"2024-05-07T02:27:32.583109+0000","last_clean":"2024-05-07T02:27:32.583109+0000","last_became_active":"2024-05-07T02:26:25.571637+0000","last_became_peered":"2024-05-07T02:26:25.571637+0000","last_unstale":"2024-05-07T02:27:32.583109+0000","last_undegraded":"2024-05-07T02:27:32.583109+0000","last_fullsized":"2024-05-07T02:27:32.583109+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:03:54.574061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:42.734993+0000","last_change":"2024-05-07T02:26:07.784160+0000","last_active":"2024-05-07T02:27:42.734993+0000","last_peered":"2024-05-07T02:27:42.734993+0000","last_clean":"2024-05-07T02:27:42.734993+0000","last_became_active":"2024-05-07T02:26:07.783646+0000","last_became_peered":"2024-05-07T02:26:07.783646+0000","last_unstale":"2024-05-07T02:27:42.734993+0000","last_undegraded":"2024-05-07T02:27:42.734993+0000","last_fullsized":"2024-05-07T02:27:42.734993+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T14:25:37.906587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.585546+0000","last_change":"2024-05-07T02:26:11.874886+0000","last_active":"2024-05-07T02:27:32.585546+0000","last_peered":"2024-05-07T02:27:32.585546+0000","last_clean":"2024-05-07T02:27:32.585546+0000","last_became_active":"2024-05-07T02:26:11.874465+0000","last_became_peered":"2024-05-07T02:26:11.874465+0000","last_unstale":"2024-05-07T02:27:32.585546+0000","last_undegraded":"2024-05-07T02:27:32.585546+0000","last_fullsized":"2024-05-07T02:27:32.585546+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:57:19.530747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.577707+0000","last_change":"2024-05-07T02:26:25.558087+0000","last_active":"2024-05-07T02:27:32.577707+0000","last_peered":"2024-05-07T02:27:32.577707+0000","last_clean":"2024-05-07T02:27:32.577707+0000","last_became_active":"2024-05-07T02:26:25.557624+0000","last_became_peered":"2024-05-07T02:26:25.557624+0000","last_unstale":"2024-05-07T02:27:32.577707+0000","last_undegraded":"2024-05-07T02:27:32.577707+0000","last_fullsized":"2024-05-07T02:27:32.577707+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:03:20.104547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":128,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:28:25.823238+0000","last_change":"2024-05-07T02:26:07.788244+0000","last_active":"2024-05-07T02:28:25.823238+0000","last_peered":"2024-05-07T02:28:25.823238+0000","last_clean":"2024-05-07T02:28:25.823238+0000","last_became_active":"2024-05-07T02:26:07.787789+0000","last_became_peered":"2024-05-07T02:26:07.787789+0000","last_unstale":"2024-05-07T02:28:25.823238+0000","last_undegraded":"2024-05-07T02:28:25.823238+0000","last_fullsized":"2024-05-07T02:28:25.823238+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:12:22.943444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":84,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.624097+0000","last_change":"2024-05-07T02:26:11.879821+0000","last_active":"2024-05-07T02:27:32.624097+0000","last_peered":"2024-05-07T02:27:32.624097+0000","last_clean":"2024-05-07T02:27:32.624097+0000","last_became_active":"2024-05-07T02:26:11.879372+0000","last_became_peered":"2024-05-07T02:26:11.879372+0000","last_unstale":"2024-05-07T02:27:32.624097+0000","last_undegraded":"2024-05-07T02:27:32.624097+0000","last_fullsized":"2024-05-07T02:27:32.624097+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:01:03.754890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.585228+0000","last_change":"2024-05-07T02:26:25.576663+0000","last_active":"2024-05-07T02:27:32.585228+0000","last_peered":"2024-05-07T02:27:32.585228+0000","last_clean":"2024-05-07T02:27:32.585228+0000","last_became_active":"2024-05-07T02:26:25.575729+0000","last_became_peered":"2024-05-07T02:26:25.575729+0000","last_unstale":"2024-05-07T02:27:32.585228+0000","last_undegraded":"2024-05-07T02:27:32.585228+0000","last_fullsized":"2024-05-07T02:27:32.585228+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:03:48.061385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":50,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:42.734377+0000","last_change":"2024-05-07T02:27:32.625336+0000","last_active":"2024-05-07T02:27:42.734377+0000","last_peered":"2024-05-07T02:27:42.734377+0000","last_clean":"2024-05-07T02:27:42.734377+0000","last_became_active":"2024-05-07T02:27:32.624805+0000","last_became_peered":"2024-05-07T02:27:32.624805+0000","last_unstale":"2024-05-07T02:27:42.734377+0000","last_undegraded":"2024-05-07T02:27:42.734377+0000","last_fullsized":"2024-05-07T02:27:42.734377+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:10:04.928297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.626374+0000","last_change":"2024-05-07T02:27:32.626374+0000","last_active":"2024-05-07T02:27:32.626374+0000","last_peered":"2024-05-07T02:27:32.626374+0000","last_clean":"2024-05-07T02:27:32.626374+0000","last_became_active":"2024-05-07T02:27:32.625489+0000","last_became_peered":"2024-05-07T02:27:32.625489+0000","last_unstale":"2024-05-07T02:27:32.626374+0000","last_undegraded":"2024-05-07T02:27:32.626374+0000","last_fullsized":"2024-05-07T02:27:32.626374+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T05:56:16.626041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.577141+0000","last_change":"2024-05-07T02:26:25.580811+0000","last_active":"2024-05-07T02:27:32.577141+0000","last_peered":"2024-05-07T02:27:32.577141+0000","last_clean":"2024-05-07T02:27:32.577141+0000","last_became_active":"2024-05-07T02:26:25.579344+0000","last_became_peered":"2024-05-07T02:26:25.579344+0000","last_unstale":"2024-05-07T02:27:32.577141+0000","last_undegraded":"2024-05-07T02:27:32.577141+0000","last_fullsized":"2024-05-07T02:27:32.577141+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:25:36.186899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:41.730235+0000","last_change":"2024-05-07T02:26:08.000010+0000","last_active":"2024-05-07T02:27:41.730235+0000","last_peered":"2024-05-07T02:27:41.730235+0000","last_clean":"2024-05-07T02:27:41.730235+0000","last_became_active":"2024-05-07T02:26:07.999576+0000","last_became_peered":"2024-05-07T02:26:07.999576+0000","last_unstale":"2024-05-07T02:27:41.730235+0000","last_undegraded":"2024-05-07T02:27:41.730235+0000","last_fullsized":"2024-05-07T02:27:41.730235+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:23:38.644205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.623974+0000","last_change":"2024-05-07T02:26:11.867098+0000","last_active":"2024-05-07T02:27:32.623974+0000","last_peered":"2024-05-07T02:27:32.623974+0000","last_clean":"2024-05-07T02:27:32.623974+0000","last_became_active":"2024-05-07T02:26:11.866036+0000","last_became_peered":"2024-05-07T02:26:11.866036+0000","last_unstale":"2024-05-07T02:27:32.623974+0000","last_undegraded":"2024-05-07T02:27:32.623974+0000","last_fullsized":"2024-05-07T02:27:32.623974+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:47:22.412323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.628243+0000","last_change":"2024-05-07T02:27:32.628243+0000","last_active":"2024-05-07T02:27:32.628243+0000","last_peered":"2024-05-07T02:27:32.628243+0000","last_clean":"2024-05-07T02:27:32.628243+0000","last_became_active":"2024-05-07T02:27:32.627048+0000","last_became_peered":"2024-05-07T02:27:32.627048+0000","last_unstale":"2024-05-07T02:27:32.628243+0000","last_undegraded":"2024-05-07T02:27:32.628243+0000","last_fullsized":"2024-05-07T02:27:32.628243+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:33:49.802873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,10],"acting":[9,8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'37","reported_seq":171,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:28:25.832713+0000","last_change":"2024-05-07T02:26:07.786841+0000","last_active":"2024-05-07T02:28:25.832713+0000","last_peered":"2024-05-07T02:28:25.832713+0000","last_clean":"2024-05-07T02:28:25.832713+0000","last_became_active":"2024-05-07T02:26:07.783836+0000","last_became_peered":"2024-05-07T02:26:07.783836+0000","last_unstale":"2024-05-07T02:28:25.832713+0000","last_undegraded":"2024-05-07T02:28:25.832713+0000","last_fullsized":"2024-05-07T02:28:25.832713+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:28:53.390009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68230,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":94,"num_read_kb":115,"num_write":39,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.578206+0000","last_change":"2024-05-07T02:26:11.857292+0000","last_active":"2024-05-07T02:27:32.578206+0000","last_peered":"2024-05-07T02:27:32.578206+0000","last_clean":"2024-05-07T02:27:32.578206+0000","last_became_active":"2024-05-07T02:26:11.856899+0000","last_became_peered":"2024-05-07T02:26:11.856899+0000","last_unstale":"2024-05-07T02:27:32.578206+0000","last_undegraded":"2024-05-07T02:27:32.578206+0000","last_fullsized":"2024-05-07T02:27:32.578206+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:45:42.279848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.624410+0000","last_change":"2024-05-07T02:26:25.570714+0000","last_active":"2024-05-07T02:27:32.624410+0000","last_peered":"2024-05-07T02:27:32.624410+0000","last_clean":"2024-05-07T02:27:32.624410+0000","last_became_active":"2024-05-07T02:26:25.570163+0000","last_became_peered":"2024-05-07T02:26:25.570163+0000","last_unstale":"2024-05-07T02:27:32.624410+0000","last_undegraded":"2024-05-07T02:27:32.624410+0000","last_fullsized":"2024-05-07T02:27:32.624410+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:55:51.903968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":51,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:44.758130+0000","last_change":"2024-05-07T02:26:07.785904+0000","last_active":"2024-05-07T02:27:44.758130+0000","last_peered":"2024-05-07T02:27:44.758130+0000","last_clean":"2024-05-07T02:27:44.758130+0000","last_became_active":"2024-05-07T02:26:07.785347+0000","last_became_peered":"2024-05-07T02:26:07.785347+0000","last_unstale":"2024-05-07T02:27:44.758130+0000","last_undegraded":"2024-05-07T02:27:44.758130+0000","last_fullsized":"2024-05-07T02:27:44.758130+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:42:05.604266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.589792+0000","last_change":"2024-05-07T02:26:11.878511+0000","last_active":"2024-05-07T02:27:32.589792+0000","last_peered":"2024-05-07T02:27:32.589792+0000","last_clean":"2024-05-07T02:27:32.589792+0000","last_became_active":"2024-05-07T02:26:11.878166+0000","last_became_peered":"2024-05-07T02:26:11.878166+0000","last_unstale":"2024-05-07T02:27:32.589792+0000","last_undegraded":"2024-05-07T02:27:32.589792+0000","last_fullsized":"2024-05-07T02:27:32.589792+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T08:14:40.436244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.580269+0000","last_change":"2024-05-07T02:26:25.579387+0000","last_active":"2024-05-07T02:27:32.580269+0000","last_peered":"2024-05-07T02:27:32.580269+0000","last_clean":"2024-05-07T02:27:32.580269+0000","last_became_active":"2024-05-07T02:26:25.578757+0000","last_became_peered":"2024-05-07T02:26:25.578757+0000","last_unstale":"2024-05-07T02:27:32.580269+0000","last_undegraded":"2024-05-07T02:27:32.580269+0000","last_fullsized":"2024-05-07T02:27:32.580269+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:19:19.650941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:42.733435+0000","last_change":"2024-05-07T02:26:07.785154+0000","last_active":"2024-05-07T02:27:42.733435+0000","last_peered":"2024-05-07T02:27:42.733435+0000","last_clean":"2024-05-07T02:27:42.733435+0000","last_became_active":"2024-05-07T02:26:07.784589+0000","last_became_peered":"2024-05-07T02:26:07.784589+0000","last_unstale":"2024-05-07T02:27:42.733435+0000","last_undegraded":"2024-05-07T02:27:42.733435+0000","last_fullsized":"2024-05-07T02:27:42.733435+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:41:41.851182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.582412+0000","last_change":"2024-05-07T02:26:11.878162+0000","last_active":"2024-05-07T02:27:32.582412+0000","last_peered":"2024-05-07T02:27:32.582412+0000","last_clean":"2024-05-07T02:27:32.582412+0000","last_became_active":"2024-05-07T02:26:11.877770+0000","last_became_peered":"2024-05-07T02:26:11.877770+0000","last_unstale":"2024-05-07T02:27:32.582412+0000","last_undegraded":"2024-05-07T02:27:32.582412+0000","last_fullsized":"2024-05-07T02:27:32.582412+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:19:08.209398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.584993+0000","last_change":"2024-05-07T02:26:25.579465+0000","last_active":"2024-05-07T02:27:32.584993+0000","last_peered":"2024-05-07T02:27:32.584993+0000","last_clean":"2024-05-07T02:27:32.584993+0000","last_became_active":"2024-05-07T02:26:25.579003+0000","last_became_peered":"2024-05-07T02:26:25.579003+0000","last_unstale":"2024-05-07T02:27:32.584993+0000","last_undegraded":"2024-05-07T02:27:32.584993+0000","last_fullsized":"2024-05-07T02:27:32.584993+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:13:24.356366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":86,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:28:21.852612+0000","last_change":"2024-05-07T02:26:07.780827+0000","last_active":"2024-05-07T02:28:21.852612+0000","last_peered":"2024-05-07T02:28:21.852612+0000","last_clean":"2024-05-07T02:28:21.852612+0000","last_became_active":"2024-05-07T02:26:07.780353+0000","last_became_peered":"2024-05-07T02:26:07.780353+0000","last_unstale":"2024-05-07T02:28:21.852612+0000","last_undegraded":"2024-05-07T02:28:21.852612+0000","last_fullsized":"2024-05-07T02:28:21.852612+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:52:40.554135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":42,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.582289+0000","last_change":"2024-05-07T02:26:11.852516+0000","last_active":"2024-05-07T02:27:32.582289+0000","last_peered":"2024-05-07T02:27:32.582289+0000","last_clean":"2024-05-07T02:27:32.582289+0000","last_became_active":"2024-05-07T02:26:11.852112+0000","last_became_peered":"2024-05-07T02:26:11.852112+0000","last_unstale":"2024-05-07T02:27:32.582289+0000","last_undegraded":"2024-05-07T02:27:32.582289+0000","last_fullsized":"2024-05-07T02:27:32.582289+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:58:46.989250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.591465+0000","last_change":"2024-05-07T02:26:25.586726+0000","last_active":"2024-05-07T02:27:32.591465+0000","last_peered":"2024-05-07T02:27:32.591465+0000","last_clean":"2024-05-07T02:27:32.591465+0000","last_became_active":"2024-05-07T02:26:25.586267+0000","last_became_peered":"2024-05-07T02:26:25.586267+0000","last_unstale":"2024-05-07T02:27:32.591465+0000","last_undegraded":"2024-05-07T02:27:32.591465+0000","last_fullsized":"2024-05-07T02:27:32.591465+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:50:00.864463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:42.733944+0000","last_change":"2024-05-07T02:27:32.599511+0000","last_active":"2024-05-07T02:27:42.733944+0000","last_peered":"2024-05-07T02:27:42.733944+0000","last_clean":"2024-05-07T02:27:42.733944+0000","last_became_active":"2024-05-07T02:27:32.590197+0000","last_became_peered":"2024-05-07T02:27:32.590197+0000","last_unstale":"2024-05-07T02:27:42.733944+0000","last_undegraded":"2024-05-07T02:27:42.733944+0000","last_fullsized":"2024-05-07T02:27:42.733944+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:23:07.401251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.623382+0000","last_change":"2024-05-07T02:26:11.857671+0000","last_active":"2024-05-07T02:27:32.623382+0000","last_peered":"2024-05-07T02:27:32.623382+0000","last_clean":"2024-05-07T02:27:32.623382+0000","last_became_active":"2024-05-07T02:26:11.857293+0000","last_became_peered":"2024-05-07T02:26:11.857293+0000","last_unstale":"2024-05-07T02:27:32.623382+0000","last_undegraded":"2024-05-07T02:27:32.623382+0000","last_fullsized":"2024-05-07T02:27:32.623382+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:48:42.267887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.623285+0000","last_change":"2024-05-07T02:26:25.563274+0000","last_active":"2024-05-07T02:27:32.623285+0000","last_peered":"2024-05-07T02:27:32.623285+0000","last_clean":"2024-05-07T02:27:32.623285+0000","last_became_active":"2024-05-07T02:26:25.562868+0000","last_became_peered":"2024-05-07T02:26:25.562868+0000","last_unstale":"2024-05-07T02:27:32.623285+0000","last_undegraded":"2024-05-07T02:27:32.623285+0000","last_fullsized":"2024-05-07T02:27:32.623285+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T14:20:03.210034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:44.758310+0000","last_change":"2024-05-07T02:26:07.784888+0000","last_active":"2024-05-07T02:27:44.758310+0000","last_peered":"2024-05-07T02:27:44.758310+0000","last_clean":"2024-05-07T02:27:44.758310+0000","last_became_active":"2024-05-07T02:26:07.784493+0000","last_became_peered":"2024-05-07T02:26:07.784493+0000","last_unstale":"2024-05-07T02:27:44.758310+0000","last_undegraded":"2024-05-07T02:27:44.758310+0000","last_fullsized":"2024-05-07T02:27:44.758310+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:21:14.054786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.579802+0000","last_change":"2024-05-07T02:26:11.858887+0000","last_active":"2024-05-07T02:27:32.579802+0000","last_peered":"2024-05-07T02:27:32.579802+0000","last_clean":"2024-05-07T02:27:32.579802+0000","last_became_active":"2024-05-07T02:26:11.858504+0000","last_became_peered":"2024-05-07T02:26:11.858504+0000","last_unstale":"2024-05-07T02:27:32.579802+0000","last_undegraded":"2024-05-07T02:27:32.579802+0000","last_fullsized":"2024-05-07T02:27:32.579802+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:53:20.290039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.627310+0000","last_change":"2024-05-07T02:27:32.627310+0000","last_active":"2024-05-07T02:27:32.627310+0000","last_peered":"2024-05-07T02:27:32.627310+0000","last_clean":"2024-05-07T02:27:32.627310+0000","last_became_active":"2024-05-07T02:27:32.626794+0000","last_became_peered":"2024-05-07T02:27:32.626794+0000","last_unstale":"2024-05-07T02:27:32.627310+0000","last_undegraded":"2024-05-07T02:27:32.627310+0000","last_fullsized":"2024-05-07T02:27:32.627310+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:06:30.466291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:41.732005+0000","last_change":"2024-05-07T02:26:08.000604+0000","last_active":"2024-05-07T02:27:41.732005+0000","last_peered":"2024-05-07T02:27:41.732005+0000","last_clean":"2024-05-07T02:27:41.732005+0000","last_became_active":"2024-05-07T02:26:08.000107+0000","last_became_peered":"2024-05-07T02:26:08.000107+0000","last_unstale":"2024-05-07T02:27:41.732005+0000","last_undegraded":"2024-05-07T02:27:41.732005+0000","last_fullsized":"2024-05-07T02:27:41.732005+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:58:06.716456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.624563+0000","last_change":"2024-05-07T02:26:11.876555+0000","last_active":"2024-05-07T02:27:32.624563+0000","last_peered":"2024-05-07T02:27:32.624563+0000","last_clean":"2024-05-07T02:27:32.624563+0000","last_became_active":"2024-05-07T02:26:11.876109+0000","last_became_peered":"2024-05-07T02:26:11.876109+0000","last_unstale":"2024-05-07T02:27:32.624563+0000","last_undegraded":"2024-05-07T02:27:32.624563+0000","last_fullsized":"2024-05-07T02:27:32.624563+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:10:24.995786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.588850+0000","last_change":"2024-05-07T02:26:25.567145+0000","last_active":"2024-05-07T02:27:32.588850+0000","last_peered":"2024-05-07T02:27:32.588850+0000","last_clean":"2024-05-07T02:27:32.588850+0000","last_became_active":"2024-05-07T02:26:25.566767+0000","last_became_peered":"2024-05-07T02:26:25.566767+0000","last_unstale":"2024-05-07T02:27:32.588850+0000","last_undegraded":"2024-05-07T02:27:32.588850+0000","last_fullsized":"2024-05-07T02:27:32.588850+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T05:10:59.333367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'6","reported_seq":90,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:28:23.835669+0000","last_change":"2024-05-07T02:26:07.789566+0000","last_active":"2024-05-07T02:28:23.835669+0000","last_peered":"2024-05-07T02:28:23.835669+0000","last_clean":"2024-05-07T02:28:23.835669+0000","last_became_active":"2024-05-07T02:26:07.789106+0000","last_became_peered":"2024-05-07T02:26:07.789106+0000","last_unstale":"2024-05-07T02:28:23.835669+0000","last_undegraded":"2024-05-07T02:28:23.835669+0000","last_fullsized":"2024-05-07T02:28:23.835669+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:45:13.620183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.624308+0000","last_change":"2024-05-07T02:26:11.869080+0000","last_active":"2024-05-07T02:27:32.624308+0000","last_peered":"2024-05-07T02:27:32.624308+0000","last_clean":"2024-05-07T02:27:32.624308+0000","last_became_active":"2024-05-07T02:26:11.868702+0000","last_became_peered":"2024-05-07T02:26:11.868702+0000","last_unstale":"2024-05-07T02:27:32.624308+0000","last_undegraded":"2024-05-07T02:27:32.624308+0000","last_fullsized":"2024-05-07T02:27:32.624308+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:05:05.497071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.588995+0000","last_change":"2024-05-07T02:26:25.867393+0000","last_active":"2024-05-07T02:27:32.588995+0000","last_peered":"2024-05-07T02:27:32.588995+0000","last_clean":"2024-05-07T02:27:32.588995+0000","last_became_active":"2024-05-07T02:26:25.866956+0000","last_became_peered":"2024-05-07T02:26:25.866956+0000","last_unstale":"2024-05-07T02:27:32.588995+0000","last_undegraded":"2024-05-07T02:27:32.588995+0000","last_fullsized":"2024-05-07T02:27:32.588995+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:23:48.821668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.623481+0000","last_change":"2024-05-07T02:26:11.867507+0000","last_active":"2024-05-07T02:27:32.623481+0000","last_peered":"2024-05-07T02:27:32.623481+0000","last_clean":"2024-05-07T02:27:32.623481+0000","last_became_active":"2024-05-07T02:26:11.867121+0000","last_became_peered":"2024-05-07T02:26:11.867121+0000","last_unstale":"2024-05-07T02:27:32.623481+0000","last_undegraded":"2024-05-07T02:27:32.623481+0000","last_fullsized":"2024-05-07T02:27:32.623481+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:44:55.442725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":89,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:28:24.858207+0000","last_change":"2024-05-07T02:26:07.999784+0000","last_active":"2024-05-07T02:28:24.858207+0000","last_peered":"2024-05-07T02:28:24.858207+0000","last_clean":"2024-05-07T02:28:24.858207+0000","last_became_active":"2024-05-07T02:26:07.999339+0000","last_became_peered":"2024-05-07T02:26:07.999339+0000","last_unstale":"2024-05-07T02:28:24.858207+0000","last_undegraded":"2024-05-07T02:28:24.858207+0000","last_fullsized":"2024-05-07T02:28:24.858207+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:50:41.622486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":43,"num_read_kb":43,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.582491+0000","last_change":"2024-05-07T02:26:25.587032+0000","last_active":"2024-05-07T02:27:32.582491+0000","last_peered":"2024-05-07T02:27:32.582491+0000","last_clean":"2024-05-07T02:27:32.582491+0000","last_became_active":"2024-05-07T02:26:25.586530+0000","last_became_peered":"2024-05-07T02:26:25.586530+0000","last_unstale":"2024-05-07T02:27:32.582491+0000","last_undegraded":"2024-05-07T02:27:32.582491+0000","last_fullsized":"2024-05-07T02:27:32.582491+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:13:13.335832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.579901+0000","last_change":"2024-05-07T02:26:11.878593+0000","last_active":"2024-05-07T02:27:32.579901+0000","last_peered":"2024-05-07T02:27:32.579901+0000","last_clean":"2024-05-07T02:27:32.579901+0000","last_became_active":"2024-05-07T02:26:11.873286+0000","last_became_peered":"2024-05-07T02:26:11.873286+0000","last_unstale":"2024-05-07T02:27:32.579901+0000","last_undegraded":"2024-05-07T02:27:32.579901+0000","last_fullsized":"2024-05-07T02:27:32.579901+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:22:33.769539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":50,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:44.758541+0000","last_change":"2024-05-07T02:26:07.790087+0000","last_active":"2024-05-07T02:27:44.758541+0000","last_peered":"2024-05-07T02:27:44.758541+0000","last_clean":"2024-05-07T02:27:44.758541+0000","last_became_active":"2024-05-07T02:26:07.789592+0000","last_became_peered":"2024-05-07T02:26:07.789592+0000","last_unstale":"2024-05-07T02:27:44.758541+0000","last_undegraded":"2024-05-07T02:27:44.758541+0000","last_fullsized":"2024-05-07T02:27:44.758541+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:44:52.569055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":114,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.581868+0000","last_change":"2024-05-07T02:24:27.877820+0000","last_active":"2024-05-07T02:27:32.581868+0000","last_peered":"2024-05-07T02:27:32.581868+0000","last_clean":"2024-05-07T02:27:32.581868+0000","last_became_active":"2024-05-07T02:24:27.873221+0000","last_became_peered":"2024-05-07T02:24:27.873221+0000","last_unstale":"2024-05-07T02:27:32.581868+0000","last_undegraded":"2024-05-07T02:27:32.581868+0000","last_fullsized":"2024-05-07T02:27:32.581868+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:20:21.582322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:20:21.582322+0000","last_clean_scrub_stamp":"2024-05-07T02:20:21.582322+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T14:16:32.772438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.624144+0000","last_change":"2024-05-07T02:26:25.868394+0000","last_active":"2024-05-07T02:27:32.624144+0000","last_peered":"2024-05-07T02:27:32.624144+0000","last_clean":"2024-05-07T02:27:32.624144+0000","last_became_active":"2024-05-07T02:26:25.867583+0000","last_became_peered":"2024-05-07T02:26:25.867583+0000","last_unstale":"2024-05-07T02:27:32.624144+0000","last_undegraded":"2024-05-07T02:27:32.624144+0000","last_fullsized":"2024-05-07T02:27:32.624144+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:55:44.705608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.589643+0000","last_change":"2024-05-07T02:26:11.868371+0000","last_active":"2024-05-07T02:27:32.589643+0000","last_peered":"2024-05-07T02:27:32.589643+0000","last_clean":"2024-05-07T02:27:32.589643+0000","last_became_active":"2024-05-07T02:26:11.867981+0000","last_became_peered":"2024-05-07T02:26:11.867981+0000","last_unstale":"2024-05-07T02:27:32.589643+0000","last_undegraded":"2024-05-07T02:27:32.589643+0000","last_fullsized":"2024-05-07T02:27:32.589643+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:08:54.687327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:42.734542+0000","last_change":"2024-05-07T02:26:07.785344+0000","last_active":"2024-05-07T02:27:42.734542+0000","last_peered":"2024-05-07T02:27:42.734542+0000","last_clean":"2024-05-07T02:27:42.734542+0000","last_became_active":"2024-05-07T02:26:07.784707+0000","last_became_peered":"2024-05-07T02:26:07.784707+0000","last_unstale":"2024-05-07T02:27:42.734542+0000","last_undegraded":"2024-05-07T02:27:42.734542+0000","last_fullsized":"2024-05-07T02:27:42.734542+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T05:00:38.754313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.579438+0000","last_change":"2024-05-07T02:26:25.571989+0000","last_active":"2024-05-07T02:27:32.579438+0000","last_peered":"2024-05-07T02:27:32.579438+0000","last_clean":"2024-05-07T02:27:32.579438+0000","last_became_active":"2024-05-07T02:26:25.571533+0000","last_became_peered":"2024-05-07T02:26:25.571533+0000","last_unstale":"2024-05-07T02:27:32.579438+0000","last_undegraded":"2024-05-07T02:27:32.579438+0000","last_fullsized":"2024-05-07T02:27:32.579438+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:54:38.669278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:44.757141+0000","last_change":"2024-05-07T02:26:07.999636+0000","last_active":"2024-05-07T02:27:44.757141+0000","last_peered":"2024-05-07T02:27:44.757141+0000","last_clean":"2024-05-07T02:27:44.757141+0000","last_became_active":"2024-05-07T02:26:07.999100+0000","last_became_peered":"2024-05-07T02:26:07.999100+0000","last_unstale":"2024-05-07T02:27:44.757141+0000","last_undegraded":"2024-05-07T02:27:44.757141+0000","last_fullsized":"2024-05-07T02:27:44.757141+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:10:24.730988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.591156+0000","last_change":"2024-05-07T02:26:11.888279+0000","last_active":"2024-05-07T02:27:32.591156+0000","last_peered":"2024-05-07T02:27:32.591156+0000","last_clean":"2024-05-07T02:27:32.591156+0000","last_became_active":"2024-05-07T02:26:11.887480+0000","last_became_peered":"2024-05-07T02:26:11.887480+0000","last_unstale":"2024-05-07T02:27:32.591156+0000","last_undegraded":"2024-05-07T02:27:32.591156+0000","last_fullsized":"2024-05-07T02:27:32.591156+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:12:12.368368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.578302+0000","last_change":"2024-05-07T02:26:25.867444+0000","last_active":"2024-05-07T02:27:32.578302+0000","last_peered":"2024-05-07T02:27:32.578302+0000","last_clean":"2024-05-07T02:27:32.578302+0000","last_became_active":"2024-05-07T02:26:25.866392+0000","last_became_peered":"2024-05-07T02:26:25.866392+0000","last_unstale":"2024-05-07T02:27:32.578302+0000","last_undegraded":"2024-05-07T02:27:32.578302+0000","last_fullsized":"2024-05-07T02:27:32.578302+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:54:36.112427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'16","reported_seq":57,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:28:16.042816+0000","last_change":"2024-05-07T02:26:07.798091+0000","last_active":"2024-05-07T02:28:16.042816+0000","last_peered":"2024-05-07T02:28:16.042816+0000","last_clean":"2024-05-07T02:28:16.042816+0000","last_became_active":"2024-05-07T02:26:07.797716+0000","last_became_peered":"2024-05-07T02:26:07.797716+0000","last_unstale":"2024-05-07T02:28:16.042816+0000","last_undegraded":"2024-05-07T02:28:16.042816+0000","last_fullsized":"2024-05-07T02:28:16.042816+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:04:54.570916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.579600+0000","last_change":"2024-05-07T02:26:11.888638+0000","last_active":"2024-05-07T02:27:32.579600+0000","last_peered":"2024-05-07T02:27:32.579600+0000","last_clean":"2024-05-07T02:27:32.579600+0000","last_became_active":"2024-05-07T02:26:11.888269+0000","last_became_peered":"2024-05-07T02:26:11.888269+0000","last_unstale":"2024-05-07T02:27:32.579600+0000","last_undegraded":"2024-05-07T02:27:32.579600+0000","last_fullsized":"2024-05-07T02:27:32.579600+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:09:38.965362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.582165+0000","last_change":"2024-05-07T02:26:25.558975+0000","last_active":"2024-05-07T02:27:32.582165+0000","last_peered":"2024-05-07T02:27:32.582165+0000","last_clean":"2024-05-07T02:27:32.582165+0000","last_became_active":"2024-05-07T02:26:25.557947+0000","last_became_peered":"2024-05-07T02:26:25.557947+0000","last_unstale":"2024-05-07T02:27:32.582165+0000","last_undegraded":"2024-05-07T02:27:32.582165+0000","last_fullsized":"2024-05-07T02:27:32.582165+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:29:32.071625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":49,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:44.759365+0000","last_change":"2024-05-07T02:26:07.796643+0000","last_active":"2024-05-07T02:27:44.759365+0000","last_peered":"2024-05-07T02:27:44.759365+0000","last_clean":"2024-05-07T02:27:44.759365+0000","last_became_active":"2024-05-07T02:26:07.796264+0000","last_became_peered":"2024-05-07T02:26:07.796264+0000","last_unstale":"2024-05-07T02:27:44.759365+0000","last_undegraded":"2024-05-07T02:27:44.759365+0000","last_fullsized":"2024-05-07T02:27:44.759365+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:29:32.454816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.583448+0000","last_change":"2024-05-07T02:26:11.873670+0000","last_active":"2024-05-07T02:27:32.583448+0000","last_peered":"2024-05-07T02:27:32.583448+0000","last_clean":"2024-05-07T02:27:32.583448+0000","last_became_active":"2024-05-07T02:26:11.871219+0000","last_became_peered":"2024-05-07T02:26:11.871219+0000","last_unstale":"2024-05-07T02:27:32.583448+0000","last_undegraded":"2024-05-07T02:27:32.583448+0000","last_fullsized":"2024-05-07T02:27:32.583448+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:49:48.048127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.577488+0000","last_change":"2024-05-07T02:26:25.868011+0000","last_active":"2024-05-07T02:27:32.577488+0000","last_peered":"2024-05-07T02:27:32.577488+0000","last_clean":"2024-05-07T02:27:32.577488+0000","last_became_active":"2024-05-07T02:26:25.867465+0000","last_became_peered":"2024-05-07T02:26:25.867465+0000","last_unstale":"2024-05-07T02:27:32.577488+0000","last_undegraded":"2024-05-07T02:27:32.577488+0000","last_fullsized":"2024-05-07T02:27:32.577488+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T05:14:40.001874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:39.701951+0000","last_change":"2024-05-07T02:26:07.999233+0000","last_active":"2024-05-07T02:27:39.701951+0000","last_peered":"2024-05-07T02:27:39.701951+0000","last_clean":"2024-05-07T02:27:39.701951+0000","last_became_active":"2024-05-07T02:26:07.998920+0000","last_became_peered":"2024-05-07T02:26:07.998920+0000","last_unstale":"2024-05-07T02:27:39.701951+0000","last_undegraded":"2024-05-07T02:27:39.701951+0000","last_fullsized":"2024-05-07T02:27:39.701951+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:33:33.878984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.577379+0000","last_change":"2024-05-07T02:26:11.856832+0000","last_active":"2024-05-07T02:27:32.577379+0000","last_peered":"2024-05-07T02:27:32.577379+0000","last_clean":"2024-05-07T02:27:32.577379+0000","last_became_active":"2024-05-07T02:26:11.856424+0000","last_became_peered":"2024-05-07T02:26:11.856424+0000","last_unstale":"2024-05-07T02:27:32.577379+0000","last_undegraded":"2024-05-07T02:27:32.577379+0000","last_fullsized":"2024-05-07T02:27:32.577379+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:00:46.839542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.627025+0000","last_change":"2024-05-07T02:27:32.626086+0000","last_active":"2024-05-07T02:27:32.627025+0000","last_peered":"2024-05-07T02:27:32.627025+0000","last_clean":"2024-05-07T02:27:32.627025+0000","last_became_active":"2024-05-07T02:27:32.625468+0000","last_became_peered":"2024-05-07T02:27:32.625468+0000","last_unstale":"2024-05-07T02:27:32.627025+0000","last_undegraded":"2024-05-07T02:27:32.627025+0000","last_fullsized":"2024-05-07T02:27:32.627025+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:17:57.245648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":48,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:42.736370+0000","last_change":"2024-05-07T02:26:07.793793+0000","last_active":"2024-05-07T02:27:42.736370+0000","last_peered":"2024-05-07T02:27:42.736370+0000","last_clean":"2024-05-07T02:27:42.736370+0000","last_became_active":"2024-05-07T02:26:07.790291+0000","last_became_peered":"2024-05-07T02:26:07.790291+0000","last_unstale":"2024-05-07T02:27:42.736370+0000","last_undegraded":"2024-05-07T02:27:42.736370+0000","last_fullsized":"2024-05-07T02:27:42.736370+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:50:14.591968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.580783+0000","last_change":"2024-05-07T02:26:11.869184+0000","last_active":"2024-05-07T02:27:32.580783+0000","last_peered":"2024-05-07T02:27:32.580783+0000","last_clean":"2024-05-07T02:27:32.580783+0000","last_became_active":"2024-05-07T02:26:11.868773+0000","last_became_peered":"2024-05-07T02:26:11.868773+0000","last_unstale":"2024-05-07T02:27:32.580783+0000","last_undegraded":"2024-05-07T02:27:32.580783+0000","last_fullsized":"2024-05-07T02:27:32.580783+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:24:04.986359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.577452+0000","last_change":"2024-05-07T02:26:25.580095+0000","last_active":"2024-05-07T02:27:32.577452+0000","last_peered":"2024-05-07T02:27:32.577452+0000","last_clean":"2024-05-07T02:27:32.577452+0000","last_became_active":"2024-05-07T02:26:25.579615+0000","last_became_peered":"2024-05-07T02:26:25.579615+0000","last_unstale":"2024-05-07T02:27:32.577452+0000","last_undegraded":"2024-05-07T02:27:32.577452+0000","last_fullsized":"2024-05-07T02:27:32.577452+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:18:05.714552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:42.721128+0000","last_change":"2024-05-07T02:26:07.784709+0000","last_active":"2024-05-07T02:27:42.721128+0000","last_peered":"2024-05-07T02:27:42.721128+0000","last_clean":"2024-05-07T02:27:42.721128+0000","last_became_active":"2024-05-07T02:26:07.784282+0000","last_became_peered":"2024-05-07T02:26:07.784282+0000","last_unstale":"2024-05-07T02:27:42.721128+0000","last_undegraded":"2024-05-07T02:27:42.721128+0000","last_fullsized":"2024-05-07T02:27:42.721128+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:01:47.023496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.579363+0000","last_change":"2024-05-07T02:26:11.879152+0000","last_active":"2024-05-07T02:27:32.579363+0000","last_peered":"2024-05-07T02:27:32.579363+0000","last_clean":"2024-05-07T02:27:32.579363+0000","last_became_active":"2024-05-07T02:26:11.878571+0000","last_became_peered":"2024-05-07T02:26:11.878571+0000","last_unstale":"2024-05-07T02:27:32.579363+0000","last_undegraded":"2024-05-07T02:27:32.579363+0000","last_fullsized":"2024-05-07T02:27:32.579363+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:04:12.637807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.577504+0000","last_change":"2024-05-07T02:26:25.867199+0000","last_active":"2024-05-07T02:27:32.577504+0000","last_peered":"2024-05-07T02:27:32.577504+0000","last_clean":"2024-05-07T02:27:32.577504+0000","last_became_active":"2024-05-07T02:26:25.866624+0000","last_became_peered":"2024-05-07T02:26:25.866624+0000","last_unstale":"2024-05-07T02:27:32.577504+0000","last_undegraded":"2024-05-07T02:27:32.577504+0000","last_fullsized":"2024-05-07T02:27:32.577504+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:14:49.513313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:44.757961+0000","last_change":"2024-05-07T02:26:07.784155+0000","last_active":"2024-05-07T02:27:44.757961+0000","last_peered":"2024-05-07T02:27:44.757961+0000","last_clean":"2024-05-07T02:27:44.757961+0000","last_became_active":"2024-05-07T02:26:07.783579+0000","last_became_peered":"2024-05-07T02:26:07.783579+0000","last_unstale":"2024-05-07T02:27:44.757961+0000","last_undegraded":"2024-05-07T02:27:44.757961+0000","last_fullsized":"2024-05-07T02:27:44.757961+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T08:51:32.037959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.582478+0000","last_change":"2024-05-07T02:26:11.868454+0000","last_active":"2024-05-07T02:27:32.582478+0000","last_peered":"2024-05-07T02:27:32.582478+0000","last_clean":"2024-05-07T02:27:32.582478+0000","last_became_active":"2024-05-07T02:26:11.868117+0000","last_became_peered":"2024-05-07T02:26:11.868117+0000","last_unstale":"2024-05-07T02:27:32.582478+0000","last_undegraded":"2024-05-07T02:27:32.582478+0000","last_fullsized":"2024-05-07T02:27:32.582478+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:24:36.703684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.623728+0000","last_change":"2024-05-07T02:26:25.573834+0000","last_active":"2024-05-07T02:27:32.623728+0000","last_peered":"2024-05-07T02:27:32.623728+0000","last_clean":"2024-05-07T02:27:32.623728+0000","last_became_active":"2024-05-07T02:26:25.573372+0000","last_became_peered":"2024-05-07T02:26:25.573372+0000","last_unstale":"2024-05-07T02:27:32.623728+0000","last_undegraded":"2024-05-07T02:27:32.623728+0000","last_fullsized":"2024-05-07T02:27:32.623728+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:03:36.916915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":90,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:28:25.849344+0000","last_change":"2024-05-07T02:26:08.000245+0000","last_active":"2024-05-07T02:28:25.849344+0000","last_peered":"2024-05-07T02:28:25.849344+0000","last_clean":"2024-05-07T02:28:25.849344+0000","last_became_active":"2024-05-07T02:26:07.999587+0000","last_became_peered":"2024-05-07T02:26:07.999587+0000","last_unstale":"2024-05-07T02:28:25.849344+0000","last_undegraded":"2024-05-07T02:28:25.849344+0000","last_fullsized":"2024-05-07T02:28:25.849344+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:44:41.754799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":44,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.582645+0000","last_change":"2024-05-07T02:26:11.858308+0000","last_active":"2024-05-07T02:27:32.582645+0000","last_peered":"2024-05-07T02:27:32.582645+0000","last_clean":"2024-05-07T02:27:32.582645+0000","last_became_active":"2024-05-07T02:26:11.857855+0000","last_became_peered":"2024-05-07T02:26:11.857855+0000","last_unstale":"2024-05-07T02:27:32.582645+0000","last_undegraded":"2024-05-07T02:27:32.582645+0000","last_fullsized":"2024-05-07T02:27:32.582645+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:11:50.021178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.583699+0000","last_change":"2024-05-07T02:26:25.584090+0000","last_active":"2024-05-07T02:27:32.583699+0000","last_peered":"2024-05-07T02:27:32.583699+0000","last_clean":"2024-05-07T02:27:32.583699+0000","last_became_active":"2024-05-07T02:26:25.580108+0000","last_became_peered":"2024-05-07T02:26:25.580108+0000","last_unstale":"2024-05-07T02:27:32.583699+0000","last_undegraded":"2024-05-07T02:27:32.583699+0000","last_fullsized":"2024-05-07T02:27:32.583699+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:56:56.964047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:44.758242+0000","last_change":"2024-05-07T02:26:07.788188+0000","last_active":"2024-05-07T02:27:44.758242+0000","last_peered":"2024-05-07T02:27:44.758242+0000","last_clean":"2024-05-07T02:27:44.758242+0000","last_became_active":"2024-05-07T02:26:07.787826+0000","last_became_peered":"2024-05-07T02:26:07.787826+0000","last_unstale":"2024-05-07T02:27:44.758242+0000","last_undegraded":"2024-05-07T02:27:44.758242+0000","last_fullsized":"2024-05-07T02:27:44.758242+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:06:14.588079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.582250+0000","last_change":"2024-05-07T02:26:11.862925+0000","last_active":"2024-05-07T02:27:32.582250+0000","last_peered":"2024-05-07T02:27:32.582250+0000","last_clean":"2024-05-07T02:27:32.582250+0000","last_became_active":"2024-05-07T02:26:11.861332+0000","last_became_peered":"2024-05-07T02:26:11.861332+0000","last_unstale":"2024-05-07T02:27:32.582250+0000","last_undegraded":"2024-05-07T02:27:32.582250+0000","last_fullsized":"2024-05-07T02:27:32.582250+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:51:11.133504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.622917+0000","last_change":"2024-05-07T02:26:25.557151+0000","last_active":"2024-05-07T02:27:32.622917+0000","last_peered":"2024-05-07T02:27:32.622917+0000","last_clean":"2024-05-07T02:27:32.622917+0000","last_became_active":"2024-05-07T02:26:25.556615+0000","last_became_peered":"2024-05-07T02:26:25.556615+0000","last_unstale":"2024-05-07T02:27:32.622917+0000","last_undegraded":"2024-05-07T02:27:32.622917+0000","last_fullsized":"2024-05-07T02:27:32.622917+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:33:20.446543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":56,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:42.737360+0000","last_change":"2024-05-07T02:27:32.598094+0000","last_active":"2024-05-07T02:27:42.737360+0000","last_peered":"2024-05-07T02:27:42.737360+0000","last_clean":"2024-05-07T02:27:42.737360+0000","last_became_active":"2024-05-07T02:27:32.594099+0000","last_became_peered":"2024-05-07T02:27:32.594099+0000","last_unstale":"2024-05-07T02:27:42.737360+0000","last_undegraded":"2024-05-07T02:27:42.737360+0000","last_fullsized":"2024-05-07T02:27:42.737360+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:44:50.979927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.582788+0000","last_change":"2024-05-07T02:26:11.860507+0000","last_active":"2024-05-07T02:27:32.582788+0000","last_peered":"2024-05-07T02:27:32.582788+0000","last_clean":"2024-05-07T02:27:32.582788+0000","last_became_active":"2024-05-07T02:26:11.860116+0000","last_became_peered":"2024-05-07T02:26:11.860116+0000","last_unstale":"2024-05-07T02:27:32.582788+0000","last_undegraded":"2024-05-07T02:27:32.582788+0000","last_fullsized":"2024-05-07T02:27:32.582788+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T05:09:38.963040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.580920+0000","last_change":"2024-05-07T02:26:11.867932+0000","last_active":"2024-05-07T02:27:32.580920+0000","last_peered":"2024-05-07T02:27:32.580920+0000","last_clean":"2024-05-07T02:27:32.580920+0000","last_became_active":"2024-05-07T02:26:11.867537+0000","last_became_peered":"2024-05-07T02:26:11.867537+0000","last_unstale":"2024-05-07T02:27:32.580920+0000","last_undegraded":"2024-05-07T02:27:32.580920+0000","last_fullsized":"2024-05-07T02:27:32.580920+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:31:35.843401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.582628+0000","last_change":"2024-05-07T02:26:11.866074+0000","last_active":"2024-05-07T02:27:32.582628+0000","last_peered":"2024-05-07T02:27:32.582628+0000","last_clean":"2024-05-07T02:27:32.582628+0000","last_became_active":"2024-05-07T02:26:11.865669+0000","last_became_peered":"2024-05-07T02:26:11.865669+0000","last_unstale":"2024-05-07T02:27:32.582628+0000","last_undegraded":"2024-05-07T02:27:32.582628+0000","last_fullsized":"2024-05-07T02:27:32.582628+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:08:51.323706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.582027+0000","last_change":"2024-05-07T02:26:11.876428+0000","last_active":"2024-05-07T02:27:32.582027+0000","last_peered":"2024-05-07T02:27:32.582027+0000","last_clean":"2024-05-07T02:27:32.582027+0000","last_became_active":"2024-05-07T02:26:11.876012+0000","last_became_peered":"2024-05-07T02:26:11.876012+0000","last_unstale":"2024-05-07T02:27:32.582027+0000","last_undegraded":"2024-05-07T02:27:32.582027+0000","last_fullsized":"2024-05-07T02:27:32.582027+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:24:58.088695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.587786+0000","last_change":"2024-05-07T02:26:11.880682+0000","last_active":"2024-05-07T02:27:32.587786+0000","last_peered":"2024-05-07T02:27:32.587786+0000","last_clean":"2024-05-07T02:27:32.587786+0000","last_became_active":"2024-05-07T02:26:11.880308+0000","last_became_peered":"2024-05-07T02:26:11.880308+0000","last_unstale":"2024-05-07T02:27:32.587786+0000","last_undegraded":"2024-05-07T02:27:32.587786+0000","last_fullsized":"2024-05-07T02:27:32.587786+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:52:45.083964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.589129+0000","last_change":"2024-05-07T02:26:11.876932+0000","last_active":"2024-05-07T02:27:32.589129+0000","last_peered":"2024-05-07T02:27:32.589129+0000","last_clean":"2024-05-07T02:27:32.589129+0000","last_became_active":"2024-05-07T02:26:11.876082+0000","last_became_peered":"2024-05-07T02:26:11.876082+0000","last_unstale":"2024-05-07T02:27:32.589129+0000","last_undegraded":"2024-05-07T02:27:32.589129+0000","last_fullsized":"2024-05-07T02:27:32.589129+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:45:21.435500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.580660+0000","last_change":"2024-05-07T02:26:11.861339+0000","last_active":"2024-05-07T02:27:32.580660+0000","last_peered":"2024-05-07T02:27:32.580660+0000","last_clean":"2024-05-07T02:27:32.580660+0000","last_became_active":"2024-05-07T02:26:11.860912+0000","last_became_peered":"2024-05-07T02:26:11.860912+0000","last_unstale":"2024-05-07T02:27:32.580660+0000","last_undegraded":"2024-05-07T02:27:32.580660+0000","last_fullsized":"2024-05-07T02:27:32.580660+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:12:57.944986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.583685+0000","last_change":"2024-05-07T02:26:11.867258+0000","last_active":"2024-05-07T02:27:32.583685+0000","last_peered":"2024-05-07T02:27:32.583685+0000","last_clean":"2024-05-07T02:27:32.583685+0000","last_became_active":"2024-05-07T02:26:11.866837+0000","last_became_peered":"2024-05-07T02:26:11.866837+0000","last_unstale":"2024-05-07T02:27:32.583685+0000","last_undegraded":"2024-05-07T02:27:32.583685+0000","last_fullsized":"2024-05-07T02:27:32.583685+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:07:58.589457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.579823+0000","last_change":"2024-05-07T02:26:11.888434+0000","last_active":"2024-05-07T02:27:32.579823+0000","last_peered":"2024-05-07T02:27:32.579823+0000","last_clean":"2024-05-07T02:27:32.579823+0000","last_became_active":"2024-05-07T02:26:11.887681+0000","last_became_peered":"2024-05-07T02:26:11.887681+0000","last_unstale":"2024-05-07T02:27:32.579823+0000","last_undegraded":"2024-05-07T02:27:32.579823+0000","last_fullsized":"2024-05-07T02:27:32.579823+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:38:58.896781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.623292+0000","last_change":"2024-05-07T02:26:11.875304+0000","last_active":"2024-05-07T02:27:32.623292+0000","last_peered":"2024-05-07T02:27:32.623292+0000","last_clean":"2024-05-07T02:27:32.623292+0000","last_became_active":"2024-05-07T02:26:11.874946+0000","last_became_peered":"2024-05-07T02:26:11.874946+0000","last_unstale":"2024-05-07T02:27:32.623292+0000","last_undegraded":"2024-05-07T02:27:32.623292+0000","last_fullsized":"2024-05-07T02:27:32.623292+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:56:04.093791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.578243+0000","last_change":"2024-05-07T02:26:11.879311+0000","last_active":"2024-05-07T02:27:32.578243+0000","last_peered":"2024-05-07T02:27:32.578243+0000","last_clean":"2024-05-07T02:27:32.578243+0000","last_became_active":"2024-05-07T02:26:11.877620+0000","last_became_peered":"2024-05-07T02:26:11.877620+0000","last_unstale":"2024-05-07T02:27:32.578243+0000","last_undegraded":"2024-05-07T02:27:32.578243+0000","last_fullsized":"2024-05-07T02:27:32.578243+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T08:41:40.525708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.623938+0000","last_change":"2024-05-07T02:26:11.876726+0000","last_active":"2024-05-07T02:27:32.623938+0000","last_peered":"2024-05-07T02:27:32.623938+0000","last_clean":"2024-05-07T02:27:32.623938+0000","last_became_active":"2024-05-07T02:26:11.876234+0000","last_became_peered":"2024-05-07T02:26:11.876234+0000","last_unstale":"2024-05-07T02:27:32.623938+0000","last_undegraded":"2024-05-07T02:27:32.623938+0000","last_fullsized":"2024-05-07T02:27:32.623938+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:26:34.422258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.590224+0000","last_change":"2024-05-07T02:26:11.879159+0000","last_active":"2024-05-07T02:27:32.590224+0000","last_peered":"2024-05-07T02:27:32.590224+0000","last_clean":"2024-05-07T02:27:32.590224+0000","last_became_active":"2024-05-07T02:26:11.878769+0000","last_became_peered":"2024-05-07T02:26:11.878769+0000","last_unstale":"2024-05-07T02:27:32.590224+0000","last_undegraded":"2024-05-07T02:27:32.590224+0000","last_fullsized":"2024-05-07T02:27:32.590224+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:05:27.089475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-07T02:27:32.590188+0000","last_change":"2024-05-07T02:27:32.590188+0000","last_active":"2024-05-07T02:27:32.590188+0000","last_peered":"2024-05-07T02:27:32.590188+0000","last_clean":"2024-05-07T02:27:32.590188+0000","last_became_active":"2024-05-07T02:27:32.589784+0000","last_became_peered":"2024-05-07T02:27:32.589784+0000","last_unstale":"2024-05-07T02:27:32.590188+0000","last_undegraded":"2024-05-07T02:27:32.590188+0000","last_fullsized":"2024-05-07T02:27:32.590188+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:32:40.488143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":75043,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":452,"num_read_kb":311,"num_write":251,"num_write_kb":148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":225129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":216,"ondisk_log_size":216,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710763,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":1007616,"data_stored":496962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906994,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32836,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712316,"statfs":{"total":95995035648,"available":95961411584,"internally_reserved":0,"allocated":1699840,"data_stored":1151896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070519,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":500876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266751,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1011712,"data_stored":497046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462980,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28596,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716556,"statfs":{"total":95995035648,"available":95965753344,"internally_reserved":0,"allocated":1617920,"data_stored":1093655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626505,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":492675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574952,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":561935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738477,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":493740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411427,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1093632,"data_stored":561764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182491,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1011712,"data_stored":491992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986263,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":1007616,"data_stored":496358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822736,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28604,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716548,"statfs":{"total":95995035648,"available":95965745152,"internally_reserved":0,"allocated":1626112,"data_stored":1088846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":69966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-07T02:28:28.635 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-07T02:28:28.635 INFO:tasks.cephadm:Setting up client nodes... 2024-05-07T02:28:28.636 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-07T02:28:28.757 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:28 smithi019 ceph-mon[29781]: pgmap v405: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-05-07T02:28:28.757 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:28 smithi019 ceph-mon[29781]: from='client.14634 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-07T02:28:28.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:28 smithi064 ceph-mon[34699]: pgmap v405: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-05-07T02:28:28.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:28 smithi064 ceph-mon[34699]: from='client.14634 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-07T02:28:28.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:28 smithi027 ceph-mon[34398]: pgmap v405: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-05-07T02:28:28.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:28 smithi027 ceph-mon[34398]: from='client.14634 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-07T02:28:29.159 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:28:30.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:30 smithi064 ceph-mon[34699]: pgmap v406: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 596 B/s wr, 10 op/s 2024-05-07T02:28:30.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:30 smithi019 ceph-mon[29781]: pgmap v406: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 596 B/s wr, 10 op/s 2024-05-07T02:28:30.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:30 smithi027 ceph-mon[34398]: pgmap v406: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 596 B/s wr, 10 op/s 2024-05-07T02:28:31.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.565+0000 7f2b02e00700 1 -- 172.21.15.19:0/1554005487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2afc0776e0 msgr2=0x7f2afc075bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:31.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.565+0000 7f2b02e00700 1 --2- 172.21.15.19:0/1554005487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2afc0776e0 0x7f2afc075bb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2af00098f0 tx=0x7f2af0009c00 comp rx=0 tx=0).stop 2024-05-07T02:28:31.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.566+0000 7f2b02e00700 1 -- 172.21.15.19:0/1554005487 shutdown_connections 2024-05-07T02:28:31.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.566+0000 7f2b02e00700 1 --2- 172.21.15.19:0/1554005487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2afc10c000 0x7f2afc10e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:31.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.566+0000 7f2b02e00700 1 --2- 172.21.15.19:0/1554005487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2afc0776e0 0x7f2afc075bb0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:31.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.566+0000 7f2b02e00700 1 --2- 172.21.15.19:0/1554005487 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2afc0760f0 0x7f2afc076580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:31.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.566+0000 7f2b02e00700 1 -- 172.21.15.19:0/1554005487 >> 172.21.15.19:0/1554005487 conn(0x7f2afc0ff7b0 msgr2=0x7f2afc101c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:28:31.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.566+0000 7f2b02e00700 1 -- 172.21.15.19:0/1554005487 shutdown_connections 2024-05-07T02:28:31.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.566+0000 7f2b02e00700 1 -- 172.21.15.19:0/1554005487 wait complete. 2024-05-07T02:28:31.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.567+0000 7f2b02e00700 1 Processor -- start 2024-05-07T02:28:31.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.568+0000 7f2b02e00700 1 -- start start 2024-05-07T02:28:31.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.568+0000 7f2b02e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2afc0760f0 0x7f2afc19c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:31.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.568+0000 7f2b02e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2afc0776e0 0x7f2afc19cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:31.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.568+0000 7f2b02e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2afc10c000 0x7f2afc1a3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:31.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.568+0000 7f2b02e00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc111dd0 con 0x7f2afc10c000 2024-05-07T02:28:31.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.568+0000 7f2b02e00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc111c50 con 0x7f2afc0760f0 2024-05-07T02:28:31.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.568+0000 7f2b02e00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc111f50 con 0x7f2afc0776e0 2024-05-07T02:28:31.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.569+0000 7f2b01a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2afc0760f0 0x7f2afc19c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:31.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.569+0000 7f2b01000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2afc0776e0 0x7f2afc19cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:31.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.569+0000 7f2b01a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2afc0760f0 0x7f2afc19c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:48402/0 (socket says 172.21.15.19:48402) 2024-05-07T02:28:31.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.569+0000 7f2b01a00700 1 -- 172.21.15.19:0/4090248272 learned_addr learned my addr 172.21.15.19:0/4090248272 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:28:31.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.569+0000 7f2b02400700 1 --2- 172.21.15.19:0/4090248272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2afc10c000 0x7f2afc1a3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:31.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.569+0000 7f2b01a00700 1 -- 172.21.15.19:0/4090248272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2afc0776e0 msgr2=0x7f2afc19cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:31.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.569+0000 7f2b01a00700 1 --2- 172.21.15.19:0/4090248272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2afc0776e0 0x7f2afc19cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:31.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.569+0000 7f2b01a00700 1 -- 172.21.15.19:0/4090248272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2afc10c000 msgr2=0x7f2afc1a3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:31.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.569+0000 7f2b01a00700 1 --2- 172.21.15.19:0/4090248272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2afc10c000 0x7f2afc1a3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:31.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.569+0000 7f2b01a00700 1 -- 172.21.15.19:0/4090248272 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2afc1a4530 con 0x7f2afc0760f0 2024-05-07T02:28:31.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.569+0000 7f2b01a00700 1 --2- 172.21.15.19:0/4090248272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2afc0760f0 0x7f2afc19c9d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2af000fc20 tx=0x7f2af0005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:28:31.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.570+0000 7f2afa000700 1 -- 172.21.15.19:0/4090248272 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af0005460 con 0x7f2afc0760f0 2024-05-07T02:28:31.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.570+0000 7f2afa000700 1 -- 172.21.15.19:0/4090248272 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2af0005600 con 0x7f2afc0760f0 2024-05-07T02:28:31.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.570+0000 7f2b02e00700 1 -- 172.21.15.19:0/4090248272 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2afc1a47c0 con 0x7f2afc0760f0 2024-05-07T02:28:31.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.570+0000 7f2afa000700 1 -- 172.21.15.19:0/4090248272 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af0003a80 con 0x7f2afc0760f0 2024-05-07T02:28:31.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.570+0000 7f2b02e00700 1 -- 172.21.15.19:0/4090248272 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2afc1a4c10 con 0x7f2afc0760f0 2024-05-07T02:28:31.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.571+0000 7f2b02e00700 1 -- 172.21.15.19:0/4090248272 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2afc0512c0 con 0x7f2afc0760f0 2024-05-07T02:28:31.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.574+0000 7f2afa000700 1 -- 172.21.15.19:0/4090248272 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f2af0003c20 con 0x7f2afc0760f0 2024-05-07T02:28:31.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.575+0000 7f2afa000700 1 --2- 172.21.15.19:0/4090248272 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f2ae40b2910 0x7f2ae40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:31.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.576+0000 7f2afa000700 1 -- 172.21.15.19:0/4090248272 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f2af00e39f0 con 0x7f2afc0760f0 2024-05-07T02:28:31.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.576+0000 7f2b01000700 1 --2- 172.21.15.19:0/4090248272 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f2ae40b2910 0x7f2ae40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:31.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.577+0000 7f2b01000700 1 --2- 172.21.15.19:0/4090248272 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f2ae40b2910 0x7f2ae40b4e20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2afc19da80 tx=0x7f2ae8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:28:31.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.578+0000 7f2afa000700 1 -- 172.21.15.19:0/4090248272 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2af0013030 con 0x7f2afc0760f0 2024-05-07T02:28:31.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.924+0000 7f2b02e00700 1 -- 172.21.15.19:0/4090248272 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f2afc0643d0 con 0x7f2afc0760f0 2024-05-07T02:28:31.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.936+0000 7f2afa000700 1 -- 172.21.15.19:0/4090248272 <== mon.2 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v33) ==== 170+0+59 (secure 0 0 0) 0x7f2af00ab280 con 0x7f2afc0760f0 2024-05-07T02:28:31.936 INFO:teuthology.orchestra.run.smithi019.stdout:[client.0] 2024-05-07T02:28:31.936 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDPkTlmnupLNxAAH1NE29cFPqMXIaEfJ22hDA== 2024-05-07T02:28:31.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.939+0000 7f2b02e00700 1 -- 172.21.15.19:0/4090248272 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f2ae40b2910 msgr2=0x7f2ae40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:31.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.939+0000 7f2b02e00700 1 --2- 172.21.15.19:0/4090248272 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f2ae40b2910 0x7f2ae40b4e20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2afc19da80 tx=0x7f2ae8009260 comp rx=0 tx=0).stop 2024-05-07T02:28:31.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.939+0000 7f2b02e00700 1 -- 172.21.15.19:0/4090248272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2afc0760f0 msgr2=0x7f2afc19c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:31.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.939+0000 7f2b02e00700 1 --2- 172.21.15.19:0/4090248272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2afc0760f0 0x7f2afc19c9d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2af000fc20 tx=0x7f2af0005270 comp rx=0 tx=0).stop 2024-05-07T02:28:31.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.940+0000 7f2b02e00700 1 -- 172.21.15.19:0/4090248272 shutdown_connections 2024-05-07T02:28:31.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.940+0000 7f2b02e00700 1 --2- 172.21.15.19:0/4090248272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2afc10c000 0x7f2afc1a3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:31.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.940+0000 7f2b02e00700 1 --2- 172.21.15.19:0/4090248272 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f2ae40b2910 0x7f2ae40b4e20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:31.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.940+0000 7f2b02e00700 1 --2- 172.21.15.19:0/4090248272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2afc0760f0 0x7f2afc19c9d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:31.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.940+0000 7f2b02e00700 1 --2- 172.21.15.19:0/4090248272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2afc0776e0 0x7f2afc19cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:31.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.940+0000 7f2b02e00700 1 -- 172.21.15.19:0/4090248272 >> 172.21.15.19:0/4090248272 conn(0x7f2afc0ff7b0 msgr2=0x7f2afc107f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:28:31.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.940+0000 7f2b02e00700 1 -- 172.21.15.19:0/4090248272 shutdown_connections 2024-05-07T02:28:31.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:31.940+0000 7f2b02e00700 1 -- 172.21.15.19:0/4090248272 wait complete. 2024-05-07T02:28:32.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:32 smithi019 ceph-mon[29781]: pgmap v407: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 596 B/s wr, 11 op/s 2024-05-07T02:28:32.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:32 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-05-07T02:28:32.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:32 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-05-07T02:28:32.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:32 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-05-07T02:28:32.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:32 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/4090248272' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-07T02:28:32.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:32 smithi019 ceph-mon[29781]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-07T02:28:32.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:32 smithi019 ceph-mon[29781]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-07T02:28:32.797 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-07T02:28:32.797 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-07T02:28:32.797 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-07T02:28:32.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:32 smithi064 ceph-mon[34699]: pgmap v407: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 596 B/s wr, 11 op/s 2024-05-07T02:28:32.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:32 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-05-07T02:28:32.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:32 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-05-07T02:28:32.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:32 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-05-07T02:28:32.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:32 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/4090248272' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-07T02:28:32.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:32 smithi064 ceph-mon[34699]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-07T02:28:32.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:32 smithi064 ceph-mon[34699]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-07T02:28:32.935 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-07T02:28:32.935 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-07T02:28:32.936 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph mgr dump --format=json 2024-05-07T02:28:32.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:32 smithi027 ceph-mon[34398]: pgmap v407: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 596 B/s wr, 11 op/s 2024-05-07T02:28:32.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:32 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-05-07T02:28:32.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:32 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-05-07T02:28:32.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:32 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-05-07T02:28:32.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:32 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/4090248272' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-07T02:28:32.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:32 smithi027 ceph-mon[34398]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-07T02:28:32.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:32 smithi027 ceph-mon[34398]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-07T02:28:33.454 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:28:33.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:33 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-05-07T02:28:33.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:33 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-05-07T02:28:33.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:33 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-05-07T02:28:33.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:33 smithi064 ceph-mon[34699]: osdmap e88: 12 total, 12 up, 12 in 2024-05-07T02:28:33.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:33 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-05-07T02:28:33.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:33 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-05-07T02:28:33.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:33 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-05-07T02:28:33.861 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:33 smithi019 ceph-mon[29781]: osdmap e88: 12 total, 12 up, 12 in 2024-05-07T02:28:33.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:33 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-05-07T02:28:33.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:33 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-05-07T02:28:33.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:33 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-05-07T02:28:33.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:33 smithi027 ceph-mon[34398]: osdmap e88: 12 total, 12 up, 12 in 2024-05-07T02:28:34.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:34 smithi019 ceph-mon[29781]: pgmap v409: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-07T02:28:34.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:34 smithi019 ceph-mon[29781]: osdmap e89: 12 total, 12 up, 12 in 2024-05-07T02:28:34.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:34 smithi027 ceph-mon[34398]: pgmap v409: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-07T02:28:34.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:34 smithi027 ceph-mon[34398]: osdmap e89: 12 total, 12 up, 12 in 2024-05-07T02:28:35.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:34 smithi064 ceph-mon[34699]: pgmap v409: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-07T02:28:35.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:34 smithi064 ceph-mon[34699]: osdmap e89: 12 total, 12 up, 12 in 2024-05-07T02:28:35.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:35 smithi019 ceph-mon[29781]: Health check failed: Reduced data availability: 2 pgs inactive (PG_AVAILABILITY) 2024-05-07T02:28:35.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:35.943+0000 7f8fe6a00700 1 -- 172.21.15.19:0/2117069759 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fe0106580 msgr2=0x7f8fe0106990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:35.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:35.943+0000 7f8fe6a00700 1 --2- 172.21.15.19:0/2117069759 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fe0106580 0x7f8fe0106990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8009990 tx=0x7f8fc8009ca0 comp rx=0 tx=0).stop 2024-05-07T02:28:35.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:35.944+0000 7f8fe6a00700 1 -- 172.21.15.19:0/2117069759 shutdown_connections 2024-05-07T02:28:35.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:35.944+0000 7f8fe6a00700 1 --2- 172.21.15.19:0/2117069759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fe0106ed0 0x7f8fe0113360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:35.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:35.944+0000 7f8fe6a00700 1 --2- 172.21.15.19:0/2117069759 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fe010c5c0 0x7f8fe010c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:35.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:35.944+0000 7f8fe6a00700 1 --2- 172.21.15.19:0/2117069759 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fe0106580 0x7f8fe0106990 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:35.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:35.944+0000 7f8fe6a00700 1 -- 172.21.15.19:0/2117069759 >> 172.21.15.19:0/2117069759 conn(0x7f8fe0102080 msgr2=0x7f8fe01044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:28:35.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:35.944+0000 7f8fe6a00700 1 -- 172.21.15.19:0/2117069759 shutdown_connections 2024-05-07T02:28:35.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:35.944+0000 7f8fe6a00700 1 -- 172.21.15.19:0/2117069759 wait complete. 2024-05-07T02:28:35.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:35.945+0000 7f8fe6a00700 1 Processor -- start 2024-05-07T02:28:35.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:35.945+0000 7f8fe6a00700 1 -- start start 2024-05-07T02:28:35.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:35.945+0000 7f8fe6a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fe0106580 0x7f8fe019d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:35.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:35.945+0000 7f8fe6a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fe0106ed0 0x7f8fe019d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:35.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:35.945+0000 7f8fe6a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fe010c5c0 0x7f8fe01a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:35.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:35.945+0000 7f8fe6a00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe01166f0 con 0x7f8fe0106580 2024-05-07T02:28:35.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:35.945+0000 7f8fe6a00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe0116570 con 0x7f8fe010c5c0 2024-05-07T02:28:35.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:35.945+0000 7f8fe6a00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe0116870 con 0x7f8fe0106ed0 2024-05-07T02:28:35.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:35.946+0000 7f8fdfe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fe0106580 0x7f8fe019d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:35.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:35.946+0000 7f8fe4a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fe010c5c0 0x7f8fe01a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:35.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:35.946+0000 7f8fe4a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fe010c5c0 0x7f8fe01a18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:48420/0 (socket says 172.21.15.19:48420) 2024-05-07T02:28:35.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:35.946+0000 7f8fe4a00700 1 -- 172.21.15.19:0/1723362344 learned_addr learned my addr 172.21.15.19:0/1723362344 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:28:35.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:35.946+0000 7f8fdf400700 1 --2- 172.21.15.19:0/1723362344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fe0106ed0 0x7f8fe019d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:35.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:35.946+0000 7f8fe4a00700 1 -- 172.21.15.19:0/1723362344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fe0106ed0 msgr2=0x7f8fe019d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:35.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:35.946+0000 7f8fe4a00700 1 --2- 172.21.15.19:0/1723362344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fe0106ed0 0x7f8fe019d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:35.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:35.946+0000 7f8fe4a00700 1 -- 172.21.15.19:0/1723362344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fe0106580 msgr2=0x7f8fe019d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:35.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:35.946+0000 7f8fe4a00700 1 --2- 172.21.15.19:0/1723362344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fe0106580 0x7f8fe019d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:35.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:35.946+0000 7f8fe4a00700 1 -- 172.21.15.19:0/1723362344 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe01a1f50 con 0x7f8fe010c5c0 2024-05-07T02:28:35.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:35.947+0000 7f8fe4a00700 1 --2- 172.21.15.19:0/1723362344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fe010c5c0 0x7f8fe01a18b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8fd800c370 tx=0x7f8fd800c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:28:35.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:35.947+0000 7f8fdcc00700 1 -- 172.21.15.19:0/1723362344 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd80092e0 con 0x7f8fe010c5c0 2024-05-07T02:28:35.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:35.947+0000 7f8fdcc00700 1 -- 172.21.15.19:0/1723362344 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fd8003a00 con 0x7f8fe010c5c0 2024-05-07T02:28:35.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:35.947+0000 7f8fdcc00700 1 -- 172.21.15.19:0/1723362344 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd80046f0 con 0x7f8fe010c5c0 2024-05-07T02:28:35.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:35.947+0000 7f8fe6a00700 1 -- 172.21.15.19:0/1723362344 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fe01a21e0 con 0x7f8fe010c5c0 2024-05-07T02:28:35.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:35.947+0000 7f8fe6a00700 1 -- 172.21.15.19:0/1723362344 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fe01a26f0 con 0x7f8fe010c5c0 2024-05-07T02:28:35.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:35.948+0000 7f8fe6a00700 1 -- 172.21.15.19:0/1723362344 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fe010e980 con 0x7f8fe010c5c0 2024-05-07T02:28:35.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:35.951+0000 7f8fdcc00700 1 -- 172.21.15.19:0/1723362344 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f8fd801a020 con 0x7f8fe010c5c0 2024-05-07T02:28:35.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:35.953+0000 7f8fdcc00700 1 --2- 172.21.15.19:0/1723362344 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f8fc40b2990 0x7f8fc40b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:35.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:35.953+0000 7f8fdcc00700 1 -- 172.21.15.19:0/1723362344 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f8fd80e2640 con 0x7f8fe010c5c0 2024-05-07T02:28:35.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:35.953+0000 7f8fdfe00700 1 --2- 172.21.15.19:0/1723362344 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f8fc40b2990 0x7f8fc40b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:35.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:35.954+0000 7f8fdfe00700 1 --2- 172.21.15.19:0/1723362344 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f8fc40b2990 0x7f8fc40b4ea0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8fe01079f0 tx=0x7f8fd0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:28:35.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:35.955+0000 7f8fdcc00700 1 -- 172.21.15.19:0/1723362344 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8fd8010040 con 0x7f8fe010c5c0 2024-05-07T02:28:35.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:35 smithi027 ceph-mon[34398]: Health check failed: Reduced data availability: 2 pgs inactive (PG_AVAILABILITY) 2024-05-07T02:28:36.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:35 smithi064 ceph-mon[34699]: Health check failed: Reduced data availability: 2 pgs inactive (PG_AVAILABILITY) 2024-05-07T02:28:36.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:36.258+0000 7f8fe6a00700 1 -- 172.21.15.19:0/1723362344 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f8fe010e360 con 0x7f8fe010c5c0 2024-05-07T02:28:36.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:36.264+0000 7f8fdcc00700 1 -- 172.21.15.19:0/1723362344 <== mon.2 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+288646 (secure 0 0 0) 0x7f8fd80b0e20 con 0x7f8fe010c5c0 2024-05-07T02:28:36.270 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:28:36.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:36.271+0000 7f8fe6a00700 1 -- 172.21.15.19:0/1723362344 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f8fc40b2990 msgr2=0x7f8fc40b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:36.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:36.271+0000 7f8fe6a00700 1 --2- 172.21.15.19:0/1723362344 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f8fc40b2990 0x7f8fc40b4ea0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8fe01079f0 tx=0x7f8fd0009260 comp rx=0 tx=0).stop 2024-05-07T02:28:36.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:36.271+0000 7f8fe6a00700 1 -- 172.21.15.19:0/1723362344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fe010c5c0 msgr2=0x7f8fe01a18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:36.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:36.271+0000 7f8fe6a00700 1 --2- 172.21.15.19:0/1723362344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fe010c5c0 0x7f8fe01a18b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8fd800c370 tx=0x7f8fd800c680 comp rx=0 tx=0).stop 2024-05-07T02:28:36.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:36.272+0000 7f8fe6a00700 1 -- 172.21.15.19:0/1723362344 shutdown_connections 2024-05-07T02:28:36.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:36.272+0000 7f8fe6a00700 1 --2- 172.21.15.19:0/1723362344 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f8fc40b2990 0x7f8fc40b4ea0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:36.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:36.272+0000 7f8fe6a00700 1 --2- 172.21.15.19:0/1723362344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fe010c5c0 0x7f8fe01a18b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:36.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:36.272+0000 7f8fe6a00700 1 --2- 172.21.15.19:0/1723362344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fe0106580 0x7f8fe019d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:36.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:36.272+0000 7f8fe6a00700 1 --2- 172.21.15.19:0/1723362344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8fe0106ed0 0x7f8fe019d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:36.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:36.272+0000 7f8fe6a00700 1 -- 172.21.15.19:0/1723362344 >> 172.21.15.19:0/1723362344 conn(0x7f8fe0102080 msgr2=0x7f8fe01048b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:28:36.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:36.272+0000 7f8fe6a00700 1 -- 172.21.15.19:0/1723362344 shutdown_connections 2024-05-07T02:28:36.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:36.272+0000 7f8fe6a00700 1 -- 172.21.15.19:0/1723362344 wait complete. 2024-05-07T02:28:36.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:36 smithi027 ceph-mon[34398]: pgmap v411: 129 pgs: 2 activating, 127 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-05-07T02:28:36.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:36 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/1723362344' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-07T02:28:37.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:36 smithi064 ceph-mon[34699]: pgmap v411: 129 pgs: 2 activating, 127 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-05-07T02:28:37.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:36 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/1723362344' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-07T02:28:37.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:36 smithi019 ceph-mon[29781]: pgmap v411: 129 pgs: 2 activating, 127 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-05-07T02:28:37.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:36 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/1723362344' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-07T02:28:37.300 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":2059564551},{"type":"v1","addr":"172.21.15.19:6801","nonce":2059564551}]},"active_addr":"172.21.15.19:6801/2059564551","active_change":"2024-05-07T02:17:31.253122+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.19:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":3466480009}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":2602127700}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":1821831886}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":686554066}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":3182889845}]}]} 2024-05-07T02:28:37.305 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-07T02:28:37.305 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-07T02:28:37.305 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-07T02:28:37.787 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:28:38.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:38 smithi027 ceph-mon[34398]: pgmap v412: 129 pgs: 2 activating, 127 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-05-07T02:28:38.985 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:38 smithi019 ceph-mon[29781]: pgmap v412: 129 pgs: 2 activating, 127 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-05-07T02:28:39.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:38 smithi064 ceph-mon[34699]: pgmap v412: 129 pgs: 2 activating, 127 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-05-07T02:28:39.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:39 smithi019 ceph-mon[29781]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive) 2024-05-07T02:28:39.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:39 smithi019 ceph-mon[29781]: Cluster is now healthy 2024-05-07T02:28:40.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:39 smithi064 ceph-mon[34699]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive) 2024-05-07T02:28:40.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:39 smithi064 ceph-mon[34699]: Cluster is now healthy 2024-05-07T02:28:40.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.147+0000 7f4c0fa00700 1 -- 172.21.15.19:0/1048277667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c08108f00 msgr2=0x7f4c08109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:40.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.147+0000 7f4c0fa00700 1 --2- 172.21.15.19:0/1048277667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c08108f00 0x7f4c08109310 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4bf40099c0 tx=0x7f4bf4009cd0 comp rx=0 tx=0).stop 2024-05-07T02:28:40.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.147+0000 7f4c0fa00700 1 -- 172.21.15.19:0/1048277667 shutdown_connections 2024-05-07T02:28:40.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.147+0000 7f4c0fa00700 1 --2- 172.21.15.19:0/1048277667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c08109850 0x7f4c081111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:40.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.147+0000 7f4c0fa00700 1 --2- 172.21.15.19:0/1048277667 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c081045d0 0x7f4c081049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:40.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.147+0000 7f4c0fa00700 1 --2- 172.21.15.19:0/1048277667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c08108f00 0x7f4c08109310 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:40.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.147+0000 7f4c0fa00700 1 -- 172.21.15.19:0/1048277667 >> 172.21.15.19:0/1048277667 conn(0x7f4c080ffea0 msgr2=0x7f4c081022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:28:40.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.147+0000 7f4c0fa00700 1 -- 172.21.15.19:0/1048277667 shutdown_connections 2024-05-07T02:28:40.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.147+0000 7f4c0fa00700 1 -- 172.21.15.19:0/1048277667 wait complete. 2024-05-07T02:28:40.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.149+0000 7f4c0fa00700 1 Processor -- start 2024-05-07T02:28:40.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.149+0000 7f4c0fa00700 1 -- start start 2024-05-07T02:28:40.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.149+0000 7f4c0fa00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c081045d0 0x7f4c08076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:40.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.149+0000 7f4c0fa00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c08108f00 0x7f4c08077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:40.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.149+0000 7f4c0fa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c08109850 0x7f4c08077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:40.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.149+0000 7f4c0fa00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4c081144b0 con 0x7f4c08108f00 2024-05-07T02:28:40.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.149+0000 7f4c0fa00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4c08114330 con 0x7f4c08109850 2024-05-07T02:28:40.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.149+0000 7f4c0fa00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4c08114630 con 0x7f4c081045d0 2024-05-07T02:28:40.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.150+0000 7f4c07e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c08108f00 0x7f4c08077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:40.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.150+0000 7f4c07e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c08108f00 0x7f4c08077110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50938/0 (socket says 172.21.15.19:50938) 2024-05-07T02:28:40.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.150+0000 7f4c07e00700 1 -- 172.21.15.19:0/3843482099 learned_addr learned my addr 172.21.15.19:0/3843482099 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:28:40.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.150+0000 7f4c0d000700 1 --2- 172.21.15.19:0/3843482099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c081045d0 0x7f4c08076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:40.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.150+0000 7f4c0da00700 1 --2- 172.21.15.19:0/3843482099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c08109850 0x7f4c08077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:40.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.150+0000 7f4c0d000700 1 -- 172.21.15.19:0/3843482099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c08109850 msgr2=0x7f4c08077650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:40.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.150+0000 7f4c0d000700 1 --2- 172.21.15.19:0/3843482099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c08109850 0x7f4c08077650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:40.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.150+0000 7f4c0d000700 1 -- 172.21.15.19:0/3843482099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c08108f00 msgr2=0x7f4c08077110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:40.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.150+0000 7f4c0d000700 1 --2- 172.21.15.19:0/3843482099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c08108f00 0x7f4c08077110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:40.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.150+0000 7f4c0d000700 1 -- 172.21.15.19:0/3843482099 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c081a9250 con 0x7f4c081045d0 2024-05-07T02:28:40.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.150+0000 7f4c07e00700 1 --2- 172.21.15.19:0/3843482099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c08108f00 0x7f4c08077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:28:40.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.151+0000 7f4c0d000700 1 --2- 172.21.15.19:0/3843482099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c081045d0 0x7f4c08076bd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc00e9f0 tx=0x7f4bfc00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:28:40.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.151+0000 7f4c05600700 1 -- 172.21.15.19:0/3843482099 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc00c9c0 con 0x7f4c081045d0 2024-05-07T02:28:40.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.151+0000 7f4c05600700 1 -- 172.21.15.19:0/3843482099 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bfc00cb60 con 0x7f4c081045d0 2024-05-07T02:28:40.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.151+0000 7f4c0fa00700 1 -- 172.21.15.19:0/3843482099 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c081a9540 con 0x7f4c081045d0 2024-05-07T02:28:40.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.151+0000 7f4c05600700 1 -- 172.21.15.19:0/3843482099 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc018680 con 0x7f4c081045d0 2024-05-07T02:28:40.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.151+0000 7f4c0fa00700 1 -- 172.21.15.19:0/3843482099 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c081a9a80 con 0x7f4c081045d0 2024-05-07T02:28:40.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.152+0000 7f4c0fa00700 1 -- 172.21.15.19:0/3843482099 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c080512c0 con 0x7f4c081045d0 2024-05-07T02:28:40.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.155+0000 7f4c05600700 1 -- 172.21.15.19:0/3843482099 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f4bfc018a30 con 0x7f4c081045d0 2024-05-07T02:28:40.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.156+0000 7f4c05600700 1 --2- 172.21.15.19:0/3843482099 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f4bf00b2910 0x7f4bf00b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:40.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.157+0000 7f4c05600700 1 -- 172.21.15.19:0/3843482099 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f4bfc0e2e80 con 0x7f4c081045d0 2024-05-07T02:28:40.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.157+0000 7f4c07e00700 1 --2- 172.21.15.19:0/3843482099 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f4bf00b2910 0x7f4bf00b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:40.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.158+0000 7f4c07e00700 1 --2- 172.21.15.19:0/3843482099 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f4bf00b2910 0x7f4bf00b4e20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4c08103590 tx=0x7f4bf401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:28:40.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.159+0000 7f4c05600700 1 -- 172.21.15.19:0/3843482099 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4bfc0aa630 con 0x7f4c081045d0 2024-05-07T02:28:40.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:39 smithi027 ceph-mon[34398]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive) 2024-05-07T02:28:40.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:39 smithi027 ceph-mon[34398]: Cluster is now healthy 2024-05-07T02:28:40.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.394+0000 7f4c0fa00700 1 -- 172.21.15.19:0/3843482099 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4c0802e5f0 con 0x7f4c081045d0 2024-05-07T02:28:40.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.396+0000 7f4c05600700 1 -- 172.21.15.19:0/3843482099 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24723 (secure 0 0 0) 0x7f4bfc0a9f30 con 0x7f4c081045d0 2024-05-07T02:28:40.397 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:28:40.397 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":89,"fsid":"9ff84c16-0c17-11ef-bc97-c7b262605968","created":"2024-05-07T02:16:19.507657+0000","modified":"2024-05-07T02:28:33.595951+0000","last_up_change":"2024-05-07T02:24:55.025124+0000","last_in_change":"2024-05-07T02:24:33.842422+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-07T02:20:21.439697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-07T02:26:06.178814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-07T02:26:10.705693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-07T02:26:23.534133+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b437cf22-ec8e-48f0-86a2-2413a0ab40dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6803","nonce":2390539683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6805","nonce":2390539683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6809","nonce":2390539683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6807","nonce":2390539683}]},"public_addr":"172.21.15.19:6803/2390539683","cluster_addr":"172.21.15.19:6805/2390539683","heartbeat_back_addr":"172.21.15.19:6809/2390539683","heartbeat_front_addr":"172.21.15.19:6807/2390539683","state":["exists","up"]},{"osd":1,"uuid":"dc753e16-634d-4c14-abd2-935281e1499e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6801","nonce":4292470615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6803","nonce":4292470615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6807","nonce":4292470615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6805","nonce":4292470615}]},"public_addr":"172.21.15.27:6801/4292470615","cluster_addr":"172.21.15.27:6803/4292470615","heartbeat_back_addr":"172.21.15.27:6807/4292470615","heartbeat_front_addr":"172.21.15.27:6805/4292470615","state":["exists","up"]},{"osd":2,"uuid":"40570071-646e-4e66-9a16-c55ab743ac86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6801","nonce":1180552939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6803","nonce":1180552939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6807","nonce":1180552939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6805","nonce":1180552939}]},"public_addr":"172.21.15.64:6801/1180552939","cluster_addr":"172.21.15.64:6803/1180552939","heartbeat_back_addr":"172.21.15.64:6807/1180552939","heartbeat_front_addr":"172.21.15.64:6805/1180552939","state":["exists","up"]},{"osd":3,"uuid":"ceaa4252-4dfa-427a-9b1a-76adf3b8e609","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6811","nonce":387118141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6813","nonce":387118141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6817","nonce":387118141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6815","nonce":387118141}]},"public_addr":"172.21.15.19:6811/387118141","cluster_addr":"172.21.15.19:6813/387118141","heartbeat_back_addr":"172.21.15.19:6817/387118141","heartbeat_front_addr":"172.21.15.19:6815/387118141","state":["exists","up"]},{"osd":4,"uuid":"5a6d1ebc-62ca-444e-8f29-1980418d0a8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6809","nonce":3662186068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6811","nonce":3662186068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6815","nonce":3662186068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6813","nonce":3662186068}]},"public_addr":"172.21.15.27:6809/3662186068","cluster_addr":"172.21.15.27:6811/3662186068","heartbeat_back_addr":"172.21.15.27:6815/3662186068","heartbeat_front_addr":"172.21.15.27:6813/3662186068","state":["exists","up"]},{"osd":5,"uuid":"60c207e9-55e8-4bed-87c3-f6a06a2a1fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6809","nonce":1264591849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6811","nonce":1264591849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6815","nonce":1264591849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6813","nonce":1264591849}]},"public_addr":"172.21.15.64:6809/1264591849","cluster_addr":"172.21.15.64:6811/1264591849","heartbeat_back_addr":"172.21.15.64:6815/1264591849","heartbeat_front_addr":"172.21.15.64:6813/1264591849","state":["exists","up"]},{"osd":6,"uuid":"f3e038e0-4026-45f1-a923-c07cbb307f64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6819","nonce":2158875543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6821","nonce":2158875543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6825","nonce":2158875543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6823","nonce":2158875543}]},"public_addr":"172.21.15.19:6819/2158875543","cluster_addr":"172.21.15.19:6821/2158875543","heartbeat_back_addr":"172.21.15.19:6825/2158875543","heartbeat_front_addr":"172.21.15.19:6823/2158875543","state":["exists","up"]},{"osd":7,"uuid":"d69e34d4-2028-43b8-8ada-ff4a9b579620","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6817","nonce":2756612091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6819","nonce":2756612091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6823","nonce":2756612091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6821","nonce":2756612091}]},"public_addr":"172.21.15.27:6817/2756612091","cluster_addr":"172.21.15.27:6819/2756612091","heartbeat_back_addr":"172.21.15.27:6823/2756612091","heartbeat_front_addr":"172.21.15.27:6821/2756612091","state":["exists","up"]},{"osd":8,"uuid":"6c1a4179-75b6-42ff-ac29-378801e6f8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6817","nonce":1987024836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6819","nonce":1987024836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6823","nonce":1987024836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6821","nonce":1987024836}]},"public_addr":"172.21.15.64:6817/1987024836","cluster_addr":"172.21.15.64:6819/1987024836","heartbeat_back_addr":"172.21.15.64:6823/1987024836","heartbeat_front_addr":"172.21.15.64:6821/1987024836","state":["exists","up"]},{"osd":9,"uuid":"9df7ff66-d39f-4709-8324-74cf9e0e052d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6827","nonce":4124921646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6829","nonce":4124921646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6833","nonce":4124921646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6831","nonce":4124921646}]},"public_addr":"172.21.15.19:6827/4124921646","cluster_addr":"172.21.15.19:6829/4124921646","heartbeat_back_addr":"172.21.15.19:6833/4124921646","heartbeat_front_addr":"172.21.15.19:6831/4124921646","state":["exists","up"]},{"osd":10,"uuid":"71eaa9a7-667d-4cb7-935c-42177d20a279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6825","nonce":1428931817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6827","nonce":1428931817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6831","nonce":1428931817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6829","nonce":1428931817}]},"public_addr":"172.21.15.27:6825/1428931817","cluster_addr":"172.21.15.27:6827/1428931817","heartbeat_back_addr":"172.21.15.27:6831/1428931817","heartbeat_front_addr":"172.21.15.27:6829/1428931817","state":["exists","up"]},{"osd":11,"uuid":"3330627f-ab5d-4e6e-83cf-fc8b74c54ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6825","nonce":4045070679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6827","nonce":4045070679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6831","nonce":4045070679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6829","nonce":4045070679}]},"public_addr":"172.21.15.64:6825/4045070679","cluster_addr":"172.21.15.64:6827/4045070679","heartbeat_back_addr":"172.21.15.64:6831/4045070679","heartbeat_front_addr":"172.21.15.64:6829/4045070679","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:19:27.902633+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:19:54.305685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:20:18.727053+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:20:53.579199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:21:21.058018+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:21:47.091494+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:22:23.269018+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:22:51.863628+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:23:18.537774+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:23:55.763993+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:24:24.956178+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:24:52.681201+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/1956516258":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/1816772181":"2024-05-08T02:17:31.252868+0000","172.21.15.19:6801/174304059":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/1514051750":"2024-05-08T02:16:54.732592+0000","172.21.15.19:6800/174304059":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/3930045871":"2024-05-08T02:16:54.732592+0000","172.21.15.19:6800/3614957309":"2024-05-08T02:16:54.732592+0000","172.21.15.19:0/1328365399":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/139593344":"2024-05-08T02:16:54.732592+0000","172.21.15.19:6801/3614957309":"2024-05-08T02:16:54.732592+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-07T02:28:40.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.400+0000 7f4c0fa00700 1 -- 172.21.15.19:0/3843482099 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f4bf00b2910 msgr2=0x7f4bf00b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:40.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.400+0000 7f4c0fa00700 1 --2- 172.21.15.19:0/3843482099 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f4bf00b2910 0x7f4bf00b4e20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4c08103590 tx=0x7f4bf401a040 comp rx=0 tx=0).stop 2024-05-07T02:28:40.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.400+0000 7f4c0fa00700 1 -- 172.21.15.19:0/3843482099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c081045d0 msgr2=0x7f4c08076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:40.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.400+0000 7f4c0fa00700 1 --2- 172.21.15.19:0/3843482099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c081045d0 0x7f4c08076bd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc00e9f0 tx=0x7f4bfc00ed00 comp rx=0 tx=0).stop 2024-05-07T02:28:40.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.401+0000 7f4c0fa00700 1 -- 172.21.15.19:0/3843482099 shutdown_connections 2024-05-07T02:28:40.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.401+0000 7f4c0fa00700 1 --2- 172.21.15.19:0/3843482099 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f4bf00b2910 0x7f4bf00b4e20 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:40.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.401+0000 7f4c0fa00700 1 --2- 172.21.15.19:0/3843482099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c08109850 0x7f4c08077650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:40.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.401+0000 7f4c0fa00700 1 --2- 172.21.15.19:0/3843482099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4c081045d0 0x7f4c08076bd0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:40.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.401+0000 7f4c0fa00700 1 --2- 172.21.15.19:0/3843482099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c08108f00 0x7f4c08077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:40.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.401+0000 7f4c0fa00700 1 -- 172.21.15.19:0/3843482099 >> 172.21.15.19:0/3843482099 conn(0x7f4c080ffea0 msgr2=0x7f4c0810a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:28:40.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.401+0000 7f4c0fa00700 1 -- 172.21.15.19:0/3843482099 shutdown_connections 2024-05-07T02:28:40.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:40.401+0000 7f4c0fa00700 1 -- 172.21.15.19:0/3843482099 wait complete. 2024-05-07T02:28:41.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:40 smithi064 ceph-mon[34699]: pgmap v413: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-05-07T02:28:41.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:40 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/3843482099' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-07T02:28:41.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:40 smithi019 ceph-mon[29781]: pgmap v413: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-05-07T02:28:41.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:40 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/3843482099' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-07T02:28:41.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:40 smithi027 ceph-mon[34398]: pgmap v413: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-05-07T02:28:41.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:40 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/3843482099' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-07T02:28:41.338 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-07T02:28:41.339 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-07T02:28:41.806 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:28:43.013 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:42 smithi019 ceph-mon[29781]: pgmap v414: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-05-07T02:28:43.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:42 smithi064 ceph-mon[34699]: pgmap v414: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-05-07T02:28:43.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:42 smithi027 ceph-mon[34398]: pgmap v414: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-05-07T02:28:44.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.249+0000 7f88ffe00700 1 -- 172.21.15.19:0/1980267059 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f890010a7c0 msgr2=0x7f890010abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:44.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.249+0000 7f88ffe00700 1 --2- 172.21.15.19:0/1980267059 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f890010a7c0 0x7f890010abd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f88ec009990 tx=0x7f88ec009ca0 comp rx=0 tx=0).stop 2024-05-07T02:28:44.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.249+0000 7f88ffe00700 1 -- 172.21.15.19:0/1980267059 shutdown_connections 2024-05-07T02:28:44.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.249+0000 7f88ffe00700 1 --2- 172.21.15.19:0/1980267059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f890010b110 0x7f8900117620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:44.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.249+0000 7f88ffe00700 1 --2- 172.21.15.19:0/1980267059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f890010f6c0 0x7f890010fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:44.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.249+0000 7f88ffe00700 1 --2- 172.21.15.19:0/1980267059 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f890010a7c0 0x7f890010abd0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:44.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.249+0000 7f88ffe00700 1 -- 172.21.15.19:0/1980267059 >> 172.21.15.19:0/1980267059 conn(0x7f8900076b00 msgr2=0x7f8900076f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:28:44.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.249+0000 7f88ffe00700 1 -- 172.21.15.19:0/1980267059 shutdown_connections 2024-05-07T02:28:44.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.249+0000 7f88ffe00700 1 -- 172.21.15.19:0/1980267059 wait complete. 2024-05-07T02:28:44.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.250+0000 7f88ffe00700 1 Processor -- start 2024-05-07T02:28:44.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.250+0000 7f88ffe00700 1 -- start start 2024-05-07T02:28:44.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.251+0000 7f88ffe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f890010a7c0 0x7f89001a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:44.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.251+0000 7f88ffe00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f890010b110 0x7f89001a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:44.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.251+0000 7f88ffe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f890010f6c0 0x7f89001a5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:44.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.251+0000 7f88ffe00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f890011a900 con 0x7f890010f6c0 2024-05-07T02:28:44.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.251+0000 7f88ffe00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f890011a780 con 0x7f890010a7c0 2024-05-07T02:28:44.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.251+0000 7f88ffe00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f890011aa80 con 0x7f890010b110 2024-05-07T02:28:44.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.252+0000 7f88ff400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f890010f6c0 0x7f89001a5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:44.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.252+0000 7f88ff400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f890010f6c0 0x7f89001a5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50952/0 (socket says 172.21.15.19:50952) 2024-05-07T02:28:44.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.252+0000 7f88ff400700 1 -- 172.21.15.19:0/3418477545 learned_addr learned my addr 172.21.15.19:0/3418477545 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:28:44.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.252+0000 7f88fe000700 1 --2- 172.21.15.19:0/3418477545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f890010b110 0x7f89001a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:44.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.252+0000 7f88fea00700 1 --2- 172.21.15.19:0/3418477545 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f890010a7c0 0x7f89001a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:44.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.252+0000 7f88fe000700 1 -- 172.21.15.19:0/3418477545 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f890010a7c0 msgr2=0x7f89001a12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:44.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.252+0000 7f88fe000700 1 --2- 172.21.15.19:0/3418477545 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f890010a7c0 0x7f89001a12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:44.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.252+0000 7f88fe000700 1 -- 172.21.15.19:0/3418477545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f890010f6c0 msgr2=0x7f89001a5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:44.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.252+0000 7f88fe000700 1 --2- 172.21.15.19:0/3418477545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f890010f6c0 0x7f89001a5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:44.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.252+0000 7f88fe000700 1 -- 172.21.15.19:0/3418477545 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89001a61a0 con 0x7f890010b110 2024-05-07T02:28:44.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.253+0000 7f88ff400700 1 --2- 172.21.15.19:0/3418477545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f890010f6c0 0x7f89001a5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:28:44.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.253+0000 7f88fe000700 1 --2- 172.21.15.19:0/3418477545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f890010b110 0x7f89001a1870 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f88ec0098e0 tx=0x7f88ec0057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:28:44.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.253+0000 7f88f7400700 1 -- 172.21.15.19:0/3418477545 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ec027070 con 0x7f890010b110 2024-05-07T02:28:44.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.253+0000 7f88f7400700 1 -- 172.21.15.19:0/3418477545 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88ec022470 con 0x7f890010b110 2024-05-07T02:28:44.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.253+0000 7f88ffe00700 1 -- 172.21.15.19:0/3418477545 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89001a6430 con 0x7f890010b110 2024-05-07T02:28:44.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.253+0000 7f88f7400700 1 -- 172.21.15.19:0/3418477545 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ec017650 con 0x7f890010b110 2024-05-07T02:28:44.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.254+0000 7f88ffe00700 1 -- 172.21.15.19:0/3418477545 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89001a6910 con 0x7f890010b110 2024-05-07T02:28:44.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.255+0000 7f88ffe00700 1 -- 172.21.15.19:0/3418477545 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89000512c0 con 0x7f890010b110 2024-05-07T02:28:44.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.258+0000 7f88f7400700 1 -- 172.21.15.19:0/3418477545 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f88ec02a070 con 0x7f890010b110 2024-05-07T02:28:44.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.259+0000 7f88f7400700 1 --2- 172.21.15.19:0/3418477545 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f88e40b2910 0x7f88e40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:44.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.259+0000 7f88f7400700 1 -- 172.21.15.19:0/3418477545 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f88ec0e37f0 con 0x7f890010b110 2024-05-07T02:28:44.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.259+0000 7f88fea00700 1 --2- 172.21.15.19:0/3418477545 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f88e40b2910 0x7f88e40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:44.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.260+0000 7f88fea00700 1 --2- 172.21.15.19:0/3418477545 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f88e40b2910 0x7f88e40b4e20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f88f0007b60 tx=0x7f88f00073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:28:44.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.261+0000 7f88f7400700 1 -- 172.21.15.19:0/3418477545 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f88ec013070 con 0x7f890010b110 2024-05-07T02:28:44.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.501+0000 7f88ffe00700 1 -- 172.21.15.19:0/3418477545 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f890002e5f0 con 0x7f890010b110 2024-05-07T02:28:44.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.503+0000 7f88f7400700 1 -- 172.21.15.19:0/3418477545 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24723 (secure 0 0 0) 0x7f88ec0ab030 con 0x7f890010b110 2024-05-07T02:28:44.505 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:28:44.505 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":89,"fsid":"9ff84c16-0c17-11ef-bc97-c7b262605968","created":"2024-05-07T02:16:19.507657+0000","modified":"2024-05-07T02:28:33.595951+0000","last_up_change":"2024-05-07T02:24:55.025124+0000","last_in_change":"2024-05-07T02:24:33.842422+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-07T02:20:21.439697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-07T02:26:06.178814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-07T02:26:10.705693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-07T02:26:23.534133+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b437cf22-ec8e-48f0-86a2-2413a0ab40dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6803","nonce":2390539683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6805","nonce":2390539683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6809","nonce":2390539683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6807","nonce":2390539683}]},"public_addr":"172.21.15.19:6803/2390539683","cluster_addr":"172.21.15.19:6805/2390539683","heartbeat_back_addr":"172.21.15.19:6809/2390539683","heartbeat_front_addr":"172.21.15.19:6807/2390539683","state":["exists","up"]},{"osd":1,"uuid":"dc753e16-634d-4c14-abd2-935281e1499e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6801","nonce":4292470615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6803","nonce":4292470615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6807","nonce":4292470615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6805","nonce":4292470615}]},"public_addr":"172.21.15.27:6801/4292470615","cluster_addr":"172.21.15.27:6803/4292470615","heartbeat_back_addr":"172.21.15.27:6807/4292470615","heartbeat_front_addr":"172.21.15.27:6805/4292470615","state":["exists","up"]},{"osd":2,"uuid":"40570071-646e-4e66-9a16-c55ab743ac86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6801","nonce":1180552939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6803","nonce":1180552939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6807","nonce":1180552939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6805","nonce":1180552939}]},"public_addr":"172.21.15.64:6801/1180552939","cluster_addr":"172.21.15.64:6803/1180552939","heartbeat_back_addr":"172.21.15.64:6807/1180552939","heartbeat_front_addr":"172.21.15.64:6805/1180552939","state":["exists","up"]},{"osd":3,"uuid":"ceaa4252-4dfa-427a-9b1a-76adf3b8e609","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6811","nonce":387118141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6813","nonce":387118141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6817","nonce":387118141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6815","nonce":387118141}]},"public_addr":"172.21.15.19:6811/387118141","cluster_addr":"172.21.15.19:6813/387118141","heartbeat_back_addr":"172.21.15.19:6817/387118141","heartbeat_front_addr":"172.21.15.19:6815/387118141","state":["exists","up"]},{"osd":4,"uuid":"5a6d1ebc-62ca-444e-8f29-1980418d0a8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6809","nonce":3662186068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6811","nonce":3662186068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6815","nonce":3662186068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6813","nonce":3662186068}]},"public_addr":"172.21.15.27:6809/3662186068","cluster_addr":"172.21.15.27:6811/3662186068","heartbeat_back_addr":"172.21.15.27:6815/3662186068","heartbeat_front_addr":"172.21.15.27:6813/3662186068","state":["exists","up"]},{"osd":5,"uuid":"60c207e9-55e8-4bed-87c3-f6a06a2a1fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6809","nonce":1264591849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6811","nonce":1264591849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6815","nonce":1264591849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6813","nonce":1264591849}]},"public_addr":"172.21.15.64:6809/1264591849","cluster_addr":"172.21.15.64:6811/1264591849","heartbeat_back_addr":"172.21.15.64:6815/1264591849","heartbeat_front_addr":"172.21.15.64:6813/1264591849","state":["exists","up"]},{"osd":6,"uuid":"f3e038e0-4026-45f1-a923-c07cbb307f64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6819","nonce":2158875543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6821","nonce":2158875543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6825","nonce":2158875543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6823","nonce":2158875543}]},"public_addr":"172.21.15.19:6819/2158875543","cluster_addr":"172.21.15.19:6821/2158875543","heartbeat_back_addr":"172.21.15.19:6825/2158875543","heartbeat_front_addr":"172.21.15.19:6823/2158875543","state":["exists","up"]},{"osd":7,"uuid":"d69e34d4-2028-43b8-8ada-ff4a9b579620","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6817","nonce":2756612091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6819","nonce":2756612091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6823","nonce":2756612091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6821","nonce":2756612091}]},"public_addr":"172.21.15.27:6817/2756612091","cluster_addr":"172.21.15.27:6819/2756612091","heartbeat_back_addr":"172.21.15.27:6823/2756612091","heartbeat_front_addr":"172.21.15.27:6821/2756612091","state":["exists","up"]},{"osd":8,"uuid":"6c1a4179-75b6-42ff-ac29-378801e6f8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6817","nonce":1987024836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6819","nonce":1987024836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6823","nonce":1987024836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6821","nonce":1987024836}]},"public_addr":"172.21.15.64:6817/1987024836","cluster_addr":"172.21.15.64:6819/1987024836","heartbeat_back_addr":"172.21.15.64:6823/1987024836","heartbeat_front_addr":"172.21.15.64:6821/1987024836","state":["exists","up"]},{"osd":9,"uuid":"9df7ff66-d39f-4709-8324-74cf9e0e052d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6827","nonce":4124921646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6829","nonce":4124921646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6833","nonce":4124921646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6831","nonce":4124921646}]},"public_addr":"172.21.15.19:6827/4124921646","cluster_addr":"172.21.15.19:6829/4124921646","heartbeat_back_addr":"172.21.15.19:6833/4124921646","heartbeat_front_addr":"172.21.15.19:6831/4124921646","state":["exists","up"]},{"osd":10,"uuid":"71eaa9a7-667d-4cb7-935c-42177d20a279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6825","nonce":1428931817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6827","nonce":1428931817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6831","nonce":1428931817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6829","nonce":1428931817}]},"public_addr":"172.21.15.27:6825/1428931817","cluster_addr":"172.21.15.27:6827/1428931817","heartbeat_back_addr":"172.21.15.27:6831/1428931817","heartbeat_front_addr":"172.21.15.27:6829/1428931817","state":["exists","up"]},{"osd":11,"uuid":"3330627f-ab5d-4e6e-83cf-fc8b74c54ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6825","nonce":4045070679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6827","nonce":4045070679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6831","nonce":4045070679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6829","nonce":4045070679}]},"public_addr":"172.21.15.64:6825/4045070679","cluster_addr":"172.21.15.64:6827/4045070679","heartbeat_back_addr":"172.21.15.64:6831/4045070679","heartbeat_front_addr":"172.21.15.64:6829/4045070679","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:19:27.902633+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:19:54.305685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:20:18.727053+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:20:53.579199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:21:21.058018+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:21:47.091494+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:22:23.269018+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:22:51.863628+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:23:18.537774+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:23:55.763993+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:24:24.956178+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:24:52.681201+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/1956516258":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/1816772181":"2024-05-08T02:17:31.252868+0000","172.21.15.19:6801/174304059":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/1514051750":"2024-05-08T02:16:54.732592+0000","172.21.15.19:6800/174304059":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/3930045871":"2024-05-08T02:16:54.732592+0000","172.21.15.19:6800/3614957309":"2024-05-08T02:16:54.732592+0000","172.21.15.19:0/1328365399":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/139593344":"2024-05-08T02:16:54.732592+0000","172.21.15.19:6801/3614957309":"2024-05-08T02:16:54.732592+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-07T02:28:44.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.506+0000 7f88ffe00700 1 -- 172.21.15.19:0/3418477545 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f88e40b2910 msgr2=0x7f88e40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:44.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.506+0000 7f88ffe00700 1 --2- 172.21.15.19:0/3418477545 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f88e40b2910 0x7f88e40b4e20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f88f0007b60 tx=0x7f88f00073d0 comp rx=0 tx=0).stop 2024-05-07T02:28:44.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.506+0000 7f88ffe00700 1 -- 172.21.15.19:0/3418477545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f890010b110 msgr2=0x7f89001a1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:44.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.506+0000 7f88ffe00700 1 --2- 172.21.15.19:0/3418477545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f890010b110 0x7f89001a1870 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f88ec0098e0 tx=0x7f88ec0057c0 comp rx=0 tx=0).stop 2024-05-07T02:28:44.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.506+0000 7f88ffe00700 1 -- 172.21.15.19:0/3418477545 shutdown_connections 2024-05-07T02:28:44.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.507+0000 7f88ffe00700 1 --2- 172.21.15.19:0/3418477545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f890010f6c0 0x7f89001a5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:44.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.507+0000 7f88ffe00700 1 --2- 172.21.15.19:0/3418477545 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f88e40b2910 0x7f88e40b4e20 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:44.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.507+0000 7f88ffe00700 1 --2- 172.21.15.19:0/3418477545 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f890010a7c0 0x7f89001a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:44.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.507+0000 7f88ffe00700 1 --2- 172.21.15.19:0/3418477545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f890010b110 0x7f89001a1870 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:44.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.507+0000 7f88ffe00700 1 -- 172.21.15.19:0/3418477545 >> 172.21.15.19:0/3418477545 conn(0x7f8900076b00 msgr2=0x7f8900108860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:28:44.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.507+0000 7f88ffe00700 1 -- 172.21.15.19:0/3418477545 shutdown_connections 2024-05-07T02:28:44.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:44.507+0000 7f88ffe00700 1 -- 172.21.15.19:0/3418477545 wait complete. 2024-05-07T02:28:44.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:44 smithi019 ceph-mon[29781]: pgmap v415: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-07T02:28:44.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:44 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/3418477545' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-07T02:28:45.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:44 smithi064 ceph-mon[34699]: pgmap v415: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-07T02:28:45.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:44 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/3418477545' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-07T02:28:45.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:44 smithi027 ceph-mon[34398]: pgmap v415: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-07T02:28:45.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:44 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/3418477545' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-07T02:28:45.503 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-07T02:28:45.503 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-07T02:28:45.504 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-07T02:28:45.504 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-07T02:28:45.505 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-07T02:28:45.505 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-07T02:28:45.505 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-07T02:28:45.506 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-07T02:28:45.506 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-07T02:28:45.507 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-07T02:28:45.507 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-07T02:28:45.507 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-07T02:28:46.523 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:28:46.542 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:28:46.548 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:28:46.672 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:28:47.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:46 smithi019 ceph-mon[29781]: pgmap v416: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-07T02:28:47.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:46 smithi064 ceph-mon[34699]: pgmap v416: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-07T02:28:47.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:46 smithi027 ceph-mon[34398]: pgmap v416: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-07T02:28:48.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:48 smithi019 ceph-mon[29781]: pgmap v417: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:28:49.055 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:28:49.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:48 smithi064 ceph-mon[34699]: pgmap v417: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:28:49.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:48 smithi027 ceph-mon[34398]: pgmap v417: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:28:50.114 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:28:50.115 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:28:50.121 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:28:50.121 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:28:50.122 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:28:50.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.672+0000 7ff94a800700 1 -- 172.21.15.19:0/3286200935 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff944107c10 msgr2=0x7ff944108020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:50.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.672+0000 7ff94a800700 1 --2- 172.21.15.19:0/3286200935 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff944107c10 0x7ff944108020 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff930009990 tx=0x7ff930009ca0 comp rx=0 tx=0).stop 2024-05-07T02:28:50.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.672+0000 7ff94a800700 1 -- 172.21.15.19:0/3286200935 shutdown_connections 2024-05-07T02:28:50.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.672+0000 7ff94a800700 1 --2- 172.21.15.19:0/3286200935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff944108560 0x7ff94410fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:50.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.672+0000 7ff94a800700 1 --2- 172.21.15.19:0/3286200935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9441032e0 0x7ff9441036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:50.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.672+0000 7ff94a800700 1 --2- 172.21.15.19:0/3286200935 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff944107c10 0x7ff944108020 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:50.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.672+0000 7ff94a800700 1 -- 172.21.15.19:0/3286200935 >> 172.21.15.19:0/3286200935 conn(0x7ff944078ac0 msgr2=0x7ff944078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:28:50.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.672+0000 7ff94a800700 1 -- 172.21.15.19:0/3286200935 shutdown_connections 2024-05-07T02:28:50.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.672+0000 7ff94a800700 1 -- 172.21.15.19:0/3286200935 wait complete. 2024-05-07T02:28:50.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.673+0000 7ff94a800700 1 Processor -- start 2024-05-07T02:28:50.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.674+0000 7ff94a800700 1 -- start start 2024-05-07T02:28:50.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.674+0000 7ff94a800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9441032e0 0x7ff94419ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:50.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.674+0000 7ff94a800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff944107c10 0x7ff94419bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:50.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.674+0000 7ff94a800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff944108560 0x7ff9441a6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:50.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.674+0000 7ff94a800700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff944113170 con 0x7ff944107c10 2024-05-07T02:28:50.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.674+0000 7ff94a800700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff944112ff0 con 0x7ff944108560 2024-05-07T02:28:50.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.674+0000 7ff94a800700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff9441132f0 con 0x7ff9441032e0 2024-05-07T02:28:50.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.675+0000 7ff948a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff944107c10 0x7ff94419bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:50.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.675+0000 7ff949e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff944108560 0x7ff9441a6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:50.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.675+0000 7ff948a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff944107c10 0x7ff94419bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54544/0 (socket says 172.21.15.19:54544) 2024-05-07T02:28:50.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.675+0000 7ff948a00700 1 -- 172.21.15.19:0/2130017771 learned_addr learned my addr 172.21.15.19:0/2130017771 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:28:50.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.675+0000 7ff949400700 1 --2- 172.21.15.19:0/2130017771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9441032e0 0x7ff94419ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:50.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.675+0000 7ff949e00700 1 -- 172.21.15.19:0/2130017771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9441032e0 msgr2=0x7ff94419ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:50.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.675+0000 7ff949e00700 1 --2- 172.21.15.19:0/2130017771 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9441032e0 0x7ff94419ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:50.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.675+0000 7ff949e00700 1 -- 172.21.15.19:0/2130017771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff944107c10 msgr2=0x7ff94419bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:50.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.675+0000 7ff949e00700 1 --2- 172.21.15.19:0/2130017771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff944107c10 0x7ff94419bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:50.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.675+0000 7ff949e00700 1 -- 172.21.15.19:0/2130017771 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff94419c880 con 0x7ff944108560 2024-05-07T02:28:50.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.675+0000 7ff949e00700 1 --2- 172.21.15.19:0/2130017771 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff944108560 0x7ff9441a6b10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff93c00efb0 tx=0x7ff93c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:28:50.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.676+0000 7ff942000700 1 -- 172.21.15.19:0/2130017771 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c003a00 con 0x7ff944108560 2024-05-07T02:28:50.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.676+0000 7ff942000700 1 -- 172.21.15.19:0/2130017771 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff93c00ce50 con 0x7ff944108560 2024-05-07T02:28:50.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.676+0000 7ff942000700 1 -- 172.21.15.19:0/2130017771 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c0047c0 con 0x7ff944108560 2024-05-07T02:28:50.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.676+0000 7ff94a800700 1 -- 172.21.15.19:0/2130017771 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9441a7170 con 0x7ff944108560 2024-05-07T02:28:50.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.676+0000 7ff94a800700 1 -- 172.21.15.19:0/2130017771 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9441a7630 con 0x7ff944108560 2024-05-07T02:28:50.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.677+0000 7ff94a800700 1 -- 172.21.15.19:0/2130017771 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff944063e90 con 0x7ff944108560 2024-05-07T02:28:50.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.680+0000 7ff942000700 1 -- 172.21.15.19:0/2130017771 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ff93c012050 con 0x7ff944108560 2024-05-07T02:28:50.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.681+0000 7ff942000700 1 --2- 172.21.15.19:0/2130017771 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7ff92c0b2910 0x7ff92c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:50.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.681+0000 7ff942000700 1 -- 172.21.15.19:0/2130017771 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7ff93c0e2520 con 0x7ff944108560 2024-05-07T02:28:50.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.682+0000 7ff949400700 1 --2- 172.21.15.19:0/2130017771 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7ff92c0b2910 0x7ff92c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:50.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.682+0000 7ff942000700 1 --2- 172.21.15.19:0/2130017771 >> [v2:172.21.15.19:6802/2390539683,v1:172.21.15.19:6803/2390539683] conn(0x7ff92c0bdc40 0x7ff92c0c0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:50.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.682+0000 7ff942000700 1 -- 172.21.15.19:0/2130017771 --> [v2:172.21.15.19:6802/2390539683,v1:172.21.15.19:6803/2390539683] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff93c01ae50 con 0x7ff92c0bdc40 2024-05-07T02:28:50.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.682+0000 7ff942000700 1 -- 172.21.15.19:0/2130017771 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7ff93c0e28d0 con 0x7ff944108560 2024-05-07T02:28:50.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.682+0000 7ff948a00700 1 --2- 172.21.15.19:0/2130017771 >> [v2:172.21.15.19:6802/2390539683,v1:172.21.15.19:6803/2390539683] conn(0x7ff92c0bdc40 0x7ff92c0c0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:50.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.683+0000 7ff949400700 1 --2- 172.21.15.19:0/2130017771 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7ff92c0b2910 0x7ff92c0b4e20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff938009c60 tx=0x7ff938009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:28:50.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.683+0000 7ff948a00700 1 --2- 172.21.15.19:0/2130017771 >> [v2:172.21.15.19:6802/2390539683,v1:172.21.15.19:6803/2390539683] conn(0x7ff92c0bdc40 0x7ff92c0c0130 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:28:50.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.684+0000 7ff942000700 1 -- 172.21.15.19:0/2130017771 <== osd.0 v2:172.21.15.19:6802/2390539683 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff93c01ae50 con 0x7ff92c0bdc40 2024-05-07T02:28:50.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.703+0000 7ff94a800700 1 -- 172.21.15.19:0/2130017771 --> [v2:172.21.15.19:6802/2390539683,v1:172.21.15.19:6803/2390539683] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff944064340 con 0x7ff92c0bdc40 2024-05-07T02:28:50.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.703+0000 7ff942000700 1 -- 172.21.15.19:0/2130017771 <== osd.0 v2:172.21.15.19:6802/2390539683 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff944064340 con 0x7ff92c0bdc40 2024-05-07T02:28:50.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.704+0000 7ff94a800700 1 -- 172.21.15.19:0/2130017771 >> [v2:172.21.15.19:6802/2390539683,v1:172.21.15.19:6803/2390539683] conn(0x7ff92c0bdc40 msgr2=0x7ff92c0c0130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:50.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.704+0000 7ff94a800700 1 --2- 172.21.15.19:0/2130017771 >> [v2:172.21.15.19:6802/2390539683,v1:172.21.15.19:6803/2390539683] conn(0x7ff92c0bdc40 0x7ff92c0c0130 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:50.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.704+0000 7ff94a800700 1 -- 172.21.15.19:0/2130017771 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7ff92c0b2910 msgr2=0x7ff92c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:50.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.704+0000 7ff94a800700 1 --2- 172.21.15.19:0/2130017771 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7ff92c0b2910 0x7ff92c0b4e20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff938009c60 tx=0x7ff938009260 comp rx=0 tx=0).stop 2024-05-07T02:28:50.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.704+0000 7ff94a800700 1 -- 172.21.15.19:0/2130017771 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff944108560 msgr2=0x7ff9441a6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:50.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.704+0000 7ff94a800700 1 --2- 172.21.15.19:0/2130017771 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff944108560 0x7ff9441a6b10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff93c00efb0 tx=0x7ff93c00c3a0 comp rx=0 tx=0).stop 2024-05-07T02:28:50.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.704+0000 7ff949e00700 1 -- 172.21.15.19:0/2130017771 reap_dead start 2024-05-07T02:28:50.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.704+0000 7ff94a800700 1 -- 172.21.15.19:0/2130017771 shutdown_connections 2024-05-07T02:28:50.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.704+0000 7ff94a800700 1 -- 172.21.15.19:0/2130017771 >> 172.21.15.19:0/2130017771 conn(0x7ff944078ac0 msgr2=0x7ff9441006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:28:50.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.704+0000 7ff94a800700 1 -- 172.21.15.19:0/2130017771 shutdown_connections 2024-05-07T02:28:50.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:50.705+0000 7ff94a800700 1 -- 172.21.15.19:0/2130017771 wait complete. 2024-05-07T02:28:50.819 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:50 smithi019 ceph-mon[29781]: pgmap v418: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:28:51.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:50 smithi064 ceph-mon[34699]: pgmap v418: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:28:51.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:50 smithi027 ceph-mon[34398]: pgmap v418: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:28:52.659 INFO:teuthology.orchestra.run.smithi019.stdout:34359738483 2024-05-07T02:28:52.659 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-07T02:28:52.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:52 smithi019 ceph-mon[29781]: pgmap v419: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:28:52.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.951+0000 7f5fb1c00700 1 -- 172.21.15.19:0/2061794796 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fac108740 msgr2=0x7f5fac1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:52.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.951+0000 7f5fb1c00700 1 --2- 172.21.15.19:0/2061794796 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fac108740 0x7f5fac1100a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4009fc0 tx=0x7f5fa400b330 comp rx=0 tx=0).stop 2024-05-07T02:28:52.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.952+0000 7f5fb1c00700 1 -- 172.21.15.19:0/2061794796 shutdown_connections 2024-05-07T02:28:52.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.952+0000 7f5fb1c00700 1 --2- 172.21.15.19:0/2061794796 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fac108740 0x7f5fac1100a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:52.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.952+0000 7f5fb1c00700 1 --2- 172.21.15.19:0/2061794796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fac1034c0 0x7f5fac1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:52.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.952+0000 7f5fb1c00700 1 --2- 172.21.15.19:0/2061794796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5fac107df0 0x7f5fac108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:52.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.952+0000 7f5fb1c00700 1 -- 172.21.15.19:0/2061794796 >> 172.21.15.19:0/2061794796 conn(0x7f5fac0fedc0 msgr2=0x7f5fac1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:28:52.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.952+0000 7f5fb1c00700 1 -- 172.21.15.19:0/2061794796 shutdown_connections 2024-05-07T02:28:52.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.952+0000 7f5fb1c00700 1 -- 172.21.15.19:0/2061794796 wait complete. 2024-05-07T02:28:52.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.953+0000 7f5fb1c00700 1 Processor -- start 2024-05-07T02:28:52.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.953+0000 7f5fb1c00700 1 -- start start 2024-05-07T02:28:52.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.954+0000 7f5fb1c00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fac1034c0 0x7f5fac19d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:52.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.954+0000 7f5fb1c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fac107df0 0x7f5fac19d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:52.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.954+0000 7f5fb1c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5fac108740 0x7f5fac1a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:52.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.954+0000 7f5fb1c00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac114510 con 0x7f5fac107df0 2024-05-07T02:28:52.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.954+0000 7f5fb1c00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac114390 con 0x7f5fac108740 2024-05-07T02:28:52.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.954+0000 7f5fb1c00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac114690 con 0x7f5fac1034c0 2024-05-07T02:28:52.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.954+0000 7f5faa000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fac107df0 0x7f5fac19d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:52.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.954+0000 7f5faaa00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fac1034c0 0x7f5fac19d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:52.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.954+0000 7f5faa000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fac107df0 0x7f5fac19d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54552/0 (socket says 172.21.15.19:54552) 2024-05-07T02:28:52.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.954+0000 7f5faa000700 1 -- 172.21.15.19:0/1332324983 learned_addr learned my addr 172.21.15.19:0/1332324983 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:28:52.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.954+0000 7f5fab400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5fac108740 0x7f5fac1a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:52.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.955+0000 7f5faaa00700 1 -- 172.21.15.19:0/1332324983 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5fac108740 msgr2=0x7f5fac1a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:52.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.955+0000 7f5faaa00700 1 --2- 172.21.15.19:0/1332324983 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5fac108740 0x7f5fac1a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:52.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.955+0000 7f5faaa00700 1 -- 172.21.15.19:0/1332324983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fac107df0 msgr2=0x7f5fac19d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:52.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.955+0000 7f5faaa00700 1 --2- 172.21.15.19:0/1332324983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fac107df0 0x7f5fac19d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:52.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.955+0000 7f5faaa00700 1 -- 172.21.15.19:0/1332324983 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fac1a1ed0 con 0x7f5fac1034c0 2024-05-07T02:28:52.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.955+0000 7f5faa000700 1 --2- 172.21.15.19:0/1332324983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fac107df0 0x7f5fac19d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:28:52.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.955+0000 7f5faaa00700 1 --2- 172.21.15.19:0/1332324983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fac1034c0 0x7f5fac19d000 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c00e9c0 tx=0x7f5f9c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:28:52.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.956+0000 7f5fa3400700 1 -- 172.21.15.19:0/1332324983 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c00c9c0 con 0x7f5fac1034c0 2024-05-07T02:28:52.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.956+0000 7f5fa3400700 1 -- 172.21.15.19:0/1332324983 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f9c00cb60 con 0x7f5fac1034c0 2024-05-07T02:28:52.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.956+0000 7f5fb1c00700 1 -- 172.21.15.19:0/1332324983 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fac1a21c0 con 0x7f5fac1034c0 2024-05-07T02:28:52.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.956+0000 7f5fa3400700 1 -- 172.21.15.19:0/1332324983 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c018420 con 0x7f5fac1034c0 2024-05-07T02:28:52.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.956+0000 7f5fb1c00700 1 -- 172.21.15.19:0/1332324983 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fac1a9ae0 con 0x7f5fac1034c0 2024-05-07T02:28:52.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.957+0000 7f5fb1c00700 1 -- 172.21.15.19:0/1332324983 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5fac111d10 con 0x7f5fac1034c0 2024-05-07T02:28:52.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.959+0000 7f5fa3400700 1 -- 172.21.15.19:0/1332324983 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f5f9c010430 con 0x7f5fac1034c0 2024-05-07T02:28:52.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.961+0000 7f5fa3400700 1 --2- 172.21.15.19:0/1332324983 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f5f900b2990 0x7f5f900b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:52.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.961+0000 7f5fa3400700 1 -- 172.21.15.19:0/1332324983 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f5f9c0e3380 con 0x7f5fac1034c0 2024-05-07T02:28:52.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.961+0000 7f5faa000700 1 --2- 172.21.15.19:0/1332324983 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f5f900b2990 0x7f5f900b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:52.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.962+0000 7f5fa3400700 1 --2- 172.21.15.19:0/1332324983 >> [v2:172.21.15.27:6816/2756612091,v1:172.21.15.27:6817/2756612091] conn(0x7f5f900bdcc0 0x7f5f900c01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:28:52.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.962+0000 7f5fa3400700 1 -- 172.21.15.19:0/1332324983 --> [v2:172.21.15.27:6816/2756612091,v1:172.21.15.27:6817/2756612091] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5f900c0880 con 0x7f5f900bdcc0 2024-05-07T02:28:52.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.962+0000 7f5fa3400700 1 -- 172.21.15.19:0/1332324983 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f5f9c0e3730 con 0x7f5fac1034c0 2024-05-07T02:28:52.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.962+0000 7f5fab400700 1 --2- 172.21.15.19:0/1332324983 >> [v2:172.21.15.27:6816/2756612091,v1:172.21.15.27:6817/2756612091] conn(0x7f5f900bdcc0 0x7f5f900c01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:28:52.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.962+0000 7f5faa000700 1 --2- 172.21.15.19:0/1332324983 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f5f900b2990 0x7f5f900b4ea0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5fac10f3e0 tx=0x7f5f98009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:28:52.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.963+0000 7f5fab400700 1 --2- 172.21.15.19:0/1332324983 >> [v2:172.21.15.27:6816/2756612091,v1:172.21.15.27:6817/2756612091] conn(0x7f5f900bdcc0 0x7f5f900c01b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:28:52.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.964+0000 7f5fa3400700 1 -- 172.21.15.19:0/1332324983 <== osd.7 v2:172.21.15.27:6816/2756612091 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5f900c0880 con 0x7f5f900bdcc0 2024-05-07T02:28:52.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.987+0000 7f5fb1c00700 1 -- 172.21.15.19:0/1332324983 --> [v2:172.21.15.27:6816/2756612091,v1:172.21.15.27:6817/2756612091] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5fac104970 con 0x7f5f900bdcc0 2024-05-07T02:28:52.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.988+0000 7f5fa3400700 1 -- 172.21.15.19:0/1332324983 <== osd.7 v2:172.21.15.27:6816/2756612091 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5fac104970 con 0x7f5f900bdcc0 2024-05-07T02:28:52.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.989+0000 7f5fb1c00700 1 -- 172.21.15.19:0/1332324983 >> [v2:172.21.15.27:6816/2756612091,v1:172.21.15.27:6817/2756612091] conn(0x7f5f900bdcc0 msgr2=0x7f5f900c01b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:52.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.989+0000 7f5fb1c00700 1 --2- 172.21.15.19:0/1332324983 >> [v2:172.21.15.27:6816/2756612091,v1:172.21.15.27:6817/2756612091] conn(0x7f5f900bdcc0 0x7f5f900c01b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:28:52.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.989+0000 7f5fb1c00700 1 -- 172.21.15.19:0/1332324983 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f5f900b2990 msgr2=0x7f5f900b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:52.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.989+0000 7f5fb1c00700 1 --2- 172.21.15.19:0/1332324983 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f5f900b2990 0x7f5f900b4ea0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5fac10f3e0 tx=0x7f5f98009260 comp rx=0 tx=0).stop 2024-05-07T02:28:52.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.989+0000 7f5fb1c00700 1 -- 172.21.15.19:0/1332324983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fac1034c0 msgr2=0x7f5fac19d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:28:52.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.989+0000 7f5fb1c00700 1 --2- 172.21.15.19:0/1332324983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5fac1034c0 0x7f5fac19d000 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c00e9c0 tx=0x7f5f9c00ecd0 comp rx=0 tx=0).stop 2024-05-07T02:28:52.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.989+0000 7f5fab400700 1 -- 172.21.15.19:0/1332324983 reap_dead start 2024-05-07T02:28:52.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.989+0000 7f5fb1c00700 1 -- 172.21.15.19:0/1332324983 shutdown_connections 2024-05-07T02:28:52.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.989+0000 7f5fb1c00700 1 -- 172.21.15.19:0/1332324983 >> 172.21.15.19:0/1332324983 conn(0x7f5fac0fedc0 msgr2=0x7f5fac1096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:28:52.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.989+0000 7f5fb1c00700 1 -- 172.21.15.19:0/1332324983 shutdown_connections 2024-05-07T02:28:52.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:28:52.989+0000 7f5fb1c00700 1 -- 172.21.15.19:0/1332324983 wait complete. 2024-05-07T02:28:53.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:52 smithi064 ceph-mon[34699]: pgmap v419: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:28:53.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:52 smithi027 ceph-mon[34398]: pgmap v419: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:28:54.920 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:54 smithi019 ceph-mon[29781]: pgmap v420: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-07T02:28:55.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:54 smithi064 ceph-mon[34699]: pgmap v420: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-07T02:28:55.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:54 smithi027 ceph-mon[34398]: pgmap v420: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-07T02:28:55.819 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:28:55.985 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:28:56.655 INFO:teuthology.orchestra.run.smithi019.stdout:201863462986 2024-05-07T02:28:56.655 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-07T02:28:56.830 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:56 smithi019 ceph-mon[29781]: pgmap v421: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:28:57.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:56 smithi064 ceph-mon[34699]: pgmap v421: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:28:57.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:56 smithi027 ceph-mon[34398]: pgmap v421: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:28:59.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:28:58 smithi064 ceph-mon[34699]: pgmap v422: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:28:59.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:28:58 smithi019 ceph-mon[29781]: pgmap v422: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:28:59.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:28:58 smithi027 ceph-mon[34398]: pgmap v422: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:29:01.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:00 smithi064 ceph-mon[34699]: pgmap v423: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:29:01.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:00 smithi019 ceph-mon[29781]: pgmap v423: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:29:01.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:00 smithi027 ceph-mon[34398]: pgmap v423: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:29:02.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.364+0000 7f7991800700 1 -- 172.21.15.19:0/3877329148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f798c105520 msgr2=0x7f798c105930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:02.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.364+0000 7f7991800700 1 --2- 172.21.15.19:0/3877329148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f798c105520 0x7f798c105930 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f797c009990 tx=0x7f797c009ca0 comp rx=0 tx=0).stop 2024-05-07T02:29:02.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.364+0000 7f7991800700 1 -- 172.21.15.19:0/3877329148 shutdown_connections 2024-05-07T02:29:02.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.364+0000 7f7991800700 1 --2- 172.21.15.19:0/3877329148 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f798c10e900 0x7f798c110dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:02.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.364+0000 7f7991800700 1 --2- 172.21.15.19:0/3877329148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f798c105520 0x7f798c105930 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:02.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.364+0000 7f7991800700 1 --2- 172.21.15.19:0/3877329148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f798c105e70 0x7f798c10e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:02.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.364+0000 7f7991800700 1 -- 172.21.15.19:0/3877329148 >> 172.21.15.19:0/3877329148 conn(0x7f798c0fedc0 msgr2=0x7f798c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:02.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.364+0000 7f7991800700 1 -- 172.21.15.19:0/3877329148 shutdown_connections 2024-05-07T02:29:02.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.364+0000 7f7991800700 1 -- 172.21.15.19:0/3877329148 wait complete. 2024-05-07T02:29:02.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.366+0000 7f7991800700 1 Processor -- start 2024-05-07T02:29:02.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.366+0000 7f7991800700 1 -- start start 2024-05-07T02:29:02.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.367+0000 7f7991800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f798c105520 0x7f798c19adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:02.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.367+0000 7f7991800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f798c105e70 0x7f798c19b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:02.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.367+0000 7f7991800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f798c10e900 0x7f798c19f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:02.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.367+0000 7f7991800700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f798c114480 con 0x7f798c105520 2024-05-07T02:29:02.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.367+0000 7f7991800700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f798c114300 con 0x7f798c10e900 2024-05-07T02:29:02.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.367+0000 7f7991800700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f798c114600 con 0x7f798c105e70 2024-05-07T02:29:02.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.367+0000 7f798aa00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f798c105520 0x7f798c19adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:02.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.367+0000 7f798aa00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f798c105520 0x7f798c19adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51500/0 (socket says 172.21.15.19:51500) 2024-05-07T02:29:02.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.367+0000 7f798aa00700 1 -- 172.21.15.19:0/3906434103 learned_addr learned my addr 172.21.15.19:0/3906434103 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:29:02.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.367+0000 7f798a000700 1 --2- 172.21.15.19:0/3906434103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f798c105e70 0x7f798c19b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:02.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.367+0000 7f798b400700 1 --2- 172.21.15.19:0/3906434103 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f798c10e900 0x7f798c19f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:02.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.368+0000 7f798b400700 1 -- 172.21.15.19:0/3906434103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f798c105e70 msgr2=0x7f798c19b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:02.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.368+0000 7f798b400700 1 --2- 172.21.15.19:0/3906434103 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f798c105e70 0x7f798c19b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:02.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.368+0000 7f798b400700 1 -- 172.21.15.19:0/3906434103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f798c105520 msgr2=0x7f798c19adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:02.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.368+0000 7f798b400700 1 --2- 172.21.15.19:0/3906434103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f798c105520 0x7f798c19adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:02.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.368+0000 7f798b400700 1 -- 172.21.15.19:0/3906434103 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f798c19fcc0 con 0x7f798c10e900 2024-05-07T02:29:02.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.368+0000 7f798b400700 1 --2- 172.21.15.19:0/3906434103 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f798c10e900 0x7f798c19f620 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f798400efc0 tx=0x7f798400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:02.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.369+0000 7f7983400700 1 -- 172.21.15.19:0/3906434103 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79840092e0 con 0x7f798c10e900 2024-05-07T02:29:02.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.369+0000 7f7983400700 1 -- 172.21.15.19:0/3906434103 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7984004bb0 con 0x7f798c10e900 2024-05-07T02:29:02.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.369+0000 7f7983400700 1 -- 172.21.15.19:0/3906434103 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7984004020 con 0x7f798c10e900 2024-05-07T02:29:02.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.369+0000 7f7991800700 1 -- 172.21.15.19:0/3906434103 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f798c19ff50 con 0x7f798c10e900 2024-05-07T02:29:02.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.369+0000 7f7991800700 1 -- 172.21.15.19:0/3906434103 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f798c1a0440 con 0x7f798c10e900 2024-05-07T02:29:02.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.370+0000 7f7991800700 1 -- 172.21.15.19:0/3906434103 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f798c063f50 con 0x7f798c10e900 2024-05-07T02:29:02.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.375+0000 7f7983400700 1 -- 172.21.15.19:0/3906434103 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f798401b030 con 0x7f798c10e900 2024-05-07T02:29:02.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.377+0000 7f7983400700 1 --2- 172.21.15.19:0/3906434103 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f79700b2990 0x7f79700b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:02.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.377+0000 7f7983400700 1 -- 172.21.15.19:0/3906434103 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f79840e2b30 con 0x7f798c10e900 2024-05-07T02:29:02.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.377+0000 7f798aa00700 1 --2- 172.21.15.19:0/3906434103 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f79700b2990 0x7f79700b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:02.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.377+0000 7f7983400700 1 --2- 172.21.15.19:0/3906434103 >> [v2:172.21.15.64:6816/1987024836,v1:172.21.15.64:6817/1987024836] conn(0x7f79700bdcc0 0x7f79700c01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:02.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.378+0000 7f7983400700 1 -- 172.21.15.19:0/3906434103 --> [v2:172.21.15.64:6816/1987024836,v1:172.21.15.64:6817/1987024836] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f798400be50 con 0x7f79700bdcc0 2024-05-07T02:29:02.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.378+0000 7f7983400700 1 -- 172.21.15.19:0/3906434103 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f79840e2ee0 con 0x7f798c10e900 2024-05-07T02:29:02.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.378+0000 7f798a000700 1 --2- 172.21.15.19:0/3906434103 >> [v2:172.21.15.64:6816/1987024836,v1:172.21.15.64:6817/1987024836] conn(0x7f79700bdcc0 0x7f79700c01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:02.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.378+0000 7f798aa00700 1 --2- 172.21.15.19:0/3906434103 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f79700b2990 0x7f79700b4ea0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f797c006010 tx=0x7f797c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:02.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.379+0000 7f798a000700 1 --2- 172.21.15.19:0/3906434103 >> [v2:172.21.15.64:6816/1987024836,v1:172.21.15.64:6817/1987024836] conn(0x7f79700bdcc0 0x7f79700c01b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:02.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.380+0000 7f7983400700 1 -- 172.21.15.19:0/3906434103 <== osd.8 v2:172.21.15.64:6816/1987024836 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f798400be50 con 0x7f79700bdcc0 2024-05-07T02:29:02.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.402+0000 7f7991800700 1 -- 172.21.15.19:0/3906434103 --> [v2:172.21.15.64:6816/1987024836,v1:172.21.15.64:6817/1987024836] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f798c0643d0 con 0x7f79700bdcc0 2024-05-07T02:29:02.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.403+0000 7f7983400700 1 -- 172.21.15.19:0/3906434103 <== osd.8 v2:172.21.15.64:6816/1987024836 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f798c0643d0 con 0x7f79700bdcc0 2024-05-07T02:29:02.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.404+0000 7f7991800700 1 -- 172.21.15.19:0/3906434103 >> [v2:172.21.15.64:6816/1987024836,v1:172.21.15.64:6817/1987024836] conn(0x7f79700bdcc0 msgr2=0x7f79700c01b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:02.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.404+0000 7f7991800700 1 --2- 172.21.15.19:0/3906434103 >> [v2:172.21.15.64:6816/1987024836,v1:172.21.15.64:6817/1987024836] conn(0x7f79700bdcc0 0x7f79700c01b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:02.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.404+0000 7f7991800700 1 -- 172.21.15.19:0/3906434103 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f79700b2990 msgr2=0x7f79700b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:02.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.404+0000 7f7991800700 1 --2- 172.21.15.19:0/3906434103 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f79700b2990 0x7f79700b4ea0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f797c006010 tx=0x7f797c00b260 comp rx=0 tx=0).stop 2024-05-07T02:29:02.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.404+0000 7f7991800700 1 -- 172.21.15.19:0/3906434103 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f798c10e900 msgr2=0x7f798c19f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:02.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.404+0000 7f7991800700 1 --2- 172.21.15.19:0/3906434103 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f798c10e900 0x7f798c19f620 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f798400efc0 tx=0x7f798400c3a0 comp rx=0 tx=0).stop 2024-05-07T02:29:02.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.404+0000 7f798b400700 1 -- 172.21.15.19:0/3906434103 reap_dead start 2024-05-07T02:29:02.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.404+0000 7f7991800700 1 -- 172.21.15.19:0/3906434103 shutdown_connections 2024-05-07T02:29:02.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.404+0000 7f7991800700 1 -- 172.21.15.19:0/3906434103 >> 172.21.15.19:0/3906434103 conn(0x7f798c0fedc0 msgr2=0x7f798c0ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:02.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.405+0000 7f7991800700 1 -- 172.21.15.19:0/3906434103 shutdown_connections 2024-05-07T02:29:02.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.405+0000 7f7991800700 1 -- 172.21.15.19:0/3906434103 wait complete. 2024-05-07T02:29:03.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.906+0000 7fea0ee00700 1 -- 172.21.15.19:0/979051543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea08104400 msgr2=0x7fea08104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:03.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.906+0000 7fea0ee00700 1 --2- 172.21.15.19:0/979051543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea08104400 0x7fea08104810 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc009fc0 tx=0x7fe9fc00b330 comp rx=0 tx=0).stop 2024-05-07T02:29:03.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.906+0000 7fea0ee00700 1 -- 172.21.15.19:0/979051543 shutdown_connections 2024-05-07T02:29:03.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.906+0000 7fea0ee00700 1 --2- 172.21.15.19:0/979051543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea08109680 0x7fea08110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:03.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.907+0000 7fea0ee00700 1 --2- 172.21.15.19:0/979051543 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea08104400 0x7fea08104810 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:03.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.907+0000 7fea0ee00700 1 --2- 172.21.15.19:0/979051543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea08108d30 0x7fea08109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:03.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.907+0000 7fea0ee00700 1 -- 172.21.15.19:0/979051543 >> 172.21.15.19:0/979051543 conn(0x7fea08079d10 msgr2=0x7fea0807a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:03.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.907+0000 7fea0ee00700 1 -- 172.21.15.19:0/979051543 shutdown_connections 2024-05-07T02:29:03.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.907+0000 7fea0ee00700 1 -- 172.21.15.19:0/979051543 wait complete. 2024-05-07T02:29:03.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.908+0000 7fea0ee00700 1 Processor -- start 2024-05-07T02:29:03.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.909+0000 7fea0ee00700 1 -- start start 2024-05-07T02:29:03.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.909+0000 7fea0ee00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea08104400 0x7fea081a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:03.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.909+0000 7fea0ee00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea08108d30 0x7fea081a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:03.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.909+0000 7fea0ee00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea08109680 0x7fea0819aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:03.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.909+0000 7fea0ee00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fea08114300 con 0x7fea08108d30 2024-05-07T02:29:03.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.909+0000 7fea0ee00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fea08114180 con 0x7fea08109680 2024-05-07T02:29:03.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.909+0000 7fea0ee00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fea08114480 con 0x7fea08104400 2024-05-07T02:29:03.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.909+0000 7fea0d000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea08108d30 0x7fea081a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:03.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.910+0000 7fea0da00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea08104400 0x7fea081a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:03.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.910+0000 7fea0d000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea08108d30 0x7fea081a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51524/0 (socket says 172.21.15.19:51524) 2024-05-07T02:29:03.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.910+0000 7fea0d000700 1 -- 172.21.15.19:0/2180952008 learned_addr learned my addr 172.21.15.19:0/2180952008 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:29:03.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.910+0000 7fea0e400700 1 --2- 172.21.15.19:0/2180952008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea08109680 0x7fea0819aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:03.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.910+0000 7fea0da00700 1 -- 172.21.15.19:0/2180952008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea08109680 msgr2=0x7fea0819aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:03.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.910+0000 7fea0da00700 1 --2- 172.21.15.19:0/2180952008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea08109680 0x7fea0819aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:03.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.910+0000 7fea0da00700 1 -- 172.21.15.19:0/2180952008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea08108d30 msgr2=0x7fea081a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:03.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.910+0000 7fea0da00700 1 --2- 172.21.15.19:0/2180952008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea08108d30 0x7fea081a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:03.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.910+0000 7fea0da00700 1 -- 172.21.15.19:0/2180952008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea0819b1c0 con 0x7fea08104400 2024-05-07T02:29:03.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.910+0000 7fea0d000700 1 --2- 172.21.15.19:0/2180952008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea08108d30 0x7fea081a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:29:03.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.910+0000 7fea0da00700 1 --2- 172.21.15.19:0/2180952008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea08104400 0x7fea081a08a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc00b6c0 tx=0x7fe9fc00ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:03.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.911+0000 7fea06000700 1 -- 172.21.15.19:0/2180952008 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc021070 con 0x7fea08104400 2024-05-07T02:29:03.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.911+0000 7fea06000700 1 -- 172.21.15.19:0/2180952008 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9fc007ac0 con 0x7fea08104400 2024-05-07T02:29:03.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.911+0000 7fea06000700 1 -- 172.21.15.19:0/2180952008 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc005040 con 0x7fea08104400 2024-05-07T02:29:03.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.911+0000 7fea0ee00700 1 -- 172.21.15.19:0/2180952008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea0819b450 con 0x7fea08104400 2024-05-07T02:29:03.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.911+0000 7fea0ee00700 1 -- 172.21.15.19:0/2180952008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea081a7750 con 0x7fea08104400 2024-05-07T02:29:03.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.912+0000 7fea0ee00700 1 -- 172.21.15.19:0/2180952008 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fea0819b940 con 0x7fea08104400 2024-05-07T02:29:03.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.914+0000 7fea06000700 1 -- 172.21.15.19:0/2180952008 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fe9fc024070 con 0x7fea08104400 2024-05-07T02:29:03.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.916+0000 7fea06000700 1 --2- 172.21.15.19:0/2180952008 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fe9f00b2990 0x7fe9f00b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:03.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.916+0000 7fea06000700 1 -- 172.21.15.19:0/2180952008 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fe9fc00f020 con 0x7fea08104400 2024-05-07T02:29:03.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.916+0000 7fea0d000700 1 --2- 172.21.15.19:0/2180952008 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fe9f00b2990 0x7fe9f00b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:03.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.916+0000 7fea06000700 1 --2- 172.21.15.19:0/2180952008 >> [v2:172.21.15.27:6800/4292470615,v1:172.21.15.27:6801/4292470615] conn(0x7fe9f00bdcc0 0x7fe9f00c01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:03.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.916+0000 7fea06000700 1 -- 172.21.15.19:0/2180952008 --> [v2:172.21.15.27:6800/4292470615,v1:172.21.15.27:6801/4292470615] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe9fc007500 con 0x7fe9f00bdcc0 2024-05-07T02:29:03.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.916+0000 7fea06000700 1 -- 172.21.15.19:0/2180952008 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fe9fc0e40d0 con 0x7fea08104400 2024-05-07T02:29:03.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.917+0000 7fea0e400700 1 --2- 172.21.15.19:0/2180952008 >> [v2:172.21.15.27:6800/4292470615,v1:172.21.15.27:6801/4292470615] conn(0x7fe9f00bdcc0 0x7fe9f00c01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:03.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.917+0000 7fea0d000700 1 --2- 172.21.15.19:0/2180952008 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fe9f00b2990 0x7fe9f00b4ea0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fea0807a240 tx=0x7fe9f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:03.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.917+0000 7fea0e400700 1 --2- 172.21.15.19:0/2180952008 >> [v2:172.21.15.27:6800/4292470615,v1:172.21.15.27:6801/4292470615] conn(0x7fe9f00bdcc0 0x7fe9f00c01b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:03.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.918+0000 7fea06000700 1 -- 172.21.15.19:0/2180952008 <== osd.1 v2:172.21.15.27:6800/4292470615 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe9fc007500 con 0x7fe9f00bdcc0 2024-05-07T02:29:03.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.937+0000 7fea0ee00700 1 -- 172.21.15.19:0/2180952008 --> [v2:172.21.15.27:6800/4292470615,v1:172.21.15.27:6801/4292470615] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fea08064270 con 0x7fe9f00bdcc0 2024-05-07T02:29:03.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.938+0000 7fea06000700 1 -- 172.21.15.19:0/2180952008 <== osd.1 v2:172.21.15.27:6800/4292470615 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fea08064270 con 0x7fe9f00bdcc0 2024-05-07T02:29:03.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.938+0000 7fea0ee00700 1 -- 172.21.15.19:0/2180952008 >> [v2:172.21.15.27:6800/4292470615,v1:172.21.15.27:6801/4292470615] conn(0x7fe9f00bdcc0 msgr2=0x7fe9f00c01b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:03.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.938+0000 7fea0ee00700 1 --2- 172.21.15.19:0/2180952008 >> [v2:172.21.15.27:6800/4292470615,v1:172.21.15.27:6801/4292470615] conn(0x7fe9f00bdcc0 0x7fe9f00c01b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:03.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.938+0000 7fea0ee00700 1 -- 172.21.15.19:0/2180952008 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fe9f00b2990 msgr2=0x7fe9f00b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:03.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.938+0000 7fea0ee00700 1 --2- 172.21.15.19:0/2180952008 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fe9f00b2990 0x7fe9f00b4ea0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fea0807a240 tx=0x7fe9f4009260 comp rx=0 tx=0).stop 2024-05-07T02:29:03.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.938+0000 7fea0ee00700 1 -- 172.21.15.19:0/2180952008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea08104400 msgr2=0x7fea081a08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:03.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.938+0000 7fea0ee00700 1 --2- 172.21.15.19:0/2180952008 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea08104400 0x7fea081a08a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc00b6c0 tx=0x7fe9fc00ccb0 comp rx=0 tx=0).stop 2024-05-07T02:29:03.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.938+0000 7fea0e400700 1 -- 172.21.15.19:0/2180952008 reap_dead start 2024-05-07T02:29:03.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.939+0000 7fea0ee00700 1 -- 172.21.15.19:0/2180952008 shutdown_connections 2024-05-07T02:29:03.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.939+0000 7fea0ee00700 1 -- 172.21.15.19:0/2180952008 >> 172.21.15.19:0/2180952008 conn(0x7fea08079d10 msgr2=0x7fea08102910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:03.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.939+0000 7fea0ee00700 1 -- 172.21.15.19:0/2180952008 shutdown_connections 2024-05-07T02:29:03.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:02.939+0000 7fea0ee00700 1 -- 172.21.15.19:0/2180952008 wait complete. 2024-05-07T02:29:03.093 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:02 smithi019 ceph-mon[29781]: pgmap v424: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:29:03.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:02 smithi064 ceph-mon[34699]: pgmap v424: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:29:03.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:02 smithi027 ceph-mon[34398]: pgmap v424: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:29:03.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.973+0000 7f2377000700 1 -- 172.21.15.19:0/3584119145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2370105520 msgr2=0x7f2370105930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:03.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.973+0000 7f2377000700 1 --2- 172.21.15.19:0/3584119145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2370105520 0x7f2370105930 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2360009990 tx=0x7f2360009ca0 comp rx=0 tx=0).stop 2024-05-07T02:29:03.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.973+0000 7f2377000700 1 -- 172.21.15.19:0/3584119145 shutdown_connections 2024-05-07T02:29:03.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.973+0000 7f2377000700 1 --2- 172.21.15.19:0/3584119145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f237010e900 0x7f2370110dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:03.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.973+0000 7f2377000700 1 --2- 172.21.15.19:0/3584119145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2370105520 0x7f2370105930 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:03.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.973+0000 7f2377000700 1 --2- 172.21.15.19:0/3584119145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2370105e70 0x7f237010e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:03.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.973+0000 7f2377000700 1 -- 172.21.15.19:0/3584119145 >> 172.21.15.19:0/3584119145 conn(0x7f23700fedc0 msgr2=0x7f23701011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:03.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.973+0000 7f2377000700 1 -- 172.21.15.19:0/3584119145 shutdown_connections 2024-05-07T02:29:03.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.973+0000 7f2377000700 1 -- 172.21.15.19:0/3584119145 wait complete. 2024-05-07T02:29:03.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.975+0000 7f2377000700 1 Processor -- start 2024-05-07T02:29:03.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.975+0000 7f2377000700 1 -- start start 2024-05-07T02:29:03.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.975+0000 7f2377000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2370105520 0x7f237019d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:03.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.975+0000 7f2377000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2370105e70 0x7f237019d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:03.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.975+0000 7f2377000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f237010e900 0x7f23701a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:03.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.975+0000 7f2377000700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2370114480 con 0x7f2370105e70 2024-05-07T02:29:03.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.975+0000 7f2377000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2370114300 con 0x7f237010e900 2024-05-07T02:29:03.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.975+0000 7f2377000700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2370114600 con 0x7f2370105520 2024-05-07T02:29:03.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.976+0000 7f236fe00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2370105520 0x7f237019d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:03.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.976+0000 7f2375000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f237010e900 0x7f23701a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:03.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.976+0000 7f236fe00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2370105520 0x7f237019d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:54026/0 (socket says 172.21.15.19:54026) 2024-05-07T02:29:03.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.976+0000 7f236f400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2370105e70 0x7f237019d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:03.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.976+0000 7f236fe00700 1 -- 172.21.15.19:0/2691713687 learned_addr learned my addr 172.21.15.19:0/2691713687 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:29:03.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.976+0000 7f2375000700 1 -- 172.21.15.19:0/2691713687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2370105520 msgr2=0x7f237019d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:03.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.976+0000 7f2375000700 1 --2- 172.21.15.19:0/2691713687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2370105520 0x7f237019d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:03.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.976+0000 7f2375000700 1 -- 172.21.15.19:0/2691713687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2370105e70 msgr2=0x7f237019d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:03.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.976+0000 7f2375000700 1 --2- 172.21.15.19:0/2691713687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2370105e70 0x7f237019d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:03.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.976+0000 7f2375000700 1 -- 172.21.15.19:0/2691713687 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23701a1f00 con 0x7f237010e900 2024-05-07T02:29:03.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.976+0000 7f236fe00700 1 --2- 172.21.15.19:0/2691713687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2370105520 0x7f237019d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:29:03.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.976+0000 7f236f400700 1 --2- 172.21.15.19:0/2691713687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2370105e70 0x7f237019d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:29:03.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.977+0000 7f2375000700 1 --2- 172.21.15.19:0/2691713687 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f237010e900 0x7f23701a1860 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2368007f10 tx=0x7f236800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:03.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.977+0000 7f236cc00700 1 -- 172.21.15.19:0/2691713687 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2368004db0 con 0x7f237010e900 2024-05-07T02:29:03.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.977+0000 7f236cc00700 1 -- 172.21.15.19:0/2691713687 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2368009e20 con 0x7f237010e900 2024-05-07T02:29:03.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.977+0000 7f236cc00700 1 -- 172.21.15.19:0/2691713687 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2368015400 con 0x7f237010e900 2024-05-07T02:29:03.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.977+0000 7f2377000700 1 -- 172.21.15.19:0/2691713687 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23701a21f0 con 0x7f237010e900 2024-05-07T02:29:03.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.977+0000 7f2377000700 1 -- 172.21.15.19:0/2691713687 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23701a9b10 con 0x7f237010e900 2024-05-07T02:29:03.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.978+0000 7f2377000700 1 -- 172.21.15.19:0/2691713687 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2370103c00 con 0x7f237010e900 2024-05-07T02:29:03.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.981+0000 7f236cc00700 1 -- 172.21.15.19:0/2691713687 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f2368005260 con 0x7f237010e900 2024-05-07T02:29:03.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.982+0000 7f236cc00700 1 --2- 172.21.15.19:0/2691713687 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f23540b2990 0x7f23540b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:03.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.982+0000 7f236cc00700 1 -- 172.21.15.19:0/2691713687 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f23680e2590 con 0x7f237010e900 2024-05-07T02:29:03.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.982+0000 7f236fe00700 1 --2- 172.21.15.19:0/2691713687 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f23540b2990 0x7f23540b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:03.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.983+0000 7f236cc00700 1 --2- 172.21.15.19:0/2691713687 >> [v2:172.21.15.64:6824/4045070679,v1:172.21.15.64:6825/4045070679] conn(0x7f23540bdcc0 0x7f23540c01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:03.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.983+0000 7f236cc00700 1 -- 172.21.15.19:0/2691713687 --> [v2:172.21.15.64:6824/4045070679,v1:172.21.15.64:6825/4045070679] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f236800ee50 con 0x7f23540bdcc0 2024-05-07T02:29:03.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.983+0000 7f236cc00700 1 -- 172.21.15.19:0/2691713687 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f23680e2940 con 0x7f237010e900 2024-05-07T02:29:03.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.983+0000 7f236f400700 1 --2- 172.21.15.19:0/2691713687 >> [v2:172.21.15.64:6824/4045070679,v1:172.21.15.64:6825/4045070679] conn(0x7f23540bdcc0 0x7f23540c01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:03.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.983+0000 7f236fe00700 1 --2- 172.21.15.19:0/2691713687 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f23540b2990 0x7f23540b4ea0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2360006010 tx=0x7f236000b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:03.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.984+0000 7f236f400700 1 --2- 172.21.15.19:0/2691713687 >> [v2:172.21.15.64:6824/4045070679,v1:172.21.15.64:6825/4045070679] conn(0x7f23540bdcc0 0x7f23540c01b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:03.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:03.985+0000 7f236cc00700 1 -- 172.21.15.19:0/2691713687 <== osd.11 v2:172.21.15.64:6824/4045070679 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f236800ee50 con 0x7f23540bdcc0 2024-05-07T02:29:04.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:04.003+0000 7f2377000700 1 -- 172.21.15.19:0/2691713687 --> [v2:172.21.15.64:6824/4045070679,v1:172.21.15.64:6825/4045070679] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f237006be30 con 0x7f23540bdcc0 2024-05-07T02:29:04.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:04.004+0000 7f236cc00700 1 -- 172.21.15.19:0/2691713687 <== osd.11 v2:172.21.15.64:6824/4045070679 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f237006be30 con 0x7f23540bdcc0 2024-05-07T02:29:04.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:04.005+0000 7f2377000700 1 -- 172.21.15.19:0/2691713687 >> [v2:172.21.15.64:6824/4045070679,v1:172.21.15.64:6825/4045070679] conn(0x7f23540bdcc0 msgr2=0x7f23540c01b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:04.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:04.005+0000 7f2377000700 1 --2- 172.21.15.19:0/2691713687 >> [v2:172.21.15.64:6824/4045070679,v1:172.21.15.64:6825/4045070679] conn(0x7f23540bdcc0 0x7f23540c01b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:04.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:04.005+0000 7f2377000700 1 -- 172.21.15.19:0/2691713687 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f23540b2990 msgr2=0x7f23540b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:04.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:04.005+0000 7f2377000700 1 --2- 172.21.15.19:0/2691713687 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f23540b2990 0x7f23540b4ea0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2360006010 tx=0x7f236000b280 comp rx=0 tx=0).stop 2024-05-07T02:29:04.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:04.005+0000 7f2377000700 1 -- 172.21.15.19:0/2691713687 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f237010e900 msgr2=0x7f23701a1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:04.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:04.005+0000 7f2377000700 1 --2- 172.21.15.19:0/2691713687 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f237010e900 0x7f23701a1860 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2368007f10 tx=0x7f236800d3b0 comp rx=0 tx=0).stop 2024-05-07T02:29:04.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:04.005+0000 7f2375000700 1 -- 172.21.15.19:0/2691713687 reap_dead start 2024-05-07T02:29:04.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:04.005+0000 7f2377000700 1 -- 172.21.15.19:0/2691713687 shutdown_connections 2024-05-07T02:29:04.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:04.005+0000 7f2377000700 1 -- 172.21.15.19:0/2691713687 >> 172.21.15.19:0/2691713687 conn(0x7f23700fedc0 msgr2=0x7f23701031a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:04.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:04.005+0000 7f2377000700 1 -- 172.21.15.19:0/2691713687 shutdown_connections 2024-05-07T02:29:04.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:04.005+0000 7f2377000700 1 -- 172.21.15.19:0/2691713687 wait complete. 2024-05-07T02:29:05.001 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:04 smithi019 ceph-mon[29781]: pgmap v425: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:29:05.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:04 smithi064 ceph-mon[34699]: pgmap v425: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:29:05.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:04 smithi027 ceph-mon[34398]: pgmap v425: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:29:06.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.253+0000 7f40f7c00700 1 -- 172.21.15.19:0/77681164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40f0106570 msgr2=0x7f40f0106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:06.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.253+0000 7f40f7c00700 1 --2- 172.21.15.19:0/77681164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40f0106570 0x7f40f0106980 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f40d8009990 tx=0x7f40d8009ca0 comp rx=0 tx=0).stop 2024-05-07T02:29:06.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.253+0000 7f40f7c00700 1 -- 172.21.15.19:0/77681164 shutdown_connections 2024-05-07T02:29:06.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.253+0000 7f40f7c00700 1 --2- 172.21.15.19:0/77681164 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40f0106ec0 0x7f40f0113350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:06.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.253+0000 7f40f7c00700 1 --2- 172.21.15.19:0/77681164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40f010c5b0 0x7f40f010c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:06.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.253+0000 7f40f7c00700 1 --2- 172.21.15.19:0/77681164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40f0106570 0x7f40f0106980 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:06.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.253+0000 7f40f7c00700 1 -- 172.21.15.19:0/77681164 >> 172.21.15.19:0/77681164 conn(0x7f40f0102050 msgr2=0x7f40f01044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:06.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.253+0000 7f40f7c00700 1 -- 172.21.15.19:0/77681164 shutdown_connections 2024-05-07T02:29:06.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.253+0000 7f40f7c00700 1 -- 172.21.15.19:0/77681164 wait complete. 2024-05-07T02:29:06.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.254+0000 7f40f7c00700 1 Processor -- start 2024-05-07T02:29:06.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.254+0000 7f40f7c00700 1 -- start start 2024-05-07T02:29:06.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.255+0000 7f40f7c00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40f0106570 0x7f40f019d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:06.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.255+0000 7f40f7c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40f0106ec0 0x7f40f019d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:06.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.255+0000 7f40f7c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40f010c5b0 0x7f40f01a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:06.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.255+0000 7f40f7c00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f40f01166e0 con 0x7f40f0106ec0 2024-05-07T02:29:06.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.255+0000 7f40f7c00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f40f0116560 con 0x7f40f010c5b0 2024-05-07T02:29:06.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.255+0000 7f40f7c00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f40f0116860 con 0x7f40f0106570 2024-05-07T02:29:06.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.255+0000 7f40f5200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40f0106570 0x7f40f019d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:06.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.255+0000 7f40f5c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40f010c5b0 0x7f40f01a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:06.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.255+0000 7f40e7e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40f0106ec0 0x7f40f019d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:06.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.255+0000 7f40f5200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40f0106570 0x7f40f019d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:54032/0 (socket says 172.21.15.19:54032) 2024-05-07T02:29:06.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.255+0000 7f40f5200700 1 -- 172.21.15.19:0/2999287321 learned_addr learned my addr 172.21.15.19:0/2999287321 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:29:06.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.256+0000 7f40f5200700 1 -- 172.21.15.19:0/2999287321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40f010c5b0 msgr2=0x7f40f01a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:06.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.256+0000 7f40f5200700 1 --2- 172.21.15.19:0/2999287321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40f010c5b0 0x7f40f01a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:06.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.256+0000 7f40f5200700 1 -- 172.21.15.19:0/2999287321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40f0106ec0 msgr2=0x7f40f019d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:06.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.256+0000 7f40f5200700 1 --2- 172.21.15.19:0/2999287321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40f0106ec0 0x7f40f019d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:06.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.256+0000 7f40f5200700 1 -- 172.21.15.19:0/2999287321 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f01a1f20 con 0x7f40f0106570 2024-05-07T02:29:06.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.256+0000 7f40f5c00700 1 --2- 172.21.15.19:0/2999287321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40f010c5b0 0x7f40f01a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:29:06.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.256+0000 7f40f5200700 1 --2- 172.21.15.19:0/2999287321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40f0106570 0x7f40f019d050 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f40e000e970 tx=0x7f40e000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:06.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.257+0000 7f40ee000700 1 -- 172.21.15.19:0/2999287321 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e0004a30 con 0x7f40f0106570 2024-05-07T02:29:06.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.257+0000 7f40ee000700 1 -- 172.21.15.19:0/2999287321 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40e000cc90 con 0x7f40f0106570 2024-05-07T02:29:06.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.257+0000 7f40f7c00700 1 -- 172.21.15.19:0/2999287321 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40f01a2210 con 0x7f40f0106570 2024-05-07T02:29:06.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.257+0000 7f40ee000700 1 -- 172.21.15.19:0/2999287321 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e0005610 con 0x7f40f0106570 2024-05-07T02:29:06.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.257+0000 7f40f7c00700 1 -- 172.21.15.19:0/2999287321 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40f01a9b30 con 0x7f40f0106570 2024-05-07T02:29:06.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.258+0000 7f40f7c00700 1 -- 172.21.15.19:0/2999287321 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f40f0063ed0 con 0x7f40f0106570 2024-05-07T02:29:06.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.261+0000 7f40ee000700 1 -- 172.21.15.19:0/2999287321 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f40e000c7e0 con 0x7f40f0106570 2024-05-07T02:29:06.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.262+0000 7f40ee000700 1 --2- 172.21.15.19:0/2999287321 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f40d40b2910 0x7f40d40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:06.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.262+0000 7f40ee000700 1 -- 172.21.15.19:0/2999287321 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f40e00e2d90 con 0x7f40f0106570 2024-05-07T02:29:06.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.262+0000 7f40e7e00700 1 --2- 172.21.15.19:0/2999287321 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f40d40b2910 0x7f40d40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:06.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.263+0000 7f40ee000700 1 --2- 172.21.15.19:0/2999287321 >> [v2:172.21.15.64:6808/1264591849,v1:172.21.15.64:6809/1264591849] conn(0x7f40d40bdc40 0x7f40d40c0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:06.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.263+0000 7f40ee000700 1 -- 172.21.15.19:0/2999287321 --> [v2:172.21.15.64:6808/1264591849,v1:172.21.15.64:6809/1264591849] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f40e0004d50 con 0x7f40d40bdc40 2024-05-07T02:29:06.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.263+0000 7f40ee000700 1 -- 172.21.15.19:0/2999287321 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f40e00e3140 con 0x7f40f0106570 2024-05-07T02:29:06.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.263+0000 7f40f5c00700 1 --2- 172.21.15.19:0/2999287321 >> [v2:172.21.15.64:6808/1264591849,v1:172.21.15.64:6809/1264591849] conn(0x7f40d40bdc40 0x7f40d40c0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:06.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.263+0000 7f40e7e00700 1 --2- 172.21.15.19:0/2999287321 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f40d40b2910 0x7f40d40b4e20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f40d800b2e0 tx=0x7f40d8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:06.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.264+0000 7f40f5c00700 1 --2- 172.21.15.19:0/2999287321 >> [v2:172.21.15.64:6808/1264591849,v1:172.21.15.64:6809/1264591849] conn(0x7f40d40bdc40 0x7f40d40c0130 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:06.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.265+0000 7f40ee000700 1 -- 172.21.15.19:0/2999287321 <== osd.5 v2:172.21.15.64:6808/1264591849 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f40e0004d50 con 0x7f40d40bdc40 2024-05-07T02:29:06.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.285+0000 7f40f7c00700 1 -- 172.21.15.19:0/2999287321 --> [v2:172.21.15.64:6808/1264591849,v1:172.21.15.64:6809/1264591849] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f40f0064350 con 0x7f40d40bdc40 2024-05-07T02:29:06.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.286+0000 7f40ee000700 1 -- 172.21.15.19:0/2999287321 <== osd.5 v2:172.21.15.64:6808/1264591849 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f40f0064350 con 0x7f40d40bdc40 2024-05-07T02:29:06.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.286+0000 7f40f7c00700 1 -- 172.21.15.19:0/2999287321 >> [v2:172.21.15.64:6808/1264591849,v1:172.21.15.64:6809/1264591849] conn(0x7f40d40bdc40 msgr2=0x7f40d40c0130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:06.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.286+0000 7f40f7c00700 1 --2- 172.21.15.19:0/2999287321 >> [v2:172.21.15.64:6808/1264591849,v1:172.21.15.64:6809/1264591849] conn(0x7f40d40bdc40 0x7f40d40c0130 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:06.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.286+0000 7f40f7c00700 1 -- 172.21.15.19:0/2999287321 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f40d40b2910 msgr2=0x7f40d40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:06.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.286+0000 7f40f7c00700 1 --2- 172.21.15.19:0/2999287321 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f40d40b2910 0x7f40d40b4e20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f40d800b2e0 tx=0x7f40d8005fd0 comp rx=0 tx=0).stop 2024-05-07T02:29:06.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.286+0000 7f40f7c00700 1 -- 172.21.15.19:0/2999287321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40f0106570 msgr2=0x7f40f019d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:06.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.286+0000 7f40f7c00700 1 --2- 172.21.15.19:0/2999287321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40f0106570 0x7f40f019d050 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f40e000e970 tx=0x7f40e000ec80 comp rx=0 tx=0).stop 2024-05-07T02:29:06.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.286+0000 7f40f5c00700 1 -- 172.21.15.19:0/2999287321 reap_dead start 2024-05-07T02:29:06.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.287+0000 7f40f7c00700 1 -- 172.21.15.19:0/2999287321 shutdown_connections 2024-05-07T02:29:06.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.287+0000 7f40f7c00700 1 -- 172.21.15.19:0/2999287321 >> 172.21.15.19:0/2999287321 conn(0x7f40f0102050 msgr2=0x7f40f01037e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:06.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.287+0000 7f40f7c00700 1 -- 172.21.15.19:0/2999287321 shutdown_connections 2024-05-07T02:29:06.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:05.287+0000 7f40f7c00700 1 -- 172.21.15.19:0/2999287321 wait complete. 2024-05-07T02:29:07.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:06 smithi064 ceph-mon[34699]: pgmap v426: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:29:07.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:06 smithi019 ceph-mon[29781]: pgmap v426: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:29:07.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:06 smithi027 ceph-mon[34398]: pgmap v426: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:29:07.689 INFO:teuthology.orchestra.run.smithi019.stdout:227633266759 2024-05-07T02:29:07.690 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-07T02:29:07.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.978+0000 7fa68c600700 1 -- 172.21.15.19:0/980855226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa684075b60 msgr2=0x7fa684075f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:07.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.978+0000 7fa68c600700 1 --2- 172.21.15.19:0/980855226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa684075b60 0x7fa684075f70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa6780098f0 tx=0x7fa678009c00 comp rx=0 tx=0).stop 2024-05-07T02:29:07.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.978+0000 7fa68c600700 1 -- 172.21.15.19:0/980855226 shutdown_connections 2024-05-07T02:29:07.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.978+0000 7fa68c600700 1 --2- 172.21.15.19:0/980855226 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa684115370 0x7fa684117790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:07.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.978+0000 7fa68c600700 1 --2- 172.21.15.19:0/980855226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa684075b60 0x7fa684075f70 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:07.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.978+0000 7fa68c600700 1 --2- 172.21.15.19:0/980855226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6840764b0 0x7fa684114e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:07.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.978+0000 7fa68c600700 1 -- 172.21.15.19:0/980855226 >> 172.21.15.19:0/980855226 conn(0x7fa684100040 msgr2=0x7fa684102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:07.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.978+0000 7fa68c600700 1 -- 172.21.15.19:0/980855226 shutdown_connections 2024-05-07T02:29:07.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.979+0000 7fa68c600700 1 -- 172.21.15.19:0/980855226 wait complete. 2024-05-07T02:29:07.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.980+0000 7fa68c600700 1 Processor -- start 2024-05-07T02:29:07.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.980+0000 7fa68c600700 1 -- start start 2024-05-07T02:29:07.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.981+0000 7fa68c600700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa684075b60 0x7fa6841a1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:07.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.981+0000 7fa68c600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6840764b0 0x7fa6841a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:07.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.981+0000 7fa68c600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa684115370 0x7fa6841a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:07.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.981+0000 7fa68c600700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa68411aa70 con 0x7fa684115370 2024-05-07T02:29:07.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.981+0000 7fa68c600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa68411a8f0 con 0x7fa6840764b0 2024-05-07T02:29:07.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.981+0000 7fa68c600700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa68411abf0 con 0x7fa684075b60 2024-05-07T02:29:07.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.982+0000 7fa689200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6840764b0 0x7fa6841a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:07.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.982+0000 7fa689c00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa684075b60 0x7fa6841a1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:07.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.982+0000 7fa689c00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa684075b60 0x7fa6841a1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:54062/0 (socket says 172.21.15.19:54062) 2024-05-07T02:29:07.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.982+0000 7fa689200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6840764b0 0x7fa6841a19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:33282/0 (socket says 172.21.15.19:33282) 2024-05-07T02:29:07.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.982+0000 7fa689c00700 1 -- 172.21.15.19:0/3295915387 learned_addr learned my addr 172.21.15.19:0/3295915387 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:29:07.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.982+0000 7fa68a600700 1 --2- 172.21.15.19:0/3295915387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa684115370 0x7fa6841a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:07.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.982+0000 7fa689200700 1 -- 172.21.15.19:0/3295915387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa684075b60 msgr2=0x7fa6841a1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:07.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.982+0000 7fa689200700 1 --2- 172.21.15.19:0/3295915387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa684075b60 0x7fa6841a1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:07.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.983+0000 7fa689200700 1 -- 172.21.15.19:0/3295915387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa684115370 msgr2=0x7fa6841a5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:07.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.983+0000 7fa689200700 1 --2- 172.21.15.19:0/3295915387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa684115370 0x7fa6841a5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:07.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.983+0000 7fa689200700 1 -- 172.21.15.19:0/3295915387 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6841a6310 con 0x7fa6840764b0 2024-05-07T02:29:07.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.983+0000 7fa689c00700 1 --2- 172.21.15.19:0/3295915387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa684075b60 0x7fa6841a1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:29:07.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.983+0000 7fa689200700 1 --2- 172.21.15.19:0/3295915387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6840764b0 0x7fa6841a19e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa67000e9c0 tx=0x7fa67000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:07.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.983+0000 7fa682000700 1 -- 172.21.15.19:0/3295915387 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67000c9c0 con 0x7fa6840764b0 2024-05-07T02:29:07.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.983+0000 7fa682000700 1 -- 172.21.15.19:0/3295915387 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa67000cb60 con 0x7fa6840764b0 2024-05-07T02:29:07.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.984+0000 7fa68c600700 1 -- 172.21.15.19:0/3295915387 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6841a6600 con 0x7fa6840764b0 2024-05-07T02:29:07.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.984+0000 7fa682000700 1 -- 172.21.15.19:0/3295915387 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa670018680 con 0x7fa6840764b0 2024-05-07T02:29:07.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.984+0000 7fa68c600700 1 -- 172.21.15.19:0/3295915387 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6841adf20 con 0x7fa6840764b0 2024-05-07T02:29:07.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.985+0000 7fa68c600700 1 -- 172.21.15.19:0/3295915387 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa684063f50 con 0x7fa6840764b0 2024-05-07T02:29:07.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.988+0000 7fa682000700 1 -- 172.21.15.19:0/3295915387 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fa670010430 con 0x7fa6840764b0 2024-05-07T02:29:07.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.989+0000 7fa682000700 1 --2- 172.21.15.19:0/3295915387 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fa66c0b2990 0x7fa66c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:07.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.989+0000 7fa682000700 1 -- 172.21.15.19:0/3295915387 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa6700e2ec0 con 0x7fa6840764b0 2024-05-07T02:29:07.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.990+0000 7fa689c00700 1 --2- 172.21.15.19:0/3295915387 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fa66c0b2990 0x7fa66c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:07.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.990+0000 7fa682000700 1 --2- 172.21.15.19:0/3295915387 >> [v2:172.21.15.64:6800/1180552939,v1:172.21.15.64:6801/1180552939] conn(0x7fa66c0bdc40 0x7fa66c0c0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:07.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.990+0000 7fa682000700 1 -- 172.21.15.19:0/3295915387 --> [v2:172.21.15.64:6800/1180552939,v1:172.21.15.64:6801/1180552939] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa66c0c0800 con 0x7fa66c0bdc40 2024-05-07T02:29:07.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.990+0000 7fa682000700 1 -- 172.21.15.19:0/3295915387 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fa6700e3270 con 0x7fa6840764b0 2024-05-07T02:29:07.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.990+0000 7fa68a600700 1 --2- 172.21.15.19:0/3295915387 >> [v2:172.21.15.64:6800/1180552939,v1:172.21.15.64:6801/1180552939] conn(0x7fa66c0bdc40 0x7fa66c0c0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:07.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.991+0000 7fa689c00700 1 --2- 172.21.15.19:0/3295915387 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fa66c0b2990 0x7fa66c0b4ea0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa678005f40 tx=0x7fa678019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:07.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.991+0000 7fa68a600700 1 --2- 172.21.15.19:0/3295915387 >> [v2:172.21.15.64:6800/1180552939,v1:172.21.15.64:6801/1180552939] conn(0x7fa66c0bdc40 0x7fa66c0c0130 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:07.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.992+0000 7fa682000700 1 -- 172.21.15.19:0/3295915387 <== osd.2 v2:172.21.15.64:6800/1180552939 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa66c0c0800 con 0x7fa66c0bdc40 2024-05-07T02:29:08.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:08.011+0000 7fa68c600700 1 -- 172.21.15.19:0/3295915387 --> [v2:172.21.15.64:6800/1180552939,v1:172.21.15.64:6801/1180552939] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa684064350 con 0x7fa66c0bdc40 2024-05-07T02:29:08.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:08.011+0000 7fa682000700 1 -- 172.21.15.19:0/3295915387 <== osd.2 v2:172.21.15.64:6800/1180552939 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa684064350 con 0x7fa66c0bdc40 2024-05-07T02:29:08.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:08.012+0000 7fa68c600700 1 -- 172.21.15.19:0/3295915387 >> [v2:172.21.15.64:6800/1180552939,v1:172.21.15.64:6801/1180552939] conn(0x7fa66c0bdc40 msgr2=0x7fa66c0c0130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:08.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:08.012+0000 7fa68c600700 1 --2- 172.21.15.19:0/3295915387 >> [v2:172.21.15.64:6800/1180552939,v1:172.21.15.64:6801/1180552939] conn(0x7fa66c0bdc40 0x7fa66c0c0130 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:08.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:08.012+0000 7fa68c600700 1 -- 172.21.15.19:0/3295915387 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fa66c0b2990 msgr2=0x7fa66c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:08.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:08.012+0000 7fa68c600700 1 --2- 172.21.15.19:0/3295915387 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fa66c0b2990 0x7fa66c0b4ea0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa678005f40 tx=0x7fa678019040 comp rx=0 tx=0).stop 2024-05-07T02:29:08.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:08.012+0000 7fa68c600700 1 -- 172.21.15.19:0/3295915387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6840764b0 msgr2=0x7fa6841a19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:08.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:08.012+0000 7fa68c600700 1 --2- 172.21.15.19:0/3295915387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6840764b0 0x7fa6841a19e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa67000e9c0 tx=0x7fa67000ecd0 comp rx=0 tx=0).stop 2024-05-07T02:29:08.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:08.012+0000 7fa68a600700 1 -- 172.21.15.19:0/3295915387 reap_dead start 2024-05-07T02:29:08.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:08.012+0000 7fa68c600700 1 -- 172.21.15.19:0/3295915387 shutdown_connections 2024-05-07T02:29:08.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:08.012+0000 7fa68c600700 1 -- 172.21.15.19:0/3295915387 >> 172.21.15.19:0/3295915387 conn(0x7fa684100040 msgr2=0x7fa684100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:08.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:08.013+0000 7fa68c600700 1 -- 172.21.15.19:0/3295915387 shutdown_connections 2024-05-07T02:29:08.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:08.013+0000 7fa68c600700 1 -- 172.21.15.19:0/3295915387 wait complete. 2024-05-07T02:29:09.098 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:08 smithi019 ceph-mon[29781]: pgmap v427: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-07T02:29:09.110 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:08 smithi064 ceph-mon[34699]: pgmap v427: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-07T02:29:09.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:08 smithi027 ceph-mon[34398]: pgmap v427: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-07T02:29:09.648 INFO:teuthology.orchestra.run.smithi019.stdout:300647710771 2024-05-07T02:29:09.648 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-07T02:29:09.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.946+0000 7f0693400700 1 -- 172.21.15.19:0/3938041685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f068c10e920 msgr2=0x7f068c110e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:09.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.946+0000 7f0693400700 1 --2- 172.21.15.19:0/3938041685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f068c10e920 0x7f068c110e00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f068400b0a0 tx=0x7f068400b3b0 comp rx=0 tx=0).stop 2024-05-07T02:29:09.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.946+0000 7f0693400700 1 -- 172.21.15.19:0/3938041685 shutdown_connections 2024-05-07T02:29:09.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.946+0000 7f0693400700 1 --2- 172.21.15.19:0/3938041685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f068c10e920 0x7f068c110e00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:09.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.946+0000 7f0693400700 1 --2- 172.21.15.19:0/3938041685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f068c1054f0 0x7f068c105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:09.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.946+0000 7f0693400700 1 --2- 172.21.15.19:0/3938041685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f068c105e40 0x7f068c10e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:09.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.946+0000 7f0693400700 1 -- 172.21.15.19:0/3938041685 >> 172.21.15.19:0/3938041685 conn(0x7f068c078e00 msgr2=0x7f068c079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:09.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.946+0000 7f0693400700 1 -- 172.21.15.19:0/3938041685 shutdown_connections 2024-05-07T02:29:09.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.947+0000 7f0693400700 1 -- 172.21.15.19:0/3938041685 wait complete. 2024-05-07T02:29:09.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.948+0000 7f0693400700 1 Processor -- start 2024-05-07T02:29:09.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.948+0000 7f0693400700 1 -- start start 2024-05-07T02:29:09.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.949+0000 7f0693400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f068c1054f0 0x7f068c1031e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:09.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.949+0000 7f0693400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f068c105e40 0x7f068c103720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:09.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.949+0000 7f0693400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f068c10e920 0x7f068c103c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:09.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.949+0000 7f0693400700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f068c1144d0 con 0x7f068c10e920 2024-05-07T02:29:09.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.949+0000 7f0693400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f068c114350 con 0x7f068c1054f0 2024-05-07T02:29:09.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.949+0000 7f0693400700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f068c114650 con 0x7f068c105e40 2024-05-07T02:29:09.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.949+0000 7f0691400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f068c10e920 0x7f068c103c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:09.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.949+0000 7f0691400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f068c10e920 0x7f068c103c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57274/0 (socket says 172.21.15.19:57274) 2024-05-07T02:29:09.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.949+0000 7f0691400700 1 -- 172.21.15.19:0/406895036 learned_addr learned my addr 172.21.15.19:0/406895036 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:29:09.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.949+0000 7f068be00700 1 --2- 172.21.15.19:0/406895036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f068c105e40 0x7f068c103720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:09.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.950+0000 7f0690a00700 1 --2- 172.21.15.19:0/406895036 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f068c1054f0 0x7f068c1031e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:09.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.950+0000 7f0691400700 1 -- 172.21.15.19:0/406895036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f068c105e40 msgr2=0x7f068c103720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:09.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.950+0000 7f0691400700 1 --2- 172.21.15.19:0/406895036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f068c105e40 0x7f068c103720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:09.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.950+0000 7f0691400700 1 -- 172.21.15.19:0/406895036 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f068c1054f0 msgr2=0x7f068c1031e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:09.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.950+0000 7f0691400700 1 --2- 172.21.15.19:0/406895036 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f068c1054f0 0x7f068c1031e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:09.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.950+0000 7f0691400700 1 -- 172.21.15.19:0/406895036 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f068c1a91e0 con 0x7f068c10e920 2024-05-07T02:29:09.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.951+0000 7f0691400700 1 --2- 172.21.15.19:0/406895036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f068c10e920 0x7f068c103c60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f068400b070 tx=0x7f0684009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:09.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.951+0000 7f0689600700 1 -- 172.21.15.19:0/406895036 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0684021070 con 0x7f068c10e920 2024-05-07T02:29:09.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.951+0000 7f0693400700 1 -- 172.21.15.19:0/406895036 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f068c1a9470 con 0x7f068c10e920 2024-05-07T02:29:09.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.951+0000 7f0689600700 1 -- 172.21.15.19:0/406895036 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0684007e30 con 0x7f068c10e920 2024-05-07T02:29:09.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.951+0000 7f0693400700 1 -- 172.21.15.19:0/406895036 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f068c1a9a00 con 0x7f068c10e920 2024-05-07T02:29:09.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.952+0000 7f0689600700 1 -- 172.21.15.19:0/406895036 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06840046a0 con 0x7f068c10e920 2024-05-07T02:29:09.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.952+0000 7f0693400700 1 -- 172.21.15.19:0/406895036 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f068c063ed0 con 0x7f068c10e920 2024-05-07T02:29:09.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.956+0000 7f0689600700 1 -- 172.21.15.19:0/406895036 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f0684007980 con 0x7f068c10e920 2024-05-07T02:29:09.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.958+0000 7f0689600700 1 --2- 172.21.15.19:0/406895036 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f06740b2910 0x7f06740b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:09.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.958+0000 7f0689600700 1 -- 172.21.15.19:0/406895036 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f068400f020 con 0x7f068c10e920 2024-05-07T02:29:09.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.958+0000 7f0689600700 1 --2- 172.21.15.19:0/406895036 >> [v2:172.21.15.19:6818/2158875543,v1:172.21.15.19:6819/2158875543] conn(0x7f06740bdb10 0x7f06740c0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:09.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.958+0000 7f0690a00700 1 --2- 172.21.15.19:0/406895036 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f06740b2910 0x7f06740b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:09.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.958+0000 7f0689600700 1 -- 172.21.15.19:0/406895036 --> [v2:172.21.15.19:6818/2158875543,v1:172.21.15.19:6819/2158875543] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f06740c06d0 con 0x7f06740bdb10 2024-05-07T02:29:09.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.958+0000 7f0689600700 1 -- 172.21.15.19:0/406895036 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f06840e43b0 con 0x7f068c10e920 2024-05-07T02:29:09.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.959+0000 7f068be00700 1 --2- 172.21.15.19:0/406895036 >> [v2:172.21.15.19:6818/2158875543,v1:172.21.15.19:6819/2158875543] conn(0x7f06740bdb10 0x7f06740c0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:09.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.959+0000 7f0690a00700 1 --2- 172.21.15.19:0/406895036 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f06740b2910 0x7f06740b4e20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0680009680 tx=0x7f0680009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:09.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.959+0000 7f068be00700 1 --2- 172.21.15.19:0/406895036 >> [v2:172.21.15.19:6818/2158875543,v1:172.21.15.19:6819/2158875543] conn(0x7f06740bdb10 0x7f06740c0000 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:09.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.961+0000 7f0689600700 1 -- 172.21.15.19:0/406895036 <== osd.6 v2:172.21.15.19:6818/2158875543 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f06740c06d0 con 0x7f06740bdb10 2024-05-07T02:29:09.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.979+0000 7f0693400700 1 -- 172.21.15.19:0/406895036 --> [v2:172.21.15.19:6818/2158875543,v1:172.21.15.19:6819/2158875543] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f068c0642d0 con 0x7f06740bdb10 2024-05-07T02:29:09.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.980+0000 7f0689600700 1 -- 172.21.15.19:0/406895036 <== osd.6 v2:172.21.15.19:6818/2158875543 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f068c0642d0 con 0x7f06740bdb10 2024-05-07T02:29:09.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.980+0000 7f0693400700 1 -- 172.21.15.19:0/406895036 >> [v2:172.21.15.19:6818/2158875543,v1:172.21.15.19:6819/2158875543] conn(0x7f06740bdb10 msgr2=0x7f06740c0000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:09.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.980+0000 7f0693400700 1 --2- 172.21.15.19:0/406895036 >> [v2:172.21.15.19:6818/2158875543,v1:172.21.15.19:6819/2158875543] conn(0x7f06740bdb10 0x7f06740c0000 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:09.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.980+0000 7f0693400700 1 -- 172.21.15.19:0/406895036 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f06740b2910 msgr2=0x7f06740b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:09.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.980+0000 7f0693400700 1 --2- 172.21.15.19:0/406895036 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f06740b2910 0x7f06740b4e20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0680009680 tx=0x7f0680009260 comp rx=0 tx=0).stop 2024-05-07T02:29:09.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.980+0000 7f0693400700 1 -- 172.21.15.19:0/406895036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f068c10e920 msgr2=0x7f068c103c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:09.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.980+0000 7f0693400700 1 --2- 172.21.15.19:0/406895036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f068c10e920 0x7f068c103c60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f068400b070 tx=0x7f0684009c90 comp rx=0 tx=0).stop 2024-05-07T02:29:09.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.980+0000 7f0691400700 1 -- 172.21.15.19:0/406895036 reap_dead start 2024-05-07T02:29:09.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.980+0000 7f0693400700 1 -- 172.21.15.19:0/406895036 shutdown_connections 2024-05-07T02:29:09.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.980+0000 7f0693400700 1 -- 172.21.15.19:0/406895036 >> 172.21.15.19:0/406895036 conn(0x7f068c078e00 msgr2=0x7f068c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:09.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.981+0000 7f0693400700 1 -- 172.21.15.19:0/406895036 shutdown_connections 2024-05-07T02:29:09.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:09.981+0000 7f0693400700 1 -- 172.21.15.19:0/406895036 wait complete. 2024-05-07T02:29:10.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.209+0000 7fc72ee00700 1 -- 172.21.15.19:0/2202540011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc728103080 msgr2=0x7fc728103490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:10.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.209+0000 7fc72ee00700 1 --2- 172.21.15.19:0/2202540011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc728103080 0x7fc728103490 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc718009990 tx=0x7fc718009ca0 comp rx=0 tx=0).stop 2024-05-07T02:29:10.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.209+0000 7fc72ee00700 1 -- 172.21.15.19:0/2202540011 shutdown_connections 2024-05-07T02:29:10.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.209+0000 7fc72ee00700 1 --2- 172.21.15.19:0/2202540011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc728110c30 0x7fc728113050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:10.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.209+0000 7fc72ee00700 1 --2- 172.21.15.19:0/2202540011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc728103080 0x7fc728103490 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:10.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.209+0000 7fc72ee00700 1 --2- 172.21.15.19:0/2202540011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc728103a60 0x7fc7281106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:10.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.209+0000 7fc72ee00700 1 -- 172.21.15.19:0/2202540011 >> 172.21.15.19:0/2202540011 conn(0x7fc7280fedc0 msgr2=0x7fc7281011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:10.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.209+0000 7fc72ee00700 1 -- 172.21.15.19:0/2202540011 shutdown_connections 2024-05-07T02:29:10.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.209+0000 7fc72ee00700 1 -- 172.21.15.19:0/2202540011 wait complete. 2024-05-07T02:29:10.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.210+0000 7fc72ee00700 1 Processor -- start 2024-05-07T02:29:10.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.210+0000 7fc72ee00700 1 -- start start 2024-05-07T02:29:10.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.211+0000 7fc72ee00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc728103080 0x7fc72819d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:10.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.211+0000 7fc72ee00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc728103a60 0x7fc72819d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:10.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.211+0000 7fc72ee00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc728110c30 0x7fc7281a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:10.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.211+0000 7fc72ee00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc728116680 con 0x7fc728103080 2024-05-07T02:29:10.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.211+0000 7fc72ee00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc728116500 con 0x7fc728110c30 2024-05-07T02:29:10.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.211+0000 7fc72ee00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc728116800 con 0x7fc728103a60 2024-05-07T02:29:10.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.211+0000 7fc727e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc728103080 0x7fc72819d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:10.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.211+0000 7fc727400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc728103a60 0x7fc72819d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:10.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.211+0000 7fc72ce00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc728110c30 0x7fc7281a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:10.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.211+0000 7fc727400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc728103a60 0x7fc72819d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:54048/0 (socket says 172.21.15.19:54048) 2024-05-07T02:29:10.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.211+0000 7fc727400700 1 -- 172.21.15.19:0/746139951 learned_addr learned my addr 172.21.15.19:0/746139951 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:29:10.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.211+0000 7fc727e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc728103080 0x7fc72819d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51598/0 (socket says 172.21.15.19:51598) 2024-05-07T02:29:10.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.212+0000 7fc72ce00700 1 -- 172.21.15.19:0/746139951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc728103a60 msgr2=0x7fc72819d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:10.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.212+0000 7fc72ce00700 1 --2- 172.21.15.19:0/746139951 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc728103a60 0x7fc72819d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:10.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.212+0000 7fc72ce00700 1 -- 172.21.15.19:0/746139951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc728103080 msgr2=0x7fc72819d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:10.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.212+0000 7fc72ce00700 1 --2- 172.21.15.19:0/746139951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc728103080 0x7fc72819d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:10.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.212+0000 7fc72ce00700 1 -- 172.21.15.19:0/746139951 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7281a1ee0 con 0x7fc728110c30 2024-05-07T02:29:10.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.212+0000 7fc727e00700 1 --2- 172.21.15.19:0/746139951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc728103080 0x7fc72819d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:29:10.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.212+0000 7fc72ce00700 1 --2- 172.21.15.19:0/746139951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc728110c30 0x7fc7281a1840 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc72000efc0 tx=0x7fc72000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:10.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.212+0000 7fc724c00700 1 -- 172.21.15.19:0/746139951 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7200092e0 con 0x7fc728110c30 2024-05-07T02:29:10.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.212+0000 7fc72ee00700 1 -- 172.21.15.19:0/746139951 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7281a2170 con 0x7fc728110c30 2024-05-07T02:29:10.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.212+0000 7fc724c00700 1 -- 172.21.15.19:0/746139951 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc720004bb0 con 0x7fc728110c30 2024-05-07T02:29:10.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.212+0000 7fc724c00700 1 -- 172.21.15.19:0/746139951 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc720004020 con 0x7fc728110c30 2024-05-07T02:29:10.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.213+0000 7fc72ee00700 1 -- 172.21.15.19:0/746139951 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7281a9af0 con 0x7fc728110c30 2024-05-07T02:29:10.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.213+0000 7fc72ee00700 1 -- 172.21.15.19:0/746139951 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc728063f10 con 0x7fc728110c30 2024-05-07T02:29:10.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.216+0000 7fc724c00700 1 -- 172.21.15.19:0/746139951 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fc72001b030 con 0x7fc728110c30 2024-05-07T02:29:10.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.218+0000 7fc724c00700 1 --2- 172.21.15.19:0/746139951 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fc70c0b2910 0x7fc70c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:10.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.218+0000 7fc724c00700 1 -- 172.21.15.19:0/746139951 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fc7200e2bb0 con 0x7fc728110c30 2024-05-07T02:29:10.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.218+0000 7fc727e00700 1 --2- 172.21.15.19:0/746139951 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fc70c0b2910 0x7fc70c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:10.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.218+0000 7fc724c00700 1 --2- 172.21.15.19:0/746139951 >> [v2:172.21.15.19:6826/4124921646,v1:172.21.15.19:6827/4124921646] conn(0x7fc70c0bdc40 0x7fc70c0c0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:10.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.218+0000 7fc724c00700 1 -- 172.21.15.19:0/746139951 --> [v2:172.21.15.19:6826/4124921646,v1:172.21.15.19:6827/4124921646] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc72000be50 con 0x7fc70c0bdc40 2024-05-07T02:29:10.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.218+0000 7fc724c00700 1 -- 172.21.15.19:0/746139951 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fc7200e2f60 con 0x7fc728110c30 2024-05-07T02:29:10.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.219+0000 7fc727400700 1 --2- 172.21.15.19:0/746139951 >> [v2:172.21.15.19:6826/4124921646,v1:172.21.15.19:6827/4124921646] conn(0x7fc70c0bdc40 0x7fc70c0c0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:10.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.219+0000 7fc727e00700 1 --2- 172.21.15.19:0/746139951 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fc70c0b2910 0x7fc70c0b4e20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc71800b2e0 tx=0x7fc718005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:10.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.219+0000 7fc727400700 1 --2- 172.21.15.19:0/746139951 >> [v2:172.21.15.19:6826/4124921646,v1:172.21.15.19:6827/4124921646] conn(0x7fc70c0bdc40 0x7fc70c0c0130 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:10.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.220+0000 7fc724c00700 1 -- 172.21.15.19:0/746139951 <== osd.9 v2:172.21.15.19:6826/4124921646 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc72000be50 con 0x7fc70c0bdc40 2024-05-07T02:29:10.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.239+0000 7fc72ee00700 1 -- 172.21.15.19:0/746139951 --> [v2:172.21.15.19:6826/4124921646,v1:172.21.15.19:6827/4124921646] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc728064390 con 0x7fc70c0bdc40 2024-05-07T02:29:10.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.240+0000 7fc724c00700 1 -- 172.21.15.19:0/746139951 <== osd.9 v2:172.21.15.19:6826/4124921646 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc728064390 con 0x7fc70c0bdc40 2024-05-07T02:29:10.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.240+0000 7fc72ee00700 1 -- 172.21.15.19:0/746139951 >> [v2:172.21.15.19:6826/4124921646,v1:172.21.15.19:6827/4124921646] conn(0x7fc70c0bdc40 msgr2=0x7fc70c0c0130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:10.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.240+0000 7fc72ee00700 1 --2- 172.21.15.19:0/746139951 >> [v2:172.21.15.19:6826/4124921646,v1:172.21.15.19:6827/4124921646] conn(0x7fc70c0bdc40 0x7fc70c0c0130 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:10.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.240+0000 7fc72ee00700 1 -- 172.21.15.19:0/746139951 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fc70c0b2910 msgr2=0x7fc70c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:10.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.240+0000 7fc72ee00700 1 --2- 172.21.15.19:0/746139951 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fc70c0b2910 0x7fc70c0b4e20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc71800b2e0 tx=0x7fc718005960 comp rx=0 tx=0).stop 2024-05-07T02:29:10.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.240+0000 7fc72ee00700 1 -- 172.21.15.19:0/746139951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc728110c30 msgr2=0x7fc7281a1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:10.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.240+0000 7fc72ee00700 1 --2- 172.21.15.19:0/746139951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc728110c30 0x7fc7281a1840 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc72000efc0 tx=0x7fc72000c3a0 comp rx=0 tx=0).stop 2024-05-07T02:29:10.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.240+0000 7fc72ce00700 1 -- 172.21.15.19:0/746139951 reap_dead start 2024-05-07T02:29:10.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.240+0000 7fc72ee00700 1 -- 172.21.15.19:0/746139951 shutdown_connections 2024-05-07T02:29:10.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.240+0000 7fc72ee00700 1 -- 172.21.15.19:0/746139951 >> 172.21.15.19:0/746139951 conn(0x7fc7280fedc0 msgr2=0x7fc728112800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:10.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.241+0000 7fc72ee00700 1 -- 172.21.15.19:0/746139951 shutdown_connections 2024-05-07T02:29:10.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:07.241+0000 7fc72ee00700 1 -- 172.21.15.19:0/746139951 wait complete. 2024-05-07T02:29:11.100 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:10 smithi019 ceph-mon[29781]: pgmap v428: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:29:11.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:10 smithi064 ceph-mon[34699]: pgmap v428: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:29:11.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.161+0000 7f4129800700 1 -- 172.21.15.19:0/1112736581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4124108f20 msgr2=0x7f4124109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:11.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.161+0000 7f4129800700 1 --2- 172.21.15.19:0/1112736581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4124108f20 0x7f4124109330 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f410c009920 tx=0x7f410c009c30 comp rx=0 tx=0).stop 2024-05-07T02:29:11.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.162+0000 7f4129800700 1 -- 172.21.15.19:0/1112736581 shutdown_connections 2024-05-07T02:29:11.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.162+0000 7f4129800700 1 --2- 172.21.15.19:0/1112736581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4124109870 0x7f41241111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:11.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.162+0000 7f4129800700 1 --2- 172.21.15.19:0/1112736581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41241045f0 0x7f4124104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:11.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.162+0000 7f4129800700 1 --2- 172.21.15.19:0/1112736581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4124108f20 0x7f4124109330 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:11.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.162+0000 7f4129800700 1 -- 172.21.15.19:0/1112736581 >> 172.21.15.19:0/1112736581 conn(0x7f4124100040 msgr2=0x7f4124102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:11.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.162+0000 7f4129800700 1 -- 172.21.15.19:0/1112736581 shutdown_connections 2024-05-07T02:29:11.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.162+0000 7f4129800700 1 -- 172.21.15.19:0/1112736581 wait complete. 2024-05-07T02:29:11.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.164+0000 7f4129800700 1 Processor -- start 2024-05-07T02:29:11.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.164+0000 7f4129800700 1 -- start start 2024-05-07T02:29:11.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.165+0000 7f4129800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41241045f0 0x7f4124075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:11.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.165+0000 7f4129800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4124108f20 0x7f4124076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:11.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.165+0000 7f4129800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4124109870 0x7f4124076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:11.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.165+0000 7f4129800700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f41241144d0 con 0x7f4124109870 2024-05-07T02:29:11.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.165+0000 7f4129800700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4124114350 con 0x7f4124108f20 2024-05-07T02:29:11.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.165+0000 7f4129800700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4124114650 con 0x7f41241045f0 2024-05-07T02:29:11.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.165+0000 7f411a000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4124108f20 0x7f4124076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:11.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.165+0000 7f4122a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41241045f0 0x7f4124075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:11.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.165+0000 7f411a000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4124108f20 0x7f4124076000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:45284/0 (socket says 172.21.15.19:45284) 2024-05-07T02:29:11.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.165+0000 7f411a000700 1 -- 172.21.15.19:0/3206292189 learned_addr learned my addr 172.21.15.19:0/3206292189 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:29:11.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.166+0000 7f4123400700 1 --2- 172.21.15.19:0/3206292189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4124109870 0x7f4124076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:11.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.166+0000 7f411a000700 1 -- 172.21.15.19:0/3206292189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41241045f0 msgr2=0x7f4124075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:11.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.166+0000 7f411a000700 1 --2- 172.21.15.19:0/3206292189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f41241045f0 0x7f4124075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:11.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.166+0000 7f411a000700 1 -- 172.21.15.19:0/3206292189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4124109870 msgr2=0x7f4124076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:11.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.166+0000 7f411a000700 1 --2- 172.21.15.19:0/3206292189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4124109870 0x7f4124076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:11.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.166+0000 7f411a000700 1 -- 172.21.15.19:0/3206292189 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41241a9360 con 0x7f4124108f20 2024-05-07T02:29:11.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.166+0000 7f411a000700 1 --2- 172.21.15.19:0/3206292189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4124108f20 0x7f4124076000 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f410c000c00 tx=0x7f410c00f950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:11.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.167+0000 7f411be00700 1 -- 172.21.15.19:0/3206292189 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410c0050c0 con 0x7f4124108f20 2024-05-07T02:29:11.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.167+0000 7f411be00700 1 -- 172.21.15.19:0/3206292189 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f410c00b560 con 0x7f4124108f20 2024-05-07T02:29:11.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.167+0000 7f4129800700 1 -- 172.21.15.19:0/3206292189 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41241a95f0 con 0x7f4124108f20 2024-05-07T02:29:11.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.167+0000 7f411be00700 1 -- 172.21.15.19:0/3206292189 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410c003970 con 0x7f4124108f20 2024-05-07T02:29:11.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.167+0000 7f4129800700 1 -- 172.21.15.19:0/3206292189 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41241a9a40 con 0x7f4124108f20 2024-05-07T02:29:11.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.168+0000 7f4129800700 1 -- 172.21.15.19:0/3206292189 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4124063e90 con 0x7f4124108f20 2024-05-07T02:29:11.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.170+0000 7f411be00700 1 -- 172.21.15.19:0/3206292189 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f410c005260 con 0x7f4124108f20 2024-05-07T02:29:11.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.172+0000 7f411be00700 1 --2- 172.21.15.19:0/3206292189 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f41080b2990 0x7f41080b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:11.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.172+0000 7f411be00700 1 -- 172.21.15.19:0/3206292189 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f410c0e3ac0 con 0x7f4124108f20 2024-05-07T02:29:11.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.172+0000 7f4122a00700 1 --2- 172.21.15.19:0/3206292189 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f41080b2990 0x7f41080b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:11.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.172+0000 7f411be00700 1 --2- 172.21.15.19:0/3206292189 >> [v2:172.21.15.27:6824/1428931817,v1:172.21.15.27:6825/1428931817] conn(0x7f41080bdcc0 0x7f41080c01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:11.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.173+0000 7f411be00700 1 -- 172.21.15.19:0/3206292189 --> [v2:172.21.15.27:6824/1428931817,v1:172.21.15.27:6825/1428931817] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f41080c0880 con 0x7f41080bdcc0 2024-05-07T02:29:11.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.173+0000 7f411be00700 1 -- 172.21.15.19:0/3206292189 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f410c0e3e70 con 0x7f4124108f20 2024-05-07T02:29:11.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.173+0000 7f4123400700 1 --2- 172.21.15.19:0/3206292189 >> [v2:172.21.15.27:6824/1428931817,v1:172.21.15.27:6825/1428931817] conn(0x7f41080bdcc0 0x7f41080c01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:11.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.173+0000 7f4122a00700 1 --2- 172.21.15.19:0/3206292189 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f41080b2990 0x7f41080b4ea0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4114009c60 tx=0x7f4114009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:11.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.174+0000 7f4123400700 1 --2- 172.21.15.19:0/3206292189 >> [v2:172.21.15.27:6824/1428931817,v1:172.21.15.27:6825/1428931817] conn(0x7f41080bdcc0 0x7f41080c01b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:11.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.175+0000 7f411be00700 1 -- 172.21.15.19:0/3206292189 <== osd.10 v2:172.21.15.27:6824/1428931817 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f41080c0880 con 0x7f41080bdcc0 2024-05-07T02:29:11.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.194+0000 7f4129800700 1 -- 172.21.15.19:0/3206292189 --> [v2:172.21.15.27:6824/1428931817,v1:172.21.15.27:6825/1428931817] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4124064290 con 0x7f41080bdcc0 2024-05-07T02:29:11.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.194+0000 7f411be00700 1 -- 172.21.15.19:0/3206292189 <== osd.10 v2:172.21.15.27:6824/1428931817 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4124064290 con 0x7f41080bdcc0 2024-05-07T02:29:11.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.195+0000 7f4129800700 1 -- 172.21.15.19:0/3206292189 >> [v2:172.21.15.27:6824/1428931817,v1:172.21.15.27:6825/1428931817] conn(0x7f41080bdcc0 msgr2=0x7f41080c01b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:11.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.195+0000 7f4129800700 1 --2- 172.21.15.19:0/3206292189 >> [v2:172.21.15.27:6824/1428931817,v1:172.21.15.27:6825/1428931817] conn(0x7f41080bdcc0 0x7f41080c01b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:11.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.195+0000 7f4129800700 1 -- 172.21.15.19:0/3206292189 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f41080b2990 msgr2=0x7f41080b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:11.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.195+0000 7f4129800700 1 --2- 172.21.15.19:0/3206292189 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f41080b2990 0x7f41080b4ea0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4114009c60 tx=0x7f4114009260 comp rx=0 tx=0).stop 2024-05-07T02:29:11.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.195+0000 7f4129800700 1 -- 172.21.15.19:0/3206292189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4124108f20 msgr2=0x7f4124076000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:11.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.195+0000 7f4129800700 1 --2- 172.21.15.19:0/3206292189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4124108f20 0x7f4124076000 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f410c000c00 tx=0x7f410c00f950 comp rx=0 tx=0).stop 2024-05-07T02:29:11.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.195+0000 7f4123400700 1 -- 172.21.15.19:0/3206292189 reap_dead start 2024-05-07T02:29:11.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.195+0000 7f4129800700 1 -- 172.21.15.19:0/3206292189 shutdown_connections 2024-05-07T02:29:11.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.195+0000 7f4129800700 1 -- 172.21.15.19:0/3206292189 >> 172.21.15.19:0/3206292189 conn(0x7f4124100040 msgr2=0x7f4124108720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:11.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.196+0000 7f4129800700 1 -- 172.21.15.19:0/3206292189 shutdown_connections 2024-05-07T02:29:11.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:11.196+0000 7f4129800700 1 -- 172.21.15.19:0/3206292189 wait complete. 2024-05-07T02:29:11.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:10 smithi027 ceph-mon[34398]: pgmap v428: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:29:12.094 INFO:teuthology.orchestra.run.smithi019.stdout:154618822745 2024-05-07T02:29:12.095 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-07T02:29:12.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.502+0000 7f5b5e800700 1 -- 172.21.15.19:0/4265249689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b58109850 msgr2=0x7f5b581111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:12.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.502+0000 7f5b5e800700 1 --2- 172.21.15.19:0/4265249689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b58109850 0x7f5b581111b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5b5000b0d0 tx=0x7f5b5000b3e0 comp rx=0 tx=0).stop 2024-05-07T02:29:12.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.503+0000 7f5b5e800700 1 -- 172.21.15.19:0/4265249689 shutdown_connections 2024-05-07T02:29:12.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.503+0000 7f5b5e800700 1 --2- 172.21.15.19:0/4265249689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b58109850 0x7f5b581111b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:12.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.503+0000 7f5b5e800700 1 --2- 172.21.15.19:0/4265249689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b581045d0 0x7f5b581049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:12.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.503+0000 7f5b5e800700 1 --2- 172.21.15.19:0/4265249689 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b58108f00 0x7f5b58109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:12.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.503+0000 7f5b5e800700 1 -- 172.21.15.19:0/4265249689 >> 172.21.15.19:0/4265249689 conn(0x7f5b580ffea0 msgr2=0x7f5b581022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:12.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.503+0000 7f5b5e800700 1 -- 172.21.15.19:0/4265249689 shutdown_connections 2024-05-07T02:29:12.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.503+0000 7f5b5e800700 1 -- 172.21.15.19:0/4265249689 wait complete. 2024-05-07T02:29:12.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.504+0000 7f5b5e800700 1 Processor -- start 2024-05-07T02:29:12.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.504+0000 7f5b5e800700 1 -- start start 2024-05-07T02:29:12.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.505+0000 7f5b5e800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b581045d0 0x7f5b5819ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:12.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.505+0000 7f5b5e800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b58108f00 0x7f5b5819b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:12.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.505+0000 7f5b5e800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b58109850 0x7f5b5819f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:12.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.505+0000 7f5b5e800700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5b581144d0 con 0x7f5b58108f00 2024-05-07T02:29:12.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.505+0000 7f5b5e800700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5b58114350 con 0x7f5b581045d0 2024-05-07T02:29:12.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.505+0000 7f5b5e800700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5b58114650 con 0x7f5b58109850 2024-05-07T02:29:12.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.506+0000 7f5b57400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b581045d0 0x7f5b5819ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:12.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.506+0000 7f5b57400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b581045d0 0x7f5b5819ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:45302/0 (socket says 172.21.15.19:45302) 2024-05-07T02:29:12.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.506+0000 7f5b57400700 1 -- 172.21.15.19:0/3074866742 learned_addr learned my addr 172.21.15.19:0/3074866742 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:29:12.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.506+0000 7f5b57e00700 1 --2- 172.21.15.19:0/3074866742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b58109850 0x7f5b5819f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:12.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.506+0000 7f5b57400700 1 -- 172.21.15.19:0/3074866742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b58109850 msgr2=0x7f5b5819f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:12.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.506+0000 7f5b57400700 1 --2- 172.21.15.19:0/3074866742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5b58109850 0x7f5b5819f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:12.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.506+0000 7f5b57400700 1 -- 172.21.15.19:0/3074866742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b58108f00 msgr2=0x7f5b5819b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:12.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.506+0000 7f5b57400700 1 --2- 172.21.15.19:0/3074866742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b58108f00 0x7f5b5819b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:12.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.506+0000 7f5b57400700 1 -- 172.21.15.19:0/3074866742 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b5819fcd0 con 0x7f5b581045d0 2024-05-07T02:29:12.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.507+0000 7f5b57400700 1 --2- 172.21.15.19:0/3074866742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b581045d0 0x7f5b5819ae00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800e9a0 tx=0x7f5b4800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:12.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.507+0000 7f5b4fe00700 1 -- 172.21.15.19:0/3074866742 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4800cbd0 con 0x7f5b581045d0 2024-05-07T02:29:12.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.507+0000 7f5b4fe00700 1 -- 172.21.15.19:0/3074866742 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b4800cd70 con 0x7f5b581045d0 2024-05-07T02:29:12.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.507+0000 7f5b4fe00700 1 -- 172.21.15.19:0/3074866742 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b48018680 con 0x7f5b581045d0 2024-05-07T02:29:12.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.508+0000 7f5b5e800700 1 -- 172.21.15.19:0/3074866742 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b5819ffc0 con 0x7f5b581045d0 2024-05-07T02:29:12.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.508+0000 7f5b5e800700 1 -- 172.21.15.19:0/3074866742 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b581a78e0 con 0x7f5b581045d0 2024-05-07T02:29:12.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.509+0000 7f5b5e800700 1 -- 172.21.15.19:0/3074866742 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5b58063f50 con 0x7f5b581045d0 2024-05-07T02:29:12.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.512+0000 7f5b4fe00700 1 -- 172.21.15.19:0/3074866742 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f5b48010430 con 0x7f5b581045d0 2024-05-07T02:29:12.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.515+0000 7f5b4fe00700 1 --2- 172.21.15.19:0/3074866742 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f5b3c0b2990 0x7f5b3c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:12.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.515+0000 7f5b4fe00700 1 -- 172.21.15.19:0/3074866742 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f5b480e36a0 con 0x7f5b581045d0 2024-05-07T02:29:12.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.515+0000 7f5b56a00700 1 --2- 172.21.15.19:0/3074866742 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f5b3c0b2990 0x7f5b3c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:12.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.515+0000 7f5b4fe00700 1 --2- 172.21.15.19:0/3074866742 >> [v2:172.21.15.27:6808/3662186068,v1:172.21.15.27:6809/3662186068] conn(0x7f5b3c0bdcc0 0x7f5b3c0c01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:12.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.515+0000 7f5b4fe00700 1 -- 172.21.15.19:0/3074866742 --> [v2:172.21.15.27:6808/3662186068,v1:172.21.15.27:6809/3662186068] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5b48016ec0 con 0x7f5b3c0bdcc0 2024-05-07T02:29:12.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.515+0000 7f5b4fe00700 1 -- 172.21.15.19:0/3074866742 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f5b480e3a50 con 0x7f5b581045d0 2024-05-07T02:29:12.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.516+0000 7f5b57e00700 1 --2- 172.21.15.19:0/3074866742 >> [v2:172.21.15.27:6808/3662186068,v1:172.21.15.27:6809/3662186068] conn(0x7f5b3c0bdcc0 0x7f5b3c0c01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:12.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.516+0000 7f5b56a00700 1 --2- 172.21.15.19:0/3074866742 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f5b3c0b2990 0x7f5b3c0b4ea0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5b58107d00 tx=0x7f5b44009620 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:12.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.516+0000 7f5b57e00700 1 --2- 172.21.15.19:0/3074866742 >> [v2:172.21.15.27:6808/3662186068,v1:172.21.15.27:6809/3662186068] conn(0x7f5b3c0bdcc0 0x7f5b3c0c01b0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:12.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.518+0000 7f5b4fe00700 1 -- 172.21.15.19:0/3074866742 <== osd.4 v2:172.21.15.27:6808/3662186068 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5b48016ec0 con 0x7f5b3c0bdcc0 2024-05-07T02:29:12.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.538+0000 7f5b5e800700 1 -- 172.21.15.19:0/3074866742 --> [v2:172.21.15.27:6808/3662186068,v1:172.21.15.27:6809/3662186068] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5b580643a0 con 0x7f5b3c0bdcc0 2024-05-07T02:29:12.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.539+0000 7f5b4fe00700 1 -- 172.21.15.19:0/3074866742 <== osd.4 v2:172.21.15.27:6808/3662186068 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5b580643a0 con 0x7f5b3c0bdcc0 2024-05-07T02:29:12.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.539+0000 7f5b5e800700 1 -- 172.21.15.19:0/3074866742 >> [v2:172.21.15.27:6808/3662186068,v1:172.21.15.27:6809/3662186068] conn(0x7f5b3c0bdcc0 msgr2=0x7f5b3c0c01b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:12.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.539+0000 7f5b5e800700 1 --2- 172.21.15.19:0/3074866742 >> [v2:172.21.15.27:6808/3662186068,v1:172.21.15.27:6809/3662186068] conn(0x7f5b3c0bdcc0 0x7f5b3c0c01b0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:12.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.539+0000 7f5b5e800700 1 -- 172.21.15.19:0/3074866742 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f5b3c0b2990 msgr2=0x7f5b3c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:12.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.539+0000 7f5b5e800700 1 --2- 172.21.15.19:0/3074866742 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f5b3c0b2990 0x7f5b3c0b4ea0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5b58107d00 tx=0x7f5b44009620 comp rx=0 tx=0).stop 2024-05-07T02:29:12.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.539+0000 7f5b5e800700 1 -- 172.21.15.19:0/3074866742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b581045d0 msgr2=0x7f5b5819ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:12.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.539+0000 7f5b5e800700 1 --2- 172.21.15.19:0/3074866742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b581045d0 0x7f5b5819ae00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800e9a0 tx=0x7f5b4800ecb0 comp rx=0 tx=0).stop 2024-05-07T02:29:12.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.539+0000 7f5b57e00700 1 -- 172.21.15.19:0/3074866742 reap_dead start 2024-05-07T02:29:12.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.540+0000 7f5b5e800700 1 -- 172.21.15.19:0/3074866742 shutdown_connections 2024-05-07T02:29:12.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.540+0000 7f5b5e800700 1 -- 172.21.15.19:0/3074866742 >> 172.21.15.19:0/3074866742 conn(0x7f5b580ffea0 msgr2=0x7f5b58108670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:12.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.540+0000 7f5b5e800700 1 -- 172.21.15.19:0/3074866742 shutdown_connections 2024-05-07T02:29:12.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:12.540+0000 7f5b5e800700 1 -- 172.21.15.19:0/3074866742 wait complete. 2024-05-07T02:29:12.947 INFO:teuthology.orchestra.run.smithi019.stdout:253403070528 2024-05-07T02:29:12.947 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-07T02:29:13.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:12 smithi064 ceph-mon[34699]: pgmap v429: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:29:13.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:12 smithi019 ceph-mon[29781]: pgmap v429: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:29:13.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:12 smithi027 ceph-mon[34398]: pgmap v429: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:29:13.768 INFO:teuthology.orchestra.run.smithi019.stdout:55834574960 2024-05-07T02:29:13.769 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-07T02:29:14.081 INFO:teuthology.orchestra.run.smithi019.stdout:274877907003 2024-05-07T02:29:14.081 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-07T02:29:14.655 INFO:teuthology.orchestra.run.smithi019.stdout:180388626514 2024-05-07T02:29:14.655 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-07T02:29:14.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.416+0000 7fa3bba00700 1 -- 172.21.15.19:0/3290843597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3b40764b0 msgr2=0x7fa3b4114e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:14.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.416+0000 7fa3bba00700 1 --2- 172.21.15.19:0/3290843597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3b40764b0 0x7fa3b4114e30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa39c0098f0 tx=0x7fa39c009c00 comp rx=0 tx=0).stop 2024-05-07T02:29:14.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.416+0000 7fa3bba00700 1 -- 172.21.15.19:0/3290843597 shutdown_connections 2024-05-07T02:29:14.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.416+0000 7fa3bba00700 1 --2- 172.21.15.19:0/3290843597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3b4115370 0x7fa3b4117790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:14.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.416+0000 7fa3bba00700 1 --2- 172.21.15.19:0/3290843597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b4075b60 0x7fa3b4075f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:14.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.416+0000 7fa3bba00700 1 --2- 172.21.15.19:0/3290843597 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3b40764b0 0x7fa3b4114e30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:14.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.416+0000 7fa3bba00700 1 -- 172.21.15.19:0/3290843597 >> 172.21.15.19:0/3290843597 conn(0x7fa3b4100040 msgr2=0x7fa3b4102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:14.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.416+0000 7fa3bba00700 1 -- 172.21.15.19:0/3290843597 shutdown_connections 2024-05-07T02:29:14.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.416+0000 7fa3bba00700 1 -- 172.21.15.19:0/3290843597 wait complete. 2024-05-07T02:29:14.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.417+0000 7fa3bba00700 1 Processor -- start 2024-05-07T02:29:14.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.417+0000 7fa3bba00700 1 -- start start 2024-05-07T02:29:14.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.418+0000 7fa3bba00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3b4075b60 0x7fa3b41a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:14.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.418+0000 7fa3bba00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b40764b0 0x7fa3b41a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:14.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.418+0000 7fa3bba00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3b4115370 0x7fa3b41a5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:14.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.418+0000 7fa3bba00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b411aac0 con 0x7fa3b40764b0 2024-05-07T02:29:14.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.418+0000 7fa3bba00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b411a940 con 0x7fa3b4075b60 2024-05-07T02:29:14.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.418+0000 7fa3bba00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b411ac40 con 0x7fa3b4115370 2024-05-07T02:29:14.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.418+0000 7fa3abe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b40764b0 0x7fa3b41a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:14.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.418+0000 7fa3b9000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3b4075b60 0x7fa3b41a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:14.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.418+0000 7fa3abe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b40764b0 0x7fa3b41a1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57320/0 (socket says 172.21.15.19:57320) 2024-05-07T02:29:14.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.418+0000 7fa3abe00700 1 -- 172.21.15.19:0/247840718 learned_addr learned my addr 172.21.15.19:0/247840718 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:29:14.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.419+0000 7fa3b9a00700 1 --2- 172.21.15.19:0/247840718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3b4115370 0x7fa3b41a5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:14.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.419+0000 7fa3abe00700 1 -- 172.21.15.19:0/247840718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3b4115370 msgr2=0x7fa3b41a5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:14.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.419+0000 7fa3abe00700 1 --2- 172.21.15.19:0/247840718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3b4115370 0x7fa3b41a5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:14.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.419+0000 7fa3abe00700 1 -- 172.21.15.19:0/247840718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3b4075b60 msgr2=0x7fa3b41a1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:14.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.419+0000 7fa3abe00700 1 --2- 172.21.15.19:0/247840718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3b4075b60 0x7fa3b41a1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:14.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.419+0000 7fa3abe00700 1 -- 172.21.15.19:0/247840718 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3b41a6420 con 0x7fa3b40764b0 2024-05-07T02:29:14.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.419+0000 7fa3b9000700 1 --2- 172.21.15.19:0/247840718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3b4075b60 0x7fa3b41a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:29:14.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.420+0000 7fa3abe00700 1 --2- 172.21.15.19:0/247840718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b40764b0 0x7fa3b41a1af0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa39c009230 tx=0x7fa39c00fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:14.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.420+0000 7fa3b2000700 1 -- 172.21.15.19:0/247840718 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39c005140 con 0x7fa3b40764b0 2024-05-07T02:29:14.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.420+0000 7fa3bba00700 1 -- 172.21.15.19:0/247840718 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3b41a66b0 con 0x7fa3b40764b0 2024-05-07T02:29:14.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.420+0000 7fa3b2000700 1 -- 172.21.15.19:0/247840718 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa39c00b560 con 0x7fa3b40764b0 2024-05-07T02:29:14.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.420+0000 7fa3bba00700 1 -- 172.21.15.19:0/247840718 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3b41ae030 con 0x7fa3b40764b0 2024-05-07T02:29:14.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.421+0000 7fa3b2000700 1 -- 172.21.15.19:0/247840718 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39c003990 con 0x7fa3b40764b0 2024-05-07T02:29:14.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.422+0000 7fa3bba00700 1 -- 172.21.15.19:0/247840718 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa3b4063ed0 con 0x7fa3b40764b0 2024-05-07T02:29:14.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.424+0000 7fa3b2000700 1 -- 172.21.15.19:0/247840718 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fa39c00ba50 con 0x7fa3b40764b0 2024-05-07T02:29:14.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.426+0000 7fa3b2000700 1 --2- 172.21.15.19:0/247840718 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fa3980b2910 0x7fa3980b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:14.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.426+0000 7fa3b2000700 1 -- 172.21.15.19:0/247840718 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fa39c0e41a0 con 0x7fa3b40764b0 2024-05-07T02:29:14.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.426+0000 7fa3b9000700 1 --2- 172.21.15.19:0/247840718 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fa3980b2910 0x7fa3980b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:14.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.426+0000 7fa3b2000700 1 --2- 172.21.15.19:0/247840718 >> [v2:172.21.15.19:6810/387118141,v1:172.21.15.19:6811/387118141] conn(0x7fa3980bdb10 0x7fa3980c0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:14.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.427+0000 7fa3b2000700 1 -- 172.21.15.19:0/247840718 --> [v2:172.21.15.19:6810/387118141,v1:172.21.15.19:6811/387118141] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa39c00b0b0 con 0x7fa3980bdb10 2024-05-07T02:29:14.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.427+0000 7fa3b2000700 1 -- 172.21.15.19:0/247840718 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fa39c0e4550 con 0x7fa3b40764b0 2024-05-07T02:29:14.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.427+0000 7fa3b9a00700 1 --2- 172.21.15.19:0/247840718 >> [v2:172.21.15.19:6810/387118141,v1:172.21.15.19:6811/387118141] conn(0x7fa3980bdb10 0x7fa3980c0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:14.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.427+0000 7fa3b9000700 1 --2- 172.21.15.19:0/247840718 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fa3980b2910 0x7fa3980b4e20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4009480 tx=0x7fa3a400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:14.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.428+0000 7fa3b9a00700 1 --2- 172.21.15.19:0/247840718 >> [v2:172.21.15.19:6810/387118141,v1:172.21.15.19:6811/387118141] conn(0x7fa3980bdb10 0x7fa3980c0000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:14.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.429+0000 7fa3b2000700 1 -- 172.21.15.19:0/247840718 <== osd.3 v2:172.21.15.19:6810/387118141 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa39c00b0b0 con 0x7fa3980bdb10 2024-05-07T02:29:14.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.447+0000 7fa3bba00700 1 -- 172.21.15.19:0/247840718 --> [v2:172.21.15.19:6810/387118141,v1:172.21.15.19:6811/387118141] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa3b40642d0 con 0x7fa3980bdb10 2024-05-07T02:29:14.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.447+0000 7fa3b2000700 1 -- 172.21.15.19:0/247840718 <== osd.3 v2:172.21.15.19:6810/387118141 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa3b40642d0 con 0x7fa3980bdb10 2024-05-07T02:29:14.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.448+0000 7fa3bba00700 1 -- 172.21.15.19:0/247840718 >> [v2:172.21.15.19:6810/387118141,v1:172.21.15.19:6811/387118141] conn(0x7fa3980bdb10 msgr2=0x7fa3980c0000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:14.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.448+0000 7fa3bba00700 1 --2- 172.21.15.19:0/247840718 >> [v2:172.21.15.19:6810/387118141,v1:172.21.15.19:6811/387118141] conn(0x7fa3980bdb10 0x7fa3980c0000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:14.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.448+0000 7fa3bba00700 1 -- 172.21.15.19:0/247840718 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fa3980b2910 msgr2=0x7fa3980b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:14.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.448+0000 7fa3bba00700 1 --2- 172.21.15.19:0/247840718 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fa3980b2910 0x7fa3980b4e20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4009480 tx=0x7fa3a400b040 comp rx=0 tx=0).stop 2024-05-07T02:29:14.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.448+0000 7fa3bba00700 1 -- 172.21.15.19:0/247840718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b40764b0 msgr2=0x7fa3b41a1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:14.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.448+0000 7fa3bba00700 1 --2- 172.21.15.19:0/247840718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b40764b0 0x7fa3b41a1af0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa39c009230 tx=0x7fa39c00fa70 comp rx=0 tx=0).stop 2024-05-07T02:29:14.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.448+0000 7fa3b9a00700 1 -- 172.21.15.19:0/247840718 reap_dead start 2024-05-07T02:29:14.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.448+0000 7fa3bba00700 1 -- 172.21.15.19:0/247840718 shutdown_connections 2024-05-07T02:29:14.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.448+0000 7fa3bba00700 1 -- 172.21.15.19:0/247840718 >> 172.21.15.19:0/247840718 conn(0x7fa3b4100040 msgr2=0x7fa3b4100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:14.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.448+0000 7fa3bba00700 1 -- 172.21.15.19:0/247840718 shutdown_connections 2024-05-07T02:29:14.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:14.448+0000 7fa3bba00700 1 -- 172.21.15.19:0/247840718 wait complete. 2024-05-07T02:29:15.080 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:14 smithi019 ceph-mon[29781]: pgmap v430: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-07T02:29:15.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:14 smithi064 ceph-mon[34699]: pgmap v430: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-07T02:29:15.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:14 smithi027 ceph-mon[34398]: pgmap v430: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-07T02:29:15.527 INFO:teuthology.orchestra.run.smithi019.stdout:77309411436 2024-05-07T02:29:15.527 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-07T02:29:16.078 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:15 smithi019 ceph-mon[29781]: pgmap v431: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:29:16.182 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:15 smithi064 ceph-mon[34699]: pgmap v431: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:29:16.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:15 smithi027 ceph-mon[34398]: pgmap v431: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:29:16.428 INFO:teuthology.orchestra.run.smithi019.stdout:133143986273 2024-05-07T02:29:16.428 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-07T02:29:16.614 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:29:16.628 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:29:16.685 INFO:teuthology.orchestra.run.smithi019.stdout:107374182501 2024-05-07T02:29:16.685 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-07T02:29:16.716 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:29:16.943 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:16 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:29:17.157 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:29:17.165 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:29:17.169 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:29:17.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:16 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:29:17.299 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:29:17.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:16 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:29:17.931 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:29:18.354 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:18 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:29:18.355 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:18 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:29:18.355 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:18 smithi019 ceph-mon[29781]: pgmap v432: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:29:18.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:18 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:29:18.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:18 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:29:18.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:18 smithi027 ceph-mon[34398]: pgmap v432: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:29:18.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:18 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:29:18.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:18 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:29:18.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:18 smithi064 ceph-mon[34699]: pgmap v432: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:29:19.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:19 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:29:19.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:19 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:29:19.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:19 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:29:19.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:19 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:29:19.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:19 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:29:19.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:19 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:29:19.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:19 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:29:19.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:19 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:29:19.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:19 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:29:20.377 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:20 smithi019 ceph-mon[29781]: pgmap v433: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-07T02:29:20.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:20 smithi027 ceph-mon[34398]: pgmap v433: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-07T02:29:20.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:20 smithi064 ceph-mon[34699]: pgmap v433: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-07T02:29:22.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:22 smithi019 ceph-mon[29781]: pgmap v434: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:29:22.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:22 smithi064 ceph-mon[34699]: pgmap v434: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:29:22.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:22 smithi027 ceph-mon[34398]: pgmap v434: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:29:24.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.041+0000 7f8786000700 1 -- 172.21.15.19:0/1947548648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8780104400 msgr2=0x7f8780104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:24.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.041+0000 7f8786000700 1 --2- 172.21.15.19:0/1947548648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8780104400 0x7f8780104810 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f87740099c0 tx=0x7f8774009cd0 comp rx=0 tx=0).stop 2024-05-07T02:29:24.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.041+0000 7f8786000700 1 -- 172.21.15.19:0/1947548648 shutdown_connections 2024-05-07T02:29:24.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.041+0000 7f8786000700 1 --2- 172.21.15.19:0/1947548648 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8780109680 0x7f8780110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:24.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.041+0000 7f8786000700 1 --2- 172.21.15.19:0/1947548648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8780104400 0x7f8780104810 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:24.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.041+0000 7f8786000700 1 --2- 172.21.15.19:0/1947548648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8780108d30 0x7f8780109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:24.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.041+0000 7f8786000700 1 -- 172.21.15.19:0/1947548648 >> 172.21.15.19:0/1947548648 conn(0x7f8780079d10 msgr2=0x7f878007a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:24.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.041+0000 7f8786000700 1 -- 172.21.15.19:0/1947548648 shutdown_connections 2024-05-07T02:29:24.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.042+0000 7f8786000700 1 -- 172.21.15.19:0/1947548648 wait complete. 2024-05-07T02:29:24.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.043+0000 7f8786000700 1 Processor -- start 2024-05-07T02:29:24.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.043+0000 7f8786000700 1 -- start start 2024-05-07T02:29:24.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.044+0000 7f8786000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8780104400 0x7f8780075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:24.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.044+0000 7f8786000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8780108d30 0x7f8780076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:24.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.044+0000 7f8786000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8780109680 0x7f8780076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:24.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.044+0000 7f8786000700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f87801142b0 con 0x7f8780109680 2024-05-07T02:29:24.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.044+0000 7f8786000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8780114130 con 0x7f8780104400 2024-05-07T02:29:24.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.044+0000 7f8786000700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8780114430 con 0x7f8780108d30 2024-05-07T02:29:24.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.044+0000 7f8785600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8780109680 0x7f8780076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:24.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.044+0000 7f8784c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8780104400 0x7f8780075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:24.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.044+0000 7f8785600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8780109680 0x7f8780076540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52706/0 (socket says 172.21.15.19:52706) 2024-05-07T02:29:24.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.044+0000 7f8785600700 1 -- 172.21.15.19:0/849813800 learned_addr learned my addr 172.21.15.19:0/849813800 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:29:24.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.044+0000 7f877fe00700 1 --2- 172.21.15.19:0/849813800 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8780108d30 0x7f8780076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:24.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.045+0000 7f8785600700 1 -- 172.21.15.19:0/849813800 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8780108d30 msgr2=0x7f8780076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:24.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.045+0000 7f8785600700 1 --2- 172.21.15.19:0/849813800 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8780108d30 0x7f8780076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:24.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.045+0000 7f8785600700 1 -- 172.21.15.19:0/849813800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8780104400 msgr2=0x7f8780075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:24.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.045+0000 7f8785600700 1 --2- 172.21.15.19:0/849813800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8780104400 0x7f8780075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:24.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.045+0000 7f8785600700 1 -- 172.21.15.19:0/849813800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87801a8fe0 con 0x7f8780109680 2024-05-07T02:29:24.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.045+0000 7f8785600700 1 --2- 172.21.15.19:0/849813800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8780109680 0x7f8780076540 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f877800c370 tx=0x7f877800c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:24.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.045+0000 7f877d600700 1 -- 172.21.15.19:0/849813800 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87780092e0 con 0x7f8780109680 2024-05-07T02:29:24.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.045+0000 7f877d600700 1 -- 172.21.15.19:0/849813800 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8778003a00 con 0x7f8780109680 2024-05-07T02:29:24.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.045+0000 7f8786000700 1 -- 172.21.15.19:0/849813800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87801a9270 con 0x7f8780109680 2024-05-07T02:29:24.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.045+0000 7f877d600700 1 -- 172.21.15.19:0/849813800 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87780046f0 con 0x7f8780109680 2024-05-07T02:29:24.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.045+0000 7f8786000700 1 -- 172.21.15.19:0/849813800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87801a9760 con 0x7f8780109680 2024-05-07T02:29:24.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.047+0000 7f8786000700 1 -- 172.21.15.19:0/849813800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87800512c0 con 0x7f8780109680 2024-05-07T02:29:24.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.051+0000 7f877d600700 1 -- 172.21.15.19:0/849813800 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f877801a020 con 0x7f8780109680 2024-05-07T02:29:24.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.053+0000 7f877d600700 1 --2- 172.21.15.19:0/849813800 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f87680b2910 0x7f87680b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:24.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.053+0000 7f877d600700 1 -- 172.21.15.19:0/849813800 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f87780e25d0 con 0x7f8780109680 2024-05-07T02:29:24.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.053+0000 7f8784c00700 1 --2- 172.21.15.19:0/849813800 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f87680b2910 0x7f87680b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:24.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.054+0000 7f8784c00700 1 --2- 172.21.15.19:0/849813800 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f87680b2910 0x7f87680b4e20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8774000c00 tx=0x7f877401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:24.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.056+0000 7f877d600700 1 -- 172.21.15.19:0/849813800 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8778010040 con 0x7f8780109680 2024-05-07T02:29:24.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.300+0000 7f8786000700 1 -- 172.21.15.19:0/849813800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f878002e5f0 con 0x7f8780109680 2024-05-07T02:29:24.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.301+0000 7f877d600700 1 -- 172.21.15.19:0/849813800 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f87780a9de0 con 0x7f8780109680 2024-05-07T02:29:24.301 INFO:teuthology.orchestra.run.smithi019.stdout:201863462992 2024-05-07T02:29:24.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.305+0000 7f8786000700 1 -- 172.21.15.19:0/849813800 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f87680b2910 msgr2=0x7f87680b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:24.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.305+0000 7f8786000700 1 --2- 172.21.15.19:0/849813800 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f87680b2910 0x7f87680b4e20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8774000c00 tx=0x7f877401a040 comp rx=0 tx=0).stop 2024-05-07T02:29:24.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.305+0000 7f8786000700 1 -- 172.21.15.19:0/849813800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8780109680 msgr2=0x7f8780076540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:24.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.305+0000 7f8786000700 1 --2- 172.21.15.19:0/849813800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8780109680 0x7f8780076540 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f877800c370 tx=0x7f877800c680 comp rx=0 tx=0).stop 2024-05-07T02:29:24.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.305+0000 7f8786000700 1 -- 172.21.15.19:0/849813800 shutdown_connections 2024-05-07T02:29:24.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.305+0000 7f8786000700 1 --2- 172.21.15.19:0/849813800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8780109680 0x7f8780076540 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:24.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.305+0000 7f8786000700 1 --2- 172.21.15.19:0/849813800 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f87680b2910 0x7f87680b4e20 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:24.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.305+0000 7f8786000700 1 --2- 172.21.15.19:0/849813800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8780104400 0x7f8780075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:24.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.305+0000 7f8786000700 1 --2- 172.21.15.19:0/849813800 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8780108d30 0x7f8780076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:24.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.305+0000 7f8786000700 1 -- 172.21.15.19:0/849813800 >> 172.21.15.19:0/849813800 conn(0x7f8780079d10 msgr2=0x7f8780108070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:24.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.305+0000 7f8786000700 1 -- 172.21.15.19:0/849813800 shutdown_connections 2024-05-07T02:29:24.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:24.306+0000 7f8786000700 1 -- 172.21.15.19:0/849813800 wait complete. 2024-05-07T02:29:24.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:24 smithi064 ceph-mon[34699]: pgmap v435: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:29:24.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:24 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/849813800' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-07T02:29:24.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:24 smithi019 ceph-mon[29781]: pgmap v435: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:29:24.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:24 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/849813800' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-07T02:29:24.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:24 smithi027 ceph-mon[34398]: pgmap v435: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:29:24.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:24 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/849813800' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-07T02:29:25.896 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:29:25.896 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:29:26.848 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:26 smithi019 ceph-mon[29781]: pgmap v436: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:29:26.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:26 smithi064 ceph-mon[34699]: pgmap v436: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:29:26.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:26 smithi027 ceph-mon[34398]: pgmap v436: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:29:28.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.465+0000 7fc463000700 1 -- 172.21.15.19:0/1811827095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc45c105dd0 msgr2=0x7fc45c10e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:28.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.465+0000 7fc463000700 1 --2- 172.21.15.19:0/1811827095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc45c105dd0 0x7fc45c10e320 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc448009990 tx=0x7fc448009ca0 comp rx=0 tx=0).stop 2024-05-07T02:29:28.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.465+0000 7fc463000700 1 -- 172.21.15.19:0/1811827095 shutdown_connections 2024-05-07T02:29:28.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.465+0000 7fc463000700 1 --2- 172.21.15.19:0/1811827095 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc45c10e8f0 0x7fc45c110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:28.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.465+0000 7fc463000700 1 --2- 172.21.15.19:0/1811827095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc45c105480 0x7fc45c105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:28.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.465+0000 7fc463000700 1 --2- 172.21.15.19:0/1811827095 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc45c105dd0 0x7fc45c10e320 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:28.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.465+0000 7fc463000700 1 -- 172.21.15.19:0/1811827095 >> 172.21.15.19:0/1811827095 conn(0x7fc45c0fef30 msgr2=0x7fc45c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:28.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.465+0000 7fc463000700 1 -- 172.21.15.19:0/1811827095 shutdown_connections 2024-05-07T02:29:28.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.465+0000 7fc463000700 1 -- 172.21.15.19:0/1811827095 wait complete. 2024-05-07T02:29:28.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.466+0000 7fc463000700 1 Processor -- start 2024-05-07T02:29:28.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.467+0000 7fc463000700 1 -- start start 2024-05-07T02:29:28.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.467+0000 7fc463000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc45c105480 0x7fc45c103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:28.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.467+0000 7fc463000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc45c105dd0 0x7fc45c1036b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:28.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.467+0000 7fc463000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc45c10e8f0 0x7fc45c103bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:28.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.467+0000 7fc463000700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc45c114450 con 0x7fc45c10e8f0 2024-05-07T02:29:28.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.467+0000 7fc463000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc45c1142d0 con 0x7fc45c105480 2024-05-07T02:29:28.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.467+0000 7fc463000700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc45c1145d0 con 0x7fc45c105dd0 2024-05-07T02:29:28.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.467+0000 7fc45be00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc45c105480 0x7fc45c103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:28.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.467+0000 7fc45be00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc45c105480 0x7fc45c103170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:46004/0 (socket says 172.21.15.19:46004) 2024-05-07T02:29:28.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.468+0000 7fc45be00700 1 -- 172.21.15.19:0/3318047450 learned_addr learned my addr 172.21.15.19:0/3318047450 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:29:28.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.468+0000 7fc45b400700 1 --2- 172.21.15.19:0/3318047450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc45c105dd0 0x7fc45c1036b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:28.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.468+0000 7fc461000700 1 --2- 172.21.15.19:0/3318047450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc45c10e8f0 0x7fc45c103bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:28.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.468+0000 7fc45be00700 1 -- 172.21.15.19:0/3318047450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc45c105dd0 msgr2=0x7fc45c1036b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:28.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.468+0000 7fc45be00700 1 --2- 172.21.15.19:0/3318047450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc45c105dd0 0x7fc45c1036b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:28.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.468+0000 7fc45be00700 1 -- 172.21.15.19:0/3318047450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc45c10e8f0 msgr2=0x7fc45c103bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:28.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.468+0000 7fc45be00700 1 --2- 172.21.15.19:0/3318047450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc45c10e8f0 0x7fc45c103bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:28.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.468+0000 7fc45be00700 1 -- 172.21.15.19:0/3318047450 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc45c1a9140 con 0x7fc45c105480 2024-05-07T02:29:28.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.469+0000 7fc45be00700 1 --2- 172.21.15.19:0/3318047450 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc45c105480 0x7fc45c103170 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc44c00e9a0 tx=0x7fc44c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:28.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.469+0000 7fc458c00700 1 -- 172.21.15.19:0/3318047450 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44c00ca10 con 0x7fc45c105480 2024-05-07T02:29:28.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.469+0000 7fc458c00700 1 -- 172.21.15.19:0/3318047450 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc44c00cbb0 con 0x7fc45c105480 2024-05-07T02:29:28.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.469+0000 7fc463000700 1 -- 172.21.15.19:0/3318047450 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc45c1a9430 con 0x7fc45c105480 2024-05-07T02:29:28.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.469+0000 7fc458c00700 1 -- 172.21.15.19:0/3318047450 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44c018680 con 0x7fc45c105480 2024-05-07T02:29:28.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.469+0000 7fc463000700 1 -- 172.21.15.19:0/3318047450 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc45c1a9990 con 0x7fc45c105480 2024-05-07T02:29:28.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.470+0000 7fc463000700 1 -- 172.21.15.19:0/3318047450 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc45c0512c0 con 0x7fc45c105480 2024-05-07T02:29:28.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.473+0000 7fc458c00700 1 -- 172.21.15.19:0/3318047450 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fc44c010430 con 0x7fc45c105480 2024-05-07T02:29:28.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.474+0000 7fc458c00700 1 --2- 172.21.15.19:0/3318047450 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fc4400b2990 0x7fc4400b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:28.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.475+0000 7fc458c00700 1 -- 172.21.15.19:0/3318047450 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fc44c0e2be0 con 0x7fc45c105480 2024-05-07T02:29:28.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.475+0000 7fc45b400700 1 --2- 172.21.15.19:0/3318047450 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fc4400b2990 0x7fc4400b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:28.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.476+0000 7fc45b400700 1 --2- 172.21.15.19:0/3318047450 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fc4400b2990 0x7fc4400b4ea0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc44800b2e0 tx=0x7fc448005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:28.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.476+0000 7fc458c00700 1 -- 172.21.15.19:0/3318047450 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc44c0aa390 con 0x7fc45c105480 2024-05-07T02:29:28.601 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:28 smithi019 ceph-mon[29781]: pgmap v437: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:29:28.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.745+0000 7fc463000700 1 -- 172.21.15.19:0/3318047450 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fc45c02e5f0 con 0x7fc45c105480 2024-05-07T02:29:28.746 INFO:teuthology.orchestra.run.smithi019.stdout:55834574965 2024-05-07T02:29:28.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.746+0000 7fc458c00700 1 -- 172.21.15.19:0/3318047450 <== mon.2 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc44c0b2410 con 0x7fc45c105480 2024-05-07T02:29:28.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.749+0000 7fc463000700 1 -- 172.21.15.19:0/3318047450 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fc4400b2990 msgr2=0x7fc4400b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:28.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.749+0000 7fc463000700 1 --2- 172.21.15.19:0/3318047450 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fc4400b2990 0x7fc4400b4ea0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc44800b2e0 tx=0x7fc448005960 comp rx=0 tx=0).stop 2024-05-07T02:29:28.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.749+0000 7fc463000700 1 -- 172.21.15.19:0/3318047450 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc45c105480 msgr2=0x7fc45c103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:28.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.749+0000 7fc463000700 1 --2- 172.21.15.19:0/3318047450 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc45c105480 0x7fc45c103170 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc44c00e9a0 tx=0x7fc44c00ecb0 comp rx=0 tx=0).stop 2024-05-07T02:29:28.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.749+0000 7fc463000700 1 -- 172.21.15.19:0/3318047450 shutdown_connections 2024-05-07T02:29:28.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.749+0000 7fc463000700 1 --2- 172.21.15.19:0/3318047450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc45c10e8f0 0x7fc45c103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:28.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.749+0000 7fc463000700 1 --2- 172.21.15.19:0/3318047450 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fc4400b2990 0x7fc4400b4ea0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:28.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.749+0000 7fc463000700 1 --2- 172.21.15.19:0/3318047450 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc45c105480 0x7fc45c103170 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:28.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.749+0000 7fc463000700 1 --2- 172.21.15.19:0/3318047450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc45c105dd0 0x7fc45c1036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:28.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.749+0000 7fc463000700 1 -- 172.21.15.19:0/3318047450 >> 172.21.15.19:0/3318047450 conn(0x7fc45c0fef30 msgr2=0x7fc45c0ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:28.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.750+0000 7fc463000700 1 -- 172.21.15.19:0/3318047450 shutdown_connections 2024-05-07T02:29:28.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.750+0000 7fc463000700 1 -- 172.21.15.19:0/3318047450 wait complete. 2024-05-07T02:29:28.788 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462986 got 201863462992 for osd.7 2024-05-07T02:29:28.789 DEBUG:teuthology.parallel:result is None 2024-05-07T02:29:28.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:28 smithi064 ceph-mon[34699]: pgmap v437: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:29:28.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:28 smithi027 ceph-mon[34398]: pgmap v437: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:29:29.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:29 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/1031535659' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-07T02:29:29.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:29 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/3318047450' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-07T02:29:29.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:29 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/1031535659' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-07T02:29:29.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:29 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/3318047450' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-07T02:29:30.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:29 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/1031535659' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-07T02:29:30.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:29 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/3318047450' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-07T02:29:30.788 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:30 smithi019 ceph-mon[29781]: pgmap v438: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:29:30.788 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:30 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/2863931741' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-07T02:29:30.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.907+0000 7fb7f1200700 1 -- 172.21.15.19:0/442131005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7ec1030c0 msgr2=0x7fb7ec1034d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:30.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.907+0000 7fb7f1200700 1 --2- 172.21.15.19:0/442131005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7ec1030c0 0x7fb7ec1034d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc009990 tx=0x7fb7dc009ca0 comp rx=0 tx=0).stop 2024-05-07T02:29:30.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.907+0000 7fb7f1200700 1 -- 172.21.15.19:0/442131005 shutdown_connections 2024-05-07T02:29:30.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.907+0000 7fb7f1200700 1 --2- 172.21.15.19:0/442131005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7ec110c70 0x7fb7ec113090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:30.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.907+0000 7fb7f1200700 1 --2- 172.21.15.19:0/442131005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7ec1030c0 0x7fb7ec1034d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:30.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.908+0000 7fb7f1200700 1 --2- 172.21.15.19:0/442131005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7ec103a10 0x7fb7ec110730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:30.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.908+0000 7fb7f1200700 1 -- 172.21.15.19:0/442131005 >> 172.21.15.19:0/442131005 conn(0x7fb7ec0fedc0 msgr2=0x7fb7ec1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:30.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.908+0000 7fb7f1200700 1 -- 172.21.15.19:0/442131005 shutdown_connections 2024-05-07T02:29:30.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.908+0000 7fb7f1200700 1 -- 172.21.15.19:0/442131005 wait complete. 2024-05-07T02:29:30.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.909+0000 7fb7f1200700 1 Processor -- start 2024-05-07T02:29:30.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.909+0000 7fb7f1200700 1 -- start start 2024-05-07T02:29:30.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.910+0000 7fb7f1200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7ec1030c0 0x7fb7ec19d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:30.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.910+0000 7fb7f1200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7ec103a10 0x7fb7ec19d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:30.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.910+0000 7fb7f1200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7ec110c70 0x7fb7ec1a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:30.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.910+0000 7fb7f1200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ec1166b0 con 0x7fb7ec110c70 2024-05-07T02:29:30.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.910+0000 7fb7f1200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ec116530 con 0x7fb7ec103a10 2024-05-07T02:29:30.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.910+0000 7fb7f1200700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ec116830 con 0x7fb7ec1030c0 2024-05-07T02:29:30.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.910+0000 7fb7ea800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7ec110c70 0x7fb7ec1a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:30.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.910+0000 7fb7e9e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7ec1030c0 0x7fb7ec19d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:30.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.910+0000 7fb7ea800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7ec110c70 0x7fb7ec1a1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40146/0 (socket says 172.21.15.19:40146) 2024-05-07T02:29:30.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.910+0000 7fb7e9e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7ec1030c0 0x7fb7ec19d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:34550/0 (socket says 172.21.15.19:34550) 2024-05-07T02:29:30.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.910+0000 7fb7ea800700 1 -- 172.21.15.19:0/4095174739 learned_addr learned my addr 172.21.15.19:0/4095174739 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:29:30.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.910+0000 7fb7e3e00700 1 --2- 172.21.15.19:0/4095174739 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7ec103a10 0x7fb7ec19d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:30.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.910+0000 7fb7e9e00700 1 -- 172.21.15.19:0/4095174739 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7ec103a10 msgr2=0x7fb7ec19d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:30.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.910+0000 7fb7e9e00700 1 --2- 172.21.15.19:0/4095174739 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7ec103a10 0x7fb7ec19d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:30.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.911+0000 7fb7e9e00700 1 -- 172.21.15.19:0/4095174739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7ec110c70 msgr2=0x7fb7ec1a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:30.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.911+0000 7fb7e9e00700 1 --2- 172.21.15.19:0/4095174739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7ec110c70 0x7fb7ec1a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:30.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.911+0000 7fb7e9e00700 1 -- 172.21.15.19:0/4095174739 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7ec1a1ed0 con 0x7fb7ec1030c0 2024-05-07T02:29:30.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.911+0000 7fb7e9e00700 1 --2- 172.21.15.19:0/4095174739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7ec1030c0 0x7fb7ec19d000 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc0098e0 tx=0x7fb7dc0057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:30.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.911+0000 7fb7e2a00700 1 -- 172.21.15.19:0/4095174739 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc027070 con 0x7fb7ec1030c0 2024-05-07T02:29:30.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.911+0000 7fb7f1200700 1 -- 172.21.15.19:0/4095174739 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7ec1a2160 con 0x7fb7ec1030c0 2024-05-07T02:29:30.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.911+0000 7fb7e2a00700 1 -- 172.21.15.19:0/4095174739 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7dc022470 con 0x7fb7ec1030c0 2024-05-07T02:29:30.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.911+0000 7fb7e2a00700 1 -- 172.21.15.19:0/4095174739 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc017650 con 0x7fb7ec1030c0 2024-05-07T02:29:30.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.912+0000 7fb7f1200700 1 -- 172.21.15.19:0/4095174739 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7ec1a9ae0 con 0x7fb7ec1030c0 2024-05-07T02:29:30.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.913+0000 7fb7f1200700 1 -- 172.21.15.19:0/4095174739 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7ec0512c0 con 0x7fb7ec1030c0 2024-05-07T02:29:30.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.915+0000 7fb7e2a00700 1 -- 172.21.15.19:0/4095174739 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fb7dc02a070 con 0x7fb7ec1030c0 2024-05-07T02:29:30.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.917+0000 7fb7e2a00700 1 --2- 172.21.15.19:0/4095174739 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fb7d00b2910 0x7fb7d00b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:30.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.917+0000 7fb7e2a00700 1 -- 172.21.15.19:0/4095174739 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fb7dc0e3860 con 0x7fb7ec1030c0 2024-05-07T02:29:30.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.917+0000 7fb7e3e00700 1 --2- 172.21.15.19:0/4095174739 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fb7d00b2910 0x7fb7d00b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:30.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.918+0000 7fb7e3e00700 1 --2- 172.21.15.19:0/4095174739 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fb7d00b2910 0x7fb7d00b4e20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec112380 tx=0x7fb7d4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:30.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.919+0000 7fb7e2a00700 1 -- 172.21.15.19:0/4095174739 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb7dc013070 con 0x7fb7ec1030c0 2024-05-07T02:29:30.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:30 smithi027 ceph-mon[34398]: pgmap v438: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:29:30.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:30 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/2863931741' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-07T02:29:31.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:30 smithi064 ceph-mon[34699]: pgmap v438: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:29:31.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:30 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/2863931741' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-07T02:29:31.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.172+0000 7fb7f1200700 1 -- 172.21.15.19:0/4095174739 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fb7ec02e5f0 con 0x7fb7ec1030c0 2024-05-07T02:29:31.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.173+0000 7fb7e2a00700 1 -- 172.21.15.19:0/4095174739 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fb7dc0ab130 con 0x7fb7ec1030c0 2024-05-07T02:29:31.173 INFO:teuthology.orchestra.run.smithi019.stdout:300647710776 2024-05-07T02:29:31.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.176+0000 7fb7f1200700 1 -- 172.21.15.19:0/4095174739 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fb7d00b2910 msgr2=0x7fb7d00b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:31.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.176+0000 7fb7f1200700 1 --2- 172.21.15.19:0/4095174739 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fb7d00b2910 0x7fb7d00b4e20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec112380 tx=0x7fb7d4009260 comp rx=0 tx=0).stop 2024-05-07T02:29:31.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.176+0000 7fb7f1200700 1 -- 172.21.15.19:0/4095174739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7ec1030c0 msgr2=0x7fb7ec19d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:31.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.176+0000 7fb7f1200700 1 --2- 172.21.15.19:0/4095174739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7ec1030c0 0x7fb7ec19d000 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc0098e0 tx=0x7fb7dc0057c0 comp rx=0 tx=0).stop 2024-05-07T02:29:31.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.176+0000 7fb7f1200700 1 -- 172.21.15.19:0/4095174739 shutdown_connections 2024-05-07T02:29:31.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.176+0000 7fb7f1200700 1 --2- 172.21.15.19:0/4095174739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7ec110c70 0x7fb7ec1a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:31.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.176+0000 7fb7f1200700 1 --2- 172.21.15.19:0/4095174739 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7ec1030c0 0x7fb7ec19d000 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:31.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.176+0000 7fb7f1200700 1 --2- 172.21.15.19:0/4095174739 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fb7d00b2910 0x7fb7d00b4e20 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:31.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.176+0000 7fb7f1200700 1 --2- 172.21.15.19:0/4095174739 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7ec103a10 0x7fb7ec19d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:31.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.176+0000 7fb7f1200700 1 -- 172.21.15.19:0/4095174739 >> 172.21.15.19:0/4095174739 conn(0x7fb7ec0fedc0 msgr2=0x7fb7ec112740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:31.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.176+0000 7fb7f1200700 1 -- 172.21.15.19:0/4095174739 shutdown_connections 2024-05-07T02:29:31.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.176+0000 7fb7f1200700 1 -- 172.21.15.19:0/4095174739 wait complete. 2024-05-07T02:29:31.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.716+0000 7f35efe00700 1 -- 172.21.15.19:0/45876117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35f0108d30 msgr2=0x7f35f0109140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:31.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.716+0000 7f35efe00700 1 --2- 172.21.15.19:0/45876117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35f0108d30 0x7f35f0109140 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f35d8009920 tx=0x7f35d8009c30 comp rx=0 tx=0).stop 2024-05-07T02:29:31.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.716+0000 7f35efe00700 1 -- 172.21.15.19:0/45876117 shutdown_connections 2024-05-07T02:29:31.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.716+0000 7f35efe00700 1 --2- 172.21.15.19:0/45876117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35f0109680 0x7f35f0110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:31.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.716+0000 7f35efe00700 1 --2- 172.21.15.19:0/45876117 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35f0104400 0x7f35f0104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:31.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.716+0000 7f35efe00700 1 --2- 172.21.15.19:0/45876117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35f0108d30 0x7f35f0109140 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:31.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.716+0000 7f35efe00700 1 -- 172.21.15.19:0/45876117 >> 172.21.15.19:0/45876117 conn(0x7f35f0079d10 msgr2=0x7f35f007a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:31.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.716+0000 7f35efe00700 1 -- 172.21.15.19:0/45876117 shutdown_connections 2024-05-07T02:29:31.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.717+0000 7f35efe00700 1 -- 172.21.15.19:0/45876117 wait complete. 2024-05-07T02:29:31.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.718+0000 7f35efe00700 1 Processor -- start 2024-05-07T02:29:31.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.718+0000 7f35efe00700 1 -- start start 2024-05-07T02:29:31.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.718+0000 7f35efe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35f0104400 0x7f35f0075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:31.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.718+0000 7f35efe00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35f0108d30 0x7f35f0076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:31.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.718+0000 7f35efe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35f0109680 0x7f35f0076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:31.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.718+0000 7f35efe00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f35f01142e0 con 0x7f35f0109680 2024-05-07T02:29:31.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.718+0000 7f35efe00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f35f0114160 con 0x7f35f0104400 2024-05-07T02:29:31.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.719+0000 7f35efe00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f35f0114460 con 0x7f35f0108d30 2024-05-07T02:29:31.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.719+0000 7f35eea00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35f0104400 0x7f35f0075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:31.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.719+0000 7f35ef400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35f0109680 0x7f35f0076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:31.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.719+0000 7f35eea00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35f0104400 0x7f35f0075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:58156/0 (socket says 172.21.15.19:58156) 2024-05-07T02:29:31.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.719+0000 7f35ef400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35f0109680 0x7f35f0076540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40158/0 (socket says 172.21.15.19:40158) 2024-05-07T02:29:31.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.719+0000 7f35eea00700 1 -- 172.21.15.19:0/3030607780 learned_addr learned my addr 172.21.15.19:0/3030607780 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:29:31.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.719+0000 7f35e7e00700 1 --2- 172.21.15.19:0/3030607780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35f0108d30 0x7f35f0076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:31.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.719+0000 7f35eea00700 1 -- 172.21.15.19:0/3030607780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35f0108d30 msgr2=0x7f35f0076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:31.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.719+0000 7f35eea00700 1 --2- 172.21.15.19:0/3030607780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35f0108d30 0x7f35f0076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:31.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.719+0000 7f35eea00700 1 -- 172.21.15.19:0/3030607780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35f0109680 msgr2=0x7f35f0076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:31.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.719+0000 7f35eea00700 1 --2- 172.21.15.19:0/3030607780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35f0109680 0x7f35f0076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:31.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.719+0000 7f35eea00700 1 -- 172.21.15.19:0/3030607780 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35f01a90f0 con 0x7f35f0104400 2024-05-07T02:29:31.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.720+0000 7f35eea00700 1 --2- 172.21.15.19:0/3030607780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35f0104400 0x7f35f0075ac0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f35e000ec10 tx=0x7f35e000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:31.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.720+0000 7f35e7400700 1 -- 172.21.15.19:0/3030607780 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e000cc90 con 0x7f35f0104400 2024-05-07T02:29:31.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.720+0000 7f35efe00700 1 -- 172.21.15.19:0/3030607780 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35f01a93e0 con 0x7f35f0104400 2024-05-07T02:29:31.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.720+0000 7f35e7400700 1 -- 172.21.15.19:0/3030607780 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35e000ce30 con 0x7f35f0104400 2024-05-07T02:29:31.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.720+0000 7f35e7400700 1 -- 172.21.15.19:0/3030607780 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e0018760 con 0x7f35f0104400 2024-05-07T02:29:31.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.720+0000 7f35efe00700 1 -- 172.21.15.19:0/3030607780 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35f01a9940 con 0x7f35f0104400 2024-05-07T02:29:31.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.721+0000 7f35efe00700 1 -- 172.21.15.19:0/3030607780 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35f00512c0 con 0x7f35f0104400 2024-05-07T02:29:31.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.724+0000 7f35e7400700 1 -- 172.21.15.19:0/3030607780 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f35e0010ac0 con 0x7f35f0104400 2024-05-07T02:29:31.724 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:31 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/4095174739' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-07T02:29:31.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.726+0000 7f35e7400700 1 --2- 172.21.15.19:0/3030607780 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f35d40b2910 0x7f35d40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:31.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.726+0000 7f35e7400700 1 -- 172.21.15.19:0/3030607780 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f35e00e2a20 con 0x7f35f0104400 2024-05-07T02:29:31.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.726+0000 7f35e7e00700 1 --2- 172.21.15.19:0/3030607780 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f35d40b2910 0x7f35d40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:31.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.727+0000 7f35e7e00700 1 --2- 172.21.15.19:0/3030607780 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f35d40b2910 0x7f35d40b4e20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f35d8009810 tx=0x7f35d8019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:31.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.730+0000 7f35e7400700 1 -- 172.21.15.19:0/3030607780 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f35e00aa1d0 con 0x7f35f0104400 2024-05-07T02:29:31.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:31 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/4095174739' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-07T02:29:31.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.985+0000 7f35efe00700 1 -- 172.21.15.19:0/3030607780 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f35f002e5f0 con 0x7f35f0104400 2024-05-07T02:29:31.986 INFO:teuthology.orchestra.run.smithi019.stdout:34359738491 2024-05-07T02:29:31.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.986+0000 7f35e7400700 1 -- 172.21.15.19:0/3030607780 <== mon.2 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f35e00b2250 con 0x7f35f0104400 2024-05-07T02:29:31.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.990+0000 7f35efe00700 1 -- 172.21.15.19:0/3030607780 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f35d40b2910 msgr2=0x7f35d40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:31.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.990+0000 7f35efe00700 1 --2- 172.21.15.19:0/3030607780 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f35d40b2910 0x7f35d40b4e20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f35d8009810 tx=0x7f35d8019040 comp rx=0 tx=0).stop 2024-05-07T02:29:31.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.990+0000 7f35efe00700 1 -- 172.21.15.19:0/3030607780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35f0104400 msgr2=0x7f35f0075ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:31.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.990+0000 7f35efe00700 1 --2- 172.21.15.19:0/3030607780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35f0104400 0x7f35f0075ac0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f35e000ec10 tx=0x7f35e000ef20 comp rx=0 tx=0).stop 2024-05-07T02:29:31.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.990+0000 7f35efe00700 1 -- 172.21.15.19:0/3030607780 shutdown_connections 2024-05-07T02:29:31.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.990+0000 7f35efe00700 1 --2- 172.21.15.19:0/3030607780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35f0109680 0x7f35f0076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:31.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.990+0000 7f35efe00700 1 --2- 172.21.15.19:0/3030607780 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f35d40b2910 0x7f35d40b4e20 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:31.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.990+0000 7f35efe00700 1 --2- 172.21.15.19:0/3030607780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35f0104400 0x7f35f0075ac0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:31.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.990+0000 7f35efe00700 1 --2- 172.21.15.19:0/3030607780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f35f0108d30 0x7f35f0076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:31.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.990+0000 7f35efe00700 1 -- 172.21.15.19:0/3030607780 >> 172.21.15.19:0/3030607780 conn(0x7f35f0079d10 msgr2=0x7f35f0101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:31.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.990+0000 7f35efe00700 1 -- 172.21.15.19:0/3030607780 shutdown_connections 2024-05-07T02:29:31.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:31.990+0000 7f35efe00700 1 -- 172.21.15.19:0/3030607780 wait complete. 2024-05-07T02:29:32.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:31 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/4095174739' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-07T02:29:32.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:32 smithi019 ceph-mon[29781]: pgmap v439: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 170 B/s wr, 10 op/s 2024-05-07T02:29:32.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:32 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/3030607780' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-07T02:29:32.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:32 smithi027 ceph-mon[34398]: pgmap v439: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 170 B/s wr, 10 op/s 2024-05-07T02:29:32.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:32 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/3030607780' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-07T02:29:33.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:32 smithi064 ceph-mon[34699]: pgmap v439: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 170 B/s wr, 10 op/s 2024-05-07T02:29:33.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:32 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/3030607780' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-07T02:29:33.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.335+0000 7f1b26a00700 1 -- 172.21.15.19:0/1525255421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b20076680 msgr2=0x7f1b20110660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:33.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.335+0000 7f1b26a00700 1 --2- 172.21.15.19:0/1525255421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b20076680 0x7f1b20110660 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c009990 tx=0x7f1b0c009ca0 comp rx=0 tx=0).stop 2024-05-07T02:29:33.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.335+0000 7f1b26a00700 1 -- 172.21.15.19:0/1525255421 shutdown_connections 2024-05-07T02:29:33.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.335+0000 7f1b26a00700 1 --2- 172.21.15.19:0/1525255421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b20110ba0 0x7f1b20113000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:33.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.335+0000 7f1b26a00700 1 --2- 172.21.15.19:0/1525255421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b20075d30 0x7f1b20076140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:33.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.335+0000 7f1b26a00700 1 --2- 172.21.15.19:0/1525255421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b20076680 0x7f1b20110660 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:33.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.335+0000 7f1b26a00700 1 -- 172.21.15.19:0/1525255421 >> 172.21.15.19:0/1525255421 conn(0x7f1b20100040 msgr2=0x7f1b20102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:33.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.335+0000 7f1b26a00700 1 -- 172.21.15.19:0/1525255421 shutdown_connections 2024-05-07T02:29:33.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.335+0000 7f1b26a00700 1 -- 172.21.15.19:0/1525255421 wait complete. 2024-05-07T02:29:33.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.337+0000 7f1b26a00700 1 Processor -- start 2024-05-07T02:29:33.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.337+0000 7f1b26a00700 1 -- start start 2024-05-07T02:29:33.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.337+0000 7f1b26a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b20075d30 0x7f1b2019d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:33.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.337+0000 7f1b26a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b20076680 0x7f1b2019d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:33.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.337+0000 7f1b26a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b20110ba0 0x7f1b201a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:33.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.337+0000 7f1b26a00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1b201166a0 con 0x7f1b20076680 2024-05-07T02:29:33.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.337+0000 7f1b26a00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1b20116520 con 0x7f1b20110ba0 2024-05-07T02:29:33.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.337+0000 7f1b26a00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1b20116820 con 0x7f1b20075d30 2024-05-07T02:29:33.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.338+0000 7f1b1f400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b20076680 0x7f1b2019d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:33.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.338+0000 7f1b1f400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b20076680 0x7f1b2019d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52720/0 (socket says 172.21.15.19:52720) 2024-05-07T02:29:33.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.338+0000 7f1b1f400700 1 -- 172.21.15.19:0/1031535659 learned_addr learned my addr 172.21.15.19:0/1031535659 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:29:33.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.338+0000 7f1b1fe00700 1 --2- 172.21.15.19:0/1031535659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b20075d30 0x7f1b2019d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:33.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.338+0000 7f1b24a00700 1 --2- 172.21.15.19:0/1031535659 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b20110ba0 0x7f1b201a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:33.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.338+0000 7f1b1fe00700 1 -- 172.21.15.19:0/1031535659 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b20110ba0 msgr2=0x7f1b201a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:33.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.338+0000 7f1b1fe00700 1 --2- 172.21.15.19:0/1031535659 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b20110ba0 0x7f1b201a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:33.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.338+0000 7f1b1fe00700 1 -- 172.21.15.19:0/1031535659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b20076680 msgr2=0x7f1b2019d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:33.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.338+0000 7f1b1fe00700 1 --2- 172.21.15.19:0/1031535659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b20076680 0x7f1b2019d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:33.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.338+0000 7f1b1fe00700 1 -- 172.21.15.19:0/1031535659 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b201a1f20 con 0x7f1b20075d30 2024-05-07T02:29:33.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.338+0000 7f1b1f400700 1 --2- 172.21.15.19:0/1031535659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b20076680 0x7f1b2019d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:29:33.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.339+0000 7f1b1fe00700 1 --2- 172.21.15.19:0/1031535659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b20075d30 0x7f1b2019d050 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1b1000e9f0 tx=0x7f1b1000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:33.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.339+0000 7f1b1cc00700 1 -- 172.21.15.19:0/1031535659 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b1000c9c0 con 0x7f1b20075d30 2024-05-07T02:29:33.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.339+0000 7f1b1cc00700 1 -- 172.21.15.19:0/1031535659 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b1000cb60 con 0x7f1b20075d30 2024-05-07T02:29:33.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.339+0000 7f1b26a00700 1 -- 172.21.15.19:0/1031535659 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b201a2210 con 0x7f1b20075d30 2024-05-07T02:29:33.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.339+0000 7f1b1cc00700 1 -- 172.21.15.19:0/1031535659 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b10018680 con 0x7f1b20075d30 2024-05-07T02:29:33.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.339+0000 7f1b26a00700 1 -- 172.21.15.19:0/1031535659 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b201a9b30 con 0x7f1b20075d30 2024-05-07T02:29:33.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.341+0000 7f1b26a00700 1 -- 172.21.15.19:0/1031535659 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b200512c0 con 0x7f1b20075d30 2024-05-07T02:29:33.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.343+0000 7f1b1cc00700 1 -- 172.21.15.19:0/1031535659 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f1b10010430 con 0x7f1b20075d30 2024-05-07T02:29:33.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.345+0000 7f1b1cc00700 1 --2- 172.21.15.19:0/1031535659 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f1b040b2910 0x7f1b040b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:33.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.345+0000 7f1b1cc00700 1 -- 172.21.15.19:0/1031535659 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f1b100e2900 con 0x7f1b20075d30 2024-05-07T02:29:33.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.345+0000 7f1b1f400700 1 --2- 172.21.15.19:0/1031535659 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f1b040b2910 0x7f1b040b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:33.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.346+0000 7f1b1f400700 1 --2- 172.21.15.19:0/1031535659 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f1b040b2910 0x7f1b040b4e20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1b201034d0 tx=0x7f1b0c004710 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:33.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.349+0000 7f1b1cc00700 1 -- 172.21.15.19:0/1031535659 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1b100aa0b0 con 0x7f1b20075d30 2024-05-07T02:29:33.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.616+0000 7f1b26a00700 1 -- 172.21.15.19:0/1031535659 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f1b2002e5f0 con 0x7f1b20075d30 2024-05-07T02:29:33.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.617+0000 7f1b1cc00700 1 -- 172.21.15.19:0/1031535659 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1b100b2130 con 0x7f1b20075d30 2024-05-07T02:29:33.937 INFO:teuthology.orchestra.run.smithi019.stdout:253403070532 2024-05-07T02:29:33.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.620+0000 7f1b26a00700 1 -- 172.21.15.19:0/1031535659 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f1b040b2910 msgr2=0x7f1b040b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:33.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.620+0000 7f1b26a00700 1 --2- 172.21.15.19:0/1031535659 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f1b040b2910 0x7f1b040b4e20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1b201034d0 tx=0x7f1b0c004710 comp rx=0 tx=0).stop 2024-05-07T02:29:33.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.620+0000 7f1b26a00700 1 -- 172.21.15.19:0/1031535659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b20075d30 msgr2=0x7f1b2019d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:33.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.620+0000 7f1b26a00700 1 --2- 172.21.15.19:0/1031535659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b20075d30 0x7f1b2019d050 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1b1000e9f0 tx=0x7f1b1000ed00 comp rx=0 tx=0).stop 2024-05-07T02:29:33.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.620+0000 7f1b26a00700 1 -- 172.21.15.19:0/1031535659 shutdown_connections 2024-05-07T02:29:33.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.620+0000 7f1b26a00700 1 --2- 172.21.15.19:0/1031535659 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f1b040b2910 0x7f1b040b4e20 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:33.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.620+0000 7f1b26a00700 1 --2- 172.21.15.19:0/1031535659 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b20110ba0 0x7f1b201a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:33.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.620+0000 7f1b26a00700 1 --2- 172.21.15.19:0/1031535659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1b20075d30 0x7f1b2019d050 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:33.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.620+0000 7f1b26a00700 1 --2- 172.21.15.19:0/1031535659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b20076680 0x7f1b2019d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:33.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.620+0000 7f1b26a00700 1 -- 172.21.15.19:0/1031535659 >> 172.21.15.19:0/1031535659 conn(0x7f1b20100040 msgr2=0x7f1b20100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:33.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.620+0000 7f1b26a00700 1 -- 172.21.15.19:0/1031535659 shutdown_connections 2024-05-07T02:29:33.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:28.620+0000 7f1b26a00700 1 -- 172.21.15.19:0/1031535659 wait complete. 2024-05-07T02:29:33.938 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:33 smithi019 ceph-mon[29781]: mgrmap e18: x(active, since 12m), standbys: z, y 2024-05-07T02:29:33.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:33 smithi027 ceph-mon[34398]: mgrmap e18: x(active, since 12m), standbys: z, y 2024-05-07T02:29:34.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:33 smithi064 ceph-mon[34699]: mgrmap e18: x(active, since 12m), standbys: z, y 2024-05-07T02:29:34.876 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:34 smithi019 ceph-mon[29781]: pgmap v440: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-07T02:29:34.876 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:34 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/3407043821' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-07T02:29:34.945 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574960 got 55834574965 for osd.1 2024-05-07T02:29:34.945 DEBUG:teuthology.parallel:result is None 2024-05-07T02:29:34.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:34 smithi027 ceph-mon[34398]: pgmap v440: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-07T02:29:34.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:34 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/3407043821' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-07T02:29:35.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.701+0000 7faf58e00700 1 -- 172.21.15.19:0/773416959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf54104400 msgr2=0x7faf54104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:35.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.701+0000 7faf58e00700 1 --2- 172.21.15.19:0/773416959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf54104400 0x7faf54104810 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7faf440099c0 tx=0x7faf44009cd0 comp rx=0 tx=0).stop 2024-05-07T02:29:35.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.701+0000 7faf58e00700 1 -- 172.21.15.19:0/773416959 shutdown_connections 2024-05-07T02:29:35.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.701+0000 7faf58e00700 1 --2- 172.21.15.19:0/773416959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf54109680 0x7faf54110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:35.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.701+0000 7faf58e00700 1 --2- 172.21.15.19:0/773416959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf54104400 0x7faf54104810 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:35.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.701+0000 7faf58e00700 1 --2- 172.21.15.19:0/773416959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf54108d30 0x7faf54109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:35.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.701+0000 7faf58e00700 1 -- 172.21.15.19:0/773416959 >> 172.21.15.19:0/773416959 conn(0x7faf54079d50 msgr2=0x7faf5407a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:35.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.701+0000 7faf58e00700 1 -- 172.21.15.19:0/773416959 shutdown_connections 2024-05-07T02:29:35.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.701+0000 7faf58e00700 1 -- 172.21.15.19:0/773416959 wait complete. 2024-05-07T02:29:35.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.702+0000 7faf58e00700 1 Processor -- start 2024-05-07T02:29:35.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.703+0000 7faf58e00700 1 -- start start 2024-05-07T02:29:35.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.703+0000 7faf58e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf54104400 0x7faf5419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:35.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.703+0000 7faf58e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf54108d30 0x7faf5419d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:35.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.703+0000 7faf58e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf54109680 0x7faf541a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:35.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.703+0000 7faf58e00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faf541142b0 con 0x7faf54108d30 2024-05-07T02:29:35.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.703+0000 7faf58e00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7faf54114130 con 0x7faf54104400 2024-05-07T02:29:35.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.703+0000 7faf58e00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7faf54114430 con 0x7faf54109680 2024-05-07T02:29:35.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.703+0000 7faf4be00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf54108d30 0x7faf5419d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:35.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.703+0000 7faf4be00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf54108d30 0x7faf5419d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40184/0 (socket says 172.21.15.19:40184) 2024-05-07T02:29:35.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.704+0000 7faf4be00700 1 -- 172.21.15.19:0/3407043821 learned_addr learned my addr 172.21.15.19:0/3407043821 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:29:35.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.704+0000 7faf53400700 1 --2- 172.21.15.19:0/3407043821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf54104400 0x7faf5419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:35.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.704+0000 7faf53e00700 1 --2- 172.21.15.19:0/3407043821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf54109680 0x7faf541a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:35.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.704+0000 7faf4be00700 1 -- 172.21.15.19:0/3407043821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf54109680 msgr2=0x7faf541a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:35.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.704+0000 7faf4be00700 1 --2- 172.21.15.19:0/3407043821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf54109680 0x7faf541a1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:35.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.704+0000 7faf4be00700 1 -- 172.21.15.19:0/3407043821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf54104400 msgr2=0x7faf5419ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:35.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.704+0000 7faf4be00700 1 --2- 172.21.15.19:0/3407043821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf54104400 0x7faf5419ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:35.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.704+0000 7faf4be00700 1 -- 172.21.15.19:0/3407043821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf541a1d10 con 0x7faf54108d30 2024-05-07T02:29:35.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.704+0000 7faf4be00700 1 --2- 172.21.15.19:0/3407043821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf54108d30 0x7faf5419d3e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7faf3c00e970 tx=0x7faf3c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:35.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.705+0000 7faf50c00700 1 -- 172.21.15.19:0/3407043821 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf3c004a30 con 0x7faf54108d30 2024-05-07T02:29:35.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.705+0000 7faf50c00700 1 -- 172.21.15.19:0/3407043821 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf3c00cc90 con 0x7faf54108d30 2024-05-07T02:29:35.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.705+0000 7faf58e00700 1 -- 172.21.15.19:0/3407043821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf541a2000 con 0x7faf54108d30 2024-05-07T02:29:35.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.705+0000 7faf50c00700 1 -- 172.21.15.19:0/3407043821 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf3c005610 con 0x7faf54108d30 2024-05-07T02:29:35.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.705+0000 7faf58e00700 1 -- 172.21.15.19:0/3407043821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf541a9920 con 0x7faf54108d30 2024-05-07T02:29:35.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.706+0000 7faf58e00700 1 -- 172.21.15.19:0/3407043821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf540512c0 con 0x7faf54108d30 2024-05-07T02:29:35.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.709+0000 7faf50c00700 1 -- 172.21.15.19:0/3407043821 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7faf3c00c7e0 con 0x7faf54108d30 2024-05-07T02:29:35.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.710+0000 7faf50c00700 1 --2- 172.21.15.19:0/3407043821 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7faf380b2880 0x7faf380b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:35.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.711+0000 7faf50c00700 1 -- 172.21.15.19:0/3407043821 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7faf3c0e29b0 con 0x7faf54108d30 2024-05-07T02:29:35.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.711+0000 7faf53400700 1 --2- 172.21.15.19:0/3407043821 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7faf380b2880 0x7faf380b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:35.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.711+0000 7faf53400700 1 --2- 172.21.15.19:0/3407043821 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7faf380b2880 0x7faf380b4d90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7faf44003760 tx=0x7faf4401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:35.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.713+0000 7faf50c00700 1 -- 172.21.15.19:0/3407043821 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faf3c054df0 con 0x7faf54108d30 2024-05-07T02:29:35.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.955+0000 7faf58e00700 1 -- 172.21.15.19:0/3407043821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7faf541a9af0 con 0x7faf54108d30 2024-05-07T02:29:35.089 INFO:teuthology.orchestra.run.smithi019.stdout:180388626519 2024-05-07T02:29:35.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.956+0000 7faf50c00700 1 -- 172.21.15.19:0/3407043821 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7faf3c0aa2c0 con 0x7faf54108d30 2024-05-07T02:29:35.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.959+0000 7faf58e00700 1 -- 172.21.15.19:0/3407043821 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7faf380b2880 msgr2=0x7faf380b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:35.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.959+0000 7faf58e00700 1 --2- 172.21.15.19:0/3407043821 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7faf380b2880 0x7faf380b4d90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7faf44003760 tx=0x7faf4401a040 comp rx=0 tx=0).stop 2024-05-07T02:29:35.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.959+0000 7faf58e00700 1 -- 172.21.15.19:0/3407043821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf54108d30 msgr2=0x7faf5419d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:35.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.959+0000 7faf58e00700 1 --2- 172.21.15.19:0/3407043821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf54108d30 0x7faf5419d3e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7faf3c00e970 tx=0x7faf3c00ec80 comp rx=0 tx=0).stop 2024-05-07T02:29:35.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.959+0000 7faf58e00700 1 -- 172.21.15.19:0/3407043821 shutdown_connections 2024-05-07T02:29:35.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.959+0000 7faf58e00700 1 --2- 172.21.15.19:0/3407043821 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faf54109680 0x7faf541a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:35.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.959+0000 7faf58e00700 1 --2- 172.21.15.19:0/3407043821 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7faf380b2880 0x7faf380b4d90 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:35.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.959+0000 7faf58e00700 1 --2- 172.21.15.19:0/3407043821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf54104400 0x7faf5419ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:35.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.959+0000 7faf58e00700 1 --2- 172.21.15.19:0/3407043821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf54108d30 0x7faf5419d3e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:35.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.959+0000 7faf58e00700 1 -- 172.21.15.19:0/3407043821 >> 172.21.15.19:0/3407043821 conn(0x7faf54079d50 msgr2=0x7faf54110ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:35.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.959+0000 7faf58e00700 1 -- 172.21.15.19:0/3407043821 shutdown_connections 2024-05-07T02:29:35.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:33.959+0000 7faf58e00700 1 -- 172.21.15.19:0/3407043821 wait complete. 2024-05-07T02:29:35.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:34 smithi064 ceph-mon[34699]: pgmap v440: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-07T02:29:35.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:34 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/3407043821' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-07T02:29:35.429 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:29:35.777 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:35 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/3424121807' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-07T02:29:35.842 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626514 got 180388626519 for osd.6 2024-05-07T02:29:35.842 DEBUG:teuthology.parallel:result is None 2024-05-07T02:29:35.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:35 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/3424121807' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-07T02:29:36.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:35 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/3424121807' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-07T02:29:36.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.240+0000 7f43ae800700 1 -- 172.21.15.19:0/188549341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43a8104400 msgr2=0x7f43a8104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:36.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.240+0000 7f43ae800700 1 --2- 172.21.15.19:0/188549341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43a8104400 0x7f43a8104810 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f439c00b0d0 tx=0x7f439c00b3e0 comp rx=0 tx=0).stop 2024-05-07T02:29:36.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.240+0000 7f43ae800700 1 -- 172.21.15.19:0/188549341 shutdown_connections 2024-05-07T02:29:36.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.240+0000 7f43ae800700 1 --2- 172.21.15.19:0/188549341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43a8109680 0x7f43a8110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:36.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.240+0000 7f43ae800700 1 --2- 172.21.15.19:0/188549341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43a8104400 0x7f43a8104810 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:36.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.240+0000 7f43ae800700 1 --2- 172.21.15.19:0/188549341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43a8108d30 0x7f43a8109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:36.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.240+0000 7f43ae800700 1 -- 172.21.15.19:0/188549341 >> 172.21.15.19:0/188549341 conn(0x7f43a8079d10 msgr2=0x7f43a807a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:36.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.240+0000 7f43ae800700 1 -- 172.21.15.19:0/188549341 shutdown_connections 2024-05-07T02:29:36.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.240+0000 7f43ae800700 1 -- 172.21.15.19:0/188549341 wait complete. 2024-05-07T02:29:36.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.241+0000 7f43ae800700 1 Processor -- start 2024-05-07T02:29:36.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.241+0000 7f43ae800700 1 -- start start 2024-05-07T02:29:36.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.242+0000 7f43ae800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43a8104400 0x7f43a8075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:36.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.242+0000 7f43ae800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43a8108d30 0x7f43a8076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:36.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.242+0000 7f43ae800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43a8109680 0x7f43a8076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:36.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.242+0000 7f43ae800700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f43a8114300 con 0x7f43a8108d30 2024-05-07T02:29:36.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.242+0000 7f43ae800700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f43a8114180 con 0x7f43a8104400 2024-05-07T02:29:36.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.242+0000 7f43ae800700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f43a8114480 con 0x7f43a8109680 2024-05-07T02:29:36.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.242+0000 7f43aca00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43a8108d30 0x7f43a8076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:36.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.242+0000 7f43ade00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43a8109680 0x7f43a8076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:36.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.242+0000 7f43ad400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43a8104400 0x7f43a8075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:36.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.242+0000 7f43aca00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43a8108d30 0x7f43a8076000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40198/0 (socket says 172.21.15.19:40198) 2024-05-07T02:29:36.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.242+0000 7f43aca00700 1 -- 172.21.15.19:0/3424121807 learned_addr learned my addr 172.21.15.19:0/3424121807 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:29:36.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.243+0000 7f43aca00700 1 -- 172.21.15.19:0/3424121807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43a8109680 msgr2=0x7f43a8076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:36.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.243+0000 7f43aca00700 1 --2- 172.21.15.19:0/3424121807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43a8109680 0x7f43a8076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:36.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.243+0000 7f43aca00700 1 -- 172.21.15.19:0/3424121807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43a8104400 msgr2=0x7f43a8075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:36.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.243+0000 7f43aca00700 1 --2- 172.21.15.19:0/3424121807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43a8104400 0x7f43a8075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:36.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.243+0000 7f43aca00700 1 -- 172.21.15.19:0/3424121807 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43a81a9050 con 0x7f43a8108d30 2024-05-07T02:29:36.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.243+0000 7f43ad400700 1 --2- 172.21.15.19:0/3424121807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43a8104400 0x7f43a8075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:29:36.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.243+0000 7f43aca00700 1 --2- 172.21.15.19:0/3424121807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43a8108d30 0x7f43a8076000 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f439400ea30 tx=0x7f439400ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:36.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.243+0000 7f43a6000700 1 -- 172.21.15.19:0/3424121807 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439400cbd0 con 0x7f43a8108d30 2024-05-07T02:29:36.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.244+0000 7f43a6000700 1 -- 172.21.15.19:0/3424121807 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43940044d0 con 0x7f43a8108d30 2024-05-07T02:29:36.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.244+0000 7f43ae800700 1 -- 172.21.15.19:0/3424121807 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43a81a9340 con 0x7f43a8108d30 2024-05-07T02:29:36.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.244+0000 7f43a6000700 1 -- 172.21.15.19:0/3424121807 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4394003dc0 con 0x7f43a8108d30 2024-05-07T02:29:36.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.244+0000 7f43ae800700 1 -- 172.21.15.19:0/3424121807 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43a81a98d0 con 0x7f43a8108d30 2024-05-07T02:29:36.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.245+0000 7f43ae800700 1 -- 172.21.15.19:0/3424121807 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43a81058b0 con 0x7f43a8108d30 2024-05-07T02:29:36.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.247+0000 7f43a6000700 1 -- 172.21.15.19:0/3424121807 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4394004000 con 0x7f43a8108d30 2024-05-07T02:29:36.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.249+0000 7f43a6000700 1 --2- 172.21.15.19:0/3424121807 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f43900b2900 0x7f43900b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:36.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.249+0000 7f43a6000700 1 -- 172.21.15.19:0/3424121807 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f43940e2780 con 0x7f43a8108d30 2024-05-07T02:29:36.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.249+0000 7f43ad400700 1 --2- 172.21.15.19:0/3424121807 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f43900b2900 0x7f43900b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:36.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.250+0000 7f43ad400700 1 --2- 172.21.15.19:0/3424121807 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f43900b2900 0x7f43900b4e10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f439c00bfd0 tx=0x7f439c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:36.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.251+0000 7f43a6000700 1 -- 172.21.15.19:0/3424121807 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f43940042f0 con 0x7f43a8108d30 2024-05-07T02:29:36.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.503+0000 7f43ae800700 1 -- 172.21.15.19:0/3424121807 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f43a8076ff0 con 0x7f43a8108d30 2024-05-07T02:29:36.224 INFO:teuthology.orchestra.run.smithi019.stdout:227633266765 2024-05-07T02:29:36.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.504+0000 7f43a6000700 1 -- 172.21.15.19:0/3424121807 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f43940aa120 con 0x7f43a8108d30 2024-05-07T02:29:36.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.507+0000 7f43ae800700 1 -- 172.21.15.19:0/3424121807 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f43900b2900 msgr2=0x7f43900b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:36.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.507+0000 7f43ae800700 1 --2- 172.21.15.19:0/3424121807 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f43900b2900 0x7f43900b4e10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f439c00bfd0 tx=0x7f439c00b040 comp rx=0 tx=0).stop 2024-05-07T02:29:36.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.507+0000 7f43ae800700 1 -- 172.21.15.19:0/3424121807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43a8108d30 msgr2=0x7f43a8076000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:36.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.507+0000 7f43ae800700 1 --2- 172.21.15.19:0/3424121807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43a8108d30 0x7f43a8076000 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f439400ea30 tx=0x7f439400ed40 comp rx=0 tx=0).stop 2024-05-07T02:29:36.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.507+0000 7f43ae800700 1 -- 172.21.15.19:0/3424121807 shutdown_connections 2024-05-07T02:29:36.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.507+0000 7f43ae800700 1 --2- 172.21.15.19:0/3424121807 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43a8109680 0x7f43a8076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:36.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.507+0000 7f43ae800700 1 --2- 172.21.15.19:0/3424121807 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f43900b2900 0x7f43900b4e10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:36.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.507+0000 7f43ae800700 1 --2- 172.21.15.19:0/3424121807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43a8104400 0x7f43a8075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:36.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.507+0000 7f43ae800700 1 --2- 172.21.15.19:0/3424121807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43a8108d30 0x7f43a8076000 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:36.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.507+0000 7f43ae800700 1 -- 172.21.15.19:0/3424121807 >> 172.21.15.19:0/3424121807 conn(0x7f43a8079d10 msgr2=0x7f43a8102910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:36.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.507+0000 7f43ae800700 1 -- 172.21.15.19:0/3424121807 shutdown_connections 2024-05-07T02:29:36.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:35.507+0000 7f43ae800700 1 -- 172.21.15.19:0/3424121807 wait complete. 2024-05-07T02:29:36.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:29.950+0000 7f79cc000700 1 -- 172.21.15.19:0/358244904 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79c4110ba0 msgr2=0x7f79c4113000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:36.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:29.950+0000 7f79cc000700 1 --2- 172.21.15.19:0/358244904 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79c4110ba0 0x7f79c4113000 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f79bc0098f0 tx=0x7f79bc009c00 comp rx=0 tx=0).stop 2024-05-07T02:29:36.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:29.950+0000 7f79cc000700 1 -- 172.21.15.19:0/358244904 shutdown_connections 2024-05-07T02:29:36.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:29.950+0000 7f79cc000700 1 --2- 172.21.15.19:0/358244904 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79c4110ba0 0x7f79c4113000 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:36.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:29.950+0000 7f79cc000700 1 --2- 172.21.15.19:0/358244904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79c4075d30 0x7f79c4076140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:36.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:29.950+0000 7f79cc000700 1 --2- 172.21.15.19:0/358244904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79c4076680 0x7f79c4110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:36.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:29.950+0000 7f79cc000700 1 -- 172.21.15.19:0/358244904 >> 172.21.15.19:0/358244904 conn(0x7f79c4100040 msgr2=0x7f79c4102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:36.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:29.950+0000 7f79cc000700 1 -- 172.21.15.19:0/358244904 shutdown_connections 2024-05-07T02:29:36.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:29.950+0000 7f79cc000700 1 -- 172.21.15.19:0/358244904 wait complete. 2024-05-07T02:29:36.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:29.952+0000 7f79cc000700 1 Processor -- start 2024-05-07T02:29:36.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:29.952+0000 7f79cc000700 1 -- start start 2024-05-07T02:29:36.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:29.952+0000 7f79cc000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79c4075d30 0x7f79c419d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:36.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:29.952+0000 7f79cc000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79c4076680 0x7f79c419d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:36.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:29.952+0000 7f79cc000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79c4110ba0 0x7f79c41a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:36.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:29.952+0000 7f79cc000700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f79c41166d0 con 0x7f79c4075d30 2024-05-07T02:29:36.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:29.952+0000 7f79cc000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f79c4116550 con 0x7f79c4076680 2024-05-07T02:29:36.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:29.952+0000 7f79cc000700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f79c4116850 con 0x7f79c4110ba0 2024-05-07T02:29:36.650 INFO:teuthology.orchestra.run.smithi019.stdout:274877907007 2024-05-07T02:29:36.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:29.953+0000 7f79c9600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79c4075d30 0x7f79c419d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:36.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:29.953+0000 7f79c9600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79c4075d30 0x7f79c419d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40126/0 (socket says 172.21.15.19:40126) 2024-05-07T02:29:36.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:29.953+0000 7f79c9600700 1 -- 172.21.15.19:0/2863931741 learned_addr learned my addr 172.21.15.19:0/2863931741 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:29:36.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:29.953+0000 7f79c8c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79c4076680 0x7f79c419d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:36.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:29.953+0000 7f79ca000700 1 --2- 172.21.15.19:0/2863931741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79c4110ba0 0x7f79c41a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:36.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:29.953+0000 7f79c9600700 1 -- 172.21.15.19:0/2863931741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79c4110ba0 msgr2=0x7f79c41a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:36.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:29.953+0000 7f79c9600700 1 --2- 172.21.15.19:0/2863931741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79c4110ba0 0x7f79c41a18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:36.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:29.953+0000 7f79c9600700 1 -- 172.21.15.19:0/2863931741 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79c4076680 msgr2=0x7f79c419d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:36.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:29.953+0000 7f79c9600700 1 --2- 172.21.15.19:0/2863931741 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79c4076680 0x7f79c419d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:36.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:29.953+0000 7f79c9600700 1 -- 172.21.15.19:0/2863931741 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79c41a1f50 con 0x7f79c4075d30 2024-05-07T02:29:36.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:29.953+0000 7f79c8c00700 1 --2- 172.21.15.19:0/2863931741 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79c4076680 0x7f79c419d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:29:36.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:29.953+0000 7f79c9600700 1 --2- 172.21.15.19:0/2863931741 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79c4075d30 0x7f79c419d080 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f79b8007f10 tx=0x7f79b800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:36.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:29.954+0000 7f79c2000700 1 -- 172.21.15.19:0/2863931741 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b800ddd0 con 0x7f79c4075d30 2024-05-07T02:29:36.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:29.954+0000 7f79cc000700 1 -- 172.21.15.19:0/2863931741 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79c41a2240 con 0x7f79c4075d30 2024-05-07T02:29:36.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:29.954+0000 7f79c2000700 1 -- 172.21.15.19:0/2863931741 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79b8004020 con 0x7f79c4075d30 2024-05-07T02:29:36.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:29.954+0000 7f79c2000700 1 -- 172.21.15.19:0/2863931741 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b8015620 con 0x7f79c4075d30 2024-05-07T02:29:36.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:29.954+0000 7f79cc000700 1 -- 172.21.15.19:0/2863931741 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79c41a9b60 con 0x7f79c4075d30 2024-05-07T02:29:36.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:29.955+0000 7f79cc000700 1 -- 172.21.15.19:0/2863931741 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79c40512c0 con 0x7f79c4075d30 2024-05-07T02:29:36.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:29.958+0000 7f79c2000700 1 -- 172.21.15.19:0/2863931741 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f79b80041d0 con 0x7f79c4075d30 2024-05-07T02:29:36.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:29.960+0000 7f79c2000700 1 --2- 172.21.15.19:0/2863931741 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f79ac0b2910 0x7f79ac0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:36.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:29.961+0000 7f79c2000700 1 -- 172.21.15.19:0/2863931741 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f79b80e2b20 con 0x7f79c4075d30 2024-05-07T02:29:36.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:29.961+0000 7f79c8c00700 1 --2- 172.21.15.19:0/2863931741 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f79ac0b2910 0x7f79ac0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:36.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:29.962+0000 7f79c8c00700 1 --2- 172.21.15.19:0/2863931741 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f79ac0b2910 0x7f79ac0b4e20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f79c4103470 tx=0x7f79b0009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:36.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:29.964+0000 7f79c2000700 1 -- 172.21.15.19:0/2863931741 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f79b8023070 con 0x7f79c4075d30 2024-05-07T02:29:36.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.222+0000 7f79cc000700 1 -- 172.21.15.19:0/2863931741 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f79c402e5f0 con 0x7f79c4075d30 2024-05-07T02:29:36.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.223+0000 7f79c2000700 1 -- 172.21.15.19:0/2863931741 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f79b8010070 con 0x7f79c4075d30 2024-05-07T02:29:36.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.226+0000 7f79cc000700 1 -- 172.21.15.19:0/2863931741 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f79ac0b2910 msgr2=0x7f79ac0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:36.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.226+0000 7f79cc000700 1 --2- 172.21.15.19:0/2863931741 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f79ac0b2910 0x7f79ac0b4e20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f79c4103470 tx=0x7f79b0009310 comp rx=0 tx=0).stop 2024-05-07T02:29:36.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.226+0000 7f79cc000700 1 -- 172.21.15.19:0/2863931741 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79c4075d30 msgr2=0x7f79c419d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:36.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.226+0000 7f79cc000700 1 --2- 172.21.15.19:0/2863931741 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79c4075d30 0x7f79c419d080 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f79b8007f10 tx=0x7f79b800d3b0 comp rx=0 tx=0).stop 2024-05-07T02:29:36.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.227+0000 7f79cc000700 1 -- 172.21.15.19:0/2863931741 shutdown_connections 2024-05-07T02:29:36.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.227+0000 7f79cc000700 1 --2- 172.21.15.19:0/2863931741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79c4110ba0 0x7f79c41a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:36.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.227+0000 7f79cc000700 1 --2- 172.21.15.19:0/2863931741 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79c4075d30 0x7f79c419d080 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:36.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.227+0000 7f79cc000700 1 --2- 172.21.15.19:0/2863931741 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f79ac0b2910 0x7f79ac0b4e20 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:36.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.227+0000 7f79cc000700 1 --2- 172.21.15.19:0/2863931741 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79c4076680 0x7f79c419d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:36.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.227+0000 7f79cc000700 1 -- 172.21.15.19:0/2863931741 >> 172.21.15.19:0/2863931741 conn(0x7f79c4100040 msgr2=0x7f79c41006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:36.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.227+0000 7f79cc000700 1 -- 172.21.15.19:0/2863931741 shutdown_connections 2024-05-07T02:29:36.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:30.227+0000 7f79cc000700 1 -- 172.21.15.19:0/2863931741 wait complete. 2024-05-07T02:29:36.828 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:36 smithi019 ceph-mon[29781]: pgmap v441: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-07T02:29:36.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:36 smithi027 ceph-mon[34398]: pgmap v441: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-07T02:29:36.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.574+0000 7fc081800700 1 -- 172.21.15.19:0/3953731704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc07c10e720 msgr2=0x7fc07c110c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:36.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.574+0000 7fc081800700 1 --2- 172.21.15.19:0/3953731704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc07c10e720 0x7fc07c110c00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc074009990 tx=0x7fc074009ca0 comp rx=0 tx=0).stop 2024-05-07T02:29:36.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.574+0000 7fc081800700 1 -- 172.21.15.19:0/3953731704 shutdown_connections 2024-05-07T02:29:36.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.574+0000 7fc081800700 1 --2- 172.21.15.19:0/3953731704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc07c10e720 0x7fc07c110c00 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:36.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.574+0000 7fc081800700 1 --2- 172.21.15.19:0/3953731704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc07c1052f0 0x7fc07c105700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:36.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.574+0000 7fc081800700 1 --2- 172.21.15.19:0/3953731704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc07c105c40 0x7fc07c10e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:36.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.574+0000 7fc081800700 1 -- 172.21.15.19:0/3953731704 >> 172.21.15.19:0/3953731704 conn(0x7fc07c078b00 msgr2=0x7fc07c078f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:36.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.575+0000 7fc081800700 1 -- 172.21.15.19:0/3953731704 shutdown_connections 2024-05-07T02:29:36.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.575+0000 7fc081800700 1 -- 172.21.15.19:0/3953731704 wait complete. 2024-05-07T02:29:36.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.576+0000 7fc081800700 1 Processor -- start 2024-05-07T02:29:36.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.576+0000 7fc081800700 1 -- start start 2024-05-07T02:29:36.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.576+0000 7fc081800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc07c1052f0 0x7fc07c1a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:36.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.577+0000 7fc081800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc07c105c40 0x7fc07c1a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:36.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.577+0000 7fc081800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc07c10e720 0x7fc07c19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:36.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.577+0000 7fc081800700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc07c1142d0 con 0x7fc07c10e720 2024-05-07T02:29:36.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.577+0000 7fc081800700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc07c114150 con 0x7fc07c105c40 2024-05-07T02:29:36.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.577+0000 7fc081800700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc07c114450 con 0x7fc07c1052f0 2024-05-07T02:29:36.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.577+0000 7fc07be00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc07c1052f0 0x7fc07c1a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:36.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.577+0000 7fc07be00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc07c1052f0 0x7fc07c1a07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:34630/0 (socket says 172.21.15.19:34630) 2024-05-07T02:29:36.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.577+0000 7fc07be00700 1 -- 172.21.15.19:0/2257470156 learned_addr learned my addr 172.21.15.19:0/2257470156 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:29:36.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.577+0000 7fc07b400700 1 --2- 172.21.15.19:0/2257470156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc07c105c40 0x7fc07c1a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:36.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.577+0000 7fc080e00700 1 --2- 172.21.15.19:0/2257470156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc07c10e720 0x7fc07c19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:36.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.577+0000 7fc07be00700 1 -- 172.21.15.19:0/2257470156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc07c105c40 msgr2=0x7fc07c1a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:36.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.577+0000 7fc07be00700 1 --2- 172.21.15.19:0/2257470156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc07c105c40 0x7fc07c1a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:36.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.577+0000 7fc07be00700 1 -- 172.21.15.19:0/2257470156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc07c10e720 msgr2=0x7fc07c19aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:36.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.577+0000 7fc07be00700 1 --2- 172.21.15.19:0/2257470156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc07c10e720 0x7fc07c19aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:36.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.577+0000 7fc07be00700 1 -- 172.21.15.19:0/2257470156 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc07c19b0c0 con 0x7fc07c1052f0 2024-05-07T02:29:36.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.577+0000 7fc080e00700 1 --2- 172.21.15.19:0/2257470156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc07c10e720 0x7fc07c19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:29:36.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.578+0000 7fc07be00700 1 --2- 172.21.15.19:0/2257470156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc07c1052f0 0x7fc07c1a07f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc06c00ef10 tx=0x7fc06c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:36.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.578+0000 7fc078c00700 1 -- 172.21.15.19:0/2257470156 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c00ca70 con 0x7fc07c1052f0 2024-05-07T02:29:36.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.578+0000 7fc078c00700 1 -- 172.21.15.19:0/2257470156 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc06c019070 con 0x7fc07c1052f0 2024-05-07T02:29:36.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.578+0000 7fc078c00700 1 -- 172.21.15.19:0/2257470156 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c014480 con 0x7fc07c1052f0 2024-05-07T02:29:36.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.578+0000 7fc081800700 1 -- 172.21.15.19:0/2257470156 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc07c19b350 con 0x7fc07c1052f0 2024-05-07T02:29:36.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.578+0000 7fc081800700 1 -- 172.21.15.19:0/2257470156 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc07c19b840 con 0x7fc07c1052f0 2024-05-07T02:29:36.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.579+0000 7fc081800700 1 -- 172.21.15.19:0/2257470156 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc07c06c8c0 con 0x7fc07c1052f0 2024-05-07T02:29:36.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.582+0000 7fc078c00700 1 -- 172.21.15.19:0/2257470156 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc06c012050 con 0x7fc07c1052f0 2024-05-07T02:29:36.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.584+0000 7fc078c00700 1 --2- 172.21.15.19:0/2257470156 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fc0600b2900 0x7fc0600b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:36.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.584+0000 7fc078c00700 1 -- 172.21.15.19:0/2257470156 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fc06c0e2c70 con 0x7fc07c1052f0 2024-05-07T02:29:36.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.584+0000 7fc07b400700 1 --2- 172.21.15.19:0/2257470156 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fc0600b2900 0x7fc0600b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:36.995 INFO:teuthology.orchestra.run.smithi019.stdout:77309411441 2024-05-07T02:29:36.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.585+0000 7fc07b400700 1 --2- 172.21.15.19:0/2257470156 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fc0600b2900 0x7fc0600b4e10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc07c102540 tx=0x7fc068009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:36.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.586+0000 7fc078c00700 1 -- 172.21.15.19:0/2257470156 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc07c19b840 con 0x7fc07c1052f0 2024-05-07T02:29:36.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.828+0000 7fc081800700 1 -- 172.21.15.19:0/2257470156 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fc07c02e5f0 con 0x7fc07c1052f0 2024-05-07T02:29:36.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.829+0000 7fc078c00700 1 -- 172.21.15.19:0/2257470156 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc06c010040 con 0x7fc07c1052f0 2024-05-07T02:29:36.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.832+0000 7fc081800700 1 -- 172.21.15.19:0/2257470156 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fc0600b2900 msgr2=0x7fc0600b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:36.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.832+0000 7fc081800700 1 --2- 172.21.15.19:0/2257470156 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fc0600b2900 0x7fc0600b4e10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc07c102540 tx=0x7fc068009260 comp rx=0 tx=0).stop 2024-05-07T02:29:36.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.832+0000 7fc081800700 1 -- 172.21.15.19:0/2257470156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc07c1052f0 msgr2=0x7fc07c1a07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:36.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.832+0000 7fc081800700 1 --2- 172.21.15.19:0/2257470156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc07c1052f0 0x7fc07c1a07f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc06c00ef10 tx=0x7fc06c00c3a0 comp rx=0 tx=0).stop 2024-05-07T02:29:36.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.832+0000 7fc081800700 1 -- 172.21.15.19:0/2257470156 shutdown_connections 2024-05-07T02:29:36.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.832+0000 7fc081800700 1 --2- 172.21.15.19:0/2257470156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc07c10e720 0x7fc07c19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:36.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.832+0000 7fc081800700 1 --2- 172.21.15.19:0/2257470156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc07c1052f0 0x7fc07c1a07f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:36.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.832+0000 7fc081800700 1 --2- 172.21.15.19:0/2257470156 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fc0600b2900 0x7fc0600b4e10 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:36.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.832+0000 7fc081800700 1 --2- 172.21.15.19:0/2257470156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc07c105c40 0x7fc07c1a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:36.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.832+0000 7fc081800700 1 -- 172.21.15.19:0/2257470156 >> 172.21.15.19:0/2257470156 conn(0x7fc07c078b00 msgr2=0x7fc07c0ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:36.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.833+0000 7fc081800700 1 -- 172.21.15.19:0/2257470156 shutdown_connections 2024-05-07T02:29:36.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:36.833+0000 7fc081800700 1 -- 172.21.15.19:0/2257470156 wait complete. 2024-05-07T02:29:37.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:36 smithi064 ceph-mon[34699]: pgmap v441: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-07T02:29:37.806 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266759 got 227633266765 for osd.8 2024-05-07T02:29:37.806 DEBUG:teuthology.parallel:result is None 2024-05-07T02:29:37.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:37 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/2257470156' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-07T02:29:38.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:37 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/2257470156' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-07T02:29:38.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:37 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/2257470156' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-07T02:29:38.323 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070528 got 253403070532 for osd.9 2024-05-07T02:29:38.323 DEBUG:teuthology.parallel:result is None 2024-05-07T02:29:38.880 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:38 smithi019 ceph-mon[29781]: pgmap v442: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-07T02:29:38.884 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877907003 got 274877907007 for osd.10 2024-05-07T02:29:38.884 DEBUG:teuthology.parallel:result is None 2024-05-07T02:29:38.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:38 smithi027 ceph-mon[34398]: pgmap v442: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-07T02:29:39.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:38 smithi064 ceph-mon[34699]: pgmap v442: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-07T02:29:39.351 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:29:39.398 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710771 got 300647710776 for osd.11 2024-05-07T02:29:39.398 DEBUG:teuthology.parallel:result is None 2024-05-07T02:29:40.628 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738483 got 34359738491 for osd.0 2024-05-07T02:29:40.628 DEBUG:teuthology.parallel:result is None 2024-05-07T02:29:40.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:40 smithi027 ceph-mon[34398]: pgmap v443: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-07T02:29:41.057 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:40 smithi019 ceph-mon[29781]: pgmap v443: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-07T02:29:41.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:40 smithi064 ceph-mon[34699]: pgmap v443: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-05-07T02:29:41.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:41 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/3081858335' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-07T02:29:42.006 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:41 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/3081858335' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-07T02:29:42.009 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411436 got 77309411441 for osd.2 2024-05-07T02:29:42.009 DEBUG:teuthology.parallel:result is None 2024-05-07T02:29:42.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:41 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/3081858335' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-07T02:29:42.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.426+0000 7f4625200700 1 -- 172.21.15.19:0/2898745315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4620075d30 msgr2=0x7f4620076140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:42.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.426+0000 7f4625200700 1 --2- 172.21.15.19:0/2898745315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4620075d30 0x7f4620076140 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4610009990 tx=0x7f4610009ca0 comp rx=0 tx=0).stop 2024-05-07T02:29:42.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.426+0000 7f4625200700 1 -- 172.21.15.19:0/2898745315 shutdown_connections 2024-05-07T02:29:42.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.426+0000 7f4625200700 1 --2- 172.21.15.19:0/2898745315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4620110ba0 0x7f4620113000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:42.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.426+0000 7f4625200700 1 --2- 172.21.15.19:0/2898745315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4620075d30 0x7f4620076140 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:42.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.426+0000 7f4625200700 1 --2- 172.21.15.19:0/2898745315 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4620076680 0x7f4620110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:42.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.426+0000 7f4625200700 1 -- 172.21.15.19:0/2898745315 >> 172.21.15.19:0/2898745315 conn(0x7f4620100040 msgr2=0x7f4620102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:42.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.426+0000 7f4625200700 1 -- 172.21.15.19:0/2898745315 shutdown_connections 2024-05-07T02:29:42.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.426+0000 7f4625200700 1 -- 172.21.15.19:0/2898745315 wait complete. 2024-05-07T02:29:42.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.428+0000 7f4625200700 1 Processor -- start 2024-05-07T02:29:42.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.428+0000 7f4625200700 1 -- start start 2024-05-07T02:29:42.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.428+0000 7f4625200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4620075d30 0x7f462019d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:42.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.428+0000 7f4625200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4620076680 0x7f462019d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:42.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.428+0000 7f4625200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4620110ba0 0x7f46201a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:42.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.428+0000 7f4625200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f46201166e0 con 0x7f4620076680 2024-05-07T02:29:42.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.428+0000 7f4625200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4620116560 con 0x7f4620075d30 2024-05-07T02:29:42.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.428+0000 7f4625200700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4620116860 con 0x7f4620110ba0 2024-05-07T02:29:42.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.428+0000 7f461d400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4620076680 0x7f462019d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:42.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.429+0000 7f461d400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4620076680 0x7f462019d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33208/0 (socket says 172.21.15.19:33208) 2024-05-07T02:29:42.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.429+0000 7f461d400700 1 -- 172.21.15.19:0/3081858335 learned_addr learned my addr 172.21.15.19:0/3081858335 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:29:42.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.429+0000 7f461e800700 1 --2- 172.21.15.19:0/3081858335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4620110ba0 0x7f46201a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:42.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.429+0000 7f461de00700 1 --2- 172.21.15.19:0/3081858335 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4620075d30 0x7f462019d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:42.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.429+0000 7f461e800700 1 -- 172.21.15.19:0/3081858335 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4620075d30 msgr2=0x7f462019d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:42.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.429+0000 7f461e800700 1 --2- 172.21.15.19:0/3081858335 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4620075d30 0x7f462019d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:42.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.429+0000 7f461e800700 1 -- 172.21.15.19:0/3081858335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4620076680 msgr2=0x7f462019d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:42.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.429+0000 7f461e800700 1 --2- 172.21.15.19:0/3081858335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4620076680 0x7f462019d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:42.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.429+0000 7f461e800700 1 -- 172.21.15.19:0/3081858335 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46201a1f00 con 0x7f4620110ba0 2024-05-07T02:29:42.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.429+0000 7f461e800700 1 --2- 172.21.15.19:0/3081858335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4620110ba0 0x7f46201a1860 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f461800ef90 tx=0x7f461800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:42.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.430+0000 7f4616a00700 1 -- 172.21.15.19:0/3081858335 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46180092e0 con 0x7f4620110ba0 2024-05-07T02:29:42.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.430+0000 7f4616a00700 1 -- 172.21.15.19:0/3081858335 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4618004bb0 con 0x7f4620110ba0 2024-05-07T02:29:42.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.430+0000 7f4625200700 1 -- 172.21.15.19:0/3081858335 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46201a2190 con 0x7f4620110ba0 2024-05-07T02:29:42.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.430+0000 7f4616a00700 1 -- 172.21.15.19:0/3081858335 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4618004020 con 0x7f4620110ba0 2024-05-07T02:29:42.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.430+0000 7f4625200700 1 -- 172.21.15.19:0/3081858335 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46201a9b10 con 0x7f4620110ba0 2024-05-07T02:29:42.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.431+0000 7f4625200700 1 -- 172.21.15.19:0/3081858335 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46200512c0 con 0x7f4620110ba0 2024-05-07T02:29:42.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.434+0000 7f4616a00700 1 -- 172.21.15.19:0/3081858335 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f461801b030 con 0x7f4620110ba0 2024-05-07T02:29:42.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.437+0000 7f4616a00700 1 --2- 172.21.15.19:0/3081858335 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f46040b2900 0x7f46040b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:42.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.437+0000 7f4616a00700 1 -- 172.21.15.19:0/3081858335 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f46180e2a80 con 0x7f4620110ba0 2024-05-07T02:29:42.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.437+0000 7f461de00700 1 --2- 172.21.15.19:0/3081858335 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f46040b2900 0x7f46040b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:42.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.437+0000 7f4616a00700 1 -- 172.21.15.19:0/3081858335 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4618022020 con 0x7f4620110ba0 2024-05-07T02:29:42.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.438+0000 7f461de00700 1 --2- 172.21.15.19:0/3081858335 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f46040b2900 0x7f46040b4e10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f461000b2e0 tx=0x7f4610005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:42.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.695+0000 7f4625200700 1 -- 172.21.15.19:0/3081858335 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f46201a9ce0 con 0x7f4620110ba0 2024-05-07T02:29:42.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.699+0000 7f4616a00700 1 -- 172.21.15.19:0/3081858335 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f4618010040 con 0x7f4620110ba0 2024-05-07T02:29:42.225 INFO:teuthology.orchestra.run.smithi019.stdout:154618822752 2024-05-07T02:29:42.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.702+0000 7f4625200700 1 -- 172.21.15.19:0/3081858335 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f46040b2900 msgr2=0x7f46040b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:42.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.702+0000 7f4625200700 1 --2- 172.21.15.19:0/3081858335 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f46040b2900 0x7f46040b4e10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f461000b2e0 tx=0x7f4610005fd0 comp rx=0 tx=0).stop 2024-05-07T02:29:42.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.702+0000 7f4625200700 1 -- 172.21.15.19:0/3081858335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4620110ba0 msgr2=0x7f46201a1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:42.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.702+0000 7f4625200700 1 --2- 172.21.15.19:0/3081858335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4620110ba0 0x7f46201a1860 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f461800ef90 tx=0x7f461800c3a0 comp rx=0 tx=0).stop 2024-05-07T02:29:42.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.702+0000 7f4625200700 1 -- 172.21.15.19:0/3081858335 shutdown_connections 2024-05-07T02:29:42.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.702+0000 7f4625200700 1 --2- 172.21.15.19:0/3081858335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4620110ba0 0x7f46201a1860 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:42.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.702+0000 7f4625200700 1 --2- 172.21.15.19:0/3081858335 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f46040b2900 0x7f46040b4e10 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:42.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.702+0000 7f4625200700 1 --2- 172.21.15.19:0/3081858335 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4620075d30 0x7f462019d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:42.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.703+0000 7f4625200700 1 --2- 172.21.15.19:0/3081858335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4620076680 0x7f462019d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:42.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.703+0000 7f4625200700 1 -- 172.21.15.19:0/3081858335 >> 172.21.15.19:0/3081858335 conn(0x7f4620100040 msgr2=0x7f4620101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:42.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.703+0000 7f4625200700 1 -- 172.21.15.19:0/3081858335 shutdown_connections 2024-05-07T02:29:42.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:40.703+0000 7f4625200700 1 -- 172.21.15.19:0/3081858335 wait complete. 2024-05-07T02:29:42.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:42 smithi019 ceph-mon[29781]: pgmap v444: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-07T02:29:42.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:42 smithi027 ceph-mon[34398]: pgmap v444: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-07T02:29:43.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:42 smithi064 ceph-mon[34699]: pgmap v444: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-05-07T02:29:44.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.482+0000 7fbc85000700 1 -- 172.21.15.19:0/1867922544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc801109b0 msgr2=0x7fbc80112e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:44.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.482+0000 7fbc85000700 1 --2- 172.21.15.19:0/1867922544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc801109b0 0x7fbc80112e10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbc78009990 tx=0x7fbc78009ca0 comp rx=0 tx=0).stop 2024-05-07T02:29:44.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.482+0000 7fbc85000700 1 -- 172.21.15.19:0/1867922544 shutdown_connections 2024-05-07T02:29:44.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.482+0000 7fbc85000700 1 --2- 172.21.15.19:0/1867922544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc801109b0 0x7fbc80112e10 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:44.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.482+0000 7fbc85000700 1 --2- 172.21.15.19:0/1867922544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc80075d30 0x7fbc80076140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:44.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.482+0000 7fbc85000700 1 --2- 172.21.15.19:0/1867922544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc80076680 0x7fbc80110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:44.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.482+0000 7fbc85000700 1 -- 172.21.15.19:0/1867922544 >> 172.21.15.19:0/1867922544 conn(0x7fbc80079d10 msgr2=0x7fbc8007a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:44.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.482+0000 7fbc85000700 1 -- 172.21.15.19:0/1867922544 shutdown_connections 2024-05-07T02:29:44.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.482+0000 7fbc85000700 1 -- 172.21.15.19:0/1867922544 wait complete. 2024-05-07T02:29:44.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.483+0000 7fbc85000700 1 Processor -- start 2024-05-07T02:29:44.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.483+0000 7fbc85000700 1 -- start start 2024-05-07T02:29:44.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.484+0000 7fbc85000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc80075d30 0x7fbc8019ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:44.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.484+0000 7fbc85000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc80076680 0x7fbc8019d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:44.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.484+0000 7fbc85000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc801109b0 0x7fbc801a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:44.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.484+0000 7fbc85000700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbc80116490 con 0x7fbc80075d30 2024-05-07T02:29:44.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.484+0000 7fbc85000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbc80116310 con 0x7fbc80076680 2024-05-07T02:29:44.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.484+0000 7fbc85000700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbc80116610 con 0x7fbc801109b0 2024-05-07T02:29:44.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.484+0000 7fbc7f400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc80075d30 0x7fbc8019ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:44.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.484+0000 7fbc76a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc80076680 0x7fbc8019d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:44.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.484+0000 7fbc7f400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc80075d30 0x7fbc8019ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33238/0 (socket says 172.21.15.19:33238) 2024-05-07T02:29:44.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.484+0000 7fbc7f400700 1 -- 172.21.15.19:0/2969599630 learned_addr learned my addr 172.21.15.19:0/2969599630 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:29:44.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.484+0000 7fbc76a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc80076680 0x7fbc8019d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:56008/0 (socket says 172.21.15.19:56008) 2024-05-07T02:29:44.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.484+0000 7fbc7fe00700 1 --2- 172.21.15.19:0/2969599630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc801109b0 0x7fbc801a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:44.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.485+0000 7fbc7f400700 1 -- 172.21.15.19:0/2969599630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc801109b0 msgr2=0x7fbc801a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:44.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.485+0000 7fbc7f400700 1 --2- 172.21.15.19:0/2969599630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc801109b0 0x7fbc801a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:44.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.485+0000 7fbc7f400700 1 -- 172.21.15.19:0/2969599630 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc80076680 msgr2=0x7fbc8019d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:44.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.485+0000 7fbc7f400700 1 --2- 172.21.15.19:0/2969599630 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc80076680 0x7fbc8019d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:44.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.485+0000 7fbc7f400700 1 -- 172.21.15.19:0/2969599630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc801a1cc0 con 0x7fbc80075d30 2024-05-07T02:29:44.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.485+0000 7fbc7f400700 1 --2- 172.21.15.19:0/2969599630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc80075d30 0x7fbc8019ce40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbc7000ef10 tx=0x7fbc7000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:44.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.486+0000 7fbc7cc00700 1 -- 172.21.15.19:0/2969599630 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7000c9c0 con 0x7fbc80075d30 2024-05-07T02:29:44.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.486+0000 7fbc85000700 1 -- 172.21.15.19:0/2969599630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc801a1f50 con 0x7fbc80075d30 2024-05-07T02:29:44.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.486+0000 7fbc7cc00700 1 -- 172.21.15.19:0/2969599630 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc70019070 con 0x7fbc80075d30 2024-05-07T02:29:44.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.486+0000 7fbc7cc00700 1 -- 172.21.15.19:0/2969599630 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc70014480 con 0x7fbc80075d30 2024-05-07T02:29:44.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.486+0000 7fbc85000700 1 -- 172.21.15.19:0/2969599630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc801a98d0 con 0x7fbc80075d30 2024-05-07T02:29:44.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.488+0000 7fbc85000700 1 -- 172.21.15.19:0/2969599630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc800772a0 con 0x7fbc80075d30 2024-05-07T02:29:44.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.490+0000 7fbc7cc00700 1 -- 172.21.15.19:0/2969599630 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fbc70012050 con 0x7fbc80075d30 2024-05-07T02:29:44.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.492+0000 7fbc7cc00700 1 --2- 172.21.15.19:0/2969599630 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fbc640b2880 0x7fbc640b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:44.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.492+0000 7fbc7cc00700 1 -- 172.21.15.19:0/2969599630 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fbc700e2550 con 0x7fbc80075d30 2024-05-07T02:29:44.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.492+0000 7fbc76a00700 1 --2- 172.21.15.19:0/2969599630 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fbc640b2880 0x7fbc640b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:44.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.493+0000 7fbc76a00700 1 --2- 172.21.15.19:0/2969599630 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fbc640b2880 0x7fbc640b4d90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbc801128b0 tx=0x7fbc68009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:44.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.497+0000 7fbc7cc00700 1 -- 172.21.15.19:0/2969599630 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbc70010040 con 0x7fbc80075d30 2024-05-07T02:29:44.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.748+0000 7fbc85000700 1 -- 172.21.15.19:0/2969599630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fbc8002e5f0 con 0x7fbc80075d30 2024-05-07T02:29:44.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.749+0000 7fbc7cc00700 1 -- 172.21.15.19:0/2969599630 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbc700a9e60 con 0x7fbc80075d30 2024-05-07T02:29:44.751 INFO:teuthology.orchestra.run.smithi019.stdout:107374182507 2024-05-07T02:29:44.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.752+0000 7fbc85000700 1 -- 172.21.15.19:0/2969599630 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fbc640b2880 msgr2=0x7fbc640b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:44.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.752+0000 7fbc85000700 1 --2- 172.21.15.19:0/2969599630 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fbc640b2880 0x7fbc640b4d90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbc801128b0 tx=0x7fbc68009260 comp rx=0 tx=0).stop 2024-05-07T02:29:44.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.752+0000 7fbc85000700 1 -- 172.21.15.19:0/2969599630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc80075d30 msgr2=0x7fbc8019ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:44.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.752+0000 7fbc85000700 1 --2- 172.21.15.19:0/2969599630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc80075d30 0x7fbc8019ce40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbc7000ef10 tx=0x7fbc7000c3a0 comp rx=0 tx=0).stop 2024-05-07T02:29:44.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.752+0000 7fbc85000700 1 -- 172.21.15.19:0/2969599630 shutdown_connections 2024-05-07T02:29:44.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.752+0000 7fbc85000700 1 --2- 172.21.15.19:0/2969599630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbc801109b0 0x7fbc801a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:44.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.752+0000 7fbc85000700 1 --2- 172.21.15.19:0/2969599630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc80075d30 0x7fbc8019ce40 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:44.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.752+0000 7fbc85000700 1 --2- 172.21.15.19:0/2969599630 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fbc640b2880 0x7fbc640b4d90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:44.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.752+0000 7fbc85000700 1 --2- 172.21.15.19:0/2969599630 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc80076680 0x7fbc8019d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:44.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.752+0000 7fbc85000700 1 -- 172.21.15.19:0/2969599630 >> 172.21.15.19:0/2969599630 conn(0x7fbc80079d10 msgr2=0x7fbc80100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:44.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.752+0000 7fbc85000700 1 -- 172.21.15.19:0/2969599630 shutdown_connections 2024-05-07T02:29:44.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:44.752+0000 7fbc85000700 1 -- 172.21.15.19:0/2969599630 wait complete. 2024-05-07T02:29:45.075 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:44 smithi019 ceph-mon[29781]: pgmap v445: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-07T02:29:45.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:44 smithi064 ceph-mon[34699]: pgmap v445: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-07T02:29:45.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.075+0000 7fe113e00700 1 -- 172.21.15.19:0/3533471780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe114108d30 msgr2=0x7fe114109140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:45.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.075+0000 7fe113e00700 1 --2- 172.21.15.19:0/3533471780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe114108d30 0x7fe114109140 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc009990 tx=0x7fe0fc009ca0 comp rx=0 tx=0).stop 2024-05-07T02:29:45.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.075+0000 7fe113e00700 1 -- 172.21.15.19:0/3533471780 shutdown_connections 2024-05-07T02:29:45.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.075+0000 7fe113e00700 1 --2- 172.21.15.19:0/3533471780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe114109680 0x7fe114110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:45.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.075+0000 7fe113e00700 1 --2- 172.21.15.19:0/3533471780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe114104400 0x7fe114104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:45.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.075+0000 7fe113e00700 1 --2- 172.21.15.19:0/3533471780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe114108d30 0x7fe114109140 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:45.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.075+0000 7fe113e00700 1 -- 172.21.15.19:0/3533471780 >> 172.21.15.19:0/3533471780 conn(0x7fe114079d10 msgr2=0x7fe11407a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:45.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.075+0000 7fe113e00700 1 -- 172.21.15.19:0/3533471780 shutdown_connections 2024-05-07T02:29:45.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.076+0000 7fe113e00700 1 -- 172.21.15.19:0/3533471780 wait complete. 2024-05-07T02:29:45.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.077+0000 7fe113e00700 1 Processor -- start 2024-05-07T02:29:45.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.077+0000 7fe113e00700 1 -- start start 2024-05-07T02:29:45.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.078+0000 7fe113e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe114104400 0x7fe11419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:45.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.078+0000 7fe113e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe114108d30 0x7fe11419d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:45.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.078+0000 7fe113e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe114109680 0x7fe1141a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:45.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.078+0000 7fe113e00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe114114290 con 0x7fe114109680 2024-05-07T02:29:45.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.078+0000 7fe113e00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe114114110 con 0x7fe114104400 2024-05-07T02:29:45.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.078+0000 7fe113e00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe114114410 con 0x7fe114108d30 2024-05-07T02:29:45.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.078+0000 7fe112a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe114104400 0x7fe11419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:45.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.078+0000 7fe10a000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe114108d30 0x7fe11419d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:45.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.078+0000 7fe112a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe114104400 0x7fe11419ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:56018/0 (socket says 172.21.15.19:56018) 2024-05-07T02:29:45.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.078+0000 7fe112a00700 1 -- 172.21.15.19:0/3231362268 learned_addr learned my addr 172.21.15.19:0/3231362268 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:29:45.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.078+0000 7fe113400700 1 --2- 172.21.15.19:0/3231362268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe114109680 0x7fe1141a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:45.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.079+0000 7fe112a00700 1 -- 172.21.15.19:0/3231362268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe114108d30 msgr2=0x7fe11419d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:45.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.079+0000 7fe112a00700 1 --2- 172.21.15.19:0/3231362268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe114108d30 0x7fe11419d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:45.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.079+0000 7fe112a00700 1 -- 172.21.15.19:0/3231362268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe114109680 msgr2=0x7fe1141a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:45.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.079+0000 7fe112a00700 1 --2- 172.21.15.19:0/3231362268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe114109680 0x7fe1141a1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:45.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.079+0000 7fe112a00700 1 -- 172.21.15.19:0/3231362268 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1141a1d10 con 0x7fe114104400 2024-05-07T02:29:45.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.079+0000 7fe112a00700 1 --2- 172.21.15.19:0/3231362268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe114104400 0x7fe11419ce40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe10400c370 tx=0x7fe10400c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:45.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.079+0000 7fe10be00700 1 -- 172.21.15.19:0/3231362268 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1040092e0 con 0x7fe114104400 2024-05-07T02:29:45.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.079+0000 7fe10be00700 1 -- 172.21.15.19:0/3231362268 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe10400cbe0 con 0x7fe114104400 2024-05-07T02:29:45.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.079+0000 7fe113e00700 1 -- 172.21.15.19:0/3231362268 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1141a1fa0 con 0x7fe114104400 2024-05-07T02:29:45.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.079+0000 7fe10be00700 1 -- 172.21.15.19:0/3231362268 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1040046b0 con 0x7fe114104400 2024-05-07T02:29:45.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.079+0000 7fe113e00700 1 -- 172.21.15.19:0/3231362268 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1141a9920 con 0x7fe114104400 2024-05-07T02:29:45.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.080+0000 7fe113e00700 1 -- 172.21.15.19:0/3231362268 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1141058b0 con 0x7fe114104400 2024-05-07T02:29:45.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.083+0000 7fe10be00700 1 -- 172.21.15.19:0/3231362268 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe10401a020 con 0x7fe114104400 2024-05-07T02:29:45.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.085+0000 7fe10be00700 1 --2- 172.21.15.19:0/3231362268 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fe0f80b2900 0x7fe0f80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:45.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.085+0000 7fe10be00700 1 -- 172.21.15.19:0/3231362268 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fe1040e24d0 con 0x7fe114104400 2024-05-07T02:29:45.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.085+0000 7fe10a000700 1 --2- 172.21.15.19:0/3231362268 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fe0f80b2900 0x7fe0f80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:45.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.086+0000 7fe10a000700 1 --2- 172.21.15.19:0/3231362268 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fe0f80b2900 0x7fe0f80b4e10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc00b2e0 tx=0x7fe0fc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:45.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.087+0000 7fe10be00700 1 -- 172.21.15.19:0/3231362268 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe104010040 con 0x7fe114104400 2024-05-07T02:29:45.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:44 smithi027 ceph-mon[34398]: pgmap v445: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-07T02:29:45.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.331+0000 7fe113e00700 1 -- 172.21.15.19:0/3231362268 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fe1141a9af0 con 0x7fe114104400 2024-05-07T02:29:45.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.332+0000 7fe10be00700 1 -- 172.21.15.19:0/3231362268 <== mon.2 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe1040a9de0 con 0x7fe114104400 2024-05-07T02:29:45.332 INFO:teuthology.orchestra.run.smithi019.stdout:133143986279 2024-05-07T02:29:45.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.335+0000 7fe113e00700 1 -- 172.21.15.19:0/3231362268 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fe0f80b2900 msgr2=0x7fe0f80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:45.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.335+0000 7fe113e00700 1 --2- 172.21.15.19:0/3231362268 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fe0f80b2900 0x7fe0f80b4e10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc00b2e0 tx=0x7fe0fc005960 comp rx=0 tx=0).stop 2024-05-07T02:29:45.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.335+0000 7fe113e00700 1 -- 172.21.15.19:0/3231362268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe114104400 msgr2=0x7fe11419ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:45.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.335+0000 7fe113e00700 1 --2- 172.21.15.19:0/3231362268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe114104400 0x7fe11419ce40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe10400c370 tx=0x7fe10400c680 comp rx=0 tx=0).stop 2024-05-07T02:29:45.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.335+0000 7fe113e00700 1 -- 172.21.15.19:0/3231362268 shutdown_connections 2024-05-07T02:29:45.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.335+0000 7fe113e00700 1 --2- 172.21.15.19:0/3231362268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe114109680 0x7fe1141a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:45.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.335+0000 7fe113e00700 1 --2- 172.21.15.19:0/3231362268 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fe0f80b2900 0x7fe0f80b4e10 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:45.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.335+0000 7fe113e00700 1 --2- 172.21.15.19:0/3231362268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe114104400 0x7fe11419ce40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:45.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.335+0000 7fe113e00700 1 --2- 172.21.15.19:0/3231362268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe114108d30 0x7fe11419d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:45.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.335+0000 7fe113e00700 1 -- 172.21.15.19:0/3231362268 >> 172.21.15.19:0/3231362268 conn(0x7fe114079d10 msgr2=0x7fe114101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:45.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.335+0000 7fe113e00700 1 -- 172.21.15.19:0/3231362268 shutdown_connections 2024-05-07T02:29:45.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:45.335+0000 7fe113e00700 1 -- 172.21.15.19:0/3231362268 wait complete. 2024-05-07T02:29:45.472 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822745 got 154618822752 for osd.5 2024-05-07T02:29:45.472 DEBUG:teuthology.parallel:result is None 2024-05-07T02:29:45.903 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182501 got 107374182507 for osd.3 2024-05-07T02:29:45.904 DEBUG:teuthology.parallel:result is None 2024-05-07T02:29:46.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:45 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/2969599630' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-07T02:29:46.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:45 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/3231362268' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-07T02:29:46.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:45 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/2969599630' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-07T02:29:46.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:45 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/3231362268' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-07T02:29:46.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:45 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/2969599630' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-07T02:29:46.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:45 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/3231362268' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-07T02:29:46.663 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986273 got 133143986279 for osd.4 2024-05-07T02:29:46.663 DEBUG:teuthology.parallel:result is None 2024-05-07T02:29:46.664 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-07T02:29:46.664 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-07T02:29:46.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:46 smithi019 ceph-mon[29781]: pgmap v446: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-07T02:29:47.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:46 smithi064 ceph-mon[34699]: pgmap v446: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-07T02:29:47.137 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:29:47.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:46 smithi027 ceph-mon[34398]: pgmap v446: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-07T02:29:48.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:48 smithi019 ceph-mon[29781]: pgmap v447: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 85 B/s wr, 10 op/s 2024-05-07T02:29:49.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:48 smithi064 ceph-mon[34699]: pgmap v447: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 85 B/s wr, 10 op/s 2024-05-07T02:29:49.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:48 smithi027 ceph-mon[34398]: pgmap v447: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 85 B/s wr, 10 op/s 2024-05-07T02:29:49.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.560+0000 7f2681400700 1 -- 172.21.15.19:0/3742761179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f267c10b110 msgr2=0x7f267c117640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:49.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.560+0000 7f2681400700 1 --2- 172.21.15.19:0/3742761179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f267c10b110 0x7f267c117640 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f267400b0a0 tx=0x7f267400b3b0 comp rx=0 tx=0).stop 2024-05-07T02:29:49.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.561+0000 7f2681400700 1 -- 172.21.15.19:0/3742761179 shutdown_connections 2024-05-07T02:29:49.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.561+0000 7f2681400700 1 --2- 172.21.15.19:0/3742761179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f267c10b110 0x7f267c117640 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:49.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.561+0000 7f2681400700 1 --2- 172.21.15.19:0/3742761179 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f267c1107a0 0x7f267c110bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:49.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.561+0000 7f2681400700 1 --2- 172.21.15.19:0/3742761179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f267c10a7c0 0x7f267c10abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:49.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.561+0000 7f2681400700 1 -- 172.21.15.19:0/3742761179 >> 172.21.15.19:0/3742761179 conn(0x7f267c076b00 msgr2=0x7f267c076f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:49.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.561+0000 7f2681400700 1 -- 172.21.15.19:0/3742761179 shutdown_connections 2024-05-07T02:29:49.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.561+0000 7f2681400700 1 -- 172.21.15.19:0/3742761179 wait complete. 2024-05-07T02:29:49.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.563+0000 7f2681400700 1 Processor -- start 2024-05-07T02:29:49.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.563+0000 7f2681400700 1 -- start start 2024-05-07T02:29:49.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.564+0000 7f2681400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f267c10a7c0 0x7f267c1a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:49.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.564+0000 7f2681400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f267c10b110 0x7f267c1a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:49.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.564+0000 7f2681400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f267c1107a0 0x7f267c1a59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:49.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.564+0000 7f2681400700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f267c11a8b0 con 0x7f267c10a7c0 2024-05-07T02:29:49.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.564+0000 7f2681400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f267c11a730 con 0x7f267c1107a0 2024-05-07T02:29:49.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.564+0000 7f2681400700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f267c11aa30 con 0x7f267c10b110 2024-05-07T02:29:49.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.564+0000 7f267be00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f267c10a7c0 0x7f267c1a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:49.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.564+0000 7f2680a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f267c1107a0 0x7f267c1a59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:49.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.564+0000 7f267b400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f267c10b110 0x7f267c1a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:49.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.564+0000 7f2680a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f267c1107a0 0x7f267c1a59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:33154/0 (socket says 172.21.15.19:33154) 2024-05-07T02:29:49.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.564+0000 7f2680a00700 1 -- 172.21.15.19:0/2675117991 learned_addr learned my addr 172.21.15.19:0/2675117991 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:29:49.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.565+0000 7f267b400700 1 -- 172.21.15.19:0/2675117991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f267c1107a0 msgr2=0x7f267c1a59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:49.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.565+0000 7f267b400700 1 --2- 172.21.15.19:0/2675117991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f267c1107a0 0x7f267c1a59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:49.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.565+0000 7f267b400700 1 -- 172.21.15.19:0/2675117991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f267c10a7c0 msgr2=0x7f267c1a11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:49.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.565+0000 7f267b400700 1 --2- 172.21.15.19:0/2675117991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f267c10a7c0 0x7f267c1a11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:49.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.565+0000 7f267b400700 1 -- 172.21.15.19:0/2675117991 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f267c1a6090 con 0x7f267c10b110 2024-05-07T02:29:49.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.565+0000 7f267b400700 1 --2- 172.21.15.19:0/2675117991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f267c10b110 0x7f267c1a1760 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f266800e9c0 tx=0x7f266800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:49.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.565+0000 7f2678c00700 1 -- 172.21.15.19:0/2675117991 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266800c9c0 con 0x7f267c10b110 2024-05-07T02:29:49.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.565+0000 7f2678c00700 1 -- 172.21.15.19:0/2675117991 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f266800cb60 con 0x7f267c10b110 2024-05-07T02:29:49.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.565+0000 7f2681400700 1 -- 172.21.15.19:0/2675117991 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f267c1a6380 con 0x7f267c10b110 2024-05-07T02:29:49.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.565+0000 7f2678c00700 1 -- 172.21.15.19:0/2675117991 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2668018680 con 0x7f267c10b110 2024-05-07T02:29:49.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.566+0000 7f2681400700 1 -- 172.21.15.19:0/2675117991 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f267c1adca0 con 0x7f267c10b110 2024-05-07T02:29:49.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.567+0000 7f2681400700 1 -- 172.21.15.19:0/2675117991 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f267c0512c0 con 0x7f267c10b110 2024-05-07T02:29:49.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.569+0000 7f2678c00700 1 -- 172.21.15.19:0/2675117991 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2668010430 con 0x7f267c10b110 2024-05-07T02:29:49.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.571+0000 7f2678c00700 1 --2- 172.21.15.19:0/2675117991 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f26600b2900 0x7f26600b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:49.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.571+0000 7f2678c00700 1 -- 172.21.15.19:0/2675117991 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f26680e2aa0 con 0x7f267c10b110 2024-05-07T02:29:49.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.571+0000 7f267be00700 1 --2- 172.21.15.19:0/2675117991 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f26600b2900 0x7f26600b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:49.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.572+0000 7f267be00700 1 --2- 172.21.15.19:0/2675117991 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f26600b2900 0x7f26600b4e10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f266c009be0 tx=0x7f266c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:49.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.573+0000 7f2678c00700 1 -- 172.21.15.19:0/2675117991 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f26680aa3b0 con 0x7f267c10b110 2024-05-07T02:29:49.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.809+0000 7f2681400700 1 -- 172.21.15.19:0/2675117991 --> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f267c063c00 con 0x7f26600b2900 2024-05-07T02:29:49.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.844+0000 7f2678c00700 1 -- 172.21.15.19:0/2675117991 <== mgr.14152 v2:172.21.15.19:6800/2059564551 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340515 (secure 0 0 0) 0x7f267c063c00 con 0x7f26600b2900 2024-05-07T02:29:49.850 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:29:49.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.851+0000 7f2681400700 1 -- 172.21.15.19:0/2675117991 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f26600b2900 msgr2=0x7f26600b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:49.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.851+0000 7f2681400700 1 --2- 172.21.15.19:0/2675117991 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f26600b2900 0x7f26600b4e10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f266c009be0 tx=0x7f266c009260 comp rx=0 tx=0).stop 2024-05-07T02:29:49.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.851+0000 7f2681400700 1 -- 172.21.15.19:0/2675117991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f267c10b110 msgr2=0x7f267c1a1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:49.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.851+0000 7f2681400700 1 --2- 172.21.15.19:0/2675117991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f267c10b110 0x7f267c1a1760 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f266800e9c0 tx=0x7f266800ecd0 comp rx=0 tx=0).stop 2024-05-07T02:29:49.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.851+0000 7f2681400700 1 -- 172.21.15.19:0/2675117991 shutdown_connections 2024-05-07T02:29:49.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.851+0000 7f2681400700 1 --2- 172.21.15.19:0/2675117991 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f26600b2900 0x7f26600b4e10 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:49.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.851+0000 7f2681400700 1 --2- 172.21.15.19:0/2675117991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f267c1107a0 0x7f267c1a59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:49.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.851+0000 7f2681400700 1 --2- 172.21.15.19:0/2675117991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f267c10a7c0 0x7f267c1a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:49.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.851+0000 7f2681400700 1 --2- 172.21.15.19:0/2675117991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f267c10b110 0x7f267c1a1760 secure :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f266800e9c0 tx=0x7f266800ecd0 comp rx=0 tx=0).stop 2024-05-07T02:29:49.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.851+0000 7f2681400700 1 -- 172.21.15.19:0/2675117991 >> 172.21.15.19:0/2675117991 conn(0x7f267c076b00 msgr2=0x7f267c108880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:49.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.852+0000 7f2681400700 1 -- 172.21.15.19:0/2675117991 shutdown_connections 2024-05-07T02:29:49.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:49.852+0000 7f2681400700 1 -- 172.21.15.19:0/2675117991 wait complete. 2024-05-07T02:29:49.855 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-05-07T02:29:50.712 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":448,"stamp":"2024-05-07T02:29:49.541780+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666035,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1349,"num_read_kb":787,"num_write":383,"num_write_kb":2305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":816,"ondisk_log_size":816,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375884,"kb_used_data":14252,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565940,"statfs":{"total":1151940427776,"available":1151555522560,"internally_reserved":0,"allocated":14594048,"data_stored":8155247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005863"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.457443+0000","last_change":"2024-05-07T02:26:11.876828+0000","last_active":"2024-05-07T02:29:31.457443+0000","last_peered":"2024-05-07T02:29:31.457443+0000","last_clean":"2024-05-07T02:29:31.457443+0000","last_became_active":"2024-05-07T02:26:11.876319+0000","last_became_peered":"2024-05-07T02:26:11.876319+0000","last_unstale":"2024-05-07T02:29:31.457443+0000","last_undegraded":"2024-05-07T02:29:31.457443+0000","last_fullsized":"2024-05-07T02:29:31.457443+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:07:33.181373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.455517+0000","last_change":"2024-05-07T02:26:11.873388+0000","last_active":"2024-05-07T02:29:31.455517+0000","last_peered":"2024-05-07T02:29:31.455517+0000","last_clean":"2024-05-07T02:29:31.455517+0000","last_became_active":"2024-05-07T02:26:11.869447+0000","last_became_peered":"2024-05-07T02:26:11.869447+0000","last_unstale":"2024-05-07T02:29:31.455517+0000","last_undegraded":"2024-05-07T02:29:31.455517+0000","last_fullsized":"2024-05-07T02:29:31.455517+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:09:52.876547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.460894+0000","last_change":"2024-05-07T02:26:11.870797+0000","last_active":"2024-05-07T02:29:31.460894+0000","last_peered":"2024-05-07T02:29:31.460894+0000","last_clean":"2024-05-07T02:29:31.460894+0000","last_became_active":"2024-05-07T02:26:11.870411+0000","last_became_peered":"2024-05-07T02:26:11.870411+0000","last_unstale":"2024-05-07T02:29:31.460894+0000","last_undegraded":"2024-05-07T02:29:31.460894+0000","last_fullsized":"2024-05-07T02:29:31.460894+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:34:12.017508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.442852+0000","last_change":"2024-05-07T02:26:11.867327+0000","last_active":"2024-05-07T02:29:31.442852+0000","last_peered":"2024-05-07T02:29:31.442852+0000","last_clean":"2024-05-07T02:29:31.442852+0000","last_became_active":"2024-05-07T02:26:11.866912+0000","last_became_peered":"2024-05-07T02:26:11.866912+0000","last_unstale":"2024-05-07T02:29:31.442852+0000","last_undegraded":"2024-05-07T02:29:31.442852+0000","last_fullsized":"2024-05-07T02:29:31.442852+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:36:36.616706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.435537+0000","last_change":"2024-05-07T02:26:11.878614+0000","last_active":"2024-05-07T02:29:31.435537+0000","last_peered":"2024-05-07T02:29:31.435537+0000","last_clean":"2024-05-07T02:29:31.435537+0000","last_became_active":"2024-05-07T02:26:11.878069+0000","last_became_peered":"2024-05-07T02:26:11.878069+0000","last_unstale":"2024-05-07T02:29:31.435537+0000","last_undegraded":"2024-05-07T02:29:31.435537+0000","last_fullsized":"2024-05-07T02:29:31.435537+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:40:15.991433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.445763+0000","last_change":"2024-05-07T02:26:11.877755+0000","last_active":"2024-05-07T02:29:31.445763+0000","last_peered":"2024-05-07T02:29:31.445763+0000","last_clean":"2024-05-07T02:29:31.445763+0000","last_became_active":"2024-05-07T02:26:11.877373+0000","last_became_peered":"2024-05-07T02:26:11.877373+0000","last_unstale":"2024-05-07T02:29:31.445763+0000","last_undegraded":"2024-05-07T02:29:31.445763+0000","last_fullsized":"2024-05-07T02:29:31.445763+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:49:55.984807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.439470+0000","last_change":"2024-05-07T02:26:11.882194+0000","last_active":"2024-05-07T02:29:31.439470+0000","last_peered":"2024-05-07T02:29:31.439470+0000","last_clean":"2024-05-07T02:29:31.439470+0000","last_became_active":"2024-05-07T02:26:11.881873+0000","last_became_peered":"2024-05-07T02:26:11.881873+0000","last_unstale":"2024-05-07T02:29:31.439470+0000","last_undegraded":"2024-05-07T02:29:31.439470+0000","last_fullsized":"2024-05-07T02:29:31.439470+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:47:31.000357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.463571+0000","last_change":"2024-05-07T02:26:11.857366+0000","last_active":"2024-05-07T02:29:31.463571+0000","last_peered":"2024-05-07T02:29:31.463571+0000","last_clean":"2024-05-07T02:29:31.463571+0000","last_became_active":"2024-05-07T02:26:11.854054+0000","last_became_peered":"2024-05-07T02:26:11.854054+0000","last_unstale":"2024-05-07T02:29:31.463571+0000","last_undegraded":"2024-05-07T02:29:31.463571+0000","last_fullsized":"2024-05-07T02:29:31.463571+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:07:04.834022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.455230+0000","last_change":"2024-05-07T02:26:11.867105+0000","last_active":"2024-05-07T02:29:31.455230+0000","last_peered":"2024-05-07T02:29:31.455230+0000","last_clean":"2024-05-07T02:29:31.455230+0000","last_became_active":"2024-05-07T02:26:11.865986+0000","last_became_peered":"2024-05-07T02:26:11.865986+0000","last_unstale":"2024-05-07T02:29:31.455230+0000","last_undegraded":"2024-05-07T02:29:31.455230+0000","last_fullsized":"2024-05-07T02:29:31.455230+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:13:01.295354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.435492+0000","last_change":"2024-05-07T02:26:11.868039+0000","last_active":"2024-05-07T02:29:31.435492+0000","last_peered":"2024-05-07T02:29:31.435492+0000","last_clean":"2024-05-07T02:29:31.435492+0000","last_became_active":"2024-05-07T02:26:11.867588+0000","last_became_peered":"2024-05-07T02:26:11.867588+0000","last_unstale":"2024-05-07T02:29:31.435492+0000","last_undegraded":"2024-05-07T02:29:31.435492+0000","last_fullsized":"2024-05-07T02:29:31.435492+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:41:45.149620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.452741+0000","last_change":"2024-05-07T02:26:11.859579+0000","last_active":"2024-05-07T02:29:31.452741+0000","last_peered":"2024-05-07T02:29:31.452741+0000","last_clean":"2024-05-07T02:29:31.452741+0000","last_became_active":"2024-05-07T02:26:11.859154+0000","last_became_peered":"2024-05-07T02:26:11.859154+0000","last_unstale":"2024-05-07T02:29:31.452741+0000","last_undegraded":"2024-05-07T02:29:31.452741+0000","last_fullsized":"2024-05-07T02:29:31.452741+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:26:40.616956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.443671+0000","last_change":"2024-05-07T02:26:11.858220+0000","last_active":"2024-05-07T02:29:31.443671+0000","last_peered":"2024-05-07T02:29:31.443671+0000","last_clean":"2024-05-07T02:29:31.443671+0000","last_became_active":"2024-05-07T02:26:11.857826+0000","last_became_peered":"2024-05-07T02:26:11.857826+0000","last_unstale":"2024-05-07T02:29:31.443671+0000","last_undegraded":"2024-05-07T02:29:31.443671+0000","last_fullsized":"2024-05-07T02:29:31.443671+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:21:36.031057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.443326+0000","last_change":"2024-05-07T02:26:11.880348+0000","last_active":"2024-05-07T02:29:31.443326+0000","last_peered":"2024-05-07T02:29:31.443326+0000","last_clean":"2024-05-07T02:29:31.443326+0000","last_became_active":"2024-05-07T02:26:11.879633+0000","last_became_peered":"2024-05-07T02:26:11.879633+0000","last_unstale":"2024-05-07T02:29:31.443326+0000","last_undegraded":"2024-05-07T02:29:31.443326+0000","last_fullsized":"2024-05-07T02:29:31.443326+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:59:34.567973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.456171+0000","last_change":"2024-05-07T02:26:11.865687+0000","last_active":"2024-05-07T02:29:31.456171+0000","last_peered":"2024-05-07T02:29:31.456171+0000","last_clean":"2024-05-07T02:29:31.456171+0000","last_became_active":"2024-05-07T02:26:11.865330+0000","last_became_peered":"2024-05-07T02:26:11.865330+0000","last_unstale":"2024-05-07T02:29:31.456171+0000","last_undegraded":"2024-05-07T02:29:31.456171+0000","last_fullsized":"2024-05-07T02:29:31.456171+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:36:10.610314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.458010+0000","last_change":"2024-05-07T02:26:11.870627+0000","last_active":"2024-05-07T02:29:31.458010+0000","last_peered":"2024-05-07T02:29:31.458010+0000","last_clean":"2024-05-07T02:29:31.458010+0000","last_became_active":"2024-05-07T02:26:11.870229+0000","last_became_peered":"2024-05-07T02:26:11.870229+0000","last_unstale":"2024-05-07T02:29:31.458010+0000","last_undegraded":"2024-05-07T02:29:31.458010+0000","last_fullsized":"2024-05-07T02:29:31.458010+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:06:46.008651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.460290+0000","last_change":"2024-05-07T02:26:11.868698+0000","last_active":"2024-05-07T02:29:31.460290+0000","last_peered":"2024-05-07T02:29:31.460290+0000","last_clean":"2024-05-07T02:29:31.460290+0000","last_became_active":"2024-05-07T02:26:11.868294+0000","last_became_peered":"2024-05-07T02:26:11.868294+0000","last_unstale":"2024-05-07T02:29:31.460290+0000","last_undegraded":"2024-05-07T02:29:31.460290+0000","last_fullsized":"2024-05-07T02:29:31.460290+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:16:22.856200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.457570+0000","last_change":"2024-05-07T02:26:11.877116+0000","last_active":"2024-05-07T02:29:31.457570+0000","last_peered":"2024-05-07T02:29:31.457570+0000","last_clean":"2024-05-07T02:29:31.457570+0000","last_became_active":"2024-05-07T02:26:11.876305+0000","last_became_peered":"2024-05-07T02:26:11.876305+0000","last_unstale":"2024-05-07T02:29:31.457570+0000","last_undegraded":"2024-05-07T02:29:31.457570+0000","last_fullsized":"2024-05-07T02:29:31.457570+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:25:46.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.452721+0000","last_change":"2024-05-07T02:26:11.877488+0000","last_active":"2024-05-07T02:29:31.452721+0000","last_peered":"2024-05-07T02:29:31.452721+0000","last_clean":"2024-05-07T02:29:31.452721+0000","last_became_active":"2024-05-07T02:26:11.874500+0000","last_became_peered":"2024-05-07T02:26:11.874500+0000","last_unstale":"2024-05-07T02:29:31.452721+0000","last_undegraded":"2024-05-07T02:29:31.452721+0000","last_fullsized":"2024-05-07T02:29:31.452721+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:53:50.046342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.439857+0000","last_change":"2024-05-07T02:26:11.876500+0000","last_active":"2024-05-07T02:29:31.439857+0000","last_peered":"2024-05-07T02:29:31.439857+0000","last_clean":"2024-05-07T02:29:31.439857+0000","last_became_active":"2024-05-07T02:26:11.876084+0000","last_became_peered":"2024-05-07T02:26:11.876084+0000","last_unstale":"2024-05-07T02:29:31.439857+0000","last_undegraded":"2024-05-07T02:29:31.439857+0000","last_fullsized":"2024-05-07T02:29:31.439857+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:05:05.681488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.868146+0000","last_change":"2024-05-07T02:26:25.579209+0000","last_active":"2024-05-07T02:28:33.868146+0000","last_peered":"2024-05-07T02:28:33.868146+0000","last_clean":"2024-05-07T02:28:33.868146+0000","last_became_active":"2024-05-07T02:26:25.578665+0000","last_became_peered":"2024-05-07T02:26:25.578665+0000","last_unstale":"2024-05-07T02:28:33.868146+0000","last_undegraded":"2024-05-07T02:28:33.868146+0000","last_fullsized":"2024-05-07T02:28:33.868146+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:06:29.784016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'14","reported_seq":184,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:47.003267+0000","last_change":"2024-05-07T02:26:07.792993+0000","last_active":"2024-05-07T02:29:47.003267+0000","last_peered":"2024-05-07T02:29:47.003267+0000","last_clean":"2024-05-07T02:29:47.003267+0000","last_became_active":"2024-05-07T02:26:07.792402+0000","last_became_peered":"2024-05-07T02:26:07.792402+0000","last_unstale":"2024-05-07T02:29:47.003267+0000","last_undegraded":"2024-05-07T02:29:47.003267+0000","last_fullsized":"2024-05-07T02:29:47.003267+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:49:02.288255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":125,"num_read_kb":125,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.442593+0000","last_change":"2024-05-07T02:28:33.869804+0000","last_active":"2024-05-07T02:29:31.442593+0000","last_peered":"2024-05-07T02:29:31.442593+0000","last_clean":"2024-05-07T02:29:31.442593+0000","last_became_active":"2024-05-07T02:28:33.869267+0000","last_became_peered":"2024-05-07T02:28:33.869267+0000","last_unstale":"2024-05-07T02:29:31.442593+0000","last_undegraded":"2024-05-07T02:29:31.442593+0000","last_fullsized":"2024-05-07T02:29:31.442593+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:10:57.384546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.611414+0000","last_change":"2024-05-07T02:26:25.579835+0000","last_active":"2024-05-07T02:28:33.611414+0000","last_peered":"2024-05-07T02:28:33.611414+0000","last_clean":"2024-05-07T02:28:33.611414+0000","last_became_active":"2024-05-07T02:26:25.579105+0000","last_became_peered":"2024-05-07T02:26:25.579105+0000","last_unstale":"2024-05-07T02:28:33.611414+0000","last_undegraded":"2024-05-07T02:28:33.611414+0000","last_fullsized":"2024-05-07T02:28:33.611414+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:32:37.745967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.621858+0000","last_change":"2024-05-07T02:26:08.001879+0000","last_active":"2024-05-07T02:28:33.621858+0000","last_peered":"2024-05-07T02:28:33.621858+0000","last_clean":"2024-05-07T02:28:33.621858+0000","last_became_active":"2024-05-07T02:26:08.000244+0000","last_became_peered":"2024-05-07T02:26:08.000244+0000","last_unstale":"2024-05-07T02:28:33.621858+0000","last_undegraded":"2024-05-07T02:28:33.621858+0000","last_fullsized":"2024-05-07T02:28:33.621858+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:56:42.591923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.460379+0000","last_change":"2024-05-07T02:26:11.882351+0000","last_active":"2024-05-07T02:29:31.460379+0000","last_peered":"2024-05-07T02:29:31.460379+0000","last_clean":"2024-05-07T02:29:31.460379+0000","last_became_active":"2024-05-07T02:26:11.881922+0000","last_became_peered":"2024-05-07T02:26:11.881922+0000","last_unstale":"2024-05-07T02:29:31.460379+0000","last_undegraded":"2024-05-07T02:29:31.460379+0000","last_fullsized":"2024-05-07T02:29:31.460379+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:58:31.993859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.611612+0000","last_change":"2024-05-07T02:26:25.580419+0000","last_active":"2024-05-07T02:28:33.611612+0000","last_peered":"2024-05-07T02:28:33.611612+0000","last_clean":"2024-05-07T02:28:33.611612+0000","last_became_active":"2024-05-07T02:26:25.579411+0000","last_became_peered":"2024-05-07T02:26:25.579411+0000","last_unstale":"2024-05-07T02:28:33.611612+0000","last_undegraded":"2024-05-07T02:28:33.611612+0000","last_fullsized":"2024-05-07T02:28:33.611612+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:43:03.552210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:34.646169+0000","last_change":"2024-05-07T02:27:32.588255+0000","last_active":"2024-05-07T02:28:34.646169+0000","last_peered":"2024-05-07T02:28:34.646169+0000","last_clean":"2024-05-07T02:28:34.646169+0000","last_became_active":"2024-05-07T02:27:32.584405+0000","last_became_peered":"2024-05-07T02:27:32.584405+0000","last_unstale":"2024-05-07T02:28:34.646169+0000","last_undegraded":"2024-05-07T02:28:34.646169+0000","last_fullsized":"2024-05-07T02:28:34.646169+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:07:22.980041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.459629+0000","last_change":"2024-05-07T02:26:11.866949+0000","last_active":"2024-05-07T02:29:31.459629+0000","last_peered":"2024-05-07T02:29:31.459629+0000","last_clean":"2024-05-07T02:29:31.459629+0000","last_became_active":"2024-05-07T02:26:11.866453+0000","last_became_peered":"2024-05-07T02:26:11.866453+0000","last_unstale":"2024-05-07T02:29:31.459629+0000","last_undegraded":"2024-05-07T02:29:31.459629+0000","last_fullsized":"2024-05-07T02:29:31.459629+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T14:22:47.291470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.611410+0000","last_change":"2024-05-07T02:26:25.570555+0000","last_active":"2024-05-07T02:28:33.611410+0000","last_peered":"2024-05-07T02:28:33.611410+0000","last_clean":"2024-05-07T02:28:33.611410+0000","last_became_active":"2024-05-07T02:26:25.570044+0000","last_became_peered":"2024-05-07T02:26:25.570044+0000","last_unstale":"2024-05-07T02:28:33.611410+0000","last_undegraded":"2024-05-07T02:28:33.611410+0000","last_fullsized":"2024-05-07T02:28:33.611410+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:48:15.701335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:34.646091+0000","last_change":"2024-05-07T02:26:07.785938+0000","last_active":"2024-05-07T02:28:34.646091+0000","last_peered":"2024-05-07T02:28:34.646091+0000","last_clean":"2024-05-07T02:28:34.646091+0000","last_became_active":"2024-05-07T02:26:07.785573+0000","last_became_peered":"2024-05-07T02:26:07.785573+0000","last_unstale":"2024-05-07T02:28:34.646091+0000","last_undegraded":"2024-05-07T02:28:34.646091+0000","last_fullsized":"2024-05-07T02:28:34.646091+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:33:24.042227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.450519+0000","last_change":"2024-05-07T02:26:11.858732+0000","last_active":"2024-05-07T02:29:31.450519+0000","last_peered":"2024-05-07T02:29:31.450519+0000","last_clean":"2024-05-07T02:29:31.450519+0000","last_became_active":"2024-05-07T02:26:11.858270+0000","last_became_peered":"2024-05-07T02:26:11.858270+0000","last_unstale":"2024-05-07T02:29:31.450519+0000","last_undegraded":"2024-05-07T02:29:31.450519+0000","last_fullsized":"2024-05-07T02:29:31.450519+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:03:53.662296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.868915+0000","last_change":"2024-05-07T02:26:25.559025+0000","last_active":"2024-05-07T02:28:33.868915+0000","last_peered":"2024-05-07T02:28:33.868915+0000","last_clean":"2024-05-07T02:28:33.868915+0000","last_became_active":"2024-05-07T02:26:25.558502+0000","last_became_peered":"2024-05-07T02:26:25.558502+0000","last_unstale":"2024-05-07T02:28:33.868915+0000","last_undegraded":"2024-05-07T02:28:33.868915+0000","last_fullsized":"2024-05-07T02:28:33.868915+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:26:59.846341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":177,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:47.962734+0000","last_change":"2024-05-07T02:26:07.783386+0000","last_active":"2024-05-07T02:29:47.962734+0000","last_peered":"2024-05-07T02:29:47.962734+0000","last_clean":"2024-05-07T02:29:47.962734+0000","last_became_active":"2024-05-07T02:26:07.782823+0000","last_became_peered":"2024-05-07T02:26:07.782823+0000","last_unstale":"2024-05-07T02:29:47.962734+0000","last_undegraded":"2024-05-07T02:29:47.962734+0000","last_fullsized":"2024-05-07T02:29:47.962734+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:40:29.350809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":127,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.433919+0000","last_change":"2024-05-07T02:26:11.877426+0000","last_active":"2024-05-07T02:29:31.433919+0000","last_peered":"2024-05-07T02:29:31.433919+0000","last_clean":"2024-05-07T02:29:31.433919+0000","last_became_active":"2024-05-07T02:26:11.876976+0000","last_became_peered":"2024-05-07T02:26:11.876976+0000","last_unstale":"2024-05-07T02:29:31.433919+0000","last_undegraded":"2024-05-07T02:29:31.433919+0000","last_fullsized":"2024-05-07T02:29:31.433919+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:18:56.077064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.868625+0000","last_change":"2024-05-07T02:27:32.625394+0000","last_active":"2024-05-07T02:28:33.868625+0000","last_peered":"2024-05-07T02:28:33.868625+0000","last_clean":"2024-05-07T02:28:33.868625+0000","last_became_active":"2024-05-07T02:27:32.624826+0000","last_became_peered":"2024-05-07T02:27:32.624826+0000","last_unstale":"2024-05-07T02:28:33.868625+0000","last_undegraded":"2024-05-07T02:28:33.868625+0000","last_fullsized":"2024-05-07T02:28:33.868625+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:16:49.694003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.840846+0000","last_change":"2024-05-07T02:26:07.786388+0000","last_active":"2024-05-07T02:28:33.840846+0000","last_peered":"2024-05-07T02:28:33.840846+0000","last_clean":"2024-05-07T02:28:33.840846+0000","last_became_active":"2024-05-07T02:26:07.784130+0000","last_became_peered":"2024-05-07T02:26:07.784130+0000","last_unstale":"2024-05-07T02:28:33.840846+0000","last_undegraded":"2024-05-07T02:28:33.840846+0000","last_fullsized":"2024-05-07T02:28:33.840846+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:44:55.229815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.453124+0000","last_change":"2024-05-07T02:28:33.886551+0000","last_active":"2024-05-07T02:29:31.453124+0000","last_peered":"2024-05-07T02:29:31.453124+0000","last_clean":"2024-05-07T02:29:31.453124+0000","last_became_active":"2024-05-07T02:28:33.886249+0000","last_became_peered":"2024-05-07T02:28:33.886249+0000","last_unstale":"2024-05-07T02:29:31.453124+0000","last_undegraded":"2024-05-07T02:29:31.453124+0000","last_fullsized":"2024-05-07T02:29:31.453124+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:56:19.851347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:34.646463+0000","last_change":"2024-05-07T02:26:25.584396+0000","last_active":"2024-05-07T02:28:34.646463+0000","last_peered":"2024-05-07T02:28:34.646463+0000","last_clean":"2024-05-07T02:28:34.646463+0000","last_became_active":"2024-05-07T02:26:25.583917+0000","last_became_peered":"2024-05-07T02:26:25.583917+0000","last_unstale":"2024-05-07T02:28:34.646463+0000","last_undegraded":"2024-05-07T02:28:34.646463+0000","last_fullsized":"2024-05-07T02:28:34.646463+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:56:34.495133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.884083+0000","last_change":"2024-05-07T02:26:07.795300+0000","last_active":"2024-05-07T02:28:33.884083+0000","last_peered":"2024-05-07T02:28:33.884083+0000","last_clean":"2024-05-07T02:28:33.884083+0000","last_became_active":"2024-05-07T02:26:07.794633+0000","last_became_peered":"2024-05-07T02:26:07.794633+0000","last_unstale":"2024-05-07T02:28:33.884083+0000","last_undegraded":"2024-05-07T02:28:33.884083+0000","last_fullsized":"2024-05-07T02:28:33.884083+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:36:44.252118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.463072+0000","last_change":"2024-05-07T02:26:11.877717+0000","last_active":"2024-05-07T02:29:31.463072+0000","last_peered":"2024-05-07T02:29:31.463072+0000","last_clean":"2024-05-07T02:29:31.463072+0000","last_became_active":"2024-05-07T02:26:11.876644+0000","last_became_peered":"2024-05-07T02:26:11.876644+0000","last_unstale":"2024-05-07T02:29:31.463072+0000","last_undegraded":"2024-05-07T02:29:31.463072+0000","last_fullsized":"2024-05-07T02:29:31.463072+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T05:20:36.176776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.619993+0000","last_change":"2024-05-07T02:26:25.572120+0000","last_active":"2024-05-07T02:28:33.619993+0000","last_peered":"2024-05-07T02:28:33.619993+0000","last_clean":"2024-05-07T02:28:33.619993+0000","last_became_active":"2024-05-07T02:26:25.571637+0000","last_became_peered":"2024-05-07T02:26:25.571637+0000","last_unstale":"2024-05-07T02:28:33.619993+0000","last_undegraded":"2024-05-07T02:28:33.619993+0000","last_fullsized":"2024-05-07T02:28:33.619993+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:03:54.574061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.613500+0000","last_change":"2024-05-07T02:26:07.784160+0000","last_active":"2024-05-07T02:28:33.613500+0000","last_peered":"2024-05-07T02:28:33.613500+0000","last_clean":"2024-05-07T02:28:33.613500+0000","last_became_active":"2024-05-07T02:26:07.783646+0000","last_became_peered":"2024-05-07T02:26:07.783646+0000","last_unstale":"2024-05-07T02:28:33.613500+0000","last_undegraded":"2024-05-07T02:28:33.613500+0000","last_fullsized":"2024-05-07T02:28:33.613500+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T14:25:37.906587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.464206+0000","last_change":"2024-05-07T02:26:11.874886+0000","last_active":"2024-05-07T02:29:31.464206+0000","last_peered":"2024-05-07T02:29:31.464206+0000","last_clean":"2024-05-07T02:29:31.464206+0000","last_became_active":"2024-05-07T02:26:11.874465+0000","last_became_peered":"2024-05-07T02:26:11.874465+0000","last_unstale":"2024-05-07T02:29:31.464206+0000","last_undegraded":"2024-05-07T02:29:31.464206+0000","last_fullsized":"2024-05-07T02:29:31.464206+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:57:19.530747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.617066+0000","last_change":"2024-05-07T02:26:25.558087+0000","last_active":"2024-05-07T02:28:33.617066+0000","last_peered":"2024-05-07T02:28:33.617066+0000","last_clean":"2024-05-07T02:28:33.617066+0000","last_became_active":"2024-05-07T02:26:25.557624+0000","last_became_peered":"2024-05-07T02:26:25.557624+0000","last_unstale":"2024-05-07T02:28:33.617066+0000","last_undegraded":"2024-05-07T02:28:33.617066+0000","last_fullsized":"2024-05-07T02:28:33.617066+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:03:20.104547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":293,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:45.958719+0000","last_change":"2024-05-07T02:26:07.788244+0000","last_active":"2024-05-07T02:29:45.958719+0000","last_peered":"2024-05-07T02:29:45.958719+0000","last_clean":"2024-05-07T02:29:45.958719+0000","last_became_active":"2024-05-07T02:26:07.787789+0000","last_became_peered":"2024-05-07T02:26:07.787789+0000","last_unstale":"2024-05-07T02:29:45.958719+0000","last_undegraded":"2024-05-07T02:29:45.958719+0000","last_fullsized":"2024-05-07T02:29:45.958719+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:12:22.943444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":244,"num_read_kb":244,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.458376+0000","last_change":"2024-05-07T02:26:11.879821+0000","last_active":"2024-05-07T02:29:31.458376+0000","last_peered":"2024-05-07T02:29:31.458376+0000","last_clean":"2024-05-07T02:29:31.458376+0000","last_became_active":"2024-05-07T02:26:11.879372+0000","last_became_peered":"2024-05-07T02:26:11.879372+0000","last_unstale":"2024-05-07T02:29:31.458376+0000","last_undegraded":"2024-05-07T02:29:31.458376+0000","last_fullsized":"2024-05-07T02:29:31.458376+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:01:03.754890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.619139+0000","last_change":"2024-05-07T02:26:25.576663+0000","last_active":"2024-05-07T02:28:33.619139+0000","last_peered":"2024-05-07T02:28:33.619139+0000","last_clean":"2024-05-07T02:28:33.619139+0000","last_became_active":"2024-05-07T02:26:25.575729+0000","last_became_peered":"2024-05-07T02:26:25.575729+0000","last_unstale":"2024-05-07T02:28:33.619139+0000","last_undegraded":"2024-05-07T02:28:33.619139+0000","last_fullsized":"2024-05-07T02:28:33.619139+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:03:48.061385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.693451+0000","last_change":"2024-05-07T02:27:32.625336+0000","last_active":"2024-05-07T02:28:33.693451+0000","last_peered":"2024-05-07T02:28:33.693451+0000","last_clean":"2024-05-07T02:28:33.693451+0000","last_became_active":"2024-05-07T02:27:32.624805+0000","last_became_peered":"2024-05-07T02:27:32.624805+0000","last_unstale":"2024-05-07T02:28:33.693451+0000","last_undegraded":"2024-05-07T02:28:33.693451+0000","last_fullsized":"2024-05-07T02:28:33.693451+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:10:04.928297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.458002+0000","last_change":"2024-05-07T02:27:32.626374+0000","last_active":"2024-05-07T02:29:31.458002+0000","last_peered":"2024-05-07T02:29:31.458002+0000","last_clean":"2024-05-07T02:29:31.458002+0000","last_became_active":"2024-05-07T02:27:32.625489+0000","last_became_peered":"2024-05-07T02:27:32.625489+0000","last_unstale":"2024-05-07T02:29:31.458002+0000","last_undegraded":"2024-05-07T02:29:31.458002+0000","last_fullsized":"2024-05-07T02:29:31.458002+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T05:56:16.626041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.611819+0000","last_change":"2024-05-07T02:26:25.580811+0000","last_active":"2024-05-07T02:28:33.611819+0000","last_peered":"2024-05-07T02:28:33.611819+0000","last_clean":"2024-05-07T02:28:33.611819+0000","last_became_active":"2024-05-07T02:26:25.579344+0000","last_became_peered":"2024-05-07T02:26:25.579344+0000","last_unstale":"2024-05-07T02:28:33.611819+0000","last_undegraded":"2024-05-07T02:28:33.611819+0000","last_fullsized":"2024-05-07T02:28:33.611819+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:25:36.186899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.614114+0000","last_change":"2024-05-07T02:26:08.000010+0000","last_active":"2024-05-07T02:28:33.614114+0000","last_peered":"2024-05-07T02:28:33.614114+0000","last_clean":"2024-05-07T02:28:33.614114+0000","last_became_active":"2024-05-07T02:26:07.999576+0000","last_became_peered":"2024-05-07T02:26:07.999576+0000","last_unstale":"2024-05-07T02:28:33.614114+0000","last_undegraded":"2024-05-07T02:28:33.614114+0000","last_fullsized":"2024-05-07T02:28:33.614114+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:23:38.644205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.458253+0000","last_change":"2024-05-07T02:26:11.867098+0000","last_active":"2024-05-07T02:29:31.458253+0000","last_peered":"2024-05-07T02:29:31.458253+0000","last_clean":"2024-05-07T02:29:31.458253+0000","last_became_active":"2024-05-07T02:26:11.866036+0000","last_became_peered":"2024-05-07T02:26:11.866036+0000","last_unstale":"2024-05-07T02:29:31.458253+0000","last_undegraded":"2024-05-07T02:29:31.458253+0000","last_fullsized":"2024-05-07T02:29:31.458253+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:47:22.412323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.617223+0000","last_change":"2024-05-07T02:27:32.628243+0000","last_active":"2024-05-07T02:28:33.617223+0000","last_peered":"2024-05-07T02:28:33.617223+0000","last_clean":"2024-05-07T02:28:33.617223+0000","last_became_active":"2024-05-07T02:27:32.627048+0000","last_became_peered":"2024-05-07T02:27:32.627048+0000","last_unstale":"2024-05-07T02:28:33.617223+0000","last_undegraded":"2024-05-07T02:28:33.617223+0000","last_fullsized":"2024-05-07T02:28:33.617223+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:33:49.802873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,10],"acting":[9,8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'40","reported_seq":340,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:45.964096+0000","last_change":"2024-05-07T02:26:07.786841+0000","last_active":"2024-05-07T02:29:45.964096+0000","last_peered":"2024-05-07T02:29:45.964096+0000","last_clean":"2024-05-07T02:29:45.964096+0000","last_became_active":"2024-05-07T02:26:07.783836+0000","last_became_peered":"2024-05-07T02:26:07.783836+0000","last_unstale":"2024-05-07T02:29:45.964096+0000","last_undegraded":"2024-05-07T02:29:45.964096+0000","last_fullsized":"2024-05-07T02:29:45.964096+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:28:53.390009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68479,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":255,"num_read_kb":196,"num_write":42,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.426089+0000","last_change":"2024-05-07T02:26:11.857292+0000","last_active":"2024-05-07T02:29:31.426089+0000","last_peered":"2024-05-07T02:29:31.426089+0000","last_clean":"2024-05-07T02:29:31.426089+0000","last_became_active":"2024-05-07T02:26:11.856899+0000","last_became_peered":"2024-05-07T02:26:11.856899+0000","last_unstale":"2024-05-07T02:29:31.426089+0000","last_undegraded":"2024-05-07T02:29:31.426089+0000","last_fullsized":"2024-05-07T02:29:31.426089+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:45:42.279848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.614502+0000","last_change":"2024-05-07T02:26:25.570714+0000","last_active":"2024-05-07T02:28:33.614502+0000","last_peered":"2024-05-07T02:28:33.614502+0000","last_clean":"2024-05-07T02:28:33.614502+0000","last_became_active":"2024-05-07T02:26:25.570163+0000","last_became_peered":"2024-05-07T02:26:25.570163+0000","last_unstale":"2024-05-07T02:28:33.614502+0000","last_undegraded":"2024-05-07T02:28:33.614502+0000","last_fullsized":"2024-05-07T02:28:33.614502+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:55:51.903968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":56,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.868547+0000","last_change":"2024-05-07T02:26:07.785904+0000","last_active":"2024-05-07T02:28:33.868547+0000","last_peered":"2024-05-07T02:28:33.868547+0000","last_clean":"2024-05-07T02:28:33.868547+0000","last_became_active":"2024-05-07T02:26:07.785347+0000","last_became_peered":"2024-05-07T02:26:07.785347+0000","last_unstale":"2024-05-07T02:28:33.868547+0000","last_undegraded":"2024-05-07T02:28:33.868547+0000","last_fullsized":"2024-05-07T02:28:33.868547+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:42:05.604266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.458104+0000","last_change":"2024-05-07T02:26:11.878511+0000","last_active":"2024-05-07T02:29:31.458104+0000","last_peered":"2024-05-07T02:29:31.458104+0000","last_clean":"2024-05-07T02:29:31.458104+0000","last_became_active":"2024-05-07T02:26:11.878166+0000","last_became_peered":"2024-05-07T02:26:11.878166+0000","last_unstale":"2024-05-07T02:29:31.458104+0000","last_undegraded":"2024-05-07T02:29:31.458104+0000","last_fullsized":"2024-05-07T02:29:31.458104+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T08:14:40.436244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:34.645774+0000","last_change":"2024-05-07T02:26:25.579387+0000","last_active":"2024-05-07T02:28:34.645774+0000","last_peered":"2024-05-07T02:28:34.645774+0000","last_clean":"2024-05-07T02:28:34.645774+0000","last_became_active":"2024-05-07T02:26:25.578757+0000","last_became_peered":"2024-05-07T02:26:25.578757+0000","last_unstale":"2024-05-07T02:28:34.645774+0000","last_undegraded":"2024-05-07T02:28:34.645774+0000","last_fullsized":"2024-05-07T02:28:34.645774+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:19:19.650941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.613626+0000","last_change":"2024-05-07T02:26:07.785154+0000","last_active":"2024-05-07T02:28:33.613626+0000","last_peered":"2024-05-07T02:28:33.613626+0000","last_clean":"2024-05-07T02:28:33.613626+0000","last_became_active":"2024-05-07T02:26:07.784589+0000","last_became_peered":"2024-05-07T02:26:07.784589+0000","last_unstale":"2024-05-07T02:28:33.613626+0000","last_undegraded":"2024-05-07T02:28:33.613626+0000","last_fullsized":"2024-05-07T02:28:33.613626+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:41:41.851182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.447695+0000","last_change":"2024-05-07T02:26:11.878162+0000","last_active":"2024-05-07T02:29:31.447695+0000","last_peered":"2024-05-07T02:29:31.447695+0000","last_clean":"2024-05-07T02:29:31.447695+0000","last_became_active":"2024-05-07T02:26:11.877770+0000","last_became_peered":"2024-05-07T02:26:11.877770+0000","last_unstale":"2024-05-07T02:29:31.447695+0000","last_undegraded":"2024-05-07T02:29:31.447695+0000","last_fullsized":"2024-05-07T02:29:31.447695+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:19:08.209398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.619945+0000","last_change":"2024-05-07T02:26:25.579465+0000","last_active":"2024-05-07T02:28:33.619945+0000","last_peered":"2024-05-07T02:28:33.619945+0000","last_clean":"2024-05-07T02:28:33.619945+0000","last_became_active":"2024-05-07T02:26:25.579003+0000","last_became_peered":"2024-05-07T02:26:25.579003+0000","last_unstale":"2024-05-07T02:28:33.619945+0000","last_undegraded":"2024-05-07T02:28:33.619945+0000","last_fullsized":"2024-05-07T02:28:33.619945+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:13:24.356366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":176,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:47.074802+0000","last_change":"2024-05-07T02:26:07.780827+0000","last_active":"2024-05-07T02:29:47.074802+0000","last_peered":"2024-05-07T02:29:47.074802+0000","last_clean":"2024-05-07T02:29:47.074802+0000","last_became_active":"2024-05-07T02:26:07.780353+0000","last_became_peered":"2024-05-07T02:26:07.780353+0000","last_unstale":"2024-05-07T02:29:47.074802+0000","last_undegraded":"2024-05-07T02:29:47.074802+0000","last_fullsized":"2024-05-07T02:29:47.074802+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:52:40.554135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":127,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.446613+0000","last_change":"2024-05-07T02:26:11.852516+0000","last_active":"2024-05-07T02:29:31.446613+0000","last_peered":"2024-05-07T02:29:31.446613+0000","last_clean":"2024-05-07T02:29:31.446613+0000","last_became_active":"2024-05-07T02:26:11.852112+0000","last_became_peered":"2024-05-07T02:26:11.852112+0000","last_unstale":"2024-05-07T02:29:31.446613+0000","last_undegraded":"2024-05-07T02:29:31.446613+0000","last_fullsized":"2024-05-07T02:29:31.446613+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:58:46.989250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.886385+0000","last_change":"2024-05-07T02:26:25.586726+0000","last_active":"2024-05-07T02:28:33.886385+0000","last_peered":"2024-05-07T02:28:33.886385+0000","last_clean":"2024-05-07T02:28:33.886385+0000","last_became_active":"2024-05-07T02:26:25.586267+0000","last_became_peered":"2024-05-07T02:26:25.586267+0000","last_unstale":"2024-05-07T02:28:33.886385+0000","last_undegraded":"2024-05-07T02:28:33.886385+0000","last_fullsized":"2024-05-07T02:28:33.886385+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:50:00.864463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:34.645472+0000","last_change":"2024-05-07T02:27:32.599511+0000","last_active":"2024-05-07T02:28:34.645472+0000","last_peered":"2024-05-07T02:28:34.645472+0000","last_clean":"2024-05-07T02:28:34.645472+0000","last_became_active":"2024-05-07T02:27:32.590197+0000","last_became_peered":"2024-05-07T02:27:32.590197+0000","last_unstale":"2024-05-07T02:28:34.645472+0000","last_undegraded":"2024-05-07T02:28:34.645472+0000","last_fullsized":"2024-05-07T02:28:34.645472+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:23:07.401251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.436771+0000","last_change":"2024-05-07T02:26:11.857671+0000","last_active":"2024-05-07T02:29:31.436771+0000","last_peered":"2024-05-07T02:29:31.436771+0000","last_clean":"2024-05-07T02:29:31.436771+0000","last_became_active":"2024-05-07T02:26:11.857293+0000","last_became_peered":"2024-05-07T02:26:11.857293+0000","last_unstale":"2024-05-07T02:29:31.436771+0000","last_undegraded":"2024-05-07T02:29:31.436771+0000","last_fullsized":"2024-05-07T02:29:31.436771+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:48:42.267887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.613506+0000","last_change":"2024-05-07T02:26:25.563274+0000","last_active":"2024-05-07T02:28:33.613506+0000","last_peered":"2024-05-07T02:28:33.613506+0000","last_clean":"2024-05-07T02:28:33.613506+0000","last_became_active":"2024-05-07T02:26:25.562868+0000","last_became_peered":"2024-05-07T02:26:25.562868+0000","last_unstale":"2024-05-07T02:28:33.613506+0000","last_undegraded":"2024-05-07T02:28:33.613506+0000","last_fullsized":"2024-05-07T02:28:33.613506+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T14:20:03.210034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:34.646544+0000","last_change":"2024-05-07T02:26:07.784888+0000","last_active":"2024-05-07T02:28:34.646544+0000","last_peered":"2024-05-07T02:28:34.646544+0000","last_clean":"2024-05-07T02:28:34.646544+0000","last_became_active":"2024-05-07T02:26:07.784493+0000","last_became_peered":"2024-05-07T02:26:07.784493+0000","last_unstale":"2024-05-07T02:28:34.646544+0000","last_undegraded":"2024-05-07T02:28:34.646544+0000","last_fullsized":"2024-05-07T02:28:34.646544+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:21:14.054786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.443398+0000","last_change":"2024-05-07T02:26:11.858887+0000","last_active":"2024-05-07T02:29:31.443398+0000","last_peered":"2024-05-07T02:29:31.443398+0000","last_clean":"2024-05-07T02:29:31.443398+0000","last_became_active":"2024-05-07T02:26:11.858504+0000","last_became_peered":"2024-05-07T02:26:11.858504+0000","last_unstale":"2024-05-07T02:29:31.443398+0000","last_undegraded":"2024-05-07T02:29:31.443398+0000","last_fullsized":"2024-05-07T02:29:31.443398+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:53:20.290039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:34.645631+0000","last_change":"2024-05-07T02:27:32.627310+0000","last_active":"2024-05-07T02:28:34.645631+0000","last_peered":"2024-05-07T02:28:34.645631+0000","last_clean":"2024-05-07T02:28:34.645631+0000","last_became_active":"2024-05-07T02:27:32.626794+0000","last_became_peered":"2024-05-07T02:27:32.626794+0000","last_unstale":"2024-05-07T02:28:34.645631+0000","last_undegraded":"2024-05-07T02:28:34.645631+0000","last_fullsized":"2024-05-07T02:28:34.645631+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:06:30.466291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.618984+0000","last_change":"2024-05-07T02:26:08.000604+0000","last_active":"2024-05-07T02:28:33.618984+0000","last_peered":"2024-05-07T02:28:33.618984+0000","last_clean":"2024-05-07T02:28:33.618984+0000","last_became_active":"2024-05-07T02:26:08.000107+0000","last_became_peered":"2024-05-07T02:26:08.000107+0000","last_unstale":"2024-05-07T02:28:33.618984+0000","last_undegraded":"2024-05-07T02:28:33.618984+0000","last_fullsized":"2024-05-07T02:28:33.618984+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:58:06.716456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.450534+0000","last_change":"2024-05-07T02:26:11.876555+0000","last_active":"2024-05-07T02:29:31.450534+0000","last_peered":"2024-05-07T02:29:31.450534+0000","last_clean":"2024-05-07T02:29:31.450534+0000","last_became_active":"2024-05-07T02:26:11.876109+0000","last_became_peered":"2024-05-07T02:26:11.876109+0000","last_unstale":"2024-05-07T02:29:31.450534+0000","last_undegraded":"2024-05-07T02:29:31.450534+0000","last_fullsized":"2024-05-07T02:29:31.450534+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:10:24.995786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.884837+0000","last_change":"2024-05-07T02:26:25.567145+0000","last_active":"2024-05-07T02:28:33.884837+0000","last_peered":"2024-05-07T02:28:33.884837+0000","last_clean":"2024-05-07T02:28:33.884837+0000","last_became_active":"2024-05-07T02:26:25.566767+0000","last_became_peered":"2024-05-07T02:26:25.566767+0000","last_unstale":"2024-05-07T02:28:33.884837+0000","last_undegraded":"2024-05-07T02:28:33.884837+0000","last_fullsized":"2024-05-07T02:28:33.884837+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T05:10:59.333367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'7","reported_seq":175,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:42.995880+0000","last_change":"2024-05-07T02:26:07.789566+0000","last_active":"2024-05-07T02:29:42.995880+0000","last_peered":"2024-05-07T02:29:42.995880+0000","last_clean":"2024-05-07T02:29:42.995880+0000","last_became_active":"2024-05-07T02:26:07.789106+0000","last_became_peered":"2024-05-07T02:26:07.789106+0000","last_unstale":"2024-05-07T02:29:42.995880+0000","last_undegraded":"2024-05-07T02:29:42.995880+0000","last_fullsized":"2024-05-07T02:29:42.995880+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:45:13.620183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":123,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.443897+0000","last_change":"2024-05-07T02:26:11.869080+0000","last_active":"2024-05-07T02:29:31.443897+0000","last_peered":"2024-05-07T02:29:31.443897+0000","last_clean":"2024-05-07T02:29:31.443897+0000","last_became_active":"2024-05-07T02:26:11.868702+0000","last_became_peered":"2024-05-07T02:26:11.868702+0000","last_unstale":"2024-05-07T02:29:31.443897+0000","last_undegraded":"2024-05-07T02:29:31.443897+0000","last_fullsized":"2024-05-07T02:29:31.443897+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:05:05.497071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.882275+0000","last_change":"2024-05-07T02:26:25.867393+0000","last_active":"2024-05-07T02:28:33.882275+0000","last_peered":"2024-05-07T02:28:33.882275+0000","last_clean":"2024-05-07T02:28:33.882275+0000","last_became_active":"2024-05-07T02:26:25.866956+0000","last_became_peered":"2024-05-07T02:26:25.866956+0000","last_unstale":"2024-05-07T02:28:33.882275+0000","last_undegraded":"2024-05-07T02:28:33.882275+0000","last_fullsized":"2024-05-07T02:28:33.882275+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:23:48.821668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.442457+0000","last_change":"2024-05-07T02:28:33.871287+0000","last_active":"2024-05-07T02:29:31.442457+0000","last_peered":"2024-05-07T02:29:31.442457+0000","last_clean":"2024-05-07T02:29:31.442457+0000","last_became_active":"2024-05-07T02:28:33.870256+0000","last_became_peered":"2024-05-07T02:28:33.870256+0000","last_unstale":"2024-05-07T02:29:31.442457+0000","last_undegraded":"2024-05-07T02:29:31.442457+0000","last_fullsized":"2024-05-07T02:29:31.442457+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T08:36:26.789348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":173,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:45.069415+0000","last_change":"2024-05-07T02:26:07.999784+0000","last_active":"2024-05-07T02:29:45.069415+0000","last_peered":"2024-05-07T02:29:45.069415+0000","last_clean":"2024-05-07T02:29:45.069415+0000","last_became_active":"2024-05-07T02:26:07.999339+0000","last_became_peered":"2024-05-07T02:26:07.999339+0000","last_unstale":"2024-05-07T02:29:45.069415+0000","last_undegraded":"2024-05-07T02:29:45.069415+0000","last_fullsized":"2024-05-07T02:29:45.069415+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:50:41.622486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":123,"num_read_kb":123,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.693213+0000","last_change":"2024-05-07T02:26:25.587032+0000","last_active":"2024-05-07T02:28:33.693213+0000","last_peered":"2024-05-07T02:28:33.693213+0000","last_clean":"2024-05-07T02:28:33.693213+0000","last_became_active":"2024-05-07T02:26:25.586530+0000","last_became_peered":"2024-05-07T02:26:25.586530+0000","last_unstale":"2024-05-07T02:28:33.693213+0000","last_undegraded":"2024-05-07T02:28:33.693213+0000","last_fullsized":"2024-05-07T02:28:33.693213+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:13:13.335832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.462997+0000","last_change":"2024-05-07T02:26:11.878593+0000","last_active":"2024-05-07T02:29:31.462997+0000","last_peered":"2024-05-07T02:29:31.462997+0000","last_clean":"2024-05-07T02:29:31.462997+0000","last_became_active":"2024-05-07T02:26:11.873286+0000","last_became_peered":"2024-05-07T02:26:11.873286+0000","last_unstale":"2024-05-07T02:29:31.462997+0000","last_undegraded":"2024-05-07T02:29:31.462997+0000","last_fullsized":"2024-05-07T02:29:31.462997+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:22:33.769539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.611714+0000","last_change":"2024-05-07T02:26:07.790087+0000","last_active":"2024-05-07T02:28:33.611714+0000","last_peered":"2024-05-07T02:28:33.611714+0000","last_clean":"2024-05-07T02:28:33.611714+0000","last_became_active":"2024-05-07T02:26:07.789592+0000","last_became_peered":"2024-05-07T02:26:07.789592+0000","last_unstale":"2024-05-07T02:28:33.611714+0000","last_undegraded":"2024-05-07T02:28:33.611714+0000","last_fullsized":"2024-05-07T02:28:33.611714+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:44:52.569055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":119,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.692667+0000","last_change":"2024-05-07T02:24:27.877820+0000","last_active":"2024-05-07T02:28:33.692667+0000","last_peered":"2024-05-07T02:28:33.692667+0000","last_clean":"2024-05-07T02:28:33.692667+0000","last_became_active":"2024-05-07T02:24:27.873221+0000","last_became_peered":"2024-05-07T02:24:27.873221+0000","last_unstale":"2024-05-07T02:28:33.692667+0000","last_undegraded":"2024-05-07T02:28:33.692667+0000","last_fullsized":"2024-05-07T02:28:33.692667+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:20:21.582322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:20:21.582322+0000","last_clean_scrub_stamp":"2024-05-07T02:20:21.582322+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T14:16:32.772438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.614338+0000","last_change":"2024-05-07T02:26:25.868394+0000","last_active":"2024-05-07T02:28:33.614338+0000","last_peered":"2024-05-07T02:28:33.614338+0000","last_clean":"2024-05-07T02:28:33.614338+0000","last_became_active":"2024-05-07T02:26:25.867583+0000","last_became_peered":"2024-05-07T02:26:25.867583+0000","last_unstale":"2024-05-07T02:28:33.614338+0000","last_undegraded":"2024-05-07T02:28:33.614338+0000","last_fullsized":"2024-05-07T02:28:33.614338+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:55:44.705608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.436288+0000","last_change":"2024-05-07T02:26:11.868371+0000","last_active":"2024-05-07T02:29:31.436288+0000","last_peered":"2024-05-07T02:29:31.436288+0000","last_clean":"2024-05-07T02:29:31.436288+0000","last_became_active":"2024-05-07T02:26:11.867981+0000","last_became_peered":"2024-05-07T02:26:11.867981+0000","last_unstale":"2024-05-07T02:29:31.436288+0000","last_undegraded":"2024-05-07T02:29:31.436288+0000","last_fullsized":"2024-05-07T02:29:31.436288+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:08:54.687327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.692876+0000","last_change":"2024-05-07T02:26:07.785344+0000","last_active":"2024-05-07T02:28:33.692876+0000","last_peered":"2024-05-07T02:28:33.692876+0000","last_clean":"2024-05-07T02:28:33.692876+0000","last_became_active":"2024-05-07T02:26:07.784707+0000","last_became_peered":"2024-05-07T02:26:07.784707+0000","last_unstale":"2024-05-07T02:28:33.692876+0000","last_undegraded":"2024-05-07T02:28:33.692876+0000","last_fullsized":"2024-05-07T02:28:33.692876+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T05:00:38.754313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:34.645559+0000","last_change":"2024-05-07T02:26:25.571989+0000","last_active":"2024-05-07T02:28:34.645559+0000","last_peered":"2024-05-07T02:28:34.645559+0000","last_clean":"2024-05-07T02:28:34.645559+0000","last_became_active":"2024-05-07T02:26:25.571533+0000","last_became_peered":"2024-05-07T02:26:25.571533+0000","last_unstale":"2024-05-07T02:28:34.645559+0000","last_undegraded":"2024-05-07T02:28:34.645559+0000","last_fullsized":"2024-05-07T02:28:34.645559+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:54:38.669278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:34.646468+0000","last_change":"2024-05-07T02:26:07.999636+0000","last_active":"2024-05-07T02:28:34.646468+0000","last_peered":"2024-05-07T02:28:34.646468+0000","last_clean":"2024-05-07T02:28:34.646468+0000","last_became_active":"2024-05-07T02:26:07.999100+0000","last_became_peered":"2024-05-07T02:26:07.999100+0000","last_unstale":"2024-05-07T02:28:34.646468+0000","last_undegraded":"2024-05-07T02:28:34.646468+0000","last_fullsized":"2024-05-07T02:28:34.646468+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:10:24.730988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.440226+0000","last_change":"2024-05-07T02:26:11.888279+0000","last_active":"2024-05-07T02:29:31.440226+0000","last_peered":"2024-05-07T02:29:31.440226+0000","last_clean":"2024-05-07T02:29:31.440226+0000","last_became_active":"2024-05-07T02:26:11.887480+0000","last_became_peered":"2024-05-07T02:26:11.887480+0000","last_unstale":"2024-05-07T02:29:31.440226+0000","last_undegraded":"2024-05-07T02:29:31.440226+0000","last_fullsized":"2024-05-07T02:29:31.440226+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:12:12.368368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.615801+0000","last_change":"2024-05-07T02:26:25.867444+0000","last_active":"2024-05-07T02:28:33.615801+0000","last_peered":"2024-05-07T02:28:33.615801+0000","last_clean":"2024-05-07T02:28:33.615801+0000","last_became_active":"2024-05-07T02:26:25.866392+0000","last_became_peered":"2024-05-07T02:26:25.866392+0000","last_unstale":"2024-05-07T02:28:33.615801+0000","last_undegraded":"2024-05-07T02:28:33.615801+0000","last_fullsized":"2024-05-07T02:28:33.615801+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:54:36.112427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'17","reported_seq":62,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.618768+0000","last_change":"2024-05-07T02:26:07.798091+0000","last_active":"2024-05-07T02:28:33.618768+0000","last_peered":"2024-05-07T02:28:33.618768+0000","last_clean":"2024-05-07T02:28:33.618768+0000","last_became_active":"2024-05-07T02:26:07.797716+0000","last_became_peered":"2024-05-07T02:26:07.797716+0000","last_unstale":"2024-05-07T02:28:33.618768+0000","last_undegraded":"2024-05-07T02:28:33.618768+0000","last_fullsized":"2024-05-07T02:28:33.618768+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:04:54.570916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.464763+0000","last_change":"2024-05-07T02:26:11.888638+0000","last_active":"2024-05-07T02:29:31.464763+0000","last_peered":"2024-05-07T02:29:31.464763+0000","last_clean":"2024-05-07T02:29:31.464763+0000","last_became_active":"2024-05-07T02:26:11.888269+0000","last_became_peered":"2024-05-07T02:26:11.888269+0000","last_unstale":"2024-05-07T02:29:31.464763+0000","last_undegraded":"2024-05-07T02:29:31.464763+0000","last_fullsized":"2024-05-07T02:29:31.464763+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:09:38.965362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.868019+0000","last_change":"2024-05-07T02:26:25.558975+0000","last_active":"2024-05-07T02:28:33.868019+0000","last_peered":"2024-05-07T02:28:33.868019+0000","last_clean":"2024-05-07T02:28:33.868019+0000","last_became_active":"2024-05-07T02:26:25.557947+0000","last_became_peered":"2024-05-07T02:26:25.557947+0000","last_unstale":"2024-05-07T02:28:33.868019+0000","last_undegraded":"2024-05-07T02:28:33.868019+0000","last_fullsized":"2024-05-07T02:28:33.868019+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:29:32.071625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.612345+0000","last_change":"2024-05-07T02:26:07.796643+0000","last_active":"2024-05-07T02:28:33.612345+0000","last_peered":"2024-05-07T02:28:33.612345+0000","last_clean":"2024-05-07T02:28:33.612345+0000","last_became_active":"2024-05-07T02:26:07.796264+0000","last_became_peered":"2024-05-07T02:26:07.796264+0000","last_unstale":"2024-05-07T02:28:33.612345+0000","last_undegraded":"2024-05-07T02:28:33.612345+0000","last_fullsized":"2024-05-07T02:28:33.612345+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:29:32.454816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.463845+0000","last_change":"2024-05-07T02:26:11.873670+0000","last_active":"2024-05-07T02:29:31.463845+0000","last_peered":"2024-05-07T02:29:31.463845+0000","last_clean":"2024-05-07T02:29:31.463845+0000","last_became_active":"2024-05-07T02:26:11.871219+0000","last_became_peered":"2024-05-07T02:26:11.871219+0000","last_unstale":"2024-05-07T02:29:31.463845+0000","last_undegraded":"2024-05-07T02:29:31.463845+0000","last_fullsized":"2024-05-07T02:29:31.463845+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:49:48.048127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.612451+0000","last_change":"2024-05-07T02:26:25.868011+0000","last_active":"2024-05-07T02:28:33.612451+0000","last_peered":"2024-05-07T02:28:33.612451+0000","last_clean":"2024-05-07T02:28:33.612451+0000","last_became_active":"2024-05-07T02:26:25.867465+0000","last_became_peered":"2024-05-07T02:26:25.867465+0000","last_unstale":"2024-05-07T02:28:33.612451+0000","last_undegraded":"2024-05-07T02:28:33.612451+0000","last_fullsized":"2024-05-07T02:28:33.612451+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T05:14:40.001874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:34.333778+0000","last_change":"2024-05-07T02:26:07.999233+0000","last_active":"2024-05-07T02:28:34.333778+0000","last_peered":"2024-05-07T02:28:34.333778+0000","last_clean":"2024-05-07T02:28:34.333778+0000","last_became_active":"2024-05-07T02:26:07.998920+0000","last_became_peered":"2024-05-07T02:26:07.998920+0000","last_unstale":"2024-05-07T02:28:34.333778+0000","last_undegraded":"2024-05-07T02:28:34.333778+0000","last_fullsized":"2024-05-07T02:28:34.333778+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:33:33.878984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.401820+0000","last_change":"2024-05-07T02:26:11.856832+0000","last_active":"2024-05-07T02:29:31.401820+0000","last_peered":"2024-05-07T02:29:31.401820+0000","last_clean":"2024-05-07T02:29:31.401820+0000","last_became_active":"2024-05-07T02:26:11.856424+0000","last_became_peered":"2024-05-07T02:26:11.856424+0000","last_unstale":"2024-05-07T02:29:31.401820+0000","last_undegraded":"2024-05-07T02:29:31.401820+0000","last_fullsized":"2024-05-07T02:29:31.401820+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:00:46.839542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:34.646542+0000","last_change":"2024-05-07T02:27:32.626086+0000","last_active":"2024-05-07T02:28:34.646542+0000","last_peered":"2024-05-07T02:28:34.646542+0000","last_clean":"2024-05-07T02:28:34.646542+0000","last_became_active":"2024-05-07T02:27:32.625468+0000","last_became_peered":"2024-05-07T02:27:32.625468+0000","last_unstale":"2024-05-07T02:28:34.646542+0000","last_undegraded":"2024-05-07T02:28:34.646542+0000","last_fullsized":"2024-05-07T02:28:34.646542+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:17:57.245648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.620116+0000","last_change":"2024-05-07T02:26:07.793793+0000","last_active":"2024-05-07T02:28:33.620116+0000","last_peered":"2024-05-07T02:28:33.620116+0000","last_clean":"2024-05-07T02:28:33.620116+0000","last_became_active":"2024-05-07T02:26:07.790291+0000","last_became_peered":"2024-05-07T02:26:07.790291+0000","last_unstale":"2024-05-07T02:28:33.620116+0000","last_undegraded":"2024-05-07T02:28:33.620116+0000","last_fullsized":"2024-05-07T02:28:33.620116+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:50:14.591968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.458904+0000","last_change":"2024-05-07T02:26:11.869184+0000","last_active":"2024-05-07T02:29:31.458904+0000","last_peered":"2024-05-07T02:29:31.458904+0000","last_clean":"2024-05-07T02:29:31.458904+0000","last_became_active":"2024-05-07T02:26:11.868773+0000","last_became_peered":"2024-05-07T02:26:11.868773+0000","last_unstale":"2024-05-07T02:29:31.458904+0000","last_undegraded":"2024-05-07T02:29:31.458904+0000","last_fullsized":"2024-05-07T02:29:31.458904+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:24:04.986359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.612145+0000","last_change":"2024-05-07T02:26:25.580095+0000","last_active":"2024-05-07T02:28:33.612145+0000","last_peered":"2024-05-07T02:28:33.612145+0000","last_clean":"2024-05-07T02:28:33.612145+0000","last_became_active":"2024-05-07T02:26:25.579615+0000","last_became_peered":"2024-05-07T02:26:25.579615+0000","last_unstale":"2024-05-07T02:28:33.612145+0000","last_undegraded":"2024-05-07T02:28:33.612145+0000","last_fullsized":"2024-05-07T02:28:33.612145+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:18:05.714552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.868184+0000","last_change":"2024-05-07T02:26:07.784709+0000","last_active":"2024-05-07T02:28:33.868184+0000","last_peered":"2024-05-07T02:28:33.868184+0000","last_clean":"2024-05-07T02:28:33.868184+0000","last_became_active":"2024-05-07T02:26:07.784282+0000","last_became_peered":"2024-05-07T02:26:07.784282+0000","last_unstale":"2024-05-07T02:28:33.868184+0000","last_undegraded":"2024-05-07T02:28:33.868184+0000","last_fullsized":"2024-05-07T02:28:33.868184+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:01:47.023496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.463214+0000","last_change":"2024-05-07T02:26:11.879152+0000","last_active":"2024-05-07T02:29:31.463214+0000","last_peered":"2024-05-07T02:29:31.463214+0000","last_clean":"2024-05-07T02:29:31.463214+0000","last_became_active":"2024-05-07T02:26:11.878571+0000","last_became_peered":"2024-05-07T02:26:11.878571+0000","last_unstale":"2024-05-07T02:29:31.463214+0000","last_undegraded":"2024-05-07T02:29:31.463214+0000","last_fullsized":"2024-05-07T02:29:31.463214+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:04:12.637807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.612216+0000","last_change":"2024-05-07T02:26:25.867199+0000","last_active":"2024-05-07T02:28:33.612216+0000","last_peered":"2024-05-07T02:28:33.612216+0000","last_clean":"2024-05-07T02:28:33.612216+0000","last_became_active":"2024-05-07T02:26:25.866624+0000","last_became_peered":"2024-05-07T02:26:25.866624+0000","last_unstale":"2024-05-07T02:28:33.612216+0000","last_undegraded":"2024-05-07T02:28:33.612216+0000","last_fullsized":"2024-05-07T02:28:33.612216+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:14:49.513313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:34.645931+0000","last_change":"2024-05-07T02:26:07.784155+0000","last_active":"2024-05-07T02:28:34.645931+0000","last_peered":"2024-05-07T02:28:34.645931+0000","last_clean":"2024-05-07T02:28:34.645931+0000","last_became_active":"2024-05-07T02:26:07.783579+0000","last_became_peered":"2024-05-07T02:26:07.783579+0000","last_unstale":"2024-05-07T02:28:34.645931+0000","last_undegraded":"2024-05-07T02:28:34.645931+0000","last_fullsized":"2024-05-07T02:28:34.645931+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T08:51:32.037959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.457140+0000","last_change":"2024-05-07T02:26:11.868454+0000","last_active":"2024-05-07T02:29:31.457140+0000","last_peered":"2024-05-07T02:29:31.457140+0000","last_clean":"2024-05-07T02:29:31.457140+0000","last_became_active":"2024-05-07T02:26:11.868117+0000","last_became_peered":"2024-05-07T02:26:11.868117+0000","last_unstale":"2024-05-07T02:29:31.457140+0000","last_undegraded":"2024-05-07T02:29:31.457140+0000","last_fullsized":"2024-05-07T02:29:31.457140+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:24:36.703684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:34.333863+0000","last_change":"2024-05-07T02:26:25.573834+0000","last_active":"2024-05-07T02:28:34.333863+0000","last_peered":"2024-05-07T02:28:34.333863+0000","last_clean":"2024-05-07T02:28:34.333863+0000","last_became_active":"2024-05-07T02:26:25.573372+0000","last_became_peered":"2024-05-07T02:26:25.573372+0000","last_unstale":"2024-05-07T02:28:34.333863+0000","last_undegraded":"2024-05-07T02:28:34.333863+0000","last_fullsized":"2024-05-07T02:28:34.333863+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:03:36.916915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":174,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:45.030448+0000","last_change":"2024-05-07T02:26:08.000245+0000","last_active":"2024-05-07T02:29:45.030448+0000","last_peered":"2024-05-07T02:29:45.030448+0000","last_clean":"2024-05-07T02:29:45.030448+0000","last_became_active":"2024-05-07T02:26:07.999587+0000","last_became_peered":"2024-05-07T02:26:07.999587+0000","last_unstale":"2024-05-07T02:29:45.030448+0000","last_undegraded":"2024-05-07T02:29:45.030448+0000","last_fullsized":"2024-05-07T02:29:45.030448+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:44:41.754799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":124,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.452891+0000","last_change":"2024-05-07T02:26:11.858308+0000","last_active":"2024-05-07T02:29:31.452891+0000","last_peered":"2024-05-07T02:29:31.452891+0000","last_clean":"2024-05-07T02:29:31.452891+0000","last_became_active":"2024-05-07T02:26:11.857855+0000","last_became_peered":"2024-05-07T02:26:11.857855+0000","last_unstale":"2024-05-07T02:29:31.452891+0000","last_undegraded":"2024-05-07T02:29:31.452891+0000","last_fullsized":"2024-05-07T02:29:31.452891+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:11:50.021178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.615521+0000","last_change":"2024-05-07T02:26:25.584090+0000","last_active":"2024-05-07T02:28:33.615521+0000","last_peered":"2024-05-07T02:28:33.615521+0000","last_clean":"2024-05-07T02:28:33.615521+0000","last_became_active":"2024-05-07T02:26:25.580108+0000","last_became_peered":"2024-05-07T02:26:25.580108+0000","last_unstale":"2024-05-07T02:28:33.615521+0000","last_undegraded":"2024-05-07T02:28:33.615521+0000","last_fullsized":"2024-05-07T02:28:33.615521+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:56:56.964047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.614227+0000","last_change":"2024-05-07T02:26:07.788188+0000","last_active":"2024-05-07T02:28:33.614227+0000","last_peered":"2024-05-07T02:28:33.614227+0000","last_clean":"2024-05-07T02:28:33.614227+0000","last_became_active":"2024-05-07T02:26:07.787826+0000","last_became_peered":"2024-05-07T02:26:07.787826+0000","last_unstale":"2024-05-07T02:28:33.614227+0000","last_undegraded":"2024-05-07T02:28:33.614227+0000","last_fullsized":"2024-05-07T02:28:33.614227+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:06:14.588079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.454390+0000","last_change":"2024-05-07T02:26:11.862925+0000","last_active":"2024-05-07T02:29:31.454390+0000","last_peered":"2024-05-07T02:29:31.454390+0000","last_clean":"2024-05-07T02:29:31.454390+0000","last_became_active":"2024-05-07T02:26:11.861332+0000","last_became_peered":"2024-05-07T02:26:11.861332+0000","last_unstale":"2024-05-07T02:29:31.454390+0000","last_undegraded":"2024-05-07T02:29:31.454390+0000","last_fullsized":"2024-05-07T02:29:31.454390+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:51:11.133504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.613240+0000","last_change":"2024-05-07T02:26:25.557151+0000","last_active":"2024-05-07T02:28:33.613240+0000","last_peered":"2024-05-07T02:28:33.613240+0000","last_clean":"2024-05-07T02:28:33.613240+0000","last_became_active":"2024-05-07T02:26:25.556615+0000","last_became_peered":"2024-05-07T02:26:25.556615+0000","last_unstale":"2024-05-07T02:28:33.613240+0000","last_undegraded":"2024-05-07T02:28:33.613240+0000","last_fullsized":"2024-05-07T02:28:33.613240+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:33:20.446543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.885890+0000","last_change":"2024-05-07T02:27:32.598094+0000","last_active":"2024-05-07T02:28:33.885890+0000","last_peered":"2024-05-07T02:28:33.885890+0000","last_clean":"2024-05-07T02:28:33.885890+0000","last_became_active":"2024-05-07T02:27:32.594099+0000","last_became_peered":"2024-05-07T02:27:32.594099+0000","last_unstale":"2024-05-07T02:28:33.885890+0000","last_undegraded":"2024-05-07T02:28:33.885890+0000","last_fullsized":"2024-05-07T02:28:33.885890+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:44:50.979927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.443992+0000","last_change":"2024-05-07T02:26:11.860507+0000","last_active":"2024-05-07T02:29:31.443992+0000","last_peered":"2024-05-07T02:29:31.443992+0000","last_clean":"2024-05-07T02:29:31.443992+0000","last_became_active":"2024-05-07T02:26:11.860116+0000","last_became_peered":"2024-05-07T02:26:11.860116+0000","last_unstale":"2024-05-07T02:29:31.443992+0000","last_undegraded":"2024-05-07T02:29:31.443992+0000","last_fullsized":"2024-05-07T02:29:31.443992+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T05:09:38.963040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.434049+0000","last_change":"2024-05-07T02:26:11.867932+0000","last_active":"2024-05-07T02:29:31.434049+0000","last_peered":"2024-05-07T02:29:31.434049+0000","last_clean":"2024-05-07T02:29:31.434049+0000","last_became_active":"2024-05-07T02:26:11.867537+0000","last_became_peered":"2024-05-07T02:26:11.867537+0000","last_unstale":"2024-05-07T02:29:31.434049+0000","last_undegraded":"2024-05-07T02:29:31.434049+0000","last_fullsized":"2024-05-07T02:29:31.434049+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:31:35.843401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.435936+0000","last_change":"2024-05-07T02:26:11.866074+0000","last_active":"2024-05-07T02:29:31.435936+0000","last_peered":"2024-05-07T02:29:31.435936+0000","last_clean":"2024-05-07T02:29:31.435936+0000","last_became_active":"2024-05-07T02:26:11.865669+0000","last_became_peered":"2024-05-07T02:26:11.865669+0000","last_unstale":"2024-05-07T02:29:31.435936+0000","last_undegraded":"2024-05-07T02:29:31.435936+0000","last_fullsized":"2024-05-07T02:29:31.435936+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:08:51.323706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.458706+0000","last_change":"2024-05-07T02:26:11.876428+0000","last_active":"2024-05-07T02:29:31.458706+0000","last_peered":"2024-05-07T02:29:31.458706+0000","last_clean":"2024-05-07T02:29:31.458706+0000","last_became_active":"2024-05-07T02:26:11.876012+0000","last_became_peered":"2024-05-07T02:26:11.876012+0000","last_unstale":"2024-05-07T02:29:31.458706+0000","last_undegraded":"2024-05-07T02:29:31.458706+0000","last_fullsized":"2024-05-07T02:29:31.458706+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:24:58.088695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.458687+0000","last_change":"2024-05-07T02:26:11.880682+0000","last_active":"2024-05-07T02:29:31.458687+0000","last_peered":"2024-05-07T02:29:31.458687+0000","last_clean":"2024-05-07T02:29:31.458687+0000","last_became_active":"2024-05-07T02:26:11.880308+0000","last_became_peered":"2024-05-07T02:26:11.880308+0000","last_unstale":"2024-05-07T02:29:31.458687+0000","last_undegraded":"2024-05-07T02:29:31.458687+0000","last_fullsized":"2024-05-07T02:29:31.458687+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:52:45.083964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.456972+0000","last_change":"2024-05-07T02:26:11.876932+0000","last_active":"2024-05-07T02:29:31.456972+0000","last_peered":"2024-05-07T02:29:31.456972+0000","last_clean":"2024-05-07T02:29:31.456972+0000","last_became_active":"2024-05-07T02:26:11.876082+0000","last_became_peered":"2024-05-07T02:26:11.876082+0000","last_unstale":"2024-05-07T02:29:31.456972+0000","last_undegraded":"2024-05-07T02:29:31.456972+0000","last_fullsized":"2024-05-07T02:29:31.456972+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:45:21.435500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.425660+0000","last_change":"2024-05-07T02:26:11.861339+0000","last_active":"2024-05-07T02:29:31.425660+0000","last_peered":"2024-05-07T02:29:31.425660+0000","last_clean":"2024-05-07T02:29:31.425660+0000","last_became_active":"2024-05-07T02:26:11.860912+0000","last_became_peered":"2024-05-07T02:26:11.860912+0000","last_unstale":"2024-05-07T02:29:31.425660+0000","last_undegraded":"2024-05-07T02:29:31.425660+0000","last_fullsized":"2024-05-07T02:29:31.425660+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:12:57.944986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.449041+0000","last_change":"2024-05-07T02:26:11.867258+0000","last_active":"2024-05-07T02:29:31.449041+0000","last_peered":"2024-05-07T02:29:31.449041+0000","last_clean":"2024-05-07T02:29:31.449041+0000","last_became_active":"2024-05-07T02:26:11.866837+0000","last_became_peered":"2024-05-07T02:26:11.866837+0000","last_unstale":"2024-05-07T02:29:31.449041+0000","last_undegraded":"2024-05-07T02:29:31.449041+0000","last_fullsized":"2024-05-07T02:29:31.449041+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:07:58.589457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.447442+0000","last_change":"2024-05-07T02:26:11.888434+0000","last_active":"2024-05-07T02:29:31.447442+0000","last_peered":"2024-05-07T02:29:31.447442+0000","last_clean":"2024-05-07T02:29:31.447442+0000","last_became_active":"2024-05-07T02:26:11.887681+0000","last_became_peered":"2024-05-07T02:26:11.887681+0000","last_unstale":"2024-05-07T02:29:31.447442+0000","last_undegraded":"2024-05-07T02:29:31.447442+0000","last_fullsized":"2024-05-07T02:29:31.447442+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:38:58.896781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.453153+0000","last_change":"2024-05-07T02:26:11.875304+0000","last_active":"2024-05-07T02:29:31.453153+0000","last_peered":"2024-05-07T02:29:31.453153+0000","last_clean":"2024-05-07T02:29:31.453153+0000","last_became_active":"2024-05-07T02:26:11.874946+0000","last_became_peered":"2024-05-07T02:26:11.874946+0000","last_unstale":"2024-05-07T02:29:31.453153+0000","last_undegraded":"2024-05-07T02:29:31.453153+0000","last_fullsized":"2024-05-07T02:29:31.453153+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:56:04.093791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.463312+0000","last_change":"2024-05-07T02:26:11.879311+0000","last_active":"2024-05-07T02:29:31.463312+0000","last_peered":"2024-05-07T02:29:31.463312+0000","last_clean":"2024-05-07T02:29:31.463312+0000","last_became_active":"2024-05-07T02:26:11.877620+0000","last_became_peered":"2024-05-07T02:26:11.877620+0000","last_unstale":"2024-05-07T02:29:31.463312+0000","last_undegraded":"2024-05-07T02:29:31.463312+0000","last_fullsized":"2024-05-07T02:29:31.463312+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T08:41:40.525708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.463659+0000","last_change":"2024-05-07T02:26:11.876726+0000","last_active":"2024-05-07T02:29:31.463659+0000","last_peered":"2024-05-07T02:29:31.463659+0000","last_clean":"2024-05-07T02:29:31.463659+0000","last_became_active":"2024-05-07T02:26:11.876234+0000","last_became_peered":"2024-05-07T02:26:11.876234+0000","last_unstale":"2024-05-07T02:29:31.463659+0000","last_undegraded":"2024-05-07T02:29:31.463659+0000","last_fullsized":"2024-05-07T02:29:31.463659+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:26:34.422258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.460248+0000","last_change":"2024-05-07T02:26:11.879159+0000","last_active":"2024-05-07T02:29:31.460248+0000","last_peered":"2024-05-07T02:29:31.460248+0000","last_clean":"2024-05-07T02:29:31.460248+0000","last_became_active":"2024-05-07T02:26:11.878769+0000","last_became_peered":"2024-05-07T02:26:11.878769+0000","last_unstale":"2024-05-07T02:29:31.460248+0000","last_undegraded":"2024-05-07T02:29:31.460248+0000","last_fullsized":"2024-05-07T02:29:31.460248+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:05:27.089475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.441690+0000","last_change":"2024-05-07T02:27:32.590188+0000","last_active":"2024-05-07T02:29:31.441690+0000","last_peered":"2024-05-07T02:29:31.441690+0000","last_clean":"2024-05-07T02:29:31.441690+0000","last_became_active":"2024-05-07T02:27:32.589784+0000","last_became_peered":"2024-05-07T02:27:32.589784+0000","last_unstale":"2024-05-07T02:29:31.441690+0000","last_undegraded":"2024-05-07T02:29:31.441690+0000","last_fullsized":"2024-05-07T02:29:31.441690+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:32:40.488143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":75433,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1268,"num_read_kb":718,"num_write":258,"num_write_kb":153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":226299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":222,"ondisk_log_size":222,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710780,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1040384,"data_stored":515823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907011,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32868,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712284,"statfs":{"total":95995035648,"available":95961378816,"internally_reserved":0,"allocated":1732608,"data_stored":1171006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070537,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":519737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266768,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1044480,"data_stored":515907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462997,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1650688,"data_stored":1112657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626522,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":511677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574969,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":581186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738494,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1056768,"data_stored":512601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411444,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1126400,"data_stored":580874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182508,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":510853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986281,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1040384,"data_stored":515219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822754,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1658880,"data_stored":1107707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":70215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-07T02:29:50.719 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-07T02:29:50.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:50 smithi019 ceph-mon[29781]: pgmap v448: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:29:50.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:50 smithi019 ceph-mon[29781]: from='client.24943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-07T02:29:51.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:50 smithi064 ceph-mon[34699]: pgmap v448: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:29:51.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:50 smithi064 ceph-mon[34699]: from='client.24943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-07T02:29:51.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:50 smithi027 ceph-mon[34398]: pgmap v448: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:29:51.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:50 smithi027 ceph-mon[34398]: from='client.24943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-07T02:29:51.232 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:29:52.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:52 smithi019 ceph-mon[29781]: pgmap v449: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:29:53.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:52 smithi064 ceph-mon[34699]: pgmap v449: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:29:53.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:52 smithi027 ceph-mon[34398]: pgmap v449: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:29:53.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.560+0000 7ffb9f200700 1 -- 172.21.15.19:0/3429026187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb980764b0 msgr2=0x7ffb98114e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:53.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.560+0000 7ffb9f200700 1 --2- 172.21.15.19:0/3429026187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb980764b0 0x7ffb98114e30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ffb80009990 tx=0x7ffb80009ca0 comp rx=0 tx=0).stop 2024-05-07T02:29:53.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.560+0000 7ffb9f200700 1 -- 172.21.15.19:0/3429026187 shutdown_connections 2024-05-07T02:29:53.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.560+0000 7ffb9f200700 1 --2- 172.21.15.19:0/3429026187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb98115370 0x7ffb98117790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:53.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.560+0000 7ffb9f200700 1 --2- 172.21.15.19:0/3429026187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb98075b60 0x7ffb98075f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:53.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.560+0000 7ffb9f200700 1 --2- 172.21.15.19:0/3429026187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb980764b0 0x7ffb98114e30 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:53.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.560+0000 7ffb9f200700 1 -- 172.21.15.19:0/3429026187 >> 172.21.15.19:0/3429026187 conn(0x7ffb98100040 msgr2=0x7ffb98102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:53.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.560+0000 7ffb9f200700 1 -- 172.21.15.19:0/3429026187 shutdown_connections 2024-05-07T02:29:53.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.560+0000 7ffb9f200700 1 -- 172.21.15.19:0/3429026187 wait complete. 2024-05-07T02:29:53.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.561+0000 7ffb9f200700 1 Processor -- start 2024-05-07T02:29:53.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.561+0000 7ffb9f200700 1 -- start start 2024-05-07T02:29:53.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.562+0000 7ffb9f200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb98075b60 0x7ffb981a1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:53.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.562+0000 7ffb9f200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb980764b0 0x7ffb981a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:53.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.562+0000 7ffb9f200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb98115370 0x7ffb981a5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:53.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.562+0000 7ffb9f200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9811aac0 con 0x7ffb98115370 2024-05-07T02:29:53.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.562+0000 7ffb9f200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9811a940 con 0x7ffb98075b60 2024-05-07T02:29:53.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.562+0000 7ffb9f200700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9811ac40 con 0x7ffb980764b0 2024-05-07T02:29:53.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.562+0000 7ffb97e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb98075b60 0x7ffb981a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:53.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.562+0000 7ffb97e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb98075b60 0x7ffb981a1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:33174/0 (socket says 172.21.15.19:33174) 2024-05-07T02:29:53.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.562+0000 7ffb97e00700 1 -- 172.21.15.19:0/1618464169 learned_addr learned my addr 172.21.15.19:0/1618464169 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:29:53.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.562+0000 7ffb9d200700 1 --2- 172.21.15.19:0/1618464169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb98115370 0x7ffb981a5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:53.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.562+0000 7ffb8f400700 1 --2- 172.21.15.19:0/1618464169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb980764b0 0x7ffb981a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:53.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.563+0000 7ffb8f400700 1 -- 172.21.15.19:0/1618464169 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb98075b60 msgr2=0x7ffb981a1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:53.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.563+0000 7ffb8f400700 1 --2- 172.21.15.19:0/1618464169 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb98075b60 0x7ffb981a1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:53.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.563+0000 7ffb8f400700 1 -- 172.21.15.19:0/1618464169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb98115370 msgr2=0x7ffb981a5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:53.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.563+0000 7ffb8f400700 1 --2- 172.21.15.19:0/1618464169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb98115370 0x7ffb981a5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:53.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.563+0000 7ffb8f400700 1 -- 172.21.15.19:0/1618464169 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb981a6320 con 0x7ffb980764b0 2024-05-07T02:29:53.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.563+0000 7ffb8f400700 1 --2- 172.21.15.19:0/1618464169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb980764b0 0x7ffb981a19f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ffb80000c00 tx=0x7ffb800047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:53.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.564+0000 7ffb95600700 1 -- 172.21.15.19:0/1618464169 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb80026070 con 0x7ffb980764b0 2024-05-07T02:29:53.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.564+0000 7ffb95600700 1 -- 172.21.15.19:0/1618464169 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb80021410 con 0x7ffb980764b0 2024-05-07T02:29:53.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.564+0000 7ffb9f200700 1 -- 172.21.15.19:0/1618464169 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb981a65b0 con 0x7ffb980764b0 2024-05-07T02:29:53.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.564+0000 7ffb95600700 1 -- 172.21.15.19:0/1618464169 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb80017650 con 0x7ffb980764b0 2024-05-07T02:29:53.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.564+0000 7ffb9f200700 1 -- 172.21.15.19:0/1618464169 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb981adf30 con 0x7ffb980764b0 2024-05-07T02:29:53.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.565+0000 7ffb9f200700 1 -- 172.21.15.19:0/1618464169 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb980772a0 con 0x7ffb980764b0 2024-05-07T02:29:53.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.568+0000 7ffb95600700 1 -- 172.21.15.19:0/1618464169 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ffb80029070 con 0x7ffb980764b0 2024-05-07T02:29:53.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.570+0000 7ffb95600700 1 --2- 172.21.15.19:0/1618464169 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7ffb7c0b2900 0x7ffb7c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:53.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.570+0000 7ffb95600700 1 -- 172.21.15.19:0/1618464169 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7ffb800e38f0 con 0x7ffb980764b0 2024-05-07T02:29:53.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.570+0000 7ffb97e00700 1 --2- 172.21.15.19:0/1618464169 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7ffb7c0b2900 0x7ffb7c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:53.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.571+0000 7ffb97e00700 1 --2- 172.21.15.19:0/1618464169 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7ffb7c0b2900 0x7ffb7c0b4e10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ffb88009d20 tx=0x7ffb88009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:53.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.572+0000 7ffb95600700 1 -- 172.21.15.19:0/1618464169 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ffb80030020 con 0x7ffb980764b0 2024-05-07T02:29:53.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.809+0000 7ffb9f200700 1 -- 172.21.15.19:0/1618464169 --> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ffb981a27b0 con 0x7ffb7c0b2900 2024-05-07T02:29:53.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.839+0000 7ffb95600700 1 -- 172.21.15.19:0/1618464169 <== mgr.14152 v2:172.21.15.19:6800/2059564551 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340515 (secure 0 0 0) 0x7ffb981a27b0 con 0x7ffb7c0b2900 2024-05-07T02:29:53.841 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:29:53.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.848+0000 7ffb9f200700 1 -- 172.21.15.19:0/1618464169 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7ffb7c0b2900 msgr2=0x7ffb7c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:53.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.848+0000 7ffb9f200700 1 --2- 172.21.15.19:0/1618464169 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7ffb7c0b2900 0x7ffb7c0b4e10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ffb88009d20 tx=0x7ffb88009260 comp rx=0 tx=0).stop 2024-05-07T02:29:53.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.848+0000 7ffb9f200700 1 -- 172.21.15.19:0/1618464169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb980764b0 msgr2=0x7ffb981a19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:53.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.848+0000 7ffb9f200700 1 --2- 172.21.15.19:0/1618464169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb980764b0 0x7ffb981a19f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ffb80000c00 tx=0x7ffb800047b0 comp rx=0 tx=0).stop 2024-05-07T02:29:53.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.848+0000 7ffb9f200700 1 -- 172.21.15.19:0/1618464169 shutdown_connections 2024-05-07T02:29:53.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.848+0000 7ffb9f200700 1 --2- 172.21.15.19:0/1618464169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb98115370 0x7ffb981a5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:53.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.848+0000 7ffb9f200700 1 --2- 172.21.15.19:0/1618464169 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7ffb7c0b2900 0x7ffb7c0b4e10 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:53.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.848+0000 7ffb9f200700 1 --2- 172.21.15.19:0/1618464169 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb98075b60 0x7ffb981a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:53.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.848+0000 7ffb9f200700 1 --2- 172.21.15.19:0/1618464169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb980764b0 0x7ffb981a19f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:53.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.848+0000 7ffb9f200700 1 -- 172.21.15.19:0/1618464169 >> 172.21.15.19:0/1618464169 conn(0x7ffb98100040 msgr2=0x7ffb98100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:53.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.849+0000 7ffb9f200700 1 -- 172.21.15.19:0/1618464169 shutdown_connections 2024-05-07T02:29:53.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:53.849+0000 7ffb9f200700 1 -- 172.21.15.19:0/1618464169 wait complete. 2024-05-07T02:29:53.851 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-05-07T02:29:54.829 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":450,"stamp":"2024-05-07T02:29:53.544127+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666035,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1384,"num_read_kb":807,"num_write":383,"num_write_kb":2305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":816,"ondisk_log_size":816,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375884,"kb_used_data":14252,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565940,"statfs":{"total":1151940427776,"available":1151555522560,"internally_reserved":0,"allocated":14594048,"data_stored":8155247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":105,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006184"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.457443+0000","last_change":"2024-05-07T02:26:11.876828+0000","last_active":"2024-05-07T02:29:31.457443+0000","last_peered":"2024-05-07T02:29:31.457443+0000","last_clean":"2024-05-07T02:29:31.457443+0000","last_became_active":"2024-05-07T02:26:11.876319+0000","last_became_peered":"2024-05-07T02:26:11.876319+0000","last_unstale":"2024-05-07T02:29:31.457443+0000","last_undegraded":"2024-05-07T02:29:31.457443+0000","last_fullsized":"2024-05-07T02:29:31.457443+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:07:33.181373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.455517+0000","last_change":"2024-05-07T02:26:11.873388+0000","last_active":"2024-05-07T02:29:31.455517+0000","last_peered":"2024-05-07T02:29:31.455517+0000","last_clean":"2024-05-07T02:29:31.455517+0000","last_became_active":"2024-05-07T02:26:11.869447+0000","last_became_peered":"2024-05-07T02:26:11.869447+0000","last_unstale":"2024-05-07T02:29:31.455517+0000","last_undegraded":"2024-05-07T02:29:31.455517+0000","last_fullsized":"2024-05-07T02:29:31.455517+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:09:52.876547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.460894+0000","last_change":"2024-05-07T02:26:11.870797+0000","last_active":"2024-05-07T02:29:31.460894+0000","last_peered":"2024-05-07T02:29:31.460894+0000","last_clean":"2024-05-07T02:29:31.460894+0000","last_became_active":"2024-05-07T02:26:11.870411+0000","last_became_peered":"2024-05-07T02:26:11.870411+0000","last_unstale":"2024-05-07T02:29:31.460894+0000","last_undegraded":"2024-05-07T02:29:31.460894+0000","last_fullsized":"2024-05-07T02:29:31.460894+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:34:12.017508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.442852+0000","last_change":"2024-05-07T02:26:11.867327+0000","last_active":"2024-05-07T02:29:31.442852+0000","last_peered":"2024-05-07T02:29:31.442852+0000","last_clean":"2024-05-07T02:29:31.442852+0000","last_became_active":"2024-05-07T02:26:11.866912+0000","last_became_peered":"2024-05-07T02:26:11.866912+0000","last_unstale":"2024-05-07T02:29:31.442852+0000","last_undegraded":"2024-05-07T02:29:31.442852+0000","last_fullsized":"2024-05-07T02:29:31.442852+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:36:36.616706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.435537+0000","last_change":"2024-05-07T02:26:11.878614+0000","last_active":"2024-05-07T02:29:31.435537+0000","last_peered":"2024-05-07T02:29:31.435537+0000","last_clean":"2024-05-07T02:29:31.435537+0000","last_became_active":"2024-05-07T02:26:11.878069+0000","last_became_peered":"2024-05-07T02:26:11.878069+0000","last_unstale":"2024-05-07T02:29:31.435537+0000","last_undegraded":"2024-05-07T02:29:31.435537+0000","last_fullsized":"2024-05-07T02:29:31.435537+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:40:15.991433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.445763+0000","last_change":"2024-05-07T02:26:11.877755+0000","last_active":"2024-05-07T02:29:31.445763+0000","last_peered":"2024-05-07T02:29:31.445763+0000","last_clean":"2024-05-07T02:29:31.445763+0000","last_became_active":"2024-05-07T02:26:11.877373+0000","last_became_peered":"2024-05-07T02:26:11.877373+0000","last_unstale":"2024-05-07T02:29:31.445763+0000","last_undegraded":"2024-05-07T02:29:31.445763+0000","last_fullsized":"2024-05-07T02:29:31.445763+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:49:55.984807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.439470+0000","last_change":"2024-05-07T02:26:11.882194+0000","last_active":"2024-05-07T02:29:31.439470+0000","last_peered":"2024-05-07T02:29:31.439470+0000","last_clean":"2024-05-07T02:29:31.439470+0000","last_became_active":"2024-05-07T02:26:11.881873+0000","last_became_peered":"2024-05-07T02:26:11.881873+0000","last_unstale":"2024-05-07T02:29:31.439470+0000","last_undegraded":"2024-05-07T02:29:31.439470+0000","last_fullsized":"2024-05-07T02:29:31.439470+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:47:31.000357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.463571+0000","last_change":"2024-05-07T02:26:11.857366+0000","last_active":"2024-05-07T02:29:31.463571+0000","last_peered":"2024-05-07T02:29:31.463571+0000","last_clean":"2024-05-07T02:29:31.463571+0000","last_became_active":"2024-05-07T02:26:11.854054+0000","last_became_peered":"2024-05-07T02:26:11.854054+0000","last_unstale":"2024-05-07T02:29:31.463571+0000","last_undegraded":"2024-05-07T02:29:31.463571+0000","last_fullsized":"2024-05-07T02:29:31.463571+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:07:04.834022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.455230+0000","last_change":"2024-05-07T02:26:11.867105+0000","last_active":"2024-05-07T02:29:31.455230+0000","last_peered":"2024-05-07T02:29:31.455230+0000","last_clean":"2024-05-07T02:29:31.455230+0000","last_became_active":"2024-05-07T02:26:11.865986+0000","last_became_peered":"2024-05-07T02:26:11.865986+0000","last_unstale":"2024-05-07T02:29:31.455230+0000","last_undegraded":"2024-05-07T02:29:31.455230+0000","last_fullsized":"2024-05-07T02:29:31.455230+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:13:01.295354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.435492+0000","last_change":"2024-05-07T02:26:11.868039+0000","last_active":"2024-05-07T02:29:31.435492+0000","last_peered":"2024-05-07T02:29:31.435492+0000","last_clean":"2024-05-07T02:29:31.435492+0000","last_became_active":"2024-05-07T02:26:11.867588+0000","last_became_peered":"2024-05-07T02:26:11.867588+0000","last_unstale":"2024-05-07T02:29:31.435492+0000","last_undegraded":"2024-05-07T02:29:31.435492+0000","last_fullsized":"2024-05-07T02:29:31.435492+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:41:45.149620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.452741+0000","last_change":"2024-05-07T02:26:11.859579+0000","last_active":"2024-05-07T02:29:31.452741+0000","last_peered":"2024-05-07T02:29:31.452741+0000","last_clean":"2024-05-07T02:29:31.452741+0000","last_became_active":"2024-05-07T02:26:11.859154+0000","last_became_peered":"2024-05-07T02:26:11.859154+0000","last_unstale":"2024-05-07T02:29:31.452741+0000","last_undegraded":"2024-05-07T02:29:31.452741+0000","last_fullsized":"2024-05-07T02:29:31.452741+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:26:40.616956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.443671+0000","last_change":"2024-05-07T02:26:11.858220+0000","last_active":"2024-05-07T02:29:31.443671+0000","last_peered":"2024-05-07T02:29:31.443671+0000","last_clean":"2024-05-07T02:29:31.443671+0000","last_became_active":"2024-05-07T02:26:11.857826+0000","last_became_peered":"2024-05-07T02:26:11.857826+0000","last_unstale":"2024-05-07T02:29:31.443671+0000","last_undegraded":"2024-05-07T02:29:31.443671+0000","last_fullsized":"2024-05-07T02:29:31.443671+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:21:36.031057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.443326+0000","last_change":"2024-05-07T02:26:11.880348+0000","last_active":"2024-05-07T02:29:31.443326+0000","last_peered":"2024-05-07T02:29:31.443326+0000","last_clean":"2024-05-07T02:29:31.443326+0000","last_became_active":"2024-05-07T02:26:11.879633+0000","last_became_peered":"2024-05-07T02:26:11.879633+0000","last_unstale":"2024-05-07T02:29:31.443326+0000","last_undegraded":"2024-05-07T02:29:31.443326+0000","last_fullsized":"2024-05-07T02:29:31.443326+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:59:34.567973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.456171+0000","last_change":"2024-05-07T02:26:11.865687+0000","last_active":"2024-05-07T02:29:31.456171+0000","last_peered":"2024-05-07T02:29:31.456171+0000","last_clean":"2024-05-07T02:29:31.456171+0000","last_became_active":"2024-05-07T02:26:11.865330+0000","last_became_peered":"2024-05-07T02:26:11.865330+0000","last_unstale":"2024-05-07T02:29:31.456171+0000","last_undegraded":"2024-05-07T02:29:31.456171+0000","last_fullsized":"2024-05-07T02:29:31.456171+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:36:10.610314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.458010+0000","last_change":"2024-05-07T02:26:11.870627+0000","last_active":"2024-05-07T02:29:31.458010+0000","last_peered":"2024-05-07T02:29:31.458010+0000","last_clean":"2024-05-07T02:29:31.458010+0000","last_became_active":"2024-05-07T02:26:11.870229+0000","last_became_peered":"2024-05-07T02:26:11.870229+0000","last_unstale":"2024-05-07T02:29:31.458010+0000","last_undegraded":"2024-05-07T02:29:31.458010+0000","last_fullsized":"2024-05-07T02:29:31.458010+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:06:46.008651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.460290+0000","last_change":"2024-05-07T02:26:11.868698+0000","last_active":"2024-05-07T02:29:31.460290+0000","last_peered":"2024-05-07T02:29:31.460290+0000","last_clean":"2024-05-07T02:29:31.460290+0000","last_became_active":"2024-05-07T02:26:11.868294+0000","last_became_peered":"2024-05-07T02:26:11.868294+0000","last_unstale":"2024-05-07T02:29:31.460290+0000","last_undegraded":"2024-05-07T02:29:31.460290+0000","last_fullsized":"2024-05-07T02:29:31.460290+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:16:22.856200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.457570+0000","last_change":"2024-05-07T02:26:11.877116+0000","last_active":"2024-05-07T02:29:31.457570+0000","last_peered":"2024-05-07T02:29:31.457570+0000","last_clean":"2024-05-07T02:29:31.457570+0000","last_became_active":"2024-05-07T02:26:11.876305+0000","last_became_peered":"2024-05-07T02:26:11.876305+0000","last_unstale":"2024-05-07T02:29:31.457570+0000","last_undegraded":"2024-05-07T02:29:31.457570+0000","last_fullsized":"2024-05-07T02:29:31.457570+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:25:46.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.452721+0000","last_change":"2024-05-07T02:26:11.877488+0000","last_active":"2024-05-07T02:29:31.452721+0000","last_peered":"2024-05-07T02:29:31.452721+0000","last_clean":"2024-05-07T02:29:31.452721+0000","last_became_active":"2024-05-07T02:26:11.874500+0000","last_became_peered":"2024-05-07T02:26:11.874500+0000","last_unstale":"2024-05-07T02:29:31.452721+0000","last_undegraded":"2024-05-07T02:29:31.452721+0000","last_fullsized":"2024-05-07T02:29:31.452721+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:53:50.046342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.439857+0000","last_change":"2024-05-07T02:26:11.876500+0000","last_active":"2024-05-07T02:29:31.439857+0000","last_peered":"2024-05-07T02:29:31.439857+0000","last_clean":"2024-05-07T02:29:31.439857+0000","last_became_active":"2024-05-07T02:26:11.876084+0000","last_became_peered":"2024-05-07T02:26:11.876084+0000","last_unstale":"2024-05-07T02:29:31.439857+0000","last_undegraded":"2024-05-07T02:29:31.439857+0000","last_fullsized":"2024-05-07T02:29:31.439857+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:05:05.681488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.868146+0000","last_change":"2024-05-07T02:26:25.579209+0000","last_active":"2024-05-07T02:28:33.868146+0000","last_peered":"2024-05-07T02:28:33.868146+0000","last_clean":"2024-05-07T02:28:33.868146+0000","last_became_active":"2024-05-07T02:26:25.578665+0000","last_became_peered":"2024-05-07T02:26:25.578665+0000","last_unstale":"2024-05-07T02:28:33.868146+0000","last_undegraded":"2024-05-07T02:28:33.868146+0000","last_fullsized":"2024-05-07T02:28:33.868146+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:06:29.784016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'14","reported_seq":184,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:47.003267+0000","last_change":"2024-05-07T02:26:07.792993+0000","last_active":"2024-05-07T02:29:47.003267+0000","last_peered":"2024-05-07T02:29:47.003267+0000","last_clean":"2024-05-07T02:29:47.003267+0000","last_became_active":"2024-05-07T02:26:07.792402+0000","last_became_peered":"2024-05-07T02:26:07.792402+0000","last_unstale":"2024-05-07T02:29:47.003267+0000","last_undegraded":"2024-05-07T02:29:47.003267+0000","last_fullsized":"2024-05-07T02:29:47.003267+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:49:02.288255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":125,"num_read_kb":125,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.442593+0000","last_change":"2024-05-07T02:28:33.869804+0000","last_active":"2024-05-07T02:29:31.442593+0000","last_peered":"2024-05-07T02:29:31.442593+0000","last_clean":"2024-05-07T02:29:31.442593+0000","last_became_active":"2024-05-07T02:28:33.869267+0000","last_became_peered":"2024-05-07T02:28:33.869267+0000","last_unstale":"2024-05-07T02:29:31.442593+0000","last_undegraded":"2024-05-07T02:29:31.442593+0000","last_fullsized":"2024-05-07T02:29:31.442593+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:10:57.384546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.611414+0000","last_change":"2024-05-07T02:26:25.579835+0000","last_active":"2024-05-07T02:28:33.611414+0000","last_peered":"2024-05-07T02:28:33.611414+0000","last_clean":"2024-05-07T02:28:33.611414+0000","last_became_active":"2024-05-07T02:26:25.579105+0000","last_became_peered":"2024-05-07T02:26:25.579105+0000","last_unstale":"2024-05-07T02:28:33.611414+0000","last_undegraded":"2024-05-07T02:28:33.611414+0000","last_fullsized":"2024-05-07T02:28:33.611414+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:32:37.745967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.621858+0000","last_change":"2024-05-07T02:26:08.001879+0000","last_active":"2024-05-07T02:28:33.621858+0000","last_peered":"2024-05-07T02:28:33.621858+0000","last_clean":"2024-05-07T02:28:33.621858+0000","last_became_active":"2024-05-07T02:26:08.000244+0000","last_became_peered":"2024-05-07T02:26:08.000244+0000","last_unstale":"2024-05-07T02:28:33.621858+0000","last_undegraded":"2024-05-07T02:28:33.621858+0000","last_fullsized":"2024-05-07T02:28:33.621858+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:56:42.591923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.460379+0000","last_change":"2024-05-07T02:26:11.882351+0000","last_active":"2024-05-07T02:29:31.460379+0000","last_peered":"2024-05-07T02:29:31.460379+0000","last_clean":"2024-05-07T02:29:31.460379+0000","last_became_active":"2024-05-07T02:26:11.881922+0000","last_became_peered":"2024-05-07T02:26:11.881922+0000","last_unstale":"2024-05-07T02:29:31.460379+0000","last_undegraded":"2024-05-07T02:29:31.460379+0000","last_fullsized":"2024-05-07T02:29:31.460379+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:58:31.993859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.611612+0000","last_change":"2024-05-07T02:26:25.580419+0000","last_active":"2024-05-07T02:28:33.611612+0000","last_peered":"2024-05-07T02:28:33.611612+0000","last_clean":"2024-05-07T02:28:33.611612+0000","last_became_active":"2024-05-07T02:26:25.579411+0000","last_became_peered":"2024-05-07T02:26:25.579411+0000","last_unstale":"2024-05-07T02:28:33.611612+0000","last_undegraded":"2024-05-07T02:28:33.611612+0000","last_fullsized":"2024-05-07T02:28:33.611612+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:43:03.552210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:34.646169+0000","last_change":"2024-05-07T02:27:32.588255+0000","last_active":"2024-05-07T02:28:34.646169+0000","last_peered":"2024-05-07T02:28:34.646169+0000","last_clean":"2024-05-07T02:28:34.646169+0000","last_became_active":"2024-05-07T02:27:32.584405+0000","last_became_peered":"2024-05-07T02:27:32.584405+0000","last_unstale":"2024-05-07T02:28:34.646169+0000","last_undegraded":"2024-05-07T02:28:34.646169+0000","last_fullsized":"2024-05-07T02:28:34.646169+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:07:22.980041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.459629+0000","last_change":"2024-05-07T02:26:11.866949+0000","last_active":"2024-05-07T02:29:31.459629+0000","last_peered":"2024-05-07T02:29:31.459629+0000","last_clean":"2024-05-07T02:29:31.459629+0000","last_became_active":"2024-05-07T02:26:11.866453+0000","last_became_peered":"2024-05-07T02:26:11.866453+0000","last_unstale":"2024-05-07T02:29:31.459629+0000","last_undegraded":"2024-05-07T02:29:31.459629+0000","last_fullsized":"2024-05-07T02:29:31.459629+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T14:22:47.291470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.611410+0000","last_change":"2024-05-07T02:26:25.570555+0000","last_active":"2024-05-07T02:28:33.611410+0000","last_peered":"2024-05-07T02:28:33.611410+0000","last_clean":"2024-05-07T02:28:33.611410+0000","last_became_active":"2024-05-07T02:26:25.570044+0000","last_became_peered":"2024-05-07T02:26:25.570044+0000","last_unstale":"2024-05-07T02:28:33.611410+0000","last_undegraded":"2024-05-07T02:28:33.611410+0000","last_fullsized":"2024-05-07T02:28:33.611410+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:48:15.701335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:34.646091+0000","last_change":"2024-05-07T02:26:07.785938+0000","last_active":"2024-05-07T02:28:34.646091+0000","last_peered":"2024-05-07T02:28:34.646091+0000","last_clean":"2024-05-07T02:28:34.646091+0000","last_became_active":"2024-05-07T02:26:07.785573+0000","last_became_peered":"2024-05-07T02:26:07.785573+0000","last_unstale":"2024-05-07T02:28:34.646091+0000","last_undegraded":"2024-05-07T02:28:34.646091+0000","last_fullsized":"2024-05-07T02:28:34.646091+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:33:24.042227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.450519+0000","last_change":"2024-05-07T02:26:11.858732+0000","last_active":"2024-05-07T02:29:31.450519+0000","last_peered":"2024-05-07T02:29:31.450519+0000","last_clean":"2024-05-07T02:29:31.450519+0000","last_became_active":"2024-05-07T02:26:11.858270+0000","last_became_peered":"2024-05-07T02:26:11.858270+0000","last_unstale":"2024-05-07T02:29:31.450519+0000","last_undegraded":"2024-05-07T02:29:31.450519+0000","last_fullsized":"2024-05-07T02:29:31.450519+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:03:53.662296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.868915+0000","last_change":"2024-05-07T02:26:25.559025+0000","last_active":"2024-05-07T02:28:33.868915+0000","last_peered":"2024-05-07T02:28:33.868915+0000","last_clean":"2024-05-07T02:28:33.868915+0000","last_became_active":"2024-05-07T02:26:25.558502+0000","last_became_peered":"2024-05-07T02:26:25.558502+0000","last_unstale":"2024-05-07T02:28:33.868915+0000","last_undegraded":"2024-05-07T02:28:33.868915+0000","last_fullsized":"2024-05-07T02:28:33.868915+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:26:59.846341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":177,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:47.962734+0000","last_change":"2024-05-07T02:26:07.783386+0000","last_active":"2024-05-07T02:29:47.962734+0000","last_peered":"2024-05-07T02:29:47.962734+0000","last_clean":"2024-05-07T02:29:47.962734+0000","last_became_active":"2024-05-07T02:26:07.782823+0000","last_became_peered":"2024-05-07T02:26:07.782823+0000","last_unstale":"2024-05-07T02:29:47.962734+0000","last_undegraded":"2024-05-07T02:29:47.962734+0000","last_fullsized":"2024-05-07T02:29:47.962734+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:40:29.350809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":127,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.433919+0000","last_change":"2024-05-07T02:26:11.877426+0000","last_active":"2024-05-07T02:29:31.433919+0000","last_peered":"2024-05-07T02:29:31.433919+0000","last_clean":"2024-05-07T02:29:31.433919+0000","last_became_active":"2024-05-07T02:26:11.876976+0000","last_became_peered":"2024-05-07T02:26:11.876976+0000","last_unstale":"2024-05-07T02:29:31.433919+0000","last_undegraded":"2024-05-07T02:29:31.433919+0000","last_fullsized":"2024-05-07T02:29:31.433919+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:18:56.077064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.868625+0000","last_change":"2024-05-07T02:27:32.625394+0000","last_active":"2024-05-07T02:28:33.868625+0000","last_peered":"2024-05-07T02:28:33.868625+0000","last_clean":"2024-05-07T02:28:33.868625+0000","last_became_active":"2024-05-07T02:27:32.624826+0000","last_became_peered":"2024-05-07T02:27:32.624826+0000","last_unstale":"2024-05-07T02:28:33.868625+0000","last_undegraded":"2024-05-07T02:28:33.868625+0000","last_fullsized":"2024-05-07T02:28:33.868625+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:16:49.694003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.840846+0000","last_change":"2024-05-07T02:26:07.786388+0000","last_active":"2024-05-07T02:28:33.840846+0000","last_peered":"2024-05-07T02:28:33.840846+0000","last_clean":"2024-05-07T02:28:33.840846+0000","last_became_active":"2024-05-07T02:26:07.784130+0000","last_became_peered":"2024-05-07T02:26:07.784130+0000","last_unstale":"2024-05-07T02:28:33.840846+0000","last_undegraded":"2024-05-07T02:28:33.840846+0000","last_fullsized":"2024-05-07T02:28:33.840846+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:44:55.229815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.453124+0000","last_change":"2024-05-07T02:28:33.886551+0000","last_active":"2024-05-07T02:29:31.453124+0000","last_peered":"2024-05-07T02:29:31.453124+0000","last_clean":"2024-05-07T02:29:31.453124+0000","last_became_active":"2024-05-07T02:28:33.886249+0000","last_became_peered":"2024-05-07T02:28:33.886249+0000","last_unstale":"2024-05-07T02:29:31.453124+0000","last_undegraded":"2024-05-07T02:29:31.453124+0000","last_fullsized":"2024-05-07T02:29:31.453124+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:56:19.851347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:34.646463+0000","last_change":"2024-05-07T02:26:25.584396+0000","last_active":"2024-05-07T02:28:34.646463+0000","last_peered":"2024-05-07T02:28:34.646463+0000","last_clean":"2024-05-07T02:28:34.646463+0000","last_became_active":"2024-05-07T02:26:25.583917+0000","last_became_peered":"2024-05-07T02:26:25.583917+0000","last_unstale":"2024-05-07T02:28:34.646463+0000","last_undegraded":"2024-05-07T02:28:34.646463+0000","last_fullsized":"2024-05-07T02:28:34.646463+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:56:34.495133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.884083+0000","last_change":"2024-05-07T02:26:07.795300+0000","last_active":"2024-05-07T02:28:33.884083+0000","last_peered":"2024-05-07T02:28:33.884083+0000","last_clean":"2024-05-07T02:28:33.884083+0000","last_became_active":"2024-05-07T02:26:07.794633+0000","last_became_peered":"2024-05-07T02:26:07.794633+0000","last_unstale":"2024-05-07T02:28:33.884083+0000","last_undegraded":"2024-05-07T02:28:33.884083+0000","last_fullsized":"2024-05-07T02:28:33.884083+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:36:44.252118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.463072+0000","last_change":"2024-05-07T02:26:11.877717+0000","last_active":"2024-05-07T02:29:31.463072+0000","last_peered":"2024-05-07T02:29:31.463072+0000","last_clean":"2024-05-07T02:29:31.463072+0000","last_became_active":"2024-05-07T02:26:11.876644+0000","last_became_peered":"2024-05-07T02:26:11.876644+0000","last_unstale":"2024-05-07T02:29:31.463072+0000","last_undegraded":"2024-05-07T02:29:31.463072+0000","last_fullsized":"2024-05-07T02:29:31.463072+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T05:20:36.176776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.619993+0000","last_change":"2024-05-07T02:26:25.572120+0000","last_active":"2024-05-07T02:28:33.619993+0000","last_peered":"2024-05-07T02:28:33.619993+0000","last_clean":"2024-05-07T02:28:33.619993+0000","last_became_active":"2024-05-07T02:26:25.571637+0000","last_became_peered":"2024-05-07T02:26:25.571637+0000","last_unstale":"2024-05-07T02:28:33.619993+0000","last_undegraded":"2024-05-07T02:28:33.619993+0000","last_fullsized":"2024-05-07T02:28:33.619993+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:03:54.574061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.613500+0000","last_change":"2024-05-07T02:26:07.784160+0000","last_active":"2024-05-07T02:28:33.613500+0000","last_peered":"2024-05-07T02:28:33.613500+0000","last_clean":"2024-05-07T02:28:33.613500+0000","last_became_active":"2024-05-07T02:26:07.783646+0000","last_became_peered":"2024-05-07T02:26:07.783646+0000","last_unstale":"2024-05-07T02:28:33.613500+0000","last_undegraded":"2024-05-07T02:28:33.613500+0000","last_fullsized":"2024-05-07T02:28:33.613500+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T14:25:37.906587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.464206+0000","last_change":"2024-05-07T02:26:11.874886+0000","last_active":"2024-05-07T02:29:31.464206+0000","last_peered":"2024-05-07T02:29:31.464206+0000","last_clean":"2024-05-07T02:29:31.464206+0000","last_became_active":"2024-05-07T02:26:11.874465+0000","last_became_peered":"2024-05-07T02:26:11.874465+0000","last_unstale":"2024-05-07T02:29:31.464206+0000","last_undegraded":"2024-05-07T02:29:31.464206+0000","last_fullsized":"2024-05-07T02:29:31.464206+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:57:19.530747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.617066+0000","last_change":"2024-05-07T02:26:25.558087+0000","last_active":"2024-05-07T02:28:33.617066+0000","last_peered":"2024-05-07T02:28:33.617066+0000","last_clean":"2024-05-07T02:28:33.617066+0000","last_became_active":"2024-05-07T02:26:25.557624+0000","last_became_peered":"2024-05-07T02:26:25.557624+0000","last_unstale":"2024-05-07T02:28:33.617066+0000","last_undegraded":"2024-05-07T02:28:33.617066+0000","last_fullsized":"2024-05-07T02:28:33.617066+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:03:20.104547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":303,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:50.967373+0000","last_change":"2024-05-07T02:26:07.788244+0000","last_active":"2024-05-07T02:29:50.967373+0000","last_peered":"2024-05-07T02:29:50.967373+0000","last_clean":"2024-05-07T02:29:50.967373+0000","last_became_active":"2024-05-07T02:26:07.787789+0000","last_became_peered":"2024-05-07T02:26:07.787789+0000","last_unstale":"2024-05-07T02:29:50.967373+0000","last_undegraded":"2024-05-07T02:29:50.967373+0000","last_fullsized":"2024-05-07T02:29:50.967373+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:12:22.943444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":254,"num_read_kb":254,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.458376+0000","last_change":"2024-05-07T02:26:11.879821+0000","last_active":"2024-05-07T02:29:31.458376+0000","last_peered":"2024-05-07T02:29:31.458376+0000","last_clean":"2024-05-07T02:29:31.458376+0000","last_became_active":"2024-05-07T02:26:11.879372+0000","last_became_peered":"2024-05-07T02:26:11.879372+0000","last_unstale":"2024-05-07T02:29:31.458376+0000","last_undegraded":"2024-05-07T02:29:31.458376+0000","last_fullsized":"2024-05-07T02:29:31.458376+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:01:03.754890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.619139+0000","last_change":"2024-05-07T02:26:25.576663+0000","last_active":"2024-05-07T02:28:33.619139+0000","last_peered":"2024-05-07T02:28:33.619139+0000","last_clean":"2024-05-07T02:28:33.619139+0000","last_became_active":"2024-05-07T02:26:25.575729+0000","last_became_peered":"2024-05-07T02:26:25.575729+0000","last_unstale":"2024-05-07T02:28:33.619139+0000","last_undegraded":"2024-05-07T02:28:33.619139+0000","last_fullsized":"2024-05-07T02:28:33.619139+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:03:48.061385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.693451+0000","last_change":"2024-05-07T02:27:32.625336+0000","last_active":"2024-05-07T02:28:33.693451+0000","last_peered":"2024-05-07T02:28:33.693451+0000","last_clean":"2024-05-07T02:28:33.693451+0000","last_became_active":"2024-05-07T02:27:32.624805+0000","last_became_peered":"2024-05-07T02:27:32.624805+0000","last_unstale":"2024-05-07T02:28:33.693451+0000","last_undegraded":"2024-05-07T02:28:33.693451+0000","last_fullsized":"2024-05-07T02:28:33.693451+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:10:04.928297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.458002+0000","last_change":"2024-05-07T02:27:32.626374+0000","last_active":"2024-05-07T02:29:31.458002+0000","last_peered":"2024-05-07T02:29:31.458002+0000","last_clean":"2024-05-07T02:29:31.458002+0000","last_became_active":"2024-05-07T02:27:32.625489+0000","last_became_peered":"2024-05-07T02:27:32.625489+0000","last_unstale":"2024-05-07T02:29:31.458002+0000","last_undegraded":"2024-05-07T02:29:31.458002+0000","last_fullsized":"2024-05-07T02:29:31.458002+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T05:56:16.626041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.611819+0000","last_change":"2024-05-07T02:26:25.580811+0000","last_active":"2024-05-07T02:28:33.611819+0000","last_peered":"2024-05-07T02:28:33.611819+0000","last_clean":"2024-05-07T02:28:33.611819+0000","last_became_active":"2024-05-07T02:26:25.579344+0000","last_became_peered":"2024-05-07T02:26:25.579344+0000","last_unstale":"2024-05-07T02:28:33.611819+0000","last_undegraded":"2024-05-07T02:28:33.611819+0000","last_fullsized":"2024-05-07T02:28:33.611819+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:25:36.186899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.614114+0000","last_change":"2024-05-07T02:26:08.000010+0000","last_active":"2024-05-07T02:28:33.614114+0000","last_peered":"2024-05-07T02:28:33.614114+0000","last_clean":"2024-05-07T02:28:33.614114+0000","last_became_active":"2024-05-07T02:26:07.999576+0000","last_became_peered":"2024-05-07T02:26:07.999576+0000","last_unstale":"2024-05-07T02:28:33.614114+0000","last_undegraded":"2024-05-07T02:28:33.614114+0000","last_fullsized":"2024-05-07T02:28:33.614114+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:23:38.644205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.458253+0000","last_change":"2024-05-07T02:26:11.867098+0000","last_active":"2024-05-07T02:29:31.458253+0000","last_peered":"2024-05-07T02:29:31.458253+0000","last_clean":"2024-05-07T02:29:31.458253+0000","last_became_active":"2024-05-07T02:26:11.866036+0000","last_became_peered":"2024-05-07T02:26:11.866036+0000","last_unstale":"2024-05-07T02:29:31.458253+0000","last_undegraded":"2024-05-07T02:29:31.458253+0000","last_fullsized":"2024-05-07T02:29:31.458253+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:47:22.412323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.617223+0000","last_change":"2024-05-07T02:27:32.628243+0000","last_active":"2024-05-07T02:28:33.617223+0000","last_peered":"2024-05-07T02:28:33.617223+0000","last_clean":"2024-05-07T02:28:33.617223+0000","last_became_active":"2024-05-07T02:27:32.627048+0000","last_became_peered":"2024-05-07T02:27:32.627048+0000","last_unstale":"2024-05-07T02:28:33.617223+0000","last_undegraded":"2024-05-07T02:28:33.617223+0000","last_fullsized":"2024-05-07T02:28:33.617223+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:33:49.802873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,10],"acting":[9,8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'40","reported_seq":350,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:50.971490+0000","last_change":"2024-05-07T02:26:07.786841+0000","last_active":"2024-05-07T02:29:50.971490+0000","last_peered":"2024-05-07T02:29:50.971490+0000","last_clean":"2024-05-07T02:29:50.971490+0000","last_became_active":"2024-05-07T02:26:07.783836+0000","last_became_peered":"2024-05-07T02:26:07.783836+0000","last_unstale":"2024-05-07T02:29:50.971490+0000","last_undegraded":"2024-05-07T02:29:50.971490+0000","last_fullsized":"2024-05-07T02:29:50.971490+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:28:53.390009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68479,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":265,"num_read_kb":201,"num_write":42,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.426089+0000","last_change":"2024-05-07T02:26:11.857292+0000","last_active":"2024-05-07T02:29:31.426089+0000","last_peered":"2024-05-07T02:29:31.426089+0000","last_clean":"2024-05-07T02:29:31.426089+0000","last_became_active":"2024-05-07T02:26:11.856899+0000","last_became_peered":"2024-05-07T02:26:11.856899+0000","last_unstale":"2024-05-07T02:29:31.426089+0000","last_undegraded":"2024-05-07T02:29:31.426089+0000","last_fullsized":"2024-05-07T02:29:31.426089+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:45:42.279848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.614502+0000","last_change":"2024-05-07T02:26:25.570714+0000","last_active":"2024-05-07T02:28:33.614502+0000","last_peered":"2024-05-07T02:28:33.614502+0000","last_clean":"2024-05-07T02:28:33.614502+0000","last_became_active":"2024-05-07T02:26:25.570163+0000","last_became_peered":"2024-05-07T02:26:25.570163+0000","last_unstale":"2024-05-07T02:28:33.614502+0000","last_undegraded":"2024-05-07T02:28:33.614502+0000","last_fullsized":"2024-05-07T02:28:33.614502+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:55:51.903968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":56,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.868547+0000","last_change":"2024-05-07T02:26:07.785904+0000","last_active":"2024-05-07T02:28:33.868547+0000","last_peered":"2024-05-07T02:28:33.868547+0000","last_clean":"2024-05-07T02:28:33.868547+0000","last_became_active":"2024-05-07T02:26:07.785347+0000","last_became_peered":"2024-05-07T02:26:07.785347+0000","last_unstale":"2024-05-07T02:28:33.868547+0000","last_undegraded":"2024-05-07T02:28:33.868547+0000","last_fullsized":"2024-05-07T02:28:33.868547+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:42:05.604266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.458104+0000","last_change":"2024-05-07T02:26:11.878511+0000","last_active":"2024-05-07T02:29:31.458104+0000","last_peered":"2024-05-07T02:29:31.458104+0000","last_clean":"2024-05-07T02:29:31.458104+0000","last_became_active":"2024-05-07T02:26:11.878166+0000","last_became_peered":"2024-05-07T02:26:11.878166+0000","last_unstale":"2024-05-07T02:29:31.458104+0000","last_undegraded":"2024-05-07T02:29:31.458104+0000","last_fullsized":"2024-05-07T02:29:31.458104+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T08:14:40.436244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:34.645774+0000","last_change":"2024-05-07T02:26:25.579387+0000","last_active":"2024-05-07T02:28:34.645774+0000","last_peered":"2024-05-07T02:28:34.645774+0000","last_clean":"2024-05-07T02:28:34.645774+0000","last_became_active":"2024-05-07T02:26:25.578757+0000","last_became_peered":"2024-05-07T02:26:25.578757+0000","last_unstale":"2024-05-07T02:28:34.645774+0000","last_undegraded":"2024-05-07T02:28:34.645774+0000","last_fullsized":"2024-05-07T02:28:34.645774+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:19:19.650941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.613626+0000","last_change":"2024-05-07T02:26:07.785154+0000","last_active":"2024-05-07T02:28:33.613626+0000","last_peered":"2024-05-07T02:28:33.613626+0000","last_clean":"2024-05-07T02:28:33.613626+0000","last_became_active":"2024-05-07T02:26:07.784589+0000","last_became_peered":"2024-05-07T02:26:07.784589+0000","last_unstale":"2024-05-07T02:28:33.613626+0000","last_undegraded":"2024-05-07T02:28:33.613626+0000","last_fullsized":"2024-05-07T02:28:33.613626+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:41:41.851182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.447695+0000","last_change":"2024-05-07T02:26:11.878162+0000","last_active":"2024-05-07T02:29:31.447695+0000","last_peered":"2024-05-07T02:29:31.447695+0000","last_clean":"2024-05-07T02:29:31.447695+0000","last_became_active":"2024-05-07T02:26:11.877770+0000","last_became_peered":"2024-05-07T02:26:11.877770+0000","last_unstale":"2024-05-07T02:29:31.447695+0000","last_undegraded":"2024-05-07T02:29:31.447695+0000","last_fullsized":"2024-05-07T02:29:31.447695+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:19:08.209398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.619945+0000","last_change":"2024-05-07T02:26:25.579465+0000","last_active":"2024-05-07T02:28:33.619945+0000","last_peered":"2024-05-07T02:28:33.619945+0000","last_clean":"2024-05-07T02:28:33.619945+0000","last_became_active":"2024-05-07T02:26:25.579003+0000","last_became_peered":"2024-05-07T02:26:25.579003+0000","last_unstale":"2024-05-07T02:28:33.619945+0000","last_undegraded":"2024-05-07T02:28:33.619945+0000","last_fullsized":"2024-05-07T02:28:33.619945+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:13:24.356366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":176,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:47.074802+0000","last_change":"2024-05-07T02:26:07.780827+0000","last_active":"2024-05-07T02:29:47.074802+0000","last_peered":"2024-05-07T02:29:47.074802+0000","last_clean":"2024-05-07T02:29:47.074802+0000","last_became_active":"2024-05-07T02:26:07.780353+0000","last_became_peered":"2024-05-07T02:26:07.780353+0000","last_unstale":"2024-05-07T02:29:47.074802+0000","last_undegraded":"2024-05-07T02:29:47.074802+0000","last_fullsized":"2024-05-07T02:29:47.074802+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:52:40.554135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":127,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.446613+0000","last_change":"2024-05-07T02:26:11.852516+0000","last_active":"2024-05-07T02:29:31.446613+0000","last_peered":"2024-05-07T02:29:31.446613+0000","last_clean":"2024-05-07T02:29:31.446613+0000","last_became_active":"2024-05-07T02:26:11.852112+0000","last_became_peered":"2024-05-07T02:26:11.852112+0000","last_unstale":"2024-05-07T02:29:31.446613+0000","last_undegraded":"2024-05-07T02:29:31.446613+0000","last_fullsized":"2024-05-07T02:29:31.446613+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:58:46.989250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.886385+0000","last_change":"2024-05-07T02:26:25.586726+0000","last_active":"2024-05-07T02:28:33.886385+0000","last_peered":"2024-05-07T02:28:33.886385+0000","last_clean":"2024-05-07T02:28:33.886385+0000","last_became_active":"2024-05-07T02:26:25.586267+0000","last_became_peered":"2024-05-07T02:26:25.586267+0000","last_unstale":"2024-05-07T02:28:33.886385+0000","last_undegraded":"2024-05-07T02:28:33.886385+0000","last_fullsized":"2024-05-07T02:28:33.886385+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:50:00.864463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:34.645472+0000","last_change":"2024-05-07T02:27:32.599511+0000","last_active":"2024-05-07T02:28:34.645472+0000","last_peered":"2024-05-07T02:28:34.645472+0000","last_clean":"2024-05-07T02:28:34.645472+0000","last_became_active":"2024-05-07T02:27:32.590197+0000","last_became_peered":"2024-05-07T02:27:32.590197+0000","last_unstale":"2024-05-07T02:28:34.645472+0000","last_undegraded":"2024-05-07T02:28:34.645472+0000","last_fullsized":"2024-05-07T02:28:34.645472+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:23:07.401251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.436771+0000","last_change":"2024-05-07T02:26:11.857671+0000","last_active":"2024-05-07T02:29:31.436771+0000","last_peered":"2024-05-07T02:29:31.436771+0000","last_clean":"2024-05-07T02:29:31.436771+0000","last_became_active":"2024-05-07T02:26:11.857293+0000","last_became_peered":"2024-05-07T02:26:11.857293+0000","last_unstale":"2024-05-07T02:29:31.436771+0000","last_undegraded":"2024-05-07T02:29:31.436771+0000","last_fullsized":"2024-05-07T02:29:31.436771+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:48:42.267887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.613506+0000","last_change":"2024-05-07T02:26:25.563274+0000","last_active":"2024-05-07T02:28:33.613506+0000","last_peered":"2024-05-07T02:28:33.613506+0000","last_clean":"2024-05-07T02:28:33.613506+0000","last_became_active":"2024-05-07T02:26:25.562868+0000","last_became_peered":"2024-05-07T02:26:25.562868+0000","last_unstale":"2024-05-07T02:28:33.613506+0000","last_undegraded":"2024-05-07T02:28:33.613506+0000","last_fullsized":"2024-05-07T02:28:33.613506+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T14:20:03.210034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:34.646544+0000","last_change":"2024-05-07T02:26:07.784888+0000","last_active":"2024-05-07T02:28:34.646544+0000","last_peered":"2024-05-07T02:28:34.646544+0000","last_clean":"2024-05-07T02:28:34.646544+0000","last_became_active":"2024-05-07T02:26:07.784493+0000","last_became_peered":"2024-05-07T02:26:07.784493+0000","last_unstale":"2024-05-07T02:28:34.646544+0000","last_undegraded":"2024-05-07T02:28:34.646544+0000","last_fullsized":"2024-05-07T02:28:34.646544+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:21:14.054786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.443398+0000","last_change":"2024-05-07T02:26:11.858887+0000","last_active":"2024-05-07T02:29:31.443398+0000","last_peered":"2024-05-07T02:29:31.443398+0000","last_clean":"2024-05-07T02:29:31.443398+0000","last_became_active":"2024-05-07T02:26:11.858504+0000","last_became_peered":"2024-05-07T02:26:11.858504+0000","last_unstale":"2024-05-07T02:29:31.443398+0000","last_undegraded":"2024-05-07T02:29:31.443398+0000","last_fullsized":"2024-05-07T02:29:31.443398+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:53:20.290039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:34.645631+0000","last_change":"2024-05-07T02:27:32.627310+0000","last_active":"2024-05-07T02:28:34.645631+0000","last_peered":"2024-05-07T02:28:34.645631+0000","last_clean":"2024-05-07T02:28:34.645631+0000","last_became_active":"2024-05-07T02:27:32.626794+0000","last_became_peered":"2024-05-07T02:27:32.626794+0000","last_unstale":"2024-05-07T02:28:34.645631+0000","last_undegraded":"2024-05-07T02:28:34.645631+0000","last_fullsized":"2024-05-07T02:28:34.645631+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:06:30.466291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.618984+0000","last_change":"2024-05-07T02:26:08.000604+0000","last_active":"2024-05-07T02:28:33.618984+0000","last_peered":"2024-05-07T02:28:33.618984+0000","last_clean":"2024-05-07T02:28:33.618984+0000","last_became_active":"2024-05-07T02:26:08.000107+0000","last_became_peered":"2024-05-07T02:26:08.000107+0000","last_unstale":"2024-05-07T02:28:33.618984+0000","last_undegraded":"2024-05-07T02:28:33.618984+0000","last_fullsized":"2024-05-07T02:28:33.618984+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:58:06.716456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.450534+0000","last_change":"2024-05-07T02:26:11.876555+0000","last_active":"2024-05-07T02:29:31.450534+0000","last_peered":"2024-05-07T02:29:31.450534+0000","last_clean":"2024-05-07T02:29:31.450534+0000","last_became_active":"2024-05-07T02:26:11.876109+0000","last_became_peered":"2024-05-07T02:26:11.876109+0000","last_unstale":"2024-05-07T02:29:31.450534+0000","last_undegraded":"2024-05-07T02:29:31.450534+0000","last_fullsized":"2024-05-07T02:29:31.450534+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:10:24.995786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.884837+0000","last_change":"2024-05-07T02:26:25.567145+0000","last_active":"2024-05-07T02:28:33.884837+0000","last_peered":"2024-05-07T02:28:33.884837+0000","last_clean":"2024-05-07T02:28:33.884837+0000","last_became_active":"2024-05-07T02:26:25.566767+0000","last_became_peered":"2024-05-07T02:26:25.566767+0000","last_unstale":"2024-05-07T02:28:33.884837+0000","last_undegraded":"2024-05-07T02:28:33.884837+0000","last_fullsized":"2024-05-07T02:28:33.884837+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T05:10:59.333367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'7","reported_seq":180,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:48.006274+0000","last_change":"2024-05-07T02:26:07.789566+0000","last_active":"2024-05-07T02:29:48.006274+0000","last_peered":"2024-05-07T02:29:48.006274+0000","last_clean":"2024-05-07T02:29:48.006274+0000","last_became_active":"2024-05-07T02:26:07.789106+0000","last_became_peered":"2024-05-07T02:26:07.789106+0000","last_unstale":"2024-05-07T02:29:48.006274+0000","last_undegraded":"2024-05-07T02:29:48.006274+0000","last_fullsized":"2024-05-07T02:29:48.006274+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:45:13.620183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.443897+0000","last_change":"2024-05-07T02:26:11.869080+0000","last_active":"2024-05-07T02:29:31.443897+0000","last_peered":"2024-05-07T02:29:31.443897+0000","last_clean":"2024-05-07T02:29:31.443897+0000","last_became_active":"2024-05-07T02:26:11.868702+0000","last_became_peered":"2024-05-07T02:26:11.868702+0000","last_unstale":"2024-05-07T02:29:31.443897+0000","last_undegraded":"2024-05-07T02:29:31.443897+0000","last_fullsized":"2024-05-07T02:29:31.443897+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:05:05.497071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.882275+0000","last_change":"2024-05-07T02:26:25.867393+0000","last_active":"2024-05-07T02:28:33.882275+0000","last_peered":"2024-05-07T02:28:33.882275+0000","last_clean":"2024-05-07T02:28:33.882275+0000","last_became_active":"2024-05-07T02:26:25.866956+0000","last_became_peered":"2024-05-07T02:26:25.866956+0000","last_unstale":"2024-05-07T02:28:33.882275+0000","last_undegraded":"2024-05-07T02:28:33.882275+0000","last_fullsized":"2024-05-07T02:28:33.882275+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:23:48.821668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.442457+0000","last_change":"2024-05-07T02:28:33.871287+0000","last_active":"2024-05-07T02:29:31.442457+0000","last_peered":"2024-05-07T02:29:31.442457+0000","last_clean":"2024-05-07T02:29:31.442457+0000","last_became_active":"2024-05-07T02:28:33.870256+0000","last_became_peered":"2024-05-07T02:28:33.870256+0000","last_unstale":"2024-05-07T02:29:31.442457+0000","last_undegraded":"2024-05-07T02:29:31.442457+0000","last_fullsized":"2024-05-07T02:29:31.442457+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T08:36:26.789348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":178,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:50.081091+0000","last_change":"2024-05-07T02:26:07.999784+0000","last_active":"2024-05-07T02:29:50.081091+0000","last_peered":"2024-05-07T02:29:50.081091+0000","last_clean":"2024-05-07T02:29:50.081091+0000","last_became_active":"2024-05-07T02:26:07.999339+0000","last_became_peered":"2024-05-07T02:26:07.999339+0000","last_unstale":"2024-05-07T02:29:50.081091+0000","last_undegraded":"2024-05-07T02:29:50.081091+0000","last_fullsized":"2024-05-07T02:29:50.081091+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:50:41.622486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":128,"num_read_kb":128,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.693213+0000","last_change":"2024-05-07T02:26:25.587032+0000","last_active":"2024-05-07T02:28:33.693213+0000","last_peered":"2024-05-07T02:28:33.693213+0000","last_clean":"2024-05-07T02:28:33.693213+0000","last_became_active":"2024-05-07T02:26:25.586530+0000","last_became_peered":"2024-05-07T02:26:25.586530+0000","last_unstale":"2024-05-07T02:28:33.693213+0000","last_undegraded":"2024-05-07T02:28:33.693213+0000","last_fullsized":"2024-05-07T02:28:33.693213+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:13:13.335832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.462997+0000","last_change":"2024-05-07T02:26:11.878593+0000","last_active":"2024-05-07T02:29:31.462997+0000","last_peered":"2024-05-07T02:29:31.462997+0000","last_clean":"2024-05-07T02:29:31.462997+0000","last_became_active":"2024-05-07T02:26:11.873286+0000","last_became_peered":"2024-05-07T02:26:11.873286+0000","last_unstale":"2024-05-07T02:29:31.462997+0000","last_undegraded":"2024-05-07T02:29:31.462997+0000","last_fullsized":"2024-05-07T02:29:31.462997+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:22:33.769539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.611714+0000","last_change":"2024-05-07T02:26:07.790087+0000","last_active":"2024-05-07T02:28:33.611714+0000","last_peered":"2024-05-07T02:28:33.611714+0000","last_clean":"2024-05-07T02:28:33.611714+0000","last_became_active":"2024-05-07T02:26:07.789592+0000","last_became_peered":"2024-05-07T02:26:07.789592+0000","last_unstale":"2024-05-07T02:28:33.611714+0000","last_undegraded":"2024-05-07T02:28:33.611714+0000","last_fullsized":"2024-05-07T02:28:33.611714+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:44:52.569055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":119,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.692667+0000","last_change":"2024-05-07T02:24:27.877820+0000","last_active":"2024-05-07T02:28:33.692667+0000","last_peered":"2024-05-07T02:28:33.692667+0000","last_clean":"2024-05-07T02:28:33.692667+0000","last_became_active":"2024-05-07T02:24:27.873221+0000","last_became_peered":"2024-05-07T02:24:27.873221+0000","last_unstale":"2024-05-07T02:28:33.692667+0000","last_undegraded":"2024-05-07T02:28:33.692667+0000","last_fullsized":"2024-05-07T02:28:33.692667+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:20:21.582322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:20:21.582322+0000","last_clean_scrub_stamp":"2024-05-07T02:20:21.582322+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T14:16:32.772438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.614338+0000","last_change":"2024-05-07T02:26:25.868394+0000","last_active":"2024-05-07T02:28:33.614338+0000","last_peered":"2024-05-07T02:28:33.614338+0000","last_clean":"2024-05-07T02:28:33.614338+0000","last_became_active":"2024-05-07T02:26:25.867583+0000","last_became_peered":"2024-05-07T02:26:25.867583+0000","last_unstale":"2024-05-07T02:28:33.614338+0000","last_undegraded":"2024-05-07T02:28:33.614338+0000","last_fullsized":"2024-05-07T02:28:33.614338+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:55:44.705608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.436288+0000","last_change":"2024-05-07T02:26:11.868371+0000","last_active":"2024-05-07T02:29:31.436288+0000","last_peered":"2024-05-07T02:29:31.436288+0000","last_clean":"2024-05-07T02:29:31.436288+0000","last_became_active":"2024-05-07T02:26:11.867981+0000","last_became_peered":"2024-05-07T02:26:11.867981+0000","last_unstale":"2024-05-07T02:29:31.436288+0000","last_undegraded":"2024-05-07T02:29:31.436288+0000","last_fullsized":"2024-05-07T02:29:31.436288+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:08:54.687327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.692876+0000","last_change":"2024-05-07T02:26:07.785344+0000","last_active":"2024-05-07T02:28:33.692876+0000","last_peered":"2024-05-07T02:28:33.692876+0000","last_clean":"2024-05-07T02:28:33.692876+0000","last_became_active":"2024-05-07T02:26:07.784707+0000","last_became_peered":"2024-05-07T02:26:07.784707+0000","last_unstale":"2024-05-07T02:28:33.692876+0000","last_undegraded":"2024-05-07T02:28:33.692876+0000","last_fullsized":"2024-05-07T02:28:33.692876+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T05:00:38.754313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:34.645559+0000","last_change":"2024-05-07T02:26:25.571989+0000","last_active":"2024-05-07T02:28:34.645559+0000","last_peered":"2024-05-07T02:28:34.645559+0000","last_clean":"2024-05-07T02:28:34.645559+0000","last_became_active":"2024-05-07T02:26:25.571533+0000","last_became_peered":"2024-05-07T02:26:25.571533+0000","last_unstale":"2024-05-07T02:28:34.645559+0000","last_undegraded":"2024-05-07T02:28:34.645559+0000","last_fullsized":"2024-05-07T02:28:34.645559+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:54:38.669278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:34.646468+0000","last_change":"2024-05-07T02:26:07.999636+0000","last_active":"2024-05-07T02:28:34.646468+0000","last_peered":"2024-05-07T02:28:34.646468+0000","last_clean":"2024-05-07T02:28:34.646468+0000","last_became_active":"2024-05-07T02:26:07.999100+0000","last_became_peered":"2024-05-07T02:26:07.999100+0000","last_unstale":"2024-05-07T02:28:34.646468+0000","last_undegraded":"2024-05-07T02:28:34.646468+0000","last_fullsized":"2024-05-07T02:28:34.646468+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:10:24.730988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.440226+0000","last_change":"2024-05-07T02:26:11.888279+0000","last_active":"2024-05-07T02:29:31.440226+0000","last_peered":"2024-05-07T02:29:31.440226+0000","last_clean":"2024-05-07T02:29:31.440226+0000","last_became_active":"2024-05-07T02:26:11.887480+0000","last_became_peered":"2024-05-07T02:26:11.887480+0000","last_unstale":"2024-05-07T02:29:31.440226+0000","last_undegraded":"2024-05-07T02:29:31.440226+0000","last_fullsized":"2024-05-07T02:29:31.440226+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:12:12.368368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.615801+0000","last_change":"2024-05-07T02:26:25.867444+0000","last_active":"2024-05-07T02:28:33.615801+0000","last_peered":"2024-05-07T02:28:33.615801+0000","last_clean":"2024-05-07T02:28:33.615801+0000","last_became_active":"2024-05-07T02:26:25.866392+0000","last_became_peered":"2024-05-07T02:26:25.866392+0000","last_unstale":"2024-05-07T02:28:33.615801+0000","last_undegraded":"2024-05-07T02:28:33.615801+0000","last_fullsized":"2024-05-07T02:28:33.615801+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:54:36.112427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'17","reported_seq":62,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.618768+0000","last_change":"2024-05-07T02:26:07.798091+0000","last_active":"2024-05-07T02:28:33.618768+0000","last_peered":"2024-05-07T02:28:33.618768+0000","last_clean":"2024-05-07T02:28:33.618768+0000","last_became_active":"2024-05-07T02:26:07.797716+0000","last_became_peered":"2024-05-07T02:26:07.797716+0000","last_unstale":"2024-05-07T02:28:33.618768+0000","last_undegraded":"2024-05-07T02:28:33.618768+0000","last_fullsized":"2024-05-07T02:28:33.618768+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:04:54.570916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.464763+0000","last_change":"2024-05-07T02:26:11.888638+0000","last_active":"2024-05-07T02:29:31.464763+0000","last_peered":"2024-05-07T02:29:31.464763+0000","last_clean":"2024-05-07T02:29:31.464763+0000","last_became_active":"2024-05-07T02:26:11.888269+0000","last_became_peered":"2024-05-07T02:26:11.888269+0000","last_unstale":"2024-05-07T02:29:31.464763+0000","last_undegraded":"2024-05-07T02:29:31.464763+0000","last_fullsized":"2024-05-07T02:29:31.464763+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:09:38.965362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.868019+0000","last_change":"2024-05-07T02:26:25.558975+0000","last_active":"2024-05-07T02:28:33.868019+0000","last_peered":"2024-05-07T02:28:33.868019+0000","last_clean":"2024-05-07T02:28:33.868019+0000","last_became_active":"2024-05-07T02:26:25.557947+0000","last_became_peered":"2024-05-07T02:26:25.557947+0000","last_unstale":"2024-05-07T02:28:33.868019+0000","last_undegraded":"2024-05-07T02:28:33.868019+0000","last_fullsized":"2024-05-07T02:28:33.868019+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:29:32.071625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.612345+0000","last_change":"2024-05-07T02:26:07.796643+0000","last_active":"2024-05-07T02:28:33.612345+0000","last_peered":"2024-05-07T02:28:33.612345+0000","last_clean":"2024-05-07T02:28:33.612345+0000","last_became_active":"2024-05-07T02:26:07.796264+0000","last_became_peered":"2024-05-07T02:26:07.796264+0000","last_unstale":"2024-05-07T02:28:33.612345+0000","last_undegraded":"2024-05-07T02:28:33.612345+0000","last_fullsized":"2024-05-07T02:28:33.612345+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:29:32.454816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.463845+0000","last_change":"2024-05-07T02:26:11.873670+0000","last_active":"2024-05-07T02:29:31.463845+0000","last_peered":"2024-05-07T02:29:31.463845+0000","last_clean":"2024-05-07T02:29:31.463845+0000","last_became_active":"2024-05-07T02:26:11.871219+0000","last_became_peered":"2024-05-07T02:26:11.871219+0000","last_unstale":"2024-05-07T02:29:31.463845+0000","last_undegraded":"2024-05-07T02:29:31.463845+0000","last_fullsized":"2024-05-07T02:29:31.463845+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:49:48.048127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.612451+0000","last_change":"2024-05-07T02:26:25.868011+0000","last_active":"2024-05-07T02:28:33.612451+0000","last_peered":"2024-05-07T02:28:33.612451+0000","last_clean":"2024-05-07T02:28:33.612451+0000","last_became_active":"2024-05-07T02:26:25.867465+0000","last_became_peered":"2024-05-07T02:26:25.867465+0000","last_unstale":"2024-05-07T02:28:33.612451+0000","last_undegraded":"2024-05-07T02:28:33.612451+0000","last_fullsized":"2024-05-07T02:28:33.612451+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T05:14:40.001874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:34.333778+0000","last_change":"2024-05-07T02:26:07.999233+0000","last_active":"2024-05-07T02:28:34.333778+0000","last_peered":"2024-05-07T02:28:34.333778+0000","last_clean":"2024-05-07T02:28:34.333778+0000","last_became_active":"2024-05-07T02:26:07.998920+0000","last_became_peered":"2024-05-07T02:26:07.998920+0000","last_unstale":"2024-05-07T02:28:34.333778+0000","last_undegraded":"2024-05-07T02:28:34.333778+0000","last_fullsized":"2024-05-07T02:28:34.333778+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:33:33.878984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.401820+0000","last_change":"2024-05-07T02:26:11.856832+0000","last_active":"2024-05-07T02:29:31.401820+0000","last_peered":"2024-05-07T02:29:31.401820+0000","last_clean":"2024-05-07T02:29:31.401820+0000","last_became_active":"2024-05-07T02:26:11.856424+0000","last_became_peered":"2024-05-07T02:26:11.856424+0000","last_unstale":"2024-05-07T02:29:31.401820+0000","last_undegraded":"2024-05-07T02:29:31.401820+0000","last_fullsized":"2024-05-07T02:29:31.401820+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:00:46.839542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:34.646623+0000","last_change":"2024-05-07T02:27:32.626086+0000","last_active":"2024-05-07T02:28:34.646623+0000","last_peered":"2024-05-07T02:28:34.646623+0000","last_clean":"2024-05-07T02:28:34.646623+0000","last_became_active":"2024-05-07T02:27:32.625468+0000","last_became_peered":"2024-05-07T02:27:32.625468+0000","last_unstale":"2024-05-07T02:28:34.646623+0000","last_undegraded":"2024-05-07T02:28:34.646623+0000","last_fullsized":"2024-05-07T02:28:34.646623+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:17:57.245648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.620116+0000","last_change":"2024-05-07T02:26:07.793793+0000","last_active":"2024-05-07T02:28:33.620116+0000","last_peered":"2024-05-07T02:28:33.620116+0000","last_clean":"2024-05-07T02:28:33.620116+0000","last_became_active":"2024-05-07T02:26:07.790291+0000","last_became_peered":"2024-05-07T02:26:07.790291+0000","last_unstale":"2024-05-07T02:28:33.620116+0000","last_undegraded":"2024-05-07T02:28:33.620116+0000","last_fullsized":"2024-05-07T02:28:33.620116+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:50:14.591968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.458904+0000","last_change":"2024-05-07T02:26:11.869184+0000","last_active":"2024-05-07T02:29:31.458904+0000","last_peered":"2024-05-07T02:29:31.458904+0000","last_clean":"2024-05-07T02:29:31.458904+0000","last_became_active":"2024-05-07T02:26:11.868773+0000","last_became_peered":"2024-05-07T02:26:11.868773+0000","last_unstale":"2024-05-07T02:29:31.458904+0000","last_undegraded":"2024-05-07T02:29:31.458904+0000","last_fullsized":"2024-05-07T02:29:31.458904+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:24:04.986359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.612145+0000","last_change":"2024-05-07T02:26:25.580095+0000","last_active":"2024-05-07T02:28:33.612145+0000","last_peered":"2024-05-07T02:28:33.612145+0000","last_clean":"2024-05-07T02:28:33.612145+0000","last_became_active":"2024-05-07T02:26:25.579615+0000","last_became_peered":"2024-05-07T02:26:25.579615+0000","last_unstale":"2024-05-07T02:28:33.612145+0000","last_undegraded":"2024-05-07T02:28:33.612145+0000","last_fullsized":"2024-05-07T02:28:33.612145+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:18:05.714552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.868184+0000","last_change":"2024-05-07T02:26:07.784709+0000","last_active":"2024-05-07T02:28:33.868184+0000","last_peered":"2024-05-07T02:28:33.868184+0000","last_clean":"2024-05-07T02:28:33.868184+0000","last_became_active":"2024-05-07T02:26:07.784282+0000","last_became_peered":"2024-05-07T02:26:07.784282+0000","last_unstale":"2024-05-07T02:28:33.868184+0000","last_undegraded":"2024-05-07T02:28:33.868184+0000","last_fullsized":"2024-05-07T02:28:33.868184+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:01:47.023496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.463214+0000","last_change":"2024-05-07T02:26:11.879152+0000","last_active":"2024-05-07T02:29:31.463214+0000","last_peered":"2024-05-07T02:29:31.463214+0000","last_clean":"2024-05-07T02:29:31.463214+0000","last_became_active":"2024-05-07T02:26:11.878571+0000","last_became_peered":"2024-05-07T02:26:11.878571+0000","last_unstale":"2024-05-07T02:29:31.463214+0000","last_undegraded":"2024-05-07T02:29:31.463214+0000","last_fullsized":"2024-05-07T02:29:31.463214+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:04:12.637807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.612216+0000","last_change":"2024-05-07T02:26:25.867199+0000","last_active":"2024-05-07T02:28:33.612216+0000","last_peered":"2024-05-07T02:28:33.612216+0000","last_clean":"2024-05-07T02:28:33.612216+0000","last_became_active":"2024-05-07T02:26:25.866624+0000","last_became_peered":"2024-05-07T02:26:25.866624+0000","last_unstale":"2024-05-07T02:28:33.612216+0000","last_undegraded":"2024-05-07T02:28:33.612216+0000","last_fullsized":"2024-05-07T02:28:33.612216+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:14:49.513313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:34.645931+0000","last_change":"2024-05-07T02:26:07.784155+0000","last_active":"2024-05-07T02:28:34.645931+0000","last_peered":"2024-05-07T02:28:34.645931+0000","last_clean":"2024-05-07T02:28:34.645931+0000","last_became_active":"2024-05-07T02:26:07.783579+0000","last_became_peered":"2024-05-07T02:26:07.783579+0000","last_unstale":"2024-05-07T02:28:34.645931+0000","last_undegraded":"2024-05-07T02:28:34.645931+0000","last_fullsized":"2024-05-07T02:28:34.645931+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T08:51:32.037959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.457140+0000","last_change":"2024-05-07T02:26:11.868454+0000","last_active":"2024-05-07T02:29:31.457140+0000","last_peered":"2024-05-07T02:29:31.457140+0000","last_clean":"2024-05-07T02:29:31.457140+0000","last_became_active":"2024-05-07T02:26:11.868117+0000","last_became_peered":"2024-05-07T02:26:11.868117+0000","last_unstale":"2024-05-07T02:29:31.457140+0000","last_undegraded":"2024-05-07T02:29:31.457140+0000","last_fullsized":"2024-05-07T02:29:31.457140+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:24:36.703684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:34.333863+0000","last_change":"2024-05-07T02:26:25.573834+0000","last_active":"2024-05-07T02:28:34.333863+0000","last_peered":"2024-05-07T02:28:34.333863+0000","last_clean":"2024-05-07T02:28:34.333863+0000","last_became_active":"2024-05-07T02:26:25.573372+0000","last_became_peered":"2024-05-07T02:26:25.573372+0000","last_unstale":"2024-05-07T02:28:34.333863+0000","last_undegraded":"2024-05-07T02:28:34.333863+0000","last_fullsized":"2024-05-07T02:28:34.333863+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:03:36.916915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":179,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:50.041011+0000","last_change":"2024-05-07T02:26:08.000245+0000","last_active":"2024-05-07T02:29:50.041011+0000","last_peered":"2024-05-07T02:29:50.041011+0000","last_clean":"2024-05-07T02:29:50.041011+0000","last_became_active":"2024-05-07T02:26:07.999587+0000","last_became_peered":"2024-05-07T02:26:07.999587+0000","last_unstale":"2024-05-07T02:29:50.041011+0000","last_undegraded":"2024-05-07T02:29:50.041011+0000","last_fullsized":"2024-05-07T02:29:50.041011+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:44:41.754799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":129,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.452891+0000","last_change":"2024-05-07T02:26:11.858308+0000","last_active":"2024-05-07T02:29:31.452891+0000","last_peered":"2024-05-07T02:29:31.452891+0000","last_clean":"2024-05-07T02:29:31.452891+0000","last_became_active":"2024-05-07T02:26:11.857855+0000","last_became_peered":"2024-05-07T02:26:11.857855+0000","last_unstale":"2024-05-07T02:29:31.452891+0000","last_undegraded":"2024-05-07T02:29:31.452891+0000","last_fullsized":"2024-05-07T02:29:31.452891+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:11:50.021178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.615521+0000","last_change":"2024-05-07T02:26:25.584090+0000","last_active":"2024-05-07T02:28:33.615521+0000","last_peered":"2024-05-07T02:28:33.615521+0000","last_clean":"2024-05-07T02:28:33.615521+0000","last_became_active":"2024-05-07T02:26:25.580108+0000","last_became_peered":"2024-05-07T02:26:25.580108+0000","last_unstale":"2024-05-07T02:28:33.615521+0000","last_undegraded":"2024-05-07T02:28:33.615521+0000","last_fullsized":"2024-05-07T02:28:33.615521+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:56:56.964047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.614227+0000","last_change":"2024-05-07T02:26:07.788188+0000","last_active":"2024-05-07T02:28:33.614227+0000","last_peered":"2024-05-07T02:28:33.614227+0000","last_clean":"2024-05-07T02:28:33.614227+0000","last_became_active":"2024-05-07T02:26:07.787826+0000","last_became_peered":"2024-05-07T02:26:07.787826+0000","last_unstale":"2024-05-07T02:28:33.614227+0000","last_undegraded":"2024-05-07T02:28:33.614227+0000","last_fullsized":"2024-05-07T02:28:33.614227+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:06:14.588079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.454390+0000","last_change":"2024-05-07T02:26:11.862925+0000","last_active":"2024-05-07T02:29:31.454390+0000","last_peered":"2024-05-07T02:29:31.454390+0000","last_clean":"2024-05-07T02:29:31.454390+0000","last_became_active":"2024-05-07T02:26:11.861332+0000","last_became_peered":"2024-05-07T02:26:11.861332+0000","last_unstale":"2024-05-07T02:29:31.454390+0000","last_undegraded":"2024-05-07T02:29:31.454390+0000","last_fullsized":"2024-05-07T02:29:31.454390+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:51:11.133504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.613240+0000","last_change":"2024-05-07T02:26:25.557151+0000","last_active":"2024-05-07T02:28:33.613240+0000","last_peered":"2024-05-07T02:28:33.613240+0000","last_clean":"2024-05-07T02:28:33.613240+0000","last_became_active":"2024-05-07T02:26:25.556615+0000","last_became_peered":"2024-05-07T02:26:25.556615+0000","last_unstale":"2024-05-07T02:28:33.613240+0000","last_undegraded":"2024-05-07T02:28:33.613240+0000","last_fullsized":"2024-05-07T02:28:33.613240+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:33:20.446543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:28:33.885890+0000","last_change":"2024-05-07T02:27:32.598094+0000","last_active":"2024-05-07T02:28:33.885890+0000","last_peered":"2024-05-07T02:28:33.885890+0000","last_clean":"2024-05-07T02:28:33.885890+0000","last_became_active":"2024-05-07T02:27:32.594099+0000","last_became_peered":"2024-05-07T02:27:32.594099+0000","last_unstale":"2024-05-07T02:28:33.885890+0000","last_undegraded":"2024-05-07T02:28:33.885890+0000","last_fullsized":"2024-05-07T02:28:33.885890+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:44:50.979927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.443992+0000","last_change":"2024-05-07T02:26:11.860507+0000","last_active":"2024-05-07T02:29:31.443992+0000","last_peered":"2024-05-07T02:29:31.443992+0000","last_clean":"2024-05-07T02:29:31.443992+0000","last_became_active":"2024-05-07T02:26:11.860116+0000","last_became_peered":"2024-05-07T02:26:11.860116+0000","last_unstale":"2024-05-07T02:29:31.443992+0000","last_undegraded":"2024-05-07T02:29:31.443992+0000","last_fullsized":"2024-05-07T02:29:31.443992+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T05:09:38.963040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.434049+0000","last_change":"2024-05-07T02:26:11.867932+0000","last_active":"2024-05-07T02:29:31.434049+0000","last_peered":"2024-05-07T02:29:31.434049+0000","last_clean":"2024-05-07T02:29:31.434049+0000","last_became_active":"2024-05-07T02:26:11.867537+0000","last_became_peered":"2024-05-07T02:26:11.867537+0000","last_unstale":"2024-05-07T02:29:31.434049+0000","last_undegraded":"2024-05-07T02:29:31.434049+0000","last_fullsized":"2024-05-07T02:29:31.434049+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:31:35.843401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.435936+0000","last_change":"2024-05-07T02:26:11.866074+0000","last_active":"2024-05-07T02:29:31.435936+0000","last_peered":"2024-05-07T02:29:31.435936+0000","last_clean":"2024-05-07T02:29:31.435936+0000","last_became_active":"2024-05-07T02:26:11.865669+0000","last_became_peered":"2024-05-07T02:26:11.865669+0000","last_unstale":"2024-05-07T02:29:31.435936+0000","last_undegraded":"2024-05-07T02:29:31.435936+0000","last_fullsized":"2024-05-07T02:29:31.435936+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:08:51.323706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.458706+0000","last_change":"2024-05-07T02:26:11.876428+0000","last_active":"2024-05-07T02:29:31.458706+0000","last_peered":"2024-05-07T02:29:31.458706+0000","last_clean":"2024-05-07T02:29:31.458706+0000","last_became_active":"2024-05-07T02:26:11.876012+0000","last_became_peered":"2024-05-07T02:26:11.876012+0000","last_unstale":"2024-05-07T02:29:31.458706+0000","last_undegraded":"2024-05-07T02:29:31.458706+0000","last_fullsized":"2024-05-07T02:29:31.458706+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:24:58.088695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.458687+0000","last_change":"2024-05-07T02:26:11.880682+0000","last_active":"2024-05-07T02:29:31.458687+0000","last_peered":"2024-05-07T02:29:31.458687+0000","last_clean":"2024-05-07T02:29:31.458687+0000","last_became_active":"2024-05-07T02:26:11.880308+0000","last_became_peered":"2024-05-07T02:26:11.880308+0000","last_unstale":"2024-05-07T02:29:31.458687+0000","last_undegraded":"2024-05-07T02:29:31.458687+0000","last_fullsized":"2024-05-07T02:29:31.458687+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:52:45.083964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.456972+0000","last_change":"2024-05-07T02:26:11.876932+0000","last_active":"2024-05-07T02:29:31.456972+0000","last_peered":"2024-05-07T02:29:31.456972+0000","last_clean":"2024-05-07T02:29:31.456972+0000","last_became_active":"2024-05-07T02:26:11.876082+0000","last_became_peered":"2024-05-07T02:26:11.876082+0000","last_unstale":"2024-05-07T02:29:31.456972+0000","last_undegraded":"2024-05-07T02:29:31.456972+0000","last_fullsized":"2024-05-07T02:29:31.456972+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:45:21.435500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.425660+0000","last_change":"2024-05-07T02:26:11.861339+0000","last_active":"2024-05-07T02:29:31.425660+0000","last_peered":"2024-05-07T02:29:31.425660+0000","last_clean":"2024-05-07T02:29:31.425660+0000","last_became_active":"2024-05-07T02:26:11.860912+0000","last_became_peered":"2024-05-07T02:26:11.860912+0000","last_unstale":"2024-05-07T02:29:31.425660+0000","last_undegraded":"2024-05-07T02:29:31.425660+0000","last_fullsized":"2024-05-07T02:29:31.425660+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:12:57.944986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.449041+0000","last_change":"2024-05-07T02:26:11.867258+0000","last_active":"2024-05-07T02:29:31.449041+0000","last_peered":"2024-05-07T02:29:31.449041+0000","last_clean":"2024-05-07T02:29:31.449041+0000","last_became_active":"2024-05-07T02:26:11.866837+0000","last_became_peered":"2024-05-07T02:26:11.866837+0000","last_unstale":"2024-05-07T02:29:31.449041+0000","last_undegraded":"2024-05-07T02:29:31.449041+0000","last_fullsized":"2024-05-07T02:29:31.449041+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:07:58.589457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.447442+0000","last_change":"2024-05-07T02:26:11.888434+0000","last_active":"2024-05-07T02:29:31.447442+0000","last_peered":"2024-05-07T02:29:31.447442+0000","last_clean":"2024-05-07T02:29:31.447442+0000","last_became_active":"2024-05-07T02:26:11.887681+0000","last_became_peered":"2024-05-07T02:26:11.887681+0000","last_unstale":"2024-05-07T02:29:31.447442+0000","last_undegraded":"2024-05-07T02:29:31.447442+0000","last_fullsized":"2024-05-07T02:29:31.447442+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:38:58.896781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.453153+0000","last_change":"2024-05-07T02:26:11.875304+0000","last_active":"2024-05-07T02:29:31.453153+0000","last_peered":"2024-05-07T02:29:31.453153+0000","last_clean":"2024-05-07T02:29:31.453153+0000","last_became_active":"2024-05-07T02:26:11.874946+0000","last_became_peered":"2024-05-07T02:26:11.874946+0000","last_unstale":"2024-05-07T02:29:31.453153+0000","last_undegraded":"2024-05-07T02:29:31.453153+0000","last_fullsized":"2024-05-07T02:29:31.453153+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:56:04.093791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.463312+0000","last_change":"2024-05-07T02:26:11.879311+0000","last_active":"2024-05-07T02:29:31.463312+0000","last_peered":"2024-05-07T02:29:31.463312+0000","last_clean":"2024-05-07T02:29:31.463312+0000","last_became_active":"2024-05-07T02:26:11.877620+0000","last_became_peered":"2024-05-07T02:26:11.877620+0000","last_unstale":"2024-05-07T02:29:31.463312+0000","last_undegraded":"2024-05-07T02:29:31.463312+0000","last_fullsized":"2024-05-07T02:29:31.463312+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T08:41:40.525708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.463659+0000","last_change":"2024-05-07T02:26:11.876726+0000","last_active":"2024-05-07T02:29:31.463659+0000","last_peered":"2024-05-07T02:29:31.463659+0000","last_clean":"2024-05-07T02:29:31.463659+0000","last_became_active":"2024-05-07T02:26:11.876234+0000","last_became_peered":"2024-05-07T02:26:11.876234+0000","last_unstale":"2024-05-07T02:29:31.463659+0000","last_undegraded":"2024-05-07T02:29:31.463659+0000","last_fullsized":"2024-05-07T02:29:31.463659+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:26:34.422258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.460248+0000","last_change":"2024-05-07T02:26:11.879159+0000","last_active":"2024-05-07T02:29:31.460248+0000","last_peered":"2024-05-07T02:29:31.460248+0000","last_clean":"2024-05-07T02:29:31.460248+0000","last_became_active":"2024-05-07T02:26:11.878769+0000","last_became_peered":"2024-05-07T02:26:11.878769+0000","last_unstale":"2024-05-07T02:29:31.460248+0000","last_undegraded":"2024-05-07T02:29:31.460248+0000","last_fullsized":"2024-05-07T02:29:31.460248+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:05:27.089475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-07T02:29:31.441690+0000","last_change":"2024-05-07T02:27:32.590188+0000","last_active":"2024-05-07T02:29:31.441690+0000","last_peered":"2024-05-07T02:29:31.441690+0000","last_clean":"2024-05-07T02:29:31.441690+0000","last_became_active":"2024-05-07T02:27:32.589784+0000","last_became_peered":"2024-05-07T02:27:32.589784+0000","last_unstale":"2024-05-07T02:29:31.441690+0000","last_undegraded":"2024-05-07T02:29:31.441690+0000","last_fullsized":"2024-05-07T02:29:31.441690+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:32:40.488143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":75433,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1303,"num_read_kb":738,"num_write":258,"num_write_kb":153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":226299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":222,"ondisk_log_size":222,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710781,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1040384,"data_stored":515823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907012,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32868,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712284,"statfs":{"total":95995035648,"available":95961378816,"internally_reserved":0,"allocated":1732608,"data_stored":1171006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070537,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":519737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266769,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1044480,"data_stored":515907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462998,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1650688,"data_stored":1112657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626523,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":511677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574970,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":581186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738495,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1056768,"data_stored":512601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411445,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1126400,"data_stored":580874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182509,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":510853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986281,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1040384,"data_stored":515219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822754,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1658880,"data_stored":1107707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":70215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-07T02:29:54.834 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-07T02:29:54.834 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-07T02:29:54.834 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-07T02:29:54.834 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph health --format=json 2024-05-07T02:29:55.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:54 smithi064 ceph-mon[34699]: pgmap v450: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:29:55.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:54 smithi064 ceph-mon[34699]: from='client.24949 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-07T02:29:55.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:54 smithi019 ceph-mon[29781]: pgmap v450: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:29:55.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:54 smithi019 ceph-mon[29781]: from='client.24949 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-07T02:29:55.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:54 smithi027 ceph-mon[34398]: pgmap v450: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:29:55.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:54 smithi027 ceph-mon[34398]: from='client.24949 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-07T02:29:55.345 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:29:56.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:56 smithi019 ceph-mon[29781]: pgmap v451: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:29:57.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:56 smithi064 ceph-mon[34699]: pgmap v451: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:29:57.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:56 smithi027 ceph-mon[34398]: pgmap v451: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:29:57.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:57.989+0000 7f0c89a00700 1 -- 172.21.15.19:0/2440096699 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c84108560 msgr2=0x7f0c8410fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:57.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:57.989+0000 7f0c89a00700 1 --2- 172.21.15.19:0/2440096699 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c84108560 0x7f0c8410fec0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c009fc0 tx=0x7f0c7c00b330 comp rx=0 tx=0).stop 2024-05-07T02:29:57.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:57.989+0000 7f0c89a00700 1 -- 172.21.15.19:0/2440096699 shutdown_connections 2024-05-07T02:29:57.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:57.989+0000 7f0c89a00700 1 --2- 172.21.15.19:0/2440096699 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c84108560 0x7f0c8410fec0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:57.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:57.989+0000 7f0c89a00700 1 --2- 172.21.15.19:0/2440096699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c841032e0 0x7f0c841036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:57.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:57.989+0000 7f0c89a00700 1 --2- 172.21.15.19:0/2440096699 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c84107c10 0x7f0c84108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:57.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:57.989+0000 7f0c89a00700 1 -- 172.21.15.19:0/2440096699 >> 172.21.15.19:0/2440096699 conn(0x7f0c84078ac0 msgr2=0x7f0c84078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:57.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:57.989+0000 7f0c89a00700 1 -- 172.21.15.19:0/2440096699 shutdown_connections 2024-05-07T02:29:57.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:57.989+0000 7f0c89a00700 1 -- 172.21.15.19:0/2440096699 wait complete. 2024-05-07T02:29:57.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:57.991+0000 7f0c89a00700 1 Processor -- start 2024-05-07T02:29:57.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:57.991+0000 7f0c89a00700 1 -- start start 2024-05-07T02:29:57.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:57.991+0000 7f0c89a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c841032e0 0x7f0c84100440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:57.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:57.991+0000 7f0c89a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c84107c10 0x7f0c84100980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:57.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:57.991+0000 7f0c89a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c84108560 0x7f0c84100ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:57.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:57.991+0000 7f0c89a00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0c841131e0 con 0x7f0c84108560 2024-05-07T02:29:57.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:57.991+0000 7f0c89a00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0c84113060 con 0x7f0c84107c10 2024-05-07T02:29:57.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:57.991+0000 7f0c89a00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0c84113360 con 0x7f0c841032e0 2024-05-07T02:29:57.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:57.992+0000 7f0c89000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c84108560 0x7f0c84100ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:57.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:57.992+0000 7f0c89000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c84108560 0x7f0c84100ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56826/0 (socket says 172.21.15.19:56826) 2024-05-07T02:29:57.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:57.992+0000 7f0c89000700 1 -- 172.21.15.19:0/36745692 learned_addr learned my addr 172.21.15.19:0/36745692 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:29:57.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:57.992+0000 7f0c83e00700 1 --2- 172.21.15.19:0/36745692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c841032e0 0x7f0c84100440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:57.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:57.992+0000 7f0c83400700 1 --2- 172.21.15.19:0/36745692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c84107c10 0x7f0c84100980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:57.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:57.992+0000 7f0c89000700 1 -- 172.21.15.19:0/36745692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c841032e0 msgr2=0x7f0c84100440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:57.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:57.992+0000 7f0c89000700 1 --2- 172.21.15.19:0/36745692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c841032e0 0x7f0c84100440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:57.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:57.992+0000 7f0c89000700 1 -- 172.21.15.19:0/36745692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c84107c10 msgr2=0x7f0c84100980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:57.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:57.992+0000 7f0c89000700 1 --2- 172.21.15.19:0/36745692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c84107c10 0x7f0c84100980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:57.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:57.992+0000 7f0c89000700 1 -- 172.21.15.19:0/36745692 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c841a90a0 con 0x7f0c84108560 2024-05-07T02:29:57.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:57.992+0000 7f0c83400700 1 --2- 172.21.15.19:0/36745692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c84107c10 0x7f0c84100980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:29:57.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:57.993+0000 7f0c89000700 1 --2- 172.21.15.19:0/36745692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c84108560 0x7f0c84100ec0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c009790 tx=0x7f0c7c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:57.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:57.993+0000 7f0c80c00700 1 -- 172.21.15.19:0/36745692 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c021070 con 0x7f0c84108560 2024-05-07T02:29:57.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:57.993+0000 7f0c80c00700 1 -- 172.21.15.19:0/36745692 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c7c007ac0 con 0x7f0c84108560 2024-05-07T02:29:57.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:57.993+0000 7f0c89a00700 1 -- 172.21.15.19:0/36745692 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c841a9330 con 0x7f0c84108560 2024-05-07T02:29:57.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:57.993+0000 7f0c80c00700 1 -- 172.21.15.19:0/36745692 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c005040 con 0x7f0c84108560 2024-05-07T02:29:57.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:57.993+0000 7f0c89a00700 1 -- 172.21.15.19:0/36745692 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c841a9830 con 0x7f0c84108560 2024-05-07T02:29:57.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:57.995+0000 7f0c89a00700 1 -- 172.21.15.19:0/36745692 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c840512c0 con 0x7f0c84108560 2024-05-07T02:29:57.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:57.997+0000 7f0c80c00700 1 -- 172.21.15.19:0/36745692 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0c7c024070 con 0x7f0c84108560 2024-05-07T02:29:57.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:57.999+0000 7f0c80c00700 1 --2- 172.21.15.19:0/36745692 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f0c680b2900 0x7f0c680b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:29:57.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:57.999+0000 7f0c80c00700 1 -- 172.21.15.19:0/36745692 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f0c7c00f020 con 0x7f0c84108560 2024-05-07T02:29:57.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:57.999+0000 7f0c83e00700 1 --2- 172.21.15.19:0/36745692 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f0c680b2900 0x7f0c680b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:29:57.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:58.000+0000 7f0c83e00700 1 --2- 172.21.15.19:0/36745692 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f0c680b2900 0x7f0c680b4e10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0c74009c60 tx=0x7f0c74009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:29:58.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:58.002+0000 7f0c80c00700 1 -- 172.21.15.19:0/36745692 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0c7c02b020 con 0x7f0c84108560 2024-05-07T02:29:58.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:58.290+0000 7f0c89a00700 1 -- 172.21.15.19:0/36745692 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f0c8402e5f0 con 0x7f0c84108560 2024-05-07T02:29:58.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:58.291+0000 7f0c80c00700 1 -- 172.21.15.19:0/36745692 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f0c7c0ab780 con 0x7f0c84108560 2024-05-07T02:29:58.291 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:29:58.292 INFO:teuthology.orchestra.run.smithi019.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-07T02:29:58.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:58.294+0000 7f0c89a00700 1 -- 172.21.15.19:0/36745692 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f0c680b2900 msgr2=0x7f0c680b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:58.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:58.294+0000 7f0c89a00700 1 --2- 172.21.15.19:0/36745692 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f0c680b2900 0x7f0c680b4e10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0c74009c60 tx=0x7f0c74009260 comp rx=0 tx=0).stop 2024-05-07T02:29:58.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:58.294+0000 7f0c89a00700 1 -- 172.21.15.19:0/36745692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c84108560 msgr2=0x7f0c84100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:29:58.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:58.294+0000 7f0c89a00700 1 --2- 172.21.15.19:0/36745692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c84108560 0x7f0c84100ec0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c009790 tx=0x7f0c7c00cc30 comp rx=0 tx=0).stop 2024-05-07T02:29:58.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:58.295+0000 7f0c89a00700 1 -- 172.21.15.19:0/36745692 shutdown_connections 2024-05-07T02:29:58.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:58.295+0000 7f0c89a00700 1 --2- 172.21.15.19:0/36745692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c84108560 0x7f0c84100ec0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:58.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:58.295+0000 7f0c89a00700 1 --2- 172.21.15.19:0/36745692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c841032e0 0x7f0c84100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:58.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:58.295+0000 7f0c89a00700 1 --2- 172.21.15.19:0/36745692 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f0c680b2900 0x7f0c680b4e10 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:58.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:58.295+0000 7f0c89a00700 1 --2- 172.21.15.19:0/36745692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c84107c10 0x7f0c84100980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:29:58.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:58.295+0000 7f0c89a00700 1 -- 172.21.15.19:0/36745692 >> 172.21.15.19:0/36745692 conn(0x7f0c84078ac0 msgr2=0x7f0c8406b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:29:58.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:58.295+0000 7f0c89a00700 1 -- 172.21.15.19:0/36745692 shutdown_connections 2024-05-07T02:29:58.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:29:58.295+0000 7f0c89a00700 1 -- 172.21.15.19:0/36745692 wait complete. 2024-05-07T02:29:59.008 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:58 smithi027 ceph-mon[34398]: pgmap v452: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:29:59.008 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:29:58 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/36745692' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-07T02:29:59.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:58 smithi064 ceph-mon[34699]: pgmap v452: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:29:59.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:29:58 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/36745692' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-07T02:29:59.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:58 smithi019 ceph-mon[29781]: pgmap v452: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:29:59.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:29:58 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/36745692' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-07T02:29:59.274 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-07T02:29:59.274 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-07T02:29:59.744 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:30:01.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:30:00 smithi064 ceph-mon[34699]: pgmap v453: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:30:01.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:30:00 smithi064 ceph-mon[34699]: overall HEALTH_OK 2024-05-07T02:30:01.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:30:00 smithi019 ceph-mon[29781]: pgmap v453: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:30:01.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:30:00 smithi019 ceph-mon[29781]: overall HEALTH_OK 2024-05-07T02:30:01.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:30:00 smithi027 ceph-mon[34398]: pgmap v453: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:30:01.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:30:00 smithi027 ceph-mon[34398]: overall HEALTH_OK 2024-05-07T02:30:02.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.277+0000 7f160b800700 1 -- 172.21.15.19:0/1696207260 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16041086c0 msgr2=0x7f1604108b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:02.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.277+0000 7f160b800700 1 --2- 172.21.15.19:0/1696207260 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16041086c0 0x7f1604108b90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f15fc00b0a0 tx=0x7f15fc00b3b0 comp rx=0 tx=0).stop 2024-05-07T02:30:02.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.277+0000 7f160b800700 1 -- 172.21.15.19:0/1696207260 shutdown_connections 2024-05-07T02:30:02.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.277+0000 7f160b800700 1 --2- 172.21.15.19:0/1696207260 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16041086c0 0x7f1604108b90 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.277+0000 7f160b800700 1 --2- 172.21.15.19:0/1696207260 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1604103270 0x7f1604103680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.277+0000 7f160b800700 1 --2- 172.21.15.19:0/1696207260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1604103bc0 0x7f16041080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.277+0000 7f160b800700 1 -- 172.21.15.19:0/1696207260 >> 172.21.15.19:0/1696207260 conn(0x7f16040fef30 msgr2=0x7f1604101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.277+0000 7f160b800700 1 -- 172.21.15.19:0/1696207260 shutdown_connections 2024-05-07T02:30:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.277+0000 7f160b800700 1 -- 172.21.15.19:0/1696207260 wait complete. 2024-05-07T02:30:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.279+0000 7f160b800700 1 Processor -- start 2024-05-07T02:30:02.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.279+0000 7f160b800700 1 -- start start 2024-05-07T02:30:02.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.280+0000 7f160b800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1604103270 0x7f16041a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:02.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.280+0000 7f160b800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1604103bc0 0x7f16041a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:02.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.280+0000 7f160b800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16041086c0 0x7f16041a5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:02.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.280+0000 7f160b800700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f160411aa90 con 0x7f1604103270 2024-05-07T02:30:02.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.280+0000 7f160b800700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f160411a910 con 0x7f1604103bc0 2024-05-07T02:30:02.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.280+0000 7f160b800700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f160411ac10 con 0x7f16041086c0 2024-05-07T02:30:02.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.280+0000 7f1608e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1604103270 0x7f16041a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:02.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.280+0000 7f1608e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1604103270 0x7f16041a13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53218/0 (socket says 172.21.15.19:53218) 2024-05-07T02:30:02.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.280+0000 7f1608e00700 1 -- 172.21.15.19:0/1483146391 learned_addr learned my addr 172.21.15.19:0/1483146391 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:30:02.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.281+0000 7f1603e00700 1 --2- 172.21.15.19:0/1483146391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1604103bc0 0x7f16041a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:02.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.281+0000 7f1609800700 1 --2- 172.21.15.19:0/1483146391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16041086c0 0x7f16041a5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:02.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.281+0000 7f1608e00700 1 -- 172.21.15.19:0/1483146391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16041086c0 msgr2=0x7f16041a5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:02.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.281+0000 7f1608e00700 1 --2- 172.21.15.19:0/1483146391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16041086c0 0x7f16041a5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:02.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.281+0000 7f1608e00700 1 -- 172.21.15.19:0/1483146391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1604103bc0 msgr2=0x7f16041a1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:02.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.281+0000 7f1608e00700 1 --2- 172.21.15.19:0/1483146391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1604103bc0 0x7f16041a1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:02.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.281+0000 7f1608e00700 1 -- 172.21.15.19:0/1483146391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16041a62b0 con 0x7f1604103270 2024-05-07T02:30:02.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.281+0000 7f1603e00700 1 --2- 172.21.15.19:0/1483146391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1604103bc0 0x7f16041a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:30:02.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.281+0000 7f1609800700 1 --2- 172.21.15.19:0/1483146391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16041086c0 0x7f16041a5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:30:02.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.281+0000 7f1608e00700 1 --2- 172.21.15.19:0/1483146391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1604103270 0x7f16041a13e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f15f800e9a0 tx=0x7f15f800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:02.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.282+0000 7f1601600700 1 -- 172.21.15.19:0/1483146391 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f800ca10 con 0x7f1604103270 2024-05-07T02:30:02.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.282+0000 7f160b800700 1 -- 172.21.15.19:0/1483146391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16041a65a0 con 0x7f1604103270 2024-05-07T02:30:02.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.282+0000 7f1601600700 1 -- 172.21.15.19:0/1483146391 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15f800cbb0 con 0x7f1604103270 2024-05-07T02:30:02.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.282+0000 7f1601600700 1 -- 172.21.15.19:0/1483146391 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f8018680 con 0x7f1604103270 2024-05-07T02:30:02.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.282+0000 7f160b800700 1 -- 172.21.15.19:0/1483146391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16041adec0 con 0x7f1604103270 2024-05-07T02:30:02.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.283+0000 7f160b800700 1 -- 172.21.15.19:0/1483146391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16040512c0 con 0x7f1604103270 2024-05-07T02:30:02.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.286+0000 7f1601600700 1 -- 172.21.15.19:0/1483146391 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f15f8010430 con 0x7f1604103270 2024-05-07T02:30:02.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.288+0000 7f1601600700 1 --2- 172.21.15.19:0/1483146391 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f15ec0b2880 0x7f15ec0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:02.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.288+0000 7f1601600700 1 -- 172.21.15.19:0/1483146391 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f15f80e2b00 con 0x7f1604103270 2024-05-07T02:30:02.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.288+0000 7f1603e00700 1 --2- 172.21.15.19:0/1483146391 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f15ec0b2880 0x7f15ec0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:02.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.289+0000 7f1603e00700 1 --2- 172.21.15.19:0/1483146391 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f15ec0b2880 0x7f15ec0b4d90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1604100b20 tx=0x7f15f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:02.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.290+0000 7f1601600700 1 -- 172.21.15.19:0/1483146391 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f15f80aa410 con 0x7f1604103270 2024-05-07T02:30:02.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.578+0000 7f160b800700 1 -- 172.21.15.19:0/1483146391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16041a2740 con 0x7f1604103270 2024-05-07T02:30:02.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.580+0000 7f1601600700 1 -- 172.21.15.19:0/1483146391 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11924 (secure 0 0 0) 0x7f16041a2740 con 0x7f1604103270 2024-05-07T02:30:02.580 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:30:02.580 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1763670446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1763670446},{"type":"v1","addr":"172.21.15.19:6839","nonce":1763670446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/477529985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":477529985},{"type":"v1","addr":"172.21.15.64:6833","nonce":477529985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-07T02:26:14.960465+0000","modified":"2024-05-07T02:27:48.914804+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24539,"mds_2":24520,"mds_3":24515,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6833/2171873162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2171873162},{"type":"v1","addr":"172.21.15.27:6833","nonce":2171873162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/2379917839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2379917839},{"type":"v1","addr":"172.21.15.19:6835","nonce":2379917839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6837/2067515538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2067515538},{"type":"v1","addr":"172.21.15.19:6837","nonce":2067515538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6835/1000540600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1000540600},{"type":"v1","addr":"172.21.15.27:6835","nonce":1000540600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/1997318872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1997318872},{"type":"v1","addr":"172.21.15.64:6835","nonce":1997318872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":34,"addr":"172.21.15.64:6837/2585907777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2585907777},{"type":"v1","addr":"172.21.15.64:6837","nonce":2585907777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":33,"addr":"172.21.15.27:6837/2583138143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2583138143},{"type":"v1","addr":"172.21.15.27:6837","nonce":2583138143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":32,"addr":"172.21.15.64:6839/1583120004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":1583120004},{"type":"v1","addr":"172.21.15.64:6839","nonce":1583120004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":29,"addr":"172.21.15.19:6841/3905645246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3905645246},{"type":"v1","addr":"172.21.15.19:6841","nonce":3905645246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6839/3963268634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3963268634},{"type":"v1","addr":"172.21.15.27:6839","nonce":3963268634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24551,24539,24520,24505,24515]},"id":1}]} 2024-05-07T02:30:02.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.583+0000 7f160b800700 1 -- 172.21.15.19:0/1483146391 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f15ec0b2880 msgr2=0x7f15ec0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:02.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.583+0000 7f160b800700 1 --2- 172.21.15.19:0/1483146391 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f15ec0b2880 0x7f15ec0b4d90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1604100b20 tx=0x7f15f0009260 comp rx=0 tx=0).stop 2024-05-07T02:30:02.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.583+0000 7f160b800700 1 -- 172.21.15.19:0/1483146391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1604103270 msgr2=0x7f16041a13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:02.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.583+0000 7f160b800700 1 --2- 172.21.15.19:0/1483146391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1604103270 0x7f16041a13e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f15f800e9a0 tx=0x7f15f800ecb0 comp rx=0 tx=0).stop 2024-05-07T02:30:02.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.583+0000 7f160b800700 1 -- 172.21.15.19:0/1483146391 shutdown_connections 2024-05-07T02:30:02.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.583+0000 7f160b800700 1 --2- 172.21.15.19:0/1483146391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16041086c0 0x7f16041a5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:02.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.583+0000 7f160b800700 1 --2- 172.21.15.19:0/1483146391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1604103270 0x7f16041a13e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:02.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.583+0000 7f160b800700 1 --2- 172.21.15.19:0/1483146391 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f15ec0b2880 0x7f15ec0b4d90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:02.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.583+0000 7f160b800700 1 --2- 172.21.15.19:0/1483146391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1604103bc0 0x7f16041a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:02.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.583+0000 7f160b800700 1 -- 172.21.15.19:0/1483146391 >> 172.21.15.19:0/1483146391 conn(0x7f16040fef30 msgr2=0x7f1604106020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:02.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.584+0000 7f160b800700 1 -- 172.21.15.19:0/1483146391 shutdown_connections 2024-05-07T02:30:02.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:02.584+0000 7f160b800700 1 -- 172.21.15.19:0/1483146391 wait complete. 2024-05-07T02:30:02.585 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 30 2024-05-07T02:30:02.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:30:02 smithi019 ceph-mon[29781]: pgmap v454: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:30:02.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:30:02 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/1483146391' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:30:03.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:30:02 smithi064 ceph-mon[34699]: pgmap v454: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:30:03.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:30:02 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/1483146391' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:30:03.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:30:02 smithi027 ceph-mon[34398]: pgmap v454: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:30:03.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:30:02 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/1483146391' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:30:03.579 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-07T02:30:04.042 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:30:05.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:30:04 smithi064 ceph-mon[34699]: pgmap v455: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:30:05.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:30:04 smithi019 ceph-mon[29781]: pgmap v455: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:30:05.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:30:04 smithi027 ceph-mon[34398]: pgmap v455: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:30:06.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.510+0000 7f767d200700 1 -- 172.21.15.19:0/4105884593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f767810b2e0 msgr2=0x7f7678117810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:06.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.510+0000 7f767d200700 1 --2- 172.21.15.19:0/4105884593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f767810b2e0 0x7f7678117810 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f767000b0a0 tx=0x7f767000b3b0 comp rx=0 tx=0).stop 2024-05-07T02:30:06.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.510+0000 7f767d200700 1 -- 172.21.15.19:0/4105884593 shutdown_connections 2024-05-07T02:30:06.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.510+0000 7f767d200700 1 --2- 172.21.15.19:0/4105884593 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f767810b2e0 0x7f7678117810 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:06.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.510+0000 7f767d200700 1 --2- 172.21.15.19:0/4105884593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7678110970 0x7f7678110d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:06.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.510+0000 7f767d200700 1 --2- 172.21.15.19:0/4105884593 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f767810a990 0x7f767810ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:06.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.510+0000 7f767d200700 1 -- 172.21.15.19:0/4105884593 >> 172.21.15.19:0/4105884593 conn(0x7f7678077c80 msgr2=0x7f76780780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:06.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.510+0000 7f767d200700 1 -- 172.21.15.19:0/4105884593 shutdown_connections 2024-05-07T02:30:06.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.511+0000 7f767d200700 1 -- 172.21.15.19:0/4105884593 wait complete. 2024-05-07T02:30:06.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.512+0000 7f767d200700 1 Processor -- start 2024-05-07T02:30:06.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.512+0000 7f767d200700 1 -- start start 2024-05-07T02:30:06.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.513+0000 7f767d200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f767810a990 0x7f76781a13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:06.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.513+0000 7f767d200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f767810b2e0 0x7f76781a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:06.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.513+0000 7f767d200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7678110970 0x7f76781a5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:06.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.513+0000 7f767d200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f767811aa90 con 0x7f767810b2e0 2024-05-07T02:30:06.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.513+0000 7f767d200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f767811a910 con 0x7f767810a990 2024-05-07T02:30:06.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.513+0000 7f767d200700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f767811ac10 con 0x7f7678110970 2024-05-07T02:30:06.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.513+0000 7f7675e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f767810a990 0x7f76781a13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:06.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.513+0000 7f7675400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f767810b2e0 0x7f76781a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:06.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.513+0000 7f7676800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7678110970 0x7f76781a5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:06.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.513+0000 7f7675e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f767810a990 0x7f76781a13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:35962/0 (socket says 172.21.15.19:35962) 2024-05-07T02:30:06.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.513+0000 7f7676800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7678110970 0x7f76781a5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:44834/0 (socket says 172.21.15.19:44834) 2024-05-07T02:30:06.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.513+0000 7f7675e00700 1 -- 172.21.15.19:0/2623200663 learned_addr learned my addr 172.21.15.19:0/2623200663 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:30:06.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.514+0000 7f7675e00700 1 -- 172.21.15.19:0/2623200663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7678110970 msgr2=0x7f76781a5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:06.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.514+0000 7f7675e00700 1 --2- 172.21.15.19:0/2623200663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7678110970 0x7f76781a5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:06.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.514+0000 7f7675e00700 1 -- 172.21.15.19:0/2623200663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f767810b2e0 msgr2=0x7f76781a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:06.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.514+0000 7f7675e00700 1 --2- 172.21.15.19:0/2623200663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f767810b2e0 0x7f76781a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:06.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.514+0000 7f7675e00700 1 -- 172.21.15.19:0/2623200663 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76781a6270 con 0x7f767810a990 2024-05-07T02:30:06.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.514+0000 7f7675e00700 1 --2- 172.21.15.19:0/2623200663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f767810a990 0x7f76781a13a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f766800e970 tx=0x7f766800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:06.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.514+0000 7f766ea00700 1 -- 172.21.15.19:0/2623200663 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7668004a30 con 0x7f767810a990 2024-05-07T02:30:06.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.514+0000 7f766ea00700 1 -- 172.21.15.19:0/2623200663 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f766800cc90 con 0x7f767810a990 2024-05-07T02:30:06.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.514+0000 7f766ea00700 1 -- 172.21.15.19:0/2623200663 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7668005610 con 0x7f767810a990 2024-05-07T02:30:06.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.514+0000 7f767d200700 1 -- 172.21.15.19:0/2623200663 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76781a6560 con 0x7f767810a990 2024-05-07T02:30:06.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.515+0000 7f767d200700 1 -- 172.21.15.19:0/2623200663 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76781ade80 con 0x7f767810a990 2024-05-07T02:30:06.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.516+0000 7f767d200700 1 -- 172.21.15.19:0/2623200663 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76780512c0 con 0x7f767810a990 2024-05-07T02:30:06.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.519+0000 7f766ea00700 1 -- 172.21.15.19:0/2623200663 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f766800c7e0 con 0x7f767810a990 2024-05-07T02:30:06.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.520+0000 7f766ea00700 1 --2- 172.21.15.19:0/2623200663 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f765c0b2900 0x7f765c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:06.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.520+0000 7f766ea00700 1 -- 172.21.15.19:0/2623200663 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f76680e28f0 con 0x7f767810a990 2024-05-07T02:30:06.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.520+0000 7f7675400700 1 --2- 172.21.15.19:0/2623200663 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f765c0b2900 0x7f765c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:06.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.521+0000 7f7675400700 1 --2- 172.21.15.19:0/2623200663 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f765c0b2900 0x7f765c0b4e10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f76781097a0 tx=0x7f7664009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:06.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.522+0000 7f766ea00700 1 -- 172.21.15.19:0/2623200663 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7668054df0 con 0x7f767810a990 2024-05-07T02:30:06.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.817+0000 7f767d200700 1 -- 172.21.15.19:0/2623200663 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76781a2700 con 0x7f767810a990 2024-05-07T02:30:06.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.819+0000 7f766ea00700 1 -- 172.21.15.19:0/2623200663 <== mon.2 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11924 (secure 0 0 0) 0x7f76680aa200 con 0x7f767810a990 2024-05-07T02:30:06.819 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:30:06.819 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1763670446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1763670446},{"type":"v1","addr":"172.21.15.19:6839","nonce":1763670446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/477529985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":477529985},{"type":"v1","addr":"172.21.15.64:6833","nonce":477529985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-07T02:26:14.960465+0000","modified":"2024-05-07T02:27:48.914804+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24539,"mds_2":24520,"mds_3":24515,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6833/2171873162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2171873162},{"type":"v1","addr":"172.21.15.27:6833","nonce":2171873162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/2379917839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2379917839},{"type":"v1","addr":"172.21.15.19:6835","nonce":2379917839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6837/2067515538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2067515538},{"type":"v1","addr":"172.21.15.19:6837","nonce":2067515538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6835/1000540600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1000540600},{"type":"v1","addr":"172.21.15.27:6835","nonce":1000540600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/1997318872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1997318872},{"type":"v1","addr":"172.21.15.64:6835","nonce":1997318872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":34,"addr":"172.21.15.64:6837/2585907777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2585907777},{"type":"v1","addr":"172.21.15.64:6837","nonce":2585907777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":33,"addr":"172.21.15.27:6837/2583138143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2583138143},{"type":"v1","addr":"172.21.15.27:6837","nonce":2583138143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":32,"addr":"172.21.15.64:6839/1583120004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":1583120004},{"type":"v1","addr":"172.21.15.64:6839","nonce":1583120004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":29,"addr":"172.21.15.19:6841/3905645246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3905645246},{"type":"v1","addr":"172.21.15.19:6841","nonce":3905645246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6839/3963268634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3963268634},{"type":"v1","addr":"172.21.15.27:6839","nonce":3963268634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24551,24539,24520,24505,24515]},"id":1}]} 2024-05-07T02:30:06.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.823+0000 7f767d200700 1 -- 172.21.15.19:0/2623200663 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f765c0b2900 msgr2=0x7f765c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:06.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.823+0000 7f767d200700 1 --2- 172.21.15.19:0/2623200663 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f765c0b2900 0x7f765c0b4e10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f76781097a0 tx=0x7f7664009260 comp rx=0 tx=0).stop 2024-05-07T02:30:06.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.823+0000 7f767d200700 1 -- 172.21.15.19:0/2623200663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f767810a990 msgr2=0x7f76781a13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:06.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.823+0000 7f767d200700 1 --2- 172.21.15.19:0/2623200663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f767810a990 0x7f76781a13a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f766800e970 tx=0x7f766800ec80 comp rx=0 tx=0).stop 2024-05-07T02:30:06.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.823+0000 7f767d200700 1 -- 172.21.15.19:0/2623200663 shutdown_connections 2024-05-07T02:30:06.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.823+0000 7f767d200700 1 --2- 172.21.15.19:0/2623200663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7678110970 0x7f76781a5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:06.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.823+0000 7f767d200700 1 --2- 172.21.15.19:0/2623200663 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f765c0b2900 0x7f765c0b4e10 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:06.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.823+0000 7f767d200700 1 --2- 172.21.15.19:0/2623200663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f767810a990 0x7f76781a13a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:06.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.823+0000 7f767d200700 1 --2- 172.21.15.19:0/2623200663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f767810b2e0 0x7f76781a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:06.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.823+0000 7f767d200700 1 -- 172.21.15.19:0/2623200663 >> 172.21.15.19:0/2623200663 conn(0x7f7678077c80 msgr2=0x7f767807b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:06.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.823+0000 7f767d200700 1 -- 172.21.15.19:0/2623200663 shutdown_connections 2024-05-07T02:30:06.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:06.823+0000 7f767d200700 1 -- 172.21.15.19:0/2623200663 wait complete. 2024-05-07T02:30:06.825 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 30 2024-05-07T02:30:06.829 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:30:06 smithi019 ceph-mon[29781]: pgmap v456: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:30:07.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:30:06 smithi064 ceph-mon[34699]: pgmap v456: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:30:07.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:30:06 smithi027 ceph-mon[34398]: pgmap v456: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:30:07.728 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-07T02:30:08.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:30:07 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/2623200663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:30:08.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:30:07 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/2623200663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:30:08.203 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:30:08.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:30:07 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/2623200663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:30:09.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:30:08 smithi064 ceph-mon[34699]: pgmap v457: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:30:09.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:30:08 smithi019 ceph-mon[29781]: pgmap v457: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:30:09.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:30:08 smithi027 ceph-mon[34398]: pgmap v457: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:30:10.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:10.773+0000 7f4b8e000700 1 -- 172.21.15.19:0/1697516949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b88076680 msgr2=0x7f4b88110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:10.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:10.773+0000 7f4b8e000700 1 --2- 172.21.15.19:0/1697516949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b88076680 0x7f4b88110470 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4b74009990 tx=0x7f4b74009ca0 comp rx=0 tx=0).stop 2024-05-07T02:30:10.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:10.773+0000 7f4b8e000700 1 -- 172.21.15.19:0/1697516949 shutdown_connections 2024-05-07T02:30:10.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:10.773+0000 7f4b8e000700 1 --2- 172.21.15.19:0/1697516949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b881109b0 0x7f4b88112e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:10.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:10.773+0000 7f4b8e000700 1 --2- 172.21.15.19:0/1697516949 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b88075d30 0x7f4b88076140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:10.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:10.773+0000 7f4b8e000700 1 --2- 172.21.15.19:0/1697516949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b88076680 0x7f4b88110470 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:10.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:10.773+0000 7f4b8e000700 1 -- 172.21.15.19:0/1697516949 >> 172.21.15.19:0/1697516949 conn(0x7f4b88079d10 msgr2=0x7f4b8807a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:10.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:10.773+0000 7f4b8e000700 1 -- 172.21.15.19:0/1697516949 shutdown_connections 2024-05-07T02:30:10.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:10.773+0000 7f4b8e000700 1 -- 172.21.15.19:0/1697516949 wait complete. 2024-05-07T02:30:10.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:10.775+0000 7f4b8e000700 1 Processor -- start 2024-05-07T02:30:10.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:10.775+0000 7f4b8e000700 1 -- start start 2024-05-07T02:30:10.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:10.776+0000 7f4b8e000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b88075d30 0x7f4b8819ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:10.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:10.776+0000 7f4b8e000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b88076680 0x7f4b8819d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:10.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:10.776+0000 7f4b8e000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b881109b0 0x7f4b881a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:10.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:10.776+0000 7f4b8e000700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4b881164b0 con 0x7f4b88075d30 2024-05-07T02:30:10.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:10.776+0000 7f4b8e000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4b88116330 con 0x7f4b881109b0 2024-05-07T02:30:10.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:10.776+0000 7f4b8e000700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4b88116630 con 0x7f4b88076680 2024-05-07T02:30:10.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:10.776+0000 7f4b8d600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b881109b0 0x7f4b881a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:10.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:10.776+0000 7f4b8d600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b881109b0 0x7f4b881a1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:57268/0 (socket says 172.21.15.19:57268) 2024-05-07T02:30:10.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:10.776+0000 7f4b87e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b88076680 0x7f4b8819d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:10.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:10.776+0000 7f4b8cc00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b88075d30 0x7f4b8819ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:10.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:10.776+0000 7f4b8d600700 1 -- 172.21.15.19:0/3488848254 learned_addr learned my addr 172.21.15.19:0/3488848254 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:30:10.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:10.777+0000 7f4b87e00700 1 -- 172.21.15.19:0/3488848254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b881109b0 msgr2=0x7f4b881a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:10.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:10.777+0000 7f4b87e00700 1 --2- 172.21.15.19:0/3488848254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b881109b0 0x7f4b881a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:10.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:10.777+0000 7f4b87e00700 1 -- 172.21.15.19:0/3488848254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b88075d30 msgr2=0x7f4b8819ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:10.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:10.777+0000 7f4b87e00700 1 --2- 172.21.15.19:0/3488848254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b88075d30 0x7f4b8819ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:10.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:10.777+0000 7f4b87e00700 1 -- 172.21.15.19:0/3488848254 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b881a1d30 con 0x7f4b88076680 2024-05-07T02:30:10.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:10.777+0000 7f4b87e00700 1 --2- 172.21.15.19:0/3488848254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b88076680 0x7f4b8819d400 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4b740059d0 tx=0x7f4b740047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:10.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:10.777+0000 7f4b85600700 1 -- 172.21.15.19:0/3488848254 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b74027070 con 0x7f4b88076680 2024-05-07T02:30:10.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:10.778+0000 7f4b8e000700 1 -- 172.21.15.19:0/3488848254 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b881a1fc0 con 0x7f4b88076680 2024-05-07T02:30:10.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:10.778+0000 7f4b85600700 1 -- 172.21.15.19:0/3488848254 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b74022470 con 0x7f4b88076680 2024-05-07T02:30:10.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:10.778+0000 7f4b85600700 1 -- 172.21.15.19:0/3488848254 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b74017650 con 0x7f4b88076680 2024-05-07T02:30:10.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:10.778+0000 7f4b8e000700 1 -- 172.21.15.19:0/3488848254 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b881a24c0 con 0x7f4b88076680 2024-05-07T02:30:10.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:10.779+0000 7f4b8e000700 1 -- 172.21.15.19:0/3488848254 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b880772a0 con 0x7f4b88076680 2024-05-07T02:30:10.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:10.782+0000 7f4b85600700 1 -- 172.21.15.19:0/3488848254 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4b7402a070 con 0x7f4b88076680 2024-05-07T02:30:10.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:10.783+0000 7f4b85600700 1 --2- 172.21.15.19:0/3488848254 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f4b700b2880 0x7f4b700b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:10.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:10.784+0000 7f4b85600700 1 -- 172.21.15.19:0/3488848254 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f4b740e3d00 con 0x7f4b88076680 2024-05-07T02:30:10.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:10.784+0000 7f4b8cc00700 1 --2- 172.21.15.19:0/3488848254 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f4b700b2880 0x7f4b700b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:10.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:10.785+0000 7f4b8cc00700 1 --2- 172.21.15.19:0/3488848254 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f4b700b2880 0x7f4b700b4d90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c009c60 tx=0x7f4b7c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:10.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:10.785+0000 7f4b85600700 1 -- 172.21.15.19:0/3488848254 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4b74031070 con 0x7f4b88076680 2024-05-07T02:30:11.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:11.026+0000 7f4b8e000700 1 -- 172.21.15.19:0/3488848254 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4b8819e1c0 con 0x7f4b88076680 2024-05-07T02:30:11.027 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:30:10 smithi019 ceph-mon[29781]: pgmap v458: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:30:11.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:11.029+0000 7f4b85600700 1 -- 172.21.15.19:0/3488848254 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24723 (secure 0 0 0) 0x7f4b881a24c0 con 0x7f4b88076680 2024-05-07T02:30:11.029 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:30:11.030 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":89,"fsid":"9ff84c16-0c17-11ef-bc97-c7b262605968","created":"2024-05-07T02:16:19.507657+0000","modified":"2024-05-07T02:28:33.595951+0000","last_up_change":"2024-05-07T02:24:55.025124+0000","last_in_change":"2024-05-07T02:24:33.842422+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-07T02:20:21.439697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-07T02:26:06.178814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-07T02:26:10.705693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-07T02:26:23.534133+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b437cf22-ec8e-48f0-86a2-2413a0ab40dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6803","nonce":2390539683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6805","nonce":2390539683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6809","nonce":2390539683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6807","nonce":2390539683}]},"public_addr":"172.21.15.19:6803/2390539683","cluster_addr":"172.21.15.19:6805/2390539683","heartbeat_back_addr":"172.21.15.19:6809/2390539683","heartbeat_front_addr":"172.21.15.19:6807/2390539683","state":["exists","up"]},{"osd":1,"uuid":"dc753e16-634d-4c14-abd2-935281e1499e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6801","nonce":4292470615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6803","nonce":4292470615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6807","nonce":4292470615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6805","nonce":4292470615}]},"public_addr":"172.21.15.27:6801/4292470615","cluster_addr":"172.21.15.27:6803/4292470615","heartbeat_back_addr":"172.21.15.27:6807/4292470615","heartbeat_front_addr":"172.21.15.27:6805/4292470615","state":["exists","up"]},{"osd":2,"uuid":"40570071-646e-4e66-9a16-c55ab743ac86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6801","nonce":1180552939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6803","nonce":1180552939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6807","nonce":1180552939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6805","nonce":1180552939}]},"public_addr":"172.21.15.64:6801/1180552939","cluster_addr":"172.21.15.64:6803/1180552939","heartbeat_back_addr":"172.21.15.64:6807/1180552939","heartbeat_front_addr":"172.21.15.64:6805/1180552939","state":["exists","up"]},{"osd":3,"uuid":"ceaa4252-4dfa-427a-9b1a-76adf3b8e609","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6811","nonce":387118141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6813","nonce":387118141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6817","nonce":387118141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6815","nonce":387118141}]},"public_addr":"172.21.15.19:6811/387118141","cluster_addr":"172.21.15.19:6813/387118141","heartbeat_back_addr":"172.21.15.19:6817/387118141","heartbeat_front_addr":"172.21.15.19:6815/387118141","state":["exists","up"]},{"osd":4,"uuid":"5a6d1ebc-62ca-444e-8f29-1980418d0a8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6809","nonce":3662186068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6811","nonce":3662186068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6815","nonce":3662186068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6813","nonce":3662186068}]},"public_addr":"172.21.15.27:6809/3662186068","cluster_addr":"172.21.15.27:6811/3662186068","heartbeat_back_addr":"172.21.15.27:6815/3662186068","heartbeat_front_addr":"172.21.15.27:6813/3662186068","state":["exists","up"]},{"osd":5,"uuid":"60c207e9-55e8-4bed-87c3-f6a06a2a1fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6809","nonce":1264591849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6811","nonce":1264591849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6815","nonce":1264591849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6813","nonce":1264591849}]},"public_addr":"172.21.15.64:6809/1264591849","cluster_addr":"172.21.15.64:6811/1264591849","heartbeat_back_addr":"172.21.15.64:6815/1264591849","heartbeat_front_addr":"172.21.15.64:6813/1264591849","state":["exists","up"]},{"osd":6,"uuid":"f3e038e0-4026-45f1-a923-c07cbb307f64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6819","nonce":2158875543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6821","nonce":2158875543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6825","nonce":2158875543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6823","nonce":2158875543}]},"public_addr":"172.21.15.19:6819/2158875543","cluster_addr":"172.21.15.19:6821/2158875543","heartbeat_back_addr":"172.21.15.19:6825/2158875543","heartbeat_front_addr":"172.21.15.19:6823/2158875543","state":["exists","up"]},{"osd":7,"uuid":"d69e34d4-2028-43b8-8ada-ff4a9b579620","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6817","nonce":2756612091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6819","nonce":2756612091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6823","nonce":2756612091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6821","nonce":2756612091}]},"public_addr":"172.21.15.27:6817/2756612091","cluster_addr":"172.21.15.27:6819/2756612091","heartbeat_back_addr":"172.21.15.27:6823/2756612091","heartbeat_front_addr":"172.21.15.27:6821/2756612091","state":["exists","up"]},{"osd":8,"uuid":"6c1a4179-75b6-42ff-ac29-378801e6f8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6817","nonce":1987024836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6819","nonce":1987024836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6823","nonce":1987024836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6821","nonce":1987024836}]},"public_addr":"172.21.15.64:6817/1987024836","cluster_addr":"172.21.15.64:6819/1987024836","heartbeat_back_addr":"172.21.15.64:6823/1987024836","heartbeat_front_addr":"172.21.15.64:6821/1987024836","state":["exists","up"]},{"osd":9,"uuid":"9df7ff66-d39f-4709-8324-74cf9e0e052d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6827","nonce":4124921646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6829","nonce":4124921646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6833","nonce":4124921646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6831","nonce":4124921646}]},"public_addr":"172.21.15.19:6827/4124921646","cluster_addr":"172.21.15.19:6829/4124921646","heartbeat_back_addr":"172.21.15.19:6833/4124921646","heartbeat_front_addr":"172.21.15.19:6831/4124921646","state":["exists","up"]},{"osd":10,"uuid":"71eaa9a7-667d-4cb7-935c-42177d20a279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6825","nonce":1428931817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6827","nonce":1428931817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6831","nonce":1428931817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6829","nonce":1428931817}]},"public_addr":"172.21.15.27:6825/1428931817","cluster_addr":"172.21.15.27:6827/1428931817","heartbeat_back_addr":"172.21.15.27:6831/1428931817","heartbeat_front_addr":"172.21.15.27:6829/1428931817","state":["exists","up"]},{"osd":11,"uuid":"3330627f-ab5d-4e6e-83cf-fc8b74c54ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6825","nonce":4045070679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6827","nonce":4045070679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6831","nonce":4045070679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6829","nonce":4045070679}]},"public_addr":"172.21.15.64:6825/4045070679","cluster_addr":"172.21.15.64:6827/4045070679","heartbeat_back_addr":"172.21.15.64:6831/4045070679","heartbeat_front_addr":"172.21.15.64:6829/4045070679","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:19:27.902633+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:19:54.305685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:20:18.727053+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:20:53.579199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:21:21.058018+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:21:47.091494+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:22:23.269018+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:22:51.863628+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:23:18.537774+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:23:55.763993+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:24:24.956178+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:24:52.681201+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/1956516258":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/1816772181":"2024-05-08T02:17:31.252868+0000","172.21.15.19:6801/174304059":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/1514051750":"2024-05-08T02:16:54.732592+0000","172.21.15.19:6800/174304059":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/3930045871":"2024-05-08T02:16:54.732592+0000","172.21.15.19:6800/3614957309":"2024-05-08T02:16:54.732592+0000","172.21.15.19:0/1328365399":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/139593344":"2024-05-08T02:16:54.732592+0000","172.21.15.19:6801/3614957309":"2024-05-08T02:16:54.732592+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-07T02:30:11.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:11.032+0000 7f4b8e000700 1 -- 172.21.15.19:0/3488848254 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f4b700b2880 msgr2=0x7f4b700b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:11.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:11.032+0000 7f4b8e000700 1 --2- 172.21.15.19:0/3488848254 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f4b700b2880 0x7f4b700b4d90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c009c60 tx=0x7f4b7c009260 comp rx=0 tx=0).stop 2024-05-07T02:30:11.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:11.032+0000 7f4b8e000700 1 -- 172.21.15.19:0/3488848254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b88076680 msgr2=0x7f4b8819d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:11.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:11.032+0000 7f4b8e000700 1 --2- 172.21.15.19:0/3488848254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b88076680 0x7f4b8819d400 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4b740059d0 tx=0x7f4b740047b0 comp rx=0 tx=0).stop 2024-05-07T02:30:11.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:11.032+0000 7f4b8e000700 1 -- 172.21.15.19:0/3488848254 shutdown_connections 2024-05-07T02:30:11.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:11.032+0000 7f4b8e000700 1 --2- 172.21.15.19:0/3488848254 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f4b700b2880 0x7f4b700b4d90 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:11.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:11.032+0000 7f4b8e000700 1 --2- 172.21.15.19:0/3488848254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b881109b0 0x7f4b881a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:11.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:11.032+0000 7f4b8e000700 1 --2- 172.21.15.19:0/3488848254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b88075d30 0x7f4b8819ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:11.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:11.032+0000 7f4b8e000700 1 --2- 172.21.15.19:0/3488848254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b88076680 0x7f4b8819d400 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:11.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:11.032+0000 7f4b8e000700 1 -- 172.21.15.19:0/3488848254 >> 172.21.15.19:0/3488848254 conn(0x7f4b88079d10 msgr2=0x7f4b88100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:11.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:11.033+0000 7f4b8e000700 1 -- 172.21.15.19:0/3488848254 shutdown_connections 2024-05-07T02:30:11.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:11.033+0000 7f4b8e000700 1 -- 172.21.15.19:0/3488848254 wait complete. 2024-05-07T02:30:11.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:30:10 smithi064 ceph-mon[34699]: pgmap v458: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:30:11.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:30:10 smithi027 ceph-mon[34398]: pgmap v458: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:30:12.016 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-07T02:30:12.092 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:30:11 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/3488848254' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-07T02:30:12.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:30:11 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/3488848254' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-07T02:30:12.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:30:11 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/3488848254' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-07T02:30:12.477 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:30:12.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:30:12 smithi019 ceph-mon[29781]: pgmap v459: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:30:13.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:30:12 smithi064 ceph-mon[34699]: pgmap v459: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:30:13.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:30:12 smithi027 ceph-mon[34398]: pgmap v459: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:30:14.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:14.854+0000 7efcfec00700 1 -- 172.21.15.19:0/2191264599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcf8110970 msgr2=0x7efcf8110d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:14.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:14.854+0000 7efcfec00700 1 --2- 172.21.15.19:0/2191264599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcf8110970 0x7efcf8110d80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efce8009990 tx=0x7efce8009ca0 comp rx=0 tx=0).stop 2024-05-07T02:30:14.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:14.855+0000 7efcfec00700 1 -- 172.21.15.19:0/2191264599 shutdown_connections 2024-05-07T02:30:14.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:14.855+0000 7efcfec00700 1 --2- 172.21.15.19:0/2191264599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcf810b2e0 0x7efcf8117810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:14.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:14.855+0000 7efcfec00700 1 --2- 172.21.15.19:0/2191264599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcf8110970 0x7efcf8110d80 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:14.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:14.855+0000 7efcfec00700 1 --2- 172.21.15.19:0/2191264599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcf810a990 0x7efcf810ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:14.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:14.855+0000 7efcfec00700 1 -- 172.21.15.19:0/2191264599 >> 172.21.15.19:0/2191264599 conn(0x7efcf8077c80 msgr2=0x7efcf80780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:14.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:14.855+0000 7efcfec00700 1 -- 172.21.15.19:0/2191264599 shutdown_connections 2024-05-07T02:30:14.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:14.855+0000 7efcfec00700 1 -- 172.21.15.19:0/2191264599 wait complete. 2024-05-07T02:30:14.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:14.856+0000 7efcfec00700 1 Processor -- start 2024-05-07T02:30:14.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:14.857+0000 7efcfec00700 1 -- start start 2024-05-07T02:30:14.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:14.857+0000 7efcfec00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcf810a990 0x7efcf81a1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:14.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:14.857+0000 7efcfec00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcf810b2e0 0x7efcf81a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:14.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:14.857+0000 7efcfec00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcf8110970 0x7efcf81a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:14.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:14.857+0000 7efcfec00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efcf811aa70 con 0x7efcf810a990 2024-05-07T02:30:14.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:14.857+0000 7efcfec00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efcf811a8f0 con 0x7efcf8110970 2024-05-07T02:30:14.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:14.857+0000 7efcfec00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7efcf811abf0 con 0x7efcf810b2e0 2024-05-07T02:30:14.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:14.857+0000 7efcf7e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcf810a990 0x7efcf81a1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:14.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:14.858+0000 7efcf7400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcf810b2e0 0x7efcf81a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:14.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:14.858+0000 7efcf7e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcf810a990 0x7efcf81a1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57928/0 (socket says 172.21.15.19:57928) 2024-05-07T02:30:14.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:14.858+0000 7efcf7e00700 1 -- 172.21.15.19:0/113232594 learned_addr learned my addr 172.21.15.19:0/113232594 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:30:14.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:14.858+0000 7efcfcc00700 1 --2- 172.21.15.19:0/113232594 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcf8110970 0x7efcf81a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:14.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:14.858+0000 7efcf7400700 1 -- 172.21.15.19:0/113232594 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcf8110970 msgr2=0x7efcf81a5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:14.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:14.858+0000 7efcf7400700 1 --2- 172.21.15.19:0/113232594 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcf8110970 0x7efcf81a5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:14.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:14.858+0000 7efcf7400700 1 -- 172.21.15.19:0/113232594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcf810a990 msgr2=0x7efcf81a1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:14.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:14.858+0000 7efcf7400700 1 --2- 172.21.15.19:0/113232594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcf810a990 0x7efcf81a1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:14.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:14.858+0000 7efcf7400700 1 -- 172.21.15.19:0/113232594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcf81a6310 con 0x7efcf810b2e0 2024-05-07T02:30:14.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:14.858+0000 7efcf7e00700 1 --2- 172.21.15.19:0/113232594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcf810a990 0x7efcf81a1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:30:14.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:14.858+0000 7efcfcc00700 1 --2- 172.21.15.19:0/113232594 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcf8110970 0x7efcf81a5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:30:14.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:14.858+0000 7efcf7400700 1 --2- 172.21.15.19:0/113232594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcf810b2e0 0x7efcf81a19e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7efce400e9c0 tx=0x7efce400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:14.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:14.859+0000 7efcf4c00700 1 -- 172.21.15.19:0/113232594 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce400c9c0 con 0x7efcf810b2e0 2024-05-07T02:30:14.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:14.859+0000 7efcf4c00700 1 -- 172.21.15.19:0/113232594 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efce400cb60 con 0x7efcf810b2e0 2024-05-07T02:30:14.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:14.859+0000 7efcfec00700 1 -- 172.21.15.19:0/113232594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcf81a6600 con 0x7efcf810b2e0 2024-05-07T02:30:14.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:14.859+0000 7efcf4c00700 1 -- 172.21.15.19:0/113232594 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce4018680 con 0x7efcf810b2e0 2024-05-07T02:30:14.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:14.859+0000 7efcfec00700 1 -- 172.21.15.19:0/113232594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcf81adf20 con 0x7efcf810b2e0 2024-05-07T02:30:14.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:14.860+0000 7efcfec00700 1 -- 172.21.15.19:0/113232594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcf80512c0 con 0x7efcf810b2e0 2024-05-07T02:30:14.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:14.862+0000 7efcf4c00700 1 -- 172.21.15.19:0/113232594 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7efce4010430 con 0x7efcf810b2e0 2024-05-07T02:30:14.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:14.864+0000 7efcf4c00700 1 --2- 172.21.15.19:0/113232594 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7efcdc0b2900 0x7efcdc0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:14.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:14.864+0000 7efcf4c00700 1 -- 172.21.15.19:0/113232594 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7efce40e2700 con 0x7efcf810b2e0 2024-05-07T02:30:14.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:14.864+0000 7efcf7e00700 1 --2- 172.21.15.19:0/113232594 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7efcdc0b2900 0x7efcdc0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:14.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:14.865+0000 7efcf7e00700 1 --2- 172.21.15.19:0/113232594 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7efcdc0b2900 0x7efcdc0b4e10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efce8006010 tx=0x7efce8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:14.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:14.869+0000 7efcf4c00700 1 -- 172.21.15.19:0/113232594 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efce40aa010 con 0x7efcf810b2e0 2024-05-07T02:30:15.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:30:14 smithi064 ceph-mon[34699]: pgmap v460: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:30:15.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:30:14 smithi019 ceph-mon[29781]: pgmap v460: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:30:15.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:15.169+0000 7efcfec00700 1 -- 172.21.15.19:0/113232594 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efcf81a27a0 con 0x7efcf810b2e0 2024-05-07T02:30:15.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:15.171+0000 7efcf4c00700 1 -- 172.21.15.19:0/113232594 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11924 (secure 0 0 0) 0x7efce40a9910 con 0x7efcf810b2e0 2024-05-07T02:30:15.171 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:30:15.171 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1763670446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1763670446},{"type":"v1","addr":"172.21.15.19:6839","nonce":1763670446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/477529985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":477529985},{"type":"v1","addr":"172.21.15.64:6833","nonce":477529985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-07T02:26:14.960465+0000","modified":"2024-05-07T02:27:48.914804+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24539,"mds_2":24520,"mds_3":24515,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6833/2171873162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2171873162},{"type":"v1","addr":"172.21.15.27:6833","nonce":2171873162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/2379917839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2379917839},{"type":"v1","addr":"172.21.15.19:6835","nonce":2379917839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6837/2067515538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2067515538},{"type":"v1","addr":"172.21.15.19:6837","nonce":2067515538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6835/1000540600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1000540600},{"type":"v1","addr":"172.21.15.27:6835","nonce":1000540600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/1997318872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1997318872},{"type":"v1","addr":"172.21.15.64:6835","nonce":1997318872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":34,"addr":"172.21.15.64:6837/2585907777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2585907777},{"type":"v1","addr":"172.21.15.64:6837","nonce":2585907777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":33,"addr":"172.21.15.27:6837/2583138143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2583138143},{"type":"v1","addr":"172.21.15.27:6837","nonce":2583138143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":32,"addr":"172.21.15.64:6839/1583120004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":1583120004},{"type":"v1","addr":"172.21.15.64:6839","nonce":1583120004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":29,"addr":"172.21.15.19:6841/3905645246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3905645246},{"type":"v1","addr":"172.21.15.19:6841","nonce":3905645246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6839/3963268634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3963268634},{"type":"v1","addr":"172.21.15.27:6839","nonce":3963268634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24551,24539,24520,24505,24515]},"id":1}]} 2024-05-07T02:30:15.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:15.174+0000 7efcfec00700 1 -- 172.21.15.19:0/113232594 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7efcdc0b2900 msgr2=0x7efcdc0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:15.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:15.174+0000 7efcfec00700 1 --2- 172.21.15.19:0/113232594 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7efcdc0b2900 0x7efcdc0b4e10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efce8006010 tx=0x7efce8005960 comp rx=0 tx=0).stop 2024-05-07T02:30:15.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:15.174+0000 7efcfec00700 1 -- 172.21.15.19:0/113232594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcf810b2e0 msgr2=0x7efcf81a19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:15.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:15.174+0000 7efcfec00700 1 --2- 172.21.15.19:0/113232594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcf810b2e0 0x7efcf81a19e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7efce400e9c0 tx=0x7efce400ecd0 comp rx=0 tx=0).stop 2024-05-07T02:30:15.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:15.174+0000 7efcfec00700 1 -- 172.21.15.19:0/113232594 shutdown_connections 2024-05-07T02:30:15.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:15.174+0000 7efcfec00700 1 --2- 172.21.15.19:0/113232594 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7efcdc0b2900 0x7efcdc0b4e10 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:15.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:15.174+0000 7efcfec00700 1 --2- 172.21.15.19:0/113232594 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcf8110970 0x7efcf81a5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:15.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:15.174+0000 7efcfec00700 1 --2- 172.21.15.19:0/113232594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcf810a990 0x7efcf81a1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:15.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:15.174+0000 7efcfec00700 1 --2- 172.21.15.19:0/113232594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcf810b2e0 0x7efcf81a19e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:15.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:15.174+0000 7efcfec00700 1 -- 172.21.15.19:0/113232594 >> 172.21.15.19:0/113232594 conn(0x7efcf8077c80 msgr2=0x7efcf807bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:15.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:15.175+0000 7efcfec00700 1 -- 172.21.15.19:0/113232594 shutdown_connections 2024-05-07T02:30:15.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:15.175+0000 7efcfec00700 1 -- 172.21.15.19:0/113232594 wait complete. 2024-05-07T02:30:15.176 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 30 2024-05-07T02:30:15.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:30:14 smithi027 ceph-mon[34398]: pgmap v460: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:30:15.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:30:15 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/113232594' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:30:16.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:30:15 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/113232594' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:30:16.129 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-07T02:26:14.960465+0000', 'modified': '2024-05-07T02:27:48.914804+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24551, 'mds_1': 24539, 'mds_2': 24520, 'mds_3': 24515, 'mds_4': 24505}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24458': {'gid': 24458, 'name': 'k', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.27:6833/2171873162', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6832', 'nonce': 2171873162}, {'type': 'v1', 'addr': '172.21.15.27:6833', 'nonce': 2171873162}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'j', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.19:6835/2379917839', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6834', 'nonce': 2379917839}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 2379917839}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'd', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.19:6837/2067515538', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6836', 'nonce': 2067515538}, {'type': 'v1', 'addr': '172.21.15.19:6837', 'nonce': 2067515538}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24488': {'gid': 24488, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.27:6835/1000540600', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6834', 'nonce': 1000540600}, {'type': 'v1', 'addr': '172.21.15.27:6835', 'nonce': 1000540600}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24499': {'gid': 24499, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.64:6835/1997318872', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6834', 'nonce': 1997318872}, {'type': 'v1', 'addr': '172.21.15.64:6835', 'nonce': 1997318872}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24505': {'gid': 24505, 'name': 'c', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 34, 'addr': '172.21.15.64:6837/2585907777', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6836', 'nonce': 2585907777}, {'type': 'v1', 'addr': '172.21.15.64:6837', 'nonce': 2585907777}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24515': {'gid': 24515, 'name': 'h', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 33, 'addr': '172.21.15.27:6837/2583138143', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6836', 'nonce': 2583138143}, {'type': 'v1', 'addr': '172.21.15.27:6837', 'nonce': 2583138143}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24520': {'gid': 24520, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 32, 'addr': '172.21.15.64:6839/1583120004', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6838', 'nonce': 1583120004}, {'type': 'v1', 'addr': '172.21.15.64:6839', 'nonce': 1583120004}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24539': {'gid': 24539, 'name': 'a', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 29, 'addr': '172.21.15.19:6841/3905645246', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6840', 'nonce': 3905645246}, {'type': 'v1', 'addr': '172.21.15.19:6841', 'nonce': 3905645246}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24551': {'gid': 24551, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.27:6839/3963268634', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6838', 'nonce': 3963268634}, {'type': 'v1', 'addr': '172.21.15.27:6839', 'nonce': 3963268634}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24551, 'qdb_cluster': [24551, 24539, 24520, 24505, 24515]} 2024-05-07T02:30:16.129 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-07T02:30:16.130 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.c status 2024-05-07T02:30:16.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:30:15 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/113232594' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:30:16.614 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:30:17.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:30:16 smithi064 ceph-mon[34699]: pgmap v461: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:30:17.139 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:30:16 smithi019 ceph-mon[29781]: pgmap v461: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:30:17.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:30:16 smithi027 ceph-mon[34398]: pgmap v461: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:30:18.110 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:30:17 smithi064 ceph-mon[34699]: pgmap v462: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:30:18.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:30:17 smithi019 ceph-mon[29781]: pgmap v462: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:30:18.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:30:17 smithi027 ceph-mon[34398]: pgmap v462: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:30:18.985 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:30:18 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:30:19.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.096+0000 7f3c7da00700 1 -- 172.21.15.19:0/1349961120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c78108f00 msgr2=0x7f3c78109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:19.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.096+0000 7f3c7da00700 1 --2- 172.21.15.19:0/1349961120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c78108f00 0x7f3c78109310 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3c64009990 tx=0x7f3c64009ca0 comp rx=0 tx=0).stop 2024-05-07T02:30:19.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.096+0000 7f3c7da00700 1 -- 172.21.15.19:0/1349961120 shutdown_connections 2024-05-07T02:30:19.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.096+0000 7f3c7da00700 1 --2- 172.21.15.19:0/1349961120 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c78109850 0x7f3c781111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:19.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.096+0000 7f3c7da00700 1 --2- 172.21.15.19:0/1349961120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c781045d0 0x7f3c781049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:19.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.096+0000 7f3c7da00700 1 --2- 172.21.15.19:0/1349961120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c78108f00 0x7f3c78109310 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:19.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.096+0000 7f3c7da00700 1 -- 172.21.15.19:0/1349961120 >> 172.21.15.19:0/1349961120 conn(0x7f3c780ffea0 msgr2=0x7f3c781022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:19.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.096+0000 7f3c7da00700 1 -- 172.21.15.19:0/1349961120 shutdown_connections 2024-05-07T02:30:19.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.096+0000 7f3c7da00700 1 -- 172.21.15.19:0/1349961120 wait complete. 2024-05-07T02:30:19.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.098+0000 7f3c7da00700 1 Processor -- start 2024-05-07T02:30:19.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.098+0000 7f3c7da00700 1 -- start start 2024-05-07T02:30:19.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.098+0000 7f3c7da00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c781045d0 0x7f3c78076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:19.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.098+0000 7f3c7da00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c78108f00 0x7f3c78077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:19.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.098+0000 7f3c7da00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c78109850 0x7f3c78077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:19.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.098+0000 7f3c7da00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3c781144b0 con 0x7f3c78108f00 2024-05-07T02:30:19.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.098+0000 7f3c7da00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3c78114330 con 0x7f3c781045d0 2024-05-07T02:30:19.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.098+0000 7f3c7da00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3c78114630 con 0x7f3c78109850 2024-05-07T02:30:19.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.099+0000 7f3c76a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c781045d0 0x7f3c78076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:19.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.099+0000 7f3c76a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c781045d0 0x7f3c78076bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:42890/0 (socket says 172.21.15.19:42890) 2024-05-07T02:30:19.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.099+0000 7f3c76a00700 1 -- 172.21.15.19:0/966138735 learned_addr learned my addr 172.21.15.19:0/966138735 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:30:19.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.099+0000 7f3c76000700 1 --2- 172.21.15.19:0/966138735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c78108f00 0x7f3c78077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:19.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.099+0000 7f3c77400700 1 --2- 172.21.15.19:0/966138735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c78109850 0x7f3c78077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:19.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.099+0000 7f3c76a00700 1 -- 172.21.15.19:0/966138735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c78109850 msgr2=0x7f3c78077650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:19.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.099+0000 7f3c76a00700 1 --2- 172.21.15.19:0/966138735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c78109850 0x7f3c78077650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:19.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.099+0000 7f3c76a00700 1 -- 172.21.15.19:0/966138735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c78108f00 msgr2=0x7f3c78077110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:19.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.099+0000 7f3c76a00700 1 --2- 172.21.15.19:0/966138735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c78108f00 0x7f3c78077110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:19.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.099+0000 7f3c76a00700 1 -- 172.21.15.19:0/966138735 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c781a9210 con 0x7f3c781045d0 2024-05-07T02:30:19.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.099+0000 7f3c76000700 1 --2- 172.21.15.19:0/966138735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c78108f00 0x7f3c78077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:30:19.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.100+0000 7f3c76a00700 1 --2- 172.21.15.19:0/966138735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c781045d0 0x7f3c78076bd0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3c6800e9a0 tx=0x7f3c6800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:19.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.100+0000 7f3c6f400700 1 -- 172.21.15.19:0/966138735 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6800cbd0 con 0x7f3c781045d0 2024-05-07T02:30:19.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.100+0000 7f3c6f400700 1 -- 172.21.15.19:0/966138735 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c6800cd70 con 0x7f3c781045d0 2024-05-07T02:30:19.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.100+0000 7f3c7da00700 1 -- 172.21.15.19:0/966138735 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c781a9500 con 0x7f3c781045d0 2024-05-07T02:30:19.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.100+0000 7f3c6f400700 1 -- 172.21.15.19:0/966138735 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c68018680 con 0x7f3c781045d0 2024-05-07T02:30:19.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.100+0000 7f3c7da00700 1 -- 172.21.15.19:0/966138735 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c781a9a90 con 0x7f3c781045d0 2024-05-07T02:30:19.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.104+0000 7f3c6f400700 1 -- 172.21.15.19:0/966138735 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3c68018820 con 0x7f3c781045d0 2024-05-07T02:30:19.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.105+0000 7f3c8c596b80 1 Processor -- start 2024-05-07T02:30:19.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.105+0000 7f3c8c596b80 1 -- start start 2024-05-07T02:30:19.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.105+0000 7f3c6f400700 1 --2- 172.21.15.19:0/966138735 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f3c5c0b2900 0x7f3c5c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:19.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.106+0000 7f3c6f400700 1 -- 172.21.15.19:0/966138735 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f3c680e2880 con 0x7f3c781045d0 2024-05-07T02:30:19.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.106+0000 7f3c8c596b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555eb4009420 0x555eb4009830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:19.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.106+0000 7f3c8c596b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555eb40035d0 0x555eb40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:19.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.106+0000 7f3c8c596b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555eb4003f20 0x555eb4010100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:19.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.106+0000 7f3c77400700 1 --2- 172.21.15.19:0/966138735 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f3c5c0b2900 0x7f3c5c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:19.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.106+0000 7f3c8c596b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555eb3f259d0 con 0x555eb40035d0 2024-05-07T02:30:19.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.106+0000 7f3c8c596b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x555eb3ec3710 con 0x555eb4003f20 2024-05-07T02:30:19.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.106+0000 7f3c8c596b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555eb3f21990 con 0x555eb4009420 2024-05-07T02:30:19.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.106+0000 7f3c76a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555eb4009420 0x555eb4009830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:19.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.106+0000 7f3c77400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555eb4003f20 0x555eb4010100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:19.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.106+0000 7f3c76a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555eb4009420 0x555eb4009830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:42956/0 (socket says 172.21.15.19:42956) 2024-05-07T02:30:19.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.106+0000 7f3c76a00700 1 -- 172.21.15.19:0/439938989 learned_addr learned my addr 172.21.15.19:0/439938989 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:30:19.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.106+0000 7f3c76000700 1 --2- 172.21.15.19:0/439938989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555eb40035d0 0x555eb40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:19.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.107+0000 7f3c76a00700 1 -- 172.21.15.19:0/439938989 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555eb4003f20 msgr2=0x555eb4010100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:30:19.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.107+0000 7f3c76a00700 1 --2- 172.21.15.19:0/439938989 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555eb4003f20 0x555eb4010100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:19.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.107+0000 7f3c76a00700 1 -- 172.21.15.19:0/439938989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555eb40035d0 msgr2=0x555eb40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:30:19.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.107+0000 7f3c76a00700 1 --2- 172.21.15.19:0/439938989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555eb40035d0 0x555eb40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:19.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.107+0000 7f3c76a00700 1 -- 172.21.15.19:0/439938989 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555eb3e72880 con 0x555eb4009420 2024-05-07T02:30:19.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.107+0000 7f3c77400700 1 --2- 172.21.15.19:0/966138735 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f3c5c0b2900 0x7f3c5c0b4e10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3c70009ec0 tx=0x7f3c70013480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:19.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.107+0000 7f3c76a00700 1 --2- 172.21.15.19:0/439938989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555eb4009420 0x555eb4009830 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3c680a6d00 tx=0x7f3c680a92d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b7cca956dbe84a server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:19.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.108+0000 7f3c63e00700 1 -- 172.21.15.19:0/439938989 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c680a95a0 con 0x555eb4009420 2024-05-07T02:30:19.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.108+0000 7f3c63e00700 1 -- 172.21.15.19:0/439938989 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c680aa190 con 0x555eb4009420 2024-05-07T02:30:19.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.108+0000 7f3c63e00700 1 -- 172.21.15.19:0/439938989 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c680a8cf0 con 0x555eb4009420 2024-05-07T02:30:19.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:30:18 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:30:19.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.109+0000 7f3c8c596b80 1 -- 172.21.15.19:0/439938989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555eb4009420 msgr2=0x555eb4009830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:19.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.109+0000 7f3c8c596b80 1 --2- 172.21.15.19:0/439938989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555eb4009420 0x555eb4009830 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3c680a6d00 tx=0x7f3c680a92d0 comp rx=0 tx=0).stop 2024-05-07T02:30:19.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.109+0000 7f3c8c596b80 1 -- 172.21.15.19:0/439938989 shutdown_connections 2024-05-07T02:30:19.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.109+0000 7f3c8c596b80 1 --2- 172.21.15.19:0/439938989 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555eb4003f20 0x555eb4010100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:19.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.109+0000 7f3c8c596b80 1 --2- 172.21.15.19:0/439938989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555eb4009420 0x555eb4009830 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:19.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.109+0000 7f3c8c596b80 1 --2- 172.21.15.19:0/439938989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555eb40035d0 0x555eb40039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:19.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.109+0000 7f3c8c596b80 1 -- 172.21.15.19:0/439938989 >> 172.21.15.19:0/439938989 conn(0x555eb3e85df0 msgr2=0x555eb3ffe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:19.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.109+0000 7f3c8c596b80 1 -- 172.21.15.19:0/439938989 shutdown_connections 2024-05-07T02:30:19.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.109+0000 7f3c8c596b80 1 -- 172.21.15.19:0/439938989 wait complete. 2024-05-07T02:30:19.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.110+0000 7f3c8c596b80 1 -- start start 2024-05-07T02:30:19.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.110+0000 7f3c8c596b80 1 Processor -- start 2024-05-07T02:30:19.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.111+0000 7f3c8c596b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555eb40035d0 0x555eb400d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:19.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.111+0000 7f3c8c596b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555eb4003f20 0x555eb400dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:19.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.111+0000 7f3c8c596b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555eb4009420 0x555eb400e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:19.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.111+0000 7f3c8c596b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555eb40107f0 con 0x555eb40035d0 2024-05-07T02:30:19.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.111+0000 7f3c8c596b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x555eb3e8bad0 con 0x555eb4003f20 2024-05-07T02:30:19.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.111+0000 7f3c8c596b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555eb4010970 con 0x555eb4009420 2024-05-07T02:30:19.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.111+0000 7f3c8c596b80 10 client.0 _ll_register_callbacks cb 0x555eb3eb9330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:30:19.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.111+0000 7f3c76000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555eb4003f20 0x555eb400dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:19.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.111+0000 7f3c76a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555eb40035d0 0x555eb400d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:19.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.111+0000 7f3c76000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555eb4003f20 0x555eb400dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:42918/0 (socket says 172.21.15.19:42918) 2024-05-07T02:30:19.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.111+0000 7f3c76000700 1 -- 172.21.15.19:0/1517482797 learned_addr learned my addr 172.21.15.19:0/1517482797 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:30:19.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.111+0000 7f3c77400700 1 --2- 172.21.15.19:0/1517482797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555eb4009420 0x555eb400e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:19.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.112+0000 7f3c76000700 1 -- 172.21.15.19:0/1517482797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555eb4009420 msgr2=0x555eb400e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:30:19.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.112+0000 7f3c76000700 1 --2- 172.21.15.19:0/1517482797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555eb4009420 0x555eb400e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:19.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.112+0000 7f3c76000700 1 -- 172.21.15.19:0/1517482797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555eb40035d0 msgr2=0x555eb400d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:30:19.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.112+0000 7f3c76000700 1 --2- 172.21.15.19:0/1517482797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555eb40035d0 0x555eb400d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:19.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.112+0000 7f3c76000700 1 -- 172.21.15.19:0/1517482797 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555eb3eeb3d0 con 0x555eb4003f20 2024-05-07T02:30:19.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.112+0000 7f3c77400700 1 --2- 172.21.15.19:0/1517482797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555eb4009420 0x555eb400e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:30:19.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.112+0000 7f3c76a00700 1 --2- 172.21.15.19:0/1517482797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555eb40035d0 0x555eb400d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:30:19.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.112+0000 7f3c76000700 1 --2- 172.21.15.19:0/1517482797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555eb4003f20 0x555eb400dad0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3c640092e0 tx=0x7f3c64005700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5ea2ee46bb34357 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:19.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.112+0000 7f3c60c00700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:30:19.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.112+0000 7f3c60c00700 1 -- 172.21.15.19:0/1517482797 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6400bdd0 con 0x555eb4003f20 2024-05-07T02:30:19.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.112+0000 7f3c8c596b80 1 -- 172.21.15.19:0/1517482797 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c680aa190 con 0x555eb4003f20 2024-05-07T02:30:19.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.112+0000 7f3c60c00700 1 -- 172.21.15.19:0/1517482797 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f3c64003680 con 0x555eb4003f20 2024-05-07T02:30:19.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.113+0000 7f3c60c00700 1 -- 172.21.15.19:0/1517482797 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3c6400b610 con 0x555eb4003f20 2024-05-07T02:30:19.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.113+0000 7f3c8c596b80 10 client.24995 fetch_fsmap learned FSMap version 30 2024-05-07T02:30:19.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.113+0000 7f3c8c596b80 1 -- 172.21.15.19:0/1517482797 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555eb401b400 con 0x555eb4003f20 2024-05-07T02:30:19.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.113+0000 7f3c60c00700 1 -- 172.21.15.19:0/1517482797 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c680aa190 con 0x555eb4003f20 2024-05-07T02:30:19.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.113+0000 7f3c60c00700 1 -- 172.21.15.19:0/1517482797 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3c6400fa30 con 0x555eb4003f20 2024-05-07T02:30:19.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.113+0000 7f3c8c596b80 10 client.24995 fetch_fsmap finished waiting for FSMap version 30 2024-05-07T02:30:19.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.113+0000 7f3c8c596b80 10 client.24995 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-07T02:30:19.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.113+0000 7f3c8c596b80 20 client.24995 populate_metadata read hostname 'smithi019' 2024-05-07T02:30:19.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.113+0000 7f3c8c596b80 1 --2- 172.21.15.19:0/1517482797 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x555eb401da70 0x555eb401eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:19.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.114+0000 7f3c8c596b80 4 client.24995 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:30:19.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.114+0000 7f3c8c596b80 1 -- 172.21.15.19:0/1517482797 --> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555eb3fd0090 con 0x555eb401da70 2024-05-07T02:30:19.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.114+0000 7f3c76a00700 1 --2- 172.21.15.19:0/1517482797 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x555eb401da70 0x555eb401eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:19.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.114+0000 7f3c76a00700 1 --2- 172.21.15.19:0/1517482797 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x555eb401da70 0x555eb401eee0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=da1b90cfa715bc57 server_cookie=8b1cdbcdfad44dc9 in_seq=0 out_seq=0 2024-05-07T02:30:19.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.114+0000 7f3c60c00700 10 client.24995 ms_handle_connect on v2:172.21.15.64:6836/2585907777 2024-05-07T02:30:19.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.115+0000 7f3c60c00700 1 -- 172.21.15.19:0/1517482797 <== mds.4 v2:172.21.15.64:6836/2585907777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555eb3fd0090 con 0x555eb401da70 2024-05-07T02:30:19.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.115+0000 7f3c60c00700 10 client.24995 handle_command_reply: tid=0 2024-05-07T02:30:19.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.115+0000 7f3c8c596b80 1 client.24995 shutdown 2024-05-07T02:30:19.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.115+0000 7f3c8c596b80 1 -- 172.21.15.19:0/1517482797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555eb4003f20 msgr2=0x555eb400dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:19.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.115+0000 7f3c8c596b80 1 --2- 172.21.15.19:0/1517482797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555eb4003f20 0x555eb400dad0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3c640092e0 tx=0x7f3c64005700 comp rx=0 tx=0).stop 2024-05-07T02:30:19.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.116+0000 7f3c8c596b80 1 -- 172.21.15.19:0/1517482797 shutdown_connections 2024-05-07T02:30:19.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.116+0000 7f3c8c596b80 1 --2- 172.21.15.19:0/1517482797 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x555eb401da70 0x555eb401eee0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:19.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.116+0000 7f3c8c596b80 1 --2- 172.21.15.19:0/1517482797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555eb4009420 0x555eb400e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:19.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.116+0000 7f3c8c596b80 1 --2- 172.21.15.19:0/1517482797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555eb40035d0 0x555eb400d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:19.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.116+0000 7f3c8c596b80 1 --2- 172.21.15.19:0/1517482797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555eb4003f20 0x555eb400dad0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:19.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.116+0000 7f3c8c596b80 1 -- 172.21.15.19:0/1517482797 >> 172.21.15.19:0/1517482797 conn(0x555eb3e85df0 msgr2=0x555eb40025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:19.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.116+0000 7f3c60c00700 0 client.24995 ms_handle_reset on v2:172.21.15.64:6836/2585907777 2024-05-07T02:30:19.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.116+0000 7f3c8c596b80 1 -- 172.21.15.19:0/1517482797 shutdown_connections 2024-05-07T02:30:19.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.116+0000 7f3c8c596b80 1 -- 172.21.15.19:0/1517482797 wait complete. 2024-05-07T02:30:19.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.116+0000 7f3c8c596b80 20 client.24995 trim_cache size 0 max 16384 2024-05-07T02:30:19.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.139+0000 7f3c8c596b80 1 Processor -- start 2024-05-07T02:30:19.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.139+0000 7f3c8c596b80 1 -- start start 2024-05-07T02:30:19.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.140+0000 7f3c8c596b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555eb40035d0 0x555eb3fff9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:19.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.140+0000 7f3c8c596b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555eb4003f20 0x555eb4019c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:19.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.140+0000 7f3c8c596b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555eb4009420 0x555eb4007d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:19.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.140+0000 7f3c8c596b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555eb401d220 con 0x555eb4003f20 2024-05-07T02:30:19.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.140+0000 7f3c8c596b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x555eb401d8f0 con 0x555eb4009420 2024-05-07T02:30:19.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.140+0000 7f3c8c596b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555eb401dfc0 con 0x555eb40035d0 2024-05-07T02:30:19.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.140+0000 7f3c76a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555eb40035d0 0x555eb3fff9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:19.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.140+0000 7f3c76a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555eb40035d0 0x555eb3fff9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:42978/0 (socket says 172.21.15.19:42978) 2024-05-07T02:30:19.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.140+0000 7f3c76000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555eb4003f20 0x555eb4019c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:19.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.140+0000 7f3c76a00700 1 -- 172.21.15.19:0/2617725163 learned_addr learned my addr 172.21.15.19:0/2617725163 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:30:19.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.140+0000 7f3c77400700 1 --2- 172.21.15.19:0/2617725163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555eb4009420 0x555eb4007d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:19.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.141+0000 7f3c76a00700 1 -- 172.21.15.19:0/2617725163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555eb4009420 msgr2=0x555eb4007d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:30:19.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.141+0000 7f3c76a00700 1 --2- 172.21.15.19:0/2617725163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555eb4009420 0x555eb4007d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:19.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.141+0000 7f3c76a00700 1 -- 172.21.15.19:0/2617725163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555eb4003f20 msgr2=0x555eb4019c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:30:19.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.141+0000 7f3c76a00700 1 --2- 172.21.15.19:0/2617725163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555eb4003f20 0x555eb4019c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:19.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.141+0000 7f3c76a00700 1 -- 172.21.15.19:0/2617725163 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555eb3ffab30 con 0x555eb40035d0 2024-05-07T02:30:19.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.141+0000 7f3c77400700 1 --2- 172.21.15.19:0/2617725163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555eb4009420 0x555eb4007d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:30:19.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.141+0000 7f3c76a00700 1 --2- 172.21.15.19:0/2617725163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555eb40035d0 0x555eb3fff9e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3c6800c5b0 tx=0x7f3c6801f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d00afbf7421fe1ef server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:19.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.141+0000 7f3c63400700 1 -- 172.21.15.19:0/2617725163 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c680a95a0 con 0x555eb40035d0 2024-05-07T02:30:19.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.141+0000 7f3c63400700 1 -- 172.21.15.19:0/2617725163 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c680ade40 con 0x555eb40035d0 2024-05-07T02:30:19.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.141+0000 7f3c63400700 1 -- 172.21.15.19:0/2617725163 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c68018d70 con 0x555eb40035d0 2024-05-07T02:30:19.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.142+0000 7f3c8c596b80 1 -- 172.21.15.19:0/2617725163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555eb40035d0 msgr2=0x555eb3fff9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:19.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.142+0000 7f3c8c596b80 1 --2- 172.21.15.19:0/2617725163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555eb40035d0 0x555eb3fff9e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3c6800c5b0 tx=0x7f3c6801f000 comp rx=0 tx=0).stop 2024-05-07T02:30:19.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.142+0000 7f3c8c596b80 1 -- 172.21.15.19:0/2617725163 shutdown_connections 2024-05-07T02:30:19.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.142+0000 7f3c8c596b80 1 --2- 172.21.15.19:0/2617725163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555eb4009420 0x555eb4007d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:19.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.142+0000 7f3c8c596b80 1 --2- 172.21.15.19:0/2617725163 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555eb40035d0 0x555eb3fff9e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:19.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.142+0000 7f3c8c596b80 1 --2- 172.21.15.19:0/2617725163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555eb4003f20 0x555eb4019c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:19.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.142+0000 7f3c8c596b80 1 -- 172.21.15.19:0/2617725163 >> 172.21.15.19:0/2617725163 conn(0x555eb3e85df0 msgr2=0x555eb3ff8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:19.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.143+0000 7f3c8c596b80 1 -- 172.21.15.19:0/2617725163 shutdown_connections 2024-05-07T02:30:19.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.143+0000 7f3c8c596b80 1 -- 172.21.15.19:0/2617725163 wait complete. 2024-05-07T02:30:19.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.143+0000 7f3c8c596b80 1 -- start start 2024-05-07T02:30:19.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.143+0000 7f3c8c596b80 1 Processor -- start 2024-05-07T02:30:19.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.144+0000 7f3c8c596b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555eb40035d0 0x555eb40216e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:19.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.144+0000 7f3c8c596b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555eb4003f20 0x555eb4021c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:19.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.144+0000 7f3c8c596b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555eb4009420 0x555eb4024c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:19.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.144+0000 7f3c8c596b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555eb401ed60 con 0x555eb40035d0 2024-05-07T02:30:19.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.144+0000 7f3c8c596b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x555eb401e690 con 0x555eb4009420 2024-05-07T02:30:19.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.144+0000 7f3c8c596b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555eb401f430 con 0x555eb4003f20 2024-05-07T02:30:19.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.144+0000 7f3c8c596b80 10 client.0 _ll_register_callbacks cb 0x555eb3eb9330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:30:19.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.144+0000 7f3c76000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555eb4003f20 0x555eb4021c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:19.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.144+0000 7f3c76a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555eb40035d0 0x555eb40216e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:19.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.144+0000 7f3c76000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555eb4003f20 0x555eb4021c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:42990/0 (socket says 172.21.15.19:42990) 2024-05-07T02:30:19.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.144+0000 7f3c76000700 1 -- 172.21.15.19:0/2543135155 learned_addr learned my addr 172.21.15.19:0/2543135155 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:30:19.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.144+0000 7f3c77400700 1 --2- 172.21.15.19:0/2543135155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555eb4009420 0x555eb4024c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:19.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.145+0000 7f3c76000700 1 -- 172.21.15.19:0/2543135155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555eb4009420 msgr2=0x555eb4024c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:30:19.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.145+0000 7f3c76000700 1 --2- 172.21.15.19:0/2543135155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555eb4009420 0x555eb4024c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:19.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.145+0000 7f3c76000700 1 -- 172.21.15.19:0/2543135155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555eb40035d0 msgr2=0x555eb40216e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:30:19.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.145+0000 7f3c76000700 1 --2- 172.21.15.19:0/2543135155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555eb40035d0 0x555eb40216e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:19.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.145+0000 7f3c76000700 1 -- 172.21.15.19:0/2543135155 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555eb40251d0 con 0x555eb4003f20 2024-05-07T02:30:19.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.145+0000 7f3c76a00700 1 --2- 172.21.15.19:0/2543135155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555eb40035d0 0x555eb40216e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:30:19.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.145+0000 7f3c76000700 1 --2- 172.21.15.19:0/2543135155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555eb4003f20 0x555eb4021c20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3c64000c00 tx=0x7f3c6400bd00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=382b6b9d749f70f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:19.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.145+0000 7f3c60c00700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:30:19.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.146+0000 7f3c60c00700 1 -- 172.21.15.19:0/2543135155 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c64037070 con 0x555eb4003f20 2024-05-07T02:30:19.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.146+0000 7f3c60c00700 1 -- 172.21.15.19:0/2543135155 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f3c6400f870 con 0x555eb4003f20 2024-05-07T02:30:19.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.146+0000 7f3c8c596b80 1 -- 172.21.15.19:0/2543135155 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c680ade40 con 0x555eb4003f20 2024-05-07T02:30:19.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.146+0000 7f3c60c00700 1 -- 172.21.15.19:0/2543135155 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3c64043050 con 0x555eb4003f20 2024-05-07T02:30:19.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.146+0000 7f3c8c596b80 10 client.24997 fetch_fsmap learned FSMap version 30 2024-05-07T02:30:19.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.146+0000 7f3c8c596b80 1 -- 172.21.15.19:0/2543135155 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555eb40264a0 con 0x555eb4003f20 2024-05-07T02:30:19.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.146+0000 7f3c60c00700 1 -- 172.21.15.19:0/2543135155 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c680ade40 con 0x555eb4003f20 2024-05-07T02:30:19.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.146+0000 7f3c60c00700 1 -- 172.21.15.19:0/2543135155 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f3c64003b30 con 0x555eb4003f20 2024-05-07T02:30:19.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.146+0000 7f3c8c596b80 10 client.24997 fetch_fsmap finished waiting for FSMap version 30 2024-05-07T02:30:19.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.147+0000 7f3c8c596b80 10 client.24997 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-07T02:30:19.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.147+0000 7f3c8c596b80 20 client.24997 populate_metadata read hostname 'smithi019' 2024-05-07T02:30:19.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.147+0000 7f3c8c596b80 1 --2- 172.21.15.19:0/2543135155 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x555eb4026790 0x555eb408bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:19.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.147+0000 7f3c8c596b80 4 client.24997 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-07T02:30:19.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.147+0000 7f3c8c596b80 1 -- 172.21.15.19:0/2543135155 --> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] -- command(tid 0: {"prefix": "status"}) -- 0x555eb3f333b0 con 0x555eb4026790 2024-05-07T02:30:19.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.147+0000 7f3c76a00700 1 --2- 172.21.15.19:0/2543135155 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x555eb4026790 0x555eb408bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:19.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.148+0000 7f3c60c00700 10 client.24997 ms_handle_connect on v2:172.21.15.64:6836/2585907777 2024-05-07T02:30:19.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.148+0000 7f3c76a00700 1 --2- 172.21.15.19:0/2543135155 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x555eb4026790 0x555eb408bdb0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=16dc0a3934d6ba39 server_cookie=eb06da88f1d309d1 in_seq=0 out_seq=0 2024-05-07T02:30:19.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.148+0000 7f3c60c00700 1 -- 172.21.15.19:0/2543135155 <== mds.4 v2:172.21.15.64:6836/2585907777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x555eb3f333b0 con 0x555eb4026790 2024-05-07T02:30:19.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.148+0000 7f3c60c00700 10 client.24997 handle_command_reply: tid=0 2024-05-07T02:30:19.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.148+0000 7f3c8c596b80 1 client.24997 shutdown 2024-05-07T02:30:19.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.148+0000 7f3c8c596b80 1 -- 172.21.15.19:0/2543135155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555eb4003f20 msgr2=0x555eb4021c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:19.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.148+0000 7f3c8c596b80 1 --2- 172.21.15.19:0/2543135155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555eb4003f20 0x555eb4021c20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3c64000c00 tx=0x7f3c6400bd00 comp rx=0 tx=0).stop 2024-05-07T02:30:19.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.148+0000 7f3c8c596b80 1 -- 172.21.15.19:0/2543135155 shutdown_connections 2024-05-07T02:30:19.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.148+0000 7f3c8c596b80 1 --2- 172.21.15.19:0/2543135155 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x555eb4026790 0x555eb408bdb0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:19.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.148+0000 7f3c8c596b80 1 --2- 172.21.15.19:0/2543135155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555eb4009420 0x555eb4024c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:19.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.148+0000 7f3c8c596b80 1 --2- 172.21.15.19:0/2543135155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555eb40035d0 0x555eb40216e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:19.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.148+0000 7f3c8c596b80 1 --2- 172.21.15.19:0/2543135155 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555eb4003f20 0x555eb4021c20 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:19.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.148+0000 7f3c60c00700 0 client.24997 ms_handle_reset on v2:172.21.15.64:6836/2585907777 2024-05-07T02:30:19.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.148+0000 7f3c8c596b80 1 -- 172.21.15.19:0/2543135155 >> 172.21.15.19:0/2543135155 conn(0x555eb3e85df0 msgr2=0x555eb40025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:19.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.149+0000 7f3c8c596b80 1 -- 172.21.15.19:0/2543135155 shutdown_connections 2024-05-07T02:30:19.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.149+0000 7f3c8c596b80 1 -- 172.21.15.19:0/2543135155 wait complete. 2024-05-07T02:30:19.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.149+0000 7f3c8c596b80 20 client.24997 trim_cache size 0 max 16384 2024-05-07T02:30:19.151 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:30:19.151 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "9ff84c16-0c17-11ef-bc97-c7b262605968", 2024-05-07T02:30:19.151 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 4, 2024-05-07T02:30:19.151 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24505, 2024-05-07T02:30:19.151 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-05-07T02:30:19.151 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-05-07T02:30:19.151 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-05-07T02:30:19.151 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 156.42589644099999, 2024-05-07T02:30:19.151 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 30, 2024-05-07T02:30:19.151 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 89, 2024-05-07T02:30:19.151 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 87, 2024-05-07T02:30:19.151 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 284.58667530500003 2024-05-07T02:30:19.151 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:30:19.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.149+0000 7f3c7da00700 1 -- 172.21.15.19:0/966138735 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f3c5c0b2900 msgr2=0x7f3c5c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:19.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.149+0000 7f3c7da00700 1 --2- 172.21.15.19:0/966138735 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f3c5c0b2900 0x7f3c5c0b4e10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3c70009ec0 tx=0x7f3c70013480 comp rx=0 tx=0).stop 2024-05-07T02:30:19.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.149+0000 7f3c7da00700 1 -- 172.21.15.19:0/966138735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c781045d0 msgr2=0x7f3c78076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:19.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.149+0000 7f3c7da00700 1 --2- 172.21.15.19:0/966138735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c781045d0 0x7f3c78076bd0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3c6800e9a0 tx=0x7f3c6800ecb0 comp rx=0 tx=0).stop 2024-05-07T02:30:19.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.150+0000 7f3c7da00700 1 -- 172.21.15.19:0/966138735 shutdown_connections 2024-05-07T02:30:19.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.150+0000 7f3c7da00700 1 --2- 172.21.15.19:0/966138735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3c78109850 0x7f3c78077650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:19.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.150+0000 7f3c7da00700 1 --2- 172.21.15.19:0/966138735 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f3c5c0b2900 0x7f3c5c0b4e10 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:19.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.150+0000 7f3c7da00700 1 --2- 172.21.15.19:0/966138735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c781045d0 0x7f3c78076bd0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:19.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.150+0000 7f3c7da00700 1 --2- 172.21.15.19:0/966138735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c78108f00 0x7f3c78077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:19.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.150+0000 7f3c7da00700 1 -- 172.21.15.19:0/966138735 >> 172.21.15.19:0/966138735 conn(0x7f3c780ffea0 msgr2=0x7f3c7810a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:19.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.150+0000 7f3c7da00700 1 -- 172.21.15.19:0/966138735 shutdown_connections 2024-05-07T02:30:19.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:19.150+0000 7f3c7da00700 1 -- 172.21.15.19:0/966138735 wait complete. 2024-05-07T02:30:19.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:30:18 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:30:19.993 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.h status 2024-05-07T02:30:20.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:30:19 smithi064 ceph-mon[34699]: pgmap v463: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:30:20.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:30:19 smithi019 ceph-mon[29781]: pgmap v463: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:30:20.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:30:19 smithi027 ceph-mon[34398]: pgmap v463: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:30:20.477 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:30:20.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:30:20 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:30:20.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:30:20 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:30:20.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:30:20 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:30:21.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:30:20 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:30:21.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:30:20 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:30:21.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:30:20 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:30:21.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:30:20 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:30:21.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:30:20 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:30:21.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:30:20 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:30:22.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:30:21 smithi019 ceph-mon[29781]: pgmap v464: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:30:22.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:30:21 smithi027 ceph-mon[34398]: pgmap v464: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:30:22.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:30:21 smithi064 ceph-mon[34699]: pgmap v464: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:30:23.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.023+0000 7f25ea000700 1 -- 172.21.15.19:0/3495154084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25e4108d30 msgr2=0x7f25e4109140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:23.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.023+0000 7f25ea000700 1 --2- 172.21.15.19:0/3495154084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25e4108d30 0x7f25e4109140 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f25d00099c0 tx=0x7f25d0009cd0 comp rx=0 tx=0).stop 2024-05-07T02:30:23.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.023+0000 7f25ea000700 1 -- 172.21.15.19:0/3495154084 shutdown_connections 2024-05-07T02:30:23.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.023+0000 7f25ea000700 1 --2- 172.21.15.19:0/3495154084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25e4109680 0x7f25e4110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:23.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.023+0000 7f25ea000700 1 --2- 172.21.15.19:0/3495154084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25e4104400 0x7f25e4104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:23.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.023+0000 7f25ea000700 1 --2- 172.21.15.19:0/3495154084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25e4108d30 0x7f25e4109140 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:23.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.023+0000 7f25ea000700 1 -- 172.21.15.19:0/3495154084 >> 172.21.15.19:0/3495154084 conn(0x7f25e4079d10 msgr2=0x7f25e407a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:23.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.023+0000 7f25ea000700 1 -- 172.21.15.19:0/3495154084 shutdown_connections 2024-05-07T02:30:23.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.023+0000 7f25ea000700 1 -- 172.21.15.19:0/3495154084 wait complete. 2024-05-07T02:30:23.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.024+0000 7f25ea000700 1 Processor -- start 2024-05-07T02:30:23.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.025+0000 7f25ea000700 1 -- start start 2024-05-07T02:30:23.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.025+0000 7f25ea000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25e4104400 0x7f25e41a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:23.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.025+0000 7f25ea000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25e4108d30 0x7f25e41a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:23.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.025+0000 7f25ea000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25e4109680 0x7f25e419ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:23.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.025+0000 7f25ea000700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f25e41142b0 con 0x7f25e4104400 2024-05-07T02:30:23.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.025+0000 7f25ea000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f25e4114130 con 0x7f25e4109680 2024-05-07T02:30:23.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.025+0000 7f25ea000700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f25e4114430 con 0x7f25e4108d30 2024-05-07T02:30:23.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.025+0000 7f25e8c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25e4104400 0x7f25e41a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:23.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.026+0000 7f25e3e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25e4108d30 0x7f25e41a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:23.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.026+0000 7f25e8c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25e4104400 0x7f25e41a09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47888/0 (socket says 172.21.15.19:47888) 2024-05-07T02:30:23.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.026+0000 7f25e8c00700 1 -- 172.21.15.19:0/1540311966 learned_addr learned my addr 172.21.15.19:0/1540311966 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:30:23.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.026+0000 7f25e9600700 1 --2- 172.21.15.19:0/1540311966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25e4109680 0x7f25e419ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:23.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.026+0000 7f25e8c00700 1 -- 172.21.15.19:0/1540311966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25e4108d30 msgr2=0x7f25e41a0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:23.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.026+0000 7f25e8c00700 1 --2- 172.21.15.19:0/1540311966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25e4108d30 0x7f25e41a0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:23.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.026+0000 7f25e8c00700 1 -- 172.21.15.19:0/1540311966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25e4109680 msgr2=0x7f25e419ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:23.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.026+0000 7f25e8c00700 1 --2- 172.21.15.19:0/1540311966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25e4109680 0x7f25e419ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:23.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.026+0000 7f25e8c00700 1 -- 172.21.15.19:0/1540311966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25e419b280 con 0x7f25e4104400 2024-05-07T02:30:23.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.026+0000 7f25e3e00700 1 --2- 172.21.15.19:0/1540311966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25e4108d30 0x7f25e41a0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:30:23.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.026+0000 7f25e9600700 1 --2- 172.21.15.19:0/1540311966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25e4109680 0x7f25e419ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:30:23.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.027+0000 7f25e8c00700 1 --2- 172.21.15.19:0/1540311966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25e4104400 0x7f25e41a09c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f25d800e9c0 tx=0x7f25d800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:23.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.027+0000 7f25e1600700 1 -- 172.21.15.19:0/1540311966 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d800cbd0 con 0x7f25e4104400 2024-05-07T02:30:23.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.027+0000 7f25e1600700 1 -- 172.21.15.19:0/1540311966 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25d800cd70 con 0x7f25e4104400 2024-05-07T02:30:23.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.027+0000 7f25e1600700 1 -- 172.21.15.19:0/1540311966 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d8005610 con 0x7f25e4104400 2024-05-07T02:30:23.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.027+0000 7f25ea000700 1 -- 172.21.15.19:0/1540311966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25e419b570 con 0x7f25e4104400 2024-05-07T02:30:23.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.027+0000 7f25ea000700 1 -- 172.21.15.19:0/1540311966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25e41a7770 con 0x7f25e4104400 2024-05-07T02:30:23.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.032+0000 7f25e1600700 1 -- 172.21.15.19:0/1540311966 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f25d8004750 con 0x7f25e4104400 2024-05-07T02:30:23.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.032+0000 7f25fa1cbb80 1 Processor -- start 2024-05-07T02:30:23.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.033+0000 7f25fa1cbb80 1 -- start start 2024-05-07T02:30:23.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.033+0000 7f25fa1cbb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560875262a40 0x560875262e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:23.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.033+0000 7f25fa1cbb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560875263390 0x56087526c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:23.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.033+0000 7f25fa1cbb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56087526cb10 0x56087526ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:23.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.033+0000 7f25fa1cbb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5608751849d0 con 0x56087526cb10 2024-05-07T02:30:23.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.033+0000 7f25fa1cbb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x560875122710 con 0x560875263390 2024-05-07T02:30:23.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.033+0000 7f25fa1cbb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560875180990 con 0x560875262a40 2024-05-07T02:30:23.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.033+0000 7f25e8c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560875263390 0x56087526c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:23.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.033+0000 7f25e3e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56087526cb10 0x56087526ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:23.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.033+0000 7f25e9600700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560875262a40 0x560875262e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:23.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.034+0000 7f25e8c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560875263390 0x56087526c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:42972/0 (socket says 172.21.15.19:42972) 2024-05-07T02:30:23.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.034+0000 7f25e8c00700 1 -- 172.21.15.19:0/518296868 learned_addr learned my addr 172.21.15.19:0/518296868 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:30:23.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.034+0000 7f25e1600700 1 --2- 172.21.15.19:0/1540311966 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f25cc0b2880 0x7f25cc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:23.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.034+0000 7f25e1600700 1 -- 172.21.15.19:0/1540311966 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f25d80e2940 con 0x7f25e4104400 2024-05-07T02:30:23.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.034+0000 7f25e8c00700 1 -- 172.21.15.19:0/518296868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560875262a40 msgr2=0x560875262e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:30:23.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.034+0000 7f25e8c00700 1 --2- 172.21.15.19:0/518296868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560875262a40 0x560875262e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:23.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.034+0000 7f25e8c00700 1 -- 172.21.15.19:0/518296868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56087526cb10 msgr2=0x56087526ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:30:23.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.034+0000 7f25e8c00700 1 --2- 172.21.15.19:0/518296868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56087526cb10 0x56087526ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:23.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.034+0000 7f25e8c00700 1 -- 172.21.15.19:0/518296868 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608750d1880 con 0x560875263390 2024-05-07T02:30:23.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.034+0000 7f25e9600700 1 --2- 172.21.15.19:0/518296868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560875262a40 0x560875262e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:30:23.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.034+0000 7f25e3e00700 1 --2- 172.21.15.19:0/518296868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56087526cb10 0x56087526ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:30:23.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.034+0000 7f25e9600700 1 --2- 172.21.15.19:0/1540311966 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f25cc0b2880 0x7f25cc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:23.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.035+0000 7f25e8c00700 1 --2- 172.21.15.19:0/518296868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560875263390 0x56087526c5d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f25d80ada40 tx=0x7f25d8005020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e18c0666d9c4069e server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:23.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.035+0000 7f25d6000700 1 -- 172.21.15.19:0/518296868 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d80a9920 con 0x560875263390 2024-05-07T02:30:23.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.035+0000 7f25d6000700 1 -- 172.21.15.19:0/518296868 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25d80b22f0 con 0x560875263390 2024-05-07T02:30:23.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.035+0000 7f25d6000700 1 -- 172.21.15.19:0/518296868 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d80a52e0 con 0x560875263390 2024-05-07T02:30:23.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.036+0000 7f25fa1cbb80 1 -- 172.21.15.19:0/518296868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560875263390 msgr2=0x56087526c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:23.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.036+0000 7f25fa1cbb80 1 --2- 172.21.15.19:0/518296868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560875263390 0x56087526c5d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f25d80ada40 tx=0x7f25d8005020 comp rx=0 tx=0).stop 2024-05-07T02:30:23.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.036+0000 7f25e9600700 1 --2- 172.21.15.19:0/1540311966 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f25cc0b2880 0x7f25cc0b4d90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x56087525edf0 tx=0x7f25dc009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:23.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.036+0000 7f25fa1cbb80 1 -- 172.21.15.19:0/518296868 shutdown_connections 2024-05-07T02:30:23.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.036+0000 7f25fa1cbb80 1 --2- 172.21.15.19:0/518296868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56087526cb10 0x56087526ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:23.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.036+0000 7f25fa1cbb80 1 --2- 172.21.15.19:0/518296868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560875262a40 0x560875262e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:23.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.036+0000 7f25fa1cbb80 1 --2- 172.21.15.19:0/518296868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560875263390 0x56087526c5d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:23.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.036+0000 7f25fa1cbb80 1 -- 172.21.15.19:0/518296868 >> 172.21.15.19:0/518296868 conn(0x5608750e4df0 msgr2=0x56087525d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:23.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.036+0000 7f25fa1cbb80 1 -- 172.21.15.19:0/518296868 shutdown_connections 2024-05-07T02:30:23.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.036+0000 7f25fa1cbb80 1 -- 172.21.15.19:0/518296868 wait complete. 2024-05-07T02:30:23.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.037+0000 7f25fa1cbb80 1 -- start start 2024-05-07T02:30:23.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.037+0000 7f25fa1cbb80 1 Processor -- start 2024-05-07T02:30:23.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.038+0000 7f25fa1cbb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560875262a40 0x560875269aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:23.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.038+0000 7f25fa1cbb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560875263390 0x560875269fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:23.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.038+0000 7f25fa1cbb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56087526cb10 0x56087526a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:23.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.038+0000 7f25fa1cbb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56087526f850 con 0x560875263390 2024-05-07T02:30:23.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.038+0000 7f25fa1cbb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5608750eaad0 con 0x56087526cb10 2024-05-07T02:30:23.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.038+0000 7f25fa1cbb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56087526f9d0 con 0x560875262a40 2024-05-07T02:30:23.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.038+0000 7f25fa1cbb80 10 client.0 _ll_register_callbacks cb 0x56087512bc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:30:23.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.038+0000 7f25e3e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560875263390 0x560875269fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:23.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.038+0000 7f25e3e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560875263390 0x560875269fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47908/0 (socket says 172.21.15.19:47908) 2024-05-07T02:30:23.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.038+0000 7f25e3e00700 1 -- 172.21.15.19:0/391295756 learned_addr learned my addr 172.21.15.19:0/391295756 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:30:23.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.038+0000 7f25e8c00700 1 --2- 172.21.15.19:0/391295756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560875262a40 0x560875269aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:23.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.038+0000 7f25e9600700 1 --2- 172.21.15.19:0/391295756 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56087526cb10 0x56087526a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:23.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.039+0000 7f25e3e00700 1 -- 172.21.15.19:0/391295756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560875262a40 msgr2=0x560875269aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:30:23.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.039+0000 7f25e3e00700 1 --2- 172.21.15.19:0/391295756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560875262a40 0x560875269aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:23.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.039+0000 7f25e3e00700 1 -- 172.21.15.19:0/391295756 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56087526cb10 msgr2=0x56087526a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:30:23.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.039+0000 7f25e3e00700 1 --2- 172.21.15.19:0/391295756 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56087526cb10 0x56087526a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:23.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.039+0000 7f25e3e00700 1 -- 172.21.15.19:0/391295756 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56087514a3d0 con 0x560875263390 2024-05-07T02:30:23.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.039+0000 7f25e3e00700 1 --2- 172.21.15.19:0/391295756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560875263390 0x560875269fe0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f25d00099c0 tx=0x7f25d000b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=512a9fd9c6201cdf server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:23.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.039+0000 7f25caa00700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-07T02:30:23.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.040+0000 7f25caa00700 1 -- 172.21.15.19:0/391295756 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d0005680 con 0x560875263390 2024-05-07T02:30:23.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.040+0000 7f25fa1cbb80 1 -- 172.21.15.19:0/391295756 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56087522f290 con 0x560875263390 2024-05-07T02:30:23.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.040+0000 7f25caa00700 1 -- 172.21.15.19:0/391295756 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f25d00037c0 con 0x560875263390 2024-05-07T02:30:23.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.040+0000 7f25caa00700 1 -- 172.21.15.19:0/391295756 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f25d0031050 con 0x560875263390 2024-05-07T02:30:23.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.040+0000 7f25fa1cbb80 10 client.14871 fetch_fsmap learned FSMap version 30 2024-05-07T02:30:23.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.040+0000 7f25fa1cbb80 1 -- 172.21.15.19:0/391295756 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56087527e8c0 con 0x560875263390 2024-05-07T02:30:23.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.040+0000 7f25caa00700 1 -- 172.21.15.19:0/391295756 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56087522f290 con 0x560875263390 2024-05-07T02:30:23.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.040+0000 7f25caa00700 1 -- 172.21.15.19:0/391295756 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f25d002c030 con 0x560875263390 2024-05-07T02:30:23.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.041+0000 7f25fa1cbb80 10 client.14871 fetch_fsmap finished waiting for FSMap version 30 2024-05-07T02:30:23.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.041+0000 7f25fa1cbb80 10 client.14871 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-07T02:30:23.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.041+0000 7f25fa1cbb80 20 client.14871 populate_metadata read hostname 'smithi019' 2024-05-07T02:30:23.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.041+0000 7f25fa1cbb80 1 --2- 172.21.15.19:0/391295756 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x560875280f10 0x560875282380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:23.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.041+0000 7f25fa1cbb80 4 client.14871 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:30:23.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.041+0000 7f25fa1cbb80 1 -- 172.21.15.19:0/391295756 --> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608751923b0 con 0x560875280f10 2024-05-07T02:30:23.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.041+0000 7f25e8c00700 1 --2- 172.21.15.19:0/391295756 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x560875280f10 0x560875282380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:23.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.042+0000 7f25e8c00700 1 --2- 172.21.15.19:0/391295756 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x560875280f10 0x560875282380 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1ac7b1da6dbc95c8 server_cookie=72b1df2d2ca047e6 in_seq=0 out_seq=0 2024-05-07T02:30:23.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.042+0000 7f25caa00700 10 client.14871 ms_handle_connect on v2:172.21.15.27:6836/2583138143 2024-05-07T02:30:23.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.043+0000 7f25caa00700 1 -- 172.21.15.19:0/391295756 <== mds.3 v2:172.21.15.27:6836/2583138143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5608751923b0 con 0x560875280f10 2024-05-07T02:30:23.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.043+0000 7f25caa00700 10 client.14871 handle_command_reply: tid=0 2024-05-07T02:30:23.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.043+0000 7f25fa1cbb80 1 client.14871 shutdown 2024-05-07T02:30:23.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.043+0000 7f25fa1cbb80 1 -- 172.21.15.19:0/391295756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560875263390 msgr2=0x560875269fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:23.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.043+0000 7f25fa1cbb80 1 --2- 172.21.15.19:0/391295756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560875263390 0x560875269fe0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f25d00099c0 tx=0x7f25d000b880 comp rx=0 tx=0).stop 2024-05-07T02:30:23.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.043+0000 7f25fa1cbb80 1 -- 172.21.15.19:0/391295756 shutdown_connections 2024-05-07T02:30:23.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.043+0000 7f25fa1cbb80 1 --2- 172.21.15.19:0/391295756 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x560875280f10 0x560875282380 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:23.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.043+0000 7f25caa00700 0 client.14871 ms_handle_reset on v2:172.21.15.27:6836/2583138143 2024-05-07T02:30:23.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.043+0000 7f25fa1cbb80 1 --2- 172.21.15.19:0/391295756 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56087526cb10 0x56087526a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:23.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.043+0000 7f25fa1cbb80 1 --2- 172.21.15.19:0/391295756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560875262a40 0x560875269aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:23.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.043+0000 7f25fa1cbb80 1 --2- 172.21.15.19:0/391295756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560875263390 0x560875269fe0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:23.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.043+0000 7f25fa1cbb80 1 -- 172.21.15.19:0/391295756 >> 172.21.15.19:0/391295756 conn(0x5608750e4df0 msgr2=0x56087525f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:23.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.043+0000 7f25fa1cbb80 1 -- 172.21.15.19:0/391295756 shutdown_connections 2024-05-07T02:30:23.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.043+0000 7f25fa1cbb80 1 -- 172.21.15.19:0/391295756 wait complete. 2024-05-07T02:30:23.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.043+0000 7f25fa1cbb80 20 client.14871 trim_cache size 0 max 16384 2024-05-07T02:30:23.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.067+0000 7f25fa1cbb80 1 Processor -- start 2024-05-07T02:30:23.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.067+0000 7f25fa1cbb80 1 -- start start 2024-05-07T02:30:23.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.067+0000 7f25fa1cbb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560875262a40 0x56087527d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:23.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.067+0000 7f25fa1cbb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560875263390 0x560875261b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:23.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.067+0000 7f25fa1cbb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56087526cb10 0x56087526a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:23.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.067+0000 7f25fa1cbb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5608752806c0 con 0x56087526cb10 2024-05-07T02:30:23.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.067+0000 7f25fa1cbb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x560875280d90 con 0x560875262a40 2024-05-07T02:30:23.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.067+0000 7f25fa1cbb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560875281460 con 0x560875263390 2024-05-07T02:30:23.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.068+0000 7f25e8c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560875262a40 0x56087527d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:23.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.068+0000 7f25e9600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56087526cb10 0x56087526a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:23.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.068+0000 7f25e8c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560875262a40 0x56087527d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:43002/0 (socket says 172.21.15.19:43002) 2024-05-07T02:30:23.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.068+0000 7f25e8c00700 1 -- 172.21.15.19:0/589953410 learned_addr learned my addr 172.21.15.19:0/589953410 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:30:23.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.068+0000 7f25e3e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560875263390 0x560875261b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:23.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.068+0000 7f25e8c00700 1 -- 172.21.15.19:0/589953410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560875263390 msgr2=0x560875261b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:30:23.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.068+0000 7f25e8c00700 1 --2- 172.21.15.19:0/589953410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560875263390 0x560875261b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:23.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.068+0000 7f25e8c00700 1 -- 172.21.15.19:0/589953410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56087526cb10 msgr2=0x56087526a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:30:23.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.068+0000 7f25e8c00700 1 --2- 172.21.15.19:0/589953410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56087526cb10 0x56087526a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:23.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.068+0000 7f25e8c00700 1 -- 172.21.15.19:0/589953410 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560875268a60 con 0x560875262a40 2024-05-07T02:30:23.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.068+0000 7f25e8c00700 1 --2- 172.21.15.19:0/589953410 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560875262a40 0x56087527d160 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f25d80a6dc0 tx=0x7f25d80e8260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e768f1d7983e51d server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:23.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.069+0000 7f25d5600700 1 -- 172.21.15.19:0/589953410 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d80c3070 con 0x560875262a40 2024-05-07T02:30:23.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.069+0000 7f25d5600700 1 -- 172.21.15.19:0/589953410 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25d80adf00 con 0x560875262a40 2024-05-07T02:30:23.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.069+0000 7f25d5600700 1 -- 172.21.15.19:0/589953410 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d80a92c0 con 0x560875262a40 2024-05-07T02:30:23.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.070+0000 7f25fa1cbb80 1 -- 172.21.15.19:0/589953410 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560875262a40 msgr2=0x56087527d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:23.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.070+0000 7f25fa1cbb80 1 --2- 172.21.15.19:0/589953410 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560875262a40 0x56087527d160 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f25d80a6dc0 tx=0x7f25d80e8260 comp rx=0 tx=0).stop 2024-05-07T02:30:23.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.070+0000 7f25fa1cbb80 1 -- 172.21.15.19:0/589953410 shutdown_connections 2024-05-07T02:30:23.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.070+0000 7f25fa1cbb80 1 --2- 172.21.15.19:0/589953410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56087526cb10 0x56087526a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:23.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.070+0000 7f25fa1cbb80 1 --2- 172.21.15.19:0/589953410 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560875262a40 0x56087527d160 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:23.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.070+0000 7f25fa1cbb80 1 --2- 172.21.15.19:0/589953410 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560875263390 0x560875261b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:23.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.070+0000 7f25fa1cbb80 1 -- 172.21.15.19:0/589953410 >> 172.21.15.19:0/589953410 conn(0x5608750e4df0 msgr2=0x560875261410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:23.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.070+0000 7f25fa1cbb80 1 -- 172.21.15.19:0/589953410 shutdown_connections 2024-05-07T02:30:23.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.070+0000 7f25fa1cbb80 1 -- 172.21.15.19:0/589953410 wait complete. 2024-05-07T02:30:23.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.070+0000 7f25fa1cbb80 1 -- start start 2024-05-07T02:30:23.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.071+0000 7f25fa1cbb80 1 Processor -- start 2024-05-07T02:30:23.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.071+0000 7f25fa1cbb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560875262a40 0x560875270280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:23.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.071+0000 7f25fa1cbb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560875263390 0x5608752707c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:23.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.071+0000 7f25fa1cbb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56087526cb10 0x560875270d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:23.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.071+0000 7f25fa1cbb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560875282200 con 0x560875262a40 2024-05-07T02:30:23.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.071+0000 7f25fa1cbb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x560875281b30 con 0x56087526cb10 2024-05-07T02:30:23.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.071+0000 7f25fa1cbb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5608752828d0 con 0x560875263390 2024-05-07T02:30:23.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.071+0000 7f25fa1cbb80 10 client.0 _ll_register_callbacks cb 0x56087512bc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:30:23.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.072+0000 7f25e8c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560875262a40 0x560875270280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:23.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.072+0000 7f25e3e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560875263390 0x5608752707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:23.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.072+0000 7f25e9600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56087526cb10 0x560875270d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:23.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.072+0000 7f25e8c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560875262a40 0x560875270280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47924/0 (socket says 172.21.15.19:47924) 2024-05-07T02:30:23.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.072+0000 7f25e8c00700 1 -- 172.21.15.19:0/1953086141 learned_addr learned my addr 172.21.15.19:0/1953086141 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:30:23.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.072+0000 7f25e3e00700 1 -- 172.21.15.19:0/1953086141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56087526cb10 msgr2=0x560875270d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:30:23.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.072+0000 7f25e3e00700 1 --2- 172.21.15.19:0/1953086141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56087526cb10 0x560875270d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:23.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.072+0000 7f25e3e00700 1 -- 172.21.15.19:0/1953086141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560875262a40 msgr2=0x560875270280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:30:23.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.072+0000 7f25e3e00700 1 --2- 172.21.15.19:0/1953086141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560875262a40 0x560875270280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:23.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.072+0000 7f25e3e00700 1 -- 172.21.15.19:0/1953086141 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56087528a8e0 con 0x560875263390 2024-05-07T02:30:23.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.072+0000 7f25e8c00700 1 --2- 172.21.15.19:0/1953086141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560875262a40 0x560875270280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:30:23.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.072+0000 7f25e3e00700 1 --2- 172.21.15.19:0/1953086141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560875263390 0x5608752707c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f25d000b580 tx=0x7f25d0005a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78422fe521a60c71 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:23.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.072+0000 7f25caa00700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:30:23.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.073+0000 7f25caa00700 1 -- 172.21.15.19:0/1953086141 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d0005690 con 0x560875263390 2024-05-07T02:30:23.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.073+0000 7f25fa1cbb80 1 -- 172.21.15.19:0/1953086141 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25d80adf00 con 0x560875263390 2024-05-07T02:30:23.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.073+0000 7f25caa00700 1 -- 172.21.15.19:0/1953086141 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f25d0024470 con 0x560875263390 2024-05-07T02:30:23.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.073+0000 7f25caa00700 1 -- 172.21.15.19:0/1953086141 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f25d0043050 con 0x560875263390 2024-05-07T02:30:23.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.073+0000 7f25fa1cbb80 10 client.25015 fetch_fsmap learned FSMap version 30 2024-05-07T02:30:23.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.073+0000 7f25fa1cbb80 1 -- 172.21.15.19:0/1953086141 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56087528bb40 con 0x560875263390 2024-05-07T02:30:23.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.074+0000 7f25caa00700 1 -- 172.21.15.19:0/1953086141 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25d80adf00 con 0x560875263390 2024-05-07T02:30:23.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.074+0000 7f25caa00700 1 -- 172.21.15.19:0/1953086141 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f25d002c040 con 0x560875263390 2024-05-07T02:30:23.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.074+0000 7f25fa1cbb80 10 client.25015 fetch_fsmap finished waiting for FSMap version 30 2024-05-07T02:30:23.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.074+0000 7f25fa1cbb80 10 client.25015 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-07T02:30:23.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.074+0000 7f25fa1cbb80 20 client.25015 populate_metadata read hostname 'smithi019' 2024-05-07T02:30:23.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.074+0000 7f25fa1cbb80 1 --2- 172.21.15.19:0/1953086141 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x560875280f10 0x5608752f3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:23.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.074+0000 7f25fa1cbb80 4 client.25015 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-07T02:30:23.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.074+0000 7f25fa1cbb80 1 -- 172.21.15.19:0/1953086141 --> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] -- command(tid 0: {"prefix": "status"}) -- 0x56087518c450 con 0x560875280f10 2024-05-07T02:30:23.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.074+0000 7f25e8c00700 1 --2- 172.21.15.19:0/1953086141 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x560875280f10 0x5608752f3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:23.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.075+0000 7f25e8c00700 1 --2- 172.21.15.19:0/1953086141 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x560875280f10 0x5608752f3ab0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6ec260607b82f740 server_cookie=ac5895d12647ff63 in_seq=0 out_seq=0 2024-05-07T02:30:23.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.075+0000 7f25caa00700 10 client.25015 ms_handle_connect on v2:172.21.15.27:6836/2583138143 2024-05-07T02:30:23.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.075+0000 7f25caa00700 1 -- 172.21.15.19:0/1953086141 <== mds.3 v2:172.21.15.27:6836/2583138143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56087518c450 con 0x560875280f10 2024-05-07T02:30:23.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.075+0000 7f25caa00700 10 client.25015 handle_command_reply: tid=0 2024-05-07T02:30:23.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.075+0000 7f25fa1cbb80 1 client.25015 shutdown 2024-05-07T02:30:23.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.076+0000 7f25fa1cbb80 1 -- 172.21.15.19:0/1953086141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560875263390 msgr2=0x5608752707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:23.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.076+0000 7f25fa1cbb80 1 --2- 172.21.15.19:0/1953086141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560875263390 0x5608752707c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f25d000b580 tx=0x7f25d0005a10 comp rx=0 tx=0).stop 2024-05-07T02:30:23.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.076+0000 7f25fa1cbb80 1 -- 172.21.15.19:0/1953086141 shutdown_connections 2024-05-07T02:30:23.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.076+0000 7f25fa1cbb80 1 --2- 172.21.15.19:0/1953086141 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x560875280f10 0x5608752f3ab0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:23.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.076+0000 7f25fa1cbb80 1 --2- 172.21.15.19:0/1953086141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56087526cb10 0x560875270d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:23.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.076+0000 7f25fa1cbb80 1 --2- 172.21.15.19:0/1953086141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560875262a40 0x560875270280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:23.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.076+0000 7f25fa1cbb80 1 --2- 172.21.15.19:0/1953086141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560875263390 0x5608752707c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:23.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.076+0000 7f25caa00700 0 client.25015 ms_handle_reset on v2:172.21.15.27:6836/2583138143 2024-05-07T02:30:23.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.076+0000 7f25fa1cbb80 1 -- 172.21.15.19:0/1953086141 >> 172.21.15.19:0/1953086141 conn(0x5608750e4df0 msgr2=0x560875267fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:23.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.076+0000 7f25fa1cbb80 1 -- 172.21.15.19:0/1953086141 shutdown_connections 2024-05-07T02:30:23.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.076+0000 7f25fa1cbb80 1 -- 172.21.15.19:0/1953086141 wait complete. 2024-05-07T02:30:23.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.076+0000 7f25fa1cbb80 20 client.25015 trim_cache size 0 max 16384 2024-05-07T02:30:23.079 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:30:23.079 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "9ff84c16-0c17-11ef-bc97-c7b262605968", 2024-05-07T02:30:23.079 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 3, 2024-05-07T02:30:23.079 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24515, 2024-05-07T02:30:23.079 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-05-07T02:30:23.079 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-05-07T02:30:23.080 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-05-07T02:30:23.080 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 161.360700141, 2024-05-07T02:30:23.080 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 30, 2024-05-07T02:30:23.080 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 89, 2024-05-07T02:30:23.080 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 87, 2024-05-07T02:30:23.080 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 285.45253392799998 2024-05-07T02:30:23.080 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:30:23.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.077+0000 7f25ea000700 1 -- 172.21.15.19:0/1540311966 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f25cc0b2880 msgr2=0x7f25cc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:23.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.077+0000 7f25ea000700 1 --2- 172.21.15.19:0/1540311966 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f25cc0b2880 0x7f25cc0b4d90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x56087525edf0 tx=0x7f25dc009040 comp rx=0 tx=0).stop 2024-05-07T02:30:23.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.077+0000 7f25ea000700 1 -- 172.21.15.19:0/1540311966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25e4104400 msgr2=0x7f25e41a09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:23.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.077+0000 7f25ea000700 1 --2- 172.21.15.19:0/1540311966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25e4104400 0x7f25e41a09c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f25d800e9c0 tx=0x7f25d800ecd0 comp rx=0 tx=0).stop 2024-05-07T02:30:23.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.077+0000 7f25ea000700 1 -- 172.21.15.19:0/1540311966 shutdown_connections 2024-05-07T02:30:23.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.077+0000 7f25ea000700 1 --2- 172.21.15.19:0/1540311966 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f25cc0b2880 0x7f25cc0b4d90 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:23.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.077+0000 7f25ea000700 1 --2- 172.21.15.19:0/1540311966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25e4109680 0x7f25e419ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:23.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.077+0000 7f25ea000700 1 --2- 172.21.15.19:0/1540311966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25e4104400 0x7f25e41a09c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:23.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.077+0000 7f25ea000700 1 --2- 172.21.15.19:0/1540311966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25e4108d30 0x7f25e41a0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:23.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.077+0000 7f25ea000700 1 -- 172.21.15.19:0/1540311966 >> 172.21.15.19:0/1540311966 conn(0x7f25e4079d10 msgr2=0x7f25e41016a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:23.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.077+0000 7f25ea000700 1 -- 172.21.15.19:0/1540311966 shutdown_connections 2024-05-07T02:30:23.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:23.077+0000 7f25ea000700 1 -- 172.21.15.19:0/1540311966 wait complete. 2024-05-07T02:30:24.069 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.i status 2024-05-07T02:30:24.545 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:30:24.699 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:30:24 smithi019 ceph-mon[29781]: pgmap v465: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:30:24.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:30:24 smithi027 ceph-mon[34398]: pgmap v465: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:30:25.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:30:24 smithi064 ceph-mon[34699]: pgmap v465: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:30:26.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:30:26 smithi019 ceph-mon[29781]: pgmap v466: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:30:26.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:30:26 smithi027 ceph-mon[34398]: pgmap v466: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:30:26.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.960+0000 7f762b400700 1 -- 172.21.15.19:0/3955315678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76241092d0 msgr2=0x7f76241096e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:26.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.960+0000 7f762b400700 1 --2- 172.21.15.19:0/3955315678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76241092d0 0x7f76241096e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7618009990 tx=0x7f7618009ca0 comp rx=0 tx=0).stop 2024-05-07T02:30:26.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.960+0000 7f762b400700 1 -- 172.21.15.19:0/3955315678 shutdown_connections 2024-05-07T02:30:26.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.960+0000 7f762b400700 1 --2- 172.21.15.19:0/3955315678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76241031c0 0x7f7624111160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:26.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.960+0000 7f762b400700 1 --2- 172.21.15.19:0/3955315678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76241092d0 0x7f76241096e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:26.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.960+0000 7f762b400700 1 --2- 172.21.15.19:0/3955315678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7624102870 0x7f7624102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:26.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.960+0000 7f762b400700 1 -- 172.21.15.19:0/3955315678 >> 172.21.15.19:0/3955315678 conn(0x7f76240fe520 msgr2=0x7f7624100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:26.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.960+0000 7f762b400700 1 -- 172.21.15.19:0/3955315678 shutdown_connections 2024-05-07T02:30:26.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.960+0000 7f762b400700 1 -- 172.21.15.19:0/3955315678 wait complete. 2024-05-07T02:30:26.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.961+0000 7f762b400700 1 Processor -- start 2024-05-07T02:30:26.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.962+0000 7f762b400700 1 -- start start 2024-05-07T02:30:26.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.962+0000 7f762b400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7624102870 0x7f762419aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:26.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.962+0000 7f762b400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76241031c0 0x7f762419b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:26.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.962+0000 7f762b400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76241092d0 0x7f762419f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:26.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.962+0000 7f762b400700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7624114580 con 0x7f7624102870 2024-05-07T02:30:26.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.962+0000 7f762b400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7624114400 con 0x7f76241031c0 2024-05-07T02:30:26.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.962+0000 7f762b400700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7624114700 con 0x7f76241092d0 2024-05-07T02:30:26.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.962+0000 7f7628a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7624102870 0x7f762419aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:26.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.962+0000 7f7628a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7624102870 0x7f762419aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47930/0 (socket says 172.21.15.19:47930) 2024-05-07T02:30:26.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.962+0000 7f7628a00700 1 -- 172.21.15.19:0/2430372538 learned_addr learned my addr 172.21.15.19:0/2430372538 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:30:26.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.963+0000 7f7623e00700 1 --2- 172.21.15.19:0/2430372538 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76241031c0 0x7f762419b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:26.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.963+0000 7f7629400700 1 --2- 172.21.15.19:0/2430372538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76241092d0 0x7f762419f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:26.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.963+0000 7f7623e00700 1 -- 172.21.15.19:0/2430372538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76241092d0 msgr2=0x7f762419f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:26.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.963+0000 7f7623e00700 1 --2- 172.21.15.19:0/2430372538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76241092d0 0x7f762419f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:26.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.963+0000 7f7623e00700 1 -- 172.21.15.19:0/2430372538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7624102870 msgr2=0x7f762419aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:26.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.963+0000 7f7623e00700 1 --2- 172.21.15.19:0/2430372538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7624102870 0x7f762419aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:26.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.963+0000 7f7623e00700 1 -- 172.21.15.19:0/2430372538 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f762419fda0 con 0x7f76241031c0 2024-05-07T02:30:26.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.963+0000 7f7628a00700 1 --2- 172.21.15.19:0/2430372538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7624102870 0x7f762419aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:30:26.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.963+0000 7f7623e00700 1 --2- 172.21.15.19:0/2430372538 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76241031c0 0x7f762419b470 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f761000e9c0 tx=0x7f761000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:26.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.964+0000 7f7621600700 1 -- 172.21.15.19:0/2430372538 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761000c9c0 con 0x7f76241031c0 2024-05-07T02:30:26.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.964+0000 7f762b400700 1 -- 172.21.15.19:0/2430372538 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76241a0090 con 0x7f76241031c0 2024-05-07T02:30:26.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.964+0000 7f7621600700 1 -- 172.21.15.19:0/2430372538 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f761000cb60 con 0x7f76241031c0 2024-05-07T02:30:26.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.964+0000 7f7621600700 1 -- 172.21.15.19:0/2430372538 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7610018680 con 0x7f76241031c0 2024-05-07T02:30:26.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.964+0000 7f762b400700 1 -- 172.21.15.19:0/2430372538 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76241a79b0 con 0x7f76241031c0 2024-05-07T02:30:26.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.968+0000 7f7621600700 1 -- 172.21.15.19:0/2430372538 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7610010430 con 0x7f76241031c0 2024-05-07T02:30:26.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.969+0000 7f7639f71b80 1 Processor -- start 2024-05-07T02:30:26.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.969+0000 7f7639f71b80 1 -- start start 2024-05-07T02:30:26.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.969+0000 7f7639f71b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a91917360 0x559a91917770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:26.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.969+0000 7f7639f71b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a91911420 0x559a91911830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:26.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.969+0000 7f7639f71b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559a91911d70 0x559a9191dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:26.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.969+0000 7f7639f71b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559a918339d0 con 0x559a91917360 2024-05-07T02:30:26.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.969+0000 7f7639f71b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559a917d1710 con 0x559a91911d70 2024-05-07T02:30:26.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.969+0000 7f7639f71b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559a9182f990 con 0x559a91911420 2024-05-07T02:30:26.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.970+0000 7f7621600700 1 --2- 172.21.15.19:0/2430372538 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f760c0b2880 0x7f760c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:26.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.970+0000 7f7628a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a91911420 0x559a91911830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:26.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.970+0000 7f7629400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a91917360 0x559a91917770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:26.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.970+0000 7f7628a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a91911420 0x559a91911830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:43072/0 (socket says 172.21.15.19:43072) 2024-05-07T02:30:26.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.970+0000 7f7628a00700 1 -- 172.21.15.19:0/3812401929 learned_addr learned my addr 172.21.15.19:0/3812401929 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:30:26.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.970+0000 7f7629400700 1 --2- 172.21.15.19:0/2430372538 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f760c0b2880 0x7f760c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:26.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.970+0000 7f7621600700 1 -- 172.21.15.19:0/2430372538 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f76100e2790 con 0x7f76241031c0 2024-05-07T02:30:26.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.970+0000 7f7623e00700 1 --2- 172.21.15.19:0/3812401929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559a91911d70 0x559a9191dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:26.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.971+0000 7f7628a00700 1 -- 172.21.15.19:0/3812401929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559a91911d70 msgr2=0x559a9191dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:30:26.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.971+0000 7f7628a00700 1 --2- 172.21.15.19:0/3812401929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559a91911d70 0x559a9191dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:26.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.971+0000 7f7628a00700 1 -- 172.21.15.19:0/3812401929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a91917360 msgr2=0x559a91917770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:30:26.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.971+0000 7f7628a00700 1 --2- 172.21.15.19:0/3812401929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a91917360 0x559a91917770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:26.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.971+0000 7f7628a00700 1 -- 172.21.15.19:0/3812401929 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a91780880 con 0x559a91911420 2024-05-07T02:30:26.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.971+0000 7f7628a00700 1 --2- 172.21.15.19:0/3812401929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a91911420 0x559a91911830 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7618009750 tx=0x7f761800bad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e7ecddc1401734d server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:26.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.971+0000 7f7629400700 1 --2- 172.21.15.19:0/2430372538 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f760c0b2880 0x7f760c0b4d90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f761c009ec0 tx=0x7f761c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:26.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.971+0000 7f7616000700 1 -- 172.21.15.19:0/3812401929 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7618005270 con 0x559a91911420 2024-05-07T02:30:26.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.971+0000 7f7616000700 1 -- 172.21.15.19:0/3812401929 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7618005410 con 0x559a91911420 2024-05-07T02:30:26.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.971+0000 7f7616000700 1 -- 172.21.15.19:0/3812401929 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761800f750 con 0x559a91911420 2024-05-07T02:30:26.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.972+0000 7f7639f71b80 1 -- 172.21.15.19:0/3812401929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a91911420 msgr2=0x559a91911830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:26.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.972+0000 7f7639f71b80 1 --2- 172.21.15.19:0/3812401929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a91911420 0x559a91911830 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7618009750 tx=0x7f761800bad0 comp rx=0 tx=0).stop 2024-05-07T02:30:26.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.972+0000 7f7639f71b80 1 -- 172.21.15.19:0/3812401929 shutdown_connections 2024-05-07T02:30:26.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.972+0000 7f7639f71b80 1 --2- 172.21.15.19:0/3812401929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559a91911d70 0x559a9191dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:26.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.972+0000 7f7639f71b80 1 --2- 172.21.15.19:0/3812401929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a91917360 0x559a91917770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:26.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.972+0000 7f7639f71b80 1 --2- 172.21.15.19:0/3812401929 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a91911420 0x559a91911830 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:26.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.972+0000 7f7639f71b80 1 -- 172.21.15.19:0/3812401929 >> 172.21.15.19:0/3812401929 conn(0x559a91793df0 msgr2=0x559a9190c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:26.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.973+0000 7f7639f71b80 1 -- 172.21.15.19:0/3812401929 shutdown_connections 2024-05-07T02:30:26.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.973+0000 7f7639f71b80 1 -- 172.21.15.19:0/3812401929 wait complete. 2024-05-07T02:30:26.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.973+0000 7f7639f71b80 1 -- start start 2024-05-07T02:30:26.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.973+0000 7f7639f71b80 1 Processor -- start 2024-05-07T02:30:26.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.974+0000 7f7639f71b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a91911420 0x559a91925b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:26.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.974+0000 7f7639f71b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a91911d70 0x559a91926080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:26.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.974+0000 7f7639f71b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559a91917360 0x559a9191ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:26.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.974+0000 7f7639f71b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559a9191e6b0 con 0x559a91911420 2024-05-07T02:30:26.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.974+0000 7f7639f71b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559a91799ad0 con 0x559a91917360 2024-05-07T02:30:26.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.974+0000 7f7639f71b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559a9191e830 con 0x559a91911d70 2024-05-07T02:30:26.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.974+0000 7f7639f71b80 10 client.0 _ll_register_callbacks cb 0x559a917dac20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:30:26.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.974+0000 7f7628a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a91911420 0x559a91925b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:26.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.974+0000 7f7628a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a91911420 0x559a91925b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47956/0 (socket says 172.21.15.19:47956) 2024-05-07T02:30:26.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.974+0000 7f7628a00700 1 -- 172.21.15.19:0/3334226990 learned_addr learned my addr 172.21.15.19:0/3334226990 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:30:26.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.974+0000 7f7623e00700 1 --2- 172.21.15.19:0/3334226990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a91911d70 0x559a91926080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:26.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.975+0000 7f7629400700 1 --2- 172.21.15.19:0/3334226990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559a91917360 0x559a9191ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:26.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.975+0000 7f7628a00700 1 -- 172.21.15.19:0/3334226990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a91911d70 msgr2=0x559a91926080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:30:26.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.975+0000 7f7628a00700 1 --2- 172.21.15.19:0/3334226990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a91911d70 0x559a91926080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:26.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.975+0000 7f7628a00700 1 -- 172.21.15.19:0/3334226990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559a91917360 msgr2=0x559a9191ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:30:26.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.975+0000 7f7628a00700 1 --2- 172.21.15.19:0/3334226990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559a91917360 0x559a9191ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:26.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.975+0000 7f7628a00700 1 -- 172.21.15.19:0/3334226990 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a917f93d0 con 0x559a91911420 2024-05-07T02:30:26.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.975+0000 7f7623e00700 1 --2- 172.21.15.19:0/3334226990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a91911d70 0x559a91926080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:30:26.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.975+0000 7f7628a00700 1 --2- 172.21.15.19:0/3334226990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a91911420 0x559a91925b40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f761800c010 tx=0x7f7618005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72d18821e93c7dd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:26.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.975+0000 7f760aa00700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-07T02:30:26.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.976+0000 7f760aa00700 1 -- 172.21.15.19:0/3334226990 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7618030020 con 0x559a91911420 2024-05-07T02:30:26.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.976+0000 7f760aa00700 1 -- 172.21.15.19:0/3334226990 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f7618003680 con 0x559a91911420 2024-05-07T02:30:26.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.976+0000 7f7639f71b80 1 -- 172.21.15.19:0/3334226990 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7618005410 con 0x559a91911420 2024-05-07T02:30:26.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.976+0000 7f760aa00700 1 -- 172.21.15.19:0/3334226990 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f76180038a0 con 0x559a91911420 2024-05-07T02:30:26.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.976+0000 7f7639f71b80 10 client.14889 fetch_fsmap learned FSMap version 30 2024-05-07T02:30:26.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.976+0000 7f7639f71b80 1 -- 172.21.15.19:0/3334226990 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559a9192d760 con 0x559a91911420 2024-05-07T02:30:26.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.976+0000 7f760aa00700 1 -- 172.21.15.19:0/3334226990 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7618005410 con 0x559a91911420 2024-05-07T02:30:26.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.977+0000 7f760aa00700 1 -- 172.21.15.19:0/3334226990 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f761802c450 con 0x559a91911420 2024-05-07T02:30:26.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.977+0000 7f7639f71b80 10 client.14889 fetch_fsmap finished waiting for FSMap version 30 2024-05-07T02:30:26.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.977+0000 7f7639f71b80 10 client.14889 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-07T02:30:26.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.977+0000 7f7639f71b80 20 client.14889 populate_metadata read hostname 'smithi019' 2024-05-07T02:30:26.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.977+0000 7f7639f71b80 1 --2- 172.21.15.19:0/3334226990 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x559a9192eab0 0x559a91930c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:26.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.977+0000 7f7639f71b80 4 client.14889 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:30:26.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.977+0000 7f7639f71b80 1 -- 172.21.15.19:0/3334226990 --> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a918de290 con 0x559a9192eab0 2024-05-07T02:30:26.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.977+0000 7f7623e00700 1 --2- 172.21.15.19:0/3334226990 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x559a9192eab0 0x559a91930c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:26.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.978+0000 7f7623e00700 1 --2- 172.21.15.19:0/3334226990 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x559a9192eab0 0x559a91930c20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cf59bd38dbc842a8 server_cookie=5c51ac88fae75fe5 in_seq=0 out_seq=0 2024-05-07T02:30:26.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.978+0000 7f760aa00700 10 client.14889 ms_handle_connect on v2:172.21.15.64:6838/1583120004 2024-05-07T02:30:26.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.979+0000 7f760aa00700 1 -- 172.21.15.19:0/3334226990 <== mds.2 v2:172.21.15.64:6838/1583120004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559a918de290 con 0x559a9192eab0 2024-05-07T02:30:26.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.979+0000 7f760aa00700 10 client.14889 handle_command_reply: tid=0 2024-05-07T02:30:26.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.979+0000 7f7639f71b80 1 client.14889 shutdown 2024-05-07T02:30:26.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.979+0000 7f7639f71b80 1 -- 172.21.15.19:0/3334226990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a91911420 msgr2=0x559a91925b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:26.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.979+0000 7f7639f71b80 1 --2- 172.21.15.19:0/3334226990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a91911420 0x559a91925b40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f761800c010 tx=0x7f7618005a40 comp rx=0 tx=0).stop 2024-05-07T02:30:26.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.979+0000 7f7639f71b80 1 -- 172.21.15.19:0/3334226990 shutdown_connections 2024-05-07T02:30:26.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.979+0000 7f7639f71b80 1 --2- 172.21.15.19:0/3334226990 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x559a9192eab0 0x559a91930c20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:26.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.979+0000 7f7639f71b80 1 --2- 172.21.15.19:0/3334226990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559a91917360 0x559a9191ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:26.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.979+0000 7f760aa00700 0 client.14889 ms_handle_reset on v2:172.21.15.64:6838/1583120004 2024-05-07T02:30:26.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.979+0000 7f7639f71b80 1 --2- 172.21.15.19:0/3334226990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a91911420 0x559a91925b40 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:26.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.979+0000 7f7639f71b80 1 --2- 172.21.15.19:0/3334226990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a91911d70 0x559a91926080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:26.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.979+0000 7f7639f71b80 1 -- 172.21.15.19:0/3334226990 >> 172.21.15.19:0/3334226990 conn(0x559a91793df0 msgr2=0x559a9190e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:26.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.980+0000 7f7639f71b80 1 -- 172.21.15.19:0/3334226990 shutdown_connections 2024-05-07T02:30:26.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.980+0000 7f7639f71b80 1 -- 172.21.15.19:0/3334226990 wait complete. 2024-05-07T02:30:26.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:26.980+0000 7f7639f71b80 20 client.14889 trim_cache size 0 max 16384 2024-05-07T02:30:27.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.003+0000 7f7639f71b80 1 Processor -- start 2024-05-07T02:30:27.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.003+0000 7f7639f71b80 1 -- start start 2024-05-07T02:30:27.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.003+0000 7f7639f71b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559a91911d70 0x559a9192f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:27.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.003+0000 7f7639f71b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a91917360 0x559a9193a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:27.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.003+0000 7f7639f71b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a9193a5c0 0x559a9191f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:27.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.003+0000 7f7639f71b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559a9192ef60 con 0x559a9193a5c0 2024-05-07T02:30:27.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.003+0000 7f7639f71b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559a9192f630 con 0x559a91911d70 2024-05-07T02:30:27.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.003+0000 7f7639f71b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559a9192fd00 con 0x559a91917360 2024-05-07T02:30:27.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.004+0000 7f7623e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a91917360 0x559a9193a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:27.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.004+0000 7f7628a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559a91911d70 0x559a9192f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:27.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.004+0000 7f7629400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a9193a5c0 0x559a9191f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:27.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.004+0000 7f7623e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a91917360 0x559a9193a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:43100/0 (socket says 172.21.15.19:43100) 2024-05-07T02:30:27.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.004+0000 7f7623e00700 1 -- 172.21.15.19:0/3781908921 learned_addr learned my addr 172.21.15.19:0/3781908921 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:30:27.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.004+0000 7f7623e00700 1 -- 172.21.15.19:0/3781908921 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559a91911d70 msgr2=0x559a9192f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:30:27.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.004+0000 7f7623e00700 1 --2- 172.21.15.19:0/3781908921 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559a91911d70 0x559a9192f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:27.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.004+0000 7f7623e00700 1 -- 172.21.15.19:0/3781908921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a9193a5c0 msgr2=0x559a9191f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:30:27.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.004+0000 7f7623e00700 1 --2- 172.21.15.19:0/3781908921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a9193a5c0 0x559a9191f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:27.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.004+0000 7f7623e00700 1 -- 172.21.15.19:0/3781908921 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a9191e330 con 0x559a91917360 2024-05-07T02:30:27.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.004+0000 7f7628a00700 1 --2- 172.21.15.19:0/3781908921 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559a91911d70 0x559a9192f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:30:27.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.005+0000 7f7623e00700 1 --2- 172.21.15.19:0/3781908921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a91917360 0x559a9193a080 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f761000c5b0 tx=0x7f76100cc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbea436052353d0d server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:27.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.005+0000 7f7615600700 1 -- 172.21.15.19:0/3781908921 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76100aa0a0 con 0x559a91917360 2024-05-07T02:30:27.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.005+0000 7f7615600700 1 -- 172.21.15.19:0/3781908921 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76100a99a0 con 0x559a91917360 2024-05-07T02:30:27.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.005+0000 7f7615600700 1 -- 172.21.15.19:0/3781908921 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76100a9c00 con 0x559a91917360 2024-05-07T02:30:27.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.006+0000 7f7639f71b80 1 -- 172.21.15.19:0/3781908921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a91917360 msgr2=0x559a9193a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:27.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.006+0000 7f7639f71b80 1 --2- 172.21.15.19:0/3781908921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a91917360 0x559a9193a080 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f761000c5b0 tx=0x7f76100cc040 comp rx=0 tx=0).stop 2024-05-07T02:30:27.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.006+0000 7f7639f71b80 1 -- 172.21.15.19:0/3781908921 shutdown_connections 2024-05-07T02:30:27.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.006+0000 7f7639f71b80 1 --2- 172.21.15.19:0/3781908921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a9193a5c0 0x559a9191f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:27.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.006+0000 7f7639f71b80 1 --2- 172.21.15.19:0/3781908921 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559a91911d70 0x559a9192f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:27.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.006+0000 7f7639f71b80 1 --2- 172.21.15.19:0/3781908921 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a91917360 0x559a9193a080 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:27.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.006+0000 7f7639f71b80 1 -- 172.21.15.19:0/3781908921 >> 172.21.15.19:0/3781908921 conn(0x559a91793df0 msgr2=0x559a91917770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:27.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.006+0000 7f7639f71b80 1 -- 172.21.15.19:0/3781908921 shutdown_connections 2024-05-07T02:30:27.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.006+0000 7f7639f71b80 1 -- 172.21.15.19:0/3781908921 wait complete. 2024-05-07T02:30:27.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.007+0000 7f7639f71b80 1 -- start start 2024-05-07T02:30:27.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.007+0000 7f7639f71b80 1 Processor -- start 2024-05-07T02:30:27.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.007+0000 7f7639f71b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a91911d70 0x559a91928470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:27.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.007+0000 7f7639f71b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a91917360 0x559a919289b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:27.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.007+0000 7f7639f71b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559a9193a5c0 0x559a91999c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:27.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.007+0000 7f7639f71b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559a91930aa0 con 0x559a91917360 2024-05-07T02:30:27.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.007+0000 7f7639f71b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559a919303d0 con 0x559a9193a5c0 2024-05-07T02:30:27.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.007+0000 7f7639f71b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559a91931170 con 0x559a91911d70 2024-05-07T02:30:27.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.008+0000 7f7639f71b80 10 client.0 _ll_register_callbacks cb 0x559a917dac20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:30:27.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.008+0000 7f7623e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a91917360 0x559a919289b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:27.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.008+0000 7f7628a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a91911d70 0x559a91928470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:27.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.008+0000 7f7628a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a91911d70 0x559a91928470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:43102/0 (socket says 172.21.15.19:43102) 2024-05-07T02:30:27.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.008+0000 7f7628a00700 1 -- 172.21.15.19:0/2643216365 learned_addr learned my addr 172.21.15.19:0/2643216365 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:30:27.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.008+0000 7f7629400700 1 --2- 172.21.15.19:0/2643216365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559a9193a5c0 0x559a91999c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:27.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.008+0000 7f7628a00700 1 -- 172.21.15.19:0/2643216365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559a9193a5c0 msgr2=0x559a91999c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:30:27.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.008+0000 7f7628a00700 1 --2- 172.21.15.19:0/2643216365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559a9193a5c0 0x559a91999c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:27.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.008+0000 7f7628a00700 1 -- 172.21.15.19:0/2643216365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a91917360 msgr2=0x559a919289b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:30:27.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.008+0000 7f7628a00700 1 --2- 172.21.15.19:0/2643216365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a91917360 0x559a919289b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:27.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.008+0000 7f7628a00700 1 -- 172.21.15.19:0/2643216365 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a9199a160 con 0x559a91911d70 2024-05-07T02:30:27.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.009+0000 7f7628a00700 1 --2- 172.21.15.19:0/2643216365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a91911d70 0x559a91928470 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f76180098e0 tx=0x7f76180059c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0e9fca75ff0882c server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:27.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.009+0000 7f760aa00700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:30:27.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.009+0000 7f760aa00700 1 -- 172.21.15.19:0/2643216365 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7618030040 con 0x559a91911d70 2024-05-07T02:30:27.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.009+0000 7f7639f71b80 1 -- 172.21.15.19:0/2643216365 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76100a99a0 con 0x559a91911d70 2024-05-07T02:30:27.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.009+0000 7f760aa00700 1 -- 172.21.15.19:0/2643216365 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f761802cc90 con 0x559a91911d70 2024-05-07T02:30:27.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.009+0000 7f760aa00700 1 -- 172.21.15.19:0/2643216365 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7618048050 con 0x559a91911d70 2024-05-07T02:30:27.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.009+0000 7f7639f71b80 10 client.25033 fetch_fsmap learned FSMap version 30 2024-05-07T02:30:27.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.009+0000 7f7639f71b80 1 -- 172.21.15.19:0/2643216365 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559a9199b2e0 con 0x559a91911d70 2024-05-07T02:30:27.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.010+0000 7f760aa00700 1 -- 172.21.15.19:0/2643216365 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76100a99a0 con 0x559a91911d70 2024-05-07T02:30:27.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.010+0000 7f760aa00700 1 -- 172.21.15.19:0/2643216365 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f761803e030 con 0x559a91911d70 2024-05-07T02:30:27.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.010+0000 7f7639f71b80 10 client.25033 fetch_fsmap finished waiting for FSMap version 30 2024-05-07T02:30:27.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.010+0000 7f7639f71b80 10 client.25033 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-07T02:30:27.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.010+0000 7f7639f71b80 20 client.25033 populate_metadata read hostname 'smithi019' 2024-05-07T02:30:27.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.010+0000 7f7639f71b80 1 --2- 172.21.15.19:0/2643216365 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x559a9199d4e0 0x559a9199e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:27.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.010+0000 7f7639f71b80 4 client.25033 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-07T02:30:27.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.010+0000 7f7639f71b80 1 -- 172.21.15.19:0/2643216365 --> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] -- command(tid 0: {"prefix": "status"}) -- 0x559a918413b0 con 0x559a9199d4e0 2024-05-07T02:30:27.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.010+0000 7f7623e00700 1 --2- 172.21.15.19:0/2643216365 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x559a9199d4e0 0x559a9199e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:27.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.011+0000 7f7623e00700 1 --2- 172.21.15.19:0/2643216365 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x559a9199d4e0 0x559a9199e950 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2770408e577bfecc server_cookie=9e4760a5c7194fe2 in_seq=0 out_seq=0 2024-05-07T02:30:27.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.011+0000 7f760aa00700 10 client.25033 ms_handle_connect on v2:172.21.15.64:6838/1583120004 2024-05-07T02:30:27.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.011+0000 7f760aa00700 1 -- 172.21.15.19:0/2643216365 <== mds.2 v2:172.21.15.64:6838/1583120004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x559a918413b0 con 0x559a9199d4e0 2024-05-07T02:30:27.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.011+0000 7f760aa00700 10 client.25033 handle_command_reply: tid=0 2024-05-07T02:30:27.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.011+0000 7f7639f71b80 1 client.25033 shutdown 2024-05-07T02:30:27.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.011+0000 7f7639f71b80 1 -- 172.21.15.19:0/2643216365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a91911d70 msgr2=0x559a91928470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:27.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.011+0000 7f7639f71b80 1 --2- 172.21.15.19:0/2643216365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a91911d70 0x559a91928470 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f76180098e0 tx=0x7f76180059c0 comp rx=0 tx=0).stop 2024-05-07T02:30:27.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.012+0000 7f7639f71b80 1 -- 172.21.15.19:0/2643216365 shutdown_connections 2024-05-07T02:30:27.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.012+0000 7f7639f71b80 1 --2- 172.21.15.19:0/2643216365 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x559a9199d4e0 0x559a9199e950 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:27.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.012+0000 7f7639f71b80 1 --2- 172.21.15.19:0/2643216365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559a9193a5c0 0x559a91999c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:27.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.012+0000 7f7639f71b80 1 --2- 172.21.15.19:0/2643216365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559a91911d70 0x559a91928470 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:27.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.012+0000 7f7639f71b80 1 --2- 172.21.15.19:0/2643216365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a91917360 0x559a919289b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:27.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.012+0000 7f760aa00700 0 client.25033 ms_handle_reset on v2:172.21.15.64:6838/1583120004 2024-05-07T02:30:27.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.012+0000 7f7639f71b80 1 -- 172.21.15.19:0/2643216365 >> 172.21.15.19:0/2643216365 conn(0x559a91793df0 msgr2=0x559a91917770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:27.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.012+0000 7f7639f71b80 1 -- 172.21.15.19:0/2643216365 shutdown_connections 2024-05-07T02:30:27.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.012+0000 7f7639f71b80 1 -- 172.21.15.19:0/2643216365 wait complete. 2024-05-07T02:30:27.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.012+0000 7f7639f71b80 20 client.25033 trim_cache size 0 max 16384 2024-05-07T02:30:27.015 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:30:27.015 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "9ff84c16-0c17-11ef-bc97-c7b262605968", 2024-05-07T02:30:27.015 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 2, 2024-05-07T02:30:27.015 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24520, 2024-05-07T02:30:27.015 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-05-07T02:30:27.015 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-05-07T02:30:27.015 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-05-07T02:30:27.015 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 166.30835286000001, 2024-05-07T02:30:27.015 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 30, 2024-05-07T02:30:27.015 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 89, 2024-05-07T02:30:27.016 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 87, 2024-05-07T02:30:27.016 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 286.55389685799997 2024-05-07T02:30:27.016 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:30:27.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.013+0000 7f762b400700 1 -- 172.21.15.19:0/2430372538 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f760c0b2880 msgr2=0x7f760c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:27.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.013+0000 7f762b400700 1 --2- 172.21.15.19:0/2430372538 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f760c0b2880 0x7f760c0b4d90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f761c009ec0 tx=0x7f761c00b040 comp rx=0 tx=0).stop 2024-05-07T02:30:27.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.013+0000 7f762b400700 1 -- 172.21.15.19:0/2430372538 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76241031c0 msgr2=0x7f762419b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:27.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.013+0000 7f762b400700 1 --2- 172.21.15.19:0/2430372538 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76241031c0 0x7f762419b470 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f761000e9c0 tx=0x7f761000ecd0 comp rx=0 tx=0).stop 2024-05-07T02:30:27.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.013+0000 7f762b400700 1 -- 172.21.15.19:0/2430372538 shutdown_connections 2024-05-07T02:30:27.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.013+0000 7f762b400700 1 --2- 172.21.15.19:0/2430372538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f76241092d0 0x7f762419f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:27.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.013+0000 7f762b400700 1 --2- 172.21.15.19:0/2430372538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7624102870 0x7f762419aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:27.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.013+0000 7f762b400700 1 --2- 172.21.15.19:0/2430372538 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f760c0b2880 0x7f760c0b4d90 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:27.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.013+0000 7f762b400700 1 --2- 172.21.15.19:0/2430372538 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76241031c0 0x7f762419b470 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:27.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.013+0000 7f762b400700 1 -- 172.21.15.19:0/2430372538 >> 172.21.15.19:0/2430372538 conn(0x7f76240fe520 msgr2=0x7f7624100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:27.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.013+0000 7f762b400700 1 -- 172.21.15.19:0/2430372538 shutdown_connections 2024-05-07T02:30:27.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:27.013+0000 7f762b400700 1 -- 172.21.15.19:0/2430372538 wait complete. 2024-05-07T02:30:27.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:30:26 smithi064 ceph-mon[34699]: pgmap v466: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:30:27.961 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.a status 2024-05-07T02:30:28.446 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:30:28.860 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:30:28 smithi019 ceph-mon[29781]: pgmap v467: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:30:28.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:30:28 smithi027 ceph-mon[34398]: pgmap v467: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:30:29.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:30:28 smithi064 ceph-mon[34699]: pgmap v467: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:30:30.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:30:30 smithi019 ceph-mon[29781]: pgmap v468: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:30:30.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:30:30 smithi027 ceph-mon[34398]: pgmap v468: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:30:30.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.987+0000 7f5ffcc00700 1 -- 172.21.15.19:0/3253251734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ff810b8a0 msgr2=0x7f5ff8113200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:30.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.987+0000 7f5ffcc00700 1 --2- 172.21.15.19:0/3253251734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ff810b8a0 0x7f5ff8113200 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0009f90 tx=0x7f5ff000b330 comp rx=0 tx=0).stop 2024-05-07T02:30:30.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.987+0000 7f5ffcc00700 1 -- 172.21.15.19:0/3253251734 shutdown_connections 2024-05-07T02:30:30.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.987+0000 7f5ffcc00700 1 --2- 172.21.15.19:0/3253251734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ff810b8a0 0x7f5ff8113200 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:30.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.987+0000 7f5ffcc00700 1 --2- 172.21.15.19:0/3253251734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ff8106620 0x7f5ff8106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:30.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.987+0000 7f5ffcc00700 1 --2- 172.21.15.19:0/3253251734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ff810af50 0x7f5ff810b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:30.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.987+0000 7f5ffcc00700 1 -- 172.21.15.19:0/3253251734 >> 172.21.15.19:0/3253251734 conn(0x7f5ff8102030 msgr2=0x7f5ff8104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:30.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.987+0000 7f5ffcc00700 1 -- 172.21.15.19:0/3253251734 shutdown_connections 2024-05-07T02:30:30.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.987+0000 7f5ffcc00700 1 -- 172.21.15.19:0/3253251734 wait complete. 2024-05-07T02:30:30.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.988+0000 7f5ffcc00700 1 Processor -- start 2024-05-07T02:30:30.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.989+0000 7f5ffcc00700 1 -- start start 2024-05-07T02:30:30.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.989+0000 7f5ffcc00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ff8106620 0x7f5ff819ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:30.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.989+0000 7f5ffcc00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ff810af50 0x7f5ff819d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:30.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.989+0000 7f5ffcc00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ff810b8a0 0x7f5ff81a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:30.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.989+0000 7f5ffcc00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff8116520 con 0x7f5ff810af50 2024-05-07T02:30:30.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.989+0000 7f5ffcc00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff81163a0 con 0x7f5ff810b8a0 2024-05-07T02:30:30.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.989+0000 7f5ffcc00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff81166a0 con 0x7f5ff8106620 2024-05-07T02:30:30.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.990+0000 7f5ff7400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ff8106620 0x7f5ff819ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:30.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.990+0000 7f5ff6a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ff810af50 0x7f5ff819d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:30.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.990+0000 7f5ff7e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ff810b8a0 0x7f5ff81a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:30.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.990+0000 7f5ff7e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ff810b8a0 0x7f5ff81a1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:36390/0 (socket says 172.21.15.19:36390) 2024-05-07T02:30:30.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.990+0000 7f5ff7400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ff8106620 0x7f5ff819ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:40220/0 (socket says 172.21.15.19:40220) 2024-05-07T02:30:30.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.990+0000 7f5ff7e00700 1 -- 172.21.15.19:0/2350565402 learned_addr learned my addr 172.21.15.19:0/2350565402 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:30:30.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.990+0000 7f5ff7e00700 1 -- 172.21.15.19:0/2350565402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ff8106620 msgr2=0x7f5ff819ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:30.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.990+0000 7f5ff7e00700 1 --2- 172.21.15.19:0/2350565402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ff8106620 0x7f5ff819ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:30.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.990+0000 7f5ff7e00700 1 -- 172.21.15.19:0/2350565402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ff810af50 msgr2=0x7f5ff819d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:30.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.990+0000 7f5ff7e00700 1 --2- 172.21.15.19:0/2350565402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ff810af50 0x7f5ff819d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:30.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.990+0000 7f5ff7e00700 1 -- 172.21.15.19:0/2350565402 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ff81a1d00 con 0x7f5ff810b8a0 2024-05-07T02:30:30.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.990+0000 7f5ff7e00700 1 --2- 172.21.15.19:0/2350565402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ff810b8a0 0x7f5ff81a1660 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5ff000b6c0 tx=0x7f5ff000cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:30.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.991+0000 7f5fefe00700 1 -- 172.21.15.19:0/2350565402 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0021070 con 0x7f5ff810b8a0 2024-05-07T02:30:30.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.991+0000 7f5fefe00700 1 -- 172.21.15.19:0/2350565402 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ff001d450 con 0x7f5ff810b8a0 2024-05-07T02:30:30.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.991+0000 7f5ffcc00700 1 -- 172.21.15.19:0/2350565402 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ff81a1f90 con 0x7f5ff810b8a0 2024-05-07T02:30:30.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.991+0000 7f5fefe00700 1 -- 172.21.15.19:0/2350565402 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff00052a0 con 0x7f5ff810b8a0 2024-05-07T02:30:30.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.991+0000 7f5ffcc00700 1 -- 172.21.15.19:0/2350565402 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ff81a9910 con 0x7f5ff810b8a0 2024-05-07T02:30:30.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.996+0000 7f5fefe00700 1 -- 172.21.15.19:0/2350565402 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5ff0024070 con 0x7f5ff810b8a0 2024-05-07T02:30:30.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.997+0000 7f600cca2b80 1 Processor -- start 2024-05-07T02:30:30.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.998+0000 7f600cca2b80 1 -- start start 2024-05-07T02:30:30.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.998+0000 7f5fefe00700 1 --2- 172.21.15.19:0/2350565402 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f5fdc0b2900 0x7f5fdc0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:30.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.998+0000 7f600cca2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633f7ecaa40 0x5633f7ecae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:30.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.998+0000 7f600cca2b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633f7ecb390 0x5633f7ed45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:30.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.998+0000 7f600cca2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633f7ed4b10 0x5633f7ed6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:30.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.998+0000 7f600cca2b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5633f7dec9d0 con 0x5633f7ed4b10 2024-05-07T02:30:30.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.998+0000 7f600cca2b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5633f7d8a710 con 0x5633f7ecb390 2024-05-07T02:30:31.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.998+0000 7f600cca2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5633f7de8990 con 0x5633f7ecaa40 2024-05-07T02:30:31.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.998+0000 7f5fefe00700 1 -- 172.21.15.19:0/2350565402 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f5ff000f020 con 0x7f5ff810b8a0 2024-05-07T02:30:31.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.999+0000 7f5ff6a00700 1 --2- 172.21.15.19:0/2350565402 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f5fdc0b2900 0x7f5fdc0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:31.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.999+0000 7f5ff7400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633f7ecaa40 0x5633f7ecae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:31.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.999+0000 7f5ff7400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633f7ecaa40 0x5633f7ecae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:40228/0 (socket says 172.21.15.19:40228) 2024-05-07T02:30:31.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.999+0000 7f5ff7400700 1 -- 172.21.15.19:0/2206106097 learned_addr learned my addr 172.21.15.19:0/2206106097 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:30:31.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.999+0000 7f5ff7e00700 1 --2- 172.21.15.19:0/2206106097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633f7ed4b10 0x5633f7ed6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:31.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.999+0000 7f5ff6a00700 1 --2- 172.21.15.19:0/2206106097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633f7ecb390 0x5633f7ed45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:31.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.999+0000 7f5ff7400700 1 -- 172.21.15.19:0/2206106097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633f7ecb390 msgr2=0x5633f7ed45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:30:31.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.999+0000 7f5ff7400700 1 --2- 172.21.15.19:0/2206106097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633f7ecb390 0x5633f7ed45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:31.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.999+0000 7f5ff7400700 1 -- 172.21.15.19:0/2206106097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633f7ed4b10 msgr2=0x5633f7ed6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:30:31.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.999+0000 7f5ff7400700 1 --2- 172.21.15.19:0/2206106097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633f7ed4b10 0x5633f7ed6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:31.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:30.999+0000 7f5ff7400700 1 -- 172.21.15.19:0/2206106097 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633f7d39880 con 0x5633f7ecaa40 2024-05-07T02:30:31.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.000+0000 7f5ff7400700 1 --2- 172.21.15.19:0/2206106097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633f7ecaa40 0x5633f7ecae50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8007cf0 tx=0x7f5fe80103d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84720b0f24a232bc server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:31.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.000+0000 7f5ff6a00700 1 --2- 172.21.15.19:0/2350565402 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f5fdc0b2900 0x7f5fdc0b4e10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x5633f7ec6c70 tx=0x7f5fe40047c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:31.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.000+0000 7f5fecc00700 1 -- 172.21.15.19:0/2206106097 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe8010d10 con 0x5633f7ecaa40 2024-05-07T02:30:31.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.000+0000 7f5fecc00700 1 -- 172.21.15.19:0/2206106097 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fe80118f0 con 0x5633f7ecaa40 2024-05-07T02:30:31.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.000+0000 7f5fecc00700 1 -- 172.21.15.19:0/2206106097 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe8019750 con 0x5633f7ecaa40 2024-05-07T02:30:31.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.001+0000 7f600cca2b80 1 -- 172.21.15.19:0/2206106097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633f7ecaa40 msgr2=0x5633f7ecae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:31.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.001+0000 7f600cca2b80 1 --2- 172.21.15.19:0/2206106097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633f7ecaa40 0x5633f7ecae50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8007cf0 tx=0x7f5fe80103d0 comp rx=0 tx=0).stop 2024-05-07T02:30:31.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.001+0000 7f600cca2b80 1 -- 172.21.15.19:0/2206106097 shutdown_connections 2024-05-07T02:30:31.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.001+0000 7f600cca2b80 1 --2- 172.21.15.19:0/2206106097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633f7ed4b10 0x5633f7ed6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:31.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.001+0000 7f600cca2b80 1 --2- 172.21.15.19:0/2206106097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633f7ecaa40 0x5633f7ecae50 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:31.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.001+0000 7f600cca2b80 1 --2- 172.21.15.19:0/2206106097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633f7ecb390 0x5633f7ed45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:31.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.001+0000 7f600cca2b80 1 -- 172.21.15.19:0/2206106097 >> 172.21.15.19:0/2206106097 conn(0x5633f7d4cdf0 msgr2=0x5633f7ec5830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:31.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.001+0000 7f600cca2b80 1 -- 172.21.15.19:0/2206106097 shutdown_connections 2024-05-07T02:30:31.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.001+0000 7f600cca2b80 1 -- 172.21.15.19:0/2206106097 wait complete. 2024-05-07T02:30:31.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.002+0000 7f600cca2b80 1 -- start start 2024-05-07T02:30:31.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.002+0000 7f600cca2b80 1 Processor -- start 2024-05-07T02:30:31.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.003+0000 7f600cca2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633f7ecaa40 0x5633f7ed19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:31.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.003+0000 7f600cca2b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633f7ecb390 0x5633f7ed1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:31.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.003+0000 7f600cca2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633f7ed4b10 0x5633f7ed2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:31.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.003+0000 7f600cca2b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5633f7ed7910 con 0x5633f7ed4b10 2024-05-07T02:30:31.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.003+0000 7f600cca2b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5633f7d52ad0 con 0x5633f7ecb390 2024-05-07T02:30:31.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.003+0000 7f600cca2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5633f7ed7a90 con 0x5633f7ecaa40 2024-05-07T02:30:31.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.003+0000 7f600cca2b80 10 client.0 _ll_register_callbacks cb 0x5633f7d93c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:30:31.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.003+0000 7f5ff6a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633f7ecb390 0x5633f7ed1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:31.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.003+0000 7f5ff7400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633f7ecaa40 0x5633f7ed19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:31.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.003+0000 7f5ff6a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633f7ecb390 0x5633f7ed1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:36414/0 (socket says 172.21.15.19:36414) 2024-05-07T02:30:31.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.003+0000 7f5ff6a00700 1 -- 172.21.15.19:0/1377906506 learned_addr learned my addr 172.21.15.19:0/1377906506 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:30:31.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.003+0000 7f5ff7e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633f7ed4b10 0x5633f7ed2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:31.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.004+0000 7f5ff6a00700 1 -- 172.21.15.19:0/1377906506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633f7ecaa40 msgr2=0x5633f7ed19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:30:31.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.004+0000 7f5ff6a00700 1 --2- 172.21.15.19:0/1377906506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633f7ecaa40 0x5633f7ed19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:31.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.004+0000 7f5ff6a00700 1 -- 172.21.15.19:0/1377906506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633f7ed4b10 msgr2=0x5633f7ed2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:30:31.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.004+0000 7f5ff6a00700 1 --2- 172.21.15.19:0/1377906506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633f7ed4b10 0x5633f7ed2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:31.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.004+0000 7f5ff6a00700 1 -- 172.21.15.19:0/1377906506 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633f7db23d0 con 0x5633f7ecb390 2024-05-07T02:30:31.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.004+0000 7f5ff7400700 1 --2- 172.21.15.19:0/1377906506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633f7ecaa40 0x5633f7ed19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:30:31.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.004+0000 7f5ff7e00700 1 --2- 172.21.15.19:0/1377906506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633f7ed4b10 0x5633f7ed2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:30:31.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.004+0000 7f5ff6a00700 1 --2- 172.21.15.19:0/1377906506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633f7ecb390 0x5633f7ed1f30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400fe00 tx=0x7f5fe4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1480969fba7afdc server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:31.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.004+0000 7f5fe1600700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:30:31.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.004+0000 7f5fe1600700 1 -- 172.21.15.19:0/1377906506 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4019040 con 0x5633f7ecb390 2024-05-07T02:30:31.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.005+0000 7f600cca2b80 1 -- 172.21.15.19:0/1377906506 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fe80118f0 con 0x5633f7ecb390 2024-05-07T02:30:31.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.005+0000 7f5fe1600700 1 -- 172.21.15.19:0/1377906506 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f5fe4021b80 con 0x5633f7ecb390 2024-05-07T02:30:31.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.005+0000 7f5fe1600700 1 -- 172.21.15.19:0/1377906506 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5fe4021d80 con 0x5633f7ecb390 2024-05-07T02:30:31.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.005+0000 7f600cca2b80 10 client.25049 fetch_fsmap learned FSMap version 30 2024-05-07T02:30:31.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.005+0000 7f600cca2b80 1 -- 172.21.15.19:0/1377906506 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5633f7ee2500 con 0x5633f7ecb390 2024-05-07T02:30:31.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.005+0000 7f5fe1600700 1 -- 172.21.15.19:0/1377906506 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fe80118f0 con 0x5633f7ecb390 2024-05-07T02:30:31.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.005+0000 7f5fe1600700 1 -- 172.21.15.19:0/1377906506 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5fe40039f0 con 0x5633f7ecb390 2024-05-07T02:30:31.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.005+0000 7f600cca2b80 10 client.25049 fetch_fsmap finished waiting for FSMap version 30 2024-05-07T02:30:31.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.005+0000 7f600cca2b80 10 client.25049 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-07T02:30:31.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.006+0000 7f600cca2b80 20 client.25049 populate_metadata read hostname 'smithi019' 2024-05-07T02:30:31.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.006+0000 7f600cca2b80 1 --2- 172.21.15.19:0/1377906506 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5633f7ee4bf0 0x5633f7ee6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:31.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.006+0000 7f600cca2b80 4 client.25049 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:30:31.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.006+0000 7f600cca2b80 1 -- 172.21.15.19:0/1377906506 --> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633f7e97290 con 0x5633f7ee4bf0 2024-05-07T02:30:31.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.006+0000 7f5ff7400700 1 --2- 172.21.15.19:0/1377906506 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5633f7ee4bf0 0x5633f7ee6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:31.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.007+0000 7f5ff7400700 1 --2- 172.21.15.19:0/1377906506 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5633f7ee4bf0 0x5633f7ee6060 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7bf571966e14a066 server_cookie=e0edf61149b78cc5 in_seq=0 out_seq=0 2024-05-07T02:30:31.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.007+0000 7f5fe1600700 10 client.25049 ms_handle_connect on v2:172.21.15.19:6840/3905645246 2024-05-07T02:30:31.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.008+0000 7f5fe1600700 1 -- 172.21.15.19:0/1377906506 <== mds.1 v2:172.21.15.19:6840/3905645246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5633f7e97290 con 0x5633f7ee4bf0 2024-05-07T02:30:31.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.008+0000 7f5fe1600700 10 client.25049 handle_command_reply: tid=0 2024-05-07T02:30:31.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.008+0000 7f600cca2b80 1 client.25049 shutdown 2024-05-07T02:30:31.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.008+0000 7f600cca2b80 1 -- 172.21.15.19:0/1377906506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633f7ecb390 msgr2=0x5633f7ed1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:31.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.008+0000 7f600cca2b80 1 --2- 172.21.15.19:0/1377906506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633f7ecb390 0x5633f7ed1f30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400fe00 tx=0x7f5fe4004750 comp rx=0 tx=0).stop 2024-05-07T02:30:31.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.008+0000 7f600cca2b80 1 -- 172.21.15.19:0/1377906506 shutdown_connections 2024-05-07T02:30:31.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.008+0000 7f600cca2b80 1 --2- 172.21.15.19:0/1377906506 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5633f7ee4bf0 0x5633f7ee6060 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:31.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.008+0000 7f600cca2b80 1 --2- 172.21.15.19:0/1377906506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633f7ed4b10 0x5633f7ed2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:31.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.008+0000 7f600cca2b80 1 --2- 172.21.15.19:0/1377906506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633f7ecaa40 0x5633f7ed19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:31.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.008+0000 7f5fe1600700 0 client.25049 ms_handle_reset on v2:172.21.15.19:6840/3905645246 2024-05-07T02:30:31.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.008+0000 7f600cca2b80 1 --2- 172.21.15.19:0/1377906506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633f7ecb390 0x5633f7ed1f30 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:31.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.008+0000 7f600cca2b80 1 -- 172.21.15.19:0/1377906506 >> 172.21.15.19:0/1377906506 conn(0x5633f7d4cdf0 msgr2=0x5633f7ec7400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:31.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.008+0000 7f600cca2b80 1 -- 172.21.15.19:0/1377906506 shutdown_connections 2024-05-07T02:30:31.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.008+0000 7f600cca2b80 1 -- 172.21.15.19:0/1377906506 wait complete. 2024-05-07T02:30:31.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.009+0000 7f600cca2b80 20 client.25049 trim_cache size 0 max 16384 2024-05-07T02:30:31.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.032+0000 7f600cca2b80 1 Processor -- start 2024-05-07T02:30:31.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.032+0000 7f600cca2b80 1 -- start start 2024-05-07T02:30:31.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.032+0000 7f600cca2b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633f7ecaa40 0x5633f7ee0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:31.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.032+0000 7f600cca2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633f7ecb390 0x5633f7ec9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:31.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.032+0000 7f600cca2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633f7ed4b10 0x5633f7ed6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:31.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.032+0000 7f600cca2b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5633f7ee43a0 con 0x5633f7ed4b10 2024-05-07T02:30:31.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.032+0000 7f600cca2b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5633f7ee4a70 con 0x5633f7ecaa40 2024-05-07T02:30:31.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.032+0000 7f600cca2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5633f7ee5140 con 0x5633f7ecb390 2024-05-07T02:30:31.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.033+0000 7f5ff7400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633f7ecaa40 0x5633f7ee0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:31.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.033+0000 7f5ff7e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633f7ed4b10 0x5633f7ed6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:31.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.033+0000 7f5ff7400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633f7ecaa40 0x5633f7ee0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:36428/0 (socket says 172.21.15.19:36428) 2024-05-07T02:30:31.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.033+0000 7f5ff7400700 1 -- 172.21.15.19:0/1905098355 learned_addr learned my addr 172.21.15.19:0/1905098355 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:30:31.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.033+0000 7f5ff6a00700 1 --2- 172.21.15.19:0/1905098355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633f7ecb390 0x5633f7ec9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:31.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.033+0000 7f5ff7400700 1 -- 172.21.15.19:0/1905098355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633f7ecb390 msgr2=0x5633f7ec9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:30:31.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.033+0000 7f5ff7400700 1 --2- 172.21.15.19:0/1905098355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633f7ecb390 0x5633f7ec9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:31.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.033+0000 7f5ff7400700 1 -- 172.21.15.19:0/1905098355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633f7ed4b10 msgr2=0x5633f7ed6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:30:31.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.033+0000 7f5ff7400700 1 --2- 172.21.15.19:0/1905098355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633f7ed4b10 0x5633f7ed6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:31.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.033+0000 7f5ff7400700 1 -- 172.21.15.19:0/1905098355 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633f7ee4890 con 0x5633f7ecaa40 2024-05-07T02:30:31.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.034+0000 7f5ff7400700 1 --2- 172.21.15.19:0/1905098355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633f7ecaa40 0x5633f7ee0d90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8007c10 tx=0x7f5fe8007610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b375dc1b2fbbda6 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:31.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.034+0000 7f5fe3e00700 1 -- 172.21.15.19:0/1905098355 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe802f070 con 0x5633f7ecaa40 2024-05-07T02:30:31.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.034+0000 7f5fe3e00700 1 -- 172.21.15.19:0/1905098355 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fe8011c40 con 0x5633f7ecaa40 2024-05-07T02:30:31.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.034+0000 7f5fe3e00700 1 -- 172.21.15.19:0/1905098355 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe80052c0 con 0x5633f7ecaa40 2024-05-07T02:30:31.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.035+0000 7f600cca2b80 1 -- 172.21.15.19:0/1905098355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633f7ecaa40 msgr2=0x5633f7ee0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:31.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.035+0000 7f600cca2b80 1 --2- 172.21.15.19:0/1905098355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633f7ecaa40 0x5633f7ee0d90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8007c10 tx=0x7f5fe8007610 comp rx=0 tx=0).stop 2024-05-07T02:30:31.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.035+0000 7f600cca2b80 1 -- 172.21.15.19:0/1905098355 shutdown_connections 2024-05-07T02:30:31.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.035+0000 7f600cca2b80 1 --2- 172.21.15.19:0/1905098355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633f7ed4b10 0x5633f7ed6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:31.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.035+0000 7f600cca2b80 1 --2- 172.21.15.19:0/1905098355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633f7ecaa40 0x5633f7ee0d90 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:31.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.035+0000 7f600cca2b80 1 --2- 172.21.15.19:0/1905098355 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633f7ecb390 0x5633f7ec9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:31.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.035+0000 7f600cca2b80 1 -- 172.21.15.19:0/1905098355 >> 172.21.15.19:0/1905098355 conn(0x5633f7d4cdf0 msgr2=0x5633f7ec9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:31.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.035+0000 7f600cca2b80 1 -- 172.21.15.19:0/1905098355 shutdown_connections 2024-05-07T02:30:31.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.035+0000 7f600cca2b80 1 -- 172.21.15.19:0/1905098355 wait complete. 2024-05-07T02:30:31.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.036+0000 7f600cca2b80 1 -- start start 2024-05-07T02:30:31.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.036+0000 7f600cca2b80 1 Processor -- start 2024-05-07T02:30:31.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.036+0000 7f600cca2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633f7ecaa40 0x5633f7eef4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:31.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.036+0000 7f600cca2b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633f7ecb390 0x5633f7eefa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:31.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.036+0000 7f600cca2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633f7ed4b10 0x5633f7ee8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:31.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.036+0000 7f600cca2b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5633f7ee5ee0 con 0x5633f7ecaa40 2024-05-07T02:30:31.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.036+0000 7f600cca2b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5633f7ee5810 con 0x5633f7ecb390 2024-05-07T02:30:31.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.036+0000 7f600cca2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5633f7ee65b0 con 0x5633f7ed4b10 2024-05-07T02:30:31.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.036+0000 7f600cca2b80 10 client.0 _ll_register_callbacks cb 0x5633f7d93c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:30:31.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.037+0000 7f5ff7400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633f7ecaa40 0x5633f7eef4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:31.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.037+0000 7f5ff6a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633f7ecb390 0x5633f7eefa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:31.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.037+0000 7f5ff6a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633f7ecb390 0x5633f7eefa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:36436/0 (socket says 172.21.15.19:36436) 2024-05-07T02:30:31.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.037+0000 7f5ff6a00700 1 -- 172.21.15.19:0/2311093615 learned_addr learned my addr 172.21.15.19:0/2311093615 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:30:31.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.037+0000 7f5ff7e00700 1 --2- 172.21.15.19:0/2311093615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633f7ed4b10 0x5633f7ee8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:31.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.037+0000 7f5ff6a00700 1 -- 172.21.15.19:0/2311093615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633f7ed4b10 msgr2=0x5633f7ee8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:30:31.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.037+0000 7f5ff6a00700 1 --2- 172.21.15.19:0/2311093615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633f7ed4b10 0x5633f7ee8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:31.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.037+0000 7f5ff6a00700 1 -- 172.21.15.19:0/2311093615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633f7ecaa40 msgr2=0x5633f7eef4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:30:31.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.037+0000 7f5ff6a00700 1 --2- 172.21.15.19:0/2311093615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633f7ecaa40 0x5633f7eef4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:31.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.037+0000 7f5ff6a00700 1 -- 172.21.15.19:0/2311093615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633f7ee9260 con 0x5633f7ecb390 2024-05-07T02:30:31.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.037+0000 7f5ff7400700 1 --2- 172.21.15.19:0/2311093615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633f7ecaa40 0x5633f7eef4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:30:31.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.038+0000 7f5ff6a00700 1 --2- 172.21.15.19:0/2311093615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633f7ecb390 0x5633f7eefa90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400a010 tx=0x7f5fe4004330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3441249a70905d0f server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:31.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.038+0000 7f5fe1600700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:30:31.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.038+0000 7f5fe1600700 1 -- 172.21.15.19:0/2311093615 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe403d060 con 0x5633f7ecb390 2024-05-07T02:30:31.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.038+0000 7f600cca2b80 1 -- 172.21.15.19:0/2311093615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fe8011c40 con 0x5633f7ecb390 2024-05-07T02:30:31.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.038+0000 7f5fe1600700 1 -- 172.21.15.19:0/2311093615 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f5fe4003a90 con 0x5633f7ecb390 2024-05-07T02:30:31.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.038+0000 7f5fe1600700 1 -- 172.21.15.19:0/2311093615 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5fe4044050 con 0x5633f7ecb390 2024-05-07T02:30:31.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.038+0000 7f600cca2b80 10 client.25055 fetch_fsmap learned FSMap version 30 2024-05-07T02:30:31.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.038+0000 7f600cca2b80 1 -- 172.21.15.19:0/2311093615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5633f7eedac0 con 0x5633f7ecb390 2024-05-07T02:30:31.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.038+0000 7f5fe1600700 1 -- 172.21.15.19:0/2311093615 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fe8011c40 con 0x5633f7ecb390 2024-05-07T02:30:31.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.039+0000 7f5fe1600700 1 -- 172.21.15.19:0/2311093615 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5fe4037c10 con 0x5633f7ecb390 2024-05-07T02:30:31.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.039+0000 7f600cca2b80 10 client.25055 fetch_fsmap finished waiting for FSMap version 30 2024-05-07T02:30:31.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.039+0000 7f600cca2b80 10 client.25055 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-07T02:30:31.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.039+0000 7f600cca2b80 20 client.25055 populate_metadata read hostname 'smithi019' 2024-05-07T02:30:31.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.039+0000 7f600cca2b80 1 --2- 172.21.15.19:0/2311093615 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5633f7f51f10 0x5633f7f53380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:31.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.039+0000 7f600cca2b80 4 client.25055 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-07T02:30:31.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.039+0000 7f600cca2b80 1 -- 172.21.15.19:0/2311093615 --> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] -- command(tid 0: {"prefix": "status"}) -- 0x5633f7dfa3b0 con 0x5633f7f51f10 2024-05-07T02:30:31.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.039+0000 7f5ff7400700 1 --2- 172.21.15.19:0/2311093615 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5633f7f51f10 0x5633f7f53380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:31.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.040+0000 7f5ff7400700 1 --2- 172.21.15.19:0/2311093615 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5633f7f51f10 0x5633f7f53380 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e0a8967eb5c1f4a5 server_cookie=8f177392dcedffd0 in_seq=0 out_seq=0 2024-05-07T02:30:31.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.040+0000 7f5fe1600700 10 client.25055 ms_handle_connect on v2:172.21.15.19:6840/3905645246 2024-05-07T02:30:31.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.040+0000 7f5fe1600700 1 -- 172.21.15.19:0/2311093615 <== mds.1 v2:172.21.15.19:6840/3905645246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5633f7dfa3b0 con 0x5633f7f51f10 2024-05-07T02:30:31.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.040+0000 7f5fe1600700 10 client.25055 handle_command_reply: tid=0 2024-05-07T02:30:31.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.040+0000 7f600cca2b80 1 client.25055 shutdown 2024-05-07T02:30:31.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.040+0000 7f600cca2b80 1 -- 172.21.15.19:0/2311093615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633f7ecb390 msgr2=0x5633f7eefa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:31.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.040+0000 7f600cca2b80 1 --2- 172.21.15.19:0/2311093615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633f7ecb390 0x5633f7eefa90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400a010 tx=0x7f5fe4004330 comp rx=0 tx=0).stop 2024-05-07T02:30:31.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.040+0000 7f600cca2b80 1 -- 172.21.15.19:0/2311093615 shutdown_connections 2024-05-07T02:30:31.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.040+0000 7f600cca2b80 1 --2- 172.21.15.19:0/2311093615 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5633f7f51f10 0x5633f7f53380 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:31.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.040+0000 7f600cca2b80 1 --2- 172.21.15.19:0/2311093615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5633f7ed4b10 0x5633f7ee8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:31.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.041+0000 7f600cca2b80 1 --2- 172.21.15.19:0/2311093615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5633f7ecaa40 0x5633f7eef4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:31.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.041+0000 7f600cca2b80 1 --2- 172.21.15.19:0/2311093615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633f7ecb390 0x5633f7eefa90 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:31.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.041+0000 7f600cca2b80 1 -- 172.21.15.19:0/2311093615 >> 172.21.15.19:0/2311093615 conn(0x5633f7d4cdf0 msgr2=0x5633f7ec4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:31.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.041+0000 7f5fe1600700 0 client.25055 ms_handle_reset on v2:172.21.15.19:6840/3905645246 2024-05-07T02:30:31.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.041+0000 7f600cca2b80 1 -- 172.21.15.19:0/2311093615 shutdown_connections 2024-05-07T02:30:31.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.041+0000 7f600cca2b80 1 -- 172.21.15.19:0/2311093615 wait complete. 2024-05-07T02:30:31.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.041+0000 7f600cca2b80 20 client.25055 trim_cache size 0 max 16384 2024-05-07T02:30:31.044 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:30:31.044 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "9ff84c16-0c17-11ef-bc97-c7b262605968", 2024-05-07T02:30:31.044 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 1, 2024-05-07T02:30:31.044 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24539, 2024-05-07T02:30:31.044 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-05-07T02:30:31.044 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-05-07T02:30:31.044 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-05-07T02:30:31.044 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 171.35461299299999, 2024-05-07T02:30:31.045 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 30, 2024-05-07T02:30:31.045 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 89, 2024-05-07T02:30:31.045 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 87, 2024-05-07T02:30:31.045 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 280.54426469800001 2024-05-07T02:30:31.045 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:30:31.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.042+0000 7f5ffcc00700 1 -- 172.21.15.19:0/2350565402 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f5fdc0b2900 msgr2=0x7f5fdc0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:31.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.042+0000 7f5ffcc00700 1 --2- 172.21.15.19:0/2350565402 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f5fdc0b2900 0x7f5fdc0b4e10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x5633f7ec6c70 tx=0x7f5fe40047c0 comp rx=0 tx=0).stop 2024-05-07T02:30:31.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.042+0000 7f5ffcc00700 1 -- 172.21.15.19:0/2350565402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ff810b8a0 msgr2=0x7f5ff81a1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:31.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.042+0000 7f5ffcc00700 1 --2- 172.21.15.19:0/2350565402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ff810b8a0 0x7f5ff81a1660 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5ff000b6c0 tx=0x7f5ff000cd70 comp rx=0 tx=0).stop 2024-05-07T02:30:31.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.042+0000 7f5ffcc00700 1 -- 172.21.15.19:0/2350565402 shutdown_connections 2024-05-07T02:30:31.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.042+0000 7f5ffcc00700 1 --2- 172.21.15.19:0/2350565402 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f5fdc0b2900 0x7f5fdc0b4e10 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:31.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.042+0000 7f5ffcc00700 1 --2- 172.21.15.19:0/2350565402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ff810b8a0 0x7f5ff81a1660 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:31.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.042+0000 7f5ffcc00700 1 --2- 172.21.15.19:0/2350565402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ff8106620 0x7f5ff819ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:31.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.042+0000 7f5ffcc00700 1 --2- 172.21.15.19:0/2350565402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ff810af50 0x7f5ff819d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:31.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.042+0000 7f5ffcc00700 1 -- 172.21.15.19:0/2350565402 >> 172.21.15.19:0/2350565402 conn(0x7f5ff8102030 msgr2=0x7f5ff810a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:31.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.042+0000 7f5ffcc00700 1 -- 172.21.15.19:0/2350565402 shutdown_connections 2024-05-07T02:30:31.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:31.042+0000 7f5ffcc00700 1 -- 172.21.15.19:0/2350565402 wait complete. 2024-05-07T02:30:31.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:30:30 smithi064 ceph-mon[34699]: pgmap v468: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:30:31.939 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.b status 2024-05-07T02:30:32.406 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:30:32.806 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:30:32 smithi019 ceph-mon[29781]: pgmap v469: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:30:32.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:30:32 smithi027 ceph-mon[34398]: pgmap v469: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:30:33.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:30:32 smithi064 ceph-mon[34699]: pgmap v469: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:30:34.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:30:34 smithi019 ceph-mon[29781]: pgmap v470: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:30:34.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:30:34 smithi027 ceph-mon[34398]: pgmap v470: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:30:35.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.002+0000 7f2d1f200700 1 -- 172.21.15.19:0/2410274201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d18108d20 msgr2=0x7f2d18109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:35.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.002+0000 7f2d1f200700 1 --2- 172.21.15.19:0/2410274201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d18108d20 0x7f2d18109130 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2d080099c0 tx=0x7f2d08009cd0 comp rx=0 tx=0).stop 2024-05-07T02:30:35.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.003+0000 7f2d1f200700 1 -- 172.21.15.19:0/2410274201 shutdown_connections 2024-05-07T02:30:35.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.003+0000 7f2d1f200700 1 --2- 172.21.15.19:0/2410274201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d18109670 0x7f2d18110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:35.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.003+0000 7f2d1f200700 1 --2- 172.21.15.19:0/2410274201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d181043f0 0x7f2d18104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:35.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.003+0000 7f2d1f200700 1 --2- 172.21.15.19:0/2410274201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d18108d20 0x7f2d18109130 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:35.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.003+0000 7f2d1f200700 1 -- 172.21.15.19:0/2410274201 >> 172.21.15.19:0/2410274201 conn(0x7f2d18079c10 msgr2=0x7f2d1807a050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:35.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.003+0000 7f2d1f200700 1 -- 172.21.15.19:0/2410274201 shutdown_connections 2024-05-07T02:30:35.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.003+0000 7f2d1f200700 1 -- 172.21.15.19:0/2410274201 wait complete. 2024-05-07T02:30:35.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.004+0000 7f2d1f200700 1 Processor -- start 2024-05-07T02:30:35.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.004+0000 7f2d1f200700 1 -- start start 2024-05-07T02:30:35.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.005+0000 7f2d1f200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d181043f0 0x7f2d18075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:35.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.005+0000 7f2d1f200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d18108d20 0x7f2d18076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:35.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.005+0000 7f2d1f200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d18109670 0x7f2d18076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:35.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.005+0000 7f2d1f200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2d18114280 con 0x7f2d181043f0 2024-05-07T02:30:35.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.005+0000 7f2d1f200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2d18114100 con 0x7f2d18108d20 2024-05-07T02:30:35.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.005+0000 7f2d1f200700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2d18114400 con 0x7f2d18109670 2024-05-07T02:30:35.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.005+0000 7f2d1de00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d181043f0 0x7f2d18075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:35.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.005+0000 7f2d1de00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d181043f0 0x7f2d18075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46558/0 (socket says 172.21.15.19:46558) 2024-05-07T02:30:35.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.005+0000 7f2d1de00700 1 -- 172.21.15.19:0/1913196291 learned_addr learned my addr 172.21.15.19:0/1913196291 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:30:35.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.005+0000 7f2d1e800700 1 --2- 172.21.15.19:0/1913196291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d18109670 0x7f2d18076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:35.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.005+0000 7f2d1d400700 1 --2- 172.21.15.19:0/1913196291 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d18108d20 0x7f2d18076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:35.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.006+0000 7f2d1de00700 1 -- 172.21.15.19:0/1913196291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d18109670 msgr2=0x7f2d18076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:35.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.006+0000 7f2d1de00700 1 --2- 172.21.15.19:0/1913196291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d18109670 0x7f2d18076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:35.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.006+0000 7f2d1de00700 1 -- 172.21.15.19:0/1913196291 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d18108d20 msgr2=0x7f2d18076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:35.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.006+0000 7f2d1de00700 1 --2- 172.21.15.19:0/1913196291 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d18108d20 0x7f2d18076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:35.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.006+0000 7f2d1de00700 1 -- 172.21.15.19:0/1913196291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d181a8fd0 con 0x7f2d181043f0 2024-05-07T02:30:35.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.006+0000 7f2d1d400700 1 --2- 172.21.15.19:0/1913196291 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d18108d20 0x7f2d18076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:30:35.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.006+0000 7f2d1de00700 1 --2- 172.21.15.19:0/1913196291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d181043f0 0x7f2d18075ac0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c00e9a0 tx=0x7f2d0c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:35.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.006+0000 7f2d12a00700 1 -- 172.21.15.19:0/1913196291 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c00ca10 con 0x7f2d181043f0 2024-05-07T02:30:35.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.006+0000 7f2d12a00700 1 -- 172.21.15.19:0/1913196291 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d0c00cbb0 con 0x7f2d181043f0 2024-05-07T02:30:35.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.006+0000 7f2d12a00700 1 -- 172.21.15.19:0/1913196291 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c018680 con 0x7f2d181043f0 2024-05-07T02:30:35.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.006+0000 7f2d1f200700 1 -- 172.21.15.19:0/1913196291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d181a92c0 con 0x7f2d181043f0 2024-05-07T02:30:35.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.007+0000 7f2d1f200700 1 -- 172.21.15.19:0/1913196291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d181a9820 con 0x7f2d181043f0 2024-05-07T02:30:35.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.010+0000 7f2d12a00700 1 -- 172.21.15.19:0/1913196291 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2d0c010430 con 0x7f2d181043f0 2024-05-07T02:30:35.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.011+0000 7f2d2f356b80 1 Processor -- start 2024-05-07T02:30:35.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.012+0000 7f2d2f356b80 1 -- start start 2024-05-07T02:30:35.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.012+0000 7f2d2f356b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f02c78a40 0x558f02c78e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:35.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.012+0000 7f2d2f356b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f02c79390 0x558f02c825d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:35.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.012+0000 7f2d2f356b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f02c82b10 0x558f02c84f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:35.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.012+0000 7f2d2f356b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558f02b9a9d0 con 0x558f02c78a40 2024-05-07T02:30:35.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.012+0000 7f2d2f356b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x558f02b38710 con 0x558f02c82b10 2024-05-07T02:30:35.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.012+0000 7f2d2f356b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558f02b96990 con 0x558f02c79390 2024-05-07T02:30:35.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.012+0000 7f2d12a00700 1 --2- 172.21.15.19:0/1913196291 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f2d000b2900 0x7f2d000b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:35.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.013+0000 7f2d1de00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f02c79390 0x558f02c825d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:35.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.013+0000 7f2d1d400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f02c82b10 0x558f02c84f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:35.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.013+0000 7f2d12a00700 1 -- 172.21.15.19:0/1913196291 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f2d0c0e2710 con 0x7f2d181043f0 2024-05-07T02:30:35.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.013+0000 7f2d1e800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f02c78a40 0x558f02c78e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:35.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.013+0000 7f2d1d400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f02c82b10 0x558f02c84f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:36472/0 (socket says 172.21.15.19:36472) 2024-05-07T02:30:35.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.013+0000 7f2d1d400700 1 -- 172.21.15.19:0/3467712440 learned_addr learned my addr 172.21.15.19:0/3467712440 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:30:35.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.013+0000 7f2d1e800700 1 --2- 172.21.15.19:0/1913196291 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f2d000b2900 0x7f2d000b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:35.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.013+0000 7f2d1d400700 1 -- 172.21.15.19:0/3467712440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f02c79390 msgr2=0x558f02c825d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:30:35.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.013+0000 7f2d1d400700 1 --2- 172.21.15.19:0/3467712440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f02c79390 0x558f02c825d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:35.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.013+0000 7f2d1d400700 1 -- 172.21.15.19:0/3467712440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f02c78a40 msgr2=0x558f02c78e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:30:35.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.013+0000 7f2d1d400700 1 --2- 172.21.15.19:0/3467712440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f02c78a40 0x558f02c78e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:35.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.013+0000 7f2d1d400700 1 -- 172.21.15.19:0/3467712440 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f02ae7880 con 0x558f02c82b10 2024-05-07T02:30:35.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.013+0000 7f2d1d400700 1 --2- 172.21.15.19:0/3467712440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f02c82b10 0x558f02c84f30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2d080099c0 tx=0x7f2d08005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6cb797535fba948 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:35.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.014+0000 7f2d07400700 1 -- 172.21.15.19:0/3467712440 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d08027070 con 0x558f02c82b10 2024-05-07T02:30:35.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.014+0000 7f2d07400700 1 -- 172.21.15.19:0/3467712440 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d08005550 con 0x558f02c82b10 2024-05-07T02:30:35.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.014+0000 7f2d1e800700 1 --2- 172.21.15.19:0/1913196291 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f2d000b2900 0x7f2d000b4e10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2d14009ec0 tx=0x7f2d1400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:35.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.014+0000 7f2d07400700 1 -- 172.21.15.19:0/3467712440 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d08003d30 con 0x558f02c82b10 2024-05-07T02:30:35.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.015+0000 7f2d2f356b80 1 -- 172.21.15.19:0/3467712440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f02c82b10 msgr2=0x558f02c84f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:35.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.015+0000 7f2d2f356b80 1 --2- 172.21.15.19:0/3467712440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f02c82b10 0x558f02c84f30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2d080099c0 tx=0x7f2d08005fb0 comp rx=0 tx=0).stop 2024-05-07T02:30:35.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.015+0000 7f2d2f356b80 1 -- 172.21.15.19:0/3467712440 shutdown_connections 2024-05-07T02:30:35.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.015+0000 7f2d2f356b80 1 --2- 172.21.15.19:0/3467712440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f02c82b10 0x558f02c84f30 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:35.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.015+0000 7f2d2f356b80 1 --2- 172.21.15.19:0/3467712440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f02c78a40 0x558f02c78e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:35.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.015+0000 7f2d2f356b80 1 --2- 172.21.15.19:0/3467712440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f02c79390 0x558f02c825d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:35.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.015+0000 7f2d2f356b80 1 -- 172.21.15.19:0/3467712440 >> 172.21.15.19:0/3467712440 conn(0x558f02afadf0 msgr2=0x558f02c73830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:35.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.015+0000 7f2d2f356b80 1 -- 172.21.15.19:0/3467712440 shutdown_connections 2024-05-07T02:30:35.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.015+0000 7f2d2f356b80 1 -- 172.21.15.19:0/3467712440 wait complete. 2024-05-07T02:30:35.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.016+0000 7f2d2f356b80 1 -- start start 2024-05-07T02:30:35.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.016+0000 7f2d2f356b80 1 Processor -- start 2024-05-07T02:30:35.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.017+0000 7f2d2f356b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f02c78a40 0x558f02c7fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:35.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.017+0000 7f2d2f356b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f02c79390 0x558f02c7ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:35.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.017+0000 7f2d2f356b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f02c82b10 0x558f02c804e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:35.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.017+0000 7f2d2f356b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558f02c858b0 con 0x558f02c78a40 2024-05-07T02:30:35.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.017+0000 7f2d2f356b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x558f02b00ad0 con 0x558f02c82b10 2024-05-07T02:30:35.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.017+0000 7f2d2f356b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558f02c85a30 con 0x558f02c79390 2024-05-07T02:30:35.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.017+0000 7f2d2f356b80 10 client.0 _ll_register_callbacks cb 0x558f02b41c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:30:35.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.017+0000 7f2d1de00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f02c78a40 0x558f02c7fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:35.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.017+0000 7f2d1e800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f02c82b10 0x558f02c804e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:35.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.017+0000 7f2d1d400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f02c79390 0x558f02c7ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:35.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.017+0000 7f2d1e800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f02c82b10 0x558f02c804e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:36482/0 (socket says 172.21.15.19:36482) 2024-05-07T02:30:35.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.017+0000 7f2d1d400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f02c79390 0x558f02c7ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:40290/0 (socket says 172.21.15.19:40290) 2024-05-07T02:30:35.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.017+0000 7f2d1e800700 1 -- 172.21.15.19:0/1038315947 learned_addr learned my addr 172.21.15.19:0/1038315947 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:30:35.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.018+0000 7f2d1d400700 1 -- 172.21.15.19:0/1038315947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f02c82b10 msgr2=0x558f02c804e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:30:35.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.018+0000 7f2d1d400700 1 --2- 172.21.15.19:0/1038315947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f02c82b10 0x558f02c804e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:35.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.018+0000 7f2d1d400700 1 -- 172.21.15.19:0/1038315947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f02c78a40 msgr2=0x558f02c7fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:30:35.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.018+0000 7f2d1d400700 1 --2- 172.21.15.19:0/1038315947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f02c78a40 0x558f02c7fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:35.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.018+0000 7f2d1d400700 1 -- 172.21.15.19:0/1038315947 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f02b603d0 con 0x558f02c79390 2024-05-07T02:30:35.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.018+0000 7f2d1de00700 1 --2- 172.21.15.19:0/1038315947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f02c78a40 0x558f02c7fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:30:35.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.018+0000 7f2d1d400700 1 --2- 172.21.15.19:0/1038315947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f02c79390 0x558f02c7ffa0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2d08025040 tx=0x7f2d08009cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2503bf2b9deaf9f server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:35.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.018+0000 7f2cfbe00700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:30:35.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.018+0000 7f2cfbe00700 1 -- 172.21.15.19:0/1038315947 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0802f020 con 0x558f02c79390 2024-05-07T02:30:35.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.018+0000 7f2d2f356b80 1 -- 172.21.15.19:0/1038315947 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d08005550 con 0x558f02c79390 2024-05-07T02:30:35.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.018+0000 7f2cfbe00700 1 -- 172.21.15.19:0/1038315947 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f2d080052c0 con 0x558f02c79390 2024-05-07T02:30:35.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.019+0000 7f2cfbe00700 1 -- 172.21.15.19:0/1038315947 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2d0803a050 con 0x558f02c79390 2024-05-07T02:30:35.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.019+0000 7f2d2f356b80 10 client.25063 fetch_fsmap learned FSMap version 30 2024-05-07T02:30:35.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.019+0000 7f2d2f356b80 1 -- 172.21.15.19:0/1038315947 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558f02c858b0 con 0x558f02c79390 2024-05-07T02:30:35.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.019+0000 7f2cfbe00700 1 -- 172.21.15.19:0/1038315947 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d08005550 con 0x558f02c79390 2024-05-07T02:30:35.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.019+0000 7f2cfbe00700 1 -- 172.21.15.19:0/1038315947 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2d0802a030 con 0x558f02c79390 2024-05-07T02:30:35.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.019+0000 7f2d2f356b80 10 client.25063 fetch_fsmap finished waiting for FSMap version 30 2024-05-07T02:30:35.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.019+0000 7f2d2f356b80 10 client.25063 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-07T02:30:35.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.019+0000 7f2d2f356b80 20 client.25063 populate_metadata read hostname 'smithi019' 2024-05-07T02:30:35.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.019+0000 7f2d2f356b80 1 --2- 172.21.15.19:0/1038315947 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x558f02c924e0 0x558f02c93950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:35.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.019+0000 7f2d2f356b80 4 client.25063 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:30:35.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.019+0000 7f2d2f356b80 1 -- 172.21.15.19:0/1038315947 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f02c45290 con 0x558f02c924e0 2024-05-07T02:30:35.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.020+0000 7f2d1de00700 1 --2- 172.21.15.19:0/1038315947 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x558f02c924e0 0x558f02c93950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:35.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.020+0000 7f2d1de00700 1 --2- 172.21.15.19:0/1038315947 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x558f02c924e0 0x558f02c93950 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a6bc380a8329794 server_cookie=fe083229a00dd4da in_seq=0 out_seq=0 2024-05-07T02:30:35.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.020+0000 7f2cfbe00700 10 client.25063 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:30:35.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.021+0000 7f2cfbe00700 1 -- 172.21.15.19:0/1038315947 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558f02c45290 con 0x558f02c924e0 2024-05-07T02:30:35.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.021+0000 7f2cfbe00700 10 client.25063 handle_command_reply: tid=0 2024-05-07T02:30:35.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.021+0000 7f2d2f356b80 1 client.25063 shutdown 2024-05-07T02:30:35.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.021+0000 7f2d2f356b80 1 -- 172.21.15.19:0/1038315947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f02c79390 msgr2=0x558f02c7ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:35.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.021+0000 7f2d2f356b80 1 --2- 172.21.15.19:0/1038315947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f02c79390 0x558f02c7ffa0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2d08025040 tx=0x7f2d08009cd0 comp rx=0 tx=0).stop 2024-05-07T02:30:35.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.022+0000 7f2d2f356b80 1 -- 172.21.15.19:0/1038315947 shutdown_connections 2024-05-07T02:30:35.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.022+0000 7f2d2f356b80 1 --2- 172.21.15.19:0/1038315947 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x558f02c924e0 0x558f02c93950 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:35.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.022+0000 7f2d2f356b80 1 --2- 172.21.15.19:0/1038315947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f02c82b10 0x558f02c804e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:35.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.022+0000 7f2d2f356b80 1 --2- 172.21.15.19:0/1038315947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f02c78a40 0x558f02c7fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:35.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.022+0000 7f2cfbe00700 0 client.25063 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:30:35.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.022+0000 7f2d2f356b80 1 --2- 172.21.15.19:0/1038315947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f02c79390 0x558f02c7ffa0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:35.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.022+0000 7f2d2f356b80 1 -- 172.21.15.19:0/1038315947 >> 172.21.15.19:0/1038315947 conn(0x558f02afadf0 msgr2=0x558f02c75400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:35.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.022+0000 7f2d2f356b80 1 -- 172.21.15.19:0/1038315947 shutdown_connections 2024-05-07T02:30:35.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.022+0000 7f2d2f356b80 1 -- 172.21.15.19:0/1038315947 wait complete. 2024-05-07T02:30:35.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.022+0000 7f2d2f356b80 20 client.25063 trim_cache size 0 max 16384 2024-05-07T02:30:35.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.045+0000 7f2d2f356b80 1 Processor -- start 2024-05-07T02:30:35.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.046+0000 7f2d2f356b80 1 -- start start 2024-05-07T02:30:35.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.046+0000 7f2d2f356b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f02c78a40 0x558f02c8ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:35.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.046+0000 7f2d2f356b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f02c79390 0x558f02c77b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:35.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.046+0000 7f2d2f356b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f02c82b10 0x558f02c9cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:35.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.046+0000 7f2d2f356b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558f02c91c90 con 0x558f02c78a40 2024-05-07T02:30:35.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.046+0000 7f2d2f356b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x558f02c92360 con 0x558f02c82b10 2024-05-07T02:30:35.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.046+0000 7f2d2f356b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558f02c92a30 con 0x558f02c79390 2024-05-07T02:30:35.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.046+0000 7f2d1e800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f02c82b10 0x558f02c9cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:35.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.046+0000 7f2d1e800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f02c82b10 0x558f02c9cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:36488/0 (socket says 172.21.15.19:36488) 2024-05-07T02:30:35.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.046+0000 7f2d1e800700 1 -- 172.21.15.19:0/3020135272 learned_addr learned my addr 172.21.15.19:0/3020135272 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:30:35.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.046+0000 7f2d1de00700 1 --2- 172.21.15.19:0/3020135272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f02c78a40 0x558f02c8ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:35.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.046+0000 7f2d1d400700 1 --2- 172.21.15.19:0/3020135272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f02c79390 0x558f02c77b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:35.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.047+0000 7f2d1e800700 1 -- 172.21.15.19:0/3020135272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f02c79390 msgr2=0x558f02c77b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:30:35.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.047+0000 7f2d1e800700 1 --2- 172.21.15.19:0/3020135272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f02c79390 0x558f02c77b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:35.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.047+0000 7f2d1e800700 1 -- 172.21.15.19:0/3020135272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f02c78a40 msgr2=0x558f02c8ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:30:35.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.047+0000 7f2d1e800700 1 --2- 172.21.15.19:0/3020135272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f02c78a40 0x558f02c8ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:35.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.047+0000 7f2d1e800700 1 -- 172.21.15.19:0/3020135272 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f02c797a0 con 0x558f02c82b10 2024-05-07T02:30:35.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.047+0000 7f2d1e800700 1 --2- 172.21.15.19:0/3020135272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f02c82b10 0x558f02c9cdb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2d140188d0 tx=0x7f2d14009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc342e0b3aa85221 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:35.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.047+0000 7f2d06a00700 1 -- 172.21.15.19:0/3020135272 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14023590 con 0x558f02c82b10 2024-05-07T02:30:35.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.048+0000 7f2d06a00700 1 -- 172.21.15.19:0/3020135272 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d14029070 con 0x558f02c82b10 2024-05-07T02:30:35.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.048+0000 7f2d06a00700 1 -- 172.21.15.19:0/3020135272 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14024930 con 0x558f02c82b10 2024-05-07T02:30:35.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.048+0000 7f2d2f356b80 1 -- 172.21.15.19:0/3020135272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f02c82b10 msgr2=0x558f02c9cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:35.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.048+0000 7f2d2f356b80 1 --2- 172.21.15.19:0/3020135272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f02c82b10 0x558f02c9cdb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2d140188d0 tx=0x7f2d14009920 comp rx=0 tx=0).stop 2024-05-07T02:30:35.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.048+0000 7f2d2f356b80 1 -- 172.21.15.19:0/3020135272 shutdown_connections 2024-05-07T02:30:35.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.048+0000 7f2d2f356b80 1 --2- 172.21.15.19:0/3020135272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f02c82b10 0x558f02c9cdb0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:35.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.048+0000 7f2d2f356b80 1 --2- 172.21.15.19:0/3020135272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f02c78a40 0x558f02c8ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:35.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.049+0000 7f2d2f356b80 1 --2- 172.21.15.19:0/3020135272 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f02c79390 0x558f02c77b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:35.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.049+0000 7f2d2f356b80 1 -- 172.21.15.19:0/3020135272 >> 172.21.15.19:0/3020135272 conn(0x558f02afadf0 msgr2=0x558f02c77410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:35.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.049+0000 7f2d2f356b80 1 -- 172.21.15.19:0/3020135272 shutdown_connections 2024-05-07T02:30:35.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.049+0000 7f2d2f356b80 1 -- 172.21.15.19:0/3020135272 wait complete. 2024-05-07T02:30:35.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.049+0000 7f2d2f356b80 1 -- start start 2024-05-07T02:30:35.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.050+0000 7f2d2f356b80 1 Processor -- start 2024-05-07T02:30:35.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.050+0000 7f2d2f356b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f02c78a40 0x558f02c97350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:35.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.050+0000 7f2d2f356b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f02c79390 0x558f02c97890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:35.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.050+0000 7f2d2f356b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f02c82b10 0x558f02c9b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:35.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.050+0000 7f2d2f356b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558f02c937d0 con 0x558f02c79390 2024-05-07T02:30:35.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.050+0000 7f2d2f356b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x558f02c93100 con 0x558f02c78a40 2024-05-07T02:30:35.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.050+0000 7f2d2f356b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558f02c93ea0 con 0x558f02c82b10 2024-05-07T02:30:35.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.050+0000 7f2d2f356b80 10 client.0 _ll_register_callbacks cb 0x558f02b41c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:30:35.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.050+0000 7f2d1de00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f02c78a40 0x558f02c97350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:35.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.050+0000 7f2d1d400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f02c79390 0x558f02c97890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:35.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.050+0000 7f2d1de00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f02c78a40 0x558f02c97350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:36504/0 (socket says 172.21.15.19:36504) 2024-05-07T02:30:35.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.050+0000 7f2d1de00700 1 -- 172.21.15.19:0/536276111 learned_addr learned my addr 172.21.15.19:0/536276111 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:30:35.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.051+0000 7f2d1e800700 1 --2- 172.21.15.19:0/536276111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f02c82b10 0x558f02c9b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:35.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.051+0000 7f2d1de00700 1 -- 172.21.15.19:0/536276111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f02c82b10 msgr2=0x558f02c9b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:30:35.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.051+0000 7f2d1de00700 1 --2- 172.21.15.19:0/536276111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f02c82b10 0x558f02c9b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:35.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.051+0000 7f2d1de00700 1 -- 172.21.15.19:0/536276111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f02c79390 msgr2=0x558f02c97890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:30:35.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.051+0000 7f2d1de00700 1 --2- 172.21.15.19:0/536276111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f02c79390 0x558f02c97890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:35.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.051+0000 7f2d1de00700 1 -- 172.21.15.19:0/536276111 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f02c98130 con 0x558f02c78a40 2024-05-07T02:30:35.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.051+0000 7f2d1e800700 1 --2- 172.21.15.19:0/536276111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f02c82b10 0x558f02c9b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:30:35.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.051+0000 7f2d1de00700 1 --2- 172.21.15.19:0/536276111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f02c78a40 0x558f02c97350 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c00c2d0 tx=0x7f2d0c01f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61866c0ffa2918e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:35.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.051+0000 7f2cfbe00700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:30:35.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.051+0000 7f2cfbe00700 1 -- 172.21.15.19:0/536276111 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c0a9920 con 0x558f02c78a40 2024-05-07T02:30:35.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.052+0000 7f2d2f356b80 1 -- 172.21.15.19:0/536276111 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d14029070 con 0x558f02c78a40 2024-05-07T02:30:35.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.052+0000 7f2cfbe00700 1 -- 172.21.15.19:0/536276111 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f2d0c0dff10 con 0x558f02c78a40 2024-05-07T02:30:35.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.052+0000 7f2cfbe00700 1 -- 172.21.15.19:0/536276111 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2d0c0adcd0 con 0x558f02c78a40 2024-05-07T02:30:35.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.052+0000 7f2d2f356b80 10 client.25073 fetch_fsmap learned FSMap version 30 2024-05-07T02:30:35.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.052+0000 7f2d2f356b80 1 -- 172.21.15.19:0/536276111 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558f02cfdf20 con 0x558f02c78a40 2024-05-07T02:30:35.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.052+0000 7f2cfbe00700 1 -- 172.21.15.19:0/536276111 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d14029070 con 0x558f02c78a40 2024-05-07T02:30:35.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.052+0000 7f2cfbe00700 1 -- 172.21.15.19:0/536276111 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f2d0c0c1150 con 0x558f02c78a40 2024-05-07T02:30:35.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.052+0000 7f2d2f356b80 10 client.25073 fetch_fsmap finished waiting for FSMap version 30 2024-05-07T02:30:35.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.052+0000 7f2d2f356b80 10 client.25073 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-07T02:30:35.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.052+0000 7f2d2f356b80 20 client.25073 populate_metadata read hostname 'smithi019' 2024-05-07T02:30:35.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.052+0000 7f2d2f356b80 1 --2- 172.21.15.19:0/536276111 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x558f02d00280 0x558f02d016f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:35.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.052+0000 7f2d2f356b80 4 client.25073 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-07T02:30:35.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.052+0000 7f2d2f356b80 1 -- 172.21.15.19:0/536276111 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "status"}) -- 0x558f02ba83b0 con 0x558f02d00280 2024-05-07T02:30:35.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.053+0000 7f2d1d400700 1 --2- 172.21.15.19:0/536276111 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x558f02d00280 0x558f02d016f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:35.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.053+0000 7f2d1d400700 1 --2- 172.21.15.19:0/536276111 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x558f02d00280 0x558f02d016f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a13c9b079edc593f server_cookie=4691a0a76b4f75e7 in_seq=0 out_seq=0 2024-05-07T02:30:35.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.053+0000 7f2cfbe00700 10 client.25073 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:30:35.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.053+0000 7f2cfbe00700 1 -- 172.21.15.19:0/536276111 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x558f02ba83b0 con 0x558f02d00280 2024-05-07T02:30:35.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.053+0000 7f2cfbe00700 10 client.25073 handle_command_reply: tid=0 2024-05-07T02:30:35.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.053+0000 7f2d2f356b80 1 client.25073 shutdown 2024-05-07T02:30:35.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.054+0000 7f2d2f356b80 1 -- 172.21.15.19:0/536276111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f02c78a40 msgr2=0x558f02c97350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:35.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.054+0000 7f2d2f356b80 1 --2- 172.21.15.19:0/536276111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f02c78a40 0x558f02c97350 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c00c2d0 tx=0x7f2d0c01f000 comp rx=0 tx=0).stop 2024-05-07T02:30:35.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.054+0000 7f2d2f356b80 1 -- 172.21.15.19:0/536276111 shutdown_connections 2024-05-07T02:30:35.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.054+0000 7f2d2f356b80 1 --2- 172.21.15.19:0/536276111 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x558f02d00280 0x558f02d016f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:35.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.054+0000 7f2d2f356b80 1 --2- 172.21.15.19:0/536276111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558f02c82b10 0x558f02c9b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:35.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.054+0000 7f2d2f356b80 1 --2- 172.21.15.19:0/536276111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558f02c78a40 0x558f02c97350 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:35.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.054+0000 7f2cfbe00700 0 client.25073 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:30:35.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.054+0000 7f2d2f356b80 1 --2- 172.21.15.19:0/536276111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558f02c79390 0x558f02c97890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:35.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.054+0000 7f2d2f356b80 1 -- 172.21.15.19:0/536276111 >> 172.21.15.19:0/536276111 conn(0x558f02afadf0 msgr2=0x558f02c7fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:35.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.054+0000 7f2d2f356b80 1 -- 172.21.15.19:0/536276111 shutdown_connections 2024-05-07T02:30:35.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.054+0000 7f2d2f356b80 1 -- 172.21.15.19:0/536276111 wait complete. 2024-05-07T02:30:35.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.054+0000 7f2d2f356b80 20 client.25073 trim_cache size 0 max 16384 2024-05-07T02:30:35.060 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:30:35.061 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "9ff84c16-0c17-11ef-bc97-c7b262605968", 2024-05-07T02:30:35.061 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-05-07T02:30:35.061 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24551, 2024-05-07T02:30:35.061 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-05-07T02:30:35.061 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-05-07T02:30:35.061 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-05-07T02:30:35.061 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 260.03850155999999, 2024-05-07T02:30:35.061 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 30, 2024-05-07T02:30:35.061 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 89, 2024-05-07T02:30:35.061 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 87, 2024-05-07T02:30:35.061 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 281.43287590800003 2024-05-07T02:30:35.061 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:30:35.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.055+0000 7f2d1f200700 1 -- 172.21.15.19:0/1913196291 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f2d000b2900 msgr2=0x7f2d000b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:35.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.055+0000 7f2d1f200700 1 --2- 172.21.15.19:0/1913196291 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f2d000b2900 0x7f2d000b4e10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2d14009ec0 tx=0x7f2d1400b040 comp rx=0 tx=0).stop 2024-05-07T02:30:35.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.055+0000 7f2d1f200700 1 -- 172.21.15.19:0/1913196291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d181043f0 msgr2=0x7f2d18075ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:35.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.055+0000 7f2d1f200700 1 --2- 172.21.15.19:0/1913196291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d181043f0 0x7f2d18075ac0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c00e9a0 tx=0x7f2d0c00ecb0 comp rx=0 tx=0).stop 2024-05-07T02:30:35.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.055+0000 7f2d1f200700 1 -- 172.21.15.19:0/1913196291 shutdown_connections 2024-05-07T02:30:35.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.055+0000 7f2d1f200700 1 --2- 172.21.15.19:0/1913196291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d18109670 0x7f2d18076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:35.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.055+0000 7f2d1f200700 1 --2- 172.21.15.19:0/1913196291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d181043f0 0x7f2d18075ac0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:35.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.055+0000 7f2d1f200700 1 --2- 172.21.15.19:0/1913196291 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f2d000b2900 0x7f2d000b4e10 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:35.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.055+0000 7f2d1f200700 1 --2- 172.21.15.19:0/1913196291 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d18108d20 0x7f2d18076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:35.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.055+0000 7f2d1f200700 1 -- 172.21.15.19:0/1913196291 >> 172.21.15.19:0/1913196291 conn(0x7f2d18079c10 msgr2=0x7f2d18101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:35.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.055+0000 7f2d1f200700 1 -- 172.21.15.19:0/1913196291 shutdown_connections 2024-05-07T02:30:35.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:35.055+0000 7f2d1f200700 1 -- 172.21.15.19:0/1913196291 wait complete. 2024-05-07T02:30:35.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:30:34 smithi064 ceph-mon[34699]: pgmap v470: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:30:36.055 INFO:tasks.cephadm:Setup complete, yielding 2024-05-07T02:30:36.055 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-07T02:30:36.066 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-05-07T02:30:36.066 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- bash -c 'ceph orch status' 2024-05-07T02:30:36.553 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:30:36.924 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:30:36 smithi019 ceph-mon[29781]: pgmap v471: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:30:36.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:30:36 smithi027 ceph-mon[34398]: pgmap v471: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:30:37.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:30:36 smithi064 ceph-mon[34699]: pgmap v471: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:30:38.930 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:30:38 smithi064 ceph-mon[34699]: pgmap v472: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:30:38.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:30:38 smithi027 ceph-mon[34398]: pgmap v472: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:30:39.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.007+0000 7f2263e00700 1 -- 172.21.15.19:0/49894704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2264106390 msgr2=0x7f22641067a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:39.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.007+0000 7f2263e00700 1 --2- 172.21.15.19:0/49894704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2264106390 0x7f22641067a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2254009990 tx=0x7f2254009ca0 comp rx=0 tx=0).stop 2024-05-07T02:30:39.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.008+0000 7f2263e00700 1 -- 172.21.15.19:0/49894704 shutdown_connections 2024-05-07T02:30:39.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.008+0000 7f2263e00700 1 --2- 172.21.15.19:0/49894704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2264107e60 0x7f226410e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:39.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.008+0000 7f2263e00700 1 --2- 172.21.15.19:0/49894704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2264106390 0x7f22641067a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:39.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.008+0000 7f2263e00700 1 --2- 172.21.15.19:0/49894704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22641074c0 0x7f22641078d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:39.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.008+0000 7f2263e00700 1 -- 172.21.15.19:0/49894704 >> 172.21.15.19:0/49894704 conn(0x7f2264101a10 msgr2=0x7f2264103e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:39.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.008+0000 7f2263e00700 1 -- 172.21.15.19:0/49894704 shutdown_connections 2024-05-07T02:30:39.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.008+0000 7f2263e00700 1 -- 172.21.15.19:0/49894704 wait complete. 2024-05-07T02:30:39.010 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:30:38 smithi019 ceph-mon[29781]: pgmap v472: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:30:39.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.009+0000 7f2263e00700 1 Processor -- start 2024-05-07T02:30:39.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.009+0000 7f2263e00700 1 -- start start 2024-05-07T02:30:39.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.010+0000 7f2263e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2264106390 0x7f226419c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:39.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.010+0000 7f2263e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22641074c0 0x7f226419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:39.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.010+0000 7f2263e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2264107e60 0x7f22641a3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:39.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.010+0000 7f2263e00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2264111dd0 con 0x7f2264106390 2024-05-07T02:30:39.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.010+0000 7f2263e00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2264111c50 con 0x7f22641074c0 2024-05-07T02:30:39.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.010+0000 7f2263e00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2264111f50 con 0x7f2264107e60 2024-05-07T02:30:39.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.010+0000 7f2262a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2264106390 0x7f226419c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:39.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.010+0000 7f2262000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22641074c0 0x7f226419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:39.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.010+0000 7f2263400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2264107e60 0x7f22641a3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:39.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.010+0000 7f2262a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2264106390 0x7f226419c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41004/0 (socket says 172.21.15.19:41004) 2024-05-07T02:30:39.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.010+0000 7f2262a00700 1 -- 172.21.15.19:0/3311829442 learned_addr learned my addr 172.21.15.19:0/3311829442 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:30:39.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.011+0000 7f2262000700 1 -- 172.21.15.19:0/3311829442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2264107e60 msgr2=0x7f22641a3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:39.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.011+0000 7f2262000700 1 --2- 172.21.15.19:0/3311829442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2264107e60 0x7f22641a3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:39.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.011+0000 7f2262000700 1 -- 172.21.15.19:0/3311829442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2264106390 msgr2=0x7f226419c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:39.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.011+0000 7f2262000700 1 --2- 172.21.15.19:0/3311829442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2264106390 0x7f226419c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:39.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.011+0000 7f2262000700 1 -- 172.21.15.19:0/3311829442 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22641a44a0 con 0x7f22641074c0 2024-05-07T02:30:39.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.011+0000 7f2262a00700 1 --2- 172.21.15.19:0/3311829442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2264106390 0x7f226419c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:30:39.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.011+0000 7f2262000700 1 --2- 172.21.15.19:0/3311829442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22641074c0 0x7f226419ce80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f224c00e9f0 tx=0x7f224c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:39.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.011+0000 7f225b400700 1 -- 172.21.15.19:0/3311829442 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224c00c9c0 con 0x7f22641074c0 2024-05-07T02:30:39.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.011+0000 7f2263e00700 1 -- 172.21.15.19:0/3311829442 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22641a4790 con 0x7f22641074c0 2024-05-07T02:30:39.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.011+0000 7f225b400700 1 -- 172.21.15.19:0/3311829442 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f224c00cb60 con 0x7f22641074c0 2024-05-07T02:30:39.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.011+0000 7f225b400700 1 -- 172.21.15.19:0/3311829442 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224c018680 con 0x7f22641074c0 2024-05-07T02:30:39.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.012+0000 7f2263e00700 1 -- 172.21.15.19:0/3311829442 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22641a4cd0 con 0x7f22641074c0 2024-05-07T02:30:39.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.013+0000 7f2263e00700 1 -- 172.21.15.19:0/3311829442 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22640512c0 con 0x7f22641074c0 2024-05-07T02:30:39.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.015+0000 7f225b400700 1 -- 172.21.15.19:0/3311829442 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f224c010430 con 0x7f22641074c0 2024-05-07T02:30:39.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.017+0000 7f225b400700 1 --2- 172.21.15.19:0/3311829442 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f22480b2880 0x7f22480b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:39.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.017+0000 7f225b400700 1 -- 172.21.15.19:0/3311829442 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f224c0e2770 con 0x7f22641074c0 2024-05-07T02:30:39.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.017+0000 7f2262a00700 1 --2- 172.21.15.19:0/3311829442 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f22480b2880 0x7f22480b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:39.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.018+0000 7f2262a00700 1 --2- 172.21.15.19:0/3311829442 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f22480b2880 0x7f22480b4d90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f225400b2e0 tx=0x7f22540095d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:39.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.022+0000 7f225b400700 1 -- 172.21.15.19:0/3311829442 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f224c0aa080 con 0x7f22641074c0 2024-05-07T02:30:39.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.279+0000 7f2263e00700 1 -- 172.21.15.19:0/3311829442 --> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f2264063c00 con 0x7f22480b2880 2024-05-07T02:30:39.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.281+0000 7f225b400700 1 -- 172.21.15.19:0/3311829442 <== mgr.14152 v2:172.21.15.19:6800/2059564551 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f2264063c00 con 0x7f22480b2880 2024-05-07T02:30:39.282 INFO:teuthology.orchestra.run.smithi019.stdout:Backend: cephadm 2024-05-07T02:30:39.282 INFO:teuthology.orchestra.run.smithi019.stdout:Available: Yes 2024-05-07T02:30:39.282 INFO:teuthology.orchestra.run.smithi019.stdout:Paused: No 2024-05-07T02:30:39.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.284+0000 7f2263e00700 1 -- 172.21.15.19:0/3311829442 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f22480b2880 msgr2=0x7f22480b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:39.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.284+0000 7f2263e00700 1 --2- 172.21.15.19:0/3311829442 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f22480b2880 0x7f22480b4d90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f225400b2e0 tx=0x7f22540095d0 comp rx=0 tx=0).stop 2024-05-07T02:30:39.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.284+0000 7f2263e00700 1 -- 172.21.15.19:0/3311829442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22641074c0 msgr2=0x7f226419ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:39.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.284+0000 7f2263e00700 1 --2- 172.21.15.19:0/3311829442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22641074c0 0x7f226419ce80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f224c00e9f0 tx=0x7f224c00ed00 comp rx=0 tx=0).stop 2024-05-07T02:30:39.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.285+0000 7f2263e00700 1 -- 172.21.15.19:0/3311829442 shutdown_connections 2024-05-07T02:30:39.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.285+0000 7f2263e00700 1 --2- 172.21.15.19:0/3311829442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2264107e60 0x7f22641a3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:39.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.285+0000 7f2263e00700 1 --2- 172.21.15.19:0/3311829442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2264106390 0x7f226419c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:39.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.285+0000 7f2263e00700 1 --2- 172.21.15.19:0/3311829442 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f22480b2880 0x7f22480b4d90 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:39.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.285+0000 7f2263e00700 1 --2- 172.21.15.19:0/3311829442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22641074c0 0x7f226419ce80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:39.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.285+0000 7f2263e00700 1 -- 172.21.15.19:0/3311829442 >> 172.21.15.19:0/3311829442 conn(0x7f2264101a10 msgr2=0x7f226410a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:39.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.285+0000 7f2263e00700 1 -- 172.21.15.19:0/3311829442 shutdown_connections 2024-05-07T02:30:39.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:39.285+0000 7f2263e00700 1 -- 172.21.15.19:0/3311829442 wait complete. 2024-05-07T02:30:40.224 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-07T02:30:40.693 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:30:40.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:30:40 smithi027 ceph-mon[34398]: from='client.25079 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:30:40.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:30:40 smithi027 ceph-mon[34398]: pgmap v473: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:30:41.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:30:40 smithi019 ceph-mon[29781]: from='client.25079 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:30:41.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:30:40 smithi019 ceph-mon[29781]: pgmap v473: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:30:41.112 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:30:40 smithi064 ceph-mon[34699]: from='client.25079 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:30:41.112 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:30:40 smithi064 ceph-mon[34699]: pgmap v473: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:30:42.884 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:30:42 smithi019 ceph-mon[29781]: pgmap v474: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-07T02:30:42.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:30:42 smithi027 ceph-mon[34398]: pgmap v474: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-07T02:30:43.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:30:42 smithi064 ceph-mon[34699]: pgmap v474: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-07T02:30:43.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.264+0000 7f15fb200700 1 -- 172.21.15.19:0/3980988802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15fc107e50 msgr2=0x7f15fc10e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:43.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.264+0000 7f15fb200700 1 --2- 172.21.15.19:0/3980988802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15fc107e50 0x7f15fc10e700 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f15f400b0a0 tx=0x7f15f400b3b0 comp rx=0 tx=0).stop 2024-05-07T02:30:43.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.264+0000 7f15fb200700 1 -- 172.21.15.19:0/3980988802 shutdown_connections 2024-05-07T02:30:43.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.264+0000 7f15fb200700 1 --2- 172.21.15.19:0/3980988802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15fc107e50 0x7f15fc10e700 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:43.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.264+0000 7f15fb200700 1 --2- 172.21.15.19:0/3980988802 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15fc106380 0x7f15fc106790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:43.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.264+0000 7f15fb200700 1 --2- 172.21.15.19:0/3980988802 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15fc1074b0 0x7f15fc1078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:43.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.264+0000 7f15fb200700 1 -- 172.21.15.19:0/3980988802 >> 172.21.15.19:0/3980988802 conn(0x7f15fc1019c0 msgr2=0x7f15fc103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:43.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.265+0000 7f15fb200700 1 -- 172.21.15.19:0/3980988802 shutdown_connections 2024-05-07T02:30:43.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.265+0000 7f15fb200700 1 -- 172.21.15.19:0/3980988802 wait complete. 2024-05-07T02:30:43.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.266+0000 7f15fb200700 1 Processor -- start 2024-05-07T02:30:43.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.266+0000 7f15fb200700 1 -- start start 2024-05-07T02:30:43.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.267+0000 7f15fb200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15fc106380 0x7f15fc19c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:43.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.267+0000 7f15fb200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15fc1074b0 0x7f15fc19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:43.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.267+0000 7f15fb200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15fc107e50 0x7f15fc1a3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:43.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.267+0000 7f15fb200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f15fc111de0 con 0x7f15fc107e50 2024-05-07T02:30:43.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.267+0000 7f15fb200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f15fc111c60 con 0x7f15fc106380 2024-05-07T02:30:43.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.267+0000 7f15fb200700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f15fc111f60 con 0x7f15fc1074b0 2024-05-07T02:30:43.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.267+0000 7f15f9e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15fc106380 0x7f15fc19c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:43.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.267+0000 7f15f9e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15fc106380 0x7f15fc19c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:59182/0 (socket says 172.21.15.19:59182) 2024-05-07T02:30:43.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.267+0000 7f15f9e00700 1 -- 172.21.15.19:0/168299998 learned_addr learned my addr 172.21.15.19:0/168299998 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:30:43.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.267+0000 7f15f9400700 1 --2- 172.21.15.19:0/168299998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15fc1074b0 0x7f15fc19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:43.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.267+0000 7f15fa800700 1 --2- 172.21.15.19:0/168299998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15fc107e50 0x7f15fc1a3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:43.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.268+0000 7f15f9e00700 1 -- 172.21.15.19:0/168299998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15fc1074b0 msgr2=0x7f15fc19cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:43.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.268+0000 7f15f9e00700 1 --2- 172.21.15.19:0/168299998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15fc1074b0 0x7f15fc19cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:43.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.268+0000 7f15f9e00700 1 -- 172.21.15.19:0/168299998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15fc107e50 msgr2=0x7f15fc1a3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:43.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.268+0000 7f15f9e00700 1 --2- 172.21.15.19:0/168299998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15fc107e50 0x7f15fc1a3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:43.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.268+0000 7f15f9e00700 1 -- 172.21.15.19:0/168299998 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15fc1a43c0 con 0x7f15fc106380 2024-05-07T02:30:43.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.268+0000 7f15f9e00700 1 --2- 172.21.15.19:0/168299998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15fc106380 0x7f15fc19c860 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f15ec00e940 tx=0x7f15ec00ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:43.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.268+0000 7f15f2a00700 1 -- 172.21.15.19:0/168299998 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ec004220 con 0x7f15fc106380 2024-05-07T02:30:43.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.269+0000 7f15fb200700 1 -- 172.21.15.19:0/168299998 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15fc1a46b0 con 0x7f15fc106380 2024-05-07T02:30:43.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.269+0000 7f15f2a00700 1 -- 172.21.15.19:0/168299998 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15ec0043c0 con 0x7f15fc106380 2024-05-07T02:30:43.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.269+0000 7f15f2a00700 1 -- 172.21.15.19:0/168299998 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ec003cc0 con 0x7f15fc106380 2024-05-07T02:30:43.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.269+0000 7f15fb200700 1 -- 172.21.15.19:0/168299998 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15fc1a4bf0 con 0x7f15fc106380 2024-05-07T02:30:43.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.270+0000 7f15fb200700 1 -- 172.21.15.19:0/168299998 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15fc0512c0 con 0x7f15fc106380 2024-05-07T02:30:43.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.273+0000 7f15f2a00700 1 -- 172.21.15.19:0/168299998 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f15ec00c7e0 con 0x7f15fc106380 2024-05-07T02:30:43.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.274+0000 7f15f2a00700 1 --2- 172.21.15.19:0/168299998 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f15e00b2880 0x7f15e00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:43.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.274+0000 7f15f2a00700 1 -- 172.21.15.19:0/168299998 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f15ec0e28d0 con 0x7f15fc106380 2024-05-07T02:30:43.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.274+0000 7f15f9400700 1 --2- 172.21.15.19:0/168299998 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f15e00b2880 0x7f15e00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:43.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.275+0000 7f15f9400700 1 --2- 172.21.15.19:0/168299998 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f15e00b2880 0x7f15e00b4d90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f15fc19d910 tx=0x7f15e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:43.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.276+0000 7f15f2a00700 1 -- 172.21.15.19:0/168299998 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f15ec0aa1e0 con 0x7f15fc106380 2024-05-07T02:30:43.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.529+0000 7f15fb200700 1 -- 172.21.15.19:0/168299998 --> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f15fc063c00 con 0x7f15e00b2880 2024-05-07T02:30:43.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.546+0000 7f15f2a00700 1 -- 172.21.15.19:0/168299998 <== mgr.14152 v2:172.21.15.19:6800/2059564551 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f15fc063c00 con 0x7f15e00b2880 2024-05-07T02:30:43.547 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-07T02:30:43.547 INFO:teuthology.orchestra.run.smithi019.stdout:mds.a smithi019 running (4m) 2m ago 4m 83.5M - 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 34df7d493449 2024-05-07T02:30:43.547 INFO:teuthology.orchestra.run.smithi019.stdout:mds.b smithi027 running (4m) 2m ago 4m 96.3M - 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 92b7019a59ce 2024-05-07T02:30:43.547 INFO:teuthology.orchestra.run.smithi019.stdout:mds.c smithi064 running (5m) 2m ago 5m 87.5M - 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 a0ff8ef5c27e 2024-05-07T02:30:43.547 INFO:teuthology.orchestra.run.smithi019.stdout:mds.d smithi019 running (5m) 2m ago 5m 82.1M - 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 0533f004eb39 2024-05-07T02:30:43.548 INFO:teuthology.orchestra.run.smithi019.stdout:mds.e smithi027 running (5m) 2m ago 5m 85.3M - 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 cbc890c768f0 2024-05-07T02:30:43.548 INFO:teuthology.orchestra.run.smithi019.stdout:mds.f smithi064 running (5m) 2m ago 5m 84.7M - 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 2dabcac5303a 2024-05-07T02:30:43.548 INFO:teuthology.orchestra.run.smithi019.stdout:mds.g smithi019 running (4m) 2m ago 4m 80.3M - 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 c5b8a079a2c9 2024-05-07T02:30:43.548 INFO:teuthology.orchestra.run.smithi019.stdout:mds.h smithi027 running (5m) 2m ago 5m 86.5M - 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 6d86f8fb2e15 2024-05-07T02:30:43.548 INFO:teuthology.orchestra.run.smithi019.stdout:mds.i smithi064 running (5m) 2m ago 5m 87.8M - 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 7d7e4fe2e2cc 2024-05-07T02:30:43.548 INFO:teuthology.orchestra.run.smithi019.stdout:mds.j smithi019 running (5m) 2m ago 5m 82.2M - 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 3b7491ac7c35 2024-05-07T02:30:43.548 INFO:teuthology.orchestra.run.smithi019.stdout:mds.k smithi027 running (5m) 2m ago 5m 83.6M - 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 56b71961f80a 2024-05-07T02:30:43.548 INFO:teuthology.orchestra.run.smithi019.stdout:mds.l smithi064 running (5m) 2m ago 5m 82.0M - 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 62bcd6ca49bc 2024-05-07T02:30:43.548 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.x smithi019 *:9283,8765 running (14m) 2m ago 14m 597M - 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 4c3f8d297faa 2024-05-07T02:30:43.548 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.y smithi027 *:8443,8765 running (11m) 2m ago 11m 535M - 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 ae5c7d337e67 2024-05-07T02:30:43.548 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.z smithi064 *:8443,8765 running (11m) 2m ago 11m 533M - 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 d79d6701e161 2024-05-07T02:30:43.548 INFO:teuthology.orchestra.run.smithi019.stdout:mon.a smithi019 running (14m) 2m ago 14m 116M 2048M 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 6ca858b3ad85 2024-05-07T02:30:43.548 INFO:teuthology.orchestra.run.smithi019.stdout:mon.b smithi027 running (12m) 2m ago 12m 111M 2048M 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 8527eb176b97 2024-05-07T02:30:43.549 INFO:teuthology.orchestra.run.smithi019.stdout:mon.c smithi064 running (12m) 2m ago 12m 114M 2048M 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 3f1234cd50e5 2024-05-07T02:30:43.549 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (11m) 2m ago 11m 120M 4273M 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 990183035c6c 2024-05-07T02:30:43.549 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi027 running (10m) 2m ago 10m 115M 4272M 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 3cb7380e3695 2024-05-07T02:30:43.549 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi064 running (10m) 2m ago 10m 115M 4272M 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 29b568914917 2024-05-07T02:30:43.549 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi019 running (9m) 2m ago 9m 118M 4273M 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 a69af090c734 2024-05-07T02:30:43.549 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi027 running (9m) 2m ago 9m 121M 4272M 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 afc34376c518 2024-05-07T02:30:43.549 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi064 running (9m) 2m ago 9m 114M 4272M 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 8e48787ef25a 2024-05-07T02:30:43.549 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6 smithi019 running (8m) 2m ago 8m 111M 4273M 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 de8d8c5971b0 2024-05-07T02:30:43.549 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7 smithi027 running (7m) 2m ago 7m 111M 4272M 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 0781010a37bd 2024-05-07T02:30:43.549 INFO:teuthology.orchestra.run.smithi019.stdout:osd.8 smithi064 running (7m) 2m ago 7m 119M 4272M 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 98e41fe21893 2024-05-07T02:30:43.549 INFO:teuthology.orchestra.run.smithi019.stdout:osd.9 smithi019 running (6m) 2m ago 6m 110M 4273M 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 af4e5352dd9e 2024-05-07T02:30:43.549 INFO:teuthology.orchestra.run.smithi019.stdout:osd.10 smithi027 running (6m) 2m ago 6m 121M 4272M 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 e18a828b22f3 2024-05-07T02:30:43.549 INFO:teuthology.orchestra.run.smithi019.stdout:osd.11 smithi064 running (5m) 2m ago 5m 122M 4272M 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 b0efb800dae0 2024-05-07T02:30:43.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.549+0000 7f15fb200700 1 -- 172.21.15.19:0/168299998 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f15e00b2880 msgr2=0x7f15e00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:43.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.549+0000 7f15fb200700 1 --2- 172.21.15.19:0/168299998 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f15e00b2880 0x7f15e00b4d90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f15fc19d910 tx=0x7f15e8009260 comp rx=0 tx=0).stop 2024-05-07T02:30:43.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.549+0000 7f15fb200700 1 -- 172.21.15.19:0/168299998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15fc106380 msgr2=0x7f15fc19c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:43.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.549+0000 7f15fb200700 1 --2- 172.21.15.19:0/168299998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15fc106380 0x7f15fc19c860 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f15ec00e940 tx=0x7f15ec00ec50 comp rx=0 tx=0).stop 2024-05-07T02:30:43.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.549+0000 7f15fb200700 1 -- 172.21.15.19:0/168299998 shutdown_connections 2024-05-07T02:30:43.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.549+0000 7f15fb200700 1 --2- 172.21.15.19:0/168299998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15fc107e50 0x7f15fc1a3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:43.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.549+0000 7f15fb200700 1 --2- 172.21.15.19:0/168299998 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f15e00b2880 0x7f15e00b4d90 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:43.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.549+0000 7f15fb200700 1 --2- 172.21.15.19:0/168299998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15fc106380 0x7f15fc19c860 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:43.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.549+0000 7f15fb200700 1 --2- 172.21.15.19:0/168299998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15fc1074b0 0x7f15fc19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:43.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.549+0000 7f15fb200700 1 -- 172.21.15.19:0/168299998 >> 172.21.15.19:0/168299998 conn(0x7f15fc1019c0 msgr2=0x7f15fc10a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:43.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.549+0000 7f15fb200700 1 -- 172.21.15.19:0/168299998 shutdown_connections 2024-05-07T02:30:43.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:43.549+0000 7f15fb200700 1 -- 172.21.15.19:0/168299998 wait complete. 2024-05-07T02:30:44.581 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-07T02:30:44.802 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:30:44 smithi019 ceph-mon[29781]: from='client.25085 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:30:44.802 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:30:44 smithi019 ceph-mon[29781]: pgmap v475: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-07T02:30:44.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:30:44 smithi027 ceph-mon[34398]: from='client.25085 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:30:44.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:30:44 smithi027 ceph-mon[34398]: pgmap v475: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-07T02:30:45.060 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:30:45.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:30:44 smithi064 ceph-mon[34699]: from='client.25085 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:30:45.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:30:44 smithi064 ceph-mon[34699]: pgmap v475: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-07T02:30:46.867 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:30:46 smithi019 ceph-mon[29781]: pgmap v476: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-07T02:30:46.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:30:46 smithi027 ceph-mon[34398]: pgmap v476: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-07T02:30:47.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:30:46 smithi064 ceph-mon[34699]: pgmap v476: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-07T02:30:47.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.497+0000 7fbb13200700 1 -- 172.21.15.19:0/2754467143 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb0c104150 msgr2=0x7fbb0c104560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:47.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.497+0000 7fbb13200700 1 --2- 172.21.15.19:0/2754467143 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb0c104150 0x7fbb0c104560 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbb00009990 tx=0x7fbb00009ca0 comp rx=0 tx=0).stop 2024-05-07T02:30:47.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.497+0000 7fbb13200700 1 -- 172.21.15.19:0/2754467143 shutdown_connections 2024-05-07T02:30:47.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.497+0000 7fbb13200700 1 --2- 172.21.15.19:0/2754467143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb0c105c20 0x7fbb0c10c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:47.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.497+0000 7fbb13200700 1 --2- 172.21.15.19:0/2754467143 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb0c104150 0x7fbb0c104560 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:47.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.497+0000 7fbb13200700 1 --2- 172.21.15.19:0/2754467143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb0c105280 0x7fbb0c105690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:47.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.497+0000 7fbb13200700 1 -- 172.21.15.19:0/2754467143 >> 172.21.15.19:0/2754467143 conn(0x7fbb0c0ff7b0 msgr2=0x7fbb0c101c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:47.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.497+0000 7fbb13200700 1 -- 172.21.15.19:0/2754467143 shutdown_connections 2024-05-07T02:30:47.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.498+0000 7fbb13200700 1 -- 172.21.15.19:0/2754467143 wait complete. 2024-05-07T02:30:47.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.499+0000 7fbb13200700 1 Processor -- start 2024-05-07T02:30:47.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.499+0000 7fbb13200700 1 -- start start 2024-05-07T02:30:47.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.499+0000 7fbb13200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb0c104150 0x7fbb0c077460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:47.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.499+0000 7fbb13200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb0c105280 0x7fbb0c075b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:47.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.499+0000 7fbb13200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb0c105c20 0x7fbb0c076050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:47.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.499+0000 7fbb13200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c10fc10 con 0x7fbb0c105280 2024-05-07T02:30:47.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.499+0000 7fbb13200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c10fa90 con 0x7fbb0c105c20 2024-05-07T02:30:47.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.499+0000 7fbb13200700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c10fd90 con 0x7fbb0c104150 2024-05-07T02:30:47.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.500+0000 7fbb11400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb0c105280 0x7fbb0c075b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:47.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.500+0000 7fbb11e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb0c104150 0x7fbb0c077460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:47.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.500+0000 7fbb11e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb0c104150 0x7fbb0c077460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:41524/0 (socket says 172.21.15.19:41524) 2024-05-07T02:30:47.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.500+0000 7fbb11400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb0c105280 0x7fbb0c075b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41040/0 (socket says 172.21.15.19:41040) 2024-05-07T02:30:47.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.500+0000 7fbb11e00700 1 -- 172.21.15.19:0/1183128238 learned_addr learned my addr 172.21.15.19:0/1183128238 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:30:47.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.500+0000 7fbb12800700 1 --2- 172.21.15.19:0/1183128238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb0c105c20 0x7fbb0c076050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:47.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.500+0000 7fbb11e00700 1 -- 172.21.15.19:0/1183128238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb0c105c20 msgr2=0x7fbb0c076050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:47.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.500+0000 7fbb11e00700 1 --2- 172.21.15.19:0/1183128238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb0c105c20 0x7fbb0c076050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:47.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.500+0000 7fbb11e00700 1 -- 172.21.15.19:0/1183128238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb0c105280 msgr2=0x7fbb0c075b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:47.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.500+0000 7fbb11e00700 1 --2- 172.21.15.19:0/1183128238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb0c105280 0x7fbb0c075b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:47.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.500+0000 7fbb11e00700 1 -- 172.21.15.19:0/1183128238 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb0c0767b0 con 0x7fbb0c104150 2024-05-07T02:30:47.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.501+0000 7fbb11e00700 1 --2- 172.21.15.19:0/1183128238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb0c104150 0x7fbb0c077460 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbb0000b6c0 tx=0x7fbb00009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:47.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.501+0000 7fbb06a00700 1 -- 172.21.15.19:0/1183128238 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb00026070 con 0x7fbb0c104150 2024-05-07T02:30:47.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.501+0000 7fbb13200700 1 -- 172.21.15.19:0/1183128238 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb0c1a4880 con 0x7fbb0c104150 2024-05-07T02:30:47.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.501+0000 7fbb06a00700 1 -- 172.21.15.19:0/1183128238 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb00021410 con 0x7fbb0c104150 2024-05-07T02:30:47.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.501+0000 7fbb06a00700 1 -- 172.21.15.19:0/1183128238 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb00017650 con 0x7fbb0c104150 2024-05-07T02:30:47.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.501+0000 7fbb13200700 1 -- 172.21.15.19:0/1183128238 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb0c1a4c70 con 0x7fbb0c104150 2024-05-07T02:30:47.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.502+0000 7fbb13200700 1 -- 172.21.15.19:0/1183128238 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb0c1103d0 con 0x7fbb0c104150 2024-05-07T02:30:47.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.505+0000 7fbb06a00700 1 -- 172.21.15.19:0/1183128238 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fbb00029070 con 0x7fbb0c104150 2024-05-07T02:30:47.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.507+0000 7fbb06a00700 1 --2- 172.21.15.19:0/1183128238 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fbaf40b2900 0x7fbaf40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:47.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.507+0000 7fbb06a00700 1 -- 172.21.15.19:0/1183128238 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fbb000e3600 con 0x7fbb0c104150 2024-05-07T02:30:47.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.507+0000 7fbb11400700 1 --2- 172.21.15.19:0/1183128238 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fbaf40b2900 0x7fbaf40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:47.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.509+0000 7fbb11400700 1 --2- 172.21.15.19:0/1183128238 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fbaf40b2900 0x7fbaf40b4e10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c076e10 tx=0x7fbafc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:47.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.509+0000 7fbb06a00700 1 -- 172.21.15.19:0/1183128238 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbb00013070 con 0x7fbb0c104150 2024-05-07T02:30:47.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.758+0000 7fbb13200700 1 -- 172.21.15.19:0/1183128238 --> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fbb0c02e990 con 0x7fbaf40b2900 2024-05-07T02:30:47.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.769+0000 7fbb06a00700 1 -- 172.21.15.19:0/1183128238 <== mgr.14152 v2:172.21.15.19:6800/2059564551 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fbb0c02e990 con 0x7fbaf40b2900 2024-05-07T02:30:47.769 INFO:teuthology.orchestra.run.smithi019.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-07T02:30:47.769 INFO:teuthology.orchestra.run.smithi019.stdout:mds.all 12/12 2m ago 5m smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi027=b;smithi027=e;smithi027=h;smithi027=k;smithi064=c;smithi064=f;smithi064=i;smithi064=l;count:12 2024-05-07T02:30:47.770 INFO:teuthology.orchestra.run.smithi019.stdout:mgr 3/3 2m ago 11m smithi019=x;smithi027=y;smithi064=z;count:3 2024-05-07T02:30:47.770 INFO:teuthology.orchestra.run.smithi019.stdout:mon 3/3 2m ago 12m smithi019:172.21.15.19=a;smithi027:172.21.15.27=b;smithi064:172.21.15.64=c;count:3 2024-05-07T02:30:47.770 INFO:teuthology.orchestra.run.smithi019.stdout:osd 12 2m ago - 2024-05-07T02:30:47.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.772+0000 7fbb13200700 1 -- 172.21.15.19:0/1183128238 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fbaf40b2900 msgr2=0x7fbaf40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:47.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.772+0000 7fbb13200700 1 --2- 172.21.15.19:0/1183128238 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fbaf40b2900 0x7fbaf40b4e10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c076e10 tx=0x7fbafc009260 comp rx=0 tx=0).stop 2024-05-07T02:30:47.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.772+0000 7fbb13200700 1 -- 172.21.15.19:0/1183128238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb0c104150 msgr2=0x7fbb0c077460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:47.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.772+0000 7fbb13200700 1 --2- 172.21.15.19:0/1183128238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb0c104150 0x7fbb0c077460 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbb0000b6c0 tx=0x7fbb00009580 comp rx=0 tx=0).stop 2024-05-07T02:30:47.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.772+0000 7fbb13200700 1 -- 172.21.15.19:0/1183128238 shutdown_connections 2024-05-07T02:30:47.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.772+0000 7fbb13200700 1 --2- 172.21.15.19:0/1183128238 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fbaf40b2900 0x7fbaf40b4e10 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:47.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.772+0000 7fbb13200700 1 --2- 172.21.15.19:0/1183128238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb0c105c20 0x7fbb0c076050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:47.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.772+0000 7fbb13200700 1 --2- 172.21.15.19:0/1183128238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb0c104150 0x7fbb0c077460 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:47.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.772+0000 7fbb13200700 1 --2- 172.21.15.19:0/1183128238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb0c105280 0x7fbb0c075b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:47.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.772+0000 7fbb13200700 1 -- 172.21.15.19:0/1183128238 >> 172.21.15.19:0/1183128238 conn(0x7fbb0c0ff7b0 msgr2=0x7fbb0c1084b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:47.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.772+0000 7fbb13200700 1 -- 172.21.15.19:0/1183128238 shutdown_connections 2024-05-07T02:30:47.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:47.772+0000 7fbb13200700 1 -- 172.21.15.19:0/1183128238 wait complete. 2024-05-07T02:30:48.735 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-07T02:30:48.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:30:48 smithi027 ceph-mon[34398]: pgmap v477: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-07T02:30:48.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:30:48 smithi027 ceph-mon[34398]: from='client.25087 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:30:49.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:30:48 smithi064 ceph-mon[34699]: pgmap v477: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-07T02:30:49.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:30:48 smithi064 ceph-mon[34699]: from='client.25087 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:30:49.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:30:48 smithi019 ceph-mon[29781]: pgmap v477: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-07T02:30:49.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:30:48 smithi019 ceph-mon[29781]: from='client.25087 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:30:49.208 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:30:50.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:30:50 smithi019 ceph-mon[29781]: pgmap v478: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-07T02:30:50.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:30:50 smithi027 ceph-mon[34398]: pgmap v478: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-07T02:30:51.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:30:50 smithi064 ceph-mon[34699]: pgmap v478: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-07T02:30:51.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.610+0000 7f48d8a00700 1 -- 172.21.15.19:0/2748766268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48d4106390 msgr2=0x7f48d41067a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:51.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.610+0000 7f48d8a00700 1 --2- 172.21.15.19:0/2748766268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48d4106390 0x7f48d41067a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f48c400b0a0 tx=0x7f48c400b3b0 comp rx=0 tx=0).stop 2024-05-07T02:30:51.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.610+0000 7f48d8a00700 1 -- 172.21.15.19:0/2748766268 shutdown_connections 2024-05-07T02:30:51.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.610+0000 7f48d8a00700 1 --2- 172.21.15.19:0/2748766268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48d4107e60 0x7f48d410e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:51.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.610+0000 7f48d8a00700 1 --2- 172.21.15.19:0/2748766268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48d4106390 0x7f48d41067a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:51.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.610+0000 7f48d8a00700 1 --2- 172.21.15.19:0/2748766268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48d41074c0 0x7f48d41078d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:51.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.610+0000 7f48d8a00700 1 -- 172.21.15.19:0/2748766268 >> 172.21.15.19:0/2748766268 conn(0x7f48d4101a10 msgr2=0x7f48d4103e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:51.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.610+0000 7f48d8a00700 1 -- 172.21.15.19:0/2748766268 shutdown_connections 2024-05-07T02:30:51.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.610+0000 7f48d8a00700 1 -- 172.21.15.19:0/2748766268 wait complete. 2024-05-07T02:30:51.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.612+0000 7f48d8a00700 1 Processor -- start 2024-05-07T02:30:51.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.612+0000 7f48d8a00700 1 -- start start 2024-05-07T02:30:51.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.613+0000 7f48d8a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48d4106390 0x7f48d419c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:51.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.613+0000 7f48d8a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48d41074c0 0x7f48d419ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:51.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.613+0000 7f48d8a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48d4107e60 0x7f48d41a3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:51.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.613+0000 7f48d8a00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f48d4111e30 con 0x7f48d4107e60 2024-05-07T02:30:51.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.613+0000 7f48d8a00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f48d4111cb0 con 0x7f48d41074c0 2024-05-07T02:30:51.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.613+0000 7f48d8a00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f48d4111fb0 con 0x7f48d4106390 2024-05-07T02:30:51.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.613+0000 7f48d3400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48d4106390 0x7f48d419c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:51.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.613+0000 7f48d2a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48d41074c0 0x7f48d419ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:51.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.613+0000 7f48d2a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48d41074c0 0x7f48d419ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:52298/0 (socket says 172.21.15.19:52298) 2024-05-07T02:30:51.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.613+0000 7f48d2a00700 1 -- 172.21.15.19:0/605035915 learned_addr learned my addr 172.21.15.19:0/605035915 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:30:51.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.613+0000 7f48d3400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48d4106390 0x7f48d419c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:51232/0 (socket says 172.21.15.19:51232) 2024-05-07T02:30:51.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.613+0000 7f48d3e00700 1 --2- 172.21.15.19:0/605035915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48d4107e60 0x7f48d41a3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:51.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.614+0000 7f48d3400700 1 -- 172.21.15.19:0/605035915 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48d41074c0 msgr2=0x7f48d419ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:51.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.614+0000 7f48d3400700 1 --2- 172.21.15.19:0/605035915 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48d41074c0 0x7f48d419ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:51.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.614+0000 7f48d3400700 1 -- 172.21.15.19:0/605035915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48d4107e60 msgr2=0x7f48d41a3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:51.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.614+0000 7f48d3400700 1 --2- 172.21.15.19:0/605035915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48d4107e60 0x7f48d41a3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:51.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.614+0000 7f48d3400700 1 -- 172.21.15.19:0/605035915 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48d41a44d0 con 0x7f48d4106390 2024-05-07T02:30:51.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.614+0000 7f48d2a00700 1 --2- 172.21.15.19:0/605035915 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48d41074c0 0x7f48d419ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:30:51.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.614+0000 7f48d3e00700 1 --2- 172.21.15.19:0/605035915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48d4107e60 0x7f48d41a3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:30:51.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.614+0000 7f48d3400700 1 --2- 172.21.15.19:0/605035915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48d4106390 0x7f48d419c970 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f48c4000f80 tx=0x7f48c4004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:51.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.615+0000 7f48cbe00700 1 -- 172.21.15.19:0/605035915 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c4021070 con 0x7f48d4106390 2024-05-07T02:30:51.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.615+0000 7f48d8a00700 1 -- 172.21.15.19:0/605035915 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48d41a4760 con 0x7f48d4106390 2024-05-07T02:30:51.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.615+0000 7f48cbe00700 1 -- 172.21.15.19:0/605035915 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48c4007b80 con 0x7f48d4106390 2024-05-07T02:30:51.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.615+0000 7f48cbe00700 1 -- 172.21.15.19:0/605035915 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c4005040 con 0x7f48d4106390 2024-05-07T02:30:51.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.615+0000 7f48d8a00700 1 -- 172.21.15.19:0/605035915 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48d41a4cc0 con 0x7f48d4106390 2024-05-07T02:30:51.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.616+0000 7f48d8a00700 1 -- 172.21.15.19:0/605035915 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48d40512c0 con 0x7f48d4106390 2024-05-07T02:30:51.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.619+0000 7f48cbe00700 1 -- 172.21.15.19:0/605035915 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f48c4024070 con 0x7f48d4106390 2024-05-07T02:30:51.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.620+0000 7f48cbe00700 1 --2- 172.21.15.19:0/605035915 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f48b80b2880 0x7f48b80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:51.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.620+0000 7f48cbe00700 1 -- 172.21.15.19:0/605035915 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f48c400f020 con 0x7f48d4106390 2024-05-07T02:30:51.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.621+0000 7f48d2a00700 1 --2- 172.21.15.19:0/605035915 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f48b80b2880 0x7f48b80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:51.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.621+0000 7f48d2a00700 1 --2- 172.21.15.19:0/605035915 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f48b80b2880 0x7f48b80b4d90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f48d419da20 tx=0x7f48c0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:51.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.623+0000 7f48cbe00700 1 -- 172.21.15.19:0/605035915 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f48c402b020 con 0x7f48d4106390 2024-05-07T02:30:51.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.870+0000 7f48d8a00700 1 -- 172.21.15.19:0/605035915 --> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f48d4063c00 con 0x7f48b80b2880 2024-05-07T02:30:51.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.875+0000 7f48cbe00700 1 -- 172.21.15.19:0/605035915 <== mgr.14152 v2:172.21.15.19:6800/2059564551 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+187 (secure 0 0 0) 0x7f48d4063c00 con 0x7f48b80b2880 2024-05-07T02:30:51.874 INFO:teuthology.orchestra.run.smithi019.stdout:HOST ADDR LABELS STATUS 2024-05-07T02:30:51.875 INFO:teuthology.orchestra.run.smithi019.stdout:smithi019 172.21.15.19 2024-05-07T02:30:51.875 INFO:teuthology.orchestra.run.smithi019.stdout:smithi027 172.21.15.27 2024-05-07T02:30:51.875 INFO:teuthology.orchestra.run.smithi019.stdout:smithi064 172.21.15.64 2024-05-07T02:30:51.875 INFO:teuthology.orchestra.run.smithi019.stdout:3 hosts in cluster 2024-05-07T02:30:51.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.878+0000 7f48d8a00700 1 -- 172.21.15.19:0/605035915 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f48b80b2880 msgr2=0x7f48b80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:51.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.878+0000 7f48d8a00700 1 --2- 172.21.15.19:0/605035915 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f48b80b2880 0x7f48b80b4d90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f48d419da20 tx=0x7f48c0009260 comp rx=0 tx=0).stop 2024-05-07T02:30:51.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.878+0000 7f48d8a00700 1 -- 172.21.15.19:0/605035915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48d4106390 msgr2=0x7f48d419c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:51.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.878+0000 7f48d8a00700 1 --2- 172.21.15.19:0/605035915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48d4106390 0x7f48d419c970 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f48c4000f80 tx=0x7f48c4004d00 comp rx=0 tx=0).stop 2024-05-07T02:30:51.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.878+0000 7f48d8a00700 1 -- 172.21.15.19:0/605035915 shutdown_connections 2024-05-07T02:30:51.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.878+0000 7f48d8a00700 1 --2- 172.21.15.19:0/605035915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f48d4107e60 0x7f48d41a3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:51.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.878+0000 7f48d8a00700 1 --2- 172.21.15.19:0/605035915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48d4106390 0x7f48d419c970 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:51.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.878+0000 7f48d8a00700 1 --2- 172.21.15.19:0/605035915 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f48b80b2880 0x7f48b80b4d90 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:51.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.878+0000 7f48d8a00700 1 --2- 172.21.15.19:0/605035915 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48d41074c0 0x7f48d419ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:51.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.878+0000 7f48d8a00700 1 -- 172.21.15.19:0/605035915 >> 172.21.15.19:0/605035915 conn(0x7f48d4101a10 msgr2=0x7f48d410a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:51.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.878+0000 7f48d8a00700 1 -- 172.21.15.19:0/605035915 shutdown_connections 2024-05-07T02:30:51.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:51.878+0000 7f48d8a00700 1 -- 172.21.15.19:0/605035915 wait complete. 2024-05-07T02:30:52.865 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-07T02:30:52.949 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:30:52 smithi019 ceph-mon[29781]: pgmap v479: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-07T02:30:52.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:30:52 smithi027 ceph-mon[34398]: pgmap v479: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-07T02:30:53.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:30:52 smithi064 ceph-mon[34699]: pgmap v479: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-07T02:30:53.335 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:30:53.835 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:30:53 smithi019 ceph-mon[29781]: from='client.25093 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:30:53.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:30:53 smithi027 ceph-mon[34398]: from='client.25093 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:30:54.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:30:53 smithi064 ceph-mon[34699]: from='client.25093 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:30:54.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:30:54 smithi019 ceph-mon[29781]: pgmap v480: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:30:54.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:30:54 smithi027 ceph-mon[34398]: pgmap v480: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:30:55.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:30:54 smithi064 ceph-mon[34699]: pgmap v480: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:30:55.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.685+0000 7fd558000700 1 -- 172.21.15.19:0/2014382578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd550104e40 msgr2=0x7fd550107260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:55.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.685+0000 7fd558000700 1 --2- 172.21.15.19:0/2014382578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd550104e40 0x7fd550107260 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd544009990 tx=0x7fd544009ca0 comp rx=0 tx=0).stop 2024-05-07T02:30:55.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.685+0000 7fd558000700 1 -- 172.21.15.19:0/2014382578 shutdown_connections 2024-05-07T02:30:55.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.685+0000 7fd558000700 1 --2- 172.21.15.19:0/2014382578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd55010a100 0x7fd55010c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:55.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.685+0000 7fd558000700 1 --2- 172.21.15.19:0/2014382578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd550104e40 0x7fd550107260 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:55.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.685+0000 7fd558000700 1 --2- 172.21.15.19:0/2014382578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5501077a0 0x7fd550109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:55.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.685+0000 7fd558000700 1 -- 172.21.15.19:0/2014382578 >> 172.21.15.19:0/2014382578 conn(0x7fd5500fe880 msgr2=0x7fd550100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:55.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.685+0000 7fd558000700 1 -- 172.21.15.19:0/2014382578 shutdown_connections 2024-05-07T02:30:55.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.685+0000 7fd558000700 1 -- 172.21.15.19:0/2014382578 wait complete. 2024-05-07T02:30:55.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.687+0000 7fd558000700 1 Processor -- start 2024-05-07T02:30:55.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.687+0000 7fd558000700 1 -- start start 2024-05-07T02:30:55.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.688+0000 7fd558000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd550104e40 0x7fd5501046a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:55.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.688+0000 7fd558000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5501077a0 0x7fd550102d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:55.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.688+0000 7fd558000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd55010a100 0x7fd550103290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:55.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.688+0000 7fd558000700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd55010fd80 con 0x7fd550104e40 2024-05-07T02:30:55.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.688+0000 7fd558000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd55010fc00 con 0x7fd5501077a0 2024-05-07T02:30:55.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.688+0000 7fd558000700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd55010ff00 con 0x7fd55010a100 2024-05-07T02:30:55.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.688+0000 7fd554c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5501077a0 0x7fd550102d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:55.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.688+0000 7fd555600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd550104e40 0x7fd5501046a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:55.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.688+0000 7fd554c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5501077a0 0x7fd550102d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:52318/0 (socket says 172.21.15.19:52318) 2024-05-07T02:30:55.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.688+0000 7fd554c00700 1 -- 172.21.15.19:0/2535651901 learned_addr learned my addr 172.21.15.19:0/2535651901 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:30:55.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.688+0000 7fd556000700 1 --2- 172.21.15.19:0/2535651901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd55010a100 0x7fd550103290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:55.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.689+0000 7fd554c00700 1 -- 172.21.15.19:0/2535651901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd55010a100 msgr2=0x7fd550103290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:55.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.689+0000 7fd554c00700 1 --2- 172.21.15.19:0/2535651901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd55010a100 0x7fd550103290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:55.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.689+0000 7fd554c00700 1 -- 172.21.15.19:0/2535651901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd550104e40 msgr2=0x7fd5501046a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:55.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.689+0000 7fd554c00700 1 --2- 172.21.15.19:0/2535651901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd550104e40 0x7fd5501046a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:55.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.689+0000 7fd554c00700 1 -- 172.21.15.19:0/2535651901 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5501039f0 con 0x7fd5501077a0 2024-05-07T02:30:55.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.689+0000 7fd554c00700 1 --2- 172.21.15.19:0/2535651901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5501077a0 0x7fd550102d50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd53c00e9a0 tx=0x7fd53c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:55.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.689+0000 7fd54e000700 1 -- 172.21.15.19:0/2535651901 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53c00ca10 con 0x7fd5501077a0 2024-05-07T02:30:55.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.689+0000 7fd54e000700 1 -- 172.21.15.19:0/2535651901 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd53c00cbb0 con 0x7fd5501077a0 2024-05-07T02:30:55.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.689+0000 7fd54e000700 1 -- 172.21.15.19:0/2535651901 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53c018680 con 0x7fd5501077a0 2024-05-07T02:30:55.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.689+0000 7fd558000700 1 -- 172.21.15.19:0/2535651901 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5501a49f0 con 0x7fd5501077a0 2024-05-07T02:30:55.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.690+0000 7fd558000700 1 -- 172.21.15.19:0/2535651901 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5501a4e80 con 0x7fd5501077a0 2024-05-07T02:30:55.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.691+0000 7fd558000700 1 -- 172.21.15.19:0/2535651901 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5500512c0 con 0x7fd5501077a0 2024-05-07T02:30:55.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.694+0000 7fd54e000700 1 -- 172.21.15.19:0/2535651901 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd53c04aab0 con 0x7fd5501077a0 2024-05-07T02:30:55.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.695+0000 7fd54e000700 1 --2- 172.21.15.19:0/2535651901 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fd5380b2900 0x7fd5380b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:55.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.696+0000 7fd54e000700 1 -- 172.21.15.19:0/2535651901 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fd53c0e2840 con 0x7fd5501077a0 2024-05-07T02:30:55.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.696+0000 7fd555600700 1 --2- 172.21.15.19:0/2535651901 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fd5380b2900 0x7fd5380b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:55.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.697+0000 7fd555600700 1 --2- 172.21.15.19:0/2535651901 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fd5380b2900 0x7fd5380b4e10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd544006010 tx=0x7fd54400b720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:55.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.698+0000 7fd54e000700 1 -- 172.21.15.19:0/2535651901 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd53c0aa150 con 0x7fd5501077a0 2024-05-07T02:30:55.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.952+0000 7fd558000700 1 -- 172.21.15.19:0/2535651901 --> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fd550063c00 con 0x7fd5380b2900 2024-05-07T02:30:55.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.956+0000 7fd54e000700 1 -- 172.21.15.19:0/2535651901 <== mgr.14152 v2:172.21.15.19:6800/2059564551 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fd550063c00 con 0x7fd5380b2900 2024-05-07T02:30:55.956 INFO:teuthology.orchestra.run.smithi019.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-07T02:30:55.956 INFO:teuthology.orchestra.run.smithi019.stdout:smithi019 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330002G400BGN 372G No 6m ago Has a FileSystem, LVM detected 2024-05-07T02:30:55.956 INFO:teuthology.orchestra.run.smithi019.stdout:smithi027 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5331001C400BGN 372G No 6m ago Has a FileSystem, LVM detected 2024-05-07T02:30:55.956 INFO:teuthology.orchestra.run.smithi019.stdout:smithi064 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT534400C2400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-05-07T02:30:55.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.960+0000 7fd558000700 1 -- 172.21.15.19:0/2535651901 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fd5380b2900 msgr2=0x7fd5380b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:55.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.960+0000 7fd558000700 1 --2- 172.21.15.19:0/2535651901 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fd5380b2900 0x7fd5380b4e10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd544006010 tx=0x7fd54400b720 comp rx=0 tx=0).stop 2024-05-07T02:30:55.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.960+0000 7fd558000700 1 -- 172.21.15.19:0/2535651901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5501077a0 msgr2=0x7fd550102d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:55.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.960+0000 7fd558000700 1 --2- 172.21.15.19:0/2535651901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5501077a0 0x7fd550102d50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd53c00e9a0 tx=0x7fd53c00ecb0 comp rx=0 tx=0).stop 2024-05-07T02:30:55.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.960+0000 7fd558000700 1 -- 172.21.15.19:0/2535651901 shutdown_connections 2024-05-07T02:30:55.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.960+0000 7fd558000700 1 --2- 172.21.15.19:0/2535651901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd55010a100 0x7fd550103290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:55.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.960+0000 7fd558000700 1 --2- 172.21.15.19:0/2535651901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd550104e40 0x7fd5501046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:55.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.960+0000 7fd558000700 1 --2- 172.21.15.19:0/2535651901 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fd5380b2900 0x7fd5380b4e10 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:55.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.960+0000 7fd558000700 1 --2- 172.21.15.19:0/2535651901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5501077a0 0x7fd550102d50 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:55.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.960+0000 7fd558000700 1 -- 172.21.15.19:0/2535651901 >> 172.21.15.19:0/2535651901 conn(0x7fd5500fe880 msgr2=0x7fd550105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:55.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.960+0000 7fd558000700 1 -- 172.21.15.19:0/2535651901 shutdown_connections 2024-05-07T02:30:55.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:55.960+0000 7fd558000700 1 -- 172.21.15.19:0/2535651901 wait complete. 2024-05-07T02:30:56.942 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-07T02:30:56.952 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-05-07T02:30:56.952 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-07T02:30:57.035 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:30:56 smithi019 ceph-mon[29781]: pgmap v481: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:30:57.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:30:56 smithi064 ceph-mon[34699]: pgmap v481: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:30:57.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:30:56 smithi027 ceph-mon[34398]: pgmap v481: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:30:57.411 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:30:57.817 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:30:57 smithi019 ceph-mon[29781]: from='client.25103 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:30:58.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:30:57 smithi064 ceph-mon[34699]: from='client.25103 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:30:58.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:30:57 smithi027 ceph-mon[34398]: from='client.25103 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:30:59.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:30:58 smithi064 ceph-mon[34699]: pgmap v482: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:30:59.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:30:58 smithi019 ceph-mon[29781]: pgmap v482: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:30:59.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:30:58 smithi027 ceph-mon[34398]: pgmap v482: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:30:59.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:59.953+0000 7f1771e00700 1 -- 172.21.15.19:0/2600293896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f176c10ba80 msgr2=0x7f176c10be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:59.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:59.953+0000 7f1771e00700 1 --2- 172.21.15.19:0/2600293896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f176c10ba80 0x7f176c10be90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1758009990 tx=0x7f1758009ca0 comp rx=0 tx=0).stop 2024-05-07T02:30:59.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:59.953+0000 7f1771e00700 1 -- 172.21.15.19:0/2600293896 shutdown_connections 2024-05-07T02:30:59.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:59.953+0000 7f1771e00700 1 --2- 172.21.15.19:0/2600293896 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f176c10c420 0x7f176c112cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:59.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:59.953+0000 7f1771e00700 1 --2- 172.21.15.19:0/2600293896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f176c10ab10 0x7f176c10af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:59.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:59.953+0000 7f1771e00700 1 --2- 172.21.15.19:0/2600293896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f176c10ba80 0x7f176c10be90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:59.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:59.953+0000 7f1771e00700 1 -- 172.21.15.19:0/2600293896 >> 172.21.15.19:0/2600293896 conn(0x7f176c0799c0 msgr2=0x7f176c07be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:30:59.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:59.953+0000 7f1771e00700 1 -- 172.21.15.19:0/2600293896 shutdown_connections 2024-05-07T02:30:59.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:59.953+0000 7f1771e00700 1 -- 172.21.15.19:0/2600293896 wait complete. 2024-05-07T02:30:59.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:59.955+0000 7f1771e00700 1 Processor -- start 2024-05-07T02:30:59.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:59.955+0000 7f1771e00700 1 -- start start 2024-05-07T02:30:59.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:59.955+0000 7f1771e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f176c10ab10 0x7f176c1a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:59.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:59.955+0000 7f1771e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f176c10ba80 0x7f176c1a1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:59.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:59.955+0000 7f1771e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f176c10c420 0x7f176c1a83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:59.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:59.955+0000 7f1771e00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f176c1163c0 con 0x7f176c10c420 2024-05-07T02:30:59.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:59.955+0000 7f1771e00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f176c116240 con 0x7f176c10ab10 2024-05-07T02:30:59.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:59.955+0000 7f1771e00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f176c116540 con 0x7f176c10ba80 2024-05-07T02:30:59.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:59.956+0000 7f176aa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f176c10ab10 0x7f176c1a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:59.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:59.956+0000 7f176b400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f176c10c420 0x7f176c1a83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:59.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:59.956+0000 7f176aa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f176c10ab10 0x7f176c1a0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:48574/0 (socket says 172.21.15.19:48574) 2024-05-07T02:30:59.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:59.956+0000 7f176aa00700 1 -- 172.21.15.19:0/1669023609 learned_addr learned my addr 172.21.15.19:0/1669023609 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:30:59.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:59.956+0000 7f176a000700 1 --2- 172.21.15.19:0/1669023609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f176c10ba80 0x7f176c1a1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:59.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:59.956+0000 7f176aa00700 1 -- 172.21.15.19:0/1669023609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f176c10ba80 msgr2=0x7f176c1a1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:59.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:59.956+0000 7f176aa00700 1 --2- 172.21.15.19:0/1669023609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f176c10ba80 0x7f176c1a1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:59.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:59.956+0000 7f176aa00700 1 -- 172.21.15.19:0/1669023609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f176c10c420 msgr2=0x7f176c1a83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:30:59.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:59.956+0000 7f176aa00700 1 --2- 172.21.15.19:0/1669023609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f176c10c420 0x7f176c1a83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:30:59.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:59.956+0000 7f176aa00700 1 -- 172.21.15.19:0/1669023609 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f176c1a8a40 con 0x7f176c10ab10 2024-05-07T02:30:59.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:59.957+0000 7f176aa00700 1 --2- 172.21.15.19:0/1669023609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f176c10ab10 0x7f176c1a0ee0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f175c00e940 tx=0x7f175c00ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:59.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:59.957+0000 7f1770a00700 1 -- 172.21.15.19:0/1669023609 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175c004220 con 0x7f176c10ab10 2024-05-07T02:30:59.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:59.957+0000 7f1770a00700 1 -- 172.21.15.19:0/1669023609 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f175c0043c0 con 0x7f176c10ab10 2024-05-07T02:30:59.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:59.957+0000 7f1771e00700 1 -- 172.21.15.19:0/1669023609 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f176c1a8d30 con 0x7f176c10ab10 2024-05-07T02:30:59.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:59.957+0000 7f1770a00700 1 -- 172.21.15.19:0/1669023609 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175c003c60 con 0x7f176c10ab10 2024-05-07T02:30:59.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:59.957+0000 7f1771e00700 1 -- 172.21.15.19:0/1669023609 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f176c1a9200 con 0x7f176c10ab10 2024-05-07T02:30:59.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:59.959+0000 7f1771e00700 1 -- 172.21.15.19:0/1669023609 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f176c0512c0 con 0x7f176c10ab10 2024-05-07T02:30:59.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:59.961+0000 7f1770a00700 1 -- 172.21.15.19:0/1669023609 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f175c00c7e0 con 0x7f176c10ab10 2024-05-07T02:30:59.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:59.963+0000 7f1770a00700 1 --2- 172.21.15.19:0/1669023609 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f17500b2900 0x7f17500b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:30:59.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:59.963+0000 7f1770a00700 1 -- 172.21.15.19:0/1669023609 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f175c0e27f0 con 0x7f176c10ab10 2024-05-07T02:30:59.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:59.963+0000 7f176a000700 1 --2- 172.21.15.19:0/1669023609 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f17500b2900 0x7f17500b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:30:59.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:59.964+0000 7f176a000700 1 --2- 172.21.15.19:0/1669023609 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f17500b2900 0x7f17500b4e10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f175800b760 tx=0x7f175800b680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:30:59.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:30:59.965+0000 7f1770a00700 1 -- 172.21.15.19:0/1669023609 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f175c0aa190 con 0x7f176c10ab10 2024-05-07T02:31:00.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:00.252+0000 7f1771e00700 1 -- 172.21.15.19:0/1669023609 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f176c02e5f0 con 0x7f176c10ab10 2024-05-07T02:31:00.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:00.253+0000 7f1770a00700 1 -- 172.21.15.19:0/1669023609 <== mon.2 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) ==== 76+0+3086 (secure 0 0 0) 0x7f175c0a5030 con 0x7f176c10ab10 2024-05-07T02:31:00.254 INFO:teuthology.orchestra.run.smithi019.stdout:e30 2024-05-07T02:31:00.254 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-07T02:31:00.254 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-07T02:31:00.254 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-05-07T02:31:00.254 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:31:00.254 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-05-07T02:31:00.254 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-05-07T02:31:00.254 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 30 2024-05-07T02:31:00.254 INFO:teuthology.orchestra.run.smithi019.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-07T02:31:00.254 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-05-07T02:26:14.960465+0000 2024-05-07T02:31:00.255 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-05-07T02:27:48.914804+0000 2024-05-07T02:31:00.255 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-05-07T02:31:00.255 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-05-07T02:31:00.255 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 300 2024-05-07T02:31:00.255 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-05-07T02:31:00.255 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-05-07T02:31:00.255 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-05-07T02:31:00.255 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-05-07T02:31:00.255 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-05-07T02:31:00.255 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 0 2024-05-07T02:31:00.255 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-07T02:31:00.255 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 5 2024-05-07T02:31:00.256 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1,2,3,4 2024-05-07T02:31:00.256 INFO:teuthology.orchestra.run.smithi019.stdout:up {0=24551,1=24539,2=24520,3=24515,4=24505} 2024-05-07T02:31:00.256 INFO:teuthology.orchestra.run.smithi019.stdout:failed 2024-05-07T02:31:00.256 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-05-07T02:31:00.256 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-05-07T02:31:00.256 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3,4] 2024-05-07T02:31:00.256 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-05-07T02:31:00.256 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-05-07T02:31:00.256 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-05-07T02:31:00.256 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-05-07T02:31:00.256 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 0 2024-05-07T02:31:00.256 INFO:teuthology.orchestra.run.smithi019.stdout:qdb_cluster leader: 24551 members: 24551,24539,24520,24505,24515 2024-05-07T02:31:00.256 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.b{0:24551} state up:active seq 7 addr [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] compat {c=[1],r=[1],i=[1fff]}] 2024-05-07T02:31:00.256 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.f{0:24499} state up:standby-replay seq 1 addr [v2:172.21.15.64:6834/1997318872,v1:172.21.15.64:6835/1997318872] compat {c=[1],r=[1],i=[1fff]}] 2024-05-07T02:31:00.257 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.a{1:24539} state up:active seq 29 addr [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] compat {c=[1],r=[1],i=[1fff]}] 2024-05-07T02:31:00.257 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.e{1:24488} state up:standby-replay seq 1 addr [v2:172.21.15.27:6834/1000540600,v1:172.21.15.27:6835/1000540600] compat {c=[1],r=[1],i=[1fff]}] 2024-05-07T02:31:00.257 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.i{2:24520} state up:active seq 32 addr [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] compat {c=[1],r=[1],i=[1fff]}] 2024-05-07T02:31:00.257 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.d{2:24481} state up:standby-replay seq 1 addr [v2:172.21.15.19:6836/2067515538,v1:172.21.15.19:6837/2067515538] compat {c=[1],r=[1],i=[1fff]}] 2024-05-07T02:31:00.257 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.h{3:24515} state up:active seq 33 addr [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] compat {c=[1],r=[1],i=[1fff]}] 2024-05-07T02:31:00.257 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.j{3:24469} state up:standby-replay seq 1 addr [v2:172.21.15.19:6834/2379917839,v1:172.21.15.19:6835/2379917839] compat {c=[1],r=[1],i=[1fff]}] 2024-05-07T02:31:00.257 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.c{4:24505} state up:active seq 34 addr [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] compat {c=[1],r=[1],i=[1fff]}] 2024-05-07T02:31:00.257 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.k{4:24458} state up:standby-replay seq 1 addr [v2:172.21.15.27:6832/2171873162,v1:172.21.15.27:6833/2171873162] compat {c=[1],r=[1],i=[1fff]}] 2024-05-07T02:31:00.257 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:31:00.257 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:31:00.257 INFO:teuthology.orchestra.run.smithi019.stdout:Standby daemons: 2024-05-07T02:31:00.257 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:31:00.257 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.g{-1:14406} state up:standby seq 1 addr [v2:172.21.15.19:6838/1763670446,v1:172.21.15.19:6839/1763670446] compat {c=[1],r=[1],i=[1fff]}] 2024-05-07T02:31:00.257 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.l{-1:24454} state up:standby seq 1 addr [v2:172.21.15.64:6832/477529985,v1:172.21.15.64:6833/477529985] compat {c=[1],r=[1],i=[1fff]}] 2024-05-07T02:31:00.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:00.257+0000 7f1771e00700 1 -- 172.21.15.19:0/1669023609 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f17500b2900 msgr2=0x7f17500b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:00.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:00.257+0000 7f1771e00700 1 --2- 172.21.15.19:0/1669023609 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f17500b2900 0x7f17500b4e10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f175800b760 tx=0x7f175800b680 comp rx=0 tx=0).stop 2024-05-07T02:31:00.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:00.257+0000 7f1771e00700 1 -- 172.21.15.19:0/1669023609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f176c10ab10 msgr2=0x7f176c1a0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:00.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:00.257+0000 7f1771e00700 1 --2- 172.21.15.19:0/1669023609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f176c10ab10 0x7f176c1a0ee0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f175c00e940 tx=0x7f175c00ec50 comp rx=0 tx=0).stop 2024-05-07T02:31:00.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:00.257+0000 7f1771e00700 1 -- 172.21.15.19:0/1669023609 shutdown_connections 2024-05-07T02:31:00.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:00.257+0000 7f1771e00700 1 --2- 172.21.15.19:0/1669023609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f176c10c420 0x7f176c1a83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:00.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:00.257+0000 7f1771e00700 1 --2- 172.21.15.19:0/1669023609 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f17500b2900 0x7f17500b4e10 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:00.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:00.257+0000 7f1771e00700 1 --2- 172.21.15.19:0/1669023609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f176c10ab10 0x7f176c1a0ee0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:00.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:00.257+0000 7f1771e00700 1 --2- 172.21.15.19:0/1669023609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f176c10ba80 0x7f176c1a1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:00.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:00.257+0000 7f1771e00700 1 -- 172.21.15.19:0/1669023609 >> 172.21.15.19:0/1669023609 conn(0x7f176c0799c0 msgr2=0x7f176c10ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:00.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:00.257+0000 7f1771e00700 1 -- 172.21.15.19:0/1669023609 shutdown_connections 2024-05-07T02:31:00.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:00.257+0000 7f1771e00700 1 -- 172.21.15.19:0/1669023609 wait complete. 2024-05-07T02:31:00.259 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 30 2024-05-07T02:31:01.009 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:00 smithi027 ceph-mon[34398]: pgmap v483: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:31:01.009 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:00 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/1669023609' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-07T02:31:01.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:00 smithi064 ceph-mon[34699]: pgmap v483: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:31:01.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:00 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/1669023609' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-07T02:31:01.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:00 smithi019 ceph-mon[29781]: pgmap v483: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:31:01.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:00 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/1669023609' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-07T02:31:01.252 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-07T02:31:01.730 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:31:02.991 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:02 smithi019 ceph-mon[29781]: pgmap v484: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:31:03.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:02 smithi064 ceph-mon[34699]: pgmap v484: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:31:03.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:02 smithi027 ceph-mon[34398]: pgmap v484: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:31:04.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.362+0000 7fc063e00700 1 -- 172.21.15.19:0/832895841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc064107e60 msgr2=0x7fc06410e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:04.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.362+0000 7fc063e00700 1 --2- 172.21.15.19:0/832895841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc064107e60 0x7fc06410e710 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc05c00b0a0 tx=0x7fc05c00b3b0 comp rx=0 tx=0).stop 2024-05-07T02:31:04.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.362+0000 7fc063e00700 1 -- 172.21.15.19:0/832895841 shutdown_connections 2024-05-07T02:31:04.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.362+0000 7fc063e00700 1 --2- 172.21.15.19:0/832895841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc064107e60 0x7fc06410e710 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:04.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.362+0000 7fc063e00700 1 --2- 172.21.15.19:0/832895841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc064106390 0x7fc0641067a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:04.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.362+0000 7fc063e00700 1 --2- 172.21.15.19:0/832895841 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0641074c0 0x7fc0641078d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:04.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.362+0000 7fc063e00700 1 -- 172.21.15.19:0/832895841 >> 172.21.15.19:0/832895841 conn(0x7fc064101a10 msgr2=0x7fc064103e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:04.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.363+0000 7fc063e00700 1 -- 172.21.15.19:0/832895841 shutdown_connections 2024-05-07T02:31:04.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.363+0000 7fc063e00700 1 -- 172.21.15.19:0/832895841 wait complete. 2024-05-07T02:31:04.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.364+0000 7fc063e00700 1 Processor -- start 2024-05-07T02:31:04.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.364+0000 7fc063e00700 1 -- start start 2024-05-07T02:31:04.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.364+0000 7fc063e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc064106390 0x7fc06419c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:04.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.364+0000 7fc063e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0641074c0 0x7fc06419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:04.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.364+0000 7fc063e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc064107e60 0x7fc0641a3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:04.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.364+0000 7fc063e00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc064111e50 con 0x7fc064106390 2024-05-07T02:31:04.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.364+0000 7fc063e00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc064111cd0 con 0x7fc064107e60 2024-05-07T02:31:04.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.364+0000 7fc063e00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc064111fd0 con 0x7fc0641074c0 2024-05-07T02:31:04.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.365+0000 7fc062000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0641074c0 0x7fc06419ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:04.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.365+0000 7fc062a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc064106390 0x7fc06419c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:04.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.365+0000 7fc062000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0641074c0 0x7fc06419ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:48246/0 (socket says 172.21.15.19:48246) 2024-05-07T02:31:04.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.365+0000 7fc062000700 1 -- 172.21.15.19:0/2244878071 learned_addr learned my addr 172.21.15.19:0/2244878071 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:31:04.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.365+0000 7fc063400700 1 --2- 172.21.15.19:0/2244878071 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc064107e60 0x7fc0641a3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:04.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.365+0000 7fc062000700 1 -- 172.21.15.19:0/2244878071 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc064107e60 msgr2=0x7fc0641a3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:04.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.365+0000 7fc062000700 1 --2- 172.21.15.19:0/2244878071 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc064107e60 0x7fc0641a3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:04.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.365+0000 7fc062000700 1 -- 172.21.15.19:0/2244878071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc064106390 msgr2=0x7fc06419c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:04.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.365+0000 7fc062000700 1 --2- 172.21.15.19:0/2244878071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc064106390 0x7fc06419c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:04.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.365+0000 7fc062000700 1 -- 172.21.15.19:0/2244878071 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0641a4440 con 0x7fc0641074c0 2024-05-07T02:31:04.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.366+0000 7fc062000700 1 --2- 172.21.15.19:0/2244878071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0641074c0 0x7fc06419ce20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc05000e9a0 tx=0x7fc05000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:04.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.366+0000 7fc068a00700 1 -- 172.21.15.19:0/2244878071 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05000ca10 con 0x7fc0641074c0 2024-05-07T02:31:04.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.366+0000 7fc068a00700 1 -- 172.21.15.19:0/2244878071 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc05000cbb0 con 0x7fc0641074c0 2024-05-07T02:31:04.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.366+0000 7fc063e00700 1 -- 172.21.15.19:0/2244878071 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0641a4730 con 0x7fc0641074c0 2024-05-07T02:31:04.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.366+0000 7fc068a00700 1 -- 172.21.15.19:0/2244878071 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc050018680 con 0x7fc0641074c0 2024-05-07T02:31:04.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.366+0000 7fc063e00700 1 -- 172.21.15.19:0/2244878071 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0641a4cc0 con 0x7fc0641074c0 2024-05-07T02:31:04.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.367+0000 7fc063e00700 1 -- 172.21.15.19:0/2244878071 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0640512c0 con 0x7fc0641074c0 2024-05-07T02:31:04.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.370+0000 7fc068a00700 1 -- 172.21.15.19:0/2244878071 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc050010430 con 0x7fc0641074c0 2024-05-07T02:31:04.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.371+0000 7fc068a00700 1 --2- 172.21.15.19:0/2244878071 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fc0480b2900 0x7fc0480b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:04.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.372+0000 7fc068a00700 1 -- 172.21.15.19:0/2244878071 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fc0500e2a50 con 0x7fc0641074c0 2024-05-07T02:31:04.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.372+0000 7fc062a00700 1 --2- 172.21.15.19:0/2244878071 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fc0480b2900 0x7fc0480b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:04.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.373+0000 7fc062a00700 1 --2- 172.21.15.19:0/2244878071 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fc0480b2900 0x7fc0480b4e10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc0641073c0 tx=0x7fc054009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:04.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.374+0000 7fc068a00700 1 -- 172.21.15.19:0/2244878071 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc0500aa360 con 0x7fc0641074c0 2024-05-07T02:31:04.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.609+0000 7fc063e00700 1 -- 172.21.15.19:0/2244878071 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fc06402e5f0 con 0x7fc0641074c0 2024-05-07T02:31:04.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.610+0000 7fc068a00700 1 -- 172.21.15.19:0/2244878071 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v89) ==== 56+0+5655 (secure 0 0 0) 0x7fc0500a9c60 con 0x7fc0641074c0 2024-05-07T02:31:04.611 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 89 2024-05-07T02:31:04.612 INFO:teuthology.orchestra.run.smithi019.stdout:fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 2024-05-07T02:31:04.612 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-05-07T02:16:19.507657+0000 2024-05-07T02:31:04.612 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-05-07T02:28:33.595951+0000 2024-05-07T02:31:04.612 INFO:teuthology.orchestra.run.smithi019.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-07T02:31:04.612 INFO:teuthology.orchestra.run.smithi019.stdout:crush_version 27 2024-05-07T02:31:04.612 INFO:teuthology.orchestra.run.smithi019.stdout:full_ratio 0.95 2024-05-07T02:31:04.612 INFO:teuthology.orchestra.run.smithi019.stdout:backfillfull_ratio 0.9 2024-05-07T02:31:04.612 INFO:teuthology.orchestra.run.smithi019.stdout:nearfull_ratio 0.85 2024-05-07T02:31:04.612 INFO:teuthology.orchestra.run.smithi019.stdout:require_min_compat_client luminous 2024-05-07T02:31:04.612 INFO:teuthology.orchestra.run.smithi019.stdout:min_compat_client luminous 2024-05-07T02:31:04.612 INFO:teuthology.orchestra.run.smithi019.stdout:require_osd_release squid 2024-05-07T02:31:04.612 INFO:teuthology.orchestra.run.smithi019.stdout:stretch_mode_enabled false 2024-05-07T02:31:04.612 INFO:teuthology.orchestra.run.smithi019.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-07T02:31:04.612 INFO:teuthology.orchestra.run.smithi019.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-05-07T02:31:04.613 INFO:teuthology.orchestra.run.smithi019.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-05-07T02:31:04.613 INFO:teuthology.orchestra.run.smithi019.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-05-07T02:31:04.613 INFO:teuthology.orchestra.run.smithi019.stdout:max_osd 12 2024-05-07T02:31:04.613 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 up in weight 1 up_from 8 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6802/2390539683,v1:172.21.15.19:6803/2390539683] [v2:172.21.15.19:6804/2390539683,v1:172.21.15.19:6805/2390539683] exists,up b437cf22-ec8e-48f0-86a2-2413a0ab40dd 2024-05-07T02:31:04.613 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 up in weight 1 up_from 13 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6800/4292470615,v1:172.21.15.27:6801/4292470615] [v2:172.21.15.27:6802/4292470615,v1:172.21.15.27:6803/4292470615] exists,up dc753e16-634d-4c14-abd2-935281e1499e 2024-05-07T02:31:04.613 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 up in weight 1 up_from 18 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6800/1180552939,v1:172.21.15.64:6801/1180552939] [v2:172.21.15.64:6802/1180552939,v1:172.21.15.64:6803/1180552939] exists,up 40570071-646e-4e66-9a16-c55ab743ac86 2024-05-07T02:31:04.613 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 up in weight 1 up_from 25 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6810/387118141,v1:172.21.15.19:6811/387118141] [v2:172.21.15.19:6812/387118141,v1:172.21.15.19:6813/387118141] exists,up ceaa4252-4dfa-427a-9b1a-76adf3b8e609 2024-05-07T02:31:04.613 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 up in weight 1 up_from 31 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6808/3662186068,v1:172.21.15.27:6809/3662186068] [v2:172.21.15.27:6810/3662186068,v1:172.21.15.27:6811/3662186068] exists,up 5a6d1ebc-62ca-444e-8f29-1980418d0a8b 2024-05-07T02:31:04.613 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 up in weight 1 up_from 36 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6808/1264591849,v1:172.21.15.64:6809/1264591849] [v2:172.21.15.64:6810/1264591849,v1:172.21.15.64:6811/1264591849] exists,up 60c207e9-55e8-4bed-87c3-f6a06a2a1fd1 2024-05-07T02:31:04.613 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6 up in weight 1 up_from 42 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6818/2158875543,v1:172.21.15.19:6819/2158875543] [v2:172.21.15.19:6820/2158875543,v1:172.21.15.19:6821/2158875543] exists,up f3e038e0-4026-45f1-a923-c07cbb307f64 2024-05-07T02:31:04.613 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7 up in weight 1 up_from 47 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6816/2756612091,v1:172.21.15.27:6817/2756612091] [v2:172.21.15.27:6818/2756612091,v1:172.21.15.27:6819/2756612091] exists,up d69e34d4-2028-43b8-8ada-ff4a9b579620 2024-05-07T02:31:04.613 INFO:teuthology.orchestra.run.smithi019.stdout:osd.8 up in weight 1 up_from 53 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6816/1987024836,v1:172.21.15.64:6817/1987024836] [v2:172.21.15.64:6818/1987024836,v1:172.21.15.64:6819/1987024836] exists,up 6c1a4179-75b6-42ff-ac29-378801e6f8f0 2024-05-07T02:31:04.613 INFO:teuthology.orchestra.run.smithi019.stdout:osd.9 up in weight 1 up_from 59 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6826/4124921646,v1:172.21.15.19:6827/4124921646] [v2:172.21.15.19:6828/4124921646,v1:172.21.15.19:6829/4124921646] exists,up 9df7ff66-d39f-4709-8324-74cf9e0e052d 2024-05-07T02:31:04.613 INFO:teuthology.orchestra.run.smithi019.stdout:osd.10 up in weight 1 up_from 64 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.27:6824/1428931817,v1:172.21.15.27:6825/1428931817] [v2:172.21.15.27:6826/1428931817,v1:172.21.15.27:6827/1428931817] exists,up 71eaa9a7-667d-4cb7-935c-42177d20a279 2024-05-07T02:31:04.613 INFO:teuthology.orchestra.run.smithi019.stdout:osd.11 up in weight 1 up_from 70 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6824/4045070679,v1:172.21.15.64:6825/4045070679] [v2:172.21.15.64:6826/4045070679,v1:172.21.15.64:6827/4045070679] exists,up 3330627f-ab5d-4e6e-83cf-fc8b74c54ee3 2024-05-07T02:31:04.614 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 2.e [2,4] 2024-05-07T02:31:04.614 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 2.10 [8,11] 2024-05-07T02:31:04.614 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 2.16 [2,4] 2024-05-07T02:31:04.614 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 2.1d [8,11] 2024-05-07T02:31:04.614 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.0 [5,9] 2024-05-07T02:31:04.614 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.17 [4,8] 2024-05-07T02:31:04.614 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.1b [4,6] 2024-05-07T02:31:04.614 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.1e [3,9] 2024-05-07T02:31:04.614 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.2c [4,9] 2024-05-07T02:31:04.614 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 4.2 [9,1] 2024-05-07T02:31:04.614 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 4.f [9,8] 2024-05-07T02:31:04.614 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 4.12 [5,10] 2024-05-07T02:31:04.614 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 4.1c [9,8] 2024-05-07T02:31:04.614 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/1956516258 expires 2024-05-08T02:17:31.252868+0000 2024-05-07T02:31:04.614 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/1816772181 expires 2024-05-08T02:17:31.252868+0000 2024-05-07T02:31:04.615 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:6801/174304059 expires 2024-05-08T02:17:31.252868+0000 2024-05-07T02:31:04.615 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/1514051750 expires 2024-05-08T02:16:54.732592+0000 2024-05-07T02:31:04.615 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:6800/174304059 expires 2024-05-08T02:17:31.252868+0000 2024-05-07T02:31:04.615 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/3930045871 expires 2024-05-08T02:16:54.732592+0000 2024-05-07T02:31:04.615 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:6800/3614957309 expires 2024-05-08T02:16:54.732592+0000 2024-05-07T02:31:04.615 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/1328365399 expires 2024-05-08T02:17:31.252868+0000 2024-05-07T02:31:04.615 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/139593344 expires 2024-05-08T02:16:54.732592+0000 2024-05-07T02:31:04.615 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:6801/3614957309 expires 2024-05-08T02:16:54.732592+0000 2024-05-07T02:31:04.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.614+0000 7fc063e00700 1 -- 172.21.15.19:0/2244878071 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fc0480b2900 msgr2=0x7fc0480b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:04.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.614+0000 7fc063e00700 1 --2- 172.21.15.19:0/2244878071 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fc0480b2900 0x7fc0480b4e10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc0641073c0 tx=0x7fc054009260 comp rx=0 tx=0).stop 2024-05-07T02:31:04.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.614+0000 7fc063e00700 1 -- 172.21.15.19:0/2244878071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0641074c0 msgr2=0x7fc06419ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:04.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.614+0000 7fc063e00700 1 --2- 172.21.15.19:0/2244878071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0641074c0 0x7fc06419ce20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc05000e9a0 tx=0x7fc05000ecb0 comp rx=0 tx=0).stop 2024-05-07T02:31:04.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.614+0000 7fc063e00700 1 -- 172.21.15.19:0/2244878071 shutdown_connections 2024-05-07T02:31:04.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.614+0000 7fc063e00700 1 --2- 172.21.15.19:0/2244878071 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fc0480b2900 0x7fc0480b4e10 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:04.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.614+0000 7fc063e00700 1 --2- 172.21.15.19:0/2244878071 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc064107e60 0x7fc0641a3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:04.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.614+0000 7fc063e00700 1 --2- 172.21.15.19:0/2244878071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc064106390 0x7fc06419c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:04.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.614+0000 7fc063e00700 1 --2- 172.21.15.19:0/2244878071 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc0641074c0 0x7fc06419ce20 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:04.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.614+0000 7fc063e00700 1 -- 172.21.15.19:0/2244878071 >> 172.21.15.19:0/2244878071 conn(0x7fc064101a10 msgr2=0x7fc06410a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:04.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.614+0000 7fc063e00700 1 -- 172.21.15.19:0/2244878071 shutdown_connections 2024-05-07T02:31:04.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:04.614+0000 7fc063e00700 1 -- 172.21.15.19:0/2244878071 wait complete. 2024-05-07T02:31:04.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:04 smithi019 ceph-mon[29781]: pgmap v485: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:31:04.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:04 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/2244878071' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-07T02:31:05.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:04 smithi064 ceph-mon[34699]: pgmap v485: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:31:05.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:04 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/2244878071' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-07T02:31:05.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:04 smithi027 ceph-mon[34398]: pgmap v485: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:31:05.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:04 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/2244878071' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-07T02:31:05.618 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-07T02:31:05.630 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-07T02:31:06.097 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:31:06.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:06 smithi019 ceph-mon[29781]: pgmap v486: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:31:07.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:06 smithi064 ceph-mon[34699]: pgmap v486: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:31:07.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:06 smithi027 ceph-mon[34398]: pgmap v486: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:31:08.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:08.715+0000 7fea56a00700 1 -- 172.21.15.19:0/3409568199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea50108740 msgr2=0x7fea501100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:08.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:08.715+0000 7fea56a00700 1 --2- 172.21.15.19:0/3409568199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea50108740 0x7fea501100a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fea4800b0d0 tx=0x7fea4800b3e0 comp rx=0 tx=0).stop 2024-05-07T02:31:08.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:08.715+0000 7fea56a00700 1 -- 172.21.15.19:0/3409568199 shutdown_connections 2024-05-07T02:31:08.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:08.715+0000 7fea56a00700 1 --2- 172.21.15.19:0/3409568199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea50108740 0x7fea501100a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:08.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:08.715+0000 7fea56a00700 1 --2- 172.21.15.19:0/3409568199 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea501034c0 0x7fea501038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:08.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:08.715+0000 7fea56a00700 1 --2- 172.21.15.19:0/3409568199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea50107df0 0x7fea50108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:08.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:08.715+0000 7fea56a00700 1 -- 172.21.15.19:0/3409568199 >> 172.21.15.19:0/3409568199 conn(0x7fea500fedc0 msgr2=0x7fea501011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:08.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:08.715+0000 7fea56a00700 1 -- 172.21.15.19:0/3409568199 shutdown_connections 2024-05-07T02:31:08.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:08.715+0000 7fea56a00700 1 -- 172.21.15.19:0/3409568199 wait complete. 2024-05-07T02:31:08.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:08.716+0000 7fea56a00700 1 Processor -- start 2024-05-07T02:31:08.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:08.716+0000 7fea56a00700 1 -- start start 2024-05-07T02:31:08.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:08.717+0000 7fea56a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea501034c0 0x7fea5019adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:08.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:08.717+0000 7fea56a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea50107df0 0x7fea5019b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:08.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:08.717+0000 7fea56a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea50108740 0x7fea5019f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:08.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:08.717+0000 7fea56a00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fea5006c3f0 con 0x7fea50107df0 2024-05-07T02:31:08.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:08.717+0000 7fea56a00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fea5006c270 con 0x7fea501034c0 2024-05-07T02:31:08.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:08.717+0000 7fea56a00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fea5006c570 con 0x7fea50108740 2024-05-07T02:31:08.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:08.717+0000 7fea47e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea50107df0 0x7fea5019b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:08.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:08.717+0000 7fea4fe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea501034c0 0x7fea5019adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:08.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:08.717+0000 7fea47e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea50107df0 0x7fea5019b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48750/0 (socket says 172.21.15.19:48750) 2024-05-07T02:31:08.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:08.717+0000 7fea4fe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea501034c0 0x7fea5019adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:48602/0 (socket says 172.21.15.19:48602) 2024-05-07T02:31:08.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:08.717+0000 7fea47e00700 1 -- 172.21.15.19:0/1135831763 learned_addr learned my addr 172.21.15.19:0/1135831763 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:31:08.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:08.717+0000 7fea54a00700 1 --2- 172.21.15.19:0/1135831763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea50108740 0x7fea5019f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:08.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:08.718+0000 7fea47e00700 1 -- 172.21.15.19:0/1135831763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea50108740 msgr2=0x7fea5019f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:08.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:08.718+0000 7fea47e00700 1 --2- 172.21.15.19:0/1135831763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea50108740 0x7fea5019f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:08.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:08.718+0000 7fea47e00700 1 -- 172.21.15.19:0/1135831763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea501034c0 msgr2=0x7fea5019adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:08.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:08.718+0000 7fea47e00700 1 --2- 172.21.15.19:0/1135831763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea501034c0 0x7fea5019adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:08.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:08.718+0000 7fea47e00700 1 -- 172.21.15.19:0/1135831763 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea5019fcc0 con 0x7fea50107df0 2024-05-07T02:31:08.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:08.718+0000 7fea4fe00700 1 --2- 172.21.15.19:0/1135831763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea501034c0 0x7fea5019adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:31:08.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:08.718+0000 7fea47e00700 1 --2- 172.21.15.19:0/1135831763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea50107df0 0x7fea5019b390 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fea3800e970 tx=0x7fea3800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:08.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:08.718+0000 7fea4d600700 1 -- 172.21.15.19:0/1135831763 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea38004a30 con 0x7fea50107df0 2024-05-07T02:31:08.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:08.719+0000 7fea4d600700 1 -- 172.21.15.19:0/1135831763 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea3800cc90 con 0x7fea50107df0 2024-05-07T02:31:08.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:08.719+0000 7fea4d600700 1 -- 172.21.15.19:0/1135831763 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea38005610 con 0x7fea50107df0 2024-05-07T02:31:08.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:08.719+0000 7fea56a00700 1 -- 172.21.15.19:0/1135831763 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea5019ffb0 con 0x7fea50107df0 2024-05-07T02:31:08.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:08.719+0000 7fea56a00700 1 -- 172.21.15.19:0/1135831763 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea501a78d0 con 0x7fea50107df0 2024-05-07T02:31:08.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:08.720+0000 7fea56a00700 1 -- 172.21.15.19:0/1135831763 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea500512c0 con 0x7fea50107df0 2024-05-07T02:31:08.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:08.723+0000 7fea4d600700 1 -- 172.21.15.19:0/1135831763 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fea3800c7e0 con 0x7fea50107df0 2024-05-07T02:31:08.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:08.724+0000 7fea4d600700 1 --2- 172.21.15.19:0/1135831763 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fea340b2900 0x7fea340b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:08.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:08.724+0000 7fea4d600700 1 -- 172.21.15.19:0/1135831763 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fea380e28e0 con 0x7fea50107df0 2024-05-07T02:31:08.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:08.725+0000 7fea4fe00700 1 --2- 172.21.15.19:0/1135831763 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fea340b2900 0x7fea340b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:08.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:08.726+0000 7fea4fe00700 1 --2- 172.21.15.19:0/1135831763 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fea340b2900 0x7fea340b4e10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fea40011490 tx=0x7fea40009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:08.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:08.727+0000 7fea4d600700 1 -- 172.21.15.19:0/1135831763 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fea38054df0 con 0x7fea50107df0 2024-05-07T02:31:08.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:08 smithi019 ceph-mon[29781]: pgmap v487: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:31:09.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:09.017+0000 7fea56a00700 1 -- 172.21.15.19:0/1135831763 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea5002e5f0 con 0x7fea50107df0 2024-05-07T02:31:09.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:09.019+0000 7fea4d600700 1 -- 172.21.15.19:0/1135831763 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11924 (secure 0 0 0) 0x7fea380aa1f0 con 0x7fea50107df0 2024-05-07T02:31:09.019 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:31:09.019 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1763670446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1763670446},{"type":"v1","addr":"172.21.15.19:6839","nonce":1763670446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/477529985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":477529985},{"type":"v1","addr":"172.21.15.64:6833","nonce":477529985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-07T02:26:14.960465+0000","modified":"2024-05-07T02:27:48.914804+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24539,"mds_2":24520,"mds_3":24515,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6833/2171873162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2171873162},{"type":"v1","addr":"172.21.15.27:6833","nonce":2171873162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/2379917839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2379917839},{"type":"v1","addr":"172.21.15.19:6835","nonce":2379917839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6837/2067515538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2067515538},{"type":"v1","addr":"172.21.15.19:6837","nonce":2067515538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6835/1000540600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1000540600},{"type":"v1","addr":"172.21.15.27:6835","nonce":1000540600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/1997318872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1997318872},{"type":"v1","addr":"172.21.15.64:6835","nonce":1997318872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":34,"addr":"172.21.15.64:6837/2585907777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2585907777},{"type":"v1","addr":"172.21.15.64:6837","nonce":2585907777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":33,"addr":"172.21.15.27:6837/2583138143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2583138143},{"type":"v1","addr":"172.21.15.27:6837","nonce":2583138143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":32,"addr":"172.21.15.64:6839/1583120004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":1583120004},{"type":"v1","addr":"172.21.15.64:6839","nonce":1583120004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":29,"addr":"172.21.15.19:6841/3905645246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3905645246},{"type":"v1","addr":"172.21.15.19:6841","nonce":3905645246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6839/3963268634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3963268634},{"type":"v1","addr":"172.21.15.27:6839","nonce":3963268634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24551,24539,24520,24505,24515]},"id":1}]} 2024-05-07T02:31:09.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:09.021+0000 7fea56a00700 1 -- 172.21.15.19:0/1135831763 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fea340b2900 msgr2=0x7fea340b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:09.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:09.021+0000 7fea56a00700 1 --2- 172.21.15.19:0/1135831763 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fea340b2900 0x7fea340b4e10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fea40011490 tx=0x7fea40009260 comp rx=0 tx=0).stop 2024-05-07T02:31:09.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:09.022+0000 7fea56a00700 1 -- 172.21.15.19:0/1135831763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea50107df0 msgr2=0x7fea5019b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:09.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:09.022+0000 7fea56a00700 1 --2- 172.21.15.19:0/1135831763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea50107df0 0x7fea5019b390 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fea3800e970 tx=0x7fea3800ec80 comp rx=0 tx=0).stop 2024-05-07T02:31:09.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:09.022+0000 7fea56a00700 1 -- 172.21.15.19:0/1135831763 shutdown_connections 2024-05-07T02:31:09.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:09.022+0000 7fea56a00700 1 --2- 172.21.15.19:0/1135831763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea50108740 0x7fea5019f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:09.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:09.022+0000 7fea56a00700 1 --2- 172.21.15.19:0/1135831763 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fea340b2900 0x7fea340b4e10 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:09.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:09.022+0000 7fea56a00700 1 --2- 172.21.15.19:0/1135831763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea501034c0 0x7fea5019adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:09.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:09.022+0000 7fea56a00700 1 --2- 172.21.15.19:0/1135831763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea50107df0 0x7fea5019b390 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:09.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:09.022+0000 7fea56a00700 1 -- 172.21.15.19:0/1135831763 >> 172.21.15.19:0/1135831763 conn(0x7fea500fedc0 msgr2=0x7fea50107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:09.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:09.022+0000 7fea56a00700 1 -- 172.21.15.19:0/1135831763 shutdown_connections 2024-05-07T02:31:09.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:09.022+0000 7fea56a00700 1 -- 172.21.15.19:0/1135831763 wait complete. 2024-05-07T02:31:09.023 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 30 2024-05-07T02:31:09.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:08 smithi064 ceph-mon[34699]: pgmap v487: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:31:09.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:08 smithi027 ceph-mon[34398]: pgmap v487: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:31:10.022 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:09 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/1135831763' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:31:10.027 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-07T02:31:10.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:09 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/1135831763' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:31:10.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:09 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/1135831763' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:31:10.501 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:31:10.909 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:10 smithi019 ceph-mon[29781]: pgmap v488: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:31:11.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:10 smithi064 ceph-mon[34699]: pgmap v488: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:31:11.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:10 smithi027 ceph-mon[34398]: pgmap v488: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:31:12.947 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:12 smithi019 ceph-mon[29781]: pgmap v489: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:31:13.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.083+0000 7fcbb8400700 1 -- 172.21.15.19:0/2728965066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbb0110970 msgr2=0x7fcbb0110d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:13.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.083+0000 7fcbb8400700 1 --2- 172.21.15.19:0/2728965066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbb0110970 0x7fcbb0110d80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcba40098f0 tx=0x7fcba4009c00 comp rx=0 tx=0).stop 2024-05-07T02:31:13.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.083+0000 7fcbb8400700 1 -- 172.21.15.19:0/2728965066 shutdown_connections 2024-05-07T02:31:13.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.083+0000 7fcbb8400700 1 --2- 172.21.15.19:0/2728965066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbb010b2e0 0x7fcbb0117810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:13.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.083+0000 7fcbb8400700 1 --2- 172.21.15.19:0/2728965066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbb0110970 0x7fcbb0110d80 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:13.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.083+0000 7fcbb8400700 1 --2- 172.21.15.19:0/2728965066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbb010a990 0x7fcbb010ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:13.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.083+0000 7fcbb8400700 1 -- 172.21.15.19:0/2728965066 >> 172.21.15.19:0/2728965066 conn(0x7fcbb0077c80 msgr2=0x7fcbb00780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:13.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.084+0000 7fcbb8400700 1 -- 172.21.15.19:0/2728965066 shutdown_connections 2024-05-07T02:31:13.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.084+0000 7fcbb8400700 1 -- 172.21.15.19:0/2728965066 wait complete. 2024-05-07T02:31:13.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.085+0000 7fcbb8400700 1 Processor -- start 2024-05-07T02:31:13.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.085+0000 7fcbb8400700 1 -- start start 2024-05-07T02:31:13.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.085+0000 7fcbb8400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbb010a990 0x7fcbb01a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:13.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.085+0000 7fcbb8400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbb010b2e0 0x7fcbb01a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:13.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.085+0000 7fcbb8400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbb0110970 0x7fcbb01a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:13.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.086+0000 7fcbb8400700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb011aa70 con 0x7fcbb0110970 2024-05-07T02:31:13.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.086+0000 7fcbb8400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb011a8f0 con 0x7fcbb010a990 2024-05-07T02:31:13.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.086+0000 7fcbb8400700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb011abf0 con 0x7fcbb010b2e0 2024-05-07T02:31:13.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.086+0000 7fcbb6400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbb0110970 0x7fcbb01a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:13.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.086+0000 7fcbb5000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbb010b2e0 0x7fcbb01a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:13.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.086+0000 7fcbb6400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbb0110970 0x7fcbb01a5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41188/0 (socket says 172.21.15.19:41188) 2024-05-07T02:31:13.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.086+0000 7fcbb6400700 1 -- 172.21.15.19:0/2610544454 learned_addr learned my addr 172.21.15.19:0/2610544454 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:31:13.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.086+0000 7fcbb5a00700 1 --2- 172.21.15.19:0/2610544454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbb010a990 0x7fcbb01a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:13.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.086+0000 7fcbb6400700 1 -- 172.21.15.19:0/2610544454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbb010b2e0 msgr2=0x7fcbb01a1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:13.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.086+0000 7fcbb6400700 1 --2- 172.21.15.19:0/2610544454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbb010b2e0 0x7fcbb01a1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:13.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.086+0000 7fcbb6400700 1 -- 172.21.15.19:0/2610544454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbb010a990 msgr2=0x7fcbb01a1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:13.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.086+0000 7fcbb6400700 1 --2- 172.21.15.19:0/2610544454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbb010a990 0x7fcbb01a1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:13.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.086+0000 7fcbb6400700 1 -- 172.21.15.19:0/2610544454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb01a6330 con 0x7fcbb0110970 2024-05-07T02:31:13.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.087+0000 7fcbb6400700 1 --2- 172.21.15.19:0/2610544454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbb0110970 0x7fcbb01a5c90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcba800c370 tx=0x7fcba800c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:13.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.087+0000 7fcbae000700 1 -- 172.21.15.19:0/2610544454 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba800ce20 con 0x7fcbb0110970 2024-05-07T02:31:13.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.087+0000 7fcbb8400700 1 -- 172.21.15.19:0/2610544454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbb01a65c0 con 0x7fcbb0110970 2024-05-07T02:31:13.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.087+0000 7fcbae000700 1 -- 172.21.15.19:0/2610544454 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcba80090d0 con 0x7fcbb0110970 2024-05-07T02:31:13.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.087+0000 7fcbae000700 1 -- 172.21.15.19:0/2610544454 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba80046b0 con 0x7fcbb0110970 2024-05-07T02:31:13.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.087+0000 7fcbb8400700 1 -- 172.21.15.19:0/2610544454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbb01adf40 con 0x7fcbb0110970 2024-05-07T02:31:13.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.088+0000 7fcbb8400700 1 -- 172.21.15.19:0/2610544454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbb010c080 con 0x7fcbb0110970 2024-05-07T02:31:13.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.091+0000 7fcbae000700 1 -- 172.21.15.19:0/2610544454 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fcba801a020 con 0x7fcbb0110970 2024-05-07T02:31:13.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.093+0000 7fcbae000700 1 --2- 172.21.15.19:0/2610544454 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fcb980b2900 0x7fcb980b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:13.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.093+0000 7fcbae000700 1 -- 172.21.15.19:0/2610544454 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fcba80e2470 con 0x7fcbb0110970 2024-05-07T02:31:13.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.093+0000 7fcbb5a00700 1 --2- 172.21.15.19:0/2610544454 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fcb980b2900 0x7fcb980b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:13.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.094+0000 7fcbb5a00700 1 --2- 172.21.15.19:0/2610544454 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fcb980b2900 0x7fcb980b4e10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fcba4009810 tx=0x7fcba4019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:13.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.096+0000 7fcbae000700 1 -- 172.21.15.19:0/2610544454 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcba8010040 con 0x7fcbb0110970 2024-05-07T02:31:13.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:12 smithi064 ceph-mon[34699]: pgmap v489: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:31:13.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:12 smithi027 ceph-mon[34398]: pgmap v489: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:31:13.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.379+0000 7fcbb8400700 1 -- 172.21.15.19:0/2610544454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcbb002e5f0 con 0x7fcbb0110970 2024-05-07T02:31:13.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.381+0000 7fcbae000700 1 -- 172.21.15.19:0/2610544454 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11924 (secure 0 0 0) 0x7fcba80a9d80 con 0x7fcbb0110970 2024-05-07T02:31:13.381 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:31:13.381 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1763670446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1763670446},{"type":"v1","addr":"172.21.15.19:6839","nonce":1763670446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/477529985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":477529985},{"type":"v1","addr":"172.21.15.64:6833","nonce":477529985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-07T02:26:14.960465+0000","modified":"2024-05-07T02:27:48.914804+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24539,"mds_2":24520,"mds_3":24515,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6833/2171873162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2171873162},{"type":"v1","addr":"172.21.15.27:6833","nonce":2171873162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/2379917839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2379917839},{"type":"v1","addr":"172.21.15.19:6835","nonce":2379917839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6837/2067515538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2067515538},{"type":"v1","addr":"172.21.15.19:6837","nonce":2067515538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6835/1000540600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1000540600},{"type":"v1","addr":"172.21.15.27:6835","nonce":1000540600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/1997318872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1997318872},{"type":"v1","addr":"172.21.15.64:6835","nonce":1997318872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":34,"addr":"172.21.15.64:6837/2585907777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2585907777},{"type":"v1","addr":"172.21.15.64:6837","nonce":2585907777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":33,"addr":"172.21.15.27:6837/2583138143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2583138143},{"type":"v1","addr":"172.21.15.27:6837","nonce":2583138143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":32,"addr":"172.21.15.64:6839/1583120004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":1583120004},{"type":"v1","addr":"172.21.15.64:6839","nonce":1583120004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":29,"addr":"172.21.15.19:6841/3905645246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3905645246},{"type":"v1","addr":"172.21.15.19:6841","nonce":3905645246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6839/3963268634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3963268634},{"type":"v1","addr":"172.21.15.27:6839","nonce":3963268634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24551,24539,24520,24505,24515]},"id":1}]} 2024-05-07T02:31:13.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.384+0000 7fcbb8400700 1 -- 172.21.15.19:0/2610544454 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fcb980b2900 msgr2=0x7fcb980b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:13.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.384+0000 7fcbb8400700 1 --2- 172.21.15.19:0/2610544454 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fcb980b2900 0x7fcb980b4e10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fcba4009810 tx=0x7fcba4019040 comp rx=0 tx=0).stop 2024-05-07T02:31:13.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.384+0000 7fcbb8400700 1 -- 172.21.15.19:0/2610544454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbb0110970 msgr2=0x7fcbb01a5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:13.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.384+0000 7fcbb8400700 1 --2- 172.21.15.19:0/2610544454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbb0110970 0x7fcbb01a5c90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcba800c370 tx=0x7fcba800c680 comp rx=0 tx=0).stop 2024-05-07T02:31:13.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.384+0000 7fcbb8400700 1 -- 172.21.15.19:0/2610544454 shutdown_connections 2024-05-07T02:31:13.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.384+0000 7fcbb8400700 1 --2- 172.21.15.19:0/2610544454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbb0110970 0x7fcbb01a5c90 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:13.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.384+0000 7fcbb8400700 1 --2- 172.21.15.19:0/2610544454 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fcb980b2900 0x7fcb980b4e10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:13.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.384+0000 7fcbb8400700 1 --2- 172.21.15.19:0/2610544454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbb010a990 0x7fcbb01a1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:13.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.384+0000 7fcbb8400700 1 --2- 172.21.15.19:0/2610544454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcbb010b2e0 0x7fcbb01a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:13.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.384+0000 7fcbb8400700 1 -- 172.21.15.19:0/2610544454 >> 172.21.15.19:0/2610544454 conn(0x7fcbb0077c80 msgr2=0x7fcbb007bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:13.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.384+0000 7fcbb8400700 1 -- 172.21.15.19:0/2610544454 shutdown_connections 2024-05-07T02:31:13.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:13.385+0000 7fcbb8400700 1 -- 172.21.15.19:0/2610544454 wait complete. 2024-05-07T02:31:13.386 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 30 2024-05-07T02:31:14.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:13 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/2610544454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:31:14.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:13 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/2610544454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:31:14.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:13 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/2610544454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:31:14.326 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-07T02:31:14.790 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:31:15.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:14 smithi064 ceph-mon[34699]: pgmap v490: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:31:15.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:14 smithi019 ceph-mon[29781]: pgmap v490: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:31:15.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:14 smithi027 ceph-mon[34398]: pgmap v490: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:31:16.984 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:16 smithi019 ceph-mon[29781]: pgmap v491: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:31:17.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:16 smithi064 ceph-mon[34699]: pgmap v491: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:31:17.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:16 smithi027 ceph-mon[34398]: pgmap v491: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:31:17.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.422+0000 7f8cc4c00700 1 -- 172.21.15.19:0/2477705697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc0108f20 msgr2=0x7f8cc0109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:17.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.422+0000 7f8cc4c00700 1 --2- 172.21.15.19:0/2477705697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc0108f20 0x7f8cc0109330 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8cac0099c0 tx=0x7f8cac009cd0 comp rx=0 tx=0).stop 2024-05-07T02:31:17.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.422+0000 7f8cc4c00700 1 -- 172.21.15.19:0/2477705697 shutdown_connections 2024-05-07T02:31:17.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.422+0000 7f8cc4c00700 1 --2- 172.21.15.19:0/2477705697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cc0109870 0x7f8cc01111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:17.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.422+0000 7f8cc4c00700 1 --2- 172.21.15.19:0/2477705697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cc01045f0 0x7f8cc0104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:17.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.422+0000 7f8cc4c00700 1 --2- 172.21.15.19:0/2477705697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc0108f20 0x7f8cc0109330 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:17.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.422+0000 7f8cc4c00700 1 -- 172.21.15.19:0/2477705697 >> 172.21.15.19:0/2477705697 conn(0x7f8cc0100040 msgr2=0x7f8cc0102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:17.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.422+0000 7f8cc4c00700 1 -- 172.21.15.19:0/2477705697 shutdown_connections 2024-05-07T02:31:17.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.422+0000 7f8cc4c00700 1 -- 172.21.15.19:0/2477705697 wait complete. 2024-05-07T02:31:17.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.423+0000 7f8cc4c00700 1 Processor -- start 2024-05-07T02:31:17.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.423+0000 7f8cc4c00700 1 -- start start 2024-05-07T02:31:17.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.424+0000 7f8cc4c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc01045f0 0x7f8cc019d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:17.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.424+0000 7f8cc4c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cc0108f20 0x7f8cc019d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:17.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.424+0000 7f8cc4c00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cc0109870 0x7f8cc01a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:17.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.424+0000 7f8cc4c00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc0114480 con 0x7f8cc01045f0 2024-05-07T02:31:17.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.424+0000 7f8cc4c00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc0114300 con 0x7f8cc0108f20 2024-05-07T02:31:17.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.424+0000 7f8cc4c00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc0114600 con 0x7f8cc0109870 2024-05-07T02:31:17.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.424+0000 7f8cbde00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc01045f0 0x7f8cc019d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:17.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.424+0000 7f8cbd400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cc0108f20 0x7f8cc019d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:17.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.424+0000 7f8cbd400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cc0108f20 0x7f8cc019d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:52396/0 (socket says 172.21.15.19:52396) 2024-05-07T02:31:17.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.424+0000 7f8cbd400700 1 -- 172.21.15.19:0/3047044169 learned_addr learned my addr 172.21.15.19:0/3047044169 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:31:17.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.424+0000 7f8cbe800700 1 --2- 172.21.15.19:0/3047044169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cc0109870 0x7f8cc01a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:17.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.425+0000 7f8cbd400700 1 -- 172.21.15.19:0/3047044169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cc0109870 msgr2=0x7f8cc01a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:17.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.425+0000 7f8cbd400700 1 --2- 172.21.15.19:0/3047044169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cc0109870 0x7f8cc01a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:17.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.425+0000 7f8cbd400700 1 -- 172.21.15.19:0/3047044169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc01045f0 msgr2=0x7f8cc019d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:17.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.425+0000 7f8cbd400700 1 --2- 172.21.15.19:0/3047044169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc01045f0 0x7f8cc019d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:17.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.425+0000 7f8cbd400700 1 -- 172.21.15.19:0/3047044169 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cc01a2000 con 0x7f8cc0108f20 2024-05-07T02:31:17.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.425+0000 7f8cbde00700 1 --2- 172.21.15.19:0/3047044169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc01045f0 0x7f8cc019d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:31:17.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.425+0000 7f8cbd400700 1 --2- 172.21.15.19:0/3047044169 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cc0108f20 0x7f8cc019d6d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8cac006010 tx=0x7f8cac005280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:17.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.426+0000 7f8cb6a00700 1 -- 172.21.15.19:0/3047044169 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cac026070 con 0x7f8cc0108f20 2024-05-07T02:31:17.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.426+0000 7f8cc4c00700 1 -- 172.21.15.19:0/3047044169 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cc01a2290 con 0x7f8cc0108f20 2024-05-07T02:31:17.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.426+0000 7f8cc4c00700 1 -- 172.21.15.19:0/3047044169 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cc01a9c10 con 0x7f8cc0108f20 2024-05-07T02:31:17.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.426+0000 7f8cb6a00700 1 -- 172.21.15.19:0/3047044169 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cac003c70 con 0x7f8cc0108f20 2024-05-07T02:31:17.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.426+0000 7f8cb6a00700 1 -- 172.21.15.19:0/3047044169 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cac00fb10 con 0x7f8cc0108f20 2024-05-07T02:31:17.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.427+0000 7f8cc4c00700 1 -- 172.21.15.19:0/3047044169 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ca0005330 con 0x7f8cc0108f20 2024-05-07T02:31:17.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.430+0000 7f8cb6a00700 1 -- 172.21.15.19:0/3047044169 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8cac029070 con 0x7f8cc0108f20 2024-05-07T02:31:17.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.431+0000 7f8cb6a00700 1 --2- 172.21.15.19:0/3047044169 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f8ca40b2880 0x7f8ca40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:17.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.431+0000 7f8cb6a00700 1 -- 172.21.15.19:0/3047044169 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f8cac0e3a30 con 0x7f8cc0108f20 2024-05-07T02:31:17.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.431+0000 7f8cbde00700 1 --2- 172.21.15.19:0/3047044169 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f8ca40b2880 0x7f8ca40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:17.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.432+0000 7f8cbde00700 1 --2- 172.21.15.19:0/3047044169 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f8ca40b2880 0x7f8ca40b4d90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0011490 tx=0x7f8cb0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:17.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.433+0000 7f8cb6a00700 1 -- 172.21.15.19:0/3047044169 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8cac030070 con 0x7f8cc0108f20 2024-05-07T02:31:17.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.672+0000 7f8cc4c00700 1 -- 172.21.15.19:0/3047044169 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8ca00058f0 con 0x7f8cc0108f20 2024-05-07T02:31:17.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.674+0000 7f8cb6a00700 1 -- 172.21.15.19:0/3047044169 <== mon.2 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24723 (secure 0 0 0) 0x7f8cac013070 con 0x7f8cc0108f20 2024-05-07T02:31:17.675 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:31:17.675 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":89,"fsid":"9ff84c16-0c17-11ef-bc97-c7b262605968","created":"2024-05-07T02:16:19.507657+0000","modified":"2024-05-07T02:28:33.595951+0000","last_up_change":"2024-05-07T02:24:55.025124+0000","last_in_change":"2024-05-07T02:24:33.842422+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-07T02:20:21.439697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-07T02:26:06.178814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-07T02:26:10.705693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-07T02:26:23.534133+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b437cf22-ec8e-48f0-86a2-2413a0ab40dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6803","nonce":2390539683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6805","nonce":2390539683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6809","nonce":2390539683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6807","nonce":2390539683}]},"public_addr":"172.21.15.19:6803/2390539683","cluster_addr":"172.21.15.19:6805/2390539683","heartbeat_back_addr":"172.21.15.19:6809/2390539683","heartbeat_front_addr":"172.21.15.19:6807/2390539683","state":["exists","up"]},{"osd":1,"uuid":"dc753e16-634d-4c14-abd2-935281e1499e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6801","nonce":4292470615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6803","nonce":4292470615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6807","nonce":4292470615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6805","nonce":4292470615}]},"public_addr":"172.21.15.27:6801/4292470615","cluster_addr":"172.21.15.27:6803/4292470615","heartbeat_back_addr":"172.21.15.27:6807/4292470615","heartbeat_front_addr":"172.21.15.27:6805/4292470615","state":["exists","up"]},{"osd":2,"uuid":"40570071-646e-4e66-9a16-c55ab743ac86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6801","nonce":1180552939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6803","nonce":1180552939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6807","nonce":1180552939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6805","nonce":1180552939}]},"public_addr":"172.21.15.64:6801/1180552939","cluster_addr":"172.21.15.64:6803/1180552939","heartbeat_back_addr":"172.21.15.64:6807/1180552939","heartbeat_front_addr":"172.21.15.64:6805/1180552939","state":["exists","up"]},{"osd":3,"uuid":"ceaa4252-4dfa-427a-9b1a-76adf3b8e609","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6811","nonce":387118141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6813","nonce":387118141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6817","nonce":387118141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6815","nonce":387118141}]},"public_addr":"172.21.15.19:6811/387118141","cluster_addr":"172.21.15.19:6813/387118141","heartbeat_back_addr":"172.21.15.19:6817/387118141","heartbeat_front_addr":"172.21.15.19:6815/387118141","state":["exists","up"]},{"osd":4,"uuid":"5a6d1ebc-62ca-444e-8f29-1980418d0a8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6809","nonce":3662186068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6811","nonce":3662186068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6815","nonce":3662186068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6813","nonce":3662186068}]},"public_addr":"172.21.15.27:6809/3662186068","cluster_addr":"172.21.15.27:6811/3662186068","heartbeat_back_addr":"172.21.15.27:6815/3662186068","heartbeat_front_addr":"172.21.15.27:6813/3662186068","state":["exists","up"]},{"osd":5,"uuid":"60c207e9-55e8-4bed-87c3-f6a06a2a1fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6809","nonce":1264591849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6811","nonce":1264591849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6815","nonce":1264591849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6813","nonce":1264591849}]},"public_addr":"172.21.15.64:6809/1264591849","cluster_addr":"172.21.15.64:6811/1264591849","heartbeat_back_addr":"172.21.15.64:6815/1264591849","heartbeat_front_addr":"172.21.15.64:6813/1264591849","state":["exists","up"]},{"osd":6,"uuid":"f3e038e0-4026-45f1-a923-c07cbb307f64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6819","nonce":2158875543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6821","nonce":2158875543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6825","nonce":2158875543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6823","nonce":2158875543}]},"public_addr":"172.21.15.19:6819/2158875543","cluster_addr":"172.21.15.19:6821/2158875543","heartbeat_back_addr":"172.21.15.19:6825/2158875543","heartbeat_front_addr":"172.21.15.19:6823/2158875543","state":["exists","up"]},{"osd":7,"uuid":"d69e34d4-2028-43b8-8ada-ff4a9b579620","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6817","nonce":2756612091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6819","nonce":2756612091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6823","nonce":2756612091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6821","nonce":2756612091}]},"public_addr":"172.21.15.27:6817/2756612091","cluster_addr":"172.21.15.27:6819/2756612091","heartbeat_back_addr":"172.21.15.27:6823/2756612091","heartbeat_front_addr":"172.21.15.27:6821/2756612091","state":["exists","up"]},{"osd":8,"uuid":"6c1a4179-75b6-42ff-ac29-378801e6f8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6817","nonce":1987024836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6819","nonce":1987024836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6823","nonce":1987024836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6821","nonce":1987024836}]},"public_addr":"172.21.15.64:6817/1987024836","cluster_addr":"172.21.15.64:6819/1987024836","heartbeat_back_addr":"172.21.15.64:6823/1987024836","heartbeat_front_addr":"172.21.15.64:6821/1987024836","state":["exists","up"]},{"osd":9,"uuid":"9df7ff66-d39f-4709-8324-74cf9e0e052d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6827","nonce":4124921646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6829","nonce":4124921646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6833","nonce":4124921646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6831","nonce":4124921646}]},"public_addr":"172.21.15.19:6827/4124921646","cluster_addr":"172.21.15.19:6829/4124921646","heartbeat_back_addr":"172.21.15.19:6833/4124921646","heartbeat_front_addr":"172.21.15.19:6831/4124921646","state":["exists","up"]},{"osd":10,"uuid":"71eaa9a7-667d-4cb7-935c-42177d20a279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6825","nonce":1428931817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6827","nonce":1428931817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6831","nonce":1428931817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6829","nonce":1428931817}]},"public_addr":"172.21.15.27:6825/1428931817","cluster_addr":"172.21.15.27:6827/1428931817","heartbeat_back_addr":"172.21.15.27:6831/1428931817","heartbeat_front_addr":"172.21.15.27:6829/1428931817","state":["exists","up"]},{"osd":11,"uuid":"3330627f-ab5d-4e6e-83cf-fc8b74c54ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6825","nonce":4045070679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6827","nonce":4045070679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6831","nonce":4045070679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6829","nonce":4045070679}]},"public_addr":"172.21.15.64:6825/4045070679","cluster_addr":"172.21.15.64:6827/4045070679","heartbeat_back_addr":"172.21.15.64:6831/4045070679","heartbeat_front_addr":"172.21.15.64:6829/4045070679","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:19:27.902633+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:19:54.305685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:20:18.727053+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:20:53.579199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:21:21.058018+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:21:47.091494+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:22:23.269018+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:22:51.863628+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:23:18.537774+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:23:55.763993+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:24:24.956178+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:24:52.681201+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/1956516258":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/1816772181":"2024-05-08T02:17:31.252868+0000","172.21.15.19:6801/174304059":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/1514051750":"2024-05-08T02:16:54.732592+0000","172.21.15.19:6800/174304059":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/3930045871":"2024-05-08T02:16:54.732592+0000","172.21.15.19:6800/3614957309":"2024-05-08T02:16:54.732592+0000","172.21.15.19:0/1328365399":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/139593344":"2024-05-08T02:16:54.732592+0000","172.21.15.19:6801/3614957309":"2024-05-08T02:16:54.732592+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-07T02:31:17.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.678+0000 7f8cc4c00700 1 -- 172.21.15.19:0/3047044169 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f8ca40b2880 msgr2=0x7f8ca40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:17.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.678+0000 7f8cc4c00700 1 --2- 172.21.15.19:0/3047044169 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f8ca40b2880 0x7f8ca40b4d90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0011490 tx=0x7f8cb0009260 comp rx=0 tx=0).stop 2024-05-07T02:31:17.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.678+0000 7f8cc4c00700 1 -- 172.21.15.19:0/3047044169 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cc0108f20 msgr2=0x7f8cc019d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:17.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.678+0000 7f8cc4c00700 1 --2- 172.21.15.19:0/3047044169 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cc0108f20 0x7f8cc019d6d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8cac006010 tx=0x7f8cac005280 comp rx=0 tx=0).stop 2024-05-07T02:31:17.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.678+0000 7f8cc4c00700 1 -- 172.21.15.19:0/3047044169 shutdown_connections 2024-05-07T02:31:17.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.678+0000 7f8cc4c00700 1 --2- 172.21.15.19:0/3047044169 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8cc0109870 0x7f8cc01a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:17.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.678+0000 7f8cc4c00700 1 --2- 172.21.15.19:0/3047044169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cc01045f0 0x7f8cc019d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:17.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.678+0000 7f8cc4c00700 1 --2- 172.21.15.19:0/3047044169 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f8ca40b2880 0x7f8ca40b4d90 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:17.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.678+0000 7f8cc4c00700 1 --2- 172.21.15.19:0/3047044169 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cc0108f20 0x7f8cc019d6d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:17.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.678+0000 7f8cc4c00700 1 -- 172.21.15.19:0/3047044169 >> 172.21.15.19:0/3047044169 conn(0x7f8cc0100040 msgr2=0x7f8cc010a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:17.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.678+0000 7f8cc4c00700 1 -- 172.21.15.19:0/3047044169 shutdown_connections 2024-05-07T02:31:17.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:17.678+0000 7f8cc4c00700 1 -- 172.21.15.19:0/3047044169 wait complete. 2024-05-07T02:31:18.063 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:17 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/3047044169' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-07T02:31:18.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:17 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/3047044169' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-07T02:31:18.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:17 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/3047044169' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-07T02:31:18.627 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-07T02:26:14.960465+0000', 'modified': '2024-05-07T02:27:48.914804+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24551, 'mds_1': 24539, 'mds_2': 24520, 'mds_3': 24515, 'mds_4': 24505}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24458': {'gid': 24458, 'name': 'k', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.27:6833/2171873162', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6832', 'nonce': 2171873162}, {'type': 'v1', 'addr': '172.21.15.27:6833', 'nonce': 2171873162}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'j', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.19:6835/2379917839', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6834', 'nonce': 2379917839}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 2379917839}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'd', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.19:6837/2067515538', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6836', 'nonce': 2067515538}, {'type': 'v1', 'addr': '172.21.15.19:6837', 'nonce': 2067515538}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24488': {'gid': 24488, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.27:6835/1000540600', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6834', 'nonce': 1000540600}, {'type': 'v1', 'addr': '172.21.15.27:6835', 'nonce': 1000540600}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24499': {'gid': 24499, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.64:6835/1997318872', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6834', 'nonce': 1997318872}, {'type': 'v1', 'addr': '172.21.15.64:6835', 'nonce': 1997318872}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24505': {'gid': 24505, 'name': 'c', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 34, 'addr': '172.21.15.64:6837/2585907777', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6836', 'nonce': 2585907777}, {'type': 'v1', 'addr': '172.21.15.64:6837', 'nonce': 2585907777}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24515': {'gid': 24515, 'name': 'h', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 33, 'addr': '172.21.15.27:6837/2583138143', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6836', 'nonce': 2583138143}, {'type': 'v1', 'addr': '172.21.15.27:6837', 'nonce': 2583138143}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24520': {'gid': 24520, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 32, 'addr': '172.21.15.64:6839/1583120004', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.64:6838', 'nonce': 1583120004}, {'type': 'v1', 'addr': '172.21.15.64:6839', 'nonce': 1583120004}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24539': {'gid': 24539, 'name': 'a', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 29, 'addr': '172.21.15.19:6841/3905645246', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6840', 'nonce': 3905645246}, {'type': 'v1', 'addr': '172.21.15.19:6841', 'nonce': 3905645246}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24551': {'gid': 24551, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.27:6839/3963268634', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6838', 'nonce': 3963268634}, {'type': 'v1', 'addr': '172.21.15.27:6839', 'nonce': 3963268634}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24551, 'qdb_cluster': [24551, 24539, 24520, 24505, 24515]} 2024-05-07T02:31:18.628 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-07T02:31:18.628 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.c status 2024-05-07T02:31:18.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:18 smithi019 ceph-mon[29781]: pgmap v492: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:31:19.097 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:31:19.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:18 smithi064 ceph-mon[34699]: pgmap v492: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:31:19.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:18 smithi027 ceph-mon[34398]: pgmap v492: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:31:20.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:20 smithi019 ceph-mon[29781]: pgmap v493: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:31:20.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:20 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:31:21.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:20 smithi064 ceph-mon[34699]: pgmap v493: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:31:21.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:20 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:31:21.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:20 smithi027 ceph-mon[34398]: pgmap v493: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:31:21.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:20 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:31:21.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.513+0000 7f9eb4600700 1 -- 172.21.15.19:0/4165669851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9eac1045f0 msgr2=0x7f9eac104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:21.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.513+0000 7f9eb4600700 1 --2- 172.21.15.19:0/4165669851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9eac1045f0 0x7f9eac104a00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9ea00098f0 tx=0x7f9ea0009c00 comp rx=0 tx=0).stop 2024-05-07T02:31:21.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.513+0000 7f9eb4600700 1 -- 172.21.15.19:0/4165669851 shutdown_connections 2024-05-07T02:31:21.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.513+0000 7f9eb4600700 1 --2- 172.21.15.19:0/4165669851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9eac109870 0x7f9eac1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:21.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.513+0000 7f9eb4600700 1 --2- 172.21.15.19:0/4165669851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9eac1045f0 0x7f9eac104a00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:21.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.513+0000 7f9eb4600700 1 --2- 172.21.15.19:0/4165669851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9eac108f20 0x7f9eac109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:21.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.513+0000 7f9eb4600700 1 -- 172.21.15.19:0/4165669851 >> 172.21.15.19:0/4165669851 conn(0x7f9eac100040 msgr2=0x7f9eac102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:21.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.513+0000 7f9eb4600700 1 -- 172.21.15.19:0/4165669851 shutdown_connections 2024-05-07T02:31:21.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.514+0000 7f9eb4600700 1 -- 172.21.15.19:0/4165669851 wait complete. 2024-05-07T02:31:21.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.515+0000 7f9eb4600700 1 Processor -- start 2024-05-07T02:31:21.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.515+0000 7f9eb4600700 1 -- start start 2024-05-07T02:31:21.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.515+0000 7f9eb4600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9eac1045f0 0x7f9eac075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:21.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.516+0000 7f9eb4600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9eac108f20 0x7f9eac076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:21.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.516+0000 7f9eb4600700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9eac109870 0x7f9eac076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:21.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.516+0000 7f9eb4600700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac1144f0 con 0x7f9eac108f20 2024-05-07T02:31:21.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.516+0000 7f9eb4600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac114370 con 0x7f9eac1045f0 2024-05-07T02:31:21.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.516+0000 7f9eb4600700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac114670 con 0x7f9eac109870 2024-05-07T02:31:21.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.516+0000 7f9eb1c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9eac1045f0 0x7f9eac075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:21.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.516+0000 7f9eb1c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9eac1045f0 0x7f9eac075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:49444/0 (socket says 172.21.15.19:49444) 2024-05-07T02:31:21.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.516+0000 7f9eb1c00700 1 -- 172.21.15.19:0/1837660153 learned_addr learned my addr 172.21.15.19:0/1837660153 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:31:21.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.516+0000 7f9eb2600700 1 --2- 172.21.15.19:0/1837660153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9eac109870 0x7f9eac076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:21.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.516+0000 7f9eb1200700 1 --2- 172.21.15.19:0/1837660153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9eac108f20 0x7f9eac076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:21.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.516+0000 7f9eb1c00700 1 -- 172.21.15.19:0/1837660153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9eac109870 msgr2=0x7f9eac076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:21.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.516+0000 7f9eb1c00700 1 --2- 172.21.15.19:0/1837660153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9eac109870 0x7f9eac076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:21.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.517+0000 7f9eb1c00700 1 -- 172.21.15.19:0/1837660153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9eac108f20 msgr2=0x7f9eac076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:21.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.517+0000 7f9eb1c00700 1 --2- 172.21.15.19:0/1837660153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9eac108f20 0x7f9eac076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:21.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.517+0000 7f9eb1c00700 1 -- 172.21.15.19:0/1837660153 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eac1a92a0 con 0x7f9eac1045f0 2024-05-07T02:31:21.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.517+0000 7f9eb1c00700 1 --2- 172.21.15.19:0/1837660153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9eac1045f0 0x7f9eac075ac0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0009230 tx=0x7f9ea000fa70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:21.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.517+0000 7f9eaa000700 1 -- 172.21.15.19:0/1837660153 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0005140 con 0x7f9eac1045f0 2024-05-07T02:31:21.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.517+0000 7f9eaa000700 1 -- 172.21.15.19:0/1837660153 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ea000b560 con 0x7f9eac1045f0 2024-05-07T02:31:21.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.517+0000 7f9eb4600700 1 -- 172.21.15.19:0/1837660153 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eac1a9530 con 0x7f9eac1045f0 2024-05-07T02:31:21.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.517+0000 7f9eaa000700 1 -- 172.21.15.19:0/1837660153 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0003ae0 con 0x7f9eac1045f0 2024-05-07T02:31:21.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.517+0000 7f9eb4600700 1 -- 172.21.15.19:0/1837660153 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eac1a99e0 con 0x7f9eac1045f0 2024-05-07T02:31:21.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.522+0000 7f9ec31a0b80 1 Processor -- start 2024-05-07T02:31:21.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.522+0000 7f9ec31a0b80 1 -- start start 2024-05-07T02:31:21.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.523+0000 7f9ec31a0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39d08d360 0x55b39d08d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:21.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.523+0000 7f9ec31a0b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b39d087420 0x55b39d087830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:21.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.523+0000 7f9ec31a0b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b39d087d70 0x55b39d093ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:21.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.523+0000 7f9ec31a0b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b39cfa99d0 con 0x55b39d08d360 2024-05-07T02:31:21.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.523+0000 7f9ec31a0b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b39cf47710 con 0x55b39d087420 2024-05-07T02:31:21.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.523+0000 7f9ec31a0b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b39cfa5990 con 0x55b39d087d70 2024-05-07T02:31:21.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.523+0000 7f9eb2600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39d08d360 0x55b39d08d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:21.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.523+0000 7f9eb2600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39d08d360 0x55b39d08d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51618/0 (socket says 172.21.15.19:51618) 2024-05-07T02:31:21.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.523+0000 7f9eb2600700 1 -- 172.21.15.19:0/846094397 learned_addr learned my addr 172.21.15.19:0/846094397 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:31:21.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.523+0000 7f9eb1200700 1 --2- 172.21.15.19:0/846094397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b39d087d70 0x55b39d093ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:21.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.524+0000 7f9eb2600700 1 -- 172.21.15.19:0/846094397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b39d087d70 msgr2=0x55b39d093ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:31:21.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.525+0000 7f9eb2600700 1 --2- 172.21.15.19:0/846094397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b39d087d70 0x55b39d093ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:21.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.525+0000 7f9eb2600700 1 -- 172.21.15.19:0/846094397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b39d087420 msgr2=0x55b39d087830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-07T02:31:21.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.525+0000 7f9eb2600700 1 --2- 172.21.15.19:0/846094397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b39d087420 0x55b39d087830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:21.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.525+0000 7f9eb2600700 1 -- 172.21.15.19:0/846094397 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b39cef6880 con 0x55b39d08d360 2024-05-07T02:31:21.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.525+0000 7f9eb1200700 1 --2- 172.21.15.19:0/846094397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b39d087d70 0x55b39d093ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-07T02:31:21.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.525+0000 7f9eaa000700 1 -- 172.21.15.19:0/1837660153 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9ea0003c80 con 0x7f9eac1045f0 2024-05-07T02:31:21.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.525+0000 7f9eb2600700 1 --2- 172.21.15.19:0/846094397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39d08d360 0x55b39d08d770 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4011660 tx=0x7f9ea4009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f7cb9cbcd22711a server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:21.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.526+0000 7f9e9ea00700 1 -- 172.21.15.19:0/846094397 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea4019030 con 0x55b39d08d360 2024-05-07T02:31:21.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.526+0000 7f9e9ea00700 1 -- 172.21.15.19:0/846094397 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ea40093a0 con 0x55b39d08d360 2024-05-07T02:31:21.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.526+0000 7f9e9ea00700 1 -- 172.21.15.19:0/846094397 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea4007500 con 0x55b39d08d360 2024-05-07T02:31:21.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.527+0000 7f9eaa000700 1 --2- 172.21.15.19:0/1837660153 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f9e940b2900 0x7f9e940b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:21.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.527+0000 7f9ec31a0b80 1 -- 172.21.15.19:0/846094397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39d08d360 msgr2=0x55b39d08d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:21.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.527+0000 7f9ec31a0b80 1 --2- 172.21.15.19:0/846094397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39d08d360 0x55b39d08d770 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4011660 tx=0x7f9ea4009920 comp rx=0 tx=0).stop 2024-05-07T02:31:21.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.527+0000 7f9eaa000700 1 -- 172.21.15.19:0/1837660153 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f9ea00e3550 con 0x7f9eac1045f0 2024-05-07T02:31:21.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.527+0000 7f9ec31a0b80 1 -- 172.21.15.19:0/846094397 shutdown_connections 2024-05-07T02:31:21.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.527+0000 7f9ec31a0b80 1 --2- 172.21.15.19:0/846094397 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b39d087d70 0x55b39d093ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:21.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.527+0000 7f9ec31a0b80 1 --2- 172.21.15.19:0/846094397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39d08d360 0x55b39d08d770 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:21.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.527+0000 7f9ec31a0b80 1 --2- 172.21.15.19:0/846094397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b39d087420 0x55b39d087830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:21.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.527+0000 7f9ec31a0b80 1 -- 172.21.15.19:0/846094397 >> 172.21.15.19:0/846094397 conn(0x55b39cf09df0 msgr2=0x55b39d0822a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:21.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.527+0000 7f9eb1c00700 1 --2- 172.21.15.19:0/1837660153 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f9e940b2900 0x7f9e940b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:21.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.528+0000 7f9ec31a0b80 1 -- 172.21.15.19:0/846094397 shutdown_connections 2024-05-07T02:31:21.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.528+0000 7f9ec31a0b80 1 -- 172.21.15.19:0/846094397 wait complete. 2024-05-07T02:31:21.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.528+0000 7f9eb1c00700 1 --2- 172.21.15.19:0/1837660153 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f9e940b2900 0x7f9e940b4e10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9ea00e0c40 tx=0x7f9ea0009c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:21.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.529+0000 7f9ec31a0b80 1 -- start start 2024-05-07T02:31:21.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.529+0000 7f9ec31a0b80 1 Processor -- start 2024-05-07T02:31:21.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.529+0000 7f9ec31a0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39d087420 0x55b39d091540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:21.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.529+0000 7f9ec31a0b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b39d087d70 0x55b39d091a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:21.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.529+0000 7f9ec31a0b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b39d08d360 0x55b39d091fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:21.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.529+0000 7f9ec31a0b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b39d094770 con 0x55b39d087420 2024-05-07T02:31:21.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.529+0000 7f9ec31a0b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b39cf0fad0 con 0x55b39d087d70 2024-05-07T02:31:21.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.529+0000 7f9ec31a0b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b39d0948f0 con 0x55b39d08d360 2024-05-07T02:31:21.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.530+0000 7f9ec31a0b80 10 client.0 _ll_register_callbacks cb 0x55b39cf50c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:31:21.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.530+0000 7f9eb1200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b39d087d70 0x55b39d091a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:21.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.530+0000 7f9eb1200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b39d087d70 0x55b39d091a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:49448/0 (socket says 172.21.15.19:49448) 2024-05-07T02:31:21.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.530+0000 7f9eb1200700 1 -- 172.21.15.19:0/1925313874 learned_addr learned my addr 172.21.15.19:0/1925313874 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:31:21.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.530+0000 7f9eb2600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39d087420 0x55b39d091540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:21.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.530+0000 7f9eb2600700 1 --2- 172.21.15.19:0/1925313874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b39d08d360 0x55b39d091fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:21.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.530+0000 7f9eb1200700 1 -- 172.21.15.19:0/1925313874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b39d08d360 msgr2=0x55b39d091fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:31:21.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.531+0000 7f9eb1200700 1 --2- 172.21.15.19:0/1925313874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b39d08d360 0x55b39d091fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:21.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.531+0000 7f9eb1200700 1 -- 172.21.15.19:0/1925313874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39d087420 msgr2=0x55b39d091540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:31:21.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.531+0000 7f9eb1200700 1 --2- 172.21.15.19:0/1925313874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39d087420 0x55b39d091540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:21.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.531+0000 7f9eb1200700 1 -- 172.21.15.19:0/1925313874 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b39cf6f3d0 con 0x55b39d087d70 2024-05-07T02:31:21.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.531+0000 7f9eb1200700 1 --2- 172.21.15.19:0/1925313874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b39d087d70 0x55b39d091a80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9e98012700 tx=0x7f9e98012a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e886cf02b2e9dedf server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:21.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.531+0000 7f9e93400700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:31:21.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.531+0000 7f9e93400700 1 -- 172.21.15.19:0/1925313874 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e98004460 con 0x55b39d087d70 2024-05-07T02:31:21.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.531+0000 7f9e93400700 1 -- 172.21.15.19:0/1925313874 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f9e98003680 con 0x55b39d087d70 2024-05-07T02:31:21.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.531+0000 7f9ec31a0b80 1 -- 172.21.15.19:0/1925313874 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b39d054290 con 0x55b39d087d70 2024-05-07T02:31:21.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.532+0000 7f9e93400700 1 -- 172.21.15.19:0/1925313874 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9e98003880 con 0x55b39d087d70 2024-05-07T02:31:21.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.532+0000 7f9ec31a0b80 10 client.25142 fetch_fsmap learned FSMap version 30 2024-05-07T02:31:21.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.532+0000 7f9ec31a0b80 1 -- 172.21.15.19:0/1925313874 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b39d09f3c0 con 0x55b39d087d70 2024-05-07T02:31:21.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.532+0000 7f9e93400700 1 -- 172.21.15.19:0/1925313874 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b39d054290 con 0x55b39d087d70 2024-05-07T02:31:21.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.532+0000 7f9e93400700 1 -- 172.21.15.19:0/1925313874 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9e98019070 con 0x55b39d087d70 2024-05-07T02:31:21.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.532+0000 7f9ec31a0b80 10 client.25142 fetch_fsmap finished waiting for FSMap version 30 2024-05-07T02:31:21.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.532+0000 7f9ec31a0b80 10 client.25142 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-07T02:31:21.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.532+0000 7f9ec31a0b80 20 client.25142 populate_metadata read hostname 'smithi019' 2024-05-07T02:31:21.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.532+0000 7f9ec31a0b80 1 --2- 172.21.15.19:0/1925313874 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x55b39d0a1a90 0x55b39d0a2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:21.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.532+0000 7f9ec31a0b80 4 client.25142 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:31:21.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.532+0000 7f9ec31a0b80 1 -- 172.21.15.19:0/1925313874 --> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b39cfb73b0 con 0x55b39d0a1a90 2024-05-07T02:31:21.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.533+0000 7f9eb2600700 1 --2- 172.21.15.19:0/1925313874 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x55b39d0a1a90 0x55b39d0a2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:21.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.534+0000 7f9eb2600700 1 --2- 172.21.15.19:0/1925313874 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x55b39d0a1a90 0x55b39d0a2f00 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8a5a4166ad90b496 server_cookie=311ae5e175698d5a in_seq=0 out_seq=0 2024-05-07T02:31:21.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.534+0000 7f9e93400700 10 client.25142 ms_handle_connect on v2:172.21.15.64:6836/2585907777 2024-05-07T02:31:21.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.534+0000 7f9e93400700 1 -- 172.21.15.19:0/1925313874 <== mds.4 v2:172.21.15.64:6836/2585907777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b39cfb73b0 con 0x55b39d0a1a90 2024-05-07T02:31:21.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.534+0000 7f9e93400700 10 client.25142 handle_command_reply: tid=0 2024-05-07T02:31:21.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.534+0000 7f9ec31a0b80 1 client.25142 shutdown 2024-05-07T02:31:21.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.535+0000 7f9ec31a0b80 1 -- 172.21.15.19:0/1925313874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b39d087d70 msgr2=0x55b39d091a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:21.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.535+0000 7f9ec31a0b80 1 --2- 172.21.15.19:0/1925313874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b39d087d70 0x55b39d091a80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9e98012700 tx=0x7f9e98012a10 comp rx=0 tx=0).stop 2024-05-07T02:31:21.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.535+0000 7f9ec31a0b80 1 -- 172.21.15.19:0/1925313874 shutdown_connections 2024-05-07T02:31:21.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.535+0000 7f9ec31a0b80 1 --2- 172.21.15.19:0/1925313874 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x55b39d0a1a90 0x55b39d0a2f00 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:21.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.535+0000 7f9ec31a0b80 1 --2- 172.21.15.19:0/1925313874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b39d08d360 0x55b39d091fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:21.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.535+0000 7f9ec31a0b80 1 --2- 172.21.15.19:0/1925313874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39d087420 0x55b39d091540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:21.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.535+0000 7f9ec31a0b80 1 --2- 172.21.15.19:0/1925313874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b39d087d70 0x55b39d091a80 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:21.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.535+0000 7f9ec31a0b80 1 -- 172.21.15.19:0/1925313874 >> 172.21.15.19:0/1925313874 conn(0x55b39cf09df0 msgr2=0x55b39d086410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:21.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.535+0000 7f9e93400700 0 client.25142 ms_handle_reset on v2:172.21.15.64:6836/2585907777 2024-05-07T02:31:21.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.535+0000 7f9ec31a0b80 1 -- 172.21.15.19:0/1925313874 shutdown_connections 2024-05-07T02:31:21.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.535+0000 7f9ec31a0b80 1 -- 172.21.15.19:0/1925313874 wait complete. 2024-05-07T02:31:21.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.535+0000 7f9ec31a0b80 20 client.25142 trim_cache size 0 max 16384 2024-05-07T02:31:21.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.558+0000 7f9ec31a0b80 1 Processor -- start 2024-05-07T02:31:21.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.558+0000 7f9ec31a0b80 1 -- start start 2024-05-07T02:31:21.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.558+0000 7f9ec31a0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39d087420 0x55b39d08d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:21.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.558+0000 7f9ec31a0b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b39d087d70 0x55b39d09db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:21.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.558+0000 7f9ec31a0b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b39d08d360 0x55b39d08ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:21.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.558+0000 7f9ec31a0b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b39d0a1240 con 0x55b39d087420 2024-05-07T02:31:21.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.558+0000 7f9ec31a0b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b39d0a1910 con 0x55b39d087d70 2024-05-07T02:31:21.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.558+0000 7f9ec31a0b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b39d0a1fe0 con 0x55b39d08d360 2024-05-07T02:31:21.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.559+0000 7f9eb2600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39d087420 0x55b39d08d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:21.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.559+0000 7f9eb2600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39d087420 0x55b39d08d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51640/0 (socket says 172.21.15.19:51640) 2024-05-07T02:31:21.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.559+0000 7f9eb2600700 1 -- 172.21.15.19:0/3294850635 learned_addr learned my addr 172.21.15.19:0/3294850635 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:31:21.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.559+0000 7f9eb1200700 1 --2- 172.21.15.19:0/3294850635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b39d087d70 0x55b39d09db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:21.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.559+0000 7f9eb2600700 1 --2- 172.21.15.19:0/3294850635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b39d08d360 0x55b39d08ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:21.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.559+0000 7f9eb2600700 1 -- 172.21.15.19:0/3294850635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b39d08d360 msgr2=0x55b39d08ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:31:21.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.559+0000 7f9eb2600700 1 --2- 172.21.15.19:0/3294850635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b39d08d360 0x55b39d08ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:21.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.559+0000 7f9eb2600700 1 -- 172.21.15.19:0/3294850635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b39d087d70 msgr2=0x55b39d09db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:31:21.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.559+0000 7f9eb2600700 1 --2- 172.21.15.19:0/3294850635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b39d087d70 0x55b39d09db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:21.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.559+0000 7f9eb2600700 1 -- 172.21.15.19:0/3294850635 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b39d09e0d0 con 0x55b39d087420 2024-05-07T02:31:21.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.559+0000 7f9eb2600700 1 --2- 172.21.15.19:0/3294850635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39d087420 0x55b39d08d770 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x55b39d08ce20 tx=0x7f9ea4024040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f97d5627416d0af server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:21.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.560+0000 7f9e9e000700 1 -- 172.21.15.19:0/3294850635 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea40047e0 con 0x55b39d087420 2024-05-07T02:31:21.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.560+0000 7f9e9e000700 1 -- 172.21.15.19:0/3294850635 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ea4004e30 con 0x55b39d087420 2024-05-07T02:31:21.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.560+0000 7f9e9e000700 1 -- 172.21.15.19:0/3294850635 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea402d9e0 con 0x55b39d087420 2024-05-07T02:31:21.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.561+0000 7f9ec31a0b80 1 -- 172.21.15.19:0/3294850635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39d087420 msgr2=0x55b39d08d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:21.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.561+0000 7f9ec31a0b80 1 --2- 172.21.15.19:0/3294850635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39d087420 0x55b39d08d770 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x55b39d08ce20 tx=0x7f9ea4024040 comp rx=0 tx=0).stop 2024-05-07T02:31:21.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.561+0000 7f9ec31a0b80 1 -- 172.21.15.19:0/3294850635 shutdown_connections 2024-05-07T02:31:21.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.561+0000 7f9ec31a0b80 1 --2- 172.21.15.19:0/3294850635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b39d08d360 0x55b39d08ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:21.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.561+0000 7f9ec31a0b80 1 --2- 172.21.15.19:0/3294850635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39d087420 0x55b39d08d770 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:21.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.561+0000 7f9ec31a0b80 1 --2- 172.21.15.19:0/3294850635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b39d087d70 0x55b39d09db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:21.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.561+0000 7f9ec31a0b80 1 -- 172.21.15.19:0/3294850635 >> 172.21.15.19:0/3294850635 conn(0x55b39cf09df0 msgr2=0x55b39d0a0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:21.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.561+0000 7f9ec31a0b80 1 -- 172.21.15.19:0/3294850635 shutdown_connections 2024-05-07T02:31:21.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.561+0000 7f9ec31a0b80 1 -- 172.21.15.19:0/3294850635 wait complete. 2024-05-07T02:31:21.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.562+0000 7f9ec31a0b80 1 -- start start 2024-05-07T02:31:21.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.562+0000 7f9ec31a0b80 1 Processor -- start 2024-05-07T02:31:21.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.562+0000 7f9ec31a0b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b39d087420 0x55b39d09cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:21.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.562+0000 7f9ec31a0b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b39d087d70 0x55b39d09d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:21.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.562+0000 7f9ec31a0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39d08d360 0x55b39d0ab370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:21.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.562+0000 7f9ec31a0b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b39d0a2d80 con 0x55b39d08d360 2024-05-07T02:31:21.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.562+0000 7f9ec31a0b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b39d0a26b0 con 0x55b39d087d70 2024-05-07T02:31:21.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.562+0000 7f9ec31a0b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b39d0a3450 con 0x55b39d087420 2024-05-07T02:31:21.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.562+0000 7f9ec31a0b80 10 client.0 _ll_register_callbacks cb 0x55b39cf50c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:31:21.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.563+0000 7f9eb2600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39d08d360 0x55b39d0ab370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:21.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.563+0000 7f9eb2600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39d08d360 0x55b39d0ab370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51650/0 (socket says 172.21.15.19:51650) 2024-05-07T02:31:21.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.563+0000 7f9eb2600700 1 -- 172.21.15.19:0/188187431 learned_addr learned my addr 172.21.15.19:0/188187431 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:31:21.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.563+0000 7f9eb1200700 1 --2- 172.21.15.19:0/188187431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b39d087d70 0x55b39d09d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:21.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.563+0000 7f9eb2600700 1 --2- 172.21.15.19:0/188187431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b39d087420 0x55b39d09cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:21.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.563+0000 7f9eb1200700 1 -- 172.21.15.19:0/188187431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b39d087420 msgr2=0x55b39d09cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:31:21.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.563+0000 7f9eb1200700 1 --2- 172.21.15.19:0/188187431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b39d087420 0x55b39d09cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:21.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.563+0000 7f9eb1200700 1 -- 172.21.15.19:0/188187431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39d08d360 msgr2=0x55b39d0ab370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:31:21.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.563+0000 7f9eb1200700 1 --2- 172.21.15.19:0/188187431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39d08d360 0x55b39d0ab370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:21.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.563+0000 7f9eb1200700 1 -- 172.21.15.19:0/188187431 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b39d09d9c0 con 0x55b39d087d70 2024-05-07T02:31:21.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.563+0000 7f9eb2600700 1 --2- 172.21.15.19:0/188187431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39d08d360 0x55b39d0ab370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:31:21.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.564+0000 7f9eb1200700 1 --2- 172.21.15.19:0/188187431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b39d087d70 0x55b39d09d0c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9e9801bd50 tx=0x7f9e98003c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edd6fff3c5235cfc server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:21.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.564+0000 7f9e93400700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:31:21.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.564+0000 7f9e93400700 1 -- 172.21.15.19:0/188187431 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9802d020 con 0x55b39d087d70 2024-05-07T02:31:21.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.564+0000 7f9ec31a0b80 1 -- 172.21.15.19:0/188187431 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ea4004e30 con 0x55b39d087d70 2024-05-07T02:31:21.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.564+0000 7f9e93400700 1 -- 172.21.15.19:0/188187431 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f9e98003e20 con 0x55b39d087d70 2024-05-07T02:31:21.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.564+0000 7f9e93400700 1 -- 172.21.15.19:0/188187431 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9e98012d60 con 0x55b39d087d70 2024-05-07T02:31:21.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.564+0000 7f9ec31a0b80 10 client.25148 fetch_fsmap learned FSMap version 30 2024-05-07T02:31:21.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.564+0000 7f9ec31a0b80 1 -- 172.21.15.19:0/188187431 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b39d0ac9c0 con 0x55b39d087d70 2024-05-07T02:31:21.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.565+0000 7f9e93400700 1 -- 172.21.15.19:0/188187431 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ea4004e30 con 0x55b39d087d70 2024-05-07T02:31:21.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.565+0000 7f9e93400700 1 -- 172.21.15.19:0/188187431 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9e98019070 con 0x55b39d087d70 2024-05-07T02:31:21.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.565+0000 7f9ec31a0b80 10 client.25148 fetch_fsmap finished waiting for FSMap version 30 2024-05-07T02:31:21.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.565+0000 7f9ec31a0b80 10 client.25148 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-07T02:31:21.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.565+0000 7f9ec31a0b80 20 client.25148 populate_metadata read hostname 'smithi019' 2024-05-07T02:31:21.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.565+0000 7f9ec31a0b80 1 --2- 172.21.15.19:0/188187431 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x55b39d10ede0 0x55b39d110250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:21.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.565+0000 7f9ec31a0b80 4 client.25148 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-07T02:31:21.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.565+0000 7f9ec31a0b80 1 -- 172.21.15.19:0/188187431 --> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] -- command(tid 0: {"prefix": "status"}) -- 0x55b39cfb1450 con 0x55b39d10ede0 2024-05-07T02:31:21.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.565+0000 7f9eb2600700 1 --2- 172.21.15.19:0/188187431 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x55b39d10ede0 0x55b39d110250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:21.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.566+0000 7f9eb2600700 1 --2- 172.21.15.19:0/188187431 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x55b39d10ede0 0x55b39d110250 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3f547ec88b4e455d server_cookie=dc452e2592a03b6b in_seq=0 out_seq=0 2024-05-07T02:31:21.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.566+0000 7f9e93400700 10 client.25148 ms_handle_connect on v2:172.21.15.64:6836/2585907777 2024-05-07T02:31:21.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.566+0000 7f9e93400700 1 -- 172.21.15.19:0/188187431 <== mds.4 v2:172.21.15.64:6836/2585907777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55b39cfb1450 con 0x55b39d10ede0 2024-05-07T02:31:21.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.566+0000 7f9e93400700 10 client.25148 handle_command_reply: tid=0 2024-05-07T02:31:21.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.566+0000 7f9ec31a0b80 1 client.25148 shutdown 2024-05-07T02:31:21.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.566+0000 7f9ec31a0b80 1 -- 172.21.15.19:0/188187431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b39d087d70 msgr2=0x55b39d09d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:21.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.566+0000 7f9ec31a0b80 1 --2- 172.21.15.19:0/188187431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b39d087d70 0x55b39d09d0c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9e9801bd50 tx=0x7f9e98003c30 comp rx=0 tx=0).stop 2024-05-07T02:31:21.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.566+0000 7f9ec31a0b80 1 -- 172.21.15.19:0/188187431 shutdown_connections 2024-05-07T02:31:21.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.566+0000 7f9ec31a0b80 1 --2- 172.21.15.19:0/188187431 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x55b39d10ede0 0x55b39d110250 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:21.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.566+0000 7f9ec31a0b80 1 --2- 172.21.15.19:0/188187431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39d08d360 0x55b39d0ab370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:21.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.566+0000 7f9ec31a0b80 1 --2- 172.21.15.19:0/188187431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b39d087420 0x55b39d09cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:21.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.566+0000 7f9ec31a0b80 1 --2- 172.21.15.19:0/188187431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b39d087d70 0x55b39d09d0c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:21.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.566+0000 7f9ec31a0b80 1 -- 172.21.15.19:0/188187431 >> 172.21.15.19:0/188187431 conn(0x55b39cf09df0 msgr2=0x55b39d07c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:21.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.566+0000 7f9e93400700 0 client.25148 ms_handle_reset on v2:172.21.15.64:6836/2585907777 2024-05-07T02:31:21.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.567+0000 7f9ec31a0b80 1 -- 172.21.15.19:0/188187431 shutdown_connections 2024-05-07T02:31:21.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.567+0000 7f9ec31a0b80 1 -- 172.21.15.19:0/188187431 wait complete. 2024-05-07T02:31:21.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.567+0000 7f9ec31a0b80 20 client.25148 trim_cache size 0 max 16384 2024-05-07T02:31:21.569 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:31:21.569 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "9ff84c16-0c17-11ef-bc97-c7b262605968", 2024-05-07T02:31:21.569 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 4, 2024-05-07T02:31:21.569 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24505, 2024-05-07T02:31:21.569 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-05-07T02:31:21.569 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-05-07T02:31:21.569 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-05-07T02:31:21.569 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 218.84398759000001, 2024-05-07T02:31:21.570 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 30, 2024-05-07T02:31:21.570 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 89, 2024-05-07T02:31:21.570 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 87, 2024-05-07T02:31:21.570 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 347.004765045 2024-05-07T02:31:21.570 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:31:21.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.567+0000 7f9eb4600700 1 -- 172.21.15.19:0/1837660153 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f9e940b2900 msgr2=0x7f9e940b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:21.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.567+0000 7f9eb4600700 1 --2- 172.21.15.19:0/1837660153 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f9e940b2900 0x7f9e940b4e10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9ea00e0c40 tx=0x7f9ea0009c00 comp rx=0 tx=0).stop 2024-05-07T02:31:21.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.567+0000 7f9eb4600700 1 -- 172.21.15.19:0/1837660153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9eac1045f0 msgr2=0x7f9eac075ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:21.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.567+0000 7f9eb4600700 1 --2- 172.21.15.19:0/1837660153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9eac1045f0 0x7f9eac075ac0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0009230 tx=0x7f9ea000fa70 comp rx=0 tx=0).stop 2024-05-07T02:31:21.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.568+0000 7f9eb4600700 1 -- 172.21.15.19:0/1837660153 shutdown_connections 2024-05-07T02:31:21.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.568+0000 7f9eb4600700 1 --2- 172.21.15.19:0/1837660153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9eac109870 0x7f9eac076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:21.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.568+0000 7f9eb4600700 1 --2- 172.21.15.19:0/1837660153 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f9e940b2900 0x7f9e940b4e10 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:21.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.568+0000 7f9eb4600700 1 --2- 172.21.15.19:0/1837660153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9eac1045f0 0x7f9eac075ac0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:21.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.568+0000 7f9eb4600700 1 --2- 172.21.15.19:0/1837660153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9eac108f20 0x7f9eac076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:21.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.568+0000 7f9eb4600700 1 -- 172.21.15.19:0/1837660153 >> 172.21.15.19:0/1837660153 conn(0x7f9eac100040 msgr2=0x7f9eac10a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:21.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.568+0000 7f9eb4600700 1 -- 172.21.15.19:0/1837660153 shutdown_connections 2024-05-07T02:31:21.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:21.568+0000 7f9eb4600700 1 -- 172.21.15.19:0/1837660153 wait complete. 2024-05-07T02:31:22.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:21 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:31:22.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:21 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:31:22.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:21 smithi064 ceph-mon[34699]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:31:22.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:21 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:31:22.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:21 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:31:22.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:21 smithi019 ceph-mon[29781]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:31:22.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:21 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:31:22.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:21 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:31:22.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:21 smithi027 ceph-mon[34398]: from='mgr.14152 172.21.15.19:0/2419208468' entity='mgr.x' 2024-05-07T02:31:22.530 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.h status 2024-05-07T02:31:23.005 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:31:23.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:22 smithi064 ceph-mon[34699]: pgmap v494: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:31:23.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:22 smithi019 ceph-mon[29781]: pgmap v494: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:31:23.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:22 smithi027 ceph-mon[34398]: pgmap v494: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:31:25.101 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:24 smithi019 ceph-mon[29781]: pgmap v495: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:31:25.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:24 smithi064 ceph-mon[34699]: pgmap v495: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:31:25.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:24 smithi027 ceph-mon[34398]: pgmap v495: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:31:25.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.515+0000 7f99a9000700 1 -- 172.21.15.19:0/1601520247 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99a4105e70 msgr2=0x7f99a410e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:25.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.515+0000 7f99a9000700 1 --2- 172.21.15.19:0/1601520247 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99a4105e70 0x7f99a410e3c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f99900098f0 tx=0x7f9990009c00 comp rx=0 tx=0).stop 2024-05-07T02:31:25.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.516+0000 7f99a9000700 1 -- 172.21.15.19:0/1601520247 shutdown_connections 2024-05-07T02:31:25.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.516+0000 7f99a9000700 1 --2- 172.21.15.19:0/1601520247 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99a410e900 0x7f99a4110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:25.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.516+0000 7f99a9000700 1 --2- 172.21.15.19:0/1601520247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99a4105520 0x7f99a4105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:25.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.516+0000 7f99a9000700 1 --2- 172.21.15.19:0/1601520247 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99a4105e70 0x7f99a410e3c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:25.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.516+0000 7f99a9000700 1 -- 172.21.15.19:0/1601520247 >> 172.21.15.19:0/1601520247 conn(0x7f99a40fedf0 msgr2=0x7f99a4101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:25.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.516+0000 7f99a9000700 1 -- 172.21.15.19:0/1601520247 shutdown_connections 2024-05-07T02:31:25.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.516+0000 7f99a9000700 1 -- 172.21.15.19:0/1601520247 wait complete. 2024-05-07T02:31:25.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.517+0000 7f99a9000700 1 Processor -- start 2024-05-07T02:31:25.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.517+0000 7f99a9000700 1 -- start start 2024-05-07T02:31:25.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.518+0000 7f99a9000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99a4105520 0x7f99a4103210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:25.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.518+0000 7f99a9000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99a4105e70 0x7f99a4103750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:25.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.518+0000 7f99a9000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99a410e900 0x7f99a4103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:25.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.518+0000 7f99a9000700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f99a41144a0 con 0x7f99a4105e70 2024-05-07T02:31:25.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.518+0000 7f99a9000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f99a4114320 con 0x7f99a4105520 2024-05-07T02:31:25.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.518+0000 7f99a9000700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f99a4114620 con 0x7f99a410e900 2024-05-07T02:31:25.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.518+0000 7f99a2800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99a410e900 0x7f99a4103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:25.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.518+0000 7f99a1400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99a4105e70 0x7f99a4103750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:25.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.518+0000 7f99a1e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99a4105520 0x7f99a4103210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:25.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.518+0000 7f99a2800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99a410e900 0x7f99a4103c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:39498/0 (socket says 172.21.15.19:39498) 2024-05-07T02:31:25.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.518+0000 7f99a2800700 1 -- 172.21.15.19:0/981918582 learned_addr learned my addr 172.21.15.19:0/981918582 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:31:25.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.519+0000 7f99a2800700 1 -- 172.21.15.19:0/981918582 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99a4105520 msgr2=0x7f99a4103210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:25.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.519+0000 7f99a2800700 1 --2- 172.21.15.19:0/981918582 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99a4105520 0x7f99a4103210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:25.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.519+0000 7f99a2800700 1 -- 172.21.15.19:0/981918582 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99a4105e70 msgr2=0x7f99a4103750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:25.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.519+0000 7f99a2800700 1 --2- 172.21.15.19:0/981918582 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99a4105e70 0x7f99a4103750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:25.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.519+0000 7f99a2800700 1 -- 172.21.15.19:0/981918582 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99a41a9250 con 0x7f99a410e900 2024-05-07T02:31:25.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.519+0000 7f99a1e00700 1 --2- 172.21.15.19:0/981918582 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99a4105520 0x7f99a4103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:31:25.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.519+0000 7f99a2800700 1 --2- 172.21.15.19:0/981918582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99a410e900 0x7f99a4103c90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f999c00ef30 tx=0x7f999c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:25.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.519+0000 7f999aa00700 1 -- 172.21.15.19:0/981918582 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999c0092e0 con 0x7f99a410e900 2024-05-07T02:31:25.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.519+0000 7f999aa00700 1 -- 172.21.15.19:0/981918582 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f999c004bb0 con 0x7f99a410e900 2024-05-07T02:31:25.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.519+0000 7f999aa00700 1 -- 172.21.15.19:0/981918582 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999c004020 con 0x7f99a410e900 2024-05-07T02:31:25.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.520+0000 7f99a9000700 1 -- 172.21.15.19:0/981918582 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99a41a94e0 con 0x7f99a410e900 2024-05-07T02:31:25.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.520+0000 7f99a9000700 1 -- 172.21.15.19:0/981918582 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99a41a99a0 con 0x7f99a410e900 2024-05-07T02:31:25.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.524+0000 7f999aa00700 1 -- 172.21.15.19:0/981918582 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f999c01b030 con 0x7f99a410e900 2024-05-07T02:31:25.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.525+0000 7f99b7b05b80 1 Processor -- start 2024-05-07T02:31:25.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.525+0000 7f99b7b05b80 1 -- start start 2024-05-07T02:31:25.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.525+0000 7f99b7b05b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608abc6a360 0x5608abc6a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:25.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.525+0000 7f99b7b05b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5608abc64420 0x5608abc64830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:25.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.525+0000 7f99b7b05b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5608abc64d70 0x5608abc70ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:25.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.525+0000 7f99b7b05b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5608abb869d0 con 0x5608abc64d70 2024-05-07T02:31:25.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.525+0000 7f99b7b05b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5608abb24710 con 0x5608abc6a360 2024-05-07T02:31:25.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.525+0000 7f99b7b05b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5608abb82990 con 0x5608abc64420 2024-05-07T02:31:25.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.526+0000 7f99a1e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5608abc64420 0x5608abc64830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:25.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.526+0000 7f99a1400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608abc6a360 0x5608abc6a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:25.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.526+0000 7f999aa00700 1 --2- 172.21.15.19:0/981918582 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f99880b2900 0x7f99880b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:25.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.526+0000 7f99a1e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5608abc64420 0x5608abc64830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:39506/0 (socket says 172.21.15.19:39506) 2024-05-07T02:31:25.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.526+0000 7f99a1e00700 1 -- 172.21.15.19:0/3749470277 learned_addr learned my addr 172.21.15.19:0/3749470277 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:31:25.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.526+0000 7f99a1400700 1 --2- 172.21.15.19:0/3749470277 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608abc6a360 0x5608abc6a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:49496/0 (socket says 172.21.15.19:49496) 2024-05-07T02:31:25.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.526+0000 7f99a1400700 1 --2- 172.21.15.19:0/3749470277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5608abc64d70 0x5608abc70ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:25.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.526+0000 7f999aa00700 1 -- 172.21.15.19:0/981918582 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f999c0e1fe0 con 0x7f99a410e900 2024-05-07T02:31:25.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.526+0000 7f99a1e00700 1 -- 172.21.15.19:0/3749470277 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608abc6a360 msgr2=0x5608abc6a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:31:25.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.526+0000 7f99a1e00700 1 --2- 172.21.15.19:0/3749470277 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608abc6a360 0x5608abc6a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:25.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.526+0000 7f99a2800700 1 --2- 172.21.15.19:0/981918582 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f99880b2900 0x7f99880b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:25.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.526+0000 7f99a1e00700 1 -- 172.21.15.19:0/3749470277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5608abc64d70 msgr2=0x5608abc70ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:31:25.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.526+0000 7f99a1e00700 1 --2- 172.21.15.19:0/3749470277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5608abc64d70 0x5608abc70ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:25.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.526+0000 7f99a1e00700 1 -- 172.21.15.19:0/3749470277 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608abad3880 con 0x5608abc64420 2024-05-07T02:31:25.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.527+0000 7f99a1e00700 1 --2- 172.21.15.19:0/3749470277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5608abc64420 0x5608abc64830 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f99940115f0 tx=0x7f9994011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11a56a90bf5954ab server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:25.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.527+0000 7f99a2800700 1 --2- 172.21.15.19:0/981918582 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f99880b2900 0x7f99880b4e10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f999c0090a0 tx=0x7f999c009ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:25.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.527+0000 7f998f400700 1 -- 172.21.15.19:0/3749470277 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9994004a30 con 0x5608abc64420 2024-05-07T02:31:25.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.527+0000 7f998f400700 1 -- 172.21.15.19:0/3749470277 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99940054d0 con 0x5608abc64420 2024-05-07T02:31:25.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.527+0000 7f998f400700 1 -- 172.21.15.19:0/3749470277 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999400e9b0 con 0x5608abc64420 2024-05-07T02:31:25.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.528+0000 7f99b7b05b80 1 -- 172.21.15.19:0/3749470277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5608abc64420 msgr2=0x5608abc64830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:25.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.528+0000 7f99b7b05b80 1 --2- 172.21.15.19:0/3749470277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5608abc64420 0x5608abc64830 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f99940115f0 tx=0x7f9994011900 comp rx=0 tx=0).stop 2024-05-07T02:31:25.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.529+0000 7f99b7b05b80 1 -- 172.21.15.19:0/3749470277 shutdown_connections 2024-05-07T02:31:25.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.529+0000 7f99b7b05b80 1 --2- 172.21.15.19:0/3749470277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5608abc64d70 0x5608abc70ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:25.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.529+0000 7f99b7b05b80 1 --2- 172.21.15.19:0/3749470277 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608abc6a360 0x5608abc6a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:25.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.529+0000 7f99b7b05b80 1 --2- 172.21.15.19:0/3749470277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5608abc64420 0x5608abc64830 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:25.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.529+0000 7f99b7b05b80 1 -- 172.21.15.19:0/3749470277 >> 172.21.15.19:0/3749470277 conn(0x5608abae6df0 msgr2=0x5608abc5f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:25.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.529+0000 7f99b7b05b80 1 -- 172.21.15.19:0/3749470277 shutdown_connections 2024-05-07T02:31:25.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.529+0000 7f99b7b05b80 1 -- 172.21.15.19:0/3749470277 wait complete. 2024-05-07T02:31:25.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.529+0000 7f99b7b05b80 1 -- start start 2024-05-07T02:31:25.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.530+0000 7f99b7b05b80 1 Processor -- start 2024-05-07T02:31:25.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.530+0000 7f99b7b05b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5608abc64420 0x5608abc6e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:25.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.530+0000 7f99b7b05b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608abc64d70 0x5608abc6e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:25.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.530+0000 7f99b7b05b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5608abc6a360 0x5608abc6ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:25.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.530+0000 7f99b7b05b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5608abc71770 con 0x5608abc6a360 2024-05-07T02:31:25.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.530+0000 7f99b7b05b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5608abaecad0 con 0x5608abc64d70 2024-05-07T02:31:25.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.530+0000 7f99b7b05b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5608abc718f0 con 0x5608abc64420 2024-05-07T02:31:25.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.530+0000 7f99b7b05b80 10 client.0 _ll_register_callbacks cb 0x5608abb2dc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:31:25.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.531+0000 7f99a1400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5608abc64420 0x5608abc6e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:25.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.531+0000 7f99a1400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5608abc64420 0x5608abc6e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:39512/0 (socket says 172.21.15.19:39512) 2024-05-07T02:31:25.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.531+0000 7f99a1400700 1 -- 172.21.15.19:0/2961821588 learned_addr learned my addr 172.21.15.19:0/2961821588 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:31:25.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.531+0000 7f99a1e00700 1 --2- 172.21.15.19:0/2961821588 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608abc64d70 0x5608abc6e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:25.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.531+0000 7f99a1400700 1 --2- 172.21.15.19:0/2961821588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5608abc6a360 0x5608abc6ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:25.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.531+0000 7f99a1e00700 1 -- 172.21.15.19:0/2961821588 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5608abc64420 msgr2=0x5608abc6e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:31:25.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.531+0000 7f99a1e00700 1 --2- 172.21.15.19:0/2961821588 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5608abc64420 0x5608abc6e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:25.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.531+0000 7f99a1e00700 1 -- 172.21.15.19:0/2961821588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5608abc6a360 msgr2=0x5608abc6ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:31:25.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.531+0000 7f99a1e00700 1 --2- 172.21.15.19:0/2961821588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5608abc6a360 0x5608abc6ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:25.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.531+0000 7f99a1e00700 1 -- 172.21.15.19:0/2961821588 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608abb4c3d0 con 0x5608abc64d70 2024-05-07T02:31:25.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.532+0000 7f99a1e00700 1 --2- 172.21.15.19:0/2961821588 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608abc64d70 0x5608abc6e9d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f999400a010 tx=0x7f9994005000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=134149e579b0223e server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:25.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.532+0000 7f9983e00700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:31:25.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.532+0000 7f9983e00700 1 -- 172.21.15.19:0/2961821588 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9994005680 con 0x5608abc64d70 2024-05-07T02:31:25.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.532+0000 7f99b7b05b80 1 -- 172.21.15.19:0/2961821588 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99940054d0 con 0x5608abc64d70 2024-05-07T02:31:25.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.532+0000 7f9983e00700 1 -- 172.21.15.19:0/2961821588 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f9994004020 con 0x5608abc64d70 2024-05-07T02:31:25.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.532+0000 7f9983e00700 1 -- 172.21.15.19:0/2961821588 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9994032050 con 0x5608abc64d70 2024-05-07T02:31:25.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.532+0000 7f99b7b05b80 10 client.25160 fetch_fsmap learned FSMap version 30 2024-05-07T02:31:25.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.532+0000 7f99b7b05b80 1 -- 172.21.15.19:0/2961821588 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5608abc7c340 con 0x5608abc64d70 2024-05-07T02:31:25.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.533+0000 7f9983e00700 1 -- 172.21.15.19:0/2961821588 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99940054d0 con 0x5608abc64d70 2024-05-07T02:31:25.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.533+0000 7f9983e00700 1 -- 172.21.15.19:0/2961821588 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9994003b40 con 0x5608abc64d70 2024-05-07T02:31:25.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.533+0000 7f99b7b05b80 10 client.25160 fetch_fsmap finished waiting for FSMap version 30 2024-05-07T02:31:25.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.533+0000 7f99b7b05b80 10 client.25160 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-07T02:31:25.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.533+0000 7f99b7b05b80 20 client.25160 populate_metadata read hostname 'smithi019' 2024-05-07T02:31:25.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.533+0000 7f99b7b05b80 1 --2- 172.21.15.19:0/2961821588 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x5608abc7e750 0x5608abc7fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:25.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.533+0000 7f99b7b05b80 4 client.25160 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:31:25.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.533+0000 7f99b7b05b80 1 -- 172.21.15.19:0/2961821588 --> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608abc31290 con 0x5608abc7e750 2024-05-07T02:31:25.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.534+0000 7f99a1400700 1 --2- 172.21.15.19:0/2961821588 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x5608abc7e750 0x5608abc7fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:25.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.534+0000 7f99a1400700 1 --2- 172.21.15.19:0/2961821588 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x5608abc7e750 0x5608abc7fbc0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7d973a7430a24c3f server_cookie=f5ae59bb3c54b0d8 in_seq=0 out_seq=0 2024-05-07T02:31:25.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.534+0000 7f9983e00700 10 client.25160 ms_handle_connect on v2:172.21.15.27:6836/2583138143 2024-05-07T02:31:25.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.535+0000 7f9983e00700 1 -- 172.21.15.19:0/2961821588 <== mds.3 v2:172.21.15.27:6836/2583138143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5608abc31290 con 0x5608abc7e750 2024-05-07T02:31:25.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.535+0000 7f9983e00700 10 client.25160 handle_command_reply: tid=0 2024-05-07T02:31:25.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.535+0000 7f99b7b05b80 1 client.25160 shutdown 2024-05-07T02:31:25.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.536+0000 7f99b7b05b80 1 -- 172.21.15.19:0/2961821588 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608abc64d70 msgr2=0x5608abc6e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:25.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.536+0000 7f99b7b05b80 1 --2- 172.21.15.19:0/2961821588 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608abc64d70 0x5608abc6e9d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f999400a010 tx=0x7f9994005000 comp rx=0 tx=0).stop 2024-05-07T02:31:25.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.536+0000 7f99b7b05b80 1 -- 172.21.15.19:0/2961821588 shutdown_connections 2024-05-07T02:31:25.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.536+0000 7f99b7b05b80 1 --2- 172.21.15.19:0/2961821588 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x5608abc7e750 0x5608abc7fbc0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:25.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.536+0000 7f99b7b05b80 1 --2- 172.21.15.19:0/2961821588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5608abc6a360 0x5608abc6ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:25.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.536+0000 7f99b7b05b80 1 --2- 172.21.15.19:0/2961821588 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5608abc64420 0x5608abc6e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:25.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.536+0000 7f99b7b05b80 1 --2- 172.21.15.19:0/2961821588 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608abc64d70 0x5608abc6e9d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:25.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.536+0000 7f99b7b05b80 1 -- 172.21.15.19:0/2961821588 >> 172.21.15.19:0/2961821588 conn(0x5608abae6df0 msgr2=0x5608abc63410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:25.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.536+0000 7f9983e00700 0 client.25160 ms_handle_reset on v2:172.21.15.27:6836/2583138143 2024-05-07T02:31:25.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.536+0000 7f99b7b05b80 1 -- 172.21.15.19:0/2961821588 shutdown_connections 2024-05-07T02:31:25.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.536+0000 7f99b7b05b80 1 -- 172.21.15.19:0/2961821588 wait complete. 2024-05-07T02:31:25.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.536+0000 7f99b7b05b80 20 client.25160 trim_cache size 0 max 16384 2024-05-07T02:31:25.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.560+0000 7f99b7b05b80 1 Processor -- start 2024-05-07T02:31:25.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.560+0000 7f99b7b05b80 1 -- start start 2024-05-07T02:31:25.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.560+0000 7f99b7b05b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5608abc64420 0x5608abc7e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:25.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.560+0000 7f99b7b05b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5608abc64d70 0x5608abc7ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:25.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.560+0000 7f99b7b05b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608abc6a360 0x5608abc7d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:25.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.560+0000 7f99b7b05b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5608abc7df00 con 0x5608abc64d70 2024-05-07T02:31:25.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.560+0000 7f99b7b05b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5608abc7e5d0 con 0x5608abc6a360 2024-05-07T02:31:25.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.560+0000 7f99b7b05b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5608abc7eca0 con 0x5608abc64420 2024-05-07T02:31:25.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.561+0000 7f99a1400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5608abc64420 0x5608abc7e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:25.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.561+0000 7f99a1e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5608abc64d70 0x5608abc7ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:25.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.561+0000 7f99a1400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5608abc64420 0x5608abc7e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:39524/0 (socket says 172.21.15.19:39524) 2024-05-07T02:31:25.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.561+0000 7f99a1400700 1 -- 172.21.15.19:0/1565283099 learned_addr learned my addr 172.21.15.19:0/1565283099 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:31:25.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.561+0000 7f99a1400700 1 --2- 172.21.15.19:0/1565283099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608abc6a360 0x5608abc7d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:25.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.561+0000 7f99a1400700 1 -- 172.21.15.19:0/1565283099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5608abc64420 msgr2=0x5608abc7e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:31:25.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.561+0000 7f99a1400700 1 --2- 172.21.15.19:0/1565283099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5608abc64420 0x5608abc7e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:25.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.561+0000 7f99a1400700 1 -- 172.21.15.19:0/1565283099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5608abc64d70 msgr2=0x5608abc7ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:31:25.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.561+0000 7f99a1400700 1 --2- 172.21.15.19:0/1565283099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5608abc64d70 0x5608abc7ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:25.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.561+0000 7f99a1400700 1 -- 172.21.15.19:0/1565283099 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608abc6ca00 con 0x5608abc6a360 2024-05-07T02:31:25.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.562+0000 7f99a1e00700 1 --2- 172.21.15.19:0/1565283099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5608abc64d70 0x5608abc7ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:31:25.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.562+0000 7f99a1400700 1 --2- 172.21.15.19:0/1565283099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608abc6a360 0x5608abc7d800 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x5608aba1b970 tx=0x7f999000bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30955ce479b67caf server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:25.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.562+0000 7f998ea00700 1 -- 172.21.15.19:0/1565283099 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9990005040 con 0x5608abc6a360 2024-05-07T02:31:25.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.562+0000 7f998ea00700 1 -- 172.21.15.19:0/1565283099 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9990005690 con 0x5608abc6a360 2024-05-07T02:31:25.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.562+0000 7f998ea00700 1 -- 172.21.15.19:0/1565283099 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9990003cd0 con 0x5608abc6a360 2024-05-07T02:31:25.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.563+0000 7f99b7b05b80 1 -- 172.21.15.19:0/1565283099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608abc6a360 msgr2=0x5608abc7d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:25.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.563+0000 7f99b7b05b80 1 --2- 172.21.15.19:0/1565283099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608abc6a360 0x5608abc7d800 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x5608aba1b970 tx=0x7f999000bf60 comp rx=0 tx=0).stop 2024-05-07T02:31:25.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.563+0000 7f99b7b05b80 1 -- 172.21.15.19:0/1565283099 shutdown_connections 2024-05-07T02:31:25.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.563+0000 7f99b7b05b80 1 --2- 172.21.15.19:0/1565283099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608abc6a360 0x5608abc7d800 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:25.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.563+0000 7f99b7b05b80 1 --2- 172.21.15.19:0/1565283099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5608abc64420 0x5608abc7e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:25.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.563+0000 7f99b7b05b80 1 --2- 172.21.15.19:0/1565283099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5608abc64d70 0x5608abc7ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:25.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.563+0000 7f99b7b05b80 1 -- 172.21.15.19:0/1565283099 >> 172.21.15.19:0/1565283099 conn(0x5608abae6df0 msgr2=0x5608abc694e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:25.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.563+0000 7f99b7b05b80 1 -- 172.21.15.19:0/1565283099 shutdown_connections 2024-05-07T02:31:25.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.564+0000 7f99b7b05b80 1 -- 172.21.15.19:0/1565283099 wait complete. 2024-05-07T02:31:25.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.564+0000 7f99b7b05b80 1 -- start start 2024-05-07T02:31:25.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.564+0000 7f99b7b05b80 1 Processor -- start 2024-05-07T02:31:25.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.565+0000 7f99b7b05b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608abc64420 0x5608abc89020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:25.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.565+0000 7f99b7b05b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5608abc64d70 0x5608abc89560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:25.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.565+0000 7f99b7b05b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5608abc6a360 0x5608abc823e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:25.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.565+0000 7f99b7b05b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5608abc7fa40 con 0x5608abc64d70 2024-05-07T02:31:25.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.565+0000 7f99b7b05b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5608abc7f370 con 0x5608abc64420 2024-05-07T02:31:25.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.565+0000 7f99b7b05b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5608abc80110 con 0x5608abc6a360 2024-05-07T02:31:25.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.565+0000 7f99b7b05b80 10 client.0 _ll_register_callbacks cb 0x5608abb2dc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:31:25.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.565+0000 7f99a1e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5608abc64d70 0x5608abc89560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:25.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.565+0000 7f99a1e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5608abc64d70 0x5608abc89560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51714/0 (socket says 172.21.15.19:51714) 2024-05-07T02:31:25.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.565+0000 7f99a1e00700 1 -- 172.21.15.19:0/3796451701 learned_addr learned my addr 172.21.15.19:0/3796451701 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:31:25.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.565+0000 7f99a1400700 1 --2- 172.21.15.19:0/3796451701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608abc64420 0x5608abc89020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:25.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.566+0000 7f99a1400700 1 --2- 172.21.15.19:0/3796451701 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5608abc6a360 0x5608abc823e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:25.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.566+0000 7f99a1e00700 1 -- 172.21.15.19:0/3796451701 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5608abc6a360 msgr2=0x5608abc823e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:31:25.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.566+0000 7f99a1e00700 1 --2- 172.21.15.19:0/3796451701 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5608abc6a360 0x5608abc823e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:25.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.566+0000 7f99a1e00700 1 -- 172.21.15.19:0/3796451701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608abc64420 msgr2=0x5608abc89020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:31:25.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.566+0000 7f99a1e00700 1 --2- 172.21.15.19:0/3796451701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608abc64420 0x5608abc89020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:25.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.566+0000 7f99a1e00700 1 -- 172.21.15.19:0/3796451701 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608abc82b20 con 0x5608abc64d70 2024-05-07T02:31:25.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.566+0000 7f99a1e00700 1 --2- 172.21.15.19:0/3796451701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5608abc64d70 0x5608abc89560 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9994000c00 tx=0x7f9994005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99cad5023fa44385 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:25.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.566+0000 7f9983e00700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-07T02:31:25.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.567+0000 7f9983e00700 1 -- 172.21.15.19:0/3796451701 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9994005580 con 0x5608abc64d70 2024-05-07T02:31:25.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.567+0000 7f99b7b05b80 1 -- 172.21.15.19:0/3796451701 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9990005690 con 0x5608abc64d70 2024-05-07T02:31:25.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.567+0000 7f9983e00700 1 -- 172.21.15.19:0/3796451701 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f999400eb50 con 0x5608abc64d70 2024-05-07T02:31:25.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.567+0000 7f9983e00700 1 -- 172.21.15.19:0/3796451701 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9994047050 con 0x5608abc64d70 2024-05-07T02:31:25.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.567+0000 7f99b7b05b80 10 client.15027 fetch_fsmap learned FSMap version 30 2024-05-07T02:31:25.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.567+0000 7f99b7b05b80 1 -- 172.21.15.19:0/3796451701 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5608abc874b0 con 0x5608abc64d70 2024-05-07T02:31:25.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.567+0000 7f9983e00700 1 -- 172.21.15.19:0/3796451701 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9990005690 con 0x5608abc64d70 2024-05-07T02:31:25.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.568+0000 7f9983e00700 1 -- 172.21.15.19:0/3796451701 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9994003680 con 0x5608abc64d70 2024-05-07T02:31:25.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.568+0000 7f99b7b05b80 10 client.15027 fetch_fsmap finished waiting for FSMap version 30 2024-05-07T02:31:25.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.568+0000 7f99b7b05b80 10 client.15027 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-07T02:31:25.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.568+0000 7f99b7b05b80 20 client.15027 populate_metadata read hostname 'smithi019' 2024-05-07T02:31:25.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.568+0000 7f99b7b05b80 1 --2- 172.21.15.19:0/3796451701 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x5608abceb8d0 0x5608abcecd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:25.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.568+0000 7f99b7b05b80 4 client.15027 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-07T02:31:25.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.568+0000 7f99b7b05b80 1 -- 172.21.15.19:0/3796451701 --> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] -- command(tid 0: {"prefix": "status"}) -- 0x5608abb943b0 con 0x5608abceb8d0 2024-05-07T02:31:25.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.568+0000 7f99a1400700 1 --2- 172.21.15.19:0/3796451701 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x5608abceb8d0 0x5608abcecd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:25.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.569+0000 7f99a1400700 1 --2- 172.21.15.19:0/3796451701 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x5608abceb8d0 0x5608abcecd40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=afe9a78f6a4dee8 server_cookie=27b20ef1171e50ae in_seq=0 out_seq=0 2024-05-07T02:31:25.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.569+0000 7f9983e00700 10 client.15027 ms_handle_connect on v2:172.21.15.27:6836/2583138143 2024-05-07T02:31:25.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.569+0000 7f9983e00700 1 -- 172.21.15.19:0/3796451701 <== mds.3 v2:172.21.15.27:6836/2583138143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5608abb943b0 con 0x5608abceb8d0 2024-05-07T02:31:25.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.569+0000 7f9983e00700 10 client.15027 handle_command_reply: tid=0 2024-05-07T02:31:25.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.569+0000 7f99b7b05b80 1 client.15027 shutdown 2024-05-07T02:31:25.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.569+0000 7f99b7b05b80 1 -- 172.21.15.19:0/3796451701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5608abc64d70 msgr2=0x5608abc89560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:25.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.569+0000 7f99b7b05b80 1 --2- 172.21.15.19:0/3796451701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5608abc64d70 0x5608abc89560 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9994000c00 tx=0x7f9994005e80 comp rx=0 tx=0).stop 2024-05-07T02:31:25.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.569+0000 7f99b7b05b80 1 -- 172.21.15.19:0/3796451701 shutdown_connections 2024-05-07T02:31:25.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.569+0000 7f99b7b05b80 1 --2- 172.21.15.19:0/3796451701 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x5608abceb8d0 0x5608abcecd40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:25.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.569+0000 7f99b7b05b80 1 --2- 172.21.15.19:0/3796451701 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5608abc6a360 0x5608abc823e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:25.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.569+0000 7f99b7b05b80 1 --2- 172.21.15.19:0/3796451701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5608abc64420 0x5608abc89020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:25.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.569+0000 7f99b7b05b80 1 --2- 172.21.15.19:0/3796451701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5608abc64d70 0x5608abc89560 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:25.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.569+0000 7f99b7b05b80 1 -- 172.21.15.19:0/3796451701 >> 172.21.15.19:0/3796451701 conn(0x5608abae6df0 msgr2=0x5608abc694e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:25.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.569+0000 7f9983e00700 0 client.15027 ms_handle_reset on v2:172.21.15.27:6836/2583138143 2024-05-07T02:31:25.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.570+0000 7f99b7b05b80 1 -- 172.21.15.19:0/3796451701 shutdown_connections 2024-05-07T02:31:25.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.570+0000 7f99b7b05b80 1 -- 172.21.15.19:0/3796451701 wait complete. 2024-05-07T02:31:25.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.570+0000 7f99b7b05b80 20 client.15027 trim_cache size 0 max 16384 2024-05-07T02:31:25.573 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:31:25.573 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "9ff84c16-0c17-11ef-bc97-c7b262605968", 2024-05-07T02:31:25.574 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 3, 2024-05-07T02:31:25.574 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24515, 2024-05-07T02:31:25.574 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-05-07T02:31:25.574 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-05-07T02:31:25.574 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-05-07T02:31:25.574 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 223.85337711700001, 2024-05-07T02:31:25.574 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 30, 2024-05-07T02:31:25.574 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 89, 2024-05-07T02:31:25.574 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 87, 2024-05-07T02:31:25.574 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 347.94521104099999 2024-05-07T02:31:25.574 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:31:25.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.571+0000 7f99a9000700 1 -- 172.21.15.19:0/981918582 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f99880b2900 msgr2=0x7f99880b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:25.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.571+0000 7f99a9000700 1 --2- 172.21.15.19:0/981918582 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f99880b2900 0x7f99880b4e10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f999c0090a0 tx=0x7f999c009ed0 comp rx=0 tx=0).stop 2024-05-07T02:31:25.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.571+0000 7f99a9000700 1 -- 172.21.15.19:0/981918582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99a410e900 msgr2=0x7f99a4103c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:25.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.571+0000 7f99a9000700 1 --2- 172.21.15.19:0/981918582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99a410e900 0x7f99a4103c90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f999c00ef30 tx=0x7f999c00c3a0 comp rx=0 tx=0).stop 2024-05-07T02:31:25.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.571+0000 7f99a9000700 1 -- 172.21.15.19:0/981918582 shutdown_connections 2024-05-07T02:31:25.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.571+0000 7f99a9000700 1 --2- 172.21.15.19:0/981918582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f99a410e900 0x7f99a4103c90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:25.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.571+0000 7f99a9000700 1 --2- 172.21.15.19:0/981918582 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f99880b2900 0x7f99880b4e10 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:25.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.571+0000 7f99a9000700 1 --2- 172.21.15.19:0/981918582 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99a4105520 0x7f99a4103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:25.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.571+0000 7f99a9000700 1 --2- 172.21.15.19:0/981918582 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99a4105e70 0x7f99a4103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:25.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.571+0000 7f99a9000700 1 -- 172.21.15.19:0/981918582 >> 172.21.15.19:0/981918582 conn(0x7f99a40fedf0 msgr2=0x7f99a40ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:25.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.571+0000 7f99a9000700 1 -- 172.21.15.19:0/981918582 shutdown_connections 2024-05-07T02:31:25.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:25.571+0000 7f99a9000700 1 -- 172.21.15.19:0/981918582 wait complete. 2024-05-07T02:31:26.615 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.i status 2024-05-07T02:31:27.052 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:31:27.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:26 smithi064 ceph-mon[34699]: pgmap v496: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:31:27.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:26 smithi019 ceph-mon[29781]: pgmap v496: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:31:27.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:26 smithi027 ceph-mon[34398]: pgmap v496: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:31:29.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:28 smithi064 ceph-mon[34699]: pgmap v497: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:31:29.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:28 smithi019 ceph-mon[29781]: pgmap v497: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:31:29.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:28 smithi027 ceph-mon[34398]: pgmap v497: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:31:29.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.602+0000 7f5f72600700 1 -- 172.21.15.19:0/502472809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f6c110c90 msgr2=0x7f5f6c1130b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:29.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.602+0000 7f5f72600700 1 --2- 172.21.15.19:0/502472809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f6c110c90 0x7f5f6c1130b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5f6400b0a0 tx=0x7f5f6400b3b0 comp rx=0 tx=0).stop 2024-05-07T02:31:29.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.602+0000 7f5f72600700 1 -- 172.21.15.19:0/502472809 shutdown_connections 2024-05-07T02:31:29.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.602+0000 7f5f72600700 1 --2- 172.21.15.19:0/502472809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f6c110c90 0x7f5f6c1130b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:29.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.602+0000 7f5f72600700 1 --2- 172.21.15.19:0/502472809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f6c1030e0 0x7f5f6c1034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:29.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.602+0000 7f5f72600700 1 --2- 172.21.15.19:0/502472809 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f6c103a30 0x7f5f6c110750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:29.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.602+0000 7f5f72600700 1 -- 172.21.15.19:0/502472809 >> 172.21.15.19:0/502472809 conn(0x7f5f6c0fedc0 msgr2=0x7f5f6c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:29.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.602+0000 7f5f72600700 1 -- 172.21.15.19:0/502472809 shutdown_connections 2024-05-07T02:31:29.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.602+0000 7f5f72600700 1 -- 172.21.15.19:0/502472809 wait complete. 2024-05-07T02:31:29.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.603+0000 7f5f72600700 1 Processor -- start 2024-05-07T02:31:29.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.604+0000 7f5f72600700 1 -- start start 2024-05-07T02:31:29.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.604+0000 7f5f72600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f6c1030e0 0x7f5f6c19d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:29.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.604+0000 7f5f72600700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f6c103a30 0x7f5f6c19d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:29.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.604+0000 7f5f72600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f6c110c90 0x7f5f6c1a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:29.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.604+0000 7f5f72600700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6c1166d0 con 0x7f5f6c110c90 2024-05-07T02:31:29.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.604+0000 7f5f72600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6c116550 con 0x7f5f6c1030e0 2024-05-07T02:31:29.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.604+0000 7f5f72600700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6c116850 con 0x7f5f6c103a30 2024-05-07T02:31:29.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.604+0000 7f5f6b400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f6c1030e0 0x7f5f6c19d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:29.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.604+0000 7f5f6b400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f6c1030e0 0x7f5f6c19d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:57586/0 (socket says 172.21.15.19:57586) 2024-05-07T02:31:29.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.604+0000 7f5f6b400700 1 -- 172.21.15.19:0/2014856127 learned_addr learned my addr 172.21.15.19:0/2014856127 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:31:29.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.605+0000 7f5f6be00700 1 --2- 172.21.15.19:0/2014856127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f6c110c90 0x7f5f6c1a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:29.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.605+0000 7f5f63e00700 1 --2- 172.21.15.19:0/2014856127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f6c103a30 0x7f5f6c19d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:29.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.605+0000 7f5f6b400700 1 -- 172.21.15.19:0/2014856127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f6c103a30 msgr2=0x7f5f6c19d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:29.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.605+0000 7f5f6b400700 1 --2- 172.21.15.19:0/2014856127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f6c103a30 0x7f5f6c19d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:29.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.605+0000 7f5f6b400700 1 -- 172.21.15.19:0/2014856127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f6c110c90 msgr2=0x7f5f6c1a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:29.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.605+0000 7f5f6b400700 1 --2- 172.21.15.19:0/2014856127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f6c110c90 0x7f5f6c1a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:29.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.605+0000 7f5f6b400700 1 -- 172.21.15.19:0/2014856127 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f6c1a1ed0 con 0x7f5f6c1030e0 2024-05-07T02:31:29.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.605+0000 7f5f6be00700 1 --2- 172.21.15.19:0/2014856127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f6c110c90 0x7f5f6c1a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:31:29.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.605+0000 7f5f6b400700 1 --2- 172.21.15.19:0/2014856127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f6c1030e0 0x7f5f6c19d0a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c00e9a0 tx=0x7f5f5c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:29.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.606+0000 7f5f68c00700 1 -- 172.21.15.19:0/2014856127 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c00ca10 con 0x7f5f6c1030e0 2024-05-07T02:31:29.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.606+0000 7f5f68c00700 1 -- 172.21.15.19:0/2014856127 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f5c00cbb0 con 0x7f5f6c1030e0 2024-05-07T02:31:29.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.606+0000 7f5f68c00700 1 -- 172.21.15.19:0/2014856127 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c018680 con 0x7f5f6c1030e0 2024-05-07T02:31:29.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.606+0000 7f5f72600700 1 -- 172.21.15.19:0/2014856127 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f6c1a21c0 con 0x7f5f6c1030e0 2024-05-07T02:31:29.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.606+0000 7f5f72600700 1 -- 172.21.15.19:0/2014856127 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f6c1a9ae0 con 0x7f5f6c1030e0 2024-05-07T02:31:29.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.612+0000 7f5f68c00700 1 -- 172.21.15.19:0/2014856127 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5f5c010430 con 0x7f5f6c1030e0 2024-05-07T02:31:29.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.612+0000 7f5f811ecb80 1 Processor -- start 2024-05-07T02:31:29.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.612+0000 7f5f811ecb80 1 -- start start 2024-05-07T02:31:29.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.613+0000 7f5f811ecb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56433428f360 0x56433428f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:29.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.613+0000 7f5f811ecb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564334289420 0x564334289830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:29.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.613+0000 7f5f811ecb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564334289d70 0x564334295ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:29.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.613+0000 7f5f811ecb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5643341ab9d0 con 0x56433428f360 2024-05-07T02:31:29.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.613+0000 7f5f811ecb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x564334149710 con 0x564334289d70 2024-05-07T02:31:29.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.613+0000 7f5f811ecb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5643341a7990 con 0x564334289420 2024-05-07T02:31:29.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.613+0000 7f5f6be00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56433428f360 0x56433428f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:29.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.613+0000 7f5f6b400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564334289420 0x564334289830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:29.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.613+0000 7f5f6be00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56433428f360 0x56433428f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53816/0 (socket says 172.21.15.19:53816) 2024-05-07T02:31:29.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.613+0000 7f5f6be00700 1 -- 172.21.15.19:0/2558148054 learned_addr learned my addr 172.21.15.19:0/2558148054 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:31:29.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.614+0000 7f5f63e00700 1 --2- 172.21.15.19:0/2558148054 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564334289d70 0x564334295ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:29.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.614+0000 7f5f6be00700 1 -- 172.21.15.19:0/2558148054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564334289420 msgr2=0x564334289830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:31:29.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.614+0000 7f5f6be00700 1 --2- 172.21.15.19:0/2558148054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564334289420 0x564334289830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:29.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.614+0000 7f5f6be00700 1 -- 172.21.15.19:0/2558148054 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564334289d70 msgr2=0x564334295ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:31:29.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.614+0000 7f5f6be00700 1 --2- 172.21.15.19:0/2558148054 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564334289d70 0x564334295ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:29.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.614+0000 7f5f6be00700 1 -- 172.21.15.19:0/2558148054 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643340f8880 con 0x56433428f360 2024-05-07T02:31:29.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.614+0000 7f5f6be00700 1 --2- 172.21.15.19:0/2558148054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56433428f360 0x56433428f770 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5f6400b0a0 tx=0x7f5f64009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc0a368194f1a6c server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:29.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.615+0000 7f5f60c00700 1 -- 172.21.15.19:0/2558148054 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f64021070 con 0x56433428f360 2024-05-07T02:31:29.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.615+0000 7f5f68c00700 1 --2- 172.21.15.19:0/2014856127 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f5f500b2900 0x7f5f500b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:29.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.615+0000 7f5f60c00700 1 -- 172.21.15.19:0/2558148054 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f640075c0 con 0x56433428f360 2024-05-07T02:31:29.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.615+0000 7f5f60c00700 1 -- 172.21.15.19:0/2558148054 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f64005240 con 0x56433428f360 2024-05-07T02:31:29.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.615+0000 7f5f68c00700 1 -- 172.21.15.19:0/2014856127 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f5f5c0e2710 con 0x7f5f6c1030e0 2024-05-07T02:31:29.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.615+0000 7f5f6b400700 1 --2- 172.21.15.19:0/2014856127 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f5f500b2900 0x7f5f500b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:29.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.616+0000 7f5f811ecb80 1 -- 172.21.15.19:0/2558148054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56433428f360 msgr2=0x56433428f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:29.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.616+0000 7f5f811ecb80 1 --2- 172.21.15.19:0/2558148054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56433428f360 0x56433428f770 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5f6400b0a0 tx=0x7f5f64009d50 comp rx=0 tx=0).stop 2024-05-07T02:31:29.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.616+0000 7f5f6b400700 1 --2- 172.21.15.19:0/2014856127 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f5f500b2900 0x7f5f500b4e10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c0a59a0 tx=0x7f5f5c0a5040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:29.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.616+0000 7f5f811ecb80 1 -- 172.21.15.19:0/2558148054 shutdown_connections 2024-05-07T02:31:29.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.616+0000 7f5f811ecb80 1 --2- 172.21.15.19:0/2558148054 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564334289d70 0x564334295ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:29.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.616+0000 7f5f811ecb80 1 --2- 172.21.15.19:0/2558148054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56433428f360 0x56433428f770 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:29.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.616+0000 7f5f811ecb80 1 --2- 172.21.15.19:0/2558148054 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564334289420 0x564334289830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:29.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.616+0000 7f5f811ecb80 1 -- 172.21.15.19:0/2558148054 >> 172.21.15.19:0/2558148054 conn(0x56433410bdf0 msgr2=0x5643342842a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:29.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.616+0000 7f5f811ecb80 1 -- 172.21.15.19:0/2558148054 shutdown_connections 2024-05-07T02:31:29.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.616+0000 7f5f811ecb80 1 -- 172.21.15.19:0/2558148054 wait complete. 2024-05-07T02:31:29.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.617+0000 7f5f811ecb80 1 -- start start 2024-05-07T02:31:29.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.617+0000 7f5f811ecb80 1 Processor -- start 2024-05-07T02:31:29.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.618+0000 7f5f811ecb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564334289420 0x564334293450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:29.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.618+0000 7f5f811ecb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564334289d70 0x564334293990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:29.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.618+0000 7f5f811ecb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56433428f360 0x564334293ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:29.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.618+0000 7f5f811ecb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564334296710 con 0x564334289420 2024-05-07T02:31:29.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.618+0000 7f5f811ecb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x564334111ad0 con 0x56433428f360 2024-05-07T02:31:29.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.618+0000 7f5f811ecb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564334296890 con 0x564334289d70 2024-05-07T02:31:29.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.618+0000 7f5f811ecb80 10 client.0 _ll_register_callbacks cb 0x564334152c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:31:29.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.618+0000 7f5f6be00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564334289420 0x564334293450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:29.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.618+0000 7f5f63e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564334289d70 0x564334293990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:29.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.618+0000 7f5f63e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564334289d70 0x564334293990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:59158/0 (socket says 172.21.15.19:59158) 2024-05-07T02:31:29.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.618+0000 7f5f6be00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564334289420 0x564334293450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53830/0 (socket says 172.21.15.19:53830) 2024-05-07T02:31:29.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.618+0000 7f5f63e00700 1 -- 172.21.15.19:0/19787007 learned_addr learned my addr 172.21.15.19:0/19787007 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:31:29.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.619+0000 7f5f6be00700 1 --2- 172.21.15.19:0/19787007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56433428f360 0x564334293ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:29.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.619+0000 7f5f63e00700 1 -- 172.21.15.19:0/19787007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56433428f360 msgr2=0x564334293ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:31:29.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.619+0000 7f5f63e00700 1 --2- 172.21.15.19:0/19787007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56433428f360 0x564334293ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:29.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.619+0000 7f5f63e00700 1 -- 172.21.15.19:0/19787007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564334289420 msgr2=0x564334293450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:31:29.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.619+0000 7f5f63e00700 1 --2- 172.21.15.19:0/19787007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564334289420 0x564334293450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:29.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.619+0000 7f5f63e00700 1 -- 172.21.15.19:0/19787007 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643341713d0 con 0x564334289d70 2024-05-07T02:31:29.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.619+0000 7f5f6be00700 1 --2- 172.21.15.19:0/19787007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564334289420 0x564334293450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:31:29.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.619+0000 7f5f63e00700 1 --2- 172.21.15.19:0/19787007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564334289d70 0x564334293990 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5f54012770 tx=0x7f5f54012a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65fed18a60e3e0c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:29.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.619+0000 7f5f59600700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:31:29.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.619+0000 7f5f59600700 1 -- 172.21.15.19:0/19787007 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f54004830 con 0x564334289d70 2024-05-07T02:31:29.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.620+0000 7f5f811ecb80 1 -- 172.21.15.19:0/19787007 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f640075c0 con 0x564334289d70 2024-05-07T02:31:29.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.620+0000 7f5f59600700 1 -- 172.21.15.19:0/19787007 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f5f540049d0 con 0x564334289d70 2024-05-07T02:31:29.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.620+0000 7f5f59600700 1 -- 172.21.15.19:0/19787007 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5f54025050 con 0x564334289d70 2024-05-07T02:31:29.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.620+0000 7f5f811ecb80 10 client.25174 fetch_fsmap learned FSMap version 30 2024-05-07T02:31:29.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.620+0000 7f5f811ecb80 1 -- 172.21.15.19:0/19787007 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5643342a1370 con 0x564334289d70 2024-05-07T02:31:29.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.620+0000 7f5f59600700 1 -- 172.21.15.19:0/19787007 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f640075c0 con 0x564334289d70 2024-05-07T02:31:29.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.620+0000 7f5f59600700 1 -- 172.21.15.19:0/19787007 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5f540106d0 con 0x564334289d70 2024-05-07T02:31:29.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.621+0000 7f5f811ecb80 10 client.25174 fetch_fsmap finished waiting for FSMap version 30 2024-05-07T02:31:29.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.621+0000 7f5f811ecb80 10 client.25174 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-07T02:31:29.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.621+0000 7f5f811ecb80 20 client.25174 populate_metadata read hostname 'smithi019' 2024-05-07T02:31:29.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.621+0000 7f5f811ecb80 1 --2- 172.21.15.19:0/19787007 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x5643342a3940 0x5643342a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:29.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.621+0000 7f5f811ecb80 4 client.25174 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:31:29.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.621+0000 7f5f811ecb80 1 -- 172.21.15.19:0/19787007 --> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564334256290 con 0x5643342a3940 2024-05-07T02:31:29.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.621+0000 7f5f6be00700 1 --2- 172.21.15.19:0/19787007 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x5643342a3940 0x5643342a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:29.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.622+0000 7f5f6be00700 1 --2- 172.21.15.19:0/19787007 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x5643342a3940 0x5643342a4db0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=483e69cc522f49ab server_cookie=c035f9efae832361 in_seq=0 out_seq=0 2024-05-07T02:31:29.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.622+0000 7f5f59600700 10 client.25174 ms_handle_connect on v2:172.21.15.64:6838/1583120004 2024-05-07T02:31:29.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.623+0000 7f5f59600700 1 -- 172.21.15.19:0/19787007 <== mds.2 v2:172.21.15.64:6838/1583120004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564334256290 con 0x5643342a3940 2024-05-07T02:31:29.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.623+0000 7f5f59600700 10 client.25174 handle_command_reply: tid=0 2024-05-07T02:31:29.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.623+0000 7f5f811ecb80 1 client.25174 shutdown 2024-05-07T02:31:29.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.623+0000 7f5f811ecb80 1 -- 172.21.15.19:0/19787007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564334289d70 msgr2=0x564334293990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:29.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.623+0000 7f5f811ecb80 1 --2- 172.21.15.19:0/19787007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564334289d70 0x564334293990 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5f54012770 tx=0x7f5f54012a80 comp rx=0 tx=0).stop 2024-05-07T02:31:29.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.623+0000 7f5f811ecb80 1 -- 172.21.15.19:0/19787007 shutdown_connections 2024-05-07T02:31:29.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.623+0000 7f5f811ecb80 1 --2- 172.21.15.19:0/19787007 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x5643342a3940 0x5643342a4db0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:29.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.623+0000 7f5f811ecb80 1 --2- 172.21.15.19:0/19787007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56433428f360 0x564334293ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:29.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.623+0000 7f5f811ecb80 1 --2- 172.21.15.19:0/19787007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564334289420 0x564334293450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:29.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.623+0000 7f5f59600700 0 client.25174 ms_handle_reset on v2:172.21.15.64:6838/1583120004 2024-05-07T02:31:29.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.623+0000 7f5f811ecb80 1 --2- 172.21.15.19:0/19787007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564334289d70 0x564334293990 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:29.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.623+0000 7f5f811ecb80 1 -- 172.21.15.19:0/19787007 >> 172.21.15.19:0/19787007 conn(0x56433410bdf0 msgr2=0x564334288410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:29.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.623+0000 7f5f811ecb80 1 -- 172.21.15.19:0/19787007 shutdown_connections 2024-05-07T02:31:29.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.623+0000 7f5f811ecb80 1 -- 172.21.15.19:0/19787007 wait complete. 2024-05-07T02:31:29.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.624+0000 7f5f811ecb80 20 client.25174 trim_cache size 0 max 16384 2024-05-07T02:31:29.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.647+0000 7f5f811ecb80 1 Processor -- start 2024-05-07T02:31:29.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.648+0000 7f5f811ecb80 1 -- start start 2024-05-07T02:31:29.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.648+0000 7f5f811ecb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564334289420 0x56433429fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:29.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.648+0000 7f5f811ecb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564334289d70 0x564334293d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:29.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.648+0000 7f5f811ecb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56433428f360 0x56433427e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:29.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.648+0000 7f5f811ecb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5643342a30f0 con 0x564334289420 2024-05-07T02:31:29.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.648+0000 7f5f811ecb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5643342a37c0 con 0x564334289d70 2024-05-07T02:31:29.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.648+0000 7f5f811ecb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5643342a3e90 con 0x56433428f360 2024-05-07T02:31:29.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.648+0000 7f5f6be00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564334289420 0x56433429fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:29.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.648+0000 7f5f6be00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564334289420 0x56433429fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53836/0 (socket says 172.21.15.19:53836) 2024-05-07T02:31:29.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.648+0000 7f5f6be00700 1 -- 172.21.15.19:0/2622154572 learned_addr learned my addr 172.21.15.19:0/2622154572 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:31:29.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.648+0000 7f5f63e00700 1 --2- 172.21.15.19:0/2622154572 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564334289d70 0x564334293d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:29.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.649+0000 7f5f6be00700 1 --2- 172.21.15.19:0/2622154572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56433428f360 0x56433427e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:29.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.649+0000 7f5f6be00700 1 -- 172.21.15.19:0/2622154572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56433428f360 msgr2=0x56433427e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:31:29.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.649+0000 7f5f6be00700 1 --2- 172.21.15.19:0/2622154572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56433428f360 0x56433427e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:29.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.649+0000 7f5f6be00700 1 -- 172.21.15.19:0/2622154572 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564334289d70 msgr2=0x564334293d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:31:29.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.649+0000 7f5f6be00700 1 --2- 172.21.15.19:0/2622154572 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564334289d70 0x564334293d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:29.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.649+0000 7f5f6be00700 1 -- 172.21.15.19:0/2622154572 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564334295cf0 con 0x564334289420 2024-05-07T02:31:29.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.649+0000 7f5f6be00700 1 --2- 172.21.15.19:0/2622154572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564334289420 0x56433429fb60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5f6400b0a0 tx=0x7f5f64006210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9819a765184c9c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:29.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.650+0000 7f5f5be00700 1 -- 172.21.15.19:0/2622154572 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f64021040 con 0x564334289420 2024-05-07T02:31:29.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.650+0000 7f5f5be00700 1 -- 172.21.15.19:0/2622154572 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f6402b040 con 0x564334289420 2024-05-07T02:31:29.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.650+0000 7f5f5be00700 1 -- 172.21.15.19:0/2622154572 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f640318a0 con 0x564334289420 2024-05-07T02:31:29.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.650+0000 7f5f811ecb80 1 -- 172.21.15.19:0/2622154572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564334289420 msgr2=0x56433429fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:29.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.650+0000 7f5f811ecb80 1 --2- 172.21.15.19:0/2622154572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564334289420 0x56433429fb60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5f6400b0a0 tx=0x7f5f64006210 comp rx=0 tx=0).stop 2024-05-07T02:31:29.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.651+0000 7f5f811ecb80 1 -- 172.21.15.19:0/2622154572 shutdown_connections 2024-05-07T02:31:29.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.651+0000 7f5f811ecb80 1 --2- 172.21.15.19:0/2622154572 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56433428f360 0x56433427e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:29.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.651+0000 7f5f811ecb80 1 --2- 172.21.15.19:0/2622154572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564334289420 0x56433429fb60 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:29.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.651+0000 7f5f811ecb80 1 --2- 172.21.15.19:0/2622154572 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564334289d70 0x564334293d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:29.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.651+0000 7f5f811ecb80 1 -- 172.21.15.19:0/2622154572 >> 172.21.15.19:0/2622154572 conn(0x56433410bdf0 msgr2=0x56433427e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:29.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.651+0000 7f5f811ecb80 1 -- 172.21.15.19:0/2622154572 shutdown_connections 2024-05-07T02:31:29.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.651+0000 7f5f811ecb80 1 -- 172.21.15.19:0/2622154572 wait complete. 2024-05-07T02:31:29.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.651+0000 7f5f811ecb80 1 -- start start 2024-05-07T02:31:29.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.651+0000 7f5f811ecb80 1 Processor -- start 2024-05-07T02:31:29.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.652+0000 7f5f811ecb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564334289420 0x564334290130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:29.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.652+0000 7f5f811ecb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564334289d70 0x564334290670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:29.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.652+0000 7f5f811ecb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56433428f360 0x564334290bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:29.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.652+0000 7f5f811ecb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5643342a4c30 con 0x564334289420 2024-05-07T02:31:29.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.652+0000 7f5f811ecb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5643342a4560 con 0x564334289d70 2024-05-07T02:31:29.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.652+0000 7f5f811ecb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5643342a5300 con 0x56433428f360 2024-05-07T02:31:29.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.652+0000 7f5f811ecb80 10 client.0 _ll_register_callbacks cb 0x564334152c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:31:29.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.652+0000 7f5f6be00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564334289420 0x564334290130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:29.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.652+0000 7f5f6be00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564334289420 0x564334290130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53848/0 (socket says 172.21.15.19:53848) 2024-05-07T02:31:29.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.652+0000 7f5f6be00700 1 -- 172.21.15.19:0/4052644434 learned_addr learned my addr 172.21.15.19:0/4052644434 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:31:29.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.652+0000 7f5f63e00700 1 --2- 172.21.15.19:0/4052644434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564334289d70 0x564334290670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:29.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.653+0000 7f5f6be00700 1 -- 172.21.15.19:0/4052644434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56433428f360 msgr2=0x564334290bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:31:29.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.653+0000 7f5f6be00700 1 --2- 172.21.15.19:0/4052644434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56433428f360 0x564334290bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:29.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.653+0000 7f5f6be00700 1 -- 172.21.15.19:0/4052644434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564334289d70 msgr2=0x564334290670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:31:29.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.653+0000 7f5f6be00700 1 --2- 172.21.15.19:0/4052644434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564334289d70 0x564334290670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:29.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.653+0000 7f5f6be00700 1 -- 172.21.15.19:0/4052644434 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643342aec20 con 0x564334289420 2024-05-07T02:31:29.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.653+0000 7f5f63e00700 1 --2- 172.21.15.19:0/4052644434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564334289d70 0x564334290670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:31:29.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.653+0000 7f5f6be00700 1 --2- 172.21.15.19:0/4052644434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564334289420 0x564334290130 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5f64009900 tx=0x7f5f640075c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ed3b6753f924655 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:29.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.653+0000 7f5f59600700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-07T02:31:29.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.654+0000 7f5f59600700 1 -- 172.21.15.19:0/4052644434 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f64021070 con 0x564334289420 2024-05-07T02:31:29.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.654+0000 7f5f811ecb80 1 -- 172.21.15.19:0/4052644434 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f6402b040 con 0x564334289420 2024-05-07T02:31:29.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.654+0000 7f5f59600700 1 -- 172.21.15.19:0/4052644434 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f5f64003c00 con 0x564334289420 2024-05-07T02:31:29.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.654+0000 7f5f59600700 1 -- 172.21.15.19:0/4052644434 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5f64003e00 con 0x564334289420 2024-05-07T02:31:29.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.654+0000 7f5f811ecb80 10 client.15045 fetch_fsmap learned FSMap version 30 2024-05-07T02:31:29.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.654+0000 7f5f811ecb80 1 -- 172.21.15.19:0/4052644434 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56433430ee10 con 0x564334289420 2024-05-07T02:31:29.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.654+0000 7f5f59600700 1 -- 172.21.15.19:0/4052644434 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f6402b040 con 0x564334289420 2024-05-07T02:31:29.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.654+0000 7f5f59600700 1 -- 172.21.15.19:0/4052644434 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5f64011070 con 0x564334289420 2024-05-07T02:31:29.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.655+0000 7f5f811ecb80 10 client.15045 fetch_fsmap finished waiting for FSMap version 30 2024-05-07T02:31:29.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.655+0000 7f5f811ecb80 10 client.15045 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-07T02:31:29.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.655+0000 7f5f811ecb80 20 client.15045 populate_metadata read hostname 'smithi019' 2024-05-07T02:31:29.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.655+0000 7f5f811ecb80 1 --2- 172.21.15.19:0/4052644434 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x564334310ae0 0x564334311f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:29.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.655+0000 7f5f811ecb80 4 client.15045 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-07T02:31:29.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.655+0000 7f5f811ecb80 1 -- 172.21.15.19:0/4052644434 --> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] -- command(tid 0: {"prefix": "status"}) -- 0x5643341b93b0 con 0x564334310ae0 2024-05-07T02:31:29.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.655+0000 7f5f63e00700 1 --2- 172.21.15.19:0/4052644434 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x564334310ae0 0x564334311f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:29.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.656+0000 7f5f63e00700 1 --2- 172.21.15.19:0/4052644434 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x564334310ae0 0x564334311f50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5ad01f7840452615 server_cookie=20b4f2718c5ece28 in_seq=0 out_seq=0 2024-05-07T02:31:29.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.656+0000 7f5f59600700 10 client.15045 ms_handle_connect on v2:172.21.15.64:6838/1583120004 2024-05-07T02:31:29.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.656+0000 7f5f59600700 1 -- 172.21.15.19:0/4052644434 <== mds.2 v2:172.21.15.64:6838/1583120004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5643341b93b0 con 0x564334310ae0 2024-05-07T02:31:29.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.656+0000 7f5f59600700 10 client.15045 handle_command_reply: tid=0 2024-05-07T02:31:29.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.656+0000 7f5f811ecb80 1 client.15045 shutdown 2024-05-07T02:31:29.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.656+0000 7f5f811ecb80 1 -- 172.21.15.19:0/4052644434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564334289420 msgr2=0x564334290130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:29.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.656+0000 7f5f811ecb80 1 --2- 172.21.15.19:0/4052644434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564334289420 0x564334290130 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5f64009900 tx=0x7f5f640075c0 comp rx=0 tx=0).stop 2024-05-07T02:31:29.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.656+0000 7f5f811ecb80 1 -- 172.21.15.19:0/4052644434 shutdown_connections 2024-05-07T02:31:29.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.656+0000 7f5f811ecb80 1 --2- 172.21.15.19:0/4052644434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56433428f360 0x564334290bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:29.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.656+0000 7f5f811ecb80 1 --2- 172.21.15.19:0/4052644434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564334289420 0x564334290130 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:29.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.656+0000 7f5f811ecb80 1 --2- 172.21.15.19:0/4052644434 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x564334310ae0 0x564334311f50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:29.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.656+0000 7f5f811ecb80 1 --2- 172.21.15.19:0/4052644434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564334289d70 0x564334290670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:29.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.656+0000 7f5f811ecb80 1 -- 172.21.15.19:0/4052644434 >> 172.21.15.19:0/4052644434 conn(0x56433410bdf0 msgr2=0x564334288410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:29.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.656+0000 7f5f59600700 0 client.15045 ms_handle_reset on v2:172.21.15.64:6838/1583120004 2024-05-07T02:31:29.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.657+0000 7f5f811ecb80 1 -- 172.21.15.19:0/4052644434 shutdown_connections 2024-05-07T02:31:29.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.657+0000 7f5f811ecb80 1 -- 172.21.15.19:0/4052644434 wait complete. 2024-05-07T02:31:29.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.657+0000 7f5f811ecb80 20 client.15045 trim_cache size 0 max 16384 2024-05-07T02:31:29.659 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:31:29.659 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "9ff84c16-0c17-11ef-bc97-c7b262605968", 2024-05-07T02:31:29.659 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 2, 2024-05-07T02:31:29.659 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24520, 2024-05-07T02:31:29.659 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-05-07T02:31:29.659 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-05-07T02:31:29.659 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-05-07T02:31:29.659 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 228.95198643699999, 2024-05-07T02:31:29.659 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 30, 2024-05-07T02:31:29.660 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 89, 2024-05-07T02:31:29.660 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 87, 2024-05-07T02:31:29.660 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 349.19753046400001 2024-05-07T02:31:29.660 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:31:29.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.657+0000 7f5f72600700 1 -- 172.21.15.19:0/2014856127 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f5f500b2900 msgr2=0x7f5f500b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:29.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.657+0000 7f5f72600700 1 --2- 172.21.15.19:0/2014856127 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f5f500b2900 0x7f5f500b4e10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c0a59a0 tx=0x7f5f5c0a5040 comp rx=0 tx=0).stop 2024-05-07T02:31:29.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.657+0000 7f5f72600700 1 -- 172.21.15.19:0/2014856127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f6c1030e0 msgr2=0x7f5f6c19d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:29.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.657+0000 7f5f72600700 1 --2- 172.21.15.19:0/2014856127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f6c1030e0 0x7f5f6c19d0a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c00e9a0 tx=0x7f5f5c00ecb0 comp rx=0 tx=0).stop 2024-05-07T02:31:29.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.658+0000 7f5f72600700 1 -- 172.21.15.19:0/2014856127 shutdown_connections 2024-05-07T02:31:29.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.658+0000 7f5f72600700 1 --2- 172.21.15.19:0/2014856127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f6c110c90 0x7f5f6c1a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:29.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.658+0000 7f5f72600700 1 --2- 172.21.15.19:0/2014856127 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f5f500b2900 0x7f5f500b4e10 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:29.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.658+0000 7f5f72600700 1 --2- 172.21.15.19:0/2014856127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f6c1030e0 0x7f5f6c19d0a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:29.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.658+0000 7f5f72600700 1 --2- 172.21.15.19:0/2014856127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5f6c103a30 0x7f5f6c19d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:29.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.658+0000 7f5f72600700 1 -- 172.21.15.19:0/2014856127 >> 172.21.15.19:0/2014856127 conn(0x7f5f6c0fedc0 msgr2=0x7f5f6c112780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:29.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.658+0000 7f5f72600700 1 -- 172.21.15.19:0/2014856127 shutdown_connections 2024-05-07T02:31:29.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:29.658+0000 7f5f72600700 1 -- 172.21.15.19:0/2014856127 wait complete. 2024-05-07T02:31:30.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:29 smithi019 ceph-mon[29781]: pgmap v498: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:31:30.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:29 smithi027 ceph-mon[34398]: pgmap v498: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:31:30.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:29 smithi064 ceph-mon[34699]: pgmap v498: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:31:30.644 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.a status 2024-05-07T02:31:31.111 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:31:32.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:32 smithi019 ceph-mon[29781]: pgmap v499: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:31:32.909 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:32 smithi027 ceph-mon[34398]: pgmap v499: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:31:33.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:32 smithi064 ceph-mon[34699]: pgmap v499: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:31:33.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.579+0000 7f955be00700 1 -- 172.21.15.19:0/1580622691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9554078640 msgr2=0x7f9554114cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:33.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.579+0000 7f955be00700 1 --2- 172.21.15.19:0/1580622691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9554078640 0x7f9554114cb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9540009990 tx=0x7f9540009ca0 comp rx=0 tx=0).stop 2024-05-07T02:31:33.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.579+0000 7f955be00700 1 -- 172.21.15.19:0/1580622691 shutdown_connections 2024-05-07T02:31:33.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.579+0000 7f955be00700 1 --2- 172.21.15.19:0/1580622691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f95541151f0 0x7f9554117610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:33.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.579+0000 7f955be00700 1 --2- 172.21.15.19:0/1580622691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9554077cf0 0x7f9554078100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:33.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.579+0000 7f955be00700 1 --2- 172.21.15.19:0/1580622691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9554078640 0x7f9554114cb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:33.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.579+0000 7f955be00700 1 -- 172.21.15.19:0/1580622691 >> 172.21.15.19:0/1580622691 conn(0x7f9554102090 msgr2=0x7f95541044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:33.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.579+0000 7f955be00700 1 -- 172.21.15.19:0/1580622691 shutdown_connections 2024-05-07T02:31:33.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.579+0000 7f955be00700 1 -- 172.21.15.19:0/1580622691 wait complete. 2024-05-07T02:31:33.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.580+0000 7f955be00700 1 Processor -- start 2024-05-07T02:31:33.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.581+0000 7f955be00700 1 -- start start 2024-05-07T02:31:33.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.581+0000 7f955be00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9554077cf0 0x7f95541a1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:33.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.581+0000 7f955be00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9554078640 0x7f95541a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:33.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.581+0000 7f955be00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95541151f0 0x7f95541a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:33.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.581+0000 7f955be00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f955411aab0 con 0x7f95541151f0 2024-05-07T02:31:33.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.581+0000 7f955be00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f955411a930 con 0x7f9554077cf0 2024-05-07T02:31:33.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.581+0000 7f955be00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f955411ac30 con 0x7f9554078640 2024-05-07T02:31:33.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.581+0000 7f9559400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9554077cf0 0x7f95541a1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:33.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.581+0000 7f9559400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9554077cf0 0x7f95541a1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:57630/0 (socket says 172.21.15.19:57630) 2024-05-07T02:31:33.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.581+0000 7f9559400700 1 -- 172.21.15.19:0/3026930430 learned_addr learned my addr 172.21.15.19:0/3026930430 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:31:33.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.582+0000 7f9559e00700 1 --2- 172.21.15.19:0/3026930430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95541151f0 0x7f95541a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:33.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.582+0000 7f9558a00700 1 --2- 172.21.15.19:0/3026930430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9554078640 0x7f95541a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:33.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.582+0000 7f9559400700 1 -- 172.21.15.19:0/3026930430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9554078640 msgr2=0x7f95541a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:33.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.582+0000 7f9559400700 1 --2- 172.21.15.19:0/3026930430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9554078640 0x7f95541a19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:33.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.582+0000 7f9559400700 1 -- 172.21.15.19:0/3026930430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95541151f0 msgr2=0x7f95541a5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:33.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.582+0000 7f9559400700 1 --2- 172.21.15.19:0/3026930430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95541151f0 0x7f95541a5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:33.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.582+0000 7f9559400700 1 -- 172.21.15.19:0/3026930430 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95541a6310 con 0x7f9554077cf0 2024-05-07T02:31:33.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.582+0000 7f9558a00700 1 --2- 172.21.15.19:0/3026930430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9554078640 0x7f95541a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:31:33.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.582+0000 7f9559e00700 1 --2- 172.21.15.19:0/3026930430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95541151f0 0x7f95541a5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:31:33.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.582+0000 7f9559400700 1 --2- 172.21.15.19:0/3026930430 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9554077cf0 0x7f95541a1440 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f954800eb30 tx=0x7f954800ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:33.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.583+0000 7f9552000700 1 -- 172.21.15.19:0/3026930430 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954800cbd0 con 0x7f9554077cf0 2024-05-07T02:31:33.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.583+0000 7f955be00700 1 -- 172.21.15.19:0/3026930430 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95541a6600 con 0x7f9554077cf0 2024-05-07T02:31:33.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.583+0000 7f9552000700 1 -- 172.21.15.19:0/3026930430 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95480044d0 con 0x7f9554077cf0 2024-05-07T02:31:33.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.583+0000 7f9552000700 1 -- 172.21.15.19:0/3026930430 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9548003e30 con 0x7f9554077cf0 2024-05-07T02:31:33.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.583+0000 7f955be00700 1 -- 172.21.15.19:0/3026930430 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95541adf20 con 0x7f9554077cf0 2024-05-07T02:31:33.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.587+0000 7f9552000700 1 -- 172.21.15.19:0/3026930430 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9548004000 con 0x7f9554077cf0 2024-05-07T02:31:33.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.588+0000 7f956a848b80 1 Processor -- start 2024-05-07T02:31:33.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.588+0000 7f956a848b80 1 -- start start 2024-05-07T02:31:33.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.589+0000 7f9552000700 1 --2- 172.21.15.19:0/3026930430 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f953c0b2900 0x7f953c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:33.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.589+0000 7f956a848b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56008d1be420 0x56008d1be830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:33.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.589+0000 7f956a848b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56008d1b85d0 0x56008d1b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:33.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.589+0000 7f956a848b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008d1b8f20 0x56008d1c5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:33.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.589+0000 7f956a848b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56008d0da9d0 con 0x56008d1b8f20 2024-05-07T02:31:33.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.589+0000 7f956a848b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56008d078710 con 0x56008d1b85d0 2024-05-07T02:31:33.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.589+0000 7f956a848b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56008d0d6990 con 0x56008d1be420 2024-05-07T02:31:33.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.589+0000 7f9552000700 1 -- 172.21.15.19:0/3026930430 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f95480e2980 con 0x7f9554077cf0 2024-05-07T02:31:33.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.589+0000 7f9559400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56008d1be420 0x56008d1be830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:33.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.589+0000 7f9559400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56008d1be420 0x56008d1be830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:59212/0 (socket says 172.21.15.19:59212) 2024-05-07T02:31:33.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.589+0000 7f9559400700 1 -- 172.21.15.19:0/3232903063 learned_addr learned my addr 172.21.15.19:0/3232903063 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:31:33.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.589+0000 7f9559e00700 1 --2- 172.21.15.19:0/3232903063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008d1b8f20 0x56008d1c5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:33.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.589+0000 7f9558a00700 1 --2- 172.21.15.19:0/3232903063 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56008d1b85d0 0x56008d1b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:33.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.589+0000 7f9559e00700 1 --2- 172.21.15.19:0/3026930430 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f953c0b2900 0x7f953c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:33.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.590+0000 7f9558a00700 1 -- 172.21.15.19:0/3232903063 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56008d1be420 msgr2=0x56008d1be830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:31:33.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.590+0000 7f9558a00700 1 --2- 172.21.15.19:0/3232903063 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56008d1be420 0x56008d1be830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:33.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.590+0000 7f9558a00700 1 -- 172.21.15.19:0/3232903063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008d1b8f20 msgr2=0x56008d1c5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:31:33.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.590+0000 7f9558a00700 1 --2- 172.21.15.19:0/3232903063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008d1b8f20 0x56008d1c5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:33.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.590+0000 7f9558a00700 1 -- 172.21.15.19:0/3232903063 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56008d027880 con 0x56008d1b85d0 2024-05-07T02:31:33.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.590+0000 7f9559400700 1 --2- 172.21.15.19:0/3232903063 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56008d1be420 0x56008d1be830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:31:33.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.590+0000 7f9559e00700 1 --2- 172.21.15.19:0/3232903063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008d1b8f20 0x56008d1c5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:31:33.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.590+0000 7f9558a00700 1 --2- 172.21.15.19:0/3232903063 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56008d1b85d0 0x56008d1b89e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9540000c00 tx=0x7f9540005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b1fbf789628a174 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:33.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.590+0000 7f9559e00700 1 --2- 172.21.15.19:0/3026930430 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f953c0b2900 0x7f953c0b4e10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f954c009fd0 tx=0x7f954c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:33.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.591+0000 7f9546a00700 1 -- 172.21.15.19:0/3232903063 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9540004830 con 0x56008d1b85d0 2024-05-07T02:31:33.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.591+0000 7f9546a00700 1 -- 172.21.15.19:0/3232903063 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9540005020 con 0x56008d1b85d0 2024-05-07T02:31:33.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.591+0000 7f9546a00700 1 -- 172.21.15.19:0/3232903063 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954000f6c0 con 0x56008d1b85d0 2024-05-07T02:31:33.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.592+0000 7f956a848b80 1 -- 172.21.15.19:0/3232903063 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56008d1b85d0 msgr2=0x56008d1b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:33.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.592+0000 7f956a848b80 1 --2- 172.21.15.19:0/3232903063 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56008d1b85d0 0x56008d1b89e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9540000c00 tx=0x7f9540005e80 comp rx=0 tx=0).stop 2024-05-07T02:31:33.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.592+0000 7f956a848b80 1 -- 172.21.15.19:0/3232903063 shutdown_connections 2024-05-07T02:31:33.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.592+0000 7f956a848b80 1 --2- 172.21.15.19:0/3232903063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008d1b8f20 0x56008d1c5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:33.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.592+0000 7f956a848b80 1 --2- 172.21.15.19:0/3232903063 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56008d1be420 0x56008d1be830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:33.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.592+0000 7f956a848b80 1 --2- 172.21.15.19:0/3232903063 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56008d1b85d0 0x56008d1b89e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:33.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.592+0000 7f956a848b80 1 -- 172.21.15.19:0/3232903063 >> 172.21.15.19:0/3232903063 conn(0x56008d03adf0 msgr2=0x56008d1b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:33.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.592+0000 7f956a848b80 1 -- 172.21.15.19:0/3232903063 shutdown_connections 2024-05-07T02:31:33.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.592+0000 7f956a848b80 1 -- 172.21.15.19:0/3232903063 wait complete. 2024-05-07T02:31:33.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.593+0000 7f956a848b80 1 -- start start 2024-05-07T02:31:33.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.593+0000 7f956a848b80 1 Processor -- start 2024-05-07T02:31:33.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.593+0000 7f956a848b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56008d1b85d0 0x56008d1c2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:33.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.593+0000 7f956a848b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008d1b8f20 0x56008d1c2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:33.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.594+0000 7f956a848b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56008d1be420 0x56008d1c2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:33.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.594+0000 7f956a848b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56008d1c5730 con 0x56008d1b8f20 2024-05-07T02:31:33.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.594+0000 7f956a848b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56008d040ad0 con 0x56008d1b85d0 2024-05-07T02:31:33.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.594+0000 7f956a848b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56008d1c58b0 con 0x56008d1be420 2024-05-07T02:31:33.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.594+0000 7f956a848b80 10 client.0 _ll_register_callbacks cb 0x56008d06e330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:31:33.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.594+0000 7f9559400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56008d1b85d0 0x56008d1c2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:33.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.594+0000 7f9558a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008d1b8f20 0x56008d1c2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:33.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.594+0000 7f9559400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56008d1b85d0 0x56008d1c2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:57660/0 (socket says 172.21.15.19:57660) 2024-05-07T02:31:33.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.594+0000 7f9559400700 1 -- 172.21.15.19:0/449564903 learned_addr learned my addr 172.21.15.19:0/449564903 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:31:33.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.594+0000 7f9559e00700 1 --2- 172.21.15.19:0/449564903 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56008d1be420 0x56008d1c2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:33.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.594+0000 7f9558a00700 1 -- 172.21.15.19:0/449564903 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56008d1be420 msgr2=0x56008d1c2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:31:33.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.594+0000 7f9558a00700 1 --2- 172.21.15.19:0/449564903 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56008d1be420 0x56008d1c2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:33.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.594+0000 7f9558a00700 1 -- 172.21.15.19:0/449564903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56008d1b85d0 msgr2=0x56008d1c2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:31:33.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.594+0000 7f9558a00700 1 --2- 172.21.15.19:0/449564903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56008d1b85d0 0x56008d1c2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:33.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.594+0000 7f9558a00700 1 -- 172.21.15.19:0/449564903 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56008d0a03d0 con 0x56008d1b8f20 2024-05-07T02:31:33.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.595+0000 7f9558a00700 1 --2- 172.21.15.19:0/449564903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008d1b8f20 0x56008d1c2ab0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f95400094b0 tx=0x7f954000bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a473a192b86d6e server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:33.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.595+0000 7f953b400700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-07T02:31:33.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.595+0000 7f953b400700 1 -- 172.21.15.19:0/449564903 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954002f020 con 0x56008d1b8f20 2024-05-07T02:31:33.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.595+0000 7f956a848b80 1 -- 172.21.15.19:0/449564903 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9540005020 con 0x56008d1b8f20 2024-05-07T02:31:33.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.595+0000 7f953b400700 1 -- 172.21.15.19:0/449564903 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f9540004a40 con 0x56008d1b8f20 2024-05-07T02:31:33.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.596+0000 7f953b400700 1 -- 172.21.15.19:0/449564903 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f954003a050 con 0x56008d1b8f20 2024-05-07T02:31:33.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.596+0000 7f956a848b80 10 client.15057 fetch_fsmap learned FSMap version 30 2024-05-07T02:31:33.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.596+0000 7f956a848b80 1 -- 172.21.15.19:0/449564903 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56008d1d02f0 con 0x56008d1b8f20 2024-05-07T02:31:33.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.596+0000 7f953b400700 1 -- 172.21.15.19:0/449564903 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9540005020 con 0x56008d1b8f20 2024-05-07T02:31:33.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.596+0000 7f953b400700 1 -- 172.21.15.19:0/449564903 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9540022440 con 0x56008d1b8f20 2024-05-07T02:31:33.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.596+0000 7f956a848b80 10 client.15057 fetch_fsmap finished waiting for FSMap version 30 2024-05-07T02:31:33.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.596+0000 7f956a848b80 10 client.15057 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-07T02:31:33.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.596+0000 7f956a848b80 20 client.15057 populate_metadata read hostname 'smithi019' 2024-05-07T02:31:33.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.597+0000 7f956a848b80 1 --2- 172.21.15.19:0/449564903 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x56008d1d1640 0x56008d1d37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:33.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.597+0000 7f956a848b80 4 client.15057 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:31:33.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.597+0000 7f956a848b80 1 -- 172.21.15.19:0/449564903 --> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56008d185090 con 0x56008d1d1640 2024-05-07T02:31:33.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.597+0000 7f9559400700 1 --2- 172.21.15.19:0/449564903 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x56008d1d1640 0x56008d1d37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:33.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.597+0000 7f9559400700 1 --2- 172.21.15.19:0/449564903 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x56008d1d1640 0x56008d1d37d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e37bf5bcb0983977 server_cookie=893e840129a1039c in_seq=0 out_seq=0 2024-05-07T02:31:33.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.597+0000 7f953b400700 10 client.15057 ms_handle_connect on v2:172.21.15.19:6840/3905645246 2024-05-07T02:31:33.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.598+0000 7f953b400700 1 -- 172.21.15.19:0/449564903 <== mds.1 v2:172.21.15.19:6840/3905645246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56008d185090 con 0x56008d1d1640 2024-05-07T02:31:33.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.598+0000 7f953b400700 10 client.15057 handle_command_reply: tid=0 2024-05-07T02:31:33.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.598+0000 7f956a848b80 1 client.15057 shutdown 2024-05-07T02:31:33.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.599+0000 7f956a848b80 1 -- 172.21.15.19:0/449564903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008d1b8f20 msgr2=0x56008d1c2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:33.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.599+0000 7f956a848b80 1 --2- 172.21.15.19:0/449564903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008d1b8f20 0x56008d1c2ab0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f95400094b0 tx=0x7f954000bfb0 comp rx=0 tx=0).stop 2024-05-07T02:31:33.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.599+0000 7f956a848b80 1 -- 172.21.15.19:0/449564903 shutdown_connections 2024-05-07T02:31:33.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.599+0000 7f956a848b80 1 --2- 172.21.15.19:0/449564903 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x56008d1d1640 0x56008d1d37d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:33.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.599+0000 7f956a848b80 1 --2- 172.21.15.19:0/449564903 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56008d1be420 0x56008d1c2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:33.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.599+0000 7f956a848b80 1 --2- 172.21.15.19:0/449564903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56008d1b85d0 0x56008d1c2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:33.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.599+0000 7f956a848b80 1 --2- 172.21.15.19:0/449564903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008d1b8f20 0x56008d1c2ab0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:33.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.599+0000 7f956a848b80 1 -- 172.21.15.19:0/449564903 >> 172.21.15.19:0/449564903 conn(0x56008d03adf0 msgr2=0x56008d1b75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:33.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.599+0000 7f953b400700 0 client.15057 ms_handle_reset on v2:172.21.15.19:6840/3905645246 2024-05-07T02:31:33.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.599+0000 7f956a848b80 1 -- 172.21.15.19:0/449564903 shutdown_connections 2024-05-07T02:31:33.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.599+0000 7f956a848b80 1 -- 172.21.15.19:0/449564903 wait complete. 2024-05-07T02:31:33.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.599+0000 7f956a848b80 20 client.15057 trim_cache size 0 max 16384 2024-05-07T02:31:33.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.623+0000 7f956a848b80 1 Processor -- start 2024-05-07T02:31:33.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.623+0000 7f956a848b80 1 -- start start 2024-05-07T02:31:33.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.623+0000 7f956a848b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56008d1b85d0 0x56008d1ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:33.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.623+0000 7f956a848b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008d1b8f20 0x56008d1bcd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:33.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.623+0000 7f956a848b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56008d1be420 0x56008d1dcc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:33.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.623+0000 7f956a848b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56008d1d1b10 con 0x56008d1b8f20 2024-05-07T02:31:33.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.623+0000 7f956a848b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56008d1d21e0 con 0x56008d1b85d0 2024-05-07T02:31:33.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.623+0000 7f956a848b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56008d1d28b0 con 0x56008d1be420 2024-05-07T02:31:33.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.624+0000 7f9559400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56008d1b85d0 0x56008d1ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:33.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.624+0000 7f9559400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56008d1b85d0 0x56008d1ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:57670/0 (socket says 172.21.15.19:57670) 2024-05-07T02:31:33.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.624+0000 7f9559400700 1 -- 172.21.15.19:0/1863335522 learned_addr learned my addr 172.21.15.19:0/1863335522 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:31:33.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.624+0000 7f9558a00700 1 --2- 172.21.15.19:0/1863335522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008d1b8f20 0x56008d1bcd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:33.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.624+0000 7f9559e00700 1 --2- 172.21.15.19:0/1863335522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56008d1be420 0x56008d1dcc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:33.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.624+0000 7f9559400700 1 -- 172.21.15.19:0/1863335522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56008d1be420 msgr2=0x56008d1dcc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:31:33.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.624+0000 7f9559400700 1 --2- 172.21.15.19:0/1863335522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56008d1be420 0x56008d1dcc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:33.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.624+0000 7f9559400700 1 -- 172.21.15.19:0/1863335522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008d1b8f20 msgr2=0x56008d1bcd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:31:33.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.624+0000 7f9559400700 1 --2- 172.21.15.19:0/1863335522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008d1b8f20 0x56008d1bcd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:33.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.624+0000 7f9559400700 1 -- 172.21.15.19:0/1863335522 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56008d1ab500 con 0x56008d1b85d0 2024-05-07T02:31:33.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.625+0000 7f9559400700 1 --2- 172.21.15.19:0/1863335522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56008d1b85d0 0x56008d1ceb80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f95480a9b20 tx=0x7f9548009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f47a7fed5ae35010 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:33.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.625+0000 7f9546000700 1 -- 172.21.15.19:0/1863335522 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95480a96a0 con 0x56008d1b85d0 2024-05-07T02:31:33.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.625+0000 7f9546000700 1 -- 172.21.15.19:0/1863335522 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95480042f0 con 0x56008d1b85d0 2024-05-07T02:31:33.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.625+0000 7f9546000700 1 -- 172.21.15.19:0/1863335522 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954801f340 con 0x56008d1b85d0 2024-05-07T02:31:33.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.626+0000 7f956a848b80 1 -- 172.21.15.19:0/1863335522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56008d1b85d0 msgr2=0x56008d1ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:33.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.626+0000 7f956a848b80 1 --2- 172.21.15.19:0/1863335522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56008d1b85d0 0x56008d1ceb80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f95480a9b20 tx=0x7f9548009f60 comp rx=0 tx=0).stop 2024-05-07T02:31:33.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.626+0000 7f956a848b80 1 -- 172.21.15.19:0/1863335522 shutdown_connections 2024-05-07T02:31:33.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.626+0000 7f956a848b80 1 --2- 172.21.15.19:0/1863335522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56008d1be420 0x56008d1dcc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:33.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.626+0000 7f956a848b80 1 --2- 172.21.15.19:0/1863335522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56008d1b85d0 0x56008d1ceb80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:33.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.626+0000 7f956a848b80 1 --2- 172.21.15.19:0/1863335522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008d1b8f20 0x56008d1bcd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:33.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.626+0000 7f956a848b80 1 -- 172.21.15.19:0/1863335522 >> 172.21.15.19:0/1863335522 conn(0x56008d03adf0 msgr2=0x56008d1bd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:33.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.626+0000 7f956a848b80 1 -- 172.21.15.19:0/1863335522 shutdown_connections 2024-05-07T02:31:33.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.626+0000 7f956a848b80 1 -- 172.21.15.19:0/1863335522 wait complete. 2024-05-07T02:31:33.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.627+0000 7f956a848b80 1 -- start start 2024-05-07T02:31:33.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.627+0000 7f956a848b80 1 Processor -- start 2024-05-07T02:31:33.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.627+0000 7f956a848b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008d1b85d0 0x56008d1dd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:33.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.627+0000 7f956a848b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56008d1b8f20 0x56008d1beff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:33.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.628+0000 7f956a848b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56008d1be420 0x56008d1bf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:33.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.628+0000 7f956a848b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56008d1d3650 con 0x56008d1b85d0 2024-05-07T02:31:33.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.628+0000 7f956a848b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56008d1d2f80 con 0x56008d1be420 2024-05-07T02:31:33.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.628+0000 7f956a848b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56008d1d3d20 con 0x56008d1b8f20 2024-05-07T02:31:33.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.628+0000 7f956a848b80 10 client.0 _ll_register_callbacks cb 0x56008d06e330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:31:33.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.628+0000 7f9559400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008d1b85d0 0x56008d1dd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:33.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.628+0000 7f9559400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008d1b85d0 0x56008d1dd070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53912/0 (socket says 172.21.15.19:53912) 2024-05-07T02:31:33.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.628+0000 7f9559400700 1 -- 172.21.15.19:0/1985624579 learned_addr learned my addr 172.21.15.19:0/1985624579 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:31:33.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.628+0000 7f9559e00700 1 --2- 172.21.15.19:0/1985624579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56008d1be420 0x56008d1bf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:33.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.628+0000 7f9558a00700 1 --2- 172.21.15.19:0/1985624579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56008d1b8f20 0x56008d1beff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:33.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.628+0000 7f9559e00700 1 -- 172.21.15.19:0/1985624579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56008d1b8f20 msgr2=0x56008d1beff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:31:33.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.628+0000 7f9559e00700 1 --2- 172.21.15.19:0/1985624579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56008d1b8f20 0x56008d1beff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:33.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.628+0000 7f9559e00700 1 -- 172.21.15.19:0/1985624579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008d1b85d0 msgr2=0x56008d1dd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:31:33.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.628+0000 7f9559e00700 1 --2- 172.21.15.19:0/1985624579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008d1b85d0 0x56008d1dd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:33.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.629+0000 7f9559e00700 1 -- 172.21.15.19:0/1985624579 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56008d1bfbd0 con 0x56008d1be420 2024-05-07T02:31:33.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.629+0000 7f9559400700 1 --2- 172.21.15.19:0/1985624579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008d1b85d0 0x56008d1dd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:31:33.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.629+0000 7f9559e00700 1 --2- 172.21.15.19:0/1985624579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56008d1be420 0x56008d1bf530 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f954c009550 tx=0x7f954c004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7c4cf99d5229a35 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:33.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.629+0000 7f953b400700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:31:33.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.629+0000 7f953b400700 1 -- 172.21.15.19:0/1985624579 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954c004630 con 0x56008d1be420 2024-05-07T02:31:33.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.629+0000 7f956a848b80 1 -- 172.21.15.19:0/1985624579 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95480042f0 con 0x56008d1be420 2024-05-07T02:31:33.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.629+0000 7f953b400700 1 -- 172.21.15.19:0/1985624579 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f954c0047d0 con 0x56008d1be420 2024-05-07T02:31:33.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.630+0000 7f953b400700 1 -- 172.21.15.19:0/1985624579 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f954c037050 con 0x56008d1be420 2024-05-07T02:31:33.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.630+0000 7f956a848b80 10 client.25202 fetch_fsmap learned FSMap version 30 2024-05-07T02:31:33.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.630+0000 7f956a848b80 1 -- 172.21.15.19:0/1985624579 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56008d23d7d0 con 0x56008d1be420 2024-05-07T02:31:33.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.630+0000 7f953b400700 1 -- 172.21.15.19:0/1985624579 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95480042f0 con 0x56008d1be420 2024-05-07T02:31:33.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.630+0000 7f953b400700 1 -- 172.21.15.19:0/1985624579 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f954c028410 con 0x56008d1be420 2024-05-07T02:31:33.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.630+0000 7f956a848b80 10 client.25202 fetch_fsmap finished waiting for FSMap version 30 2024-05-07T02:31:33.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.630+0000 7f956a848b80 10 client.25202 resolve_mds: resolved name 'a' to daemon mds.a 2024-05-07T02:31:33.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.630+0000 7f956a848b80 20 client.25202 populate_metadata read hostname 'smithi019' 2024-05-07T02:31:33.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.630+0000 7f956a848b80 1 --2- 172.21.15.19:0/1985624579 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x56008d23fa90 0x56008d240f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:33.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.630+0000 7f956a848b80 4 client.25202 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-07T02:31:33.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.630+0000 7f956a848b80 1 -- 172.21.15.19:0/1985624579 --> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] -- command(tid 0: {"prefix": "status"}) -- 0x56008d0e83b0 con 0x56008d23fa90 2024-05-07T02:31:33.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.631+0000 7f9559400700 1 --2- 172.21.15.19:0/1985624579 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x56008d23fa90 0x56008d240f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:33.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.631+0000 7f9559400700 1 --2- 172.21.15.19:0/1985624579 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x56008d23fa90 0x56008d240f00 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=19e2c771879d1783 server_cookie=4d9c418764b49de0 in_seq=0 out_seq=0 2024-05-07T02:31:33.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.631+0000 7f953b400700 10 client.25202 ms_handle_connect on v2:172.21.15.19:6840/3905645246 2024-05-07T02:31:33.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.631+0000 7f953b400700 1 -- 172.21.15.19:0/1985624579 <== mds.1 v2:172.21.15.19:6840/3905645246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56008d0e83b0 con 0x56008d23fa90 2024-05-07T02:31:33.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.632+0000 7f953b400700 10 client.25202 handle_command_reply: tid=0 2024-05-07T02:31:33.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.632+0000 7f956a848b80 1 client.25202 shutdown 2024-05-07T02:31:33.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.632+0000 7f956a848b80 1 -- 172.21.15.19:0/1985624579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56008d1be420 msgr2=0x56008d1bf530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:33.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.632+0000 7f956a848b80 1 --2- 172.21.15.19:0/1985624579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56008d1be420 0x56008d1bf530 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f954c009550 tx=0x7f954c004b40 comp rx=0 tx=0).stop 2024-05-07T02:31:33.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.632+0000 7f956a848b80 1 -- 172.21.15.19:0/1985624579 shutdown_connections 2024-05-07T02:31:33.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.632+0000 7f956a848b80 1 --2- 172.21.15.19:0/1985624579 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x56008d23fa90 0x56008d240f00 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:33.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.632+0000 7f956a848b80 1 --2- 172.21.15.19:0/1985624579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56008d1be420 0x56008d1bf530 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:33.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.632+0000 7f956a848b80 1 --2- 172.21.15.19:0/1985624579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56008d1b85d0 0x56008d1dd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:33.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.632+0000 7f953b400700 0 client.25202 ms_handle_reset on v2:172.21.15.19:6840/3905645246 2024-05-07T02:31:33.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.632+0000 7f956a848b80 1 --2- 172.21.15.19:0/1985624579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56008d1b8f20 0x56008d1beff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:33.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.632+0000 7f956a848b80 1 -- 172.21.15.19:0/1985624579 >> 172.21.15.19:0/1985624579 conn(0x56008d03adf0 msgr2=0x56008d1ad170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:33.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.632+0000 7f956a848b80 1 -- 172.21.15.19:0/1985624579 shutdown_connections 2024-05-07T02:31:33.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.632+0000 7f956a848b80 1 -- 172.21.15.19:0/1985624579 wait complete. 2024-05-07T02:31:33.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.632+0000 7f956a848b80 20 client.25202 trim_cache size 0 max 16384 2024-05-07T02:31:33.635 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:31:33.635 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "9ff84c16-0c17-11ef-bc97-c7b262605968", 2024-05-07T02:31:33.635 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 1, 2024-05-07T02:31:33.635 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24539, 2024-05-07T02:31:33.635 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-05-07T02:31:33.635 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-05-07T02:31:33.635 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-05-07T02:31:33.635 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 233.944992093, 2024-05-07T02:31:33.635 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 30, 2024-05-07T02:31:33.635 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 89, 2024-05-07T02:31:33.635 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 87, 2024-05-07T02:31:33.636 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 343.13464307200002 2024-05-07T02:31:33.636 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:31:33.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.633+0000 7f955be00700 1 -- 172.21.15.19:0/3026930430 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f953c0b2900 msgr2=0x7f953c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:33.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.633+0000 7f955be00700 1 --2- 172.21.15.19:0/3026930430 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f953c0b2900 0x7f953c0b4e10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f954c009fd0 tx=0x7f954c00b040 comp rx=0 tx=0).stop 2024-05-07T02:31:33.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.633+0000 7f955be00700 1 -- 172.21.15.19:0/3026930430 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9554077cf0 msgr2=0x7f95541a1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:33.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.633+0000 7f955be00700 1 --2- 172.21.15.19:0/3026930430 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9554077cf0 0x7f95541a1440 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f954800eb30 tx=0x7f954800ee40 comp rx=0 tx=0).stop 2024-05-07T02:31:33.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.634+0000 7f955be00700 1 -- 172.21.15.19:0/3026930430 shutdown_connections 2024-05-07T02:31:33.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.634+0000 7f955be00700 1 --2- 172.21.15.19:0/3026930430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95541151f0 0x7f95541a5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:33.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.634+0000 7f955be00700 1 --2- 172.21.15.19:0/3026930430 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f953c0b2900 0x7f953c0b4e10 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:33.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.634+0000 7f955be00700 1 --2- 172.21.15.19:0/3026930430 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9554077cf0 0x7f95541a1440 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:33.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.634+0000 7f955be00700 1 --2- 172.21.15.19:0/3026930430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9554078640 0x7f95541a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:33.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.634+0000 7f955be00700 1 -- 172.21.15.19:0/3026930430 >> 172.21.15.19:0/3026930430 conn(0x7f9554102090 msgr2=0x7f95541044b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:33.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.634+0000 7f955be00700 1 -- 172.21.15.19:0/3026930430 shutdown_connections 2024-05-07T02:31:33.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:33.634+0000 7f955be00700 1 -- 172.21.15.19:0/3026930430 wait complete. 2024-05-07T02:31:34.637 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.b status 2024-05-07T02:31:34.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:34 smithi019 ceph-mon[29781]: pgmap v500: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:31:34.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:34 smithi027 ceph-mon[34398]: pgmap v500: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:31:35.105 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:31:35.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:34 smithi064 ceph-mon[34699]: pgmap v500: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:31:36.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:36 smithi019 ceph-mon[29781]: pgmap v501: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:31:36.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:36 smithi027 ceph-mon[34398]: pgmap v501: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:31:37.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:36 smithi064 ceph-mon[34699]: pgmap v501: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:31:37.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.666+0000 7f5a89400700 1 -- 172.21.15.19:0/1120321887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a84107df0 msgr2=0x7f5a84108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:37.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.666+0000 7f5a89400700 1 --2- 172.21.15.19:0/1120321887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a84107df0 0x7f5a84108200 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5a70009990 tx=0x7f5a70009ca0 comp rx=0 tx=0).stop 2024-05-07T02:31:37.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.667+0000 7f5a89400700 1 -- 172.21.15.19:0/1120321887 shutdown_connections 2024-05-07T02:31:37.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.667+0000 7f5a89400700 1 --2- 172.21.15.19:0/1120321887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a84108740 0x7f5a841100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:37.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.667+0000 7f5a89400700 1 --2- 172.21.15.19:0/1120321887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a841034c0 0x7f5a841038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:37.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.667+0000 7f5a89400700 1 --2- 172.21.15.19:0/1120321887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a84107df0 0x7f5a84108200 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:37.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.667+0000 7f5a89400700 1 -- 172.21.15.19:0/1120321887 >> 172.21.15.19:0/1120321887 conn(0x7f5a840fedc0 msgr2=0x7f5a841011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:37.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.667+0000 7f5a89400700 1 -- 172.21.15.19:0/1120321887 shutdown_connections 2024-05-07T02:31:37.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.667+0000 7f5a89400700 1 -- 172.21.15.19:0/1120321887 wait complete. 2024-05-07T02:31:37.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.668+0000 7f5a89400700 1 Processor -- start 2024-05-07T02:31:37.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.668+0000 7f5a89400700 1 -- start start 2024-05-07T02:31:37.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.669+0000 7f5a89400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a841034c0 0x7f5a840ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:37.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.669+0000 7f5a89400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a84107df0 0x7f5a840ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:37.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.669+0000 7f5a89400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a84108740 0x7f5a840ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:37.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.669+0000 7f5a89400700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5a841133a0 con 0x7f5a84107df0 2024-05-07T02:31:37.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.669+0000 7f5a89400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5a84113220 con 0x7f5a84108740 2024-05-07T02:31:37.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.669+0000 7f5a89400700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5a84113520 con 0x7f5a841034c0 2024-05-07T02:31:37.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.669+0000 7f5a82800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a841034c0 0x7f5a840ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:37.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.669+0000 7f5a82800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a841034c0 0x7f5a840ff2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:59258/0 (socket says 172.21.15.19:59258) 2024-05-07T02:31:37.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.669+0000 7f5a82800700 1 -- 172.21.15.19:0/3507221522 learned_addr learned my addr 172.21.15.19:0/3507221522 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:31:37.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.669+0000 7f5a81e00700 1 --2- 172.21.15.19:0/3507221522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a84107df0 0x7f5a840ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:37.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.669+0000 7f5a83200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a84108740 0x7f5a840ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:37.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.670+0000 7f5a82800700 1 -- 172.21.15.19:0/3507221522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a84108740 msgr2=0x7f5a840ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:37.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.670+0000 7f5a82800700 1 --2- 172.21.15.19:0/3507221522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a84108740 0x7f5a840ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:37.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.670+0000 7f5a82800700 1 -- 172.21.15.19:0/3507221522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a84107df0 msgr2=0x7f5a840ff7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:37.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.670+0000 7f5a82800700 1 --2- 172.21.15.19:0/3507221522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a84107df0 0x7f5a840ff7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:37.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.670+0000 7f5a82800700 1 -- 172.21.15.19:0/3507221522 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a841a9280 con 0x7f5a841034c0 2024-05-07T02:31:37.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.670+0000 7f5a82800700 1 --2- 172.21.15.19:0/3507221522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a841034c0 0x7f5a840ff2a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5a7400e9a0 tx=0x7f5a7400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:37.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.674+0000 7f5a7b400700 1 -- 172.21.15.19:0/3507221522 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7400ca10 con 0x7f5a841034c0 2024-05-07T02:31:37.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.675+0000 7f5a7b400700 1 -- 172.21.15.19:0/3507221522 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a7400cbb0 con 0x7f5a841034c0 2024-05-07T02:31:37.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.675+0000 7f5a89400700 1 -- 172.21.15.19:0/3507221522 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a841a9570 con 0x7f5a841034c0 2024-05-07T02:31:37.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.675+0000 7f5a7b400700 1 -- 172.21.15.19:0/3507221522 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a74018680 con 0x7f5a841034c0 2024-05-07T02:31:37.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.675+0000 7f5a89400700 1 -- 172.21.15.19:0/3507221522 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a841a9b00 con 0x7f5a841034c0 2024-05-07T02:31:37.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.680+0000 7f5a7b400700 1 -- 172.21.15.19:0/3507221522 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5a74018820 con 0x7f5a841034c0 2024-05-07T02:31:37.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.681+0000 7f5a97eccb80 1 Processor -- start 2024-05-07T02:31:37.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.682+0000 7f5a97eccb80 1 -- start start 2024-05-07T02:31:37.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.682+0000 7f5a97eccb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630d9db6420 0x5630d9db6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:37.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.682+0000 7f5a97eccb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630d9db05d0 0x5630d9db09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:37.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.682+0000 7f5a97eccb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5630d9db0f20 0x5630d9dbd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:37.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.682+0000 7f5a97eccb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5630d9cd29d0 con 0x5630d9db6420 2024-05-07T02:31:37.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.682+0000 7f5a97eccb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5630d9c70710 con 0x5630d9db05d0 2024-05-07T02:31:37.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.682+0000 7f5a97eccb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5630d9cce990 con 0x5630d9db0f20 2024-05-07T02:31:37.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.682+0000 7f5a83200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630d9db6420 0x5630d9db6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:37.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.683+0000 7f5a83200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630d9db6420 0x5630d9db6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53950/0 (socket says 172.21.15.19:53950) 2024-05-07T02:31:37.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.683+0000 7f5a83200700 1 -- 172.21.15.19:0/3166354086 learned_addr learned my addr 172.21.15.19:0/3166354086 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:31:37.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.683+0000 7f5a82800700 1 --2- 172.21.15.19:0/3166354086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630d9db05d0 0x5630d9db09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:37.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.683+0000 7f5a81e00700 1 --2- 172.21.15.19:0/3166354086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5630d9db0f20 0x5630d9dbd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:37.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.683+0000 7f5a81e00700 1 -- 172.21.15.19:0/3166354086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630d9db05d0 msgr2=0x5630d9db09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:31:37.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.683+0000 7f5a81e00700 1 --2- 172.21.15.19:0/3166354086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630d9db05d0 0x5630d9db09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:37.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.683+0000 7f5a81e00700 1 -- 172.21.15.19:0/3166354086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630d9db6420 msgr2=0x5630d9db6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:31:37.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.683+0000 7f5a81e00700 1 --2- 172.21.15.19:0/3166354086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630d9db6420 0x5630d9db6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:37.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.683+0000 7f5a81e00700 1 -- 172.21.15.19:0/3166354086 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630d9c1f880 con 0x5630d9db0f20 2024-05-07T02:31:37.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.683+0000 7f5a7b400700 1 --2- 172.21.15.19:0/3507221522 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f5a680b2900 0x7f5a680b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:37.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.684+0000 7f5a81e00700 1 --2- 172.21.15.19:0/3166354086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5630d9db0f20 0x5630d9dbd100 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5a700098e0 tx=0x7f5a70005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52eff3b3c312e9b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:37.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.684+0000 7f5a7b400700 1 -- 172.21.15.19:0/3507221522 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f5a740e2750 con 0x7f5a841034c0 2024-05-07T02:31:37.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.684+0000 7f5a83200700 1 --2- 172.21.15.19:0/3507221522 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f5a680b2900 0x7f5a680b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:37.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.684+0000 7f5a6fe00700 1 -- 172.21.15.19:0/3166354086 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a70027070 con 0x5630d9db0f20 2024-05-07T02:31:37.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.684+0000 7f5a6fe00700 1 -- 172.21.15.19:0/3166354086 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a700056b0 con 0x5630d9db0f20 2024-05-07T02:31:37.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.684+0000 7f5a6fe00700 1 -- 172.21.15.19:0/3166354086 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a70003ae0 con 0x5630d9db0f20 2024-05-07T02:31:37.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.685+0000 7f5a97eccb80 1 -- 172.21.15.19:0/3166354086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5630d9db0f20 msgr2=0x5630d9dbd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:37.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.685+0000 7f5a97eccb80 1 --2- 172.21.15.19:0/3166354086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5630d9db0f20 0x5630d9dbd100 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5a700098e0 tx=0x7f5a70005f00 comp rx=0 tx=0).stop 2024-05-07T02:31:37.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.685+0000 7f5a83200700 1 --2- 172.21.15.19:0/3507221522 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f5a680b2900 0x7f5a680b4e10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x5630d9db5fa0 tx=0x7f5a7c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:37.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.685+0000 7f5a97eccb80 1 -- 172.21.15.19:0/3166354086 shutdown_connections 2024-05-07T02:31:37.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.685+0000 7f5a97eccb80 1 --2- 172.21.15.19:0/3166354086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5630d9db0f20 0x5630d9dbd100 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:37.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.685+0000 7f5a97eccb80 1 --2- 172.21.15.19:0/3166354086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630d9db6420 0x5630d9db6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:37.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.685+0000 7f5a97eccb80 1 --2- 172.21.15.19:0/3166354086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630d9db05d0 0x5630d9db09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:37.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.685+0000 7f5a97eccb80 1 -- 172.21.15.19:0/3166354086 >> 172.21.15.19:0/3166354086 conn(0x5630d9c32df0 msgr2=0x5630d9dab330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:37.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.685+0000 7f5a97eccb80 1 -- 172.21.15.19:0/3166354086 shutdown_connections 2024-05-07T02:31:37.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.686+0000 7f5a97eccb80 1 -- 172.21.15.19:0/3166354086 wait complete. 2024-05-07T02:31:37.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.686+0000 7f5a97eccb80 1 -- start start 2024-05-07T02:31:37.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.686+0000 7f5a97eccb80 1 Processor -- start 2024-05-07T02:31:37.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.687+0000 7f5a97eccb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630d9db05d0 0x5630d9dba5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:37.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.687+0000 7f5a97eccb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5630d9db0f20 0x5630d9dbab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:37.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.687+0000 7f5a97eccb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630d9db6420 0x5630d9dbb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:37.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.687+0000 7f5a97eccb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5630d9dbd790 con 0x5630d9db6420 2024-05-07T02:31:37.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.687+0000 7f5a97eccb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5630d9c38ad0 con 0x5630d9db05d0 2024-05-07T02:31:37.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.687+0000 7f5a97eccb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5630d9dbd910 con 0x5630d9db0f20 2024-05-07T02:31:37.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.687+0000 7f5a97eccb80 10 client.0 _ll_register_callbacks cb 0x5630d9c66330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:31:37.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.687+0000 7f5a82800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630d9db05d0 0x5630d9dba5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:37.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.687+0000 7f5a82800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630d9db05d0 0x5630d9dba5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:57720/0 (socket says 172.21.15.19:57720) 2024-05-07T02:31:37.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.687+0000 7f5a82800700 1 -- 172.21.15.19:0/2198875953 learned_addr learned my addr 172.21.15.19:0/2198875953 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:31:37.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.687+0000 7f5a83200700 1 --2- 172.21.15.19:0/2198875953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630d9db6420 0x5630d9dbb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:37.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.688+0000 7f5a81e00700 1 --2- 172.21.15.19:0/2198875953 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5630d9db0f20 0x5630d9dbab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:37.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.688+0000 7f5a82800700 1 -- 172.21.15.19:0/2198875953 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5630d9db0f20 msgr2=0x5630d9dbab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:31:37.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.688+0000 7f5a82800700 1 --2- 172.21.15.19:0/2198875953 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5630d9db0f20 0x5630d9dbab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:37.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.688+0000 7f5a82800700 1 -- 172.21.15.19:0/2198875953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630d9db6420 msgr2=0x5630d9dbb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:31:37.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.688+0000 7f5a82800700 1 --2- 172.21.15.19:0/2198875953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630d9db6420 0x5630d9dbb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:37.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.688+0000 7f5a82800700 1 -- 172.21.15.19:0/2198875953 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630d9c983d0 con 0x5630d9db05d0 2024-05-07T02:31:37.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.688+0000 7f5a83200700 1 --2- 172.21.15.19:0/2198875953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630d9db6420 0x5630d9dbb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:31:37.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.688+0000 7f5a82800700 1 --2- 172.21.15.19:0/2198875953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630d9db05d0 0x5630d9dba5f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5a740a98f0 tx=0x7f5a740a58f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35f6bf31dd2a951b server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:37.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.688+0000 7f5a6cc00700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:31:37.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.689+0000 7f5a6cc00700 1 -- 172.21.15.19:0/2198875953 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a740a92b0 con 0x5630d9db05d0 2024-05-07T02:31:37.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.689+0000 7f5a6cc00700 1 -- 172.21.15.19:0/2198875953 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f5a740e2400 con 0x5630d9db05d0 2024-05-07T02:31:37.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.689+0000 7f5a97eccb80 1 -- 172.21.15.19:0/2198875953 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a700056b0 con 0x5630d9db05d0 2024-05-07T02:31:37.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.689+0000 7f5a6cc00700 1 -- 172.21.15.19:0/2198875953 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5a740c3050 con 0x5630d9db05d0 2024-05-07T02:31:37.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.689+0000 7f5a97eccb80 10 client.25214 fetch_fsmap learned FSMap version 30 2024-05-07T02:31:37.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.689+0000 7f5a97eccb80 1 -- 172.21.15.19:0/2198875953 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5630d9dc83c0 con 0x5630d9db05d0 2024-05-07T02:31:37.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.689+0000 7f5a6cc00700 1 -- 172.21.15.19:0/2198875953 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a700056b0 con 0x5630d9db05d0 2024-05-07T02:31:37.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.690+0000 7f5a6cc00700 1 -- 172.21.15.19:0/2198875953 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5a740cc0f0 con 0x5630d9db05d0 2024-05-07T02:31:37.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.690+0000 7f5a97eccb80 10 client.25214 fetch_fsmap finished waiting for FSMap version 30 2024-05-07T02:31:37.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.690+0000 7f5a97eccb80 10 client.25214 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-07T02:31:37.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.690+0000 7f5a97eccb80 20 client.25214 populate_metadata read hostname 'smithi019' 2024-05-07T02:31:37.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.690+0000 7f5a97eccb80 1 --2- 172.21.15.19:0/2198875953 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5630d9dca9c0 0x5630d9dcbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:37.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.690+0000 7f5a97eccb80 4 client.25214 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:31:37.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.690+0000 7f5a97eccb80 1 -- 172.21.15.19:0/2198875953 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630d9d7d090 con 0x5630d9dca9c0 2024-05-07T02:31:37.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.690+0000 7f5a81e00700 1 --2- 172.21.15.19:0/2198875953 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5630d9dca9c0 0x5630d9dcbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:37.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.691+0000 7f5a81e00700 1 --2- 172.21.15.19:0/2198875953 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5630d9dca9c0 0x5630d9dcbe30 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=516c507ef6819bd4 server_cookie=87e00da76dc2ffd6 in_seq=0 out_seq=0 2024-05-07T02:31:37.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.691+0000 7f5a6cc00700 10 client.25214 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:31:37.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.692+0000 7f5a6cc00700 1 -- 172.21.15.19:0/2198875953 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5630d9d7d090 con 0x5630d9dca9c0 2024-05-07T02:31:37.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.692+0000 7f5a6cc00700 10 client.25214 handle_command_reply: tid=0 2024-05-07T02:31:37.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.692+0000 7f5a97eccb80 1 client.25214 shutdown 2024-05-07T02:31:37.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.692+0000 7f5a97eccb80 1 -- 172.21.15.19:0/2198875953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630d9db05d0 msgr2=0x5630d9dba5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:37.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.692+0000 7f5a97eccb80 1 --2- 172.21.15.19:0/2198875953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630d9db05d0 0x5630d9dba5f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5a740a98f0 tx=0x7f5a740a58f0 comp rx=0 tx=0).stop 2024-05-07T02:31:37.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.692+0000 7f5a97eccb80 1 -- 172.21.15.19:0/2198875953 shutdown_connections 2024-05-07T02:31:37.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.692+0000 7f5a97eccb80 1 --2- 172.21.15.19:0/2198875953 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5630d9dca9c0 0x5630d9dcbe30 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:37.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.692+0000 7f5a97eccb80 1 --2- 172.21.15.19:0/2198875953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630d9db6420 0x5630d9dbb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:37.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.692+0000 7f5a6cc00700 0 client.25214 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:31:37.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.692+0000 7f5a97eccb80 1 --2- 172.21.15.19:0/2198875953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630d9db05d0 0x5630d9dba5f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:37.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.692+0000 7f5a97eccb80 1 --2- 172.21.15.19:0/2198875953 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5630d9db0f20 0x5630d9dbab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:37.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.692+0000 7f5a97eccb80 1 -- 172.21.15.19:0/2198875953 >> 172.21.15.19:0/2198875953 conn(0x5630d9c32df0 msgr2=0x5630d9daf5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:37.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.693+0000 7f5a97eccb80 1 -- 172.21.15.19:0/2198875953 shutdown_connections 2024-05-07T02:31:37.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.693+0000 7f5a97eccb80 1 -- 172.21.15.19:0/2198875953 wait complete. 2024-05-07T02:31:37.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.693+0000 7f5a97eccb80 20 client.25214 trim_cache size 0 max 16384 2024-05-07T02:31:37.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.716+0000 7f5a97eccb80 1 Processor -- start 2024-05-07T02:31:37.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.716+0000 7f5a97eccb80 1 -- start start 2024-05-07T02:31:37.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.717+0000 7f5a97eccb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5630d9db05d0 0x5630d9dac9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:37.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.717+0000 7f5a97eccb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630d9db0f20 0x5630d9db4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:37.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.717+0000 7f5a97eccb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630d9db6420 0x5630d9db95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:37.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.717+0000 7f5a97eccb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5630d9dca170 con 0x5630d9db0f20 2024-05-07T02:31:37.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.717+0000 7f5a97eccb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5630d9dca840 con 0x5630d9db6420 2024-05-07T02:31:37.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.717+0000 7f5a97eccb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5630d9dcaf10 con 0x5630d9db05d0 2024-05-07T02:31:37.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.717+0000 7f5a81e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630d9db0f20 0x5630d9db4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:37.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.717+0000 7f5a81e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630d9db0f20 0x5630d9db4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53972/0 (socket says 172.21.15.19:53972) 2024-05-07T02:31:37.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.717+0000 7f5a81e00700 1 -- 172.21.15.19:0/1908325190 learned_addr learned my addr 172.21.15.19:0/1908325190 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:31:37.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.717+0000 7f5a83200700 1 --2- 172.21.15.19:0/1908325190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630d9db6420 0x5630d9db95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:37.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.717+0000 7f5a82800700 1 --2- 172.21.15.19:0/1908325190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5630d9db05d0 0x5630d9dac9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:37.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.718+0000 7f5a81e00700 1 -- 172.21.15.19:0/1908325190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5630d9db05d0 msgr2=0x5630d9dac9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:31:37.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.718+0000 7f5a81e00700 1 --2- 172.21.15.19:0/1908325190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5630d9db05d0 0x5630d9dac9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:37.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.718+0000 7f5a81e00700 1 -- 172.21.15.19:0/1908325190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630d9db6420 msgr2=0x5630d9db95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:31:37.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.718+0000 7f5a81e00700 1 --2- 172.21.15.19:0/1908325190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630d9db6420 0x5630d9db95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:37.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.718+0000 7f5a81e00700 1 -- 172.21.15.19:0/1908325190 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630d9da7b30 con 0x5630d9db0f20 2024-05-07T02:31:37.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.718+0000 7f5a81e00700 1 --2- 172.21.15.19:0/1908325190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630d9db0f20 0x5630d9db4d40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5a70006010 tx=0x7f5a7001a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa06cc9ea0aefce9 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:37.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.718+0000 7f5a6f400700 1 -- 172.21.15.19:0/1908325190 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a70027040 con 0x5630d9db0f20 2024-05-07T02:31:37.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.718+0000 7f5a6f400700 1 -- 172.21.15.19:0/1908325190 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a70005690 con 0x5630d9db0f20 2024-05-07T02:31:37.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.718+0000 7f5a6f400700 1 -- 172.21.15.19:0/1908325190 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a70032160 con 0x5630d9db0f20 2024-05-07T02:31:37.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.719+0000 7f5a97eccb80 1 -- 172.21.15.19:0/1908325190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630d9db0f20 msgr2=0x5630d9db4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:37.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.719+0000 7f5a97eccb80 1 --2- 172.21.15.19:0/1908325190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630d9db0f20 0x5630d9db4d40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5a70006010 tx=0x7f5a7001a040 comp rx=0 tx=0).stop 2024-05-07T02:31:37.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.719+0000 7f5a97eccb80 1 -- 172.21.15.19:0/1908325190 shutdown_connections 2024-05-07T02:31:37.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.719+0000 7f5a97eccb80 1 --2- 172.21.15.19:0/1908325190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630d9db6420 0x5630d9db95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:37.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.719+0000 7f5a97eccb80 1 --2- 172.21.15.19:0/1908325190 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5630d9db05d0 0x5630d9dac9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:37.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.719+0000 7f5a97eccb80 1 --2- 172.21.15.19:0/1908325190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630d9db0f20 0x5630d9db4d40 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:37.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.719+0000 7f5a97eccb80 1 -- 172.21.15.19:0/1908325190 >> 172.21.15.19:0/1908325190 conn(0x5630d9c32df0 msgr2=0x5630d9db8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:37.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.720+0000 7f5a97eccb80 1 -- 172.21.15.19:0/1908325190 shutdown_connections 2024-05-07T02:31:37.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.720+0000 7f5a97eccb80 1 -- 172.21.15.19:0/1908325190 wait complete. 2024-05-07T02:31:37.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.720+0000 7f5a97eccb80 1 -- start start 2024-05-07T02:31:37.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.720+0000 7f5a97eccb80 1 Processor -- start 2024-05-07T02:31:37.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.721+0000 7f5a97eccb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5630d9db05d0 0x5630d9db6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:37.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.721+0000 7f5a97eccb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630d9db0f20 0x5630d9db7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:37.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.721+0000 7f5a97eccb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630d9db6420 0x5630d9dd42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:37.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.721+0000 7f5a97eccb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5630d9dcbcb0 con 0x5630d9db0f20 2024-05-07T02:31:37.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.721+0000 7f5a97eccb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5630d9dcb5e0 con 0x5630d9db6420 2024-05-07T02:31:37.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.721+0000 7f5a97eccb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5630d9dcc380 con 0x5630d9db05d0 2024-05-07T02:31:37.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.721+0000 7f5a97eccb80 10 client.0 _ll_register_callbacks cb 0x5630d9c66330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:31:37.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.721+0000 7f5a81e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630d9db0f20 0x5630d9db7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:37.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.721+0000 7f5a82800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5630d9db05d0 0x5630d9db6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:37.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.721+0000 7f5a82800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5630d9db05d0 0x5630d9db6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:59300/0 (socket says 172.21.15.19:59300) 2024-05-07T02:31:37.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.721+0000 7f5a82800700 1 -- 172.21.15.19:0/1618366765 learned_addr learned my addr 172.21.15.19:0/1618366765 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:31:37.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.721+0000 7f5a81e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630d9db0f20 0x5630d9db7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53984/0 (socket says 172.21.15.19:53984) 2024-05-07T02:31:37.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.721+0000 7f5a83200700 1 --2- 172.21.15.19:0/1618366765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630d9db6420 0x5630d9dd42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:37.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.722+0000 7f5a82800700 1 -- 172.21.15.19:0/1618366765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630d9db6420 msgr2=0x5630d9dd42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:31:37.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.722+0000 7f5a82800700 1 --2- 172.21.15.19:0/1618366765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630d9db6420 0x5630d9dd42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:37.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.722+0000 7f5a82800700 1 -- 172.21.15.19:0/1618366765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630d9db0f20 msgr2=0x5630d9db7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:31:37.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.722+0000 7f5a82800700 1 --2- 172.21.15.19:0/1618366765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630d9db0f20 0x5630d9db7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:37.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.722+0000 7f5a82800700 1 -- 172.21.15.19:0/1618366765 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630d9dac2f0 con 0x5630d9db05d0 2024-05-07T02:31:37.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.722+0000 7f5a82800700 1 --2- 172.21.15.19:0/1618366765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5630d9db05d0 0x5630d9db6e30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5a740a59e0 tx=0x7f5a74009790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30bfb62a31a2db71 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:37.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.722+0000 7f5a6cc00700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:31:37.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.722+0000 7f5a6cc00700 1 -- 172.21.15.19:0/1618366765 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a740a50f0 con 0x5630d9db05d0 2024-05-07T02:31:37.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.723+0000 7f5a6cc00700 1 -- 172.21.15.19:0/1618366765 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f5a740a8dd0 con 0x5630d9db05d0 2024-05-07T02:31:37.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.723+0000 7f5a97eccb80 1 -- 172.21.15.19:0/1618366765 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a70005690 con 0x5630d9db05d0 2024-05-07T02:31:37.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.723+0000 7f5a6cc00700 1 -- 172.21.15.19:0/1618366765 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5a740a90d0 con 0x5630d9db05d0 2024-05-07T02:31:37.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.723+0000 7f5a97eccb80 10 client.25213 fetch_fsmap learned FSMap version 30 2024-05-07T02:31:37.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.723+0000 7f5a97eccb80 1 -- 172.21.15.19:0/1618366765 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5630d9dd5640 con 0x5630d9db05d0 2024-05-07T02:31:37.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.723+0000 7f5a6cc00700 1 -- 172.21.15.19:0/1618366765 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a70005690 con 0x5630d9db05d0 2024-05-07T02:31:37.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.723+0000 7f5a6cc00700 1 -- 172.21.15.19:0/1618366765 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f5a740a8630 con 0x5630d9db05d0 2024-05-07T02:31:37.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.723+0000 7f5a97eccb80 10 client.25213 fetch_fsmap finished waiting for FSMap version 30 2024-05-07T02:31:37.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.723+0000 7f5a97eccb80 10 client.25213 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-07T02:31:37.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.723+0000 7f5a97eccb80 20 client.25213 populate_metadata read hostname 'smithi019' 2024-05-07T02:31:37.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.724+0000 7f5a97eccb80 1 --2- 172.21.15.19:0/1618366765 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5630d9e37a10 0x5630d9e38e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:37.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.724+0000 7f5a97eccb80 4 client.25213 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-07T02:31:37.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.724+0000 7f5a97eccb80 1 -- 172.21.15.19:0/1618366765 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "status"}) -- 0x5630d9ce03b0 con 0x5630d9e37a10 2024-05-07T02:31:37.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.724+0000 7f5a81e00700 1 --2- 172.21.15.19:0/1618366765 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5630d9e37a10 0x5630d9e38e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:37.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.724+0000 7f5a81e00700 1 --2- 172.21.15.19:0/1618366765 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5630d9e37a10 0x5630d9e38e80 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=969bd7a1c866a0dc server_cookie=82027dfedb33f7f7 in_seq=0 out_seq=0 2024-05-07T02:31:37.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.724+0000 7f5a6cc00700 10 client.25213 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:31:37.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.725+0000 7f5a6cc00700 1 -- 172.21.15.19:0/1618366765 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5630d9ce03b0 con 0x5630d9e37a10 2024-05-07T02:31:37.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.725+0000 7f5a6cc00700 10 client.25213 handle_command_reply: tid=0 2024-05-07T02:31:37.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.725+0000 7f5a97eccb80 1 client.25213 shutdown 2024-05-07T02:31:37.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.725+0000 7f5a97eccb80 1 -- 172.21.15.19:0/1618366765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5630d9db05d0 msgr2=0x5630d9db6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:37.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.725+0000 7f5a97eccb80 1 --2- 172.21.15.19:0/1618366765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5630d9db05d0 0x5630d9db6e30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5a740a59e0 tx=0x7f5a74009790 comp rx=0 tx=0).stop 2024-05-07T02:31:37.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.725+0000 7f5a97eccb80 1 -- 172.21.15.19:0/1618366765 shutdown_connections 2024-05-07T02:31:37.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.725+0000 7f5a97eccb80 1 --2- 172.21.15.19:0/1618366765 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5630d9e37a10 0x5630d9e38e80 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:37.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.725+0000 7f5a97eccb80 1 --2- 172.21.15.19:0/1618366765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5630d9db6420 0x5630d9dd42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:37.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.725+0000 7f5a97eccb80 1 --2- 172.21.15.19:0/1618366765 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5630d9db05d0 0x5630d9db6e30 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:37.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.725+0000 7f5a97eccb80 1 --2- 172.21.15.19:0/1618366765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5630d9db0f20 0x5630d9db7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:37.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.725+0000 7f5a97eccb80 1 -- 172.21.15.19:0/1618366765 >> 172.21.15.19:0/1618366765 conn(0x5630d9c32df0 msgr2=0x5630d9da5170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:37.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.725+0000 7f5a6cc00700 0 client.25213 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:31:37.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.725+0000 7f5a97eccb80 1 -- 172.21.15.19:0/1618366765 shutdown_connections 2024-05-07T02:31:37.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.725+0000 7f5a97eccb80 1 -- 172.21.15.19:0/1618366765 wait complete. 2024-05-07T02:31:37.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.726+0000 7f5a97eccb80 20 client.25213 trim_cache size 0 max 16384 2024-05-07T02:31:37.728 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:31:37.728 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "9ff84c16-0c17-11ef-bc97-c7b262605968", 2024-05-07T02:31:37.728 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-05-07T02:31:37.728 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24551, 2024-05-07T02:31:37.728 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-05-07T02:31:37.728 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-05-07T02:31:37.728 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-05-07T02:31:37.728 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 322.70881768300001, 2024-05-07T02:31:37.728 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 30, 2024-05-07T02:31:37.729 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 89, 2024-05-07T02:31:37.729 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 87, 2024-05-07T02:31:37.729 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 344.10319175000001 2024-05-07T02:31:37.729 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:31:37.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.726+0000 7f5a89400700 1 -- 172.21.15.19:0/3507221522 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f5a680b2900 msgr2=0x7f5a680b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:37.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.726+0000 7f5a89400700 1 --2- 172.21.15.19:0/3507221522 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f5a680b2900 0x7f5a680b4e10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x5630d9db5fa0 tx=0x7f5a7c00b040 comp rx=0 tx=0).stop 2024-05-07T02:31:37.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.726+0000 7f5a89400700 1 -- 172.21.15.19:0/3507221522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a841034c0 msgr2=0x7f5a840ff2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:37.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.726+0000 7f5a89400700 1 --2- 172.21.15.19:0/3507221522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a841034c0 0x7f5a840ff2a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5a7400e9a0 tx=0x7f5a7400ecb0 comp rx=0 tx=0).stop 2024-05-07T02:31:37.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.726+0000 7f5a89400700 1 -- 172.21.15.19:0/3507221522 shutdown_connections 2024-05-07T02:31:37.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.726+0000 7f5a89400700 1 --2- 172.21.15.19:0/3507221522 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f5a680b2900 0x7f5a680b4e10 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:37.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.726+0000 7f5a89400700 1 --2- 172.21.15.19:0/3507221522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a84108740 0x7f5a840ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:37.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.726+0000 7f5a89400700 1 --2- 172.21.15.19:0/3507221522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a841034c0 0x7f5a840ff2a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:37.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.726+0000 7f5a89400700 1 --2- 172.21.15.19:0/3507221522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a84107df0 0x7f5a840ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:37.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.726+0000 7f5a89400700 1 -- 172.21.15.19:0/3507221522 >> 172.21.15.19:0/3507221522 conn(0x7f5a840fedc0 msgr2=0x7f5a8406b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:37.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.727+0000 7f5a89400700 1 -- 172.21.15.19:0/3507221522 shutdown_connections 2024-05-07T02:31:37.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:37.727+0000 7f5a89400700 1 -- 172.21.15.19:0/3507221522 wait complete. 2024-05-07T02:31:38.928 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:38 smithi019 ceph-mon[29781]: pgmap v502: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:31:38.944 INFO:teuthology.run_tasks:Running task sequential... 2024-05-07T02:31:38.953 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-07T02:31:38.953 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-07T02:31:38.953 INFO:teuthology.task.print:Enabling mgr modules 2024-05-07T02:31:38.954 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-05-07T02:31:38.954 INFO:teuthology.task.exec:Executing custom commands... 2024-05-07T02:31:38.954 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-05-07T02:31:38.955 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-05-07T02:31:38.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:38 smithi027 ceph-mon[34398]: pgmap v502: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:31:39.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:38 smithi064 ceph-mon[34699]: pgmap v502: 129 pgs: 129 active+clean; 650 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-07T02:31:39.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.225+0000 7fea29400640 1 -- 172.21.15.19:0/3892145197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea24105cc0 msgr2=0x7fea241069c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:39.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.225+0000 7fea29400640 1 --2- 172.21.15.19:0/3892145197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea24105cc0 0x7fea241069c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fea14009900 tx=0x7fea1402a3f0 comp rx=0 tx=0).stop 2024-05-07T02:31:39.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.225+0000 7fea29400640 1 -- 172.21.15.19:0/3892145197 shutdown_connections 2024-05-07T02:31:39.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.225+0000 7fea29400640 1 --2- 172.21.15.19:0/3892145197 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea2410bc00 0x7fea2410e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:39.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.225+0000 7fea29400640 1 --2- 172.21.15.19:0/3892145197 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea24106f00 0x7fea24107310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:39.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.225+0000 7fea29400640 1 --2- 172.21.15.19:0/3892145197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea24105cc0 0x7fea241069c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:39.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.225+0000 7fea29400640 1 -- 172.21.15.19:0/3892145197 >> 172.21.15.19:0/3892145197 conn(0x7fea24100ed0 msgr2=0x7fea24103360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:39.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.226+0000 7fea29400640 1 -- 172.21.15.19:0/3892145197 shutdown_connections 2024-05-07T02:31:39.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.226+0000 7fea29400640 1 -- 172.21.15.19:0/3892145197 wait complete. 2024-05-07T02:31:39.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.227+0000 7fea29400640 1 Processor -- start 2024-05-07T02:31:39.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.227+0000 7fea29400640 1 -- start start 2024-05-07T02:31:39.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.227+0000 7fea29400640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea24105cc0 0x7fea2419c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:39.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.227+0000 7fea29400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea24106f00 0x7fea2419c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:39.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.228+0000 7fea29400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea2410bc00 0x7fea241a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:39.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.228+0000 7fea29400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fea24111890 con 0x7fea24106f00 2024-05-07T02:31:39.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.228+0000 7fea29400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fea24111710 con 0x7fea2410bc00 2024-05-07T02:31:39.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.228+0000 7fea29400640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fea24111a10 con 0x7fea24105cc0 2024-05-07T02:31:39.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.228+0000 7fea23400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea24106f00 0x7fea2419c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:39.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.228+0000 7fea23e00640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea24105cc0 0x7fea2419c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:39.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.228+0000 7fea23400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea24106f00 0x7fea2419c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42300/0 (socket says 172.21.15.19:42300) 2024-05-07T02:31:39.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.228+0000 7fea23400640 1 -- 172.21.15.19:0/1163446966 learned_addr learned my addr 172.21.15.19:0/1163446966 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:31:39.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.228+0000 7fea28a00640 1 --2- 172.21.15.19:0/1163446966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea2410bc00 0x7fea241a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:39.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.228+0000 7fea23400640 1 -- 172.21.15.19:0/1163446966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea24105cc0 msgr2=0x7fea2419c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:39.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.228+0000 7fea23400640 1 --2- 172.21.15.19:0/1163446966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea24105cc0 0x7fea2419c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:39.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.228+0000 7fea23400640 1 -- 172.21.15.19:0/1163446966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea2410bc00 msgr2=0x7fea241a3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:39.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.228+0000 7fea23400640 1 --2- 172.21.15.19:0/1163446966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea2410bc00 0x7fea241a3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:39.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.228+0000 7fea23400640 1 -- 172.21.15.19:0/1163446966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea241a3ec0 con 0x7fea24106f00 2024-05-07T02:31:39.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.229+0000 7fea23400640 1 --2- 172.21.15.19:0/1163446966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea24106f00 0x7fea2419c8c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fea1000ea60 tx=0x7fea1000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:39.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.229+0000 7fea20c00640 1 -- 172.21.15.19:0/1163446966 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1000cd20 con 0x7fea24106f00 2024-05-07T02:31:39.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.229+0000 7fea20c00640 1 -- 172.21.15.19:0/1163446966 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea100035f0 con 0x7fea24106f00 2024-05-07T02:31:39.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.229+0000 7fea29400640 1 -- 172.21.15.19:0/1163446966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea241a41b0 con 0x7fea24106f00 2024-05-07T02:31:39.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.229+0000 7fea20c00640 1 -- 172.21.15.19:0/1163446966 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea10010690 con 0x7fea24106f00 2024-05-07T02:31:39.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.229+0000 7fea29400640 1 -- 172.21.15.19:0/1163446966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea241a4700 con 0x7fea24106f00 2024-05-07T02:31:39.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.230+0000 7fea29400640 1 -- 172.21.15.19:0/1163446966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea2410aa10 con 0x7fea24106f00 2024-05-07T02:31:39.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.234+0000 7fea20c00640 1 -- 172.21.15.19:0/1163446966 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fea100037a0 con 0x7fea24106f00 2024-05-07T02:31:39.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.235+0000 7fea20c00640 1 --2- 172.21.15.19:0/1163446966 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fea080b28a0 0x7fea080b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:39.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.235+0000 7fea20c00640 1 -- 172.21.15.19:0/1163446966 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fea100e2ac0 con 0x7fea24106f00 2024-05-07T02:31:39.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.235+0000 7fea23e00640 1 --2- 172.21.15.19:0/1163446966 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fea080b28a0 0x7fea080b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:39.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.236+0000 7fea23e00640 1 --2- 172.21.15.19:0/1163446966 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fea080b28a0 0x7fea080b4db0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fea1402a9e0 tx=0x7fea140314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:39.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.237+0000 7fea20c00640 1 -- 172.21.15.19:0/1163446966 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fea10054df0 con 0x7fea24106f00 2024-05-07T02:31:39.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.511+0000 7fea29400640 1 -- 172.21.15.19:0/1163446966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fea240638d0 con 0x7fea24106f00 2024-05-07T02:31:39.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.692+0000 7fea20c00640 1 -- 172.21.15.19:0/1163446966 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7fea100aa3d0 con 0x7fea24106f00 2024-05-07T02:31:39.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.697+0000 7fea29400640 1 -- 172.21.15.19:0/1163446966 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fea080b28a0 msgr2=0x7fea080b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:39.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.697+0000 7fea29400640 1 --2- 172.21.15.19:0/1163446966 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fea080b28a0 0x7fea080b4db0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fea1402a9e0 tx=0x7fea140314a0 comp rx=0 tx=0).stop 2024-05-07T02:31:39.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.697+0000 7fea29400640 1 -- 172.21.15.19:0/1163446966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea24106f00 msgr2=0x7fea2419c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:39.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.697+0000 7fea29400640 1 --2- 172.21.15.19:0/1163446966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea24106f00 0x7fea2419c8c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fea1000ea60 tx=0x7fea1000ee60 comp rx=0 tx=0).stop 2024-05-07T02:31:39.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.700+0000 7fea29400640 1 -- 172.21.15.19:0/1163446966 shutdown_connections 2024-05-07T02:31:39.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.701+0000 7fea29400640 1 --2- 172.21.15.19:0/1163446966 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fea080b28a0 0x7fea080b4db0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:39.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.701+0000 7fea29400640 1 --2- 172.21.15.19:0/1163446966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea2410bc00 0x7fea241a3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:39.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.701+0000 7fea29400640 1 --2- 172.21.15.19:0/1163446966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea24106f00 0x7fea2419c8c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:39.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.701+0000 7fea29400640 1 --2- 172.21.15.19:0/1163446966 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea24105cc0 0x7fea2419c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:39.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.701+0000 7fea29400640 1 -- 172.21.15.19:0/1163446966 >> 172.21.15.19:0/1163446966 conn(0x7fea24100ed0 msgr2=0x7fea24107be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:39.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.702+0000 7fea29400640 1 -- 172.21.15.19:0/1163446966 shutdown_connections 2024-05-07T02:31:39.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:39.703+0000 7fea29400640 1 -- 172.21.15.19:0/1163446966 wait complete. 2024-05-07T02:31:39.737 INFO:teuthology.run_tasks:Running task kclient... 2024-05-07T02:31:39.750 INFO:tasks.kclient:Mounting kernel clients... 2024-05-07T02:31:39.750 INFO:tasks.kclient:config is {'client.0': {}} 2024-05-07T02:31:39.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-07T02:31:39.750 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-05-07T02:31:39.799 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-05-07T02:31:39.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-07T02:31:39.800 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-07T02:31:39.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-07T02:31:39.874 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link delete ceph-brx 2024-05-07T02:31:39.936 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:39 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/1163446966' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-07T02:31:39.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:39 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/1163446966' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-07T02:31:39.956 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:31:39 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: ignoring --setuser ceph since I am not root 2024-05-07T02:31:39.956 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:31:39 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: ignoring --setgroup ceph since I am not root 2024-05-07T02:31:39.956 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:31:39 smithi027 ceph-mgr[35799]: -- 172.21.15.27:0/2293010942 <== mon.2 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5633202085a0 con 0x563324f92000 2024-05-07T02:31:39.956 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:31:39 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:31:39.917+0000 7f0f047fb200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-07T02:31:39.996 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-05-07T02:31:39.996 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-07T02:31:39.997 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-07T02:31:39.997 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph fs ls 2024-05-07T02:31:40.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:39 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/1163446966' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-07T02:31:40.108 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:31:39 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: ignoring --setuser ceph since I am not root 2024-05-07T02:31:40.108 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:31:39 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: ignoring --setgroup ceph since I am not root 2024-05-07T02:31:40.108 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:31:39 smithi064 ceph-mgr[36576]: -- 172.21.15.64:0/178566172 <== mon.1 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5640eebc45a0 con 0x5640f394e480 2024-05-07T02:31:40.108 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:31:39 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:31:39.915+0000 7fa4ddeb5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-07T02:31:40.394 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:31:39 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: ignoring --setuser ceph since I am not root 2024-05-07T02:31:40.394 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:31:39 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: ignoring --setgroup ceph since I am not root 2024-05-07T02:31:40.395 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:31:40 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:31:40.135+0000 7fa7ea0ad200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-07T02:31:40.417 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:31:40.456 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:31:40 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:31:40.111+0000 7f0f047fb200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-07T02:31:40.456 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:31:40 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:31:40.218+0000 7f0f047fb200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-07T02:31:40.539 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:31:40 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:31:40.111+0000 7fa4ddeb5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-07T02:31:40.539 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:31:40 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:31:40.219+0000 7fa4ddeb5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-07T02:31:40.694 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:31:40 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:31:40.406+0000 7fa7ea0ad200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-07T02:31:40.695 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:31:40 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:31:40.551+0000 7fa7ea0ad200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-07T02:31:40.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:40 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/1163446966' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-07T02:31:40.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:40 smithi064 ceph-mon[34699]: mgrmap e19: x(active, since 14m), standbys: z, y 2024-05-07T02:31:40.858 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:31:40 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:31:40.540+0000 7fa4ddeb5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-07T02:31:40.858 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:31:40 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:31:40.630+0000 7fa4ddeb5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-07T02:31:40.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:40 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/1163446966' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-07T02:31:40.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:40 smithi027 ceph-mon[34398]: mgrmap e19: x(active, since 14m), standbys: z, y 2024-05-07T02:31:40.956 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:31:40 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:31:40.539+0000 7f0f047fb200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-07T02:31:40.956 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:31:40 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:31:40.628+0000 7f0f047fb200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-07T02:31:40.967 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:40 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/1163446966' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-07T02:31:40.967 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:40 smithi019 ceph-mon[29781]: mgrmap e19: x(active, since 14m), standbys: z, y 2024-05-07T02:31:41.394 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:31:40 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:31:40.973+0000 7fa7ea0ad200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-07T02:31:41.395 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:31:41 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:31:41.093+0000 7fa7ea0ad200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-07T02:31:42.206 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:31:41 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:31:41.876+0000 7f0f047fb200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-07T02:31:42.206 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:31:42 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:31:42.056+0000 7f0f047fb200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-07T02:31:42.237 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:31:41 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:31:41.880+0000 7fa4ddeb5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-07T02:31:42.237 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:31:42 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:31:42.060+0000 7fa4ddeb5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-07T02:31:42.538 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:31:42 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:31:42.234+0000 7f0f047fb200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-07T02:31:42.541 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:31:42 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:31:42.237+0000 7fa4ddeb5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-07T02:31:42.846 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:31:42 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:31:42.538+0000 7f0f047fb200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-07T02:31:42.846 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:31:42 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:31:42.635+0000 7f0f047fb200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-07T02:31:42.849 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:31:42 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:31:42.542+0000 7fa4ddeb5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-07T02:31:42.850 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:31:42 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:31:42.639+0000 7fa4ddeb5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-07T02:31:42.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:42.904+0000 7f67b4000700 1 -- 172.21.15.19:0/1955527796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67ac1045f0 msgr2=0x7f67ac104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:42.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:42.904+0000 7f67b4000700 1 --2- 172.21.15.19:0/1955527796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67ac1045f0 0x7f67ac104a00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f67a0009990 tx=0x7f67a0009ca0 comp rx=0 tx=0).stop 2024-05-07T02:31:42.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:42.904+0000 7f67b4000700 1 -- 172.21.15.19:0/1955527796 shutdown_connections 2024-05-07T02:31:42.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:42.904+0000 7f67b4000700 1 --2- 172.21.15.19:0/1955527796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67ac109870 0x7f67ac1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:42.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:42.904+0000 7f67b4000700 1 --2- 172.21.15.19:0/1955527796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67ac1045f0 0x7f67ac104a00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:42.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:42.904+0000 7f67b4000700 1 --2- 172.21.15.19:0/1955527796 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67ac108f20 0x7f67ac109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:42.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:42.904+0000 7f67b4000700 1 -- 172.21.15.19:0/1955527796 >> 172.21.15.19:0/1955527796 conn(0x7f67ac100040 msgr2=0x7f67ac102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:42.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:42.904+0000 7f67b4000700 1 -- 172.21.15.19:0/1955527796 shutdown_connections 2024-05-07T02:31:42.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:42.904+0000 7f67b4000700 1 -- 172.21.15.19:0/1955527796 wait complete. 2024-05-07T02:31:42.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:42.905+0000 7f67b4000700 1 Processor -- start 2024-05-07T02:31:42.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:42.906+0000 7f67b4000700 1 -- start start 2024-05-07T02:31:42.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:42.906+0000 7f67b4000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67ac1045f0 0x7f67ac19d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:42.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:42.906+0000 7f67b4000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67ac108f20 0x7f67ac19d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:42.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:42.906+0000 7f67b4000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67ac109870 0x7f67ac1a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:42.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:42.906+0000 7f67b4000700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f67ac1144f0 con 0x7f67ac108f20 2024-05-07T02:31:42.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:42.906+0000 7f67b4000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f67ac114370 con 0x7f67ac109870 2024-05-07T02:31:42.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:42.906+0000 7f67b4000700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f67ac114670 con 0x7f67ac1045f0 2024-05-07T02:31:42.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:42.907+0000 7f67b1600700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67ac1045f0 0x7f67ac19d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:42.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:42.907+0000 7f67b0c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67ac108f20 0x7f67ac19d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:42.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:42.907+0000 7f67b2000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67ac109870 0x7f67ac1a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:42.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:42.907+0000 7f67b1600700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67ac1045f0 0x7f67ac19d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:46660/0 (socket says 172.21.15.19:46660) 2024-05-07T02:31:42.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:42.907+0000 7f67b1600700 1 -- 172.21.15.19:0/51203273 learned_addr learned my addr 172.21.15.19:0/51203273 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:31:42.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:42.907+0000 7f67b1600700 1 -- 172.21.15.19:0/51203273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67ac109870 msgr2=0x7f67ac1a1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:42.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:42.907+0000 7f67b1600700 1 --2- 172.21.15.19:0/51203273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67ac109870 0x7f67ac1a1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:42.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:42.907+0000 7f67b1600700 1 -- 172.21.15.19:0/51203273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67ac108f20 msgr2=0x7f67ac19d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:42.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:42.907+0000 7f67b1600700 1 --2- 172.21.15.19:0/51203273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67ac108f20 0x7f67ac19d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:42.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:42.907+0000 7f67b1600700 1 -- 172.21.15.19:0/51203273 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67ac1a2010 con 0x7f67ac1045f0 2024-05-07T02:31:42.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:42.907+0000 7f67b1600700 1 --2- 172.21.15.19:0/51203273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67ac1045f0 0x7f67ac19d140 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f67a0000c00 tx=0x7f67a00057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:42.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:42.908+0000 7f67aa000700 1 -- 172.21.15.19:0/51203273 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a0027070 con 0x7f67ac1045f0 2024-05-07T02:31:42.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:42.908+0000 7f67b4000700 1 -- 172.21.15.19:0/51203273 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67ac1a22a0 con 0x7f67ac1045f0 2024-05-07T02:31:42.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:42.908+0000 7f67aa000700 1 -- 172.21.15.19:0/51203273 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67a000b610 con 0x7f67ac1045f0 2024-05-07T02:31:42.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:42.908+0000 7f67aa000700 1 -- 172.21.15.19:0/51203273 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a00173f0 con 0x7f67ac1045f0 2024-05-07T02:31:42.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:42.908+0000 7f67b4000700 1 -- 172.21.15.19:0/51203273 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67ac1a9c20 con 0x7f67ac1045f0 2024-05-07T02:31:42.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:42.909+0000 7f67b4000700 1 -- 172.21.15.19:0/51203273 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67ac0512c0 con 0x7f67ac1045f0 2024-05-07T02:31:42.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:42.911+0000 7f67aa000700 1 -- 172.21.15.19:0/51203273 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7f67a002a070 con 0x7f67ac1045f0 2024-05-07T02:31:42.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:42.913+0000 7f67aa000700 1 --2- 172.21.15.19:0/51203273 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f67940b2950 0x7f67940b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:42.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:42.913+0000 7f67b0c00700 1 -- 172.21.15.19:0/51203273 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f67940b2950 msgr2=0x7f67940b4e60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2059564551 2024-05-07T02:31:42.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:42.913+0000 7f67aa000700 1 -- 172.21.15.19:0/51203273 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f67a00e37b0 con 0x7f67ac1045f0 2024-05-07T02:31:42.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:42.913+0000 7f67b0c00700 1 --2- 172.21.15.19:0/51203273 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f67940b2950 0x7f67940b4e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-07T02:31:42.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:42.915+0000 7f67aa000700 1 -- 172.21.15.19:0/51203273 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f67a005fdf0 con 0x7f67ac1045f0 2024-05-07T02:31:42.992 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:31:42 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:31:42.740+0000 7fa7ea0ad200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-07T02:31:43.108 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:31:42 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:31:42.850+0000 7fa4ddeb5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-07T02:31:43.108 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:31:43 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:31:43.053+0000 7fa4ddeb5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-07T02:31:43.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:43.114+0000 7f67b0c00700 1 -- 172.21.15.19:0/51203273 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f67940b2950 msgr2=0x7f67940b4e60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2059564551 2024-05-07T02:31:43.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:43.114+0000 7f67b0c00700 1 --2- 172.21.15.19:0/51203273 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f67940b2950 0x7f67940b4e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-07T02:31:43.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:43.199+0000 7f67b4000700 1 -- 172.21.15.19:0/51203273 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f67ac1a9df0 con 0x7f67ac1045f0 2024-05-07T02:31:43.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:43.200+0000 7f67aa000700 1 -- 172.21.15.19:0/51203273 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v30) ==== 53+0+88 (secure 0 0 0) 0x7f67a0013070 con 0x7f67ac1045f0 2024-05-07T02:31:43.200 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-07T02:31:43.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:43.204+0000 7f67b4000700 1 -- 172.21.15.19:0/51203273 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f67940b2950 msgr2=0x7f67940b4e60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-07T02:31:43.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:43.204+0000 7f67b4000700 1 --2- 172.21.15.19:0/51203273 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f67940b2950 0x7f67940b4e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:43.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:43.204+0000 7f67b4000700 1 -- 172.21.15.19:0/51203273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67ac1045f0 msgr2=0x7f67ac19d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:43.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:43.204+0000 7f67b4000700 1 --2- 172.21.15.19:0/51203273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67ac1045f0 0x7f67ac19d140 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f67a0000c00 tx=0x7f67a00057c0 comp rx=0 tx=0).stop 2024-05-07T02:31:43.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:43.204+0000 7f67b4000700 1 -- 172.21.15.19:0/51203273 shutdown_connections 2024-05-07T02:31:43.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:43.204+0000 7f67b4000700 1 --2- 172.21.15.19:0/51203273 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f67940b2950 0x7f67940b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:43.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:43.204+0000 7f67b4000700 1 --2- 172.21.15.19:0/51203273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67ac109870 0x7f67ac1a1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:43.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:43.204+0000 7f67b4000700 1 --2- 172.21.15.19:0/51203273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67ac1045f0 0x7f67ac19d140 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:43.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:43.204+0000 7f67b4000700 1 --2- 172.21.15.19:0/51203273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67ac108f20 0x7f67ac19d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:43.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:43.204+0000 7f67b4000700 1 -- 172.21.15.19:0/51203273 >> 172.21.15.19:0/51203273 conn(0x7f67ac100040 msgr2=0x7f67ac1086c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:43.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:43.204+0000 7f67b4000700 1 -- 172.21.15.19:0/51203273 shutdown_connections 2024-05-07T02:31:43.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:43.204+0000 7f67b4000700 1 -- 172.21.15.19:0/51203273 wait complete. 2024-05-07T02:31:43.206 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:31:42 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:31:42.846+0000 7f0f047fb200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-07T02:31:43.206 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:31:43 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:31:43.049+0000 7f0f047fb200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-07T02:31:43.252 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:31:42 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:31:42.992+0000 7fa7ea0ad200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-07T02:31:43.252 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:31:43 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:31:43.221+0000 7fa7ea0ad200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-07T02:31:43.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:43 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/51203273' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-07T02:31:43.628 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:43 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/51203273' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-07T02:31:43.675 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:43 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/51203273' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-07T02:31:43.895 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:31:43 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:31:43.637+0000 7fa7ea0ad200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-07T02:31:43.895 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:31:43 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:31:43.769+0000 7fa7ea0ad200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-07T02:31:43.956 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:31:43 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:31:43.675+0000 7f0f047fb200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-07T02:31:43.956 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:31:43 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:31:43.909+0000 7f0f047fb200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-07T02:31:44.108 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:31:43 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:31:43.682+0000 7fa4ddeb5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-07T02:31:44.108 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:31:43 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:31:43.916+0000 7fa4ddeb5200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-07T02:31:44.201 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-07T02:31:44.201 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-07T02:31:44.201 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-05-07T02:31:44.201 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-07T02:31:44.201 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-07T02:31:44.201 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-07T02:31:44.202 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-07T02:31:44.202 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-07T02:31:44.202 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-07T02:31:44.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-07T02:31:44.202 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-05-07T02:31:44.256 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-07T02:31:44.256 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-07T02:31:44.256 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-05-07T02:31:44.256 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-05-07T02:31:44.256 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-05-07T02:31:44.256 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-05-07T02:31:44.256 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-07T02:31:44.257 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-05-07T02:31:44.257 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-05-07T02:31:44.257 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-07T02:31:44.257 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-05-07T02:31:44.257 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-05-07T02:31:44.257 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-07T02:31:44.257 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-05-07T02:31:44.257 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-05-07T02:31:44.257 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-05-07T02:31:44.257 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 41710sec preferred_lft 41710sec 2024-05-07T02:31:44.257 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-05-07T02:31:44.257 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-05-07T02:31:44.257 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-07T02:31:44.257 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-05-07T02:31:44.257 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-05-07T02:31:44.258 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-07T02:31:44.258 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-07T02:31:44.258 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-05-07T02:31:44.259 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link add name ceph-brx type bridge 2024-05-07T02:31:44.259 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip addr flush dev ceph-brx 2024-05-07T02:31:44.259 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set ceph-brx up 2024-05-07T02:31:44.259 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-07T02:31:44.259 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-05-07T02:31:44.310 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:31:44 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:31:44.035+0000 7fa7ea0ad200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-07T02:31:44.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-07T02:31:44.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-07T02:31:44.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-07T02:31:44.488 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-07T02:31:44.545 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-05-07T02:31:44.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-07T02:31:44.549 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-05-07T02:31:44.587 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-05-07T02:31:44.587 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-07T02:31:44.587 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-07T02:31:44.587 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-07T02:31:44.587 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-07T02:31:44.587 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-07T02:31:44.587 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-05-07T02:31:44.588 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-07T02:31:44.588 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-07T02:31:44.588 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-07T02:31:44.588 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-05-07T02:31:44.644 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:31:44 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:31:44.310+0000 7fa7ea0ad200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-07T02:31:44.706 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:31:44 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:31:44.352+0000 7f0f047fb200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-07T02:31:44.706 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:31:44 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:31:44.454+0000 7f0f047fb200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-07T02:31:44.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-07T02:31:44.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-07T02:31:44.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-07T02:31:44.831 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-05-07T02:31:44.849 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:31:44 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:31:44.361+0000 7fa4ddeb5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-07T02:31:44.849 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:31:44 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:31:44.463+0000 7fa4ddeb5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-07T02:31:44.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-07T02:31:44.926 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list-id 2024-05-07T02:31:44.975 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-07T02:31:44.975 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-05-07T02:31:44.975 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-07T02:31:44.975 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-07T02:31:44.975 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-05-07T02:31:45.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-07T02:31:45.108 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:31:44 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:31:44.849+0000 7fa4ddeb5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-07T02:31:45.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-07T02:31:45.151 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-07T02:31:45.151 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-07T02:31:45.151 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-05-07T02:31:45.151 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-07T02:31:45.151 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-07T02:31:45.151 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-07T02:31:45.151 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-07T02:31:45.151 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-07T02:31:45.152 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-05-07T02:31:45.206 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:31:44 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:31:44.840+0000 7f0f047fb200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-07T02:31:45.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-07T02:31:45.356 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:31:45 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:31:45.160+0000 7fa7ea0ad200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-07T02:31:45.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-07T02:31:45.411 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-07T02:31:45.411 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-05-07T02:31:45.411 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-05-07T02:31:45.411 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set dev brx.0 master ceph-brx 2024-05-07T02:31:45.411 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-05-07T02:31:45.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-07T02:31:45.565 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:31:45 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:31:45.214+0000 7f0f047fb200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-07T02:31:45.566 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:31:45 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:31:45.311+0000 7f0f047fb200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-07T02:31:45.577 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:31:45 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:31:45.224+0000 7fa4ddeb5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-07T02:31:45.577 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:31:45 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:31:45.322+0000 7fa4ddeb5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-07T02:31:45.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-07T02:31:45.608 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-07T02:31:45.608 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-07T02:31:45.644 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:31:45 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:31:45.475+0000 7fa7ea0ad200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-07T02:31:45.650 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-07T02:31:45.650 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-07T02:31:45.650 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-07T02:31:45.738 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-05-07T02:31:45.738 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-05-07T02:31:45.738 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/1d507078-2e76-4466-9990-b1c17f741a27 2024-05-07T02:31:45.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-07T02:31:45.738 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/1d507078-2e76-4466-9990-b1c17f741a27 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-05-07T02:31:45.858 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:31:45 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:31:45.577+0000 7fa4ddeb5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-07T02:31:45.858 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:31:45 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:31:45.718+0000 7fa4ddeb5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-07T02:31:45.928 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-05-07T02:31:45.928 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-05-07T02:31:45.928 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-05-07T02:31:45.956 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:31:45 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:31:45.565+0000 7f0f047fb200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-07T02:31:45.956 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:31:45 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:31:45.706+0000 7f0f047fb200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-07T02:31:46.358 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:31:46 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:31:46.035+0000 7fa4ddeb5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-07T02:31:46.358 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:31:46 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:31:46.155+0000 7fa4ddeb5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-07T02:31:46.358 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:31:46 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:31:46.251+0000 7fa4ddeb5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-07T02:31:46.390 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:31:46 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:31:46.083+0000 7fa7ea0ad200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-07T02:31:46.390 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:31:46 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:31:46.223+0000 7fa7ea0ad200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-07T02:31:46.456 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:31:46 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:31:46.025+0000 7f0f047fb200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-07T02:31:46.456 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:31:46 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:31:46.146+0000 7f0f047fb200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-07T02:31:46.456 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:31:46 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:31:46.242+0000 7f0f047fb200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-07T02:31:46.726 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-05-07T02:31:46.726 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-05-07T02:31:46.726 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-05-07T02:31:46.726 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.27:3300,172.21.15.64:3300" 2024-05-07T02:31:46.727 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.19:3300,172.21.15.27:3300,172.21.15.64:3300:/volumes/qa/sv_1/1d507078-2e76-4466-9990-b1c17f741a27 2024-05-07T02:31:46.727 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,key=0,fsid=9ff84c16-0c17-11ef-bc97-c7b262605968" will pass to kernel 2024-05-07T02:31:46.729 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-07T02:31:46.729 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-05-07T02:31:46.730 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-07T02:31:46.730 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-05-07T02:31:46.730 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-07T02:31:46.734 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-07T02:31:46.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-07T02:31:46.735 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-07T02:31:46.810 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-07T02:31:46.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-07T02:31:46.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-07T02:31:46.926 INFO:teuthology.orchestra.run.smithi019.stdout:client25250 2024-05-07T02:31:46.926 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-07T02:31:46.926 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-07T02:31:46.977 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-07T02:31:47.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-07T02:31:47.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-07T02:31:47.093 INFO:teuthology.orchestra.run.smithi019.stdout:client25250 2024-05-07T02:31:47.093 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-07T02:31:47.144 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:31:46 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:31:46.747+0000 7fa7ea0ad200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-07T02:31:47.172 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:31:46 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:31:46.883+0000 7fa4ddeb5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-07T02:31:47.172 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:31:47 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z[36572]: 2024-05-07T02:31:47.111+0000 7fa4ddeb5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-07T02:31:47.173 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:31:46 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:31:46.872+0000 7f0f047fb200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-07T02:31:47.173 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:31:47 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y[35795]: 2024-05-07T02:31:47.099+0000 7f0f047fb200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-07T02:31:47.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-07T02:31:47.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-07T02:31:47.208 INFO:teuthology.orchestra.run.smithi019.stdout:client25250 2024-05-07T02:31:47.208 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968.client25250/status') 2024-05-07T02:31:47.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-07T02:31:47.335 INFO:teuthology.orchestra.run.smithi019.stdout:instance: client.25250 (3)172.21.15.19:0/3164981075 2024-05-07T02:31:47.335 INFO:teuthology.orchestra.run.smithi019.stdout:blocklisted: false 2024-05-07T02:31:47.335 INFO:teuthology.orchestra.run.smithi019.stderr:0+1 records in 2024-05-07T02:31:47.335 INFO:teuthology.orchestra.run.smithi019.stderr:0+1 records out 2024-05-07T02:31:47.335 INFO:teuthology.orchestra.run.smithi019.stderr:71 bytes copied, 8.5221e-05 s, 833 kB/s 2024-05-07T02:31:47.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-07T02:31:47.351 INFO:teuthology.run_tasks:Running task exec... 2024-05-07T02:31:47.361 INFO:teuthology.task.exec:Executing custom commands... 2024-05-07T02:31:47.361 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-05-07T02:31:47.361 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-05-07T02:31:47.426 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:31:47 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:31:47.254+0000 7fa7ea0ad200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-07T02:31:47.427 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:31:47 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:31:47.393+0000 7fa7ea0ad200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-07T02:31:47.427 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:47 smithi019 ceph-mon[29781]: Standby manager daemon y restarted 2024-05-07T02:31:47.427 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:47 smithi019 ceph-mon[29781]: Standby manager daemon y started 2024-05-07T02:31:47.427 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:47 smithi019 ceph-mon[29781]: from='mgr.? 172.21.15.27:0/2920214166' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-07T02:31:47.427 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:47 smithi019 ceph-mon[29781]: from='mgr.? 172.21.15.27:0/2920214166' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-07T02:31:47.427 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:47 smithi019 ceph-mon[29781]: from='mgr.? 172.21.15.27:0/2920214166' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-07T02:31:47.427 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:47 smithi019 ceph-mon[29781]: from='mgr.? 172.21.15.27:0/2920214166' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-07T02:31:47.427 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:47 smithi019 ceph-mon[29781]: Standby manager daemon z restarted 2024-05-07T02:31:47.427 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:47 smithi019 ceph-mon[29781]: Standby manager daemon z started 2024-05-07T02:31:47.427 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:47 smithi019 ceph-mon[29781]: from='mgr.? 172.21.15.64:0/2467817579' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-07T02:31:47.428 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:47 smithi019 ceph-mon[29781]: from='mgr.? 172.21.15.64:0/2467817579' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-07T02:31:47.428 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:47 smithi019 ceph-mon[29781]: from='mgr.? 172.21.15.64:0/2467817579' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-07T02:31:47.428 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:47 smithi019 ceph-mon[29781]: from='mgr.? 172.21.15.64:0/2467817579' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-07T02:31:47.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:47 smithi027 ceph-mon[34398]: Standby manager daemon y restarted 2024-05-07T02:31:47.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:47 smithi027 ceph-mon[34398]: Standby manager daemon y started 2024-05-07T02:31:47.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:47 smithi027 ceph-mon[34398]: from='mgr.? 172.21.15.27:0/2920214166' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-07T02:31:47.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:47 smithi027 ceph-mon[34398]: from='mgr.? 172.21.15.27:0/2920214166' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-07T02:31:47.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:47 smithi027 ceph-mon[34398]: from='mgr.? 172.21.15.27:0/2920214166' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-07T02:31:47.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:47 smithi027 ceph-mon[34398]: from='mgr.? 172.21.15.27:0/2920214166' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-07T02:31:47.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:47 smithi027 ceph-mon[34398]: Standby manager daemon z restarted 2024-05-07T02:31:47.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:47 smithi027 ceph-mon[34398]: Standby manager daemon z started 2024-05-07T02:31:47.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:47 smithi027 ceph-mon[34398]: from='mgr.? 172.21.15.64:0/2467817579' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-07T02:31:47.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:47 smithi027 ceph-mon[34398]: from='mgr.? 172.21.15.64:0/2467817579' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-07T02:31:47.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:47 smithi027 ceph-mon[34398]: from='mgr.? 172.21.15.64:0/2467817579' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-07T02:31:47.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:47 smithi027 ceph-mon[34398]: from='mgr.? 172.21.15.64:0/2467817579' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-07T02:31:47.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:47 smithi064 ceph-mon[34699]: Standby manager daemon y restarted 2024-05-07T02:31:47.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:47 smithi064 ceph-mon[34699]: Standby manager daemon y started 2024-05-07T02:31:47.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:47 smithi064 ceph-mon[34699]: from='mgr.? 172.21.15.27:0/2920214166' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-07T02:31:47.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:47 smithi064 ceph-mon[34699]: from='mgr.? 172.21.15.27:0/2920214166' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-07T02:31:47.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:47 smithi064 ceph-mon[34699]: from='mgr.? 172.21.15.27:0/2920214166' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-07T02:31:47.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:47 smithi064 ceph-mon[34699]: from='mgr.? 172.21.15.27:0/2920214166' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-07T02:31:47.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:47 smithi064 ceph-mon[34699]: Standby manager daemon z restarted 2024-05-07T02:31:47.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:47 smithi064 ceph-mon[34699]: Standby manager daemon z started 2024-05-07T02:31:47.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:47 smithi064 ceph-mon[34699]: from='mgr.? 172.21.15.64:0/2467817579' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-07T02:31:47.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:47 smithi064 ceph-mon[34699]: from='mgr.? 172.21.15.64:0/2467817579' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-07T02:31:47.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:47 smithi064 ceph-mon[34699]: from='mgr.? 172.21.15.64:0/2467817579' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-07T02:31:47.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:47 smithi064 ceph-mon[34699]: from='mgr.? 172.21.15.64:0/2467817579' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-07T02:31:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47.627+0000 7f94e0c00640 1 -- 172.21.15.19:0/3182628931 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94dc103d90 msgr2=0x7f94dc1041a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47.627+0000 7f94e0c00640 1 --2- 172.21.15.19:0/3182628931 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94dc103d90 0x7f94dc1041a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f94c8009900 tx=0x7f94c802a410 comp rx=0 tx=0).stop 2024-05-07T02:31:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47.627+0000 7f94e0c00640 1 -- 172.21.15.19:0/3182628931 shutdown_connections 2024-05-07T02:31:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47.627+0000 7f94e0c00640 1 --2- 172.21.15.19:0/3182628931 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94dc108a90 0x7f94dc10aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47.627+0000 7f94e0c00640 1 --2- 172.21.15.19:0/3182628931 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94dc103d90 0x7f94dc1041a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47.627+0000 7f94e0c00640 1 --2- 172.21.15.19:0/3182628931 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94dc102b50 0x7f94dc103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47.627+0000 7f94e0c00640 1 -- 172.21.15.19:0/3182628931 >> 172.21.15.19:0/3182628931 conn(0x7f94dc0fdd40 msgr2=0x7f94dc1001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47.627+0000 7f94e0c00640 1 -- 172.21.15.19:0/3182628931 shutdown_connections 2024-05-07T02:31:47.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47.627+0000 7f94e0c00640 1 -- 172.21.15.19:0/3182628931 wait complete. 2024-05-07T02:31:47.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47.629+0000 7f94e0c00640 1 Processor -- start 2024-05-07T02:31:47.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47.629+0000 7f94e0c00640 1 -- start start 2024-05-07T02:31:47.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47.629+0000 7f94e0c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94dc102b50 0x7f94dc19a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:47.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47.629+0000 7f94e0c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94dc103d90 0x7f94dc19a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:47.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47.629+0000 7f94e0c00640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94dc108a90 0x7f94dc1a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:47.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47.629+0000 7f94e0c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc10f780 con 0x7f94dc103d90 2024-05-07T02:31:47.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47.629+0000 7f94e0c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc10f600 con 0x7f94dc102b50 2024-05-07T02:31:47.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47.629+0000 7f94e0c00640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc10f900 con 0x7f94dc108a90 2024-05-07T02:31:47.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47.629+0000 7f94d9400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94dc103d90 0x7f94dc19a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:47.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47.629+0000 7f94d9e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94dc102b50 0x7f94dc19a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:47.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47.630+0000 7f94d9e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94dc102b50 0x7f94dc19a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:42646/0 (socket says 172.21.15.19:42646) 2024-05-07T02:31:47.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47.630+0000 7f94d9e00640 1 -- 172.21.15.19:0/1076768361 learned_addr learned my addr 172.21.15.19:0/1076768361 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:31:47.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47.630+0000 7f94da800640 1 --2- 172.21.15.19:0/1076768361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94dc108a90 0x7f94dc1a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:47.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47.630+0000 7f94da800640 1 -- 172.21.15.19:0/1076768361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94dc102b50 msgr2=0x7f94dc19a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:47.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47.630+0000 7f94da800640 1 --2- 172.21.15.19:0/1076768361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94dc102b50 0x7f94dc19a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:47.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47.630+0000 7f94da800640 1 -- 172.21.15.19:0/1076768361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94dc103d90 msgr2=0x7f94dc19a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:47.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47.630+0000 7f94da800640 1 --2- 172.21.15.19:0/1076768361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94dc103d90 0x7f94dc19a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:47.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47.630+0000 7f94da800640 1 -- 172.21.15.19:0/1076768361 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94dc1a1e40 con 0x7f94dc108a90 2024-05-07T02:31:47.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47.630+0000 7f94d9e00640 1 --2- 172.21.15.19:0/1076768361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94dc102b50 0x7f94dc19a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:31:47.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47.630+0000 7f94d9400640 1 --2- 172.21.15.19:0/1076768361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94dc103d90 0x7f94dc19a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:31:47.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47.630+0000 7f94da800640 1 --2- 172.21.15.19:0/1076768361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94dc108a90 0x7f94dc1a17d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f94d4009fd0 tx=0x7f94d400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:47.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47.631+0000 7f94d2a00640 1 -- 172.21.15.19:0/1076768361 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d400ce20 con 0x7f94dc108a90 2024-05-07T02:31:47.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47.631+0000 7f94d2a00640 1 -- 172.21.15.19:0/1076768361 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94d4005020 con 0x7f94dc108a90 2024-05-07T02:31:47.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47.631+0000 7f94e0c00640 1 -- 172.21.15.19:0/1076768361 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94dc1a2130 con 0x7f94dc108a90 2024-05-07T02:31:47.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47.631+0000 7f94d2a00640 1 -- 172.21.15.19:0/1076768361 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d4010690 con 0x7f94dc108a90 2024-05-07T02:31:47.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47.631+0000 7f94e0c00640 1 -- 172.21.15.19:0/1076768361 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94dc1a2680 con 0x7f94dc108a90 2024-05-07T02:31:47.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47.632+0000 7f94e0c00640 1 -- 172.21.15.19:0/1076768361 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94dc107190 con 0x7f94dc108a90 2024-05-07T02:31:47.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47.635+0000 7f94d2a00640 1 -- 172.21.15.19:0/1076768361 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7f94d4004b20 con 0x7f94dc108a90 2024-05-07T02:31:47.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47.637+0000 7f94d2a00640 1 --2- 172.21.15.19:0/1076768361 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f94c00b28f0 0x7f94c00b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:47.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47.637+0000 7f94d9e00640 1 -- 172.21.15.19:0/1076768361 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f94c00b28f0 msgr2=0x7f94c00b4e00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2059564551 2024-05-07T02:31:47.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47.637+0000 7f94d2a00640 1 -- 172.21.15.19:0/1076768361 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f94d40e2c90 con 0x7f94dc108a90 2024-05-07T02:31:47.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47.637+0000 7f94d9e00640 1 --2- 172.21.15.19:0/1076768361 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f94c00b28f0 0x7f94c00b4e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-07T02:31:47.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47.641+0000 7f94d2a00640 1 -- 172.21.15.19:0/1076768361 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f94d40aa5a0 con 0x7f94dc108a90 2024-05-07T02:31:47.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47.838+0000 7f94d9e00640 1 -- 172.21.15.19:0/1076768361 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f94c00b28f0 msgr2=0x7f94c00b4e00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2059564551 2024-05-07T02:31:47.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47.838+0000 7f94d9e00640 1 --2- 172.21.15.19:0/1076768361 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f94c00b28f0 0x7f94c00b4e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-07T02:31:47.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:47.902+0000 7f94e0c00640 1 -- 172.21.15.19:0/1076768361 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f94dc0638d0 con 0x7f94dc108a90 2024-05-07T02:31:48.144 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:31:47 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:31:47.747+0000 7fa7ea0ad200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-07T02:31:48.145 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:31:47 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:31:47.948+0000 7fa7ea0ad200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-07T02:31:48.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.180+0000 7f94d2a00640 1 -- 172.21.15.19:0/1076768361 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v31) ==== 117+0+0 (secure 0 0 0) 0x7f94d40a9ea0 con 0x7f94dc108a90 2024-05-07T02:31:48.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.183+0000 7f94e0c00640 1 -- 172.21.15.19:0/1076768361 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f94c00b28f0 msgr2=0x7f94c00b4e00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-07T02:31:48.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.183+0000 7f94e0c00640 1 --2- 172.21.15.19:0/1076768361 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f94c00b28f0 0x7f94c00b4e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:48.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.183+0000 7f94e0c00640 1 -- 172.21.15.19:0/1076768361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94dc108a90 msgr2=0x7f94dc1a17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:48.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.183+0000 7f94e0c00640 1 --2- 172.21.15.19:0/1076768361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94dc108a90 0x7f94dc1a17d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f94d4009fd0 tx=0x7f94d400ef10 comp rx=0 tx=0).stop 2024-05-07T02:31:48.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.183+0000 7f94e0c00640 1 -- 172.21.15.19:0/1076768361 shutdown_connections 2024-05-07T02:31:48.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.183+0000 7f94e0c00640 1 --2- 172.21.15.19:0/1076768361 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f94c00b28f0 0x7f94c00b4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:48.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.183+0000 7f94e0c00640 1 --2- 172.21.15.19:0/1076768361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f94dc108a90 0x7f94dc1a17d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:48.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.183+0000 7f94e0c00640 1 --2- 172.21.15.19:0/1076768361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94dc103d90 0x7f94dc19a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:48.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.183+0000 7f94e0c00640 1 --2- 172.21.15.19:0/1076768361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94dc102b50 0x7f94dc19a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:48.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.183+0000 7f94e0c00640 1 -- 172.21.15.19:0/1076768361 >> 172.21.15.19:0/1076768361 conn(0x7f94dc0fdd40 msgr2=0x7f94dc101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:48.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.184+0000 7f94e0c00640 1 -- 172.21.15.19:0/1076768361 shutdown_connections 2024-05-07T02:31:48.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.184+0000 7f94e0c00640 1 -- 172.21.15.19:0/1076768361 wait complete. 2024-05-07T02:31:48.209 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-07T02:31:48.220 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-05-07T02:31:48.230 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-07T02:31:48.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:48 smithi027 ceph-mon[34398]: mgrmap e20: x(active, since 14m), standbys: z, y 2024-05-07T02:31:48.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:48 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/1076768361' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-07T02:31:48.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:48 smithi027 ceph-mon[34398]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-07T02:31:48.513 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:31:48 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:31:48.344+0000 7fa7ea0ad200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-07T02:31:48.514 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:48 smithi019 ceph-mon[29781]: mgrmap e20: x(active, since 14m), standbys: z, y 2024-05-07T02:31:48.514 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:48 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/1076768361' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-07T02:31:48.514 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:48 smithi019 ceph-mon[29781]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-07T02:31:48.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.523+0000 7fe1bbe00640 1 -- 172.21.15.19:0/1537077709 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1bc104d00 msgr2=0x7fe1bc105110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:48.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.523+0000 7fe1bbe00640 1 --2- 172.21.15.19:0/1537077709 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1bc104d00 0x7fe1bc105110 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8009860 tx=0x7fe1a802a340 comp rx=0 tx=0).stop 2024-05-07T02:31:48.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.523+0000 7fe1bbe00640 1 -- 172.21.15.19:0/1537077709 shutdown_connections 2024-05-07T02:31:48.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.523+0000 7fe1bbe00640 1 --2- 172.21.15.19:0/1537077709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1bc109a00 0x7fe1bc10be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:48.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.523+0000 7fe1bbe00640 1 --2- 172.21.15.19:0/1537077709 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1bc104d00 0x7fe1bc105110 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:48.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.523+0000 7fe1bbe00640 1 --2- 172.21.15.19:0/1537077709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1bc103ac0 0x7fe1bc1047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:48.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.523+0000 7fe1bbe00640 1 -- 172.21.15.19:0/1537077709 >> 172.21.15.19:0/1537077709 conn(0x7fe1bc0fecb0 msgr2=0x7fe1bc101160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:48.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.523+0000 7fe1bbe00640 1 -- 172.21.15.19:0/1537077709 shutdown_connections 2024-05-07T02:31:48.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.523+0000 7fe1bbe00640 1 -- 172.21.15.19:0/1537077709 wait complete. 2024-05-07T02:31:48.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.524+0000 7fe1bbe00640 1 Processor -- start 2024-05-07T02:31:48.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.525+0000 7fe1bbe00640 1 -- start start 2024-05-07T02:31:48.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.525+0000 7fe1bbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1bc103ac0 0x7fe1bc19a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:48.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.525+0000 7fe1bbe00640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1bc104d00 0x7fe1bc19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:48.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.525+0000 7fe1bbe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1bc109a00 0x7fe1bc1a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:48.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.525+0000 7fe1bbe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe1bc10f580 con 0x7fe1bc103ac0 2024-05-07T02:31:48.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.525+0000 7fe1bbe00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe1bc10f400 con 0x7fe1bc109a00 2024-05-07T02:31:48.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.525+0000 7fe1bbe00640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe1bc10f700 con 0x7fe1bc104d00 2024-05-07T02:31:48.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.525+0000 7fe1baa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1bc103ac0 0x7fe1bc19a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:48.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.525+0000 7fe1baa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1bc103ac0 0x7fe1bc19a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42364/0 (socket says 172.21.15.19:42364) 2024-05-07T02:31:48.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.525+0000 7fe1baa00640 1 -- 172.21.15.19:0/2129649450 learned_addr learned my addr 172.21.15.19:0/2129649450 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:31:48.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.525+0000 7fe1ba000640 1 --2- 172.21.15.19:0/2129649450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1bc104d00 0x7fe1bc19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:48.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.525+0000 7fe1bb400640 1 --2- 172.21.15.19:0/2129649450 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1bc109a00 0x7fe1bc1a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:48.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.526+0000 7fe1baa00640 1 -- 172.21.15.19:0/2129649450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1bc104d00 msgr2=0x7fe1bc19a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:48.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.526+0000 7fe1baa00640 1 --2- 172.21.15.19:0/2129649450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1bc104d00 0x7fe1bc19a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:48.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.526+0000 7fe1baa00640 1 -- 172.21.15.19:0/2129649450 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1bc109a00 msgr2=0x7fe1bc1a1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:48.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.526+0000 7fe1baa00640 1 --2- 172.21.15.19:0/2129649450 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1bc109a00 0x7fe1bc1a1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:48.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.526+0000 7fe1baa00640 1 -- 172.21.15.19:0/2129649450 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1bc1a1c00 con 0x7fe1bc103ac0 2024-05-07T02:31:48.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.526+0000 7fe1baa00640 1 --2- 172.21.15.19:0/2129649450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1bc103ac0 0x7fe1bc19a0c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac00e980 tx=0x7fe1ac00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:48.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.526+0000 7fe1b3400640 1 -- 172.21.15.19:0/2129649450 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac00cd80 con 0x7fe1bc103ac0 2024-05-07T02:31:48.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.526+0000 7fe1b3400640 1 -- 172.21.15.19:0/2129649450 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1ac0108e0 con 0x7fe1bc103ac0 2024-05-07T02:31:48.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.526+0000 7fe1bbe00640 1 -- 172.21.15.19:0/2129649450 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1bc1a1ef0 con 0x7fe1bc103ac0 2024-05-07T02:31:48.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.526+0000 7fe1b3400640 1 -- 172.21.15.19:0/2129649450 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac018940 con 0x7fe1bc103ac0 2024-05-07T02:31:48.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.527+0000 7fe1bbe00640 1 -- 172.21.15.19:0/2129649450 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1bc1a2350 con 0x7fe1bc103ac0 2024-05-07T02:31:48.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.528+0000 7fe1bbe00640 1 -- 172.21.15.19:0/2129649450 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1bc069e80 con 0x7fe1bc103ac0 2024-05-07T02:31:48.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.530+0000 7fe1b3400640 1 -- 172.21.15.19:0/2129649450 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7fe1ac01f020 con 0x7fe1bc103ac0 2024-05-07T02:31:48.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.533+0000 7fe1b3400640 1 --2- 172.21.15.19:0/2129649450 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fe1a00b28f0 0x7fe1a00b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:48.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.533+0000 7fe1ba000640 1 -- 172.21.15.19:0/2129649450 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fe1a00b28f0 msgr2=0x7fe1a00b4e00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2059564551 2024-05-07T02:31:48.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.533+0000 7fe1b3400640 1 -- 172.21.15.19:0/2129649450 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7fe1ac0e2d50 con 0x7fe1bc103ac0 2024-05-07T02:31:48.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.533+0000 7fe1ba000640 1 --2- 172.21.15.19:0/2129649450 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fe1a00b28f0 0x7fe1a00b4e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-07T02:31:48.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.536+0000 7fe1b3400640 1 -- 172.21.15.19:0/2129649450 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe1ac0aa660 con 0x7fe1bc103ac0 2024-05-07T02:31:48.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:48 smithi064 ceph-mon[34699]: mgrmap e20: x(active, since 14m), standbys: z, y 2024-05-07T02:31:48.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:48 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/1076768361' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-07T02:31:48.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:48 smithi064 ceph-mon[34699]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-07T02:31:48.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.734+0000 7fe1ba000640 1 -- 172.21.15.19:0/2129649450 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fe1a00b28f0 msgr2=0x7fe1a00b4e00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2059564551 2024-05-07T02:31:48.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.734+0000 7fe1ba000640 1 --2- 172.21.15.19:0/2129649450 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fe1a00b28f0 0x7fe1a00b4e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-07T02:31:48.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.749+0000 7fe1bbe00640 1 -- 172.21.15.19:0/2129649450 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe1bc02dd90 con 0x7fe1bc103ac0 2024-05-07T02:31:48.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.752+0000 7fe1b3400640 1 -- 172.21.15.19:0/2129649450 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24723 (secure 0 0 0) 0x7fe1bc02dd90 con 0x7fe1bc103ac0 2024-05-07T02:31:48.752 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:31:48.752 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":89,"fsid":"9ff84c16-0c17-11ef-bc97-c7b262605968","created":"2024-05-07T02:16:19.507657+0000","modified":"2024-05-07T02:28:33.595951+0000","last_up_change":"2024-05-07T02:24:55.025124+0000","last_in_change":"2024-05-07T02:24:33.842422+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-07T02:20:21.439697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-07T02:26:06.178814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-07T02:26:10.705693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-07T02:26:23.534133+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b437cf22-ec8e-48f0-86a2-2413a0ab40dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6803","nonce":2390539683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6805","nonce":2390539683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6809","nonce":2390539683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6807","nonce":2390539683}]},"public_addr":"172.21.15.19:6803/2390539683","cluster_addr":"172.21.15.19:6805/2390539683","heartbeat_back_addr":"172.21.15.19:6809/2390539683","heartbeat_front_addr":"172.21.15.19:6807/2390539683","state":["exists","up"]},{"osd":1,"uuid":"dc753e16-634d-4c14-abd2-935281e1499e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6801","nonce":4292470615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6803","nonce":4292470615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6807","nonce":4292470615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6805","nonce":4292470615}]},"public_addr":"172.21.15.27:6801/4292470615","cluster_addr":"172.21.15.27:6803/4292470615","heartbeat_back_addr":"172.21.15.27:6807/4292470615","heartbeat_front_addr":"172.21.15.27:6805/4292470615","state":["exists","up"]},{"osd":2,"uuid":"40570071-646e-4e66-9a16-c55ab743ac86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6801","nonce":1180552939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6803","nonce":1180552939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6807","nonce":1180552939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6805","nonce":1180552939}]},"public_addr":"172.21.15.64:6801/1180552939","cluster_addr":"172.21.15.64:6803/1180552939","heartbeat_back_addr":"172.21.15.64:6807/1180552939","heartbeat_front_addr":"172.21.15.64:6805/1180552939","state":["exists","up"]},{"osd":3,"uuid":"ceaa4252-4dfa-427a-9b1a-76adf3b8e609","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6811","nonce":387118141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6813","nonce":387118141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6817","nonce":387118141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6815","nonce":387118141}]},"public_addr":"172.21.15.19:6811/387118141","cluster_addr":"172.21.15.19:6813/387118141","heartbeat_back_addr":"172.21.15.19:6817/387118141","heartbeat_front_addr":"172.21.15.19:6815/387118141","state":["exists","up"]},{"osd":4,"uuid":"5a6d1ebc-62ca-444e-8f29-1980418d0a8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6809","nonce":3662186068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6811","nonce":3662186068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6815","nonce":3662186068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6813","nonce":3662186068}]},"public_addr":"172.21.15.27:6809/3662186068","cluster_addr":"172.21.15.27:6811/3662186068","heartbeat_back_addr":"172.21.15.27:6815/3662186068","heartbeat_front_addr":"172.21.15.27:6813/3662186068","state":["exists","up"]},{"osd":5,"uuid":"60c207e9-55e8-4bed-87c3-f6a06a2a1fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6809","nonce":1264591849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6811","nonce":1264591849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6815","nonce":1264591849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6813","nonce":1264591849}]},"public_addr":"172.21.15.64:6809/1264591849","cluster_addr":"172.21.15.64:6811/1264591849","heartbeat_back_addr":"172.21.15.64:6815/1264591849","heartbeat_front_addr":"172.21.15.64:6813/1264591849","state":["exists","up"]},{"osd":6,"uuid":"f3e038e0-4026-45f1-a923-c07cbb307f64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6819","nonce":2158875543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6821","nonce":2158875543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6825","nonce":2158875543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6823","nonce":2158875543}]},"public_addr":"172.21.15.19:6819/2158875543","cluster_addr":"172.21.15.19:6821/2158875543","heartbeat_back_addr":"172.21.15.19:6825/2158875543","heartbeat_front_addr":"172.21.15.19:6823/2158875543","state":["exists","up"]},{"osd":7,"uuid":"d69e34d4-2028-43b8-8ada-ff4a9b579620","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6817","nonce":2756612091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6819","nonce":2756612091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6823","nonce":2756612091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6821","nonce":2756612091}]},"public_addr":"172.21.15.27:6817/2756612091","cluster_addr":"172.21.15.27:6819/2756612091","heartbeat_back_addr":"172.21.15.27:6823/2756612091","heartbeat_front_addr":"172.21.15.27:6821/2756612091","state":["exists","up"]},{"osd":8,"uuid":"6c1a4179-75b6-42ff-ac29-378801e6f8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6817","nonce":1987024836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6819","nonce":1987024836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6823","nonce":1987024836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6821","nonce":1987024836}]},"public_addr":"172.21.15.64:6817/1987024836","cluster_addr":"172.21.15.64:6819/1987024836","heartbeat_back_addr":"172.21.15.64:6823/1987024836","heartbeat_front_addr":"172.21.15.64:6821/1987024836","state":["exists","up"]},{"osd":9,"uuid":"9df7ff66-d39f-4709-8324-74cf9e0e052d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6827","nonce":4124921646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6829","nonce":4124921646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6833","nonce":4124921646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6831","nonce":4124921646}]},"public_addr":"172.21.15.19:6827/4124921646","cluster_addr":"172.21.15.19:6829/4124921646","heartbeat_back_addr":"172.21.15.19:6833/4124921646","heartbeat_front_addr":"172.21.15.19:6831/4124921646","state":["exists","up"]},{"osd":10,"uuid":"71eaa9a7-667d-4cb7-935c-42177d20a279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6825","nonce":1428931817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6827","nonce":1428931817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6831","nonce":1428931817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6829","nonce":1428931817}]},"public_addr":"172.21.15.27:6825/1428931817","cluster_addr":"172.21.15.27:6827/1428931817","heartbeat_back_addr":"172.21.15.27:6831/1428931817","heartbeat_front_addr":"172.21.15.27:6829/1428931817","state":["exists","up"]},{"osd":11,"uuid":"3330627f-ab5d-4e6e-83cf-fc8b74c54ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6825","nonce":4045070679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6827","nonce":4045070679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6831","nonce":4045070679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6829","nonce":4045070679}]},"public_addr":"172.21.15.64:6825/4045070679","cluster_addr":"172.21.15.64:6827/4045070679","heartbeat_back_addr":"172.21.15.64:6831/4045070679","heartbeat_front_addr":"172.21.15.64:6829/4045070679","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:19:27.902633+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:19:54.305685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:20:18.727053+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:20:53.579199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:21:21.058018+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:21:47.091494+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:22:23.269018+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:22:51.863628+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:23:18.537774+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:23:55.763993+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:24:24.956178+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:24:52.681201+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/1956516258":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/1816772181":"2024-05-08T02:17:31.252868+0000","172.21.15.19:6801/174304059":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/1514051750":"2024-05-08T02:16:54.732592+0000","172.21.15.19:6800/174304059":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/3930045871":"2024-05-08T02:16:54.732592+0000","172.21.15.19:6800/3614957309":"2024-05-08T02:16:54.732592+0000","172.21.15.19:0/1328365399":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/139593344":"2024-05-08T02:16:54.732592+0000","172.21.15.19:6801/3614957309":"2024-05-08T02:16:54.732592+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-07T02:31:48.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.754+0000 7fe1bbe00640 1 -- 172.21.15.19:0/2129649450 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fe1a00b28f0 msgr2=0x7fe1a00b4e00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-07T02:31:48.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.754+0000 7fe1bbe00640 1 --2- 172.21.15.19:0/2129649450 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fe1a00b28f0 0x7fe1a00b4e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:48.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.754+0000 7fe1bbe00640 1 -- 172.21.15.19:0/2129649450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1bc103ac0 msgr2=0x7fe1bc19a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:48.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.754+0000 7fe1bbe00640 1 --2- 172.21.15.19:0/2129649450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1bc103ac0 0x7fe1bc19a0c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac00e980 tx=0x7fe1ac00ee40 comp rx=0 tx=0).stop 2024-05-07T02:31:48.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.754+0000 7fe1bbe00640 1 -- 172.21.15.19:0/2129649450 shutdown_connections 2024-05-07T02:31:48.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.754+0000 7fe1bbe00640 1 --2- 172.21.15.19:0/2129649450 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7fe1a00b28f0 0x7fe1a00b4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:48.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.754+0000 7fe1bbe00640 1 --2- 172.21.15.19:0/2129649450 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1bc109a00 0x7fe1bc1a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:48.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.754+0000 7fe1bbe00640 1 --2- 172.21.15.19:0/2129649450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe1bc104d00 0x7fe1bc19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:48.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.754+0000 7fe1bbe00640 1 --2- 172.21.15.19:0/2129649450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe1bc103ac0 0x7fe1bc19a0c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:48.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.754+0000 7fe1bbe00640 1 -- 172.21.15.19:0/2129649450 >> 172.21.15.19:0/2129649450 conn(0x7fe1bc0fecb0 msgr2=0x7fe1bc1059e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:48.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.755+0000 7fe1bbe00640 1 -- 172.21.15.19:0/2129649450 shutdown_connections 2024-05-07T02:31:48.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:48.755+0000 7fe1bbe00640 1 -- 172.21.15.19:0/2129649450 wait complete. 2024-05-07T02:31:48.776 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:31:48 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:31:48.513+0000 7fa7ea0ad200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-07T02:31:48.776 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:31:48 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:31:48.651+0000 7fa7ea0ad200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-07T02:31:48.779 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-07T02:20:21.439697+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-07T02:26:06.178814+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-07T02:26:10.705693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-07T02:26:23.534133+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-07T02:31:48.780 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-07T02:31:49.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.054+0000 7f1448a00640 1 -- 172.21.15.19:0/2651242263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f144410ba30 msgr2=0x7f144410de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:49.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.054+0000 7f1448a00640 1 --2- 172.21.15.19:0/2651242263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f144410ba30 0x7f144410de50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f143c009ca0 tx=0x7f143c02a8c0 comp rx=0 tx=0).stop 2024-05-07T02:31:49.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.054+0000 7f1448a00640 1 -- 172.21.15.19:0/2651242263 shutdown_connections 2024-05-07T02:31:49.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.054+0000 7f1448a00640 1 --2- 172.21.15.19:0/2651242263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f144410ba30 0x7f144410de50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:49.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.054+0000 7f1448a00640 1 --2- 172.21.15.19:0/2651242263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14440759d0 0x7f144410b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:49.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.054+0000 7f1448a00640 1 --2- 172.21.15.19:0/2651242263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1444075080 0x7f1444075490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:49.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.054+0000 7f1448a00640 1 -- 172.21.15.19:0/2651242263 >> 172.21.15.19:0/2651242263 conn(0x7f14440fec70 msgr2=0x7f1444101120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:49.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.054+0000 7f1448a00640 1 -- 172.21.15.19:0/2651242263 shutdown_connections 2024-05-07T02:31:49.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.054+0000 7f1448a00640 1 -- 172.21.15.19:0/2651242263 wait complete. 2024-05-07T02:31:49.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.056+0000 7f1448a00640 1 Processor -- start 2024-05-07T02:31:49.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.056+0000 7f1448a00640 1 -- start start 2024-05-07T02:31:49.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.056+0000 7f1448a00640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1444075080 0x7f144419c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:49.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.056+0000 7f1448a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14440759d0 0x7f144419c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:49.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.057+0000 7f1448a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f144410ba30 0x7f14441a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:49.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.057+0000 7f1448a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1444111810 con 0x7f14440759d0 2024-05-07T02:31:49.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.057+0000 7f1448a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1444111690 con 0x7f144410ba30 2024-05-07T02:31:49.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.057+0000 7f1448a00640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1444111990 con 0x7f1444075080 2024-05-07T02:31:49.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.057+0000 7f1442a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14440759d0 0x7f144419c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:49.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.057+0000 7f1442a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14440759d0 0x7f144419c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50608/0 (socket says 172.21.15.19:50608) 2024-05-07T02:31:49.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.057+0000 7f1442a00640 1 -- 172.21.15.19:0/3855056991 learned_addr learned my addr 172.21.15.19:0/3855056991 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:31:49.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.057+0000 7f1443400640 1 --2- 172.21.15.19:0/3855056991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1444075080 0x7f144419c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:49.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.057+0000 7f1443e00640 1 --2- 172.21.15.19:0/3855056991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f144410ba30 0x7f14441a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:49.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.057+0000 7f1442a00640 1 -- 172.21.15.19:0/3855056991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1444075080 msgr2=0x7f144419c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:49.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.057+0000 7f1442a00640 1 --2- 172.21.15.19:0/3855056991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1444075080 0x7f144419c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:49.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.057+0000 7f1442a00640 1 -- 172.21.15.19:0/3855056991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f144410ba30 msgr2=0x7f14441a3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:49.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.057+0000 7f1442a00640 1 --2- 172.21.15.19:0/3855056991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f144410ba30 0x7f14441a3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:49.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.057+0000 7f1442a00640 1 -- 172.21.15.19:0/3855056991 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14441a3f00 con 0x7f14440759d0 2024-05-07T02:31:49.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.058+0000 7f1442a00640 1 --2- 172.21.15.19:0/3855056991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14440759d0 0x7f144419c950 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f143000e9f0 tx=0x7f143000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:49.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.058+0000 7f143be00640 1 -- 172.21.15.19:0/3855056991 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143000cd80 con 0x7f14440759d0 2024-05-07T02:31:49.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.058+0000 7f143be00640 1 -- 172.21.15.19:0/3855056991 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14300054d0 con 0x7f14440759d0 2024-05-07T02:31:49.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.058+0000 7f143be00640 1 -- 172.21.15.19:0/3855056991 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1430010690 con 0x7f14440759d0 2024-05-07T02:31:49.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.058+0000 7f1448a00640 1 -- 172.21.15.19:0/3855056991 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14441a41f0 con 0x7f14440759d0 2024-05-07T02:31:49.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.058+0000 7f1448a00640 1 -- 172.21.15.19:0/3855056991 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14441a46f0 con 0x7f14440759d0 2024-05-07T02:31:49.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.059+0000 7f1448a00640 1 -- 172.21.15.19:0/3855056991 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f144410abf0 con 0x7f14440759d0 2024-05-07T02:31:49.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.063+0000 7f143be00640 1 -- 172.21.15.19:0/3855056991 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7f1430004e90 con 0x7f14440759d0 2024-05-07T02:31:49.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.065+0000 7f143be00640 1 --2- 172.21.15.19:0/3855056991 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f14280b2840 0x7f14280b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:49.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.065+0000 7f1443400640 1 -- 172.21.15.19:0/3855056991 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f14280b2840 msgr2=0x7f14280b4d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2059564551 2024-05-07T02:31:49.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.065+0000 7f143be00640 1 -- 172.21.15.19:0/3855056991 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f14300e2aa0 con 0x7f14440759d0 2024-05-07T02:31:49.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.065+0000 7f1443400640 1 --2- 172.21.15.19:0/3855056991 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f14280b2840 0x7f14280b4d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-07T02:31:49.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.070+0000 7f143be00640 1 -- 172.21.15.19:0/3855056991 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f14300aa3b0 con 0x7f14440759d0 2024-05-07T02:31:49.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.266+0000 7f1443400640 1 -- 172.21.15.19:0/3855056991 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f14280b2840 msgr2=0x7f14280b4d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2059564551 2024-05-07T02:31:49.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.266+0000 7f1443400640 1 --2- 172.21.15.19:0/3855056991 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f14280b2840 0x7f14280b4d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-07T02:31:49.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.282+0000 7f1448a00640 1 -- 172.21.15.19:0/3855056991 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f14440638d0 con 0x7f14440759d0 2024-05-07T02:31:49.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.283+0000 7f143be00640 1 -- 172.21.15.19:0/3855056991 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f14300a9cb0 con 0x7f14440759d0 2024-05-07T02:31:49.283 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-05-07T02:31:49.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.286+0000 7f1448a00640 1 -- 172.21.15.19:0/3855056991 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f14280b2840 msgr2=0x7f14280b4d50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-07T02:31:49.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.286+0000 7f1448a00640 1 --2- 172.21.15.19:0/3855056991 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f14280b2840 0x7f14280b4d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:49.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.286+0000 7f1448a00640 1 -- 172.21.15.19:0/3855056991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14440759d0 msgr2=0x7f144419c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:49.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.286+0000 7f1448a00640 1 --2- 172.21.15.19:0/3855056991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14440759d0 0x7f144419c950 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f143000e9f0 tx=0x7f143000edf0 comp rx=0 tx=0).stop 2024-05-07T02:31:49.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.286+0000 7f1448a00640 1 -- 172.21.15.19:0/3855056991 shutdown_connections 2024-05-07T02:31:49.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.286+0000 7f1448a00640 1 --2- 172.21.15.19:0/3855056991 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f14280b2840 0x7f14280b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:49.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.286+0000 7f1448a00640 1 --2- 172.21.15.19:0/3855056991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f144410ba30 0x7f14441a3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:49.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.286+0000 7f1448a00640 1 --2- 172.21.15.19:0/3855056991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14440759d0 0x7f144419c950 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:49.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.286+0000 7f1448a00640 1 --2- 172.21.15.19:0/3855056991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1444075080 0x7f144419c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:49.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.286+0000 7f1448a00640 1 -- 172.21.15.19:0/3855056991 >> 172.21.15.19:0/3855056991 conn(0x7f14440fec70 msgr2=0x7f14440ffab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:49.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.286+0000 7f1448a00640 1 -- 172.21.15.19:0/3855056991 shutdown_connections 2024-05-07T02:31:49.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.286+0000 7f1448a00640 1 -- 172.21.15.19:0/3855056991 wait complete. 2024-05-07T02:31:49.309 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:49 smithi019 ceph-mon[29781]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-07T02:31:49.309 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:49 smithi019 ceph-mon[29781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:31:49.309 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:49 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/2129649450' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-07T02:31:49.312 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-07T02:31:49.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:49 smithi027 ceph-mon[34398]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-07T02:31:49.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:49 smithi027 ceph-mon[34398]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:31:49.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:49 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/2129649450' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-07T02:31:49.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.591+0000 7f49f6200640 1 -- 172.21.15.19:0/1279808779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49f0104ca0 msgr2=0x7f49f01050b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:49.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.591+0000 7f49f6200640 1 --2- 172.21.15.19:0/1279808779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49f0104ca0 0x7f49f01050b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f49dc009900 tx=0x7f49dc02a3f0 comp rx=0 tx=0).stop 2024-05-07T02:31:49.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.591+0000 7f49f6200640 1 -- 172.21.15.19:0/1279808779 shutdown_connections 2024-05-07T02:31:49.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.591+0000 7f49f6200640 1 --2- 172.21.15.19:0/1279808779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49f01099a0 0x7f49f010bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:49.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.591+0000 7f49f6200640 1 --2- 172.21.15.19:0/1279808779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49f0104ca0 0x7f49f01050b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:49.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.591+0000 7f49f6200640 1 --2- 172.21.15.19:0/1279808779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49f0103a60 0x7f49f0104760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:49.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.591+0000 7f49f6200640 1 -- 172.21.15.19:0/1279808779 >> 172.21.15.19:0/1279808779 conn(0x7f49f00fec50 msgr2=0x7f49f0101100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:49.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.591+0000 7f49f6200640 1 -- 172.21.15.19:0/1279808779 shutdown_connections 2024-05-07T02:31:49.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.592+0000 7f49f6200640 1 -- 172.21.15.19:0/1279808779 wait complete. 2024-05-07T02:31:49.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.593+0000 7f49f6200640 1 Processor -- start 2024-05-07T02:31:49.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.593+0000 7f49f6200640 1 -- start start 2024-05-07T02:31:49.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.594+0000 7f49f6200640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49f0103a60 0x7f49f019c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:49.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.594+0000 7f49f6200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49f0104ca0 0x7f49f019c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:49.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.594+0000 7f49f6200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49f01099a0 0x7f49f01a3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:49.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.594+0000 7f49f6200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f49f010f580 con 0x7f49f01099a0 2024-05-07T02:31:49.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.594+0000 7f49f6200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f49f010f400 con 0x7f49f0104ca0 2024-05-07T02:31:49.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.594+0000 7f49f6200640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f49f010f700 con 0x7f49f0103a60 2024-05-07T02:31:49.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.594+0000 7f49f4e00640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49f0103a60 0x7f49f019c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:49.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.594+0000 7f49f4e00640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49f0103a60 0x7f49f019c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:35980/0 (socket says 172.21.15.19:35980) 2024-05-07T02:31:49.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.594+0000 7f49f4e00640 1 -- 172.21.15.19:0/2632498344 learned_addr learned my addr 172.21.15.19:0/2632498344 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:31:49.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.594+0000 7f49efe00640 1 --2- 172.21.15.19:0/2632498344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49f0104ca0 0x7f49f019c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:49.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.594+0000 7f49f5800640 1 --2- 172.21.15.19:0/2632498344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49f01099a0 0x7f49f01a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:49.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.595+0000 7f49f4e00640 1 -- 172.21.15.19:0/2632498344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49f0104ca0 msgr2=0x7f49f019c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:49.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.595+0000 7f49f4e00640 1 --2- 172.21.15.19:0/2632498344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49f0104ca0 0x7f49f019c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:49.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.595+0000 7f49f4e00640 1 -- 172.21.15.19:0/2632498344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49f01099a0 msgr2=0x7f49f01a3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:49.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.595+0000 7f49f4e00640 1 --2- 172.21.15.19:0/2632498344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49f01099a0 0x7f49f01a3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:49.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.595+0000 7f49f4e00640 1 -- 172.21.15.19:0/2632498344 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f01a3ef0 con 0x7f49f0103a60 2024-05-07T02:31:49.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.595+0000 7f49f5800640 1 --2- 172.21.15.19:0/2632498344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49f01099a0 0x7f49f01a3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-07T02:31:49.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.595+0000 7f49f4e00640 1 --2- 172.21.15.19:0/2632498344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49f0103a60 0x7f49f019c3b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f49e400e9f0 tx=0x7f49e400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:49.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.595+0000 7f49ed600640 1 -- 172.21.15.19:0/2632498344 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e400ccc0 con 0x7f49f0103a60 2024-05-07T02:31:49.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.595+0000 7f49ed600640 1 -- 172.21.15.19:0/2632498344 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49e40035f0 con 0x7f49f0103a60 2024-05-07T02:31:49.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.596+0000 7f49f6200640 1 -- 172.21.15.19:0/2632498344 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49f01a41e0 con 0x7f49f0103a60 2024-05-07T02:31:49.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.596+0000 7f49ed600640 1 -- 172.21.15.19:0/2632498344 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e400ce60 con 0x7f49f0103a60 2024-05-07T02:31:49.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.596+0000 7f49f6200640 1 -- 172.21.15.19:0/2632498344 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49f01a46e0 con 0x7f49f0103a60 2024-05-07T02:31:49.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.597+0000 7f49f6200640 1 -- 172.21.15.19:0/2632498344 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49f01a49a0 con 0x7f49f0103a60 2024-05-07T02:31:49.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.599+0000 7f49ed600640 1 -- 172.21.15.19:0/2632498344 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7f49e4004030 con 0x7f49f0103a60 2024-05-07T02:31:49.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.601+0000 7f49ed600640 1 --2- 172.21.15.19:0/2632498344 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f49d80b2840 0x7f49d80b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:49.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.601+0000 7f49efe00640 1 -- 172.21.15.19:0/2632498344 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f49d80b2840 msgr2=0x7f49d80b4d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2059564551 2024-05-07T02:31:49.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.601+0000 7f49ed600640 1 -- 172.21.15.19:0/2632498344 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9179+0+0 (secure 0 0 0) 0x7f49e40e2ab0 con 0x7f49f0103a60 2024-05-07T02:31:49.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.601+0000 7f49efe00640 1 --2- 172.21.15.19:0/2632498344 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f49d80b2840 0x7f49d80b4d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-07T02:31:49.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.605+0000 7f49ed600640 1 -- 172.21.15.19:0/2632498344 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f49e40aa3c0 con 0x7f49f0103a60 2024-05-07T02:31:49.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:49 smithi064 ceph-mon[34699]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-07T02:31:49.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:49 smithi064 ceph-mon[34699]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:31:49.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:49 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/2129649450' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-07T02:31:49.644 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:31:49 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:31:49.496+0000 7fa7ea0ad200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-07T02:31:49.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.802+0000 7f49efe00640 1 -- 172.21.15.19:0/2632498344 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f49d80b2840 msgr2=0x7f49d80b4d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2059564551 2024-05-07T02:31:49.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.802+0000 7f49efe00640 1 --2- 172.21.15.19:0/2632498344 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f49d80b2840 0x7f49d80b4d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-07T02:31:49.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.815+0000 7f49f6200640 1 -- 172.21.15.19:0/2632498344 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f49f01a4b70 con 0x7f49f0103a60 2024-05-07T02:31:49.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.815+0000 7f49ed600640 1 -- 172.21.15.19:0/2632498344 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f49e40a9cc0 con 0x7f49f0103a60 2024-05-07T02:31:49.815 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-05-07T02:31:49.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.818+0000 7f49f6200640 1 -- 172.21.15.19:0/2632498344 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f49d80b2840 msgr2=0x7f49d80b4d50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-07T02:31:49.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.818+0000 7f49f6200640 1 --2- 172.21.15.19:0/2632498344 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f49d80b2840 0x7f49d80b4d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:49.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.818+0000 7f49f6200640 1 -- 172.21.15.19:0/2632498344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49f0103a60 msgr2=0x7f49f019c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:49.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.818+0000 7f49f6200640 1 --2- 172.21.15.19:0/2632498344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49f0103a60 0x7f49f019c3b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f49e400e9f0 tx=0x7f49e400eeb0 comp rx=0 tx=0).stop 2024-05-07T02:31:49.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.818+0000 7f49f6200640 1 -- 172.21.15.19:0/2632498344 shutdown_connections 2024-05-07T02:31:49.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.818+0000 7f49f6200640 1 --2- 172.21.15.19:0/2632498344 >> [v2:172.21.15.19:6800/2059564551,v1:172.21.15.19:6801/2059564551] conn(0x7f49d80b2840 0x7f49d80b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:49.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.818+0000 7f49f6200640 1 --2- 172.21.15.19:0/2632498344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49f01099a0 0x7f49f01a3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:49.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.818+0000 7f49f6200640 1 --2- 172.21.15.19:0/2632498344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49f0104ca0 0x7f49f019c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:49.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.818+0000 7f49f6200640 1 --2- 172.21.15.19:0/2632498344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f49f0103a60 0x7f49f019c3b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:49.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.818+0000 7f49f6200640 1 -- 172.21.15.19:0/2632498344 >> 172.21.15.19:0/2632498344 conn(0x7f49f00fec50 msgr2=0x7f49f00ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:49.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.818+0000 7f49f6200640 1 -- 172.21.15.19:0/2632498344 shutdown_connections 2024-05-07T02:31:49.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:49.818+0000 7f49f6200640 1 -- 172.21.15.19:0/2632498344 wait complete. 2024-05-07T02:31:49.852 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-07T02:31:50.078 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:31:49 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:31:49.810+0000 7fa7ea0ad200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-07T02:31:50.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.185+0000 7f292ac00640 1 -- 172.21.15.19:0/404313372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2924103c20 msgr2=0x7f2924104920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:50.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.185+0000 7f292ac00640 1 --2- 172.21.15.19:0/404313372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2924103c20 0x7f2924104920 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2914009900 tx=0x7f291402a3f0 comp rx=0 tx=0).stop 2024-05-07T02:31:50.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.185+0000 7f292ac00640 1 -- 172.21.15.19:0/404313372 shutdown_connections 2024-05-07T02:31:50.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.185+0000 7f292ac00640 1 --2- 172.21.15.19:0/404313372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2924109b60 0x7f292410bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:50.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.185+0000 7f292ac00640 1 --2- 172.21.15.19:0/404313372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2924104e60 0x7f2924105270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:50.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.185+0000 7f292ac00640 1 --2- 172.21.15.19:0/404313372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2924103c20 0x7f2924104920 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:50.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.185+0000 7f292ac00640 1 -- 172.21.15.19:0/404313372 >> 172.21.15.19:0/404313372 conn(0x7f29240fedf0 msgr2=0x7f29241012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:50.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.185+0000 7f292ac00640 1 -- 172.21.15.19:0/404313372 shutdown_connections 2024-05-07T02:31:50.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.185+0000 7f292ac00640 1 -- 172.21.15.19:0/404313372 wait complete. 2024-05-07T02:31:50.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.186+0000 7f292ac00640 1 Processor -- start 2024-05-07T02:31:50.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.187+0000 7f292ac00640 1 -- start start 2024-05-07T02:31:50.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.187+0000 7f292ac00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2924103c20 0x7f292419a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:50.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.187+0000 7f292ac00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2924104e60 0x7f292419a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:50.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.187+0000 7f292ac00640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2924109b60 0x7f29241a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:50.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.187+0000 7f292ac00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f292410f700 con 0x7f2924104e60 2024-05-07T02:31:50.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.187+0000 7f292ac00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f292410f580 con 0x7f2924103c20 2024-05-07T02:31:50.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.187+0000 7f292ac00640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f292410f880 con 0x7f2924109b60 2024-05-07T02:31:50.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.187+0000 7f2928c00640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2924109b60 0x7f29241a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:50.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.188+0000 7f2928c00640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2924109b60 0x7f29241a1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:36088/0 (socket says 172.21.15.19:36088) 2024-05-07T02:31:50.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.188+0000 7f2928c00640 1 -- 172.21.15.19:0/2260517036 learned_addr learned my addr 172.21.15.19:0/2260517036 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:31:50.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.188+0000 7f2923400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2924104e60 0x7f292419a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:50.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.188+0000 7f2923e00640 1 --2- 172.21.15.19:0/2260517036 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2924103c20 0x7f292419a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:50.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.188+0000 7f2928c00640 1 -- 172.21.15.19:0/2260517036 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2924103c20 msgr2=0x7f292419a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:50.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.188+0000 7f2928c00640 1 --2- 172.21.15.19:0/2260517036 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2924103c20 0x7f292419a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:50.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.188+0000 7f2928c00640 1 -- 172.21.15.19:0/2260517036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2924104e60 msgr2=0x7f292419a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:50.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.188+0000 7f2928c00640 1 --2- 172.21.15.19:0/2260517036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2924104e60 0x7f292419a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:50.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.188+0000 7f2928c00640 1 -- 172.21.15.19:0/2260517036 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29241a1e70 con 0x7f2924109b60 2024-05-07T02:31:50.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.188+0000 7f2923400640 1 --2- 172.21.15.19:0/2260517036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2924104e60 0x7f292419a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:31:50.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.189+0000 7f2928c00640 1 --2- 172.21.15.19:0/2260517036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2924109b60 0x7f29241a1800 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f291c00eb10 tx=0x7f291c00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:50.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.194+0000 7f2920c00640 1 -- 172.21.15.19:0/2260517036 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291c004b20 con 0x7f2924109b60 2024-05-07T02:31:50.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.194+0000 7f2920c00640 1 -- 172.21.15.19:0/2260517036 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f291c004cc0 con 0x7f2924109b60 2024-05-07T02:31:50.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.194+0000 7f2920c00640 1 -- 172.21.15.19:0/2260517036 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291c010670 con 0x7f2924109b60 2024-05-07T02:31:50.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.194+0000 7f292ac00640 1 -- 172.21.15.19:0/2260517036 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29241a2160 con 0x7f2924109b60 2024-05-07T02:31:50.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.194+0000 7f292ac00640 1 -- 172.21.15.19:0/2260517036 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29241a2660 con 0x7f2924109b60 2024-05-07T02:31:50.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.195+0000 7f292ac00640 1 -- 172.21.15.19:0/2260517036 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2924108970 con 0x7f2924109b60 2024-05-07T02:31:50.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.200+0000 7f2920c00640 1 -- 172.21.15.19:0/2260517036 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 21) ==== 149337+0+0 (secure 0 0 0) 0x7f291c005000 con 0x7f2924109b60 2024-05-07T02:31:50.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.202+0000 7f2920c00640 1 -- 172.21.15.19:0/2260517036 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f291c0e2830 con 0x7f2924109b60 2024-05-07T02:31:50.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.204+0000 7f2920c00640 1 -- 172.21.15.19:0/2260517036 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f291c0a6050 con 0x7f2924109b60 2024-05-07T02:31:50.211 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:50 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/3855056991' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-07T02:31:50.211 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:50 smithi027 ceph-mon[34398]: Active manager daemon x restarted 2024-05-07T02:31:50.211 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:50 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/2632498344' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-07T02:31:50.211 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:50 smithi027 ceph-mon[34398]: Activating manager daemon x 2024-05-07T02:31:50.211 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:50 smithi027 ceph-mon[34398]: osdmap e90: 12 total, 12 up, 12 in 2024-05-07T02:31:50.211 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:50 smithi027 ceph-mon[34398]: mgrmap e21: x(active, starting, since 0.0129999s), standbys: z, y 2024-05-07T02:31:50.211 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:50 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-07T02:31:50.211 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:50 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-07T02:31:50.212 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:50 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-07T02:31:50.212 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:50 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-07T02:31:50.212 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:50 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-07T02:31:50.212 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:50 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-07T02:31:50.212 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:50 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-07T02:31:50.212 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:50 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-07T02:31:50.212 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:50 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-07T02:31:50.212 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:50 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-07T02:31:50.212 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:50 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-07T02:31:50.212 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:50 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-07T02:31:50.212 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:50 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-07T02:31:50.212 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:50 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-07T02:31:50.212 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:50 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-07T02:31:50.212 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:50 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-07T02:31:50.212 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:50 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-07T02:31:50.213 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:50 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-07T02:31:50.213 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:50 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-07T02:31:50.213 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:50 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-07T02:31:50.213 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:50 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-07T02:31:50.213 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:50 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-07T02:31:50.213 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:50 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-07T02:31:50.213 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:50 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-07T02:31:50.213 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:50 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-07T02:31:50.213 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:50 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-07T02:31:50.214 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:50 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-07T02:31:50.214 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:50 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-07T02:31:50.214 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:50 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-07T02:31:50.214 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:50 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-07T02:31:50.214 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:50 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-07T02:31:50.214 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:50 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-07T02:31:50.214 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:50 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-07T02:31:50.214 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:50 smithi027 ceph-mon[34398]: Manager daemon x is now available 2024-05-07T02:31:50.214 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:50 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:31:50.214 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:50 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-07T02:31:50.214 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:50 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-07T02:31:50.214 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:50 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-07T02:31:50.214 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:50 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-07T02:31:50.337 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:50 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/3855056991' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-07T02:31:50.337 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:50 smithi019 ceph-mon[29781]: Active manager daemon x restarted 2024-05-07T02:31:50.337 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:50 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/2632498344' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-07T02:31:50.337 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:50 smithi019 ceph-mon[29781]: Activating manager daemon x 2024-05-07T02:31:50.337 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:50 smithi019 ceph-mon[29781]: osdmap e90: 12 total, 12 up, 12 in 2024-05-07T02:31:50.337 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:50 smithi019 ceph-mon[29781]: mgrmap e21: x(active, starting, since 0.0129999s), standbys: z, y 2024-05-07T02:31:50.337 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:50 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-07T02:31:50.337 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:50 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-07T02:31:50.337 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:50 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-07T02:31:50.337 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:50 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-07T02:31:50.338 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:50 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-07T02:31:50.338 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:50 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-07T02:31:50.338 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:50 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-07T02:31:50.338 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:50 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-07T02:31:50.338 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:50 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-07T02:31:50.338 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:50 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-07T02:31:50.338 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:50 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-07T02:31:50.338 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:50 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-07T02:31:50.338 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:50 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-07T02:31:50.338 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:50 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-07T02:31:50.338 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:50 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-07T02:31:50.338 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:50 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-07T02:31:50.338 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:50 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-07T02:31:50.339 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:50 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-07T02:31:50.339 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:50 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-07T02:31:50.339 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:50 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-07T02:31:50.339 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:50 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-07T02:31:50.339 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:50 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-07T02:31:50.339 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:50 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-07T02:31:50.339 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:50 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-07T02:31:50.339 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:50 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-07T02:31:50.339 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:50 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-07T02:31:50.339 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:50 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-07T02:31:50.339 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:50 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-07T02:31:50.339 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:50 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-07T02:31:50.339 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:50 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-07T02:31:50.339 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:50 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-07T02:31:50.339 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:50 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-07T02:31:50.340 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:50 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-07T02:31:50.340 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:50 smithi019 ceph-mon[29781]: Manager daemon x is now available 2024-05-07T02:31:50.340 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:50 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:31:50.340 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:50 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-07T02:31:50.340 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:50 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-07T02:31:50.340 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:50 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-07T02:31:50.340 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:50 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-07T02:31:50.340 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:31:50 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:31:50.202+0000 7fa73aa00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-07T02:31:50.340 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:31:50 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:31:50.202+0000 7fa73aa00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-07T02:31:50.340 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:31:50 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:31:50.202+0000 7fa73aa00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-07T02:31:50.340 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:31:50 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:31:50.202+0000 7fa73aa00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-07T02:31:50.340 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:31:50 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:31:50.202+0000 7fa73aa00700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-07T02:31:50.341 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:31:50 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:31:50.206+0000 7fa73e600700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-07T02:31:50.341 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:31:50 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:31:50.207+0000 7fa73e600700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-07T02:31:50.341 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:31:50 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:31:50.207+0000 7fa73e600700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-07T02:31:50.341 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:31:50 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:31:50.207+0000 7fa73e600700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-07T02:31:50.341 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:31:50 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:31:50.207+0000 7fa73e600700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-07T02:31:50.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.457+0000 7f292ac00640 1 -- 172.21.15.19:0/2260517036 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f292402ddc0 con 0x7f2924109b60 2024-05-07T02:31:50.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.458+0000 7f2920c00640 1 -- 172.21.15.19:0/2260517036 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f291c0aa4c0 con 0x7f2924109b60 2024-05-07T02:31:50.458 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-05-07T02:31:50.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.461+0000 7f292ac00640 1 -- 172.21.15.19:0/2260517036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2924109b60 msgr2=0x7f29241a1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:50.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.461+0000 7f292ac00640 1 --2- 172.21.15.19:0/2260517036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2924109b60 0x7f29241a1800 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f291c00eb10 tx=0x7f291c00c2d0 comp rx=0 tx=0).stop 2024-05-07T02:31:50.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.461+0000 7f292ac00640 1 -- 172.21.15.19:0/2260517036 shutdown_connections 2024-05-07T02:31:50.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.461+0000 7f292ac00640 1 --2- 172.21.15.19:0/2260517036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2924109b60 0x7f29241a1800 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:50.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.461+0000 7f292ac00640 1 --2- 172.21.15.19:0/2260517036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2924104e60 0x7f292419a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:50.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.461+0000 7f292ac00640 1 --2- 172.21.15.19:0/2260517036 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2924103c20 0x7f292419a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:50.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.461+0000 7f292ac00640 1 -- 172.21.15.19:0/2260517036 >> 172.21.15.19:0/2260517036 conn(0x7f29240fedf0 msgr2=0x7f2924105b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:50.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.461+0000 7f292ac00640 1 -- 172.21.15.19:0/2260517036 shutdown_connections 2024-05-07T02:31:50.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.461+0000 7f292ac00640 1 -- 172.21.15.19:0/2260517036 wait complete. 2024-05-07T02:31:50.499 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-07T02:31:50.610 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:50 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/3855056991' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-07T02:31:50.610 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:50 smithi064 ceph-mon[34699]: Active manager daemon x restarted 2024-05-07T02:31:50.610 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:50 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/2632498344' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-07T02:31:50.610 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:50 smithi064 ceph-mon[34699]: Activating manager daemon x 2024-05-07T02:31:50.610 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:50 smithi064 ceph-mon[34699]: osdmap e90: 12 total, 12 up, 12 in 2024-05-07T02:31:50.611 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:50 smithi064 ceph-mon[34699]: mgrmap e21: x(active, starting, since 0.0129999s), standbys: z, y 2024-05-07T02:31:50.611 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:50 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-07T02:31:50.611 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:50 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-07T02:31:50.611 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:50 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-07T02:31:50.611 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:50 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-07T02:31:50.611 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:50 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-07T02:31:50.611 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:50 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-07T02:31:50.611 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:50 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-07T02:31:50.611 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:50 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-07T02:31:50.611 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:50 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-07T02:31:50.611 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:50 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-07T02:31:50.611 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:50 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-07T02:31:50.611 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:50 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-07T02:31:50.611 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:50 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-07T02:31:50.611 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:50 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-07T02:31:50.612 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:50 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-07T02:31:50.612 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:50 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-07T02:31:50.612 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:50 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-07T02:31:50.612 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:50 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-07T02:31:50.612 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:50 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-07T02:31:50.612 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:50 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-07T02:31:50.612 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:50 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-07T02:31:50.612 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:50 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-07T02:31:50.612 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:50 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-07T02:31:50.612 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:50 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-07T02:31:50.612 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:50 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-07T02:31:50.613 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:50 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-07T02:31:50.613 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:50 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-07T02:31:50.613 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:50 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-07T02:31:50.613 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:50 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-07T02:31:50.613 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:50 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-07T02:31:50.613 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:50 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-07T02:31:50.613 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:50 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-07T02:31:50.613 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:50 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-07T02:31:50.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:50 smithi064 ceph-mon[34699]: Manager daemon x is now available 2024-05-07T02:31:50.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:50 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:31:50.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:50 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-07T02:31:50.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:50 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-07T02:31:50.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:50 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-07T02:31:50.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:50 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-07T02:31:50.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.758+0000 7f6549200640 1 -- 172.21.15.19:0/2897327157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6544103a60 msgr2=0x7f6544104760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:50.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.758+0000 7f6549200640 1 --2- 172.21.15.19:0/2897327157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6544103a60 0x7f6544104760 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6534009900 tx=0x7f653402a3f0 comp rx=0 tx=0).stop 2024-05-07T02:31:50.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.758+0000 7f6549200640 1 -- 172.21.15.19:0/2897327157 shutdown_connections 2024-05-07T02:31:50.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.758+0000 7f6549200640 1 --2- 172.21.15.19:0/2897327157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65441099a0 0x7f654410bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:50.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.758+0000 7f6549200640 1 --2- 172.21.15.19:0/2897327157 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6544104ca0 0x7f65441050b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:50.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.758+0000 7f6549200640 1 --2- 172.21.15.19:0/2897327157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6544103a60 0x7f6544104760 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:50.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.758+0000 7f6549200640 1 -- 172.21.15.19:0/2897327157 >> 172.21.15.19:0/2897327157 conn(0x7f65440fec50 msgr2=0x7f6544101100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:50.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.758+0000 7f6549200640 1 -- 172.21.15.19:0/2897327157 shutdown_connections 2024-05-07T02:31:50.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.758+0000 7f6549200640 1 -- 172.21.15.19:0/2897327157 wait complete. 2024-05-07T02:31:50.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.759+0000 7f6549200640 1 Processor -- start 2024-05-07T02:31:50.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.759+0000 7f6549200640 1 -- start start 2024-05-07T02:31:50.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.760+0000 7f6549200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6544103a60 0x7f654419c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:50.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.760+0000 7f6549200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6544104ca0 0x7f654419c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:50.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.760+0000 7f6549200640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65441099a0 0x7f65441a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:50.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.760+0000 7f6549200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f654410f560 con 0x7f6544104ca0 2024-05-07T02:31:50.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.760+0000 7f6549200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f654410f3e0 con 0x7f6544103a60 2024-05-07T02:31:50.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.760+0000 7f6549200640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f654410f6e0 con 0x7f65441099a0 2024-05-07T02:31:50.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.760+0000 7f6543400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6544103a60 0x7f654419c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:50.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.760+0000 7f6543400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6544103a60 0x7f654419c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:57516/0 (socket says 172.21.15.19:57516) 2024-05-07T02:31:50.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.760+0000 7f6543400640 1 -- 172.21.15.19:0/3121807016 learned_addr learned my addr 172.21.15.19:0/3121807016 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:31:50.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.760+0000 7f6543e00640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65441099a0 0x7f65441a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:50.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.760+0000 7f6542a00640 1 --2- 172.21.15.19:0/3121807016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6544104ca0 0x7f654419c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:50.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.760+0000 7f6543400640 1 -- 172.21.15.19:0/3121807016 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65441099a0 msgr2=0x7f65441a38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:50.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.761+0000 7f6543400640 1 --2- 172.21.15.19:0/3121807016 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65441099a0 0x7f65441a38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:50.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.761+0000 7f6543400640 1 -- 172.21.15.19:0/3121807016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6544104ca0 msgr2=0x7f654419c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:50.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.761+0000 7f6543400640 1 --2- 172.21.15.19:0/3121807016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6544104ca0 0x7f654419c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:50.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.761+0000 7f6543400640 1 -- 172.21.15.19:0/3121807016 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65441a3f10 con 0x7f6544103a60 2024-05-07T02:31:50.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.761+0000 7f6543e00640 1 --2- 172.21.15.19:0/3121807016 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65441099a0 0x7f65441a38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:31:50.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.761+0000 7f6543400640 1 --2- 172.21.15.19:0/3121807016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6544103a60 0x7f654419c3d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6534006010 tx=0x7f6534004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:50.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.761+0000 7f653be00640 1 -- 172.21.15.19:0/3121807016 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653403f030 con 0x7f6544103a60 2024-05-07T02:31:50.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.761+0000 7f653be00640 1 -- 172.21.15.19:0/3121807016 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6534003aa0 con 0x7f6544103a60 2024-05-07T02:31:50.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.761+0000 7f6549200640 1 -- 172.21.15.19:0/3121807016 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65441a41a0 con 0x7f6544103a60 2024-05-07T02:31:50.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.761+0000 7f653be00640 1 -- 172.21.15.19:0/3121807016 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6534035070 con 0x7f6544103a60 2024-05-07T02:31:50.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.762+0000 7f6549200640 1 -- 172.21.15.19:0/3121807016 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65441a46e0 con 0x7f6544103a60 2024-05-07T02:31:50.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.763+0000 7f6549200640 1 -- 172.21.15.19:0/3121807016 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65441a4a00 con 0x7f6544103a60 2024-05-07T02:31:50.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.767+0000 7f653be00640 1 -- 172.21.15.19:0/3121807016 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 21) ==== 149337+0+0 (secure 0 0 0) 0x7f6534042030 con 0x7f6544103a60 2024-05-07T02:31:50.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.769+0000 7f653be00640 1 -- 172.21.15.19:0/3121807016 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f6534102b40 con 0x7f6544103a60 2024-05-07T02:31:50.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.769+0000 7f653be00640 1 -- 172.21.15.19:0/3121807016 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f65340ca100 con 0x7f6544103a60 2024-05-07T02:31:50.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.844+0000 7f653be00640 1 -- 172.21.15.19:0/3121807016 <== mon.2 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7f6534003c50 con 0x7f6544103a60 2024-05-07T02:31:50.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.850+0000 7f653be00640 1 --2- 172.21.15.19:0/3121807016 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f65280bad80 0x7f65280bd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:50.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.854+0000 7f6542a00640 1 --2- 172.21.15.19:0/3121807016 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f65280bad80 0x7f65280bd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:50.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.856+0000 7f6542a00640 1 --2- 172.21.15.19:0/3121807016 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f65280bad80 0x7f65280bd1a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f654419d470 tx=0x7f6530005f90 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:50.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.995+0000 7f6549200640 1 -- 172.21.15.19:0/3121807016 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f65441a4bd0 con 0x7f6544103a60 2024-05-07T02:31:50.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.996+0000 7f653be00640 1 -- 172.21.15.19:0/3121807016 <== mon.2 v2:172.21.15.27:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) ==== 103+0+11 (secure 0 0 0) 0x7f65340be6f0 con 0x7f6544103a60 2024-05-07T02:31:50.996 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-05-07T02:31:50.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.998+0000 7f6549200640 1 -- 172.21.15.19:0/3121807016 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f65280bad80 msgr2=0x7f65280bd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:50.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.999+0000 7f6549200640 1 --2- 172.21.15.19:0/3121807016 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f65280bad80 0x7f65280bd1a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f654419d470 tx=0x7f6530005f90 comp rx=0 tx=0).stop 2024-05-07T02:31:50.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.999+0000 7f6549200640 1 -- 172.21.15.19:0/3121807016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6544103a60 msgr2=0x7f654419c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:50.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.999+0000 7f6549200640 1 --2- 172.21.15.19:0/3121807016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6544103a60 0x7f654419c3d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6534006010 tx=0x7f6534004270 comp rx=0 tx=0).stop 2024-05-07T02:31:50.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.999+0000 7f6549200640 1 -- 172.21.15.19:0/3121807016 shutdown_connections 2024-05-07T02:31:50.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.999+0000 7f6549200640 1 --2- 172.21.15.19:0/3121807016 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f65280bad80 0x7f65280bd1a0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:50.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.999+0000 7f6549200640 1 --2- 172.21.15.19:0/3121807016 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65441099a0 0x7f65441a38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:50.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.999+0000 7f6549200640 1 --2- 172.21.15.19:0/3121807016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6544104ca0 0x7f654419c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:50.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.999+0000 7f6549200640 1 --2- 172.21.15.19:0/3121807016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6544103a60 0x7f654419c3d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:50.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.999+0000 7f6549200640 1 -- 172.21.15.19:0/3121807016 >> 172.21.15.19:0/3121807016 conn(0x7f65440fec50 msgr2=0x7f6544101070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:50.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.999+0000 7f6549200640 1 -- 172.21.15.19:0/3121807016 shutdown_connections 2024-05-07T02:31:50.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:50.999+0000 7f6549200640 1 -- 172.21.15.19:0/3121807016 wait complete. 2024-05-07T02:31:51.023 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-05-07T02:31:51.024 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-07T02:31:51.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:51 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/2260517036' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-07T02:31:51.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:51 smithi019 ceph-mon[29781]: [07/May/2024:02:31:50] ENGINE Bus STARTING 2024-05-07T02:31:51.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:51 smithi019 ceph-mon[29781]: [07/May/2024:02:31:50] ENGINE Serving on http://172.21.15.19:8765 2024-05-07T02:31:51.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:51 smithi019 ceph-mon[29781]: [07/May/2024:02:31:50] ENGINE Serving on https://172.21.15.19:7150 2024-05-07T02:31:51.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:51 smithi019 ceph-mon[29781]: [07/May/2024:02:31:50] ENGINE Bus STARTED 2024-05-07T02:31:51.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:51 smithi019 ceph-mon[29781]: mgrmap e22: x(active, since 1.02536s), standbys: z, y 2024-05-07T02:31:51.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:51 smithi019 ceph-mon[29781]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:31:51.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:51 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/3121807016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-07T02:31:51.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:51 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/2260517036' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-07T02:31:51.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:51 smithi027 ceph-mon[34398]: [07/May/2024:02:31:50] ENGINE Bus STARTING 2024-05-07T02:31:51.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:51 smithi027 ceph-mon[34398]: [07/May/2024:02:31:50] ENGINE Serving on http://172.21.15.19:8765 2024-05-07T02:31:51.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:51 smithi027 ceph-mon[34398]: [07/May/2024:02:31:50] ENGINE Serving on https://172.21.15.19:7150 2024-05-07T02:31:51.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:51 smithi027 ceph-mon[34398]: [07/May/2024:02:31:50] ENGINE Bus STARTED 2024-05-07T02:31:51.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:51 smithi027 ceph-mon[34398]: mgrmap e22: x(active, since 1.02536s), standbys: z, y 2024-05-07T02:31:51.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:51 smithi027 ceph-mon[34398]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:31:51.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:51 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/3121807016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-07T02:31:51.465 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:31:51.574 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:51 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/2260517036' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-07T02:31:51.574 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:51 smithi064 ceph-mon[34699]: [07/May/2024:02:31:50] ENGINE Bus STARTING 2024-05-07T02:31:51.574 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:51 smithi064 ceph-mon[34699]: [07/May/2024:02:31:50] ENGINE Serving on http://172.21.15.19:8765 2024-05-07T02:31:51.574 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:51 smithi064 ceph-mon[34699]: [07/May/2024:02:31:50] ENGINE Serving on https://172.21.15.19:7150 2024-05-07T02:31:51.574 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:51 smithi064 ceph-mon[34699]: [07/May/2024:02:31:50] ENGINE Bus STARTED 2024-05-07T02:31:51.574 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:51 smithi064 ceph-mon[34699]: mgrmap e22: x(active, since 1.02536s), standbys: z, y 2024-05-07T02:31:51.574 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:51 smithi064 ceph-mon[34699]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:31:51.574 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:51 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/3121807016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-07T02:31:53.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:52 smithi064 ceph-mon[34699]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:31:53.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:52 smithi064 ceph-mon[34699]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-05-07T02:31:53.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:52 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:31:53.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:52 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:31:53.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:52 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:31:53.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:52 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:31:53.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:52 smithi019 ceph-mon[29781]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:31:53.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:52 smithi019 ceph-mon[29781]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-05-07T02:31:53.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:52 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:31:53.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:52 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:31:53.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:52 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:31:53.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:52 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:31:53.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:52 smithi027 ceph-mon[34398]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:31:53.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:52 smithi027 ceph-mon[34398]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-05-07T02:31:53.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:52 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:31:53.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:52 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:31:53.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:52 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:31:53.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:52 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:31:54.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:54.597+0000 7fa3fae00700 1 -- 172.21.15.19:0/266513107 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3f4105be0 msgr2=0x7fa3f410e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:54.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:54.597+0000 7fa3fae00700 1 --2- 172.21.15.19:0/266513107 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3f4105be0 0x7fa3f410e130 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0009990 tx=0x7fa3e0009ca0 comp rx=0 tx=0).stop 2024-05-07T02:31:54.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:54.597+0000 7fa3fae00700 1 -- 172.21.15.19:0/266513107 shutdown_connections 2024-05-07T02:31:54.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:54.597+0000 7fa3fae00700 1 --2- 172.21.15.19:0/266513107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3f410e700 0x7fa3f4110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:54.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:54.597+0000 7fa3fae00700 1 --2- 172.21.15.19:0/266513107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3f4105290 0x7fa3f41056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:54.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:54.597+0000 7fa3fae00700 1 --2- 172.21.15.19:0/266513107 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3f4105be0 0x7fa3f410e130 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:54.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:54.597+0000 7fa3fae00700 1 -- 172.21.15.19:0/266513107 >> 172.21.15.19:0/266513107 conn(0x7fa3f4078ac0 msgr2=0x7fa3f4078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:54.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:54.597+0000 7fa3fae00700 1 -- 172.21.15.19:0/266513107 shutdown_connections 2024-05-07T02:31:54.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:54.597+0000 7fa3fae00700 1 -- 172.21.15.19:0/266513107 wait complete. 2024-05-07T02:31:54.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:54.599+0000 7fa3fae00700 1 Processor -- start 2024-05-07T02:31:54.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:54.599+0000 7fa3fae00700 1 -- start start 2024-05-07T02:31:54.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:54.600+0000 7fa3fae00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3f4105290 0x7fa3f4102f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:54.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:54.600+0000 7fa3fae00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3f4105be0 0x7fa3f41034c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:54.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:54.600+0000 7fa3fae00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3f410e700 0x7fa3f4103a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:54.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:54.600+0000 7fa3fae00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f41142b0 con 0x7fa3f410e700 2024-05-07T02:31:54.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:54.600+0000 7fa3fae00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f4114130 con 0x7fa3f4105290 2024-05-07T02:31:54.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:54.600+0000 7fa3fae00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f4114430 con 0x7fa3f4105be0 2024-05-07T02:31:54.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:54.600+0000 7fa3f9a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3f4105290 0x7fa3f4102f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:54.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:54.600+0000 7fa3f9a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3f4105290 0x7fa3f4102f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:57538/0 (socket says 172.21.15.19:57538) 2024-05-07T02:31:54.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:54.600+0000 7fa3f9a00700 1 -- 172.21.15.19:0/2052175588 learned_addr learned my addr 172.21.15.19:0/2052175588 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:31:54.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:54.601+0000 7fa3f9000700 1 --2- 172.21.15.19:0/2052175588 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3f4105be0 0x7fa3f41034c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:54.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:54.601+0000 7fa3fa400700 1 --2- 172.21.15.19:0/2052175588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3f410e700 0x7fa3f4103a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:54.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:54.601+0000 7fa3f9000700 1 -- 172.21.15.19:0/2052175588 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3f4105290 msgr2=0x7fa3f4102f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:54.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:54.601+0000 7fa3f9000700 1 --2- 172.21.15.19:0/2052175588 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3f4105290 0x7fa3f4102f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:54.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:54.601+0000 7fa3f9000700 1 -- 172.21.15.19:0/2052175588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3f410e700 msgr2=0x7fa3f4103a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:54.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:54.601+0000 7fa3f9000700 1 --2- 172.21.15.19:0/2052175588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3f410e700 0x7fa3f4103a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:54.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:54.601+0000 7fa3f9000700 1 -- 172.21.15.19:0/2052175588 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f41a9020 con 0x7fa3f4105be0 2024-05-07T02:31:54.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:54.601+0000 7fa3fa400700 1 --2- 172.21.15.19:0/2052175588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3f410e700 0x7fa3f4103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:31:54.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:54.602+0000 7fa3f9000700 1 --2- 172.21.15.19:0/2052175588 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3f4105be0 0x7fa3f41034c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0000c00 tx=0x7fa3e00047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:54.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:54.602+0000 7fa3f2000700 1 -- 172.21.15.19:0/2052175588 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0026070 con 0x7fa3f4105be0 2024-05-07T02:31:54.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:54.602+0000 7fa3f2000700 1 -- 172.21.15.19:0/2052175588 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3e0021410 con 0x7fa3f4105be0 2024-05-07T02:31:54.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:54.602+0000 7fa3f2000700 1 -- 172.21.15.19:0/2052175588 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0017650 con 0x7fa3f4105be0 2024-05-07T02:31:54.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:54.602+0000 7fa3fae00700 1 -- 172.21.15.19:0/2052175588 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3f41a92b0 con 0x7fa3f4105be0 2024-05-07T02:31:54.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:54.602+0000 7fa3fae00700 1 -- 172.21.15.19:0/2052175588 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3f41a9740 con 0x7fa3f4105be0 2024-05-07T02:31:54.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:54.603+0000 7fa3fae00700 1 -- 172.21.15.19:0/2052175588 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3f40512c0 con 0x7fa3f4105be0 2024-05-07T02:31:54.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:54.607+0000 7fa3f2000700 1 -- 172.21.15.19:0/2052175588 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 23) ==== 149772+0+0 (secure 0 0 0) 0x7fa3e0009d10 con 0x7fa3f4105be0 2024-05-07T02:31:54.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:54.609+0000 7fa3f2000700 1 --2- 172.21.15.19:0/2052175588 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fa3dc0b2a70 0x7fa3dc0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:54.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:54.609+0000 7fa3f2000700 1 -- 172.21.15.19:0/2052175588 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fa3e00e3d20 con 0x7fa3f4105be0 2024-05-07T02:31:54.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:54.610+0000 7fa3f9a00700 1 --2- 172.21.15.19:0/2052175588 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fa3dc0b2a70 0x7fa3dc0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:54.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:54.611+0000 7fa3f9a00700 1 --2- 172.21.15.19:0/2052175588 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fa3dc0b2a70 0x7fa3dc0b4f80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8009d20 tx=0x7fa3e8009260 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:54.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:54.611+0000 7fa3f2000700 1 -- 172.21.15.19:0/2052175588 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa3e002f070 con 0x7fa3f4105be0 2024-05-07T02:31:54.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:54.642+0000 7fa3f2000700 1 -- 172.21.15.19:0/2052175588 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7fa3e00a6f30 con 0x7fa3f4105be0 2024-05-07T02:31:54.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:54 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:31:54.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:54 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:31:54.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:54 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:54 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:54 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:54 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:54 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:54 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:54 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:54 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:54 smithi064 ceph-mon[34699]: Adjusting osd_memory_target on smithi027 to 176.2M 2024-05-07T02:31:54.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:54 smithi064 ceph-mon[34699]: Unable to set osd_memory_target on smithi027 to 184763084: error parsing value: Value '184763084' is below minimum 939524096 2024-05-07T02:31:54.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:54 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:31:54.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:54 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:31:54.860 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:54 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.860 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:54 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.860 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:54 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.860 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:54 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.860 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:54 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.860 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:54 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.860 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:54 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.860 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:54 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.860 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:54 smithi064 ceph-mon[34699]: Adjusting osd_memory_target on smithi064 to 176.2M 2024-05-07T02:31:54.860 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:54 smithi064 ceph-mon[34699]: Unable to set osd_memory_target on smithi064 to 184793907: error parsing value: Value '184793907' is below minimum 939524096 2024-05-07T02:31:54.860 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:54 smithi064 ceph-mon[34699]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:31:54.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:54 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:31:54.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:54 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:31:54.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:54 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:54 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:54 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:54 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:54 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:54 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:54 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:54 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:54 smithi019 ceph-mon[29781]: Adjusting osd_memory_target on smithi027 to 176.2M 2024-05-07T02:31:54.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:54 smithi019 ceph-mon[29781]: Unable to set osd_memory_target on smithi027 to 184763084: error parsing value: Value '184763084' is below minimum 939524096 2024-05-07T02:31:54.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:54 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:31:54.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:54 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:31:54.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:54 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:54 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:54 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:54 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:54 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:54 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:54 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:54 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:54 smithi019 ceph-mon[29781]: Adjusting osd_memory_target on smithi064 to 176.2M 2024-05-07T02:31:54.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:54 smithi019 ceph-mon[29781]: Unable to set osd_memory_target on smithi064 to 184793907: error parsing value: Value '184793907' is below minimum 939524096 2024-05-07T02:31:54.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:54 smithi019 ceph-mon[29781]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:31:54.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:54 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:31:54.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:54 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:31:54.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:54 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:54 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:54 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:54 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:54 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:54 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:54 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:54 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:54 smithi027 ceph-mon[34398]: Adjusting osd_memory_target on smithi027 to 176.2M 2024-05-07T02:31:54.958 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:54 smithi027 ceph-mon[34398]: Unable to set osd_memory_target on smithi027 to 184763084: error parsing value: Value '184763084' is below minimum 939524096 2024-05-07T02:31:54.958 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:54 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:31:54.958 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:54 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:31:54.958 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:54 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.958 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:54 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.958 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:54 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.958 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:54 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.958 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:54 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.958 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:54 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.958 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:54 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.958 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:54 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:54.958 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:54 smithi027 ceph-mon[34398]: Adjusting osd_memory_target on smithi064 to 176.2M 2024-05-07T02:31:54.958 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:54 smithi027 ceph-mon[34398]: Unable to set osd_memory_target on smithi064 to 184793907: error parsing value: Value '184793907' is below minimum 939524096 2024-05-07T02:31:54.958 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:54 smithi027 ceph-mon[34398]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:31:54.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:54.999+0000 7fa3fae00700 1 -- 172.21.15.19:0/2052175588 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3f402e5f0 con 0x7fa3f4105be0 2024-05-07T02:31:55.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:55.000+0000 7fa3f2000700 1 -- 172.21.15.19:0/2052175588 <== mon.1 v2:172.21.15.64:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11924 (secure 0 0 0) 0x7fa3e0013070 con 0x7fa3f4105be0 2024-05-07T02:31:55.000 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:31:55.000 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1763670446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1763670446},{"type":"v1","addr":"172.21.15.19:6839","nonce":1763670446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/477529985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":477529985},{"type":"v1","addr":"172.21.15.64:6833","nonce":477529985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-07T02:26:14.960465+0000","modified":"2024-05-07T02:31:47.904333+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24539,"mds_2":24520,"mds_3":24515,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6833/2171873162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2171873162},{"type":"v1","addr":"172.21.15.27:6833","nonce":2171873162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/2379917839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2379917839},{"type":"v1","addr":"172.21.15.19:6835","nonce":2379917839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6837/2067515538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2067515538},{"type":"v1","addr":"172.21.15.19:6837","nonce":2067515538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6835/1000540600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1000540600},{"type":"v1","addr":"172.21.15.27:6835","nonce":1000540600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/1997318872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1997318872},{"type":"v1","addr":"172.21.15.64:6835","nonce":1997318872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":34,"addr":"172.21.15.64:6837/2585907777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2585907777},{"type":"v1","addr":"172.21.15.64:6837","nonce":2585907777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":33,"addr":"172.21.15.27:6837/2583138143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2583138143},{"type":"v1","addr":"172.21.15.27:6837","nonce":2583138143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":32,"addr":"172.21.15.64:6839/1583120004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":1583120004},{"type":"v1","addr":"172.21.15.64:6839","nonce":1583120004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":29,"addr":"172.21.15.19:6841/3905645246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3905645246},{"type":"v1","addr":"172.21.15.19:6841","nonce":3905645246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6839/3963268634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3963268634},{"type":"v1","addr":"172.21.15.27:6839","nonce":3963268634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24515,24505,24520,24551,24539]},"id":1}]} 2024-05-07T02:31:55.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:55.003+0000 7fa3fae00700 1 -- 172.21.15.19:0/2052175588 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fa3dc0b2a70 msgr2=0x7fa3dc0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:55.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:55.003+0000 7fa3fae00700 1 --2- 172.21.15.19:0/2052175588 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fa3dc0b2a70 0x7fa3dc0b4f80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8009d20 tx=0x7fa3e8009260 comp rx=0 tx=0).stop 2024-05-07T02:31:55.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:55.003+0000 7fa3fae00700 1 -- 172.21.15.19:0/2052175588 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3f4105be0 msgr2=0x7fa3f41034c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:55.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:55.003+0000 7fa3fae00700 1 --2- 172.21.15.19:0/2052175588 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3f4105be0 0x7fa3f41034c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0000c00 tx=0x7fa3e00047b0 comp rx=0 tx=0).stop 2024-05-07T02:31:55.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:55.003+0000 7fa3fae00700 1 -- 172.21.15.19:0/2052175588 shutdown_connections 2024-05-07T02:31:55.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:55.003+0000 7fa3fae00700 1 --2- 172.21.15.19:0/2052175588 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fa3dc0b2a70 0x7fa3dc0b4f80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:55.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:55.003+0000 7fa3fae00700 1 --2- 172.21.15.19:0/2052175588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3f410e700 0x7fa3f4103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:55.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:55.003+0000 7fa3fae00700 1 --2- 172.21.15.19:0/2052175588 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3f4105290 0x7fa3f4102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:55.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:55.003+0000 7fa3fae00700 1 --2- 172.21.15.19:0/2052175588 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3f4105be0 0x7fa3f41034c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:55.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:55.004+0000 7fa3fae00700 1 -- 172.21.15.19:0/2052175588 >> 172.21.15.19:0/2052175588 conn(0x7fa3f4078ac0 msgr2=0x7fa3f40fffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:55.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:55.004+0000 7fa3fae00700 1 -- 172.21.15.19:0/2052175588 shutdown_connections 2024-05-07T02:31:55.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:55.004+0000 7fa3fae00700 1 -- 172.21.15.19:0/2052175588 wait complete. 2024-05-07T02:31:55.005 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 31 2024-05-07T02:31:55.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:55 smithi019 ceph-mon[29781]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-05-07T02:31:55.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:55 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:31:55.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:55 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/2052175588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:31:55.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:55 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:31:55.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:55 smithi027 ceph-mon[34398]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-05-07T02:31:55.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:55 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:31:55.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:55 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/2052175588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:31:55.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:55 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:31:56.065 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-05-07T02:31:56.065 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-05-07T02:31:56.065 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-07T02:31:56.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:55 smithi064 ceph-mon[34699]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-05-07T02:31:56.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:55 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:31:56.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:55 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/2052175588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:31:56.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:55 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:31:56.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.331+0000 7f7445000640 1 -- 172.21.15.19:0/3167587127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7440102b30 msgr2=0x7f7440103830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:56.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.331+0000 7f7445000640 1 --2- 172.21.15.19:0/3167587127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7440102b30 0x7f7440103830 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7430009860 tx=0x7f743002a340 comp rx=0 tx=0).stop 2024-05-07T02:31:56.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.331+0000 7f7445000640 1 -- 172.21.15.19:0/3167587127 shutdown_connections 2024-05-07T02:31:56.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.331+0000 7f7445000640 1 --2- 172.21.15.19:0/3167587127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7440108a70 0x7f744010ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:56.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.331+0000 7f7445000640 1 --2- 172.21.15.19:0/3167587127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7440103d70 0x7f7440104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:56.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.331+0000 7f7445000640 1 --2- 172.21.15.19:0/3167587127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7440102b30 0x7f7440103830 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:56.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.331+0000 7f7445000640 1 -- 172.21.15.19:0/3167587127 >> 172.21.15.19:0/3167587127 conn(0x7f74400fdd20 msgr2=0x7f74401001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:56.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.331+0000 7f7445000640 1 -- 172.21.15.19:0/3167587127 shutdown_connections 2024-05-07T02:31:56.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.331+0000 7f7445000640 1 -- 172.21.15.19:0/3167587127 wait complete. 2024-05-07T02:31:56.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.332+0000 7f7445000640 1 Processor -- start 2024-05-07T02:31:56.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.333+0000 7f7445000640 1 -- start start 2024-05-07T02:31:56.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.333+0000 7f7445000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7440102b30 0x7f744019c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:56.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.333+0000 7f7445000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7440103d70 0x7f744019ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:56.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.333+0000 7f7445000640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7440108a70 0x7f74401a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:56.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.333+0000 7f7445000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f744010e610 con 0x7f7440102b30 2024-05-07T02:31:56.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.333+0000 7f7445000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f744010e490 con 0x7f7440103d70 2024-05-07T02:31:56.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.333+0000 7f7445000640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f744010e790 con 0x7f7440108a70 2024-05-07T02:31:56.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.333+0000 7f743de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7440102b30 0x7f744019c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:56.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.333+0000 7f743de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7440102b30 0x7f744019c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50856/0 (socket says 172.21.15.19:50856) 2024-05-07T02:31:56.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.333+0000 7f743de00640 1 -- 172.21.15.19:0/2504962352 learned_addr learned my addr 172.21.15.19:0/2504962352 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:31:56.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.333+0000 7f7437e00640 1 --2- 172.21.15.19:0/2504962352 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7440103d70 0x7f744019ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:56.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.334+0000 7f743e800640 1 --2- 172.21.15.19:0/2504962352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7440108a70 0x7f74401a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:56.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.334+0000 7f743de00640 1 -- 172.21.15.19:0/2504962352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7440108a70 msgr2=0x7f74401a39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:56.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.334+0000 7f743de00640 1 --2- 172.21.15.19:0/2504962352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7440108a70 0x7f74401a39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:56.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.334+0000 7f743de00640 1 -- 172.21.15.19:0/2504962352 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7440103d70 msgr2=0x7f744019ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:56.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.334+0000 7f743de00640 1 --2- 172.21.15.19:0/2504962352 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7440103d70 0x7f744019ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:56.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.334+0000 7f743de00640 1 -- 172.21.15.19:0/2504962352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74401a4040 con 0x7f7440102b30 2024-05-07T02:31:56.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.334+0000 7f743de00640 1 --2- 172.21.15.19:0/2504962352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7440102b30 0x7f744019c500 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f743002a860 tx=0x7f7430032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:56.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.334+0000 7f7436a00640 1 -- 172.21.15.19:0/2504962352 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7430037040 con 0x7f7440102b30 2024-05-07T02:31:56.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.335+0000 7f7436a00640 1 -- 172.21.15.19:0/2504962352 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7430030c10 con 0x7f7440102b30 2024-05-07T02:31:56.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.335+0000 7f7445000640 1 -- 172.21.15.19:0/2504962352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74401a42d0 con 0x7f7440102b30 2024-05-07T02:31:56.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.335+0000 7f7436a00640 1 -- 172.21.15.19:0/2504962352 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74300052b0 con 0x7f7440102b30 2024-05-07T02:31:56.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.335+0000 7f7445000640 1 -- 172.21.15.19:0/2504962352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74401a4760 con 0x7f7440102b30 2024-05-07T02:31:56.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.336+0000 7f7445000640 1 -- 172.21.15.19:0/2504962352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7440106160 con 0x7f7440102b30 2024-05-07T02:31:56.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.338+0000 7f7436a00640 1 -- 172.21.15.19:0/2504962352 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f7430043020 con 0x7f7440102b30 2024-05-07T02:31:56.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.340+0000 7f7436a00640 1 --2- 172.21.15.19:0/2504962352 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f74240b2a90 0x7f74240b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:56.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.340+0000 7f7436a00640 1 -- 172.21.15.19:0/2504962352 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f7430101e90 con 0x7f7440102b30 2024-05-07T02:31:56.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.340+0000 7f7437e00640 1 --2- 172.21.15.19:0/2504962352 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f74240b2a90 0x7f74240b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:56.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.341+0000 7f7437e00640 1 --2- 172.21.15.19:0/2504962352 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f74240b2a90 0x7f74240b4fa0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f74280096c0 tx=0x7f7428009260 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:56.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.345+0000 7f7436a00640 1 -- 172.21.15.19:0/2504962352 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f743004a060 con 0x7f7440102b30 2024-05-07T02:31:56.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.554+0000 7f7445000640 1 -- 172.21.15.19:0/2504962352 --> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7440063110 con 0x7f74240b2a90 2024-05-07T02:31:56.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.578+0000 7f7436a00640 1 -- 172.21.15.19:0/2504962352 <== mgr.25234 v2:172.21.15.19:6800/3265825478 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339447 (secure 0 0 0) 0x7f7440063110 con 0x7f74240b2a90 2024-05-07T02:31:56.584 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:31:56.586 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-05-07T02:31:56.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.585+0000 7f7445000640 1 -- 172.21.15.19:0/2504962352 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f74240b2a90 msgr2=0x7f74240b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:56.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.585+0000 7f7445000640 1 --2- 172.21.15.19:0/2504962352 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f74240b2a90 0x7f74240b4fa0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f74280096c0 tx=0x7f7428009260 comp rx=0 tx=0).stop 2024-05-07T02:31:56.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.585+0000 7f7445000640 1 -- 172.21.15.19:0/2504962352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7440102b30 msgr2=0x7f744019c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:56.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.585+0000 7f7445000640 1 --2- 172.21.15.19:0/2504962352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7440102b30 0x7f744019c500 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f743002a860 tx=0x7f7430032d50 comp rx=0 tx=0).stop 2024-05-07T02:31:56.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.586+0000 7f7445000640 1 -- 172.21.15.19:0/2504962352 shutdown_connections 2024-05-07T02:31:56.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.586+0000 7f7445000640 1 --2- 172.21.15.19:0/2504962352 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f74240b2a90 0x7f74240b4fa0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:56.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.586+0000 7f7445000640 1 --2- 172.21.15.19:0/2504962352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7440108a70 0x7f74401a39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:56.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.586+0000 7f7445000640 1 --2- 172.21.15.19:0/2504962352 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7440103d70 0x7f744019ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:56.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.586+0000 7f7445000640 1 --2- 172.21.15.19:0/2504962352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7440102b30 0x7f744019c500 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:56.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.586+0000 7f7445000640 1 -- 172.21.15.19:0/2504962352 >> 172.21.15.19:0/2504962352 conn(0x7f74400fdd20 msgr2=0x7f7440100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:56.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.586+0000 7f7445000640 1 -- 172.21.15.19:0/2504962352 shutdown_connections 2024-05-07T02:31:56.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.586+0000 7f7445000640 1 -- 172.21.15.19:0/2504962352 wait complete. 2024-05-07T02:31:56.615 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-05-07T02:31:55.850763+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1198131,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2682,"num_read_kb":1494,"num_write":513,"num_write_kb":3633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":906,"ondisk_log_size":906,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":377672,"kb_used_data":16004,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124564152,"statfs":{"total":1151940427776,"available":1151553691648,"internally_reserved":0,"allocated":16388096,"data_stored":9868320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.978312"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.093082+0000","last_change":"2024-05-07T02:26:11.876828+0000","last_active":"2024-05-07T02:31:50.093082+0000","last_peered":"2024-05-07T02:31:50.093082+0000","last_clean":"2024-05-07T02:31:50.093082+0000","last_became_active":"2024-05-07T02:26:11.876319+0000","last_became_peered":"2024-05-07T02:26:11.876319+0000","last_unstale":"2024-05-07T02:31:50.093082+0000","last_undegraded":"2024-05-07T02:31:50.093082+0000","last_fullsized":"2024-05-07T02:31:50.093082+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:07:33.181373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.094443+0000","last_change":"2024-05-07T02:26:11.873388+0000","last_active":"2024-05-07T02:31:50.094443+0000","last_peered":"2024-05-07T02:31:50.094443+0000","last_clean":"2024-05-07T02:31:50.094443+0000","last_became_active":"2024-05-07T02:26:11.869447+0000","last_became_peered":"2024-05-07T02:26:11.869447+0000","last_unstale":"2024-05-07T02:31:50.094443+0000","last_undegraded":"2024-05-07T02:31:50.094443+0000","last_fullsized":"2024-05-07T02:31:50.094443+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:09:52.876547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.094158+0000","last_change":"2024-05-07T02:26:11.870797+0000","last_active":"2024-05-07T02:31:50.094158+0000","last_peered":"2024-05-07T02:31:50.094158+0000","last_clean":"2024-05-07T02:31:50.094158+0000","last_became_active":"2024-05-07T02:26:11.870411+0000","last_became_peered":"2024-05-07T02:26:11.870411+0000","last_unstale":"2024-05-07T02:31:50.094158+0000","last_undegraded":"2024-05-07T02:31:50.094158+0000","last_fullsized":"2024-05-07T02:31:50.094158+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:34:12.017508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.826540+0000","last_change":"2024-05-07T02:26:11.867327+0000","last_active":"2024-05-07T02:31:49.826540+0000","last_peered":"2024-05-07T02:31:49.826540+0000","last_clean":"2024-05-07T02:31:49.826540+0000","last_became_active":"2024-05-07T02:26:11.866912+0000","last_became_peered":"2024-05-07T02:26:11.866912+0000","last_unstale":"2024-05-07T02:31:49.826540+0000","last_undegraded":"2024-05-07T02:31:49.826540+0000","last_fullsized":"2024-05-07T02:31:49.826540+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:36:36.616706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.988361+0000","last_change":"2024-05-07T02:26:11.878614+0000","last_active":"2024-05-07T02:31:49.988361+0000","last_peered":"2024-05-07T02:31:49.988361+0000","last_clean":"2024-05-07T02:31:49.988361+0000","last_became_active":"2024-05-07T02:26:11.878069+0000","last_became_peered":"2024-05-07T02:26:11.878069+0000","last_unstale":"2024-05-07T02:31:49.988361+0000","last_undegraded":"2024-05-07T02:31:49.988361+0000","last_fullsized":"2024-05-07T02:31:49.988361+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:40:15.991433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.003199+0000","last_change":"2024-05-07T02:26:11.877755+0000","last_active":"2024-05-07T02:31:50.003199+0000","last_peered":"2024-05-07T02:31:50.003199+0000","last_clean":"2024-05-07T02:31:50.003199+0000","last_became_active":"2024-05-07T02:26:11.877373+0000","last_became_peered":"2024-05-07T02:26:11.877373+0000","last_unstale":"2024-05-07T02:31:50.003199+0000","last_undegraded":"2024-05-07T02:31:50.003199+0000","last_fullsized":"2024-05-07T02:31:50.003199+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:49:55.984807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.136082+0000","last_change":"2024-05-07T02:26:11.882194+0000","last_active":"2024-05-07T02:31:50.136082+0000","last_peered":"2024-05-07T02:31:50.136082+0000","last_clean":"2024-05-07T02:31:50.136082+0000","last_became_active":"2024-05-07T02:26:11.881873+0000","last_became_peered":"2024-05-07T02:26:11.881873+0000","last_unstale":"2024-05-07T02:31:50.136082+0000","last_undegraded":"2024-05-07T02:31:50.136082+0000","last_fullsized":"2024-05-07T02:31:50.136082+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:47:31.000357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.823872+0000","last_change":"2024-05-07T02:26:11.857366+0000","last_active":"2024-05-07T02:31:49.823872+0000","last_peered":"2024-05-07T02:31:49.823872+0000","last_clean":"2024-05-07T02:31:49.823872+0000","last_became_active":"2024-05-07T02:26:11.854054+0000","last_became_peered":"2024-05-07T02:26:11.854054+0000","last_unstale":"2024-05-07T02:31:49.823872+0000","last_undegraded":"2024-05-07T02:31:49.823872+0000","last_fullsized":"2024-05-07T02:31:49.823872+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:07:04.834022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092885+0000","last_change":"2024-05-07T02:26:11.867105+0000","last_active":"2024-05-07T02:31:50.092885+0000","last_peered":"2024-05-07T02:31:50.092885+0000","last_clean":"2024-05-07T02:31:50.092885+0000","last_became_active":"2024-05-07T02:26:11.865986+0000","last_became_peered":"2024-05-07T02:26:11.865986+0000","last_unstale":"2024-05-07T02:31:50.092885+0000","last_undegraded":"2024-05-07T02:31:50.092885+0000","last_fullsized":"2024-05-07T02:31:50.092885+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:13:01.295354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.003769+0000","last_change":"2024-05-07T02:26:11.868039+0000","last_active":"2024-05-07T02:31:50.003769+0000","last_peered":"2024-05-07T02:31:50.003769+0000","last_clean":"2024-05-07T02:31:50.003769+0000","last_became_active":"2024-05-07T02:26:11.867588+0000","last_became_peered":"2024-05-07T02:26:11.867588+0000","last_unstale":"2024-05-07T02:31:50.003769+0000","last_undegraded":"2024-05-07T02:31:50.003769+0000","last_fullsized":"2024-05-07T02:31:50.003769+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:41:45.149620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.136641+0000","last_change":"2024-05-07T02:26:11.859579+0000","last_active":"2024-05-07T02:31:50.136641+0000","last_peered":"2024-05-07T02:31:50.136641+0000","last_clean":"2024-05-07T02:31:50.136641+0000","last_became_active":"2024-05-07T02:26:11.859154+0000","last_became_peered":"2024-05-07T02:26:11.859154+0000","last_unstale":"2024-05-07T02:31:50.136641+0000","last_undegraded":"2024-05-07T02:31:50.136641+0000","last_fullsized":"2024-05-07T02:31:50.136641+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:26:40.616956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825795+0000","last_change":"2024-05-07T02:26:11.858220+0000","last_active":"2024-05-07T02:31:49.825795+0000","last_peered":"2024-05-07T02:31:49.825795+0000","last_clean":"2024-05-07T02:31:49.825795+0000","last_became_active":"2024-05-07T02:26:11.857826+0000","last_became_peered":"2024-05-07T02:26:11.857826+0000","last_unstale":"2024-05-07T02:31:49.825795+0000","last_undegraded":"2024-05-07T02:31:49.825795+0000","last_fullsized":"2024-05-07T02:31:49.825795+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:21:36.031057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.097826+0000","last_change":"2024-05-07T02:26:11.880348+0000","last_active":"2024-05-07T02:31:50.097826+0000","last_peered":"2024-05-07T02:31:50.097826+0000","last_clean":"2024-05-07T02:31:50.097826+0000","last_became_active":"2024-05-07T02:26:11.879633+0000","last_became_peered":"2024-05-07T02:26:11.879633+0000","last_unstale":"2024-05-07T02:31:50.097826+0000","last_undegraded":"2024-05-07T02:31:50.097826+0000","last_fullsized":"2024-05-07T02:31:50.097826+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:59:34.567973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.093652+0000","last_change":"2024-05-07T02:26:11.865687+0000","last_active":"2024-05-07T02:31:50.093652+0000","last_peered":"2024-05-07T02:31:50.093652+0000","last_clean":"2024-05-07T02:31:50.093652+0000","last_became_active":"2024-05-07T02:26:11.865330+0000","last_became_peered":"2024-05-07T02:26:11.865330+0000","last_unstale":"2024-05-07T02:31:50.093652+0000","last_undegraded":"2024-05-07T02:31:50.093652+0000","last_fullsized":"2024-05-07T02:31:50.093652+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:36:10.610314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.093991+0000","last_change":"2024-05-07T02:26:11.870627+0000","last_active":"2024-05-07T02:31:50.093991+0000","last_peered":"2024-05-07T02:31:50.093991+0000","last_clean":"2024-05-07T02:31:50.093991+0000","last_became_active":"2024-05-07T02:26:11.870229+0000","last_became_peered":"2024-05-07T02:26:11.870229+0000","last_unstale":"2024-05-07T02:31:50.093991+0000","last_undegraded":"2024-05-07T02:31:50.093991+0000","last_fullsized":"2024-05-07T02:31:50.093991+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:06:46.008651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092311+0000","last_change":"2024-05-07T02:26:11.868698+0000","last_active":"2024-05-07T02:31:50.092311+0000","last_peered":"2024-05-07T02:31:50.092311+0000","last_clean":"2024-05-07T02:31:50.092311+0000","last_became_active":"2024-05-07T02:26:11.868294+0000","last_became_peered":"2024-05-07T02:26:11.868294+0000","last_unstale":"2024-05-07T02:31:50.092311+0000","last_undegraded":"2024-05-07T02:31:50.092311+0000","last_fullsized":"2024-05-07T02:31:50.092311+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:16:22.856200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092865+0000","last_change":"2024-05-07T02:26:11.877116+0000","last_active":"2024-05-07T02:31:50.092865+0000","last_peered":"2024-05-07T02:31:50.092865+0000","last_clean":"2024-05-07T02:31:50.092865+0000","last_became_active":"2024-05-07T02:26:11.876305+0000","last_became_peered":"2024-05-07T02:26:11.876305+0000","last_unstale":"2024-05-07T02:31:50.092865+0000","last_undegraded":"2024-05-07T02:31:50.092865+0000","last_fullsized":"2024-05-07T02:31:50.092865+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:25:46.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.985509+0000","last_change":"2024-05-07T02:26:11.877488+0000","last_active":"2024-05-07T02:31:49.985509+0000","last_peered":"2024-05-07T02:31:49.985509+0000","last_clean":"2024-05-07T02:31:49.985509+0000","last_became_active":"2024-05-07T02:26:11.874500+0000","last_became_peered":"2024-05-07T02:26:11.874500+0000","last_unstale":"2024-05-07T02:31:49.985509+0000","last_undegraded":"2024-05-07T02:31:49.985509+0000","last_fullsized":"2024-05-07T02:31:49.985509+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:53:50.046342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825931+0000","last_change":"2024-05-07T02:26:11.876500+0000","last_active":"2024-05-07T02:31:49.825931+0000","last_peered":"2024-05-07T02:31:49.825931+0000","last_clean":"2024-05-07T02:31:49.825931+0000","last_became_active":"2024-05-07T02:26:11.876084+0000","last_became_peered":"2024-05-07T02:26:11.876084+0000","last_unstale":"2024-05-07T02:31:49.825931+0000","last_undegraded":"2024-05-07T02:31:49.825931+0000","last_fullsized":"2024-05-07T02:31:49.825931+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:05:05.681488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825186+0000","last_change":"2024-05-07T02:26:25.579209+0000","last_active":"2024-05-07T02:31:49.825186+0000","last_peered":"2024-05-07T02:31:49.825186+0000","last_clean":"2024-05-07T02:31:49.825186+0000","last_became_active":"2024-05-07T02:26:25.578665+0000","last_became_peered":"2024-05-07T02:26:25.578665+0000","last_unstale":"2024-05-07T02:31:49.825186+0000","last_undegraded":"2024-05-07T02:31:49.825186+0000","last_fullsized":"2024-05-07T02:31:49.825186+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:06:29.784016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'15","reported_seq":310,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.215535+0000","last_change":"2024-05-07T02:26:07.792993+0000","last_active":"2024-05-07T02:31:50.215535+0000","last_peered":"2024-05-07T02:31:50.215535+0000","last_clean":"2024-05-07T02:31:50.215535+0000","last_became_active":"2024-05-07T02:26:07.792402+0000","last_became_peered":"2024-05-07T02:26:07.792402+0000","last_unstale":"2024-05-07T02:31:50.215535+0000","last_undegraded":"2024-05-07T02:31:50.215535+0000","last_fullsized":"2024-05-07T02:31:50.215535+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:49:02.288255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":247,"num_read_kb":247,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.985441+0000","last_change":"2024-05-07T02:28:33.869804+0000","last_active":"2024-05-07T02:31:49.985441+0000","last_peered":"2024-05-07T02:31:49.985441+0000","last_clean":"2024-05-07T02:31:49.985441+0000","last_became_active":"2024-05-07T02:28:33.869267+0000","last_became_peered":"2024-05-07T02:28:33.869267+0000","last_unstale":"2024-05-07T02:31:49.985441+0000","last_undegraded":"2024-05-07T02:31:49.985441+0000","last_fullsized":"2024-05-07T02:31:49.985441+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:10:57.384546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825312+0000","last_change":"2024-05-07T02:26:25.579835+0000","last_active":"2024-05-07T02:31:49.825312+0000","last_peered":"2024-05-07T02:31:49.825312+0000","last_clean":"2024-05-07T02:31:49.825312+0000","last_became_active":"2024-05-07T02:26:25.579105+0000","last_became_peered":"2024-05-07T02:26:25.579105+0000","last_unstale":"2024-05-07T02:31:49.825312+0000","last_undegraded":"2024-05-07T02:31:49.825312+0000","last_fullsized":"2024-05-07T02:31:49.825312+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:32:37.745967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825500+0000","last_change":"2024-05-07T02:26:08.001879+0000","last_active":"2024-05-07T02:31:49.825500+0000","last_peered":"2024-05-07T02:31:49.825500+0000","last_clean":"2024-05-07T02:31:49.825500+0000","last_became_active":"2024-05-07T02:26:08.000244+0000","last_became_peered":"2024-05-07T02:26:08.000244+0000","last_unstale":"2024-05-07T02:31:49.825500+0000","last_undegraded":"2024-05-07T02:31:49.825500+0000","last_fullsized":"2024-05-07T02:31:49.825500+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:56:42.591923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.094664+0000","last_change":"2024-05-07T02:26:11.882351+0000","last_active":"2024-05-07T02:31:50.094664+0000","last_peered":"2024-05-07T02:31:50.094664+0000","last_clean":"2024-05-07T02:31:50.094664+0000","last_became_active":"2024-05-07T02:26:11.881922+0000","last_became_peered":"2024-05-07T02:26:11.881922+0000","last_unstale":"2024-05-07T02:31:50.094664+0000","last_undegraded":"2024-05-07T02:31:50.094664+0000","last_fullsized":"2024-05-07T02:31:50.094664+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:58:31.993859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825489+0000","last_change":"2024-05-07T02:26:25.580419+0000","last_active":"2024-05-07T02:31:49.825489+0000","last_peered":"2024-05-07T02:31:49.825489+0000","last_clean":"2024-05-07T02:31:49.825489+0000","last_became_active":"2024-05-07T02:26:25.579411+0000","last_became_peered":"2024-05-07T02:26:25.579411+0000","last_unstale":"2024-05-07T02:31:49.825489+0000","last_undegraded":"2024-05-07T02:31:49.825489+0000","last_fullsized":"2024-05-07T02:31:49.825489+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:43:03.552210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092822+0000","last_change":"2024-05-07T02:27:32.588255+0000","last_active":"2024-05-07T02:31:50.092822+0000","last_peered":"2024-05-07T02:31:50.092822+0000","last_clean":"2024-05-07T02:31:50.092822+0000","last_became_active":"2024-05-07T02:27:32.584405+0000","last_became_peered":"2024-05-07T02:27:32.584405+0000","last_unstale":"2024-05-07T02:31:50.092822+0000","last_undegraded":"2024-05-07T02:31:50.092822+0000","last_fullsized":"2024-05-07T02:31:50.092822+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:07:22.980041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092515+0000","last_change":"2024-05-07T02:26:11.866949+0000","last_active":"2024-05-07T02:31:50.092515+0000","last_peered":"2024-05-07T02:31:50.092515+0000","last_clean":"2024-05-07T02:31:50.092515+0000","last_became_active":"2024-05-07T02:26:11.866453+0000","last_became_peered":"2024-05-07T02:26:11.866453+0000","last_unstale":"2024-05-07T02:31:50.092515+0000","last_undegraded":"2024-05-07T02:31:50.092515+0000","last_fullsized":"2024-05-07T02:31:50.092515+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T14:22:47.291470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825287+0000","last_change":"2024-05-07T02:26:25.570555+0000","last_active":"2024-05-07T02:31:49.825287+0000","last_peered":"2024-05-07T02:31:49.825287+0000","last_clean":"2024-05-07T02:31:49.825287+0000","last_became_active":"2024-05-07T02:26:25.570044+0000","last_became_peered":"2024-05-07T02:26:25.570044+0000","last_unstale":"2024-05-07T02:31:49.825287+0000","last_undegraded":"2024-05-07T02:31:49.825287+0000","last_fullsized":"2024-05-07T02:31:49.825287+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:48:15.701335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.988509+0000","last_change":"2024-05-07T02:26:07.785938+0000","last_active":"2024-05-07T02:31:49.988509+0000","last_peered":"2024-05-07T02:31:49.988509+0000","last_clean":"2024-05-07T02:31:49.988509+0000","last_became_active":"2024-05-07T02:26:07.785573+0000","last_became_peered":"2024-05-07T02:26:07.785573+0000","last_unstale":"2024-05-07T02:31:49.988509+0000","last_undegraded":"2024-05-07T02:31:49.988509+0000","last_fullsized":"2024-05-07T02:31:49.988509+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:33:24.042227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.999531+0000","last_change":"2024-05-07T02:26:11.858732+0000","last_active":"2024-05-07T02:31:49.999531+0000","last_peered":"2024-05-07T02:31:49.999531+0000","last_clean":"2024-05-07T02:31:49.999531+0000","last_became_active":"2024-05-07T02:26:11.858270+0000","last_became_peered":"2024-05-07T02:26:11.858270+0000","last_unstale":"2024-05-07T02:31:49.999531+0000","last_undegraded":"2024-05-07T02:31:49.999531+0000","last_fullsized":"2024-05-07T02:31:49.999531+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:03:53.662296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825701+0000","last_change":"2024-05-07T02:26:25.559025+0000","last_active":"2024-05-07T02:31:49.825701+0000","last_peered":"2024-05-07T02:31:49.825701+0000","last_clean":"2024-05-07T02:31:49.825701+0000","last_became_active":"2024-05-07T02:26:25.558502+0000","last_became_peered":"2024-05-07T02:26:25.558502+0000","last_unstale":"2024-05-07T02:31:49.825701+0000","last_undegraded":"2024-05-07T02:31:49.825701+0000","last_fullsized":"2024-05-07T02:31:49.825701+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:26:59.846341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":302,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.147359+0000","last_change":"2024-05-07T02:26:07.783386+0000","last_active":"2024-05-07T02:31:50.147359+0000","last_peered":"2024-05-07T02:31:50.147359+0000","last_clean":"2024-05-07T02:31:50.147359+0000","last_became_active":"2024-05-07T02:26:07.782823+0000","last_became_peered":"2024-05-07T02:26:07.782823+0000","last_unstale":"2024-05-07T02:31:50.147359+0000","last_undegraded":"2024-05-07T02:31:50.147359+0000","last_fullsized":"2024-05-07T02:31:50.147359+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:40:29.350809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":248,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.988372+0000","last_change":"2024-05-07T02:26:11.877426+0000","last_active":"2024-05-07T02:31:49.988372+0000","last_peered":"2024-05-07T02:31:49.988372+0000","last_clean":"2024-05-07T02:31:49.988372+0000","last_became_active":"2024-05-07T02:26:11.876976+0000","last_became_peered":"2024-05-07T02:26:11.876976+0000","last_unstale":"2024-05-07T02:31:49.988372+0000","last_undegraded":"2024-05-07T02:31:49.988372+0000","last_fullsized":"2024-05-07T02:31:49.988372+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:18:56.077064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825760+0000","last_change":"2024-05-07T02:27:32.625394+0000","last_active":"2024-05-07T02:31:49.825760+0000","last_peered":"2024-05-07T02:31:49.825760+0000","last_clean":"2024-05-07T02:31:49.825760+0000","last_became_active":"2024-05-07T02:27:32.624826+0000","last_became_peered":"2024-05-07T02:27:32.624826+0000","last_unstale":"2024-05-07T02:31:49.825760+0000","last_undegraded":"2024-05-07T02:31:49.825760+0000","last_fullsized":"2024-05-07T02:31:49.825760+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:16:49.694003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.003155+0000","last_change":"2024-05-07T02:26:07.786388+0000","last_active":"2024-05-07T02:31:50.003155+0000","last_peered":"2024-05-07T02:31:50.003155+0000","last_clean":"2024-05-07T02:31:50.003155+0000","last_became_active":"2024-05-07T02:26:07.784130+0000","last_became_peered":"2024-05-07T02:26:07.784130+0000","last_unstale":"2024-05-07T02:31:50.003155+0000","last_undegraded":"2024-05-07T02:31:50.003155+0000","last_fullsized":"2024-05-07T02:31:50.003155+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:44:55.229815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.999374+0000","last_change":"2024-05-07T02:28:33.886551+0000","last_active":"2024-05-07T02:31:49.999374+0000","last_peered":"2024-05-07T02:31:49.999374+0000","last_clean":"2024-05-07T02:31:49.999374+0000","last_became_active":"2024-05-07T02:28:33.886249+0000","last_became_peered":"2024-05-07T02:28:33.886249+0000","last_unstale":"2024-05-07T02:31:49.999374+0000","last_undegraded":"2024-05-07T02:31:49.999374+0000","last_fullsized":"2024-05-07T02:31:49.999374+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:56:19.851347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092828+0000","last_change":"2024-05-07T02:26:25.584396+0000","last_active":"2024-05-07T02:31:50.092828+0000","last_peered":"2024-05-07T02:31:50.092828+0000","last_clean":"2024-05-07T02:31:50.092828+0000","last_became_active":"2024-05-07T02:26:25.583917+0000","last_became_peered":"2024-05-07T02:26:25.583917+0000","last_unstale":"2024-05-07T02:31:50.092828+0000","last_undegraded":"2024-05-07T02:31:50.092828+0000","last_fullsized":"2024-05-07T02:31:50.092828+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:56:34.495133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.097562+0000","last_change":"2024-05-07T02:26:07.795300+0000","last_active":"2024-05-07T02:31:50.097562+0000","last_peered":"2024-05-07T02:31:50.097562+0000","last_clean":"2024-05-07T02:31:50.097562+0000","last_became_active":"2024-05-07T02:26:07.794633+0000","last_became_peered":"2024-05-07T02:26:07.794633+0000","last_unstale":"2024-05-07T02:31:50.097562+0000","last_undegraded":"2024-05-07T02:31:50.097562+0000","last_fullsized":"2024-05-07T02:31:50.097562+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:36:44.252118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.824253+0000","last_change":"2024-05-07T02:26:11.877717+0000","last_active":"2024-05-07T02:31:49.824253+0000","last_peered":"2024-05-07T02:31:49.824253+0000","last_clean":"2024-05-07T02:31:49.824253+0000","last_became_active":"2024-05-07T02:26:11.876644+0000","last_became_peered":"2024-05-07T02:26:11.876644+0000","last_unstale":"2024-05-07T02:31:49.824253+0000","last_undegraded":"2024-05-07T02:31:49.824253+0000","last_fullsized":"2024-05-07T02:31:49.824253+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T05:20:36.176776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.094315+0000","last_change":"2024-05-07T02:26:25.572120+0000","last_active":"2024-05-07T02:31:50.094315+0000","last_peered":"2024-05-07T02:31:50.094315+0000","last_clean":"2024-05-07T02:31:50.094315+0000","last_became_active":"2024-05-07T02:26:25.571637+0000","last_became_peered":"2024-05-07T02:26:25.571637+0000","last_unstale":"2024-05-07T02:31:50.094315+0000","last_undegraded":"2024-05-07T02:31:50.094315+0000","last_fullsized":"2024-05-07T02:31:50.094315+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:03:54.574061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.999043+0000","last_change":"2024-05-07T02:26:07.784160+0000","last_active":"2024-05-07T02:31:49.999043+0000","last_peered":"2024-05-07T02:31:49.999043+0000","last_clean":"2024-05-07T02:31:49.999043+0000","last_became_active":"2024-05-07T02:26:07.783646+0000","last_became_peered":"2024-05-07T02:26:07.783646+0000","last_unstale":"2024-05-07T02:31:49.999043+0000","last_undegraded":"2024-05-07T02:31:49.999043+0000","last_fullsized":"2024-05-07T02:31:49.999043+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T14:25:37.906587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.984924+0000","last_change":"2024-05-07T02:26:11.874886+0000","last_active":"2024-05-07T02:31:49.984924+0000","last_peered":"2024-05-07T02:31:49.984924+0000","last_clean":"2024-05-07T02:31:49.984924+0000","last_became_active":"2024-05-07T02:26:11.874465+0000","last_became_peered":"2024-05-07T02:26:11.874465+0000","last_unstale":"2024-05-07T02:31:49.984924+0000","last_undegraded":"2024-05-07T02:31:49.984924+0000","last_fullsized":"2024-05-07T02:31:49.984924+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:57:19.530747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.824559+0000","last_change":"2024-05-07T02:26:25.558087+0000","last_active":"2024-05-07T02:31:49.824559+0000","last_peered":"2024-05-07T02:31:49.824559+0000","last_clean":"2024-05-07T02:31:49.824559+0000","last_became_active":"2024-05-07T02:26:25.557624+0000","last_became_peered":"2024-05-07T02:26:25.557624+0000","last_unstale":"2024-05-07T02:31:49.824559+0000","last_undegraded":"2024-05-07T02:31:49.824559+0000","last_fullsized":"2024-05-07T02:31:49.824559+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:03:20.104547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'6","reported_seq":546,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.219941+0000","last_change":"2024-05-07T02:26:07.788244+0000","last_active":"2024-05-07T02:31:50.219941+0000","last_peered":"2024-05-07T02:31:50.219941+0000","last_clean":"2024-05-07T02:31:50.219941+0000","last_became_active":"2024-05-07T02:26:07.787789+0000","last_became_peered":"2024-05-07T02:26:07.787789+0000","last_unstale":"2024-05-07T02:31:50.219941+0000","last_undegraded":"2024-05-07T02:31:50.219941+0000","last_fullsized":"2024-05-07T02:31:50.219941+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:12:22.943444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":492,"num_read_kb":492,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092921+0000","last_change":"2024-05-07T02:26:11.879821+0000","last_active":"2024-05-07T02:31:50.092921+0000","last_peered":"2024-05-07T02:31:50.092921+0000","last_clean":"2024-05-07T02:31:50.092921+0000","last_became_active":"2024-05-07T02:26:11.879372+0000","last_became_peered":"2024-05-07T02:26:11.879372+0000","last_unstale":"2024-05-07T02:31:50.092921+0000","last_undegraded":"2024-05-07T02:31:50.092921+0000","last_fullsized":"2024-05-07T02:31:50.092921+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:01:03.754890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.985163+0000","last_change":"2024-05-07T02:26:25.576663+0000","last_active":"2024-05-07T02:31:49.985163+0000","last_peered":"2024-05-07T02:31:49.985163+0000","last_clean":"2024-05-07T02:31:49.985163+0000","last_became_active":"2024-05-07T02:26:25.575729+0000","last_became_peered":"2024-05-07T02:26:25.575729+0000","last_unstale":"2024-05-07T02:31:49.985163+0000","last_undegraded":"2024-05-07T02:31:49.985163+0000","last_fullsized":"2024-05-07T02:31:49.985163+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:03:48.061385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.136515+0000","last_change":"2024-05-07T02:27:32.625336+0000","last_active":"2024-05-07T02:31:50.136515+0000","last_peered":"2024-05-07T02:31:50.136515+0000","last_clean":"2024-05-07T02:31:50.136515+0000","last_became_active":"2024-05-07T02:27:32.624805+0000","last_became_peered":"2024-05-07T02:27:32.624805+0000","last_unstale":"2024-05-07T02:31:50.136515+0000","last_undegraded":"2024-05-07T02:31:50.136515+0000","last_fullsized":"2024-05-07T02:31:50.136515+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:10:04.928297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.094830+0000","last_change":"2024-05-07T02:27:32.626374+0000","last_active":"2024-05-07T02:31:50.094830+0000","last_peered":"2024-05-07T02:31:50.094830+0000","last_clean":"2024-05-07T02:31:50.094830+0000","last_became_active":"2024-05-07T02:27:32.625489+0000","last_became_peered":"2024-05-07T02:27:32.625489+0000","last_unstale":"2024-05-07T02:31:50.094830+0000","last_undegraded":"2024-05-07T02:31:50.094830+0000","last_fullsized":"2024-05-07T02:31:50.094830+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T05:56:16.626041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825653+0000","last_change":"2024-05-07T02:26:25.580811+0000","last_active":"2024-05-07T02:31:49.825653+0000","last_peered":"2024-05-07T02:31:49.825653+0000","last_clean":"2024-05-07T02:31:49.825653+0000","last_became_active":"2024-05-07T02:26:25.579344+0000","last_became_peered":"2024-05-07T02:26:25.579344+0000","last_unstale":"2024-05-07T02:31:49.825653+0000","last_undegraded":"2024-05-07T02:31:49.825653+0000","last_fullsized":"2024-05-07T02:31:49.825653+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:25:36.186899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.999415+0000","last_change":"2024-05-07T02:26:08.000010+0000","last_active":"2024-05-07T02:31:49.999415+0000","last_peered":"2024-05-07T02:31:49.999415+0000","last_clean":"2024-05-07T02:31:49.999415+0000","last_became_active":"2024-05-07T02:26:07.999576+0000","last_became_peered":"2024-05-07T02:26:07.999576+0000","last_unstale":"2024-05-07T02:31:49.999415+0000","last_undegraded":"2024-05-07T02:31:49.999415+0000","last_fullsized":"2024-05-07T02:31:49.999415+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:23:38.644205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092735+0000","last_change":"2024-05-07T02:26:11.867098+0000","last_active":"2024-05-07T02:31:50.092735+0000","last_peered":"2024-05-07T02:31:50.092735+0000","last_clean":"2024-05-07T02:31:50.092735+0000","last_became_active":"2024-05-07T02:26:11.866036+0000","last_became_peered":"2024-05-07T02:26:11.866036+0000","last_unstale":"2024-05-07T02:31:50.092735+0000","last_undegraded":"2024-05-07T02:31:50.092735+0000","last_fullsized":"2024-05-07T02:31:50.092735+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:47:22.412323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.985140+0000","last_change":"2024-05-07T02:27:32.628243+0000","last_active":"2024-05-07T02:31:49.985140+0000","last_peered":"2024-05-07T02:31:49.985140+0000","last_clean":"2024-05-07T02:31:49.985140+0000","last_became_active":"2024-05-07T02:27:32.627048+0000","last_became_peered":"2024-05-07T02:27:32.627048+0000","last_unstale":"2024-05-07T02:31:49.985140+0000","last_undegraded":"2024-05-07T02:31:49.985140+0000","last_fullsized":"2024-05-07T02:31:49.985140+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:33:49.802873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,10],"acting":[9,8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'44","reported_seq":597,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.148780+0000","last_change":"2024-05-07T02:26:07.786841+0000","last_active":"2024-05-07T02:31:50.148780+0000","last_peered":"2024-05-07T02:31:50.148780+0000","last_clean":"2024-05-07T02:31:50.148780+0000","last_became_active":"2024-05-07T02:26:07.783836+0000","last_became_peered":"2024-05-07T02:26:07.783836+0000","last_unstale":"2024-05-07T02:31:50.148780+0000","last_undegraded":"2024-05-07T02:31:50.148780+0000","last_fullsized":"2024-05-07T02:31:50.148780+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:28:53.390009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":73494,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":505,"num_read_kb":326,"num_write":46,"num_write_kb":94,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.988712+0000","last_change":"2024-05-07T02:26:11.857292+0000","last_active":"2024-05-07T02:31:49.988712+0000","last_peered":"2024-05-07T02:31:49.988712+0000","last_clean":"2024-05-07T02:31:49.988712+0000","last_became_active":"2024-05-07T02:26:11.856899+0000","last_became_peered":"2024-05-07T02:26:11.856899+0000","last_unstale":"2024-05-07T02:31:49.988712+0000","last_undegraded":"2024-05-07T02:31:49.988712+0000","last_fullsized":"2024-05-07T02:31:49.988712+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:45:42.279848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.999888+0000","last_change":"2024-05-07T02:26:25.570714+0000","last_active":"2024-05-07T02:31:49.999888+0000","last_peered":"2024-05-07T02:31:49.999888+0000","last_clean":"2024-05-07T02:31:49.999888+0000","last_became_active":"2024-05-07T02:26:25.570163+0000","last_became_peered":"2024-05-07T02:26:25.570163+0000","last_unstale":"2024-05-07T02:31:49.999888+0000","last_undegraded":"2024-05-07T02:31:49.999888+0000","last_fullsized":"2024-05-07T02:31:49.999888+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:55:51.903968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":59,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.826064+0000","last_change":"2024-05-07T02:26:07.785904+0000","last_active":"2024-05-07T02:31:49.826064+0000","last_peered":"2024-05-07T02:31:49.826064+0000","last_clean":"2024-05-07T02:31:49.826064+0000","last_became_active":"2024-05-07T02:26:07.785347+0000","last_became_peered":"2024-05-07T02:26:07.785347+0000","last_unstale":"2024-05-07T02:31:49.826064+0000","last_undegraded":"2024-05-07T02:31:49.826064+0000","last_fullsized":"2024-05-07T02:31:49.826064+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:42:05.604266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.097758+0000","last_change":"2024-05-07T02:26:11.878511+0000","last_active":"2024-05-07T02:31:50.097758+0000","last_peered":"2024-05-07T02:31:50.097758+0000","last_clean":"2024-05-07T02:31:50.097758+0000","last_became_active":"2024-05-07T02:26:11.878166+0000","last_became_peered":"2024-05-07T02:26:11.878166+0000","last_unstale":"2024-05-07T02:31:50.097758+0000","last_undegraded":"2024-05-07T02:31:50.097758+0000","last_fullsized":"2024-05-07T02:31:50.097758+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T08:14:40.436244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092718+0000","last_change":"2024-05-07T02:26:25.579387+0000","last_active":"2024-05-07T02:31:50.092718+0000","last_peered":"2024-05-07T02:31:50.092718+0000","last_clean":"2024-05-07T02:31:50.092718+0000","last_became_active":"2024-05-07T02:26:25.578757+0000","last_became_peered":"2024-05-07T02:26:25.578757+0000","last_unstale":"2024-05-07T02:31:50.092718+0000","last_undegraded":"2024-05-07T02:31:50.092718+0000","last_fullsized":"2024-05-07T02:31:50.092718+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:19:19.650941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.999163+0000","last_change":"2024-05-07T02:26:07.785154+0000","last_active":"2024-05-07T02:31:49.999163+0000","last_peered":"2024-05-07T02:31:49.999163+0000","last_clean":"2024-05-07T02:31:49.999163+0000","last_became_active":"2024-05-07T02:26:07.784589+0000","last_became_peered":"2024-05-07T02:26:07.784589+0000","last_unstale":"2024-05-07T02:31:49.999163+0000","last_undegraded":"2024-05-07T02:31:49.999163+0000","last_fullsized":"2024-05-07T02:31:49.999163+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:41:41.851182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.136271+0000","last_change":"2024-05-07T02:26:11.878162+0000","last_active":"2024-05-07T02:31:50.136271+0000","last_peered":"2024-05-07T02:31:50.136271+0000","last_clean":"2024-05-07T02:31:50.136271+0000","last_became_active":"2024-05-07T02:26:11.877770+0000","last_became_peered":"2024-05-07T02:26:11.877770+0000","last_unstale":"2024-05-07T02:31:50.136271+0000","last_undegraded":"2024-05-07T02:31:50.136271+0000","last_fullsized":"2024-05-07T02:31:50.136271+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:19:08.209398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.985823+0000","last_change":"2024-05-07T02:26:25.579465+0000","last_active":"2024-05-07T02:31:49.985823+0000","last_peered":"2024-05-07T02:31:49.985823+0000","last_clean":"2024-05-07T02:31:49.985823+0000","last_became_active":"2024-05-07T02:26:25.579003+0000","last_became_peered":"2024-05-07T02:26:25.579003+0000","last_unstale":"2024-05-07T02:31:49.985823+0000","last_undegraded":"2024-05-07T02:31:49.985823+0000","last_fullsized":"2024-05-07T02:31:49.985823+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:13:24.356366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'8","reported_seq":306,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.379222+0000","last_change":"2024-05-07T02:26:07.780827+0000","last_active":"2024-05-07T02:31:50.379222+0000","last_peered":"2024-05-07T02:31:50.379222+0000","last_clean":"2024-05-07T02:31:50.379222+0000","last_became_active":"2024-05-07T02:26:07.780353+0000","last_became_peered":"2024-05-07T02:26:07.780353+0000","last_unstale":"2024-05-07T02:31:50.379222+0000","last_undegraded":"2024-05-07T02:31:50.379222+0000","last_fullsized":"2024-05-07T02:31:50.379222+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:52:40.554135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1680,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":251,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.136134+0000","last_change":"2024-05-07T02:26:11.852516+0000","last_active":"2024-05-07T02:31:50.136134+0000","last_peered":"2024-05-07T02:31:50.136134+0000","last_clean":"2024-05-07T02:31:50.136134+0000","last_became_active":"2024-05-07T02:26:11.852112+0000","last_became_peered":"2024-05-07T02:26:11.852112+0000","last_unstale":"2024-05-07T02:31:50.136134+0000","last_undegraded":"2024-05-07T02:31:50.136134+0000","last_fullsized":"2024-05-07T02:31:50.136134+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:58:46.989250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.098786+0000","last_change":"2024-05-07T02:26:25.586726+0000","last_active":"2024-05-07T02:31:50.098786+0000","last_peered":"2024-05-07T02:31:50.098786+0000","last_clean":"2024-05-07T02:31:50.098786+0000","last_became_active":"2024-05-07T02:26:25.586267+0000","last_became_peered":"2024-05-07T02:26:25.586267+0000","last_unstale":"2024-05-07T02:31:50.098786+0000","last_undegraded":"2024-05-07T02:31:50.098786+0000","last_fullsized":"2024-05-07T02:31:50.098786+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:50:00.864463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092578+0000","last_change":"2024-05-07T02:27:32.599511+0000","last_active":"2024-05-07T02:31:50.092578+0000","last_peered":"2024-05-07T02:31:50.092578+0000","last_clean":"2024-05-07T02:31:50.092578+0000","last_became_active":"2024-05-07T02:27:32.590197+0000","last_became_peered":"2024-05-07T02:27:32.590197+0000","last_unstale":"2024-05-07T02:31:50.092578+0000","last_undegraded":"2024-05-07T02:31:50.092578+0000","last_fullsized":"2024-05-07T02:31:50.092578+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:23:07.401251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.999070+0000","last_change":"2024-05-07T02:26:11.857671+0000","last_active":"2024-05-07T02:31:49.999070+0000","last_peered":"2024-05-07T02:31:49.999070+0000","last_clean":"2024-05-07T02:31:49.999070+0000","last_became_active":"2024-05-07T02:26:11.857293+0000","last_became_peered":"2024-05-07T02:26:11.857293+0000","last_unstale":"2024-05-07T02:31:49.999070+0000","last_undegraded":"2024-05-07T02:31:49.999070+0000","last_fullsized":"2024-05-07T02:31:49.999070+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:48:42.267887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.999012+0000","last_change":"2024-05-07T02:26:25.563274+0000","last_active":"2024-05-07T02:31:49.999012+0000","last_peered":"2024-05-07T02:31:49.999012+0000","last_clean":"2024-05-07T02:31:49.999012+0000","last_became_active":"2024-05-07T02:26:25.562868+0000","last_became_peered":"2024-05-07T02:26:25.562868+0000","last_unstale":"2024-05-07T02:31:49.999012+0000","last_undegraded":"2024-05-07T02:31:49.999012+0000","last_fullsized":"2024-05-07T02:31:49.999012+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T14:20:03.210034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'4","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.988931+0000","last_change":"2024-05-07T02:26:07.784888+0000","last_active":"2024-05-07T02:31:49.988931+0000","last_peered":"2024-05-07T02:31:49.988931+0000","last_clean":"2024-05-07T02:31:49.988931+0000","last_became_active":"2024-05-07T02:26:07.784493+0000","last_became_peered":"2024-05-07T02:26:07.784493+0000","last_unstale":"2024-05-07T02:31:49.988931+0000","last_undegraded":"2024-05-07T02:31:49.988931+0000","last_fullsized":"2024-05-07T02:31:49.988931+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:21:14.054786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.093166+0000","last_change":"2024-05-07T02:26:11.858887+0000","last_active":"2024-05-07T02:31:50.093166+0000","last_peered":"2024-05-07T02:31:50.093166+0000","last_clean":"2024-05-07T02:31:50.093166+0000","last_became_active":"2024-05-07T02:26:11.858504+0000","last_became_peered":"2024-05-07T02:26:11.858504+0000","last_unstale":"2024-05-07T02:31:50.093166+0000","last_undegraded":"2024-05-07T02:31:50.093166+0000","last_fullsized":"2024-05-07T02:31:50.093166+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:53:20.290039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.988326+0000","last_change":"2024-05-07T02:27:32.627310+0000","last_active":"2024-05-07T02:31:49.988326+0000","last_peered":"2024-05-07T02:31:49.988326+0000","last_clean":"2024-05-07T02:31:49.988326+0000","last_became_active":"2024-05-07T02:27:32.626794+0000","last_became_peered":"2024-05-07T02:27:32.626794+0000","last_unstale":"2024-05-07T02:31:49.988326+0000","last_undegraded":"2024-05-07T02:31:49.988326+0000","last_fullsized":"2024-05-07T02:31:49.988326+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:06:30.466291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.985349+0000","last_change":"2024-05-07T02:26:08.000604+0000","last_active":"2024-05-07T02:31:49.985349+0000","last_peered":"2024-05-07T02:31:49.985349+0000","last_clean":"2024-05-07T02:31:49.985349+0000","last_became_active":"2024-05-07T02:26:08.000107+0000","last_became_peered":"2024-05-07T02:26:08.000107+0000","last_unstale":"2024-05-07T02:31:49.985349+0000","last_undegraded":"2024-05-07T02:31:49.985349+0000","last_fullsized":"2024-05-07T02:31:49.985349+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:58:06.716456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.003473+0000","last_change":"2024-05-07T02:26:11.876555+0000","last_active":"2024-05-07T02:31:50.003473+0000","last_peered":"2024-05-07T02:31:50.003473+0000","last_clean":"2024-05-07T02:31:50.003473+0000","last_became_active":"2024-05-07T02:26:11.876109+0000","last_became_peered":"2024-05-07T02:26:11.876109+0000","last_unstale":"2024-05-07T02:31:50.003473+0000","last_undegraded":"2024-05-07T02:31:50.003473+0000","last_fullsized":"2024-05-07T02:31:50.003473+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:10:24.995786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.098087+0000","last_change":"2024-05-07T02:26:25.567145+0000","last_active":"2024-05-07T02:31:50.098087+0000","last_peered":"2024-05-07T02:31:50.098087+0000","last_clean":"2024-05-07T02:31:50.098087+0000","last_became_active":"2024-05-07T02:26:25.566767+0000","last_became_peered":"2024-05-07T02:26:25.566767+0000","last_unstale":"2024-05-07T02:31:50.098087+0000","last_undegraded":"2024-05-07T02:31:50.098087+0000","last_fullsized":"2024-05-07T02:31:50.098087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T05:10:59.333367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'8","reported_seq":306,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.260546+0000","last_change":"2024-05-07T02:26:07.789566+0000","last_active":"2024-05-07T02:31:50.260546+0000","last_peered":"2024-05-07T02:31:50.260546+0000","last_clean":"2024-05-07T02:31:50.260546+0000","last_became_active":"2024-05-07T02:26:07.789106+0000","last_became_peered":"2024-05-07T02:26:07.789106+0000","last_unstale":"2024-05-07T02:31:50.260546+0000","last_undegraded":"2024-05-07T02:31:50.260546+0000","last_fullsized":"2024-05-07T02:31:50.260546+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:45:13.620183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":250,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.003155+0000","last_change":"2024-05-07T02:26:11.869080+0000","last_active":"2024-05-07T02:31:50.003155+0000","last_peered":"2024-05-07T02:31:50.003155+0000","last_clean":"2024-05-07T02:31:50.003155+0000","last_became_active":"2024-05-07T02:26:11.868702+0000","last_became_peered":"2024-05-07T02:26:11.868702+0000","last_unstale":"2024-05-07T02:31:50.003155+0000","last_undegraded":"2024-05-07T02:31:50.003155+0000","last_fullsized":"2024-05-07T02:31:50.003155+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:05:05.497071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.098519+0000","last_change":"2024-05-07T02:26:25.867393+0000","last_active":"2024-05-07T02:31:50.098519+0000","last_peered":"2024-05-07T02:31:50.098519+0000","last_clean":"2024-05-07T02:31:50.098519+0000","last_became_active":"2024-05-07T02:26:25.866956+0000","last_became_peered":"2024-05-07T02:26:25.866956+0000","last_unstale":"2024-05-07T02:31:50.098519+0000","last_undegraded":"2024-05-07T02:31:50.098519+0000","last_fullsized":"2024-05-07T02:31:50.098519+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:23:48.821668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.984780+0000","last_change":"2024-05-07T02:28:33.871287+0000","last_active":"2024-05-07T02:31:49.984780+0000","last_peered":"2024-05-07T02:31:49.984780+0000","last_clean":"2024-05-07T02:31:49.984780+0000","last_became_active":"2024-05-07T02:28:33.870256+0000","last_became_peered":"2024-05-07T02:28:33.870256+0000","last_unstale":"2024-05-07T02:31:49.984780+0000","last_undegraded":"2024-05-07T02:31:49.984780+0000","last_fullsized":"2024-05-07T02:31:49.984780+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T08:36:26.789348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":301,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.097586+0000","last_change":"2024-05-07T02:26:07.999784+0000","last_active":"2024-05-07T02:31:50.097586+0000","last_peered":"2024-05-07T02:31:50.097586+0000","last_clean":"2024-05-07T02:31:50.097586+0000","last_became_active":"2024-05-07T02:26:07.999339+0000","last_became_peered":"2024-05-07T02:26:07.999339+0000","last_unstale":"2024-05-07T02:31:50.097586+0000","last_undegraded":"2024-05-07T02:31:50.097586+0000","last_fullsized":"2024-05-07T02:31:50.097586+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:50:41.622486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":247,"num_read_kb":247,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.136478+0000","last_change":"2024-05-07T02:26:25.587032+0000","last_active":"2024-05-07T02:31:50.136478+0000","last_peered":"2024-05-07T02:31:50.136478+0000","last_clean":"2024-05-07T02:31:50.136478+0000","last_became_active":"2024-05-07T02:26:25.586530+0000","last_became_peered":"2024-05-07T02:26:25.586530+0000","last_unstale":"2024-05-07T02:31:50.136478+0000","last_undegraded":"2024-05-07T02:31:50.136478+0000","last_fullsized":"2024-05-07T02:31:50.136478+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:13:13.335832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825819+0000","last_change":"2024-05-07T02:26:11.878593+0000","last_active":"2024-05-07T02:31:49.825819+0000","last_peered":"2024-05-07T02:31:49.825819+0000","last_clean":"2024-05-07T02:31:49.825819+0000","last_became_active":"2024-05-07T02:26:11.873286+0000","last_became_peered":"2024-05-07T02:26:11.873286+0000","last_unstale":"2024-05-07T02:31:49.825819+0000","last_undegraded":"2024-05-07T02:31:49.825819+0000","last_fullsized":"2024-05-07T02:31:49.825819+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:22:33.769539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825626+0000","last_change":"2024-05-07T02:26:07.790087+0000","last_active":"2024-05-07T02:31:49.825626+0000","last_peered":"2024-05-07T02:31:49.825626+0000","last_clean":"2024-05-07T02:31:49.825626+0000","last_became_active":"2024-05-07T02:26:07.789592+0000","last_became_peered":"2024-05-07T02:26:07.789592+0000","last_unstale":"2024-05-07T02:31:49.825626+0000","last_undegraded":"2024-05-07T02:31:49.825626+0000","last_fullsized":"2024-05-07T02:31:49.825626+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:44:52.569055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":122,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.135972+0000","last_change":"2024-05-07T02:24:27.877820+0000","last_active":"2024-05-07T02:31:50.135972+0000","last_peered":"2024-05-07T02:31:50.135972+0000","last_clean":"2024-05-07T02:31:50.135972+0000","last_became_active":"2024-05-07T02:24:27.873221+0000","last_became_peered":"2024-05-07T02:24:27.873221+0000","last_unstale":"2024-05-07T02:31:50.135972+0000","last_undegraded":"2024-05-07T02:31:50.135972+0000","last_fullsized":"2024-05-07T02:31:50.135972+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:20:21.582322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:20:21.582322+0000","last_clean_scrub_stamp":"2024-05-07T02:20:21.582322+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T14:16:32.772438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.999518+0000","last_change":"2024-05-07T02:26:25.868394+0000","last_active":"2024-05-07T02:31:49.999518+0000","last_peered":"2024-05-07T02:31:49.999518+0000","last_clean":"2024-05-07T02:31:49.999518+0000","last_became_active":"2024-05-07T02:26:25.867583+0000","last_became_peered":"2024-05-07T02:26:25.867583+0000","last_unstale":"2024-05-07T02:31:49.999518+0000","last_undegraded":"2024-05-07T02:31:49.999518+0000","last_fullsized":"2024-05-07T02:31:49.999518+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:55:44.705608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.097400+0000","last_change":"2024-05-07T02:26:11.868371+0000","last_active":"2024-05-07T02:31:50.097400+0000","last_peered":"2024-05-07T02:31:50.097400+0000","last_clean":"2024-05-07T02:31:50.097400+0000","last_became_active":"2024-05-07T02:26:11.867981+0000","last_became_peered":"2024-05-07T02:26:11.867981+0000","last_unstale":"2024-05-07T02:31:50.097400+0000","last_undegraded":"2024-05-07T02:31:50.097400+0000","last_fullsized":"2024-05-07T02:31:50.097400+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:08:54.687327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.136059+0000","last_change":"2024-05-07T02:26:07.785344+0000","last_active":"2024-05-07T02:31:50.136059+0000","last_peered":"2024-05-07T02:31:50.136059+0000","last_clean":"2024-05-07T02:31:50.136059+0000","last_became_active":"2024-05-07T02:26:07.784707+0000","last_became_peered":"2024-05-07T02:26:07.784707+0000","last_unstale":"2024-05-07T02:31:50.136059+0000","last_undegraded":"2024-05-07T02:31:50.136059+0000","last_fullsized":"2024-05-07T02:31:50.136059+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T05:00:38.754313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092632+0000","last_change":"2024-05-07T02:26:25.571989+0000","last_active":"2024-05-07T02:31:50.092632+0000","last_peered":"2024-05-07T02:31:50.092632+0000","last_clean":"2024-05-07T02:31:50.092632+0000","last_became_active":"2024-05-07T02:26:25.571533+0000","last_became_peered":"2024-05-07T02:26:25.571533+0000","last_unstale":"2024-05-07T02:31:50.092632+0000","last_undegraded":"2024-05-07T02:31:50.092632+0000","last_fullsized":"2024-05-07T02:31:50.092632+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:54:38.669278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.988658+0000","last_change":"2024-05-07T02:26:07.999636+0000","last_active":"2024-05-07T02:31:49.988658+0000","last_peered":"2024-05-07T02:31:49.988658+0000","last_clean":"2024-05-07T02:31:49.988658+0000","last_became_active":"2024-05-07T02:26:07.999100+0000","last_became_peered":"2024-05-07T02:26:07.999100+0000","last_unstale":"2024-05-07T02:31:49.988658+0000","last_undegraded":"2024-05-07T02:31:49.988658+0000","last_fullsized":"2024-05-07T02:31:49.988658+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:10:24.730988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.098303+0000","last_change":"2024-05-07T02:26:11.888279+0000","last_active":"2024-05-07T02:31:50.098303+0000","last_peered":"2024-05-07T02:31:50.098303+0000","last_clean":"2024-05-07T02:31:50.098303+0000","last_became_active":"2024-05-07T02:26:11.887480+0000","last_became_peered":"2024-05-07T02:26:11.887480+0000","last_unstale":"2024-05-07T02:31:50.098303+0000","last_undegraded":"2024-05-07T02:31:50.098303+0000","last_fullsized":"2024-05-07T02:31:50.098303+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:12:12.368368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.824134+0000","last_change":"2024-05-07T02:26:25.867444+0000","last_active":"2024-05-07T02:31:49.824134+0000","last_peered":"2024-05-07T02:31:49.824134+0000","last_clean":"2024-05-07T02:31:49.824134+0000","last_became_active":"2024-05-07T02:26:25.866392+0000","last_became_peered":"2024-05-07T02:26:25.866392+0000","last_unstale":"2024-05-07T02:31:49.824134+0000","last_undegraded":"2024-05-07T02:31:49.824134+0000","last_fullsized":"2024-05-07T02:31:49.824134+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:54:36.112427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'52","reported_seq":123,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.150979+0000","last_change":"2024-05-07T02:26:07.798091+0000","last_active":"2024-05-07T02:31:50.150979+0000","last_peered":"2024-05-07T02:31:50.150979+0000","last_clean":"2024-05-07T02:31:50.150979+0000","last_became_active":"2024-05-07T02:26:07.797716+0000","last_became_peered":"2024-05-07T02:26:07.797716+0000","last_unstale":"2024-05-07T02:31:50.150979+0000","last_undegraded":"2024-05-07T02:31:50.150979+0000","last_fullsized":"2024-05-07T02:31:50.150979+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:04:54.570916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":92,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825760+0000","last_change":"2024-05-07T02:26:11.888638+0000","last_active":"2024-05-07T02:31:49.825760+0000","last_peered":"2024-05-07T02:31:49.825760+0000","last_clean":"2024-05-07T02:31:49.825760+0000","last_became_active":"2024-05-07T02:26:11.888269+0000","last_became_peered":"2024-05-07T02:26:11.888269+0000","last_unstale":"2024-05-07T02:31:49.825760+0000","last_undegraded":"2024-05-07T02:31:49.825760+0000","last_fullsized":"2024-05-07T02:31:49.825760+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:09:38.965362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825175+0000","last_change":"2024-05-07T02:26:25.558975+0000","last_active":"2024-05-07T02:31:49.825175+0000","last_peered":"2024-05-07T02:31:49.825175+0000","last_clean":"2024-05-07T02:31:49.825175+0000","last_became_active":"2024-05-07T02:26:25.557947+0000","last_became_peered":"2024-05-07T02:26:25.557947+0000","last_unstale":"2024-05-07T02:31:49.825175+0000","last_undegraded":"2024-05-07T02:31:49.825175+0000","last_fullsized":"2024-05-07T02:31:49.825175+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:29:32.071625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.826855+0000","last_change":"2024-05-07T02:26:07.796643+0000","last_active":"2024-05-07T02:31:49.826855+0000","last_peered":"2024-05-07T02:31:49.826855+0000","last_clean":"2024-05-07T02:31:49.826855+0000","last_became_active":"2024-05-07T02:26:07.796264+0000","last_became_peered":"2024-05-07T02:26:07.796264+0000","last_unstale":"2024-05-07T02:31:49.826855+0000","last_undegraded":"2024-05-07T02:31:49.826855+0000","last_fullsized":"2024-05-07T02:31:49.826855+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:29:32.454816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.094595+0000","last_change":"2024-05-07T02:26:11.873670+0000","last_active":"2024-05-07T02:31:50.094595+0000","last_peered":"2024-05-07T02:31:50.094595+0000","last_clean":"2024-05-07T02:31:50.094595+0000","last_became_active":"2024-05-07T02:26:11.871219+0000","last_became_peered":"2024-05-07T02:26:11.871219+0000","last_unstale":"2024-05-07T02:31:50.094595+0000","last_undegraded":"2024-05-07T02:31:50.094595+0000","last_fullsized":"2024-05-07T02:31:50.094595+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:49:48.048127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.826601+0000","last_change":"2024-05-07T02:26:25.868011+0000","last_active":"2024-05-07T02:31:49.826601+0000","last_peered":"2024-05-07T02:31:49.826601+0000","last_clean":"2024-05-07T02:31:49.826601+0000","last_became_active":"2024-05-07T02:26:25.867465+0000","last_became_peered":"2024-05-07T02:26:25.867465+0000","last_unstale":"2024-05-07T02:31:49.826601+0000","last_undegraded":"2024-05-07T02:31:49.826601+0000","last_fullsized":"2024-05-07T02:31:49.826601+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T05:14:40.001874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092382+0000","last_change":"2024-05-07T02:26:07.999233+0000","last_active":"2024-05-07T02:31:50.092382+0000","last_peered":"2024-05-07T02:31:50.092382+0000","last_clean":"2024-05-07T02:31:50.092382+0000","last_became_active":"2024-05-07T02:26:07.998920+0000","last_became_peered":"2024-05-07T02:26:07.998920+0000","last_unstale":"2024-05-07T02:31:50.092382+0000","last_undegraded":"2024-05-07T02:31:50.092382+0000","last_fullsized":"2024-05-07T02:31:50.092382+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:33:33.878984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.988283+0000","last_change":"2024-05-07T02:26:11.856832+0000","last_active":"2024-05-07T02:31:49.988283+0000","last_peered":"2024-05-07T02:31:49.988283+0000","last_clean":"2024-05-07T02:31:49.988283+0000","last_became_active":"2024-05-07T02:26:11.856424+0000","last_became_peered":"2024-05-07T02:26:11.856424+0000","last_unstale":"2024-05-07T02:31:49.988283+0000","last_undegraded":"2024-05-07T02:31:49.988283+0000","last_fullsized":"2024-05-07T02:31:49.988283+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:00:46.839542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.988845+0000","last_change":"2024-05-07T02:27:32.626086+0000","last_active":"2024-05-07T02:31:49.988845+0000","last_peered":"2024-05-07T02:31:49.988845+0000","last_clean":"2024-05-07T02:31:49.988845+0000","last_became_active":"2024-05-07T02:27:32.625468+0000","last_became_peered":"2024-05-07T02:27:32.625468+0000","last_unstale":"2024-05-07T02:31:49.988845+0000","last_undegraded":"2024-05-07T02:31:49.988845+0000","last_fullsized":"2024-05-07T02:31:49.988845+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:17:57.245648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.824755+0000","last_change":"2024-05-07T02:26:07.793793+0000","last_active":"2024-05-07T02:31:49.824755+0000","last_peered":"2024-05-07T02:31:49.824755+0000","last_clean":"2024-05-07T02:31:49.824755+0000","last_became_active":"2024-05-07T02:26:07.790291+0000","last_became_peered":"2024-05-07T02:26:07.790291+0000","last_unstale":"2024-05-07T02:31:49.824755+0000","last_undegraded":"2024-05-07T02:31:49.824755+0000","last_fullsized":"2024-05-07T02:31:49.824755+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:50:14.591968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092657+0000","last_change":"2024-05-07T02:26:11.869184+0000","last_active":"2024-05-07T02:31:50.092657+0000","last_peered":"2024-05-07T02:31:50.092657+0000","last_clean":"2024-05-07T02:31:50.092657+0000","last_became_active":"2024-05-07T02:26:11.868773+0000","last_became_peered":"2024-05-07T02:26:11.868773+0000","last_unstale":"2024-05-07T02:31:50.092657+0000","last_undegraded":"2024-05-07T02:31:50.092657+0000","last_fullsized":"2024-05-07T02:31:50.092657+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:24:04.986359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825656+0000","last_change":"2024-05-07T02:26:25.580095+0000","last_active":"2024-05-07T02:31:49.825656+0000","last_peered":"2024-05-07T02:31:49.825656+0000","last_clean":"2024-05-07T02:31:49.825656+0000","last_became_active":"2024-05-07T02:26:25.579615+0000","last_became_peered":"2024-05-07T02:26:25.579615+0000","last_unstale":"2024-05-07T02:31:49.825656+0000","last_undegraded":"2024-05-07T02:31:49.825656+0000","last_fullsized":"2024-05-07T02:31:49.825656+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:18:05.714552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'4","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825368+0000","last_change":"2024-05-07T02:26:07.784709+0000","last_active":"2024-05-07T02:31:49.825368+0000","last_peered":"2024-05-07T02:31:49.825368+0000","last_clean":"2024-05-07T02:31:49.825368+0000","last_became_active":"2024-05-07T02:26:07.784282+0000","last_became_peered":"2024-05-07T02:26:07.784282+0000","last_unstale":"2024-05-07T02:31:49.825368+0000","last_undegraded":"2024-05-07T02:31:49.825368+0000","last_fullsized":"2024-05-07T02:31:49.825368+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:01:47.023496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.826271+0000","last_change":"2024-05-07T02:26:11.879152+0000","last_active":"2024-05-07T02:31:49.826271+0000","last_peered":"2024-05-07T02:31:49.826271+0000","last_clean":"2024-05-07T02:31:49.826271+0000","last_became_active":"2024-05-07T02:26:11.878571+0000","last_became_peered":"2024-05-07T02:26:11.878571+0000","last_unstale":"2024-05-07T02:31:49.826271+0000","last_undegraded":"2024-05-07T02:31:49.826271+0000","last_fullsized":"2024-05-07T02:31:49.826271+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:04:12.637807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.826575+0000","last_change":"2024-05-07T02:26:25.867199+0000","last_active":"2024-05-07T02:31:49.826575+0000","last_peered":"2024-05-07T02:31:49.826575+0000","last_clean":"2024-05-07T02:31:49.826575+0000","last_became_active":"2024-05-07T02:26:25.866624+0000","last_became_peered":"2024-05-07T02:26:25.866624+0000","last_unstale":"2024-05-07T02:31:49.826575+0000","last_undegraded":"2024-05-07T02:31:49.826575+0000","last_fullsized":"2024-05-07T02:31:49.826575+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:14:49.513313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.988386+0000","last_change":"2024-05-07T02:26:07.784155+0000","last_active":"2024-05-07T02:31:49.988386+0000","last_peered":"2024-05-07T02:31:49.988386+0000","last_clean":"2024-05-07T02:31:49.988386+0000","last_became_active":"2024-05-07T02:26:07.783579+0000","last_became_peered":"2024-05-07T02:26:07.783579+0000","last_unstale":"2024-05-07T02:31:49.988386+0000","last_undegraded":"2024-05-07T02:31:49.988386+0000","last_fullsized":"2024-05-07T02:31:49.988386+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T08:51:32.037959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.136365+0000","last_change":"2024-05-07T02:26:11.868454+0000","last_active":"2024-05-07T02:31:50.136365+0000","last_peered":"2024-05-07T02:31:50.136365+0000","last_clean":"2024-05-07T02:31:50.136365+0000","last_became_active":"2024-05-07T02:26:11.868117+0000","last_became_peered":"2024-05-07T02:26:11.868117+0000","last_unstale":"2024-05-07T02:31:50.136365+0000","last_undegraded":"2024-05-07T02:31:50.136365+0000","last_fullsized":"2024-05-07T02:31:50.136365+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:24:36.703684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092403+0000","last_change":"2024-05-07T02:26:25.573834+0000","last_active":"2024-05-07T02:31:50.092403+0000","last_peered":"2024-05-07T02:31:50.092403+0000","last_clean":"2024-05-07T02:31:50.092403+0000","last_became_active":"2024-05-07T02:26:25.573372+0000","last_became_peered":"2024-05-07T02:26:25.573372+0000","last_unstale":"2024-05-07T02:31:50.092403+0000","last_undegraded":"2024-05-07T02:31:50.092403+0000","last_fullsized":"2024-05-07T02:31:50.092403+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:03:36.916915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'6","reported_seq":303,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.305895+0000","last_change":"2024-05-07T02:26:08.000245+0000","last_active":"2024-05-07T02:31:50.305895+0000","last_peered":"2024-05-07T02:31:50.305895+0000","last_clean":"2024-05-07T02:31:50.305895+0000","last_became_active":"2024-05-07T02:26:07.999587+0000","last_became_peered":"2024-05-07T02:26:07.999587+0000","last_unstale":"2024-05-07T02:31:50.305895+0000","last_undegraded":"2024-05-07T02:31:50.305895+0000","last_fullsized":"2024-05-07T02:31:50.305895+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:44:41.754799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":249,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.136693+0000","last_change":"2024-05-07T02:26:11.858308+0000","last_active":"2024-05-07T02:31:50.136693+0000","last_peered":"2024-05-07T02:31:50.136693+0000","last_clean":"2024-05-07T02:31:50.136693+0000","last_became_active":"2024-05-07T02:26:11.857855+0000","last_became_peered":"2024-05-07T02:26:11.857855+0000","last_unstale":"2024-05-07T02:31:50.136693+0000","last_undegraded":"2024-05-07T02:31:50.136693+0000","last_fullsized":"2024-05-07T02:31:50.136693+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:11:50.021178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.095840+0000","last_change":"2024-05-07T02:26:25.584090+0000","last_active":"2024-05-07T02:31:50.095840+0000","last_peered":"2024-05-07T02:31:50.095840+0000","last_clean":"2024-05-07T02:31:50.095840+0000","last_became_active":"2024-05-07T02:26:25.580108+0000","last_became_peered":"2024-05-07T02:26:25.580108+0000","last_unstale":"2024-05-07T02:31:50.095840+0000","last_undegraded":"2024-05-07T02:31:50.095840+0000","last_fullsized":"2024-05-07T02:31:50.095840+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:56:56.964047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'41","reported_seq":97,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.147728+0000","last_change":"2024-05-07T02:26:07.788188+0000","last_active":"2024-05-07T02:31:50.147728+0000","last_peered":"2024-05-07T02:31:50.147728+0000","last_clean":"2024-05-07T02:31:50.147728+0000","last_became_active":"2024-05-07T02:26:07.787826+0000","last_became_peered":"2024-05-07T02:26:07.787826+0000","last_unstale":"2024-05-07T02:31:50.147728+0000","last_undegraded":"2024-05-07T02:31:50.147728+0000","last_fullsized":"2024-05-07T02:31:50.147728+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:06:14.588079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.136460+0000","last_change":"2024-05-07T02:26:11.862925+0000","last_active":"2024-05-07T02:31:50.136460+0000","last_peered":"2024-05-07T02:31:50.136460+0000","last_clean":"2024-05-07T02:31:50.136460+0000","last_became_active":"2024-05-07T02:26:11.861332+0000","last_became_peered":"2024-05-07T02:26:11.861332+0000","last_unstale":"2024-05-07T02:31:50.136460+0000","last_undegraded":"2024-05-07T02:31:50.136460+0000","last_fullsized":"2024-05-07T02:31:50.136460+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:51:11.133504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.998822+0000","last_change":"2024-05-07T02:26:25.557151+0000","last_active":"2024-05-07T02:31:49.998822+0000","last_peered":"2024-05-07T02:31:49.998822+0000","last_clean":"2024-05-07T02:31:49.998822+0000","last_became_active":"2024-05-07T02:26:25.556615+0000","last_became_peered":"2024-05-07T02:26:25.556615+0000","last_unstale":"2024-05-07T02:31:49.998822+0000","last_undegraded":"2024-05-07T02:31:49.998822+0000","last_fullsized":"2024-05-07T02:31:49.998822+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:33:20.446543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":63,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.098780+0000","last_change":"2024-05-07T02:27:32.598094+0000","last_active":"2024-05-07T02:31:50.098780+0000","last_peered":"2024-05-07T02:31:50.098780+0000","last_clean":"2024-05-07T02:31:50.098780+0000","last_became_active":"2024-05-07T02:27:32.594099+0000","last_became_peered":"2024-05-07T02:27:32.594099+0000","last_unstale":"2024-05-07T02:31:50.098780+0000","last_undegraded":"2024-05-07T02:31:50.098780+0000","last_fullsized":"2024-05-07T02:31:50.098780+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:44:50.979927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.136717+0000","last_change":"2024-05-07T02:26:11.860507+0000","last_active":"2024-05-07T02:31:50.136717+0000","last_peered":"2024-05-07T02:31:50.136717+0000","last_clean":"2024-05-07T02:31:50.136717+0000","last_became_active":"2024-05-07T02:26:11.860116+0000","last_became_peered":"2024-05-07T02:26:11.860116+0000","last_unstale":"2024-05-07T02:31:50.136717+0000","last_undegraded":"2024-05-07T02:31:50.136717+0000","last_fullsized":"2024-05-07T02:31:50.136717+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T05:09:38.963040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092816+0000","last_change":"2024-05-07T02:26:11.867932+0000","last_active":"2024-05-07T02:31:50.092816+0000","last_peered":"2024-05-07T02:31:50.092816+0000","last_clean":"2024-05-07T02:31:50.092816+0000","last_became_active":"2024-05-07T02:26:11.867537+0000","last_became_peered":"2024-05-07T02:26:11.867537+0000","last_unstale":"2024-05-07T02:31:50.092816+0000","last_undegraded":"2024-05-07T02:31:50.092816+0000","last_fullsized":"2024-05-07T02:31:50.092816+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:31:35.843401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825700+0000","last_change":"2024-05-07T02:26:11.866074+0000","last_active":"2024-05-07T02:31:49.825700+0000","last_peered":"2024-05-07T02:31:49.825700+0000","last_clean":"2024-05-07T02:31:49.825700+0000","last_became_active":"2024-05-07T02:26:11.865669+0000","last_became_peered":"2024-05-07T02:26:11.865669+0000","last_unstale":"2024-05-07T02:31:49.825700+0000","last_undegraded":"2024-05-07T02:31:49.825700+0000","last_fullsized":"2024-05-07T02:31:49.825700+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:08:51.323706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825297+0000","last_change":"2024-05-07T02:26:11.876428+0000","last_active":"2024-05-07T02:31:49.825297+0000","last_peered":"2024-05-07T02:31:49.825297+0000","last_clean":"2024-05-07T02:31:49.825297+0000","last_became_active":"2024-05-07T02:26:11.876012+0000","last_became_peered":"2024-05-07T02:26:11.876012+0000","last_unstale":"2024-05-07T02:31:49.825297+0000","last_undegraded":"2024-05-07T02:31:49.825297+0000","last_fullsized":"2024-05-07T02:31:49.825297+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:24:58.088695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.094427+0000","last_change":"2024-05-07T02:26:11.880682+0000","last_active":"2024-05-07T02:31:50.094427+0000","last_peered":"2024-05-07T02:31:50.094427+0000","last_clean":"2024-05-07T02:31:50.094427+0000","last_became_active":"2024-05-07T02:26:11.880308+0000","last_became_peered":"2024-05-07T02:26:11.880308+0000","last_unstale":"2024-05-07T02:31:50.094427+0000","last_undegraded":"2024-05-07T02:31:50.094427+0000","last_fullsized":"2024-05-07T02:31:50.094427+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:52:45.083964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.098433+0000","last_change":"2024-05-07T02:26:11.876932+0000","last_active":"2024-05-07T02:31:50.098433+0000","last_peered":"2024-05-07T02:31:50.098433+0000","last_clean":"2024-05-07T02:31:50.098433+0000","last_became_active":"2024-05-07T02:26:11.876082+0000","last_became_peered":"2024-05-07T02:26:11.876082+0000","last_unstale":"2024-05-07T02:31:50.098433+0000","last_undegraded":"2024-05-07T02:31:50.098433+0000","last_fullsized":"2024-05-07T02:31:50.098433+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:45:21.435500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092847+0000","last_change":"2024-05-07T02:26:11.861339+0000","last_active":"2024-05-07T02:31:50.092847+0000","last_peered":"2024-05-07T02:31:50.092847+0000","last_clean":"2024-05-07T02:31:50.092847+0000","last_became_active":"2024-05-07T02:26:11.860912+0000","last_became_peered":"2024-05-07T02:26:11.860912+0000","last_unstale":"2024-05-07T02:31:50.092847+0000","last_undegraded":"2024-05-07T02:31:50.092847+0000","last_fullsized":"2024-05-07T02:31:50.092847+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:12:57.944986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825780+0000","last_change":"2024-05-07T02:26:11.867258+0000","last_active":"2024-05-07T02:31:49.825780+0000","last_peered":"2024-05-07T02:31:49.825780+0000","last_clean":"2024-05-07T02:31:49.825780+0000","last_became_active":"2024-05-07T02:26:11.866837+0000","last_became_peered":"2024-05-07T02:26:11.866837+0000","last_unstale":"2024-05-07T02:31:49.825780+0000","last_undegraded":"2024-05-07T02:31:49.825780+0000","last_fullsized":"2024-05-07T02:31:49.825780+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:07:58.589457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825979+0000","last_change":"2024-05-07T02:26:11.888434+0000","last_active":"2024-05-07T02:31:49.825979+0000","last_peered":"2024-05-07T02:31:49.825979+0000","last_clean":"2024-05-07T02:31:49.825979+0000","last_became_active":"2024-05-07T02:26:11.887681+0000","last_became_peered":"2024-05-07T02:26:11.887681+0000","last_unstale":"2024-05-07T02:31:49.825979+0000","last_undegraded":"2024-05-07T02:31:49.825979+0000","last_fullsized":"2024-05-07T02:31:49.825979+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:38:58.896781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.999103+0000","last_change":"2024-05-07T02:26:11.875304+0000","last_active":"2024-05-07T02:31:49.999103+0000","last_peered":"2024-05-07T02:31:49.999103+0000","last_clean":"2024-05-07T02:31:49.999103+0000","last_became_active":"2024-05-07T02:26:11.874946+0000","last_became_peered":"2024-05-07T02:26:11.874946+0000","last_unstale":"2024-05-07T02:31:49.999103+0000","last_undegraded":"2024-05-07T02:31:49.999103+0000","last_fullsized":"2024-05-07T02:31:49.999103+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:56:04.093791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.824887+0000","last_change":"2024-05-07T02:26:11.879311+0000","last_active":"2024-05-07T02:31:49.824887+0000","last_peered":"2024-05-07T02:31:49.824887+0000","last_clean":"2024-05-07T02:31:49.824887+0000","last_became_active":"2024-05-07T02:26:11.877620+0000","last_became_peered":"2024-05-07T02:26:11.877620+0000","last_unstale":"2024-05-07T02:31:49.824887+0000","last_undegraded":"2024-05-07T02:31:49.824887+0000","last_fullsized":"2024-05-07T02:31:49.824887+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T08:41:40.525708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092753+0000","last_change":"2024-05-07T02:26:11.876726+0000","last_active":"2024-05-07T02:31:50.092753+0000","last_peered":"2024-05-07T02:31:50.092753+0000","last_clean":"2024-05-07T02:31:50.092753+0000","last_became_active":"2024-05-07T02:26:11.876234+0000","last_became_peered":"2024-05-07T02:26:11.876234+0000","last_unstale":"2024-05-07T02:31:50.092753+0000","last_undegraded":"2024-05-07T02:31:50.092753+0000","last_fullsized":"2024-05-07T02:31:50.092753+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:26:34.422258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.098131+0000","last_change":"2024-05-07T02:26:11.879159+0000","last_active":"2024-05-07T02:31:50.098131+0000","last_peered":"2024-05-07T02:31:50.098131+0000","last_clean":"2024-05-07T02:31:50.098131+0000","last_became_active":"2024-05-07T02:26:11.878769+0000","last_became_peered":"2024-05-07T02:26:11.878769+0000","last_unstale":"2024-05-07T02:31:50.098131+0000","last_undegraded":"2024-05-07T02:31:50.098131+0000","last_fullsized":"2024-05-07T02:31:50.098131+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:05:27.089475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092884+0000","last_change":"2024-05-07T02:27:32.590188+0000","last_active":"2024-05-07T02:31:50.092884+0000","last_peered":"2024-05-07T02:31:50.092884+0000","last_clean":"2024-05-07T02:31:50.092884+0000","last_became_active":"2024-05-07T02:27:32.589784+0000","last_became_peered":"2024-05-07T02:27:32.589784+0000","last_unstale":"2024-05-07T02:31:50.092884+0000","last_undegraded":"2024-05-07T02:31:50.092884+0000","last_fullsized":"2024-05-07T02:31:50.092884+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:32:40.488143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":607529,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2601,"num_read_kb":1425,"num_write":388,"num_write_kb":1481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2187264,"data_stored":1822587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":312,"ondisk_log_size":312,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710803,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1056768,"data_stored":525718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907034,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":33020,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712132,"statfs":{"total":95995035648,"available":95961223168,"internally_reserved":0,"allocated":1888256,"data_stored":1318608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070560,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28456,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716696,"statfs":{"total":95995035648,"available":95965896704,"internally_reserved":0,"allocated":1470464,"data_stored":922848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266791,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36760,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708392,"statfs":{"total":95995035648,"available":95957393408,"internally_reserved":0,"allocated":1454080,"data_stored":919018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863463020,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1667072,"data_stored":1122862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626545,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":521963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574992,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1126400,"data_stored":596487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738517,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1073152,"data_stored":522887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1146880,"data_stored":596566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182531,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32728,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712424,"statfs":{"total":95995035648,"available":95961522176,"internally_reserved":0,"allocated":1589248,"data_stored":1045564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986304,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1056768,"data_stored":526206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822777,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28784,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716368,"statfs":{"total":95995035648,"available":95965560832,"internally_reserved":0,"allocated":1810432,"data_stored":1249593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":76169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":76248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":134811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":394604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":207922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-07T02:31:56.621 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-07T02:31:56.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:56 smithi019 ceph-mon[29781]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:31:56.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.930+0000 7f05c6600640 1 -- 172.21.15.19:0/3417771107 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05c0104ea0 msgr2=0x7f05c01052b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:56.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.930+0000 7f05c6600640 1 --2- 172.21.15.19:0/3417771107 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05c0104ea0 0x7f05c01052b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f05a8009900 tx=0x7f05a802a3f0 comp rx=0 tx=0).stop 2024-05-07T02:31:56.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.930+0000 7f05c6600640 1 -- 172.21.15.19:0/3417771107 shutdown_connections 2024-05-07T02:31:56.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.930+0000 7f05c6600640 1 --2- 172.21.15.19:0/3417771107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05c0109ba0 0x7f05c010bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:56.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.930+0000 7f05c6600640 1 --2- 172.21.15.19:0/3417771107 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05c0104ea0 0x7f05c01052b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:56.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.930+0000 7f05c6600640 1 --2- 172.21.15.19:0/3417771107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05c0103c60 0x7f05c0104960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:56.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.930+0000 7f05c6600640 1 -- 172.21.15.19:0/3417771107 >> 172.21.15.19:0/3417771107 conn(0x7f05c00fee70 msgr2=0x7f05c0101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:56.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.930+0000 7f05c6600640 1 -- 172.21.15.19:0/3417771107 shutdown_connections 2024-05-07T02:31:56.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.930+0000 7f05c6600640 1 -- 172.21.15.19:0/3417771107 wait complete. 2024-05-07T02:31:56.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.931+0000 7f05c6600640 1 Processor -- start 2024-05-07T02:31:56.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.931+0000 7f05c6600640 1 -- start start 2024-05-07T02:31:56.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.932+0000 7f05c6600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05c0103c60 0x7f05c0077a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:56.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.932+0000 7f05c6600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05c0104ea0 0x7f05c0076130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:56.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.932+0000 7f05c6600640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05c0109ba0 0x7f05c0076670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:56.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.932+0000 7f05c6600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f05c010f7b0 con 0x7f05c0104ea0 2024-05-07T02:31:56.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.932+0000 7f05c6600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f05c010f630 con 0x7f05c0103c60 2024-05-07T02:31:56.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.932+0000 7f05c6600640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f05c010f930 con 0x7f05c0109ba0 2024-05-07T02:31:56.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.932+0000 7f05b7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05c0104ea0 0x7f05c0076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:56.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.932+0000 7f05b7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05c0104ea0 0x7f05c0076130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50860/0 (socket says 172.21.15.19:50860) 2024-05-07T02:31:56.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.932+0000 7f05b7e00640 1 -- 172.21.15.19:0/3159354536 learned_addr learned my addr 172.21.15.19:0/3159354536 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:31:56.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.932+0000 7f05bf400640 1 --2- 172.21.15.19:0/3159354536 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05c0103c60 0x7f05c0077a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:56.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.932+0000 7f05bfe00640 1 --2- 172.21.15.19:0/3159354536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05c0109ba0 0x7f05c0076670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:56.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.933+0000 7f05b7e00640 1 -- 172.21.15.19:0/3159354536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05c0109ba0 msgr2=0x7f05c0076670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:56.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.933+0000 7f05b7e00640 1 --2- 172.21.15.19:0/3159354536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05c0109ba0 0x7f05c0076670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:56.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.933+0000 7f05b7e00640 1 -- 172.21.15.19:0/3159354536 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05c0103c60 msgr2=0x7f05c0077a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:56.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.933+0000 7f05b7e00640 1 --2- 172.21.15.19:0/3159354536 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05c0103c60 0x7f05c0077a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:56.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.933+0000 7f05b7e00640 1 -- 172.21.15.19:0/3159354536 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c0076da0 con 0x7f05c0104ea0 2024-05-07T02:31:56.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.933+0000 7f05bf400640 1 --2- 172.21.15.19:0/3159354536 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05c0103c60 0x7f05c0077a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:31:56.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.933+0000 7f05b7e00640 1 --2- 172.21.15.19:0/3159354536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05c0104ea0 0x7f05c0076130 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f05a8031440 tx=0x7f05a8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:56.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.933+0000 7f05bcc00640 1 -- 172.21.15.19:0/3159354536 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a803f070 con 0x7f05c0104ea0 2024-05-07T02:31:56.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.934+0000 7f05c6600640 1 -- 172.21.15.19:0/3159354536 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05c01a4350 con 0x7f05c0104ea0 2024-05-07T02:31:56.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.934+0000 7f05bcc00640 1 -- 172.21.15.19:0/3159354536 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05a8031c50 con 0x7f05c0104ea0 2024-05-07T02:31:56.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.934+0000 7f05bcc00640 1 -- 172.21.15.19:0/3159354536 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a8035030 con 0x7f05c0104ea0 2024-05-07T02:31:56.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.934+0000 7f05c6600640 1 -- 172.21.15.19:0/3159354536 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05c01a4750 con 0x7f05c0104ea0 2024-05-07T02:31:56.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.935+0000 7f05c6600640 1 -- 172.21.15.19:0/3159354536 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05c006a340 con 0x7f05c0104ea0 2024-05-07T02:31:56.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.938+0000 7f05bcc00640 1 -- 172.21.15.19:0/3159354536 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f05a8042070 con 0x7f05c0104ea0 2024-05-07T02:31:56.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.940+0000 7f05bcc00640 1 --2- 172.21.15.19:0/3159354536 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f05a40b2a90 0x7f05a40b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:56.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.940+0000 7f05bcc00640 1 -- 172.21.15.19:0/3159354536 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f05a81032c0 con 0x7f05c0104ea0 2024-05-07T02:31:56.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.941+0000 7f05bf400640 1 --2- 172.21.15.19:0/3159354536 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f05a40b2a90 0x7f05a40b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:56.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.941+0000 7f05bf400640 1 --2- 172.21.15.19:0/3159354536 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f05a40b2a90 0x7f05a40b4fa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f05c0104860 tx=0x7f05b0005f90 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:56.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:56.944+0000 7f05bcc00640 1 -- 172.21.15.19:0/3159354536 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f05a80ca150 con 0x7f05c0104ea0 2024-05-07T02:31:56.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:56 smithi027 ceph-mon[34398]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:31:57.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:56 smithi064 ceph-mon[34699]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:31:57.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.157+0000 7f05c6600640 1 -- 172.21.15.19:0/3159354536 --> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f05c002e010 con 0x7f05a40b2a90 2024-05-07T02:31:57.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.175+0000 7f05bcc00640 1 -- 172.21.15.19:0/3159354536 <== mgr.25234 v2:172.21.15.19:6800/3265825478 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339447 (secure 0 0 0) 0x7f05c002e010 con 0x7f05a40b2a90 2024-05-07T02:31:57.180 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:31:57.182 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-05-07T02:31:57.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.183+0000 7f05c6600640 1 -- 172.21.15.19:0/3159354536 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f05a40b2a90 msgr2=0x7f05a40b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:57.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.183+0000 7f05c6600640 1 --2- 172.21.15.19:0/3159354536 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f05a40b2a90 0x7f05a40b4fa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f05c0104860 tx=0x7f05b0005f90 comp rx=0 tx=0).stop 2024-05-07T02:31:57.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.183+0000 7f05c6600640 1 -- 172.21.15.19:0/3159354536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05c0104ea0 msgr2=0x7f05c0076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:57.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.183+0000 7f05c6600640 1 --2- 172.21.15.19:0/3159354536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05c0104ea0 0x7f05c0076130 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f05a8031440 tx=0x7f05a8003c10 comp rx=0 tx=0).stop 2024-05-07T02:31:57.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.183+0000 7f05c6600640 1 -- 172.21.15.19:0/3159354536 shutdown_connections 2024-05-07T02:31:57.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.183+0000 7f05c6600640 1 --2- 172.21.15.19:0/3159354536 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f05a40b2a90 0x7f05a40b4fa0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:57.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.183+0000 7f05c6600640 1 --2- 172.21.15.19:0/3159354536 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05c0109ba0 0x7f05c0076670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:57.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.183+0000 7f05c6600640 1 --2- 172.21.15.19:0/3159354536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05c0104ea0 0x7f05c0076130 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:57.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.183+0000 7f05c6600640 1 --2- 172.21.15.19:0/3159354536 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05c0103c60 0x7f05c0077a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:57.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.183+0000 7f05c6600640 1 -- 172.21.15.19:0/3159354536 >> 172.21.15.19:0/3159354536 conn(0x7f05c00fee70 msgr2=0x7f05c0105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:57.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.183+0000 7f05c6600640 1 -- 172.21.15.19:0/3159354536 shutdown_connections 2024-05-07T02:31:57.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.184+0000 7f05c6600640 1 -- 172.21.15.19:0/3159354536 wait complete. 2024-05-07T02:31:57.210 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-05-07T02:31:55.850763+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1198131,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2682,"num_read_kb":1494,"num_write":513,"num_write_kb":3633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":906,"ondisk_log_size":906,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":377672,"kb_used_data":16004,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124564152,"statfs":{"total":1151940427776,"available":1151553691648,"internally_reserved":0,"allocated":16388096,"data_stored":9868320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.978312"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.093082+0000","last_change":"2024-05-07T02:26:11.876828+0000","last_active":"2024-05-07T02:31:50.093082+0000","last_peered":"2024-05-07T02:31:50.093082+0000","last_clean":"2024-05-07T02:31:50.093082+0000","last_became_active":"2024-05-07T02:26:11.876319+0000","last_became_peered":"2024-05-07T02:26:11.876319+0000","last_unstale":"2024-05-07T02:31:50.093082+0000","last_undegraded":"2024-05-07T02:31:50.093082+0000","last_fullsized":"2024-05-07T02:31:50.093082+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:07:33.181373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.094443+0000","last_change":"2024-05-07T02:26:11.873388+0000","last_active":"2024-05-07T02:31:50.094443+0000","last_peered":"2024-05-07T02:31:50.094443+0000","last_clean":"2024-05-07T02:31:50.094443+0000","last_became_active":"2024-05-07T02:26:11.869447+0000","last_became_peered":"2024-05-07T02:26:11.869447+0000","last_unstale":"2024-05-07T02:31:50.094443+0000","last_undegraded":"2024-05-07T02:31:50.094443+0000","last_fullsized":"2024-05-07T02:31:50.094443+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:09:52.876547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.094158+0000","last_change":"2024-05-07T02:26:11.870797+0000","last_active":"2024-05-07T02:31:50.094158+0000","last_peered":"2024-05-07T02:31:50.094158+0000","last_clean":"2024-05-07T02:31:50.094158+0000","last_became_active":"2024-05-07T02:26:11.870411+0000","last_became_peered":"2024-05-07T02:26:11.870411+0000","last_unstale":"2024-05-07T02:31:50.094158+0000","last_undegraded":"2024-05-07T02:31:50.094158+0000","last_fullsized":"2024-05-07T02:31:50.094158+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:34:12.017508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.826540+0000","last_change":"2024-05-07T02:26:11.867327+0000","last_active":"2024-05-07T02:31:49.826540+0000","last_peered":"2024-05-07T02:31:49.826540+0000","last_clean":"2024-05-07T02:31:49.826540+0000","last_became_active":"2024-05-07T02:26:11.866912+0000","last_became_peered":"2024-05-07T02:26:11.866912+0000","last_unstale":"2024-05-07T02:31:49.826540+0000","last_undegraded":"2024-05-07T02:31:49.826540+0000","last_fullsized":"2024-05-07T02:31:49.826540+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:36:36.616706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.988361+0000","last_change":"2024-05-07T02:26:11.878614+0000","last_active":"2024-05-07T02:31:49.988361+0000","last_peered":"2024-05-07T02:31:49.988361+0000","last_clean":"2024-05-07T02:31:49.988361+0000","last_became_active":"2024-05-07T02:26:11.878069+0000","last_became_peered":"2024-05-07T02:26:11.878069+0000","last_unstale":"2024-05-07T02:31:49.988361+0000","last_undegraded":"2024-05-07T02:31:49.988361+0000","last_fullsized":"2024-05-07T02:31:49.988361+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:40:15.991433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.003199+0000","last_change":"2024-05-07T02:26:11.877755+0000","last_active":"2024-05-07T02:31:50.003199+0000","last_peered":"2024-05-07T02:31:50.003199+0000","last_clean":"2024-05-07T02:31:50.003199+0000","last_became_active":"2024-05-07T02:26:11.877373+0000","last_became_peered":"2024-05-07T02:26:11.877373+0000","last_unstale":"2024-05-07T02:31:50.003199+0000","last_undegraded":"2024-05-07T02:31:50.003199+0000","last_fullsized":"2024-05-07T02:31:50.003199+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:49:55.984807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.136082+0000","last_change":"2024-05-07T02:26:11.882194+0000","last_active":"2024-05-07T02:31:50.136082+0000","last_peered":"2024-05-07T02:31:50.136082+0000","last_clean":"2024-05-07T02:31:50.136082+0000","last_became_active":"2024-05-07T02:26:11.881873+0000","last_became_peered":"2024-05-07T02:26:11.881873+0000","last_unstale":"2024-05-07T02:31:50.136082+0000","last_undegraded":"2024-05-07T02:31:50.136082+0000","last_fullsized":"2024-05-07T02:31:50.136082+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:47:31.000357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.823872+0000","last_change":"2024-05-07T02:26:11.857366+0000","last_active":"2024-05-07T02:31:49.823872+0000","last_peered":"2024-05-07T02:31:49.823872+0000","last_clean":"2024-05-07T02:31:49.823872+0000","last_became_active":"2024-05-07T02:26:11.854054+0000","last_became_peered":"2024-05-07T02:26:11.854054+0000","last_unstale":"2024-05-07T02:31:49.823872+0000","last_undegraded":"2024-05-07T02:31:49.823872+0000","last_fullsized":"2024-05-07T02:31:49.823872+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:07:04.834022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092885+0000","last_change":"2024-05-07T02:26:11.867105+0000","last_active":"2024-05-07T02:31:50.092885+0000","last_peered":"2024-05-07T02:31:50.092885+0000","last_clean":"2024-05-07T02:31:50.092885+0000","last_became_active":"2024-05-07T02:26:11.865986+0000","last_became_peered":"2024-05-07T02:26:11.865986+0000","last_unstale":"2024-05-07T02:31:50.092885+0000","last_undegraded":"2024-05-07T02:31:50.092885+0000","last_fullsized":"2024-05-07T02:31:50.092885+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:13:01.295354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.003769+0000","last_change":"2024-05-07T02:26:11.868039+0000","last_active":"2024-05-07T02:31:50.003769+0000","last_peered":"2024-05-07T02:31:50.003769+0000","last_clean":"2024-05-07T02:31:50.003769+0000","last_became_active":"2024-05-07T02:26:11.867588+0000","last_became_peered":"2024-05-07T02:26:11.867588+0000","last_unstale":"2024-05-07T02:31:50.003769+0000","last_undegraded":"2024-05-07T02:31:50.003769+0000","last_fullsized":"2024-05-07T02:31:50.003769+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:41:45.149620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.136641+0000","last_change":"2024-05-07T02:26:11.859579+0000","last_active":"2024-05-07T02:31:50.136641+0000","last_peered":"2024-05-07T02:31:50.136641+0000","last_clean":"2024-05-07T02:31:50.136641+0000","last_became_active":"2024-05-07T02:26:11.859154+0000","last_became_peered":"2024-05-07T02:26:11.859154+0000","last_unstale":"2024-05-07T02:31:50.136641+0000","last_undegraded":"2024-05-07T02:31:50.136641+0000","last_fullsized":"2024-05-07T02:31:50.136641+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:26:40.616956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825795+0000","last_change":"2024-05-07T02:26:11.858220+0000","last_active":"2024-05-07T02:31:49.825795+0000","last_peered":"2024-05-07T02:31:49.825795+0000","last_clean":"2024-05-07T02:31:49.825795+0000","last_became_active":"2024-05-07T02:26:11.857826+0000","last_became_peered":"2024-05-07T02:26:11.857826+0000","last_unstale":"2024-05-07T02:31:49.825795+0000","last_undegraded":"2024-05-07T02:31:49.825795+0000","last_fullsized":"2024-05-07T02:31:49.825795+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:21:36.031057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.097826+0000","last_change":"2024-05-07T02:26:11.880348+0000","last_active":"2024-05-07T02:31:50.097826+0000","last_peered":"2024-05-07T02:31:50.097826+0000","last_clean":"2024-05-07T02:31:50.097826+0000","last_became_active":"2024-05-07T02:26:11.879633+0000","last_became_peered":"2024-05-07T02:26:11.879633+0000","last_unstale":"2024-05-07T02:31:50.097826+0000","last_undegraded":"2024-05-07T02:31:50.097826+0000","last_fullsized":"2024-05-07T02:31:50.097826+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:59:34.567973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.093652+0000","last_change":"2024-05-07T02:26:11.865687+0000","last_active":"2024-05-07T02:31:50.093652+0000","last_peered":"2024-05-07T02:31:50.093652+0000","last_clean":"2024-05-07T02:31:50.093652+0000","last_became_active":"2024-05-07T02:26:11.865330+0000","last_became_peered":"2024-05-07T02:26:11.865330+0000","last_unstale":"2024-05-07T02:31:50.093652+0000","last_undegraded":"2024-05-07T02:31:50.093652+0000","last_fullsized":"2024-05-07T02:31:50.093652+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:36:10.610314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.093991+0000","last_change":"2024-05-07T02:26:11.870627+0000","last_active":"2024-05-07T02:31:50.093991+0000","last_peered":"2024-05-07T02:31:50.093991+0000","last_clean":"2024-05-07T02:31:50.093991+0000","last_became_active":"2024-05-07T02:26:11.870229+0000","last_became_peered":"2024-05-07T02:26:11.870229+0000","last_unstale":"2024-05-07T02:31:50.093991+0000","last_undegraded":"2024-05-07T02:31:50.093991+0000","last_fullsized":"2024-05-07T02:31:50.093991+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:06:46.008651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092311+0000","last_change":"2024-05-07T02:26:11.868698+0000","last_active":"2024-05-07T02:31:50.092311+0000","last_peered":"2024-05-07T02:31:50.092311+0000","last_clean":"2024-05-07T02:31:50.092311+0000","last_became_active":"2024-05-07T02:26:11.868294+0000","last_became_peered":"2024-05-07T02:26:11.868294+0000","last_unstale":"2024-05-07T02:31:50.092311+0000","last_undegraded":"2024-05-07T02:31:50.092311+0000","last_fullsized":"2024-05-07T02:31:50.092311+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:16:22.856200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092865+0000","last_change":"2024-05-07T02:26:11.877116+0000","last_active":"2024-05-07T02:31:50.092865+0000","last_peered":"2024-05-07T02:31:50.092865+0000","last_clean":"2024-05-07T02:31:50.092865+0000","last_became_active":"2024-05-07T02:26:11.876305+0000","last_became_peered":"2024-05-07T02:26:11.876305+0000","last_unstale":"2024-05-07T02:31:50.092865+0000","last_undegraded":"2024-05-07T02:31:50.092865+0000","last_fullsized":"2024-05-07T02:31:50.092865+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:25:46.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.985509+0000","last_change":"2024-05-07T02:26:11.877488+0000","last_active":"2024-05-07T02:31:49.985509+0000","last_peered":"2024-05-07T02:31:49.985509+0000","last_clean":"2024-05-07T02:31:49.985509+0000","last_became_active":"2024-05-07T02:26:11.874500+0000","last_became_peered":"2024-05-07T02:26:11.874500+0000","last_unstale":"2024-05-07T02:31:49.985509+0000","last_undegraded":"2024-05-07T02:31:49.985509+0000","last_fullsized":"2024-05-07T02:31:49.985509+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:53:50.046342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825931+0000","last_change":"2024-05-07T02:26:11.876500+0000","last_active":"2024-05-07T02:31:49.825931+0000","last_peered":"2024-05-07T02:31:49.825931+0000","last_clean":"2024-05-07T02:31:49.825931+0000","last_became_active":"2024-05-07T02:26:11.876084+0000","last_became_peered":"2024-05-07T02:26:11.876084+0000","last_unstale":"2024-05-07T02:31:49.825931+0000","last_undegraded":"2024-05-07T02:31:49.825931+0000","last_fullsized":"2024-05-07T02:31:49.825931+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:05:05.681488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825186+0000","last_change":"2024-05-07T02:26:25.579209+0000","last_active":"2024-05-07T02:31:49.825186+0000","last_peered":"2024-05-07T02:31:49.825186+0000","last_clean":"2024-05-07T02:31:49.825186+0000","last_became_active":"2024-05-07T02:26:25.578665+0000","last_became_peered":"2024-05-07T02:26:25.578665+0000","last_unstale":"2024-05-07T02:31:49.825186+0000","last_undegraded":"2024-05-07T02:31:49.825186+0000","last_fullsized":"2024-05-07T02:31:49.825186+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:06:29.784016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'15","reported_seq":310,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.215535+0000","last_change":"2024-05-07T02:26:07.792993+0000","last_active":"2024-05-07T02:31:50.215535+0000","last_peered":"2024-05-07T02:31:50.215535+0000","last_clean":"2024-05-07T02:31:50.215535+0000","last_became_active":"2024-05-07T02:26:07.792402+0000","last_became_peered":"2024-05-07T02:26:07.792402+0000","last_unstale":"2024-05-07T02:31:50.215535+0000","last_undegraded":"2024-05-07T02:31:50.215535+0000","last_fullsized":"2024-05-07T02:31:50.215535+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:49:02.288255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":247,"num_read_kb":247,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.985441+0000","last_change":"2024-05-07T02:28:33.869804+0000","last_active":"2024-05-07T02:31:49.985441+0000","last_peered":"2024-05-07T02:31:49.985441+0000","last_clean":"2024-05-07T02:31:49.985441+0000","last_became_active":"2024-05-07T02:28:33.869267+0000","last_became_peered":"2024-05-07T02:28:33.869267+0000","last_unstale":"2024-05-07T02:31:49.985441+0000","last_undegraded":"2024-05-07T02:31:49.985441+0000","last_fullsized":"2024-05-07T02:31:49.985441+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:10:57.384546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825312+0000","last_change":"2024-05-07T02:26:25.579835+0000","last_active":"2024-05-07T02:31:49.825312+0000","last_peered":"2024-05-07T02:31:49.825312+0000","last_clean":"2024-05-07T02:31:49.825312+0000","last_became_active":"2024-05-07T02:26:25.579105+0000","last_became_peered":"2024-05-07T02:26:25.579105+0000","last_unstale":"2024-05-07T02:31:49.825312+0000","last_undegraded":"2024-05-07T02:31:49.825312+0000","last_fullsized":"2024-05-07T02:31:49.825312+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:32:37.745967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825500+0000","last_change":"2024-05-07T02:26:08.001879+0000","last_active":"2024-05-07T02:31:49.825500+0000","last_peered":"2024-05-07T02:31:49.825500+0000","last_clean":"2024-05-07T02:31:49.825500+0000","last_became_active":"2024-05-07T02:26:08.000244+0000","last_became_peered":"2024-05-07T02:26:08.000244+0000","last_unstale":"2024-05-07T02:31:49.825500+0000","last_undegraded":"2024-05-07T02:31:49.825500+0000","last_fullsized":"2024-05-07T02:31:49.825500+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:56:42.591923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.094664+0000","last_change":"2024-05-07T02:26:11.882351+0000","last_active":"2024-05-07T02:31:50.094664+0000","last_peered":"2024-05-07T02:31:50.094664+0000","last_clean":"2024-05-07T02:31:50.094664+0000","last_became_active":"2024-05-07T02:26:11.881922+0000","last_became_peered":"2024-05-07T02:26:11.881922+0000","last_unstale":"2024-05-07T02:31:50.094664+0000","last_undegraded":"2024-05-07T02:31:50.094664+0000","last_fullsized":"2024-05-07T02:31:50.094664+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:58:31.993859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825489+0000","last_change":"2024-05-07T02:26:25.580419+0000","last_active":"2024-05-07T02:31:49.825489+0000","last_peered":"2024-05-07T02:31:49.825489+0000","last_clean":"2024-05-07T02:31:49.825489+0000","last_became_active":"2024-05-07T02:26:25.579411+0000","last_became_peered":"2024-05-07T02:26:25.579411+0000","last_unstale":"2024-05-07T02:31:49.825489+0000","last_undegraded":"2024-05-07T02:31:49.825489+0000","last_fullsized":"2024-05-07T02:31:49.825489+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:43:03.552210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092822+0000","last_change":"2024-05-07T02:27:32.588255+0000","last_active":"2024-05-07T02:31:50.092822+0000","last_peered":"2024-05-07T02:31:50.092822+0000","last_clean":"2024-05-07T02:31:50.092822+0000","last_became_active":"2024-05-07T02:27:32.584405+0000","last_became_peered":"2024-05-07T02:27:32.584405+0000","last_unstale":"2024-05-07T02:31:50.092822+0000","last_undegraded":"2024-05-07T02:31:50.092822+0000","last_fullsized":"2024-05-07T02:31:50.092822+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:07:22.980041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092515+0000","last_change":"2024-05-07T02:26:11.866949+0000","last_active":"2024-05-07T02:31:50.092515+0000","last_peered":"2024-05-07T02:31:50.092515+0000","last_clean":"2024-05-07T02:31:50.092515+0000","last_became_active":"2024-05-07T02:26:11.866453+0000","last_became_peered":"2024-05-07T02:26:11.866453+0000","last_unstale":"2024-05-07T02:31:50.092515+0000","last_undegraded":"2024-05-07T02:31:50.092515+0000","last_fullsized":"2024-05-07T02:31:50.092515+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T14:22:47.291470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825287+0000","last_change":"2024-05-07T02:26:25.570555+0000","last_active":"2024-05-07T02:31:49.825287+0000","last_peered":"2024-05-07T02:31:49.825287+0000","last_clean":"2024-05-07T02:31:49.825287+0000","last_became_active":"2024-05-07T02:26:25.570044+0000","last_became_peered":"2024-05-07T02:26:25.570044+0000","last_unstale":"2024-05-07T02:31:49.825287+0000","last_undegraded":"2024-05-07T02:31:49.825287+0000","last_fullsized":"2024-05-07T02:31:49.825287+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:48:15.701335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.988509+0000","last_change":"2024-05-07T02:26:07.785938+0000","last_active":"2024-05-07T02:31:49.988509+0000","last_peered":"2024-05-07T02:31:49.988509+0000","last_clean":"2024-05-07T02:31:49.988509+0000","last_became_active":"2024-05-07T02:26:07.785573+0000","last_became_peered":"2024-05-07T02:26:07.785573+0000","last_unstale":"2024-05-07T02:31:49.988509+0000","last_undegraded":"2024-05-07T02:31:49.988509+0000","last_fullsized":"2024-05-07T02:31:49.988509+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:33:24.042227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.999531+0000","last_change":"2024-05-07T02:26:11.858732+0000","last_active":"2024-05-07T02:31:49.999531+0000","last_peered":"2024-05-07T02:31:49.999531+0000","last_clean":"2024-05-07T02:31:49.999531+0000","last_became_active":"2024-05-07T02:26:11.858270+0000","last_became_peered":"2024-05-07T02:26:11.858270+0000","last_unstale":"2024-05-07T02:31:49.999531+0000","last_undegraded":"2024-05-07T02:31:49.999531+0000","last_fullsized":"2024-05-07T02:31:49.999531+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:03:53.662296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825701+0000","last_change":"2024-05-07T02:26:25.559025+0000","last_active":"2024-05-07T02:31:49.825701+0000","last_peered":"2024-05-07T02:31:49.825701+0000","last_clean":"2024-05-07T02:31:49.825701+0000","last_became_active":"2024-05-07T02:26:25.558502+0000","last_became_peered":"2024-05-07T02:26:25.558502+0000","last_unstale":"2024-05-07T02:31:49.825701+0000","last_undegraded":"2024-05-07T02:31:49.825701+0000","last_fullsized":"2024-05-07T02:31:49.825701+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:26:59.846341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":302,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.147359+0000","last_change":"2024-05-07T02:26:07.783386+0000","last_active":"2024-05-07T02:31:50.147359+0000","last_peered":"2024-05-07T02:31:50.147359+0000","last_clean":"2024-05-07T02:31:50.147359+0000","last_became_active":"2024-05-07T02:26:07.782823+0000","last_became_peered":"2024-05-07T02:26:07.782823+0000","last_unstale":"2024-05-07T02:31:50.147359+0000","last_undegraded":"2024-05-07T02:31:50.147359+0000","last_fullsized":"2024-05-07T02:31:50.147359+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:40:29.350809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":248,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.988372+0000","last_change":"2024-05-07T02:26:11.877426+0000","last_active":"2024-05-07T02:31:49.988372+0000","last_peered":"2024-05-07T02:31:49.988372+0000","last_clean":"2024-05-07T02:31:49.988372+0000","last_became_active":"2024-05-07T02:26:11.876976+0000","last_became_peered":"2024-05-07T02:26:11.876976+0000","last_unstale":"2024-05-07T02:31:49.988372+0000","last_undegraded":"2024-05-07T02:31:49.988372+0000","last_fullsized":"2024-05-07T02:31:49.988372+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:18:56.077064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825760+0000","last_change":"2024-05-07T02:27:32.625394+0000","last_active":"2024-05-07T02:31:49.825760+0000","last_peered":"2024-05-07T02:31:49.825760+0000","last_clean":"2024-05-07T02:31:49.825760+0000","last_became_active":"2024-05-07T02:27:32.624826+0000","last_became_peered":"2024-05-07T02:27:32.624826+0000","last_unstale":"2024-05-07T02:31:49.825760+0000","last_undegraded":"2024-05-07T02:31:49.825760+0000","last_fullsized":"2024-05-07T02:31:49.825760+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:16:49.694003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.003155+0000","last_change":"2024-05-07T02:26:07.786388+0000","last_active":"2024-05-07T02:31:50.003155+0000","last_peered":"2024-05-07T02:31:50.003155+0000","last_clean":"2024-05-07T02:31:50.003155+0000","last_became_active":"2024-05-07T02:26:07.784130+0000","last_became_peered":"2024-05-07T02:26:07.784130+0000","last_unstale":"2024-05-07T02:31:50.003155+0000","last_undegraded":"2024-05-07T02:31:50.003155+0000","last_fullsized":"2024-05-07T02:31:50.003155+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:44:55.229815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.999374+0000","last_change":"2024-05-07T02:28:33.886551+0000","last_active":"2024-05-07T02:31:49.999374+0000","last_peered":"2024-05-07T02:31:49.999374+0000","last_clean":"2024-05-07T02:31:49.999374+0000","last_became_active":"2024-05-07T02:28:33.886249+0000","last_became_peered":"2024-05-07T02:28:33.886249+0000","last_unstale":"2024-05-07T02:31:49.999374+0000","last_undegraded":"2024-05-07T02:31:49.999374+0000","last_fullsized":"2024-05-07T02:31:49.999374+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:56:19.851347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092828+0000","last_change":"2024-05-07T02:26:25.584396+0000","last_active":"2024-05-07T02:31:50.092828+0000","last_peered":"2024-05-07T02:31:50.092828+0000","last_clean":"2024-05-07T02:31:50.092828+0000","last_became_active":"2024-05-07T02:26:25.583917+0000","last_became_peered":"2024-05-07T02:26:25.583917+0000","last_unstale":"2024-05-07T02:31:50.092828+0000","last_undegraded":"2024-05-07T02:31:50.092828+0000","last_fullsized":"2024-05-07T02:31:50.092828+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:56:34.495133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.097562+0000","last_change":"2024-05-07T02:26:07.795300+0000","last_active":"2024-05-07T02:31:50.097562+0000","last_peered":"2024-05-07T02:31:50.097562+0000","last_clean":"2024-05-07T02:31:50.097562+0000","last_became_active":"2024-05-07T02:26:07.794633+0000","last_became_peered":"2024-05-07T02:26:07.794633+0000","last_unstale":"2024-05-07T02:31:50.097562+0000","last_undegraded":"2024-05-07T02:31:50.097562+0000","last_fullsized":"2024-05-07T02:31:50.097562+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:36:44.252118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.824253+0000","last_change":"2024-05-07T02:26:11.877717+0000","last_active":"2024-05-07T02:31:49.824253+0000","last_peered":"2024-05-07T02:31:49.824253+0000","last_clean":"2024-05-07T02:31:49.824253+0000","last_became_active":"2024-05-07T02:26:11.876644+0000","last_became_peered":"2024-05-07T02:26:11.876644+0000","last_unstale":"2024-05-07T02:31:49.824253+0000","last_undegraded":"2024-05-07T02:31:49.824253+0000","last_fullsized":"2024-05-07T02:31:49.824253+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T05:20:36.176776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.094315+0000","last_change":"2024-05-07T02:26:25.572120+0000","last_active":"2024-05-07T02:31:50.094315+0000","last_peered":"2024-05-07T02:31:50.094315+0000","last_clean":"2024-05-07T02:31:50.094315+0000","last_became_active":"2024-05-07T02:26:25.571637+0000","last_became_peered":"2024-05-07T02:26:25.571637+0000","last_unstale":"2024-05-07T02:31:50.094315+0000","last_undegraded":"2024-05-07T02:31:50.094315+0000","last_fullsized":"2024-05-07T02:31:50.094315+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:03:54.574061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.999043+0000","last_change":"2024-05-07T02:26:07.784160+0000","last_active":"2024-05-07T02:31:49.999043+0000","last_peered":"2024-05-07T02:31:49.999043+0000","last_clean":"2024-05-07T02:31:49.999043+0000","last_became_active":"2024-05-07T02:26:07.783646+0000","last_became_peered":"2024-05-07T02:26:07.783646+0000","last_unstale":"2024-05-07T02:31:49.999043+0000","last_undegraded":"2024-05-07T02:31:49.999043+0000","last_fullsized":"2024-05-07T02:31:49.999043+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T14:25:37.906587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.984924+0000","last_change":"2024-05-07T02:26:11.874886+0000","last_active":"2024-05-07T02:31:49.984924+0000","last_peered":"2024-05-07T02:31:49.984924+0000","last_clean":"2024-05-07T02:31:49.984924+0000","last_became_active":"2024-05-07T02:26:11.874465+0000","last_became_peered":"2024-05-07T02:26:11.874465+0000","last_unstale":"2024-05-07T02:31:49.984924+0000","last_undegraded":"2024-05-07T02:31:49.984924+0000","last_fullsized":"2024-05-07T02:31:49.984924+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:57:19.530747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.824559+0000","last_change":"2024-05-07T02:26:25.558087+0000","last_active":"2024-05-07T02:31:49.824559+0000","last_peered":"2024-05-07T02:31:49.824559+0000","last_clean":"2024-05-07T02:31:49.824559+0000","last_became_active":"2024-05-07T02:26:25.557624+0000","last_became_peered":"2024-05-07T02:26:25.557624+0000","last_unstale":"2024-05-07T02:31:49.824559+0000","last_undegraded":"2024-05-07T02:31:49.824559+0000","last_fullsized":"2024-05-07T02:31:49.824559+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:03:20.104547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'6","reported_seq":546,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.219941+0000","last_change":"2024-05-07T02:26:07.788244+0000","last_active":"2024-05-07T02:31:50.219941+0000","last_peered":"2024-05-07T02:31:50.219941+0000","last_clean":"2024-05-07T02:31:50.219941+0000","last_became_active":"2024-05-07T02:26:07.787789+0000","last_became_peered":"2024-05-07T02:26:07.787789+0000","last_unstale":"2024-05-07T02:31:50.219941+0000","last_undegraded":"2024-05-07T02:31:50.219941+0000","last_fullsized":"2024-05-07T02:31:50.219941+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:12:22.943444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":492,"num_read_kb":492,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092921+0000","last_change":"2024-05-07T02:26:11.879821+0000","last_active":"2024-05-07T02:31:50.092921+0000","last_peered":"2024-05-07T02:31:50.092921+0000","last_clean":"2024-05-07T02:31:50.092921+0000","last_became_active":"2024-05-07T02:26:11.879372+0000","last_became_peered":"2024-05-07T02:26:11.879372+0000","last_unstale":"2024-05-07T02:31:50.092921+0000","last_undegraded":"2024-05-07T02:31:50.092921+0000","last_fullsized":"2024-05-07T02:31:50.092921+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:01:03.754890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.985163+0000","last_change":"2024-05-07T02:26:25.576663+0000","last_active":"2024-05-07T02:31:49.985163+0000","last_peered":"2024-05-07T02:31:49.985163+0000","last_clean":"2024-05-07T02:31:49.985163+0000","last_became_active":"2024-05-07T02:26:25.575729+0000","last_became_peered":"2024-05-07T02:26:25.575729+0000","last_unstale":"2024-05-07T02:31:49.985163+0000","last_undegraded":"2024-05-07T02:31:49.985163+0000","last_fullsized":"2024-05-07T02:31:49.985163+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:03:48.061385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.136515+0000","last_change":"2024-05-07T02:27:32.625336+0000","last_active":"2024-05-07T02:31:50.136515+0000","last_peered":"2024-05-07T02:31:50.136515+0000","last_clean":"2024-05-07T02:31:50.136515+0000","last_became_active":"2024-05-07T02:27:32.624805+0000","last_became_peered":"2024-05-07T02:27:32.624805+0000","last_unstale":"2024-05-07T02:31:50.136515+0000","last_undegraded":"2024-05-07T02:31:50.136515+0000","last_fullsized":"2024-05-07T02:31:50.136515+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:10:04.928297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.094830+0000","last_change":"2024-05-07T02:27:32.626374+0000","last_active":"2024-05-07T02:31:50.094830+0000","last_peered":"2024-05-07T02:31:50.094830+0000","last_clean":"2024-05-07T02:31:50.094830+0000","last_became_active":"2024-05-07T02:27:32.625489+0000","last_became_peered":"2024-05-07T02:27:32.625489+0000","last_unstale":"2024-05-07T02:31:50.094830+0000","last_undegraded":"2024-05-07T02:31:50.094830+0000","last_fullsized":"2024-05-07T02:31:50.094830+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T05:56:16.626041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825653+0000","last_change":"2024-05-07T02:26:25.580811+0000","last_active":"2024-05-07T02:31:49.825653+0000","last_peered":"2024-05-07T02:31:49.825653+0000","last_clean":"2024-05-07T02:31:49.825653+0000","last_became_active":"2024-05-07T02:26:25.579344+0000","last_became_peered":"2024-05-07T02:26:25.579344+0000","last_unstale":"2024-05-07T02:31:49.825653+0000","last_undegraded":"2024-05-07T02:31:49.825653+0000","last_fullsized":"2024-05-07T02:31:49.825653+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:25:36.186899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.999415+0000","last_change":"2024-05-07T02:26:08.000010+0000","last_active":"2024-05-07T02:31:49.999415+0000","last_peered":"2024-05-07T02:31:49.999415+0000","last_clean":"2024-05-07T02:31:49.999415+0000","last_became_active":"2024-05-07T02:26:07.999576+0000","last_became_peered":"2024-05-07T02:26:07.999576+0000","last_unstale":"2024-05-07T02:31:49.999415+0000","last_undegraded":"2024-05-07T02:31:49.999415+0000","last_fullsized":"2024-05-07T02:31:49.999415+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:23:38.644205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092735+0000","last_change":"2024-05-07T02:26:11.867098+0000","last_active":"2024-05-07T02:31:50.092735+0000","last_peered":"2024-05-07T02:31:50.092735+0000","last_clean":"2024-05-07T02:31:50.092735+0000","last_became_active":"2024-05-07T02:26:11.866036+0000","last_became_peered":"2024-05-07T02:26:11.866036+0000","last_unstale":"2024-05-07T02:31:50.092735+0000","last_undegraded":"2024-05-07T02:31:50.092735+0000","last_fullsized":"2024-05-07T02:31:50.092735+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:47:22.412323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.985140+0000","last_change":"2024-05-07T02:27:32.628243+0000","last_active":"2024-05-07T02:31:49.985140+0000","last_peered":"2024-05-07T02:31:49.985140+0000","last_clean":"2024-05-07T02:31:49.985140+0000","last_became_active":"2024-05-07T02:27:32.627048+0000","last_became_peered":"2024-05-07T02:27:32.627048+0000","last_unstale":"2024-05-07T02:31:49.985140+0000","last_undegraded":"2024-05-07T02:31:49.985140+0000","last_fullsized":"2024-05-07T02:31:49.985140+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:33:49.802873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,10],"acting":[9,8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'44","reported_seq":597,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.148780+0000","last_change":"2024-05-07T02:26:07.786841+0000","last_active":"2024-05-07T02:31:50.148780+0000","last_peered":"2024-05-07T02:31:50.148780+0000","last_clean":"2024-05-07T02:31:50.148780+0000","last_became_active":"2024-05-07T02:26:07.783836+0000","last_became_peered":"2024-05-07T02:26:07.783836+0000","last_unstale":"2024-05-07T02:31:50.148780+0000","last_undegraded":"2024-05-07T02:31:50.148780+0000","last_fullsized":"2024-05-07T02:31:50.148780+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:28:53.390009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":73494,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":505,"num_read_kb":326,"num_write":46,"num_write_kb":94,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.988712+0000","last_change":"2024-05-07T02:26:11.857292+0000","last_active":"2024-05-07T02:31:49.988712+0000","last_peered":"2024-05-07T02:31:49.988712+0000","last_clean":"2024-05-07T02:31:49.988712+0000","last_became_active":"2024-05-07T02:26:11.856899+0000","last_became_peered":"2024-05-07T02:26:11.856899+0000","last_unstale":"2024-05-07T02:31:49.988712+0000","last_undegraded":"2024-05-07T02:31:49.988712+0000","last_fullsized":"2024-05-07T02:31:49.988712+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:45:42.279848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.999888+0000","last_change":"2024-05-07T02:26:25.570714+0000","last_active":"2024-05-07T02:31:49.999888+0000","last_peered":"2024-05-07T02:31:49.999888+0000","last_clean":"2024-05-07T02:31:49.999888+0000","last_became_active":"2024-05-07T02:26:25.570163+0000","last_became_peered":"2024-05-07T02:26:25.570163+0000","last_unstale":"2024-05-07T02:31:49.999888+0000","last_undegraded":"2024-05-07T02:31:49.999888+0000","last_fullsized":"2024-05-07T02:31:49.999888+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:55:51.903968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":59,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.826064+0000","last_change":"2024-05-07T02:26:07.785904+0000","last_active":"2024-05-07T02:31:49.826064+0000","last_peered":"2024-05-07T02:31:49.826064+0000","last_clean":"2024-05-07T02:31:49.826064+0000","last_became_active":"2024-05-07T02:26:07.785347+0000","last_became_peered":"2024-05-07T02:26:07.785347+0000","last_unstale":"2024-05-07T02:31:49.826064+0000","last_undegraded":"2024-05-07T02:31:49.826064+0000","last_fullsized":"2024-05-07T02:31:49.826064+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:42:05.604266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.097758+0000","last_change":"2024-05-07T02:26:11.878511+0000","last_active":"2024-05-07T02:31:50.097758+0000","last_peered":"2024-05-07T02:31:50.097758+0000","last_clean":"2024-05-07T02:31:50.097758+0000","last_became_active":"2024-05-07T02:26:11.878166+0000","last_became_peered":"2024-05-07T02:26:11.878166+0000","last_unstale":"2024-05-07T02:31:50.097758+0000","last_undegraded":"2024-05-07T02:31:50.097758+0000","last_fullsized":"2024-05-07T02:31:50.097758+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T08:14:40.436244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092718+0000","last_change":"2024-05-07T02:26:25.579387+0000","last_active":"2024-05-07T02:31:50.092718+0000","last_peered":"2024-05-07T02:31:50.092718+0000","last_clean":"2024-05-07T02:31:50.092718+0000","last_became_active":"2024-05-07T02:26:25.578757+0000","last_became_peered":"2024-05-07T02:26:25.578757+0000","last_unstale":"2024-05-07T02:31:50.092718+0000","last_undegraded":"2024-05-07T02:31:50.092718+0000","last_fullsized":"2024-05-07T02:31:50.092718+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:19:19.650941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.999163+0000","last_change":"2024-05-07T02:26:07.785154+0000","last_active":"2024-05-07T02:31:49.999163+0000","last_peered":"2024-05-07T02:31:49.999163+0000","last_clean":"2024-05-07T02:31:49.999163+0000","last_became_active":"2024-05-07T02:26:07.784589+0000","last_became_peered":"2024-05-07T02:26:07.784589+0000","last_unstale":"2024-05-07T02:31:49.999163+0000","last_undegraded":"2024-05-07T02:31:49.999163+0000","last_fullsized":"2024-05-07T02:31:49.999163+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:41:41.851182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.136271+0000","last_change":"2024-05-07T02:26:11.878162+0000","last_active":"2024-05-07T02:31:50.136271+0000","last_peered":"2024-05-07T02:31:50.136271+0000","last_clean":"2024-05-07T02:31:50.136271+0000","last_became_active":"2024-05-07T02:26:11.877770+0000","last_became_peered":"2024-05-07T02:26:11.877770+0000","last_unstale":"2024-05-07T02:31:50.136271+0000","last_undegraded":"2024-05-07T02:31:50.136271+0000","last_fullsized":"2024-05-07T02:31:50.136271+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:19:08.209398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.985823+0000","last_change":"2024-05-07T02:26:25.579465+0000","last_active":"2024-05-07T02:31:49.985823+0000","last_peered":"2024-05-07T02:31:49.985823+0000","last_clean":"2024-05-07T02:31:49.985823+0000","last_became_active":"2024-05-07T02:26:25.579003+0000","last_became_peered":"2024-05-07T02:26:25.579003+0000","last_unstale":"2024-05-07T02:31:49.985823+0000","last_undegraded":"2024-05-07T02:31:49.985823+0000","last_fullsized":"2024-05-07T02:31:49.985823+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:13:24.356366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'8","reported_seq":306,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.379222+0000","last_change":"2024-05-07T02:26:07.780827+0000","last_active":"2024-05-07T02:31:50.379222+0000","last_peered":"2024-05-07T02:31:50.379222+0000","last_clean":"2024-05-07T02:31:50.379222+0000","last_became_active":"2024-05-07T02:26:07.780353+0000","last_became_peered":"2024-05-07T02:26:07.780353+0000","last_unstale":"2024-05-07T02:31:50.379222+0000","last_undegraded":"2024-05-07T02:31:50.379222+0000","last_fullsized":"2024-05-07T02:31:50.379222+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:52:40.554135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1680,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":251,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.136134+0000","last_change":"2024-05-07T02:26:11.852516+0000","last_active":"2024-05-07T02:31:50.136134+0000","last_peered":"2024-05-07T02:31:50.136134+0000","last_clean":"2024-05-07T02:31:50.136134+0000","last_became_active":"2024-05-07T02:26:11.852112+0000","last_became_peered":"2024-05-07T02:26:11.852112+0000","last_unstale":"2024-05-07T02:31:50.136134+0000","last_undegraded":"2024-05-07T02:31:50.136134+0000","last_fullsized":"2024-05-07T02:31:50.136134+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:58:46.989250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.098786+0000","last_change":"2024-05-07T02:26:25.586726+0000","last_active":"2024-05-07T02:31:50.098786+0000","last_peered":"2024-05-07T02:31:50.098786+0000","last_clean":"2024-05-07T02:31:50.098786+0000","last_became_active":"2024-05-07T02:26:25.586267+0000","last_became_peered":"2024-05-07T02:26:25.586267+0000","last_unstale":"2024-05-07T02:31:50.098786+0000","last_undegraded":"2024-05-07T02:31:50.098786+0000","last_fullsized":"2024-05-07T02:31:50.098786+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:50:00.864463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092578+0000","last_change":"2024-05-07T02:27:32.599511+0000","last_active":"2024-05-07T02:31:50.092578+0000","last_peered":"2024-05-07T02:31:50.092578+0000","last_clean":"2024-05-07T02:31:50.092578+0000","last_became_active":"2024-05-07T02:27:32.590197+0000","last_became_peered":"2024-05-07T02:27:32.590197+0000","last_unstale":"2024-05-07T02:31:50.092578+0000","last_undegraded":"2024-05-07T02:31:50.092578+0000","last_fullsized":"2024-05-07T02:31:50.092578+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:23:07.401251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.999070+0000","last_change":"2024-05-07T02:26:11.857671+0000","last_active":"2024-05-07T02:31:49.999070+0000","last_peered":"2024-05-07T02:31:49.999070+0000","last_clean":"2024-05-07T02:31:49.999070+0000","last_became_active":"2024-05-07T02:26:11.857293+0000","last_became_peered":"2024-05-07T02:26:11.857293+0000","last_unstale":"2024-05-07T02:31:49.999070+0000","last_undegraded":"2024-05-07T02:31:49.999070+0000","last_fullsized":"2024-05-07T02:31:49.999070+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:48:42.267887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.999012+0000","last_change":"2024-05-07T02:26:25.563274+0000","last_active":"2024-05-07T02:31:49.999012+0000","last_peered":"2024-05-07T02:31:49.999012+0000","last_clean":"2024-05-07T02:31:49.999012+0000","last_became_active":"2024-05-07T02:26:25.562868+0000","last_became_peered":"2024-05-07T02:26:25.562868+0000","last_unstale":"2024-05-07T02:31:49.999012+0000","last_undegraded":"2024-05-07T02:31:49.999012+0000","last_fullsized":"2024-05-07T02:31:49.999012+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T14:20:03.210034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'4","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.988931+0000","last_change":"2024-05-07T02:26:07.784888+0000","last_active":"2024-05-07T02:31:49.988931+0000","last_peered":"2024-05-07T02:31:49.988931+0000","last_clean":"2024-05-07T02:31:49.988931+0000","last_became_active":"2024-05-07T02:26:07.784493+0000","last_became_peered":"2024-05-07T02:26:07.784493+0000","last_unstale":"2024-05-07T02:31:49.988931+0000","last_undegraded":"2024-05-07T02:31:49.988931+0000","last_fullsized":"2024-05-07T02:31:49.988931+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:21:14.054786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.093166+0000","last_change":"2024-05-07T02:26:11.858887+0000","last_active":"2024-05-07T02:31:50.093166+0000","last_peered":"2024-05-07T02:31:50.093166+0000","last_clean":"2024-05-07T02:31:50.093166+0000","last_became_active":"2024-05-07T02:26:11.858504+0000","last_became_peered":"2024-05-07T02:26:11.858504+0000","last_unstale":"2024-05-07T02:31:50.093166+0000","last_undegraded":"2024-05-07T02:31:50.093166+0000","last_fullsized":"2024-05-07T02:31:50.093166+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:53:20.290039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.988326+0000","last_change":"2024-05-07T02:27:32.627310+0000","last_active":"2024-05-07T02:31:49.988326+0000","last_peered":"2024-05-07T02:31:49.988326+0000","last_clean":"2024-05-07T02:31:49.988326+0000","last_became_active":"2024-05-07T02:27:32.626794+0000","last_became_peered":"2024-05-07T02:27:32.626794+0000","last_unstale":"2024-05-07T02:31:49.988326+0000","last_undegraded":"2024-05-07T02:31:49.988326+0000","last_fullsized":"2024-05-07T02:31:49.988326+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:06:30.466291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.985349+0000","last_change":"2024-05-07T02:26:08.000604+0000","last_active":"2024-05-07T02:31:49.985349+0000","last_peered":"2024-05-07T02:31:49.985349+0000","last_clean":"2024-05-07T02:31:49.985349+0000","last_became_active":"2024-05-07T02:26:08.000107+0000","last_became_peered":"2024-05-07T02:26:08.000107+0000","last_unstale":"2024-05-07T02:31:49.985349+0000","last_undegraded":"2024-05-07T02:31:49.985349+0000","last_fullsized":"2024-05-07T02:31:49.985349+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:58:06.716456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.003473+0000","last_change":"2024-05-07T02:26:11.876555+0000","last_active":"2024-05-07T02:31:50.003473+0000","last_peered":"2024-05-07T02:31:50.003473+0000","last_clean":"2024-05-07T02:31:50.003473+0000","last_became_active":"2024-05-07T02:26:11.876109+0000","last_became_peered":"2024-05-07T02:26:11.876109+0000","last_unstale":"2024-05-07T02:31:50.003473+0000","last_undegraded":"2024-05-07T02:31:50.003473+0000","last_fullsized":"2024-05-07T02:31:50.003473+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:10:24.995786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.098087+0000","last_change":"2024-05-07T02:26:25.567145+0000","last_active":"2024-05-07T02:31:50.098087+0000","last_peered":"2024-05-07T02:31:50.098087+0000","last_clean":"2024-05-07T02:31:50.098087+0000","last_became_active":"2024-05-07T02:26:25.566767+0000","last_became_peered":"2024-05-07T02:26:25.566767+0000","last_unstale":"2024-05-07T02:31:50.098087+0000","last_undegraded":"2024-05-07T02:31:50.098087+0000","last_fullsized":"2024-05-07T02:31:50.098087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T05:10:59.333367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'8","reported_seq":306,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.260546+0000","last_change":"2024-05-07T02:26:07.789566+0000","last_active":"2024-05-07T02:31:50.260546+0000","last_peered":"2024-05-07T02:31:50.260546+0000","last_clean":"2024-05-07T02:31:50.260546+0000","last_became_active":"2024-05-07T02:26:07.789106+0000","last_became_peered":"2024-05-07T02:26:07.789106+0000","last_unstale":"2024-05-07T02:31:50.260546+0000","last_undegraded":"2024-05-07T02:31:50.260546+0000","last_fullsized":"2024-05-07T02:31:50.260546+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:45:13.620183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":250,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.003155+0000","last_change":"2024-05-07T02:26:11.869080+0000","last_active":"2024-05-07T02:31:50.003155+0000","last_peered":"2024-05-07T02:31:50.003155+0000","last_clean":"2024-05-07T02:31:50.003155+0000","last_became_active":"2024-05-07T02:26:11.868702+0000","last_became_peered":"2024-05-07T02:26:11.868702+0000","last_unstale":"2024-05-07T02:31:50.003155+0000","last_undegraded":"2024-05-07T02:31:50.003155+0000","last_fullsized":"2024-05-07T02:31:50.003155+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:05:05.497071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.098519+0000","last_change":"2024-05-07T02:26:25.867393+0000","last_active":"2024-05-07T02:31:50.098519+0000","last_peered":"2024-05-07T02:31:50.098519+0000","last_clean":"2024-05-07T02:31:50.098519+0000","last_became_active":"2024-05-07T02:26:25.866956+0000","last_became_peered":"2024-05-07T02:26:25.866956+0000","last_unstale":"2024-05-07T02:31:50.098519+0000","last_undegraded":"2024-05-07T02:31:50.098519+0000","last_fullsized":"2024-05-07T02:31:50.098519+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:23:48.821668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.984780+0000","last_change":"2024-05-07T02:28:33.871287+0000","last_active":"2024-05-07T02:31:49.984780+0000","last_peered":"2024-05-07T02:31:49.984780+0000","last_clean":"2024-05-07T02:31:49.984780+0000","last_became_active":"2024-05-07T02:28:33.870256+0000","last_became_peered":"2024-05-07T02:28:33.870256+0000","last_unstale":"2024-05-07T02:31:49.984780+0000","last_undegraded":"2024-05-07T02:31:49.984780+0000","last_fullsized":"2024-05-07T02:31:49.984780+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T08:36:26.789348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":301,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.097586+0000","last_change":"2024-05-07T02:26:07.999784+0000","last_active":"2024-05-07T02:31:50.097586+0000","last_peered":"2024-05-07T02:31:50.097586+0000","last_clean":"2024-05-07T02:31:50.097586+0000","last_became_active":"2024-05-07T02:26:07.999339+0000","last_became_peered":"2024-05-07T02:26:07.999339+0000","last_unstale":"2024-05-07T02:31:50.097586+0000","last_undegraded":"2024-05-07T02:31:50.097586+0000","last_fullsized":"2024-05-07T02:31:50.097586+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:50:41.622486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":247,"num_read_kb":247,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.136478+0000","last_change":"2024-05-07T02:26:25.587032+0000","last_active":"2024-05-07T02:31:50.136478+0000","last_peered":"2024-05-07T02:31:50.136478+0000","last_clean":"2024-05-07T02:31:50.136478+0000","last_became_active":"2024-05-07T02:26:25.586530+0000","last_became_peered":"2024-05-07T02:26:25.586530+0000","last_unstale":"2024-05-07T02:31:50.136478+0000","last_undegraded":"2024-05-07T02:31:50.136478+0000","last_fullsized":"2024-05-07T02:31:50.136478+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:13:13.335832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825819+0000","last_change":"2024-05-07T02:26:11.878593+0000","last_active":"2024-05-07T02:31:49.825819+0000","last_peered":"2024-05-07T02:31:49.825819+0000","last_clean":"2024-05-07T02:31:49.825819+0000","last_became_active":"2024-05-07T02:26:11.873286+0000","last_became_peered":"2024-05-07T02:26:11.873286+0000","last_unstale":"2024-05-07T02:31:49.825819+0000","last_undegraded":"2024-05-07T02:31:49.825819+0000","last_fullsized":"2024-05-07T02:31:49.825819+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:22:33.769539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825626+0000","last_change":"2024-05-07T02:26:07.790087+0000","last_active":"2024-05-07T02:31:49.825626+0000","last_peered":"2024-05-07T02:31:49.825626+0000","last_clean":"2024-05-07T02:31:49.825626+0000","last_became_active":"2024-05-07T02:26:07.789592+0000","last_became_peered":"2024-05-07T02:26:07.789592+0000","last_unstale":"2024-05-07T02:31:49.825626+0000","last_undegraded":"2024-05-07T02:31:49.825626+0000","last_fullsized":"2024-05-07T02:31:49.825626+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:44:52.569055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":122,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.135972+0000","last_change":"2024-05-07T02:24:27.877820+0000","last_active":"2024-05-07T02:31:50.135972+0000","last_peered":"2024-05-07T02:31:50.135972+0000","last_clean":"2024-05-07T02:31:50.135972+0000","last_became_active":"2024-05-07T02:24:27.873221+0000","last_became_peered":"2024-05-07T02:24:27.873221+0000","last_unstale":"2024-05-07T02:31:50.135972+0000","last_undegraded":"2024-05-07T02:31:50.135972+0000","last_fullsized":"2024-05-07T02:31:50.135972+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:20:21.582322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:20:21.582322+0000","last_clean_scrub_stamp":"2024-05-07T02:20:21.582322+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T14:16:32.772438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.999518+0000","last_change":"2024-05-07T02:26:25.868394+0000","last_active":"2024-05-07T02:31:49.999518+0000","last_peered":"2024-05-07T02:31:49.999518+0000","last_clean":"2024-05-07T02:31:49.999518+0000","last_became_active":"2024-05-07T02:26:25.867583+0000","last_became_peered":"2024-05-07T02:26:25.867583+0000","last_unstale":"2024-05-07T02:31:49.999518+0000","last_undegraded":"2024-05-07T02:31:49.999518+0000","last_fullsized":"2024-05-07T02:31:49.999518+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:55:44.705608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.097400+0000","last_change":"2024-05-07T02:26:11.868371+0000","last_active":"2024-05-07T02:31:50.097400+0000","last_peered":"2024-05-07T02:31:50.097400+0000","last_clean":"2024-05-07T02:31:50.097400+0000","last_became_active":"2024-05-07T02:26:11.867981+0000","last_became_peered":"2024-05-07T02:26:11.867981+0000","last_unstale":"2024-05-07T02:31:50.097400+0000","last_undegraded":"2024-05-07T02:31:50.097400+0000","last_fullsized":"2024-05-07T02:31:50.097400+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:08:54.687327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.136059+0000","last_change":"2024-05-07T02:26:07.785344+0000","last_active":"2024-05-07T02:31:50.136059+0000","last_peered":"2024-05-07T02:31:50.136059+0000","last_clean":"2024-05-07T02:31:50.136059+0000","last_became_active":"2024-05-07T02:26:07.784707+0000","last_became_peered":"2024-05-07T02:26:07.784707+0000","last_unstale":"2024-05-07T02:31:50.136059+0000","last_undegraded":"2024-05-07T02:31:50.136059+0000","last_fullsized":"2024-05-07T02:31:50.136059+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T05:00:38.754313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092632+0000","last_change":"2024-05-07T02:26:25.571989+0000","last_active":"2024-05-07T02:31:50.092632+0000","last_peered":"2024-05-07T02:31:50.092632+0000","last_clean":"2024-05-07T02:31:50.092632+0000","last_became_active":"2024-05-07T02:26:25.571533+0000","last_became_peered":"2024-05-07T02:26:25.571533+0000","last_unstale":"2024-05-07T02:31:50.092632+0000","last_undegraded":"2024-05-07T02:31:50.092632+0000","last_fullsized":"2024-05-07T02:31:50.092632+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:54:38.669278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.988658+0000","last_change":"2024-05-07T02:26:07.999636+0000","last_active":"2024-05-07T02:31:49.988658+0000","last_peered":"2024-05-07T02:31:49.988658+0000","last_clean":"2024-05-07T02:31:49.988658+0000","last_became_active":"2024-05-07T02:26:07.999100+0000","last_became_peered":"2024-05-07T02:26:07.999100+0000","last_unstale":"2024-05-07T02:31:49.988658+0000","last_undegraded":"2024-05-07T02:31:49.988658+0000","last_fullsized":"2024-05-07T02:31:49.988658+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:10:24.730988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.098303+0000","last_change":"2024-05-07T02:26:11.888279+0000","last_active":"2024-05-07T02:31:50.098303+0000","last_peered":"2024-05-07T02:31:50.098303+0000","last_clean":"2024-05-07T02:31:50.098303+0000","last_became_active":"2024-05-07T02:26:11.887480+0000","last_became_peered":"2024-05-07T02:26:11.887480+0000","last_unstale":"2024-05-07T02:31:50.098303+0000","last_undegraded":"2024-05-07T02:31:50.098303+0000","last_fullsized":"2024-05-07T02:31:50.098303+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:12:12.368368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.824134+0000","last_change":"2024-05-07T02:26:25.867444+0000","last_active":"2024-05-07T02:31:49.824134+0000","last_peered":"2024-05-07T02:31:49.824134+0000","last_clean":"2024-05-07T02:31:49.824134+0000","last_became_active":"2024-05-07T02:26:25.866392+0000","last_became_peered":"2024-05-07T02:26:25.866392+0000","last_unstale":"2024-05-07T02:31:49.824134+0000","last_undegraded":"2024-05-07T02:31:49.824134+0000","last_fullsized":"2024-05-07T02:31:49.824134+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:54:36.112427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'52","reported_seq":123,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.150979+0000","last_change":"2024-05-07T02:26:07.798091+0000","last_active":"2024-05-07T02:31:50.150979+0000","last_peered":"2024-05-07T02:31:50.150979+0000","last_clean":"2024-05-07T02:31:50.150979+0000","last_became_active":"2024-05-07T02:26:07.797716+0000","last_became_peered":"2024-05-07T02:26:07.797716+0000","last_unstale":"2024-05-07T02:31:50.150979+0000","last_undegraded":"2024-05-07T02:31:50.150979+0000","last_fullsized":"2024-05-07T02:31:50.150979+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:04:54.570916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":92,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825760+0000","last_change":"2024-05-07T02:26:11.888638+0000","last_active":"2024-05-07T02:31:49.825760+0000","last_peered":"2024-05-07T02:31:49.825760+0000","last_clean":"2024-05-07T02:31:49.825760+0000","last_became_active":"2024-05-07T02:26:11.888269+0000","last_became_peered":"2024-05-07T02:26:11.888269+0000","last_unstale":"2024-05-07T02:31:49.825760+0000","last_undegraded":"2024-05-07T02:31:49.825760+0000","last_fullsized":"2024-05-07T02:31:49.825760+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:09:38.965362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825175+0000","last_change":"2024-05-07T02:26:25.558975+0000","last_active":"2024-05-07T02:31:49.825175+0000","last_peered":"2024-05-07T02:31:49.825175+0000","last_clean":"2024-05-07T02:31:49.825175+0000","last_became_active":"2024-05-07T02:26:25.557947+0000","last_became_peered":"2024-05-07T02:26:25.557947+0000","last_unstale":"2024-05-07T02:31:49.825175+0000","last_undegraded":"2024-05-07T02:31:49.825175+0000","last_fullsized":"2024-05-07T02:31:49.825175+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:29:32.071625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.826855+0000","last_change":"2024-05-07T02:26:07.796643+0000","last_active":"2024-05-07T02:31:49.826855+0000","last_peered":"2024-05-07T02:31:49.826855+0000","last_clean":"2024-05-07T02:31:49.826855+0000","last_became_active":"2024-05-07T02:26:07.796264+0000","last_became_peered":"2024-05-07T02:26:07.796264+0000","last_unstale":"2024-05-07T02:31:49.826855+0000","last_undegraded":"2024-05-07T02:31:49.826855+0000","last_fullsized":"2024-05-07T02:31:49.826855+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:29:32.454816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.094595+0000","last_change":"2024-05-07T02:26:11.873670+0000","last_active":"2024-05-07T02:31:50.094595+0000","last_peered":"2024-05-07T02:31:50.094595+0000","last_clean":"2024-05-07T02:31:50.094595+0000","last_became_active":"2024-05-07T02:26:11.871219+0000","last_became_peered":"2024-05-07T02:26:11.871219+0000","last_unstale":"2024-05-07T02:31:50.094595+0000","last_undegraded":"2024-05-07T02:31:50.094595+0000","last_fullsized":"2024-05-07T02:31:50.094595+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:49:48.048127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.826601+0000","last_change":"2024-05-07T02:26:25.868011+0000","last_active":"2024-05-07T02:31:49.826601+0000","last_peered":"2024-05-07T02:31:49.826601+0000","last_clean":"2024-05-07T02:31:49.826601+0000","last_became_active":"2024-05-07T02:26:25.867465+0000","last_became_peered":"2024-05-07T02:26:25.867465+0000","last_unstale":"2024-05-07T02:31:49.826601+0000","last_undegraded":"2024-05-07T02:31:49.826601+0000","last_fullsized":"2024-05-07T02:31:49.826601+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T05:14:40.001874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092382+0000","last_change":"2024-05-07T02:26:07.999233+0000","last_active":"2024-05-07T02:31:50.092382+0000","last_peered":"2024-05-07T02:31:50.092382+0000","last_clean":"2024-05-07T02:31:50.092382+0000","last_became_active":"2024-05-07T02:26:07.998920+0000","last_became_peered":"2024-05-07T02:26:07.998920+0000","last_unstale":"2024-05-07T02:31:50.092382+0000","last_undegraded":"2024-05-07T02:31:50.092382+0000","last_fullsized":"2024-05-07T02:31:50.092382+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:33:33.878984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.988283+0000","last_change":"2024-05-07T02:26:11.856832+0000","last_active":"2024-05-07T02:31:49.988283+0000","last_peered":"2024-05-07T02:31:49.988283+0000","last_clean":"2024-05-07T02:31:49.988283+0000","last_became_active":"2024-05-07T02:26:11.856424+0000","last_became_peered":"2024-05-07T02:26:11.856424+0000","last_unstale":"2024-05-07T02:31:49.988283+0000","last_undegraded":"2024-05-07T02:31:49.988283+0000","last_fullsized":"2024-05-07T02:31:49.988283+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:00:46.839542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.988845+0000","last_change":"2024-05-07T02:27:32.626086+0000","last_active":"2024-05-07T02:31:49.988845+0000","last_peered":"2024-05-07T02:31:49.988845+0000","last_clean":"2024-05-07T02:31:49.988845+0000","last_became_active":"2024-05-07T02:27:32.625468+0000","last_became_peered":"2024-05-07T02:27:32.625468+0000","last_unstale":"2024-05-07T02:31:49.988845+0000","last_undegraded":"2024-05-07T02:31:49.988845+0000","last_fullsized":"2024-05-07T02:31:49.988845+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:17:57.245648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.824755+0000","last_change":"2024-05-07T02:26:07.793793+0000","last_active":"2024-05-07T02:31:49.824755+0000","last_peered":"2024-05-07T02:31:49.824755+0000","last_clean":"2024-05-07T02:31:49.824755+0000","last_became_active":"2024-05-07T02:26:07.790291+0000","last_became_peered":"2024-05-07T02:26:07.790291+0000","last_unstale":"2024-05-07T02:31:49.824755+0000","last_undegraded":"2024-05-07T02:31:49.824755+0000","last_fullsized":"2024-05-07T02:31:49.824755+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:50:14.591968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092657+0000","last_change":"2024-05-07T02:26:11.869184+0000","last_active":"2024-05-07T02:31:50.092657+0000","last_peered":"2024-05-07T02:31:50.092657+0000","last_clean":"2024-05-07T02:31:50.092657+0000","last_became_active":"2024-05-07T02:26:11.868773+0000","last_became_peered":"2024-05-07T02:26:11.868773+0000","last_unstale":"2024-05-07T02:31:50.092657+0000","last_undegraded":"2024-05-07T02:31:50.092657+0000","last_fullsized":"2024-05-07T02:31:50.092657+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:24:04.986359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825656+0000","last_change":"2024-05-07T02:26:25.580095+0000","last_active":"2024-05-07T02:31:49.825656+0000","last_peered":"2024-05-07T02:31:49.825656+0000","last_clean":"2024-05-07T02:31:49.825656+0000","last_became_active":"2024-05-07T02:26:25.579615+0000","last_became_peered":"2024-05-07T02:26:25.579615+0000","last_unstale":"2024-05-07T02:31:49.825656+0000","last_undegraded":"2024-05-07T02:31:49.825656+0000","last_fullsized":"2024-05-07T02:31:49.825656+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:18:05.714552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'4","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825368+0000","last_change":"2024-05-07T02:26:07.784709+0000","last_active":"2024-05-07T02:31:49.825368+0000","last_peered":"2024-05-07T02:31:49.825368+0000","last_clean":"2024-05-07T02:31:49.825368+0000","last_became_active":"2024-05-07T02:26:07.784282+0000","last_became_peered":"2024-05-07T02:26:07.784282+0000","last_unstale":"2024-05-07T02:31:49.825368+0000","last_undegraded":"2024-05-07T02:31:49.825368+0000","last_fullsized":"2024-05-07T02:31:49.825368+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:01:47.023496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.826271+0000","last_change":"2024-05-07T02:26:11.879152+0000","last_active":"2024-05-07T02:31:49.826271+0000","last_peered":"2024-05-07T02:31:49.826271+0000","last_clean":"2024-05-07T02:31:49.826271+0000","last_became_active":"2024-05-07T02:26:11.878571+0000","last_became_peered":"2024-05-07T02:26:11.878571+0000","last_unstale":"2024-05-07T02:31:49.826271+0000","last_undegraded":"2024-05-07T02:31:49.826271+0000","last_fullsized":"2024-05-07T02:31:49.826271+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:04:12.637807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.826575+0000","last_change":"2024-05-07T02:26:25.867199+0000","last_active":"2024-05-07T02:31:49.826575+0000","last_peered":"2024-05-07T02:31:49.826575+0000","last_clean":"2024-05-07T02:31:49.826575+0000","last_became_active":"2024-05-07T02:26:25.866624+0000","last_became_peered":"2024-05-07T02:26:25.866624+0000","last_unstale":"2024-05-07T02:31:49.826575+0000","last_undegraded":"2024-05-07T02:31:49.826575+0000","last_fullsized":"2024-05-07T02:31:49.826575+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:14:49.513313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.988386+0000","last_change":"2024-05-07T02:26:07.784155+0000","last_active":"2024-05-07T02:31:49.988386+0000","last_peered":"2024-05-07T02:31:49.988386+0000","last_clean":"2024-05-07T02:31:49.988386+0000","last_became_active":"2024-05-07T02:26:07.783579+0000","last_became_peered":"2024-05-07T02:26:07.783579+0000","last_unstale":"2024-05-07T02:31:49.988386+0000","last_undegraded":"2024-05-07T02:31:49.988386+0000","last_fullsized":"2024-05-07T02:31:49.988386+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T08:51:32.037959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.136365+0000","last_change":"2024-05-07T02:26:11.868454+0000","last_active":"2024-05-07T02:31:50.136365+0000","last_peered":"2024-05-07T02:31:50.136365+0000","last_clean":"2024-05-07T02:31:50.136365+0000","last_became_active":"2024-05-07T02:26:11.868117+0000","last_became_peered":"2024-05-07T02:26:11.868117+0000","last_unstale":"2024-05-07T02:31:50.136365+0000","last_undegraded":"2024-05-07T02:31:50.136365+0000","last_fullsized":"2024-05-07T02:31:50.136365+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:24:36.703684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092403+0000","last_change":"2024-05-07T02:26:25.573834+0000","last_active":"2024-05-07T02:31:50.092403+0000","last_peered":"2024-05-07T02:31:50.092403+0000","last_clean":"2024-05-07T02:31:50.092403+0000","last_became_active":"2024-05-07T02:26:25.573372+0000","last_became_peered":"2024-05-07T02:26:25.573372+0000","last_unstale":"2024-05-07T02:31:50.092403+0000","last_undegraded":"2024-05-07T02:31:50.092403+0000","last_fullsized":"2024-05-07T02:31:50.092403+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:03:36.916915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'6","reported_seq":303,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.305895+0000","last_change":"2024-05-07T02:26:08.000245+0000","last_active":"2024-05-07T02:31:50.305895+0000","last_peered":"2024-05-07T02:31:50.305895+0000","last_clean":"2024-05-07T02:31:50.305895+0000","last_became_active":"2024-05-07T02:26:07.999587+0000","last_became_peered":"2024-05-07T02:26:07.999587+0000","last_unstale":"2024-05-07T02:31:50.305895+0000","last_undegraded":"2024-05-07T02:31:50.305895+0000","last_fullsized":"2024-05-07T02:31:50.305895+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:44:41.754799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":249,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.136693+0000","last_change":"2024-05-07T02:26:11.858308+0000","last_active":"2024-05-07T02:31:50.136693+0000","last_peered":"2024-05-07T02:31:50.136693+0000","last_clean":"2024-05-07T02:31:50.136693+0000","last_became_active":"2024-05-07T02:26:11.857855+0000","last_became_peered":"2024-05-07T02:26:11.857855+0000","last_unstale":"2024-05-07T02:31:50.136693+0000","last_undegraded":"2024-05-07T02:31:50.136693+0000","last_fullsized":"2024-05-07T02:31:50.136693+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:11:50.021178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.095840+0000","last_change":"2024-05-07T02:26:25.584090+0000","last_active":"2024-05-07T02:31:50.095840+0000","last_peered":"2024-05-07T02:31:50.095840+0000","last_clean":"2024-05-07T02:31:50.095840+0000","last_became_active":"2024-05-07T02:26:25.580108+0000","last_became_peered":"2024-05-07T02:26:25.580108+0000","last_unstale":"2024-05-07T02:31:50.095840+0000","last_undegraded":"2024-05-07T02:31:50.095840+0000","last_fullsized":"2024-05-07T02:31:50.095840+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:56:56.964047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'41","reported_seq":97,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.147728+0000","last_change":"2024-05-07T02:26:07.788188+0000","last_active":"2024-05-07T02:31:50.147728+0000","last_peered":"2024-05-07T02:31:50.147728+0000","last_clean":"2024-05-07T02:31:50.147728+0000","last_became_active":"2024-05-07T02:26:07.787826+0000","last_became_peered":"2024-05-07T02:26:07.787826+0000","last_unstale":"2024-05-07T02:31:50.147728+0000","last_undegraded":"2024-05-07T02:31:50.147728+0000","last_fullsized":"2024-05-07T02:31:50.147728+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:06:14.588079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.136460+0000","last_change":"2024-05-07T02:26:11.862925+0000","last_active":"2024-05-07T02:31:50.136460+0000","last_peered":"2024-05-07T02:31:50.136460+0000","last_clean":"2024-05-07T02:31:50.136460+0000","last_became_active":"2024-05-07T02:26:11.861332+0000","last_became_peered":"2024-05-07T02:26:11.861332+0000","last_unstale":"2024-05-07T02:31:50.136460+0000","last_undegraded":"2024-05-07T02:31:50.136460+0000","last_fullsized":"2024-05-07T02:31:50.136460+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:51:11.133504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.998822+0000","last_change":"2024-05-07T02:26:25.557151+0000","last_active":"2024-05-07T02:31:49.998822+0000","last_peered":"2024-05-07T02:31:49.998822+0000","last_clean":"2024-05-07T02:31:49.998822+0000","last_became_active":"2024-05-07T02:26:25.556615+0000","last_became_peered":"2024-05-07T02:26:25.556615+0000","last_unstale":"2024-05-07T02:31:49.998822+0000","last_undegraded":"2024-05-07T02:31:49.998822+0000","last_fullsized":"2024-05-07T02:31:49.998822+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:33:20.446543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":63,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.098780+0000","last_change":"2024-05-07T02:27:32.598094+0000","last_active":"2024-05-07T02:31:50.098780+0000","last_peered":"2024-05-07T02:31:50.098780+0000","last_clean":"2024-05-07T02:31:50.098780+0000","last_became_active":"2024-05-07T02:27:32.594099+0000","last_became_peered":"2024-05-07T02:27:32.594099+0000","last_unstale":"2024-05-07T02:31:50.098780+0000","last_undegraded":"2024-05-07T02:31:50.098780+0000","last_fullsized":"2024-05-07T02:31:50.098780+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:44:50.979927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.136717+0000","last_change":"2024-05-07T02:26:11.860507+0000","last_active":"2024-05-07T02:31:50.136717+0000","last_peered":"2024-05-07T02:31:50.136717+0000","last_clean":"2024-05-07T02:31:50.136717+0000","last_became_active":"2024-05-07T02:26:11.860116+0000","last_became_peered":"2024-05-07T02:26:11.860116+0000","last_unstale":"2024-05-07T02:31:50.136717+0000","last_undegraded":"2024-05-07T02:31:50.136717+0000","last_fullsized":"2024-05-07T02:31:50.136717+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T05:09:38.963040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092816+0000","last_change":"2024-05-07T02:26:11.867932+0000","last_active":"2024-05-07T02:31:50.092816+0000","last_peered":"2024-05-07T02:31:50.092816+0000","last_clean":"2024-05-07T02:31:50.092816+0000","last_became_active":"2024-05-07T02:26:11.867537+0000","last_became_peered":"2024-05-07T02:26:11.867537+0000","last_unstale":"2024-05-07T02:31:50.092816+0000","last_undegraded":"2024-05-07T02:31:50.092816+0000","last_fullsized":"2024-05-07T02:31:50.092816+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:31:35.843401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825700+0000","last_change":"2024-05-07T02:26:11.866074+0000","last_active":"2024-05-07T02:31:49.825700+0000","last_peered":"2024-05-07T02:31:49.825700+0000","last_clean":"2024-05-07T02:31:49.825700+0000","last_became_active":"2024-05-07T02:26:11.865669+0000","last_became_peered":"2024-05-07T02:26:11.865669+0000","last_unstale":"2024-05-07T02:31:49.825700+0000","last_undegraded":"2024-05-07T02:31:49.825700+0000","last_fullsized":"2024-05-07T02:31:49.825700+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:08:51.323706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825297+0000","last_change":"2024-05-07T02:26:11.876428+0000","last_active":"2024-05-07T02:31:49.825297+0000","last_peered":"2024-05-07T02:31:49.825297+0000","last_clean":"2024-05-07T02:31:49.825297+0000","last_became_active":"2024-05-07T02:26:11.876012+0000","last_became_peered":"2024-05-07T02:26:11.876012+0000","last_unstale":"2024-05-07T02:31:49.825297+0000","last_undegraded":"2024-05-07T02:31:49.825297+0000","last_fullsized":"2024-05-07T02:31:49.825297+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:24:58.088695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.094427+0000","last_change":"2024-05-07T02:26:11.880682+0000","last_active":"2024-05-07T02:31:50.094427+0000","last_peered":"2024-05-07T02:31:50.094427+0000","last_clean":"2024-05-07T02:31:50.094427+0000","last_became_active":"2024-05-07T02:26:11.880308+0000","last_became_peered":"2024-05-07T02:26:11.880308+0000","last_unstale":"2024-05-07T02:31:50.094427+0000","last_undegraded":"2024-05-07T02:31:50.094427+0000","last_fullsized":"2024-05-07T02:31:50.094427+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:52:45.083964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.098433+0000","last_change":"2024-05-07T02:26:11.876932+0000","last_active":"2024-05-07T02:31:50.098433+0000","last_peered":"2024-05-07T02:31:50.098433+0000","last_clean":"2024-05-07T02:31:50.098433+0000","last_became_active":"2024-05-07T02:26:11.876082+0000","last_became_peered":"2024-05-07T02:26:11.876082+0000","last_unstale":"2024-05-07T02:31:50.098433+0000","last_undegraded":"2024-05-07T02:31:50.098433+0000","last_fullsized":"2024-05-07T02:31:50.098433+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:45:21.435500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092847+0000","last_change":"2024-05-07T02:26:11.861339+0000","last_active":"2024-05-07T02:31:50.092847+0000","last_peered":"2024-05-07T02:31:50.092847+0000","last_clean":"2024-05-07T02:31:50.092847+0000","last_became_active":"2024-05-07T02:26:11.860912+0000","last_became_peered":"2024-05-07T02:26:11.860912+0000","last_unstale":"2024-05-07T02:31:50.092847+0000","last_undegraded":"2024-05-07T02:31:50.092847+0000","last_fullsized":"2024-05-07T02:31:50.092847+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:12:57.944986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825780+0000","last_change":"2024-05-07T02:26:11.867258+0000","last_active":"2024-05-07T02:31:49.825780+0000","last_peered":"2024-05-07T02:31:49.825780+0000","last_clean":"2024-05-07T02:31:49.825780+0000","last_became_active":"2024-05-07T02:26:11.866837+0000","last_became_peered":"2024-05-07T02:26:11.866837+0000","last_unstale":"2024-05-07T02:31:49.825780+0000","last_undegraded":"2024-05-07T02:31:49.825780+0000","last_fullsized":"2024-05-07T02:31:49.825780+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:07:58.589457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825979+0000","last_change":"2024-05-07T02:26:11.888434+0000","last_active":"2024-05-07T02:31:49.825979+0000","last_peered":"2024-05-07T02:31:49.825979+0000","last_clean":"2024-05-07T02:31:49.825979+0000","last_became_active":"2024-05-07T02:26:11.887681+0000","last_became_peered":"2024-05-07T02:26:11.887681+0000","last_unstale":"2024-05-07T02:31:49.825979+0000","last_undegraded":"2024-05-07T02:31:49.825979+0000","last_fullsized":"2024-05-07T02:31:49.825979+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:38:58.896781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.999103+0000","last_change":"2024-05-07T02:26:11.875304+0000","last_active":"2024-05-07T02:31:49.999103+0000","last_peered":"2024-05-07T02:31:49.999103+0000","last_clean":"2024-05-07T02:31:49.999103+0000","last_became_active":"2024-05-07T02:26:11.874946+0000","last_became_peered":"2024-05-07T02:26:11.874946+0000","last_unstale":"2024-05-07T02:31:49.999103+0000","last_undegraded":"2024-05-07T02:31:49.999103+0000","last_fullsized":"2024-05-07T02:31:49.999103+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:56:04.093791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.824887+0000","last_change":"2024-05-07T02:26:11.879311+0000","last_active":"2024-05-07T02:31:49.824887+0000","last_peered":"2024-05-07T02:31:49.824887+0000","last_clean":"2024-05-07T02:31:49.824887+0000","last_became_active":"2024-05-07T02:26:11.877620+0000","last_became_peered":"2024-05-07T02:26:11.877620+0000","last_unstale":"2024-05-07T02:31:49.824887+0000","last_undegraded":"2024-05-07T02:31:49.824887+0000","last_fullsized":"2024-05-07T02:31:49.824887+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T08:41:40.525708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092753+0000","last_change":"2024-05-07T02:26:11.876726+0000","last_active":"2024-05-07T02:31:50.092753+0000","last_peered":"2024-05-07T02:31:50.092753+0000","last_clean":"2024-05-07T02:31:50.092753+0000","last_became_active":"2024-05-07T02:26:11.876234+0000","last_became_peered":"2024-05-07T02:26:11.876234+0000","last_unstale":"2024-05-07T02:31:50.092753+0000","last_undegraded":"2024-05-07T02:31:50.092753+0000","last_fullsized":"2024-05-07T02:31:50.092753+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:26:34.422258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.098131+0000","last_change":"2024-05-07T02:26:11.879159+0000","last_active":"2024-05-07T02:31:50.098131+0000","last_peered":"2024-05-07T02:31:50.098131+0000","last_clean":"2024-05-07T02:31:50.098131+0000","last_became_active":"2024-05-07T02:26:11.878769+0000","last_became_peered":"2024-05-07T02:26:11.878769+0000","last_unstale":"2024-05-07T02:31:50.098131+0000","last_undegraded":"2024-05-07T02:31:50.098131+0000","last_fullsized":"2024-05-07T02:31:50.098131+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:05:27.089475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092884+0000","last_change":"2024-05-07T02:27:32.590188+0000","last_active":"2024-05-07T02:31:50.092884+0000","last_peered":"2024-05-07T02:31:50.092884+0000","last_clean":"2024-05-07T02:31:50.092884+0000","last_became_active":"2024-05-07T02:27:32.589784+0000","last_became_peered":"2024-05-07T02:27:32.589784+0000","last_unstale":"2024-05-07T02:31:50.092884+0000","last_undegraded":"2024-05-07T02:31:50.092884+0000","last_fullsized":"2024-05-07T02:31:50.092884+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:32:40.488143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":607529,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2601,"num_read_kb":1425,"num_write":388,"num_write_kb":1481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2187264,"data_stored":1822587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":312,"ondisk_log_size":312,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710803,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1056768,"data_stored":525718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907034,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":33020,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712132,"statfs":{"total":95995035648,"available":95961223168,"internally_reserved":0,"allocated":1888256,"data_stored":1318608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070560,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28456,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716696,"statfs":{"total":95995035648,"available":95965896704,"internally_reserved":0,"allocated":1470464,"data_stored":922848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266791,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36760,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708392,"statfs":{"total":95995035648,"available":95957393408,"internally_reserved":0,"allocated":1454080,"data_stored":919018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863463020,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1667072,"data_stored":1122862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626545,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":521963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574992,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1126400,"data_stored":596487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738517,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1073152,"data_stored":522887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1146880,"data_stored":596566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182531,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32728,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712424,"statfs":{"total":95995035648,"available":95961522176,"internally_reserved":0,"allocated":1589248,"data_stored":1045564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986304,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1056768,"data_stored":526206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822777,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28784,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716368,"statfs":{"total":95995035648,"available":95965560832,"internally_reserved":0,"allocated":1810432,"data_stored":1249593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":76169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":76248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":134811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":394604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":207922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-07T02:31:57.215 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-05-07T02:31:57.215 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-07T02:31:57.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.528+0000 7f54caa00640 1 -- 172.21.15.19:0/3260810836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54c4076b20 msgr2=0x7f54c410fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:57.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.528+0000 7f54caa00640 1 --2- 172.21.15.19:0/3260810836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54c4076b20 0x7f54c410fe70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f54b0009860 tx=0x7f54b002a340 comp rx=0 tx=0).stop 2024-05-07T02:31:57.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.529+0000 7f54caa00640 1 -- 172.21.15.19:0/3260810836 shutdown_connections 2024-05-07T02:31:57.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.529+0000 7f54caa00640 1 --2- 172.21.15.19:0/3260810836 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54c41103b0 0x7f54c41127d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:57.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.529+0000 7f54caa00640 1 --2- 172.21.15.19:0/3260810836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54c4076b20 0x7f54c410fe70 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:57.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.529+0000 7f54caa00640 1 --2- 172.21.15.19:0/3260810836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54c40761d0 0x7f54c40765e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:57.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.529+0000 7f54caa00640 1 -- 172.21.15.19:0/3260810836 >> 172.21.15.19:0/3260810836 conn(0x7f54c40fee60 msgr2=0x7f54c41012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:57.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.529+0000 7f54caa00640 1 -- 172.21.15.19:0/3260810836 shutdown_connections 2024-05-07T02:31:57.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.529+0000 7f54caa00640 1 -- 172.21.15.19:0/3260810836 wait complete. 2024-05-07T02:31:57.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.530+0000 7f54caa00640 1 Processor -- start 2024-05-07T02:31:57.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.530+0000 7f54caa00640 1 -- start start 2024-05-07T02:31:57.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.530+0000 7f54caa00640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54c40761d0 0x7f54c41a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:57.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.530+0000 7f54caa00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54c4076b20 0x7f54c41a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:57.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.530+0000 7f54caa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54c41103b0 0x7f54c41a7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:57.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.530+0000 7f54caa00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f54c4115d90 con 0x7f54c41103b0 2024-05-07T02:31:57.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.530+0000 7f54caa00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f54c4115c10 con 0x7f54c4076b20 2024-05-07T02:31:57.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.530+0000 7f54caa00640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f54c4115f10 con 0x7f54c40761d0 2024-05-07T02:31:57.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.530+0000 7f54c3400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54c4076b20 0x7f54c41a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:57.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.530+0000 7f54c8a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54c41103b0 0x7f54c41a7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:57.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.530+0000 7f54c3e00640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54c40761d0 0x7f54c41a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:57.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.530+0000 7f54c3400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54c4076b20 0x7f54c41a0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:57590/0 (socket says 172.21.15.19:57590) 2024-05-07T02:31:57.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.530+0000 7f54c3400640 1 -- 172.21.15.19:0/565111391 learned_addr learned my addr 172.21.15.19:0/565111391 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:31:57.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.531+0000 7f54c3e00640 1 -- 172.21.15.19:0/565111391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54c4076b20 msgr2=0x7f54c41a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:57.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.531+0000 7f54c3e00640 1 --2- 172.21.15.19:0/565111391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54c4076b20 0x7f54c41a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:57.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.531+0000 7f54c3e00640 1 -- 172.21.15.19:0/565111391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54c41103b0 msgr2=0x7f54c41a7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:57.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.531+0000 7f54c3e00640 1 --2- 172.21.15.19:0/565111391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54c41103b0 0x7f54c41a7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:57.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.531+0000 7f54c3e00640 1 -- 172.21.15.19:0/565111391 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54c41a83a0 con 0x7f54c40761d0 2024-05-07T02:31:57.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.531+0000 7f54c3e00640 1 --2- 172.21.15.19:0/565111391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54c40761d0 0x7f54c41a0860 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f54b4005fd0 tx=0x7f54b400ec20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:57.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.531+0000 7f54c0c00640 1 -- 172.21.15.19:0/565111391 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b400ccc0 con 0x7f54c40761d0 2024-05-07T02:31:57.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.531+0000 7f54caa00640 1 -- 172.21.15.19:0/565111391 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54c41a8690 con 0x7f54c40761d0 2024-05-07T02:31:57.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.532+0000 7f54c0c00640 1 -- 172.21.15.19:0/565111391 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54b400ee70 con 0x7f54c40761d0 2024-05-07T02:31:57.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.532+0000 7f54c0c00640 1 -- 172.21.15.19:0/565111391 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b4018750 con 0x7f54c40761d0 2024-05-07T02:31:57.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.532+0000 7f54caa00640 1 -- 172.21.15.19:0/565111391 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54c41a8c30 con 0x7f54c40761d0 2024-05-07T02:31:57.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.533+0000 7f54caa00640 1 -- 172.21.15.19:0/565111391 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54c4105f30 con 0x7f54c40761d0 2024-05-07T02:31:57.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.538+0000 7f54c0c00640 1 -- 172.21.15.19:0/565111391 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f54b4010430 con 0x7f54c40761d0 2024-05-07T02:31:57.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.539+0000 7f54c0c00640 1 --2- 172.21.15.19:0/565111391 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f54a80b2b10 0x7f54a80b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:31:57.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.539+0000 7f54c0c00640 1 -- 172.21.15.19:0/565111391 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f54b40e2f10 con 0x7f54c40761d0 2024-05-07T02:31:57.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.540+0000 7f54c3400640 1 --2- 172.21.15.19:0/565111391 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f54a80b2b10 0x7f54a80b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:31:57.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.540+0000 7f54c0c00640 1 -- 172.21.15.19:0/565111391 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f54b40a9d40 con 0x7f54c40761d0 2024-05-07T02:31:57.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.541+0000 7f54c3400640 1 --2- 172.21.15.19:0/565111391 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f54a80b2b10 0x7f54a80b5020 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f54b00303f0 tx=0x7f54b0030470 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:31:57.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.776+0000 7f54caa00640 1 -- 172.21.15.19:0/565111391 --> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f54c4063110 con 0x7f54a80b2b10 2024-05-07T02:31:57.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.790+0000 7f54c0c00640 1 -- 172.21.15.19:0/565111391 <== mgr.25234 v2:172.21.15.19:6800/3265825478 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339447 (secure 0 0 0) 0x7f54c4063110 con 0x7f54a80b2b10 2024-05-07T02:31:57.796 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:31:57.797 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-05-07T02:31:57.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.797+0000 7f54caa00640 1 -- 172.21.15.19:0/565111391 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f54a80b2b10 msgr2=0x7f54a80b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:57.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.797+0000 7f54caa00640 1 --2- 172.21.15.19:0/565111391 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f54a80b2b10 0x7f54a80b5020 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f54b00303f0 tx=0x7f54b0030470 comp rx=0 tx=0).stop 2024-05-07T02:31:57.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.797+0000 7f54caa00640 1 -- 172.21.15.19:0/565111391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54c40761d0 msgr2=0x7f54c41a0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:31:57.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.797+0000 7f54caa00640 1 --2- 172.21.15.19:0/565111391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54c40761d0 0x7f54c41a0860 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f54b4005fd0 tx=0x7f54b400ec20 comp rx=0 tx=0).stop 2024-05-07T02:31:57.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.798+0000 7f54caa00640 1 -- 172.21.15.19:0/565111391 shutdown_connections 2024-05-07T02:31:57.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.798+0000 7f54caa00640 1 --2- 172.21.15.19:0/565111391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54c41103b0 0x7f54c41a7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:57.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.798+0000 7f54caa00640 1 --2- 172.21.15.19:0/565111391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54c4076b20 0x7f54c41a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:57.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.798+0000 7f54caa00640 1 --2- 172.21.15.19:0/565111391 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f54a80b2b10 0x7f54a80b5020 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:57.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.798+0000 7f54caa00640 1 --2- 172.21.15.19:0/565111391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f54c40761d0 0x7f54c41a0860 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:31:57.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.798+0000 7f54caa00640 1 -- 172.21.15.19:0/565111391 >> 172.21.15.19:0/565111391 conn(0x7f54c40fee60 msgr2=0x7f54c4103100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:31:57.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.798+0000 7f54caa00640 1 -- 172.21.15.19:0/565111391 shutdown_connections 2024-05-07T02:31:57.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:31:57.798+0000 7f54caa00640 1 -- 172.21.15.19:0/565111391 wait complete. 2024-05-07T02:31:57.820 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-05-07T02:31:55.850763+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1198131,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2682,"num_read_kb":1494,"num_write":513,"num_write_kb":3633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":906,"ondisk_log_size":906,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":377672,"kb_used_data":16004,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124564152,"statfs":{"total":1151940427776,"available":1151553691648,"internally_reserved":0,"allocated":16388096,"data_stored":9868320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.978312"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.093082+0000","last_change":"2024-05-07T02:26:11.876828+0000","last_active":"2024-05-07T02:31:50.093082+0000","last_peered":"2024-05-07T02:31:50.093082+0000","last_clean":"2024-05-07T02:31:50.093082+0000","last_became_active":"2024-05-07T02:26:11.876319+0000","last_became_peered":"2024-05-07T02:26:11.876319+0000","last_unstale":"2024-05-07T02:31:50.093082+0000","last_undegraded":"2024-05-07T02:31:50.093082+0000","last_fullsized":"2024-05-07T02:31:50.093082+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:07:33.181373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.094443+0000","last_change":"2024-05-07T02:26:11.873388+0000","last_active":"2024-05-07T02:31:50.094443+0000","last_peered":"2024-05-07T02:31:50.094443+0000","last_clean":"2024-05-07T02:31:50.094443+0000","last_became_active":"2024-05-07T02:26:11.869447+0000","last_became_peered":"2024-05-07T02:26:11.869447+0000","last_unstale":"2024-05-07T02:31:50.094443+0000","last_undegraded":"2024-05-07T02:31:50.094443+0000","last_fullsized":"2024-05-07T02:31:50.094443+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:09:52.876547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.094158+0000","last_change":"2024-05-07T02:26:11.870797+0000","last_active":"2024-05-07T02:31:50.094158+0000","last_peered":"2024-05-07T02:31:50.094158+0000","last_clean":"2024-05-07T02:31:50.094158+0000","last_became_active":"2024-05-07T02:26:11.870411+0000","last_became_peered":"2024-05-07T02:26:11.870411+0000","last_unstale":"2024-05-07T02:31:50.094158+0000","last_undegraded":"2024-05-07T02:31:50.094158+0000","last_fullsized":"2024-05-07T02:31:50.094158+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:34:12.017508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.826540+0000","last_change":"2024-05-07T02:26:11.867327+0000","last_active":"2024-05-07T02:31:49.826540+0000","last_peered":"2024-05-07T02:31:49.826540+0000","last_clean":"2024-05-07T02:31:49.826540+0000","last_became_active":"2024-05-07T02:26:11.866912+0000","last_became_peered":"2024-05-07T02:26:11.866912+0000","last_unstale":"2024-05-07T02:31:49.826540+0000","last_undegraded":"2024-05-07T02:31:49.826540+0000","last_fullsized":"2024-05-07T02:31:49.826540+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:36:36.616706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.988361+0000","last_change":"2024-05-07T02:26:11.878614+0000","last_active":"2024-05-07T02:31:49.988361+0000","last_peered":"2024-05-07T02:31:49.988361+0000","last_clean":"2024-05-07T02:31:49.988361+0000","last_became_active":"2024-05-07T02:26:11.878069+0000","last_became_peered":"2024-05-07T02:26:11.878069+0000","last_unstale":"2024-05-07T02:31:49.988361+0000","last_undegraded":"2024-05-07T02:31:49.988361+0000","last_fullsized":"2024-05-07T02:31:49.988361+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:40:15.991433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.003199+0000","last_change":"2024-05-07T02:26:11.877755+0000","last_active":"2024-05-07T02:31:50.003199+0000","last_peered":"2024-05-07T02:31:50.003199+0000","last_clean":"2024-05-07T02:31:50.003199+0000","last_became_active":"2024-05-07T02:26:11.877373+0000","last_became_peered":"2024-05-07T02:26:11.877373+0000","last_unstale":"2024-05-07T02:31:50.003199+0000","last_undegraded":"2024-05-07T02:31:50.003199+0000","last_fullsized":"2024-05-07T02:31:50.003199+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:49:55.984807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.136082+0000","last_change":"2024-05-07T02:26:11.882194+0000","last_active":"2024-05-07T02:31:50.136082+0000","last_peered":"2024-05-07T02:31:50.136082+0000","last_clean":"2024-05-07T02:31:50.136082+0000","last_became_active":"2024-05-07T02:26:11.881873+0000","last_became_peered":"2024-05-07T02:26:11.881873+0000","last_unstale":"2024-05-07T02:31:50.136082+0000","last_undegraded":"2024-05-07T02:31:50.136082+0000","last_fullsized":"2024-05-07T02:31:50.136082+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:47:31.000357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.823872+0000","last_change":"2024-05-07T02:26:11.857366+0000","last_active":"2024-05-07T02:31:49.823872+0000","last_peered":"2024-05-07T02:31:49.823872+0000","last_clean":"2024-05-07T02:31:49.823872+0000","last_became_active":"2024-05-07T02:26:11.854054+0000","last_became_peered":"2024-05-07T02:26:11.854054+0000","last_unstale":"2024-05-07T02:31:49.823872+0000","last_undegraded":"2024-05-07T02:31:49.823872+0000","last_fullsized":"2024-05-07T02:31:49.823872+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:07:04.834022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092885+0000","last_change":"2024-05-07T02:26:11.867105+0000","last_active":"2024-05-07T02:31:50.092885+0000","last_peered":"2024-05-07T02:31:50.092885+0000","last_clean":"2024-05-07T02:31:50.092885+0000","last_became_active":"2024-05-07T02:26:11.865986+0000","last_became_peered":"2024-05-07T02:26:11.865986+0000","last_unstale":"2024-05-07T02:31:50.092885+0000","last_undegraded":"2024-05-07T02:31:50.092885+0000","last_fullsized":"2024-05-07T02:31:50.092885+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:13:01.295354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.003769+0000","last_change":"2024-05-07T02:26:11.868039+0000","last_active":"2024-05-07T02:31:50.003769+0000","last_peered":"2024-05-07T02:31:50.003769+0000","last_clean":"2024-05-07T02:31:50.003769+0000","last_became_active":"2024-05-07T02:26:11.867588+0000","last_became_peered":"2024-05-07T02:26:11.867588+0000","last_unstale":"2024-05-07T02:31:50.003769+0000","last_undegraded":"2024-05-07T02:31:50.003769+0000","last_fullsized":"2024-05-07T02:31:50.003769+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:41:45.149620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.136641+0000","last_change":"2024-05-07T02:26:11.859579+0000","last_active":"2024-05-07T02:31:50.136641+0000","last_peered":"2024-05-07T02:31:50.136641+0000","last_clean":"2024-05-07T02:31:50.136641+0000","last_became_active":"2024-05-07T02:26:11.859154+0000","last_became_peered":"2024-05-07T02:26:11.859154+0000","last_unstale":"2024-05-07T02:31:50.136641+0000","last_undegraded":"2024-05-07T02:31:50.136641+0000","last_fullsized":"2024-05-07T02:31:50.136641+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:26:40.616956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825795+0000","last_change":"2024-05-07T02:26:11.858220+0000","last_active":"2024-05-07T02:31:49.825795+0000","last_peered":"2024-05-07T02:31:49.825795+0000","last_clean":"2024-05-07T02:31:49.825795+0000","last_became_active":"2024-05-07T02:26:11.857826+0000","last_became_peered":"2024-05-07T02:26:11.857826+0000","last_unstale":"2024-05-07T02:31:49.825795+0000","last_undegraded":"2024-05-07T02:31:49.825795+0000","last_fullsized":"2024-05-07T02:31:49.825795+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:21:36.031057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.097826+0000","last_change":"2024-05-07T02:26:11.880348+0000","last_active":"2024-05-07T02:31:50.097826+0000","last_peered":"2024-05-07T02:31:50.097826+0000","last_clean":"2024-05-07T02:31:50.097826+0000","last_became_active":"2024-05-07T02:26:11.879633+0000","last_became_peered":"2024-05-07T02:26:11.879633+0000","last_unstale":"2024-05-07T02:31:50.097826+0000","last_undegraded":"2024-05-07T02:31:50.097826+0000","last_fullsized":"2024-05-07T02:31:50.097826+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:59:34.567973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.093652+0000","last_change":"2024-05-07T02:26:11.865687+0000","last_active":"2024-05-07T02:31:50.093652+0000","last_peered":"2024-05-07T02:31:50.093652+0000","last_clean":"2024-05-07T02:31:50.093652+0000","last_became_active":"2024-05-07T02:26:11.865330+0000","last_became_peered":"2024-05-07T02:26:11.865330+0000","last_unstale":"2024-05-07T02:31:50.093652+0000","last_undegraded":"2024-05-07T02:31:50.093652+0000","last_fullsized":"2024-05-07T02:31:50.093652+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:36:10.610314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.093991+0000","last_change":"2024-05-07T02:26:11.870627+0000","last_active":"2024-05-07T02:31:50.093991+0000","last_peered":"2024-05-07T02:31:50.093991+0000","last_clean":"2024-05-07T02:31:50.093991+0000","last_became_active":"2024-05-07T02:26:11.870229+0000","last_became_peered":"2024-05-07T02:26:11.870229+0000","last_unstale":"2024-05-07T02:31:50.093991+0000","last_undegraded":"2024-05-07T02:31:50.093991+0000","last_fullsized":"2024-05-07T02:31:50.093991+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:06:46.008651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092311+0000","last_change":"2024-05-07T02:26:11.868698+0000","last_active":"2024-05-07T02:31:50.092311+0000","last_peered":"2024-05-07T02:31:50.092311+0000","last_clean":"2024-05-07T02:31:50.092311+0000","last_became_active":"2024-05-07T02:26:11.868294+0000","last_became_peered":"2024-05-07T02:26:11.868294+0000","last_unstale":"2024-05-07T02:31:50.092311+0000","last_undegraded":"2024-05-07T02:31:50.092311+0000","last_fullsized":"2024-05-07T02:31:50.092311+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:16:22.856200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092865+0000","last_change":"2024-05-07T02:26:11.877116+0000","last_active":"2024-05-07T02:31:50.092865+0000","last_peered":"2024-05-07T02:31:50.092865+0000","last_clean":"2024-05-07T02:31:50.092865+0000","last_became_active":"2024-05-07T02:26:11.876305+0000","last_became_peered":"2024-05-07T02:26:11.876305+0000","last_unstale":"2024-05-07T02:31:50.092865+0000","last_undegraded":"2024-05-07T02:31:50.092865+0000","last_fullsized":"2024-05-07T02:31:50.092865+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:25:46.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.985509+0000","last_change":"2024-05-07T02:26:11.877488+0000","last_active":"2024-05-07T02:31:49.985509+0000","last_peered":"2024-05-07T02:31:49.985509+0000","last_clean":"2024-05-07T02:31:49.985509+0000","last_became_active":"2024-05-07T02:26:11.874500+0000","last_became_peered":"2024-05-07T02:26:11.874500+0000","last_unstale":"2024-05-07T02:31:49.985509+0000","last_undegraded":"2024-05-07T02:31:49.985509+0000","last_fullsized":"2024-05-07T02:31:49.985509+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:53:50.046342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825931+0000","last_change":"2024-05-07T02:26:11.876500+0000","last_active":"2024-05-07T02:31:49.825931+0000","last_peered":"2024-05-07T02:31:49.825931+0000","last_clean":"2024-05-07T02:31:49.825931+0000","last_became_active":"2024-05-07T02:26:11.876084+0000","last_became_peered":"2024-05-07T02:26:11.876084+0000","last_unstale":"2024-05-07T02:31:49.825931+0000","last_undegraded":"2024-05-07T02:31:49.825931+0000","last_fullsized":"2024-05-07T02:31:49.825931+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:05:05.681488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825186+0000","last_change":"2024-05-07T02:26:25.579209+0000","last_active":"2024-05-07T02:31:49.825186+0000","last_peered":"2024-05-07T02:31:49.825186+0000","last_clean":"2024-05-07T02:31:49.825186+0000","last_became_active":"2024-05-07T02:26:25.578665+0000","last_became_peered":"2024-05-07T02:26:25.578665+0000","last_unstale":"2024-05-07T02:31:49.825186+0000","last_undegraded":"2024-05-07T02:31:49.825186+0000","last_fullsized":"2024-05-07T02:31:49.825186+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:06:29.784016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'15","reported_seq":310,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.215535+0000","last_change":"2024-05-07T02:26:07.792993+0000","last_active":"2024-05-07T02:31:50.215535+0000","last_peered":"2024-05-07T02:31:50.215535+0000","last_clean":"2024-05-07T02:31:50.215535+0000","last_became_active":"2024-05-07T02:26:07.792402+0000","last_became_peered":"2024-05-07T02:26:07.792402+0000","last_unstale":"2024-05-07T02:31:50.215535+0000","last_undegraded":"2024-05-07T02:31:50.215535+0000","last_fullsized":"2024-05-07T02:31:50.215535+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:49:02.288255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":247,"num_read_kb":247,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.985441+0000","last_change":"2024-05-07T02:28:33.869804+0000","last_active":"2024-05-07T02:31:49.985441+0000","last_peered":"2024-05-07T02:31:49.985441+0000","last_clean":"2024-05-07T02:31:49.985441+0000","last_became_active":"2024-05-07T02:28:33.869267+0000","last_became_peered":"2024-05-07T02:28:33.869267+0000","last_unstale":"2024-05-07T02:31:49.985441+0000","last_undegraded":"2024-05-07T02:31:49.985441+0000","last_fullsized":"2024-05-07T02:31:49.985441+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:10:57.384546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825312+0000","last_change":"2024-05-07T02:26:25.579835+0000","last_active":"2024-05-07T02:31:49.825312+0000","last_peered":"2024-05-07T02:31:49.825312+0000","last_clean":"2024-05-07T02:31:49.825312+0000","last_became_active":"2024-05-07T02:26:25.579105+0000","last_became_peered":"2024-05-07T02:26:25.579105+0000","last_unstale":"2024-05-07T02:31:49.825312+0000","last_undegraded":"2024-05-07T02:31:49.825312+0000","last_fullsized":"2024-05-07T02:31:49.825312+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:32:37.745967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825500+0000","last_change":"2024-05-07T02:26:08.001879+0000","last_active":"2024-05-07T02:31:49.825500+0000","last_peered":"2024-05-07T02:31:49.825500+0000","last_clean":"2024-05-07T02:31:49.825500+0000","last_became_active":"2024-05-07T02:26:08.000244+0000","last_became_peered":"2024-05-07T02:26:08.000244+0000","last_unstale":"2024-05-07T02:31:49.825500+0000","last_undegraded":"2024-05-07T02:31:49.825500+0000","last_fullsized":"2024-05-07T02:31:49.825500+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:56:42.591923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.094664+0000","last_change":"2024-05-07T02:26:11.882351+0000","last_active":"2024-05-07T02:31:50.094664+0000","last_peered":"2024-05-07T02:31:50.094664+0000","last_clean":"2024-05-07T02:31:50.094664+0000","last_became_active":"2024-05-07T02:26:11.881922+0000","last_became_peered":"2024-05-07T02:26:11.881922+0000","last_unstale":"2024-05-07T02:31:50.094664+0000","last_undegraded":"2024-05-07T02:31:50.094664+0000","last_fullsized":"2024-05-07T02:31:50.094664+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:58:31.993859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825489+0000","last_change":"2024-05-07T02:26:25.580419+0000","last_active":"2024-05-07T02:31:49.825489+0000","last_peered":"2024-05-07T02:31:49.825489+0000","last_clean":"2024-05-07T02:31:49.825489+0000","last_became_active":"2024-05-07T02:26:25.579411+0000","last_became_peered":"2024-05-07T02:26:25.579411+0000","last_unstale":"2024-05-07T02:31:49.825489+0000","last_undegraded":"2024-05-07T02:31:49.825489+0000","last_fullsized":"2024-05-07T02:31:49.825489+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:43:03.552210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092822+0000","last_change":"2024-05-07T02:27:32.588255+0000","last_active":"2024-05-07T02:31:50.092822+0000","last_peered":"2024-05-07T02:31:50.092822+0000","last_clean":"2024-05-07T02:31:50.092822+0000","last_became_active":"2024-05-07T02:27:32.584405+0000","last_became_peered":"2024-05-07T02:27:32.584405+0000","last_unstale":"2024-05-07T02:31:50.092822+0000","last_undegraded":"2024-05-07T02:31:50.092822+0000","last_fullsized":"2024-05-07T02:31:50.092822+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:07:22.980041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092515+0000","last_change":"2024-05-07T02:26:11.866949+0000","last_active":"2024-05-07T02:31:50.092515+0000","last_peered":"2024-05-07T02:31:50.092515+0000","last_clean":"2024-05-07T02:31:50.092515+0000","last_became_active":"2024-05-07T02:26:11.866453+0000","last_became_peered":"2024-05-07T02:26:11.866453+0000","last_unstale":"2024-05-07T02:31:50.092515+0000","last_undegraded":"2024-05-07T02:31:50.092515+0000","last_fullsized":"2024-05-07T02:31:50.092515+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T14:22:47.291470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825287+0000","last_change":"2024-05-07T02:26:25.570555+0000","last_active":"2024-05-07T02:31:49.825287+0000","last_peered":"2024-05-07T02:31:49.825287+0000","last_clean":"2024-05-07T02:31:49.825287+0000","last_became_active":"2024-05-07T02:26:25.570044+0000","last_became_peered":"2024-05-07T02:26:25.570044+0000","last_unstale":"2024-05-07T02:31:49.825287+0000","last_undegraded":"2024-05-07T02:31:49.825287+0000","last_fullsized":"2024-05-07T02:31:49.825287+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:48:15.701335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.988509+0000","last_change":"2024-05-07T02:26:07.785938+0000","last_active":"2024-05-07T02:31:49.988509+0000","last_peered":"2024-05-07T02:31:49.988509+0000","last_clean":"2024-05-07T02:31:49.988509+0000","last_became_active":"2024-05-07T02:26:07.785573+0000","last_became_peered":"2024-05-07T02:26:07.785573+0000","last_unstale":"2024-05-07T02:31:49.988509+0000","last_undegraded":"2024-05-07T02:31:49.988509+0000","last_fullsized":"2024-05-07T02:31:49.988509+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:33:24.042227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.999531+0000","last_change":"2024-05-07T02:26:11.858732+0000","last_active":"2024-05-07T02:31:49.999531+0000","last_peered":"2024-05-07T02:31:49.999531+0000","last_clean":"2024-05-07T02:31:49.999531+0000","last_became_active":"2024-05-07T02:26:11.858270+0000","last_became_peered":"2024-05-07T02:26:11.858270+0000","last_unstale":"2024-05-07T02:31:49.999531+0000","last_undegraded":"2024-05-07T02:31:49.999531+0000","last_fullsized":"2024-05-07T02:31:49.999531+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:03:53.662296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825701+0000","last_change":"2024-05-07T02:26:25.559025+0000","last_active":"2024-05-07T02:31:49.825701+0000","last_peered":"2024-05-07T02:31:49.825701+0000","last_clean":"2024-05-07T02:31:49.825701+0000","last_became_active":"2024-05-07T02:26:25.558502+0000","last_became_peered":"2024-05-07T02:26:25.558502+0000","last_unstale":"2024-05-07T02:31:49.825701+0000","last_undegraded":"2024-05-07T02:31:49.825701+0000","last_fullsized":"2024-05-07T02:31:49.825701+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:26:59.846341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":302,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.147359+0000","last_change":"2024-05-07T02:26:07.783386+0000","last_active":"2024-05-07T02:31:50.147359+0000","last_peered":"2024-05-07T02:31:50.147359+0000","last_clean":"2024-05-07T02:31:50.147359+0000","last_became_active":"2024-05-07T02:26:07.782823+0000","last_became_peered":"2024-05-07T02:26:07.782823+0000","last_unstale":"2024-05-07T02:31:50.147359+0000","last_undegraded":"2024-05-07T02:31:50.147359+0000","last_fullsized":"2024-05-07T02:31:50.147359+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:40:29.350809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":248,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.988372+0000","last_change":"2024-05-07T02:26:11.877426+0000","last_active":"2024-05-07T02:31:49.988372+0000","last_peered":"2024-05-07T02:31:49.988372+0000","last_clean":"2024-05-07T02:31:49.988372+0000","last_became_active":"2024-05-07T02:26:11.876976+0000","last_became_peered":"2024-05-07T02:26:11.876976+0000","last_unstale":"2024-05-07T02:31:49.988372+0000","last_undegraded":"2024-05-07T02:31:49.988372+0000","last_fullsized":"2024-05-07T02:31:49.988372+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:18:56.077064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825760+0000","last_change":"2024-05-07T02:27:32.625394+0000","last_active":"2024-05-07T02:31:49.825760+0000","last_peered":"2024-05-07T02:31:49.825760+0000","last_clean":"2024-05-07T02:31:49.825760+0000","last_became_active":"2024-05-07T02:27:32.624826+0000","last_became_peered":"2024-05-07T02:27:32.624826+0000","last_unstale":"2024-05-07T02:31:49.825760+0000","last_undegraded":"2024-05-07T02:31:49.825760+0000","last_fullsized":"2024-05-07T02:31:49.825760+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:16:49.694003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.003155+0000","last_change":"2024-05-07T02:26:07.786388+0000","last_active":"2024-05-07T02:31:50.003155+0000","last_peered":"2024-05-07T02:31:50.003155+0000","last_clean":"2024-05-07T02:31:50.003155+0000","last_became_active":"2024-05-07T02:26:07.784130+0000","last_became_peered":"2024-05-07T02:26:07.784130+0000","last_unstale":"2024-05-07T02:31:50.003155+0000","last_undegraded":"2024-05-07T02:31:50.003155+0000","last_fullsized":"2024-05-07T02:31:50.003155+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:44:55.229815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.999374+0000","last_change":"2024-05-07T02:28:33.886551+0000","last_active":"2024-05-07T02:31:49.999374+0000","last_peered":"2024-05-07T02:31:49.999374+0000","last_clean":"2024-05-07T02:31:49.999374+0000","last_became_active":"2024-05-07T02:28:33.886249+0000","last_became_peered":"2024-05-07T02:28:33.886249+0000","last_unstale":"2024-05-07T02:31:49.999374+0000","last_undegraded":"2024-05-07T02:31:49.999374+0000","last_fullsized":"2024-05-07T02:31:49.999374+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:56:19.851347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092828+0000","last_change":"2024-05-07T02:26:25.584396+0000","last_active":"2024-05-07T02:31:50.092828+0000","last_peered":"2024-05-07T02:31:50.092828+0000","last_clean":"2024-05-07T02:31:50.092828+0000","last_became_active":"2024-05-07T02:26:25.583917+0000","last_became_peered":"2024-05-07T02:26:25.583917+0000","last_unstale":"2024-05-07T02:31:50.092828+0000","last_undegraded":"2024-05-07T02:31:50.092828+0000","last_fullsized":"2024-05-07T02:31:50.092828+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:56:34.495133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.097562+0000","last_change":"2024-05-07T02:26:07.795300+0000","last_active":"2024-05-07T02:31:50.097562+0000","last_peered":"2024-05-07T02:31:50.097562+0000","last_clean":"2024-05-07T02:31:50.097562+0000","last_became_active":"2024-05-07T02:26:07.794633+0000","last_became_peered":"2024-05-07T02:26:07.794633+0000","last_unstale":"2024-05-07T02:31:50.097562+0000","last_undegraded":"2024-05-07T02:31:50.097562+0000","last_fullsized":"2024-05-07T02:31:50.097562+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:36:44.252118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.824253+0000","last_change":"2024-05-07T02:26:11.877717+0000","last_active":"2024-05-07T02:31:49.824253+0000","last_peered":"2024-05-07T02:31:49.824253+0000","last_clean":"2024-05-07T02:31:49.824253+0000","last_became_active":"2024-05-07T02:26:11.876644+0000","last_became_peered":"2024-05-07T02:26:11.876644+0000","last_unstale":"2024-05-07T02:31:49.824253+0000","last_undegraded":"2024-05-07T02:31:49.824253+0000","last_fullsized":"2024-05-07T02:31:49.824253+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T05:20:36.176776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.094315+0000","last_change":"2024-05-07T02:26:25.572120+0000","last_active":"2024-05-07T02:31:50.094315+0000","last_peered":"2024-05-07T02:31:50.094315+0000","last_clean":"2024-05-07T02:31:50.094315+0000","last_became_active":"2024-05-07T02:26:25.571637+0000","last_became_peered":"2024-05-07T02:26:25.571637+0000","last_unstale":"2024-05-07T02:31:50.094315+0000","last_undegraded":"2024-05-07T02:31:50.094315+0000","last_fullsized":"2024-05-07T02:31:50.094315+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:03:54.574061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.999043+0000","last_change":"2024-05-07T02:26:07.784160+0000","last_active":"2024-05-07T02:31:49.999043+0000","last_peered":"2024-05-07T02:31:49.999043+0000","last_clean":"2024-05-07T02:31:49.999043+0000","last_became_active":"2024-05-07T02:26:07.783646+0000","last_became_peered":"2024-05-07T02:26:07.783646+0000","last_unstale":"2024-05-07T02:31:49.999043+0000","last_undegraded":"2024-05-07T02:31:49.999043+0000","last_fullsized":"2024-05-07T02:31:49.999043+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T14:25:37.906587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.984924+0000","last_change":"2024-05-07T02:26:11.874886+0000","last_active":"2024-05-07T02:31:49.984924+0000","last_peered":"2024-05-07T02:31:49.984924+0000","last_clean":"2024-05-07T02:31:49.984924+0000","last_became_active":"2024-05-07T02:26:11.874465+0000","last_became_peered":"2024-05-07T02:26:11.874465+0000","last_unstale":"2024-05-07T02:31:49.984924+0000","last_undegraded":"2024-05-07T02:31:49.984924+0000","last_fullsized":"2024-05-07T02:31:49.984924+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:57:19.530747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.824559+0000","last_change":"2024-05-07T02:26:25.558087+0000","last_active":"2024-05-07T02:31:49.824559+0000","last_peered":"2024-05-07T02:31:49.824559+0000","last_clean":"2024-05-07T02:31:49.824559+0000","last_became_active":"2024-05-07T02:26:25.557624+0000","last_became_peered":"2024-05-07T02:26:25.557624+0000","last_unstale":"2024-05-07T02:31:49.824559+0000","last_undegraded":"2024-05-07T02:31:49.824559+0000","last_fullsized":"2024-05-07T02:31:49.824559+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:03:20.104547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'6","reported_seq":546,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.219941+0000","last_change":"2024-05-07T02:26:07.788244+0000","last_active":"2024-05-07T02:31:50.219941+0000","last_peered":"2024-05-07T02:31:50.219941+0000","last_clean":"2024-05-07T02:31:50.219941+0000","last_became_active":"2024-05-07T02:26:07.787789+0000","last_became_peered":"2024-05-07T02:26:07.787789+0000","last_unstale":"2024-05-07T02:31:50.219941+0000","last_undegraded":"2024-05-07T02:31:50.219941+0000","last_fullsized":"2024-05-07T02:31:50.219941+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:12:22.943444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":492,"num_read_kb":492,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092921+0000","last_change":"2024-05-07T02:26:11.879821+0000","last_active":"2024-05-07T02:31:50.092921+0000","last_peered":"2024-05-07T02:31:50.092921+0000","last_clean":"2024-05-07T02:31:50.092921+0000","last_became_active":"2024-05-07T02:26:11.879372+0000","last_became_peered":"2024-05-07T02:26:11.879372+0000","last_unstale":"2024-05-07T02:31:50.092921+0000","last_undegraded":"2024-05-07T02:31:50.092921+0000","last_fullsized":"2024-05-07T02:31:50.092921+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:01:03.754890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.985163+0000","last_change":"2024-05-07T02:26:25.576663+0000","last_active":"2024-05-07T02:31:49.985163+0000","last_peered":"2024-05-07T02:31:49.985163+0000","last_clean":"2024-05-07T02:31:49.985163+0000","last_became_active":"2024-05-07T02:26:25.575729+0000","last_became_peered":"2024-05-07T02:26:25.575729+0000","last_unstale":"2024-05-07T02:31:49.985163+0000","last_undegraded":"2024-05-07T02:31:49.985163+0000","last_fullsized":"2024-05-07T02:31:49.985163+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:03:48.061385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.136515+0000","last_change":"2024-05-07T02:27:32.625336+0000","last_active":"2024-05-07T02:31:50.136515+0000","last_peered":"2024-05-07T02:31:50.136515+0000","last_clean":"2024-05-07T02:31:50.136515+0000","last_became_active":"2024-05-07T02:27:32.624805+0000","last_became_peered":"2024-05-07T02:27:32.624805+0000","last_unstale":"2024-05-07T02:31:50.136515+0000","last_undegraded":"2024-05-07T02:31:50.136515+0000","last_fullsized":"2024-05-07T02:31:50.136515+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:10:04.928297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.094830+0000","last_change":"2024-05-07T02:27:32.626374+0000","last_active":"2024-05-07T02:31:50.094830+0000","last_peered":"2024-05-07T02:31:50.094830+0000","last_clean":"2024-05-07T02:31:50.094830+0000","last_became_active":"2024-05-07T02:27:32.625489+0000","last_became_peered":"2024-05-07T02:27:32.625489+0000","last_unstale":"2024-05-07T02:31:50.094830+0000","last_undegraded":"2024-05-07T02:31:50.094830+0000","last_fullsized":"2024-05-07T02:31:50.094830+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T05:56:16.626041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825653+0000","last_change":"2024-05-07T02:26:25.580811+0000","last_active":"2024-05-07T02:31:49.825653+0000","last_peered":"2024-05-07T02:31:49.825653+0000","last_clean":"2024-05-07T02:31:49.825653+0000","last_became_active":"2024-05-07T02:26:25.579344+0000","last_became_peered":"2024-05-07T02:26:25.579344+0000","last_unstale":"2024-05-07T02:31:49.825653+0000","last_undegraded":"2024-05-07T02:31:49.825653+0000","last_fullsized":"2024-05-07T02:31:49.825653+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:25:36.186899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.999415+0000","last_change":"2024-05-07T02:26:08.000010+0000","last_active":"2024-05-07T02:31:49.999415+0000","last_peered":"2024-05-07T02:31:49.999415+0000","last_clean":"2024-05-07T02:31:49.999415+0000","last_became_active":"2024-05-07T02:26:07.999576+0000","last_became_peered":"2024-05-07T02:26:07.999576+0000","last_unstale":"2024-05-07T02:31:49.999415+0000","last_undegraded":"2024-05-07T02:31:49.999415+0000","last_fullsized":"2024-05-07T02:31:49.999415+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:23:38.644205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092735+0000","last_change":"2024-05-07T02:26:11.867098+0000","last_active":"2024-05-07T02:31:50.092735+0000","last_peered":"2024-05-07T02:31:50.092735+0000","last_clean":"2024-05-07T02:31:50.092735+0000","last_became_active":"2024-05-07T02:26:11.866036+0000","last_became_peered":"2024-05-07T02:26:11.866036+0000","last_unstale":"2024-05-07T02:31:50.092735+0000","last_undegraded":"2024-05-07T02:31:50.092735+0000","last_fullsized":"2024-05-07T02:31:50.092735+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:47:22.412323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.985140+0000","last_change":"2024-05-07T02:27:32.628243+0000","last_active":"2024-05-07T02:31:49.985140+0000","last_peered":"2024-05-07T02:31:49.985140+0000","last_clean":"2024-05-07T02:31:49.985140+0000","last_became_active":"2024-05-07T02:27:32.627048+0000","last_became_peered":"2024-05-07T02:27:32.627048+0000","last_unstale":"2024-05-07T02:31:49.985140+0000","last_undegraded":"2024-05-07T02:31:49.985140+0000","last_fullsized":"2024-05-07T02:31:49.985140+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:33:49.802873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,10],"acting":[9,8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'44","reported_seq":597,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.148780+0000","last_change":"2024-05-07T02:26:07.786841+0000","last_active":"2024-05-07T02:31:50.148780+0000","last_peered":"2024-05-07T02:31:50.148780+0000","last_clean":"2024-05-07T02:31:50.148780+0000","last_became_active":"2024-05-07T02:26:07.783836+0000","last_became_peered":"2024-05-07T02:26:07.783836+0000","last_unstale":"2024-05-07T02:31:50.148780+0000","last_undegraded":"2024-05-07T02:31:50.148780+0000","last_fullsized":"2024-05-07T02:31:50.148780+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:28:53.390009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":73494,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":505,"num_read_kb":326,"num_write":46,"num_write_kb":94,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.988712+0000","last_change":"2024-05-07T02:26:11.857292+0000","last_active":"2024-05-07T02:31:49.988712+0000","last_peered":"2024-05-07T02:31:49.988712+0000","last_clean":"2024-05-07T02:31:49.988712+0000","last_became_active":"2024-05-07T02:26:11.856899+0000","last_became_peered":"2024-05-07T02:26:11.856899+0000","last_unstale":"2024-05-07T02:31:49.988712+0000","last_undegraded":"2024-05-07T02:31:49.988712+0000","last_fullsized":"2024-05-07T02:31:49.988712+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:45:42.279848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.999888+0000","last_change":"2024-05-07T02:26:25.570714+0000","last_active":"2024-05-07T02:31:49.999888+0000","last_peered":"2024-05-07T02:31:49.999888+0000","last_clean":"2024-05-07T02:31:49.999888+0000","last_became_active":"2024-05-07T02:26:25.570163+0000","last_became_peered":"2024-05-07T02:26:25.570163+0000","last_unstale":"2024-05-07T02:31:49.999888+0000","last_undegraded":"2024-05-07T02:31:49.999888+0000","last_fullsized":"2024-05-07T02:31:49.999888+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:55:51.903968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":59,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.826064+0000","last_change":"2024-05-07T02:26:07.785904+0000","last_active":"2024-05-07T02:31:49.826064+0000","last_peered":"2024-05-07T02:31:49.826064+0000","last_clean":"2024-05-07T02:31:49.826064+0000","last_became_active":"2024-05-07T02:26:07.785347+0000","last_became_peered":"2024-05-07T02:26:07.785347+0000","last_unstale":"2024-05-07T02:31:49.826064+0000","last_undegraded":"2024-05-07T02:31:49.826064+0000","last_fullsized":"2024-05-07T02:31:49.826064+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:42:05.604266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.097758+0000","last_change":"2024-05-07T02:26:11.878511+0000","last_active":"2024-05-07T02:31:50.097758+0000","last_peered":"2024-05-07T02:31:50.097758+0000","last_clean":"2024-05-07T02:31:50.097758+0000","last_became_active":"2024-05-07T02:26:11.878166+0000","last_became_peered":"2024-05-07T02:26:11.878166+0000","last_unstale":"2024-05-07T02:31:50.097758+0000","last_undegraded":"2024-05-07T02:31:50.097758+0000","last_fullsized":"2024-05-07T02:31:50.097758+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T08:14:40.436244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092718+0000","last_change":"2024-05-07T02:26:25.579387+0000","last_active":"2024-05-07T02:31:50.092718+0000","last_peered":"2024-05-07T02:31:50.092718+0000","last_clean":"2024-05-07T02:31:50.092718+0000","last_became_active":"2024-05-07T02:26:25.578757+0000","last_became_peered":"2024-05-07T02:26:25.578757+0000","last_unstale":"2024-05-07T02:31:50.092718+0000","last_undegraded":"2024-05-07T02:31:50.092718+0000","last_fullsized":"2024-05-07T02:31:50.092718+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:19:19.650941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.999163+0000","last_change":"2024-05-07T02:26:07.785154+0000","last_active":"2024-05-07T02:31:49.999163+0000","last_peered":"2024-05-07T02:31:49.999163+0000","last_clean":"2024-05-07T02:31:49.999163+0000","last_became_active":"2024-05-07T02:26:07.784589+0000","last_became_peered":"2024-05-07T02:26:07.784589+0000","last_unstale":"2024-05-07T02:31:49.999163+0000","last_undegraded":"2024-05-07T02:31:49.999163+0000","last_fullsized":"2024-05-07T02:31:49.999163+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:41:41.851182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.136271+0000","last_change":"2024-05-07T02:26:11.878162+0000","last_active":"2024-05-07T02:31:50.136271+0000","last_peered":"2024-05-07T02:31:50.136271+0000","last_clean":"2024-05-07T02:31:50.136271+0000","last_became_active":"2024-05-07T02:26:11.877770+0000","last_became_peered":"2024-05-07T02:26:11.877770+0000","last_unstale":"2024-05-07T02:31:50.136271+0000","last_undegraded":"2024-05-07T02:31:50.136271+0000","last_fullsized":"2024-05-07T02:31:50.136271+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:19:08.209398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.985823+0000","last_change":"2024-05-07T02:26:25.579465+0000","last_active":"2024-05-07T02:31:49.985823+0000","last_peered":"2024-05-07T02:31:49.985823+0000","last_clean":"2024-05-07T02:31:49.985823+0000","last_became_active":"2024-05-07T02:26:25.579003+0000","last_became_peered":"2024-05-07T02:26:25.579003+0000","last_unstale":"2024-05-07T02:31:49.985823+0000","last_undegraded":"2024-05-07T02:31:49.985823+0000","last_fullsized":"2024-05-07T02:31:49.985823+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:13:24.356366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'8","reported_seq":306,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.379222+0000","last_change":"2024-05-07T02:26:07.780827+0000","last_active":"2024-05-07T02:31:50.379222+0000","last_peered":"2024-05-07T02:31:50.379222+0000","last_clean":"2024-05-07T02:31:50.379222+0000","last_became_active":"2024-05-07T02:26:07.780353+0000","last_became_peered":"2024-05-07T02:26:07.780353+0000","last_unstale":"2024-05-07T02:31:50.379222+0000","last_undegraded":"2024-05-07T02:31:50.379222+0000","last_fullsized":"2024-05-07T02:31:50.379222+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:52:40.554135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1680,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":251,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.136134+0000","last_change":"2024-05-07T02:26:11.852516+0000","last_active":"2024-05-07T02:31:50.136134+0000","last_peered":"2024-05-07T02:31:50.136134+0000","last_clean":"2024-05-07T02:31:50.136134+0000","last_became_active":"2024-05-07T02:26:11.852112+0000","last_became_peered":"2024-05-07T02:26:11.852112+0000","last_unstale":"2024-05-07T02:31:50.136134+0000","last_undegraded":"2024-05-07T02:31:50.136134+0000","last_fullsized":"2024-05-07T02:31:50.136134+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:58:46.989250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.098786+0000","last_change":"2024-05-07T02:26:25.586726+0000","last_active":"2024-05-07T02:31:50.098786+0000","last_peered":"2024-05-07T02:31:50.098786+0000","last_clean":"2024-05-07T02:31:50.098786+0000","last_became_active":"2024-05-07T02:26:25.586267+0000","last_became_peered":"2024-05-07T02:26:25.586267+0000","last_unstale":"2024-05-07T02:31:50.098786+0000","last_undegraded":"2024-05-07T02:31:50.098786+0000","last_fullsized":"2024-05-07T02:31:50.098786+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:50:00.864463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092578+0000","last_change":"2024-05-07T02:27:32.599511+0000","last_active":"2024-05-07T02:31:50.092578+0000","last_peered":"2024-05-07T02:31:50.092578+0000","last_clean":"2024-05-07T02:31:50.092578+0000","last_became_active":"2024-05-07T02:27:32.590197+0000","last_became_peered":"2024-05-07T02:27:32.590197+0000","last_unstale":"2024-05-07T02:31:50.092578+0000","last_undegraded":"2024-05-07T02:31:50.092578+0000","last_fullsized":"2024-05-07T02:31:50.092578+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:23:07.401251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.999070+0000","last_change":"2024-05-07T02:26:11.857671+0000","last_active":"2024-05-07T02:31:49.999070+0000","last_peered":"2024-05-07T02:31:49.999070+0000","last_clean":"2024-05-07T02:31:49.999070+0000","last_became_active":"2024-05-07T02:26:11.857293+0000","last_became_peered":"2024-05-07T02:26:11.857293+0000","last_unstale":"2024-05-07T02:31:49.999070+0000","last_undegraded":"2024-05-07T02:31:49.999070+0000","last_fullsized":"2024-05-07T02:31:49.999070+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:48:42.267887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.999012+0000","last_change":"2024-05-07T02:26:25.563274+0000","last_active":"2024-05-07T02:31:49.999012+0000","last_peered":"2024-05-07T02:31:49.999012+0000","last_clean":"2024-05-07T02:31:49.999012+0000","last_became_active":"2024-05-07T02:26:25.562868+0000","last_became_peered":"2024-05-07T02:26:25.562868+0000","last_unstale":"2024-05-07T02:31:49.999012+0000","last_undegraded":"2024-05-07T02:31:49.999012+0000","last_fullsized":"2024-05-07T02:31:49.999012+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T14:20:03.210034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'4","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.988931+0000","last_change":"2024-05-07T02:26:07.784888+0000","last_active":"2024-05-07T02:31:49.988931+0000","last_peered":"2024-05-07T02:31:49.988931+0000","last_clean":"2024-05-07T02:31:49.988931+0000","last_became_active":"2024-05-07T02:26:07.784493+0000","last_became_peered":"2024-05-07T02:26:07.784493+0000","last_unstale":"2024-05-07T02:31:49.988931+0000","last_undegraded":"2024-05-07T02:31:49.988931+0000","last_fullsized":"2024-05-07T02:31:49.988931+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:21:14.054786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.093166+0000","last_change":"2024-05-07T02:26:11.858887+0000","last_active":"2024-05-07T02:31:50.093166+0000","last_peered":"2024-05-07T02:31:50.093166+0000","last_clean":"2024-05-07T02:31:50.093166+0000","last_became_active":"2024-05-07T02:26:11.858504+0000","last_became_peered":"2024-05-07T02:26:11.858504+0000","last_unstale":"2024-05-07T02:31:50.093166+0000","last_undegraded":"2024-05-07T02:31:50.093166+0000","last_fullsized":"2024-05-07T02:31:50.093166+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:53:20.290039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.988326+0000","last_change":"2024-05-07T02:27:32.627310+0000","last_active":"2024-05-07T02:31:49.988326+0000","last_peered":"2024-05-07T02:31:49.988326+0000","last_clean":"2024-05-07T02:31:49.988326+0000","last_became_active":"2024-05-07T02:27:32.626794+0000","last_became_peered":"2024-05-07T02:27:32.626794+0000","last_unstale":"2024-05-07T02:31:49.988326+0000","last_undegraded":"2024-05-07T02:31:49.988326+0000","last_fullsized":"2024-05-07T02:31:49.988326+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:06:30.466291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.985349+0000","last_change":"2024-05-07T02:26:08.000604+0000","last_active":"2024-05-07T02:31:49.985349+0000","last_peered":"2024-05-07T02:31:49.985349+0000","last_clean":"2024-05-07T02:31:49.985349+0000","last_became_active":"2024-05-07T02:26:08.000107+0000","last_became_peered":"2024-05-07T02:26:08.000107+0000","last_unstale":"2024-05-07T02:31:49.985349+0000","last_undegraded":"2024-05-07T02:31:49.985349+0000","last_fullsized":"2024-05-07T02:31:49.985349+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:58:06.716456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.003473+0000","last_change":"2024-05-07T02:26:11.876555+0000","last_active":"2024-05-07T02:31:50.003473+0000","last_peered":"2024-05-07T02:31:50.003473+0000","last_clean":"2024-05-07T02:31:50.003473+0000","last_became_active":"2024-05-07T02:26:11.876109+0000","last_became_peered":"2024-05-07T02:26:11.876109+0000","last_unstale":"2024-05-07T02:31:50.003473+0000","last_undegraded":"2024-05-07T02:31:50.003473+0000","last_fullsized":"2024-05-07T02:31:50.003473+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:10:24.995786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.098087+0000","last_change":"2024-05-07T02:26:25.567145+0000","last_active":"2024-05-07T02:31:50.098087+0000","last_peered":"2024-05-07T02:31:50.098087+0000","last_clean":"2024-05-07T02:31:50.098087+0000","last_became_active":"2024-05-07T02:26:25.566767+0000","last_became_peered":"2024-05-07T02:26:25.566767+0000","last_unstale":"2024-05-07T02:31:50.098087+0000","last_undegraded":"2024-05-07T02:31:50.098087+0000","last_fullsized":"2024-05-07T02:31:50.098087+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T05:10:59.333367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'8","reported_seq":306,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.260546+0000","last_change":"2024-05-07T02:26:07.789566+0000","last_active":"2024-05-07T02:31:50.260546+0000","last_peered":"2024-05-07T02:31:50.260546+0000","last_clean":"2024-05-07T02:31:50.260546+0000","last_became_active":"2024-05-07T02:26:07.789106+0000","last_became_peered":"2024-05-07T02:26:07.789106+0000","last_unstale":"2024-05-07T02:31:50.260546+0000","last_undegraded":"2024-05-07T02:31:50.260546+0000","last_fullsized":"2024-05-07T02:31:50.260546+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:45:13.620183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":250,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.003155+0000","last_change":"2024-05-07T02:26:11.869080+0000","last_active":"2024-05-07T02:31:50.003155+0000","last_peered":"2024-05-07T02:31:50.003155+0000","last_clean":"2024-05-07T02:31:50.003155+0000","last_became_active":"2024-05-07T02:26:11.868702+0000","last_became_peered":"2024-05-07T02:26:11.868702+0000","last_unstale":"2024-05-07T02:31:50.003155+0000","last_undegraded":"2024-05-07T02:31:50.003155+0000","last_fullsized":"2024-05-07T02:31:50.003155+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:05:05.497071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.098519+0000","last_change":"2024-05-07T02:26:25.867393+0000","last_active":"2024-05-07T02:31:50.098519+0000","last_peered":"2024-05-07T02:31:50.098519+0000","last_clean":"2024-05-07T02:31:50.098519+0000","last_became_active":"2024-05-07T02:26:25.866956+0000","last_became_peered":"2024-05-07T02:26:25.866956+0000","last_unstale":"2024-05-07T02:31:50.098519+0000","last_undegraded":"2024-05-07T02:31:50.098519+0000","last_fullsized":"2024-05-07T02:31:50.098519+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:23:48.821668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.984780+0000","last_change":"2024-05-07T02:28:33.871287+0000","last_active":"2024-05-07T02:31:49.984780+0000","last_peered":"2024-05-07T02:31:49.984780+0000","last_clean":"2024-05-07T02:31:49.984780+0000","last_became_active":"2024-05-07T02:28:33.870256+0000","last_became_peered":"2024-05-07T02:28:33.870256+0000","last_unstale":"2024-05-07T02:31:49.984780+0000","last_undegraded":"2024-05-07T02:31:49.984780+0000","last_fullsized":"2024-05-07T02:31:49.984780+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T08:36:26.789348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"89'6","reported_seq":301,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.097586+0000","last_change":"2024-05-07T02:26:07.999784+0000","last_active":"2024-05-07T02:31:50.097586+0000","last_peered":"2024-05-07T02:31:50.097586+0000","last_clean":"2024-05-07T02:31:50.097586+0000","last_became_active":"2024-05-07T02:26:07.999339+0000","last_became_peered":"2024-05-07T02:26:07.999339+0000","last_unstale":"2024-05-07T02:31:50.097586+0000","last_undegraded":"2024-05-07T02:31:50.097586+0000","last_fullsized":"2024-05-07T02:31:50.097586+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:50:41.622486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":247,"num_read_kb":247,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.136478+0000","last_change":"2024-05-07T02:26:25.587032+0000","last_active":"2024-05-07T02:31:50.136478+0000","last_peered":"2024-05-07T02:31:50.136478+0000","last_clean":"2024-05-07T02:31:50.136478+0000","last_became_active":"2024-05-07T02:26:25.586530+0000","last_became_peered":"2024-05-07T02:26:25.586530+0000","last_unstale":"2024-05-07T02:31:50.136478+0000","last_undegraded":"2024-05-07T02:31:50.136478+0000","last_fullsized":"2024-05-07T02:31:50.136478+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:13:13.335832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825819+0000","last_change":"2024-05-07T02:26:11.878593+0000","last_active":"2024-05-07T02:31:49.825819+0000","last_peered":"2024-05-07T02:31:49.825819+0000","last_clean":"2024-05-07T02:31:49.825819+0000","last_became_active":"2024-05-07T02:26:11.873286+0000","last_became_peered":"2024-05-07T02:26:11.873286+0000","last_unstale":"2024-05-07T02:31:49.825819+0000","last_undegraded":"2024-05-07T02:31:49.825819+0000","last_fullsized":"2024-05-07T02:31:49.825819+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:22:33.769539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825626+0000","last_change":"2024-05-07T02:26:07.790087+0000","last_active":"2024-05-07T02:31:49.825626+0000","last_peered":"2024-05-07T02:31:49.825626+0000","last_clean":"2024-05-07T02:31:49.825626+0000","last_became_active":"2024-05-07T02:26:07.789592+0000","last_became_peered":"2024-05-07T02:26:07.789592+0000","last_unstale":"2024-05-07T02:31:49.825626+0000","last_undegraded":"2024-05-07T02:31:49.825626+0000","last_fullsized":"2024-05-07T02:31:49.825626+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:44:52.569055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":122,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.135972+0000","last_change":"2024-05-07T02:24:27.877820+0000","last_active":"2024-05-07T02:31:50.135972+0000","last_peered":"2024-05-07T02:31:50.135972+0000","last_clean":"2024-05-07T02:31:50.135972+0000","last_became_active":"2024-05-07T02:24:27.873221+0000","last_became_peered":"2024-05-07T02:24:27.873221+0000","last_unstale":"2024-05-07T02:31:50.135972+0000","last_undegraded":"2024-05-07T02:31:50.135972+0000","last_fullsized":"2024-05-07T02:31:50.135972+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:20:21.582322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:20:21.582322+0000","last_clean_scrub_stamp":"2024-05-07T02:20:21.582322+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T14:16:32.772438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.999518+0000","last_change":"2024-05-07T02:26:25.868394+0000","last_active":"2024-05-07T02:31:49.999518+0000","last_peered":"2024-05-07T02:31:49.999518+0000","last_clean":"2024-05-07T02:31:49.999518+0000","last_became_active":"2024-05-07T02:26:25.867583+0000","last_became_peered":"2024-05-07T02:26:25.867583+0000","last_unstale":"2024-05-07T02:31:49.999518+0000","last_undegraded":"2024-05-07T02:31:49.999518+0000","last_fullsized":"2024-05-07T02:31:49.999518+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:55:44.705608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.097400+0000","last_change":"2024-05-07T02:26:11.868371+0000","last_active":"2024-05-07T02:31:50.097400+0000","last_peered":"2024-05-07T02:31:50.097400+0000","last_clean":"2024-05-07T02:31:50.097400+0000","last_became_active":"2024-05-07T02:26:11.867981+0000","last_became_peered":"2024-05-07T02:26:11.867981+0000","last_unstale":"2024-05-07T02:31:50.097400+0000","last_undegraded":"2024-05-07T02:31:50.097400+0000","last_fullsized":"2024-05-07T02:31:50.097400+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:08:54.687327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.136059+0000","last_change":"2024-05-07T02:26:07.785344+0000","last_active":"2024-05-07T02:31:50.136059+0000","last_peered":"2024-05-07T02:31:50.136059+0000","last_clean":"2024-05-07T02:31:50.136059+0000","last_became_active":"2024-05-07T02:26:07.784707+0000","last_became_peered":"2024-05-07T02:26:07.784707+0000","last_unstale":"2024-05-07T02:31:50.136059+0000","last_undegraded":"2024-05-07T02:31:50.136059+0000","last_fullsized":"2024-05-07T02:31:50.136059+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T05:00:38.754313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092632+0000","last_change":"2024-05-07T02:26:25.571989+0000","last_active":"2024-05-07T02:31:50.092632+0000","last_peered":"2024-05-07T02:31:50.092632+0000","last_clean":"2024-05-07T02:31:50.092632+0000","last_became_active":"2024-05-07T02:26:25.571533+0000","last_became_peered":"2024-05-07T02:26:25.571533+0000","last_unstale":"2024-05-07T02:31:50.092632+0000","last_undegraded":"2024-05-07T02:31:50.092632+0000","last_fullsized":"2024-05-07T02:31:50.092632+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:54:38.669278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.988658+0000","last_change":"2024-05-07T02:26:07.999636+0000","last_active":"2024-05-07T02:31:49.988658+0000","last_peered":"2024-05-07T02:31:49.988658+0000","last_clean":"2024-05-07T02:31:49.988658+0000","last_became_active":"2024-05-07T02:26:07.999100+0000","last_became_peered":"2024-05-07T02:26:07.999100+0000","last_unstale":"2024-05-07T02:31:49.988658+0000","last_undegraded":"2024-05-07T02:31:49.988658+0000","last_fullsized":"2024-05-07T02:31:49.988658+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:10:24.730988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.098303+0000","last_change":"2024-05-07T02:26:11.888279+0000","last_active":"2024-05-07T02:31:50.098303+0000","last_peered":"2024-05-07T02:31:50.098303+0000","last_clean":"2024-05-07T02:31:50.098303+0000","last_became_active":"2024-05-07T02:26:11.887480+0000","last_became_peered":"2024-05-07T02:26:11.887480+0000","last_unstale":"2024-05-07T02:31:50.098303+0000","last_undegraded":"2024-05-07T02:31:50.098303+0000","last_fullsized":"2024-05-07T02:31:50.098303+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:12:12.368368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.824134+0000","last_change":"2024-05-07T02:26:25.867444+0000","last_active":"2024-05-07T02:31:49.824134+0000","last_peered":"2024-05-07T02:31:49.824134+0000","last_clean":"2024-05-07T02:31:49.824134+0000","last_became_active":"2024-05-07T02:26:25.866392+0000","last_became_peered":"2024-05-07T02:26:25.866392+0000","last_unstale":"2024-05-07T02:31:49.824134+0000","last_undegraded":"2024-05-07T02:31:49.824134+0000","last_fullsized":"2024-05-07T02:31:49.824134+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:54:36.112427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'52","reported_seq":123,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.150979+0000","last_change":"2024-05-07T02:26:07.798091+0000","last_active":"2024-05-07T02:31:50.150979+0000","last_peered":"2024-05-07T02:31:50.150979+0000","last_clean":"2024-05-07T02:31:50.150979+0000","last_became_active":"2024-05-07T02:26:07.797716+0000","last_became_peered":"2024-05-07T02:26:07.797716+0000","last_unstale":"2024-05-07T02:31:50.150979+0000","last_undegraded":"2024-05-07T02:31:50.150979+0000","last_fullsized":"2024-05-07T02:31:50.150979+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:04:54.570916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":92,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825760+0000","last_change":"2024-05-07T02:26:11.888638+0000","last_active":"2024-05-07T02:31:49.825760+0000","last_peered":"2024-05-07T02:31:49.825760+0000","last_clean":"2024-05-07T02:31:49.825760+0000","last_became_active":"2024-05-07T02:26:11.888269+0000","last_became_peered":"2024-05-07T02:26:11.888269+0000","last_unstale":"2024-05-07T02:31:49.825760+0000","last_undegraded":"2024-05-07T02:31:49.825760+0000","last_fullsized":"2024-05-07T02:31:49.825760+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:09:38.965362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825175+0000","last_change":"2024-05-07T02:26:25.558975+0000","last_active":"2024-05-07T02:31:49.825175+0000","last_peered":"2024-05-07T02:31:49.825175+0000","last_clean":"2024-05-07T02:31:49.825175+0000","last_became_active":"2024-05-07T02:26:25.557947+0000","last_became_peered":"2024-05-07T02:26:25.557947+0000","last_unstale":"2024-05-07T02:31:49.825175+0000","last_undegraded":"2024-05-07T02:31:49.825175+0000","last_fullsized":"2024-05-07T02:31:49.825175+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:29:32.071625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.826855+0000","last_change":"2024-05-07T02:26:07.796643+0000","last_active":"2024-05-07T02:31:49.826855+0000","last_peered":"2024-05-07T02:31:49.826855+0000","last_clean":"2024-05-07T02:31:49.826855+0000","last_became_active":"2024-05-07T02:26:07.796264+0000","last_became_peered":"2024-05-07T02:26:07.796264+0000","last_unstale":"2024-05-07T02:31:49.826855+0000","last_undegraded":"2024-05-07T02:31:49.826855+0000","last_fullsized":"2024-05-07T02:31:49.826855+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:29:32.454816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.094595+0000","last_change":"2024-05-07T02:26:11.873670+0000","last_active":"2024-05-07T02:31:50.094595+0000","last_peered":"2024-05-07T02:31:50.094595+0000","last_clean":"2024-05-07T02:31:50.094595+0000","last_became_active":"2024-05-07T02:26:11.871219+0000","last_became_peered":"2024-05-07T02:26:11.871219+0000","last_unstale":"2024-05-07T02:31:50.094595+0000","last_undegraded":"2024-05-07T02:31:50.094595+0000","last_fullsized":"2024-05-07T02:31:50.094595+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:49:48.048127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.826601+0000","last_change":"2024-05-07T02:26:25.868011+0000","last_active":"2024-05-07T02:31:49.826601+0000","last_peered":"2024-05-07T02:31:49.826601+0000","last_clean":"2024-05-07T02:31:49.826601+0000","last_became_active":"2024-05-07T02:26:25.867465+0000","last_became_peered":"2024-05-07T02:26:25.867465+0000","last_unstale":"2024-05-07T02:31:49.826601+0000","last_undegraded":"2024-05-07T02:31:49.826601+0000","last_fullsized":"2024-05-07T02:31:49.826601+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T05:14:40.001874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092382+0000","last_change":"2024-05-07T02:26:07.999233+0000","last_active":"2024-05-07T02:31:50.092382+0000","last_peered":"2024-05-07T02:31:50.092382+0000","last_clean":"2024-05-07T02:31:50.092382+0000","last_became_active":"2024-05-07T02:26:07.998920+0000","last_became_peered":"2024-05-07T02:26:07.998920+0000","last_unstale":"2024-05-07T02:31:50.092382+0000","last_undegraded":"2024-05-07T02:31:50.092382+0000","last_fullsized":"2024-05-07T02:31:50.092382+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:33:33.878984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.988283+0000","last_change":"2024-05-07T02:26:11.856832+0000","last_active":"2024-05-07T02:31:49.988283+0000","last_peered":"2024-05-07T02:31:49.988283+0000","last_clean":"2024-05-07T02:31:49.988283+0000","last_became_active":"2024-05-07T02:26:11.856424+0000","last_became_peered":"2024-05-07T02:26:11.856424+0000","last_unstale":"2024-05-07T02:31:49.988283+0000","last_undegraded":"2024-05-07T02:31:49.988283+0000","last_fullsized":"2024-05-07T02:31:49.988283+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:00:46.839542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.988845+0000","last_change":"2024-05-07T02:27:32.626086+0000","last_active":"2024-05-07T02:31:49.988845+0000","last_peered":"2024-05-07T02:31:49.988845+0000","last_clean":"2024-05-07T02:31:49.988845+0000","last_became_active":"2024-05-07T02:27:32.625468+0000","last_became_peered":"2024-05-07T02:27:32.625468+0000","last_unstale":"2024-05-07T02:31:49.988845+0000","last_undegraded":"2024-05-07T02:31:49.988845+0000","last_fullsized":"2024-05-07T02:31:49.988845+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:17:57.245648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.824755+0000","last_change":"2024-05-07T02:26:07.793793+0000","last_active":"2024-05-07T02:31:49.824755+0000","last_peered":"2024-05-07T02:31:49.824755+0000","last_clean":"2024-05-07T02:31:49.824755+0000","last_became_active":"2024-05-07T02:26:07.790291+0000","last_became_peered":"2024-05-07T02:26:07.790291+0000","last_unstale":"2024-05-07T02:31:49.824755+0000","last_undegraded":"2024-05-07T02:31:49.824755+0000","last_fullsized":"2024-05-07T02:31:49.824755+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:50:14.591968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092657+0000","last_change":"2024-05-07T02:26:11.869184+0000","last_active":"2024-05-07T02:31:50.092657+0000","last_peered":"2024-05-07T02:31:50.092657+0000","last_clean":"2024-05-07T02:31:50.092657+0000","last_became_active":"2024-05-07T02:26:11.868773+0000","last_became_peered":"2024-05-07T02:26:11.868773+0000","last_unstale":"2024-05-07T02:31:50.092657+0000","last_undegraded":"2024-05-07T02:31:50.092657+0000","last_fullsized":"2024-05-07T02:31:50.092657+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:24:04.986359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825656+0000","last_change":"2024-05-07T02:26:25.580095+0000","last_active":"2024-05-07T02:31:49.825656+0000","last_peered":"2024-05-07T02:31:49.825656+0000","last_clean":"2024-05-07T02:31:49.825656+0000","last_became_active":"2024-05-07T02:26:25.579615+0000","last_became_peered":"2024-05-07T02:26:25.579615+0000","last_unstale":"2024-05-07T02:31:49.825656+0000","last_undegraded":"2024-05-07T02:31:49.825656+0000","last_fullsized":"2024-05-07T02:31:49.825656+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:18:05.714552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'4","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825368+0000","last_change":"2024-05-07T02:26:07.784709+0000","last_active":"2024-05-07T02:31:49.825368+0000","last_peered":"2024-05-07T02:31:49.825368+0000","last_clean":"2024-05-07T02:31:49.825368+0000","last_became_active":"2024-05-07T02:26:07.784282+0000","last_became_peered":"2024-05-07T02:26:07.784282+0000","last_unstale":"2024-05-07T02:31:49.825368+0000","last_undegraded":"2024-05-07T02:31:49.825368+0000","last_fullsized":"2024-05-07T02:31:49.825368+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:01:47.023496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.826271+0000","last_change":"2024-05-07T02:26:11.879152+0000","last_active":"2024-05-07T02:31:49.826271+0000","last_peered":"2024-05-07T02:31:49.826271+0000","last_clean":"2024-05-07T02:31:49.826271+0000","last_became_active":"2024-05-07T02:26:11.878571+0000","last_became_peered":"2024-05-07T02:26:11.878571+0000","last_unstale":"2024-05-07T02:31:49.826271+0000","last_undegraded":"2024-05-07T02:31:49.826271+0000","last_fullsized":"2024-05-07T02:31:49.826271+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:04:12.637807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.826575+0000","last_change":"2024-05-07T02:26:25.867199+0000","last_active":"2024-05-07T02:31:49.826575+0000","last_peered":"2024-05-07T02:31:49.826575+0000","last_clean":"2024-05-07T02:31:49.826575+0000","last_became_active":"2024-05-07T02:26:25.866624+0000","last_became_peered":"2024-05-07T02:26:25.866624+0000","last_unstale":"2024-05-07T02:31:49.826575+0000","last_undegraded":"2024-05-07T02:31:49.826575+0000","last_fullsized":"2024-05-07T02:31:49.826575+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:14:49.513313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.988386+0000","last_change":"2024-05-07T02:26:07.784155+0000","last_active":"2024-05-07T02:31:49.988386+0000","last_peered":"2024-05-07T02:31:49.988386+0000","last_clean":"2024-05-07T02:31:49.988386+0000","last_became_active":"2024-05-07T02:26:07.783579+0000","last_became_peered":"2024-05-07T02:26:07.783579+0000","last_unstale":"2024-05-07T02:31:49.988386+0000","last_undegraded":"2024-05-07T02:31:49.988386+0000","last_fullsized":"2024-05-07T02:31:49.988386+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T08:51:32.037959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.136365+0000","last_change":"2024-05-07T02:26:11.868454+0000","last_active":"2024-05-07T02:31:50.136365+0000","last_peered":"2024-05-07T02:31:50.136365+0000","last_clean":"2024-05-07T02:31:50.136365+0000","last_became_active":"2024-05-07T02:26:11.868117+0000","last_became_peered":"2024-05-07T02:26:11.868117+0000","last_unstale":"2024-05-07T02:31:50.136365+0000","last_undegraded":"2024-05-07T02:31:50.136365+0000","last_fullsized":"2024-05-07T02:31:50.136365+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T06:24:36.703684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092403+0000","last_change":"2024-05-07T02:26:25.573834+0000","last_active":"2024-05-07T02:31:50.092403+0000","last_peered":"2024-05-07T02:31:50.092403+0000","last_clean":"2024-05-07T02:31:50.092403+0000","last_became_active":"2024-05-07T02:26:25.573372+0000","last_became_peered":"2024-05-07T02:26:25.573372+0000","last_unstale":"2024-05-07T02:31:50.092403+0000","last_undegraded":"2024-05-07T02:31:50.092403+0000","last_fullsized":"2024-05-07T02:31:50.092403+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T12:03:36.916915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'6","reported_seq":303,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.305895+0000","last_change":"2024-05-07T02:26:08.000245+0000","last_active":"2024-05-07T02:31:50.305895+0000","last_peered":"2024-05-07T02:31:50.305895+0000","last_clean":"2024-05-07T02:31:50.305895+0000","last_became_active":"2024-05-07T02:26:07.999587+0000","last_became_peered":"2024-05-07T02:26:07.999587+0000","last_unstale":"2024-05-07T02:31:50.305895+0000","last_undegraded":"2024-05-07T02:31:50.305895+0000","last_fullsized":"2024-05-07T02:31:50.305895+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:44:41.754799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":249,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.136693+0000","last_change":"2024-05-07T02:26:11.858308+0000","last_active":"2024-05-07T02:31:50.136693+0000","last_peered":"2024-05-07T02:31:50.136693+0000","last_clean":"2024-05-07T02:31:50.136693+0000","last_became_active":"2024-05-07T02:26:11.857855+0000","last_became_peered":"2024-05-07T02:26:11.857855+0000","last_unstale":"2024-05-07T02:31:50.136693+0000","last_undegraded":"2024-05-07T02:31:50.136693+0000","last_fullsized":"2024-05-07T02:31:50.136693+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:11:50.021178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.095840+0000","last_change":"2024-05-07T02:26:25.584090+0000","last_active":"2024-05-07T02:31:50.095840+0000","last_peered":"2024-05-07T02:31:50.095840+0000","last_clean":"2024-05-07T02:31:50.095840+0000","last_became_active":"2024-05-07T02:26:25.580108+0000","last_became_peered":"2024-05-07T02:26:25.580108+0000","last_unstale":"2024-05-07T02:31:50.095840+0000","last_undegraded":"2024-05-07T02:31:50.095840+0000","last_fullsized":"2024-05-07T02:31:50.095840+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:56:56.964047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'41","reported_seq":97,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.147728+0000","last_change":"2024-05-07T02:26:07.788188+0000","last_active":"2024-05-07T02:31:50.147728+0000","last_peered":"2024-05-07T02:31:50.147728+0000","last_clean":"2024-05-07T02:31:50.147728+0000","last_became_active":"2024-05-07T02:26:07.787826+0000","last_became_peered":"2024-05-07T02:26:07.787826+0000","last_unstale":"2024-05-07T02:31:50.147728+0000","last_undegraded":"2024-05-07T02:31:50.147728+0000","last_fullsized":"2024-05-07T02:31:50.147728+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:06:14.588079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.136460+0000","last_change":"2024-05-07T02:26:11.862925+0000","last_active":"2024-05-07T02:31:50.136460+0000","last_peered":"2024-05-07T02:31:50.136460+0000","last_clean":"2024-05-07T02:31:50.136460+0000","last_became_active":"2024-05-07T02:26:11.861332+0000","last_became_peered":"2024-05-07T02:26:11.861332+0000","last_unstale":"2024-05-07T02:31:50.136460+0000","last_undegraded":"2024-05-07T02:31:50.136460+0000","last_fullsized":"2024-05-07T02:31:50.136460+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T09:51:11.133504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.998822+0000","last_change":"2024-05-07T02:26:25.557151+0000","last_active":"2024-05-07T02:31:49.998822+0000","last_peered":"2024-05-07T02:31:49.998822+0000","last_clean":"2024-05-07T02:31:49.998822+0000","last_became_active":"2024-05-07T02:26:25.556615+0000","last_became_peered":"2024-05-07T02:26:25.556615+0000","last_unstale":"2024-05-07T02:31:49.998822+0000","last_undegraded":"2024-05-07T02:31:49.998822+0000","last_fullsized":"2024-05-07T02:31:49.998822+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:24.504365+0000","last_clean_scrub_stamp":"2024-05-07T02:26:24.504365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:33:20.446543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":63,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.098780+0000","last_change":"2024-05-07T02:27:32.598094+0000","last_active":"2024-05-07T02:31:50.098780+0000","last_peered":"2024-05-07T02:31:50.098780+0000","last_clean":"2024-05-07T02:31:50.098780+0000","last_became_active":"2024-05-07T02:27:32.594099+0000","last_became_peered":"2024-05-07T02:27:32.594099+0000","last_unstale":"2024-05-07T02:31:50.098780+0000","last_undegraded":"2024-05-07T02:31:50.098780+0000","last_fullsized":"2024-05-07T02:31:50.098780+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:06.742138+0000","last_clean_scrub_stamp":"2024-05-07T02:26:06.742138+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:44:50.979927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.136717+0000","last_change":"2024-05-07T02:26:11.860507+0000","last_active":"2024-05-07T02:31:50.136717+0000","last_peered":"2024-05-07T02:31:50.136717+0000","last_clean":"2024-05-07T02:31:50.136717+0000","last_became_active":"2024-05-07T02:26:11.860116+0000","last_became_peered":"2024-05-07T02:26:11.860116+0000","last_unstale":"2024-05-07T02:31:50.136717+0000","last_undegraded":"2024-05-07T02:31:50.136717+0000","last_fullsized":"2024-05-07T02:31:50.136717+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T05:09:38.963040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092816+0000","last_change":"2024-05-07T02:26:11.867932+0000","last_active":"2024-05-07T02:31:50.092816+0000","last_peered":"2024-05-07T02:31:50.092816+0000","last_clean":"2024-05-07T02:31:50.092816+0000","last_became_active":"2024-05-07T02:26:11.867537+0000","last_became_peered":"2024-05-07T02:26:11.867537+0000","last_unstale":"2024-05-07T02:31:50.092816+0000","last_undegraded":"2024-05-07T02:31:50.092816+0000","last_fullsized":"2024-05-07T02:31:50.092816+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:31:35.843401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825700+0000","last_change":"2024-05-07T02:26:11.866074+0000","last_active":"2024-05-07T02:31:49.825700+0000","last_peered":"2024-05-07T02:31:49.825700+0000","last_clean":"2024-05-07T02:31:49.825700+0000","last_became_active":"2024-05-07T02:26:11.865669+0000","last_became_peered":"2024-05-07T02:26:11.865669+0000","last_unstale":"2024-05-07T02:31:49.825700+0000","last_undegraded":"2024-05-07T02:31:49.825700+0000","last_fullsized":"2024-05-07T02:31:49.825700+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T07:08:51.323706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825297+0000","last_change":"2024-05-07T02:26:11.876428+0000","last_active":"2024-05-07T02:31:49.825297+0000","last_peered":"2024-05-07T02:31:49.825297+0000","last_clean":"2024-05-07T02:31:49.825297+0000","last_became_active":"2024-05-07T02:26:11.876012+0000","last_became_peered":"2024-05-07T02:26:11.876012+0000","last_unstale":"2024-05-07T02:31:49.825297+0000","last_undegraded":"2024-05-07T02:31:49.825297+0000","last_fullsized":"2024-05-07T02:31:49.825297+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:24:58.088695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.094427+0000","last_change":"2024-05-07T02:26:11.880682+0000","last_active":"2024-05-07T02:31:50.094427+0000","last_peered":"2024-05-07T02:31:50.094427+0000","last_clean":"2024-05-07T02:31:50.094427+0000","last_became_active":"2024-05-07T02:26:11.880308+0000","last_became_peered":"2024-05-07T02:26:11.880308+0000","last_unstale":"2024-05-07T02:31:50.094427+0000","last_undegraded":"2024-05-07T02:31:50.094427+0000","last_fullsized":"2024-05-07T02:31:50.094427+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:52:45.083964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.098433+0000","last_change":"2024-05-07T02:26:11.876932+0000","last_active":"2024-05-07T02:31:50.098433+0000","last_peered":"2024-05-07T02:31:50.098433+0000","last_clean":"2024-05-07T02:31:50.098433+0000","last_became_active":"2024-05-07T02:26:11.876082+0000","last_became_peered":"2024-05-07T02:26:11.876082+0000","last_unstale":"2024-05-07T02:31:50.098433+0000","last_undegraded":"2024-05-07T02:31:50.098433+0000","last_fullsized":"2024-05-07T02:31:50.098433+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T04:45:21.435500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092847+0000","last_change":"2024-05-07T02:26:11.861339+0000","last_active":"2024-05-07T02:31:50.092847+0000","last_peered":"2024-05-07T02:31:50.092847+0000","last_clean":"2024-05-07T02:31:50.092847+0000","last_became_active":"2024-05-07T02:26:11.860912+0000","last_became_peered":"2024-05-07T02:26:11.860912+0000","last_unstale":"2024-05-07T02:31:50.092847+0000","last_undegraded":"2024-05-07T02:31:50.092847+0000","last_fullsized":"2024-05-07T02:31:50.092847+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:12:57.944986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825780+0000","last_change":"2024-05-07T02:26:11.867258+0000","last_active":"2024-05-07T02:31:49.825780+0000","last_peered":"2024-05-07T02:31:49.825780+0000","last_clean":"2024-05-07T02:31:49.825780+0000","last_became_active":"2024-05-07T02:26:11.866837+0000","last_became_peered":"2024-05-07T02:26:11.866837+0000","last_unstale":"2024-05-07T02:31:49.825780+0000","last_undegraded":"2024-05-07T02:31:49.825780+0000","last_fullsized":"2024-05-07T02:31:49.825780+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T10:07:58.589457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.825979+0000","last_change":"2024-05-07T02:26:11.888434+0000","last_active":"2024-05-07T02:31:49.825979+0000","last_peered":"2024-05-07T02:31:49.825979+0000","last_clean":"2024-05-07T02:31:49.825979+0000","last_became_active":"2024-05-07T02:26:11.887681+0000","last_became_peered":"2024-05-07T02:26:11.887681+0000","last_unstale":"2024-05-07T02:31:49.825979+0000","last_undegraded":"2024-05-07T02:31:49.825979+0000","last_fullsized":"2024-05-07T02:31:49.825979+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T02:38:58.896781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.999103+0000","last_change":"2024-05-07T02:26:11.875304+0000","last_active":"2024-05-07T02:31:49.999103+0000","last_peered":"2024-05-07T02:31:49.999103+0000","last_clean":"2024-05-07T02:31:49.999103+0000","last_became_active":"2024-05-07T02:26:11.874946+0000","last_became_peered":"2024-05-07T02:26:11.874946+0000","last_unstale":"2024-05-07T02:31:49.999103+0000","last_undegraded":"2024-05-07T02:31:49.999103+0000","last_fullsized":"2024-05-07T02:31:49.999103+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:56:04.093791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:49.824887+0000","last_change":"2024-05-07T02:26:11.879311+0000","last_active":"2024-05-07T02:31:49.824887+0000","last_peered":"2024-05-07T02:31:49.824887+0000","last_clean":"2024-05-07T02:31:49.824887+0000","last_became_active":"2024-05-07T02:26:11.877620+0000","last_became_peered":"2024-05-07T02:26:11.877620+0000","last_unstale":"2024-05-07T02:31:49.824887+0000","last_undegraded":"2024-05-07T02:31:49.824887+0000","last_fullsized":"2024-05-07T02:31:49.824887+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T08:41:40.525708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092753+0000","last_change":"2024-05-07T02:26:11.876726+0000","last_active":"2024-05-07T02:31:50.092753+0000","last_peered":"2024-05-07T02:31:50.092753+0000","last_clean":"2024-05-07T02:31:50.092753+0000","last_became_active":"2024-05-07T02:26:11.876234+0000","last_became_peered":"2024-05-07T02:26:11.876234+0000","last_unstale":"2024-05-07T02:31:50.092753+0000","last_undegraded":"2024-05-07T02:31:50.092753+0000","last_fullsized":"2024-05-07T02:31:50.092753+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T11:26:34.422258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.098131+0000","last_change":"2024-05-07T02:26:11.879159+0000","last_active":"2024-05-07T02:31:50.098131+0000","last_peered":"2024-05-07T02:31:50.098131+0000","last_clean":"2024-05-07T02:31:50.098131+0000","last_became_active":"2024-05-07T02:26:11.878769+0000","last_became_peered":"2024-05-07T02:26:11.878769+0000","last_unstale":"2024-05-07T02:31:50.098131+0000","last_undegraded":"2024-05-07T02:31:50.098131+0000","last_fullsized":"2024-05-07T02:31:50.098131+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T03:05:27.089475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-07T02:31:50.092884+0000","last_change":"2024-05-07T02:27:32.590188+0000","last_active":"2024-05-07T02:31:50.092884+0000","last_peered":"2024-05-07T02:31:50.092884+0000","last_clean":"2024-05-07T02:31:50.092884+0000","last_became_active":"2024-05-07T02:27:32.589784+0000","last_became_peered":"2024-05-07T02:27:32.589784+0000","last_unstale":"2024-05-07T02:31:50.092884+0000","last_undegraded":"2024-05-07T02:31:50.092884+0000","last_fullsized":"2024-05-07T02:31:50.092884+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-07T02:26:10.810482+0000","last_clean_scrub_stamp":"2024-05-07T02:26:10.810482+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-08T13:32:40.488143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":607529,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2601,"num_read_kb":1425,"num_write":388,"num_write_kb":1481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2187264,"data_stored":1822587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":312,"ondisk_log_size":312,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710803,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1056768,"data_stored":525718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907034,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":33020,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712132,"statfs":{"total":95995035648,"available":95961223168,"internally_reserved":0,"allocated":1888256,"data_stored":1318608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070560,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28456,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716696,"statfs":{"total":95995035648,"available":95965896704,"internally_reserved":0,"allocated":1470464,"data_stored":922848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266791,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36760,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708392,"statfs":{"total":95995035648,"available":95957393408,"internally_reserved":0,"allocated":1454080,"data_stored":919018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863463020,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1667072,"data_stored":1122862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626545,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":521963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574992,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1126400,"data_stored":596487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738517,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1073152,"data_stored":522887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1146880,"data_stored":596566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182531,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32728,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712424,"statfs":{"total":95995035648,"available":95961522176,"internally_reserved":0,"allocated":1589248,"data_stored":1045564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986304,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1056768,"data_stored":526206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822777,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28784,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716368,"statfs":{"total":95995035648,"available":95965560832,"internally_reserved":0,"allocated":1810432,"data_stored":1249593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":76169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":76248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":134811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":394604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":207922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-07T02:31:57.825 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-07T02:31:57.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:57 smithi019 ceph-mon[29781]: from='client.15192 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-07T02:31:57.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:57 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:31:57.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:57 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:31:57.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:57 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:57.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:57 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:57.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:57 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:57.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:57 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:57.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:57 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:57.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:57 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:57.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:57 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:57.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:57 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:57.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:57 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:31:57.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:57 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:31:57.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:57 smithi027 ceph-mon[34398]: from='client.15192 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-07T02:31:57.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:57 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:31:57.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:57 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:31:57.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:57 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:57.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:57 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:57.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:57 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:57.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:57 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:57.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:57 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:57.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:57 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:57.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:57 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:57.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:57 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:57.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:57 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:31:57.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:57 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:31:58.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:57 smithi064 ceph-mon[34699]: from='client.15192 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-07T02:31:58.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:57 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:31:58.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:57 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:31:58.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:57 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:58.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:57 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:58.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:57 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:58.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:57 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:58.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:57 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:58.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:57 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:58.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:57 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:58.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:57 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:31:58.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:57 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:31:58.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:57 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:31:58.272 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:31:58.782 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:58 smithi019 ceph-mon[29781]: from='client.15198 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-07T02:31:58.783 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:58 smithi019 ceph-mon[29781]: Adjusting osd_memory_target on smithi019 to 177.5M 2024-05-07T02:31:58.783 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:58 smithi019 ceph-mon[29781]: Unable to set osd_memory_target on smithi019 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-05-07T02:31:58.783 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:58 smithi019 ceph-mon[29781]: Updating smithi019:/etc/ceph/ceph.conf 2024-05-07T02:31:58.783 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:58 smithi019 ceph-mon[29781]: Updating smithi027:/etc/ceph/ceph.conf 2024-05-07T02:31:58.783 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:58 smithi019 ceph-mon[29781]: Updating smithi064:/etc/ceph/ceph.conf 2024-05-07T02:31:58.783 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:58 smithi019 ceph-mon[29781]: from='client.25330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-07T02:31:58.783 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:58 smithi019 ceph-mon[29781]: Updating smithi027:/var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-07T02:31:58.783 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:58 smithi019 ceph-mon[29781]: Updating smithi064:/var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-07T02:31:58.783 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:58 smithi019 ceph-mon[29781]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 0 B/s wr, 18 op/s 2024-05-07T02:31:58.783 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:58 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:31:58.783 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:58 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:31:58.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:58 smithi027 ceph-mon[34398]: from='client.15198 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-07T02:31:58.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:58 smithi027 ceph-mon[34398]: Adjusting osd_memory_target on smithi019 to 177.5M 2024-05-07T02:31:58.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:58 smithi027 ceph-mon[34398]: Unable to set osd_memory_target on smithi019 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-05-07T02:31:58.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:58 smithi027 ceph-mon[34398]: Updating smithi019:/etc/ceph/ceph.conf 2024-05-07T02:31:58.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:58 smithi027 ceph-mon[34398]: Updating smithi027:/etc/ceph/ceph.conf 2024-05-07T02:31:58.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:58 smithi027 ceph-mon[34398]: Updating smithi064:/etc/ceph/ceph.conf 2024-05-07T02:31:58.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:58 smithi027 ceph-mon[34398]: from='client.25330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-07T02:31:58.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:58 smithi027 ceph-mon[34398]: Updating smithi027:/var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-07T02:31:58.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:58 smithi027 ceph-mon[34398]: Updating smithi064:/var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-07T02:31:58.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:58 smithi027 ceph-mon[34398]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 0 B/s wr, 18 op/s 2024-05-07T02:31:58.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:58 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:31:58.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:58 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:31:59.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:58 smithi064 ceph-mon[34699]: from='client.15198 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-07T02:31:59.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:58 smithi064 ceph-mon[34699]: Adjusting osd_memory_target on smithi019 to 177.5M 2024-05-07T02:31:59.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:58 smithi064 ceph-mon[34699]: Unable to set osd_memory_target on smithi019 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-05-07T02:31:59.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:58 smithi064 ceph-mon[34699]: Updating smithi019:/etc/ceph/ceph.conf 2024-05-07T02:31:59.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:58 smithi064 ceph-mon[34699]: Updating smithi027:/etc/ceph/ceph.conf 2024-05-07T02:31:59.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:58 smithi064 ceph-mon[34699]: Updating smithi064:/etc/ceph/ceph.conf 2024-05-07T02:31:59.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:58 smithi064 ceph-mon[34699]: from='client.25330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-07T02:31:59.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:58 smithi064 ceph-mon[34699]: Updating smithi027:/var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-07T02:31:59.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:58 smithi064 ceph-mon[34699]: Updating smithi064:/var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-07T02:31:59.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:58 smithi064 ceph-mon[34699]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 0 B/s wr, 18 op/s 2024-05-07T02:31:59.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:58 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:31:59.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:58 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:31:59.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:59 smithi027 ceph-mon[34398]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2024-05-07T02:31:59.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:59 smithi027 ceph-mon[34398]: Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2024-05-07T02:31:59.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:59 smithi027 ceph-mon[34398]: Updating smithi019:/var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-07T02:31:59.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:59 smithi027 ceph-mon[34398]: Updating smithi027:/var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-07T02:31:59.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:59 smithi027 ceph-mon[34398]: Updating smithi064:/var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-07T02:31:59.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:59 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:31:59.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:59 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:31:59.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:31:59 smithi027 ceph-mon[34398]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-05-07T02:32:00.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:59 smithi064 ceph-mon[34699]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2024-05-07T02:32:00.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:59 smithi064 ceph-mon[34699]: Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2024-05-07T02:32:00.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:59 smithi064 ceph-mon[34699]: Updating smithi019:/var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-07T02:32:00.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:59 smithi064 ceph-mon[34699]: Updating smithi027:/var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-07T02:32:00.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:59 smithi064 ceph-mon[34699]: Updating smithi064:/var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-07T02:32:00.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:59 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:32:00.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:59 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:32:00.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:31:59 smithi064 ceph-mon[34699]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-05-07T02:32:00.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:59 smithi019 ceph-mon[29781]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2024-05-07T02:32:00.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:59 smithi019 ceph-mon[29781]: Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2024-05-07T02:32:00.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:59 smithi019 ceph-mon[29781]: Updating smithi019:/var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-07T02:32:00.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:59 smithi019 ceph-mon[29781]: Updating smithi027:/var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-07T02:32:00.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:59 smithi019 ceph-mon[29781]: Updating smithi064:/var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-07T02:32:00.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:59 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:32:00.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:59 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:32:00.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:31:59 smithi019 ceph-mon[29781]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-05-07T02:32:00.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:00.875+0000 7fce78e00700 1 -- 172.21.15.19:0/2194799502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce74077560 msgr2=0x7fce741106a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:00.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:00.875+0000 7fce78e00700 1 --2- 172.21.15.19:0/2194799502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce74077560 0x7fce741106a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fce60009990 tx=0x7fce60009ca0 comp rx=0 tx=0).stop 2024-05-07T02:32:00.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:00.875+0000 7fce78e00700 1 -- 172.21.15.19:0/2194799502 shutdown_connections 2024-05-07T02:32:00.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:00.875+0000 7fce78e00700 1 --2- 172.21.15.19:0/2194799502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce74110be0 0x7fce74113000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:00.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:00.875+0000 7fce78e00700 1 --2- 172.21.15.19:0/2194799502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce74076c10 0x7fce74077020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:00.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:00.875+0000 7fce78e00700 1 --2- 172.21.15.19:0/2194799502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce74077560 0x7fce741106a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:00.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:00.875+0000 7fce78e00700 1 -- 172.21.15.19:0/2194799502 >> 172.21.15.19:0/2194799502 conn(0x7fce740ffea0 msgr2=0x7fce741022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:00.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:00.875+0000 7fce78e00700 1 -- 172.21.15.19:0/2194799502 shutdown_connections 2024-05-07T02:32:00.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:00.875+0000 7fce78e00700 1 -- 172.21.15.19:0/2194799502 wait complete. 2024-05-07T02:32:00.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:00.876+0000 7fce78e00700 1 Processor -- start 2024-05-07T02:32:00.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:00.876+0000 7fce78e00700 1 -- start start 2024-05-07T02:32:00.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:00.877+0000 7fce78e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce74076c10 0x7fce7419cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:00.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:00.877+0000 7fce78e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce74077560 0x7fce7419d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:00.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:00.877+0000 7fce78e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce74110be0 0x7fce741a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:00.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:00.877+0000 7fce78e00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fce741166a0 con 0x7fce74077560 2024-05-07T02:32:00.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:00.877+0000 7fce78e00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fce74116520 con 0x7fce74110be0 2024-05-07T02:32:00.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:00.877+0000 7fce78e00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fce74116820 con 0x7fce74076c10 2024-05-07T02:32:00.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:00.877+0000 7fce71400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce74077560 0x7fce7419d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:00.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:00.877+0000 7fce71e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce74076c10 0x7fce7419cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:00.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:00.877+0000 7fce71400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce74077560 0x7fce7419d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38824/0 (socket says 172.21.15.19:38824) 2024-05-07T02:32:00.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:00.877+0000 7fce71400700 1 -- 172.21.15.19:0/753762112 learned_addr learned my addr 172.21.15.19:0/753762112 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:32:00.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:00.877+0000 7fce72800700 1 --2- 172.21.15.19:0/753762112 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce74110be0 0x7fce741a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:00.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:00.877+0000 7fce71e00700 1 -- 172.21.15.19:0/753762112 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce74110be0 msgr2=0x7fce741a1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:00.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:00.877+0000 7fce71e00700 1 --2- 172.21.15.19:0/753762112 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce74110be0 0x7fce741a1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:00.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:00.877+0000 7fce71e00700 1 -- 172.21.15.19:0/753762112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce74077560 msgr2=0x7fce7419d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:00.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:00.877+0000 7fce71e00700 1 --2- 172.21.15.19:0/753762112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce74077560 0x7fce7419d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:00.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:00.877+0000 7fce71e00700 1 -- 172.21.15.19:0/753762112 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce741a1ec0 con 0x7fce74076c10 2024-05-07T02:32:00.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:00.878+0000 7fce71400700 1 --2- 172.21.15.19:0/753762112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce74077560 0x7fce7419d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:32:00.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:00.878+0000 7fce71e00700 1 --2- 172.21.15.19:0/753762112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce74076c10 0x7fce7419cff0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fce6400e9f0 tx=0x7fce6400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:00.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:00.878+0000 7fce6aa00700 1 -- 172.21.15.19:0/753762112 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6400c9c0 con 0x7fce74076c10 2024-05-07T02:32:00.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:00.878+0000 7fce78e00700 1 -- 172.21.15.19:0/753762112 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce741a21b0 con 0x7fce74076c10 2024-05-07T02:32:00.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:00.878+0000 7fce6aa00700 1 -- 172.21.15.19:0/753762112 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce6400cb60 con 0x7fce74076c10 2024-05-07T02:32:00.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:00.878+0000 7fce6aa00700 1 -- 172.21.15.19:0/753762112 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce64018680 con 0x7fce74076c10 2024-05-07T02:32:00.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:00.878+0000 7fce78e00700 1 -- 172.21.15.19:0/753762112 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce741a9ad0 con 0x7fce74076c10 2024-05-07T02:32:00.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:00.879+0000 7fce78e00700 1 -- 172.21.15.19:0/753762112 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce740512c0 con 0x7fce74076c10 2024-05-07T02:32:00.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:00.882+0000 7fce6aa00700 1 -- 172.21.15.19:0/753762112 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7fce64010430 con 0x7fce74076c10 2024-05-07T02:32:00.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:00.884+0000 7fce6aa00700 1 --2- 172.21.15.19:0/753762112 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fce580b2a80 0x7fce580b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:00.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:00.884+0000 7fce6aa00700 1 -- 172.21.15.19:0/753762112 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fce640e3080 con 0x7fce74076c10 2024-05-07T02:32:00.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:00.884+0000 7fce71400700 1 --2- 172.21.15.19:0/753762112 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fce580b2a80 0x7fce580b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:00.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:00.885+0000 7fce71400700 1 --2- 172.21.15.19:0/753762112 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fce580b2a80 0x7fce580b4f90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fce741033f0 tx=0x7fce60005960 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:00.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:00.886+0000 7fce6aa00700 1 -- 172.21.15.19:0/753762112 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fce640aa500 con 0x7fce74076c10 2024-05-07T02:32:00.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:00 smithi027 ceph-mon[34398]: Updating smithi019:/var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-07T02:32:00.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:00 smithi027 ceph-mon[34398]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 0 B/s wr, 14 op/s 2024-05-07T02:32:00.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:00 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:32:00.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:00 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:32:00.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:00 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:32:00.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:00 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:32:00.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:00 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:32:00.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:00 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:32:00.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:00 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:32:01.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:00 smithi064 ceph-mon[34699]: Updating smithi019:/var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-07T02:32:01.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:00 smithi064 ceph-mon[34699]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 0 B/s wr, 14 op/s 2024-05-07T02:32:01.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:00 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:32:01.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:00 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:32:01.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:00 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:32:01.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:00 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:32:01.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:00 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:32:01.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:00 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:32:01.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:00 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:32:01.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:00 smithi019 ceph-mon[29781]: Updating smithi019:/var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-07T02:32:01.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:00 smithi019 ceph-mon[29781]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 0 B/s wr, 14 op/s 2024-05-07T02:32:01.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:00 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:32:01.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:00 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:32:01.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:00 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:32:01.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:00 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:32:01.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:00 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:32:01.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:00 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:32:01.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:00 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:32:01.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:01.182+0000 7fce78e00700 1 -- 172.21.15.19:0/753762112 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fce7419e350 con 0x7fce74076c10 2024-05-07T02:32:01.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:01.184+0000 7fce6aa00700 1 -- 172.21.15.19:0/753762112 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11924 (secure 0 0 0) 0x7fce640a9e00 con 0x7fce74076c10 2024-05-07T02:32:01.185 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:32:01.185 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1763670446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1763670446},{"type":"v1","addr":"172.21.15.19:6839","nonce":1763670446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/477529985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":477529985},{"type":"v1","addr":"172.21.15.64:6833","nonce":477529985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-07T02:26:14.960465+0000","modified":"2024-05-07T02:31:47.904333+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24539,"mds_2":24520,"mds_3":24515,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6833/2171873162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2171873162},{"type":"v1","addr":"172.21.15.27:6833","nonce":2171873162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/2379917839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2379917839},{"type":"v1","addr":"172.21.15.19:6835","nonce":2379917839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6837/2067515538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2067515538},{"type":"v1","addr":"172.21.15.19:6837","nonce":2067515538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6835/1000540600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1000540600},{"type":"v1","addr":"172.21.15.27:6835","nonce":1000540600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/1997318872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1997318872},{"type":"v1","addr":"172.21.15.64:6835","nonce":1997318872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":34,"addr":"172.21.15.64:6837/2585907777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2585907777},{"type":"v1","addr":"172.21.15.64:6837","nonce":2585907777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":33,"addr":"172.21.15.27:6837/2583138143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2583138143},{"type":"v1","addr":"172.21.15.27:6837","nonce":2583138143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":32,"addr":"172.21.15.64:6839/1583120004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":1583120004},{"type":"v1","addr":"172.21.15.64:6839","nonce":1583120004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":29,"addr":"172.21.15.19:6841/3905645246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3905645246},{"type":"v1","addr":"172.21.15.19:6841","nonce":3905645246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6839/3963268634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3963268634},{"type":"v1","addr":"172.21.15.27:6839","nonce":3963268634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24515,24505,24520,24551,24539]},"id":1}]} 2024-05-07T02:32:01.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:01.187+0000 7fce78e00700 1 -- 172.21.15.19:0/753762112 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fce580b2a80 msgr2=0x7fce580b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:01.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:01.187+0000 7fce78e00700 1 --2- 172.21.15.19:0/753762112 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fce580b2a80 0x7fce580b4f90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fce741033f0 tx=0x7fce60005960 comp rx=0 tx=0).stop 2024-05-07T02:32:01.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:01.187+0000 7fce78e00700 1 -- 172.21.15.19:0/753762112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce74076c10 msgr2=0x7fce7419cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:01.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:01.187+0000 7fce78e00700 1 --2- 172.21.15.19:0/753762112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce74076c10 0x7fce7419cff0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fce6400e9f0 tx=0x7fce6400ed00 comp rx=0 tx=0).stop 2024-05-07T02:32:01.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:01.187+0000 7fce78e00700 1 -- 172.21.15.19:0/753762112 shutdown_connections 2024-05-07T02:32:01.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:01.187+0000 7fce78e00700 1 --2- 172.21.15.19:0/753762112 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce74110be0 0x7fce741a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:01.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:01.187+0000 7fce78e00700 1 --2- 172.21.15.19:0/753762112 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce74076c10 0x7fce7419cff0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:01.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:01.187+0000 7fce78e00700 1 --2- 172.21.15.19:0/753762112 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fce580b2a80 0x7fce580b4f90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:01.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:01.187+0000 7fce78e00700 1 --2- 172.21.15.19:0/753762112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce74077560 0x7fce7419d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:01.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:01.187+0000 7fce78e00700 1 -- 172.21.15.19:0/753762112 >> 172.21.15.19:0/753762112 conn(0x7fce740ffea0 msgr2=0x7fce74101d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:01.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:01.188+0000 7fce78e00700 1 -- 172.21.15.19:0/753762112 shutdown_connections 2024-05-07T02:32:01.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:01.188+0000 7fce78e00700 1 -- 172.21.15.19:0/753762112 wait complete. 2024-05-07T02:32:01.190 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 31 2024-05-07T02:32:01.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:01 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/753762112' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:32:02.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:01 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/753762112' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:32:02.118 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:01 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/753762112' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:32:02.182 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-07T02:32:02.695 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:32:02.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:02 smithi027 ceph-mon[34398]: pgmap v9: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 0 B/s wr, 16 op/s 2024-05-07T02:32:03.037 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:02 smithi019 ceph-mon[29781]: pgmap v9: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 0 B/s wr, 16 op/s 2024-05-07T02:32:03.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:02 smithi064 ceph-mon[34699]: pgmap v9: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 0 B/s wr, 16 op/s 2024-05-07T02:32:05.098 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:04 smithi019 ceph-mon[29781]: pgmap v10: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 0 B/s wr, 14 op/s 2024-05-07T02:32:05.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:04 smithi027 ceph-mon[34398]: pgmap v10: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 0 B/s wr, 14 op/s 2024-05-07T02:32:05.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.235+0000 7f4936000700 1 -- 172.21.15.19:0/4102344910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4930104400 msgr2=0x7f4930104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:05.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.235+0000 7f4936000700 1 --2- 172.21.15.19:0/4102344910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4930104400 0x7f4930104810 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4924009920 tx=0x7f4924009c30 comp rx=0 tx=0).stop 2024-05-07T02:32:05.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.235+0000 7f4936000700 1 -- 172.21.15.19:0/4102344910 shutdown_connections 2024-05-07T02:32:05.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.235+0000 7f4936000700 1 --2- 172.21.15.19:0/4102344910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4930109680 0x7f4930110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:05.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.235+0000 7f4936000700 1 --2- 172.21.15.19:0/4102344910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4930104400 0x7f4930104810 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:05.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.235+0000 7f4936000700 1 --2- 172.21.15.19:0/4102344910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4930108d30 0x7f4930109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:05.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.235+0000 7f4936000700 1 -- 172.21.15.19:0/4102344910 >> 172.21.15.19:0/4102344910 conn(0x7f4930079d10 msgr2=0x7f493007a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:05.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.235+0000 7f4936000700 1 -- 172.21.15.19:0/4102344910 shutdown_connections 2024-05-07T02:32:05.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.235+0000 7f4936000700 1 -- 172.21.15.19:0/4102344910 wait complete. 2024-05-07T02:32:05.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.236+0000 7f4936000700 1 Processor -- start 2024-05-07T02:32:05.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.236+0000 7f4936000700 1 -- start start 2024-05-07T02:32:05.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.237+0000 7f4936000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4930104400 0x7f493019adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:05.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.237+0000 7f4936000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4930108d30 0x7f493019b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:05.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.237+0000 7f4936000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4930109680 0x7f493019f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:05.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.237+0000 7f4936000700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4930114300 con 0x7f4930109680 2024-05-07T02:32:05.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.237+0000 7f4936000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4930114180 con 0x7f4930108d30 2024-05-07T02:32:05.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.237+0000 7f4936000700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4930114480 con 0x7f4930104400 2024-05-07T02:32:05.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.237+0000 7f4934c00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4930104400 0x7f493019adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:05.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.237+0000 7f4934c00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4930104400 0x7f493019adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:36660/0 (socket says 172.21.15.19:36660) 2024-05-07T02:32:05.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.237+0000 7f4934c00700 1 -- 172.21.15.19:0/448584901 learned_addr learned my addr 172.21.15.19:0/448584901 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:32:05.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.237+0000 7f492fe00700 1 --2- 172.21.15.19:0/448584901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4930108d30 0x7f493019b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:05.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.237+0000 7f4935600700 1 --2- 172.21.15.19:0/448584901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4930109680 0x7f493019f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:05.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.238+0000 7f492fe00700 1 -- 172.21.15.19:0/448584901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4930104400 msgr2=0x7f493019adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:05.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.238+0000 7f492fe00700 1 --2- 172.21.15.19:0/448584901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4930104400 0x7f493019adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:05.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.238+0000 7f492fe00700 1 -- 172.21.15.19:0/448584901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4930109680 msgr2=0x7f493019f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:05.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.238+0000 7f492fe00700 1 --2- 172.21.15.19:0/448584901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4930109680 0x7f493019f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:05.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.238+0000 7f492fe00700 1 -- 172.21.15.19:0/448584901 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f493019fc80 con 0x7f4930108d30 2024-05-07T02:32:05.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.238+0000 7f492fe00700 1 --2- 172.21.15.19:0/448584901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4930108d30 0x7f493019b350 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f491c005fd0 tx=0x7f491c00ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:05.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.238+0000 7f492d600700 1 -- 172.21.15.19:0/448584901 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f491c00cbd0 con 0x7f4930108d30 2024-05-07T02:32:05.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.238+0000 7f492d600700 1 -- 172.21.15.19:0/448584901 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f491c0044d0 con 0x7f4930108d30 2024-05-07T02:32:05.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.238+0000 7f492d600700 1 -- 172.21.15.19:0/448584901 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f491c010430 con 0x7f4930108d30 2024-05-07T02:32:05.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.238+0000 7f4936000700 1 -- 172.21.15.19:0/448584901 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f493019ff70 con 0x7f4930108d30 2024-05-07T02:32:05.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.239+0000 7f4936000700 1 -- 172.21.15.19:0/448584901 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49301a7890 con 0x7f4930108d30 2024-05-07T02:32:05.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.240+0000 7f4936000700 1 -- 172.21.15.19:0/448584901 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49300512c0 con 0x7f4930108d30 2024-05-07T02:32:05.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.242+0000 7f492d600700 1 -- 172.21.15.19:0/448584901 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f491c003680 con 0x7f4930108d30 2024-05-07T02:32:05.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.244+0000 7f492d600700 1 --2- 172.21.15.19:0/448584901 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f49180b2b00 0x7f49180b5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:05.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.244+0000 7f492d600700 1 -- 172.21.15.19:0/448584901 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f491c0e2d10 con 0x7f4930108d30 2024-05-07T02:32:05.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.244+0000 7f4934c00700 1 --2- 172.21.15.19:0/448584901 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f49180b2b00 0x7f49180b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:05.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.245+0000 7f4934c00700 1 --2- 172.21.15.19:0/448584901 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f49180b2b00 0x7f49180b5010 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4924009810 tx=0x7f4924019040 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:05.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.246+0000 7f492d600700 1 -- 172.21.15.19:0/448584901 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f491c0aa190 con 0x7f4930108d30 2024-05-07T02:32:05.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:04 smithi064 ceph-mon[34699]: pgmap v10: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 0 B/s wr, 14 op/s 2024-05-07T02:32:05.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.514+0000 7f4936000700 1 -- 172.21.15.19:0/448584901 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f493019c110 con 0x7f4930108d30 2024-05-07T02:32:05.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.516+0000 7f492d600700 1 -- 172.21.15.19:0/448584901 <== mon.2 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+25100 (secure 0 0 0) 0x7f491c0a5030 con 0x7f4930108d30 2024-05-07T02:32:05.518 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:32:05.518 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":90,"fsid":"9ff84c16-0c17-11ef-bc97-c7b262605968","created":"2024-05-07T02:16:19.507657+0000","modified":"2024-05-07T02:31:49.815728+0000","last_up_change":"2024-05-07T02:24:55.025124+0000","last_in_change":"2024-05-07T02:24:33.842422+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-07T02:20:21.439697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-07T02:26:06.178814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-07T02:26:10.705693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-07T02:26:23.534133+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b437cf22-ec8e-48f0-86a2-2413a0ab40dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6803","nonce":2390539683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6805","nonce":2390539683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6809","nonce":2390539683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6807","nonce":2390539683}]},"public_addr":"172.21.15.19:6803/2390539683","cluster_addr":"172.21.15.19:6805/2390539683","heartbeat_back_addr":"172.21.15.19:6809/2390539683","heartbeat_front_addr":"172.21.15.19:6807/2390539683","state":["exists","up"]},{"osd":1,"uuid":"dc753e16-634d-4c14-abd2-935281e1499e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6801","nonce":4292470615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6803","nonce":4292470615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6807","nonce":4292470615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6805","nonce":4292470615}]},"public_addr":"172.21.15.27:6801/4292470615","cluster_addr":"172.21.15.27:6803/4292470615","heartbeat_back_addr":"172.21.15.27:6807/4292470615","heartbeat_front_addr":"172.21.15.27:6805/4292470615","state":["exists","up"]},{"osd":2,"uuid":"40570071-646e-4e66-9a16-c55ab743ac86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6801","nonce":1180552939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6803","nonce":1180552939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6807","nonce":1180552939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6805","nonce":1180552939}]},"public_addr":"172.21.15.64:6801/1180552939","cluster_addr":"172.21.15.64:6803/1180552939","heartbeat_back_addr":"172.21.15.64:6807/1180552939","heartbeat_front_addr":"172.21.15.64:6805/1180552939","state":["exists","up"]},{"osd":3,"uuid":"ceaa4252-4dfa-427a-9b1a-76adf3b8e609","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6811","nonce":387118141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6813","nonce":387118141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6817","nonce":387118141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6815","nonce":387118141}]},"public_addr":"172.21.15.19:6811/387118141","cluster_addr":"172.21.15.19:6813/387118141","heartbeat_back_addr":"172.21.15.19:6817/387118141","heartbeat_front_addr":"172.21.15.19:6815/387118141","state":["exists","up"]},{"osd":4,"uuid":"5a6d1ebc-62ca-444e-8f29-1980418d0a8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6809","nonce":3662186068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6811","nonce":3662186068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6815","nonce":3662186068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6813","nonce":3662186068}]},"public_addr":"172.21.15.27:6809/3662186068","cluster_addr":"172.21.15.27:6811/3662186068","heartbeat_back_addr":"172.21.15.27:6815/3662186068","heartbeat_front_addr":"172.21.15.27:6813/3662186068","state":["exists","up"]},{"osd":5,"uuid":"60c207e9-55e8-4bed-87c3-f6a06a2a1fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6809","nonce":1264591849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6811","nonce":1264591849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6815","nonce":1264591849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6813","nonce":1264591849}]},"public_addr":"172.21.15.64:6809/1264591849","cluster_addr":"172.21.15.64:6811/1264591849","heartbeat_back_addr":"172.21.15.64:6815/1264591849","heartbeat_front_addr":"172.21.15.64:6813/1264591849","state":["exists","up"]},{"osd":6,"uuid":"f3e038e0-4026-45f1-a923-c07cbb307f64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6819","nonce":2158875543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6821","nonce":2158875543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6825","nonce":2158875543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6823","nonce":2158875543}]},"public_addr":"172.21.15.19:6819/2158875543","cluster_addr":"172.21.15.19:6821/2158875543","heartbeat_back_addr":"172.21.15.19:6825/2158875543","heartbeat_front_addr":"172.21.15.19:6823/2158875543","state":["exists","up"]},{"osd":7,"uuid":"d69e34d4-2028-43b8-8ada-ff4a9b579620","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6817","nonce":2756612091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6819","nonce":2756612091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6823","nonce":2756612091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6821","nonce":2756612091}]},"public_addr":"172.21.15.27:6817/2756612091","cluster_addr":"172.21.15.27:6819/2756612091","heartbeat_back_addr":"172.21.15.27:6823/2756612091","heartbeat_front_addr":"172.21.15.27:6821/2756612091","state":["exists","up"]},{"osd":8,"uuid":"6c1a4179-75b6-42ff-ac29-378801e6f8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6817","nonce":1987024836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6819","nonce":1987024836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6823","nonce":1987024836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6821","nonce":1987024836}]},"public_addr":"172.21.15.64:6817/1987024836","cluster_addr":"172.21.15.64:6819/1987024836","heartbeat_back_addr":"172.21.15.64:6823/1987024836","heartbeat_front_addr":"172.21.15.64:6821/1987024836","state":["exists","up"]},{"osd":9,"uuid":"9df7ff66-d39f-4709-8324-74cf9e0e052d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6827","nonce":4124921646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6829","nonce":4124921646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6833","nonce":4124921646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6831","nonce":4124921646}]},"public_addr":"172.21.15.19:6827/4124921646","cluster_addr":"172.21.15.19:6829/4124921646","heartbeat_back_addr":"172.21.15.19:6833/4124921646","heartbeat_front_addr":"172.21.15.19:6831/4124921646","state":["exists","up"]},{"osd":10,"uuid":"71eaa9a7-667d-4cb7-935c-42177d20a279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6825","nonce":1428931817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6827","nonce":1428931817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6831","nonce":1428931817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6829","nonce":1428931817}]},"public_addr":"172.21.15.27:6825/1428931817","cluster_addr":"172.21.15.27:6827/1428931817","heartbeat_back_addr":"172.21.15.27:6831/1428931817","heartbeat_front_addr":"172.21.15.27:6829/1428931817","state":["exists","up"]},{"osd":11,"uuid":"3330627f-ab5d-4e6e-83cf-fc8b74c54ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6825","nonce":4045070679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6827","nonce":4045070679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6831","nonce":4045070679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6829","nonce":4045070679}]},"public_addr":"172.21.15.64:6825/4045070679","cluster_addr":"172.21.15.64:6827/4045070679","heartbeat_back_addr":"172.21.15.64:6831/4045070679","heartbeat_front_addr":"172.21.15.64:6829/4045070679","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:19:27.902633+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:19:54.305685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:20:18.727053+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:20:53.579199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:21:21.058018+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:21:47.091494+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:22:23.269018+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:22:51.863628+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:23:18.537774+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:23:55.763993+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:24:24.956178+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:24:52.681201+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/1821831886":"2024-05-08T02:31:49.815663+0000","172.21.15.19:6801/2059564551":"2024-05-08T02:31:49.815663+0000","172.21.15.19:6800/2059564551":"2024-05-08T02:31:49.815663+0000","172.21.15.19:0/1956516258":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/1816772181":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/686554066":"2024-05-08T02:31:49.815663+0000","172.21.15.19:0/2602127700":"2024-05-08T02:31:49.815663+0000","172.21.15.19:6801/174304059":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/3466480009":"2024-05-08T02:31:49.815663+0000","172.21.15.19:0/1514051750":"2024-05-08T02:16:54.732592+0000","172.21.15.19:6800/174304059":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/3930045871":"2024-05-08T02:16:54.732592+0000","172.21.15.19:6800/3614957309":"2024-05-08T02:16:54.732592+0000","172.21.15.19:0/1328365399":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/139593344":"2024-05-08T02:16:54.732592+0000","172.21.15.19:6801/3614957309":"2024-05-08T02:16:54.732592+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-07T02:32:05.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.519+0000 7f4936000700 1 -- 172.21.15.19:0/448584901 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f49180b2b00 msgr2=0x7f49180b5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:05.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.519+0000 7f4936000700 1 --2- 172.21.15.19:0/448584901 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f49180b2b00 0x7f49180b5010 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4924009810 tx=0x7f4924019040 comp rx=0 tx=0).stop 2024-05-07T02:32:05.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.519+0000 7f4936000700 1 -- 172.21.15.19:0/448584901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4930108d30 msgr2=0x7f493019b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:05.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.519+0000 7f4936000700 1 --2- 172.21.15.19:0/448584901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4930108d30 0x7f493019b350 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f491c005fd0 tx=0x7f491c00ee70 comp rx=0 tx=0).stop 2024-05-07T02:32:05.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.520+0000 7f4936000700 1 -- 172.21.15.19:0/448584901 shutdown_connections 2024-05-07T02:32:05.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.520+0000 7f4936000700 1 --2- 172.21.15.19:0/448584901 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f49180b2b00 0x7f49180b5010 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:05.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.520+0000 7f4936000700 1 --2- 172.21.15.19:0/448584901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4930109680 0x7f493019f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:05.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.520+0000 7f4936000700 1 --2- 172.21.15.19:0/448584901 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4930104400 0x7f493019adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:05.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.520+0000 7f4936000700 1 --2- 172.21.15.19:0/448584901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4930108d30 0x7f493019b350 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:05.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.520+0000 7f4936000700 1 -- 172.21.15.19:0/448584901 >> 172.21.15.19:0/448584901 conn(0x7f4930079d10 msgr2=0x7f4930101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:05.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.520+0000 7f4936000700 1 -- 172.21.15.19:0/448584901 shutdown_connections 2024-05-07T02:32:05.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:05.520+0000 7f4936000700 1 -- 172.21.15.19:0/448584901 wait complete. 2024-05-07T02:32:06.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:05 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/448584901' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-07T02:32:06.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:05 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/448584901' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-07T02:32:06.378 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:05 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/448584901' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-07T02:32:06.451 DEBUG:tasks.fwd_scrub:Yielding 2024-05-07T02:32:06.451 INFO:teuthology.run_tasks:Running task exec... 2024-05-07T02:32:06.461 INFO:teuthology.task.exec:Executing custom commands... 2024-05-07T02:32:06.461 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-05-07T02:32:06.461 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-05-07T02:32:06.462 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-05-07T02:32:06.462 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-07T02:32:06.462 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-07T02:32:06.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:06.736+0000 7f464f200640 1 -- 172.21.15.19:0/672492796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f465010bd10 msgr2=0x7f46501058b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:06.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:06.736+0000 7f464f200640 1 --2- 172.21.15.19:0/672492796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f465010bd10 0x7f46501058b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4640009ca0 tx=0x7f464002a8c0 comp rx=0 tx=0).stop 2024-05-07T02:32:06.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:06.736+0000 7f464f200640 1 -- 172.21.15.19:0/672492796 shutdown_connections 2024-05-07T02:32:06.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:06.736+0000 7f464f200640 1 --2- 172.21.15.19:0/672492796 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4650106460 0x7f4650112830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:06.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:06.736+0000 7f464f200640 1 --2- 172.21.15.19:0/672492796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f465010c1b0 0x7f4650105df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:06.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:06.736+0000 7f464f200640 1 --2- 172.21.15.19:0/672492796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f465010bd10 0x7f46501058b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:06.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:06.736+0000 7f464f200640 1 -- 172.21.15.19:0/672492796 >> 172.21.15.19:0/672492796 conn(0x7f46501013e0 msgr2=0x7f4650103800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:06.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:06.736+0000 7f464f200640 1 -- 172.21.15.19:0/672492796 shutdown_connections 2024-05-07T02:32:06.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:06.736+0000 7f464f200640 1 -- 172.21.15.19:0/672492796 wait complete. 2024-05-07T02:32:06.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:06.737+0000 7f464f200640 1 Processor -- start 2024-05-07T02:32:06.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:06.738+0000 7f464f200640 1 -- start start 2024-05-07T02:32:06.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:06.738+0000 7f464f200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4650106460 0x7f46501a25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:06.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:06.738+0000 7f464f200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f465010bd10 0x7f46501a2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:06.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:06.738+0000 7f464f200640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f465010c1b0 0x7f465019c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:06.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:06.738+0000 7f464f200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4650116000 con 0x7f4650106460 2024-05-07T02:32:06.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:06.738+0000 7f464f200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4650115e80 con 0x7f465010bd10 2024-05-07T02:32:06.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:06.738+0000 7f464f200640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4650116180 con 0x7f465010c1b0 2024-05-07T02:32:06.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:06.739+0000 7f464de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4650106460 0x7f46501a25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:06.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:06.739+0000 7f464de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4650106460 0x7f46501a25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38850/0 (socket says 172.21.15.19:38850) 2024-05-07T02:32:06.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:06.739+0000 7f464de00640 1 -- 172.21.15.19:0/3186350787 learned_addr learned my addr 172.21.15.19:0/3186350787 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:32:06.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:06.739+0000 7f464e800640 1 --2- 172.21.15.19:0/3186350787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f465010c1b0 0x7f465019c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:06.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:06.739+0000 7f464d400640 1 --2- 172.21.15.19:0/3186350787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f465010bd10 0x7f46501a2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:06.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:06.739+0000 7f464e800640 1 -- 172.21.15.19:0/3186350787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f465010bd10 msgr2=0x7f46501a2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:06.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:06.739+0000 7f464e800640 1 --2- 172.21.15.19:0/3186350787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f465010bd10 0x7f46501a2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:06.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:06.739+0000 7f464e800640 1 -- 172.21.15.19:0/3186350787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4650106460 msgr2=0x7f46501a25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:06.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:06.739+0000 7f464e800640 1 --2- 172.21.15.19:0/3186350787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4650106460 0x7f46501a25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:06.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:06.739+0000 7f464e800640 1 -- 172.21.15.19:0/3186350787 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f465019ce90 con 0x7f465010c1b0 2024-05-07T02:32:06.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:06.740+0000 7f464e800640 1 --2- 172.21.15.19:0/3186350787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f465010c1b0 0x7f465019c760 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f464800e970 tx=0x7f464800ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:06.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:06.740+0000 7f4646a00640 1 -- 172.21.15.19:0/3186350787 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464800ccc0 con 0x7f465010c1b0 2024-05-07T02:32:06.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:06.740+0000 7f4646a00640 1 -- 172.21.15.19:0/3186350787 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4648005020 con 0x7f465010c1b0 2024-05-07T02:32:06.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:06.740+0000 7f4646a00640 1 -- 172.21.15.19:0/3186350787 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4648004a00 con 0x7f465010c1b0 2024-05-07T02:32:06.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:06.740+0000 7f464f200640 1 -- 172.21.15.19:0/3186350787 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f465019d120 con 0x7f465010c1b0 2024-05-07T02:32:06.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:06.740+0000 7f464f200640 1 -- 172.21.15.19:0/3186350787 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46501a9380 con 0x7f465010c1b0 2024-05-07T02:32:06.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:06.741+0000 7f464f200640 1 -- 172.21.15.19:0/3186350787 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4650106ff0 con 0x7f465010c1b0 2024-05-07T02:32:06.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:06.745+0000 7f4646a00640 1 -- 172.21.15.19:0/3186350787 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f4648004ba0 con 0x7f465010c1b0 2024-05-07T02:32:06.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:06.747+0000 7f4646a00640 1 --2- 172.21.15.19:0/3186350787 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f46340b2a90 0x7f46340b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:06.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:06.747+0000 7f4646a00640 1 -- 172.21.15.19:0/3186350787 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f46480e2a60 con 0x7f465010c1b0 2024-05-07T02:32:06.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:06.747+0000 7f464de00640 1 --2- 172.21.15.19:0/3186350787 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f46340b2a90 0x7f46340b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:06.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:06.748+0000 7f464de00640 1 --2- 172.21.15.19:0/3186350787 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f46340b2a90 0x7f46340b4fa0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4640009a60 tx=0x7f4640009820 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:06.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:06.752+0000 7f4646a00640 1 -- 172.21.15.19:0/3186350787 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f46480a9f40 con 0x7f465010c1b0 2024-05-07T02:32:06.928 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:32:06.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:06.990+0000 7f464f200640 1 -- 172.21.15.19:0/3186350787 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f46501a95c0 con 0x7f465010c1b0 2024-05-07T02:32:06.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:06.999+0000 7f4646a00640 1 -- 172.21.15.19:0/3186350787 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v29)=0 v29) ==== 129+0+0 (secure 0 0 0) 0x7f46480a9840 con 0x7f465010c1b0 2024-05-07T02:32:07.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.002+0000 7f464f200640 1 -- 172.21.15.19:0/3186350787 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f46340b2a90 msgr2=0x7f46340b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:07.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.002+0000 7f464f200640 1 --2- 172.21.15.19:0/3186350787 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f46340b2a90 0x7f46340b4fa0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4640009a60 tx=0x7f4640009820 comp rx=0 tx=0).stop 2024-05-07T02:32:07.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.002+0000 7f464f200640 1 -- 172.21.15.19:0/3186350787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f465010c1b0 msgr2=0x7f465019c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:07.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.002+0000 7f464f200640 1 --2- 172.21.15.19:0/3186350787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f465010c1b0 0x7f465019c760 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f464800e970 tx=0x7f464800ee30 comp rx=0 tx=0).stop 2024-05-07T02:32:07.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.002+0000 7f464f200640 1 -- 172.21.15.19:0/3186350787 shutdown_connections 2024-05-07T02:32:07.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.002+0000 7f464f200640 1 --2- 172.21.15.19:0/3186350787 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f46340b2a90 0x7f46340b4fa0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:07.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.002+0000 7f464f200640 1 --2- 172.21.15.19:0/3186350787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f465010c1b0 0x7f465019c760 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:07.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.002+0000 7f464f200640 1 --2- 172.21.15.19:0/3186350787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f465010bd10 0x7f46501a2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:07.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.002+0000 7f464f200640 1 --2- 172.21.15.19:0/3186350787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4650106460 0x7f46501a25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:07.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.002+0000 7f464f200640 1 -- 172.21.15.19:0/3186350787 >> 172.21.15.19:0/3186350787 conn(0x7f46501013e0 msgr2=0x7f46501017f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:07.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.002+0000 7f464f200640 1 -- 172.21.15.19:0/3186350787 shutdown_connections 2024-05-07T02:32:07.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.003+0000 7f464f200640 1 -- 172.21.15.19:0/3186350787 wait complete. 2024-05-07T02:32:07.034 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-05-07T02:32:07.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:06 smithi027 ceph-mon[34398]: pgmap v11: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 0 B/s wr, 15 op/s 2024-05-07T02:32:07.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:06 smithi019 ceph-mon[29781]: pgmap v11: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 0 B/s wr, 15 op/s 2024-05-07T02:32:07.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.315+0000 7fda45600640 1 -- 172.21.15.19:0/3975972465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda40104410 msgr2=0x7fda401107e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:07.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.315+0000 7fda45600640 1 --2- 172.21.15.19:0/3975972465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda40104410 0x7fda401107e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fda38009ca0 tx=0x7fda3802a8c0 comp rx=0 tx=0).stop 2024-05-07T02:32:07.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.315+0000 7fda45600640 1 -- 172.21.15.19:0/3975972465 shutdown_connections 2024-05-07T02:32:07.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.315+0000 7fda45600640 1 --2- 172.21.15.19:0/3975972465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda40104410 0x7fda401107e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:07.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.315+0000 7fda45600640 1 --2- 172.21.15.19:0/3975972465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda4010a160 0x7fda40103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:07.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.315+0000 7fda45600640 1 --2- 172.21.15.19:0/3975972465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda40109cc0 0x7fda40103860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:07.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.315+0000 7fda45600640 1 -- 172.21.15.19:0/3975972465 >> 172.21.15.19:0/3975972465 conn(0x7fda400ff490 msgr2=0x7fda401018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:07.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.315+0000 7fda45600640 1 -- 172.21.15.19:0/3975972465 shutdown_connections 2024-05-07T02:32:07.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.315+0000 7fda45600640 1 -- 172.21.15.19:0/3975972465 wait complete. 2024-05-07T02:32:07.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.316+0000 7fda45600640 1 Processor -- start 2024-05-07T02:32:07.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.317+0000 7fda45600640 1 -- start start 2024-05-07T02:32:07.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.317+0000 7fda45600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda40104410 0x7fda401a25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:07.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.317+0000 7fda45600640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda40109cc0 0x7fda401a2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:07.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.317+0000 7fda45600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda4010a160 0x7fda4019c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:07.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.317+0000 7fda45600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fda40113eb0 con 0x7fda4010a160 2024-05-07T02:32:07.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.317+0000 7fda45600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fda40113d30 con 0x7fda40104410 2024-05-07T02:32:07.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.317+0000 7fda45600640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fda40114030 con 0x7fda40109cc0 2024-05-07T02:32:07.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.317+0000 7fda3e000640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda40109cc0 0x7fda401a2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:07.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.317+0000 7fda3f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda4010a160 0x7fda4019c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:07.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.317+0000 7fda3e000640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda40109cc0 0x7fda401a2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:36692/0 (socket says 172.21.15.19:36692) 2024-05-07T02:32:07.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.317+0000 7fda3e000640 1 -- 172.21.15.19:0/3076236371 learned_addr learned my addr 172.21.15.19:0/3076236371 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:32:07.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.317+0000 7fda3ea00640 1 --2- 172.21.15.19:0/3076236371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda40104410 0x7fda401a25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:07.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.318+0000 7fda3e000640 1 -- 172.21.15.19:0/3076236371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda40104410 msgr2=0x7fda401a25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:07.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.318+0000 7fda3e000640 1 --2- 172.21.15.19:0/3076236371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda40104410 0x7fda401a25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:07.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.318+0000 7fda3e000640 1 -- 172.21.15.19:0/3076236371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda4010a160 msgr2=0x7fda4019c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:07.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.318+0000 7fda3e000640 1 --2- 172.21.15.19:0/3076236371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda4010a160 0x7fda4019c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:07.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.318+0000 7fda3e000640 1 -- 172.21.15.19:0/3076236371 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda4019ceb0 con 0x7fda40109cc0 2024-05-07T02:32:07.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.318+0000 7fda3e000640 1 --2- 172.21.15.19:0/3076236371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda40109cc0 0x7fda401a2ae0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fda2c00e9f0 tx=0x7fda2c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:07.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.318+0000 7fda37400640 1 -- 172.21.15.19:0/3076236371 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2c00ccc0 con 0x7fda40109cc0 2024-05-07T02:32:07.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.318+0000 7fda45600640 1 -- 172.21.15.19:0/3076236371 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda4019d1a0 con 0x7fda40109cc0 2024-05-07T02:32:07.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.318+0000 7fda37400640 1 -- 172.21.15.19:0/3076236371 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda2c0035f0 con 0x7fda40109cc0 2024-05-07T02:32:07.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.319+0000 7fda37400640 1 -- 172.21.15.19:0/3076236371 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2c00ce60 con 0x7fda40109cc0 2024-05-07T02:32:07.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.319+0000 7fda45600640 1 -- 172.21.15.19:0/3076236371 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda401a93f0 con 0x7fda40109cc0 2024-05-07T02:32:07.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.320+0000 7fda45600640 1 -- 172.21.15.19:0/3076236371 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda40104fa0 con 0x7fda40109cc0 2024-05-07T02:32:07.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.323+0000 7fda37400640 1 -- 172.21.15.19:0/3076236371 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7fda2c003aa0 con 0x7fda40109cc0 2024-05-07T02:32:07.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.324+0000 7fda37400640 1 --2- 172.21.15.19:0/3076236371 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fda240b2a90 0x7fda240b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:07.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.324+0000 7fda37400640 1 -- 172.21.15.19:0/3076236371 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fda2c0e2e10 con 0x7fda40109cc0 2024-05-07T02:32:07.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.324+0000 7fda3ea00640 1 --2- 172.21.15.19:0/3076236371 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fda240b2a90 0x7fda240b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:07.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.325+0000 7fda3ea00640 1 --2- 172.21.15.19:0/3076236371 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fda240b2a90 0x7fda240b4fa0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fda4019db00 tx=0x7fda30005f90 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:07.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.329+0000 7fda37400640 1 -- 172.21.15.19:0/3076236371 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fda2c0e6030 con 0x7fda40109cc0 2024-05-07T02:32:07.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:06 smithi064 ceph-mon[34699]: pgmap v11: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 0 B/s wr, 15 op/s 2024-05-07T02:32:07.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.554+0000 7fda45600640 1 -- 172.21.15.19:0/3076236371 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fda401a9620 con 0x7fda40109cc0 2024-05-07T02:32:07.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.563+0000 7fda37400640 1 -- 172.21.15.19:0/3076236371 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v30)=0 v30) ==== 138+0+0 (secure 0 0 0) 0x7fda2c0aa290 con 0x7fda40109cc0 2024-05-07T02:32:07.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.567+0000 7fda45600640 1 -- 172.21.15.19:0/3076236371 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fda240b2a90 msgr2=0x7fda240b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:07.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.567+0000 7fda45600640 1 --2- 172.21.15.19:0/3076236371 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fda240b2a90 0x7fda240b4fa0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fda4019db00 tx=0x7fda30005f90 comp rx=0 tx=0).stop 2024-05-07T02:32:07.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.567+0000 7fda45600640 1 -- 172.21.15.19:0/3076236371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda40109cc0 msgr2=0x7fda401a2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:07.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.567+0000 7fda45600640 1 --2- 172.21.15.19:0/3076236371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda40109cc0 0x7fda401a2ae0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fda2c00e9f0 tx=0x7fda2c00eeb0 comp rx=0 tx=0).stop 2024-05-07T02:32:07.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.568+0000 7fda45600640 1 -- 172.21.15.19:0/3076236371 shutdown_connections 2024-05-07T02:32:07.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.568+0000 7fda45600640 1 --2- 172.21.15.19:0/3076236371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda4010a160 0x7fda4019c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:07.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.568+0000 7fda45600640 1 --2- 172.21.15.19:0/3076236371 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fda240b2a90 0x7fda240b4fa0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:07.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.568+0000 7fda45600640 1 --2- 172.21.15.19:0/3076236371 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda40109cc0 0x7fda401a2ae0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:07.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.568+0000 7fda45600640 1 --2- 172.21.15.19:0/3076236371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda40104410 0x7fda401a25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:07.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.568+0000 7fda45600640 1 -- 172.21.15.19:0/3076236371 >> 172.21.15.19:0/3076236371 conn(0x7fda400ff490 msgr2=0x7fda401061c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:07.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.568+0000 7fda45600640 1 -- 172.21.15.19:0/3076236371 shutdown_connections 2024-05-07T02:32:07.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.568+0000 7fda45600640 1 -- 172.21.15.19:0/3076236371 wait complete. 2024-05-07T02:32:07.598 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-05-07T02:32:07.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.864+0000 7f21f9000640 1 -- 172.21.15.19:0/2071213108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21f410b8c0 msgr2=0x7f21f410dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:07.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.864+0000 7f21f9000640 1 --2- 172.21.15.19:0/2071213108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21f410b8c0 0x7f21f410dce0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f21e0009900 tx=0x7f21e002a480 comp rx=0 tx=0).stop 2024-05-07T02:32:07.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.865+0000 7f21f9000640 1 -- 172.21.15.19:0/2071213108 shutdown_connections 2024-05-07T02:32:07.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.865+0000 7f21f9000640 1 --2- 172.21.15.19:0/2071213108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21f410e220 0x7f21f41106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:07.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.865+0000 7f21f9000640 1 --2- 172.21.15.19:0/2071213108 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21f410b8c0 0x7f21f410dce0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:07.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.865+0000 7f21f9000640 1 --2- 172.21.15.19:0/2071213108 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21f4104e60 0x7f21f4105270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:07.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.865+0000 7f21f9000640 1 -- 172.21.15.19:0/2071213108 >> 172.21.15.19:0/2071213108 conn(0x7f21f40782e0 msgr2=0x7f21f4078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:07.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.865+0000 7f21f9000640 1 -- 172.21.15.19:0/2071213108 shutdown_connections 2024-05-07T02:32:07.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.865+0000 7f21f9000640 1 -- 172.21.15.19:0/2071213108 wait complete. 2024-05-07T02:32:07.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.866+0000 7f21f9000640 1 Processor -- start 2024-05-07T02:32:07.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.867+0000 7f21f9000640 1 -- start start 2024-05-07T02:32:07.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.867+0000 7f21f9000640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21f4104e60 0x7f21f4105940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:07.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.867+0000 7f21f9000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21f410b8c0 0x7f21f4105e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:07.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.867+0000 7f21f9000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21f410e220 0x7f21f41063c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:07.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.867+0000 7f21f9000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f21f4113eb0 con 0x7f21f410b8c0 2024-05-07T02:32:07.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.867+0000 7f21f9000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f21f4113d30 con 0x7f21f410e220 2024-05-07T02:32:07.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.867+0000 7f21f9000640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f21f4114030 con 0x7f21f4104e60 2024-05-07T02:32:07.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.867+0000 7f21f1e00640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21f4104e60 0x7f21f4105940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:07.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.867+0000 7f21f1e00640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21f4104e60 0x7f21f4105940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:36702/0 (socket says 172.21.15.19:36702) 2024-05-07T02:32:07.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.867+0000 7f21f1e00640 1 -- 172.21.15.19:0/2778211430 learned_addr learned my addr 172.21.15.19:0/2778211430 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:32:07.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.867+0000 7f21f1400640 1 --2- 172.21.15.19:0/2778211430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21f410b8c0 0x7f21f4105e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:07.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.867+0000 7f21f2800640 1 --2- 172.21.15.19:0/2778211430 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21f410e220 0x7f21f41063c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:07.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.868+0000 7f21f1e00640 1 -- 172.21.15.19:0/2778211430 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21f410e220 msgr2=0x7f21f41063c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:07.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.868+0000 7f21f1e00640 1 --2- 172.21.15.19:0/2778211430 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21f410e220 0x7f21f41063c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:07.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.868+0000 7f21f1e00640 1 -- 172.21.15.19:0/2778211430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21f410b8c0 msgr2=0x7f21f4105e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:07.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.868+0000 7f21f1e00640 1 --2- 172.21.15.19:0/2778211430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21f410b8c0 0x7f21f4105e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:07.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.868+0000 7f21f1e00640 1 -- 172.21.15.19:0/2778211430 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21f41acfe0 con 0x7f21f4104e60 2024-05-07T02:32:07.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.868+0000 7f21f1e00640 1 --2- 172.21.15.19:0/2778211430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21f4104e60 0x7f21f4105940 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f21e400e9f0 tx=0x7f21e400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:07.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.869+0000 7f21eaa00640 1 -- 172.21.15.19:0/2778211430 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e400cd80 con 0x7f21f4104e60 2024-05-07T02:32:07.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.869+0000 7f21eaa00640 1 -- 172.21.15.19:0/2778211430 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21e40054d0 con 0x7f21f4104e60 2024-05-07T02:32:07.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.869+0000 7f21eaa00640 1 -- 172.21.15.19:0/2778211430 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e4010690 con 0x7f21f4104e60 2024-05-07T02:32:07.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.869+0000 7f21f9000640 1 -- 172.21.15.19:0/2778211430 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21f41ad210 con 0x7f21f4104e60 2024-05-07T02:32:07.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.869+0000 7f21f9000640 1 -- 172.21.15.19:0/2778211430 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21f41ad730 con 0x7f21f4104e60 2024-05-07T02:32:07.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.870+0000 7f21f9000640 1 -- 172.21.15.19:0/2778211430 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21f406ae80 con 0x7f21f4104e60 2024-05-07T02:32:07.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.873+0000 7f21eaa00640 1 -- 172.21.15.19:0/2778211430 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f21e4047a10 con 0x7f21f4104e60 2024-05-07T02:32:07.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.875+0000 7f21eaa00640 1 --2- 172.21.15.19:0/2778211430 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f21d80b2b10 0x7f21d80b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:07.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.876+0000 7f21eaa00640 1 -- 172.21.15.19:0/2778211430 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f21e40e2f90 con 0x7f21f4104e60 2024-05-07T02:32:07.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.876+0000 7f21f1400640 1 --2- 172.21.15.19:0/2778211430 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f21d80b2b10 0x7f21d80b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:07.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.876+0000 7f21eaa00640 1 -- 172.21.15.19:0/2778211430 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f21e40aa410 con 0x7f21f4104e60 2024-05-07T02:32:07.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:07.877+0000 7f21f1400640 1 --2- 172.21.15.19:0/2778211430 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f21d80b2b10 0x7f21d80b5020 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f21e0005ab0 tx=0x7f21e0005a20 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:08.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.099+0000 7f21f9000640 1 -- 172.21.15.19:0/2778211430 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f21f41ada20 con 0x7f21f4104e60 2024-05-07T02:32:08.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.108+0000 7f21eaa00640 1 -- 172.21.15.19:0/2778211430 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v31)=0 v31) ==== 133+0+0 (secure 0 0 0) 0x7f21e40a9d10 con 0x7f21f4104e60 2024-05-07T02:32:08.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.110+0000 7f21f9000640 1 -- 172.21.15.19:0/2778211430 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f21d80b2b10 msgr2=0x7f21d80b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:08.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.110+0000 7f21f9000640 1 --2- 172.21.15.19:0/2778211430 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f21d80b2b10 0x7f21d80b5020 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f21e0005ab0 tx=0x7f21e0005a20 comp rx=0 tx=0).stop 2024-05-07T02:32:08.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.110+0000 7f21f9000640 1 -- 172.21.15.19:0/2778211430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21f4104e60 msgr2=0x7f21f4105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:08.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.110+0000 7f21f9000640 1 --2- 172.21.15.19:0/2778211430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21f4104e60 0x7f21f4105940 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f21e400e9f0 tx=0x7f21e400edf0 comp rx=0 tx=0).stop 2024-05-07T02:32:08.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.111+0000 7f21f9000640 1 -- 172.21.15.19:0/2778211430 shutdown_connections 2024-05-07T02:32:08.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.111+0000 7f21f9000640 1 --2- 172.21.15.19:0/2778211430 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21f410e220 0x7f21f41063c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:08.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.111+0000 7f21f9000640 1 --2- 172.21.15.19:0/2778211430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21f410b8c0 0x7f21f4105e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:08.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.111+0000 7f21f9000640 1 --2- 172.21.15.19:0/2778211430 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f21d80b2b10 0x7f21d80b5020 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:08.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.111+0000 7f21f9000640 1 --2- 172.21.15.19:0/2778211430 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21f4104e60 0x7f21f4105940 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:08.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.111+0000 7f21f9000640 1 -- 172.21.15.19:0/2778211430 >> 172.21.15.19:0/2778211430 conn(0x7f21f40782e0 msgr2=0x7f21f410f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:08.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.111+0000 7f21f9000640 1 -- 172.21.15.19:0/2778211430 shutdown_connections 2024-05-07T02:32:08.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.111+0000 7f21f9000640 1 -- 172.21.15.19:0/2778211430 wait complete. 2024-05-07T02:32:08.142 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-05-07T02:32:08.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:07 smithi064 ceph-mon[34699]: from='client.? ' entity='client.admin' 2024-05-07T02:32:08.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:07 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:32:08.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:07 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:32:08.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:07 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:32:08.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:07 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:32:08.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:07 smithi064 ceph-mon[34699]: from='client.? ' entity='client.admin' 2024-05-07T02:32:08.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:07 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:32:08.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:07 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:32:08.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:07 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:32:08.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:07 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:32:08.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:07 smithi064 ceph-mon[34699]: pgmap v12: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 0 B/s wr, 18 op/s 2024-05-07T02:32:08.387 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:07 smithi019 ceph-mon[29781]: from='client.? ' entity='client.admin' 2024-05-07T02:32:08.387 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:07 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:32:08.387 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:07 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:32:08.387 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:07 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:32:08.387 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:07 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:32:08.387 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:07 smithi019 ceph-mon[29781]: from='client.? ' entity='client.admin' 2024-05-07T02:32:08.387 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:07 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:32:08.387 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:07 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:32:08.387 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:07 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:32:08.387 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:07 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:32:08.387 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:07 smithi019 ceph-mon[29781]: pgmap v12: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 0 B/s wr, 18 op/s 2024-05-07T02:32:08.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.404+0000 7f28fae00640 1 -- 172.21.15.19:0/453980142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28f410b800 msgr2=0x7f28f410dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:08.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.404+0000 7f28fae00640 1 --2- 172.21.15.19:0/453980142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28f410b800 0x7f28f410dc20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f28dc009900 tx=0x7f28dc02a3f0 comp rx=0 tx=0).stop 2024-05-07T02:32:08.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.404+0000 7f28fae00640 1 -- 172.21.15.19:0/453980142 shutdown_connections 2024-05-07T02:32:08.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.404+0000 7f28fae00640 1 --2- 172.21.15.19:0/453980142 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28f410e1f0 0x7f28f41106e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:08.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.404+0000 7f28fae00640 1 --2- 172.21.15.19:0/453980142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28f410b800 0x7f28f410dc20 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:08.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.404+0000 7f28fae00640 1 --2- 172.21.15.19:0/453980142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28f4104da0 0x7f28f41051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:08.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.404+0000 7f28fae00640 1 -- 172.21.15.19:0/453980142 >> 172.21.15.19:0/453980142 conn(0x7f28f40782e0 msgr2=0x7f28f4078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:08.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.404+0000 7f28fae00640 1 -- 172.21.15.19:0/453980142 shutdown_connections 2024-05-07T02:32:08.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.404+0000 7f28fae00640 1 -- 172.21.15.19:0/453980142 wait complete. 2024-05-07T02:32:08.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.406+0000 7f28fae00640 1 Processor -- start 2024-05-07T02:32:08.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.406+0000 7f28fae00640 1 -- start start 2024-05-07T02:32:08.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.407+0000 7f28fae00640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28f4104da0 0x7f28f41a25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:08.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.407+0000 7f28fae00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28f410b800 0x7f28f41a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:08.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.407+0000 7f28fae00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28f410e1f0 0x7f28f419c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:08.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.407+0000 7f28fae00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f28f4113e50 con 0x7f28f410b800 2024-05-07T02:32:08.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.407+0000 7f28fae00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f28f4113cd0 con 0x7f28f410e1f0 2024-05-07T02:32:08.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.407+0000 7f28fae00640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f28f4113fd0 con 0x7f28f4104da0 2024-05-07T02:32:08.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.407+0000 7f28f3e00640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28f4104da0 0x7f28f41a25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:08.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.407+0000 7f28f3e00640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28f4104da0 0x7f28f41a25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:36708/0 (socket says 172.21.15.19:36708) 2024-05-07T02:32:08.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.407+0000 7f28f3e00640 1 -- 172.21.15.19:0/1525683234 learned_addr learned my addr 172.21.15.19:0/1525683234 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:32:08.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.407+0000 7f28ebe00640 1 --2- 172.21.15.19:0/1525683234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28f410b800 0x7f28f41a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:08.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.407+0000 7f28f8e00640 1 --2- 172.21.15.19:0/1525683234 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28f410e1f0 0x7f28f419c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:08.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.408+0000 7f28f3e00640 1 -- 172.21.15.19:0/1525683234 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28f410e1f0 msgr2=0x7f28f419c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:08.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.408+0000 7f28f3e00640 1 --2- 172.21.15.19:0/1525683234 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28f410e1f0 0x7f28f419c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:08.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.408+0000 7f28f3e00640 1 -- 172.21.15.19:0/1525683234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28f410b800 msgr2=0x7f28f41a2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:08.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.408+0000 7f28f3e00640 1 --2- 172.21.15.19:0/1525683234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28f410b800 0x7f28f41a2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:08.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.408+0000 7f28f3e00640 1 -- 172.21.15.19:0/1525683234 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28f419ced0 con 0x7f28f4104da0 2024-05-07T02:32:08.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.408+0000 7f28f3e00640 1 --2- 172.21.15.19:0/1525683234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28f4104da0 0x7f28f41a25c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f28e400e9f0 tx=0x7f28e400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:08.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.409+0000 7f28f1600640 1 -- 172.21.15.19:0/1525683234 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e400cd80 con 0x7f28f4104da0 2024-05-07T02:32:08.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.409+0000 7f28f1600640 1 -- 172.21.15.19:0/1525683234 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28e40108e0 con 0x7f28f4104da0 2024-05-07T02:32:08.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.409+0000 7f28f1600640 1 -- 172.21.15.19:0/1525683234 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e4018980 con 0x7f28f4104da0 2024-05-07T02:32:08.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.409+0000 7f28fae00640 1 -- 172.21.15.19:0/1525683234 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28f419d1c0 con 0x7f28f4104da0 2024-05-07T02:32:08.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.409+0000 7f28fae00640 1 -- 172.21.15.19:0/1525683234 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28f41a9410 con 0x7f28f4104da0 2024-05-07T02:32:08.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.410+0000 7f28fae00640 1 -- 172.21.15.19:0/1525683234 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28f406af40 con 0x7f28f4104da0 2024-05-07T02:32:08.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.414+0000 7f28f1600640 1 -- 172.21.15.19:0/1525683234 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f28e4010430 con 0x7f28f4104da0 2024-05-07T02:32:08.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.416+0000 7f28f1600640 1 --2- 172.21.15.19:0/1525683234 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f28d80b2a90 0x7f28d80b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:08.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.416+0000 7f28f1600640 1 -- 172.21.15.19:0/1525683234 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f28e40e2e70 con 0x7f28f4104da0 2024-05-07T02:32:08.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.416+0000 7f28ebe00640 1 --2- 172.21.15.19:0/1525683234 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f28d80b2a90 0x7f28d80b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:08.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.417+0000 7f28ebe00640 1 --2- 172.21.15.19:0/1525683234 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f28d80b2a90 0x7f28d80b4fa0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f28dc031440 tx=0x7f28dc009560 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:08.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.420+0000 7f28f1600640 1 -- 172.21.15.19:0/1525683234 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f28e40a9d00 con 0x7f28f4104da0 2024-05-07T02:32:08.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:07 smithi027 ceph-mon[34398]: from='client.? ' entity='client.admin' 2024-05-07T02:32:08.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:07 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:32:08.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:07 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:32:08.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:07 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:32:08.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:07 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:32:08.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:07 smithi027 ceph-mon[34398]: from='client.? ' entity='client.admin' 2024-05-07T02:32:08.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:07 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:32:08.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:07 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:32:08.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:07 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:32:08.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:07 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:32:08.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:07 smithi027 ceph-mon[34398]: pgmap v12: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 0 B/s wr, 18 op/s 2024-05-07T02:32:08.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.682+0000 7f28fae00640 1 -- 172.21.15.19:0/1525683234 --> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f28f410f6e0 con 0x7f28d80b2a90 2024-05-07T02:32:08.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.790+0000 7f28f1600640 1 -- 172.21.15.19:0/1525683234 <== mgr.25234 v2:172.21.15.19:6800/3265825478 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f28f410f6e0 con 0x7f28d80b2a90 2024-05-07T02:32:08.791 INFO:teuthology.orchestra.run.smithi019.stdout:Schedule set for path / 2024-05-07T02:32:08.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.793+0000 7f28fae00640 1 -- 172.21.15.19:0/1525683234 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f28d80b2a90 msgr2=0x7f28d80b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:08.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.793+0000 7f28fae00640 1 --2- 172.21.15.19:0/1525683234 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f28d80b2a90 0x7f28d80b4fa0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f28dc031440 tx=0x7f28dc009560 comp rx=0 tx=0).stop 2024-05-07T02:32:08.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.793+0000 7f28fae00640 1 -- 172.21.15.19:0/1525683234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28f4104da0 msgr2=0x7f28f41a25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:08.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.793+0000 7f28fae00640 1 --2- 172.21.15.19:0/1525683234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28f4104da0 0x7f28f41a25c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f28e400e9f0 tx=0x7f28e400eeb0 comp rx=0 tx=0).stop 2024-05-07T02:32:08.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.793+0000 7f28fae00640 1 -- 172.21.15.19:0/1525683234 shutdown_connections 2024-05-07T02:32:08.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.793+0000 7f28fae00640 1 --2- 172.21.15.19:0/1525683234 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28f410e1f0 0x7f28f419c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:08.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.793+0000 7f28fae00640 1 --2- 172.21.15.19:0/1525683234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28f410b800 0x7f28f41a2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:08.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.793+0000 7f28fae00640 1 --2- 172.21.15.19:0/1525683234 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f28d80b2a90 0x7f28d80b4fa0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:08.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.793+0000 7f28fae00640 1 --2- 172.21.15.19:0/1525683234 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28f4104da0 0x7f28f41a25c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:08.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.793+0000 7f28fae00640 1 -- 172.21.15.19:0/1525683234 >> 172.21.15.19:0/1525683234 conn(0x7f28f40782e0 msgr2=0x7f28f410a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:08.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.793+0000 7f28fae00640 1 -- 172.21.15.19:0/1525683234 shutdown_connections 2024-05-07T02:32:08.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:08.793+0000 7f28fae00640 1 -- 172.21.15.19:0/1525683234 wait complete. 2024-05-07T02:32:08.821 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-05-07T02:32:09.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.093+0000 7f7472000640 1 -- 172.21.15.19:0/2443509955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f746c075670 msgr2=0x7f746c075a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:09.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.093+0000 7f7472000640 1 --2- 172.21.15.19:0/2443509955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f746c075670 0x7f746c075a80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7460009900 tx=0x7f746002a480 comp rx=0 tx=0).stop 2024-05-07T02:32:09.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.093+0000 7f7472000640 1 -- 172.21.15.19:0/2443509955 shutdown_connections 2024-05-07T02:32:09.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.093+0000 7f7472000640 1 --2- 172.21.15.19:0/2443509955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f746c1102b0 0x7f746c112710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.093+0000 7f7472000640 1 --2- 172.21.15.19:0/2443509955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f746c10d950 0x7f746c10fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.093+0000 7f7472000640 1 --2- 172.21.15.19:0/2443509955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f746c075670 0x7f746c075a80 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.093+0000 7f7472000640 1 -- 172.21.15.19:0/2443509955 >> 172.21.15.19:0/2443509955 conn(0x7f746c0ff160 msgr2=0x7f746c1015b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:09.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.093+0000 7f7472000640 1 -- 172.21.15.19:0/2443509955 shutdown_connections 2024-05-07T02:32:09.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.093+0000 7f7472000640 1 -- 172.21.15.19:0/2443509955 wait complete. 2024-05-07T02:32:09.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.094+0000 7f7472000640 1 Processor -- start 2024-05-07T02:32:09.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.094+0000 7f7472000640 1 -- start start 2024-05-07T02:32:09.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.094+0000 7f7472000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f746c075670 0x7f746c1a2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:09.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.094+0000 7f7472000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f746c10d950 0x7f746c1a2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:09.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.094+0000 7f7472000640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f746c1102b0 0x7f746c19c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:09.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.095+0000 7f7472000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f746c115f10 con 0x7f746c10d950 2024-05-07T02:32:09.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.095+0000 7f7472000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f746c115d90 con 0x7f746c075670 2024-05-07T02:32:09.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.095+0000 7f7472000640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f746c116090 con 0x7f746c1102b0 2024-05-07T02:32:09.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.095+0000 7f746be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f746c10d950 0x7f746c1a2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:09.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.095+0000 7f7471600640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f746c1102b0 0x7f746c19c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:09.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.095+0000 7f7470c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f746c075670 0x7f746c1a2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:09.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.095+0000 7f7470c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f746c075670 0x7f746c1a2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:44070/0 (socket says 172.21.15.19:44070) 2024-05-07T02:32:09.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.095+0000 7f7470c00640 1 -- 172.21.15.19:0/694357704 learned_addr learned my addr 172.21.15.19:0/694357704 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:32:09.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.095+0000 7f7471600640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f746c1102b0 0x7f746c19c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:60482/0 (socket says 172.21.15.19:60482) 2024-05-07T02:32:09.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.095+0000 7f7470c00640 1 -- 172.21.15.19:0/694357704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f746c1102b0 msgr2=0x7f746c19c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:09.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.095+0000 7f7470c00640 1 --2- 172.21.15.19:0/694357704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f746c1102b0 0x7f746c19c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.095+0000 7f7470c00640 1 -- 172.21.15.19:0/694357704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f746c10d950 msgr2=0x7f746c1a2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:09.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.095+0000 7f7470c00640 1 --2- 172.21.15.19:0/694357704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f746c10d950 0x7f746c1a2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.095+0000 7f7470c00640 1 -- 172.21.15.19:0/694357704 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f746c19ce00 con 0x7f746c075670 2024-05-07T02:32:09.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.096+0000 7f7470c00640 1 --2- 172.21.15.19:0/694357704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f746c075670 0x7f746c1a2550 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7460005940 tx=0x7f74600040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:09.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.096+0000 7f7469600640 1 -- 172.21.15.19:0/694357704 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746003f030 con 0x7f746c075670 2024-05-07T02:32:09.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.096+0000 7f7472000640 1 -- 172.21.15.19:0/694357704 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f746c19d030 con 0x7f746c075670 2024-05-07T02:32:09.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.096+0000 7f7469600640 1 -- 172.21.15.19:0/694357704 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74600044c0 con 0x7f746c075670 2024-05-07T02:32:09.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.096+0000 7f7469600640 1 -- 172.21.15.19:0/694357704 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7460035070 con 0x7f746c075670 2024-05-07T02:32:09.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.096+0000 7f7472000640 1 -- 172.21.15.19:0/694357704 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f746c1a92f0 con 0x7f746c075670 2024-05-07T02:32:09.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.097+0000 7f7472000640 1 -- 172.21.15.19:0/694357704 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f746c0767b0 con 0x7f746c075670 2024-05-07T02:32:09.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.101+0000 7f7469600640 1 -- 172.21.15.19:0/694357704 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f7460042030 con 0x7f746c075670 2024-05-07T02:32:09.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.102+0000 7f7469600640 1 --2- 172.21.15.19:0/694357704 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f74540b2a90 0x7f74540b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:09.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.103+0000 7f7469600640 1 -- 172.21.15.19:0/694357704 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f7460103180 con 0x7f746c075670 2024-05-07T02:32:09.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.103+0000 7f746be00640 1 --2- 172.21.15.19:0/694357704 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f74540b2a90 0x7f74540b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:09.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.103+0000 7f746be00640 1 --2- 172.21.15.19:0/694357704 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f74540b2a90 0x7f74540b4fa0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f746c19de70 tx=0x7f7458005f90 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:09.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.104+0000 7f7469600640 1 -- 172.21.15.19:0/694357704 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f74600c9fb0 con 0x7f746c075670 2024-05-07T02:32:09.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.310+0000 7fe8c5a00700 1 -- 172.21.15.19:0/1083157692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8c0103250 msgr2=0x7fe8c0103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:09.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.310+0000 7fe8c5a00700 1 --2- 172.21.15.19:0/1083157692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8c0103250 0x7fe8c0103660 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00098f0 tx=0x7fe8b0009c00 comp rx=0 tx=0).stop 2024-05-07T02:32:09.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.311+0000 7fe8c5a00700 1 -- 172.21.15.19:0/1083157692 shutdown_connections 2024-05-07T02:32:09.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.311+0000 7fe8c5a00700 1 --2- 172.21.15.19:0/1083157692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8c01086a0 0x7fe8c0108b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.311+0000 7fe8c5a00700 1 --2- 172.21.15.19:0/1083157692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8c0103250 0x7fe8c0103660 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.311+0000 7fe8c5a00700 1 --2- 172.21.15.19:0/1083157692 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8c0103ba0 0x7fe8c01080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.311+0000 7fe8c5a00700 1 -- 172.21.15.19:0/1083157692 >> 172.21.15.19:0/1083157692 conn(0x7fe8c00fedc0 msgr2=0x7fe8c01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:09.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.311+0000 7fe8c5a00700 1 -- 172.21.15.19:0/1083157692 shutdown_connections 2024-05-07T02:32:09.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.311+0000 7fe8c5a00700 1 -- 172.21.15.19:0/1083157692 wait complete. 2024-05-07T02:32:09.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.312+0000 7fe8c5a00700 1 Processor -- start 2024-05-07T02:32:09.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.312+0000 7fe8c5a00700 1 -- start start 2024-05-07T02:32:09.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.312+0000 7fe8c5a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8c0103250 0x7fe8c01a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:09.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.312+0000 7fe8c5a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8c0103ba0 0x7fe8c01a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:09.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.312+0000 7fe8c5a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8c01086a0 0x7fe8c01a5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:09.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.312+0000 7fe8c5a00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c011aa70 con 0x7fe8c01086a0 2024-05-07T02:32:09.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.312+0000 7fe8c5a00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c011a8f0 con 0x7fe8c0103ba0 2024-05-07T02:32:09.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.313+0000 7fe8c5a00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c011abf0 con 0x7fe8c0103250 2024-05-07T02:32:09.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.313+0000 7fe8bea00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8c0103250 0x7fe8c01a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:09.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.313+0000 7fe8bf400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8c01086a0 0x7fe8c01a5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:09.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.313+0000 7fe8be000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8c0103ba0 0x7fe8c01a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:09.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.313+0000 7fe8bf400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8c01086a0 0x7fe8c01a5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53786/0 (socket says 172.21.15.19:53786) 2024-05-07T02:32:09.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.313+0000 7fe8be000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8c0103ba0 0x7fe8c01a1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:44080/0 (socket says 172.21.15.19:44080) 2024-05-07T02:32:09.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.313+0000 7fe8bea00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8c0103250 0x7fe8c01a14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:60510/0 (socket says 172.21.15.19:60510) 2024-05-07T02:32:09.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.313+0000 7fe8be000700 1 -- 172.21.15.19:0/586722726 learned_addr learned my addr 172.21.15.19:0/586722726 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:32:09.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.313+0000 7fe8be000700 1 -- 172.21.15.19:0/586722726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8c0103250 msgr2=0x7fe8c01a14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:09.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.313+0000 7fe8be000700 1 --2- 172.21.15.19:0/586722726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8c0103250 0x7fe8c01a14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.313+0000 7fe8be000700 1 -- 172.21.15.19:0/586722726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8c01086a0 msgr2=0x7fe8c01a5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:09.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.313+0000 7fe8be000700 1 --2- 172.21.15.19:0/586722726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8c01086a0 0x7fe8c01a5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.313+0000 7fe8be000700 1 -- 172.21.15.19:0/586722726 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8c01a6370 con 0x7fe8c0103ba0 2024-05-07T02:32:09.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.314+0000 7fe8be000700 1 --2- 172.21.15.19:0/586722726 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8c0103ba0 0x7fe8c01a1a40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac00e9a0 tx=0x7fe8ac00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:09.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.314+0000 7fe8b7400700 1 -- 172.21.15.19:0/586722726 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac00ca10 con 0x7fe8c0103ba0 2024-05-07T02:32:09.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.314+0000 7fe8b7400700 1 -- 172.21.15.19:0/586722726 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8ac00cbb0 con 0x7fe8c0103ba0 2024-05-07T02:32:09.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.314+0000 7fe8c5a00700 1 -- 172.21.15.19:0/586722726 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8c01a6660 con 0x7fe8c0103ba0 2024-05-07T02:32:09.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.314+0000 7fe8c5a00700 1 -- 172.21.15.19:0/586722726 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8c01adf80 con 0x7fe8c0103ba0 2024-05-07T02:32:09.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.314+0000 7fe8b7400700 1 -- 172.21.15.19:0/586722726 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac018680 con 0x7fe8c0103ba0 2024-05-07T02:32:09.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.318+0000 7fe8b7400700 1 -- 172.21.15.19:0/586722726 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7fe8ac018820 con 0x7fe8c0103ba0 2024-05-07T02:32:09.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.319+0000 7fe8d451ab80 1 Processor -- start 2024-05-07T02:32:09.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.319+0000 7fe8d451ab80 1 -- start start 2024-05-07T02:32:09.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.320+0000 7fe8d451ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c318694f0 0x563c31869900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:09.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.320+0000 7fe8d451ab80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c318636a0 0x563c31863ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:09.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.320+0000 7fe8d451ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c31863ff0 0x563c318701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:09.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.320+0000 7fe8d451ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563c31785aa0 con 0x563c31863ff0 2024-05-07T02:32:09.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.320+0000 7fe8d451ab80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x563c317237e0 con 0x563c318636a0 2024-05-07T02:32:09.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.320+0000 7fe8d451ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563c31781a60 con 0x563c318694f0 2024-05-07T02:32:09.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.320+0000 7fe8bf400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c318694f0 0x563c31869900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:09.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.320+0000 7fe8bea00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c318636a0 0x563c31863ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:09.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.320+0000 7fe8bf400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c318694f0 0x563c31869900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:60524/0 (socket says 172.21.15.19:60524) 2024-05-07T02:32:09.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.320+0000 7fe8bf400700 1 -- 172.21.15.19:0/1254288949 learned_addr learned my addr 172.21.15.19:0/1254288949 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:32:09.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.320+0000 7fe8be000700 1 --2- 172.21.15.19:0/1254288949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c31863ff0 0x563c318701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:09.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.321+0000 7fe8bf400700 1 -- 172.21.15.19:0/1254288949 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c318636a0 msgr2=0x563c31863ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:32:09.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.321+0000 7fe8bf400700 1 --2- 172.21.15.19:0/1254288949 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c318636a0 0x563c31863ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.321+0000 7fe8bf400700 1 -- 172.21.15.19:0/1254288949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c31863ff0 msgr2=0x563c318701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:32:09.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.321+0000 7fe8bf400700 1 --2- 172.21.15.19:0/1254288949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c31863ff0 0x563c318701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.321+0000 7fe8bf400700 1 -- 172.21.15.19:0/1254288949 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c316d2950 con 0x563c318694f0 2024-05-07T02:32:09.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.321+0000 7fe8b7400700 1 --2- 172.21.15.19:0/586722726 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fe8a40b2b00 0x7fe8a40b5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:09.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.321+0000 7fe8b7400700 1 -- 172.21.15.19:0/586722726 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fe8ac0e2cf0 con 0x7fe8c0103ba0 2024-05-07T02:32:09.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.321+0000 7fe8bea00700 1 --2- 172.21.15.19:0/586722726 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fe8a40b2b00 0x7fe8a40b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:09.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.321+0000 7fe8bf400700 1 --2- 172.21.15.19:0/1254288949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c318694f0 0x563c31869900 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe8b80093a0 tx=0x7fe8b80099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfab2a3bbe1a5f43 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:09.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.321+0000 7fe8abe00700 1 -- 172.21.15.19:0/1254288949 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8003c00 con 0x563c318694f0 2024-05-07T02:32:09.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.322+0000 7fe8abe00700 1 -- 172.21.15.19:0/1254288949 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8b8003da0 con 0x563c318694f0 2024-05-07T02:32:09.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.322+0000 7fe8abe00700 1 -- 172.21.15.19:0/1254288949 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8017880 con 0x563c318694f0 2024-05-07T02:32:09.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.322+0000 7fe8d451ab80 1 -- 172.21.15.19:0/1254288949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c318694f0 msgr2=0x563c31869900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:09.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.322+0000 7fe8d451ab80 1 --2- 172.21.15.19:0/1254288949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c318694f0 0x563c31869900 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe8b80093a0 tx=0x7fe8b80099e0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.322+0000 7fe8bea00700 1 --2- 172.21.15.19:0/586722726 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fe8a40b2b00 0x7fe8a40b5010 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00098c0 tx=0x7fe8b0019040 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:09.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.323+0000 7fe8d451ab80 1 -- 172.21.15.19:0/1254288949 shutdown_connections 2024-05-07T02:32:09.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.323+0000 7fe8d451ab80 1 --2- 172.21.15.19:0/1254288949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c31863ff0 0x563c318701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.323+0000 7fe8d451ab80 1 --2- 172.21.15.19:0/1254288949 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c318694f0 0x563c31869900 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.323+0000 7fe8d451ab80 1 --2- 172.21.15.19:0/1254288949 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c318636a0 0x563c31863ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.323+0000 7fe8d451ab80 1 -- 172.21.15.19:0/1254288949 >> 172.21.15.19:0/1254288949 conn(0x563c316e5ec0 msgr2=0x563c3185e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:09.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.323+0000 7fe8d451ab80 1 -- 172.21.15.19:0/1254288949 shutdown_connections 2024-05-07T02:32:09.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.323+0000 7fe8d451ab80 1 -- 172.21.15.19:0/1254288949 wait complete. 2024-05-07T02:32:09.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.323+0000 7fe8d451ab80 1 -- start start 2024-05-07T02:32:09.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.324+0000 7fe8d451ab80 1 Processor -- start 2024-05-07T02:32:09.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.324+0000 7fe8d451ab80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c318636a0 0x563c3186d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:09.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.324+0000 7fe8d451ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c31863ff0 0x563c3186dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:09.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.324+0000 7fe8d451ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c318694f0 0x563c3186e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:09.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.324+0000 7fe8d451ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563c318708c0 con 0x563c31863ff0 2024-05-07T02:32:09.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.324+0000 7fe8d451ab80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x563c316ebba0 con 0x563c318636a0 2024-05-07T02:32:09.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.324+0000 7fe8d451ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563c31870a40 con 0x563c318694f0 2024-05-07T02:32:09.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.324+0000 7fe8d451ab80 10 client.0 _ll_register_callbacks cb 0x563c31719400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:32:09.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.325+0000 7fe8be000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c31863ff0 0x563c3186dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:09.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.325+0000 7fe8be000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c31863ff0 0x563c3186dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53796/0 (socket says 172.21.15.19:53796) 2024-05-07T02:32:09.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.325+0000 7fe8bea00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c318636a0 0x563c3186d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:09.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.325+0000 7fe8bf400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c318694f0 0x563c3186e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:09.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.325+0000 7fe8be000700 1 -- 172.21.15.19:0/3461626421 learned_addr learned my addr 172.21.15.19:0/3461626421 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:32:09.327 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:09 smithi019 ceph-mon[29781]: from='client.? ' entity='client.admin' 2024-05-07T02:32:09.327 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:09 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:32:09.327 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:09 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:32:09.327 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:09 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:32:09.327 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:09 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:32:09.327 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:09 smithi019 ceph-mon[29781]: from='client.25366 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:32:09.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.326+0000 7fe8bea00700 1 -- 172.21.15.19:0/3461626421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c318694f0 msgr2=0x563c3186e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:32:09.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.326+0000 7fe8bea00700 1 --2- 172.21.15.19:0/3461626421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c318694f0 0x563c3186e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.326+0000 7fe8bea00700 1 -- 172.21.15.19:0/3461626421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c31863ff0 msgr2=0x563c3186dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:32:09.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.326+0000 7fe8bea00700 1 --2- 172.21.15.19:0/3461626421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c31863ff0 0x563c3186dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.326+0000 7fe8bea00700 1 -- 172.21.15.19:0/3461626421 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c3174b4a0 con 0x563c318636a0 2024-05-07T02:32:09.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.326+0000 7fe8be000700 1 --2- 172.21.15.19:0/3461626421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c31863ff0 0x563c3186dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:32:09.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.326+0000 7fe8bea00700 1 --2- 172.21.15.19:0/3461626421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c318636a0 0x563c3186d6c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe8b000c010 tx=0x7fe8b0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d09c6e3d59a4bcc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:09.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.326+0000 7fe8a8c00700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:32:09.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.326+0000 7fe8a8c00700 1 -- 172.21.15.19:0/3461626421 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b0034060 con 0x563c318636a0 2024-05-07T02:32:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.326+0000 7fe8d451ab80 1 -- 172.21.15.19:0/3461626421 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8b8003da0 con 0x563c318636a0 2024-05-07T02:32:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.326+0000 7fe8a8c00700 1 -- 172.21.15.19:0/3461626421 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fe8b00052d0 con 0x563c318636a0 2024-05-07T02:32:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.327+0000 7fe8a8c00700 1 -- 172.21.15.19:0/3461626421 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe8b00054d0 con 0x563c318636a0 2024-05-07T02:32:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.327+0000 7fe8d451ab80 10 client.25403 fetch_fsmap learned FSMap version 31 2024-05-07T02:32:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.327+0000 7fe8d451ab80 1 -- 172.21.15.19:0/3461626421 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563c3187b4e0 con 0x563c318636a0 2024-05-07T02:32:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.327+0000 7fe8a8c00700 1 -- 172.21.15.19:0/3461626421 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8b8003da0 con 0x563c318636a0 2024-05-07T02:32:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.327+0000 7fe8a8c00700 1 -- 172.21.15.19:0/3461626421 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fe8b000b790 con 0x563c318636a0 2024-05-07T02:32:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.328+0000 7fe8d451ab80 10 client.25403 fetch_fsmap finished waiting for FSMap version 31 2024-05-07T02:32:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.328+0000 7fe8d451ab80 10 client.25403 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:32:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.328+0000 7fe8d451ab80 20 client.25403 populate_metadata read hostname 'smithi019' 2024-05-07T02:32:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.328+0000 7fe8d451ab80 1 --2- 172.21.15.19:0/3461626421 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x563c3187da70 0x563c3187fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.328+0000 7fe8d451ab80 4 client.25403 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:32:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.328+0000 7fe8d451ab80 1 -- 172.21.15.19:0/3461626421 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c31830160 con 0x563c3187da70 2024-05-07T02:32:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.328+0000 7fe8be000700 1 --2- 172.21.15.19:0/3461626421 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x563c3187da70 0x563c3187fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:09.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.329+0000 7fe8be000700 1 --2- 172.21.15.19:0/3461626421 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x563c3187da70 0x563c3187fe90 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a39dbe778412ea10 server_cookie=50dbc75256882ee3 in_seq=0 out_seq=0 2024-05-07T02:32:09.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.329+0000 7fe8a8c00700 10 client.25403 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:32:09.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.329+0000 7fe8a8c00700 1 -- 172.21.15.19:0/3461626421 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563c31830160 con 0x563c3187da70 2024-05-07T02:32:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.329+0000 7fe8a8c00700 10 client.25403 handle_command_reply: tid=0 2024-05-07T02:32:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.330+0000 7fe8d451ab80 1 client.25403 shutdown 2024-05-07T02:32:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.330+0000 7fe8d451ab80 1 -- 172.21.15.19:0/3461626421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c318636a0 msgr2=0x563c3186d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.330+0000 7fe8d451ab80 1 --2- 172.21.15.19:0/3461626421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c318636a0 0x563c3186d6c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe8b000c010 tx=0x7fe8b0009c00 comp rx=0 tx=0).stop 2024-05-07T02:32:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.330+0000 7fe8d451ab80 1 -- 172.21.15.19:0/3461626421 shutdown_connections 2024-05-07T02:32:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.330+0000 7fe8d451ab80 1 --2- 172.21.15.19:0/3461626421 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x563c3187da70 0x563c3187fe90 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.330+0000 7fe8d451ab80 1 --2- 172.21.15.19:0/3461626421 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c318694f0 0x563c3186e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.330+0000 7fe8d451ab80 1 --2- 172.21.15.19:0/3461626421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c318636a0 0x563c3186d6c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.330+0000 7fe8d451ab80 1 --2- 172.21.15.19:0/3461626421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c31863ff0 0x563c3186dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.330+0000 7fe8a8c00700 0 client.25403 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:32:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.330+0000 7fe8d451ab80 1 -- 172.21.15.19:0/3461626421 >> 172.21.15.19:0/3461626421 conn(0x563c316e5ec0 msgr2=0x563c31862690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.330+0000 7fe8d451ab80 1 -- 172.21.15.19:0/3461626421 shutdown_connections 2024-05-07T02:32:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.330+0000 7fe8d451ab80 1 -- 172.21.15.19:0/3461626421 wait complete. 2024-05-07T02:32:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.330+0000 7fe8d451ab80 20 client.25403 trim_cache size 0 max 16384 2024-05-07T02:32:09.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.349+0000 7f7472000640 1 -- 172.21.15.19:0/694357704 --> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f746c110e10 con 0x7f74540b2a90 2024-05-07T02:32:09.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.354+0000 7fe8d451ab80 1 Processor -- start 2024-05-07T02:32:09.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.354+0000 7fe8d451ab80 1 -- start start 2024-05-07T02:32:09.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.354+0000 7fe8d451ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c318636a0 0x563c3186e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:09.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.354+0000 7fe8d451ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c31863ff0 0x563c31879950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:09.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.354+0000 7fe8d451ab80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c318694f0 0x563c3186f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:09.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.354+0000 7fe8d451ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c01b6c80 con 0x563c318636a0 2024-05-07T02:32:09.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.354+0000 7fe8d451ab80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c01b87c0 con 0x563c318694f0 2024-05-07T02:32:09.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.354+0000 7fe8d451ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c01b7350 con 0x563c31863ff0 2024-05-07T02:32:09.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.355+0000 7fe8be000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c31863ff0 0x563c31879950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:09.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.355+0000 7fe8bea00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c318636a0 0x563c3186e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:09.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.355+0000 7fe8be000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c31863ff0 0x563c31879950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:60542/0 (socket says 172.21.15.19:60542) 2024-05-07T02:32:09.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.355+0000 7fe8be000700 1 -- 172.21.15.19:0/1818674254 learned_addr learned my addr 172.21.15.19:0/1818674254 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:32:09.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.355+0000 7fe8bf400700 1 --2- 172.21.15.19:0/1818674254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c318694f0 0x563c3186f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:09.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.355+0000 7fe8be000700 1 -- 172.21.15.19:0/1818674254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c318694f0 msgr2=0x563c3186f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:32:09.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.355+0000 7fe8be000700 1 --2- 172.21.15.19:0/1818674254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c318694f0 0x563c3186f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.355+0000 7fe8be000700 1 -- 172.21.15.19:0/1818674254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c318636a0 msgr2=0x563c3186e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:32:09.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.355+0000 7fe8be000700 1 --2- 172.21.15.19:0/1818674254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c318636a0 0x563c3186e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.355+0000 7fe8be000700 1 -- 172.21.15.19:0/1818674254 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c3185ac00 con 0x563c31863ff0 2024-05-07T02:32:09.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.355+0000 7fe8be000700 1 --2- 172.21.15.19:0/1818674254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c31863ff0 0x563c31879950 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac00e970 tx=0x7fe8ac014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2c196f9bc3b5d69 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:09.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.356+0000 7fe8ab400700 1 -- 172.21.15.19:0/1818674254 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac0b2210 con 0x563c31863ff0 2024-05-07T02:32:09.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.356+0000 7fe8ab400700 1 -- 172.21.15.19:0/1818674254 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8ac0a9580 con 0x563c31863ff0 2024-05-07T02:32:09.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.356+0000 7fe8ab400700 1 -- 172.21.15.19:0/1818674254 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac0a9cd0 con 0x563c31863ff0 2024-05-07T02:32:09.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.357+0000 7fe8d451ab80 1 -- 172.21.15.19:0/1818674254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c31863ff0 msgr2=0x563c31879950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:09.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.357+0000 7fe8d451ab80 1 --2- 172.21.15.19:0/1818674254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c31863ff0 0x563c31879950 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac00e970 tx=0x7fe8ac014000 comp rx=0 tx=0).stop 2024-05-07T02:32:09.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.357+0000 7fe8d451ab80 1 -- 172.21.15.19:0/1818674254 shutdown_connections 2024-05-07T02:32:09.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.357+0000 7fe8d451ab80 1 --2- 172.21.15.19:0/1818674254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c318694f0 0x563c3186f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.357+0000 7fe8d451ab80 1 --2- 172.21.15.19:0/1818674254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c318636a0 0x563c3186e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.357+0000 7fe8d451ab80 1 --2- 172.21.15.19:0/1818674254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c31863ff0 0x563c31879950 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.357+0000 7fe8d451ab80 1 -- 172.21.15.19:0/1818674254 >> 172.21.15.19:0/1818674254 conn(0x563c316e5ec0 msgr2=0x563c3186ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:09.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.357+0000 7fe8d451ab80 1 -- 172.21.15.19:0/1818674254 shutdown_connections 2024-05-07T02:32:09.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.357+0000 7fe8d451ab80 1 -- 172.21.15.19:0/1818674254 wait complete. 2024-05-07T02:32:09.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:09 smithi064 ceph-mon[34699]: from='client.? ' entity='client.admin' 2024-05-07T02:32:09.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:09 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:32:09.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:09 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:32:09.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:09 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:32:09.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:09 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:32:09.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:09 smithi064 ceph-mon[34699]: from='client.25366 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:32:09.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.358+0000 7fe8d451ab80 1 -- start start 2024-05-07T02:32:09.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.358+0000 7fe8d451ab80 1 Processor -- start 2024-05-07T02:32:09.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.359+0000 7fe8d451ab80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c318636a0 0x563c31876000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:09.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.359+0000 7fe8d451ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c31863ff0 0x563c318723c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:09.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.359+0000 7fe8d451ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c318694f0 0x563c31872900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:09.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.359+0000 7fe8d451ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c01b80f0 con 0x563c31863ff0 2024-05-07T02:32:09.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.359+0000 7fe8d451ab80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c01b7a20 con 0x563c318636a0 2024-05-07T02:32:09.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.359+0000 7fe8d451ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c01b4c20 con 0x563c318694f0 2024-05-07T02:32:09.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.359+0000 7fe8d451ab80 10 client.0 _ll_register_callbacks cb 0x563c31719400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:32:09.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.359+0000 7fe8bea00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c318636a0 0x563c31876000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:09.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.359+0000 7fe8bea00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c318636a0 0x563c31876000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:44104/0 (socket says 172.21.15.19:44104) 2024-05-07T02:32:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.359+0000 7fe8bea00700 1 -- 172.21.15.19:0/2373889576 learned_addr learned my addr 172.21.15.19:0/2373889576 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:32:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.359+0000 7fe8be000700 1 --2- 172.21.15.19:0/2373889576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c31863ff0 0x563c318723c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.359+0000 7fe8bf400700 1 --2- 172.21.15.19:0/2373889576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c318694f0 0x563c31872900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.359+0000 7fe8bea00700 1 -- 172.21.15.19:0/2373889576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c318694f0 msgr2=0x563c31872900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:32:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.359+0000 7fe8bea00700 1 --2- 172.21.15.19:0/2373889576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c318694f0 0x563c31872900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.359+0000 7fe8bea00700 1 -- 172.21.15.19:0/2373889576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c31863ff0 msgr2=0x563c318723c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:32:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.360+0000 7fe8bea00700 1 --2- 172.21.15.19:0/2373889576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c31863ff0 0x563c318723c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.360+0000 7fe8bea00700 1 -- 172.21.15.19:0/2373889576 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c31872e40 con 0x563c318636a0 2024-05-07T02:32:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.360+0000 7fe8be000700 1 --2- 172.21.15.19:0/2373889576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c31863ff0 0x563c318723c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:32:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.360+0000 7fe8bea00700 1 --2- 172.21.15.19:0/2373889576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c318636a0 0x563c31876000 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00098f0 tx=0x7fe8b000ba60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9ceb489bc263dcc server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.360+0000 7fe8a8c00700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:32:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.360+0000 7fe8a8c00700 1 -- 172.21.15.19:0/2373889576 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b0028070 con 0x563c318636a0 2024-05-07T02:32:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.360+0000 7fe8d451ab80 1 -- 172.21.15.19:0/2373889576 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8ac0a9580 con 0x563c318636a0 2024-05-07T02:32:09.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.361+0000 7fe8a8c00700 1 -- 172.21.15.19:0/2373889576 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fe8b00052d0 con 0x563c318636a0 2024-05-07T02:32:09.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.361+0000 7fe8a8c00700 1 -- 172.21.15.19:0/2373889576 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe8b0046050 con 0x563c318636a0 2024-05-07T02:32:09.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.361+0000 7fe8d451ab80 10 client.25409 fetch_fsmap learned FSMap version 31 2024-05-07T02:32:09.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.361+0000 7fe8d451ab80 1 -- 172.21.15.19:0/2373889576 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563c318e8f20 con 0x563c318636a0 2024-05-07T02:32:09.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.361+0000 7fe8a8c00700 1 -- 172.21.15.19:0/2373889576 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8ac0a9580 con 0x563c318636a0 2024-05-07T02:32:09.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.361+0000 7fe8a8c00700 1 -- 172.21.15.19:0/2373889576 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fe8b0024720 con 0x563c318636a0 2024-05-07T02:32:09.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.361+0000 7fe8d451ab80 10 client.25409 fetch_fsmap finished waiting for FSMap version 31 2024-05-07T02:32:09.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.361+0000 7fe8d451ab80 10 client.25409 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:32:09.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.361+0000 7fe8d451ab80 20 client.25409 populate_metadata read hostname 'smithi019' 2024-05-07T02:32:09.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.361+0000 7fe8d451ab80 1 --2- 172.21.15.19:0/2373889576 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x563c318e95a0 0x563c318eb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:09.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.362+0000 7fe8d451ab80 4 client.25409 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-07T02:32:09.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.362+0000 7fe8d451ab80 1 -- 172.21.15.19:0/2373889576 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563c31793480 con 0x563c318e95a0 2024-05-07T02:32:09.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.362+0000 7fe8be000700 1 --2- 172.21.15.19:0/2373889576 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x563c318e95a0 0x563c318eb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:09.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.362+0000 7fe8a8c00700 10 client.25409 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:32:09.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.362+0000 7fe8be000700 1 --2- 172.21.15.19:0/2373889576 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x563c318e95a0 0x563c318eb9c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adab7368efa0d6af server_cookie=9eb919829991d739 in_seq=0 out_seq=0 2024-05-07T02:32:09.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.364+0000 7fe8a8c00700 1 -- 172.21.15.19:0/2373889576 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563c31793480 con 0x563c318e95a0 2024-05-07T02:32:09.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.364+0000 7fe8a8c00700 10 client.25409 handle_command_reply: tid=0 2024-05-07T02:32:09.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.364+0000 7fe8d451ab80 1 client.25409 shutdown 2024-05-07T02:32:09.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.364+0000 7fe8d451ab80 1 -- 172.21.15.19:0/2373889576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c318636a0 msgr2=0x563c31876000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:09.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.364+0000 7fe8d451ab80 1 --2- 172.21.15.19:0/2373889576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c318636a0 0x563c31876000 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00098f0 tx=0x7fe8b000ba60 comp rx=0 tx=0).stop 2024-05-07T02:32:09.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.364+0000 7fe8d451ab80 1 -- 172.21.15.19:0/2373889576 shutdown_connections 2024-05-07T02:32:09.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.364+0000 7fe8d451ab80 1 --2- 172.21.15.19:0/2373889576 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x563c318e95a0 0x563c318eb9c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.364+0000 7fe8d451ab80 1 --2- 172.21.15.19:0/2373889576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563c318694f0 0x563c31872900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.365+0000 7fe8d451ab80 1 --2- 172.21.15.19:0/2373889576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563c318636a0 0x563c31876000 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.365+0000 7fe8d451ab80 1 --2- 172.21.15.19:0/2373889576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563c31863ff0 0x563c318723c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.365+0000 7fe8d451ab80 1 -- 172.21.15.19:0/2373889576 >> 172.21.15.19:0/2373889576 conn(0x563c316e5ec0 msgr2=0x563c3186de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:09.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.365+0000 7fe8a8c00700 0 client.25409 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:32:09.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.365+0000 7fe8d451ab80 1 -- 172.21.15.19:0/2373889576 shutdown_connections 2024-05-07T02:32:09.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.365+0000 7fe8d451ab80 1 -- 172.21.15.19:0/2373889576 wait complete. 2024-05-07T02:32:09.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.365+0000 7fe8d451ab80 20 client.25409 trim_cache size 0 max 16384 2024-05-07T02:32:09.367 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:32:09.367 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-07T02:32:09.367 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "ed137034-a0c9-4805-8bfd-2512803393e4", 2024-05-07T02:32:09.367 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-07T02:32:09.367 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:32:09.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.366+0000 7fe8c5a00700 1 -- 172.21.15.19:0/586722726 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fe8a40b2b00 msgr2=0x7fe8a40b5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:09.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.366+0000 7fe8c5a00700 1 --2- 172.21.15.19:0/586722726 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fe8a40b2b00 0x7fe8a40b5010 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00098c0 tx=0x7fe8b0019040 comp rx=0 tx=0).stop 2024-05-07T02:32:09.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.366+0000 7fe8c5a00700 1 -- 172.21.15.19:0/586722726 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8c0103ba0 msgr2=0x7fe8c01a1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:09.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.366+0000 7fe8c5a00700 1 --2- 172.21.15.19:0/586722726 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8c0103ba0 0x7fe8c01a1a40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac00e9a0 tx=0x7fe8ac00ecb0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.366+0000 7fe8c5a00700 1 -- 172.21.15.19:0/586722726 shutdown_connections 2024-05-07T02:32:09.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.366+0000 7fe8c5a00700 1 --2- 172.21.15.19:0/586722726 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fe8a40b2b00 0x7fe8a40b5010 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.366+0000 7fe8c5a00700 1 --2- 172.21.15.19:0/586722726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8c01086a0 0x7fe8c01a5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.366+0000 7fe8c5a00700 1 --2- 172.21.15.19:0/586722726 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe8c0103250 0x7fe8c01a14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.366+0000 7fe8c5a00700 1 --2- 172.21.15.19:0/586722726 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8c0103ba0 0x7fe8c01a1a40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.366+0000 7fe8c5a00700 1 -- 172.21.15.19:0/586722726 >> 172.21.15.19:0/586722726 conn(0x7fe8c00fedc0 msgr2=0x7fe8c0106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:09.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.366+0000 7fe8c5a00700 1 -- 172.21.15.19:0/586722726 shutdown_connections 2024-05-07T02:32:09.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.366+0000 7fe8c5a00700 1 -- 172.21.15.19:0/586722726 wait complete. 2024-05-07T02:32:09.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.433+0000 7f7469600640 1 -- 172.21.15.19:0/694357704 <== mgr.25234 v2:172.21.15.19:6800/3265825478 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f746c110e10 con 0x7f74540b2a90 2024-05-07T02:32:09.434 INFO:teuthology.orchestra.run.smithi019.stdout:Retention added to path / 2024-05-07T02:32:09.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.436+0000 7f7472000640 1 -- 172.21.15.19:0/694357704 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f74540b2a90 msgr2=0x7f74540b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:09.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.436+0000 7f7472000640 1 --2- 172.21.15.19:0/694357704 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f74540b2a90 0x7f74540b4fa0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f746c19de70 tx=0x7f7458005f90 comp rx=0 tx=0).stop 2024-05-07T02:32:09.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.436+0000 7f7472000640 1 -- 172.21.15.19:0/694357704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f746c075670 msgr2=0x7f746c1a2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:09.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.436+0000 7f7472000640 1 --2- 172.21.15.19:0/694357704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f746c075670 0x7f746c1a2550 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7460005940 tx=0x7f74600040c0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.436+0000 7f7472000640 1 -- 172.21.15.19:0/694357704 shutdown_connections 2024-05-07T02:32:09.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.436+0000 7f7472000640 1 --2- 172.21.15.19:0/694357704 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f74540b2a90 0x7f74540b4fa0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.436+0000 7f7472000640 1 --2- 172.21.15.19:0/694357704 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f746c1102b0 0x7f746c19c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.436+0000 7f7472000640 1 --2- 172.21.15.19:0/694357704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f746c10d950 0x7f746c1a2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.436+0000 7f7472000640 1 --2- 172.21.15.19:0/694357704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f746c075670 0x7f746c1a2550 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.436+0000 7f7472000640 1 -- 172.21.15.19:0/694357704 >> 172.21.15.19:0/694357704 conn(0x7f746c0ff160 msgr2=0x7f746c110700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:09.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.436+0000 7f7472000640 1 -- 172.21.15.19:0/694357704 shutdown_connections 2024-05-07T02:32:09.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.436+0000 7f7472000640 1 -- 172.21.15.19:0/694357704 wait complete. 2024-05-07T02:32:09.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:09 smithi027 ceph-mon[34398]: from='client.? ' entity='client.admin' 2024-05-07T02:32:09.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:09 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:32:09.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:09 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:32:09.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:09 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:32:09.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:09 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:32:09.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:09 smithi027 ceph-mon[34398]: from='client.25366 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:32:09.463 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-07T02:32:09.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.711+0000 7f1f97400640 1 -- 172.21.15.19:0/2764299668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f90104d90 msgr2=0x7f1f901051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:09.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.711+0000 7f1f97400640 1 --2- 172.21.15.19:0/2764299668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f90104d90 0x7f1f901051a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1f88009be0 tx=0x7f1f8802a6c0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.712+0000 7f1f97400640 1 -- 172.21.15.19:0/2764299668 shutdown_connections 2024-05-07T02:32:09.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.712+0000 7f1f97400640 1 --2- 172.21.15.19:0/2764299668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f9010e1e0 0x7f1f901106d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.712+0000 7f1f97400640 1 --2- 172.21.15.19:0/2764299668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f9010b7f0 0x7f1f9010dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.712+0000 7f1f97400640 1 --2- 172.21.15.19:0/2764299668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f90104d90 0x7f1f901051a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.712+0000 7f1f97400640 1 -- 172.21.15.19:0/2764299668 >> 172.21.15.19:0/2764299668 conn(0x7f1f900fe390 msgr2=0x7f1f901007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:09.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.712+0000 7f1f97400640 1 -- 172.21.15.19:0/2764299668 shutdown_connections 2024-05-07T02:32:09.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.712+0000 7f1f97400640 1 -- 172.21.15.19:0/2764299668 wait complete. 2024-05-07T02:32:09.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.713+0000 7f1f97400640 1 Processor -- start 2024-05-07T02:32:09.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.713+0000 7f1f97400640 1 -- start start 2024-05-07T02:32:09.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.714+0000 7f1f97400640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f90104d90 0x7f1f9019a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:09.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.714+0000 7f1f97400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f9010b7f0 0x7f1f9019ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:09.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.714+0000 7f1f97400640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f9010e1e0 0x7f1f9019f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:09.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.714+0000 7f1f97400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1f90113e40 con 0x7f1f9010b7f0 2024-05-07T02:32:09.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.714+0000 7f1f97400640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1f90113cc0 con 0x7f1f9010e1e0 2024-05-07T02:32:09.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.714+0000 7f1f97400640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1f90113fc0 con 0x7f1f90104d90 2024-05-07T02:32:09.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.714+0000 7f1f94a00640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f90104d90 0x7f1f9019a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:09.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.714+0000 7f1f8fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f9010b7f0 0x7f1f9019ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:09.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.714+0000 7f1f94a00640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f90104d90 0x7f1f9019a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:60564/0 (socket says 172.21.15.19:60564) 2024-05-07T02:32:09.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.714+0000 7f1f94a00640 1 -- 172.21.15.19:0/3708081837 learned_addr learned my addr 172.21.15.19:0/3708081837 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:32:09.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.714+0000 7f1f95400640 1 --2- 172.21.15.19:0/3708081837 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f9010e1e0 0x7f1f9019f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:09.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.714+0000 7f1f94a00640 1 -- 172.21.15.19:0/3708081837 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f9010e1e0 msgr2=0x7f1f9019f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:09.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.714+0000 7f1f94a00640 1 --2- 172.21.15.19:0/3708081837 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f9010e1e0 0x7f1f9019f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.714+0000 7f1f94a00640 1 -- 172.21.15.19:0/3708081837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f9010b7f0 msgr2=0x7f1f9019ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:09.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.714+0000 7f1f94a00640 1 --2- 172.21.15.19:0/3708081837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f9010b7f0 0x7f1f9019ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.714+0000 7f1f94a00640 1 -- 172.21.15.19:0/3708081837 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f9019f660 con 0x7f1f90104d90 2024-05-07T02:32:09.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.715+0000 7f1f94a00640 1 --2- 172.21.15.19:0/3708081837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f90104d90 0x7f1f9019a800 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1f88004af0 tx=0x7f1f88003c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:09.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.715+0000 7f1f8d600640 1 -- 172.21.15.19:0/3708081837 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f88040020 con 0x7f1f90104d90 2024-05-07T02:32:09.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.715+0000 7f1f8d600640 1 -- 172.21.15.19:0/3708081837 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f88030970 con 0x7f1f90104d90 2024-05-07T02:32:09.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.715+0000 7f1f97400640 1 -- 172.21.15.19:0/3708081837 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f9019f8f0 con 0x7f1f90104d90 2024-05-07T02:32:09.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.715+0000 7f1f8d600640 1 -- 172.21.15.19:0/3708081837 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f88036040 con 0x7f1f90104d90 2024-05-07T02:32:09.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.715+0000 7f1f97400640 1 -- 172.21.15.19:0/3708081837 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f9019fe30 con 0x7f1f90104d90 2024-05-07T02:32:09.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.716+0000 7f1f97400640 1 -- 172.21.15.19:0/3708081837 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f90050870 con 0x7f1f90104d90 2024-05-07T02:32:09.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.719+0000 7f1f8d600640 1 -- 172.21.15.19:0/3708081837 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f1f88043020 con 0x7f1f90104d90 2024-05-07T02:32:09.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.720+0000 7f1f8d600640 1 --2- 172.21.15.19:0/3708081837 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f1f740b2b10 0x7f1f740b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:09.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.721+0000 7f1f8d600640 1 -- 172.21.15.19:0/3708081837 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f1f88103a90 con 0x7f1f90104d90 2024-05-07T02:32:09.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.721+0000 7f1f8fe00640 1 --2- 172.21.15.19:0/3708081837 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f1f740b2b10 0x7f1f740b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:09.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.722+0000 7f1f8fe00640 1 --2- 172.21.15.19:0/3708081837 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f1f740b2b10 0x7f1f740b5020 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1f90101ea0 tx=0x7f1f78005f90 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:09.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.723+0000 7f1f8d600640 1 -- 172.21.15.19:0/3708081837 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1f8804a020 con 0x7f1f90104d90 2024-05-07T02:32:09.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.963+0000 7f1f97400640 1 -- 172.21.15.19:0/3708081837 --> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f1f9010edd0 con 0x7f1f740b2b10 2024-05-07T02:32:09.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.976+0000 7f1f8d600640 1 -- 172.21.15.19:0/3708081837 <== mgr.25234 v2:172.21.15.19:6800/3265825478 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f1f9010edd0 con 0x7f1f740b2b10 2024-05-07T02:32:09.977 INFO:teuthology.orchestra.run.smithi019.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-07T00:00:00", "created": "2024-05-07T02:32:08", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-05-07T02:32:09.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.979+0000 7f1f97400640 1 -- 172.21.15.19:0/3708081837 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f1f740b2b10 msgr2=0x7f1f740b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:09.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.979+0000 7f1f97400640 1 --2- 172.21.15.19:0/3708081837 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f1f740b2b10 0x7f1f740b5020 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1f90101ea0 tx=0x7f1f78005f90 comp rx=0 tx=0).stop 2024-05-07T02:32:09.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.979+0000 7f1f97400640 1 -- 172.21.15.19:0/3708081837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f90104d90 msgr2=0x7f1f9019a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:09.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.979+0000 7f1f97400640 1 --2- 172.21.15.19:0/3708081837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f90104d90 0x7f1f9019a800 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1f88004af0 tx=0x7f1f88003c20 comp rx=0 tx=0).stop 2024-05-07T02:32:09.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.979+0000 7f1f97400640 1 -- 172.21.15.19:0/3708081837 shutdown_connections 2024-05-07T02:32:09.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.979+0000 7f1f97400640 1 --2- 172.21.15.19:0/3708081837 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f9010e1e0 0x7f1f9019f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.979+0000 7f1f97400640 1 --2- 172.21.15.19:0/3708081837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f9010b7f0 0x7f1f9019ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.979+0000 7f1f97400640 1 --2- 172.21.15.19:0/3708081837 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f1f740b2b10 0x7f1f740b5020 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.979+0000 7f1f97400640 1 --2- 172.21.15.19:0/3708081837 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f90104d90 0x7f1f9019a800 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:09.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.979+0000 7f1f97400640 1 -- 172.21.15.19:0/3708081837 >> 172.21.15.19:0/3708081837 conn(0x7f1f900fe390 msgr2=0x7f1f9010e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:09.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.979+0000 7f1f97400640 1 -- 172.21.15.19:0/3708081837 shutdown_connections 2024-05-07T02:32:09.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:09.979+0000 7f1f97400640 1 -- 172.21.15.19:0/3708081837 wait complete. 2024-05-07T02:32:10.005 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-07T02:32:10.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.270+0000 7fb599c00640 1 -- 172.21.15.19:0/2921901553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb59410e910 msgr2=0x7fb59410ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:10.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.270+0000 7fb599c00640 1 --2- 172.21.15.19:0/2921901553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb59410e910 0x7fb59410ed40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb57c009900 tx=0x7fb57c02a410 comp rx=0 tx=0).stop 2024-05-07T02:32:10.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.270+0000 7fb599c00640 1 -- 172.21.15.19:0/2921901553 shutdown_connections 2024-05-07T02:32:10.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.270+0000 7fb599c00640 1 --2- 172.21.15.19:0/2921901553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb59410f3d0 0x7fb594116b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:10.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.270+0000 7fb599c00640 1 --2- 172.21.15.19:0/2921901553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb59410e910 0x7fb59410ed40 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:10.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.270+0000 7fb599c00640 1 --2- 172.21.15.19:0/2921901553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb59410a3a0 0x7fb59410a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:10.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.270+0000 7fb599c00640 1 -- 172.21.15.19:0/2921901553 >> 172.21.15.19:0/2921901553 conn(0x7fb594076080 msgr2=0x7fb594076490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:10.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.270+0000 7fb599c00640 1 -- 172.21.15.19:0/2921901553 shutdown_connections 2024-05-07T02:32:10.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.270+0000 7fb599c00640 1 -- 172.21.15.19:0/2921901553 wait complete. 2024-05-07T02:32:10.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.271+0000 7fb599c00640 1 Processor -- start 2024-05-07T02:32:10.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.272+0000 7fb599c00640 1 -- start start 2024-05-07T02:32:10.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.272+0000 7fb599c00640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb59410a3a0 0x7fb5941a69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:10.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.272+0000 7fb599c00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb59410e910 0x7fb5941a6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:10.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.272+0000 7fb599c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb59410f3d0 0x7fb5941a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:10.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.272+0000 7fb599c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb59411a3f0 con 0x7fb59410f3d0 2024-05-07T02:32:10.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.272+0000 7fb599c00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb59411a270 con 0x7fb59410e910 2024-05-07T02:32:10.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.272+0000 7fb599c00640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb59411a570 con 0x7fb59410a3a0 2024-05-07T02:32:10.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.273+0000 7fb58be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb59410e910 0x7fb5941a6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:10.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.273+0000 7fb58be00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb59410e910 0x7fb5941a6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:44148/0 (socket says 172.21.15.19:44148) 2024-05-07T02:32:10.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.273+0000 7fb58be00640 1 -- 172.21.15.19:0/4006178664 learned_addr learned my addr 172.21.15.19:0/4006178664 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:32:10.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.273+0000 7fb593e00640 1 --2- 172.21.15.19:0/4006178664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb59410a3a0 0x7fb5941a69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:10.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.273+0000 7fb599200640 1 --2- 172.21.15.19:0/4006178664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb59410f3d0 0x7fb5941a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:10.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.273+0000 7fb58be00640 1 -- 172.21.15.19:0/4006178664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb59410a3a0 msgr2=0x7fb5941a69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:10.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.273+0000 7fb58be00640 1 --2- 172.21.15.19:0/4006178664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb59410a3a0 0x7fb5941a69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:10.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.273+0000 7fb58be00640 1 -- 172.21.15.19:0/4006178664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb59410f3d0 msgr2=0x7fb5941a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:10.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.273+0000 7fb58be00640 1 --2- 172.21.15.19:0/4006178664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb59410f3d0 0x7fb5941a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:10.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.273+0000 7fb58be00640 1 -- 172.21.15.19:0/4006178664 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5941a1260 con 0x7fb59410e910 2024-05-07T02:32:10.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.273+0000 7fb58be00640 1 --2- 172.21.15.19:0/4006178664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb59410e910 0x7fb5941a6ef0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb57c005ab0 tx=0x7fb57c003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:10.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.274+0000 7fb591600640 1 -- 172.21.15.19:0/4006178664 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c03f070 con 0x7fb59410e910 2024-05-07T02:32:10.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.274+0000 7fb591600640 1 -- 172.21.15.19:0/4006178664 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb57c031b50 con 0x7fb59410e910 2024-05-07T02:32:10.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.274+0000 7fb591600640 1 -- 172.21.15.19:0/4006178664 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c035030 con 0x7fb59410e910 2024-05-07T02:32:10.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.274+0000 7fb599c00640 1 -- 172.21.15.19:0/4006178664 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5941a1490 con 0x7fb59410e910 2024-05-07T02:32:10.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.274+0000 7fb599c00640 1 -- 172.21.15.19:0/4006178664 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5941ad750 con 0x7fb59410e910 2024-05-07T02:32:10.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.275+0000 7fb599c00640 1 -- 172.21.15.19:0/4006178664 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb59410b400 con 0x7fb59410e910 2024-05-07T02:32:10.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.277+0000 7fb591600640 1 -- 172.21.15.19:0/4006178664 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7fb57c042070 con 0x7fb59410e910 2024-05-07T02:32:10.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.279+0000 7fb591600640 1 --2- 172.21.15.19:0/4006178664 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fb5780b2b10 0x7fb5780b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:10.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.279+0000 7fb591600640 1 -- 172.21.15.19:0/4006178664 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fb57c103500 con 0x7fb59410e910 2024-05-07T02:32:10.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.279+0000 7fb593e00640 1 --2- 172.21.15.19:0/4006178664 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fb5780b2b10 0x7fb5780b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:10.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.280+0000 7fb593e00640 1 --2- 172.21.15.19:0/4006178664 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fb5780b2b10 0x7fb5780b5020 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb5840095f0 tx=0x7fb584005f90 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:10.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.281+0000 7fb591600640 1 -- 172.21.15.19:0/4006178664 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb57c0ca390 con 0x7fb59410e910 2024-05-07T02:32:10.327 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:10 smithi019 ceph-mon[29781]: from='client.25391 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:32:10.328 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:10 smithi019 ceph-mon[29781]: scrub queued for path: / 2024-05-07T02:32:10.328 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:10 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:32:10.328 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:10 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:32:10.328 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:10 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:32:10.328 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:10 smithi019 ceph-mon[29781]: pgmap v13: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:32:10.331 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-07T02:32:10.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:10 smithi027 ceph-mon[34398]: from='client.25391 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:32:10.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:10 smithi027 ceph-mon[34398]: scrub queued for path: / 2024-05-07T02:32:10.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:10 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:32:10.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:10 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:32:10.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:10 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:32:10.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:10 smithi027 ceph-mon[34398]: pgmap v13: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:32:10.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.517+0000 7fb599c00640 1 -- 172.21.15.19:0/4006178664 --> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fb5941a1fd0 con 0x7fb5780b2b10 2024-05-07T02:32:10.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.528+0000 7fb591600640 1 -- 172.21.15.19:0/4006178664 <== mgr.25234 v2:172.21.15.19:6800/3265825478 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fb5941a1fd0 con 0x7fb5780b2b10 2024-05-07T02:32:10.529 INFO:teuthology.orchestra.run.smithi019.stdout:/ 1m 6m3h 2024-05-07T02:32:10.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.531+0000 7fb599c00640 1 -- 172.21.15.19:0/4006178664 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fb5780b2b10 msgr2=0x7fb5780b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:10.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.531+0000 7fb599c00640 1 --2- 172.21.15.19:0/4006178664 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fb5780b2b10 0x7fb5780b5020 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb5840095f0 tx=0x7fb584005f90 comp rx=0 tx=0).stop 2024-05-07T02:32:10.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.531+0000 7fb599c00640 1 -- 172.21.15.19:0/4006178664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb59410e910 msgr2=0x7fb5941a6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:10.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.531+0000 7fb599c00640 1 --2- 172.21.15.19:0/4006178664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb59410e910 0x7fb5941a6ef0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb57c005ab0 tx=0x7fb57c003b90 comp rx=0 tx=0).stop 2024-05-07T02:32:10.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.531+0000 7fb599c00640 1 -- 172.21.15.19:0/4006178664 shutdown_connections 2024-05-07T02:32:10.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.531+0000 7fb599c00640 1 --2- 172.21.15.19:0/4006178664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb59410f3d0 0x7fb5941a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:10.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.531+0000 7fb599c00640 1 --2- 172.21.15.19:0/4006178664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb59410e910 0x7fb5941a6ef0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:10.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.531+0000 7fb599c00640 1 --2- 172.21.15.19:0/4006178664 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fb5780b2b10 0x7fb5780b5020 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:10.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.531+0000 7fb599c00640 1 --2- 172.21.15.19:0/4006178664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb59410a3a0 0x7fb5941a69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:10.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.531+0000 7fb599c00640 1 -- 172.21.15.19:0/4006178664 >> 172.21.15.19:0/4006178664 conn(0x7fb594076080 msgr2=0x7fb594107af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:10.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.531+0000 7fb599c00640 1 -- 172.21.15.19:0/4006178664 shutdown_connections 2024-05-07T02:32:10.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:10.531+0000 7fb599c00640 1 -- 172.21.15.19:0/4006178664 wait complete. 2024-05-07T02:32:10.559 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-05-07T02:32:10.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:10 smithi064 ceph-mon[34699]: from='client.25391 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:32:10.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:10 smithi064 ceph-mon[34699]: scrub queued for path: / 2024-05-07T02:32:10.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:10 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:32:10.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:10 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:32:10.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:10 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:32:10.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:10 smithi064 ceph-mon[34699]: pgmap v13: 129 pgs: 129 active+clean; 1.1 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:32:10.632 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-05-07T02:32:10.645 INFO:teuthology.run_tasks:Running task workunit... 2024-05-07T02:32:10.700 INFO:tasks.workunit:Pulling workunits from ref 366c9130c497dee9e73178650cf710ed21269b18 2024-05-07T02:32:10.700 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-07T02:32:10.700 INFO:tasks.workunit:timeout=3h 2024-05-07T02:32:10.700 INFO:tasks.workunit:cleanup=True 2024-05-07T02:32:10.701 DEBUG:teuthology.orchestra.run.smithi019:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-07T02:32:10.742 INFO:teuthology.orchestra.run.smithi019.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-07T02:32:10.742 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-05-07T02:32:10.742 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 72h/114d Inode: 1099511628283 Links: 2 2024-05-07T02:32:10.742 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-07T02:32:10.743 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-05-07T02:32:10.743 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-05-07 02:28:06.079484686 +0000 2024-05-07T02:32:10.743 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-05-07 02:28:06.079484686 +0000 2024-05-07T02:32:10.743 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-05-07 02:31:46.801138689 +0000 2024-05-07T02:32:10.743 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-05-07 02:28:06.079484686 +0000 2024-05-07T02:32:10.743 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-07T02:32:10.743 DEBUG:teuthology.orchestra.run.smithi019:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-07T02:32:10.752 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:32:10.844 DEBUG:teuthology.orchestra.run.smithi019:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/batrick/ceph /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 366c9130c497dee9e73178650cf710ed21269b18 2024-05-07T02:32:10.899 INFO:tasks.workunit.client.0.smithi019.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-07T02:32:11.170 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:11 smithi019 ceph-mon[29781]: from='client.25396 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:32:11.170 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:11 smithi019 ceph-mon[29781]: from='client.25421 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:32:11.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:11 smithi027 ceph-mon[34398]: from='client.25396 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:32:11.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:11 smithi027 ceph-mon[34398]: from='client.25421 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:32:11.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:11 smithi064 ceph-mon[34699]: from='client.25396 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:32:11.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:11 smithi064 ceph-mon[34699]: from='client.25421 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:32:12.383 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:12 smithi019 ceph-mon[29781]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-05-07T02:32:12.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:12 smithi027 ceph-mon[34398]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-05-07T02:32:12.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:12 smithi064 ceph-mon[34699]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-05-07T02:32:13.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.367+0000 7efc70e00700 1 -- 172.21.15.19:0/860216408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc6c104400 msgr2=0x7efc6c104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:13.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.367+0000 7efc70e00700 1 --2- 172.21.15.19:0/860216408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc6c104400 0x7efc6c104810 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7efc5c009990 tx=0x7efc5c009ca0 comp rx=0 tx=0).stop 2024-05-07T02:32:13.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.368+0000 7efc70e00700 1 -- 172.21.15.19:0/860216408 shutdown_connections 2024-05-07T02:32:13.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.368+0000 7efc70e00700 1 --2- 172.21.15.19:0/860216408 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc6c109680 0x7efc6c110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:13.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.368+0000 7efc70e00700 1 --2- 172.21.15.19:0/860216408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc6c104400 0x7efc6c104810 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:13.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.368+0000 7efc70e00700 1 --2- 172.21.15.19:0/860216408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc6c108d30 0x7efc6c109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:13.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.368+0000 7efc70e00700 1 -- 172.21.15.19:0/860216408 >> 172.21.15.19:0/860216408 conn(0x7efc6c079d10 msgr2=0x7efc6c07a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:13.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.368+0000 7efc70e00700 1 -- 172.21.15.19:0/860216408 shutdown_connections 2024-05-07T02:32:13.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.368+0000 7efc70e00700 1 -- 172.21.15.19:0/860216408 wait complete. 2024-05-07T02:32:13.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.369+0000 7efc70e00700 1 Processor -- start 2024-05-07T02:32:13.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.370+0000 7efc70e00700 1 -- start start 2024-05-07T02:32:13.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.370+0000 7efc70e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc6c104400 0x7efc6c19cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:13.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.370+0000 7efc70e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc6c108d30 0x7efc6c19d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:13.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.370+0000 7efc70e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc6c109680 0x7efc6c1a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:13.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.370+0000 7efc70e00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efc6c114300 con 0x7efc6c109680 2024-05-07T02:32:13.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.370+0000 7efc70e00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efc6c114180 con 0x7efc6c108d30 2024-05-07T02:32:13.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.370+0000 7efc70e00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7efc6c114480 con 0x7efc6c104400 2024-05-07T02:32:13.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.371+0000 7efc6b400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc6c104400 0x7efc6c19cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:13.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.371+0000 7efc6b400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc6c104400 0x7efc6c19cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:60610/0 (socket says 172.21.15.19:60610) 2024-05-07T02:32:13.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.371+0000 7efc6b400700 1 -- 172.21.15.19:0/1220288120 learned_addr learned my addr 172.21.15.19:0/1220288120 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:32:13.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.371+0000 7efc6be00700 1 --2- 172.21.15.19:0/1220288120 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc6c109680 0x7efc6c1a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:13.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.371+0000 7efc6aa00700 1 --2- 172.21.15.19:0/1220288120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc6c108d30 0x7efc6c19d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:13.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.371+0000 7efc6b400700 1 -- 172.21.15.19:0/1220288120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc6c108d30 msgr2=0x7efc6c19d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:13.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.371+0000 7efc6b400700 1 --2- 172.21.15.19:0/1220288120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc6c108d30 0x7efc6c19d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:13.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.371+0000 7efc6b400700 1 -- 172.21.15.19:0/1220288120 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc6c109680 msgr2=0x7efc6c1a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:13.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.371+0000 7efc6b400700 1 --2- 172.21.15.19:0/1220288120 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc6c109680 0x7efc6c1a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:13.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.371+0000 7efc6b400700 1 -- 172.21.15.19:0/1220288120 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc6c1a1e00 con 0x7efc6c104400 2024-05-07T02:32:13.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.371+0000 7efc6aa00700 1 --2- 172.21.15.19:0/1220288120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc6c108d30 0x7efc6c19d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:32:13.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.371+0000 7efc6be00700 1 --2- 172.21.15.19:0/1220288120 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc6c109680 0x7efc6c1a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:32:13.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.372+0000 7efc6b400700 1 --2- 172.21.15.19:0/1220288120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc6c104400 0x7efc6c19cf30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efc5c00b2e0 tx=0x7efc5c00be40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:13.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.372+0000 7efc63e00700 1 -- 172.21.15.19:0/1220288120 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5c005da0 con 0x7efc6c104400 2024-05-07T02:32:13.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.372+0000 7efc70e00700 1 -- 172.21.15.19:0/1220288120 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc6c1a2090 con 0x7efc6c104400 2024-05-07T02:32:13.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.372+0000 7efc63e00700 1 -- 172.21.15.19:0/1220288120 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc5c003d40 con 0x7efc6c104400 2024-05-07T02:32:13.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.372+0000 7efc63e00700 1 -- 172.21.15.19:0/1220288120 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5c004a50 con 0x7efc6c104400 2024-05-07T02:32:13.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.372+0000 7efc70e00700 1 -- 172.21.15.19:0/1220288120 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc6c1a2560 con 0x7efc6c104400 2024-05-07T02:32:13.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.377+0000 7efc63e00700 1 -- 172.21.15.19:0/1220288120 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7efc5c009d10 con 0x7efc6c104400 2024-05-07T02:32:13.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.377+0000 7efc80feab80 1 Processor -- start 2024-05-07T02:32:13.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.377+0000 7efc80feab80 1 -- start start 2024-05-07T02:32:13.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.378+0000 7efc80feab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fd03c4c40 0x556fd03c5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:13.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.378+0000 7efc80feab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fd03c5590 0x556fd03ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:13.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.378+0000 7efc80feab80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fd03cec30 0x556fd03d1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:13.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.378+0000 7efc80feab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556fd02e6a20 con 0x556fd03c5590 2024-05-07T02:32:13.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.378+0000 7efc80feab80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556fd0284760 con 0x556fd03cec30 2024-05-07T02:32:13.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.378+0000 7efc80feab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556fd02e29e0 con 0x556fd03c4c40 2024-05-07T02:32:13.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.378+0000 7efc6b400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fd03c5590 0x556fd03ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:13.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.378+0000 7efc6be00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fd03c4c40 0x556fd03c5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:13.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.378+0000 7efc6aa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fd03cec30 0x556fd03d1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:13.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.378+0000 7efc6b400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fd03c5590 0x556fd03ce6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53906/0 (socket says 172.21.15.19:53906) 2024-05-07T02:32:13.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.378+0000 7efc6b400700 1 -- 172.21.15.19:0/4038093764 learned_addr learned my addr 172.21.15.19:0/4038093764 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:32:13.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.379+0000 7efc63e00700 1 --2- 172.21.15.19:0/1220288120 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7efc500b2a80 0x7efc500b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:13.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.379+0000 7efc6be00700 1 -- 172.21.15.19:0/4038093764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fd03cec30 msgr2=0x556fd03d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:32:13.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.379+0000 7efc6be00700 1 --2- 172.21.15.19:0/4038093764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fd03cec30 0x556fd03d1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:13.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.379+0000 7efc6be00700 1 -- 172.21.15.19:0/4038093764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fd03c5590 msgr2=0x556fd03ce6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:32:13.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.379+0000 7efc6be00700 1 --2- 172.21.15.19:0/4038093764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fd03c5590 0x556fd03ce6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:13.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.379+0000 7efc6be00700 1 -- 172.21.15.19:0/4038093764 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fd02338d0 con 0x556fd03c4c40 2024-05-07T02:32:13.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.379+0000 7efc63e00700 1 -- 172.21.15.19:0/1220288120 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7efc5c0e3580 con 0x7efc6c104400 2024-05-07T02:32:13.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.379+0000 7efc6be00700 1 --2- 172.21.15.19:0/1220288120 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7efc500b2a80 0x7efc500b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:13.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.380+0000 7efc6be00700 1 --2- 172.21.15.19:0/4038093764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fd03c4c40 0x556fd03c5050 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7efc640093a0 tx=0x7efc64009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d301ae98dad82c28 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:13.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.380+0000 7efc60c00700 1 -- 172.21.15.19:0/4038093764 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6401b030 con 0x556fd03c4c40 2024-05-07T02:32:13.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.380+0000 7efc60c00700 1 -- 172.21.15.19:0/4038093764 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc640218d0 con 0x556fd03c4c40 2024-05-07T02:32:13.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.380+0000 7efc60c00700 1 -- 172.21.15.19:0/4038093764 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc64020710 con 0x556fd03c4c40 2024-05-07T02:32:13.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.381+0000 7efc80feab80 1 -- 172.21.15.19:0/4038093764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fd03c4c40 msgr2=0x556fd03c5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:13.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.381+0000 7efc80feab80 1 --2- 172.21.15.19:0/4038093764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fd03c4c40 0x556fd03c5050 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7efc640093a0 tx=0x7efc64009920 comp rx=0 tx=0).stop 2024-05-07T02:32:13.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.381+0000 7efc6be00700 1 --2- 172.21.15.19:0/1220288120 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7efc500b2a80 0x7efc500b4f90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7efc64009fd0 tx=0x7efc64022000 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:13.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.381+0000 7efc80feab80 1 -- 172.21.15.19:0/4038093764 shutdown_connections 2024-05-07T02:32:13.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.381+0000 7efc80feab80 1 --2- 172.21.15.19:0/4038093764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fd03cec30 0x556fd03d1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:13.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.381+0000 7efc80feab80 1 --2- 172.21.15.19:0/4038093764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fd03c4c40 0x556fd03c5050 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:13.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.381+0000 7efc80feab80 1 --2- 172.21.15.19:0/4038093764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fd03c5590 0x556fd03ce6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:13.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.381+0000 7efc80feab80 1 -- 172.21.15.19:0/4038093764 >> 172.21.15.19:0/4038093764 conn(0x556fd0246e40 msgr2=0x556fd03bf980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:13.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.381+0000 7efc80feab80 1 -- 172.21.15.19:0/4038093764 shutdown_connections 2024-05-07T02:32:13.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.382+0000 7efc80feab80 1 -- 172.21.15.19:0/4038093764 wait complete. 2024-05-07T02:32:13.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.382+0000 7efc80feab80 1 -- start start 2024-05-07T02:32:13.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.382+0000 7efc80feab80 1 Processor -- start 2024-05-07T02:32:13.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.383+0000 7efc80feab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fd03c4c40 0x556fd03cbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:13.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.383+0000 7efc80feab80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fd03c5590 0x556fd03cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:13.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.383+0000 7efc80feab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fd03cec30 0x556fd03cc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:13.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.383+0000 7efc80feab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556fd03d1a00 con 0x556fd03cec30 2024-05-07T02:32:13.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.383+0000 7efc80feab80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556fd024cb20 con 0x556fd03c5590 2024-05-07T02:32:13.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.383+0000 7efc80feab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556fd03d1b80 con 0x556fd03c4c40 2024-05-07T02:32:13.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.383+0000 7efc80feab80 10 client.0 _ll_register_callbacks cb 0x556fd027a380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:32:13.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.384+0000 7efc6be00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fd03cec30 0x556fd03cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:13.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.384+0000 7efc6b400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fd03c4c40 0x556fd03cbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:13.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.384+0000 7efc6b400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fd03c4c40 0x556fd03cbba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:60636/0 (socket says 172.21.15.19:60636) 2024-05-07T02:32:13.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.384+0000 7efc6b400700 1 -- 172.21.15.19:0/2768784865 learned_addr learned my addr 172.21.15.19:0/2768784865 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:32:13.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.384+0000 7efc6aa00700 1 --2- 172.21.15.19:0/2768784865 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fd03c5590 0x556fd03cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:13.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.384+0000 7efc6be00700 1 -- 172.21.15.19:0/2768784865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fd03c4c40 msgr2=0x556fd03cbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:32:13.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.384+0000 7efc6be00700 1 --2- 172.21.15.19:0/2768784865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fd03c4c40 0x556fd03cbba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:13.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.384+0000 7efc6be00700 1 -- 172.21.15.19:0/2768784865 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fd03c5590 msgr2=0x556fd03cc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:32:13.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.384+0000 7efc6be00700 1 --2- 172.21.15.19:0/2768784865 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fd03c5590 0x556fd03cc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:13.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.384+0000 7efc6be00700 1 -- 172.21.15.19:0/2768784865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fd02ac420 con 0x556fd03cec30 2024-05-07T02:32:13.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.384+0000 7efc6aa00700 1 --2- 172.21.15.19:0/2768784865 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fd03c5590 0x556fd03cc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:32:13.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.385+0000 7efc6be00700 1 --2- 172.21.15.19:0/2768784865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fd03cec30 0x556fd03cc620 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7efc64009370 tx=0x7efc64020600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8163818e52116d4f server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:13.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.385+0000 7efc55600700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-07T02:32:13.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.385+0000 7efc55600700 1 -- 172.21.15.19:0/2768784865 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc64025070 con 0x556fd03cec30 2024-05-07T02:32:13.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.385+0000 7efc55600700 1 -- 172.21.15.19:0/2768784865 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7efc64021630 con 0x556fd03cec30 2024-05-07T02:32:13.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.386+0000 7efc80feab80 1 -- 172.21.15.19:0/2768784865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc640218d0 con 0x556fd03cec30 2024-05-07T02:32:13.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.386+0000 7efc55600700 1 -- 172.21.15.19:0/2768784865 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7efc6403d050 con 0x556fd03cec30 2024-05-07T02:32:13.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.386+0000 7efc80feab80 10 client.15288 fetch_fsmap learned FSMap version 31 2024-05-07T02:32:13.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.386+0000 7efc80feab80 1 -- 172.21.15.19:0/2768784865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556fd03dc600 con 0x556fd03cec30 2024-05-07T02:32:13.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.386+0000 7efc55600700 1 -- 172.21.15.19:0/2768784865 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc640218d0 con 0x556fd03cec30 2024-05-07T02:32:13.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.387+0000 7efc55600700 1 -- 172.21.15.19:0/2768784865 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7efc64020b30 con 0x556fd03cec30 2024-05-07T02:32:13.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.387+0000 7efc80feab80 10 client.15288 fetch_fsmap finished waiting for FSMap version 31 2024-05-07T02:32:13.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.387+0000 7efc80feab80 10 client.15288 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:32:13.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.387+0000 7efc80feab80 20 client.15288 populate_metadata read hostname 'smithi019' 2024-05-07T02:32:13.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.387+0000 7efc80feab80 1 --2- 172.21.15.19:0/2768784865 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x556fd03debe0 0x556fd03e1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:13.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.387+0000 7efc80feab80 4 client.15288 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:32:13.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.387+0000 7efc80feab80 1 -- 172.21.15.19:0/2768784865 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556fd03910e0 con 0x556fd03debe0 2024-05-07T02:32:13.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.387+0000 7efc6b400700 1 --2- 172.21.15.19:0/2768784865 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x556fd03debe0 0x556fd03e1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:13.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.388+0000 7efc6b400700 1 --2- 172.21.15.19:0/2768784865 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x556fd03debe0 0x556fd03e1000 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=599069cec9e87c7 server_cookie=cf211719ce666078 in_seq=0 out_seq=0 2024-05-07T02:32:13.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.388+0000 7efc55600700 10 client.15288 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:32:13.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.389+0000 7efc55600700 1 -- 172.21.15.19:0/2768784865 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556fd03910e0 con 0x556fd03debe0 2024-05-07T02:32:13.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.389+0000 7efc55600700 10 client.15288 handle_command_reply: tid=0 2024-05-07T02:32:13.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.389+0000 7efc80feab80 1 client.15288 shutdown 2024-05-07T02:32:13.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.389+0000 7efc80feab80 1 -- 172.21.15.19:0/2768784865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fd03cec30 msgr2=0x556fd03cc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:13.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.389+0000 7efc80feab80 1 --2- 172.21.15.19:0/2768784865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fd03cec30 0x556fd03cc620 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7efc64009370 tx=0x7efc64020600 comp rx=0 tx=0).stop 2024-05-07T02:32:13.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.389+0000 7efc80feab80 1 -- 172.21.15.19:0/2768784865 shutdown_connections 2024-05-07T02:32:13.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.389+0000 7efc80feab80 1 --2- 172.21.15.19:0/2768784865 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x556fd03debe0 0x556fd03e1000 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:13.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.389+0000 7efc80feab80 1 --2- 172.21.15.19:0/2768784865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fd03cec30 0x556fd03cc620 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:13.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.389+0000 7efc80feab80 1 --2- 172.21.15.19:0/2768784865 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fd03c4c40 0x556fd03cbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:13.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.389+0000 7efc80feab80 1 --2- 172.21.15.19:0/2768784865 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fd03c5590 0x556fd03cc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:13.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.389+0000 7efc55600700 0 client.15288 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:32:13.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.389+0000 7efc80feab80 1 -- 172.21.15.19:0/2768784865 >> 172.21.15.19:0/2768784865 conn(0x556fd0246e40 msgr2=0x556fd03c1600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:13.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.390+0000 7efc80feab80 1 -- 172.21.15.19:0/2768784865 shutdown_connections 2024-05-07T02:32:13.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.390+0000 7efc80feab80 1 -- 172.21.15.19:0/2768784865 wait complete. 2024-05-07T02:32:13.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.390+0000 7efc80feab80 20 client.15288 trim_cache size 0 max 16384 2024-05-07T02:32:13.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.416+0000 7efc80feab80 1 Processor -- start 2024-05-07T02:32:13.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.416+0000 7efc80feab80 1 -- start start 2024-05-07T02:32:13.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.417+0000 7efc80feab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fd03c4c40 0x556fd03dae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:13.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.417+0000 7efc80feab80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fd03c5590 0x556fd03c4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:13.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.417+0000 7efc80feab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fd03cec30 0x556fd03cc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:13.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.417+0000 7efc80feab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efc6c1b2500 con 0x556fd03cec30 2024-05-07T02:32:13.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.417+0000 7efc80feab80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efc6c1b4040 con 0x556fd03c5590 2024-05-07T02:32:13.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.417+0000 7efc80feab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7efc6c1b2bd0 con 0x556fd03c4c40 2024-05-07T02:32:13.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.417+0000 7efc6b400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fd03c4c40 0x556fd03dae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:13.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.417+0000 7efc6b400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fd03c4c40 0x556fd03dae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:60642/0 (socket says 172.21.15.19:60642) 2024-05-07T02:32:13.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.417+0000 7efc6b400700 1 -- 172.21.15.19:0/3164155339 learned_addr learned my addr 172.21.15.19:0/3164155339 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:32:13.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.417+0000 7efc6aa00700 1 --2- 172.21.15.19:0/3164155339 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fd03c5590 0x556fd03c4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:13.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.417+0000 7efc6be00700 1 --2- 172.21.15.19:0/3164155339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fd03cec30 0x556fd03cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:13.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.418+0000 7efc6aa00700 1 -- 172.21.15.19:0/3164155339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fd03c4c40 msgr2=0x556fd03dae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:32:13.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.418+0000 7efc6aa00700 1 --2- 172.21.15.19:0/3164155339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fd03c4c40 0x556fd03dae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:13.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.418+0000 7efc6aa00700 1 -- 172.21.15.19:0/3164155339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fd03cec30 msgr2=0x556fd03cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:32:13.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.418+0000 7efc6aa00700 1 --2- 172.21.15.19:0/3164155339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fd03cec30 0x556fd03cc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:13.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.418+0000 7efc6aa00700 1 -- 172.21.15.19:0/3164155339 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fd03bb730 con 0x556fd03c5590 2024-05-07T02:32:13.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.418+0000 7efc6be00700 1 --2- 172.21.15.19:0/3164155339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fd03cec30 0x556fd03cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:32:13.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.418+0000 7efc6aa00700 1 --2- 172.21.15.19:0/3164155339 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fd03c5590 0x556fd03c4040 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efc6c103c00 tx=0x7efc58009e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22b277a45ad0284b server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:13.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.419+0000 7efc57e00700 1 -- 172.21.15.19:0/3164155339 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc58003b00 con 0x556fd03c5590 2024-05-07T02:32:13.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.419+0000 7efc57e00700 1 -- 172.21.15.19:0/3164155339 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc58003ca0 con 0x556fd03c5590 2024-05-07T02:32:13.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.419+0000 7efc57e00700 1 -- 172.21.15.19:0/3164155339 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc580116a0 con 0x556fd03c5590 2024-05-07T02:32:13.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.420+0000 7efc80feab80 1 -- 172.21.15.19:0/3164155339 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fd03c5590 msgr2=0x556fd03c4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:13.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.420+0000 7efc80feab80 1 --2- 172.21.15.19:0/3164155339 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fd03c5590 0x556fd03c4040 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efc6c103c00 tx=0x7efc58009e50 comp rx=0 tx=0).stop 2024-05-07T02:32:13.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.420+0000 7efc80feab80 1 -- 172.21.15.19:0/3164155339 shutdown_connections 2024-05-07T02:32:13.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.420+0000 7efc80feab80 1 --2- 172.21.15.19:0/3164155339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fd03cec30 0x556fd03cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:13.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.420+0000 7efc80feab80 1 --2- 172.21.15.19:0/3164155339 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fd03c4c40 0x556fd03dae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:13.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.420+0000 7efc80feab80 1 --2- 172.21.15.19:0/3164155339 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fd03c5590 0x556fd03c4040 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:13.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.420+0000 7efc80feab80 1 -- 172.21.15.19:0/3164155339 >> 172.21.15.19:0/3164155339 conn(0x556fd0246e40 msgr2=0x556fd03cd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:13.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.420+0000 7efc80feab80 1 -- 172.21.15.19:0/3164155339 shutdown_connections 2024-05-07T02:32:13.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.420+0000 7efc80feab80 1 -- 172.21.15.19:0/3164155339 wait complete. 2024-05-07T02:32:13.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.420+0000 7efc80feab80 1 -- start start 2024-05-07T02:32:13.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.421+0000 7efc80feab80 1 Processor -- start 2024-05-07T02:32:13.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.421+0000 7efc80feab80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fd03c4c40 0x556fd03e14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:13.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.421+0000 7efc80feab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fd03c5590 0x556fd03e19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:13.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.421+0000 7efc80feab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fd03cec30 0x556fd0441160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:13.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.421+0000 7efc80feab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efc6c1b3970 con 0x556fd03c5590 2024-05-07T02:32:13.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.421+0000 7efc80feab80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efc6c1b32a0 con 0x556fd03c4c40 2024-05-07T02:32:13.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.421+0000 7efc80feab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7efc6c1b04a0 con 0x556fd03cec30 2024-05-07T02:32:13.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.422+0000 7efc80feab80 10 client.0 _ll_register_callbacks cb 0x556fd027a380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:32:13.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.422+0000 7efc6b400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fd03c4c40 0x556fd03e14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:13.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.422+0000 7efc6aa00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fd03c5590 0x556fd03e19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:13.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.422+0000 7efc6b400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fd03c4c40 0x556fd03e14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:44196/0 (socket says 172.21.15.19:44196) 2024-05-07T02:32:13.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.422+0000 7efc6b400700 1 -- 172.21.15.19:0/1812854950 learned_addr learned my addr 172.21.15.19:0/1812854950 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:32:13.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.422+0000 7efc6be00700 1 --2- 172.21.15.19:0/1812854950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fd03cec30 0x556fd0441160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:13.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.422+0000 7efc6aa00700 1 -- 172.21.15.19:0/1812854950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fd03cec30 msgr2=0x556fd0441160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:32:13.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.422+0000 7efc6aa00700 1 --2- 172.21.15.19:0/1812854950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fd03cec30 0x556fd0441160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:13.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.422+0000 7efc6aa00700 1 -- 172.21.15.19:0/1812854950 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fd03c4c40 msgr2=0x556fd03e14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:32:13.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.422+0000 7efc6aa00700 1 --2- 172.21.15.19:0/1812854950 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fd03c4c40 0x556fd03e14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:13.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.422+0000 7efc6aa00700 1 -- 172.21.15.19:0/1812854950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fd03c53c0 con 0x556fd03c5590 2024-05-07T02:32:13.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.422+0000 7efc6b400700 1 --2- 172.21.15.19:0/1812854950 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fd03c4c40 0x556fd03e14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:32:13.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.423+0000 7efc6aa00700 1 --2- 172.21.15.19:0/1812854950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fd03c5590 0x556fd03e19e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x556fd03c0c90 tx=0x7efc58012b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2aef1a92c3f20eb server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:13.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.423+0000 7efc55600700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-07T02:32:13.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.423+0000 7efc55600700 1 -- 172.21.15.19:0/1812854950 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc58003b00 con 0x556fd03c5590 2024-05-07T02:32:13.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.423+0000 7efc80feab80 1 -- 172.21.15.19:0/1812854950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc58003ca0 con 0x556fd03c5590 2024-05-07T02:32:13.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.423+0000 7efc55600700 1 -- 172.21.15.19:0/1812854950 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7efc58019040 con 0x556fd03c5590 2024-05-07T02:32:13.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.424+0000 7efc55600700 1 -- 172.21.15.19:0/1812854950 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7efc58033050 con 0x556fd03c5590 2024-05-07T02:32:13.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.424+0000 7efc80feab80 10 client.15294 fetch_fsmap learned FSMap version 31 2024-05-07T02:32:13.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.424+0000 7efc80feab80 1 -- 172.21.15.19:0/1812854950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556fd03bbb20 con 0x556fd03c5590 2024-05-07T02:32:13.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.424+0000 7efc55600700 1 -- 172.21.15.19:0/1812854950 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc58003ca0 con 0x556fd03c5590 2024-05-07T02:32:13.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.424+0000 7efc55600700 1 -- 172.21.15.19:0/1812854950 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7efc58004cb0 con 0x556fd03c5590 2024-05-07T02:32:13.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.424+0000 7efc80feab80 10 client.15294 fetch_fsmap finished waiting for FSMap version 31 2024-05-07T02:32:13.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.424+0000 7efc80feab80 10 client.15294 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:32:13.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.424+0000 7efc80feab80 20 client.15294 populate_metadata read hostname 'smithi019' 2024-05-07T02:32:13.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.424+0000 7efc80feab80 1 --2- 172.21.15.19:0/1812854950 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x556fd0448c70 0x556fd044b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:13.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.425+0000 7efc80feab80 4 client.15294 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-07T02:32:13.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.425+0000 7efc80feab80 1 -- 172.21.15.19:0/1812854950 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556fd02f4400 con 0x556fd0448c70 2024-05-07T02:32:13.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.425+0000 7efc6b400700 1 --2- 172.21.15.19:0/1812854950 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x556fd0448c70 0x556fd044b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:13.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.425+0000 7efc6b400700 1 --2- 172.21.15.19:0/1812854950 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x556fd0448c70 0x556fd044b090 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39198bd893f6497 server_cookie=9240779d43da8d97 in_seq=0 out_seq=0 2024-05-07T02:32:13.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.425+0000 7efc55600700 10 client.15294 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:32:13.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.426+0000 7efc55600700 1 -- 172.21.15.19:0/1812854950 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556fd02f4400 con 0x556fd0448c70 2024-05-07T02:32:13.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.426+0000 7efc55600700 10 client.15294 handle_command_reply: tid=0 2024-05-07T02:32:13.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.426+0000 7efc80feab80 1 client.15294 shutdown 2024-05-07T02:32:13.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.426+0000 7efc80feab80 1 -- 172.21.15.19:0/1812854950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fd03c5590 msgr2=0x556fd03e19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:13.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.426+0000 7efc80feab80 1 --2- 172.21.15.19:0/1812854950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fd03c5590 0x556fd03e19e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x556fd03c0c90 tx=0x7efc58012b20 comp rx=0 tx=0).stop 2024-05-07T02:32:13.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.426+0000 7efc80feab80 1 -- 172.21.15.19:0/1812854950 shutdown_connections 2024-05-07T02:32:13.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.426+0000 7efc80feab80 1 --2- 172.21.15.19:0/1812854950 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x556fd0448c70 0x556fd044b090 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:13.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.426+0000 7efc80feab80 1 --2- 172.21.15.19:0/1812854950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556fd03cec30 0x556fd0441160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:13.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.426+0000 7efc80feab80 1 --2- 172.21.15.19:0/1812854950 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x556fd03c4c40 0x556fd03e14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:13.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.426+0000 7efc80feab80 1 --2- 172.21.15.19:0/1812854950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556fd03c5590 0x556fd03e19e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:13.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.426+0000 7efc80feab80 1 -- 172.21.15.19:0/1812854950 >> 172.21.15.19:0/1812854950 conn(0x556fd0246e40 msgr2=0x556fd03c3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:13.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.426+0000 7efc55600700 0 client.15294 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:32:13.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.427+0000 7efc80feab80 1 -- 172.21.15.19:0/1812854950 shutdown_connections 2024-05-07T02:32:13.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.427+0000 7efc80feab80 1 -- 172.21.15.19:0/1812854950 wait complete. 2024-05-07T02:32:13.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.427+0000 7efc80feab80 20 client.15294 trim_cache size 0 max 16384 2024-05-07T02:32:13.429 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:32:13.429 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-07T02:32:13.429 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-05-07T02:32:13.429 INFO:teuthology.orchestra.run.smithi019.stdout: "ed137034-a0c9-4805-8bfd-2512803393e4": { 2024-05-07T02:32:13.429 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-05-07T02:32:13.429 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "ed137034-a0c9-4805-8bfd-2512803393e4", 2024-05-07T02:32:13.429 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-05-07T02:32:13.429 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:32:13.429 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:32:13.429 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:32:13.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.427+0000 7efc70e00700 1 -- 172.21.15.19:0/1220288120 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7efc500b2a80 msgr2=0x7efc500b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:13.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.427+0000 7efc70e00700 1 --2- 172.21.15.19:0/1220288120 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7efc500b2a80 0x7efc500b4f90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7efc64009fd0 tx=0x7efc64022000 comp rx=0 tx=0).stop 2024-05-07T02:32:13.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.428+0000 7efc70e00700 1 -- 172.21.15.19:0/1220288120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc6c104400 msgr2=0x7efc6c19cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:13.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.428+0000 7efc70e00700 1 --2- 172.21.15.19:0/1220288120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc6c104400 0x7efc6c19cf30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efc5c00b2e0 tx=0x7efc5c00be40 comp rx=0 tx=0).stop 2024-05-07T02:32:13.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.428+0000 7efc70e00700 1 -- 172.21.15.19:0/1220288120 shutdown_connections 2024-05-07T02:32:13.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.428+0000 7efc70e00700 1 --2- 172.21.15.19:0/1220288120 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7efc500b2a80 0x7efc500b4f90 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:13.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.428+0000 7efc70e00700 1 --2- 172.21.15.19:0/1220288120 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc6c109680 0x7efc6c1a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:13.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.428+0000 7efc70e00700 1 --2- 172.21.15.19:0/1220288120 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efc6c104400 0x7efc6c19cf30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:13.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.428+0000 7efc70e00700 1 --2- 172.21.15.19:0/1220288120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efc6c108d30 0x7efc6c19d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:13.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.428+0000 7efc70e00700 1 -- 172.21.15.19:0/1220288120 >> 172.21.15.19:0/1220288120 conn(0x7efc6c079d10 msgr2=0x7efc6c101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:13.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.428+0000 7efc70e00700 1 -- 172.21.15.19:0/1220288120 shutdown_connections 2024-05-07T02:32:13.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:13.428+0000 7efc70e00700 1 -- 172.21.15.19:0/1220288120 wait complete. 2024-05-07T02:32:14.519 INFO:tasks.cephfs.filesystem:scrub status for tag:ed137034-a0c9-4805-8bfd-2512803393e4 - {'path': '/', 'tag': 'ed137034-a0c9-4805-8bfd-2512803393e4', 'options': 'recursive,force'} 2024-05-07T02:32:15.192 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:14 smithi027 ceph-mon[34398]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-05-07T02:32:15.311 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:14 smithi064 ceph-mon[34699]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-05-07T02:32:15.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:14 smithi019 ceph-mon[29781]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-05-07T02:32:17.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:16 smithi027 ceph-mon[34398]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 41 op/s 2024-05-07T02:32:17.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:16 smithi064 ceph-mon[34699]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 41 op/s 2024-05-07T02:32:17.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:16 smithi019 ceph-mon[29781]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 41 op/s 2024-05-07T02:32:19.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:18 smithi019 ceph-mon[29781]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-05-07T02:32:19.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:18 smithi027 ceph-mon[34398]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-05-07T02:32:19.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:18 smithi064 ceph-mon[34699]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-05-07T02:32:21.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:20 smithi027 ceph-mon[34398]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-05-07T02:32:21.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:20 smithi064 ceph-mon[34699]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-05-07T02:32:21.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:20 smithi019 ceph-mon[29781]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-05-07T02:32:23.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:22 smithi027 ceph-mon[34398]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-05-07T02:32:23.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:22 smithi064 ceph-mon[34699]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-05-07T02:32:23.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:22 smithi019 ceph-mon[29781]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-05-07T02:32:24.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:23 smithi027 ceph-mon[34398]: scrub summary: idle 2024-05-07T02:32:24.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:23 smithi019 ceph-mon[29781]: scrub summary: idle 2024-05-07T02:32:24.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:23 smithi064 ceph-mon[34699]: scrub summary: idle 2024-05-07T02:32:25.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:24 smithi027 ceph-mon[34398]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-07T02:32:25.325 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:24 smithi019 ceph-mon[29781]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-07T02:32:25.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:24 smithi064 ceph-mon[34699]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 8 op/s 2024-05-07T02:32:27.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:26 smithi027 ceph-mon[34398]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-07T02:32:27.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:26 smithi064 ceph-mon[34699]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-07T02:32:27.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:26 smithi019 ceph-mon[29781]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-07T02:32:29.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:28 smithi027 ceph-mon[34398]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 170 B/s wr, 12 op/s 2024-05-07T02:32:29.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:28 smithi064 ceph-mon[34699]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 170 B/s wr, 12 op/s 2024-05-07T02:32:29.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:28 smithi019 ceph-mon[29781]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 170 B/s wr, 12 op/s 2024-05-07T02:32:31.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:30 smithi064 ceph-mon[34699]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:32:31.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:30 smithi019 ceph-mon[29781]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:32:31.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:30 smithi027 ceph-mon[34398]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:32:33.259 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:32 smithi027 ceph-mon[34398]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:32:33.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:32 smithi064 ceph-mon[34699]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:32:33.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:32 smithi019 ceph-mon[29781]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:32:35.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:34 smithi064 ceph-mon[34699]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:32:35.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:34 smithi019 ceph-mon[29781]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:32:35.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:34 smithi027 ceph-mon[34398]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:32:37.272 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:36 smithi019 ceph-mon[29781]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-07T02:32:37.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:36 smithi064 ceph-mon[34699]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-07T02:32:37.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:36 smithi027 ceph-mon[34398]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-07T02:32:38.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:37 smithi064 ceph-mon[34699]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-07T02:32:38.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:37 smithi019 ceph-mon[29781]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-07T02:32:38.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:37 smithi027 ceph-mon[34398]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-05-07T02:32:41.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:40 smithi027 ceph-mon[34398]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:32:41.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:40 smithi064 ceph-mon[34699]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:32:41.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:40 smithi019 ceph-mon[29781]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:32:43.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:42 smithi027 ceph-mon[34398]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:32:43.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:42 smithi064 ceph-mon[34699]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:32:43.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:42 smithi019 ceph-mon[29781]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:32:44.520 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-07T02:32:45.007 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:32:45.204 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:44 smithi019 ceph-mon[29781]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:32:45.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:44 smithi027 ceph-mon[34398]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:32:45.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:44 smithi064 ceph-mon[34699]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:32:47.120 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:46 smithi019 ceph-mon[29781]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 10 op/s 2024-05-07T02:32:47.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:46 smithi027 ceph-mon[34398]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 10 op/s 2024-05-07T02:32:47.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:46 smithi064 ceph-mon[34699]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 10 op/s 2024-05-07T02:32:47.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.471+0000 7f0080a00700 1 -- 172.21.15.19:0/15715782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f007c109680 msgr2=0x7f007c110fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:47.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.471+0000 7f0080a00700 1 --2- 172.21.15.19:0/15715782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f007c109680 0x7f007c110fe0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f007400b0a0 tx=0x7f007400b3b0 comp rx=0 tx=0).stop 2024-05-07T02:32:47.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.471+0000 7f0080a00700 1 -- 172.21.15.19:0/15715782 shutdown_connections 2024-05-07T02:32:47.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.471+0000 7f0080a00700 1 --2- 172.21.15.19:0/15715782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f007c109680 0x7f007c110fe0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:47.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.471+0000 7f0080a00700 1 --2- 172.21.15.19:0/15715782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f007c104400 0x7f007c104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:47.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.471+0000 7f0080a00700 1 --2- 172.21.15.19:0/15715782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f007c108d30 0x7f007c109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:47.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.471+0000 7f0080a00700 1 -- 172.21.15.19:0/15715782 >> 172.21.15.19:0/15715782 conn(0x7f007c079d10 msgr2=0x7f007c07a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:47.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.472+0000 7f0080a00700 1 -- 172.21.15.19:0/15715782 shutdown_connections 2024-05-07T02:32:47.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.472+0000 7f0080a00700 1 -- 172.21.15.19:0/15715782 wait complete. 2024-05-07T02:32:47.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.474+0000 7f0080a00700 1 Processor -- start 2024-05-07T02:32:47.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.474+0000 7f0080a00700 1 -- start start 2024-05-07T02:32:47.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.475+0000 7f0080a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f007c104400 0x7f007c199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:47.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.475+0000 7f0080a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f007c108d30 0x7f007c199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:47.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.475+0000 7f0080a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f007c109680 0x7f007c1a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:47.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.475+0000 7f0080a00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f007c1142b0 con 0x7f007c109680 2024-05-07T02:32:47.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.475+0000 7f0080a00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f007c114130 con 0x7f007c104400 2024-05-07T02:32:47.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.475+0000 7f0080a00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f007c114430 con 0x7f007c108d30 2024-05-07T02:32:47.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.475+0000 7f0079e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f007c108d30 0x7f007c199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:47.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.475+0000 7f007a800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f007c104400 0x7f007c199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:47.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.476+0000 7f007a800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f007c104400 0x7f007c199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:52676/0 (socket says 172.21.15.19:52676) 2024-05-07T02:32:47.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.476+0000 7f007a800700 1 -- 172.21.15.19:0/22433172 learned_addr learned my addr 172.21.15.19:0/22433172 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:32:47.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.476+0000 7f007b200700 1 --2- 172.21.15.19:0/22433172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f007c109680 0x7f007c1a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:47.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.476+0000 7f0079e00700 1 -- 172.21.15.19:0/22433172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f007c104400 msgr2=0x7f007c199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:47.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.476+0000 7f0079e00700 1 --2- 172.21.15.19:0/22433172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f007c104400 0x7f007c199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:47.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.476+0000 7f0079e00700 1 -- 172.21.15.19:0/22433172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f007c109680 msgr2=0x7f007c1a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:47.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.476+0000 7f0079e00700 1 --2- 172.21.15.19:0/22433172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f007c109680 0x7f007c1a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:47.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.476+0000 7f0079e00700 1 -- 172.21.15.19:0/22433172 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f007c1a0d20 con 0x7f007c108d30 2024-05-07T02:32:47.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.476+0000 7f007b200700 1 --2- 172.21.15.19:0/22433172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f007c109680 0x7f007c1a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:32:47.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.477+0000 7f0079e00700 1 --2- 172.21.15.19:0/22433172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f007c108d30 0x7f007c199cc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f007c06c3c0 tx=0x7f006800dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:47.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.477+0000 7f0073400700 1 -- 172.21.15.19:0/22433172 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0068017070 con 0x7f007c108d30 2024-05-07T02:32:47.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.477+0000 7f0080a00700 1 -- 172.21.15.19:0/22433172 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f007c1a1010 con 0x7f007c108d30 2024-05-07T02:32:47.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.477+0000 7f0080a00700 1 -- 172.21.15.19:0/22433172 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f007c1a15a0 con 0x7f007c108d30 2024-05-07T02:32:47.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.479+0000 7f0073400700 1 -- 172.21.15.19:0/22433172 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0068009de0 con 0x7f007c108d30 2024-05-07T02:32:47.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.479+0000 7f0073400700 1 -- 172.21.15.19:0/22433172 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0068010df0 con 0x7f007c108d30 2024-05-07T02:32:47.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.483+0000 7f008fef3b80 1 Processor -- start 2024-05-07T02:32:47.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.484+0000 7f008fef3b80 1 -- start start 2024-05-07T02:32:47.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.484+0000 7f0073400700 1 -- 172.21.15.19:0/22433172 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f006801a070 con 0x7f007c108d30 2024-05-07T02:32:47.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.484+0000 7f008fef3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55897465da90 0x55897465dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:47.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.484+0000 7f008fef3b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55897465e3e0 0x558974667620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:47.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.484+0000 7f008fef3b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558974667b60 0x558974669f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:47.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.484+0000 7f008fef3b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55897457fa20 con 0x558974667b60 2024-05-07T02:32:47.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.484+0000 7f008fef3b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55897451d760 con 0x55897465e3e0 2024-05-07T02:32:47.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.484+0000 7f008fef3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55897457b9e0 con 0x55897465da90 2024-05-07T02:32:47.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.485+0000 7f007b200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55897465da90 0x55897465dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:47.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.485+0000 7f007b200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55897465da90 0x55897465dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:41862/0 (socket says 172.21.15.19:41862) 2024-05-07T02:32:47.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.485+0000 7f007b200700 1 -- 172.21.15.19:0/976504259 learned_addr learned my addr 172.21.15.19:0/976504259 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:32:47.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.485+0000 7f0079e00700 1 --2- 172.21.15.19:0/976504259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558974667b60 0x558974669f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:47.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.486+0000 7f007b200700 1 -- 172.21.15.19:0/976504259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55897465e3e0 msgr2=0x558974667620 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-07T02:32:47.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.486+0000 7f007b200700 1 --2- 172.21.15.19:0/976504259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55897465e3e0 0x558974667620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:47.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.486+0000 7f007b200700 1 -- 172.21.15.19:0/976504259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558974667b60 msgr2=0x558974669f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:32:47.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.486+0000 7f007b200700 1 --2- 172.21.15.19:0/976504259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558974667b60 0x558974669f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:47.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.486+0000 7f007b200700 1 -- 172.21.15.19:0/976504259 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589744cc8d0 con 0x55897465da90 2024-05-07T02:32:47.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.486+0000 7f0079e00700 1 --2- 172.21.15.19:0/976504259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558974667b60 0x558974669f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:32:47.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.487+0000 7f0073400700 1 --2- 172.21.15.19:0/22433172 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f00600b2b80 0x7f00600b5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:47.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.487+0000 7f0073400700 1 -- 172.21.15.19:0/22433172 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f00680e22a0 con 0x7f007c108d30 2024-05-07T02:32:47.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.487+0000 7f007a800700 1 --2- 172.21.15.19:0/22433172 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f00600b2b80 0x7f00600b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:47.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.487+0000 7f007b200700 1 --2- 172.21.15.19:0/976504259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55897465da90 0x55897465dea0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0074009610 tx=0x7f0074009cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f3765a1df06af18 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:47.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.488+0000 7f0067e00700 1 -- 172.21.15.19:0/976504259 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0074004cb0 con 0x55897465da90 2024-05-07T02:32:47.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.489+0000 7f0067e00700 1 -- 172.21.15.19:0/976504259 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0074006060 con 0x55897465da90 2024-05-07T02:32:47.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.489+0000 7f0067e00700 1 -- 172.21.15.19:0/976504259 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0074005780 con 0x55897465da90 2024-05-07T02:32:47.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.490+0000 7f008fef3b80 1 -- 172.21.15.19:0/976504259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55897465da90 msgr2=0x55897465dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:47.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.490+0000 7f007a800700 1 --2- 172.21.15.19:0/22433172 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f00600b2b80 0x7f00600b5090 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f006c0046c0 tx=0x7f006c004060 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:47.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.490+0000 7f008fef3b80 1 --2- 172.21.15.19:0/976504259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55897465da90 0x55897465dea0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0074009610 tx=0x7f0074009cb0 comp rx=0 tx=0).stop 2024-05-07T02:32:47.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.490+0000 7f008fef3b80 1 -- 172.21.15.19:0/976504259 shutdown_connections 2024-05-07T02:32:47.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.490+0000 7f008fef3b80 1 --2- 172.21.15.19:0/976504259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558974667b60 0x558974669f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:47.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.490+0000 7f008fef3b80 1 --2- 172.21.15.19:0/976504259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55897465da90 0x55897465dea0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:47.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.490+0000 7f008fef3b80 1 --2- 172.21.15.19:0/976504259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55897465e3e0 0x558974667620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:47.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.490+0000 7f008fef3b80 1 -- 172.21.15.19:0/976504259 >> 172.21.15.19:0/976504259 conn(0x5589744dfe40 msgr2=0x558974658880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:47.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.490+0000 7f008fef3b80 1 -- 172.21.15.19:0/976504259 shutdown_connections 2024-05-07T02:32:47.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.490+0000 7f008fef3b80 1 -- 172.21.15.19:0/976504259 wait complete. 2024-05-07T02:32:47.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.491+0000 7f008fef3b80 1 -- start start 2024-05-07T02:32:47.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.492+0000 7f008fef3b80 1 Processor -- start 2024-05-07T02:32:47.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.492+0000 7f008fef3b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55897465da90 0x558974664a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:47.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.492+0000 7f008fef3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55897465e3e0 0x558974664fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:47.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.492+0000 7f008fef3b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558974667b60 0x558974665500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:47.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.492+0000 7f008fef3b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55897466a900 con 0x558974667b60 2024-05-07T02:32:47.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.492+0000 7f008fef3b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5589744e5b20 con 0x55897465da90 2024-05-07T02:32:47.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.492+0000 7f008fef3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55897466aa80 con 0x55897465e3e0 2024-05-07T02:32:47.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.492+0000 7f008fef3b80 10 client.0 _ll_register_callbacks cb 0x558974526c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:32:47.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.493+0000 7f007a800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55897465da90 0x558974664a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:47.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.493+0000 7f007a800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55897465da90 0x558974664a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:52692/0 (socket says 172.21.15.19:52692) 2024-05-07T02:32:47.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.493+0000 7f007a800700 1 -- 172.21.15.19:0/1006075828 learned_addr learned my addr 172.21.15.19:0/1006075828 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:32:47.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.493+0000 7f0079e00700 1 --2- 172.21.15.19:0/1006075828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55897465e3e0 0x558974664fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:47.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.493+0000 7f007b200700 1 --2- 172.21.15.19:0/1006075828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558974667b60 0x558974665500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:47.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.494+0000 7f0079e00700 1 -- 172.21.15.19:0/1006075828 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55897465da90 msgr2=0x558974664a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:32:47.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.494+0000 7f0079e00700 1 --2- 172.21.15.19:0/1006075828 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55897465da90 0x558974664a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:47.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.494+0000 7f0079e00700 1 -- 172.21.15.19:0/1006075828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558974667b60 msgr2=0x558974665500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:32:47.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.494+0000 7f0079e00700 1 --2- 172.21.15.19:0/1006075828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558974667b60 0x558974665500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:47.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.494+0000 7f0079e00700 1 -- 172.21.15.19:0/1006075828 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558974545420 con 0x55897465e3e0 2024-05-07T02:32:47.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.494+0000 7f0079e00700 1 --2- 172.21.15.19:0/1006075828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55897465e3e0 0x558974664fc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f00680df540 tx=0x7f006800b680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47f220e9b2d65064 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:47.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.494+0000 7f0064c00700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:32:47.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.494+0000 7f007b200700 1 --2- 172.21.15.19:0/1006075828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558974667b60 0x558974665500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-07T02:32:47.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.495+0000 7f0064c00700 1 -- 172.21.15.19:0/1006075828 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0068021070 con 0x55897465e3e0 2024-05-07T02:32:47.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.495+0000 7f0064c00700 1 -- 172.21.15.19:0/1006075828 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f00680a9490 con 0x55897465e3e0 2024-05-07T02:32:47.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.495+0000 7f008fef3b80 1 -- 172.21.15.19:0/1006075828 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0074006060 con 0x55897465e3e0 2024-05-07T02:32:47.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.495+0000 7f0064c00700 1 -- 172.21.15.19:0/1006075828 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f00680a8790 con 0x55897465e3e0 2024-05-07T02:32:47.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.495+0000 7f008fef3b80 10 client.25432 fetch_fsmap learned FSMap version 31 2024-05-07T02:32:47.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.495+0000 7f008fef3b80 1 -- 172.21.15.19:0/1006075828 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558974675510 con 0x55897465e3e0 2024-05-07T02:32:47.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.496+0000 7f0064c00700 1 -- 172.21.15.19:0/1006075828 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0074006060 con 0x55897465e3e0 2024-05-07T02:32:47.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.496+0000 7f0064c00700 1 -- 172.21.15.19:0/1006075828 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f00680b19f0 con 0x55897465e3e0 2024-05-07T02:32:47.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.496+0000 7f008fef3b80 10 client.25432 fetch_fsmap finished waiting for FSMap version 31 2024-05-07T02:32:47.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.496+0000 7f008fef3b80 10 client.25432 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:32:47.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.496+0000 7f008fef3b80 20 client.25432 populate_metadata read hostname 'smithi019' 2024-05-07T02:32:47.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.496+0000 7f008fef3b80 1 --2- 172.21.15.19:0/1006075828 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x558974677b50 0x558974679f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:47.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.496+0000 7f008fef3b80 4 client.25432 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:32:47.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.496+0000 7f008fef3b80 1 -- 172.21.15.19:0/1006075828 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55897462a2e0 con 0x558974677b50 2024-05-07T02:32:47.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.497+0000 7f007a800700 1 --2- 172.21.15.19:0/1006075828 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x558974677b50 0x558974679f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:47.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.498+0000 7f007a800700 1 --2- 172.21.15.19:0/1006075828 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x558974677b50 0x558974679f70 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=361884d7af11a93a server_cookie=db9ecad53db553fb in_seq=0 out_seq=0 2024-05-07T02:32:47.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.498+0000 7f0064c00700 10 client.25432 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:32:47.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.498+0000 7f0064c00700 1 -- 172.21.15.19:0/1006075828 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55897462a2e0 con 0x558974677b50 2024-05-07T02:32:47.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.499+0000 7f0064c00700 10 client.25432 handle_command_reply: tid=0 2024-05-07T02:32:47.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.499+0000 7f008fef3b80 1 client.25432 shutdown 2024-05-07T02:32:47.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.499+0000 7f008fef3b80 1 -- 172.21.15.19:0/1006075828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55897465e3e0 msgr2=0x558974664fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:47.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.499+0000 7f008fef3b80 1 --2- 172.21.15.19:0/1006075828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55897465e3e0 0x558974664fc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f00680df540 tx=0x7f006800b680 comp rx=0 tx=0).stop 2024-05-07T02:32:47.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.499+0000 7f008fef3b80 1 -- 172.21.15.19:0/1006075828 shutdown_connections 2024-05-07T02:32:47.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.499+0000 7f008fef3b80 1 --2- 172.21.15.19:0/1006075828 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x558974677b50 0x558974679f70 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:47.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.499+0000 7f008fef3b80 1 --2- 172.21.15.19:0/1006075828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558974667b60 0x558974665500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:47.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.499+0000 7f008fef3b80 1 --2- 172.21.15.19:0/1006075828 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55897465da90 0x558974664a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:47.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.499+0000 7f008fef3b80 1 --2- 172.21.15.19:0/1006075828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55897465e3e0 0x558974664fc0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:47.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.499+0000 7f008fef3b80 1 -- 172.21.15.19:0/1006075828 >> 172.21.15.19:0/1006075828 conn(0x5589744dfe40 msgr2=0x55897465a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:47.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.499+0000 7f0064c00700 0 client.25432 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:32:47.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.500+0000 7f008fef3b80 1 -- 172.21.15.19:0/1006075828 shutdown_connections 2024-05-07T02:32:47.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.500+0000 7f008fef3b80 1 -- 172.21.15.19:0/1006075828 wait complete. 2024-05-07T02:32:47.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.501+0000 7f008fef3b80 20 client.25432 trim_cache size 0 max 16384 2024-05-07T02:32:47.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.523+0000 7f008fef3b80 1 Processor -- start 2024-05-07T02:32:47.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.523+0000 7f008fef3b80 1 -- start start 2024-05-07T02:32:47.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.523+0000 7f008fef3b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55897465da90 0x5589746639e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:47.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.523+0000 7f008fef3b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558974667b60 0x558974665340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:47.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.523+0000 7f008fef3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558974673a20 0x55897467f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:47.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.523+0000 7f008fef3b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f005c00ee00 con 0x55897465da90 2024-05-07T02:32:47.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.523+0000 7f008fef3b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f005c010940 con 0x558974667b60 2024-05-07T02:32:47.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.523+0000 7f008fef3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f005c00f4d0 con 0x558974673a20 2024-05-07T02:32:47.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.524+0000 7f007a800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55897465da90 0x5589746639e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:47.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.524+0000 7f007a800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55897465da90 0x5589746639e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60312/0 (socket says 172.21.15.19:60312) 2024-05-07T02:32:47.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.524+0000 7f007a800700 1 -- 172.21.15.19:0/4083996075 learned_addr learned my addr 172.21.15.19:0/4083996075 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:32:47.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.525+0000 7f007a800700 1 -- 172.21.15.19:0/4083996075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558974673a20 msgr2=0x55897467f6a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-07T02:32:47.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.525+0000 7f007a800700 1 --2- 172.21.15.19:0/4083996075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558974673a20 0x55897467f6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:47.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.525+0000 7f007a800700 1 -- 172.21.15.19:0/4083996075 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558974667b60 msgr2=0x558974665340 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-07T02:32:47.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.525+0000 7f007a800700 1 --2- 172.21.15.19:0/4083996075 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558974667b60 0x558974665340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:47.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.525+0000 7f007a800700 1 -- 172.21.15.19:0/4083996075 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55897465cb30 con 0x55897465da90 2024-05-07T02:32:47.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.525+0000 7f007a800700 1 --2- 172.21.15.19:0/4083996075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55897465da90 0x5589746639e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f006c004360 tx=0x7f006c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c936d62db9769089 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:47.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.526+0000 7f0067400700 1 -- 172.21.15.19:0/4083996075 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006c02d070 con 0x55897465da90 2024-05-07T02:32:47.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.526+0000 7f0067400700 1 -- 172.21.15.19:0/4083996075 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f006c00fdf0 con 0x55897465da90 2024-05-07T02:32:47.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.526+0000 7f0067400700 1 -- 172.21.15.19:0/4083996075 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006c027840 con 0x55897465da90 2024-05-07T02:32:47.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.527+0000 7f008fef3b80 1 -- 172.21.15.19:0/4083996075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55897465da90 msgr2=0x5589746639e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:47.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.527+0000 7f008fef3b80 1 --2- 172.21.15.19:0/4083996075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55897465da90 0x5589746639e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f006c004360 tx=0x7f006c015000 comp rx=0 tx=0).stop 2024-05-07T02:32:47.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.527+0000 7f008fef3b80 1 -- 172.21.15.19:0/4083996075 shutdown_connections 2024-05-07T02:32:47.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.527+0000 7f008fef3b80 1 --2- 172.21.15.19:0/4083996075 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558974673a20 0x55897467f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:47.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.527+0000 7f008fef3b80 1 --2- 172.21.15.19:0/4083996075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55897465da90 0x5589746639e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:47.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.527+0000 7f008fef3b80 1 --2- 172.21.15.19:0/4083996075 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558974667b60 0x558974665340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:47.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.527+0000 7f008fef3b80 1 -- 172.21.15.19:0/4083996075 >> 172.21.15.19:0/4083996075 conn(0x5589744dfe40 msgr2=0x558974666ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:47.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.528+0000 7f008fef3b80 1 -- 172.21.15.19:0/4083996075 shutdown_connections 2024-05-07T02:32:47.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.528+0000 7f008fef3b80 1 -- 172.21.15.19:0/4083996075 wait complete. 2024-05-07T02:32:47.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.528+0000 7f008fef3b80 1 -- start start 2024-05-07T02:32:47.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.529+0000 7f008fef3b80 1 Processor -- start 2024-05-07T02:32:47.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.529+0000 7f008fef3b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55897465da90 0x5589746da530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:47.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.529+0000 7f008fef3b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558974667b60 0x5589746daa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:47.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.529+0000 7f008fef3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558974673a20 0x55897466c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:47.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.529+0000 7f008fef3b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f005c010270 con 0x55897465da90 2024-05-07T02:32:47.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.529+0000 7f008fef3b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f005c00fba0 con 0x558974667b60 2024-05-07T02:32:47.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.529+0000 7f008fef3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f005c00cda0 con 0x558974673a20 2024-05-07T02:32:47.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.530+0000 7f008fef3b80 10 client.0 _ll_register_callbacks cb 0x558974526c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:32:47.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.530+0000 7f007a800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55897465da90 0x5589746da530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:47.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.530+0000 7f007a800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55897465da90 0x5589746da530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60322/0 (socket says 172.21.15.19:60322) 2024-05-07T02:32:47.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.530+0000 7f007a800700 1 -- 172.21.15.19:0/2204096081 learned_addr learned my addr 172.21.15.19:0/2204096081 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:32:47.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.531+0000 7f007a800700 1 -- 172.21.15.19:0/2204096081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558974673a20 msgr2=0x55897466c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:32:47.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.531+0000 7f007a800700 1 --2- 172.21.15.19:0/2204096081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558974673a20 0x55897466c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:47.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.531+0000 7f007a800700 1 -- 172.21.15.19:0/2204096081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558974667b60 msgr2=0x5589746daa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:32:47.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.531+0000 7f007a800700 1 --2- 172.21.15.19:0/2204096081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558974667b60 0x5589746daa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:47.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.531+0000 7f007a800700 1 -- 172.21.15.19:0/2204096081 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55897466c830 con 0x55897465da90 2024-05-07T02:32:47.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.531+0000 7f007a800700 1 --2- 172.21.15.19:0/2204096081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55897465da90 0x5589746da530 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f006c000c00 tx=0x7f006c00fb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cb5b052e5c128bf server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:47.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.531+0000 7f0064c00700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-07T02:32:47.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.532+0000 7f0064c00700 1 -- 172.21.15.19:0/2204096081 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006c02d040 con 0x55897465da90 2024-05-07T02:32:47.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.532+0000 7f0064c00700 1 -- 172.21.15.19:0/2204096081 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f006c005030 con 0x55897465da90 2024-05-07T02:32:47.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.532+0000 7f008fef3b80 1 -- 172.21.15.19:0/2204096081 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f006c00fdf0 con 0x55897465da90 2024-05-07T02:32:47.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.533+0000 7f0064c00700 1 -- 172.21.15.19:0/2204096081 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f006c005230 con 0x55897465da90 2024-05-07T02:32:47.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.533+0000 7f008fef3b80 10 client.15309 fetch_fsmap learned FSMap version 31 2024-05-07T02:32:47.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.533+0000 7f008fef3b80 1 -- 172.21.15.19:0/2204096081 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5589746e2ce0 con 0x55897465da90 2024-05-07T02:32:47.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.533+0000 7f0064c00700 1 -- 172.21.15.19:0/2204096081 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f006c00fdf0 con 0x55897465da90 2024-05-07T02:32:47.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.533+0000 7f0064c00700 1 -- 172.21.15.19:0/2204096081 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f006c027220 con 0x55897465da90 2024-05-07T02:32:47.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.534+0000 7f008fef3b80 10 client.15309 fetch_fsmap finished waiting for FSMap version 31 2024-05-07T02:32:47.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.534+0000 7f008fef3b80 10 client.15309 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:32:47.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.534+0000 7f008fef3b80 20 client.15309 populate_metadata read hostname 'smithi019' 2024-05-07T02:32:47.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.534+0000 7f008fef3b80 1 --2- 172.21.15.19:0/2204096081 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5589746e33a0 0x5589746e57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:47.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.534+0000 7f008fef3b80 4 client.15309 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-07T02:32:47.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.534+0000 7f008fef3b80 1 -- 172.21.15.19:0/2204096081 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55897458d400 con 0x5589746e33a0 2024-05-07T02:32:47.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.534+0000 7f0079e00700 1 --2- 172.21.15.19:0/2204096081 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5589746e33a0 0x5589746e57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:47.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.535+0000 7f0064c00700 10 client.15309 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:32:47.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.535+0000 7f0079e00700 1 --2- 172.21.15.19:0/2204096081 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5589746e33a0 0x5589746e57c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b911779d6f44f70d server_cookie=72087eefcf6b3e97 in_seq=0 out_seq=0 2024-05-07T02:32:47.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.535+0000 7f0064c00700 1 -- 172.21.15.19:0/2204096081 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55897458d400 con 0x5589746e33a0 2024-05-07T02:32:47.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.535+0000 7f0064c00700 10 client.15309 handle_command_reply: tid=0 2024-05-07T02:32:47.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.536+0000 7f008fef3b80 1 client.15309 shutdown 2024-05-07T02:32:47.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.536+0000 7f008fef3b80 1 -- 172.21.15.19:0/2204096081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55897465da90 msgr2=0x5589746da530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:47.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.536+0000 7f008fef3b80 1 --2- 172.21.15.19:0/2204096081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55897465da90 0x5589746da530 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f006c000c00 tx=0x7f006c00fb90 comp rx=0 tx=0).stop 2024-05-07T02:32:47.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.536+0000 7f008fef3b80 1 -- 172.21.15.19:0/2204096081 shutdown_connections 2024-05-07T02:32:47.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.536+0000 7f008fef3b80 1 --2- 172.21.15.19:0/2204096081 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5589746e33a0 0x5589746e57c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:47.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.536+0000 7f008fef3b80 1 --2- 172.21.15.19:0/2204096081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558974673a20 0x55897466c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:47.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.536+0000 7f008fef3b80 1 --2- 172.21.15.19:0/2204096081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55897465da90 0x5589746da530 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:47.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.536+0000 7f0064c00700 0 client.15309 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:32:47.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.536+0000 7f008fef3b80 1 --2- 172.21.15.19:0/2204096081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558974667b60 0x5589746daa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:47.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.536+0000 7f008fef3b80 1 -- 172.21.15.19:0/2204096081 >> 172.21.15.19:0/2204096081 conn(0x5589744dfe40 msgr2=0x558974666ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:47.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.537+0000 7f008fef3b80 1 -- 172.21.15.19:0/2204096081 shutdown_connections 2024-05-07T02:32:47.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.537+0000 7f008fef3b80 1 -- 172.21.15.19:0/2204096081 wait complete. 2024-05-07T02:32:47.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.537+0000 7f008fef3b80 20 client.15309 trim_cache size 0 max 16384 2024-05-07T02:32:47.537 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:32:47.537 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-07T02:32:47.538 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-07T02:32:47.538 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:32:47.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.538+0000 7f0080a00700 1 -- 172.21.15.19:0/22433172 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f00600b2b80 msgr2=0x7f00600b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:47.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.538+0000 7f0080a00700 1 --2- 172.21.15.19:0/22433172 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f00600b2b80 0x7f00600b5090 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f006c0046c0 tx=0x7f006c004060 comp rx=0 tx=0).stop 2024-05-07T02:32:47.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.538+0000 7f0080a00700 1 -- 172.21.15.19:0/22433172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f007c108d30 msgr2=0x7f007c199cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:47.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.538+0000 7f0080a00700 1 --2- 172.21.15.19:0/22433172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f007c108d30 0x7f007c199cc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f007c06c3c0 tx=0x7f006800dc30 comp rx=0 tx=0).stop 2024-05-07T02:32:47.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.538+0000 7f0080a00700 1 -- 172.21.15.19:0/22433172 shutdown_connections 2024-05-07T02:32:47.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.538+0000 7f0080a00700 1 --2- 172.21.15.19:0/22433172 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f00600b2b80 0x7f00600b5090 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:47.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.538+0000 7f0080a00700 1 --2- 172.21.15.19:0/22433172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f007c109680 0x7f007c1a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:47.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.538+0000 7f0080a00700 1 --2- 172.21.15.19:0/22433172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f007c104400 0x7f007c199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:47.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.538+0000 7f0080a00700 1 --2- 172.21.15.19:0/22433172 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f007c108d30 0x7f007c199cc0 secure :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f007c06c3c0 tx=0x7f006800dc30 comp rx=0 tx=0).stop 2024-05-07T02:32:47.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.538+0000 7f0080a00700 1 -- 172.21.15.19:0/22433172 >> 172.21.15.19:0/22433172 conn(0x7f007c079d10 msgr2=0x7f007c101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:47.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.539+0000 7f0080a00700 1 -- 172.21.15.19:0/22433172 shutdown_connections 2024-05-07T02:32:47.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:47.539+0000 7f0080a00700 1 -- 172.21.15.19:0/22433172 wait complete. 2024-05-07T02:32:48.382 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-07T02:32:48.382 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-07T02:32:48.936 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:32:49.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:48 smithi027 ceph-mon[34398]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-07T02:32:49.284 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:48 smithi019 ceph-mon[29781]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-07T02:32:49.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:48 smithi064 ceph-mon[34699]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-07T02:32:51.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:50 smithi027 ceph-mon[34398]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-07T02:32:51.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.349+0000 7fd72be00700 1 -- 172.21.15.19:0/2959362652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd72c108d30 msgr2=0x7fd72c109140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:51.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.349+0000 7fd72be00700 1 --2- 172.21.15.19:0/2959362652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd72c108d30 0x7fd72c109140 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd7180099c0 tx=0x7fd718009cd0 comp rx=0 tx=0).stop 2024-05-07T02:32:51.350 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:50 smithi019 ceph-mon[29781]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-07T02:32:51.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.349+0000 7fd72be00700 1 -- 172.21.15.19:0/2959362652 shutdown_connections 2024-05-07T02:32:51.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.349+0000 7fd72be00700 1 --2- 172.21.15.19:0/2959362652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd72c109680 0x7fd72c110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:51.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.349+0000 7fd72be00700 1 --2- 172.21.15.19:0/2959362652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd72c104400 0x7fd72c104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:51.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.349+0000 7fd72be00700 1 --2- 172.21.15.19:0/2959362652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd72c108d30 0x7fd72c109140 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:51.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.349+0000 7fd72be00700 1 -- 172.21.15.19:0/2959362652 >> 172.21.15.19:0/2959362652 conn(0x7fd72c079d10 msgr2=0x7fd72c07a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:51.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.350+0000 7fd72be00700 1 -- 172.21.15.19:0/2959362652 shutdown_connections 2024-05-07T02:32:51.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.350+0000 7fd72be00700 1 -- 172.21.15.19:0/2959362652 wait complete. 2024-05-07T02:32:51.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.351+0000 7fd72be00700 1 Processor -- start 2024-05-07T02:32:51.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.351+0000 7fd72be00700 1 -- start start 2024-05-07T02:32:51.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.352+0000 7fd72be00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd72c104400 0x7fd72c072960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:51.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.352+0000 7fd72be00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd72c108d30 0x7fd72c072ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:51.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.352+0000 7fd72be00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd72c109680 0x7fd72c0733e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:51.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.352+0000 7fd72be00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd72c114290 con 0x7fd72c104400 2024-05-07T02:32:51.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.352+0000 7fd72be00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd72c114110 con 0x7fd72c109680 2024-05-07T02:32:51.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.352+0000 7fd72be00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd72c114410 con 0x7fd72c108d30 2024-05-07T02:32:51.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.352+0000 7fd72b400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd72c109680 0x7fd72c0733e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:51.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.353+0000 7fd72aa00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd72c104400 0x7fd72c072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:51.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.353+0000 7fd72b400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd72c109680 0x7fd72c0733e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:37786/0 (socket says 172.21.15.19:37786) 2024-05-07T02:32:51.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.353+0000 7fd72b400700 1 -- 172.21.15.19:0/3533034404 learned_addr learned my addr 172.21.15.19:0/3533034404 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:32:51.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.353+0000 7fd72a000700 1 --2- 172.21.15.19:0/3533034404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd72c108d30 0x7fd72c072ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:51.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.353+0000 7fd72b400700 1 -- 172.21.15.19:0/3533034404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd72c108d30 msgr2=0x7fd72c072ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:51.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.353+0000 7fd72b400700 1 --2- 172.21.15.19:0/3533034404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd72c108d30 0x7fd72c072ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:51.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.353+0000 7fd72b400700 1 -- 172.21.15.19:0/3533034404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd72c104400 msgr2=0x7fd72c072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:51.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.353+0000 7fd72b400700 1 --2- 172.21.15.19:0/3533034404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd72c104400 0x7fd72c072960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:51.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.353+0000 7fd72b400700 1 -- 172.21.15.19:0/3533034404 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd72c1b17e0 con 0x7fd72c109680 2024-05-07T02:32:51.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.353+0000 7fd72a000700 1 --2- 172.21.15.19:0/3533034404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd72c108d30 0x7fd72c072ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:32:51.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.353+0000 7fd72aa00700 1 --2- 172.21.15.19:0/3533034404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd72c104400 0x7fd72c072960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:32:51.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.354+0000 7fd72b400700 1 --2- 172.21.15.19:0/3533034404 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd72c109680 0x7fd72c0733e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd72400c370 tx=0x7fd72400c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:51.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.354+0000 7fd723400700 1 -- 172.21.15.19:0/3533034404 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7240092e0 con 0x7fd72c109680 2024-05-07T02:32:51.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.354+0000 7fd723400700 1 -- 172.21.15.19:0/3533034404 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd724003a00 con 0x7fd72c109680 2024-05-07T02:32:51.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.354+0000 7fd72be00700 1 -- 172.21.15.19:0/3533034404 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd72c1b1a70 con 0x7fd72c109680 2024-05-07T02:32:51.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.354+0000 7fd723400700 1 -- 172.21.15.19:0/3533034404 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7240046e0 con 0x7fd72c109680 2024-05-07T02:32:51.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.354+0000 7fd72be00700 1 -- 172.21.15.19:0/3533034404 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd72c1b1f80 con 0x7fd72c109680 2024-05-07T02:32:51.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.356+0000 7fd72be00700 1 -- 172.21.15.19:0/3533034404 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd72c1058b0 con 0x7fd72c109680 2024-05-07T02:32:51.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.358+0000 7fd723400700 1 -- 172.21.15.19:0/3533034404 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7fd72401a020 con 0x7fd72c109680 2024-05-07T02:32:51.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:50 smithi064 ceph-mon[34699]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-07T02:32:51.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.361+0000 7fd723400700 1 --2- 172.21.15.19:0/3533034404 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fd7100b2b00 0x7fd7100b5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:51.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.361+0000 7fd723400700 1 -- 172.21.15.19:0/3533034404 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fd7240e2950 con 0x7fd72c109680 2024-05-07T02:32:51.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.362+0000 7fd723400700 1 -- 172.21.15.19:0/3533034404 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd724010040 con 0x7fd72c109680 2024-05-07T02:32:51.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.365+0000 7fd72aa00700 1 --2- 172.21.15.19:0/3533034404 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fd7100b2b00 0x7fd7100b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:51.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.365+0000 7fd72aa00700 1 --2- 172.21.15.19:0/3533034404 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fd7100b2b00 0x7fd7100b5010 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd71c011490 tx=0x7fd71c009260 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:51.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.641+0000 7fd72be00700 1 -- 172.21.15.19:0/3533034404 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd72c073e90 con 0x7fd72c109680 2024-05-07T02:32:51.642 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:32:51.643 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1763670446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1763670446},{"type":"v1","addr":"172.21.15.19:6839","nonce":1763670446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/477529985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":477529985},{"type":"v1","addr":"172.21.15.64:6833","nonce":477529985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-07T02:26:14.960465+0000","modified":"2024-05-07T02:31:47.904333+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24539,"mds_2":24520,"mds_3":24515,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6833/2171873162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2171873162},{"type":"v1","addr":"172.21.15.27:6833","nonce":2171873162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/2379917839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2379917839},{"type":"v1","addr":"172.21.15.19:6835","nonce":2379917839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6837/2067515538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2067515538},{"type":"v1","addr":"172.21.15.19:6837","nonce":2067515538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6835/1000540600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1000540600},{"type":"v1","addr":"172.21.15.27:6835","nonce":1000540600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/1997318872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1997318872},{"type":"v1","addr":"172.21.15.64:6835","nonce":1997318872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":34,"addr":"172.21.15.64:6837/2585907777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2585907777},{"type":"v1","addr":"172.21.15.64:6837","nonce":2585907777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":33,"addr":"172.21.15.27:6837/2583138143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2583138143},{"type":"v1","addr":"172.21.15.27:6837","nonce":2583138143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":32,"addr":"172.21.15.64:6839/1583120004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":1583120004},{"type":"v1","addr":"172.21.15.64:6839","nonce":1583120004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":29,"addr":"172.21.15.19:6841/3905645246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3905645246},{"type":"v1","addr":"172.21.15.19:6841","nonce":3905645246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6839/3963268634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3963268634},{"type":"v1","addr":"172.21.15.27:6839","nonce":3963268634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24515,24505,24520,24551,24539]},"id":1}]} 2024-05-07T02:32:51.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.642+0000 7fd723400700 1 -- 172.21.15.19:0/3533034404 <== mon.2 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11924 (secure 0 0 0) 0x7fd7240a9dd0 con 0x7fd72c109680 2024-05-07T02:32:51.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.645+0000 7fd72be00700 1 -- 172.21.15.19:0/3533034404 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fd7100b2b00 msgr2=0x7fd7100b5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:51.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.645+0000 7fd72be00700 1 --2- 172.21.15.19:0/3533034404 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fd7100b2b00 0x7fd7100b5010 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd71c011490 tx=0x7fd71c009260 comp rx=0 tx=0).stop 2024-05-07T02:32:51.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.645+0000 7fd72be00700 1 -- 172.21.15.19:0/3533034404 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd72c109680 msgr2=0x7fd72c0733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:51.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.645+0000 7fd72be00700 1 --2- 172.21.15.19:0/3533034404 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd72c109680 0x7fd72c0733e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd72400c370 tx=0x7fd72400c680 comp rx=0 tx=0).stop 2024-05-07T02:32:51.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.645+0000 7fd72be00700 1 -- 172.21.15.19:0/3533034404 shutdown_connections 2024-05-07T02:32:51.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.645+0000 7fd72be00700 1 --2- 172.21.15.19:0/3533034404 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd72c109680 0x7fd72c0733e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:51.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.645+0000 7fd72be00700 1 --2- 172.21.15.19:0/3533034404 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fd7100b2b00 0x7fd7100b5010 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:51.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.645+0000 7fd72be00700 1 --2- 172.21.15.19:0/3533034404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd72c104400 0x7fd72c072960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:51.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.645+0000 7fd72be00700 1 --2- 172.21.15.19:0/3533034404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd72c108d30 0x7fd72c072ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:51.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.645+0000 7fd72be00700 1 -- 172.21.15.19:0/3533034404 >> 172.21.15.19:0/3533034404 conn(0x7fd72c079d10 msgr2=0x7fd72c101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:51.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.645+0000 7fd72be00700 1 -- 172.21.15.19:0/3533034404 shutdown_connections 2024-05-07T02:32:51.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:51.645+0000 7fd72be00700 1 -- 172.21.15.19:0/3533034404 wait complete. 2024-05-07T02:32:51.649 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 31 2024-05-07T02:32:52.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:51 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/3533034404' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:32:52.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:51 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/3533034404' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:32:52.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:51 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/3533034404' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:32:52.568 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-07T02:32:52.998 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:32:53.208 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:52 smithi019 ceph-mon[29781]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-05-07T02:32:53.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:52 smithi064 ceph-mon[34699]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-05-07T02:32:53.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:52 smithi027 ceph-mon[34398]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-05-07T02:32:55.135 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:54 smithi019 ceph-mon[29781]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-07T02:32:55.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:54 smithi064 ceph-mon[34699]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-07T02:32:55.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:54 smithi027 ceph-mon[34398]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-07T02:32:55.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.637+0000 7f291c000700 1 -- 172.21.15.19:0/288739253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2914106570 msgr2=0x7f2914106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:55.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.637+0000 7f291c000700 1 --2- 172.21.15.19:0/288739253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2914106570 0x7f2914106980 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f29000098f0 tx=0x7f2900009c00 comp rx=0 tx=0).stop 2024-05-07T02:32:55.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.637+0000 7f291c000700 1 -- 172.21.15.19:0/288739253 shutdown_connections 2024-05-07T02:32:55.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.637+0000 7f291c000700 1 --2- 172.21.15.19:0/288739253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2914106ec0 0x7f2914113350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:55.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.637+0000 7f291c000700 1 --2- 172.21.15.19:0/288739253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f291410c5b0 0x7f291410c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:55.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.637+0000 7f291c000700 1 --2- 172.21.15.19:0/288739253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2914106570 0x7f2914106980 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:55.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.637+0000 7f291c000700 1 -- 172.21.15.19:0/288739253 >> 172.21.15.19:0/288739253 conn(0x7f2914102050 msgr2=0x7f29141044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:55.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.637+0000 7f291c000700 1 -- 172.21.15.19:0/288739253 shutdown_connections 2024-05-07T02:32:55.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.639+0000 7f291c000700 1 -- 172.21.15.19:0/288739253 wait complete. 2024-05-07T02:32:55.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.640+0000 7f291c000700 1 Processor -- start 2024-05-07T02:32:55.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.640+0000 7f291c000700 1 -- start start 2024-05-07T02:32:55.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.641+0000 7f291c000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2914106570 0x7f291419d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:55.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.641+0000 7f291c000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2914106ec0 0x7f291419d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:55.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.641+0000 7f291c000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f291410c5b0 0x7f29141a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:55.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.641+0000 7f291c000700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2914116690 con 0x7f2914106ec0 2024-05-07T02:32:55.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.641+0000 7f291c000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2914116510 con 0x7f2914106570 2024-05-07T02:32:55.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.641+0000 7f291c000700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2914116810 con 0x7f291410c5b0 2024-05-07T02:32:55.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.641+0000 7f2919600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2914106570 0x7f291419d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:55.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.641+0000 7f2918c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2914106ec0 0x7f291419d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:55.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.641+0000 7f2919600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2914106570 0x7f291419d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:37804/0 (socket says 172.21.15.19:37804) 2024-05-07T02:32:55.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.641+0000 7f2919600700 1 -- 172.21.15.19:0/3254278702 learned_addr learned my addr 172.21.15.19:0/3254278702 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:32:55.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.641+0000 7f291a000700 1 --2- 172.21.15.19:0/3254278702 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f291410c5b0 0x7f29141a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:55.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.642+0000 7f2919600700 1 -- 172.21.15.19:0/3254278702 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f291410c5b0 msgr2=0x7f29141a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:55.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.642+0000 7f2919600700 1 --2- 172.21.15.19:0/3254278702 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f291410c5b0 0x7f29141a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:55.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.642+0000 7f2919600700 1 -- 172.21.15.19:0/3254278702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2914106ec0 msgr2=0x7f291419d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:55.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.642+0000 7f2919600700 1 --2- 172.21.15.19:0/3254278702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2914106ec0 0x7f291419d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:55.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.642+0000 7f2919600700 1 -- 172.21.15.19:0/3254278702 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29141a1ef0 con 0x7f2914106570 2024-05-07T02:32:55.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.642+0000 7f2918c00700 1 --2- 172.21.15.19:0/3254278702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2914106ec0 0x7f291419d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:32:55.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.643+0000 7f2919600700 1 --2- 172.21.15.19:0/3254278702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2914106570 0x7f291419d020 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f290800ea30 tx=0x7f290800ed40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:55.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.643+0000 7f2912000700 1 -- 172.21.15.19:0/3254278702 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290800cbd0 con 0x7f2914106570 2024-05-07T02:32:55.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.643+0000 7f291c000700 1 -- 172.21.15.19:0/3254278702 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29141a21e0 con 0x7f2914106570 2024-05-07T02:32:55.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.643+0000 7f2912000700 1 -- 172.21.15.19:0/3254278702 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29080044d0 con 0x7f2914106570 2024-05-07T02:32:55.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.643+0000 7f2912000700 1 -- 172.21.15.19:0/3254278702 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2908010430 con 0x7f2914106570 2024-05-07T02:32:55.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.644+0000 7f291c000700 1 -- 172.21.15.19:0/3254278702 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29141a9b00 con 0x7f2914106570 2024-05-07T02:32:55.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.650+0000 7f2912000700 1 -- 172.21.15.19:0/3254278702 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f2908003e80 con 0x7f2914106570 2024-05-07T02:32:55.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.650+0000 7f292abc7b80 1 Processor -- start 2024-05-07T02:32:55.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.650+0000 7f292abc7b80 1 -- start start 2024-05-07T02:32:55.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.651+0000 7f292abc7b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6a51a3ae0 0x55c6a51a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:55.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.651+0000 7f292abc7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6a519dc80 0x55c6a519e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:55.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.651+0000 7f292abc7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6a519e5d0 0x55c6a51aa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:55.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.651+0000 7f292abc7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c6a50b7a20 con 0x55c6a519e5d0 2024-05-07T02:32:55.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.651+0000 7f292abc7b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c6a5055760 con 0x55c6a51a3ae0 2024-05-07T02:32:55.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.651+0000 7f292abc7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c6a50b39e0 con 0x55c6a519dc80 2024-05-07T02:32:55.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.651+0000 7f2912000700 1 --2- 172.21.15.19:0/3254278702 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f28fc0b2a80 0x7f28fc0b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:55.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.651+0000 7f291a000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6a51a3ae0 0x55c6a51a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:55.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.652+0000 7f2918c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6a519e5d0 0x55c6a51aa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:55.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.652+0000 7f2912000700 1 -- 172.21.15.19:0/3254278702 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f29080e2de0 con 0x7f2914106570 2024-05-07T02:32:55.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.652+0000 7f291a000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6a51a3ae0 0x55c6a51a3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:37818/0 (socket says 172.21.15.19:37818) 2024-05-07T02:32:55.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.652+0000 7f291a000700 1 -- 172.21.15.19:0/4080771312 learned_addr learned my addr 172.21.15.19:0/4080771312 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:32:55.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.652+0000 7f2919600700 1 --2- 172.21.15.19:0/4080771312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6a519dc80 0x55c6a519e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:55.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.652+0000 7f291a000700 1 --2- 172.21.15.19:0/3254278702 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f28fc0b2a80 0x7f28fc0b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:55.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.652+0000 7f291a000700 1 -- 172.21.15.19:0/4080771312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6a519dc80 msgr2=0x55c6a519e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:32:55.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.652+0000 7f291a000700 1 --2- 172.21.15.19:0/4080771312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6a519dc80 0x55c6a519e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:55.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.652+0000 7f291a000700 1 -- 172.21.15.19:0/4080771312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6a519e5d0 msgr2=0x55c6a51aa8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:32:55.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.652+0000 7f291a000700 1 --2- 172.21.15.19:0/4080771312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6a519e5d0 0x55c6a51aa8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:55.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.652+0000 7f291a000700 1 -- 172.21.15.19:0/4080771312 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6a50048d0 con 0x55c6a51a3ae0 2024-05-07T02:32:55.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.652+0000 7f2918c00700 1 --2- 172.21.15.19:0/4080771312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6a519e5d0 0x55c6a51aa8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:32:55.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.653+0000 7f291a000700 1 --2- 172.21.15.19:0/4080771312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6a51a3ae0 0x55c6a51a3ef0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f290c007880 tx=0x7f290c007b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdcc1bfccf555e02 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:55.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.653+0000 7f2906a00700 1 -- 172.21.15.19:0/4080771312 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c01a470 con 0x55c6a51a3ae0 2024-05-07T02:32:55.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.653+0000 7f291a000700 1 --2- 172.21.15.19:0/3254278702 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f28fc0b2a80 0x7f28fc0b4f90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f290c010000 tx=0x7f290c01d000 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:55.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.653+0000 7f2906a00700 1 -- 172.21.15.19:0/4080771312 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f290c01aac0 con 0x55c6a51a3ae0 2024-05-07T02:32:55.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.653+0000 7f2906a00700 1 -- 172.21.15.19:0/4080771312 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c0228c0 con 0x55c6a51a3ae0 2024-05-07T02:32:55.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.655+0000 7f292abc7b80 1 -- 172.21.15.19:0/4080771312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6a51a3ae0 msgr2=0x55c6a51a3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:55.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.655+0000 7f292abc7b80 1 --2- 172.21.15.19:0/4080771312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6a51a3ae0 0x55c6a51a3ef0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f290c007880 tx=0x7f290c007b90 comp rx=0 tx=0).stop 2024-05-07T02:32:55.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.655+0000 7f292abc7b80 1 -- 172.21.15.19:0/4080771312 shutdown_connections 2024-05-07T02:32:55.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.655+0000 7f292abc7b80 1 --2- 172.21.15.19:0/4080771312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6a519e5d0 0x55c6a51aa8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:55.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.655+0000 7f292abc7b80 1 --2- 172.21.15.19:0/4080771312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6a51a3ae0 0x55c6a51a3ef0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:55.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.655+0000 7f292abc7b80 1 --2- 172.21.15.19:0/4080771312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6a519dc80 0x55c6a519e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:55.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.655+0000 7f292abc7b80 1 -- 172.21.15.19:0/4080771312 >> 172.21.15.19:0/4080771312 conn(0x55c6a5017e40 msgr2=0x55c6a51902f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:55.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.655+0000 7f292abc7b80 1 -- 172.21.15.19:0/4080771312 shutdown_connections 2024-05-07T02:32:55.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.655+0000 7f292abc7b80 1 -- 172.21.15.19:0/4080771312 wait complete. 2024-05-07T02:32:55.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.656+0000 7f292abc7b80 1 -- start start 2024-05-07T02:32:55.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.656+0000 7f292abc7b80 1 Processor -- start 2024-05-07T02:32:55.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.657+0000 7f292abc7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6a519dc80 0x55c6a51a7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:55.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.657+0000 7f292abc7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6a519e5d0 0x55c6a51a83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:55.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.657+0000 7f292abc7b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6a51a3ae0 0x55c6a51a8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:55.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.657+0000 7f292abc7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c6a51aae20 con 0x55c6a519e5d0 2024-05-07T02:32:55.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.657+0000 7f292abc7b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c6a501db20 con 0x55c6a51a3ae0 2024-05-07T02:32:55.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.657+0000 7f292abc7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c6a51aafa0 con 0x55c6a519dc80 2024-05-07T02:32:55.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.657+0000 7f292abc7b80 10 client.0 _ll_register_callbacks cb 0x55c6a505ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:32:55.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.657+0000 7f291a000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6a51a3ae0 0x55c6a51a8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:55.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.657+0000 7f2919600700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6a519dc80 0x55c6a51a7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:55.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.657+0000 7f291a000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6a51a3ae0 0x55c6a51a8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:37830/0 (socket says 172.21.15.19:37830) 2024-05-07T02:32:55.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.657+0000 7f291a000700 1 -- 172.21.15.19:0/1018715849 learned_addr learned my addr 172.21.15.19:0/1018715849 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:32:55.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.658+0000 7f291a000700 1 -- 172.21.15.19:0/1018715849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6a519dc80 msgr2=0x55c6a51a7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:32:55.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.658+0000 7f291a000700 1 --2- 172.21.15.19:0/1018715849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6a519dc80 0x55c6a51a7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:55.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.658+0000 7f291a000700 1 -- 172.21.15.19:0/1018715849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6a519e5d0 msgr2=0x55c6a51a83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:32:55.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.658+0000 7f2918c00700 1 --2- 172.21.15.19:0/1018715849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6a519e5d0 0x55c6a51a83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:55.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.658+0000 7f291a000700 1 --2- 172.21.15.19:0/1018715849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6a519e5d0 0x55c6a51a83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:55.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.658+0000 7f291a000700 1 -- 172.21.15.19:0/1018715849 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6a507d420 con 0x55c6a51a3ae0 2024-05-07T02:32:55.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.658+0000 7f2918c00700 1 --2- 172.21.15.19:0/1018715849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6a519e5d0 0x55c6a51a83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-07T02:32:55.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.659+0000 7f291a000700 1 --2- 172.21.15.19:0/1018715849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6a51a3ae0 0x55c6a51a8900 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f290c0224a0 tx=0x7f290c023e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fdf663887e4206b server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:55.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.659+0000 7f28fb400700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:32:55.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.659+0000 7f28fb400700 1 -- 172.21.15.19:0/1018715849 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c004d60 con 0x55c6a51a3ae0 2024-05-07T02:32:55.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.659+0000 7f292abc7b80 1 -- 172.21.15.19:0/1018715849 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f290c01aac0 con 0x55c6a51a3ae0 2024-05-07T02:32:55.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.659+0000 7f28fb400700 1 -- 172.21.15.19:0/1018715849 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f290c01a470 con 0x55c6a51a3ae0 2024-05-07T02:32:55.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.659+0000 7f28fb400700 1 -- 172.21.15.19:0/1018715849 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f290c03c050 con 0x55c6a51a3ae0 2024-05-07T02:32:55.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.659+0000 7f292abc7b80 10 client.25466 fetch_fsmap learned FSMap version 31 2024-05-07T02:32:55.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.659+0000 7f292abc7b80 1 -- 172.21.15.19:0/1018715849 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c6a51b5b40 con 0x55c6a51a3ae0 2024-05-07T02:32:55.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.660+0000 7f28fb400700 1 -- 172.21.15.19:0/1018715849 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f290c01aac0 con 0x55c6a51a3ae0 2024-05-07T02:32:55.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.660+0000 7f28fb400700 1 -- 172.21.15.19:0/1018715849 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f290c018040 con 0x55c6a51a3ae0 2024-05-07T02:32:55.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.660+0000 7f292abc7b80 10 client.25466 fetch_fsmap finished waiting for FSMap version 31 2024-05-07T02:32:55.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.660+0000 7f292abc7b80 10 client.25466 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-07T02:32:55.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.660+0000 7f292abc7b80 20 client.25466 populate_metadata read hostname 'smithi019' 2024-05-07T02:32:55.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.660+0000 7f292abc7b80 1 --2- 172.21.15.19:0/1018715849 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x55c6a51b80d0 0x55c6a51ba4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:55.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.660+0000 7f292abc7b80 4 client.25466 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:32:55.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.660+0000 7f292abc7b80 1 -- 172.21.15.19:0/1018715849 --> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6a51622e0 con 0x55c6a51b80d0 2024-05-07T02:32:55.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.660+0000 7f2919600700 1 --2- 172.21.15.19:0/1018715849 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x55c6a51b80d0 0x55c6a51ba4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:55.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.661+0000 7f28fb400700 10 client.25466 ms_handle_connect on v2:172.21.15.64:6836/2585907777 2024-05-07T02:32:55.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.661+0000 7f2919600700 1 --2- 172.21.15.19:0/1018715849 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x55c6a51b80d0 0x55c6a51ba4f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7b879ab3195e5958 server_cookie=79baae4237242bd5 in_seq=0 out_seq=0 2024-05-07T02:32:55.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.662+0000 7f28fb400700 1 -- 172.21.15.19:0/1018715849 <== mds.4 v2:172.21.15.64:6836/2585907777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c6a51622e0 con 0x55c6a51b80d0 2024-05-07T02:32:55.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.662+0000 7f28fb400700 10 client.25466 handle_command_reply: tid=0 2024-05-07T02:32:55.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.662+0000 7f292abc7b80 1 client.25466 shutdown 2024-05-07T02:32:55.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.662+0000 7f292abc7b80 1 -- 172.21.15.19:0/1018715849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6a51a3ae0 msgr2=0x55c6a51a8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:55.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.662+0000 7f292abc7b80 1 --2- 172.21.15.19:0/1018715849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6a51a3ae0 0x55c6a51a8900 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f290c0224a0 tx=0x7f290c023e00 comp rx=0 tx=0).stop 2024-05-07T02:32:55.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.662+0000 7f292abc7b80 1 -- 172.21.15.19:0/1018715849 shutdown_connections 2024-05-07T02:32:55.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.662+0000 7f292abc7b80 1 --2- 172.21.15.19:0/1018715849 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x55c6a51b80d0 0x55c6a51ba4f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:55.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.662+0000 7f292abc7b80 1 --2- 172.21.15.19:0/1018715849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6a51a3ae0 0x55c6a51a8900 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:55.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.662+0000 7f292abc7b80 1 --2- 172.21.15.19:0/1018715849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6a519dc80 0x55c6a51a7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:55.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.662+0000 7f292abc7b80 1 --2- 172.21.15.19:0/1018715849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6a519e5d0 0x55c6a51a83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:55.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.662+0000 7f28fb400700 0 client.25466 ms_handle_reset on v2:172.21.15.64:6836/2585907777 2024-05-07T02:32:55.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.662+0000 7f292abc7b80 1 -- 172.21.15.19:0/1018715849 >> 172.21.15.19:0/1018715849 conn(0x55c6a5017e40 msgr2=0x55c6a5194460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:55.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.662+0000 7f292abc7b80 1 -- 172.21.15.19:0/1018715849 shutdown_connections 2024-05-07T02:32:55.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.663+0000 7f292abc7b80 1 -- 172.21.15.19:0/1018715849 wait complete. 2024-05-07T02:32:55.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.663+0000 7f292abc7b80 20 client.25466 trim_cache size 0 max 16384 2024-05-07T02:32:55.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.679+0000 7f292abc7b80 1 Processor -- start 2024-05-07T02:32:55.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.679+0000 7f292abc7b80 1 -- start start 2024-05-07T02:32:55.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.679+0000 7f292abc7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6a519dc80 0x55c6a5190440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:55.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.679+0000 7f292abc7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6a519e5d0 0x55c6a51b4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:55.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.679+0000 7f292abc7b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6a51a3ae0 0x55c6a51a8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:55.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.679+0000 7f292abc7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f29141ae4a0 con 0x55c6a519e5d0 2024-05-07T02:32:55.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.679+0000 7f292abc7b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f29141affe0 con 0x55c6a51a3ae0 2024-05-07T02:32:55.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.679+0000 7f292abc7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f29141aeb70 con 0x55c6a519dc80 2024-05-07T02:32:55.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.680+0000 7f2918c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6a519e5d0 0x55c6a51b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:55.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.680+0000 7f2918c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6a519e5d0 0x55c6a51b4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40650/0 (socket says 172.21.15.19:40650) 2024-05-07T02:32:55.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.680+0000 7f2918c00700 1 -- 172.21.15.19:0/1616579734 learned_addr learned my addr 172.21.15.19:0/1616579734 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:32:55.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.680+0000 7f291a000700 1 --2- 172.21.15.19:0/1616579734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6a51a3ae0 0x55c6a51a8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:55.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.680+0000 7f2919600700 1 --2- 172.21.15.19:0/1616579734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6a519dc80 0x55c6a5190440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:55.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.681+0000 7f2918c00700 1 -- 172.21.15.19:0/1616579734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6a519dc80 msgr2=0x55c6a5190440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:32:55.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.681+0000 7f2918c00700 1 --2- 172.21.15.19:0/1616579734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6a519dc80 0x55c6a5190440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:55.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.681+0000 7f2918c00700 1 -- 172.21.15.19:0/1616579734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6a51a3ae0 msgr2=0x55c6a51a8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:32:55.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.681+0000 7f2918c00700 1 --2- 172.21.15.19:0/1616579734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6a51a3ae0 0x55c6a51a8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:55.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.681+0000 7f2918c00700 1 -- 172.21.15.19:0/1616579734 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6a5191920 con 0x55c6a519e5d0 2024-05-07T02:32:55.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.681+0000 7f2918c00700 1 --2- 172.21.15.19:0/1616579734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6a519e5d0 0x55c6a51b4240 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2914075b80 tx=0x7f2900005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=590e8ed6314e6f74 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:55.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.681+0000 7f2906000700 1 -- 172.21.15.19:0/1616579734 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290000f660 con 0x55c6a519e5d0 2024-05-07T02:32:55.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.682+0000 7f2906000700 1 -- 172.21.15.19:0/1616579734 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f290000f800 con 0x55c6a519e5d0 2024-05-07T02:32:55.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.682+0000 7f2906000700 1 -- 172.21.15.19:0/1616579734 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2900003bf0 con 0x55c6a519e5d0 2024-05-07T02:32:55.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.683+0000 7f292abc7b80 1 -- 172.21.15.19:0/1616579734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6a519e5d0 msgr2=0x55c6a51b4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:55.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.683+0000 7f292abc7b80 1 --2- 172.21.15.19:0/1616579734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6a519e5d0 0x55c6a51b4240 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2914075b80 tx=0x7f2900005dc0 comp rx=0 tx=0).stop 2024-05-07T02:32:55.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.683+0000 7f292abc7b80 1 -- 172.21.15.19:0/1616579734 shutdown_connections 2024-05-07T02:32:55.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.683+0000 7f292abc7b80 1 --2- 172.21.15.19:0/1616579734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6a51a3ae0 0x55c6a51a8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:55.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.683+0000 7f292abc7b80 1 --2- 172.21.15.19:0/1616579734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6a519dc80 0x55c6a5190440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:55.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.683+0000 7f292abc7b80 1 --2- 172.21.15.19:0/1616579734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6a519e5d0 0x55c6a51b4240 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:55.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.683+0000 7f292abc7b80 1 -- 172.21.15.19:0/1616579734 >> 172.21.15.19:0/1616579734 conn(0x55c6a5017e40 msgr2=0x55c6a5194460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:55.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.683+0000 7f292abc7b80 1 -- 172.21.15.19:0/1616579734 shutdown_connections 2024-05-07T02:32:55.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.683+0000 7f292abc7b80 1 -- 172.21.15.19:0/1616579734 wait complete. 2024-05-07T02:32:55.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.684+0000 7f292abc7b80 1 -- start start 2024-05-07T02:32:55.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.684+0000 7f292abc7b80 1 Processor -- start 2024-05-07T02:32:55.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.685+0000 7f292abc7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6a519dc80 0x55c6a51bd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:55.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.685+0000 7f292abc7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6a519e5d0 0x55c6a51bad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:55.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.685+0000 7f292abc7b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6a51a3ae0 0x55c6a51bb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:55.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.685+0000 7f292abc7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f29141af910 con 0x55c6a519e5d0 2024-05-07T02:32:55.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.685+0000 7f292abc7b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f29141af240 con 0x55c6a51a3ae0 2024-05-07T02:32:55.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.685+0000 7f292abc7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f29141ac440 con 0x55c6a519dc80 2024-05-07T02:32:55.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.685+0000 7f292abc7b80 10 client.0 _ll_register_callbacks cb 0x55c6a505ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:32:55.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.685+0000 7f2919600700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6a519dc80 0x55c6a51bd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:55.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.685+0000 7f291a000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6a51a3ae0 0x55c6a51bb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:55.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.685+0000 7f291a000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6a51a3ae0 0x55c6a51bb2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:37844/0 (socket says 172.21.15.19:37844) 2024-05-07T02:32:55.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.685+0000 7f291a000700 1 -- 172.21.15.19:0/1314275019 learned_addr learned my addr 172.21.15.19:0/1314275019 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:32:55.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.685+0000 7f2918c00700 1 --2- 172.21.15.19:0/1314275019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6a519e5d0 0x55c6a51bad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:55.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.686+0000 7f2919600700 1 -- 172.21.15.19:0/1314275019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6a51a3ae0 msgr2=0x55c6a51bb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:32:55.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.686+0000 7f2919600700 1 --2- 172.21.15.19:0/1314275019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6a51a3ae0 0x55c6a51bb2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:55.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.686+0000 7f2919600700 1 -- 172.21.15.19:0/1314275019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6a519e5d0 msgr2=0x55c6a51bad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:32:55.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.686+0000 7f2919600700 1 --2- 172.21.15.19:0/1314275019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6a519e5d0 0x55c6a51bad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:55.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.686+0000 7f2919600700 1 -- 172.21.15.19:0/1314275019 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6a51bb990 con 0x55c6a519dc80 2024-05-07T02:32:55.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.686+0000 7f291a000700 1 --2- 172.21.15.19:0/1314275019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6a51a3ae0 0x55c6a51bb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:32:55.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.686+0000 7f2919600700 1 --2- 172.21.15.19:0/1314275019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6a519dc80 0x55c6a51bd9f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f29080e0080 tx=0x7f29080a6020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f0021820731e5dd server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:55.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.686+0000 7f28fb400700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:32:55.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.686+0000 7f28fb400700 1 -- 172.21.15.19:0/1314275019 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29080a52f0 con 0x55c6a519dc80 2024-05-07T02:32:55.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.686+0000 7f28fb400700 1 -- 172.21.15.19:0/1314275019 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f29080a6150 con 0x55c6a519dc80 2024-05-07T02:32:55.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.687+0000 7f292abc7b80 1 -- 172.21.15.19:0/1314275019 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f290000f800 con 0x55c6a519dc80 2024-05-07T02:32:55.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.687+0000 7f28fb400700 1 -- 172.21.15.19:0/1314275019 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f29080b2300 con 0x55c6a519dc80 2024-05-07T02:32:55.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.687+0000 7f292abc7b80 10 client.25456 fetch_fsmap learned FSMap version 31 2024-05-07T02:32:55.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.687+0000 7f292abc7b80 1 -- 172.21.15.19:0/1314275019 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c6a5222540 con 0x55c6a519dc80 2024-05-07T02:32:55.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.687+0000 7f28fb400700 1 -- 172.21.15.19:0/1314275019 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f290000f800 con 0x55c6a519dc80 2024-05-07T02:32:55.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.687+0000 7f28fb400700 1 -- 172.21.15.19:0/1314275019 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f29080b2530 con 0x55c6a519dc80 2024-05-07T02:32:55.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.687+0000 7f292abc7b80 10 client.25456 fetch_fsmap finished waiting for FSMap version 31 2024-05-07T02:32:55.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.687+0000 7f292abc7b80 10 client.25456 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-07T02:32:55.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.688+0000 7f292abc7b80 20 client.25456 populate_metadata read hostname 'smithi019' 2024-05-07T02:32:55.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.688+0000 7f292abc7b80 1 --2- 172.21.15.19:0/1314275019 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x55c6a5222ba0 0x55c6a5224fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:55.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.688+0000 7f292abc7b80 4 client.25456 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:32:55.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.688+0000 7f292abc7b80 1 -- 172.21.15.19:0/1314275019 --> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c6a50c5400 con 0x55c6a5222ba0 2024-05-07T02:32:55.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.688+0000 7f2918c00700 1 --2- 172.21.15.19:0/1314275019 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x55c6a5222ba0 0x55c6a5224fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:55.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.689+0000 7f2918c00700 1 --2- 172.21.15.19:0/1314275019 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x55c6a5222ba0 0x55c6a5224fc0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=955ec1b52a8643ba server_cookie=eba1c470644c1ddb in_seq=0 out_seq=0 2024-05-07T02:32:55.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.689+0000 7f28fb400700 10 client.25456 ms_handle_connect on v2:172.21.15.64:6836/2585907777 2024-05-07T02:32:55.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.689+0000 7f28fb400700 1 -- 172.21.15.19:0/1314275019 <== mds.4 v2:172.21.15.64:6836/2585907777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c6a50c5400 con 0x55c6a5222ba0 2024-05-07T02:32:55.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.689+0000 7f28fb400700 10 client.25456 handle_command_reply: tid=0 2024-05-07T02:32:55.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.689+0000 7f292abc7b80 1 client.25456 shutdown 2024-05-07T02:32:55.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.689+0000 7f292abc7b80 1 -- 172.21.15.19:0/1314275019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6a519dc80 msgr2=0x55c6a51bd9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:55.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.689+0000 7f292abc7b80 1 --2- 172.21.15.19:0/1314275019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6a519dc80 0x55c6a51bd9f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f29080e0080 tx=0x7f29080a6020 comp rx=0 tx=0).stop 2024-05-07T02:32:55.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.689+0000 7f292abc7b80 1 -- 172.21.15.19:0/1314275019 shutdown_connections 2024-05-07T02:32:55.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.689+0000 7f292abc7b80 1 --2- 172.21.15.19:0/1314275019 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x55c6a5222ba0 0x55c6a5224fc0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:55.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.689+0000 7f292abc7b80 1 --2- 172.21.15.19:0/1314275019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6a51a3ae0 0x55c6a51bb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:55.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.689+0000 7f292abc7b80 1 --2- 172.21.15.19:0/1314275019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6a519dc80 0x55c6a51bd9f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:55.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.689+0000 7f292abc7b80 1 --2- 172.21.15.19:0/1314275019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6a519e5d0 0x55c6a51bad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:55.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.689+0000 7f292abc7b80 1 -- 172.21.15.19:0/1314275019 >> 172.21.15.19:0/1314275019 conn(0x55c6a5017e40 msgr2=0x55c6a518a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:55.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.689+0000 7f28fb400700 0 client.25456 ms_handle_reset on v2:172.21.15.64:6836/2585907777 2024-05-07T02:32:55.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.690+0000 7f292abc7b80 1 -- 172.21.15.19:0/1314275019 shutdown_connections 2024-05-07T02:32:55.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.690+0000 7f292abc7b80 1 -- 172.21.15.19:0/1314275019 wait complete. 2024-05-07T02:32:55.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.690+0000 7f292abc7b80 20 client.25456 trim_cache size 0 max 16384 2024-05-07T02:32:55.691 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:32:55.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.691+0000 7f291c000700 1 -- 172.21.15.19:0/3254278702 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f28fc0b2a80 msgr2=0x7f28fc0b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:55.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.691+0000 7f291c000700 1 --2- 172.21.15.19:0/3254278702 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f28fc0b2a80 0x7f28fc0b4f90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f290c010000 tx=0x7f290c01d000 comp rx=0 tx=0).stop 2024-05-07T02:32:55.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.691+0000 7f291c000700 1 -- 172.21.15.19:0/3254278702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2914106570 msgr2=0x7f291419d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:55.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.691+0000 7f291c000700 1 --2- 172.21.15.19:0/3254278702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2914106570 0x7f291419d020 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f290800ea30 tx=0x7f290800ed40 comp rx=0 tx=0).stop 2024-05-07T02:32:55.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.691+0000 7f291c000700 1 -- 172.21.15.19:0/3254278702 shutdown_connections 2024-05-07T02:32:55.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.691+0000 7f291c000700 1 --2- 172.21.15.19:0/3254278702 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f291410c5b0 0x7f29141a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:55.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.691+0000 7f291c000700 1 --2- 172.21.15.19:0/3254278702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2914106570 0x7f291419d020 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:55.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.691+0000 7f291c000700 1 --2- 172.21.15.19:0/3254278702 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f28fc0b2a80 0x7f28fc0b4f90 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:55.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.691+0000 7f291c000700 1 --2- 172.21.15.19:0/3254278702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2914106ec0 0x7f291419d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:55.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.691+0000 7f291c000700 1 -- 172.21.15.19:0/3254278702 >> 172.21.15.19:0/3254278702 conn(0x7f2914102050 msgr2=0x7f2914103850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:55.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.691+0000 7f291c000700 1 -- 172.21.15.19:0/3254278702 shutdown_connections 2024-05-07T02:32:55.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:55.691+0000 7f291c000700 1 -- 172.21.15.19:0/3254278702 wait complete. 2024-05-07T02:32:56.610 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-07T02:32:57.098 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:32:57.249 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:56 smithi019 ceph-mon[29781]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-07T02:32:57.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:56 smithi064 ceph-mon[34699]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-07T02:32:57.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:56 smithi027 ceph-mon[34398]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 682 B/s wr, 11 op/s 2024-05-07T02:32:59.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:32:58 smithi019 ceph-mon[29781]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-07T02:32:59.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:32:58 smithi064 ceph-mon[34699]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-07T02:32:59.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:32:58 smithi027 ceph-mon[34398]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-07T02:32:59.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.944+0000 7fce94c00700 1 -- 172.21.15.19:0/543008523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce90109870 msgr2=0x7fce901111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:59.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.944+0000 7fce94c00700 1 --2- 172.21.15.19:0/543008523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce90109870 0x7fce901111d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fce8800b0a0 tx=0x7fce8800b3b0 comp rx=0 tx=0).stop 2024-05-07T02:32:59.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.944+0000 7fce94c00700 1 -- 172.21.15.19:0/543008523 shutdown_connections 2024-05-07T02:32:59.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.944+0000 7fce94c00700 1 --2- 172.21.15.19:0/543008523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce90109870 0x7fce901111d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:59.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.944+0000 7fce94c00700 1 --2- 172.21.15.19:0/543008523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce901045f0 0x7fce90104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:59.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.944+0000 7fce94c00700 1 --2- 172.21.15.19:0/543008523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce90108f20 0x7fce90109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:59.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.944+0000 7fce94c00700 1 -- 172.21.15.19:0/543008523 >> 172.21.15.19:0/543008523 conn(0x7fce90100040 msgr2=0x7fce90102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:59.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.944+0000 7fce94c00700 1 -- 172.21.15.19:0/543008523 shutdown_connections 2024-05-07T02:32:59.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.945+0000 7fce94c00700 1 -- 172.21.15.19:0/543008523 wait complete. 2024-05-07T02:32:59.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.946+0000 7fce94c00700 1 Processor -- start 2024-05-07T02:32:59.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.946+0000 7fce94c00700 1 -- start start 2024-05-07T02:32:59.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.947+0000 7fce94c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce901045f0 0x7fce90075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:59.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.947+0000 7fce94c00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce90108f20 0x7fce90076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:59.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.947+0000 7fce94c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce90109870 0x7fce90076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:59.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.947+0000 7fce94c00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fce901144f0 con 0x7fce901045f0 2024-05-07T02:32:59.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.947+0000 7fce94c00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fce90114370 con 0x7fce90109870 2024-05-07T02:32:59.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.947+0000 7fce94c00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fce90114670 con 0x7fce90108f20 2024-05-07T02:32:59.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.947+0000 7fce85400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce90108f20 0x7fce90076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:59.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.947+0000 7fce8e800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce90109870 0x7fce90076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:59.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.947+0000 7fce8de00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce901045f0 0x7fce90075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:59.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.947+0000 7fce85400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce90108f20 0x7fce90076000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:46768/0 (socket says 172.21.15.19:46768) 2024-05-07T02:32:59.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.947+0000 7fce8e800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce90109870 0x7fce90076540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:55440/0 (socket says 172.21.15.19:55440) 2024-05-07T02:32:59.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.947+0000 7fce85400700 1 -- 172.21.15.19:0/901966528 learned_addr learned my addr 172.21.15.19:0/901966528 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:32:59.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.948+0000 7fce8e800700 1 -- 172.21.15.19:0/901966528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce90108f20 msgr2=0x7fce90076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:59.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.948+0000 7fce8e800700 1 --2- 172.21.15.19:0/901966528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce90108f20 0x7fce90076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:59.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.948+0000 7fce8e800700 1 -- 172.21.15.19:0/901966528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce901045f0 msgr2=0x7fce90075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:59.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.948+0000 7fce8e800700 1 --2- 172.21.15.19:0/901966528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce901045f0 0x7fce90075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:59.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.948+0000 7fce8e800700 1 -- 172.21.15.19:0/901966528 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce901a91c0 con 0x7fce90109870 2024-05-07T02:32:59.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.948+0000 7fce85400700 1 --2- 172.21.15.19:0/901966528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce90108f20 0x7fce90076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:32:59.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.948+0000 7fce8e800700 1 --2- 172.21.15.19:0/901966528 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce90109870 0x7fce90076540 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fce8800b070 tx=0x7fce88004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:59.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.949+0000 7fce87400700 1 -- 172.21.15.19:0/901966528 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce88021070 con 0x7fce90109870 2024-05-07T02:32:59.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.949+0000 7fce87400700 1 -- 172.21.15.19:0/901966528 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce88007b80 con 0x7fce90109870 2024-05-07T02:32:59.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.949+0000 7fce94c00700 1 -- 172.21.15.19:0/901966528 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce901a9450 con 0x7fce90109870 2024-05-07T02:32:59.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.949+0000 7fce87400700 1 -- 172.21.15.19:0/901966528 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce88005040 con 0x7fce90109870 2024-05-07T02:32:59.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.949+0000 7fce94c00700 1 -- 172.21.15.19:0/901966528 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce901a9950 con 0x7fce90109870 2024-05-07T02:32:59.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.954+0000 7fcea37a8b80 1 Processor -- start 2024-05-07T02:32:59.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.955+0000 7fce87400700 1 -- 172.21.15.19:0/901966528 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7fce88024070 con 0x7fce90109870 2024-05-07T02:32:59.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.955+0000 7fcea37a8b80 1 -- start start 2024-05-07T02:32:59.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.955+0000 7fcea37a8b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec13981470 0x55ec13981880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:59.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.955+0000 7fcea37a8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec1397b620 0x55ec1397ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:59.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.955+0000 7fcea37a8b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec1397bf70 0x55ec13988150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:59.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.955+0000 7fcea37a8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ec1389da20 con 0x55ec1397b620 2024-05-07T02:32:59.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.955+0000 7fcea37a8b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ec1383b760 con 0x55ec1397bf70 2024-05-07T02:32:59.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.955+0000 7fcea37a8b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ec138999e0 con 0x55ec13981470 2024-05-07T02:32:59.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.956+0000 7fce85400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec13981470 0x55ec13981880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:59.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.956+0000 7fce8de00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec1397b620 0x55ec1397ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:59.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.956+0000 7fce85400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec13981470 0x55ec13981880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:46776/0 (socket says 172.21.15.19:46776) 2024-05-07T02:32:59.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.956+0000 7fce85400700 1 -- 172.21.15.19:0/239133283 learned_addr learned my addr 172.21.15.19:0/239133283 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:32:59.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.956+0000 7fce85400700 1 --2- 172.21.15.19:0/239133283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec1397bf70 0x55ec13988150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:59.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.956+0000 7fce8de00700 1 -- 172.21.15.19:0/239133283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec13981470 msgr2=0x55ec13981880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:32:59.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.956+0000 7fce8de00700 1 --2- 172.21.15.19:0/239133283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec13981470 0x55ec13981880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:59.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.956+0000 7fce8de00700 1 -- 172.21.15.19:0/239133283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec1397bf70 msgr2=0x55ec13988150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:32:59.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.956+0000 7fce8de00700 1 --2- 172.21.15.19:0/239133283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec1397bf70 0x55ec13988150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:59.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.956+0000 7fce8de00700 1 -- 172.21.15.19:0/239133283 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec137ea8d0 con 0x55ec1397b620 2024-05-07T02:32:59.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.957+0000 7fce87400700 1 --2- 172.21.15.19:0/901966528 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fce740b2b00 0x7fce740b5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:59.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.957+0000 7fce87400700 1 -- 172.21.15.19:0/901966528 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fce8800f020 con 0x7fce90109870 2024-05-07T02:32:59.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.957+0000 7fce8de00700 1 --2- 172.21.15.19:0/239133283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec1397b620 0x55ec1397ba30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fce80009cb0 tx=0x7fce80011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=357e0fa4f872aedd server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:59.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.957+0000 7fce85400700 1 --2- 172.21.15.19:0/901966528 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fce740b2b00 0x7fce740b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:59.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.957+0000 7fce7f400700 1 -- 172.21.15.19:0/239133283 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce80011d20 con 0x55ec1397b620 2024-05-07T02:32:59.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.958+0000 7fce7f400700 1 -- 172.21.15.19:0/239133283 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce80004500 con 0x55ec1397b620 2024-05-07T02:32:59.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.958+0000 7fce7f400700 1 -- 172.21.15.19:0/239133283 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce80005280 con 0x55ec1397b620 2024-05-07T02:32:59.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.959+0000 7fcea37a8b80 1 -- 172.21.15.19:0/239133283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec1397b620 msgr2=0x55ec1397ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:59.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.959+0000 7fcea37a8b80 1 --2- 172.21.15.19:0/239133283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec1397b620 0x55ec1397ba30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fce80009cb0 tx=0x7fce80011440 comp rx=0 tx=0).stop 2024-05-07T02:32:59.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.959+0000 7fce85400700 1 --2- 172.21.15.19:0/901966528 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fce740b2b00 0x7fce740b5010 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x55ec13980ff0 tx=0x7fce78009260 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:59.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.959+0000 7fcea37a8b80 1 -- 172.21.15.19:0/239133283 shutdown_connections 2024-05-07T02:32:59.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.959+0000 7fcea37a8b80 1 --2- 172.21.15.19:0/239133283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec1397bf70 0x55ec13988150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:59.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.959+0000 7fcea37a8b80 1 --2- 172.21.15.19:0/239133283 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec13981470 0x55ec13981880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:59.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.959+0000 7fcea37a8b80 1 --2- 172.21.15.19:0/239133283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec1397b620 0x55ec1397ba30 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:59.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.959+0000 7fcea37a8b80 1 -- 172.21.15.19:0/239133283 >> 172.21.15.19:0/239133283 conn(0x55ec137fde40 msgr2=0x55ec13976380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:59.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.960+0000 7fcea37a8b80 1 -- 172.21.15.19:0/239133283 shutdown_connections 2024-05-07T02:32:59.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.960+0000 7fcea37a8b80 1 -- 172.21.15.19:0/239133283 wait complete. 2024-05-07T02:32:59.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.961+0000 7fcea37a8b80 1 -- start start 2024-05-07T02:32:59.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.961+0000 7fcea37a8b80 1 Processor -- start 2024-05-07T02:32:59.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.962+0000 7fcea37a8b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec1397b620 0x55ec13985650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:59.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.962+0000 7fcea37a8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec1397bf70 0x55ec13985b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:59.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.962+0000 7fcea37a8b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec13981470 0x55ec139860d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:59.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.962+0000 7fcea37a8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ec13988840 con 0x55ec1397bf70 2024-05-07T02:32:59.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.962+0000 7fcea37a8b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ec13803b20 con 0x55ec1397b620 2024-05-07T02:32:59.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.962+0000 7fcea37a8b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ec139889c0 con 0x55ec13981470 2024-05-07T02:32:59.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.962+0000 7fcea37a8b80 10 client.0 _ll_register_callbacks cb 0x55ec13831380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:32:59.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.962+0000 7fce85400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec1397bf70 0x55ec13985b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:59.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.962+0000 7fce8de00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec1397b620 0x55ec13985650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:59.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.962+0000 7fce8de00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec1397b620 0x55ec13985650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:55472/0 (socket says 172.21.15.19:55472) 2024-05-07T02:32:59.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.962+0000 7fce8de00700 1 -- 172.21.15.19:0/3276849326 learned_addr learned my addr 172.21.15.19:0/3276849326 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:32:59.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.963+0000 7fce8e800700 1 --2- 172.21.15.19:0/3276849326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec13981470 0x55ec139860d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:59.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.963+0000 7fce8e800700 1 -- 172.21.15.19:0/3276849326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec1397b620 msgr2=0x55ec13985650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:32:59.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.963+0000 7fce8e800700 1 --2- 172.21.15.19:0/3276849326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec1397b620 0x55ec13985650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:59.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.963+0000 7fce8e800700 1 -- 172.21.15.19:0/3276849326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec1397bf70 msgr2=0x55ec13985b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:32:59.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.963+0000 7fce8e800700 1 --2- 172.21.15.19:0/3276849326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec1397bf70 0x55ec13985b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:59.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.963+0000 7fce8e800700 1 -- 172.21.15.19:0/3276849326 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec13863420 con 0x55ec13981470 2024-05-07T02:32:59.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.963+0000 7fce8de00700 1 --2- 172.21.15.19:0/3276849326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec1397b620 0x55ec13985650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:32:59.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.963+0000 7fce85400700 1 --2- 172.21.15.19:0/3276849326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec1397bf70 0x55ec13985b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:32:59.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.963+0000 7fce8e800700 1 --2- 172.21.15.19:0/3276849326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec13981470 0x55ec139860d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fce880e1140 tx=0x7fce880aa4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7d7e6e71f93bd8a server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:32:59.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.963+0000 7fce73e00700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:32:59.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.964+0000 7fce73e00700 1 -- 172.21.15.19:0/3276849326 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce880ab3b0 con 0x55ec13981470 2024-05-07T02:32:59.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.964+0000 7fce73e00700 1 -- 172.21.15.19:0/3276849326 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fce880a72a0 con 0x55ec13981470 2024-05-07T02:32:59.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.964+0000 7fcea37a8b80 1 -- 172.21.15.19:0/3276849326 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce80004500 con 0x55ec13981470 2024-05-07T02:32:59.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.964+0000 7fce73e00700 1 -- 172.21.15.19:0/3276849326 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fce880aacb0 con 0x55ec13981470 2024-05-07T02:32:59.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.964+0000 7fcea37a8b80 10 client.25468 fetch_fsmap learned FSMap version 31 2024-05-07T02:32:59.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.964+0000 7fcea37a8b80 1 -- 172.21.15.19:0/3276849326 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ec13988840 con 0x55ec13981470 2024-05-07T02:32:59.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.964+0000 7fce73e00700 1 -- 172.21.15.19:0/3276849326 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce80004500 con 0x55ec13981470 2024-05-07T02:32:59.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.965+0000 7fce73e00700 1 -- 172.21.15.19:0/3276849326 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fce88011070 con 0x55ec13981470 2024-05-07T02:32:59.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.965+0000 7fcea37a8b80 10 client.25468 fetch_fsmap finished waiting for FSMap version 31 2024-05-07T02:32:59.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.965+0000 7fcea37a8b80 10 client.25468 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-05-07T02:32:59.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.965+0000 7fcea37a8b80 20 client.25468 populate_metadata read hostname 'smithi019' 2024-05-07T02:32:59.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.965+0000 7fcea37a8b80 1 --2- 172.21.15.19:0/3276849326 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x55ec1399e150 0x55ec139a0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:32:59.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.965+0000 7fcea37a8b80 4 client.25468 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:32:59.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.965+0000 7fcea37a8b80 1 -- 172.21.15.19:0/3276849326 --> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec139480e0 con 0x55ec1399e150 2024-05-07T02:32:59.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.965+0000 7fce8de00700 1 --2- 172.21.15.19:0/3276849326 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x55ec1399e150 0x55ec139a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:32:59.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.966+0000 7fce8de00700 1 --2- 172.21.15.19:0/3276849326 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x55ec1399e150 0x55ec139a0600 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=296ec6eba9d0b05f server_cookie=97482eb42ac64466 in_seq=0 out_seq=0 2024-05-07T02:32:59.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.966+0000 7fce73e00700 10 client.25468 ms_handle_connect on v2:172.21.15.27:6836/2583138143 2024-05-07T02:32:59.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.967+0000 7fce73e00700 1 -- 172.21.15.19:0/3276849326 <== mds.3 v2:172.21.15.27:6836/2583138143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ec139480e0 con 0x55ec1399e150 2024-05-07T02:32:59.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.967+0000 7fce73e00700 10 client.25468 handle_command_reply: tid=0 2024-05-07T02:32:59.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.967+0000 7fcea37a8b80 1 client.25468 shutdown 2024-05-07T02:32:59.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.967+0000 7fcea37a8b80 1 -- 172.21.15.19:0/3276849326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec13981470 msgr2=0x55ec139860d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:32:59.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.967+0000 7fcea37a8b80 1 --2- 172.21.15.19:0/3276849326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec13981470 0x55ec139860d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fce880e1140 tx=0x7fce880aa4a0 comp rx=0 tx=0).stop 2024-05-07T02:32:59.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.967+0000 7fcea37a8b80 1 -- 172.21.15.19:0/3276849326 shutdown_connections 2024-05-07T02:32:59.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.967+0000 7fcea37a8b80 1 --2- 172.21.15.19:0/3276849326 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x55ec1399e150 0x55ec139a0600 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:59.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.967+0000 7fcea37a8b80 1 --2- 172.21.15.19:0/3276849326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec13981470 0x55ec139860d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:59.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.967+0000 7fcea37a8b80 1 --2- 172.21.15.19:0/3276849326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec1397b620 0x55ec13985650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:59.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.967+0000 7fcea37a8b80 1 --2- 172.21.15.19:0/3276849326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec1397bf70 0x55ec13985b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:32:59.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.967+0000 7fce73e00700 0 client.25468 ms_handle_reset on v2:172.21.15.27:6836/2583138143 2024-05-07T02:32:59.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.967+0000 7fcea37a8b80 1 -- 172.21.15.19:0/3276849326 >> 172.21.15.19:0/3276849326 conn(0x55ec137fde40 msgr2=0x55ec1397a610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:32:59.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.967+0000 7fcea37a8b80 1 -- 172.21.15.19:0/3276849326 shutdown_connections 2024-05-07T02:32:59.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.968+0000 7fcea37a8b80 1 -- 172.21.15.19:0/3276849326 wait complete. 2024-05-07T02:32:59.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:32:59.968+0000 7fcea37a8b80 20 client.25468 trim_cache size 0 max 16384 2024-05-07T02:33:00.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.002+0000 7fcea37a8b80 1 Processor -- start 2024-05-07T02:33:00.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.002+0000 7fcea37a8b80 1 -- start start 2024-05-07T02:33:00.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.003+0000 7fcea37a8b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec1397b620 0x55ec1397b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:00.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.003+0000 7fcea37a8b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec1397bf70 0x55ec1397fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:00.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.003+0000 7fcea37a8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec13981470 0x55ec1399a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:00.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.003+0000 7fcea37a8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fce901ae310 con 0x55ec13981470 2024-05-07T02:33:00.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.003+0000 7fcea37a8b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fce901afe50 con 0x55ec1397bf70 2024-05-07T02:33:00.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.003+0000 7fcea37a8b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fce901ae9e0 con 0x55ec1397b620 2024-05-07T02:33:00.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.003+0000 7fce85400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec1397bf70 0x55ec1397fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:00.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.003+0000 7fce8de00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec1397b620 0x55ec1397b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:00.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.004+0000 7fce8e800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec13981470 0x55ec1399a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:00.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.004+0000 7fce85400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec1397bf70 0x55ec1397fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:55484/0 (socket says 172.21.15.19:55484) 2024-05-07T02:33:00.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.004+0000 7fce8de00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec1397b620 0x55ec1397b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:46792/0 (socket says 172.21.15.19:46792) 2024-05-07T02:33:00.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.004+0000 7fce85400700 1 -- 172.21.15.19:0/2626749468 learned_addr learned my addr 172.21.15.19:0/2626749468 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:00.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.004+0000 7fce85400700 1 -- 172.21.15.19:0/2626749468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec1397b620 msgr2=0x55ec1397b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:00.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.004+0000 7fce85400700 1 --2- 172.21.15.19:0/2626749468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec1397b620 0x55ec1397b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:00.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.004+0000 7fce85400700 1 -- 172.21.15.19:0/2626749468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec13981470 msgr2=0x55ec1399a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:00.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.004+0000 7fce85400700 1 --2- 172.21.15.19:0/2626749468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec13981470 0x55ec1399a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:00.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.004+0000 7fce85400700 1 -- 172.21.15.19:0/2626749468 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec139802d0 con 0x55ec1397bf70 2024-05-07T02:33:00.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.004+0000 7fce8de00700 1 --2- 172.21.15.19:0/2626749468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec1397b620 0x55ec1397b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:33:00.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.004+0000 7fce85400700 1 --2- 172.21.15.19:0/2626749468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec1397bf70 0x55ec1397fd90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fce78009690 tx=0x7fce780047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae0025315f15d47b server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:00.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.005+0000 7fce7ea00700 1 -- 172.21.15.19:0/2626749468 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce78010840 con 0x55ec1397bf70 2024-05-07T02:33:00.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.005+0000 7fce7ea00700 1 -- 172.21.15.19:0/2626749468 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce78004830 con 0x55ec1397bf70 2024-05-07T02:33:00.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.005+0000 7fce7ea00700 1 -- 172.21.15.19:0/2626749468 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce78005110 con 0x55ec1397bf70 2024-05-07T02:33:00.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.006+0000 7fcea37a8b80 1 -- 172.21.15.19:0/2626749468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec1397bf70 msgr2=0x55ec1397fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:00.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.006+0000 7fcea37a8b80 1 --2- 172.21.15.19:0/2626749468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec1397bf70 0x55ec1397fd90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fce78009690 tx=0x7fce780047c0 comp rx=0 tx=0).stop 2024-05-07T02:33:00.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.006+0000 7fcea37a8b80 1 -- 172.21.15.19:0/2626749468 shutdown_connections 2024-05-07T02:33:00.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.006+0000 7fcea37a8b80 1 --2- 172.21.15.19:0/2626749468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec13981470 0x55ec1399a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:00.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.006+0000 7fcea37a8b80 1 --2- 172.21.15.19:0/2626749468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec1397b620 0x55ec1397b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:00.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.006+0000 7fcea37a8b80 1 --2- 172.21.15.19:0/2626749468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec1397bf70 0x55ec1397fd90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:00.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.006+0000 7fcea37a8b80 1 -- 172.21.15.19:0/2626749468 >> 172.21.15.19:0/2626749468 conn(0x55ec137fde40 msgr2=0x55ec13970830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:00.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.006+0000 7fcea37a8b80 1 -- 172.21.15.19:0/2626749468 shutdown_connections 2024-05-07T02:33:00.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.006+0000 7fcea37a8b80 1 -- 172.21.15.19:0/2626749468 wait complete. 2024-05-07T02:33:00.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.007+0000 7fcea37a8b80 1 -- start start 2024-05-07T02:33:00.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.007+0000 7fcea37a8b80 1 Processor -- start 2024-05-07T02:33:00.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.008+0000 7fcea37a8b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec1397b620 0x55ec139968b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:00.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.008+0000 7fcea37a8b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec1397bf70 0x55ec13991c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:00.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.008+0000 7fcea37a8b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec13981470 0x55ec13992180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:00.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.008+0000 7fcea37a8b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fce901af780 con 0x55ec13981470 2024-05-07T02:33:00.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.008+0000 7fcea37a8b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fce901af0b0 con 0x55ec1397bf70 2024-05-07T02:33:00.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.008+0000 7fcea37a8b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fce901ac2b0 con 0x55ec1397b620 2024-05-07T02:33:00.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.008+0000 7fcea37a8b80 10 client.0 _ll_register_callbacks cb 0x55ec13831380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:33:00.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.009+0000 7fce8de00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec1397b620 0x55ec139968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:00.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.009+0000 7fce8de00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec1397b620 0x55ec139968b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:46798/0 (socket says 172.21.15.19:46798) 2024-05-07T02:33:00.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.009+0000 7fce8de00700 1 -- 172.21.15.19:0/1775874826 learned_addr learned my addr 172.21.15.19:0/1775874826 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:00.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.009+0000 7fce85400700 1 --2- 172.21.15.19:0/1775874826 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec1397bf70 0x55ec13991c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:00.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.009+0000 7fce8e800700 1 --2- 172.21.15.19:0/1775874826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec13981470 0x55ec13992180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:00.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.009+0000 7fce8de00700 1 -- 172.21.15.19:0/1775874826 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec1397bf70 msgr2=0x55ec13991c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:00.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.009+0000 7fce8de00700 1 --2- 172.21.15.19:0/1775874826 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec1397bf70 0x55ec13991c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:00.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.009+0000 7fce8de00700 1 -- 172.21.15.19:0/1775874826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec13981470 msgr2=0x55ec13992180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:00.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.009+0000 7fce8de00700 1 --2- 172.21.15.19:0/1775874826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec13981470 0x55ec13992180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:00.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.009+0000 7fce8de00700 1 -- 172.21.15.19:0/1775874826 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec13992770 con 0x55ec1397b620 2024-05-07T02:33:00.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.009+0000 7fce8e800700 1 --2- 172.21.15.19:0/1775874826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec13981470 0x55ec13992180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:33:00.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.010+0000 7fce8de00700 1 --2- 172.21.15.19:0/1775874826 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec1397b620 0x55ec139968b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fce80000c00 tx=0x7fce80009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85e9c78b56b22415 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:00.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.010+0000 7fce73e00700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:33:00.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.010+0000 7fce73e00700 1 -- 172.21.15.19:0/1775874826 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce80011b20 con 0x55ec1397b620 2024-05-07T02:33:00.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.010+0000 7fce73e00700 1 -- 172.21.15.19:0/1775874826 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fce80011cc0 con 0x55ec1397b620 2024-05-07T02:33:00.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.010+0000 7fcea37a8b80 1 -- 172.21.15.19:0/1775874826 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce78004830 con 0x55ec1397b620 2024-05-07T02:33:00.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.010+0000 7fce73e00700 1 -- 172.21.15.19:0/1775874826 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fce8003b050 con 0x55ec1397b620 2024-05-07T02:33:00.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.010+0000 7fcea37a8b80 10 client.25474 fetch_fsmap learned FSMap version 31 2024-05-07T02:33:00.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.010+0000 7fcea37a8b80 1 -- 172.21.15.19:0/1775874826 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ec13a086e0 con 0x55ec1397b620 2024-05-07T02:33:00.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.011+0000 7fce73e00700 1 -- 172.21.15.19:0/1775874826 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce78004830 con 0x55ec1397b620 2024-05-07T02:33:00.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.011+0000 7fce73e00700 1 -- 172.21.15.19:0/1775874826 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fce80018070 con 0x55ec1397b620 2024-05-07T02:33:00.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.011+0000 7fcea37a8b80 10 client.25474 fetch_fsmap finished waiting for FSMap version 31 2024-05-07T02:33:00.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.011+0000 7fcea37a8b80 10 client.25474 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-05-07T02:33:00.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.011+0000 7fcea37a8b80 20 client.25474 populate_metadata read hostname 'smithi019' 2024-05-07T02:33:00.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.011+0000 7fcea37a8b80 1 --2- 172.21.15.19:0/1775874826 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x55ec13a08c60 0x55ec13a0b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:00.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.011+0000 7fcea37a8b80 4 client.25474 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:33:00.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.011+0000 7fcea37a8b80 1 -- 172.21.15.19:0/1775874826 --> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ec138ab400 con 0x55ec13a08c60 2024-05-07T02:33:00.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.012+0000 7fce85400700 1 --2- 172.21.15.19:0/1775874826 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x55ec13a08c60 0x55ec13a0b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:00.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.012+0000 7fce85400700 1 --2- 172.21.15.19:0/1775874826 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x55ec13a08c60 0x55ec13a0b080 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=83d7c76a876d7711 server_cookie=d36a7278955c29ae in_seq=0 out_seq=0 2024-05-07T02:33:00.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.012+0000 7fce73e00700 10 client.25474 ms_handle_connect on v2:172.21.15.27:6836/2583138143 2024-05-07T02:33:00.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.012+0000 7fce73e00700 1 -- 172.21.15.19:0/1775874826 <== mds.3 v2:172.21.15.27:6836/2583138143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ec138ab400 con 0x55ec13a08c60 2024-05-07T02:33:00.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.012+0000 7fce73e00700 10 client.25474 handle_command_reply: tid=0 2024-05-07T02:33:00.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.012+0000 7fcea37a8b80 1 client.25474 shutdown 2024-05-07T02:33:00.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.013+0000 7fcea37a8b80 1 -- 172.21.15.19:0/1775874826 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec1397b620 msgr2=0x55ec139968b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:00.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.013+0000 7fcea37a8b80 1 --2- 172.21.15.19:0/1775874826 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec1397b620 0x55ec139968b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fce80000c00 tx=0x7fce80009780 comp rx=0 tx=0).stop 2024-05-07T02:33:00.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.013+0000 7fcea37a8b80 1 -- 172.21.15.19:0/1775874826 shutdown_connections 2024-05-07T02:33:00.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.013+0000 7fcea37a8b80 1 --2- 172.21.15.19:0/1775874826 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x55ec13a08c60 0x55ec13a0b080 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:00.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.013+0000 7fcea37a8b80 1 --2- 172.21.15.19:0/1775874826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec13981470 0x55ec13992180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:00.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.013+0000 7fcea37a8b80 1 --2- 172.21.15.19:0/1775874826 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ec1397b620 0x55ec139968b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:00.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.013+0000 7fcea37a8b80 1 --2- 172.21.15.19:0/1775874826 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ec1397bf70 0x55ec13991c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:00.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.013+0000 7fce73e00700 0 client.25474 ms_handle_reset on v2:172.21.15.27:6836/2583138143 2024-05-07T02:33:00.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.013+0000 7fcea37a8b80 1 -- 172.21.15.19:0/1775874826 >> 172.21.15.19:0/1775874826 conn(0x55ec137fde40 msgr2=0x55ec13986c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:00.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.013+0000 7fcea37a8b80 1 -- 172.21.15.19:0/1775874826 shutdown_connections 2024-05-07T02:33:00.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.013+0000 7fcea37a8b80 1 -- 172.21.15.19:0/1775874826 wait complete. 2024-05-07T02:33:00.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.013+0000 7fcea37a8b80 20 client.25474 trim_cache size 0 max 16384 2024-05-07T02:33:00.015 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:33:00.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.014+0000 7fce94c00700 1 -- 172.21.15.19:0/901966528 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fce740b2b00 msgr2=0x7fce740b5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:00.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.014+0000 7fce94c00700 1 --2- 172.21.15.19:0/901966528 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fce740b2b00 0x7fce740b5010 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x55ec13980ff0 tx=0x7fce78009260 comp rx=0 tx=0).stop 2024-05-07T02:33:00.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.014+0000 7fce94c00700 1 -- 172.21.15.19:0/901966528 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce90109870 msgr2=0x7fce90076540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:00.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.014+0000 7fce94c00700 1 --2- 172.21.15.19:0/901966528 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce90109870 0x7fce90076540 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fce8800b070 tx=0x7fce88004c80 comp rx=0 tx=0).stop 2024-05-07T02:33:00.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.014+0000 7fce94c00700 1 -- 172.21.15.19:0/901966528 shutdown_connections 2024-05-07T02:33:00.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.014+0000 7fce94c00700 1 --2- 172.21.15.19:0/901966528 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce90109870 0x7fce90076540 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:00.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.014+0000 7fce94c00700 1 --2- 172.21.15.19:0/901966528 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fce740b2b00 0x7fce740b5010 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:00.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.014+0000 7fce94c00700 1 --2- 172.21.15.19:0/901966528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce901045f0 0x7fce90075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:00.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.014+0000 7fce94c00700 1 --2- 172.21.15.19:0/901966528 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce90108f20 0x7fce90076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:00.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.014+0000 7fce94c00700 1 -- 172.21.15.19:0/901966528 >> 172.21.15.19:0/901966528 conn(0x7fce90100040 msgr2=0x7fce9010a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:00.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.015+0000 7fce94c00700 1 -- 172.21.15.19:0/901966528 shutdown_connections 2024-05-07T02:33:00.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:00.015+0000 7fce94c00700 1 -- 172.21.15.19:0/901966528 wait complete. 2024-05-07T02:33:00.895 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:33:00 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:33:00.460+0000 7fa70f800700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-07T02:33:00.895 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:33:00 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:33:00.460+0000 7fa70f800700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-07T02:33:00.895 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:33:00 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:33:00.460+0000 7fa70f800700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-07T02:33:00.895 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:33:00 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:33:00.460+0000 7fa70f800700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-07T02:33:00.895 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:33:00 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:33:00.460+0000 7fa70f800700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-07T02:33:01.122 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-07T02:33:01.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:00 smithi064 ceph-mon[34699]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:33:01.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:00 smithi019 ceph-mon[29781]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:33:01.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:00 smithi027 ceph-mon[34398]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:33:01.564 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:33:02.101 INFO:tasks.workunit.client.0.smithi019.stderr:Note: switching to '366c9130c497dee9e73178650cf710ed21269b18'. 2024-05-07T02:33:02.101 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-05-07T02:33:02.102 INFO:tasks.workunit.client.0.smithi019.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-07T02:33:02.102 INFO:tasks.workunit.client.0.smithi019.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-07T02:33:02.102 INFO:tasks.workunit.client.0.smithi019.stderr:state without impacting any branches by switching back to a branch. 2024-05-07T02:33:02.102 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-05-07T02:33:02.102 INFO:tasks.workunit.client.0.smithi019.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-07T02:33:02.102 INFO:tasks.workunit.client.0.smithi019.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-07T02:33:02.102 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-05-07T02:33:02.102 INFO:tasks.workunit.client.0.smithi019.stderr: git switch -c 2024-05-07T02:33:02.102 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-05-07T02:33:02.102 INFO:tasks.workunit.client.0.smithi019.stderr:Or undo this operation with: 2024-05-07T02:33:02.102 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-05-07T02:33:02.102 INFO:tasks.workunit.client.0.smithi019.stderr: git switch - 2024-05-07T02:33:02.102 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-05-07T02:33:02.102 INFO:tasks.workunit.client.0.smithi019.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-07T02:33:02.103 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-05-07T02:33:02.103 INFO:tasks.workunit.client.0.smithi019.stderr:HEAD is now at 366c9130c49 SQUASH:noclientremote 2024-05-07T02:33:02.116 DEBUG:teuthology.orchestra.run.smithi019:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-07T02:33:02.200 INFO:tasks.workunit.client.0.smithi019.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-07T02:33:02.208 INFO:tasks.workunit.client.0.smithi019.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-07T02:33:02.208 INFO:tasks.workunit.client.0.smithi019.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-07T02:33:02.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:02 smithi064 ceph-mon[34699]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-07T02:33:02.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:02 smithi064 ceph-mon[34699]: mgrmap e25: x(active, since 72s), standbys: z, y 2024-05-07T02:33:02.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:02 smithi019 ceph-mon[29781]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-07T02:33:02.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:02 smithi019 ceph-mon[29781]: mgrmap e25: x(active, since 72s), standbys: z, y 2024-05-07T02:33:02.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:02 smithi027 ceph-mon[34398]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-07T02:33:02.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:02 smithi027 ceph-mon[34398]: mgrmap e25: x(active, since 72s), standbys: z, y 2024-05-07T02:33:04.048 INFO:tasks.workunit.client.0.smithi019.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-07T02:33:04.284 INFO:tasks.workunit.client.0.smithi019.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-07T02:33:04.372 INFO:tasks.workunit.client.0.smithi019.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-07T02:33:04.377 INFO:tasks.workunit.client.0.smithi019.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-07T02:33:04.378 INFO:tasks.workunit.client.0.smithi019.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-07T02:33:04.462 INFO:tasks.workunit.client.0.smithi019.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-07T02:33:04.470 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-07T02:33:04.471 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-07T02:33:04.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.488+0000 7f9667e00700 1 -- 172.21.15.19:0/935041435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96681032e0 msgr2=0x7f96681036f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:04.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.488+0000 7f9667e00700 1 --2- 172.21.15.19:0/935041435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96681032e0 0x7f96681036f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9658009990 tx=0x7f9658009ca0 comp rx=0 tx=0).stop 2024-05-07T02:33:04.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.488+0000 7f9667e00700 1 -- 172.21.15.19:0/935041435 shutdown_connections 2024-05-07T02:33:04.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.488+0000 7f9667e00700 1 --2- 172.21.15.19:0/935041435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9668108560 0x7f966810fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:04.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.488+0000 7f9667e00700 1 --2- 172.21.15.19:0/935041435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96681032e0 0x7f96681036f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:04.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.488+0000 7f9667e00700 1 --2- 172.21.15.19:0/935041435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9668107c10 0x7f9668108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:04.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.488+0000 7f9667e00700 1 -- 172.21.15.19:0/935041435 >> 172.21.15.19:0/935041435 conn(0x7f9668078ac0 msgr2=0x7f9668078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:04.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.488+0000 7f9667e00700 1 -- 172.21.15.19:0/935041435 shutdown_connections 2024-05-07T02:33:04.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.488+0000 7f9667e00700 1 -- 172.21.15.19:0/935041435 wait complete. 2024-05-07T02:33:04.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.489+0000 7f9667e00700 1 Processor -- start 2024-05-07T02:33:04.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.489+0000 7f9667e00700 1 -- start start 2024-05-07T02:33:04.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.490+0000 7f9667e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96681032e0 0x7f966819cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:04.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.490+0000 7f9667e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9668107c10 0x7f966819d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:04.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.490+0000 7f9667e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9668108560 0x7f96681a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:04.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.490+0000 7f9667e00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f96681131b0 con 0x7f96681032e0 2024-05-07T02:33:04.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.490+0000 7f9667e00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9668113030 con 0x7f9668108560 2024-05-07T02:33:04.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.490+0000 7f9667e00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9668113330 con 0x7f9668107c10 2024-05-07T02:33:04.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.490+0000 7f9666a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96681032e0 0x7f966819cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:04.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.490+0000 7f9666000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9668107c10 0x7f966819d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:04.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.490+0000 7f9666a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96681032e0 0x7f966819cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40028/0 (socket says 172.21.15.19:40028) 2024-05-07T02:33:04.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.490+0000 7f9666a00700 1 -- 172.21.15.19:0/2534904105 learned_addr learned my addr 172.21.15.19:0/2534904105 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:04.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.490+0000 7f9667400700 1 --2- 172.21.15.19:0/2534904105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9668108560 0x7f96681a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:04.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.491+0000 7f9666000700 1 -- 172.21.15.19:0/2534904105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9668108560 msgr2=0x7f96681a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:04.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.491+0000 7f9666000700 1 --2- 172.21.15.19:0/2534904105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9668108560 0x7f96681a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:04.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.491+0000 7f9666000700 1 -- 172.21.15.19:0/2534904105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96681032e0 msgr2=0x7f966819cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:04.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.491+0000 7f9666000700 1 --2- 172.21.15.19:0/2534904105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96681032e0 0x7f966819cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:04.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.491+0000 7f9666000700 1 -- 172.21.15.19:0/2534904105 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96681a1d70 con 0x7f9668107c10 2024-05-07T02:33:04.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.491+0000 7f9666a00700 1 --2- 172.21.15.19:0/2534904105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96681032e0 0x7f966819cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:33:04.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.491+0000 7f9666000700 1 --2- 172.21.15.19:0/2534904105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9668107c10 0x7f966819d440 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f965400e9c0 tx=0x7f965400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:04.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.492+0000 7f965f400700 1 -- 172.21.15.19:0/2534904105 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965400cb70 con 0x7f9668107c10 2024-05-07T02:33:04.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.492+0000 7f965f400700 1 -- 172.21.15.19:0/2534904105 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f965400ee60 con 0x7f9668107c10 2024-05-07T02:33:04.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.492+0000 7f965f400700 1 -- 172.21.15.19:0/2534904105 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9654003d60 con 0x7f9668107c10 2024-05-07T02:33:04.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.492+0000 7f9667e00700 1 -- 172.21.15.19:0/2534904105 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96681a2060 con 0x7f9668107c10 2024-05-07T02:33:04.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.492+0000 7f9667e00700 1 -- 172.21.15.19:0/2534904105 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96681a9980 con 0x7f9668107c10 2024-05-07T02:33:04.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.496+0000 7f965f400700 1 -- 172.21.15.19:0/2534904105 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f9654003f00 con 0x7f9668107c10 2024-05-07T02:33:04.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.497+0000 7f967c23ab80 1 Processor -- start 2024-05-07T02:33:04.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.497+0000 7f967c23ab80 1 -- start start 2024-05-07T02:33:04.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.497+0000 7f965f400700 1 --2- 172.21.15.19:0/2534904105 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f964c0b2b10 0x7f964c0b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:04.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.497+0000 7f967c23ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b17fe11c40 0x55b17fe12050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:04.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.498+0000 7f967c23ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b17fe12590 0x55b17fe1b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:04.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.498+0000 7f967c23ab80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b17fe1bc30 0x55b17fe1e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:04.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.498+0000 7f967c23ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b17fd33a20 con 0x55b17fe12590 2024-05-07T02:33:04.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.498+0000 7f967c23ab80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b17fcd1760 con 0x55b17fe1bc30 2024-05-07T02:33:04.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.498+0000 7f967c23ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b17fd2f9e0 con 0x55b17fe11c40 2024-05-07T02:33:04.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.498+0000 7f9667400700 1 --2- 172.21.15.19:0/2534904105 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f964c0b2b10 0x7f964c0b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:04.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.498+0000 7f965f400700 1 -- 172.21.15.19:0/2534904105 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f96540e2db0 con 0x7f9668107c10 2024-05-07T02:33:04.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.498+0000 7f9666a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b17fe11c40 0x55b17fe12050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:04.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.498+0000 7f9667400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b17fe1bc30 0x55b17fe1e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:04.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.498+0000 7f9666a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b17fe11c40 0x55b17fe12050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:46842/0 (socket says 172.21.15.19:46842) 2024-05-07T02:33:04.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.498+0000 7f9666a00700 1 -- 172.21.15.19:0/2393150585 learned_addr learned my addr 172.21.15.19:0/2393150585 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:04.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.498+0000 7f9666000700 1 --2- 172.21.15.19:0/2393150585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b17fe12590 0x55b17fe1b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:04.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.499+0000 7f9666a00700 1 -- 172.21.15.19:0/2393150585 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b17fe1bc30 msgr2=0x55b17fe1e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:04.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.499+0000 7f9666a00700 1 --2- 172.21.15.19:0/2393150585 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b17fe1bc30 0x55b17fe1e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:04.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.499+0000 7f9666a00700 1 -- 172.21.15.19:0/2393150585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b17fe12590 msgr2=0x55b17fe1b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:04.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.499+0000 7f9666a00700 1 --2- 172.21.15.19:0/2393150585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b17fe12590 0x55b17fe1b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:04.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.499+0000 7f9666a00700 1 -- 172.21.15.19:0/2393150585 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b17fc808d0 con 0x55b17fe11c40 2024-05-07T02:33:04.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.499+0000 7f9667400700 1 --2- 172.21.15.19:0/2534904105 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f964c0b2b10 0x7f964c0b5020 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9660009ec0 tx=0x7f96600134c0 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:04.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.499+0000 7f9666a00700 1 --2- 172.21.15.19:0/2393150585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b17fe11c40 0x55b17fe12050 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9658009700 tx=0x7f965800b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b3430f9b2278ad4 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:04.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.500+0000 7f9653e00700 1 -- 172.21.15.19:0/2393150585 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9658005270 con 0x55b17fe11c40 2024-05-07T02:33:04.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.500+0000 7f9653e00700 1 -- 172.21.15.19:0/2393150585 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9658005410 con 0x55b17fe11c40 2024-05-07T02:33:04.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.500+0000 7f9653e00700 1 -- 172.21.15.19:0/2393150585 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965800f6c0 con 0x55b17fe11c40 2024-05-07T02:33:04.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.501+0000 7f967c23ab80 1 -- 172.21.15.19:0/2393150585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b17fe11c40 msgr2=0x55b17fe12050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:04.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.501+0000 7f967c23ab80 1 --2- 172.21.15.19:0/2393150585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b17fe11c40 0x55b17fe12050 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9658009700 tx=0x7f965800b710 comp rx=0 tx=0).stop 2024-05-07T02:33:04.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.501+0000 7f967c23ab80 1 -- 172.21.15.19:0/2393150585 shutdown_connections 2024-05-07T02:33:04.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.501+0000 7f967c23ab80 1 --2- 172.21.15.19:0/2393150585 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b17fe1bc30 0x55b17fe1e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:04.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.501+0000 7f967c23ab80 1 --2- 172.21.15.19:0/2393150585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b17fe11c40 0x55b17fe12050 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:04.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.501+0000 7f967c23ab80 1 --2- 172.21.15.19:0/2393150585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b17fe12590 0x55b17fe1b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:04.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.501+0000 7f967c23ab80 1 -- 172.21.15.19:0/2393150585 >> 172.21.15.19:0/2393150585 conn(0x55b17fc93e40 msgr2=0x55b17fe0c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:04.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.501+0000 7f967c23ab80 1 -- 172.21.15.19:0/2393150585 shutdown_connections 2024-05-07T02:33:04.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.501+0000 7f967c23ab80 1 -- 172.21.15.19:0/2393150585 wait complete. 2024-05-07T02:33:04.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.502+0000 7f967c23ab80 1 -- start start 2024-05-07T02:33:04.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.502+0000 7f967c23ab80 1 Processor -- start 2024-05-07T02:33:04.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.503+0000 7f967c23ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b17fe11c40 0x55b17fe18bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:04.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.503+0000 7f967c23ab80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b17fe12590 0x55b17fe19100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:04.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.503+0000 7f967c23ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b17fe1bc30 0x55b17fe19640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:04.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.503+0000 7f967c23ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b17fe1e940 con 0x55b17fe1bc30 2024-05-07T02:33:04.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.503+0000 7f967c23ab80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b17fc99b20 con 0x55b17fe12590 2024-05-07T02:33:04.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.503+0000 7f967c23ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b17fe1eac0 con 0x55b17fe11c40 2024-05-07T02:33:04.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.503+0000 7f967c23ab80 10 client.0 _ll_register_callbacks cb 0x55b17fcc7380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:33:04.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.503+0000 7f9666a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b17fe11c40 0x55b17fe18bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:04.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.503+0000 7f9666000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b17fe12590 0x55b17fe19100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:04.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.503+0000 7f9666a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b17fe11c40 0x55b17fe18bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:46846/0 (socket says 172.21.15.19:46846) 2024-05-07T02:33:04.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.503+0000 7f9666000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b17fe12590 0x55b17fe19100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:55562/0 (socket says 172.21.15.19:55562) 2024-05-07T02:33:04.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.503+0000 7f9666a00700 1 -- 172.21.15.19:0/45799489 learned_addr learned my addr 172.21.15.19:0/45799489 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:04.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.504+0000 7f9667400700 1 --2- 172.21.15.19:0/45799489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b17fe1bc30 0x55b17fe19640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:04.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.504+0000 7f9666000700 1 -- 172.21.15.19:0/45799489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b17fe11c40 msgr2=0x55b17fe18bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:04.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.504+0000 7f9666000700 1 --2- 172.21.15.19:0/45799489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b17fe11c40 0x55b17fe18bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:04.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.504+0000 7f9666000700 1 -- 172.21.15.19:0/45799489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b17fe1bc30 msgr2=0x55b17fe19640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:04.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.504+0000 7f9666000700 1 --2- 172.21.15.19:0/45799489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b17fe1bc30 0x55b17fe19640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:04.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.504+0000 7f9666000700 1 -- 172.21.15.19:0/45799489 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b17fcf9420 con 0x55b17fe12590 2024-05-07T02:33:04.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.504+0000 7f9666000700 1 --2- 172.21.15.19:0/45799489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b17fe12590 0x55b17fe19100 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f96540dff90 tx=0x7f9654009630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45c5278f0e3cea80 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:04.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.504+0000 7f9650c00700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:33:04.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.505+0000 7f9650c00700 1 -- 172.21.15.19:0/45799489 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96540a5040 con 0x55b17fe12590 2024-05-07T02:33:04.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.505+0000 7f9650c00700 1 -- 172.21.15.19:0/45799489 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f965401e2e0 con 0x55b17fe12590 2024-05-07T02:33:04.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.505+0000 7f967c23ab80 1 -- 172.21.15.19:0/45799489 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9658005410 con 0x55b17fe12590 2024-05-07T02:33:04.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.505+0000 7f9650c00700 1 -- 172.21.15.19:0/45799489 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f965401e4e0 con 0x55b17fe12590 2024-05-07T02:33:04.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.505+0000 7f967c23ab80 10 client.25508 fetch_fsmap learned FSMap version 31 2024-05-07T02:33:04.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.505+0000 7f967c23ab80 1 -- 172.21.15.19:0/45799489 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b17fe295a0 con 0x55b17fe12590 2024-05-07T02:33:04.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.505+0000 7f9650c00700 1 -- 172.21.15.19:0/45799489 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9658005410 con 0x55b17fe12590 2024-05-07T02:33:04.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.506+0000 7f9650c00700 1 -- 172.21.15.19:0/45799489 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f96540a9cb0 con 0x55b17fe12590 2024-05-07T02:33:04.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.506+0000 7f967c23ab80 10 client.25508 fetch_fsmap finished waiting for FSMap version 31 2024-05-07T02:33:04.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.506+0000 7f967c23ab80 10 client.25508 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-07T02:33:04.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.506+0000 7f967c23ab80 20 client.25508 populate_metadata read hostname 'smithi019' 2024-05-07T02:33:04.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.506+0000 7f967c23ab80 1 --2- 172.21.15.19:0/45799489 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55b17fe2bb30 0x55b17fe2dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:04.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.506+0000 7f967c23ab80 4 client.25508 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:33:04.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.506+0000 7f967c23ab80 1 -- 172.21.15.19:0/45799489 --> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b17fdde0e0 con 0x55b17fe2bb30 2024-05-07T02:33:04.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.506+0000 7f9666a00700 1 --2- 172.21.15.19:0/45799489 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55b17fe2bb30 0x55b17fe2dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:04.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.507+0000 7f9666a00700 1 --2- 172.21.15.19:0/45799489 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55b17fe2bb30 0x55b17fe2dfb0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=57ca81f8810d1bfb server_cookie=20c97e1495c3945b in_seq=0 out_seq=0 2024-05-07T02:33:04.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.507+0000 7f9650c00700 10 client.25508 ms_handle_connect on v2:172.21.15.64:6838/1583120004 2024-05-07T02:33:04.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.508+0000 7f9650c00700 1 -- 172.21.15.19:0/45799489 <== mds.2 v2:172.21.15.64:6838/1583120004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b17fdde0e0 con 0x55b17fe2bb30 2024-05-07T02:33:04.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.508+0000 7f9650c00700 10 client.25508 handle_command_reply: tid=0 2024-05-07T02:33:04.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.508+0000 7f967c23ab80 1 client.25508 shutdown 2024-05-07T02:33:04.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.508+0000 7f967c23ab80 1 -- 172.21.15.19:0/45799489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b17fe12590 msgr2=0x55b17fe19100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:04.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.508+0000 7f967c23ab80 1 --2- 172.21.15.19:0/45799489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b17fe12590 0x55b17fe19100 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f96540dff90 tx=0x7f9654009630 comp rx=0 tx=0).stop 2024-05-07T02:33:04.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.508+0000 7f967c23ab80 1 -- 172.21.15.19:0/45799489 shutdown_connections 2024-05-07T02:33:04.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.508+0000 7f967c23ab80 1 --2- 172.21.15.19:0/45799489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b17fe1bc30 0x55b17fe19640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:04.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.508+0000 7f967c23ab80 1 --2- 172.21.15.19:0/45799489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b17fe11c40 0x55b17fe18bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:04.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.508+0000 7f967c23ab80 1 --2- 172.21.15.19:0/45799489 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55b17fe2bb30 0x55b17fe2dfb0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:04.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.508+0000 7f967c23ab80 1 --2- 172.21.15.19:0/45799489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b17fe12590 0x55b17fe19100 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:04.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.508+0000 7f967c23ab80 1 -- 172.21.15.19:0/45799489 >> 172.21.15.19:0/45799489 conn(0x55b17fc93e40 msgr2=0x55b17fe0e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:04.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.509+0000 7f9650c00700 0 client.25508 ms_handle_reset on v2:172.21.15.64:6838/1583120004 2024-05-07T02:33:04.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.509+0000 7f967c23ab80 1 -- 172.21.15.19:0/45799489 shutdown_connections 2024-05-07T02:33:04.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.509+0000 7f967c23ab80 1 -- 172.21.15.19:0/45799489 wait complete. 2024-05-07T02:33:04.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.509+0000 7f967c23ab80 20 client.25508 trim_cache size 0 max 16384 2024-05-07T02:33:04.516 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-05-07T02:33:04.518 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-05-07T02:33:04.518 DEBUG:teuthology.orchestra.run.smithi019:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=366c9130c497dee9e73178650cf710ed21269b18 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-05-07T02:33:04.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.526+0000 7f967c23ab80 1 Processor -- start 2024-05-07T02:33:04.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.526+0000 7f967c23ab80 1 -- start start 2024-05-07T02:33:04.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.527+0000 7f967c23ab80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b17fc93e40 0x55b17fe17bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:04.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.527+0000 7f967c23ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b17fe11c40 0x55b17fe279d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:04.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.527+0000 7f967c23ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b17fe12590 0x55b17fe33770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:04.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.527+0000 7f967c23ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f96681b2550 con 0x55b17fe12590 2024-05-07T02:33:04.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.527+0000 7f967c23ab80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f96681b4090 con 0x55b17fc93e40 2024-05-07T02:33:04.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.527+0000 7f967c23ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f96681b2c20 con 0x55b17fe11c40 2024-05-07T02:33:04.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.527+0000 7f9666a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b17fc93e40 0x55b17fe17bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:04.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.527+0000 7f9666a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b17fc93e40 0x55b17fe17bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:55574/0 (socket says 172.21.15.19:55574) 2024-05-07T02:33:04.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.527+0000 7f9666a00700 1 -- 172.21.15.19:0/3955160185 learned_addr learned my addr 172.21.15.19:0/3955160185 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:04.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.527+0000 7f9666000700 1 --2- 172.21.15.19:0/3955160185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b17fe11c40 0x55b17fe279d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:04.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.527+0000 7f9667400700 1 --2- 172.21.15.19:0/3955160185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b17fe12590 0x55b17fe33770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:04.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.527+0000 7f9666a00700 1 -- 172.21.15.19:0/3955160185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b17fe11c40 msgr2=0x55b17fe279d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:04.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.527+0000 7f9666a00700 1 --2- 172.21.15.19:0/3955160185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b17fe11c40 0x55b17fe279d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:04.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.527+0000 7f9666a00700 1 -- 172.21.15.19:0/3955160185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b17fe12590 msgr2=0x55b17fe33770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:04.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.527+0000 7f9666a00700 1 --2- 172.21.15.19:0/3955160185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b17fe12590 0x55b17fe33770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:04.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.528+0000 7f9666a00700 1 -- 172.21.15.19:0/3955160185 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b17fe12fa0 con 0x55b17fc93e40 2024-05-07T02:33:04.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.528+0000 7f9667400700 1 --2- 172.21.15.19:0/3955160185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b17fe12590 0x55b17fe33770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:33:04.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.528+0000 7f9666a00700 1 --2- 172.21.15.19:0/3955160185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b17fc93e40 0x55b17fe17bc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9658009750 tx=0x7f965801a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdfcdca8700565fd server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:04.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.528+0000 7f9653400700 1 -- 172.21.15.19:0/3955160185 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9658003e60 con 0x55b17fc93e40 2024-05-07T02:33:04.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.528+0000 7f9653400700 1 -- 172.21.15.19:0/3955160185 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9658003b30 con 0x55b17fc93e40 2024-05-07T02:33:04.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.528+0000 7f9653400700 1 -- 172.21.15.19:0/3955160185 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9658033340 con 0x55b17fc93e40 2024-05-07T02:33:04.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.529+0000 7f967c23ab80 1 -- 172.21.15.19:0/3955160185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b17fc93e40 msgr2=0x55b17fe17bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:04.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.529+0000 7f967c23ab80 1 --2- 172.21.15.19:0/3955160185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b17fc93e40 0x55b17fe17bc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9658009750 tx=0x7f965801a040 comp rx=0 tx=0).stop 2024-05-07T02:33:04.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.529+0000 7f967c23ab80 1 -- 172.21.15.19:0/3955160185 shutdown_connections 2024-05-07T02:33:04.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.529+0000 7f967c23ab80 1 --2- 172.21.15.19:0/3955160185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b17fe12590 0x55b17fe33770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:04.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.529+0000 7f967c23ab80 1 --2- 172.21.15.19:0/3955160185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b17fc93e40 0x55b17fe17bc0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:04.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.529+0000 7f967c23ab80 1 --2- 172.21.15.19:0/3955160185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b17fe11c40 0x55b17fe279d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:04.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.529+0000 7f967c23ab80 1 -- 172.21.15.19:0/3955160185 >> 172.21.15.19:0/3955160185 conn(0x55b17fe05d10 msgr2=0x55b17fe174b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:04.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.530+0000 7f967c23ab80 1 -- 172.21.15.19:0/3955160185 shutdown_connections 2024-05-07T02:33:04.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.530+0000 7f967c23ab80 1 -- 172.21.15.19:0/3955160185 wait complete. 2024-05-07T02:33:04.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.530+0000 7f967c23ab80 1 -- start start 2024-05-07T02:33:04.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.531+0000 7f967c23ab80 1 Processor -- start 2024-05-07T02:33:04.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.531+0000 7f967c23ab80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b17fc93e40 0x55b17fe1eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:04.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.531+0000 7f967c23ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b17fe11c40 0x55b17fe1f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:04.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.531+0000 7f967c23ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b17fe12590 0x55b17fe94310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:04.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.531+0000 7f967c23ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f96681b39c0 con 0x55b17fe11c40 2024-05-07T02:33:04.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.531+0000 7f967c23ab80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f96681b32f0 con 0x55b17fc93e40 2024-05-07T02:33:04.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.531+0000 7f967c23ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f96681b04f0 con 0x55b17fe12590 2024-05-07T02:33:04.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.531+0000 7f967c23ab80 10 client.0 _ll_register_callbacks cb 0x55b17fcc7380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:33:04.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.532+0000 7f9666000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b17fe11c40 0x55b17fe1f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:04.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.532+0000 7f9666000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b17fe11c40 0x55b17fe1f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40054/0 (socket says 172.21.15.19:40054) 2024-05-07T02:33:04.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.532+0000 7f9666000700 1 -- 172.21.15.19:0/4026638466 learned_addr learned my addr 172.21.15.19:0/4026638466 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:04.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.532+0000 7f9667400700 1 --2- 172.21.15.19:0/4026638466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b17fe12590 0x55b17fe94310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:04.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.532+0000 7f9666a00700 1 --2- 172.21.15.19:0/4026638466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b17fc93e40 0x55b17fe1eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:04.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.532+0000 7f9666a00700 1 -- 172.21.15.19:0/4026638466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b17fe12590 msgr2=0x55b17fe94310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:04.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.532+0000 7f9666a00700 1 --2- 172.21.15.19:0/4026638466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b17fe12590 0x55b17fe94310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:04.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.532+0000 7f9666a00700 1 -- 172.21.15.19:0/4026638466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b17fe11c40 msgr2=0x55b17fe1f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:04.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.532+0000 7f9666a00700 1 --2- 172.21.15.19:0/4026638466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b17fe11c40 0x55b17fe1f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:04.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.532+0000 7f9666a00700 1 -- 172.21.15.19:0/4026638466 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b17fe94850 con 0x55b17fc93e40 2024-05-07T02:33:04.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.532+0000 7f9666000700 1 --2- 172.21.15.19:0/4026638466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b17fe11c40 0x55b17fe1f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:33:04.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.533+0000 7f9666a00700 1 --2- 172.21.15.19:0/4026638466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b17fc93e40 0x55b17fe1eff0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f965800b750 tx=0x7f9658005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94a37ebca11f7bff server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:04.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.533+0000 7f9650c00700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:33:04.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.533+0000 7f9650c00700 1 -- 172.21.15.19:0/4026638466 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9658003e60 con 0x55b17fc93e40 2024-05-07T02:33:04.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.533+0000 7f9650c00700 1 -- 172.21.15.19:0/4026638466 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f9658003680 con 0x55b17fc93e40 2024-05-07T02:33:04.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.533+0000 7f967c23ab80 1 -- 172.21.15.19:0/4026638466 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9658003b30 con 0x55b17fc93e40 2024-05-07T02:33:04.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.533+0000 7f9650c00700 1 -- 172.21.15.19:0/4026638466 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f96580038a0 con 0x55b17fc93e40 2024-05-07T02:33:04.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.533+0000 7f967c23ab80 10 client.25514 fetch_fsmap learned FSMap version 31 2024-05-07T02:33:04.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.533+0000 7f967c23ab80 1 -- 172.21.15.19:0/4026638466 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b17fe95a60 con 0x55b17fc93e40 2024-05-07T02:33:04.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.534+0000 7f9650c00700 1 -- 172.21.15.19:0/4026638466 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9658003b30 con 0x55b17fc93e40 2024-05-07T02:33:04.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.534+0000 7f9650c00700 1 -- 172.21.15.19:0/4026638466 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f965800f460 con 0x55b17fc93e40 2024-05-07T02:33:04.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.534+0000 7f967c23ab80 10 client.25514 fetch_fsmap finished waiting for FSMap version 31 2024-05-07T02:33:04.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.534+0000 7f967c23ab80 10 client.25514 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-07T02:33:04.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.534+0000 7f967c23ab80 20 client.25514 populate_metadata read hostname 'smithi019' 2024-05-07T02:33:04.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.534+0000 7f967c23ab80 1 --2- 172.21.15.19:0/4026638466 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55b17fe1bc30 0x55b17fe97c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:04.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.534+0000 7f967c23ab80 4 client.25514 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:33:04.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.534+0000 7f967c23ab80 1 -- 172.21.15.19:0/4026638466 --> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b17fd41400 con 0x55b17fe1bc30 2024-05-07T02:33:04.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.535+0000 7f9666000700 1 --2- 172.21.15.19:0/4026638466 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55b17fe1bc30 0x55b17fe97c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:04.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.535+0000 7f9666000700 1 --2- 172.21.15.19:0/4026638466 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55b17fe1bc30 0x55b17fe97c90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e8e4fe8d8077be6e server_cookie=35a6fd20f5cdc8df in_seq=0 out_seq=0 2024-05-07T02:33:04.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.535+0000 7f9650c00700 10 client.25514 ms_handle_connect on v2:172.21.15.64:6838/1583120004 2024-05-07T02:33:04.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.535+0000 7f9650c00700 1 -- 172.21.15.19:0/4026638466 <== mds.2 v2:172.21.15.64:6838/1583120004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b17fd41400 con 0x55b17fe1bc30 2024-05-07T02:33:04.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.535+0000 7f9650c00700 10 client.25514 handle_command_reply: tid=0 2024-05-07T02:33:04.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.535+0000 7f967c23ab80 1 client.25514 shutdown 2024-05-07T02:33:04.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.536+0000 7f967c23ab80 1 -- 172.21.15.19:0/4026638466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b17fc93e40 msgr2=0x55b17fe1eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:04.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.536+0000 7f967c23ab80 1 --2- 172.21.15.19:0/4026638466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b17fc93e40 0x55b17fe1eff0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f965800b750 tx=0x7f9658005f10 comp rx=0 tx=0).stop 2024-05-07T02:33:04.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.536+0000 7f967c23ab80 1 -- 172.21.15.19:0/4026638466 shutdown_connections 2024-05-07T02:33:04.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.536+0000 7f967c23ab80 1 --2- 172.21.15.19:0/4026638466 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b17fe12590 0x55b17fe94310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:04.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.536+0000 7f967c23ab80 1 --2- 172.21.15.19:0/4026638466 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55b17fe1bc30 0x55b17fe97c90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:04.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.536+0000 7f967c23ab80 1 --2- 172.21.15.19:0/4026638466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b17fc93e40 0x55b17fe1eff0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:04.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.536+0000 7f967c23ab80 1 --2- 172.21.15.19:0/4026638466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b17fe11c40 0x55b17fe1f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:04.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.536+0000 7f967c23ab80 1 -- 172.21.15.19:0/4026638466 >> 172.21.15.19:0/4026638466 conn(0x55b17fe05d10 msgr2=0x55b17fe10610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:04.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.536+0000 7f9650c00700 0 client.25514 ms_handle_reset on v2:172.21.15.64:6838/1583120004 2024-05-07T02:33:04.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.536+0000 7f967c23ab80 1 -- 172.21.15.19:0/4026638466 shutdown_connections 2024-05-07T02:33:04.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.536+0000 7f967c23ab80 1 -- 172.21.15.19:0/4026638466 wait complete. 2024-05-07T02:33:04.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.537+0000 7f967c23ab80 20 client.25514 trim_cache size 0 max 16384 2024-05-07T02:33:04.539 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:33:04.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.538+0000 7f9667e00700 1 -- 172.21.15.19:0/2534904105 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f964c0b2b10 msgr2=0x7f964c0b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:04.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.538+0000 7f9667e00700 1 --2- 172.21.15.19:0/2534904105 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f964c0b2b10 0x7f964c0b5020 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9660009ec0 tx=0x7f96600134c0 comp rx=0 tx=0).stop 2024-05-07T02:33:04.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.538+0000 7f9667e00700 1 -- 172.21.15.19:0/2534904105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9668107c10 msgr2=0x7f966819d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:04.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.538+0000 7f9667e00700 1 --2- 172.21.15.19:0/2534904105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9668107c10 0x7f966819d440 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f965400e9c0 tx=0x7f965400ecd0 comp rx=0 tx=0).stop 2024-05-07T02:33:04.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.538+0000 7f9667e00700 1 -- 172.21.15.19:0/2534904105 shutdown_connections 2024-05-07T02:33:04.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.538+0000 7f9667e00700 1 --2- 172.21.15.19:0/2534904105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9668108560 0x7f96681a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:04.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.538+0000 7f9667e00700 1 --2- 172.21.15.19:0/2534904105 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f964c0b2b10 0x7f964c0b5020 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:04.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.538+0000 7f9667e00700 1 --2- 172.21.15.19:0/2534904105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96681032e0 0x7f966819cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:04.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.538+0000 7f9667e00700 1 --2- 172.21.15.19:0/2534904105 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9668107c10 0x7f966819d440 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:04.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.538+0000 7f9667e00700 1 -- 172.21.15.19:0/2534904105 >> 172.21.15.19:0/2534904105 conn(0x7f9668078ac0 msgr2=0x7f966806b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:04.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.538+0000 7f9667e00700 1 -- 172.21.15.19:0/2534904105 shutdown_connections 2024-05-07T02:33:04.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:04.538+0000 7f9667e00700 1 -- 172.21.15.19:0/2534904105 wait complete. 2024-05-07T02:33:04.583 INFO:tasks.workunit.client.0.smithi019.stderr:+ dbench 1 2024-05-07T02:33:04.605 INFO:tasks.workunit.client.0.smithi019.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-05-07T02:33:04.605 INFO:tasks.workunit.client.0.smithi019.stdout: 2024-05-07T02:33:04.605 INFO:tasks.workunit.client.0.smithi019.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-05-07T02:33:04.619 INFO:tasks.workunit.client.0.smithi019.stdout:0 of 1 processes prepared for launch 0 sec 2024-05-07T02:33:04.719 INFO:tasks.workunit.client.0.smithi019.stdout:1 of 1 processes prepared for launch 0 sec 2024-05-07T02:33:04.719 INFO:tasks.workunit.client.0.smithi019.stdout:releasing clients 2024-05-07T02:33:05.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:04 smithi027 ceph-mon[34398]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-07T02:33:05.249 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:04 smithi019 ceph-mon[29781]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-07T02:33:05.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:04 smithi064 ceph-mon[34699]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-07T02:33:05.673 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-07T02:33:05.721 INFO:tasks.workunit.client.0.smithi019.stdout: 1 1855 26.36 MB/sec warmup 1 sec latency 20.861 ms 2024-05-07T02:33:06.208 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:33:06.720 INFO:tasks.workunit.client.0.smithi019.stdout: 1 4245 20.15 MB/sec warmup 2 sec latency 14.311 ms 2024-05-07T02:33:07.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:06 smithi019 ceph-mon[29781]: pgmap v41: 129 pgs: 129 active+clean; 2.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 131 KiB/s wr, 42 op/s 2024-05-07T02:33:07.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:06 smithi027 ceph-mon[34398]: pgmap v41: 129 pgs: 129 active+clean; 2.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 131 KiB/s wr, 42 op/s 2024-05-07T02:33:07.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:06 smithi064 ceph-mon[34699]: pgmap v41: 129 pgs: 129 active+clean; 2.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 131 KiB/s wr, 42 op/s 2024-05-07T02:33:07.720 INFO:tasks.workunit.client.0.smithi019.stdout: 1 6844 18.15 MB/sec warmup 3 sec latency 10.541 ms 2024-05-07T02:33:08.720 INFO:tasks.workunit.client.0.smithi019.stdout: 1 8487 16.06 MB/sec warmup 4 sec latency 21.027 ms 2024-05-07T02:33:08.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.742+0000 7fb982200700 1 -- 172.21.15.19:0/1545195192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb97c10b2a0 msgr2=0x7fb97c117820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:08.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.742+0000 7fb982200700 1 --2- 172.21.15.19:0/1545195192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb97c10b2a0 0x7fb97c117820 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb974009f90 tx=0x7fb97400b330 comp rx=0 tx=0).stop 2024-05-07T02:33:08.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.743+0000 7fb982200700 1 -- 172.21.15.19:0/1545195192 shutdown_connections 2024-05-07T02:33:08.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.743+0000 7fb982200700 1 --2- 172.21.15.19:0/1545195192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb97c10b2a0 0x7fb97c117820 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:08.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.743+0000 7fb982200700 1 --2- 172.21.15.19:0/1545195192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb97c110980 0x7fb97c110d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:08.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.743+0000 7fb982200700 1 --2- 172.21.15.19:0/1545195192 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb97c10a950 0x7fb97c10ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:08.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.743+0000 7fb982200700 1 -- 172.21.15.19:0/1545195192 >> 172.21.15.19:0/1545195192 conn(0x7fb97c076d30 msgr2=0x7fb97c077140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:08.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.743+0000 7fb982200700 1 -- 172.21.15.19:0/1545195192 shutdown_connections 2024-05-07T02:33:08.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.744+0000 7fb982200700 1 -- 172.21.15.19:0/1545195192 wait complete. 2024-05-07T02:33:08.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.746+0000 7fb982200700 1 Processor -- start 2024-05-07T02:33:08.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.746+0000 7fb982200700 1 -- start start 2024-05-07T02:33:08.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.746+0000 7fb982200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb97c10a950 0x7fb97c1a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:08.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.746+0000 7fb982200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb97c10b2a0 0x7fb97c1a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:08.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.746+0000 7fb982200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb97c110980 0x7fb97c1a5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:08.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.746+0000 7fb982200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c11ab00 con 0x7fb97c10b2a0 2024-05-07T02:33:08.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.746+0000 7fb982200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c11a980 con 0x7fb97c110980 2024-05-07T02:33:08.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.746+0000 7fb982200700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c11ac80 con 0x7fb97c10a950 2024-05-07T02:33:08.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.747+0000 7fb97aa00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb97c10b2a0 0x7fb97c1a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:08.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.747+0000 7fb97be00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb97c110980 0x7fb97c1a5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:08.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.747+0000 7fb97b400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb97c10a950 0x7fb97c1a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:08.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.747+0000 7fb97be00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb97c110980 0x7fb97c1a5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:55594/0 (socket says 172.21.15.19:55594) 2024-05-07T02:33:08.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.747+0000 7fb97be00700 1 -- 172.21.15.19:0/2897649107 learned_addr learned my addr 172.21.15.19:0/2897649107 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:08.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.747+0000 7fb97b400700 1 -- 172.21.15.19:0/2897649107 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb97c110980 msgr2=0x7fb97c1a5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:08.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.747+0000 7fb97b400700 1 --2- 172.21.15.19:0/2897649107 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb97c110980 0x7fb97c1a5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:08.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.747+0000 7fb97b400700 1 -- 172.21.15.19:0/2897649107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb97c10b2a0 msgr2=0x7fb97c1a1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:08.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.747+0000 7fb97b400700 1 --2- 172.21.15.19:0/2897649107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb97c10b2a0 0x7fb97c1a1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:08.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.747+0000 7fb97b400700 1 -- 172.21.15.19:0/2897649107 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb97c1a63a0 con 0x7fb97c10a950 2024-05-07T02:33:08.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.748+0000 7fb97b400700 1 --2- 172.21.15.19:0/2897649107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb97c10a950 0x7fb97c1a14d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb96c005fd0 tx=0x7fb96c00eca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:08.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.748+0000 7fb973e00700 1 -- 172.21.15.19:0/2897649107 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c010900 con 0x7fb97c10a950 2024-05-07T02:33:08.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.748+0000 7fb973e00700 1 -- 172.21.15.19:0/2897649107 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb96c018420 con 0x7fb97c10a950 2024-05-07T02:33:08.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.748+0000 7fb973e00700 1 -- 172.21.15.19:0/2897649107 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c00f680 con 0x7fb97c10a950 2024-05-07T02:33:08.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.748+0000 7fb982200700 1 -- 172.21.15.19:0/2897649107 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb97c1a6690 con 0x7fb97c10a950 2024-05-07T02:33:08.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.748+0000 7fb982200700 1 -- 172.21.15.19:0/2897649107 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb97c1adfb0 con 0x7fb97c10a950 2024-05-07T02:33:08.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.752+0000 7fb973e00700 1 -- 172.21.15.19:0/2897649107 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7fb96c00f820 con 0x7fb97c10a950 2024-05-07T02:33:08.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.754+0000 7fb973e00700 1 --2- 172.21.15.19:0/2897649107 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fb9600b2b10 0x7fb9600b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:08.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.754+0000 7fb990de4b80 1 Processor -- start 2024-05-07T02:33:08.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.754+0000 7fb973e00700 1 -- 172.21.15.19:0/2897649107 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fb96c0e2c30 con 0x7fb97c10a950 2024-05-07T02:33:08.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.754+0000 7fb97aa00700 1 --2- 172.21.15.19:0/2897649107 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fb9600b2b10 0x7fb9600b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:08.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.754+0000 7fb990de4b80 1 -- start start 2024-05-07T02:33:08.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.755+0000 7fb990de4b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603f613f3b0 0x5603f613f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:08.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.755+0000 7fb990de4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5603f6139470 0x5603f6139880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:08.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.755+0000 7fb990de4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603f6139dc0 0x5603f6146040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:08.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.755+0000 7fb990de4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5603f605ba20 con 0x5603f6139dc0 2024-05-07T02:33:08.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.755+0000 7fb990de4b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5603f5ff9760 con 0x5603f613f3b0 2024-05-07T02:33:08.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.755+0000 7fb990de4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5603f60579e0 con 0x5603f6139470 2024-05-07T02:33:08.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.755+0000 7fb97b400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603f613f3b0 0x5603f613f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:08.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.755+0000 7fb97b400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603f613f3b0 0x5603f613f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:55610/0 (socket says 172.21.15.19:55610) 2024-05-07T02:33:08.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.755+0000 7fb97b400700 1 -- 172.21.15.19:0/2921311729 learned_addr learned my addr 172.21.15.19:0/2921311729 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:08.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.755+0000 7fb97be00700 1 --2- 172.21.15.19:0/2921311729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603f6139dc0 0x5603f6146040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:08.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.755+0000 7fb97aa00700 1 --2- 172.21.15.19:0/2921311729 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5603f6139470 0x5603f6139880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:08.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.755+0000 7fb97aa00700 1 --2- 172.21.15.19:0/2897649107 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fb9600b2b10 0x7fb9600b5020 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb97c06aab0 tx=0x7fb964009260 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:08.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.756+0000 7fb97b400700 1 -- 172.21.15.19:0/2921311729 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5603f6139470 msgr2=0x5603f6139880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:08.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.756+0000 7fb97b400700 1 --2- 172.21.15.19:0/2921311729 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5603f6139470 0x5603f6139880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:08.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.756+0000 7fb97b400700 1 -- 172.21.15.19:0/2921311729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603f6139dc0 msgr2=0x5603f6146040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:08.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.756+0000 7fb97b400700 1 --2- 172.21.15.19:0/2921311729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603f6139dc0 0x5603f6146040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:08.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.756+0000 7fb97b400700 1 -- 172.21.15.19:0/2921311729 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603f5fa88d0 con 0x5603f613f3b0 2024-05-07T02:33:08.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.756+0000 7fb97aa00700 1 --2- 172.21.15.19:0/2921311729 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5603f6139470 0x5603f6139880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:33:08.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.756+0000 7fb97b400700 1 --2- 172.21.15.19:0/2921311729 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603f613f3b0 0x5603f613f7c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0a9630 tx=0x7fb96c0a93b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3ca011256f4930d server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:08.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.756+0000 7fb970c00700 1 -- 172.21.15.19:0/2921311729 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c0adce0 con 0x5603f613f3b0 2024-05-07T02:33:08.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.756+0000 7fb970c00700 1 -- 172.21.15.19:0/2921311729 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb96c08fe50 con 0x5603f613f3b0 2024-05-07T02:33:08.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.756+0000 7fb970c00700 1 -- 172.21.15.19:0/2921311729 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c0cae40 con 0x5603f613f3b0 2024-05-07T02:33:08.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.757+0000 7fb990de4b80 1 -- 172.21.15.19:0/2921311729 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603f613f3b0 msgr2=0x5603f613f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:08.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.757+0000 7fb990de4b80 1 --2- 172.21.15.19:0/2921311729 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603f613f3b0 0x5603f613f7c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0a9630 tx=0x7fb96c0a93b0 comp rx=0 tx=0).stop 2024-05-07T02:33:08.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.757+0000 7fb990de4b80 1 -- 172.21.15.19:0/2921311729 shutdown_connections 2024-05-07T02:33:08.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.757+0000 7fb990de4b80 1 --2- 172.21.15.19:0/2921311729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603f6139dc0 0x5603f6146040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:08.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.758+0000 7fb990de4b80 1 --2- 172.21.15.19:0/2921311729 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603f613f3b0 0x5603f613f7c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:08.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.758+0000 7fb990de4b80 1 --2- 172.21.15.19:0/2921311729 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5603f6139470 0x5603f6139880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:08.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.758+0000 7fb990de4b80 1 -- 172.21.15.19:0/2921311729 >> 172.21.15.19:0/2921311729 conn(0x5603f5fbbe40 msgr2=0x5603f61342f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:08.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.758+0000 7fb990de4b80 1 -- 172.21.15.19:0/2921311729 shutdown_connections 2024-05-07T02:33:08.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.758+0000 7fb990de4b80 1 -- 172.21.15.19:0/2921311729 wait complete. 2024-05-07T02:33:08.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.758+0000 7fb990de4b80 1 -- start start 2024-05-07T02:33:08.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.759+0000 7fb990de4b80 1 Processor -- start 2024-05-07T02:33:08.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.759+0000 7fb990de4b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603f6139470 0x5603f6143640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:08.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.759+0000 7fb990de4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603f6139dc0 0x5603f6143b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:08.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.759+0000 7fb990de4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5603f613f3b0 0x5603f61440c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:08.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.759+0000 7fb990de4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5603f6146760 con 0x5603f6139dc0 2024-05-07T02:33:08.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.759+0000 7fb990de4b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5603f5fc1b20 con 0x5603f6139470 2024-05-07T02:33:08.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.759+0000 7fb990de4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5603f61468e0 con 0x5603f613f3b0 2024-05-07T02:33:08.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.759+0000 7fb990de4b80 10 client.0 _ll_register_callbacks cb 0x5603f6002c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:33:08.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.760+0000 7fb97b400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603f6139470 0x5603f6143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:08.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.760+0000 7fb97b400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603f6139470 0x5603f6143640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:55626/0 (socket says 172.21.15.19:55626) 2024-05-07T02:33:08.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.760+0000 7fb97b400700 1 -- 172.21.15.19:0/4153098812 learned_addr learned my addr 172.21.15.19:0/4153098812 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:08.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.760+0000 7fb97be00700 1 --2- 172.21.15.19:0/4153098812 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5603f613f3b0 0x5603f61440c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:08.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.760+0000 7fb97aa00700 1 --2- 172.21.15.19:0/4153098812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603f6139dc0 0x5603f6143b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:08.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.760+0000 7fb97b400700 1 -- 172.21.15.19:0/4153098812 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5603f613f3b0 msgr2=0x5603f61440c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:08.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.760+0000 7fb97b400700 1 --2- 172.21.15.19:0/4153098812 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5603f613f3b0 0x5603f61440c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:08.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.760+0000 7fb97b400700 1 -- 172.21.15.19:0/4153098812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603f6139dc0 msgr2=0x5603f6143b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:08.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.760+0000 7fb97b400700 1 --2- 172.21.15.19:0/4153098812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603f6139dc0 0x5603f6143b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:08.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.760+0000 7fb97b400700 1 -- 172.21.15.19:0/4153098812 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603f6021420 con 0x5603f6139470 2024-05-07T02:33:08.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.760+0000 7fb97aa00700 1 --2- 172.21.15.19:0/4153098812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603f6139dc0 0x5603f6143b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-07T02:33:08.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.761+0000 7fb97b400700 1 --2- 172.21.15.19:0/4153098812 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603f6139470 0x5603f6143640 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0e2f50 tx=0x7fb96c018b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16b08925e5366fe9 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:08.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.761+0000 7fb969600700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:33:08.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.761+0000 7fb969600700 1 -- 172.21.15.19:0/4153098812 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c0c2060 con 0x5603f6139470 2024-05-07T02:33:08.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.761+0000 7fb990de4b80 1 -- 172.21.15.19:0/4153098812 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb96c08fe50 con 0x5603f6139470 2024-05-07T02:33:08.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.761+0000 7fb969600700 1 -- 172.21.15.19:0/4153098812 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fb96c0a9b90 con 0x5603f6139470 2024-05-07T02:33:08.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.761+0000 7fb969600700 1 -- 172.21.15.19:0/4153098812 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb96c0adce0 con 0x5603f6139470 2024-05-07T02:33:08.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.761+0000 7fb990de4b80 10 client.25526 fetch_fsmap learned FSMap version 31 2024-05-07T02:33:08.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.761+0000 7fb990de4b80 1 -- 172.21.15.19:0/4153098812 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5603f6151430 con 0x5603f6139470 2024-05-07T02:33:08.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.761+0000 7fb969600700 1 -- 172.21.15.19:0/4153098812 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb96c08fe50 con 0x5603f6139470 2024-05-07T02:33:08.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.762+0000 7fb969600700 1 -- 172.21.15.19:0/4153098812 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fb96c0e0150 con 0x5603f6139470 2024-05-07T02:33:08.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.762+0000 7fb990de4b80 10 client.25526 fetch_fsmap finished waiting for FSMap version 31 2024-05-07T02:33:08.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.762+0000 7fb990de4b80 10 client.25526 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-07T02:33:08.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.762+0000 7fb990de4b80 20 client.25526 populate_metadata read hostname 'smithi019' 2024-05-07T02:33:08.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.762+0000 7fb990de4b80 1 --2- 172.21.15.19:0/4153098812 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5603f6153900 0x5603f6155d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:08.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.762+0000 7fb990de4b80 4 client.25526 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:33:08.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.762+0000 7fb990de4b80 1 -- 172.21.15.19:0/4153098812 --> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603f61062e0 con 0x5603f6153900 2024-05-07T02:33:08.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.762+0000 7fb97aa00700 1 --2- 172.21.15.19:0/4153098812 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5603f6153900 0x5603f6155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:08.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.763+0000 7fb97aa00700 1 --2- 172.21.15.19:0/4153098812 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5603f6153900 0x5603f6155d20 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fb9e2001a61a1ad0 server_cookie=fe27e25f606224f1 in_seq=0 out_seq=0 2024-05-07T02:33:08.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.763+0000 7fb969600700 10 client.25526 ms_handle_connect on v2:172.21.15.19:6840/3905645246 2024-05-07T02:33:08.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.764+0000 7fb969600700 1 -- 172.21.15.19:0/4153098812 <== mds.1 v2:172.21.15.19:6840/3905645246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5603f61062e0 con 0x5603f6153900 2024-05-07T02:33:08.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.764+0000 7fb969600700 10 client.25526 handle_command_reply: tid=0 2024-05-07T02:33:08.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.764+0000 7fb990de4b80 1 client.25526 shutdown 2024-05-07T02:33:08.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.765+0000 7fb990de4b80 1 -- 172.21.15.19:0/4153098812 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603f6139470 msgr2=0x5603f6143640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:08.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.765+0000 7fb990de4b80 1 --2- 172.21.15.19:0/4153098812 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603f6139470 0x5603f6143640 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0e2f50 tx=0x7fb96c018b00 comp rx=0 tx=0).stop 2024-05-07T02:33:08.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.765+0000 7fb990de4b80 1 -- 172.21.15.19:0/4153098812 shutdown_connections 2024-05-07T02:33:08.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.765+0000 7fb990de4b80 1 --2- 172.21.15.19:0/4153098812 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5603f6153900 0x5603f6155d20 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:08.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.765+0000 7fb990de4b80 1 --2- 172.21.15.19:0/4153098812 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5603f613f3b0 0x5603f61440c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:08.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.765+0000 7fb990de4b80 1 --2- 172.21.15.19:0/4153098812 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603f6139470 0x5603f6143640 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:08.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.765+0000 7fb990de4b80 1 --2- 172.21.15.19:0/4153098812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603f6139dc0 0x5603f6143b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:08.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.765+0000 7fb990de4b80 1 -- 172.21.15.19:0/4153098812 >> 172.21.15.19:0/4153098812 conn(0x5603f5fbbe40 msgr2=0x5603f6138460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:08.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.765+0000 7fb969600700 0 client.25526 ms_handle_reset on v2:172.21.15.19:6840/3905645246 2024-05-07T02:33:08.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.765+0000 7fb990de4b80 1 -- 172.21.15.19:0/4153098812 shutdown_connections 2024-05-07T02:33:08.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.765+0000 7fb990de4b80 1 -- 172.21.15.19:0/4153098812 wait complete. 2024-05-07T02:33:08.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.765+0000 7fb990de4b80 20 client.25526 trim_cache size 0 max 16384 2024-05-07T02:33:08.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.783+0000 7fb990de4b80 1 Processor -- start 2024-05-07T02:33:08.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.783+0000 7fb990de4b80 1 -- start start 2024-05-07T02:33:08.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.783+0000 7fb990de4b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603f6139470 0x5603f614fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:08.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.783+0000 7fb990de4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5603f6139dc0 0x5603f6141000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:08.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.783+0000 7fb990de4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603f613f3b0 0x5603f6143600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:08.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.783+0000 7fb990de4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c1b6c30 con 0x5603f613f3b0 2024-05-07T02:33:08.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.783+0000 7fb990de4b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c1b8770 con 0x5603f6139470 2024-05-07T02:33:08.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.783+0000 7fb990de4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c1b7300 con 0x5603f6139dc0 2024-05-07T02:33:08.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.784+0000 7fb97b400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603f6139470 0x5603f614fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:08.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.784+0000 7fb97aa00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5603f6139dc0 0x5603f6141000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:08.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.784+0000 7fb97b400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603f6139470 0x5603f614fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:55636/0 (socket says 172.21.15.19:55636) 2024-05-07T02:33:08.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.784+0000 7fb97aa00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5603f6139dc0 0x5603f6141000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:46906/0 (socket says 172.21.15.19:46906) 2024-05-07T02:33:08.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.784+0000 7fb97b400700 1 -- 172.21.15.19:0/85107533 learned_addr learned my addr 172.21.15.19:0/85107533 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:08.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.784+0000 7fb97be00700 1 --2- 172.21.15.19:0/85107533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603f613f3b0 0x5603f6143600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:08.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.784+0000 7fb97aa00700 1 -- 172.21.15.19:0/85107533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603f6139470 msgr2=0x5603f614fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:08.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.784+0000 7fb97aa00700 1 --2- 172.21.15.19:0/85107533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603f6139470 0x5603f614fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:08.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.784+0000 7fb97aa00700 1 -- 172.21.15.19:0/85107533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603f613f3b0 msgr2=0x5603f6143600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:08.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.784+0000 7fb97aa00700 1 --2- 172.21.15.19:0/85107533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603f613f3b0 0x5603f6143600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:08.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.784+0000 7fb97aa00700 1 -- 172.21.15.19:0/85107533 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603f613a1d0 con 0x5603f6139dc0 2024-05-07T02:33:08.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.784+0000 7fb97be00700 1 --2- 172.21.15.19:0/85107533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603f613f3b0 0x5603f6143600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:33:08.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.785+0000 7fb97aa00700 1 --2- 172.21.15.19:0/85107533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5603f6139dc0 0x5603f6141000 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x5603f61468d0 tx=0x7fb964014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c8a04fcc0750230 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:08.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.785+0000 7fb96be00700 1 -- 172.21.15.19:0/85107533 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb964004830 con 0x5603f6139dc0 2024-05-07T02:33:08.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.785+0000 7fb96be00700 1 -- 172.21.15.19:0/85107533 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb964005020 con 0x5603f6139dc0 2024-05-07T02:33:08.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.785+0000 7fb96be00700 1 -- 172.21.15.19:0/85107533 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb964003680 con 0x5603f6139dc0 2024-05-07T02:33:08.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.786+0000 7fb990de4b80 1 -- 172.21.15.19:0/85107533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5603f6139dc0 msgr2=0x5603f6141000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:08.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.786+0000 7fb990de4b80 1 --2- 172.21.15.19:0/85107533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5603f6139dc0 0x5603f6141000 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x5603f61468d0 tx=0x7fb964014000 comp rx=0 tx=0).stop 2024-05-07T02:33:08.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.786+0000 7fb990de4b80 1 -- 172.21.15.19:0/85107533 shutdown_connections 2024-05-07T02:33:08.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.786+0000 7fb990de4b80 1 --2- 172.21.15.19:0/85107533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603f613f3b0 0x5603f6143600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:08.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.786+0000 7fb990de4b80 1 --2- 172.21.15.19:0/85107533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603f6139470 0x5603f614fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:08.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.786+0000 7fb990de4b80 1 --2- 172.21.15.19:0/85107533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5603f6139dc0 0x5603f6141000 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:08.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.786+0000 7fb990de4b80 1 -- 172.21.15.19:0/85107533 >> 172.21.15.19:0/85107533 conn(0x5603f5fbbe40 msgr2=0x5603f6133d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:08.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.786+0000 7fb990de4b80 1 -- 172.21.15.19:0/85107533 shutdown_connections 2024-05-07T02:33:08.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.787+0000 7fb990de4b80 1 -- 172.21.15.19:0/85107533 wait complete. 2024-05-07T02:33:08.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.787+0000 7fb990de4b80 1 -- start start 2024-05-07T02:33:08.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.787+0000 7fb990de4b80 1 Processor -- start 2024-05-07T02:33:08.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.788+0000 7fb990de4b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603f6139470 0x5603f6158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:08.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.788+0000 7fb990de4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603f6139dc0 0x5603f61591f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:08.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.788+0000 7fb990de4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5603f613f3b0 0x5603f6156120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:08.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.788+0000 7fb990de4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c1b80a0 con 0x5603f6139dc0 2024-05-07T02:33:08.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.788+0000 7fb990de4b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c1b79d0 con 0x5603f6139470 2024-05-07T02:33:08.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.788+0000 7fb990de4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c1b4bd0 con 0x5603f613f3b0 2024-05-07T02:33:08.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.788+0000 7fb990de4b80 10 client.0 _ll_register_callbacks cb 0x5603f6002c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:33:08.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.788+0000 7fb97b400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603f6139470 0x5603f6158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:08.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.788+0000 7fb97be00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5603f613f3b0 0x5603f6156120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:08.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.788+0000 7fb97b400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603f6139470 0x5603f6158cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:55648/0 (socket says 172.21.15.19:55648) 2024-05-07T02:33:08.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.789+0000 7fb97b400700 1 -- 172.21.15.19:0/373319751 learned_addr learned my addr 172.21.15.19:0/373319751 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:08.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.789+0000 7fb97aa00700 1 --2- 172.21.15.19:0/373319751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603f6139dc0 0x5603f61591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:08.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.789+0000 7fb97b400700 1 -- 172.21.15.19:0/373319751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5603f613f3b0 msgr2=0x5603f6156120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:08.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.789+0000 7fb97b400700 1 --2- 172.21.15.19:0/373319751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5603f613f3b0 0x5603f6156120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:08.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.789+0000 7fb97b400700 1 -- 172.21.15.19:0/373319751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603f6139dc0 msgr2=0x5603f61591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:08.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.789+0000 7fb97b400700 1 --2- 172.21.15.19:0/373319751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603f6139dc0 0x5603f61591f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:08.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.789+0000 7fb97b400700 1 -- 172.21.15.19:0/373319751 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603f6156830 con 0x5603f6139470 2024-05-07T02:33:08.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.789+0000 7fb97b400700 1 --2- 172.21.15.19:0/373319751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603f6139470 0x5603f6158cb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0a59b0 tx=0x7fb96c0a9280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fac2e0b93557b57a server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:08.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.789+0000 7fb969600700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:33:08.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.790+0000 7fb969600700 1 -- 172.21.15.19:0/373319751 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c0c2070 con 0x5603f6139470 2024-05-07T02:33:08.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.790+0000 7fb990de4b80 1 -- 172.21.15.19:0/373319751 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb964005020 con 0x5603f6139470 2024-05-07T02:33:08.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.790+0000 7fb969600700 1 -- 172.21.15.19:0/373319751 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fb96c0b2300 con 0x5603f6139470 2024-05-07T02:33:08.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.790+0000 7fb969600700 1 -- 172.21.15.19:0/373319751 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb96c0a57f0 con 0x5603f6139470 2024-05-07T02:33:08.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.790+0000 7fb990de4b80 10 client.25532 fetch_fsmap learned FSMap version 31 2024-05-07T02:33:08.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.790+0000 7fb990de4b80 1 -- 172.21.15.19:0/373319751 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5603f61bd7d0 con 0x5603f6139470 2024-05-07T02:33:08.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.790+0000 7fb969600700 1 -- 172.21.15.19:0/373319751 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb964005020 con 0x5603f6139470 2024-05-07T02:33:08.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.790+0000 7fb969600700 1 -- 172.21.15.19:0/373319751 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fb96c0e0150 con 0x5603f6139470 2024-05-07T02:33:08.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.790+0000 7fb990de4b80 10 client.25532 fetch_fsmap finished waiting for FSMap version 31 2024-05-07T02:33:08.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.791+0000 7fb990de4b80 10 client.25532 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-07T02:33:08.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.791+0000 7fb990de4b80 20 client.25532 populate_metadata read hostname 'smithi019' 2024-05-07T02:33:08.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.791+0000 7fb990de4b80 1 --2- 172.21.15.19:0/373319751 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5603f61bdd70 0x5603f61c0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:08.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.791+0000 7fb990de4b80 4 client.25532 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:33:08.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.791+0000 7fb990de4b80 1 -- 172.21.15.19:0/373319751 --> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5603f6069400 con 0x5603f61bdd70 2024-05-07T02:33:08.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.791+0000 7fb97aa00700 1 --2- 172.21.15.19:0/373319751 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5603f61bdd70 0x5603f61c0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:08.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.792+0000 7fb97aa00700 1 --2- 172.21.15.19:0/373319751 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5603f61bdd70 0x5603f61c0190 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=10db9fc6423f15d3 server_cookie=6728f301c631b4fe in_seq=0 out_seq=0 2024-05-07T02:33:08.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.792+0000 7fb969600700 10 client.25532 ms_handle_connect on v2:172.21.15.19:6840/3905645246 2024-05-07T02:33:08.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.792+0000 7fb969600700 1 -- 172.21.15.19:0/373319751 <== mds.1 v2:172.21.15.19:6840/3905645246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5603f6069400 con 0x5603f61bdd70 2024-05-07T02:33:08.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.792+0000 7fb969600700 10 client.25532 handle_command_reply: tid=0 2024-05-07T02:33:08.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.792+0000 7fb990de4b80 1 client.25532 shutdown 2024-05-07T02:33:08.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.792+0000 7fb990de4b80 1 -- 172.21.15.19:0/373319751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603f6139470 msgr2=0x5603f6158cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:08.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.792+0000 7fb990de4b80 1 --2- 172.21.15.19:0/373319751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603f6139470 0x5603f6158cb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0a59b0 tx=0x7fb96c0a9280 comp rx=0 tx=0).stop 2024-05-07T02:33:08.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.792+0000 7fb990de4b80 1 -- 172.21.15.19:0/373319751 shutdown_connections 2024-05-07T02:33:08.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.792+0000 7fb990de4b80 1 --2- 172.21.15.19:0/373319751 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5603f61bdd70 0x5603f61c0190 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:08.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.792+0000 7fb990de4b80 1 --2- 172.21.15.19:0/373319751 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5603f613f3b0 0x5603f6156120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:08.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.792+0000 7fb990de4b80 1 --2- 172.21.15.19:0/373319751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5603f6139470 0x5603f6158cb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:08.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.792+0000 7fb990de4b80 1 --2- 172.21.15.19:0/373319751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603f6139dc0 0x5603f61591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:08.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.792+0000 7fb969600700 0 client.25532 ms_handle_reset on v2:172.21.15.19:6840/3905645246 2024-05-07T02:33:08.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.792+0000 7fb990de4b80 1 -- 172.21.15.19:0/373319751 >> 172.21.15.19:0/373319751 conn(0x5603f5fbbe40 msgr2=0x5603f6138460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:08.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.793+0000 7fb990de4b80 1 -- 172.21.15.19:0/373319751 shutdown_connections 2024-05-07T02:33:08.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.793+0000 7fb990de4b80 1 -- 172.21.15.19:0/373319751 wait complete. 2024-05-07T02:33:08.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.793+0000 7fb990de4b80 20 client.25532 trim_cache size 0 max 16384 2024-05-07T02:33:08.794 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:33:08.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.794+0000 7fb982200700 1 -- 172.21.15.19:0/2897649107 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fb9600b2b10 msgr2=0x7fb9600b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:08.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.794+0000 7fb982200700 1 --2- 172.21.15.19:0/2897649107 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fb9600b2b10 0x7fb9600b5020 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb97c06aab0 tx=0x7fb964009260 comp rx=0 tx=0).stop 2024-05-07T02:33:08.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.794+0000 7fb982200700 1 -- 172.21.15.19:0/2897649107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb97c10a950 msgr2=0x7fb97c1a14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:08.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.794+0000 7fb982200700 1 --2- 172.21.15.19:0/2897649107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb97c10a950 0x7fb97c1a14d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb96c005fd0 tx=0x7fb96c00eca0 comp rx=0 tx=0).stop 2024-05-07T02:33:08.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.794+0000 7fb982200700 1 -- 172.21.15.19:0/2897649107 shutdown_connections 2024-05-07T02:33:08.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.794+0000 7fb982200700 1 --2- 172.21.15.19:0/2897649107 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb97c110980 0x7fb97c1a5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:08.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.794+0000 7fb982200700 1 --2- 172.21.15.19:0/2897649107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb97c10a950 0x7fb97c1a14d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:08.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.794+0000 7fb982200700 1 --2- 172.21.15.19:0/2897649107 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fb9600b2b10 0x7fb9600b5020 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:08.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.794+0000 7fb982200700 1 --2- 172.21.15.19:0/2897649107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb97c10b2a0 0x7fb97c1a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:08.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.794+0000 7fb982200700 1 -- 172.21.15.19:0/2897649107 >> 172.21.15.19:0/2897649107 conn(0x7fb97c076d30 msgr2=0x7fb97c1088c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:08.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.794+0000 7fb982200700 1 -- 172.21.15.19:0/2897649107 shutdown_connections 2024-05-07T02:33:08.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:08.794+0000 7fb982200700 1 -- 172.21.15.19:0/2897649107 wait complete. 2024-05-07T02:33:09.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:08 smithi019 ceph-mon[29781]: pgmap v42: 129 pgs: 129 active+clean; 2.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 131 KiB/s wr, 39 op/s 2024-05-07T02:33:09.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:08 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:33:09.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:08 smithi027 ceph-mon[34398]: pgmap v42: 129 pgs: 129 active+clean; 2.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 131 KiB/s wr, 39 op/s 2024-05-07T02:33:09.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:08 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:33:09.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:08 smithi064 ceph-mon[34699]: pgmap v42: 129 pgs: 129 active+clean; 2.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 131 KiB/s wr, 39 op/s 2024-05-07T02:33:09.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:08 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:33:09.720 INFO:tasks.workunit.client.0.smithi019.stdout: 1 10958 15.60 MB/sec warmup 5 sec latency 30.174 ms 2024-05-07T02:33:09.848 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-07T02:33:10.410 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:33:10.720 INFO:tasks.workunit.client.0.smithi019.stdout: 1 13067 14.23 MB/sec warmup 6 sec latency 19.466 ms 2024-05-07T02:33:11.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:10 smithi019 ceph-mon[29781]: pgmap v43: 129 pgs: 129 active+clean; 2.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 131 KiB/s wr, 38 op/s 2024-05-07T02:33:11.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:10 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:33:11.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:10 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:33:11.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:10 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:33:11.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:10 smithi027 ceph-mon[34398]: pgmap v43: 129 pgs: 129 active+clean; 2.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 131 KiB/s wr, 38 op/s 2024-05-07T02:33:11.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:10 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:33:11.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:10 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:33:11.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:10 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:33:11.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:10 smithi064 ceph-mon[34699]: pgmap v43: 129 pgs: 129 active+clean; 2.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 131 KiB/s wr, 38 op/s 2024-05-07T02:33:11.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:10 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:33:11.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:10 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:33:11.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:10 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:33:11.720 INFO:tasks.workunit.client.0.smithi019.stdout: 1 14917 13.95 MB/sec warmup 7 sec latency 12.065 ms 2024-05-07T02:33:12.720 INFO:tasks.workunit.client.0.smithi019.stdout: 1 17404 13.84 MB/sec warmup 8 sec latency 10.707 ms 2024-05-07T02:33:12.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:12.995+0000 7f89ba600700 1 -- 172.21.15.19:0/288931031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89b4108f20 msgr2=0x7f89b4109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:12.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:12.995+0000 7f89ba600700 1 --2- 172.21.15.19:0/288931031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89b4108f20 0x7f89b4109330 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f899c009990 tx=0x7f899c009ca0 comp rx=0 tx=0).stop 2024-05-07T02:33:12.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:12.995+0000 7f89ba600700 1 -- 172.21.15.19:0/288931031 shutdown_connections 2024-05-07T02:33:12.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:12.995+0000 7f89ba600700 1 --2- 172.21.15.19:0/288931031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89b4109870 0x7f89b41111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:12.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:12.995+0000 7f89ba600700 1 --2- 172.21.15.19:0/288931031 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89b41045f0 0x7f89b4104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:12.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:12.995+0000 7f89ba600700 1 --2- 172.21.15.19:0/288931031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89b4108f20 0x7f89b4109330 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:12.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:12.995+0000 7f89ba600700 1 -- 172.21.15.19:0/288931031 >> 172.21.15.19:0/288931031 conn(0x7f89b4100040 msgr2=0x7f89b4102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:12.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:12.995+0000 7f89ba600700 1 -- 172.21.15.19:0/288931031 shutdown_connections 2024-05-07T02:33:12.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:12.995+0000 7f89ba600700 1 -- 172.21.15.19:0/288931031 wait complete. 2024-05-07T02:33:12.996 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:12 smithi019 ceph-mon[29781]: pgmap v44: 129 pgs: 129 active+clean; 8.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 702 KiB/s wr, 104 op/s 2024-05-07T02:33:12.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:12.997+0000 7f89ba600700 1 Processor -- start 2024-05-07T02:33:12.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:12.997+0000 7f89ba600700 1 -- start start 2024-05-07T02:33:12.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:12.998+0000 7f89ba600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89b41045f0 0x7f89b419d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:12.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:12.998+0000 7f89ba600700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89b4108f20 0x7f89b419d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:12.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:12.998+0000 7f89ba600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89b4109870 0x7f89b41a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:12.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:12.998+0000 7f89ba600700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f89b4114480 con 0x7f89b4109870 2024-05-07T02:33:12.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:12.998+0000 7f89ba600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f89b4114300 con 0x7f89b41045f0 2024-05-07T02:33:12.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:12.998+0000 7f89ba600700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f89b4114600 con 0x7f89b4108f20 2024-05-07T02:33:12.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:12.998+0000 7f89b3400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89b41045f0 0x7f89b419d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:12.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:12.998+0000 7f89b3400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89b41045f0 0x7f89b419d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:56298/0 (socket says 172.21.15.19:56298) 2024-05-07T02:33:12.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:12.998+0000 7f89b3400700 1 -- 172.21.15.19:0/2035876666 learned_addr learned my addr 172.21.15.19:0/2035876666 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:13.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:12.999+0000 7f89aaa00700 1 --2- 172.21.15.19:0/2035876666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89b4108f20 0x7f89b419d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:13.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:12.999+0000 7f89b3400700 1 -- 172.21.15.19:0/2035876666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89b4108f20 msgr2=0x7f89b419d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:13.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:12.999+0000 7f89b3400700 1 --2- 172.21.15.19:0/2035876666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89b4108f20 0x7f89b419d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:13.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:12.999+0000 7f89b3e00700 1 --2- 172.21.15.19:0/2035876666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89b4109870 0x7f89b41a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:13.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:12.999+0000 7f89b3400700 1 -- 172.21.15.19:0/2035876666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89b4109870 msgr2=0x7f89b41a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:13.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:12.999+0000 7f89b3400700 1 --2- 172.21.15.19:0/2035876666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89b4109870 0x7f89b41a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:13.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:12.999+0000 7f89b3400700 1 -- 172.21.15.19:0/2035876666 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b41a1f20 con 0x7f89b41045f0 2024-05-07T02:33:13.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:12.999+0000 7f89b3e00700 1 --2- 172.21.15.19:0/2035876666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89b4109870 0x7f89b41a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-07T02:33:13.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:12.999+0000 7f89b3400700 1 --2- 172.21.15.19:0/2035876666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89b41045f0 0x7f89b419d050 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f89a400e9a0 tx=0x7f89a400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:13.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.000+0000 7f89b0c00700 1 -- 172.21.15.19:0/2035876666 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a400cbd0 con 0x7f89b41045f0 2024-05-07T02:33:13.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.000+0000 7f89b0c00700 1 -- 172.21.15.19:0/2035876666 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89a400cd70 con 0x7f89b41045f0 2024-05-07T02:33:13.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.000+0000 7f89ba600700 1 -- 172.21.15.19:0/2035876666 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89b41a2210 con 0x7f89b41045f0 2024-05-07T02:33:13.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.000+0000 7f89b0c00700 1 -- 172.21.15.19:0/2035876666 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a4018680 con 0x7f89b41045f0 2024-05-07T02:33:13.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.000+0000 7f89ba600700 1 -- 172.21.15.19:0/2035876666 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89b41a9b30 con 0x7f89b41045f0 2024-05-07T02:33:13.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.004+0000 7f89b0c00700 1 -- 172.21.15.19:0/2035876666 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f89a4010430 con 0x7f89b41045f0 2024-05-07T02:33:13.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.005+0000 7f89c9173b80 1 Processor -- start 2024-05-07T02:33:13.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.006+0000 7f89c9173b80 1 -- start start 2024-05-07T02:33:13.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.006+0000 7f89c9173b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555aa8669470 0x555aa8669880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:13.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.006+0000 7f89c9173b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555aa8663620 0x555aa8663a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:13.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.006+0000 7f89c9173b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555aa8663f70 0x555aa8670150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:13.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.006+0000 7f89c9173b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555aa8585a20 con 0x555aa8663f70 2024-05-07T02:33:13.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.006+0000 7f89c9173b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x555aa8523760 con 0x555aa8669470 2024-05-07T02:33:13.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.006+0000 7f89c9173b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555aa85819e0 con 0x555aa8663620 2024-05-07T02:33:13.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.007+0000 7f89b0c00700 1 --2- 172.21.15.19:0/2035876666 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f89980b2b90 0x7f89980b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:13.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.007+0000 7f89b3e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555aa8669470 0x555aa8669880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:13.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.007+0000 7f89aaa00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555aa8663f70 0x555aa8670150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:13.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.007+0000 7f89b3e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555aa8669470 0x555aa8669880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:56304/0 (socket says 172.21.15.19:56304) 2024-05-07T02:33:13.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.007+0000 7f89b3e00700 1 -- 172.21.15.19:0/2145380970 learned_addr learned my addr 172.21.15.19:0/2145380970 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:13.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.007+0000 7f89b0c00700 1 -- 172.21.15.19:0/2035876666 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f89a40e2ed0 con 0x7f89b41045f0 2024-05-07T02:33:13.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.007+0000 7f89b3400700 1 --2- 172.21.15.19:0/2145380970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555aa8663620 0x555aa8663a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:13.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.007+0000 7f89b3e00700 1 --2- 172.21.15.19:0/2035876666 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f89980b2b90 0x7f89980b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:13.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.007+0000 7f89b3e00700 1 -- 172.21.15.19:0/2145380970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555aa8663620 msgr2=0x555aa8663a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:13.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.007+0000 7f89b3e00700 1 --2- 172.21.15.19:0/2145380970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555aa8663620 0x555aa8663a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:13.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.007+0000 7f89b3e00700 1 -- 172.21.15.19:0/2145380970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555aa8663f70 msgr2=0x555aa8670150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:13.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.007+0000 7f89b3e00700 1 --2- 172.21.15.19:0/2145380970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555aa8663f70 0x555aa8670150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:13.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.007+0000 7f89b3e00700 1 -- 172.21.15.19:0/2145380970 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555aa84d28d0 con 0x555aa8669470 2024-05-07T02:33:13.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.008+0000 7f89b3e00700 1 --2- 172.21.15.19:0/2145380970 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555aa8669470 0x555aa8669880 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f89ac007900 tx=0x7f89ac007c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c910485573bc07f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:13.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.008+0000 7f89a8c00700 1 -- 172.21.15.19:0/2145380970 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac01d070 con 0x555aa8669470 2024-05-07T02:33:13.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.008+0000 7f89a8c00700 1 -- 172.21.15.19:0/2145380970 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89ac013040 con 0x555aa8669470 2024-05-07T02:33:13.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.008+0000 7f89a8c00700 1 -- 172.21.15.19:0/2145380970 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac012050 con 0x555aa8669470 2024-05-07T02:33:13.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.009+0000 7f89c9173b80 1 -- 172.21.15.19:0/2145380970 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555aa8669470 msgr2=0x555aa8669880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:13.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.009+0000 7f89c9173b80 1 --2- 172.21.15.19:0/2145380970 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555aa8669470 0x555aa8669880 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f89ac007900 tx=0x7f89ac007c10 comp rx=0 tx=0).stop 2024-05-07T02:33:13.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.009+0000 7f89b3e00700 1 --2- 172.21.15.19:0/2035876666 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f89980b2b90 0x7f89980b50a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f89ac01f000 tx=0x7f89ac01d000 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:13.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.009+0000 7f89c9173b80 1 -- 172.21.15.19:0/2145380970 shutdown_connections 2024-05-07T02:33:13.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.009+0000 7f89c9173b80 1 --2- 172.21.15.19:0/2145380970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555aa8663f70 0x555aa8670150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:13.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.009+0000 7f89c9173b80 1 --2- 172.21.15.19:0/2145380970 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555aa8669470 0x555aa8669880 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:13.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.009+0000 7f89c9173b80 1 --2- 172.21.15.19:0/2145380970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555aa8663620 0x555aa8663a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:13.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.009+0000 7f89c9173b80 1 -- 172.21.15.19:0/2145380970 >> 172.21.15.19:0/2145380970 conn(0x555aa84e5e40 msgr2=0x555aa865e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:13.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.009+0000 7f89c9173b80 1 -- 172.21.15.19:0/2145380970 shutdown_connections 2024-05-07T02:33:13.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.009+0000 7f89c9173b80 1 -- 172.21.15.19:0/2145380970 wait complete. 2024-05-07T02:33:13.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.010+0000 7f89c9173b80 1 -- start start 2024-05-07T02:33:13.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.010+0000 7f89c9173b80 1 Processor -- start 2024-05-07T02:33:13.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.011+0000 7f89c9173b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555aa8663620 0x555aa866d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:13.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.011+0000 7f89c9173b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555aa8663f70 0x555aa866db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:13.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.011+0000 7f89c9173b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555aa8669470 0x555aa866e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:13.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.011+0000 7f89c9173b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555aa8670840 con 0x555aa8663f70 2024-05-07T02:33:13.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.011+0000 7f89c9173b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x555aa84ebb20 con 0x555aa8663620 2024-05-07T02:33:13.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.011+0000 7f89c9173b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555aa86709c0 con 0x555aa8669470 2024-05-07T02:33:13.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.011+0000 7f89c9173b80 10 client.0 _ll_register_callbacks cb 0x555aa8519380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:33:13.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.011+0000 7f89b3e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555aa8669470 0x555aa866e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:13.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.011+0000 7f89aaa00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555aa8663f70 0x555aa866db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:13.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.011+0000 7f89b3400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555aa8663620 0x555aa866d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:13.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.011+0000 7f89b3e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555aa8669470 0x555aa866e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:32828/0 (socket says 172.21.15.19:32828) 2024-05-07T02:33:13.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.011+0000 7f89b3e00700 1 -- 172.21.15.19:0/3246758405 learned_addr learned my addr 172.21.15.19:0/3246758405 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:13.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.012+0000 7f89b3e00700 1 -- 172.21.15.19:0/3246758405 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555aa8663620 msgr2=0x555aa866d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:13.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.012+0000 7f89b3e00700 1 --2- 172.21.15.19:0/3246758405 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555aa8663620 0x555aa866d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:13.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.012+0000 7f89b3e00700 1 -- 172.21.15.19:0/3246758405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555aa8663f70 msgr2=0x555aa866db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:13.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.012+0000 7f89b3e00700 1 --2- 172.21.15.19:0/3246758405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555aa8663f70 0x555aa866db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:13.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.012+0000 7f89b3e00700 1 -- 172.21.15.19:0/3246758405 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555aa854b420 con 0x555aa8669470 2024-05-07T02:33:13.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.012+0000 7f89aaa00700 1 --2- 172.21.15.19:0/3246758405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555aa8663f70 0x555aa866db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:33:13.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.012+0000 7f89b3400700 1 --2- 172.21.15.19:0/3246758405 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555aa8663620 0x555aa866d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:33:13.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.012+0000 7f89b3e00700 1 --2- 172.21.15.19:0/3246758405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555aa8669470 0x555aa866e060 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f89ac015f90 tx=0x7f89ac022b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39262fbb0f773af9 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:13.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.012+0000 7f89a1600700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:33:13.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.012+0000 7f89a1600700 1 -- 172.21.15.19:0/3246758405 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac019670 con 0x555aa8669470 2024-05-07T02:33:13.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.013+0000 7f89c9173b80 1 -- 172.21.15.19:0/3246758405 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89ac013040 con 0x555aa8669470 2024-05-07T02:33:13.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.013+0000 7f89a1600700 1 -- 172.21.15.19:0/3246758405 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f89ac019810 con 0x555aa8669470 2024-05-07T02:33:13.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.013+0000 7f89a1600700 1 -- 172.21.15.19:0/3246758405 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f89ac039050 con 0x555aa8669470 2024-05-07T02:33:13.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.013+0000 7f89c9173b80 10 client.25528 fetch_fsmap learned FSMap version 31 2024-05-07T02:33:13.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.013+0000 7f89c9173b80 1 -- 172.21.15.19:0/3246758405 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555aa867b440 con 0x555aa8669470 2024-05-07T02:33:13.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.013+0000 7f89a1600700 1 -- 172.21.15.19:0/3246758405 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89ac013040 con 0x555aa8669470 2024-05-07T02:33:13.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.013+0000 7f89a1600700 1 -- 172.21.15.19:0/3246758405 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f89ac018070 con 0x555aa8669470 2024-05-07T02:33:13.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.013+0000 7f89c9173b80 10 client.25528 fetch_fsmap finished waiting for FSMap version 31 2024-05-07T02:33:13.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.013+0000 7f89c9173b80 10 client.25528 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:33:13.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.013+0000 7f89c9173b80 20 client.25528 populate_metadata read hostname 'smithi019' 2024-05-07T02:33:13.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.014+0000 7f89c9173b80 1 --2- 172.21.15.19:0/3246758405 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x555aa867da20 0x555aa867fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:13.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.014+0000 7f89c9173b80 4 client.25528 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:33:13.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.014+0000 7f89c9173b80 1 -- 172.21.15.19:0/3246758405 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555aa86300e0 con 0x555aa867da20 2024-05-07T02:33:13.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.014+0000 7f89b3400700 1 --2- 172.21.15.19:0/3246758405 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x555aa867da20 0x555aa867fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:13.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.014+0000 7f89b3400700 1 --2- 172.21.15.19:0/3246758405 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x555aa867da20 0x555aa867fe40 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3112c612cac9b021 server_cookie=9fd68fd8ffa5dad in_seq=0 out_seq=0 2024-05-07T02:33:13.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.014+0000 7f89a1600700 10 client.25528 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:33:13.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.015+0000 7f89a1600700 1 -- 172.21.15.19:0/3246758405 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555aa86300e0 con 0x555aa867da20 2024-05-07T02:33:13.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.015+0000 7f89a1600700 10 client.25528 handle_command_reply: tid=0 2024-05-07T02:33:13.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.015+0000 7f89c9173b80 1 client.25528 shutdown 2024-05-07T02:33:13.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.015+0000 7f89c9173b80 1 -- 172.21.15.19:0/3246758405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555aa8669470 msgr2=0x555aa866e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:13.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.015+0000 7f89c9173b80 1 --2- 172.21.15.19:0/3246758405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555aa8669470 0x555aa866e060 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f89ac015f90 tx=0x7f89ac022b20 comp rx=0 tx=0).stop 2024-05-07T02:33:13.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.015+0000 7f89c9173b80 1 -- 172.21.15.19:0/3246758405 shutdown_connections 2024-05-07T02:33:13.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.015+0000 7f89c9173b80 1 --2- 172.21.15.19:0/3246758405 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x555aa867da20 0x555aa867fe40 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:13.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.015+0000 7f89c9173b80 1 --2- 172.21.15.19:0/3246758405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555aa8669470 0x555aa866e060 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:13.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.015+0000 7f89c9173b80 1 --2- 172.21.15.19:0/3246758405 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555aa8663620 0x555aa866d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:13.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.015+0000 7f89c9173b80 1 --2- 172.21.15.19:0/3246758405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555aa8663f70 0x555aa866db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:13.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.015+0000 7f89a1600700 0 client.25528 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:33:13.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.015+0000 7f89c9173b80 1 -- 172.21.15.19:0/3246758405 >> 172.21.15.19:0/3246758405 conn(0x555aa84e5e40 msgr2=0x555aa8662610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:13.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.016+0000 7f89c9173b80 1 -- 172.21.15.19:0/3246758405 shutdown_connections 2024-05-07T02:33:13.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.016+0000 7f89c9173b80 1 -- 172.21.15.19:0/3246758405 wait complete. 2024-05-07T02:33:13.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.016+0000 7f89c9173b80 20 client.25528 trim_cache size 0 max 16384 2024-05-07T02:33:13.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.034+0000 7f89c9173b80 1 Processor -- start 2024-05-07T02:33:13.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.034+0000 7f89c9173b80 1 -- start start 2024-05-07T02:33:13.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.034+0000 7f89c9173b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555aa8663620 0x555aa8679c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:13.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.034+0000 7f89c9173b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555aa8663f70 0x555aa8667d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:13.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.034+0000 7f89c9173b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555aa8669470 0x555aa866dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:13.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.034+0000 7f89c9173b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f89b41b2850 con 0x555aa8663f70 2024-05-07T02:33:13.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.034+0000 7f89c9173b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f89b41b4390 con 0x555aa8663620 2024-05-07T02:33:13.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.034+0000 7f89c9173b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f89b41b2f20 con 0x555aa8669470 2024-05-07T02:33:13.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.035+0000 7f89b3e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555aa8669470 0x555aa866dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:13.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.035+0000 7f89b3400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555aa8663620 0x555aa8679c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:13.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.035+0000 7f89aaa00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555aa8663f70 0x555aa8667d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:13.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.035+0000 7f89b3e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555aa8669470 0x555aa866dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:32830/0 (socket says 172.21.15.19:32830) 2024-05-07T02:33:13.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.035+0000 7f89b3400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555aa8663620 0x555aa8679c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:56314/0 (socket says 172.21.15.19:56314) 2024-05-07T02:33:13.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.035+0000 7f89b3e00700 1 -- 172.21.15.19:0/2115448147 learned_addr learned my addr 172.21.15.19:0/2115448147 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:13.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.035+0000 7f89b3e00700 1 -- 172.21.15.19:0/2115448147 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555aa8663620 msgr2=0x555aa8679c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:13.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.035+0000 7f89b3e00700 1 --2- 172.21.15.19:0/2115448147 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555aa8663620 0x555aa8679c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:13.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.035+0000 7f89b3e00700 1 -- 172.21.15.19:0/2115448147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555aa8663f70 msgr2=0x555aa8667d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:13.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.035+0000 7f89b3e00700 1 --2- 172.21.15.19:0/2115448147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555aa8663f70 0x555aa8667d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:13.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.035+0000 7f89b3e00700 1 -- 172.21.15.19:0/2115448147 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555aa865ab80 con 0x555aa8669470 2024-05-07T02:33:13.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.035+0000 7f89b3400700 1 --2- 172.21.15.19:0/2115448147 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555aa8663620 0x555aa8679c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:33:13.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.035+0000 7f89b3e00700 1 --2- 172.21.15.19:0/2115448147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555aa8669470 0x555aa866dcb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f89ac0095e0 tx=0x7f89ac009610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2566ccc86a82751f server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:13.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.036+0000 7f89a3e00700 1 -- 172.21.15.19:0/2115448147 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac041020 con 0x555aa8669470 2024-05-07T02:33:13.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.036+0000 7f89a3e00700 1 -- 172.21.15.19:0/2115448147 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89ac030070 con 0x555aa8669470 2024-05-07T02:33:13.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.036+0000 7f89a3e00700 1 -- 172.21.15.19:0/2115448147 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac012070 con 0x555aa8669470 2024-05-07T02:33:13.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.036+0000 7f89c9173b80 1 -- 172.21.15.19:0/2115448147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555aa8669470 msgr2=0x555aa866dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:13.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.036+0000 7f89c9173b80 1 --2- 172.21.15.19:0/2115448147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555aa8669470 0x555aa866dcb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f89ac0095e0 tx=0x7f89ac009610 comp rx=0 tx=0).stop 2024-05-07T02:33:13.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.037+0000 7f89c9173b80 1 -- 172.21.15.19:0/2115448147 shutdown_connections 2024-05-07T02:33:13.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.037+0000 7f89c9173b80 1 --2- 172.21.15.19:0/2115448147 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555aa8669470 0x555aa866dcb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:13.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.037+0000 7f89c9173b80 1 --2- 172.21.15.19:0/2115448147 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555aa8663620 0x555aa8679c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:13.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.037+0000 7f89c9173b80 1 --2- 172.21.15.19:0/2115448147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555aa8663f70 0x555aa8667d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:13.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.037+0000 7f89c9173b80 1 -- 172.21.15.19:0/2115448147 >> 172.21.15.19:0/2115448147 conn(0x555aa84e5e40 msgr2=0x555aa866d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:13.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.037+0000 7f89c9173b80 1 -- 172.21.15.19:0/2115448147 shutdown_connections 2024-05-07T02:33:13.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.037+0000 7f89c9173b80 1 -- 172.21.15.19:0/2115448147 wait complete. 2024-05-07T02:33:13.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.037+0000 7f89c9173b80 1 -- start start 2024-05-07T02:33:13.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.037+0000 7f89c9173b80 1 Processor -- start 2024-05-07T02:33:13.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.038+0000 7f89c9173b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555aa8663620 0x555aa867fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:13.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.038+0000 7f89c9173b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555aa8663f70 0x555aa8680380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:13.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.038+0000 7f89c9173b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555aa8669470 0x555aa86808c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:13.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.038+0000 7f89c9173b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f89b41b3cc0 con 0x555aa8669470 2024-05-07T02:33:13.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.038+0000 7f89c9173b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f89b41b35f0 con 0x555aa8663f70 2024-05-07T02:33:13.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.038+0000 7f89c9173b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f89b41b07f0 con 0x555aa8663620 2024-05-07T02:33:13.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.038+0000 7f89c9173b80 10 client.0 _ll_register_callbacks cb 0x555aa8519380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:33:13.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.038+0000 7f89aaa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555aa8663f70 0x555aa8680380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:13.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.038+0000 7f89b3400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555aa8663620 0x555aa867fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:13.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.038+0000 7f89b3400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555aa8663620 0x555aa867fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:32846/0 (socket says 172.21.15.19:32846) 2024-05-07T02:33:13.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.038+0000 7f89aaa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555aa8663f70 0x555aa8680380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:56326/0 (socket says 172.21.15.19:56326) 2024-05-07T02:33:13.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.038+0000 7f89b3e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555aa8669470 0x555aa86808c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:13.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.038+0000 7f89b3400700 1 -- 172.21.15.19:0/359035817 learned_addr learned my addr 172.21.15.19:0/359035817 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:13.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.039+0000 7f89b3400700 1 -- 172.21.15.19:0/359035817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555aa8663f70 msgr2=0x555aa8680380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:13.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.039+0000 7f89b3400700 1 --2- 172.21.15.19:0/359035817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555aa8663f70 0x555aa8680380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:13.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.039+0000 7f89b3400700 1 -- 172.21.15.19:0/359035817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555aa8669470 msgr2=0x555aa86808c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:13.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.039+0000 7f89b3400700 1 --2- 172.21.15.19:0/359035817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555aa8669470 0x555aa86808c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:13.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.039+0000 7f89b3400700 1 -- 172.21.15.19:0/359035817 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555aa86e6170 con 0x555aa8663620 2024-05-07T02:33:13.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.039+0000 7f89b3400700 1 --2- 172.21.15.19:0/359035817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555aa8663620 0x555aa867fe40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f89a40e0110 tx=0x7f89a40cc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8995c0f36730377 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:13.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.039+0000 7f89a1600700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:33:13.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.039+0000 7f89a1600700 1 -- 172.21.15.19:0/359035817 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a40a9680 con 0x555aa8663620 2024-05-07T02:33:13.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.039+0000 7f89a1600700 1 -- 172.21.15.19:0/359035817 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f89a40aa000 con 0x555aa8663620 2024-05-07T02:33:13.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.040+0000 7f89c9173b80 1 -- 172.21.15.19:0/359035817 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89ac030070 con 0x555aa8663620 2024-05-07T02:33:13.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.040+0000 7f89a1600700 1 -- 172.21.15.19:0/359035817 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f89a40aa270 con 0x555aa8663620 2024-05-07T02:33:13.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.040+0000 7f89c9173b80 10 client.25534 fetch_fsmap learned FSMap version 31 2024-05-07T02:33:13.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.040+0000 7f89c9173b80 1 -- 172.21.15.19:0/359035817 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555aa86e7430 con 0x555aa8663620 2024-05-07T02:33:13.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.040+0000 7f89a1600700 1 -- 172.21.15.19:0/359035817 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89ac030070 con 0x555aa8663620 2024-05-07T02:33:13.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.040+0000 7f89a1600700 1 -- 172.21.15.19:0/359035817 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f89a40a54e0 con 0x555aa8663620 2024-05-07T02:33:13.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.041+0000 7f89c9173b80 10 client.25534 fetch_fsmap finished waiting for FSMap version 31 2024-05-07T02:33:13.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.041+0000 7f89c9173b80 10 client.25534 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:33:13.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.041+0000 7f89c9173b80 20 client.25534 populate_metadata read hostname 'smithi019' 2024-05-07T02:33:13.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.041+0000 7f89c9173b80 1 --2- 172.21.15.19:0/359035817 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x555aa86e7ae0 0x555aa86e9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:13.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.041+0000 7f89c9173b80 4 client.25534 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:33:13.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.041+0000 7f89c9173b80 1 -- 172.21.15.19:0/359035817 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555aa8593400 con 0x555aa86e7ae0 2024-05-07T02:33:13.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.041+0000 7f89aaa00700 1 --2- 172.21.15.19:0/359035817 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x555aa86e7ae0 0x555aa86e9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:13.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.041+0000 7f89aaa00700 1 --2- 172.21.15.19:0/359035817 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x555aa86e7ae0 0x555aa86e9f00 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84ed7e32743ca5fe server_cookie=4ead6414f857045f in_seq=0 out_seq=0 2024-05-07T02:33:13.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.041+0000 7f89a1600700 10 client.25534 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:33:13.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.042+0000 7f89a1600700 1 -- 172.21.15.19:0/359035817 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555aa8593400 con 0x555aa86e7ae0 2024-05-07T02:33:13.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.042+0000 7f89a1600700 10 client.25534 handle_command_reply: tid=0 2024-05-07T02:33:13.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.042+0000 7f89c9173b80 1 client.25534 shutdown 2024-05-07T02:33:13.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.042+0000 7f89c9173b80 1 -- 172.21.15.19:0/359035817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555aa8663620 msgr2=0x555aa867fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:13.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.042+0000 7f89c9173b80 1 --2- 172.21.15.19:0/359035817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555aa8663620 0x555aa867fe40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f89a40e0110 tx=0x7f89a40cc040 comp rx=0 tx=0).stop 2024-05-07T02:33:13.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.042+0000 7f89c9173b80 1 -- 172.21.15.19:0/359035817 shutdown_connections 2024-05-07T02:33:13.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.042+0000 7f89c9173b80 1 --2- 172.21.15.19:0/359035817 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x555aa86e7ae0 0x555aa86e9f00 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:13.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.042+0000 7f89c9173b80 1 --2- 172.21.15.19:0/359035817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555aa8669470 0x555aa86808c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:13.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.042+0000 7f89c9173b80 1 --2- 172.21.15.19:0/359035817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555aa8663620 0x555aa867fe40 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:13.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.042+0000 7f89c9173b80 1 --2- 172.21.15.19:0/359035817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555aa8663f70 0x555aa8680380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:13.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.042+0000 7f89a1600700 0 client.25534 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:33:13.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.042+0000 7f89c9173b80 1 -- 172.21.15.19:0/359035817 >> 172.21.15.19:0/359035817 conn(0x555aa84e5e40 msgr2=0x555aa8662610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:13.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.042+0000 7f89c9173b80 1 -- 172.21.15.19:0/359035817 shutdown_connections 2024-05-07T02:33:13.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.043+0000 7f89c9173b80 1 -- 172.21.15.19:0/359035817 wait complete. 2024-05-07T02:33:13.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.043+0000 7f89c9173b80 20 client.25534 trim_cache size 0 max 16384 2024-05-07T02:33:13.045 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:33:13.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.043+0000 7f89ba600700 1 -- 172.21.15.19:0/2035876666 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f89980b2b90 msgr2=0x7f89980b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:13.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.043+0000 7f89ba600700 1 --2- 172.21.15.19:0/2035876666 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f89980b2b90 0x7f89980b50a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f89ac01f000 tx=0x7f89ac01d000 comp rx=0 tx=0).stop 2024-05-07T02:33:13.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.043+0000 7f89ba600700 1 -- 172.21.15.19:0/2035876666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89b41045f0 msgr2=0x7f89b419d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:13.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.043+0000 7f89ba600700 1 --2- 172.21.15.19:0/2035876666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89b41045f0 0x7f89b419d050 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f89a400e9a0 tx=0x7f89a400ecb0 comp rx=0 tx=0).stop 2024-05-07T02:33:13.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.044+0000 7f89ba600700 1 -- 172.21.15.19:0/2035876666 shutdown_connections 2024-05-07T02:33:13.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.044+0000 7f89ba600700 1 --2- 172.21.15.19:0/2035876666 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f89980b2b90 0x7f89980b50a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:13.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.044+0000 7f89ba600700 1 --2- 172.21.15.19:0/2035876666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89b4109870 0x7f89b41a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:13.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.044+0000 7f89ba600700 1 --2- 172.21.15.19:0/2035876666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89b41045f0 0x7f89b419d050 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:13.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.044+0000 7f89ba600700 1 --2- 172.21.15.19:0/2035876666 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f89b4108f20 0x7f89b419d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:13.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.044+0000 7f89ba600700 1 -- 172.21.15.19:0/2035876666 >> 172.21.15.19:0/2035876666 conn(0x7f89b4100040 msgr2=0x7f89b410a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:13.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.044+0000 7f89ba600700 1 -- 172.21.15.19:0/2035876666 shutdown_connections 2024-05-07T02:33:13.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:13.044+0000 7f89ba600700 1 -- 172.21.15.19:0/2035876666 wait complete. 2024-05-07T02:33:13.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:12 smithi064 ceph-mon[34699]: pgmap v44: 129 pgs: 129 active+clean; 8.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 702 KiB/s wr, 104 op/s 2024-05-07T02:33:13.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:12 smithi027 ceph-mon[34398]: pgmap v44: 129 pgs: 129 active+clean; 8.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 702 KiB/s wr, 104 op/s 2024-05-07T02:33:13.720 INFO:tasks.workunit.client.0.smithi019.stdout: 1 19144 13.51 MB/sec warmup 9 sec latency 22.206 ms 2024-05-07T02:33:14.720 INFO:tasks.workunit.client.0.smithi019.stdout: 1 21566 13.54 MB/sec warmup 10 sec latency 24.250 ms 2024-05-07T02:33:15.043 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-07T02:33:15.044 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-07T02:33:15.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:14 smithi064 ceph-mon[34699]: pgmap v45: 129 pgs: 129 active+clean; 8.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 680 KiB/s wr, 85 op/s 2024-05-07T02:33:15.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:14 smithi019 ceph-mon[29781]: pgmap v45: 129 pgs: 129 active+clean; 8.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 680 KiB/s wr, 85 op/s 2024-05-07T02:33:15.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:14 smithi027 ceph-mon[34398]: pgmap v45: 129 pgs: 129 active+clean; 8.0 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 680 KiB/s wr, 85 op/s 2024-05-07T02:33:15.607 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:33:15.720 INFO:tasks.workunit.client.0.smithi019.stdout: 1 23150 12.83 MB/sec warmup 11 sec latency 22.480 ms 2024-05-07T02:33:16.721 INFO:tasks.workunit.client.0.smithi019.stdout: 1 25290 12.90 MB/sec warmup 12 sec latency 12.919 ms 2024-05-07T02:33:16.971 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:16 smithi019 ceph-mon[29781]: pgmap v46: 129 pgs: 129 active+clean; 13 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 1.2 MiB/s wr, 149 op/s 2024-05-07T02:33:17.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:16 smithi064 ceph-mon[34699]: pgmap v46: 129 pgs: 129 active+clean; 13 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 1.2 MiB/s wr, 149 op/s 2024-05-07T02:33:17.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:16 smithi027 ceph-mon[34398]: pgmap v46: 129 pgs: 129 active+clean; 13 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 1.2 MiB/s wr, 149 op/s 2024-05-07T02:33:17.721 INFO:tasks.workunit.client.0.smithi019.stdout: 1 27308 12.46 MB/sec warmup 13 sec latency 12.678 ms 2024-05-07T02:33:18.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.107+0000 7f5cc3200700 1 -- 172.21.15.19:0/158798394 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cc4109680 msgr2=0x7f5cc4110fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:18.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.107+0000 7f5cc3200700 1 --2- 172.21.15.19:0/158798394 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cc4109680 0x7f5cc4110fe0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc00b0a0 tx=0x7f5cbc00b3b0 comp rx=0 tx=0).stop 2024-05-07T02:33:18.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.107+0000 7f5cc3200700 1 -- 172.21.15.19:0/158798394 shutdown_connections 2024-05-07T02:33:18.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.107+0000 7f5cc3200700 1 --2- 172.21.15.19:0/158798394 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cc4109680 0x7f5cc4110fe0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:18.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.107+0000 7f5cc3200700 1 --2- 172.21.15.19:0/158798394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5cc4104400 0x7f5cc4104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:18.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.107+0000 7f5cc3200700 1 --2- 172.21.15.19:0/158798394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cc4108d30 0x7f5cc4109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:18.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.107+0000 7f5cc3200700 1 -- 172.21.15.19:0/158798394 >> 172.21.15.19:0/158798394 conn(0x7f5cc4079d10 msgr2=0x7f5cc407a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:18.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.107+0000 7f5cc3200700 1 -- 172.21.15.19:0/158798394 shutdown_connections 2024-05-07T02:33:18.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.107+0000 7f5cc3200700 1 -- 172.21.15.19:0/158798394 wait complete. 2024-05-07T02:33:18.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.109+0000 7f5cc3200700 1 Processor -- start 2024-05-07T02:33:18.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.109+0000 7f5cc3200700 1 -- start start 2024-05-07T02:33:18.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.110+0000 7f5cc3200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cc4104400 0x7f5cc41a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:18.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.110+0000 7f5cc3200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5cc4108d30 0x7f5cc41a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:18.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.110+0000 7f5cc3200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cc4109680 0x7f5cc419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:18.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.110+0000 7f5cc3200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc4114300 con 0x7f5cc4109680 2024-05-07T02:33:18.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.110+0000 7f5cc3200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc4114180 con 0x7f5cc4104400 2024-05-07T02:33:18.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.110+0000 7f5cc3200700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc4114480 con 0x7f5cc4108d30 2024-05-07T02:33:18.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.110+0000 7f5cc1e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cc4104400 0x7f5cc41a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:18.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.110+0000 7f5cb9400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5cc4108d30 0x7f5cc41a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:18.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.111+0000 7f5cc1e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cc4104400 0x7f5cc41a0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:56346/0 (socket says 172.21.15.19:56346) 2024-05-07T02:33:18.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.111+0000 7f5cc1e00700 1 -- 172.21.15.19:0/2897378101 learned_addr learned my addr 172.21.15.19:0/2897378101 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:18.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.111+0000 7f5cc2800700 1 --2- 172.21.15.19:0/2897378101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cc4109680 0x7f5cc419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:18.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.111+0000 7f5cb9400700 1 -- 172.21.15.19:0/2897378101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cc4104400 msgr2=0x7f5cc41a0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:18.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.111+0000 7f5cb9400700 1 --2- 172.21.15.19:0/2897378101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cc4104400 0x7f5cc41a0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:18.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.111+0000 7f5cb9400700 1 -- 172.21.15.19:0/2897378101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cc4109680 msgr2=0x7f5cc419ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:18.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.111+0000 7f5cb9400700 1 --2- 172.21.15.19:0/2897378101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cc4109680 0x7f5cc419ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:18.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.111+0000 7f5cb9400700 1 -- 172.21.15.19:0/2897378101 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc419b1b0 con 0x7f5cc4108d30 2024-05-07T02:33:18.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.111+0000 7f5cc1e00700 1 --2- 172.21.15.19:0/2897378101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cc4104400 0x7f5cc41a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:33:18.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.111+0000 7f5cc2800700 1 --2- 172.21.15.19:0/2897378101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cc4109680 0x7f5cc419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:33:18.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.112+0000 7f5cb9400700 1 --2- 172.21.15.19:0/2897378101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5cc4108d30 0x7f5cc41a0d80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5cc406c3c0 tx=0x7f5cac00dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:18.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.112+0000 7f5cbb400700 1 -- 172.21.15.19:0/2897378101 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cac017070 con 0x7f5cc4108d30 2024-05-07T02:33:18.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.112+0000 7f5cc3200700 1 -- 172.21.15.19:0/2897378101 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cc419b4a0 con 0x7f5cc4108d30 2024-05-07T02:33:18.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.112+0000 7f5cbb400700 1 -- 172.21.15.19:0/2897378101 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cac009de0 con 0x7f5cc4108d30 2024-05-07T02:33:18.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.112+0000 7f5cbb400700 1 -- 172.21.15.19:0/2897378101 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cac010d90 con 0x7f5cc4108d30 2024-05-07T02:33:18.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.112+0000 7f5cc3200700 1 -- 172.21.15.19:0/2897378101 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cc41a76d0 con 0x7f5cc4108d30 2024-05-07T02:33:18.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.116+0000 7f5cbb400700 1 -- 172.21.15.19:0/2897378101 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f5cac01a070 con 0x7f5cc4108d30 2024-05-07T02:33:18.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.117+0000 7f5cbb400700 1 --2- 172.21.15.19:0/2897378101 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f5ca80b2b10 0x7f5ca80b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:18.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.117+0000 7f5cbb400700 1 -- 172.21.15.19:0/2897378101 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f5cac0e24c0 con 0x7f5cc4108d30 2024-05-07T02:33:18.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.117+0000 7f5cc1e00700 1 --2- 172.21.15.19:0/2897378101 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f5ca80b2b10 0x7f5ca80b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:18.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.117+0000 7f5cd7d8ab80 1 Processor -- start 2024-05-07T02:33:18.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.118+0000 7f5cd7d8ab80 1 -- start start 2024-05-07T02:33:18.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.118+0000 7f5cd7d8ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559db44a6830 0x559db44a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:18.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.118+0000 7f5cd7d8ab80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559db44a7180 0x559db44b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:18.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.118+0000 7f5cd7d8ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db44b4ff0 0x559db44b7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:18.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.118+0000 7f5cd7d8ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559db43c8aa0 con 0x559db44b4ff0 2024-05-07T02:33:18.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.118+0000 7f5cd7d8ab80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559db43667e0 con 0x559db44a7180 2024-05-07T02:33:18.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.118+0000 7f5cd7d8ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559db43c4a60 con 0x559db44a6830 2024-05-07T02:33:18.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.118+0000 7f5cc1e00700 1 --2- 172.21.15.19:0/2897378101 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f5ca80b2b10 0x7f5ca80b5020 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4102b60 tx=0x7f5cb400b500 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:18.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.119+0000 7f5cc1e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559db44a6830 0x559db44a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:18.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.119+0000 7f5cb9400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559db44a7180 0x559db44b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:18.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.119+0000 7f5cb9400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559db44a7180 0x559db44b4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:56356/0 (socket says 172.21.15.19:56356) 2024-05-07T02:33:18.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.119+0000 7f5cc1e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559db44a6830 0x559db44a6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:32872/0 (socket says 172.21.15.19:32872) 2024-05-07T02:33:18.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.119+0000 7f5cb9400700 1 -- 172.21.15.19:0/1018225458 learned_addr learned my addr 172.21.15.19:0/1018225458 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:18.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.119+0000 7f5cc2800700 1 --2- 172.21.15.19:0/1018225458 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db44b4ff0 0x559db44b7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:18.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.119+0000 7f5cb9400700 1 -- 172.21.15.19:0/1018225458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559db44a6830 msgr2=0x559db44a6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:18.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.119+0000 7f5cb9400700 1 --2- 172.21.15.19:0/1018225458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559db44a6830 0x559db44a6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:18.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.119+0000 7f5cb9400700 1 -- 172.21.15.19:0/1018225458 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db44b4ff0 msgr2=0x559db44b7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:18.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.119+0000 7f5cb9400700 1 --2- 172.21.15.19:0/1018225458 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db44b4ff0 0x559db44b7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:18.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.119+0000 7f5cb9400700 1 -- 172.21.15.19:0/1018225458 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559db4315950 con 0x559db44a7180 2024-05-07T02:33:18.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.119+0000 7f5cb9400700 1 --2- 172.21.15.19:0/1018225458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559db44a7180 0x559db44b4ab0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5cac0a52b0 tx=0x7f5cac0a8a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d70834dfdb7155cf server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:18.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.120+0000 7f5cb3400700 1 -- 172.21.15.19:0/1018225458 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cac021070 con 0x559db44a7180 2024-05-07T02:33:18.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.120+0000 7f5cb3400700 1 -- 172.21.15.19:0/1018225458 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cac0a9230 con 0x559db44a7180 2024-05-07T02:33:18.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.120+0000 7f5cb3400700 1 -- 172.21.15.19:0/1018225458 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cac0ad3c0 con 0x559db44a7180 2024-05-07T02:33:18.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.121+0000 7f5cd7d8ab80 1 -- 172.21.15.19:0/1018225458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559db44a7180 msgr2=0x559db44b4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:18.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.121+0000 7f5cd7d8ab80 1 --2- 172.21.15.19:0/1018225458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559db44a7180 0x559db44b4ab0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5cac0a52b0 tx=0x7f5cac0a8a20 comp rx=0 tx=0).stop 2024-05-07T02:33:18.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.121+0000 7f5cd7d8ab80 1 -- 172.21.15.19:0/1018225458 shutdown_connections 2024-05-07T02:33:18.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.121+0000 7f5cd7d8ab80 1 --2- 172.21.15.19:0/1018225458 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db44b4ff0 0x559db44b7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:18.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.121+0000 7f5cd7d8ab80 1 --2- 172.21.15.19:0/1018225458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559db44a6830 0x559db44a6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:18.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.121+0000 7f5cd7d8ab80 1 --2- 172.21.15.19:0/1018225458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559db44a7180 0x559db44b4ab0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:18.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.121+0000 7f5cd7d8ab80 1 -- 172.21.15.19:0/1018225458 >> 172.21.15.19:0/1018225458 conn(0x559db4328ec0 msgr2=0x559db44a1900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:18.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.121+0000 7f5cd7d8ab80 1 -- 172.21.15.19:0/1018225458 shutdown_connections 2024-05-07T02:33:18.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.121+0000 7f5cd7d8ab80 1 -- 172.21.15.19:0/1018225458 wait complete. 2024-05-07T02:33:18.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.122+0000 7f5cd7d8ab80 1 -- start start 2024-05-07T02:33:18.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.122+0000 7f5cd7d8ab80 1 Processor -- start 2024-05-07T02:33:18.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.123+0000 7f5cd7d8ab80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559db44a6830 0x559db44b1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:18.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.123+0000 7f5cd7d8ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559db44a7180 0x559db44b24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:18.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.123+0000 7f5cd7d8ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db44b4ff0 0x559db44b2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:18.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.123+0000 7f5cd7d8ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559db44b7d50 con 0x559db44b4ff0 2024-05-07T02:33:18.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.123+0000 7f5cd7d8ab80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559db432eba0 con 0x559db44a6830 2024-05-07T02:33:18.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.123+0000 7f5cd7d8ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559db44b7ed0 con 0x559db44a7180 2024-05-07T02:33:18.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.123+0000 7f5cd7d8ab80 10 client.0 _ll_register_callbacks cb 0x559db436fcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:33:18.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.123+0000 7f5cc1e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559db44a6830 0x559db44b1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:18.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.123+0000 7f5cc1e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559db44a6830 0x559db44b1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:56364/0 (socket says 172.21.15.19:56364) 2024-05-07T02:33:18.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.123+0000 7f5cc1e00700 1 -- 172.21.15.19:0/1852546990 learned_addr learned my addr 172.21.15.19:0/1852546990 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:18.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.123+0000 7f5cc2800700 1 --2- 172.21.15.19:0/1852546990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db44b4ff0 0x559db44b2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:18.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.123+0000 7f5cb9400700 1 --2- 172.21.15.19:0/1852546990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559db44a7180 0x559db44b24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:18.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.123+0000 7f5cc1e00700 1 -- 172.21.15.19:0/1852546990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559db44a7180 msgr2=0x559db44b24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:18.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.123+0000 7f5cc1e00700 1 --2- 172.21.15.19:0/1852546990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559db44a7180 0x559db44b24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:18.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.123+0000 7f5cc1e00700 1 -- 172.21.15.19:0/1852546990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db44b4ff0 msgr2=0x559db44b2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:18.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.123+0000 7f5cc1e00700 1 --2- 172.21.15.19:0/1852546990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db44b4ff0 0x559db44b2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:18.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.123+0000 7f5cc1e00700 1 -- 172.21.15.19:0/1852546990 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559db438e4a0 con 0x559db44a6830 2024-05-07T02:33:18.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.124+0000 7f5cc1e00700 1 --2- 172.21.15.19:0/1852546990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559db44a6830 0x559db44b1f80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4011ab0 tx=0x7f5cb40047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de1b5c53b42b2099 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:18.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.124+0000 7f5ca7e00700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:33:18.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.124+0000 7f5ca7e00700 1 -- 172.21.15.19:0/1852546990 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb401fcd0 con 0x559db44a6830 2024-05-07T02:33:18.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.124+0000 7f5cd7d8ab80 1 -- 172.21.15.19:0/1852546990 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cac0a9230 con 0x559db44a6830 2024-05-07T02:33:18.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.124+0000 7f5ca7e00700 1 -- 172.21.15.19:0/1852546990 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f5cb4004020 con 0x559db44a6830 2024-05-07T02:33:18.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.124+0000 7f5ca7e00700 1 -- 172.21.15.19:0/1852546990 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5cb4004220 con 0x559db44a6830 2024-05-07T02:33:18.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.124+0000 7f5cd7d8ab80 10 client.25562 fetch_fsmap learned FSMap version 31 2024-05-07T02:33:18.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.124+0000 7f5cd7d8ab80 1 -- 172.21.15.19:0/1852546990 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559db44c29b0 con 0x559db44a6830 2024-05-07T02:33:18.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.125+0000 7f5ca7e00700 1 -- 172.21.15.19:0/1852546990 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cac0a9230 con 0x559db44a6830 2024-05-07T02:33:18.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.125+0000 7f5ca7e00700 1 -- 172.21.15.19:0/1852546990 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f5cb4026070 con 0x559db44a6830 2024-05-07T02:33:18.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.125+0000 7f5cd7d8ab80 10 client.25562 fetch_fsmap finished waiting for FSMap version 31 2024-05-07T02:33:18.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.125+0000 7f5cd7d8ab80 10 client.25562 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:33:18.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.125+0000 7f5cd7d8ab80 20 client.25562 populate_metadata read hostname 'smithi019' 2024-05-07T02:33:18.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.125+0000 7f5cd7d8ab80 1 --2- 172.21.15.19:0/1852546990 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x559db44c4ff0 0x559db44c7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:18.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.125+0000 7f5cd7d8ab80 4 client.25562 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:33:18.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.125+0000 7f5cd7d8ab80 1 -- 172.21.15.19:0/1852546990 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559db4473360 con 0x559db44c4ff0 2024-05-07T02:33:18.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.125+0000 7f5cb9400700 1 --2- 172.21.15.19:0/1852546990 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x559db44c4ff0 0x559db44c7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:18.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.126+0000 7f5cb9400700 1 --2- 172.21.15.19:0/1852546990 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x559db44c4ff0 0x559db44c7410 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4d3d131a94a9a13 server_cookie=d4eaf0ffc54014e4 in_seq=0 out_seq=0 2024-05-07T02:33:18.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.126+0000 7f5ca7e00700 10 client.25562 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:33:18.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.127+0000 7f5ca7e00700 1 -- 172.21.15.19:0/1852546990 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559db4473360 con 0x559db44c4ff0 2024-05-07T02:33:18.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.127+0000 7f5ca7e00700 10 client.25562 handle_command_reply: tid=0 2024-05-07T02:33:18.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.127+0000 7f5cd7d8ab80 1 client.25562 shutdown 2024-05-07T02:33:18.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.128+0000 7f5cd7d8ab80 1 -- 172.21.15.19:0/1852546990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559db44a6830 msgr2=0x559db44b1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:18.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.128+0000 7f5cd7d8ab80 1 --2- 172.21.15.19:0/1852546990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559db44a6830 0x559db44b1f80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4011ab0 tx=0x7f5cb40047c0 comp rx=0 tx=0).stop 2024-05-07T02:33:18.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.128+0000 7f5cd7d8ab80 1 -- 172.21.15.19:0/1852546990 shutdown_connections 2024-05-07T02:33:18.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.128+0000 7f5cd7d8ab80 1 --2- 172.21.15.19:0/1852546990 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x559db44c4ff0 0x559db44c7410 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:18.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.128+0000 7f5cd7d8ab80 1 --2- 172.21.15.19:0/1852546990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db44b4ff0 0x559db44b2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:18.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.128+0000 7f5cd7d8ab80 1 --2- 172.21.15.19:0/1852546990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559db44a6830 0x559db44b1f80 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:18.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.128+0000 7f5cd7d8ab80 1 --2- 172.21.15.19:0/1852546990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559db44a7180 0x559db44b24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:18.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.128+0000 7f5ca7e00700 0 client.25562 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:33:18.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.128+0000 7f5cd7d8ab80 1 -- 172.21.15.19:0/1852546990 >> 172.21.15.19:0/1852546990 conn(0x559db4328ec0 msgr2=0x559db44a34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:18.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.128+0000 7f5cd7d8ab80 1 -- 172.21.15.19:0/1852546990 shutdown_connections 2024-05-07T02:33:18.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.128+0000 7f5cd7d8ab80 1 -- 172.21.15.19:0/1852546990 wait complete. 2024-05-07T02:33:18.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.128+0000 7f5cd7d8ab80 20 client.25562 trim_cache size 0 max 16384 2024-05-07T02:33:18.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.151+0000 7f5cd7d8ab80 1 Processor -- start 2024-05-07T02:33:18.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.151+0000 7f5cd7d8ab80 1 -- start start 2024-05-07T02:33:18.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.152+0000 7f5cd7d8ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db44a6830 0x559db44c0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:18.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.152+0000 7f5cd7d8ab80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559db44a7180 0x559db44ad160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:18.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.152+0000 7f5cd7d8ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559db44b4ff0 0x559db44ad6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:18.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.152+0000 7f5cd7d8ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc41ae370 con 0x559db44a6830 2024-05-07T02:33:18.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.152+0000 7f5cd7d8ab80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc41afeb0 con 0x559db44a7180 2024-05-07T02:33:18.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.152+0000 7f5cd7d8ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc41aea40 con 0x559db44b4ff0 2024-05-07T02:33:18.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.152+0000 7f5cb9400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559db44a7180 0x559db44ad160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:18.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.152+0000 7f5cb9400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559db44a7180 0x559db44ad160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:56378/0 (socket says 172.21.15.19:56378) 2024-05-07T02:33:18.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.152+0000 7f5cb9400700 1 -- 172.21.15.19:0/2297296915 learned_addr learned my addr 172.21.15.19:0/2297296915 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:18.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.152+0000 7f5cc1e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db44a6830 0x559db44c0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:18.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.152+0000 7f5cc2800700 1 --2- 172.21.15.19:0/2297296915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559db44b4ff0 0x559db44ad6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:18.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.153+0000 7f5cb9400700 1 -- 172.21.15.19:0/2297296915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559db44b4ff0 msgr2=0x559db44ad6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:18.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.153+0000 7f5cb9400700 1 --2- 172.21.15.19:0/2297296915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559db44b4ff0 0x559db44ad6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:18.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.153+0000 7f5cb9400700 1 -- 172.21.15.19:0/2297296915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db44a6830 msgr2=0x559db44c0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:18.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.153+0000 7f5cb9400700 1 --2- 172.21.15.19:0/2297296915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db44a6830 0x559db44c0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:18.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.153+0000 7f5cb9400700 1 -- 172.21.15.19:0/2297296915 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559db44b04f0 con 0x559db44a7180 2024-05-07T02:33:18.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.153+0000 7f5cb9400700 1 --2- 172.21.15.19:0/2297296915 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559db44a7180 0x559db44ad160 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5cac0a91c0 tx=0x7f5cac00df60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c689bb1b5e9242f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:18.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.154+0000 7f5cb2a00700 1 -- 172.21.15.19:0/2297296915 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cac0c2070 con 0x559db44a7180 2024-05-07T02:33:18.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.154+0000 7f5cb2a00700 1 -- 172.21.15.19:0/2297296915 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cac0ad5e0 con 0x559db44a7180 2024-05-07T02:33:18.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.154+0000 7f5cb2a00700 1 -- 172.21.15.19:0/2297296915 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cac021040 con 0x559db44a7180 2024-05-07T02:33:18.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.155+0000 7f5cd7d8ab80 1 -- 172.21.15.19:0/2297296915 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559db44a7180 msgr2=0x559db44ad160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:18.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.155+0000 7f5cd7d8ab80 1 --2- 172.21.15.19:0/2297296915 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559db44a7180 0x559db44ad160 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5cac0a91c0 tx=0x7f5cac00df60 comp rx=0 tx=0).stop 2024-05-07T02:33:18.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.155+0000 7f5cd7d8ab80 1 -- 172.21.15.19:0/2297296915 shutdown_connections 2024-05-07T02:33:18.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.155+0000 7f5cd7d8ab80 1 --2- 172.21.15.19:0/2297296915 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559db44b4ff0 0x559db44ad6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:18.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.155+0000 7f5cd7d8ab80 1 --2- 172.21.15.19:0/2297296915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db44a6830 0x559db44c0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:18.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.155+0000 7f5cd7d8ab80 1 --2- 172.21.15.19:0/2297296915 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559db44a7180 0x559db44ad160 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:18.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.155+0000 7f5cd7d8ab80 1 -- 172.21.15.19:0/2297296915 >> 172.21.15.19:0/2297296915 conn(0x559db4328ec0 msgr2=0x559db44aca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:18.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.155+0000 7f5cd7d8ab80 1 -- 172.21.15.19:0/2297296915 shutdown_connections 2024-05-07T02:33:18.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.155+0000 7f5cd7d8ab80 1 -- 172.21.15.19:0/2297296915 wait complete. 2024-05-07T02:33:18.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.156+0000 7f5cd7d8ab80 1 -- start start 2024-05-07T02:33:18.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.156+0000 7f5cd7d8ab80 1 Processor -- start 2024-05-07T02:33:18.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.156+0000 7f5cd7d8ab80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559db44a6830 0x559db44bd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:18.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.156+0000 7f5cd7d8ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db44a7180 0x559db44b9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:18.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.156+0000 7f5cd7d8ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559db44b4ff0 0x559db44b9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:18.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.156+0000 7f5cd7d8ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc41af7e0 con 0x559db44a7180 2024-05-07T02:33:18.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.156+0000 7f5cd7d8ab80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc41af110 con 0x559db44a6830 2024-05-07T02:33:18.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.156+0000 7f5cd7d8ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc41ac310 con 0x559db44b4ff0 2024-05-07T02:33:18.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.157+0000 7f5cd7d8ab80 10 client.0 _ll_register_callbacks cb 0x559db436fcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:33:18.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.157+0000 7f5cc1e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559db44a6830 0x559db44bd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:18.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.157+0000 7f5cb9400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db44a7180 0x559db44b9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:18.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.157+0000 7f5cc1e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559db44a6830 0x559db44bd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:56388/0 (socket says 172.21.15.19:56388) 2024-05-07T02:33:18.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.157+0000 7f5cc1e00700 1 -- 172.21.15.19:0/2319663830 learned_addr learned my addr 172.21.15.19:0/2319663830 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:18.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.157+0000 7f5cc2800700 1 --2- 172.21.15.19:0/2319663830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559db44b4ff0 0x559db44b9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:18.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.157+0000 7f5cc1e00700 1 -- 172.21.15.19:0/2319663830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559db44b4ff0 msgr2=0x559db44b9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:18.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.157+0000 7f5cc1e00700 1 --2- 172.21.15.19:0/2319663830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559db44b4ff0 0x559db44b9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:18.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.157+0000 7f5cc1e00700 1 -- 172.21.15.19:0/2319663830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db44a7180 msgr2=0x559db44b9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:18.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.157+0000 7f5cc1e00700 1 --2- 172.21.15.19:0/2319663830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db44a7180 0x559db44b9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:18.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.157+0000 7f5cc1e00700 1 -- 172.21.15.19:0/2319663830 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559db44ba550 con 0x559db44a6830 2024-05-07T02:33:18.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.158+0000 7f5cc1e00700 1 --2- 172.21.15.19:0/2319663830 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559db44a6830 0x559db44bd600 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4003ee0 tx=0x7f5cb4005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc78de44f4b0f12c server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:18.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.158+0000 7f5ca7e00700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:33:18.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.158+0000 7f5ca7e00700 1 -- 172.21.15.19:0/2319663830 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb401fcd0 con 0x559db44a6830 2024-05-07T02:33:18.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.158+0000 7f5ca7e00700 1 -- 172.21.15.19:0/2319663830 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f5cb4027910 con 0x559db44a6830 2024-05-07T02:33:18.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.158+0000 7f5cd7d8ab80 1 -- 172.21.15.19:0/2319663830 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559db43d6480 con 0x559db44a6830 2024-05-07T02:33:18.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.158+0000 7f5ca7e00700 1 -- 172.21.15.19:0/2319663830 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5cb401f710 con 0x559db44a6830 2024-05-07T02:33:18.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.158+0000 7f5cd7d8ab80 10 client.25568 fetch_fsmap learned FSMap version 31 2024-05-07T02:33:18.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.158+0000 7f5cd7d8ab80 1 -- 172.21.15.19:0/2319663830 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559db45306a0 con 0x559db44a6830 2024-05-07T02:33:18.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.158+0000 7f5ca7e00700 1 -- 172.21.15.19:0/2319663830 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559db43d6480 con 0x559db44a6830 2024-05-07T02:33:18.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.159+0000 7f5ca7e00700 1 -- 172.21.15.19:0/2319663830 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f5cb4026040 con 0x559db44a6830 2024-05-07T02:33:18.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.159+0000 7f5cd7d8ab80 10 client.25568 fetch_fsmap finished waiting for FSMap version 31 2024-05-07T02:33:18.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.159+0000 7f5cd7d8ab80 10 client.25568 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:33:18.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.159+0000 7f5cd7d8ab80 20 client.25568 populate_metadata read hostname 'smithi019' 2024-05-07T02:33:18.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.159+0000 7f5cd7d8ab80 1 --2- 172.21.15.19:0/2319663830 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x559db4530db0 0x559db45331d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:18.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.159+0000 7f5cd7d8ab80 4 client.25568 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-07T02:33:18.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.159+0000 7f5cd7d8ab80 1 -- 172.21.15.19:0/2319663830 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559db43d0520 con 0x559db4530db0 2024-05-07T02:33:18.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.159+0000 7f5cb9400700 1 --2- 172.21.15.19:0/2319663830 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x559db4530db0 0x559db45331d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:18.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.160+0000 7f5cb9400700 1 --2- 172.21.15.19:0/2319663830 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x559db4530db0 0x559db45331d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=332c33bb69409e7d server_cookie=608a656839e73355 in_seq=0 out_seq=0 2024-05-07T02:33:18.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.160+0000 7f5ca7e00700 10 client.25568 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:33:18.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.164+0000 7f5ca7e00700 1 -- 172.21.15.19:0/2319663830 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559db43d0520 con 0x559db4530db0 2024-05-07T02:33:18.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.164+0000 7f5ca7e00700 10 client.25568 handle_command_reply: tid=0 2024-05-07T02:33:18.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.164+0000 7f5cd7d8ab80 1 client.25568 shutdown 2024-05-07T02:33:18.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.164+0000 7f5cd7d8ab80 1 -- 172.21.15.19:0/2319663830 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559db44a6830 msgr2=0x559db44bd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:18.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.164+0000 7f5cd7d8ab80 1 --2- 172.21.15.19:0/2319663830 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559db44a6830 0x559db44bd600 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4003ee0 tx=0x7f5cb4005a20 comp rx=0 tx=0).stop 2024-05-07T02:33:18.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.164+0000 7f5cd7d8ab80 1 -- 172.21.15.19:0/2319663830 shutdown_connections 2024-05-07T02:33:18.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.164+0000 7f5cd7d8ab80 1 --2- 172.21.15.19:0/2319663830 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x559db4530db0 0x559db45331d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:18.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.164+0000 7f5cd7d8ab80 1 --2- 172.21.15.19:0/2319663830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559db44b4ff0 0x559db44b9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:18.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.164+0000 7f5cd7d8ab80 1 --2- 172.21.15.19:0/2319663830 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559db44a6830 0x559db44bd600 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:18.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.164+0000 7f5cd7d8ab80 1 --2- 172.21.15.19:0/2319663830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559db44a7180 0x559db44b9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:18.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.164+0000 7f5ca7e00700 0 client.25568 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:33:18.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.164+0000 7f5cd7d8ab80 1 -- 172.21.15.19:0/2319663830 >> 172.21.15.19:0/2319663830 conn(0x559db4328ec0 msgr2=0x559db44ad480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:18.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.164+0000 7f5cd7d8ab80 1 -- 172.21.15.19:0/2319663830 shutdown_connections 2024-05-07T02:33:18.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.164+0000 7f5cd7d8ab80 1 -- 172.21.15.19:0/2319663830 wait complete. 2024-05-07T02:33:18.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.165+0000 7f5cd7d8ab80 20 client.25568 trim_cache size 0 max 16384 2024-05-07T02:33:18.166 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:33:18.166 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-07T02:33:18.166 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "3df64ffd-792c-426a-89f2-9260c97d313a", 2024-05-07T02:33:18.166 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-07T02:33:18.166 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:33:18.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.165+0000 7f5cc3200700 1 -- 172.21.15.19:0/2897378101 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f5ca80b2b10 msgr2=0x7f5ca80b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:18.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.165+0000 7f5cc3200700 1 --2- 172.21.15.19:0/2897378101 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f5ca80b2b10 0x7f5ca80b5020 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4102b60 tx=0x7f5cb400b500 comp rx=0 tx=0).stop 2024-05-07T02:33:18.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.165+0000 7f5cc3200700 1 -- 172.21.15.19:0/2897378101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5cc4108d30 msgr2=0x7f5cc41a0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:18.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.165+0000 7f5cc3200700 1 --2- 172.21.15.19:0/2897378101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5cc4108d30 0x7f5cc41a0d80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5cc406c3c0 tx=0x7f5cac00dc30 comp rx=0 tx=0).stop 2024-05-07T02:33:18.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.166+0000 7f5cc3200700 1 -- 172.21.15.19:0/2897378101 shutdown_connections 2024-05-07T02:33:18.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.166+0000 7f5cc3200700 1 --2- 172.21.15.19:0/2897378101 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f5ca80b2b10 0x7f5ca80b5020 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:18.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.166+0000 7f5cc3200700 1 --2- 172.21.15.19:0/2897378101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cc4109680 0x7f5cc419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:18.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.166+0000 7f5cc3200700 1 --2- 172.21.15.19:0/2897378101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cc4104400 0x7f5cc41a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:18.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.166+0000 7f5cc3200700 1 --2- 172.21.15.19:0/2897378101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5cc4108d30 0x7f5cc41a0d80 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:18.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.166+0000 7f5cc3200700 1 -- 172.21.15.19:0/2897378101 >> 172.21.15.19:0/2897378101 conn(0x7f5cc4079d10 msgr2=0x7f5cc4101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:18.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.166+0000 7f5cc3200700 1 -- 172.21.15.19:0/2897378101 shutdown_connections 2024-05-07T02:33:18.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:18.166+0000 7f5cc3200700 1 -- 172.21.15.19:0/2897378101 wait complete. 2024-05-07T02:33:18.721 INFO:tasks.workunit.client.0.smithi019.stdout: 1 29081 12.43 MB/sec warmup 14 sec latency 10.923 ms 2024-05-07T02:33:19.173 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-07T02:33:19.235 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:18 smithi019 ceph-mon[29781]: pgmap v47: 129 pgs: 129 active+clean; 13 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-05-07T02:33:19.235 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:18 smithi019 ceph-mon[29781]: scrub queued for path: / 2024-05-07T02:33:19.235 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:18 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:33:19.236 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:18 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:33:19.236 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:18 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:33:19.236 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:18 smithi019 ceph-mon[29781]: scrub summary: idle 2024-05-07T02:33:19.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:18 smithi064 ceph-mon[34699]: pgmap v47: 129 pgs: 129 active+clean; 13 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-05-07T02:33:19.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:18 smithi064 ceph-mon[34699]: scrub queued for path: / 2024-05-07T02:33:19.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:18 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:33:19.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:18 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:33:19.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:18 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:33:19.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:18 smithi064 ceph-mon[34699]: scrub summary: idle 2024-05-07T02:33:19.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:18 smithi027 ceph-mon[34398]: pgmap v47: 129 pgs: 129 active+clean; 13 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-05-07T02:33:19.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:18 smithi027 ceph-mon[34398]: scrub queued for path: / 2024-05-07T02:33:19.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:18 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:33:19.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:18 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:33:19.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:18 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:33:19.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:18 smithi027 ceph-mon[34398]: scrub summary: idle 2024-05-07T02:33:19.640 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:33:19.721 INFO:tasks.workunit.client.0.smithi019.stdout: 1 31336 12.43 MB/sec warmup 15 sec latency 23.002 ms 2024-05-07T02:33:20.069 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:20 smithi019 ceph-mon[29781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:33:20.069 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:20 smithi019 ceph-mon[29781]: pgmap v48: 129 pgs: 129 active+clean; 13 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-05-07T02:33:20.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:20 smithi064 ceph-mon[34699]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:33:20.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:20 smithi064 ceph-mon[34699]: pgmap v48: 129 pgs: 129 active+clean; 13 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-05-07T02:33:20.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:20 smithi027 ceph-mon[34398]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:33:20.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:20 smithi027 ceph-mon[34398]: pgmap v48: 129 pgs: 129 active+clean; 13 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-05-07T02:33:20.722 INFO:tasks.workunit.client.0.smithi019.stdout: 1 32257 12.05 MB/sec warmup 16 sec latency 286.121 ms 2024-05-07T02:33:21.721 INFO:tasks.workunit.client.0.smithi019.stdout: 1 32338 11.36 MB/sec warmup 17 sec latency 779.681 ms 2024-05-07T02:33:22.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.138+0000 7ff714200700 1 -- 172.21.15.19:0/635540243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff70c108f00 msgr2=0x7ff70c109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:22.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.138+0000 7ff714200700 1 --2- 172.21.15.19:0/635540243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff70c108f00 0x7ff70c109310 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff6f80099c0 tx=0x7ff6f8009cd0 comp rx=0 tx=0).stop 2024-05-07T02:33:22.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.138+0000 7ff714200700 1 -- 172.21.15.19:0/635540243 shutdown_connections 2024-05-07T02:33:22.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.138+0000 7ff714200700 1 --2- 172.21.15.19:0/635540243 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff70c109850 0x7ff70c1111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:22.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.138+0000 7ff714200700 1 --2- 172.21.15.19:0/635540243 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff70c1045d0 0x7ff70c1049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:22.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.138+0000 7ff714200700 1 --2- 172.21.15.19:0/635540243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff70c108f00 0x7ff70c109310 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:22.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.138+0000 7ff714200700 1 -- 172.21.15.19:0/635540243 >> 172.21.15.19:0/635540243 conn(0x7ff70c0ffe80 msgr2=0x7ff70c1022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:22.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.138+0000 7ff714200700 1 -- 172.21.15.19:0/635540243 shutdown_connections 2024-05-07T02:33:22.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.139+0000 7ff714200700 1 -- 172.21.15.19:0/635540243 wait complete. 2024-05-07T02:33:22.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.140+0000 7ff714200700 1 Processor -- start 2024-05-07T02:33:22.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.140+0000 7ff714200700 1 -- start start 2024-05-07T02:33:22.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.141+0000 7ff714200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff70c1045d0 0x7ff70c19ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:22.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.141+0000 7ff714200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff70c108f00 0x7ff70c19b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:22.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.141+0000 7ff714200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff70c109850 0x7ff70c19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:22.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.141+0000 7ff714200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff70c114460 con 0x7ff70c1045d0 2024-05-07T02:33:22.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.141+0000 7ff714200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff70c1142e0 con 0x7ff70c109850 2024-05-07T02:33:22.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.141+0000 7ff714200700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff70c1145e0 con 0x7ff70c108f00 2024-05-07T02:33:22.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.141+0000 7ff711800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff70c1045d0 0x7ff70c19ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:22.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.141+0000 7ff710e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff70c108f00 0x7ff70c19b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:22.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.141+0000 7ff711800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff70c1045d0 0x7ff70c19ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52722/0 (socket says 172.21.15.19:52722) 2024-05-07T02:33:22.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.141+0000 7ff711800700 1 -- 172.21.15.19:0/1257508824 learned_addr learned my addr 172.21.15.19:0/1257508824 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:22.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.141+0000 7ff712200700 1 --2- 172.21.15.19:0/1257508824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff70c109850 0x7ff70c19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:22.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.142+0000 7ff710e00700 1 -- 172.21.15.19:0/1257508824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff70c109850 msgr2=0x7ff70c19f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:22.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.142+0000 7ff710e00700 1 --2- 172.21.15.19:0/1257508824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff70c109850 0x7ff70c19f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:22.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.142+0000 7ff710e00700 1 -- 172.21.15.19:0/1257508824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff70c1045d0 msgr2=0x7ff70c19ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:22.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.142+0000 7ff710e00700 1 --2- 172.21.15.19:0/1257508824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff70c1045d0 0x7ff70c19ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:22.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.142+0000 7ff710e00700 1 -- 172.21.15.19:0/1257508824 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff70c19fce0 con 0x7ff70c108f00 2024-05-07T02:33:22.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.142+0000 7ff711800700 1 --2- 172.21.15.19:0/1257508824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff70c1045d0 0x7ff70c19ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:33:22.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.142+0000 7ff710e00700 1 --2- 172.21.15.19:0/1257508824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff70c108f00 0x7ff70c19b3b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff6f800b580 tx=0x7ff6f8005280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:22.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.142+0000 7ff70a000700 1 -- 172.21.15.19:0/1257508824 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f8026070 con 0x7ff70c108f00 2024-05-07T02:33:22.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.142+0000 7ff714200700 1 -- 172.21.15.19:0/1257508824 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff70c19ff70 con 0x7ff70c108f00 2024-05-07T02:33:22.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.142+0000 7ff70a000700 1 -- 172.21.15.19:0/1257508824 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6f80056c0 con 0x7ff70c108f00 2024-05-07T02:33:22.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.142+0000 7ff70a000700 1 -- 172.21.15.19:0/1257508824 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f800f960 con 0x7ff70c108f00 2024-05-07T02:33:22.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.143+0000 7ff714200700 1 -- 172.21.15.19:0/1257508824 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff70c076b00 con 0x7ff70c108f00 2024-05-07T02:33:22.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.146+0000 7ff70a000700 1 -- 172.21.15.19:0/1257508824 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff6f8029070 con 0x7ff70c108f00 2024-05-07T02:33:22.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.147+0000 7ff722cccb80 1 Processor -- start 2024-05-07T02:33:22.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.148+0000 7ff722cccb80 1 -- start start 2024-05-07T02:33:22.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.148+0000 7ff70a000700 1 --2- 172.21.15.19:0/1257508824 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff6f40b29f0 0x7ff6f40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:22.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.148+0000 7ff70a000700 1 -- 172.21.15.19:0/1257508824 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7ff6f80e3220 con 0x7ff70c108f00 2024-05-07T02:33:22.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.148+0000 7ff722cccb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f819a143b0 0x55f819a147c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:22.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.148+0000 7ff722cccb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f819a0e470 0x55f819a0e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:22.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.148+0000 7ff722cccb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f819a0edc0 0x55f819a1b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:22.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.148+0000 7ff722cccb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f819930a20 con 0x55f819a143b0 2024-05-07T02:33:22.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.148+0000 7ff722cccb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f8198ce760 con 0x55f819a0edc0 2024-05-07T02:33:22.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.148+0000 7ff722cccb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f81992c9e0 con 0x55f819a0e470 2024-05-07T02:33:22.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.148+0000 7ff712200700 1 --2- 172.21.15.19:0/1257508824 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff6f40b29f0 0x7ff6f40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:22.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.149+0000 7ff711800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f819a143b0 0x55f819a147c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:22.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.149+0000 7ff711800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f819a143b0 0x55f819a147c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52736/0 (socket says 172.21.15.19:52736) 2024-05-07T02:33:22.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.149+0000 7ff711800700 1 -- 172.21.15.19:0/2504715445 learned_addr learned my addr 172.21.15.19:0/2504715445 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:22.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.149+0000 7ff710e00700 1 --2- 172.21.15.19:0/2504715445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f819a0e470 0x55f819a0e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:22.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.149+0000 7ff712200700 1 --2- 172.21.15.19:0/2504715445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f819a0edc0 0x55f819a1b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:22.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.149+0000 7ff711800700 1 -- 172.21.15.19:0/2504715445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f819a0e470 msgr2=0x55f819a0e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:22.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.149+0000 7ff711800700 1 --2- 172.21.15.19:0/2504715445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f819a0e470 0x55f819a0e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:22.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.149+0000 7ff711800700 1 -- 172.21.15.19:0/2504715445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f819a0edc0 msgr2=0x55f819a1b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:22.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.149+0000 7ff711800700 1 --2- 172.21.15.19:0/2504715445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f819a0edc0 0x55f819a1b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:22.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.149+0000 7ff711800700 1 -- 172.21.15.19:0/2504715445 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f81987d8d0 con 0x55f819a143b0 2024-05-07T02:33:22.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.149+0000 7ff712200700 1 --2- 172.21.15.19:0/2504715445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f819a0edc0 0x55f819a1b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-07T02:33:22.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.149+0000 7ff710e00700 1 --2- 172.21.15.19:0/2504715445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f819a0e470 0x55f819a0e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:33:22.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.150+0000 7ff711800700 1 --2- 172.21.15.19:0/2504715445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f819a143b0 0x55f819a147c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff7000117e0 tx=0x7ff700011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b01b5edee9784b4b server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:22.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.150+0000 7ff712200700 1 --2- 172.21.15.19:0/1257508824 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff6f40b29f0 0x7ff6f40b4f00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff70400f6e0 tx=0x7ff704009920 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:22.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.150+0000 7ff6fea00700 1 -- 172.21.15.19:0/2504715445 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70001e070 con 0x55f819a143b0 2024-05-07T02:33:22.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.150+0000 7ff6fea00700 1 -- 172.21.15.19:0/2504715445 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff70000f930 con 0x55f819a143b0 2024-05-07T02:33:22.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.150+0000 7ff6fea00700 1 -- 172.21.15.19:0/2504715445 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff700019910 con 0x55f819a143b0 2024-05-07T02:33:22.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.151+0000 7ff722cccb80 1 -- 172.21.15.19:0/2504715445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f819a143b0 msgr2=0x55f819a147c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:22.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.151+0000 7ff722cccb80 1 --2- 172.21.15.19:0/2504715445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f819a143b0 0x55f819a147c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff7000117e0 tx=0x7ff700011af0 comp rx=0 tx=0).stop 2024-05-07T02:33:22.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.151+0000 7ff722cccb80 1 -- 172.21.15.19:0/2504715445 shutdown_connections 2024-05-07T02:33:22.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.151+0000 7ff722cccb80 1 --2- 172.21.15.19:0/2504715445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f819a0edc0 0x55f819a1b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:22.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.151+0000 7ff722cccb80 1 --2- 172.21.15.19:0/2504715445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f819a143b0 0x55f819a147c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:22.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.151+0000 7ff722cccb80 1 --2- 172.21.15.19:0/2504715445 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f819a0e470 0x55f819a0e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:22.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.151+0000 7ff722cccb80 1 -- 172.21.15.19:0/2504715445 >> 172.21.15.19:0/2504715445 conn(0x55f819890e40 msgr2=0x55f819a092f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:22.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.151+0000 7ff722cccb80 1 -- 172.21.15.19:0/2504715445 shutdown_connections 2024-05-07T02:33:22.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.151+0000 7ff722cccb80 1 -- 172.21.15.19:0/2504715445 wait complete. 2024-05-07T02:33:22.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.152+0000 7ff722cccb80 1 -- start start 2024-05-07T02:33:22.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.152+0000 7ff722cccb80 1 Processor -- start 2024-05-07T02:33:22.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.153+0000 7ff722cccb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f819a0e470 0x55f819a18590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:22.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.153+0000 7ff722cccb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f819a0edc0 0x55f819a18ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:22.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.153+0000 7ff722cccb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f819a143b0 0x55f819a19010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:22.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.153+0000 7ff722cccb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f819a1b7c0 con 0x55f819a143b0 2024-05-07T02:33:22.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.153+0000 7ff722cccb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f819896b20 con 0x55f819a0e470 2024-05-07T02:33:22.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.153+0000 7ff722cccb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f819a1b940 con 0x55f819a0edc0 2024-05-07T02:33:22.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.153+0000 7ff722cccb80 10 client.0 _ll_register_callbacks cb 0x55f8198d7c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:33:22.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.153+0000 7ff710e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f819a0edc0 0x55f819a18ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:22.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.153+0000 7ff710e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f819a0edc0 0x55f819a18ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:52460/0 (socket says 172.21.15.19:52460) 2024-05-07T02:33:22.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.153+0000 7ff710e00700 1 -- 172.21.15.19:0/1056607470 learned_addr learned my addr 172.21.15.19:0/1056607470 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:22.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.153+0000 7ff711800700 1 --2- 172.21.15.19:0/1056607470 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f819a0e470 0x55f819a18590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:22.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.153+0000 7ff712200700 1 --2- 172.21.15.19:0/1056607470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f819a143b0 0x55f819a19010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:22.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.154+0000 7ff710e00700 1 -- 172.21.15.19:0/1056607470 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f819a0e470 msgr2=0x55f819a18590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:22.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.154+0000 7ff710e00700 1 --2- 172.21.15.19:0/1056607470 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f819a0e470 0x55f819a18590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:22.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.154+0000 7ff710e00700 1 -- 172.21.15.19:0/1056607470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f819a143b0 msgr2=0x55f819a19010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:22.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.154+0000 7ff710e00700 1 --2- 172.21.15.19:0/1056607470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f819a143b0 0x55f819a19010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:22.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.154+0000 7ff710e00700 1 -- 172.21.15.19:0/1056607470 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8198f6420 con 0x55f819a0edc0 2024-05-07T02:33:22.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.154+0000 7ff710e00700 1 --2- 172.21.15.19:0/1056607470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f819a0edc0 0x55f819a18ad0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff6f80e05e0 tx=0x7ff6f80aa9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acbc647755e0ef6b server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:22.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.154+0000 7ff6f3400700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:33:22.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.154+0000 7ff6f3400700 1 -- 172.21.15.19:0/1056607470 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f80b2f70 con 0x55f819a0edc0 2024-05-07T02:33:22.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.154+0000 7ff6f3400700 1 -- 172.21.15.19:0/1056607470 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7ff6f80c4640 con 0x55f819a0edc0 2024-05-07T02:33:22.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.154+0000 7ff722cccb80 1 -- 172.21.15.19:0/1056607470 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff70000f930 con 0x55f819a0edc0 2024-05-07T02:33:22.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.155+0000 7ff6f3400700 1 -- 172.21.15.19:0/1056607470 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff6f80ae120 con 0x55f819a0edc0 2024-05-07T02:33:22.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.155+0000 7ff722cccb80 10 client.25564 fetch_fsmap learned FSMap version 32 2024-05-07T02:33:22.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.155+0000 7ff722cccb80 1 -- 172.21.15.19:0/1056607470 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f819a1b7c0 con 0x55f819a0edc0 2024-05-07T02:33:22.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.155+0000 7ff6f3400700 1 -- 172.21.15.19:0/1056607470 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff70000f930 con 0x55f819a0edc0 2024-05-07T02:33:22.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.155+0000 7ff6f3400700 1 -- 172.21.15.19:0/1056607470 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7ff6f800fb20 con 0x55f819a0edc0 2024-05-07T02:33:22.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.155+0000 7ff722cccb80 10 client.25564 fetch_fsmap finished waiting for FSMap version 32 2024-05-07T02:33:22.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.155+0000 7ff722cccb80 10 client.25564 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:33:22.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.155+0000 7ff722cccb80 20 client.25564 populate_metadata read hostname 'smithi019' 2024-05-07T02:33:22.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.155+0000 7ff722cccb80 1 --2- 172.21.15.19:0/1056607470 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55f819a288e0 0x55f819a2ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:22.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.156+0000 7ff722cccb80 4 client.25564 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:33:22.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.156+0000 7ff722cccb80 1 -- 172.21.15.19:0/1056607470 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8199db2e0 con 0x55f819a288e0 2024-05-07T02:33:22.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.156+0000 7ff711800700 1 --2- 172.21.15.19:0/1056607470 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55f819a288e0 0x55f819a2ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:22.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.157+0000 7ff711800700 1 --2- 172.21.15.19:0/1056607470 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55f819a288e0 0x55f819a2ad00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5810da6194a0659d server_cookie=eb64a5808bcc5c3c in_seq=0 out_seq=0 2024-05-07T02:33:22.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.157+0000 7ff6f3400700 10 client.25564 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:33:22.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.157+0000 7ff6f3400700 1 -- 172.21.15.19:0/1056607470 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f8199db2e0 con 0x55f819a288e0 2024-05-07T02:33:22.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.157+0000 7ff6f3400700 10 client.25564 handle_command_reply: tid=0 2024-05-07T02:33:22.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.157+0000 7ff722cccb80 1 client.25564 shutdown 2024-05-07T02:33:22.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.158+0000 7ff722cccb80 1 -- 172.21.15.19:0/1056607470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f819a0edc0 msgr2=0x55f819a18ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:22.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.158+0000 7ff722cccb80 1 --2- 172.21.15.19:0/1056607470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f819a0edc0 0x55f819a18ad0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff6f80e05e0 tx=0x7ff6f80aa9f0 comp rx=0 tx=0).stop 2024-05-07T02:33:22.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.158+0000 7ff722cccb80 1 -- 172.21.15.19:0/1056607470 shutdown_connections 2024-05-07T02:33:22.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.158+0000 7ff722cccb80 1 --2- 172.21.15.19:0/1056607470 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55f819a288e0 0x55f819a2ad00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:22.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.158+0000 7ff722cccb80 1 --2- 172.21.15.19:0/1056607470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f819a143b0 0x55f819a19010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:22.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.158+0000 7ff6f3400700 0 client.25564 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:33:22.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.158+0000 7ff722cccb80 1 --2- 172.21.15.19:0/1056607470 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f819a0e470 0x55f819a18590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:22.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.158+0000 7ff722cccb80 1 --2- 172.21.15.19:0/1056607470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f819a0edc0 0x55f819a18ad0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:22.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.158+0000 7ff722cccb80 1 -- 172.21.15.19:0/1056607470 >> 172.21.15.19:0/1056607470 conn(0x55f819890e40 msgr2=0x55f819a0d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:22.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.158+0000 7ff722cccb80 1 -- 172.21.15.19:0/1056607470 shutdown_connections 2024-05-07T02:33:22.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.158+0000 7ff722cccb80 1 -- 172.21.15.19:0/1056607470 wait complete. 2024-05-07T02:33:22.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.158+0000 7ff722cccb80 20 client.25564 trim_cache size 0 max 16384 2024-05-07T02:33:22.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.181+0000 7ff722cccb80 1 Processor -- start 2024-05-07T02:33:22.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.181+0000 7ff722cccb80 1 -- start start 2024-05-07T02:33:22.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.181+0000 7ff722cccb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f819a0e470 0x55f819a17510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:22.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.181+0000 7ff722cccb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f819a0edc0 0x55f819a18e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:22.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.181+0000 7ff722cccb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f819a143b0 0x55f819a1a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:22.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.181+0000 7ff722cccb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff70c1b26e0 con 0x55f819a143b0 2024-05-07T02:33:22.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.181+0000 7ff722cccb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff70c1b4250 con 0x55f819a0e470 2024-05-07T02:33:22.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.181+0000 7ff722cccb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff70c1b2db0 con 0x55f819a0edc0 2024-05-07T02:33:22.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.182+0000 7ff711800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f819a0e470 0x55f819a17510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:22.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.182+0000 7ff711800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f819a0e470 0x55f819a17510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:39300/0 (socket says 172.21.15.19:39300) 2024-05-07T02:33:22.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.182+0000 7ff711800700 1 -- 172.21.15.19:0/478158185 learned_addr learned my addr 172.21.15.19:0/478158185 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:22.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.182+0000 7ff710e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f819a0edc0 0x55f819a18e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:22.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.182+0000 7ff712200700 1 --2- 172.21.15.19:0/478158185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f819a143b0 0x55f819a1a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:22.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.182+0000 7ff711800700 1 -- 172.21.15.19:0/478158185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f819a0edc0 msgr2=0x55f819a18e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:22.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.182+0000 7ff711800700 1 --2- 172.21.15.19:0/478158185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f819a0edc0 0x55f819a18e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:22.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.182+0000 7ff711800700 1 -- 172.21.15.19:0/478158185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f819a143b0 msgr2=0x55f819a1a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:22.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.182+0000 7ff711800700 1 --2- 172.21.15.19:0/478158185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f819a143b0 0x55f819a1a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:22.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.182+0000 7ff711800700 1 -- 172.21.15.19:0/478158185 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f819a093c0 con 0x55f819a0e470 2024-05-07T02:33:22.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.182+0000 7ff710e00700 1 --2- 172.21.15.19:0/478158185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f819a0edc0 0x55f819a18e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:33:22.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.183+0000 7ff711800700 1 --2- 172.21.15.19:0/478158185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f819a0e470 0x55f819a17510 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff700009d20 tx=0x7ff700019550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fac0bfca9de02507 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:22.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.183+0000 7ff6fe000700 1 -- 172.21.15.19:0/478158185 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70001e040 con 0x55f819a0e470 2024-05-07T02:33:22.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.183+0000 7ff6fe000700 1 -- 172.21.15.19:0/478158185 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff70002a030 con 0x55f819a0e470 2024-05-07T02:33:22.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.183+0000 7ff6fe000700 1 -- 172.21.15.19:0/478158185 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff700003680 con 0x55f819a0e470 2024-05-07T02:33:22.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.184+0000 7ff722cccb80 1 -- 172.21.15.19:0/478158185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f819a0e470 msgr2=0x55f819a17510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:22.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.184+0000 7ff722cccb80 1 --2- 172.21.15.19:0/478158185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f819a0e470 0x55f819a17510 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff700009d20 tx=0x7ff700019550 comp rx=0 tx=0).stop 2024-05-07T02:33:22.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.184+0000 7ff722cccb80 1 -- 172.21.15.19:0/478158185 shutdown_connections 2024-05-07T02:33:22.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.184+0000 7ff722cccb80 1 --2- 172.21.15.19:0/478158185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f819a143b0 0x55f819a1a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:22.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.184+0000 7ff722cccb80 1 --2- 172.21.15.19:0/478158185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f819a0e470 0x55f819a17510 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:22.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.184+0000 7ff722cccb80 1 --2- 172.21.15.19:0/478158185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f819a0edc0 0x55f819a18e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:22.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.184+0000 7ff722cccb80 1 -- 172.21.15.19:0/478158185 >> 172.21.15.19:0/478158185 conn(0x55f819890e40 msgr2=0x55f819a19c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:22.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.184+0000 7ff722cccb80 1 -- 172.21.15.19:0/478158185 shutdown_connections 2024-05-07T02:33:22.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.184+0000 7ff722cccb80 1 -- 172.21.15.19:0/478158185 wait complete. 2024-05-07T02:33:22.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.185+0000 7ff722cccb80 1 -- start start 2024-05-07T02:33:22.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.185+0000 7ff722cccb80 1 Processor -- start 2024-05-07T02:33:22.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.186+0000 7ff722cccb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f819a0e470 0x55f819a2ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:22.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.186+0000 7ff722cccb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f819a0edc0 0x55f819a30450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:22.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.186+0000 7ff722cccb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f819a143b0 0x55f819a30990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:22.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.186+0000 7ff722cccb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff70c1b3b50 con 0x55f819a0edc0 2024-05-07T02:33:22.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.186+0000 7ff722cccb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff70c1b3480 con 0x55f819a0e470 2024-05-07T02:33:22.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.186+0000 7ff722cccb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff70c1b0680 con 0x55f819a143b0 2024-05-07T02:33:22.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.186+0000 7ff722cccb80 10 client.0 _ll_register_callbacks cb 0x55f8198d7c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:33:22.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.186+0000 7ff710e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f819a0edc0 0x55f819a30450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:22.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.186+0000 7ff711800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f819a0e470 0x55f819a2ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:22.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.186+0000 7ff710e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f819a0edc0 0x55f819a30450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52760/0 (socket says 172.21.15.19:52760) 2024-05-07T02:33:22.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.186+0000 7ff710e00700 1 -- 172.21.15.19:0/4135853064 learned_addr learned my addr 172.21.15.19:0/4135853064 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:22.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.186+0000 7ff712200700 1 --2- 172.21.15.19:0/4135853064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f819a143b0 0x55f819a30990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:22.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.186+0000 7ff710e00700 1 -- 172.21.15.19:0/4135853064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f819a143b0 msgr2=0x55f819a30990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:22.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.186+0000 7ff710e00700 1 --2- 172.21.15.19:0/4135853064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f819a143b0 0x55f819a30990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:22.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.186+0000 7ff710e00700 1 -- 172.21.15.19:0/4135853064 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f819a0e470 msgr2=0x55f819a2ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:22.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.186+0000 7ff710e00700 1 --2- 172.21.15.19:0/4135853064 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f819a0e470 0x55f819a2ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:22.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.186+0000 7ff710e00700 1 -- 172.21.15.19:0/4135853064 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f819a92230 con 0x55f819a0edc0 2024-05-07T02:33:22.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.187+0000 7ff711800700 1 --2- 172.21.15.19:0/4135853064 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f819a0e470 0x55f819a2ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:33:22.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.187+0000 7ff710e00700 1 --2- 172.21.15.19:0/4135853064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f819a0edc0 0x55f819a30450 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff6f8000c00 tx=0x7ff6f8009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f5239c1a1546074 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:22.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.187+0000 7ff6f3400700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-07T02:33:22.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.187+0000 7ff6f3400700 1 -- 172.21.15.19:0/4135853064 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f80a6060 con 0x55f819a0edc0 2024-05-07T02:33:22.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.187+0000 7ff6f3400700 1 -- 172.21.15.19:0/4135853064 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7ff6f80b2cf0 con 0x55f819a0edc0 2024-05-07T02:33:22.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.187+0000 7ff722cccb80 1 -- 172.21.15.19:0/4135853064 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff70002a030 con 0x55f819a0edc0 2024-05-07T02:33:22.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.187+0000 7ff6f3400700 1 -- 172.21.15.19:0/4135853064 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff6f80a5b10 con 0x55f819a0edc0 2024-05-07T02:33:22.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.187+0000 7ff722cccb80 10 client.15438 fetch_fsmap learned FSMap version 32 2024-05-07T02:33:22.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.187+0000 7ff722cccb80 1 -- 172.21.15.19:0/4135853064 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f819a934a0 con 0x55f819a0edc0 2024-05-07T02:33:22.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.188+0000 7ff6f3400700 1 -- 172.21.15.19:0/4135853064 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff70002a030 con 0x55f819a0edc0 2024-05-07T02:33:22.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.188+0000 7ff6f3400700 1 -- 172.21.15.19:0/4135853064 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8366+0+0 (secure 0 0 0) 0x7ff6f80a8e70 con 0x55f819a0edc0 2024-05-07T02:33:22.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.188+0000 7ff722cccb80 10 client.15438 fetch_fsmap finished waiting for FSMap version 32 2024-05-07T02:33:22.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.188+0000 7ff722cccb80 10 client.15438 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:33:22.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.188+0000 7ff722cccb80 20 client.15438 populate_metadata read hostname 'smithi019' 2024-05-07T02:33:22.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.188+0000 7ff722cccb80 1 --2- 172.21.15.19:0/4135853064 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55f819a93a50 0x55f819a95e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:22.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.188+0000 7ff722cccb80 4 client.15438 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-07T02:33:22.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.188+0000 7ff722cccb80 1 -- 172.21.15.19:0/4135853064 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f81993e400 con 0x55f819a93a50 2024-05-07T02:33:22.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.189+0000 7ff711800700 1 --2- 172.21.15.19:0/4135853064 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55f819a93a50 0x55f819a95e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:22.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.189+0000 7ff711800700 1 --2- 172.21.15.19:0/4135853064 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55f819a93a50 0x55f819a95e70 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=266b756d9827a4dc server_cookie=2c2d9a7d9b3d9164 in_seq=0 out_seq=0 2024-05-07T02:33:22.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.189+0000 7ff6f3400700 10 client.15438 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:33:22.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.190+0000 7ff6f3400700 1 -- 172.21.15.19:0/4135853064 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f81993e400 con 0x55f819a93a50 2024-05-07T02:33:22.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.190+0000 7ff6f3400700 10 client.15438 handle_command_reply: tid=0 2024-05-07T02:33:22.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.190+0000 7ff722cccb80 1 client.15438 shutdown 2024-05-07T02:33:22.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.190+0000 7ff722cccb80 1 -- 172.21.15.19:0/4135853064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f819a0edc0 msgr2=0x55f819a30450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:22.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.190+0000 7ff722cccb80 1 --2- 172.21.15.19:0/4135853064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f819a0edc0 0x55f819a30450 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff6f8000c00 tx=0x7ff6f8009310 comp rx=0 tx=0).stop 2024-05-07T02:33:22.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.190+0000 7ff722cccb80 1 -- 172.21.15.19:0/4135853064 shutdown_connections 2024-05-07T02:33:22.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.190+0000 7ff722cccb80 1 --2- 172.21.15.19:0/4135853064 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55f819a93a50 0x55f819a95e70 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:22.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.190+0000 7ff722cccb80 1 --2- 172.21.15.19:0/4135853064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f819a143b0 0x55f819a30990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:22.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.190+0000 7ff722cccb80 1 --2- 172.21.15.19:0/4135853064 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f819a0e470 0x55f819a2ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:22.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.190+0000 7ff722cccb80 1 --2- 172.21.15.19:0/4135853064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f819a0edc0 0x55f819a30450 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:22.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.190+0000 7ff722cccb80 1 -- 172.21.15.19:0/4135853064 >> 172.21.15.19:0/4135853064 conn(0x55f819890e40 msgr2=0x55f819a038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:22.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.190+0000 7ff6f3400700 0 client.15438 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:33:22.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.190+0000 7ff722cccb80 1 -- 172.21.15.19:0/4135853064 shutdown_connections 2024-05-07T02:33:22.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.190+0000 7ff722cccb80 1 -- 172.21.15.19:0/4135853064 wait complete. 2024-05-07T02:33:22.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.190+0000 7ff722cccb80 20 client.15438 trim_cache size 0 max 16384 2024-05-07T02:33:22.193 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:33:22.193 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-07T02:33:22.193 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-07T02:33:22.193 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:33:22.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.191+0000 7ff714200700 1 -- 172.21.15.19:0/1257508824 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff6f40b29f0 msgr2=0x7ff6f40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:22.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.191+0000 7ff714200700 1 --2- 172.21.15.19:0/1257508824 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff6f40b29f0 0x7ff6f40b4f00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff70400f6e0 tx=0x7ff704009920 comp rx=0 tx=0).stop 2024-05-07T02:33:22.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.191+0000 7ff714200700 1 -- 172.21.15.19:0/1257508824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff70c108f00 msgr2=0x7ff70c19b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:22.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.191+0000 7ff714200700 1 --2- 172.21.15.19:0/1257508824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff70c108f00 0x7ff70c19b3b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff6f800b580 tx=0x7ff6f8005280 comp rx=0 tx=0).stop 2024-05-07T02:33:22.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.191+0000 7ff714200700 1 -- 172.21.15.19:0/1257508824 shutdown_connections 2024-05-07T02:33:22.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.191+0000 7ff714200700 1 --2- 172.21.15.19:0/1257508824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff70c109850 0x7ff70c19f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:22.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.191+0000 7ff714200700 1 --2- 172.21.15.19:0/1257508824 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff6f40b29f0 0x7ff6f40b4f00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:22.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.191+0000 7ff714200700 1 --2- 172.21.15.19:0/1257508824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff70c1045d0 0x7ff70c19ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:22.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.191+0000 7ff714200700 1 --2- 172.21.15.19:0/1257508824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff70c108f00 0x7ff70c19b3b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:22.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.191+0000 7ff714200700 1 -- 172.21.15.19:0/1257508824 >> 172.21.15.19:0/1257508824 conn(0x7ff70c0ffe80 msgr2=0x7ff70c10a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:22.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.192+0000 7ff714200700 1 -- 172.21.15.19:0/1257508824 shutdown_connections 2024-05-07T02:33:22.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:22.192+0000 7ff714200700 1 -- 172.21.15.19:0/1257508824 wait complete. 2024-05-07T02:33:22.722 INFO:tasks.workunit.client.0.smithi019.stdout: 1 32338 10.73 MB/sec warmup 18 sec latency 1779.796 ms 2024-05-07T02:33:23.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:22 smithi027 ceph-mon[34398]: pgmap v49: 129 pgs: 129 active+clean; 16 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 1.5 MiB/s wr, 189 op/s 2024-05-07T02:33:23.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:22 smithi027 ceph-mon[34398]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-05-07T02:33:23.227 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:22 smithi019 ceph-mon[29781]: pgmap v49: 129 pgs: 129 active+clean; 16 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 1.5 MiB/s wr, 189 op/s 2024-05-07T02:33:23.227 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:22 smithi019 ceph-mon[29781]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-05-07T02:33:23.231 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-07T02:33:23.231 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-07T02:33:23.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:22 smithi064 ceph-mon[34699]: pgmap v49: 129 pgs: 129 active+clean; 16 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 1.5 MiB/s wr, 189 op/s 2024-05-07T02:33:23.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:22 smithi064 ceph-mon[34699]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-05-07T02:33:23.722 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:33:23.722 INFO:tasks.workunit.client.0.smithi019.stdout: 1 32338 10.16 MB/sec warmup 19 sec latency 2779.893 ms 2024-05-07T02:33:24.722 INFO:tasks.workunit.client.0.smithi019.stdout: 1 32338 9.65 MB/sec warmup 20 sec latency 3779.993 ms 2024-05-07T02:33:25.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:24 smithi019 ceph-mon[29781]: pgmap v50: 129 pgs: 129 active+clean; 16 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 930 KiB/s wr, 123 op/s 2024-05-07T02:33:25.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:24 smithi019 ceph-mon[29781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:33:25.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:24 smithi027 ceph-mon[34398]: pgmap v50: 129 pgs: 129 active+clean; 16 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 930 KiB/s wr, 123 op/s 2024-05-07T02:33:25.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:24 smithi027 ceph-mon[34398]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:33:25.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:24 smithi064 ceph-mon[34699]: pgmap v50: 129 pgs: 129 active+clean; 16 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 930 KiB/s wr, 123 op/s 2024-05-07T02:33:25.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:24 smithi064 ceph-mon[34699]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:33:25.722 INFO:tasks.workunit.client.0.smithi019.stdout: 1 32350 9.19 MB/sec warmup 21 sec latency 4598.104 ms 2024-05-07T02:33:26.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.505+0000 7f7d7be00700 1 -- 172.21.15.19:0/2737952922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d7c10b110 msgr2=0x7f7d7c117620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:26.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.505+0000 7f7d7be00700 1 --2- 172.21.15.19:0/2737952922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d7c10b110 0x7f7d7c117620 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7d7400b0a0 tx=0x7f7d7400b3b0 comp rx=0 tx=0).stop 2024-05-07T02:33:26.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.505+0000 7f7d7be00700 1 -- 172.21.15.19:0/2737952922 shutdown_connections 2024-05-07T02:33:26.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.505+0000 7f7d7be00700 1 --2- 172.21.15.19:0/2737952922 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d7c10b110 0x7f7d7c117620 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:26.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.505+0000 7f7d7be00700 1 --2- 172.21.15.19:0/2737952922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d7c10f6c0 0x7f7d7c10fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:26.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.505+0000 7f7d7be00700 1 --2- 172.21.15.19:0/2737952922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d7c10a7c0 0x7f7d7c10abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:26.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.505+0000 7f7d7be00700 1 -- 172.21.15.19:0/2737952922 >> 172.21.15.19:0/2737952922 conn(0x7f7d7c076b00 msgr2=0x7f7d7c076f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:26.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.505+0000 7f7d7be00700 1 -- 172.21.15.19:0/2737952922 shutdown_connections 2024-05-07T02:33:26.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.505+0000 7f7d7be00700 1 -- 172.21.15.19:0/2737952922 wait complete. 2024-05-07T02:33:26.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.507+0000 7f7d7be00700 1 Processor -- start 2024-05-07T02:33:26.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.507+0000 7f7d7be00700 1 -- start start 2024-05-07T02:33:26.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.508+0000 7f7d7be00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d7c10a7c0 0x7f7d7c1a1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:26.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.508+0000 7f7d7be00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d7c10b110 0x7f7d7c1a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:26.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.508+0000 7f7d7be00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d7c10f6c0 0x7f7d7c1a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:26.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.508+0000 7f7d7be00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c11a8f0 con 0x7f7d7c10f6c0 2024-05-07T02:33:26.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.508+0000 7f7d7be00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c11a770 con 0x7f7d7c10a7c0 2024-05-07T02:33:26.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.508+0000 7f7d7be00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c11aa70 con 0x7f7d7c10b110 2024-05-07T02:33:26.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.508+0000 7f7d7aa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d7c10a7c0 0x7f7d7c1a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:26.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.508+0000 7f7d7aa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d7c10a7c0 0x7f7d7c1a1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:39318/0 (socket says 172.21.15.19:39318) 2024-05-07T02:33:26.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.508+0000 7f7d7aa00700 1 -- 172.21.15.19:0/1059580090 learned_addr learned my addr 172.21.15.19:0/1059580090 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:26.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.508+0000 7f7d7a000700 1 --2- 172.21.15.19:0/1059580090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d7c10b110 0x7f7d7c1a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:26.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.508+0000 7f7d7b400700 1 --2- 172.21.15.19:0/1059580090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d7c10f6c0 0x7f7d7c1a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:26.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.509+0000 7f7d7aa00700 1 -- 172.21.15.19:0/1059580090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d7c10b110 msgr2=0x7f7d7c1a17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:26.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.509+0000 7f7d7aa00700 1 --2- 172.21.15.19:0/1059580090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d7c10b110 0x7f7d7c1a17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:26.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.509+0000 7f7d7aa00700 1 -- 172.21.15.19:0/1059580090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d7c10f6c0 msgr2=0x7f7d7c1a5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:26.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.509+0000 7f7d7aa00700 1 --2- 172.21.15.19:0/1059580090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d7c10f6c0 0x7f7d7c1a5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:26.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.509+0000 7f7d7aa00700 1 -- 172.21.15.19:0/1059580090 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d7c1a60a0 con 0x7f7d7c10a7c0 2024-05-07T02:33:26.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.509+0000 7f7d7aa00700 1 --2- 172.21.15.19:0/1059580090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d7c10a7c0 0x7f7d7c1a1220 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c00eb30 tx=0x7f7d6c00ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:26.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.509+0000 7f7d73400700 1 -- 172.21.15.19:0/1059580090 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c00cbd0 con 0x7f7d7c10a7c0 2024-05-07T02:33:26.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.509+0000 7f7d7be00700 1 -- 172.21.15.19:0/1059580090 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d7c1a6390 con 0x7f7d7c10a7c0 2024-05-07T02:33:26.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.509+0000 7f7d73400700 1 -- 172.21.15.19:0/1059580090 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d6c0044d0 con 0x7f7d7c10a7c0 2024-05-07T02:33:26.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.509+0000 7f7d73400700 1 -- 172.21.15.19:0/1059580090 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c003e30 con 0x7f7d7c10a7c0 2024-05-07T02:33:26.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.510+0000 7f7d7be00700 1 -- 172.21.15.19:0/1059580090 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d7c1adcb0 con 0x7f7d7c10a7c0 2024-05-07T02:33:26.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.511+0000 7f7d70c00700 1 -- 172.21.15.19:0/1059580090 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d7c0512c0 con 0x7f7d7c10a7c0 2024-05-07T02:33:26.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.513+0000 7f7d73400700 1 -- 172.21.15.19:0/1059580090 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7d6c004000 con 0x7f7d7c10a7c0 2024-05-07T02:33:26.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.515+0000 7f7d73400700 1 --2- 172.21.15.19:0/1059580090 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f7d600b29f0 0x7f7d600b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:26.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.515+0000 7f7d73400700 1 -- 172.21.15.19:0/1059580090 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f7d6c0e2f50 con 0x7f7d7c10a7c0 2024-05-07T02:33:26.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.515+0000 7f7d7a000700 1 --2- 172.21.15.19:0/1059580090 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f7d600b29f0 0x7f7d600b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:26.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.516+0000 7f7d7a000700 1 --2- 172.21.15.19:0/1059580090 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f7d600b29f0 0x7f7d600b4f00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c077ec0 tx=0x7f7d68009260 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:26.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.517+0000 7f7d73400700 1 -- 172.21.15.19:0/1059580090 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7d6c054df0 con 0x7f7d7c10a7c0 2024-05-07T02:33:26.722 INFO:tasks.workunit.client.0.smithi019.stdout: 1 32350 8.78 MB/sec warmup 22 sec latency 1157.378 ms 2024-05-07T02:33:26.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.798+0000 7f7d70c00700 1 -- 172.21.15.19:0/1059580090 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d7c1a2530 con 0x7f7d7c10a7c0 2024-05-07T02:33:26.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.799+0000 7f7d73400700 1 -- 172.21.15.19:0/1059580090 <== mon.2 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11927 (secure 0 0 0) 0x7f7d6c0042f0 con 0x7f7d7c10a7c0 2024-05-07T02:33:26.800 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:33:26.800 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1763670446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1763670446},{"type":"v1","addr":"172.21.15.19:6839","nonce":1763670446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/477529985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":477529985},{"type":"v1","addr":"172.21.15.64:6833","nonce":477529985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-07T02:26:14.960465+0000","modified":"2024-05-07T02:33:23.639249+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24539,"mds_2":24520,"mds_3":24515,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6833/2171873162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2171873162},{"type":"v1","addr":"172.21.15.27:6833","nonce":2171873162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/2379917839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2379917839},{"type":"v1","addr":"172.21.15.19:6835","nonce":2379917839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6837/2067515538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2067515538},{"type":"v1","addr":"172.21.15.19:6837","nonce":2067515538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6835/1000540600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1000540600},{"type":"v1","addr":"172.21.15.27:6835","nonce":1000540600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/1997318872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1997318872},{"type":"v1","addr":"172.21.15.64:6835","nonce":1997318872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":34,"addr":"172.21.15.64:6837/2585907777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2585907777},{"type":"v1","addr":"172.21.15.64:6837","nonce":2585907777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":33,"addr":"172.21.15.27:6837/2583138143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2583138143},{"type":"v1","addr":"172.21.15.27:6837","nonce":2583138143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":32,"addr":"172.21.15.64:6839/1583120004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":1583120004},{"type":"v1","addr":"172.21.15.64:6839","nonce":1583120004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":29,"addr":"172.21.15.19:6841/3905645246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3905645246},{"type":"v1","addr":"172.21.15.19:6841","nonce":3905645246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6839/3963268634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3963268634},{"type":"v1","addr":"172.21.15.27:6839","nonce":3963268634}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24505,24515,24520,24539,24551]},"id":1}]} 2024-05-07T02:33:26.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.802+0000 7f7d70c00700 1 -- 172.21.15.19:0/1059580090 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f7d600b29f0 msgr2=0x7f7d600b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:26.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.802+0000 7f7d70c00700 1 --2- 172.21.15.19:0/1059580090 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f7d600b29f0 0x7f7d600b4f00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c077ec0 tx=0x7f7d68009260 comp rx=0 tx=0).stop 2024-05-07T02:33:26.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.802+0000 7f7d70c00700 1 -- 172.21.15.19:0/1059580090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d7c10a7c0 msgr2=0x7f7d7c1a1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:26.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.802+0000 7f7d70c00700 1 --2- 172.21.15.19:0/1059580090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d7c10a7c0 0x7f7d7c1a1220 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c00eb30 tx=0x7f7d6c00ee40 comp rx=0 tx=0).stop 2024-05-07T02:33:26.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.802+0000 7f7d70c00700 1 -- 172.21.15.19:0/1059580090 shutdown_connections 2024-05-07T02:33:26.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.802+0000 7f7d70c00700 1 --2- 172.21.15.19:0/1059580090 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f7d600b29f0 0x7f7d600b4f00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:26.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.802+0000 7f7d70c00700 1 --2- 172.21.15.19:0/1059580090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d7c10f6c0 0x7f7d7c1a5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:26.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.803+0000 7f7d70c00700 1 --2- 172.21.15.19:0/1059580090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d7c10a7c0 0x7f7d7c1a1220 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:26.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.803+0000 7f7d70c00700 1 --2- 172.21.15.19:0/1059580090 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7d7c10b110 0x7f7d7c1a17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:26.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.803+0000 7f7d70c00700 1 -- 172.21.15.19:0/1059580090 >> 172.21.15.19:0/1059580090 conn(0x7f7d7c076b00 msgr2=0x7f7d7c108860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:26.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.803+0000 7f7d70c00700 1 -- 172.21.15.19:0/1059580090 shutdown_connections 2024-05-07T02:33:26.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:26.803+0000 7f7d70c00700 1 -- 172.21.15.19:0/1059580090 wait complete. 2024-05-07T02:33:26.804 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 33 2024-05-07T02:33:27.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:26 smithi019 ceph-mon[29781]: pgmap v51: 129 pgs: 129 active+clean; 17 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 944 KiB/s wr, 131 op/s 2024-05-07T02:33:27.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:26 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/1059580090' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:33:27.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:26 smithi027 ceph-mon[34398]: pgmap v51: 129 pgs: 129 active+clean; 17 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 944 KiB/s wr, 131 op/s 2024-05-07T02:33:27.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:26 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/1059580090' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:33:27.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:26 smithi064 ceph-mon[34699]: pgmap v51: 129 pgs: 129 active+clean; 17 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 944 KiB/s wr, 131 op/s 2024-05-07T02:33:27.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:26 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/1059580090' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:33:27.722 INFO:tasks.workunit.client.0.smithi019.stdout: 1 32350 8.39 MB/sec warmup 23 sec latency 2157.499 ms 2024-05-07T02:33:27.751 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-07T02:33:28.237 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:33:28.722 INFO:tasks.workunit.client.0.smithi019.stdout: 1 32350 8.04 MB/sec warmup 24 sec latency 3157.665 ms 2024-05-07T02:33:29.015 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:28 smithi019 ceph-mon[29781]: pgmap v52: 129 pgs: 129 active+clean; 17 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 439 KiB/s wr, 67 op/s 2024-05-07T02:33:29.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:28 smithi027 ceph-mon[34398]: pgmap v52: 129 pgs: 129 active+clean; 17 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 439 KiB/s wr, 67 op/s 2024-05-07T02:33:29.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:28 smithi064 ceph-mon[34699]: pgmap v52: 129 pgs: 129 active+clean; 17 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 439 KiB/s wr, 67 op/s 2024-05-07T02:33:29.722 INFO:tasks.workunit.client.0.smithi019.stdout: 1 32350 7.72 MB/sec warmup 25 sec latency 4157.768 ms 2024-05-07T02:33:30.271 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:29 smithi019 ceph-mon[29781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:33:30.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:29 smithi064 ceph-mon[34699]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:33:30.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:29 smithi027 ceph-mon[34398]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:33:30.723 INFO:tasks.workunit.client.0.smithi019.stdout: 1 32396 7.43 MB/sec warmup 26 sec latency 5043.249 ms 2024-05-07T02:33:30.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.963+0000 7f148fe00700 1 -- 172.21.15.19:0/597288212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f148810e910 msgr2=0x7f1488110df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:30.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.963+0000 7f148fe00700 1 --2- 172.21.15.19:0/597288212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f148810e910 0x7f1488110df0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f148000b0a0 tx=0x7f148000b3b0 comp rx=0 tx=0).stop 2024-05-07T02:33:30.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.963+0000 7f148fe00700 1 -- 172.21.15.19:0/597288212 shutdown_connections 2024-05-07T02:33:30.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.963+0000 7f148fe00700 1 --2- 172.21.15.19:0/597288212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f148810e910 0x7f1488110df0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:30.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.963+0000 7f148fe00700 1 --2- 172.21.15.19:0/597288212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14881054e0 0x7f14881058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:30.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.963+0000 7f148fe00700 1 --2- 172.21.15.19:0/597288212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1488105e30 0x7f148810e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:30.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.963+0000 7f148fe00700 1 -- 172.21.15.19:0/597288212 >> 172.21.15.19:0/597288212 conn(0x7f14880fedc0 msgr2=0x7f14881011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:30.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.964+0000 7f148fe00700 1 -- 172.21.15.19:0/597288212 shutdown_connections 2024-05-07T02:33:30.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.964+0000 7f148fe00700 1 -- 172.21.15.19:0/597288212 wait complete. 2024-05-07T02:33:30.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.965+0000 7f148fe00700 1 Processor -- start 2024-05-07T02:33:30.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.966+0000 7f148fe00700 1 -- start start 2024-05-07T02:33:30.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.966+0000 7f148fe00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14881054e0 0x7f148819ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:30.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.966+0000 7f148fe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1488105e30 0x7f148819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:30.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.966+0000 7f148fe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f148810e910 0x7f148819f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:30.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.966+0000 7f148fe00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1488114490 con 0x7f1488105e30 2024-05-07T02:33:30.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.966+0000 7f148fe00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1488114310 con 0x7f148810e910 2024-05-07T02:33:30.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.966+0000 7f148fe00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1488114610 con 0x7f14881054e0 2024-05-07T02:33:30.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.967+0000 7f148d400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14881054e0 0x7f148819ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:30.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.967+0000 7f148d400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14881054e0 0x7f148819ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:59392/0 (socket says 172.21.15.19:59392) 2024-05-07T02:33:30.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.967+0000 7f148d400700 1 -- 172.21.15.19:0/200851360 learned_addr learned my addr 172.21.15.19:0/200851360 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:30.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.967+0000 7f148ca00700 1 --2- 172.21.15.19:0/200851360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1488105e30 0x7f148819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:30.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.967+0000 7f148de00700 1 --2- 172.21.15.19:0/200851360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f148810e910 0x7f148819f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:30.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.967+0000 7f148d400700 1 -- 172.21.15.19:0/200851360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f148810e910 msgr2=0x7f148819f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:30.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.967+0000 7f148d400700 1 --2- 172.21.15.19:0/200851360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f148810e910 0x7f148819f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:30.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.967+0000 7f148d400700 1 -- 172.21.15.19:0/200851360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1488105e30 msgr2=0x7f148819b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:30.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.967+0000 7f148d400700 1 --2- 172.21.15.19:0/200851360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1488105e30 0x7f148819b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:30.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.967+0000 7f148d400700 1 -- 172.21.15.19:0/200851360 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f148819fc70 con 0x7f14881054e0 2024-05-07T02:33:30.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.967+0000 7f148ca00700 1 --2- 172.21.15.19:0/200851360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1488105e30 0x7f148819b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:33:30.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.967+0000 7f148d400700 1 --2- 172.21.15.19:0/200851360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14881054e0 0x7f148819ada0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f147c00e9a0 tx=0x7f147c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:30.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.968+0000 7f1486000700 1 -- 172.21.15.19:0/200851360 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c00cbd0 con 0x7f14881054e0 2024-05-07T02:33:30.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.968+0000 7f1486000700 1 -- 172.21.15.19:0/200851360 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f147c00cd70 con 0x7f14881054e0 2024-05-07T02:33:30.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.968+0000 7f148fe00700 1 -- 172.21.15.19:0/200851360 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f148819ff60 con 0x7f14881054e0 2024-05-07T02:33:30.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.968+0000 7f1486000700 1 -- 172.21.15.19:0/200851360 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c018680 con 0x7f14881054e0 2024-05-07T02:33:30.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.968+0000 7f148fe00700 1 -- 172.21.15.19:0/200851360 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14881a7880 con 0x7f14881054e0 2024-05-07T02:33:30.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.973+0000 7f1486000700 1 -- 172.21.15.19:0/200851360 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f147c010430 con 0x7f14881054e0 2024-05-07T02:33:30.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.974+0000 7f149ea41b80 1 Processor -- start 2024-05-07T02:33:30.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.974+0000 7f149ea41b80 1 -- start start 2024-05-07T02:33:30.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.974+0000 7f1486000700 1 --2- 172.21.15.19:0/200851360 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f14700b29f0 0x7f14700b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:30.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.975+0000 7f1486000700 1 -- 172.21.15.19:0/200851360 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f147c0e2c90 con 0x7f14881054e0 2024-05-07T02:33:30.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.975+0000 7f149ea41b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a0e11c470 0x563a0e11c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:30.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.975+0000 7f149ea41b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e116620 0x563a0e116a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:30.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.975+0000 7f149ea41b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a0e116f70 0x563a0e127560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:30.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.975+0000 7f149ea41b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563a0e038a20 con 0x563a0e116f70 2024-05-07T02:33:30.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.975+0000 7f149ea41b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x563a0dfd6760 con 0x563a0e11c470 2024-05-07T02:33:30.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.975+0000 7f149ea41b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563a0e0349e0 con 0x563a0e116620 2024-05-07T02:33:30.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.975+0000 7f148d400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a0e11c470 0x563a0e11c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:30.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.975+0000 7f148d400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a0e11c470 0x563a0e11c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:50454/0 (socket says 172.21.15.19:50454) 2024-05-07T02:33:30.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.975+0000 7f148d400700 1 -- 172.21.15.19:0/245953705 learned_addr learned my addr 172.21.15.19:0/245953705 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:30.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.975+0000 7f148de00700 1 --2- 172.21.15.19:0/200851360 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f14700b29f0 0x7f14700b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:30.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.975+0000 7f148ca00700 1 --2- 172.21.15.19:0/245953705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e116620 0x563a0e116a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:30.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.976+0000 7f148d400700 1 -- 172.21.15.19:0/245953705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e116620 msgr2=0x563a0e116a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:30.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.976+0000 7f148d400700 1 --2- 172.21.15.19:0/245953705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e116620 0x563a0e116a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:30.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.976+0000 7f148d400700 1 -- 172.21.15.19:0/245953705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a0e116f70 msgr2=0x563a0e127560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:30.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.976+0000 7f148d400700 1 --2- 172.21.15.19:0/245953705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a0e116f70 0x563a0e127560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:30.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.976+0000 7f148d400700 1 -- 172.21.15.19:0/245953705 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a0df858d0 con 0x563a0e11c470 2024-05-07T02:33:30.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.976+0000 7f148ca00700 1 --2- 172.21.15.19:0/245953705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e116620 0x563a0e116a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-07T02:33:30.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.976+0000 7f148d400700 1 --2- 172.21.15.19:0/245953705 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a0e11c470 0x563a0e11c880 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f147c010c60 tx=0x7f147c0b2320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2229a6df30a55d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:30.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.976+0000 7f147aa00700 1 -- 172.21.15.19:0/245953705 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c0a9b80 con 0x563a0e11c470 2024-05-07T02:33:30.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.977+0000 7f147aa00700 1 -- 172.21.15.19:0/245953705 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f147c0adf30 con 0x563a0e11c470 2024-05-07T02:33:30.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.977+0000 7f147aa00700 1 -- 172.21.15.19:0/245953705 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c0b27b0 con 0x563a0e11c470 2024-05-07T02:33:30.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.978+0000 7f149ea41b80 1 -- 172.21.15.19:0/245953705 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a0e11c470 msgr2=0x563a0e11c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:30.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.978+0000 7f148de00700 1 --2- 172.21.15.19:0/200851360 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f14700b29f0 0x7f14700b4f00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f148000b780 tx=0x7f1480009d80 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:30.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.978+0000 7f149ea41b80 1 --2- 172.21.15.19:0/245953705 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a0e11c470 0x563a0e11c880 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f147c010c60 tx=0x7f147c0b2320 comp rx=0 tx=0).stop 2024-05-07T02:33:30.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.978+0000 7f149ea41b80 1 -- 172.21.15.19:0/245953705 shutdown_connections 2024-05-07T02:33:30.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.978+0000 7f149ea41b80 1 --2- 172.21.15.19:0/245953705 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a0e116f70 0x563a0e127560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:30.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.978+0000 7f149ea41b80 1 --2- 172.21.15.19:0/245953705 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a0e11c470 0x563a0e11c880 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:30.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.978+0000 7f149ea41b80 1 --2- 172.21.15.19:0/245953705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e116620 0x563a0e116a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:30.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.978+0000 7f149ea41b80 1 -- 172.21.15.19:0/245953705 >> 172.21.15.19:0/245953705 conn(0x563a0df98e40 msgr2=0x563a0e111380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:30.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.978+0000 7f149ea41b80 1 -- 172.21.15.19:0/245953705 shutdown_connections 2024-05-07T02:33:30.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.978+0000 7f149ea41b80 1 -- 172.21.15.19:0/245953705 wait complete. 2024-05-07T02:33:30.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.979+0000 7f149ea41b80 1 -- start start 2024-05-07T02:33:30.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.979+0000 7f149ea41b80 1 Processor -- start 2024-05-07T02:33:30.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.980+0000 7f149ea41b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e116620 0x563a0e1218e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:30.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.980+0000 7f149ea41b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a0e116f70 0x563a0e121e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:30.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.980+0000 7f149ea41b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a0e11c470 0x563a0e126730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:30.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.980+0000 7f149ea41b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563a0e121290 con 0x563a0e116f70 2024-05-07T02:33:30.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.980+0000 7f149ea41b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x563a0df9eb20 con 0x563a0e11c470 2024-05-07T02:33:30.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.980+0000 7f149ea41b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563a0e121410 con 0x563a0e116620 2024-05-07T02:33:30.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.980+0000 7f149ea41b80 10 client.0 _ll_register_callbacks cb 0x563a0dfcc380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:33:30.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.980+0000 7f148d400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e116620 0x563a0e1218e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:30.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.980+0000 7f148d400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e116620 0x563a0e1218e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:59406/0 (socket says 172.21.15.19:59406) 2024-05-07T02:33:30.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.980+0000 7f148d400700 1 -- 172.21.15.19:0/808551 learned_addr learned my addr 172.21.15.19:0/808551 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:30.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.980+0000 7f148de00700 1 --2- 172.21.15.19:0/808551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a0e11c470 0x563a0e126730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:30.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.981+0000 7f148d400700 1 -- 172.21.15.19:0/808551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a0e11c470 msgr2=0x563a0e126730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:30.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.981+0000 7f148d400700 1 --2- 172.21.15.19:0/808551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a0e11c470 0x563a0e126730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:30.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.981+0000 7f148d400700 1 -- 172.21.15.19:0/808551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a0e116f70 msgr2=0x563a0e121e20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-07T02:33:30.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.981+0000 7f148d400700 1 --2- 172.21.15.19:0/808551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a0e116f70 0x563a0e121e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:30.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.981+0000 7f148d400700 1 -- 172.21.15.19:0/808551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a0dffe420 con 0x563a0e116620 2024-05-07T02:33:30.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.981+0000 7f148de00700 1 --2- 172.21.15.19:0/808551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a0e11c470 0x563a0e126730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:33:30.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.981+0000 7f148d400700 1 --2- 172.21.15.19:0/808551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e116620 0x563a0e1218e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f147c0a9880 tx=0x7f147c0b2890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9e7af8fabd1e973 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:30.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.981+0000 7f146f400700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:33:30.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.981+0000 7f146f400700 1 -- 172.21.15.19:0/808551 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c06e060 con 0x563a0e116620 2024-05-07T02:33:30.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.981+0000 7f149ea41b80 1 -- 172.21.15.19:0/808551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f147c0adf30 con 0x563a0e116620 2024-05-07T02:33:30.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.981+0000 7f146f400700 1 -- 172.21.15.19:0/808551 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f147c010930 con 0x563a0e116620 2024-05-07T02:33:30.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.981+0000 7f146f400700 1 -- 172.21.15.19:0/808551 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f147c0b29e0 con 0x563a0e116620 2024-05-07T02:33:30.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.981+0000 7f149ea41b80 10 client.25585 fetch_fsmap learned FSMap version 34 2024-05-07T02:33:30.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.981+0000 7f149ea41b80 1 -- 172.21.15.19:0/808551 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563a0e121290 con 0x563a0e116620 2024-05-07T02:33:30.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.982+0000 7f146f400700 1 -- 172.21.15.19:0/808551 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f147c0adf30 con 0x563a0e116620 2024-05-07T02:33:30.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.982+0000 7f146f400700 1 -- 172.21.15.19:0/808551 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f147c06f430 con 0x563a0e116620 2024-05-07T02:33:30.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.982+0000 7f149ea41b80 10 client.25585 fetch_fsmap finished waiting for FSMap version 34 2024-05-07T02:33:30.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.982+0000 7f149ea41b80 10 client.25585 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-07T02:33:30.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.982+0000 7f149ea41b80 20 client.25585 populate_metadata read hostname 'smithi019' 2024-05-07T02:33:30.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.982+0000 7f149ea41b80 1 --2- 172.21.15.19:0/808551 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x563a0e138f40 0x563a0e13b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:30.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.982+0000 7f149ea41b80 4 client.25585 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:33:30.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.982+0000 7f149ea41b80 1 -- 172.21.15.19:0/808551 --> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a0e0e30e0 con 0x563a0e138f40 2024-05-07T02:33:30.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.983+0000 7f148ca00700 1 --2- 172.21.15.19:0/808551 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x563a0e138f40 0x563a0e13b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:30.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.983+0000 7f148ca00700 1 --2- 172.21.15.19:0/808551 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x563a0e138f40 0x563a0e13b360 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=db677d68e11a6ce5 server_cookie=7bb59405ad3da2e0 in_seq=0 out_seq=0 2024-05-07T02:33:30.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.983+0000 7f146f400700 10 client.25585 ms_handle_connect on v2:172.21.15.64:6836/2585907777 2024-05-07T02:33:30.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.984+0000 7f146f400700 1 -- 172.21.15.19:0/808551 <== mds.4 v2:172.21.15.64:6836/2585907777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563a0e0e30e0 con 0x563a0e138f40 2024-05-07T02:33:30.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.984+0000 7f146f400700 10 client.25585 handle_command_reply: tid=0 2024-05-07T02:33:30.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.984+0000 7f149ea41b80 1 client.25585 shutdown 2024-05-07T02:33:30.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.984+0000 7f149ea41b80 1 -- 172.21.15.19:0/808551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e116620 msgr2=0x563a0e1218e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:30.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.984+0000 7f149ea41b80 1 --2- 172.21.15.19:0/808551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e116620 0x563a0e1218e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f147c0a9880 tx=0x7f147c0b2890 comp rx=0 tx=0).stop 2024-05-07T02:33:30.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.985+0000 7f149ea41b80 1 -- 172.21.15.19:0/808551 shutdown_connections 2024-05-07T02:33:30.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.985+0000 7f149ea41b80 1 --2- 172.21.15.19:0/808551 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x563a0e138f40 0x563a0e13b360 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:30.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.985+0000 7f149ea41b80 1 --2- 172.21.15.19:0/808551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a0e11c470 0x563a0e126730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:30.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.985+0000 7f149ea41b80 1 --2- 172.21.15.19:0/808551 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e116620 0x563a0e1218e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:30.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.985+0000 7f149ea41b80 1 --2- 172.21.15.19:0/808551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a0e116f70 0x563a0e121e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:30.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.985+0000 7f146f400700 0 client.25585 ms_handle_reset on v2:172.21.15.64:6836/2585907777 2024-05-07T02:33:30.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.985+0000 7f149ea41b80 1 -- 172.21.15.19:0/808551 >> 172.21.15.19:0/808551 conn(0x563a0df98e40 msgr2=0x563a0e115610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:30.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.985+0000 7f149ea41b80 1 -- 172.21.15.19:0/808551 shutdown_connections 2024-05-07T02:33:30.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.985+0000 7f149ea41b80 1 -- 172.21.15.19:0/808551 wait complete. 2024-05-07T02:33:30.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:30.985+0000 7f149ea41b80 20 client.25585 trim_cache size 0 max 16384 2024-05-07T02:33:31.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.006+0000 7f149ea41b80 1 Processor -- start 2024-05-07T02:33:31.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.006+0000 7f149ea41b80 1 -- start start 2024-05-07T02:33:31.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.007+0000 7f149ea41b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a0e116620 0x563a0e120b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:31.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.007+0000 7f149ea41b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e116f70 0x563a0e11fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:31.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.007+0000 7f149ea41b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a0e11c470 0x563a0e10b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:31.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.007+0000 7f149ea41b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f14881ae340 con 0x563a0e116620 2024-05-07T02:33:31.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.007+0000 7f149ea41b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f14881aff40 con 0x563a0e11c470 2024-05-07T02:33:31.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.007+0000 7f149ea41b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f14881aea10 con 0x563a0e116f70 2024-05-07T02:33:31.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.007+0000 7f148d400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a0e116620 0x563a0e120b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:31.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.007+0000 7f148ca00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e116f70 0x563a0e11fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:31.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.007+0000 7f148d400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a0e116620 0x563a0e120b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46970/0 (socket says 172.21.15.19:46970) 2024-05-07T02:33:31.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.007+0000 7f148d400700 1 -- 172.21.15.19:0/1467291383 learned_addr learned my addr 172.21.15.19:0/1467291383 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:31.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.007+0000 7f148de00700 1 --2- 172.21.15.19:0/1467291383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a0e11c470 0x563a0e10b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:31.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.008+0000 7f148ca00700 1 -- 172.21.15.19:0/1467291383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a0e11c470 msgr2=0x563a0e10b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:31.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.008+0000 7f148ca00700 1 --2- 172.21.15.19:0/1467291383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a0e11c470 0x563a0e10b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:31.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.008+0000 7f148ca00700 1 -- 172.21.15.19:0/1467291383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a0e116620 msgr2=0x563a0e120b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:31.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.008+0000 7f148ca00700 1 --2- 172.21.15.19:0/1467291383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a0e116620 0x563a0e120b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:31.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.008+0000 7f148ca00700 1 -- 172.21.15.19:0/1467291383 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a0e10db80 con 0x563a0e116f70 2024-05-07T02:33:31.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.008+0000 7f148ca00700 1 --2- 172.21.15.19:0/1467291383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e116f70 0x563a0e11fe80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x563a0e13b8a0 tx=0x7f1474009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72480d39e79a3e3c server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:31.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.009+0000 7f147a000700 1 -- 172.21.15.19:0/1467291383 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1474010e10 con 0x563a0e116f70 2024-05-07T02:33:31.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.009+0000 7f147a000700 1 -- 172.21.15.19:0/1467291383 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f147401d030 con 0x563a0e116f70 2024-05-07T02:33:31.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.009+0000 7f147a000700 1 -- 172.21.15.19:0/1467291383 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147400f9f0 con 0x563a0e116f70 2024-05-07T02:33:31.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.010+0000 7f149ea41b80 1 -- 172.21.15.19:0/1467291383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e116f70 msgr2=0x563a0e11fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:31.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.010+0000 7f149ea41b80 1 --2- 172.21.15.19:0/1467291383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e116f70 0x563a0e11fe80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x563a0e13b8a0 tx=0x7f1474009260 comp rx=0 tx=0).stop 2024-05-07T02:33:31.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.010+0000 7f149ea41b80 1 -- 172.21.15.19:0/1467291383 shutdown_connections 2024-05-07T02:33:31.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.010+0000 7f149ea41b80 1 --2- 172.21.15.19:0/1467291383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a0e11c470 0x563a0e10b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:31.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.010+0000 7f149ea41b80 1 --2- 172.21.15.19:0/1467291383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a0e116620 0x563a0e120b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:31.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.010+0000 7f149ea41b80 1 --2- 172.21.15.19:0/1467291383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e116f70 0x563a0e11fe80 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:31.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.010+0000 7f149ea41b80 1 -- 172.21.15.19:0/1467291383 >> 172.21.15.19:0/1467291383 conn(0x563a0df98e40 msgr2=0x563a0e10b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:31.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.010+0000 7f149ea41b80 1 -- 172.21.15.19:0/1467291383 shutdown_connections 2024-05-07T02:33:31.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.010+0000 7f149ea41b80 1 -- 172.21.15.19:0/1467291383 wait complete. 2024-05-07T02:33:31.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.010+0000 7f149ea41b80 1 -- start start 2024-05-07T02:33:31.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.011+0000 7f149ea41b80 1 Processor -- start 2024-05-07T02:33:31.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.011+0000 7f149ea41b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e116620 0x563a0e132aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:31.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.011+0000 7f149ea41b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a0e116f70 0x563a0e132fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:31.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.011+0000 7f149ea41b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a0e11c470 0x563a0e1a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:31.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.011+0000 7f149ea41b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f14881af7b0 con 0x563a0e11c470 2024-05-07T02:33:31.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.011+0000 7f149ea41b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f14881af0e0 con 0x563a0e116f70 2024-05-07T02:33:31.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.011+0000 7f149ea41b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f14881ac300 con 0x563a0e116620 2024-05-07T02:33:31.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.011+0000 7f149ea41b80 10 client.0 _ll_register_callbacks cb 0x563a0dfcc380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:33:31.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.012+0000 7f148d400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e116620 0x563a0e132aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:31.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.012+0000 7f148ca00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a0e116f70 0x563a0e132fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:31.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.012+0000 7f148d400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e116620 0x563a0e132aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:59414/0 (socket says 172.21.15.19:59414) 2024-05-07T02:33:31.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.012+0000 7f148d400700 1 -- 172.21.15.19:0/3114078017 learned_addr learned my addr 172.21.15.19:0/3114078017 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:31.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.012+0000 7f148de00700 1 --2- 172.21.15.19:0/3114078017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a0e11c470 0x563a0e1a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:31.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.012+0000 7f148d400700 1 -- 172.21.15.19:0/3114078017 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a0e116f70 msgr2=0x563a0e132fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:31.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.012+0000 7f148d400700 1 --2- 172.21.15.19:0/3114078017 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a0e116f70 0x563a0e132fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:31.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.012+0000 7f148d400700 1 -- 172.21.15.19:0/3114078017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a0e11c470 msgr2=0x563a0e1a14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:31.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.012+0000 7f148d400700 1 --2- 172.21.15.19:0/3114078017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a0e11c470 0x563a0e1a14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:31.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.012+0000 7f148d400700 1 -- 172.21.15.19:0/3114078017 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a0e1a1a40 con 0x563a0e116620 2024-05-07T02:33:31.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.012+0000 7f148de00700 1 --2- 172.21.15.19:0/3114078017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a0e11c470 0x563a0e1a14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-07T02:33:31.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.012+0000 7f148d400700 1 --2- 172.21.15.19:0/3114078017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e116620 0x563a0e132aa0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f147c0a6df0 tx=0x7f147c0e00d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a4bfea49d4677bc server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:31.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.012+0000 7f146f400700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:33:31.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.013+0000 7f146f400700 1 -- 172.21.15.19:0/3114078017 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c06e040 con 0x563a0e116620 2024-05-07T02:33:31.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.013+0000 7f146f400700 1 -- 172.21.15.19:0/3114078017 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f147c0c4170 con 0x563a0e116620 2024-05-07T02:33:31.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.013+0000 7f149ea41b80 1 -- 172.21.15.19:0/3114078017 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f147401d030 con 0x563a0e116620 2024-05-07T02:33:31.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.013+0000 7f146f400700 1 -- 172.21.15.19:0/3114078017 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f147c08e050 con 0x563a0e116620 2024-05-07T02:33:31.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.013+0000 7f149ea41b80 10 client.25591 fetch_fsmap learned FSMap version 34 2024-05-07T02:33:31.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.013+0000 7f149ea41b80 1 -- 172.21.15.19:0/3114078017 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563a0e1a1a40 con 0x563a0e116620 2024-05-07T02:33:31.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.013+0000 7f146f400700 1 -- 172.21.15.19:0/3114078017 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f147401d030 con 0x563a0e116620 2024-05-07T02:33:31.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.013+0000 7f146f400700 1 -- 172.21.15.19:0/3114078017 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f147c06f250 con 0x563a0e116620 2024-05-07T02:33:31.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.014+0000 7f149ea41b80 10 client.25591 fetch_fsmap finished waiting for FSMap version 34 2024-05-07T02:33:31.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.014+0000 7f149ea41b80 10 client.25591 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-07T02:33:31.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.014+0000 7f149ea41b80 20 client.25591 populate_metadata read hostname 'smithi019' 2024-05-07T02:33:31.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.014+0000 7f149ea41b80 1 --2- 172.21.15.19:0/3114078017 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x563a0e1a2f90 0x563a0e1a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:31.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.014+0000 7f149ea41b80 4 client.25591 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:33:31.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.014+0000 7f149ea41b80 1 -- 172.21.15.19:0/3114078017 --> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563a0e046400 con 0x563a0e1a2f90 2024-05-07T02:33:31.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.015+0000 7f148ca00700 1 --2- 172.21.15.19:0/3114078017 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x563a0e1a2f90 0x563a0e1a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:31.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.016+0000 7f148ca00700 1 --2- 172.21.15.19:0/3114078017 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x563a0e1a2f90 0x563a0e1a53b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=aa00bdda05602a96 server_cookie=fbed64db4d3f4e80 in_seq=0 out_seq=0 2024-05-07T02:33:31.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.016+0000 7f146f400700 10 client.25591 ms_handle_connect on v2:172.21.15.64:6836/2585907777 2024-05-07T02:33:31.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.016+0000 7f146f400700 1 -- 172.21.15.19:0/3114078017 <== mds.4 v2:172.21.15.64:6836/2585907777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563a0e046400 con 0x563a0e1a2f90 2024-05-07T02:33:31.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.016+0000 7f146f400700 10 client.25591 handle_command_reply: tid=0 2024-05-07T02:33:31.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.017+0000 7f149ea41b80 1 client.25591 shutdown 2024-05-07T02:33:31.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.017+0000 7f149ea41b80 1 -- 172.21.15.19:0/3114078017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e116620 msgr2=0x563a0e132aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:31.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.017+0000 7f149ea41b80 1 --2- 172.21.15.19:0/3114078017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e116620 0x563a0e132aa0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f147c0a6df0 tx=0x7f147c0e00d0 comp rx=0 tx=0).stop 2024-05-07T02:33:31.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.017+0000 7f149ea41b80 1 -- 172.21.15.19:0/3114078017 shutdown_connections 2024-05-07T02:33:31.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.017+0000 7f149ea41b80 1 --2- 172.21.15.19:0/3114078017 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x563a0e1a2f90 0x563a0e1a53b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:31.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.017+0000 7f149ea41b80 1 --2- 172.21.15.19:0/3114078017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a0e11c470 0x563a0e1a14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:31.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.017+0000 7f149ea41b80 1 --2- 172.21.15.19:0/3114078017 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563a0e116620 0x563a0e132aa0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:31.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.017+0000 7f149ea41b80 1 --2- 172.21.15.19:0/3114078017 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563a0e116f70 0x563a0e132fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:31.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.017+0000 7f149ea41b80 1 -- 172.21.15.19:0/3114078017 >> 172.21.15.19:0/3114078017 conn(0x563a0df98e40 msgr2=0x563a0e119d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:31.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.017+0000 7f146f400700 0 client.25591 ms_handle_reset on v2:172.21.15.64:6836/2585907777 2024-05-07T02:33:31.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.017+0000 7f149ea41b80 1 -- 172.21.15.19:0/3114078017 shutdown_connections 2024-05-07T02:33:31.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.017+0000 7f149ea41b80 1 -- 172.21.15.19:0/3114078017 wait complete. 2024-05-07T02:33:31.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.017+0000 7f149ea41b80 20 client.25591 trim_cache size 0 max 16384 2024-05-07T02:33:31.019 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:33:31.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.018+0000 7f148fe00700 1 -- 172.21.15.19:0/200851360 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f14700b29f0 msgr2=0x7f14700b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:31.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.018+0000 7f148fe00700 1 --2- 172.21.15.19:0/200851360 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f14700b29f0 0x7f14700b4f00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f148000b780 tx=0x7f1480009d80 comp rx=0 tx=0).stop 2024-05-07T02:33:31.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.018+0000 7f148fe00700 1 -- 172.21.15.19:0/200851360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14881054e0 msgr2=0x7f148819ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:31.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.018+0000 7f148fe00700 1 --2- 172.21.15.19:0/200851360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14881054e0 0x7f148819ada0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f147c00e9a0 tx=0x7f147c00ecb0 comp rx=0 tx=0).stop 2024-05-07T02:33:31.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.018+0000 7f148fe00700 1 -- 172.21.15.19:0/200851360 shutdown_connections 2024-05-07T02:33:31.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.018+0000 7f148fe00700 1 --2- 172.21.15.19:0/200851360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f148810e910 0x7f148819f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:31.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.018+0000 7f148fe00700 1 --2- 172.21.15.19:0/200851360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14881054e0 0x7f148819ada0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:31.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.018+0000 7f148fe00700 1 --2- 172.21.15.19:0/200851360 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f14700b29f0 0x7f14700b4f00 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:31.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.018+0000 7f148fe00700 1 --2- 172.21.15.19:0/200851360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1488105e30 0x7f148819b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:31.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.018+0000 7f148fe00700 1 -- 172.21.15.19:0/200851360 >> 172.21.15.19:0/200851360 conn(0x7f14880fedc0 msgr2=0x7f14880ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:31.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.019+0000 7f148fe00700 1 -- 172.21.15.19:0/200851360 shutdown_connections 2024-05-07T02:33:31.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:31.019+0000 7f148fe00700 1 -- 172.21.15.19:0/200851360 wait complete. 2024-05-07T02:33:31.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:30 smithi064 ceph-mon[34699]: pgmap v53: 129 pgs: 129 active+clean; 17 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 439 KiB/s wr, 67 op/s 2024-05-07T02:33:31.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:30 smithi019 ceph-mon[29781]: pgmap v53: 129 pgs: 129 active+clean; 17 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 439 KiB/s wr, 67 op/s 2024-05-07T02:33:31.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:30 smithi027 ceph-mon[34398]: pgmap v53: 129 pgs: 129 active+clean; 17 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 439 KiB/s wr, 67 op/s 2024-05-07T02:33:31.722 INFO:tasks.workunit.client.0.smithi019.stdout: 1 32686 7.16 MB/sec warmup 27 sec latency 623.456 ms 2024-05-07T02:33:32.723 INFO:tasks.workunit.client.0.smithi019.stdout: 1 32686 6.90 MB/sec warmup 28 sec latency 1623.570 ms 2024-05-07T02:33:33.238 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:32 smithi019 ceph-mon[29781]: pgmap v54: 129 pgs: 129 active+clean; 37 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 2.1 MiB/s wr, 84 op/s 2024-05-07T02:33:33.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:32 smithi064 ceph-mon[34699]: pgmap v54: 129 pgs: 129 active+clean; 37 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 2.1 MiB/s wr, 84 op/s 2024-05-07T02:33:33.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:32 smithi027 ceph-mon[34398]: pgmap v54: 129 pgs: 129 active+clean; 37 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 2.1 MiB/s wr, 84 op/s 2024-05-07T02:33:33.723 INFO:tasks.workunit.client.0.smithi019.stdout: 1 32686 6.67 MB/sec warmup 29 sec latency 2623.741 ms 2024-05-07T02:33:33.856 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-07T02:33:34.350 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:33:34.723 INFO:tasks.workunit.client.0.smithi019.stdout: 1 32686 6.44 MB/sec warmup 30 sec latency 3623.855 ms 2024-05-07T02:33:35.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:34 smithi019 ceph-mon[29781]: pgmap v55: 129 pgs: 129 active+clean; 37 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 1.7 MiB/s wr, 24 op/s 2024-05-07T02:33:35.258 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:34 smithi027 ceph-mon[34398]: pgmap v55: 129 pgs: 129 active+clean; 37 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 1.7 MiB/s wr, 24 op/s 2024-05-07T02:33:35.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:34 smithi064 ceph-mon[34699]: pgmap v55: 129 pgs: 129 active+clean; 37 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 1.7 MiB/s wr, 24 op/s 2024-05-07T02:33:35.723 INFO:tasks.workunit.client.0.smithi019.stdout: 1 32745 6.24 MB/sec warmup 31 sec latency 4390.813 ms 2024-05-07T02:33:36.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:35 smithi019 ceph-mon[29781]: pgmap v56: 129 pgs: 129 active+clean; 61 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 3.8 MiB/s wr, 74 op/s 2024-05-07T02:33:36.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:35 smithi064 ceph-mon[34699]: pgmap v56: 129 pgs: 129 active+clean; 61 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 3.8 MiB/s wr, 74 op/s 2024-05-07T02:33:36.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:35 smithi027 ceph-mon[34398]: pgmap v56: 129 pgs: 129 active+clean; 61 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 3.8 MiB/s wr, 74 op/s 2024-05-07T02:33:36.724 INFO:tasks.workunit.client.0.smithi019.stdout: 1 32755 6.05 MB/sec warmup 32 sec latency 708.481 ms 2024-05-07T02:33:36.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.831+0000 7f9106800700 1 -- 172.21.15.19:0/2838766757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9100105e50 msgr2=0x7f910010e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:36.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.831+0000 7f9106800700 1 --2- 172.21.15.19:0/2838766757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9100105e50 0x7f910010e3a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f90e8009f90 tx=0x7f90e800b330 comp rx=0 tx=0).stop 2024-05-07T02:33:36.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.831+0000 7f9106800700 1 -- 172.21.15.19:0/2838766757 shutdown_connections 2024-05-07T02:33:36.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.831+0000 7f9106800700 1 --2- 172.21.15.19:0/2838766757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f910010e8e0 0x7f9100110dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:36.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.831+0000 7f9106800700 1 --2- 172.21.15.19:0/2838766757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9100105500 0x7f9100105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:36.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.831+0000 7f9106800700 1 --2- 172.21.15.19:0/2838766757 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9100105e50 0x7f910010e3a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:36.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.831+0000 7f9106800700 1 -- 172.21.15.19:0/2838766757 >> 172.21.15.19:0/2838766757 conn(0x7f91000fedc0 msgr2=0x7f91001011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:36.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.831+0000 7f9106800700 1 -- 172.21.15.19:0/2838766757 shutdown_connections 2024-05-07T02:33:36.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.831+0000 7f9106800700 1 -- 172.21.15.19:0/2838766757 wait complete. 2024-05-07T02:33:36.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.832+0000 7f9106800700 1 Processor -- start 2024-05-07T02:33:36.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.833+0000 7f9106800700 1 -- start start 2024-05-07T02:33:36.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.833+0000 7f9106800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9100105500 0x7f91001031f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:36.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.833+0000 7f9106800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9100105e50 0x7f9100103730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:36.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.833+0000 7f9106800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f910010e8e0 0x7f9100103c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:36.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.833+0000 7f9106800700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9100114490 con 0x7f910010e8e0 2024-05-07T02:33:36.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.833+0000 7f9106800700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9100114310 con 0x7f9100105500 2024-05-07T02:33:36.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.833+0000 7f9106800700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9100114610 con 0x7f9100105e50 2024-05-07T02:33:36.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.833+0000 7f90f6a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9100105e50 0x7f9100103730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:36.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.833+0000 7f90ff400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9100105500 0x7f91001031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:36.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.833+0000 7f90ff400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9100105500 0x7f91001031f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:50502/0 (socket says 172.21.15.19:50502) 2024-05-07T02:33:36.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.833+0000 7f90f6a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9100105e50 0x7f9100103730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:59436/0 (socket says 172.21.15.19:59436) 2024-05-07T02:33:36.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.833+0000 7f90ffe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f910010e8e0 0x7f9100103c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:36.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.833+0000 7f90ff400700 1 -- 172.21.15.19:0/3276851019 learned_addr learned my addr 172.21.15.19:0/3276851019 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:36.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.834+0000 7f90ff400700 1 -- 172.21.15.19:0/3276851019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9100105e50 msgr2=0x7f9100103730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:36.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.834+0000 7f90ff400700 1 --2- 172.21.15.19:0/3276851019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9100105e50 0x7f9100103730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:36.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.834+0000 7f90ff400700 1 -- 172.21.15.19:0/3276851019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f910010e8e0 msgr2=0x7f9100103c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:36.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.834+0000 7f90ff400700 1 --2- 172.21.15.19:0/3276851019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f910010e8e0 0x7f9100103c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:36.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.834+0000 7f90ff400700 1 -- 172.21.15.19:0/3276851019 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91001a91a0 con 0x7f9100105500 2024-05-07T02:33:36.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.834+0000 7f90f6a00700 1 --2- 172.21.15.19:0/3276851019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9100105e50 0x7f9100103730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:33:36.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.834+0000 7f90ff400700 1 --2- 172.21.15.19:0/3276851019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9100105500 0x7f91001031f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f90f000e9a0 tx=0x7f90f000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:36.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.834+0000 7f90fcc00700 1 -- 172.21.15.19:0/3276851019 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f000c9c0 con 0x7f9100105500 2024-05-07T02:33:36.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.834+0000 7f90fcc00700 1 -- 172.21.15.19:0/3276851019 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90f000cb60 con 0x7f9100105500 2024-05-07T02:33:36.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.834+0000 7f90fcc00700 1 -- 172.21.15.19:0/3276851019 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f0018680 con 0x7f9100105500 2024-05-07T02:33:36.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.835+0000 7f9106800700 1 -- 172.21.15.19:0/3276851019 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91001a9490 con 0x7f9100105500 2024-05-07T02:33:36.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.835+0000 7f9106800700 1 -- 172.21.15.19:0/3276851019 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91001a9a20 con 0x7f9100105500 2024-05-07T02:33:36.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.838+0000 7f90fcc00700 1 -- 172.21.15.19:0/3276851019 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f90f0010430 con 0x7f9100105500 2024-05-07T02:33:36.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.839+0000 7f91152cdb80 1 Processor -- start 2024-05-07T02:33:36.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.840+0000 7f91152cdb80 1 -- start start 2024-05-07T02:33:36.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.840+0000 7f90fcc00700 1 --2- 172.21.15.19:0/3276851019 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f90e40b2a70 0x7f90e40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:36.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.840+0000 7f91152cdb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d4a3e13b0 0x555d4a3e17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:36.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.840+0000 7f90fcc00700 1 -- 172.21.15.19:0/3276851019 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f90f00e2c00 con 0x7f9100105500 2024-05-07T02:33:36.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.840+0000 7f91152cdb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d4a3db470 0x555d4a3db880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:36.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.840+0000 7f91152cdb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d4a3dbdc0 0x555d4a3e8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:36.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.840+0000 7f91152cdb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555d4a2fda20 con 0x555d4a3e13b0 2024-05-07T02:33:36.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.840+0000 7f91152cdb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x555d4a29b760 con 0x555d4a3dbdc0 2024-05-07T02:33:36.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.840+0000 7f91152cdb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555d4a2f99e0 con 0x555d4a3db470 2024-05-07T02:33:36.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.840+0000 7f90ffe00700 1 --2- 172.21.15.19:0/3276851019 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f90e40b2a70 0x7f90e40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:36.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.840+0000 7f90ff400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d4a3e13b0 0x555d4a3e17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:36.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.840+0000 7f90f6a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d4a3db470 0x555d4a3db880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:36.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.840+0000 7f90f6a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d4a3db470 0x555d4a3db880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:59452/0 (socket says 172.21.15.19:59452) 2024-05-07T02:33:36.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.840+0000 7f90ff400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d4a3e13b0 0x555d4a3e17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47004/0 (socket says 172.21.15.19:47004) 2024-05-07T02:33:36.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.840+0000 7f90f6a00700 1 -- 172.21.15.19:0/3791381521 learned_addr learned my addr 172.21.15.19:0/3791381521 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:36.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.841+0000 7f90ffe00700 1 --2- 172.21.15.19:0/3791381521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d4a3dbdc0 0x555d4a3e8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:36.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.841+0000 7f90f6a00700 1 -- 172.21.15.19:0/3791381521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d4a3dbdc0 msgr2=0x555d4a3e8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:36.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.841+0000 7f90f6a00700 1 --2- 172.21.15.19:0/3791381521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d4a3dbdc0 0x555d4a3e8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:36.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.841+0000 7f90f6a00700 1 -- 172.21.15.19:0/3791381521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d4a3e13b0 msgr2=0x555d4a3e17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:36.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.841+0000 7f90f6a00700 1 --2- 172.21.15.19:0/3791381521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d4a3e13b0 0x555d4a3e17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:36.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.841+0000 7f90f6a00700 1 -- 172.21.15.19:0/3791381521 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d4a24a8d0 con 0x555d4a3db470 2024-05-07T02:33:36.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.841+0000 7f90ffe00700 1 --2- 172.21.15.19:0/3276851019 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f90e40b2a70 0x7f90e40b4f80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f90f8005fd0 tx=0x7f90f80137f0 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:36.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.841+0000 7f90f6a00700 1 --2- 172.21.15.19:0/3791381521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d4a3db470 0x555d4a3db880 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f90e8009f90 tx=0x7f90e8004f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d40f5dfaa98bc3eb server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:36.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.842+0000 7f90f4c00700 1 -- 172.21.15.19:0/3791381521 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e800b9f0 con 0x555d4a3db470 2024-05-07T02:33:36.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.842+0000 7f90f4c00700 1 -- 172.21.15.19:0/3791381521 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90e800bb90 con 0x555d4a3db470 2024-05-07T02:33:36.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.842+0000 7f90f4c00700 1 -- 172.21.15.19:0/3791381521 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e80052a0 con 0x555d4a3db470 2024-05-07T02:33:36.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.843+0000 7f91152cdb80 1 -- 172.21.15.19:0/3791381521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d4a3db470 msgr2=0x555d4a3db880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:36.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.843+0000 7f91152cdb80 1 --2- 172.21.15.19:0/3791381521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d4a3db470 0x555d4a3db880 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f90e8009f90 tx=0x7f90e8004f90 comp rx=0 tx=0).stop 2024-05-07T02:33:36.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.843+0000 7f91152cdb80 1 -- 172.21.15.19:0/3791381521 shutdown_connections 2024-05-07T02:33:36.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.843+0000 7f91152cdb80 1 --2- 172.21.15.19:0/3791381521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d4a3dbdc0 0x555d4a3e8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:36.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.843+0000 7f91152cdb80 1 --2- 172.21.15.19:0/3791381521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d4a3e13b0 0x555d4a3e17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:36.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.843+0000 7f91152cdb80 1 --2- 172.21.15.19:0/3791381521 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d4a3db470 0x555d4a3db880 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:36.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.843+0000 7f91152cdb80 1 -- 172.21.15.19:0/3791381521 >> 172.21.15.19:0/3791381521 conn(0x555d4a25de40 msgr2=0x555d4a3d62f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:36.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.843+0000 7f91152cdb80 1 -- 172.21.15.19:0/3791381521 shutdown_connections 2024-05-07T02:33:36.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.843+0000 7f91152cdb80 1 -- 172.21.15.19:0/3791381521 wait complete. 2024-05-07T02:33:36.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.843+0000 7f91152cdb80 1 -- start start 2024-05-07T02:33:36.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.844+0000 7f91152cdb80 1 Processor -- start 2024-05-07T02:33:36.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.844+0000 7f91152cdb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d4a3db470 0x555d4a3e55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:36.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.844+0000 7f91152cdb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d4a3dbdc0 0x555d4a3e5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:36.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.844+0000 7f91152cdb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d4a3e13b0 0x555d4a3e6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:36.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.844+0000 7f91152cdb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555d4a3e8760 con 0x555d4a3db470 2024-05-07T02:33:36.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.844+0000 7f91152cdb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x555d4a263b20 con 0x555d4a3dbdc0 2024-05-07T02:33:36.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.844+0000 7f91152cdb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555d4a3e88e0 con 0x555d4a3e13b0 2024-05-07T02:33:36.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.844+0000 7f91152cdb80 10 client.0 _ll_register_callbacks cb 0x555d4a2a4c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:33:36.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.845+0000 7f90ffe00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d4a3e13b0 0x555d4a3e6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:36.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.845+0000 7f90ff400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d4a3db470 0x555d4a3e55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:36.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.845+0000 7f90f6a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d4a3dbdc0 0x555d4a3e5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:36.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.845+0000 7f90ffe00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d4a3e13b0 0x555d4a3e6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:59458/0 (socket says 172.21.15.19:59458) 2024-05-07T02:33:36.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.845+0000 7f90ffe00700 1 -- 172.21.15.19:0/3162565184 learned_addr learned my addr 172.21.15.19:0/3162565184 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:36.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.845+0000 7f90ffe00700 1 -- 172.21.15.19:0/3162565184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d4a3dbdc0 msgr2=0x555d4a3e5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:36.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.845+0000 7f90ffe00700 1 --2- 172.21.15.19:0/3162565184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d4a3dbdc0 0x555d4a3e5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:36.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.845+0000 7f90ffe00700 1 -- 172.21.15.19:0/3162565184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d4a3db470 msgr2=0x555d4a3e55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:36.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.845+0000 7f90ffe00700 1 --2- 172.21.15.19:0/3162565184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d4a3db470 0x555d4a3e55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:36.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.845+0000 7f90ffe00700 1 -- 172.21.15.19:0/3162565184 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d4a2c3420 con 0x555d4a3e13b0 2024-05-07T02:33:36.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.845+0000 7f90f6a00700 1 --2- 172.21.15.19:0/3162565184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d4a3dbdc0 0x555d4a3e5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:33:36.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.845+0000 7f90ffe00700 1 --2- 172.21.15.19:0/3162565184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d4a3e13b0 0x555d4a3e6030 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f90f8012240 tx=0x7f90f80047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52eee4df382de2c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:36.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.845+0000 7f90ed600700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:33:36.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.846+0000 7f90ed600700 1 -- 172.21.15.19:0/3162565184 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f8024030 con 0x555d4a3e13b0 2024-05-07T02:33:36.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.846+0000 7f91152cdb80 1 -- 172.21.15.19:0/3162565184 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90e800bb90 con 0x555d4a3e13b0 2024-05-07T02:33:36.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.846+0000 7f90ed600700 1 -- 172.21.15.19:0/3162565184 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f90f8004830 con 0x555d4a3e13b0 2024-05-07T02:33:36.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.846+0000 7f90ed600700 1 -- 172.21.15.19:0/3162565184 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f90f8030050 con 0x555d4a3e13b0 2024-05-07T02:33:36.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.846+0000 7f91152cdb80 10 client.25603 fetch_fsmap learned FSMap version 34 2024-05-07T02:33:36.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.846+0000 7f91152cdb80 1 -- 172.21.15.19:0/3162565184 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555d4a3f3400 con 0x555d4a3e13b0 2024-05-07T02:33:36.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.846+0000 7f90ed600700 1 -- 172.21.15.19:0/3162565184 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90e800bb90 con 0x555d4a3e13b0 2024-05-07T02:33:36.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.847+0000 7f90ed600700 1 -- 172.21.15.19:0/3162565184 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f90f801f070 con 0x555d4a3e13b0 2024-05-07T02:33:36.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.847+0000 7f91152cdb80 10 client.25603 fetch_fsmap finished waiting for FSMap version 34 2024-05-07T02:33:36.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.847+0000 7f91152cdb80 10 client.25603 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-05-07T02:33:36.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.847+0000 7f91152cdb80 20 client.25603 populate_metadata read hostname 'smithi019' 2024-05-07T02:33:36.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.847+0000 7f91152cdb80 1 --2- 172.21.15.19:0/3162565184 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x555d4a3f5990 0x555d4a3f7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:36.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.847+0000 7f91152cdb80 4 client.25603 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:33:36.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.847+0000 7f91152cdb80 1 -- 172.21.15.19:0/3162565184 --> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d4a3a82e0 con 0x555d4a3f5990 2024-05-07T02:33:36.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.847+0000 7f90ff400700 1 --2- 172.21.15.19:0/3162565184 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x555d4a3f5990 0x555d4a3f7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:36.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.848+0000 7f90ff400700 1 --2- 172.21.15.19:0/3162565184 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x555d4a3f5990 0x555d4a3f7df0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6edde0f49641de24 server_cookie=e668fb3a52eb0b3e in_seq=0 out_seq=0 2024-05-07T02:33:36.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.848+0000 7f90ed600700 10 client.25603 ms_handle_connect on v2:172.21.15.27:6836/2583138143 2024-05-07T02:33:36.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.849+0000 7f90ed600700 1 -- 172.21.15.19:0/3162565184 <== mds.3 v2:172.21.15.27:6836/2583138143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555d4a3a82e0 con 0x555d4a3f5990 2024-05-07T02:33:36.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.849+0000 7f90ed600700 10 client.25603 handle_command_reply: tid=0 2024-05-07T02:33:36.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.849+0000 7f91152cdb80 1 client.25603 shutdown 2024-05-07T02:33:36.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.849+0000 7f91152cdb80 1 -- 172.21.15.19:0/3162565184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d4a3e13b0 msgr2=0x555d4a3e6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:36.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.849+0000 7f91152cdb80 1 --2- 172.21.15.19:0/3162565184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d4a3e13b0 0x555d4a3e6030 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f90f8012240 tx=0x7f90f80047c0 comp rx=0 tx=0).stop 2024-05-07T02:33:36.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.849+0000 7f91152cdb80 1 -- 172.21.15.19:0/3162565184 shutdown_connections 2024-05-07T02:33:36.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.849+0000 7f91152cdb80 1 --2- 172.21.15.19:0/3162565184 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x555d4a3f5990 0x555d4a3f7df0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:36.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.849+0000 7f91152cdb80 1 --2- 172.21.15.19:0/3162565184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d4a3e13b0 0x555d4a3e6030 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:36.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.849+0000 7f91152cdb80 1 --2- 172.21.15.19:0/3162565184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d4a3db470 0x555d4a3e55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:36.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.849+0000 7f91152cdb80 1 --2- 172.21.15.19:0/3162565184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d4a3dbdc0 0x555d4a3e5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:36.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.849+0000 7f90ed600700 0 client.25603 ms_handle_reset on v2:172.21.15.27:6836/2583138143 2024-05-07T02:33:36.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.849+0000 7f91152cdb80 1 -- 172.21.15.19:0/3162565184 >> 172.21.15.19:0/3162565184 conn(0x555d4a25de40 msgr2=0x555d4a3da460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:36.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.849+0000 7f91152cdb80 1 -- 172.21.15.19:0/3162565184 shutdown_connections 2024-05-07T02:33:36.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.849+0000 7f91152cdb80 1 -- 172.21.15.19:0/3162565184 wait complete. 2024-05-07T02:33:36.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.850+0000 7f91152cdb80 20 client.25603 trim_cache size 0 max 16384 2024-05-07T02:33:36.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.865+0000 7f91152cdb80 1 Processor -- start 2024-05-07T02:33:36.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.866+0000 7f91152cdb80 1 -- start start 2024-05-07T02:33:36.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.866+0000 7f91152cdb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d4a3db470 0x555d4a3e4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:36.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.866+0000 7f91152cdb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d4a3dbdc0 0x555d4a3e5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:36.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.866+0000 7f91152cdb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d4a3e13b0 0x555d4a3dab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:36.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.866+0000 7f91152cdb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f91001b2790 con 0x555d4a3dbdc0 2024-05-07T02:33:36.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.866+0000 7f91152cdb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f91001b4390 con 0x555d4a3db470 2024-05-07T02:33:36.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.866+0000 7f91152cdb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f91001b2e60 con 0x555d4a3e13b0 2024-05-07T02:33:36.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.866+0000 7f90f6a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d4a3dbdc0 0x555d4a3e5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:36.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.866+0000 7f90f6a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d4a3dbdc0 0x555d4a3e5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47020/0 (socket says 172.21.15.19:47020) 2024-05-07T02:33:36.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.866+0000 7f90f6a00700 1 -- 172.21.15.19:0/2664254185 learned_addr learned my addr 172.21.15.19:0/2664254185 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:36.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.866+0000 7f90ff400700 1 --2- 172.21.15.19:0/2664254185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d4a3db470 0x555d4a3e4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:36.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.866+0000 7f90ffe00700 1 --2- 172.21.15.19:0/2664254185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d4a3e13b0 0x555d4a3dab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:36.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.867+0000 7f90f6a00700 1 -- 172.21.15.19:0/2664254185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d4a3e13b0 msgr2=0x555d4a3dab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:36.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.867+0000 7f90f6a00700 1 --2- 172.21.15.19:0/2664254185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d4a3e13b0 0x555d4a3dab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:36.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.867+0000 7f90f6a00700 1 -- 172.21.15.19:0/2664254185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d4a3db470 msgr2=0x555d4a3e4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:36.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.867+0000 7f90f6a00700 1 --2- 172.21.15.19:0/2664254185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d4a3db470 0x555d4a3e4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:36.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.867+0000 7f90f6a00700 1 -- 172.21.15.19:0/2664254185 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d4a3d7240 con 0x555d4a3dbdc0 2024-05-07T02:33:36.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.867+0000 7f90ff400700 1 --2- 172.21.15.19:0/2664254185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d4a3db470 0x555d4a3e4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:33:36.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.867+0000 7f90ffe00700 1 --2- 172.21.15.19:0/2664254185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d4a3e13b0 0x555d4a3dab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:33:36.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.867+0000 7f90f6a00700 1 --2- 172.21.15.19:0/2664254185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d4a3dbdc0 0x555d4a3e5e70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f90e8009f90 tx=0x7f90e8007eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b7d521d93459c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:36.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.867+0000 7f90efe00700 1 -- 172.21.15.19:0/2664254185 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e80092e0 con 0x555d4a3dbdc0 2024-05-07T02:33:36.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.868+0000 7f90efe00700 1 -- 172.21.15.19:0/2664254185 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90e800b9f0 con 0x555d4a3dbdc0 2024-05-07T02:33:36.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.868+0000 7f90efe00700 1 -- 172.21.15.19:0/2664254185 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e8004020 con 0x555d4a3dbdc0 2024-05-07T02:33:36.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.868+0000 7f91152cdb80 1 -- 172.21.15.19:0/2664254185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d4a3dbdc0 msgr2=0x555d4a3e5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:36.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.868+0000 7f91152cdb80 1 --2- 172.21.15.19:0/2664254185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d4a3dbdc0 0x555d4a3e5e70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f90e8009f90 tx=0x7f90e8007eb0 comp rx=0 tx=0).stop 2024-05-07T02:33:36.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.868+0000 7f91152cdb80 1 -- 172.21.15.19:0/2664254185 shutdown_connections 2024-05-07T02:33:36.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.868+0000 7f91152cdb80 1 --2- 172.21.15.19:0/2664254185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d4a3e13b0 0x555d4a3dab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:36.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.868+0000 7f91152cdb80 1 --2- 172.21.15.19:0/2664254185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d4a3db470 0x555d4a3e4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:36.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.868+0000 7f91152cdb80 1 --2- 172.21.15.19:0/2664254185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d4a3dbdc0 0x555d4a3e5e70 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:36.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.868+0000 7f91152cdb80 1 -- 172.21.15.19:0/2664254185 >> 172.21.15.19:0/2664254185 conn(0x555d4a25de40 msgr2=0x555d4a3da460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:36.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.869+0000 7f91152cdb80 1 -- 172.21.15.19:0/2664254185 shutdown_connections 2024-05-07T02:33:36.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.869+0000 7f91152cdb80 1 -- 172.21.15.19:0/2664254185 wait complete. 2024-05-07T02:33:36.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.869+0000 7f91152cdb80 1 -- start start 2024-05-07T02:33:36.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.869+0000 7f91152cdb80 1 Processor -- start 2024-05-07T02:33:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.870+0000 7f91152cdb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d4a3db470 0x555d4a3ee0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:36.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.870+0000 7f91152cdb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d4a3dbdc0 0x555d4a3fcb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:36.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.870+0000 7f91152cdb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d4a3e13b0 0x555d4a3fd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:36.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.870+0000 7f91152cdb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f91001b3c00 con 0x555d4a3e13b0 2024-05-07T02:33:36.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.870+0000 7f91152cdb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f91001b3530 con 0x555d4a3dbdc0 2024-05-07T02:33:36.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.870+0000 7f91152cdb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f91001b0730 con 0x555d4a3db470 2024-05-07T02:33:36.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.870+0000 7f91152cdb80 10 client.0 _ll_register_callbacks cb 0x555d4a3f82d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:33:36.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.870+0000 7f90ffe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d4a3e13b0 0x555d4a3fd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:36.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.870+0000 7f90ff400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d4a3db470 0x555d4a3ee0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:36.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.870+0000 7f90ffe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d4a3e13b0 0x555d4a3fd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47024/0 (socket says 172.21.15.19:47024) 2024-05-07T02:33:36.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.870+0000 7f90f6a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d4a3dbdc0 0x555d4a3fcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:36.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.870+0000 7f90ffe00700 1 -- 172.21.15.19:0/2862077582 learned_addr learned my addr 172.21.15.19:0/2862077582 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:36.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.871+0000 7f90ff400700 1 -- 172.21.15.19:0/2862077582 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d4a3dbdc0 msgr2=0x555d4a3fcb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:36.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.871+0000 7f90ff400700 1 --2- 172.21.15.19:0/2862077582 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d4a3dbdc0 0x555d4a3fcb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:36.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.871+0000 7f90ff400700 1 -- 172.21.15.19:0/2862077582 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d4a3e13b0 msgr2=0x555d4a3fd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:36.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.871+0000 7f90ff400700 1 --2- 172.21.15.19:0/2862077582 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d4a3e13b0 0x555d4a3fd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:36.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.871+0000 7f90ff400700 1 -- 172.21.15.19:0/2862077582 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d4a3fd7a0 con 0x555d4a3db470 2024-05-07T02:33:36.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.871+0000 7f90ffe00700 1 --2- 172.21.15.19:0/2862077582 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d4a3e13b0 0x555d4a3fd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:33:36.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.871+0000 7f90f6a00700 1 --2- 172.21.15.19:0/2862077582 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d4a3dbdc0 0x555d4a3fcb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:33:36.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.871+0000 7f90ff400700 1 --2- 172.21.15.19:0/2862077582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d4a3db470 0x555d4a3ee0a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f90f00dff60 tx=0x7f90f0011f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5023eef18da9ff1c server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:36.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.871+0000 7f90ed600700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:33:36.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.871+0000 7f90ed600700 1 -- 172.21.15.19:0/2862077582 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f00adea0 con 0x555d4a3db470 2024-05-07T02:33:36.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.871+0000 7f90ed600700 1 -- 172.21.15.19:0/2862077582 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f90f00a9d50 con 0x555d4a3db470 2024-05-07T02:33:36.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.871+0000 7f91152cdb80 1 -- 172.21.15.19:0/2862077582 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90e800b9f0 con 0x555d4a3db470 2024-05-07T02:33:36.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.872+0000 7f90ed600700 1 -- 172.21.15.19:0/2862077582 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f90f00b2950 con 0x555d4a3db470 2024-05-07T02:33:36.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.872+0000 7f91152cdb80 10 client.25609 fetch_fsmap learned FSMap version 34 2024-05-07T02:33:36.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.872+0000 7f91152cdb80 1 -- 172.21.15.19:0/2862077582 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555d4a45ff90 con 0x555d4a3db470 2024-05-07T02:33:36.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.872+0000 7f90ed600700 1 -- 172.21.15.19:0/2862077582 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90e800b9f0 con 0x555d4a3db470 2024-05-07T02:33:36.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.872+0000 7f90ed600700 1 -- 172.21.15.19:0/2862077582 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f90f00a5460 con 0x555d4a3db470 2024-05-07T02:33:36.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.872+0000 7f91152cdb80 10 client.25609 fetch_fsmap finished waiting for FSMap version 34 2024-05-07T02:33:36.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.872+0000 7f91152cdb80 10 client.25609 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-05-07T02:33:36.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.872+0000 7f91152cdb80 20 client.25609 populate_metadata read hostname 'smithi019' 2024-05-07T02:33:36.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.872+0000 7f91152cdb80 1 --2- 172.21.15.19:0/2862077582 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x555d4a460620 0x555d4a462a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:36.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.872+0000 7f91152cdb80 4 client.25609 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:33:36.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.872+0000 7f91152cdb80 1 -- 172.21.15.19:0/2862077582 --> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555d4a30b400 con 0x555d4a460620 2024-05-07T02:33:36.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.872+0000 7f90f6a00700 1 --2- 172.21.15.19:0/2862077582 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x555d4a460620 0x555d4a462a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:36.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.873+0000 7f90f6a00700 1 --2- 172.21.15.19:0/2862077582 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x555d4a460620 0x555d4a462a40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ccae4449d754fdd5 server_cookie=7049807c12005344 in_seq=0 out_seq=0 2024-05-07T02:33:36.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.873+0000 7f90ed600700 10 client.25609 ms_handle_connect on v2:172.21.15.27:6836/2583138143 2024-05-07T02:33:36.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.873+0000 7f90ed600700 1 -- 172.21.15.19:0/2862077582 <== mds.3 v2:172.21.15.27:6836/2583138143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555d4a30b400 con 0x555d4a460620 2024-05-07T02:33:36.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.873+0000 7f90ed600700 10 client.25609 handle_command_reply: tid=0 2024-05-07T02:33:36.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.873+0000 7f91152cdb80 1 client.25609 shutdown 2024-05-07T02:33:36.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.873+0000 7f91152cdb80 1 -- 172.21.15.19:0/2862077582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d4a3db470 msgr2=0x555d4a3ee0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:36.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.873+0000 7f91152cdb80 1 --2- 172.21.15.19:0/2862077582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d4a3db470 0x555d4a3ee0a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f90f00dff60 tx=0x7f90f0011f90 comp rx=0 tx=0).stop 2024-05-07T02:33:36.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.873+0000 7f91152cdb80 1 -- 172.21.15.19:0/2862077582 shutdown_connections 2024-05-07T02:33:36.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.873+0000 7f91152cdb80 1 --2- 172.21.15.19:0/2862077582 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x555d4a460620 0x555d4a462a40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:36.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.874+0000 7f91152cdb80 1 --2- 172.21.15.19:0/2862077582 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d4a3e13b0 0x555d4a3fd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:36.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.874+0000 7f91152cdb80 1 --2- 172.21.15.19:0/2862077582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d4a3db470 0x555d4a3ee0a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:36.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.874+0000 7f91152cdb80 1 --2- 172.21.15.19:0/2862077582 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d4a3dbdc0 0x555d4a3fcb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:36.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.874+0000 7f91152cdb80 1 -- 172.21.15.19:0/2862077582 >> 172.21.15.19:0/2862077582 conn(0x555d4a25de40 msgr2=0x555d4a3d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:36.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.874+0000 7f90ed600700 0 client.25609 ms_handle_reset on v2:172.21.15.27:6836/2583138143 2024-05-07T02:33:36.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.874+0000 7f91152cdb80 1 -- 172.21.15.19:0/2862077582 shutdown_connections 2024-05-07T02:33:36.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.874+0000 7f91152cdb80 1 -- 172.21.15.19:0/2862077582 wait complete. 2024-05-07T02:33:36.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.874+0000 7f91152cdb80 20 client.25609 trim_cache size 0 max 16384 2024-05-07T02:33:36.877 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:33:36.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.875+0000 7f9106800700 1 -- 172.21.15.19:0/3276851019 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f90e40b2a70 msgr2=0x7f90e40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:36.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.875+0000 7f9106800700 1 --2- 172.21.15.19:0/3276851019 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f90e40b2a70 0x7f90e40b4f80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f90f8005fd0 tx=0x7f90f80137f0 comp rx=0 tx=0).stop 2024-05-07T02:33:36.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.875+0000 7f9106800700 1 -- 172.21.15.19:0/3276851019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9100105500 msgr2=0x7f91001031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:36.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.875+0000 7f9106800700 1 --2- 172.21.15.19:0/3276851019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9100105500 0x7f91001031f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f90f000e9a0 tx=0x7f90f000ecb0 comp rx=0 tx=0).stop 2024-05-07T02:33:36.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.875+0000 7f9106800700 1 -- 172.21.15.19:0/3276851019 shutdown_connections 2024-05-07T02:33:36.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.875+0000 7f9106800700 1 --2- 172.21.15.19:0/3276851019 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f90e40b2a70 0x7f90e40b4f80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:36.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.875+0000 7f9106800700 1 --2- 172.21.15.19:0/3276851019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f910010e8e0 0x7f9100103c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:36.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.875+0000 7f9106800700 1 --2- 172.21.15.19:0/3276851019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9100105500 0x7f91001031f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:36.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.875+0000 7f9106800700 1 --2- 172.21.15.19:0/3276851019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9100105e50 0x7f9100103730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:36.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.875+0000 7f9106800700 1 -- 172.21.15.19:0/3276851019 >> 172.21.15.19:0/3276851019 conn(0x7f91000fedc0 msgr2=0x7f91000ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:36.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.875+0000 7f9106800700 1 -- 172.21.15.19:0/3276851019 shutdown_connections 2024-05-07T02:33:36.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:36.875+0000 7f9106800700 1 -- 172.21.15.19:0/3276851019 wait complete. 2024-05-07T02:33:37.724 INFO:tasks.workunit.client.0.smithi019.stdout: 1 32755 5.86 MB/sec warmup 33 sec latency 1709.086 ms 2024-05-07T02:33:37.856 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-07T02:33:38.313 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:33:38.724 INFO:tasks.workunit.client.0.smithi019.stdout: 1 32755 5.69 MB/sec warmup 34 sec latency 2709.151 ms 2024-05-07T02:33:39.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:38 smithi019 ceph-mon[29781]: pgmap v57: 129 pgs: 129 active+clean; 61 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 3.8 MiB/s wr, 66 op/s 2024-05-07T02:33:39.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:38 smithi064 ceph-mon[34699]: pgmap v57: 129 pgs: 129 active+clean; 61 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 3.8 MiB/s wr, 66 op/s 2024-05-07T02:33:39.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:38 smithi027 ceph-mon[34398]: pgmap v57: 129 pgs: 129 active+clean; 61 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 3.8 MiB/s wr, 66 op/s 2024-05-07T02:33:39.724 INFO:tasks.workunit.client.0.smithi019.stdout: 1 32755 5.53 MB/sec warmup 35 sec latency 3709.239 ms 2024-05-07T02:33:40.724 INFO:tasks.workunit.client.0.smithi019.stdout: 1 32877 5.38 MB/sec warmup 36 sec latency 4472.873 ms 2024-05-07T02:33:40.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.982+0000 7f02d7200700 1 -- 172.21.15.19:0/3755633364 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02d0106580 msgr2=0x7f02d0106990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:40.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.982+0000 7f02d7200700 1 --2- 172.21.15.19:0/3755633364 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02d0106580 0x7f02d0106990 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f02bc009990 tx=0x7f02bc009ca0 comp rx=0 tx=0).stop 2024-05-07T02:33:40.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.982+0000 7f02d7200700 1 -- 172.21.15.19:0/3755633364 shutdown_connections 2024-05-07T02:33:40.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.982+0000 7f02d7200700 1 --2- 172.21.15.19:0/3755633364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02d0106ed0 0x7f02d0113360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:40.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.982+0000 7f02d7200700 1 --2- 172.21.15.19:0/3755633364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02d010c5c0 0x7f02d010c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:40.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.982+0000 7f02d7200700 1 --2- 172.21.15.19:0/3755633364 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02d0106580 0x7f02d0106990 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:40.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.982+0000 7f02d7200700 1 -- 172.21.15.19:0/3755633364 >> 172.21.15.19:0/3755633364 conn(0x7f02d0102080 msgr2=0x7f02d01044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:40.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.982+0000 7f02d7200700 1 -- 172.21.15.19:0/3755633364 shutdown_connections 2024-05-07T02:33:40.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.982+0000 7f02d7200700 1 -- 172.21.15.19:0/3755633364 wait complete. 2024-05-07T02:33:40.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.984+0000 7f02d7200700 1 Processor -- start 2024-05-07T02:33:40.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.984+0000 7f02d7200700 1 -- start start 2024-05-07T02:33:40.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.985+0000 7f02d7200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02d0106580 0x7f02d019d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:40.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.985+0000 7f02d7200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02d0106ed0 0x7f02d019d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:40.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.985+0000 7f02d7200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02d010c5c0 0x7f02d01a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:40.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.985+0000 7f02d7200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f02d01166f0 con 0x7f02d0106580 2024-05-07T02:33:40.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.985+0000 7f02d7200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f02d0116570 con 0x7f02d0106ed0 2024-05-07T02:33:40.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.985+0000 7f02d7200700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f02d0116870 con 0x7f02d010c5c0 2024-05-07T02:33:40.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.985+0000 7f02cf400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02d0106ed0 0x7f02d019d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:40.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.985+0000 7f02cf400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02d0106ed0 0x7f02d019d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:49732/0 (socket says 172.21.15.19:49732) 2024-05-07T02:33:40.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.985+0000 7f02cf400700 1 -- 172.21.15.19:0/3163591303 learned_addr learned my addr 172.21.15.19:0/3163591303 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:40.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.985+0000 7f02cfe00700 1 --2- 172.21.15.19:0/3163591303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02d0106580 0x7f02d019d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:40.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.985+0000 7f02d5200700 1 --2- 172.21.15.19:0/3163591303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02d010c5c0 0x7f02d01a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:40.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.986+0000 7f02cf400700 1 -- 172.21.15.19:0/3163591303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02d010c5c0 msgr2=0x7f02d01a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:40.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.986+0000 7f02cf400700 1 --2- 172.21.15.19:0/3163591303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02d010c5c0 0x7f02d01a1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:40.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.986+0000 7f02cf400700 1 -- 172.21.15.19:0/3163591303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02d0106580 msgr2=0x7f02d019d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:40.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.986+0000 7f02cf400700 1 --2- 172.21.15.19:0/3163591303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02d0106580 0x7f02d019d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:40.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.986+0000 7f02cf400700 1 -- 172.21.15.19:0/3163591303 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02d01a1f30 con 0x7f02d0106ed0 2024-05-07T02:33:40.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.986+0000 7f02cf400700 1 --2- 172.21.15.19:0/3163591303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02d0106ed0 0x7f02d019d600 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f02bc0098e0 tx=0x7f02bc0057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:40.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.987+0000 7f02ccc00700 1 -- 172.21.15.19:0/3163591303 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02bc027070 con 0x7f02d0106ed0 2024-05-07T02:33:40.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.987+0000 7f02ccc00700 1 -- 172.21.15.19:0/3163591303 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02bc022470 con 0x7f02d0106ed0 2024-05-07T02:33:40.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.987+0000 7f02d7200700 1 -- 172.21.15.19:0/3163591303 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02d01a21c0 con 0x7f02d0106ed0 2024-05-07T02:33:40.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.987+0000 7f02ccc00700 1 -- 172.21.15.19:0/3163591303 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02bc017650 con 0x7f02d0106ed0 2024-05-07T02:33:40.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.987+0000 7f02d7200700 1 -- 172.21.15.19:0/3163591303 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02d01a9b40 con 0x7f02d0106ed0 2024-05-07T02:33:40.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.992+0000 7f02ccc00700 1 -- 172.21.15.19:0/3163591303 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f02bc02a070 con 0x7f02d0106ed0 2024-05-07T02:33:40.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.993+0000 7f02e5c50b80 1 Processor -- start 2024-05-07T02:33:40.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.993+0000 7f02e5c50b80 1 -- start start 2024-05-07T02:33:40.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.994+0000 7f02e5c50b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756fbd9470 0x55756fbd9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:40.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.994+0000 7f02e5c50b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55756fbd3620 0x55756fbd3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:40.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.994+0000 7f02e5c50b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55756fbd3f70 0x55756fbe0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:40.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.994+0000 7f02e5c50b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55756faf5a20 con 0x55756fbd9470 2024-05-07T02:33:40.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.994+0000 7f02e5c50b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55756fa93760 con 0x55756fbd3f70 2024-05-07T02:33:40.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.994+0000 7f02e5c50b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55756faf19e0 con 0x55756fbd3620 2024-05-07T02:33:40.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.994+0000 7f02ccc00700 1 --2- 172.21.15.19:0/3163591303 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f02b40b2a70 0x7f02b40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:40.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.994+0000 7f02cfe00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55756fbd3620 0x55756fbd3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:40.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.994+0000 7f02cfe00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55756fbd3620 0x55756fbd3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:51090/0 (socket says 172.21.15.19:51090) 2024-05-07T02:33:40.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.995+0000 7f02cfe00700 1 -- 172.21.15.19:0/1737966308 learned_addr learned my addr 172.21.15.19:0/1737966308 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:40.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.995+0000 7f02ccc00700 1 -- 172.21.15.19:0/3163591303 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f02bc0e3610 con 0x7f02d0106ed0 2024-05-07T02:33:40.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.995+0000 7f02cf400700 1 --2- 172.21.15.19:0/1737966308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55756fbd3f70 0x55756fbe0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:40.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.995+0000 7f02d5200700 1 --2- 172.21.15.19:0/1737966308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756fbd9470 0x55756fbd9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:40.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.995+0000 7f02d5200700 1 --2- 172.21.15.19:0/3163591303 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f02b40b2a70 0x7f02b40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:40.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.995+0000 7f02cfe00700 1 -- 172.21.15.19:0/1737966308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55756fbd3f70 msgr2=0x55756fbe0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:40.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.995+0000 7f02cfe00700 1 --2- 172.21.15.19:0/1737966308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55756fbd3f70 0x55756fbe0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:40.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.995+0000 7f02cfe00700 1 -- 172.21.15.19:0/1737966308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756fbd9470 msgr2=0x55756fbd9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:40.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.995+0000 7f02cfe00700 1 --2- 172.21.15.19:0/1737966308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756fbd9470 0x55756fbd9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:40.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.995+0000 7f02cfe00700 1 -- 172.21.15.19:0/1737966308 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55756fa428d0 con 0x55756fbd3620 2024-05-07T02:33:40.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.996+0000 7f02cfe00700 1 --2- 172.21.15.19:0/1737966308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55756fbd3620 0x55756fbd3a30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f02c0009d20 tx=0x7f02c0011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c391d98e2a7fe624 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:40.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.996+0000 7f02c5600700 1 -- 172.21.15.19:0/1737966308 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c001e070 con 0x55756fbd3620 2024-05-07T02:33:40.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.996+0000 7f02d5200700 1 --2- 172.21.15.19:0/3163591303 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f02b40b2a70 0x7f02b40b4f80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f02c8009ec0 tx=0x7f02c800b040 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:40.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.996+0000 7f02c5600700 1 -- 172.21.15.19:0/1737966308 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02c0019ae0 con 0x55756fbd3620 2024-05-07T02:33:40.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.996+0000 7f02c5600700 1 -- 172.21.15.19:0/1737966308 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c000ea90 con 0x55756fbd3620 2024-05-07T02:33:40.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.997+0000 7f02e5c50b80 1 -- 172.21.15.19:0/1737966308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55756fbd3620 msgr2=0x55756fbd3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:40.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.997+0000 7f02e5c50b80 1 --2- 172.21.15.19:0/1737966308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55756fbd3620 0x55756fbd3a30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f02c0009d20 tx=0x7f02c0011720 comp rx=0 tx=0).stop 2024-05-07T02:33:40.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.997+0000 7f02e5c50b80 1 -- 172.21.15.19:0/1737966308 shutdown_connections 2024-05-07T02:33:40.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.997+0000 7f02e5c50b80 1 --2- 172.21.15.19:0/1737966308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55756fbd3f70 0x55756fbe0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:40.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.997+0000 7f02e5c50b80 1 --2- 172.21.15.19:0/1737966308 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756fbd9470 0x55756fbd9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:40.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.997+0000 7f02e5c50b80 1 --2- 172.21.15.19:0/1737966308 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55756fbd3620 0x55756fbd3a30 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:40.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.997+0000 7f02e5c50b80 1 -- 172.21.15.19:0/1737966308 >> 172.21.15.19:0/1737966308 conn(0x55756fa55e40 msgr2=0x55756fbce380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:40.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.997+0000 7f02e5c50b80 1 -- 172.21.15.19:0/1737966308 shutdown_connections 2024-05-07T02:33:40.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.997+0000 7f02e5c50b80 1 -- 172.21.15.19:0/1737966308 wait complete. 2024-05-07T02:33:40.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.998+0000 7f02e5c50b80 1 -- start start 2024-05-07T02:33:40.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.998+0000 7f02e5c50b80 1 Processor -- start 2024-05-07T02:33:40.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.999+0000 7f02e5c50b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55756fbd3620 0x55756fbdd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:40.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.999+0000 7f02e5c50b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756fbd3f70 0x55756fbddb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:41.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.999+0000 7f02e5c50b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55756fbd9470 0x55756fbde0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:41.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.999+0000 7f02e5c50b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55756fbe0840 con 0x55756fbd3f70 2024-05-07T02:33:41.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.999+0000 7f02e5c50b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55756fa5bb20 con 0x55756fbd9470 2024-05-07T02:33:41.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.999+0000 7f02e5c50b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55756fbe09c0 con 0x55756fbd3620 2024-05-07T02:33:41.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.999+0000 7f02e5c50b80 10 client.0 _ll_register_callbacks cb 0x55756fa89380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:33:41.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.999+0000 7f02cfe00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55756fbd3620 0x55756fbdd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:41.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.999+0000 7f02cf400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756fbd3f70 0x55756fbddb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:41.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.999+0000 7f02cfe00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55756fbd3620 0x55756fbdd640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:51096/0 (socket says 172.21.15.19:51096) 2024-05-07T02:33:41.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.999+0000 7f02cfe00700 1 -- 172.21.15.19:0/646209709 learned_addr learned my addr 172.21.15.19:0/646209709 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:41.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.999+0000 7f02d5200700 1 --2- 172.21.15.19:0/646209709 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55756fbd9470 0x55756fbde0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:41.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.999+0000 7f02cfe00700 1 -- 172.21.15.19:0/646209709 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55756fbd9470 msgr2=0x55756fbde0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:41.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.999+0000 7f02cfe00700 1 --2- 172.21.15.19:0/646209709 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55756fbd9470 0x55756fbde0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:41.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.999+0000 7f02cfe00700 1 -- 172.21.15.19:0/646209709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756fbd3f70 msgr2=0x55756fbddb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:41.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.999+0000 7f02cfe00700 1 --2- 172.21.15.19:0/646209709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756fbd3f70 0x55756fbddb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:41.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:40.999+0000 7f02cfe00700 1 -- 172.21.15.19:0/646209709 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55756fabb420 con 0x55756fbd3620 2024-05-07T02:33:41.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.000+0000 7f02cfe00700 1 --2- 172.21.15.19:0/646209709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55756fbd3620 0x55756fbdd640 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f02c0005c50 tx=0x7f02c0019660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d57d8985f3acbe36 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:41.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.000+0000 7f02ba000700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:33:41.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.000+0000 7f02ba000700 1 -- 172.21.15.19:0/646209709 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c001e050 con 0x55756fbd3620 2024-05-07T02:33:41.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.000+0000 7f02ba000700 1 -- 172.21.15.19:0/646209709 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f02c0005250 con 0x55756fbd3620 2024-05-07T02:33:41.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.000+0000 7f02e5c50b80 1 -- 172.21.15.19:0/646209709 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02c0019ae0 con 0x55756fbd3620 2024-05-07T02:33:41.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.000+0000 7f02ba000700 1 -- 172.21.15.19:0/646209709 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f02c0032050 con 0x55756fbd3620 2024-05-07T02:33:41.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.000+0000 7f02e5c50b80 10 client.25621 fetch_fsmap learned FSMap version 34 2024-05-07T02:33:41.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.000+0000 7f02e5c50b80 1 -- 172.21.15.19:0/646209709 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55756fbeb410 con 0x55756fbd3620 2024-05-07T02:33:41.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.001+0000 7f02ba000700 1 -- 172.21.15.19:0/646209709 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02c0019ae0 con 0x55756fbd3620 2024-05-07T02:33:41.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.001+0000 7f02ba000700 1 -- 172.21.15.19:0/646209709 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f02c000e6d0 con 0x55756fbd3620 2024-05-07T02:33:41.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.001+0000 7f02e5c50b80 10 client.25621 fetch_fsmap finished waiting for FSMap version 34 2024-05-07T02:33:41.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.001+0000 7f02e5c50b80 10 client.25621 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-07T02:33:41.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.001+0000 7f02e5c50b80 20 client.25621 populate_metadata read hostname 'smithi019' 2024-05-07T02:33:41.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.001+0000 7f02e5c50b80 1 --2- 172.21.15.19:0/646209709 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55756fbed990 0x55756fbefdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:41.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.001+0000 7f02e5c50b80 4 client.25621 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:33:41.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.001+0000 7f02e5c50b80 1 -- 172.21.15.19:0/646209709 --> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55756fba00e0 con 0x55756fbed990 2024-05-07T02:33:41.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.002+0000 7f02cf400700 1 --2- 172.21.15.19:0/646209709 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55756fbed990 0x55756fbefdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:41.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.002+0000 7f02cf400700 1 --2- 172.21.15.19:0/646209709 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55756fbed990 0x55756fbefdb0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b99a6a882b297088 server_cookie=4877a4daabf2c797 in_seq=0 out_seq=0 2024-05-07T02:33:41.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.002+0000 7f02ba000700 10 client.25621 ms_handle_connect on v2:172.21.15.64:6838/1583120004 2024-05-07T02:33:41.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.003+0000 7f02ba000700 1 -- 172.21.15.19:0/646209709 <== mds.2 v2:172.21.15.64:6838/1583120004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55756fba00e0 con 0x55756fbed990 2024-05-07T02:33:41.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.003+0000 7f02ba000700 10 client.25621 handle_command_reply: tid=0 2024-05-07T02:33:41.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.003+0000 7f02e5c50b80 1 client.25621 shutdown 2024-05-07T02:33:41.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.003+0000 7f02e5c50b80 1 -- 172.21.15.19:0/646209709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55756fbd3620 msgr2=0x55756fbdd640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:41.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.003+0000 7f02e5c50b80 1 --2- 172.21.15.19:0/646209709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55756fbd3620 0x55756fbdd640 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f02c0005c50 tx=0x7f02c0019660 comp rx=0 tx=0).stop 2024-05-07T02:33:41.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.004+0000 7f02e5c50b80 1 -- 172.21.15.19:0/646209709 shutdown_connections 2024-05-07T02:33:41.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.004+0000 7f02e5c50b80 1 --2- 172.21.15.19:0/646209709 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55756fbed990 0x55756fbefdb0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:41.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.004+0000 7f02e5c50b80 1 --2- 172.21.15.19:0/646209709 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55756fbd9470 0x55756fbde0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:41.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.004+0000 7f02e5c50b80 1 --2- 172.21.15.19:0/646209709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55756fbd3620 0x55756fbdd640 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:41.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.004+0000 7f02e5c50b80 1 --2- 172.21.15.19:0/646209709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756fbd3f70 0x55756fbddb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:41.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.004+0000 7f02e5c50b80 1 -- 172.21.15.19:0/646209709 >> 172.21.15.19:0/646209709 conn(0x55756fa55e40 msgr2=0x55756fbd2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:41.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.004+0000 7f02ba000700 0 client.25621 ms_handle_reset on v2:172.21.15.64:6838/1583120004 2024-05-07T02:33:41.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.004+0000 7f02e5c50b80 1 -- 172.21.15.19:0/646209709 shutdown_connections 2024-05-07T02:33:41.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.004+0000 7f02e5c50b80 1 -- 172.21.15.19:0/646209709 wait complete. 2024-05-07T02:33:41.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.004+0000 7f02e5c50b80 20 client.25621 trim_cache size 0 max 16384 2024-05-07T02:33:41.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.021+0000 7f02e5c50b80 1 Processor -- start 2024-05-07T02:33:41.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.021+0000 7f02e5c50b80 1 -- start start 2024-05-07T02:33:41.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.022+0000 7f02e5c50b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756fbd3620 0x55756fbd3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:41.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.022+0000 7f02e5c50b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55756fbd3f70 0x55756fbd7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:41.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.022+0000 7f02e5c50b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55756fbd9470 0x55756fbddf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:41.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.022+0000 7f02e5c50b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f02d01b2690 con 0x55756fbd3620 2024-05-07T02:33:41.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.022+0000 7f02e5c50b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f02d01b4290 con 0x55756fbd3f70 2024-05-07T02:33:41.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.022+0000 7f02e5c50b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f02d01b2d60 con 0x55756fbd9470 2024-05-07T02:33:41.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.022+0000 7f02cfe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756fbd3620 0x55756fbd3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:41.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.022+0000 7f02cfe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756fbd3620 0x55756fbd3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44974/0 (socket says 172.21.15.19:44974) 2024-05-07T02:33:41.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.022+0000 7f02d5200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55756fbd9470 0x55756fbddf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:41.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.022+0000 7f02cfe00700 1 -- 172.21.15.19:0/1757608944 learned_addr learned my addr 172.21.15.19:0/1757608944 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:41.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.022+0000 7f02cf400700 1 --2- 172.21.15.19:0/1757608944 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55756fbd3f70 0x55756fbd7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:41.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.023+0000 7f02d5200700 1 -- 172.21.15.19:0/1757608944 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55756fbd3f70 msgr2=0x55756fbd7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:41.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.023+0000 7f02d5200700 1 --2- 172.21.15.19:0/1757608944 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55756fbd3f70 0x55756fbd7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:41.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.023+0000 7f02d5200700 1 -- 172.21.15.19:0/1757608944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756fbd3620 msgr2=0x55756fbd3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:41.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.023+0000 7f02d5200700 1 --2- 172.21.15.19:0/1757608944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756fbd3620 0x55756fbd3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:41.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.023+0000 7f02d5200700 1 -- 172.21.15.19:0/1757608944 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55756fbc8f40 con 0x55756fbd9470 2024-05-07T02:33:41.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.023+0000 7f02cfe00700 1 --2- 172.21.15.19:0/1757608944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756fbd3620 0x55756fbd3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:33:41.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.023+0000 7f02cf400700 1 --2- 172.21.15.19:0/1757608944 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55756fbd3f70 0x55756fbd7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:33:41.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.023+0000 7f02d5200700 1 --2- 172.21.15.19:0/1757608944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55756fbd9470 0x55756fbddf00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f02c80193b0 tx=0x7f02c8009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16382f9eac9c7968 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:41.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.023+0000 7f02c4c00700 1 -- 172.21.15.19:0/1757608944 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c8019e10 con 0x55756fbd9470 2024-05-07T02:33:41.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.023+0000 7f02c4c00700 1 -- 172.21.15.19:0/1757608944 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02c8004bb0 con 0x55756fbd9470 2024-05-07T02:33:41.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.023+0000 7f02c4c00700 1 -- 172.21.15.19:0/1757608944 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c8025420 con 0x55756fbd9470 2024-05-07T02:33:41.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.024+0000 7f02e5c50b80 1 -- 172.21.15.19:0/1757608944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55756fbd9470 msgr2=0x55756fbddf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:41.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.024+0000 7f02e5c50b80 1 --2- 172.21.15.19:0/1757608944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55756fbd9470 0x55756fbddf00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f02c80193b0 tx=0x7f02c8009920 comp rx=0 tx=0).stop 2024-05-07T02:33:41.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.024+0000 7f02e5c50b80 1 -- 172.21.15.19:0/1757608944 shutdown_connections 2024-05-07T02:33:41.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.024+0000 7f02e5c50b80 1 --2- 172.21.15.19:0/1757608944 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55756fbd9470 0x55756fbddf00 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:41.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.024+0000 7f02e5c50b80 1 --2- 172.21.15.19:0/1757608944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756fbd3620 0x55756fbd3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:41.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.024+0000 7f02e5c50b80 1 --2- 172.21.15.19:0/1757608944 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55756fbd3f70 0x55756fbd7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:41.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.024+0000 7f02e5c50b80 1 -- 172.21.15.19:0/1757608944 >> 172.21.15.19:0/1757608944 conn(0x55756fa55e40 msgr2=0x55756fbc8830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:41.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.024+0000 7f02e5c50b80 1 -- 172.21.15.19:0/1757608944 shutdown_connections 2024-05-07T02:33:41.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.024+0000 7f02e5c50b80 1 -- 172.21.15.19:0/1757608944 wait complete. 2024-05-07T02:33:41.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.025+0000 7f02e5c50b80 1 -- start start 2024-05-07T02:33:41.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.025+0000 7f02e5c50b80 1 Processor -- start 2024-05-07T02:33:41.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.026+0000 7f02e5c50b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756fbd3620 0x55756fbe0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:41.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.026+0000 7f02e5c50b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55756fbd3f70 0x55756fbe1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:41.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.026+0000 7f02e5c50b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55756fbd9470 0x55756fc55fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:41.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.026+0000 7f02e5c50b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f02d01b3b00 con 0x55756fbd3620 2024-05-07T02:33:41.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.026+0000 7f02e5c50b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f02d01b3430 con 0x55756fbd3f70 2024-05-07T02:33:41.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.026+0000 7f02e5c50b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f02d01b0630 con 0x55756fbd9470 2024-05-07T02:33:41.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.026+0000 7f02e5c50b80 10 client.0 _ll_register_callbacks cb 0x55756fa89380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:33:41.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.026+0000 7f02cfe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756fbd3620 0x55756fbe0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:41.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.026+0000 7f02cfe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756fbd3620 0x55756fbe0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44980/0 (socket says 172.21.15.19:44980) 2024-05-07T02:33:41.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.026+0000 7f02d5200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55756fbd9470 0x55756fc55fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:41.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.026+0000 7f02cfe00700 1 -- 172.21.15.19:0/1912033927 learned_addr learned my addr 172.21.15.19:0/1912033927 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:41.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.026+0000 7f02cf400700 1 --2- 172.21.15.19:0/1912033927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55756fbd3f70 0x55756fbe1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:41.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.027+0000 7f02d5200700 1 -- 172.21.15.19:0/1912033927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55756fbd3f70 msgr2=0x55756fbe1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:41.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.027+0000 7f02d5200700 1 --2- 172.21.15.19:0/1912033927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55756fbd3f70 0x55756fbe1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:41.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.027+0000 7f02d5200700 1 -- 172.21.15.19:0/1912033927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756fbd3620 msgr2=0x55756fbe0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:41.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.027+0000 7f02d5200700 1 --2- 172.21.15.19:0/1912033927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756fbd3620 0x55756fbe0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:41.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.027+0000 7f02d5200700 1 -- 172.21.15.19:0/1912033927 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55756fc564f0 con 0x55756fbd9470 2024-05-07T02:33:41.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.027+0000 7f02cfe00700 1 --2- 172.21.15.19:0/1912033927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756fbd3620 0x55756fbe0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:33:41.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.027+0000 7f02d5200700 1 --2- 172.21.15.19:0/1912033927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55756fbd9470 0x55756fc55fb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f02c80193b0 tx=0x7f02c8005010 comp rx=0 tx=0).ready entity=mon.1 client_cookie=432eddff6726f438 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:41.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.027+0000 7f02ba000700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:33:41.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.028+0000 7f02ba000700 1 -- 172.21.15.19:0/1912033927 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c8019e10 con 0x55756fbd9470 2024-05-07T02:33:41.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.028+0000 7f02e5c50b80 1 -- 172.21.15.19:0/1912033927 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02c8004bb0 con 0x55756fbd9470 2024-05-07T02:33:41.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.028+0000 7f02ba000700 1 -- 172.21.15.19:0/1912033927 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f02c802a070 con 0x55756fbd9470 2024-05-07T02:33:41.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.028+0000 7f02ba000700 1 -- 172.21.15.19:0/1912033927 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f02c803a050 con 0x55756fbd9470 2024-05-07T02:33:41.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.028+0000 7f02e5c50b80 10 client.25627 fetch_fsmap learned FSMap version 34 2024-05-07T02:33:41.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.028+0000 7f02e5c50b80 1 -- 172.21.15.19:0/1912033927 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55756fc575b0 con 0x55756fbd9470 2024-05-07T02:33:41.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.028+0000 7f02ba000700 1 -- 172.21.15.19:0/1912033927 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02c8004bb0 con 0x55756fbd9470 2024-05-07T02:33:41.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.029+0000 7f02ba000700 1 -- 172.21.15.19:0/1912033927 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f02c8025c50 con 0x55756fbd9470 2024-05-07T02:33:41.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.029+0000 7f02e5c50b80 10 client.25627 fetch_fsmap finished waiting for FSMap version 34 2024-05-07T02:33:41.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.029+0000 7f02e5c50b80 10 client.25627 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-07T02:33:41.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.029+0000 7f02e5c50b80 20 client.25627 populate_metadata read hostname 'smithi019' 2024-05-07T02:33:41.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.029+0000 7f02e5c50b80 1 --2- 172.21.15.19:0/1912033927 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55756fc57c30 0x55756fc5a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:41.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.029+0000 7f02e5c50b80 4 client.25627 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:33:41.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.029+0000 7f02e5c50b80 1 -- 172.21.15.19:0/1912033927 --> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55756fb03400 con 0x55756fc57c30 2024-05-07T02:33:41.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.029+0000 7f02cfe00700 1 --2- 172.21.15.19:0/1912033927 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55756fc57c30 0x55756fc5a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:41.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.030+0000 7f02cfe00700 1 --2- 172.21.15.19:0/1912033927 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55756fc57c30 0x55756fc5a050 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3e1277f0e4b7c1a2 server_cookie=9a6b8cbb2c1d0aa4 in_seq=0 out_seq=0 2024-05-07T02:33:41.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.030+0000 7f02ba000700 10 client.25627 ms_handle_connect on v2:172.21.15.64:6838/1583120004 2024-05-07T02:33:41.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.030+0000 7f02ba000700 1 -- 172.21.15.19:0/1912033927 <== mds.2 v2:172.21.15.64:6838/1583120004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55756fb03400 con 0x55756fc57c30 2024-05-07T02:33:41.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.030+0000 7f02ba000700 10 client.25627 handle_command_reply: tid=0 2024-05-07T02:33:41.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.030+0000 7f02e5c50b80 1 client.25627 shutdown 2024-05-07T02:33:41.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.030+0000 7f02e5c50b80 1 -- 172.21.15.19:0/1912033927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55756fbd9470 msgr2=0x55756fc55fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:41.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.030+0000 7f02e5c50b80 1 --2- 172.21.15.19:0/1912033927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55756fbd9470 0x55756fc55fb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f02c80193b0 tx=0x7f02c8005010 comp rx=0 tx=0).stop 2024-05-07T02:33:41.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.030+0000 7f02e5c50b80 1 -- 172.21.15.19:0/1912033927 shutdown_connections 2024-05-07T02:33:41.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.030+0000 7f02e5c50b80 1 --2- 172.21.15.19:0/1912033927 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55756fbd9470 0x55756fc55fb0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:41.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.030+0000 7f02e5c50b80 1 --2- 172.21.15.19:0/1912033927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55756fbd3620 0x55756fbe0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.030+0000 7f02e5c50b80 1 --2- 172.21.15.19:0/1912033927 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55756fc57c30 0x55756fc5a050 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.030+0000 7f02e5c50b80 1 --2- 172.21.15.19:0/1912033927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55756fbd3f70 0x55756fbe1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.030+0000 7f02e5c50b80 1 -- 172.21.15.19:0/1912033927 >> 172.21.15.19:0/1912033927 conn(0x55756fa55e40 msgr2=0x55756fbc8830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.030+0000 7f02ba000700 0 client.25627 ms_handle_reset on v2:172.21.15.64:6838/1583120004 2024-05-07T02:33:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.031+0000 7f02e5c50b80 1 -- 172.21.15.19:0/1912033927 shutdown_connections 2024-05-07T02:33:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.031+0000 7f02e5c50b80 1 -- 172.21.15.19:0/1912033927 wait complete. 2024-05-07T02:33:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.031+0000 7f02e5c50b80 20 client.25627 trim_cache size 0 max 16384 2024-05-07T02:33:41.033 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:33:41.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.031+0000 7f02d7200700 1 -- 172.21.15.19:0/3163591303 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f02b40b2a70 msgr2=0x7f02b40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:41.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.031+0000 7f02d7200700 1 --2- 172.21.15.19:0/3163591303 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f02b40b2a70 0x7f02b40b4f80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f02c8009ec0 tx=0x7f02c800b040 comp rx=0 tx=0).stop 2024-05-07T02:33:41.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.032+0000 7f02d7200700 1 -- 172.21.15.19:0/3163591303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02d0106ed0 msgr2=0x7f02d019d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:41.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.032+0000 7f02d7200700 1 --2- 172.21.15.19:0/3163591303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02d0106ed0 0x7f02d019d600 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f02bc0098e0 tx=0x7f02bc0057c0 comp rx=0 tx=0).stop 2024-05-07T02:33:41.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.032+0000 7f02d7200700 1 -- 172.21.15.19:0/3163591303 shutdown_connections 2024-05-07T02:33:41.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.032+0000 7f02d7200700 1 --2- 172.21.15.19:0/3163591303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f02d010c5c0 0x7f02d01a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:41.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.032+0000 7f02d7200700 1 --2- 172.21.15.19:0/3163591303 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f02b40b2a70 0x7f02b40b4f80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:41.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.032+0000 7f02d7200700 1 --2- 172.21.15.19:0/3163591303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02d0106580 0x7f02d019d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:41.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.032+0000 7f02d7200700 1 --2- 172.21.15.19:0/3163591303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02d0106ed0 0x7f02d019d600 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:41.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.032+0000 7f02d7200700 1 -- 172.21.15.19:0/3163591303 >> 172.21.15.19:0/3163591303 conn(0x7f02d0102080 msgr2=0x7f02d01048b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:41.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.032+0000 7f02d7200700 1 -- 172.21.15.19:0/3163591303 shutdown_connections 2024-05-07T02:33:41.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:41.032+0000 7f02d7200700 1 -- 172.21.15.19:0/3163591303 wait complete. 2024-05-07T02:33:41.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:40 smithi019 ceph-mon[29781]: pgmap v58: 129 pgs: 129 active+clean; 61 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 3.8 MiB/s wr, 66 op/s 2024-05-07T02:33:41.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:40 smithi064 ceph-mon[34699]: pgmap v58: 129 pgs: 129 active+clean; 61 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 3.8 MiB/s wr, 66 op/s 2024-05-07T02:33:41.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:40 smithi027 ceph-mon[34398]: pgmap v58: 129 pgs: 129 active+clean; 61 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 3.8 MiB/s wr, 66 op/s 2024-05-07T02:33:41.724 INFO:tasks.workunit.client.0.smithi019.stdout: 1 32877 5.24 MB/sec warmup 37 sec latency 1129.664 ms 2024-05-07T02:33:42.174 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-07T02:33:42.639 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:33:42.724 INFO:tasks.workunit.client.0.smithi019.stdout: 1 32877 5.10 MB/sec warmup 38 sec latency 2129.785 ms 2024-05-07T02:33:43.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:42 smithi019 ceph-mon[29781]: pgmap v59: 129 pgs: 129 active+clean; 34 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 3.9 MiB/s wr, 114 op/s 2024-05-07T02:33:43.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:42 smithi064 ceph-mon[34699]: pgmap v59: 129 pgs: 129 active+clean; 34 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 3.9 MiB/s wr, 114 op/s 2024-05-07T02:33:43.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:42 smithi027 ceph-mon[34398]: pgmap v59: 129 pgs: 129 active+clean; 34 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 3.9 MiB/s wr, 114 op/s 2024-05-07T02:33:43.724 INFO:tasks.workunit.client.0.smithi019.stdout: 1 32877 4.97 MB/sec warmup 39 sec latency 3129.894 ms 2024-05-07T02:33:44.725 INFO:tasks.workunit.client.0.smithi019.stdout: 1 32877 4.85 MB/sec warmup 40 sec latency 4129.989 ms 2024-05-07T02:33:45.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.169+0000 7f6c9d000700 1 -- 172.21.15.19:0/2458176240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c980764b0 msgr2=0x7f6c98114e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:45.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.169+0000 7f6c9d000700 1 --2- 172.21.15.19:0/2458176240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c980764b0 0x7f6c98114e30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6c800098f0 tx=0x7f6c80009c00 comp rx=0 tx=0).stop 2024-05-07T02:33:45.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.170+0000 7f6c9d000700 1 -- 172.21.15.19:0/2458176240 shutdown_connections 2024-05-07T02:33:45.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.170+0000 7f6c9d000700 1 --2- 172.21.15.19:0/2458176240 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c98115370 0x7f6c98117790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:45.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.170+0000 7f6c9d000700 1 --2- 172.21.15.19:0/2458176240 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c98075b60 0x7f6c98075f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:45.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.170+0000 7f6c9d000700 1 --2- 172.21.15.19:0/2458176240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c980764b0 0x7f6c98114e30 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:45.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.170+0000 7f6c9d000700 1 -- 172.21.15.19:0/2458176240 >> 172.21.15.19:0/2458176240 conn(0x7f6c98100040 msgr2=0x7f6c98102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:45.171 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:44 smithi019 ceph-mon[29781]: pgmap v60: 129 pgs: 129 active+clean; 34 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 2.2 MiB/s wr, 97 op/s 2024-05-07T02:33:45.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.170+0000 7f6c9d000700 1 -- 172.21.15.19:0/2458176240 shutdown_connections 2024-05-07T02:33:45.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.170+0000 7f6c9d000700 1 -- 172.21.15.19:0/2458176240 wait complete. 2024-05-07T02:33:45.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.172+0000 7f6c9d000700 1 Processor -- start 2024-05-07T02:33:45.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.172+0000 7f6c9d000700 1 -- start start 2024-05-07T02:33:45.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.172+0000 7f6c9d000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c98075b60 0x7f6c981a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:45.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.172+0000 7f6c9d000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c980764b0 0x7f6c981a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:45.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.172+0000 7f6c9d000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c98115370 0x7f6c981a5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:45.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.172+0000 7f6c9d000700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9811aac0 con 0x7f6c98115370 2024-05-07T02:33:45.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.172+0000 7f6c9d000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9811a940 con 0x7f6c980764b0 2024-05-07T02:33:45.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.172+0000 7f6c9d000700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9811ac40 con 0x7f6c98075b60 2024-05-07T02:33:45.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.173+0000 7f6c96800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c98115370 0x7f6c981a5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:45.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.173+0000 7f6c8fe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c980764b0 0x7f6c981a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:45.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.173+0000 7f6c95e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c98075b60 0x7f6c981a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:45.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.173+0000 7f6c96800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c98115370 0x7f6c981a5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44998/0 (socket says 172.21.15.19:44998) 2024-05-07T02:33:45.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.173+0000 7f6c96800700 1 -- 172.21.15.19:0/840737685 learned_addr learned my addr 172.21.15.19:0/840737685 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:45.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.173+0000 7f6c8fe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c980764b0 0x7f6c981a1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:49778/0 (socket says 172.21.15.19:49778) 2024-05-07T02:33:45.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.173+0000 7f6c8fe00700 1 -- 172.21.15.19:0/840737685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c98075b60 msgr2=0x7f6c981a1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:45.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.173+0000 7f6c8fe00700 1 --2- 172.21.15.19:0/840737685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c98075b60 0x7f6c981a1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:45.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.173+0000 7f6c8fe00700 1 -- 172.21.15.19:0/840737685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c98115370 msgr2=0x7f6c981a5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:45.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.173+0000 7f6c8fe00700 1 --2- 172.21.15.19:0/840737685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c98115370 0x7f6c981a5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:45.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.173+0000 7f6c8fe00700 1 -- 172.21.15.19:0/840737685 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c981a6420 con 0x7f6c980764b0 2024-05-07T02:33:45.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.173+0000 7f6c8fe00700 1 --2- 172.21.15.19:0/840737685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c980764b0 0x7f6c981a1af0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6c8000c010 tx=0x7f6c8000fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:45.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.174+0000 7f6c8ea00700 1 -- 172.21.15.19:0/840737685 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c800050c0 con 0x7f6c980764b0 2024-05-07T02:33:45.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.174+0000 7f6c8ea00700 1 -- 172.21.15.19:0/840737685 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c8000b560 con 0x7f6c980764b0 2024-05-07T02:33:45.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.174+0000 7f6c9d000700 1 -- 172.21.15.19:0/840737685 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c981a66b0 con 0x7f6c980764b0 2024-05-07T02:33:45.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.174+0000 7f6c8ea00700 1 -- 172.21.15.19:0/840737685 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c80003a00 con 0x7f6c980764b0 2024-05-07T02:33:45.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.174+0000 7f6c9d000700 1 -- 172.21.15.19:0/840737685 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c981ae030 con 0x7f6c980764b0 2024-05-07T02:33:45.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.178+0000 7f6c8ea00700 1 -- 172.21.15.19:0/840737685 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f6c8000b710 con 0x7f6c980764b0 2024-05-07T02:33:45.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.179+0000 7f6cabc2db80 1 Processor -- start 2024-05-07T02:33:45.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.179+0000 7f6cabc2db80 1 -- start start 2024-05-07T02:33:45.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.179+0000 7f6c8ea00700 1 --2- 172.21.15.19:0/840737685 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f6c7c0b2a70 0x7f6c7c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:45.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.180+0000 7f6cabc2db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583ed2233b0 0x5583ed2237c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:45.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.180+0000 7f6cabc2db80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583ed21d470 0x5583ed21d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:45.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.180+0000 7f6cabc2db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583ed21ddc0 0x5583ed22a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:45.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.180+0000 7f6cabc2db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5583ed13fa20 con 0x5583ed2233b0 2024-05-07T02:33:45.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.180+0000 7f6cabc2db80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5583ed0dd760 con 0x5583ed21d470 2024-05-07T02:33:45.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.180+0000 7f6cabc2db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5583ed13b9e0 con 0x5583ed21ddc0 2024-05-07T02:33:45.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.180+0000 7f6c8ea00700 1 -- 172.21.15.19:0/840737685 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f6c800e3720 con 0x7f6c980764b0 2024-05-07T02:33:45.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.180+0000 7f6c8fe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583ed21d470 0x5583ed21d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:45.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.180+0000 7f6c96800700 1 --2- 172.21.15.19:0/840737685 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f6c7c0b2a70 0x7f6c7c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:45.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.180+0000 7f6c8fe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583ed21d470 0x5583ed21d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:49784/0 (socket says 172.21.15.19:49784) 2024-05-07T02:33:45.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.180+0000 7f6c8fe00700 1 -- 172.21.15.19:0/148972263 learned_addr learned my addr 172.21.15.19:0/148972263 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:45.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.180+0000 7f6c95e00700 1 --2- 172.21.15.19:0/148972263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583ed2233b0 0x5583ed2237c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:45.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.180+0000 7f6c96800700 1 --2- 172.21.15.19:0/148972263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583ed21ddc0 0x5583ed22a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:45.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.180+0000 7f6c8fe00700 1 -- 172.21.15.19:0/148972263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583ed21ddc0 msgr2=0x5583ed22a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:45.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.180+0000 7f6c8fe00700 1 --2- 172.21.15.19:0/148972263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583ed21ddc0 0x5583ed22a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:45.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.180+0000 7f6c8fe00700 1 -- 172.21.15.19:0/148972263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583ed2233b0 msgr2=0x5583ed2237c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:45.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.180+0000 7f6c8fe00700 1 --2- 172.21.15.19:0/148972263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583ed2233b0 0x5583ed2237c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:45.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.181+0000 7f6c8fe00700 1 -- 172.21.15.19:0/148972263 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583ed08c8d0 con 0x5583ed21d470 2024-05-07T02:33:45.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.181+0000 7f6c95e00700 1 --2- 172.21.15.19:0/148972263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583ed2233b0 0x5583ed2237c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:33:45.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.181+0000 7f6c8fe00700 1 --2- 172.21.15.19:0/148972263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583ed21d470 0x5583ed21d880 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6c80009680 tx=0x7f6c800a9e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=697df54072ea9a99 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:45.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.181+0000 7f6c96800700 1 --2- 172.21.15.19:0/840737685 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f6c7c0b2a70 0x7f6c7c0b4f80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6c90009e60 tx=0x7f6c900099e0 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:45.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.181+0000 7f6c87400700 1 -- 172.21.15.19:0/148972263 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c800aa6a0 con 0x5583ed21d470 2024-05-07T02:33:45.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.181+0000 7f6c87400700 1 -- 172.21.15.19:0/148972263 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c800aea50 con 0x5583ed21d470 2024-05-07T02:33:45.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.181+0000 7f6c87400700 1 -- 172.21.15.19:0/148972263 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c800afe40 con 0x5583ed21d470 2024-05-07T02:33:45.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.182+0000 7f6cabc2db80 1 -- 172.21.15.19:0/148972263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583ed21d470 msgr2=0x5583ed21d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:45.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.182+0000 7f6cabc2db80 1 --2- 172.21.15.19:0/148972263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583ed21d470 0x5583ed21d880 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6c80009680 tx=0x7f6c800a9e90 comp rx=0 tx=0).stop 2024-05-07T02:33:45.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.182+0000 7f6cabc2db80 1 -- 172.21.15.19:0/148972263 shutdown_connections 2024-05-07T02:33:45.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.182+0000 7f6cabc2db80 1 --2- 172.21.15.19:0/148972263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583ed21ddc0 0x5583ed22a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:45.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.182+0000 7f6cabc2db80 1 --2- 172.21.15.19:0/148972263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583ed2233b0 0x5583ed2237c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:45.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.182+0000 7f6cabc2db80 1 --2- 172.21.15.19:0/148972263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583ed21d470 0x5583ed21d880 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:45.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.182+0000 7f6cabc2db80 1 -- 172.21.15.19:0/148972263 >> 172.21.15.19:0/148972263 conn(0x5583ed09fe40 msgr2=0x5583ed2182f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:45.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.183+0000 7f6cabc2db80 1 -- 172.21.15.19:0/148972263 shutdown_connections 2024-05-07T02:33:45.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.183+0000 7f6cabc2db80 1 -- 172.21.15.19:0/148972263 wait complete. 2024-05-07T02:33:45.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.183+0000 7f6cabc2db80 1 -- start start 2024-05-07T02:33:45.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.183+0000 7f6cabc2db80 1 Processor -- start 2024-05-07T02:33:45.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.184+0000 7f6cabc2db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583ed21d470 0x5583ed2275b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:45.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.184+0000 7f6cabc2db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583ed21ddc0 0x5583ed227af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:45.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.184+0000 7f6cabc2db80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583ed2233b0 0x5583ed228030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:45.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.184+0000 7f6cabc2db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5583ed22a580 con 0x5583ed21ddc0 2024-05-07T02:33:45.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.184+0000 7f6cabc2db80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5583ed0a5b20 con 0x5583ed2233b0 2024-05-07T02:33:45.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.184+0000 7f6cabc2db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5583ed22a700 con 0x5583ed21d470 2024-05-07T02:33:45.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.184+0000 7f6cabc2db80 10 client.0 _ll_register_callbacks cb 0x5583ed0e6c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:33:45.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.185+0000 7f6c95e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583ed21d470 0x5583ed2275b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:45.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.185+0000 7f6c8fe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583ed21ddc0 0x5583ed227af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:45.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.185+0000 7f6c96800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583ed2233b0 0x5583ed228030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:45.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.185+0000 7f6c96800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583ed2233b0 0x5583ed228030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:49800/0 (socket says 172.21.15.19:49800) 2024-05-07T02:33:45.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.185+0000 7f6c96800700 1 -- 172.21.15.19:0/2633212250 learned_addr learned my addr 172.21.15.19:0/2633212250 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:45.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.185+0000 7f6c95e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583ed21d470 0x5583ed2275b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:51148/0 (socket says 172.21.15.19:51148) 2024-05-07T02:33:45.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.185+0000 7f6c8fe00700 1 -- 172.21.15.19:0/2633212250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583ed21d470 msgr2=0x5583ed2275b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:45.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.185+0000 7f6c8fe00700 1 --2- 172.21.15.19:0/2633212250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583ed21d470 0x5583ed2275b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:45.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.185+0000 7f6c8fe00700 1 -- 172.21.15.19:0/2633212250 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583ed2233b0 msgr2=0x5583ed228030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:45.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.185+0000 7f6c8fe00700 1 --2- 172.21.15.19:0/2633212250 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583ed2233b0 0x5583ed228030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:45.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.185+0000 7f6c8fe00700 1 -- 172.21.15.19:0/2633212250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583ed105420 con 0x5583ed21ddc0 2024-05-07T02:33:45.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.185+0000 7f6c8fe00700 1 --2- 172.21.15.19:0/2633212250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583ed21ddc0 0x5583ed227af0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x5583ed219a40 tx=0x7f6c80009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57edde32a377b489 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:45.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.185+0000 7f6c7be00700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-07T02:33:45.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.186+0000 7f6c7be00700 1 -- 172.21.15.19:0/2633212250 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c800c4060 con 0x5583ed21ddc0 2024-05-07T02:33:45.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.186+0000 7f6c7be00700 1 -- 172.21.15.19:0/2633212250 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f6c800a6c90 con 0x5583ed21ddc0 2024-05-07T02:33:45.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.186+0000 7f6cabc2db80 1 -- 172.21.15.19:0/2633212250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c800aea50 con 0x5583ed21ddc0 2024-05-07T02:33:45.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.186+0000 7f6c7be00700 1 -- 172.21.15.19:0/2633212250 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6c800e0f10 con 0x5583ed21ddc0 2024-05-07T02:33:45.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.186+0000 7f6cabc2db80 10 client.15501 fetch_fsmap learned FSMap version 34 2024-05-07T02:33:45.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.186+0000 7f6cabc2db80 1 -- 172.21.15.19:0/2633212250 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5583ed22a580 con 0x5583ed21ddc0 2024-05-07T02:33:45.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.186+0000 7f6c7be00700 1 -- 172.21.15.19:0/2633212250 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c800aea50 con 0x5583ed21ddc0 2024-05-07T02:33:45.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.187+0000 7f6c7be00700 1 -- 172.21.15.19:0/2633212250 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f6c800aa900 con 0x5583ed21ddc0 2024-05-07T02:33:45.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.187+0000 7f6cabc2db80 10 client.15501 fetch_fsmap finished waiting for FSMap version 34 2024-05-07T02:33:45.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.187+0000 7f6cabc2db80 10 client.15501 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-07T02:33:45.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.187+0000 7f6cabc2db80 20 client.15501 populate_metadata read hostname 'smithi019' 2024-05-07T02:33:45.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.187+0000 7f6cabc2db80 1 --2- 172.21.15.19:0/2633212250 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5583ed237500 0x5583ed239950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:45.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.187+0000 7f6cabc2db80 4 client.15501 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:33:45.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.187+0000 7f6cabc2db80 1 -- 172.21.15.19:0/2633212250 --> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583ed1ea2e0 con 0x5583ed237500 2024-05-07T02:33:45.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.187+0000 7f6c95e00700 1 --2- 172.21.15.19:0/2633212250 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5583ed237500 0x5583ed239950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:45.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.188+0000 7f6c95e00700 1 --2- 172.21.15.19:0/2633212250 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5583ed237500 0x5583ed239950 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=58d05d1e4be2e804 server_cookie=43e3577bf57d0a14 in_seq=0 out_seq=0 2024-05-07T02:33:45.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.188+0000 7f6c7be00700 10 client.15501 ms_handle_connect on v2:172.21.15.19:6840/3905645246 2024-05-07T02:33:45.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.188+0000 7f6c7be00700 1 -- 172.21.15.19:0/2633212250 <== mds.1 v2:172.21.15.19:6840/3905645246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5583ed1ea2e0 con 0x5583ed237500 2024-05-07T02:33:45.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.188+0000 7f6c7be00700 10 client.15501 handle_command_reply: tid=0 2024-05-07T02:33:45.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.189+0000 7f6cabc2db80 1 client.15501 shutdown 2024-05-07T02:33:45.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.189+0000 7f6cabc2db80 1 -- 172.21.15.19:0/2633212250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583ed21ddc0 msgr2=0x5583ed227af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:45.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.189+0000 7f6cabc2db80 1 --2- 172.21.15.19:0/2633212250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583ed21ddc0 0x5583ed227af0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x5583ed219a40 tx=0x7f6c80009c00 comp rx=0 tx=0).stop 2024-05-07T02:33:45.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.189+0000 7f6cabc2db80 1 -- 172.21.15.19:0/2633212250 shutdown_connections 2024-05-07T02:33:45.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.189+0000 7f6cabc2db80 1 --2- 172.21.15.19:0/2633212250 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5583ed237500 0x5583ed239950 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:45.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.189+0000 7f6cabc2db80 1 --2- 172.21.15.19:0/2633212250 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583ed2233b0 0x5583ed228030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:45.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.189+0000 7f6c7be00700 0 client.15501 ms_handle_reset on v2:172.21.15.19:6840/3905645246 2024-05-07T02:33:45.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.189+0000 7f6cabc2db80 1 --2- 172.21.15.19:0/2633212250 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583ed21d470 0x5583ed2275b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:45.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.189+0000 7f6cabc2db80 1 --2- 172.21.15.19:0/2633212250 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583ed21ddc0 0x5583ed227af0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:45.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.189+0000 7f6cabc2db80 1 -- 172.21.15.19:0/2633212250 >> 172.21.15.19:0/2633212250 conn(0x5583ed09fe40 msgr2=0x5583ed21c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:45.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.189+0000 7f6cabc2db80 1 -- 172.21.15.19:0/2633212250 shutdown_connections 2024-05-07T02:33:45.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.189+0000 7f6cabc2db80 1 -- 172.21.15.19:0/2633212250 wait complete. 2024-05-07T02:33:45.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.190+0000 7f6cabc2db80 20 client.15501 trim_cache size 0 max 16384 2024-05-07T02:33:45.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.208+0000 7f6cabc2db80 1 Processor -- start 2024-05-07T02:33:45.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.208+0000 7f6cabc2db80 1 -- start start 2024-05-07T02:33:45.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.208+0000 7f6cabc2db80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583ed21d470 0x5583ed218360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:45.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.208+0000 7f6cabc2db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583ed21ddc0 0x5583ed227e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:45.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.208+0000 7f6cabc2db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583ed2233b0 0x5583ed224eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:45.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.208+0000 7f6cabc2db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6c981b6c50 con 0x5583ed21ddc0 2024-05-07T02:33:45.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.208+0000 7f6cabc2db80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6c981b8850 con 0x5583ed21d470 2024-05-07T02:33:45.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.208+0000 7f6cabc2db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6c981b7320 con 0x5583ed2233b0 2024-05-07T02:33:45.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.208+0000 7f6c96800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583ed2233b0 0x5583ed224eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:45.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.209+0000 7f6c96800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583ed2233b0 0x5583ed224eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:51162/0 (socket says 172.21.15.19:51162) 2024-05-07T02:33:45.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.208+0000 7f6c8fe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583ed21ddc0 0x5583ed227e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:45.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.209+0000 7f6c96800700 1 -- 172.21.15.19:0/649275398 learned_addr learned my addr 172.21.15.19:0/649275398 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:45.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.209+0000 7f6c95e00700 1 --2- 172.21.15.19:0/649275398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583ed21d470 0x5583ed218360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:45.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.209+0000 7f6c96800700 1 -- 172.21.15.19:0/649275398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583ed21d470 msgr2=0x5583ed218360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:45.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.209+0000 7f6c96800700 1 --2- 172.21.15.19:0/649275398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583ed21d470 0x5583ed218360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:45.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.209+0000 7f6c96800700 1 -- 172.21.15.19:0/649275398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583ed21ddc0 msgr2=0x5583ed227e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:45.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.209+0000 7f6c96800700 1 --2- 172.21.15.19:0/649275398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583ed21ddc0 0x5583ed227e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:45.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.209+0000 7f6c96800700 1 -- 172.21.15.19:0/649275398 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583ed21e1d0 con 0x5583ed2233b0 2024-05-07T02:33:45.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.210+0000 7f6c96800700 1 --2- 172.21.15.19:0/649275398 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583ed2233b0 0x5583ed224eb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6c9001a760 tx=0x7f6c9000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1c2fcae89ad090f server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:45.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.210+0000 7f6c86a00700 1 -- 172.21.15.19:0/649275398 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c90026650 con 0x5583ed2233b0 2024-05-07T02:33:45.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.210+0000 7f6c86a00700 1 -- 172.21.15.19:0/649275398 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c9002b070 con 0x5583ed2233b0 2024-05-07T02:33:45.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.210+0000 7f6c86a00700 1 -- 172.21.15.19:0/649275398 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c90019aa0 con 0x5583ed2233b0 2024-05-07T02:33:45.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.211+0000 7f6cabc2db80 1 -- 172.21.15.19:0/649275398 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583ed2233b0 msgr2=0x5583ed224eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:45.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.211+0000 7f6cabc2db80 1 --2- 172.21.15.19:0/649275398 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583ed2233b0 0x5583ed224eb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6c9001a760 tx=0x7f6c9000b040 comp rx=0 tx=0).stop 2024-05-07T02:33:45.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.211+0000 7f6cabc2db80 1 -- 172.21.15.19:0/649275398 shutdown_connections 2024-05-07T02:33:45.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.211+0000 7f6cabc2db80 1 --2- 172.21.15.19:0/649275398 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583ed2233b0 0x5583ed224eb0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:45.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.211+0000 7f6cabc2db80 1 --2- 172.21.15.19:0/649275398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583ed21d470 0x5583ed218360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:45.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.211+0000 7f6cabc2db80 1 --2- 172.21.15.19:0/649275398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583ed21ddc0 0x5583ed227e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:45.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.211+0000 7f6cabc2db80 1 -- 172.21.15.19:0/649275398 >> 172.21.15.19:0/649275398 conn(0x5583ed09fe40 msgr2=0x5583ed21c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:45.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.211+0000 7f6cabc2db80 1 -- 172.21.15.19:0/649275398 shutdown_connections 2024-05-07T02:33:45.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.211+0000 7f6cabc2db80 1 -- 172.21.15.19:0/649275398 wait complete. 2024-05-07T02:33:45.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.212+0000 7f6cabc2db80 1 -- start start 2024-05-07T02:33:45.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.212+0000 7f6cabc2db80 1 Processor -- start 2024-05-07T02:33:45.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.212+0000 7f6cabc2db80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583ed21d470 0x5583ed230040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:45.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.212+0000 7f6cabc2db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583ed21ddc0 0x5583ed23e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:45.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.213+0000 7f6cabc2db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583ed2233b0 0x5583ed23eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:45.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.213+0000 7f6cabc2db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6c981b80c0 con 0x5583ed21ddc0 2024-05-07T02:33:45.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.213+0000 7f6cabc2db80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6c981b79f0 con 0x5583ed21d470 2024-05-07T02:33:45.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.213+0000 7f6cabc2db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6c981b4bf0 con 0x5583ed2233b0 2024-05-07T02:33:45.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.213+0000 7f6cabc2db80 10 client.0 _ll_register_callbacks cb 0x5583ed0e6c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:33:45.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.213+0000 7f6c8fe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583ed21ddc0 0x5583ed23e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:45.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.213+0000 7f6c8fe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583ed21ddc0 0x5583ed23e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45040/0 (socket says 172.21.15.19:45040) 2024-05-07T02:33:45.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.213+0000 7f6c8fe00700 1 -- 172.21.15.19:0/2216809729 learned_addr learned my addr 172.21.15.19:0/2216809729 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:45.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.213+0000 7f6c95e00700 1 --2- 172.21.15.19:0/2216809729 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583ed21d470 0x5583ed230040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:45.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.213+0000 7f6c96800700 1 --2- 172.21.15.19:0/2216809729 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583ed2233b0 0x5583ed23eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:45.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.213+0000 7f6c95e00700 1 -- 172.21.15.19:0/2216809729 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583ed2233b0 msgr2=0x5583ed23eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:45.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.213+0000 7f6c95e00700 1 --2- 172.21.15.19:0/2216809729 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583ed2233b0 0x5583ed23eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:45.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.213+0000 7f6c95e00700 1 -- 172.21.15.19:0/2216809729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583ed21ddc0 msgr2=0x5583ed23e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:45.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.213+0000 7f6c95e00700 1 --2- 172.21.15.19:0/2216809729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583ed21ddc0 0x5583ed23e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:45.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.213+0000 7f6c95e00700 1 -- 172.21.15.19:0/2216809729 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583ed23f620 con 0x5583ed21d470 2024-05-07T02:33:45.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.213+0000 7f6c8fe00700 1 --2- 172.21.15.19:0/2216809729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583ed21ddc0 0x5583ed23e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:33:45.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.214+0000 7f6c95e00700 1 --2- 172.21.15.19:0/2216809729 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583ed21d470 0x5583ed230040 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x5583ed237910 tx=0x7f6c88010980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb041982c84087f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:45.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.214+0000 7f6c7be00700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:33:45.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.214+0000 7f6c7be00700 1 -- 172.21.15.19:0/2216809729 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c88004230 con 0x5583ed21d470 2024-05-07T02:33:45.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.214+0000 7f6cabc2db80 1 -- 172.21.15.19:0/2216809729 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c9002b070 con 0x5583ed21d470 2024-05-07T02:33:45.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.214+0000 7f6c7be00700 1 -- 172.21.15.19:0/2216809729 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f6c880043d0 con 0x5583ed21d470 2024-05-07T02:33:45.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.214+0000 7f6c7be00700 1 -- 172.21.15.19:0/2216809729 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6c88004830 con 0x5583ed21d470 2024-05-07T02:33:45.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.214+0000 7f6cabc2db80 10 client.25661 fetch_fsmap learned FSMap version 34 2024-05-07T02:33:45.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.214+0000 7f6cabc2db80 1 -- 172.21.15.19:0/2216809729 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5583ed2a1dc0 con 0x5583ed21d470 2024-05-07T02:33:45.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.215+0000 7f6c7be00700 1 -- 172.21.15.19:0/2216809729 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c9002b070 con 0x5583ed21d470 2024-05-07T02:33:45.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.215+0000 7f6c7be00700 1 -- 172.21.15.19:0/2216809729 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f6c88003680 con 0x5583ed21d470 2024-05-07T02:33:45.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.215+0000 7f6cabc2db80 10 client.25661 fetch_fsmap finished waiting for FSMap version 34 2024-05-07T02:33:45.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.215+0000 7f6cabc2db80 10 client.25661 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-07T02:33:45.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.215+0000 7f6cabc2db80 20 client.25661 populate_metadata read hostname 'smithi019' 2024-05-07T02:33:45.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.215+0000 7f6cabc2db80 1 --2- 172.21.15.19:0/2216809729 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5583ed2a24a0 0x5583ed2a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:45.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.215+0000 7f6cabc2db80 4 client.25661 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:33:45.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.215+0000 7f6cabc2db80 1 -- 172.21.15.19:0/2216809729 --> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5583ed14d400 con 0x5583ed2a24a0 2024-05-07T02:33:45.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.215+0000 7f6c8fe00700 1 --2- 172.21.15.19:0/2216809729 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5583ed2a24a0 0x5583ed2a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:45.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.216+0000 7f6c8fe00700 1 --2- 172.21.15.19:0/2216809729 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5583ed2a24a0 0x5583ed2a48c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c4072a8b54dac6be server_cookie=5846a4cfaac37546 in_seq=0 out_seq=0 2024-05-07T02:33:45.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.216+0000 7f6c7be00700 10 client.25661 ms_handle_connect on v2:172.21.15.19:6840/3905645246 2024-05-07T02:33:45.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.216+0000 7f6c7be00700 1 -- 172.21.15.19:0/2216809729 <== mds.1 v2:172.21.15.19:6840/3905645246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5583ed14d400 con 0x5583ed2a24a0 2024-05-07T02:33:45.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.216+0000 7f6c7be00700 10 client.25661 handle_command_reply: tid=0 2024-05-07T02:33:45.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.216+0000 7f6cabc2db80 1 client.25661 shutdown 2024-05-07T02:33:45.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.216+0000 7f6cabc2db80 1 -- 172.21.15.19:0/2216809729 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583ed21d470 msgr2=0x5583ed230040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:45.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.216+0000 7f6cabc2db80 1 --2- 172.21.15.19:0/2216809729 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583ed21d470 0x5583ed230040 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x5583ed237910 tx=0x7f6c88010980 comp rx=0 tx=0).stop 2024-05-07T02:33:45.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.217+0000 7f6cabc2db80 1 -- 172.21.15.19:0/2216809729 shutdown_connections 2024-05-07T02:33:45.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.217+0000 7f6cabc2db80 1 --2- 172.21.15.19:0/2216809729 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5583ed2a24a0 0x5583ed2a48c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:45.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.217+0000 7f6cabc2db80 1 --2- 172.21.15.19:0/2216809729 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583ed2233b0 0x5583ed23eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:45.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.217+0000 7f6cabc2db80 1 --2- 172.21.15.19:0/2216809729 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5583ed21d470 0x5583ed230040 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:45.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.217+0000 7f6cabc2db80 1 --2- 172.21.15.19:0/2216809729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5583ed21ddc0 0x5583ed23e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:45.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.217+0000 7f6c7be00700 0 client.25661 ms_handle_reset on v2:172.21.15.19:6840/3905645246 2024-05-07T02:33:45.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.217+0000 7f6cabc2db80 1 -- 172.21.15.19:0/2216809729 >> 172.21.15.19:0/2216809729 conn(0x5583ed09fe40 msgr2=0x5583ed2128b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:45.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.217+0000 7f6cabc2db80 1 -- 172.21.15.19:0/2216809729 shutdown_connections 2024-05-07T02:33:45.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.217+0000 7f6cabc2db80 1 -- 172.21.15.19:0/2216809729 wait complete. 2024-05-07T02:33:45.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.217+0000 7f6cabc2db80 20 client.25661 trim_cache size 0 max 16384 2024-05-07T02:33:45.221 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:33:45.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.218+0000 7f6c9d000700 1 -- 172.21.15.19:0/840737685 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f6c7c0b2a70 msgr2=0x7f6c7c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:45.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.218+0000 7f6c9d000700 1 --2- 172.21.15.19:0/840737685 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f6c7c0b2a70 0x7f6c7c0b4f80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6c90009e60 tx=0x7f6c900099e0 comp rx=0 tx=0).stop 2024-05-07T02:33:45.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.218+0000 7f6c9d000700 1 -- 172.21.15.19:0/840737685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c980764b0 msgr2=0x7f6c981a1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:45.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.218+0000 7f6c9d000700 1 --2- 172.21.15.19:0/840737685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c980764b0 0x7f6c981a1af0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6c8000c010 tx=0x7f6c8000fa00 comp rx=0 tx=0).stop 2024-05-07T02:33:45.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.218+0000 7f6c9d000700 1 -- 172.21.15.19:0/840737685 shutdown_connections 2024-05-07T02:33:45.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.218+0000 7f6c9d000700 1 --2- 172.21.15.19:0/840737685 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f6c7c0b2a70 0x7f6c7c0b4f80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:45.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.218+0000 7f6c9d000700 1 --2- 172.21.15.19:0/840737685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c98115370 0x7f6c981a5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:45.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.218+0000 7f6c9d000700 1 --2- 172.21.15.19:0/840737685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c98075b60 0x7f6c981a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:45.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.218+0000 7f6c9d000700 1 --2- 172.21.15.19:0/840737685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c980764b0 0x7f6c981a1af0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:45.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.218+0000 7f6c9d000700 1 -- 172.21.15.19:0/840737685 >> 172.21.15.19:0/840737685 conn(0x7f6c98100040 msgr2=0x7f6c98100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:45.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.218+0000 7f6c9d000700 1 -- 172.21.15.19:0/840737685 shutdown_connections 2024-05-07T02:33:45.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:45.218+0000 7f6c9d000700 1 -- 172.21.15.19:0/840737685 wait complete. 2024-05-07T02:33:45.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:44 smithi064 ceph-mon[34699]: pgmap v60: 129 pgs: 129 active+clean; 34 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 2.2 MiB/s wr, 97 op/s 2024-05-07T02:33:45.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:44 smithi027 ceph-mon[34398]: pgmap v60: 129 pgs: 129 active+clean; 34 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 2.2 MiB/s wr, 97 op/s 2024-05-07T02:33:45.725 INFO:tasks.workunit.client.0.smithi019.stdout: 1 33119 4.76 MB/sec warmup 41 sec latency 4894.664 ms 2024-05-07T02:33:46.291 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-07T02:33:46.725 INFO:tasks.workunit.client.0.smithi019.stdout: 1 34985 4.90 MB/sec warmup 42 sec latency 13.254 ms 2024-05-07T02:33:46.770 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:33:47.181 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:46 smithi019 ceph-mon[29781]: pgmap v61: 129 pgs: 129 active+clean; 34 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 2.3 MiB/s wr, 109 op/s 2024-05-07T02:33:47.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:46 smithi064 ceph-mon[34699]: pgmap v61: 129 pgs: 129 active+clean; 34 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 2.3 MiB/s wr, 109 op/s 2024-05-07T02:33:47.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:46 smithi027 ceph-mon[34398]: pgmap v61: 129 pgs: 129 active+clean; 34 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 2.3 MiB/s wr, 109 op/s 2024-05-07T02:33:47.725 INFO:tasks.workunit.client.0.smithi019.stdout: 1 35581 4.90 MB/sec warmup 43 sec latency 740.211 ms 2024-05-07T02:33:48.725 INFO:tasks.workunit.client.0.smithi019.stdout: 1 35581 4.79 MB/sec warmup 44 sec latency 1740.356 ms 2024-05-07T02:33:49.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.333+0000 7fdc63200700 1 -- 172.21.15.19:0/3280782933 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc64105c40 msgr2=0x7fdc6410e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:49.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.333+0000 7fdc63200700 1 --2- 172.21.15.19:0/3280782933 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc64105c40 0x7fdc6410e190 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdc500098f0 tx=0x7fdc50009c00 comp rx=0 tx=0).stop 2024-05-07T02:33:49.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.333+0000 7fdc63200700 1 -- 172.21.15.19:0/3280782933 shutdown_connections 2024-05-07T02:33:49.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.333+0000 7fdc63200700 1 --2- 172.21.15.19:0/3280782933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc6410e720 0x7fdc64110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:49.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.333+0000 7fdc63200700 1 --2- 172.21.15.19:0/3280782933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc641052f0 0x7fdc64105700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:49.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.333+0000 7fdc63200700 1 --2- 172.21.15.19:0/3280782933 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc64105c40 0x7fdc6410e190 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:49.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.333+0000 7fdc63200700 1 -- 172.21.15.19:0/3280782933 >> 172.21.15.19:0/3280782933 conn(0x7fdc64078ac0 msgr2=0x7fdc64078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:49.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.334+0000 7fdc63200700 1 -- 172.21.15.19:0/3280782933 shutdown_connections 2024-05-07T02:33:49.336 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:48 smithi019 ceph-mon[29781]: pgmap v62: 129 pgs: 129 active+clean; 34 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 166 KiB/s wr, 59 op/s 2024-05-07T02:33:49.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.334+0000 7fdc63200700 1 -- 172.21.15.19:0/3280782933 wait complete. 2024-05-07T02:33:49.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.335+0000 7fdc63200700 1 Processor -- start 2024-05-07T02:33:49.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.335+0000 7fdc63200700 1 -- start start 2024-05-07T02:33:49.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.336+0000 7fdc63200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc641052f0 0x7fdc64102fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:49.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.336+0000 7fdc63200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc64105c40 0x7fdc64103520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:49.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.336+0000 7fdc63200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc6410e720 0x7fdc64103a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:49.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.336+0000 7fdc63200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdc641142b0 con 0x7fdc6410e720 2024-05-07T02:33:49.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.336+0000 7fdc63200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdc64114130 con 0x7fdc64105c40 2024-05-07T02:33:49.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.336+0000 7fdc63200700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdc64114430 con 0x7fdc641052f0 2024-05-07T02:33:49.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.336+0000 7fdc61e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc641052f0 0x7fdc64102fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:49.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.336+0000 7fdc61400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc64105c40 0x7fdc64103520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:49.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.336+0000 7fdc62800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc6410e720 0x7fdc64103a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:49.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.336+0000 7fdc61e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc641052f0 0x7fdc64102fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:45622/0 (socket says 172.21.15.19:45622) 2024-05-07T02:33:49.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.336+0000 7fdc61e00700 1 -- 172.21.15.19:0/1830072372 learned_addr learned my addr 172.21.15.19:0/1830072372 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:49.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.337+0000 7fdc61400700 1 -- 172.21.15.19:0/1830072372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc641052f0 msgr2=0x7fdc64102fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:49.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.337+0000 7fdc61400700 1 --2- 172.21.15.19:0/1830072372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc641052f0 0x7fdc64102fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:49.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.337+0000 7fdc61400700 1 -- 172.21.15.19:0/1830072372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc6410e720 msgr2=0x7fdc64103a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:49.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.337+0000 7fdc61400700 1 --2- 172.21.15.19:0/1830072372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc6410e720 0x7fdc64103a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:49.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.337+0000 7fdc61400700 1 -- 172.21.15.19:0/1830072372 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc641a9040 con 0x7fdc64105c40 2024-05-07T02:33:49.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.337+0000 7fdc61e00700 1 --2- 172.21.15.19:0/1830072372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc641052f0 0x7fdc64102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:33:49.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.337+0000 7fdc61400700 1 --2- 172.21.15.19:0/1830072372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc64105c40 0x7fdc64103520 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000c010 tx=0x7fdc5000fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:49.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.337+0000 7fdc5aa00700 1 -- 172.21.15.19:0/1830072372 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc500050c0 con 0x7fdc64105c40 2024-05-07T02:33:49.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.337+0000 7fdc63200700 1 -- 172.21.15.19:0/1830072372 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc641a92d0 con 0x7fdc64105c40 2024-05-07T02:33:49.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.337+0000 7fdc5aa00700 1 -- 172.21.15.19:0/1830072372 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc5000b560 con 0x7fdc64105c40 2024-05-07T02:33:49.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.337+0000 7fdc5aa00700 1 -- 172.21.15.19:0/1830072372 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc50003a00 con 0x7fdc64105c40 2024-05-07T02:33:49.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.338+0000 7fdc63200700 1 -- 172.21.15.19:0/1830072372 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc641a9800 con 0x7fdc64105c40 2024-05-07T02:33:49.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.341+0000 7fdc5aa00700 1 -- 172.21.15.19:0/1830072372 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fdc5000b710 con 0x7fdc64105c40 2024-05-07T02:33:49.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.343+0000 7fdc77df5b80 1 Processor -- start 2024-05-07T02:33:49.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.344+0000 7fdc77df5b80 1 -- start start 2024-05-07T02:33:49.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.344+0000 7fdc77df5b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567b853fa90 0x5567b853fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:49.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.344+0000 7fdc77df5b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567b85403e0 0x5567b8549620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:49.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.344+0000 7fdc5aa00700 1 --2- 172.21.15.19:0/1830072372 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fdc480b2a70 0x7fdc480b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:49.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.344+0000 7fdc77df5b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b8549b60 0x5567b854bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:49.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.344+0000 7fdc77df5b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5567b8461a20 con 0x5567b8549b60 2024-05-07T02:33:49.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.344+0000 7fdc77df5b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5567b83ff760 con 0x5567b853fa90 2024-05-07T02:33:49.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.344+0000 7fdc77df5b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5567b845d9e0 con 0x5567b85403e0 2024-05-07T02:33:49.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.344+0000 7fdc5aa00700 1 -- 172.21.15.19:0/1830072372 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fdc500e3720 con 0x7fdc64105c40 2024-05-07T02:33:49.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.345+0000 7fdc62800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567b853fa90 0x5567b853fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:49.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.345+0000 7fdc61e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567b85403e0 0x5567b8549620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:49.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.345+0000 7fdc61400700 1 --2- 172.21.15.19:0/1830072372 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fdc480b2a70 0x7fdc480b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:49.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.345+0000 7fdc62800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567b853fa90 0x5567b853fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:32794/0 (socket says 172.21.15.19:32794) 2024-05-07T02:33:49.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.345+0000 7fdc62800700 1 -- 172.21.15.19:0/1631628158 learned_addr learned my addr 172.21.15.19:0/1631628158 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:49.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.345+0000 7fdc62800700 1 --2- 172.21.15.19:0/1631628158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b8549b60 0x5567b854bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:49.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.346+0000 7fdc62800700 1 -- 172.21.15.19:0/1631628158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567b85403e0 msgr2=0x5567b8549620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:49.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.346+0000 7fdc62800700 1 --2- 172.21.15.19:0/1631628158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567b85403e0 0x5567b8549620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:49.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.346+0000 7fdc62800700 1 -- 172.21.15.19:0/1631628158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b8549b60 msgr2=0x5567b854bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:49.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.346+0000 7fdc62800700 1 --2- 172.21.15.19:0/1631628158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b8549b60 0x5567b854bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:49.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.346+0000 7fdc62800700 1 -- 172.21.15.19:0/1631628158 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567b83ae8d0 con 0x5567b853fa90 2024-05-07T02:33:49.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.346+0000 7fdc61400700 1 --2- 172.21.15.19:0/1830072372 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fdc480b2a70 0x7fdc480b4f80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdc500e0ae0 tx=0x7fdc50009f50 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:49.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.346+0000 7fdc62800700 1 --2- 172.21.15.19:0/1631628158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567b853fa90 0x5567b853fea0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c013880 tx=0x7fdc5c009860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47c2ec6afd546823 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:49.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.346+0000 7fdc4f400700 1 -- 172.21.15.19:0/1631628158 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c010460 con 0x5567b853fa90 2024-05-07T02:33:49.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.346+0000 7fdc4f400700 1 -- 172.21.15.19:0/1631628158 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc5c010ab0 con 0x5567b853fa90 2024-05-07T02:33:49.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.346+0000 7fdc4f400700 1 -- 172.21.15.19:0/1631628158 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c01f5f0 con 0x5567b853fa90 2024-05-07T02:33:49.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.347+0000 7fdc77df5b80 1 -- 172.21.15.19:0/1631628158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567b853fa90 msgr2=0x5567b853fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:49.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.347+0000 7fdc77df5b80 1 --2- 172.21.15.19:0/1631628158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567b853fa90 0x5567b853fea0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c013880 tx=0x7fdc5c009860 comp rx=0 tx=0).stop 2024-05-07T02:33:49.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.347+0000 7fdc77df5b80 1 -- 172.21.15.19:0/1631628158 shutdown_connections 2024-05-07T02:33:49.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.348+0000 7fdc77df5b80 1 --2- 172.21.15.19:0/1631628158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b8549b60 0x5567b854bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:49.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.348+0000 7fdc77df5b80 1 --2- 172.21.15.19:0/1631628158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567b853fa90 0x5567b853fea0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:49.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.348+0000 7fdc77df5b80 1 --2- 172.21.15.19:0/1631628158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567b85403e0 0x5567b8549620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:49.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.348+0000 7fdc77df5b80 1 -- 172.21.15.19:0/1631628158 >> 172.21.15.19:0/1631628158 conn(0x5567b83c1e40 msgr2=0x5567b853a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:49.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.348+0000 7fdc77df5b80 1 -- 172.21.15.19:0/1631628158 shutdown_connections 2024-05-07T02:33:49.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.348+0000 7fdc77df5b80 1 -- 172.21.15.19:0/1631628158 wait complete. 2024-05-07T02:33:49.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.348+0000 7fdc77df5b80 1 -- start start 2024-05-07T02:33:49.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.349+0000 7fdc77df5b80 1 Processor -- start 2024-05-07T02:33:49.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.349+0000 7fdc77df5b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567b853fa90 0x5567b8546b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:49.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.349+0000 7fdc77df5b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b85403e0 0x5567b8547050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:49.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.349+0000 7fdc77df5b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567b8549b60 0x5567b8547590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:49.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.349+0000 7fdc77df5b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5567b854c960 con 0x5567b85403e0 2024-05-07T02:33:49.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.349+0000 7fdc77df5b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5567b83c7b20 con 0x5567b853fa90 2024-05-07T02:33:49.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.349+0000 7fdc77df5b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5567b854cae0 con 0x5567b8549b60 2024-05-07T02:33:49.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.349+0000 7fdc77df5b80 10 client.0 _ll_register_callbacks cb 0x5567b8408c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:33:49.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.350+0000 7fdc62800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567b853fa90 0x5567b8546b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:49.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.350+0000 7fdc62800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567b853fa90 0x5567b8546b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:32800/0 (socket says 172.21.15.19:32800) 2024-05-07T02:33:49.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.350+0000 7fdc61e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b85403e0 0x5567b8547050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:49.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.350+0000 7fdc62800700 1 -- 172.21.15.19:0/2922997424 learned_addr learned my addr 172.21.15.19:0/2922997424 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:49.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.350+0000 7fdc62800700 1 --2- 172.21.15.19:0/2922997424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567b8549b60 0x5567b8547590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:49.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.350+0000 7fdc62800700 1 -- 172.21.15.19:0/2922997424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567b8549b60 msgr2=0x5567b8547590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:49.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.350+0000 7fdc62800700 1 --2- 172.21.15.19:0/2922997424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567b8549b60 0x5567b8547590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:49.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.350+0000 7fdc62800700 1 -- 172.21.15.19:0/2922997424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b85403e0 msgr2=0x5567b8547050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:49.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.350+0000 7fdc62800700 1 --2- 172.21.15.19:0/2922997424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b85403e0 0x5567b8547050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:49.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.350+0000 7fdc62800700 1 -- 172.21.15.19:0/2922997424 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567b8427420 con 0x5567b853fa90 2024-05-07T02:33:49.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.350+0000 7fdc61e00700 1 --2- 172.21.15.19:0/2922997424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b85403e0 0x5567b8547050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:33:49.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.350+0000 7fdc62800700 1 --2- 172.21.15.19:0/2922997424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567b853fa90 0x5567b8546b10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c0057a0 tx=0x7fdc5c018040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99eec84faf894091 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:49.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.350+0000 7fdc43e00700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:33:49.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.351+0000 7fdc43e00700 1 -- 172.21.15.19:0/2922997424 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c01f910 con 0x5567b853fa90 2024-05-07T02:33:49.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.351+0000 7fdc77df5b80 1 -- 172.21.15.19:0/2922997424 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc5c010ab0 con 0x5567b853fa90 2024-05-07T02:33:49.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.351+0000 7fdc43e00700 1 -- 172.21.15.19:0/2922997424 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fdc5c01fab0 con 0x5567b853fa90 2024-05-07T02:33:49.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.351+0000 7fdc43e00700 1 -- 172.21.15.19:0/2922997424 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fdc5c034050 con 0x5567b853fa90 2024-05-07T02:33:49.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.351+0000 7fdc77df5b80 10 client.25673 fetch_fsmap learned FSMap version 34 2024-05-07T02:33:49.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.351+0000 7fdc77df5b80 1 -- 172.21.15.19:0/2922997424 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5567b85575b0 con 0x5567b853fa90 2024-05-07T02:33:49.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.352+0000 7fdc43e00700 1 -- 172.21.15.19:0/2922997424 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc5c010ab0 con 0x5567b853fa90 2024-05-07T02:33:49.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.352+0000 7fdc43e00700 1 -- 172.21.15.19:0/2922997424 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7fdc5c020c20 con 0x5567b853fa90 2024-05-07T02:33:49.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.352+0000 7fdc77df5b80 10 client.25673 fetch_fsmap finished waiting for FSMap version 34 2024-05-07T02:33:49.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.352+0000 7fdc77df5b80 10 client.25673 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:33:49.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.352+0000 7fdc77df5b80 20 client.25673 populate_metadata read hostname 'smithi019' 2024-05-07T02:33:49.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.352+0000 7fdc77df5b80 1 --2- 172.21.15.19:0/2922997424 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5567b8559a80 0x5567b855bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:49.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.352+0000 7fdc77df5b80 4 client.25673 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:33:49.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.352+0000 7fdc77df5b80 1 -- 172.21.15.19:0/2922997424 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567b850c2e0 con 0x5567b8559a80 2024-05-07T02:33:49.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.353+0000 7fdc61e00700 1 --2- 172.21.15.19:0/2922997424 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5567b8559a80 0x5567b855bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:49.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.353+0000 7fdc61e00700 1 --2- 172.21.15.19:0/2922997424 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5567b8559a80 0x5567b855bea0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b79ce6da5d9082b server_cookie=8c58c496fe447427 in_seq=0 out_seq=0 2024-05-07T02:33:49.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.353+0000 7fdc43e00700 10 client.25673 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:33:49.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.354+0000 7fdc43e00700 1 -- 172.21.15.19:0/2922997424 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5567b850c2e0 con 0x5567b8559a80 2024-05-07T02:33:49.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.354+0000 7fdc43e00700 10 client.25673 handle_command_reply: tid=0 2024-05-07T02:33:49.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.354+0000 7fdc77df5b80 1 client.25673 shutdown 2024-05-07T02:33:49.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.354+0000 7fdc77df5b80 1 -- 172.21.15.19:0/2922997424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567b853fa90 msgr2=0x5567b8546b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:49.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.354+0000 7fdc77df5b80 1 --2- 172.21.15.19:0/2922997424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567b853fa90 0x5567b8546b10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c0057a0 tx=0x7fdc5c018040 comp rx=0 tx=0).stop 2024-05-07T02:33:49.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.354+0000 7fdc77df5b80 1 -- 172.21.15.19:0/2922997424 shutdown_connections 2024-05-07T02:33:49.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.354+0000 7fdc77df5b80 1 --2- 172.21.15.19:0/2922997424 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5567b8559a80 0x5567b855bea0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:49.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.354+0000 7fdc77df5b80 1 --2- 172.21.15.19:0/2922997424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567b8549b60 0x5567b8547590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:49.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.355+0000 7fdc77df5b80 1 --2- 172.21.15.19:0/2922997424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567b853fa90 0x5567b8546b10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:49.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.355+0000 7fdc77df5b80 1 --2- 172.21.15.19:0/2922997424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b85403e0 0x5567b8547050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:49.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.355+0000 7fdc43e00700 0 client.25673 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:33:49.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.355+0000 7fdc77df5b80 1 -- 172.21.15.19:0/2922997424 >> 172.21.15.19:0/2922997424 conn(0x5567b83c1e40 msgr2=0x5567b853c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:49.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.355+0000 7fdc77df5b80 1 -- 172.21.15.19:0/2922997424 shutdown_connections 2024-05-07T02:33:49.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.355+0000 7fdc77df5b80 1 -- 172.21.15.19:0/2922997424 wait complete. 2024-05-07T02:33:49.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.355+0000 7fdc77df5b80 20 client.25673 trim_cache size 0 max 16384 2024-05-07T02:33:49.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:48 smithi064 ceph-mon[34699]: pgmap v62: 129 pgs: 129 active+clean; 34 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 166 KiB/s wr, 59 op/s 2024-05-07T02:33:49.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.372+0000 7fdc77df5b80 1 Processor -- start 2024-05-07T02:33:49.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.372+0000 7fdc77df5b80 1 -- start start 2024-05-07T02:33:49.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.372+0000 7fdc77df5b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567b853fa90 0x5567b8545e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:49.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.372+0000 7fdc77df5b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567b85403e0 0x5567b85471e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:49.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.372+0000 7fdc77df5b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b8549b60 0x5567b8555a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:49.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.372+0000 7fdc77df5b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdc641b25d0 con 0x5567b8549b60 2024-05-07T02:33:49.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.372+0000 7fdc77df5b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdc641b41d0 con 0x5567b85403e0 2024-05-07T02:33:49.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.372+0000 7fdc77df5b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdc641b2ca0 con 0x5567b853fa90 2024-05-07T02:33:49.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.373+0000 7fdc62800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b8549b60 0x5567b8555a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:49.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.373+0000 7fdc61e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567b85403e0 0x5567b85471e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:49.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.373+0000 7fdc61e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567b85403e0 0x5567b85471e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:32802/0 (socket says 172.21.15.19:32802) 2024-05-07T02:33:49.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.373+0000 7fdc61e00700 1 -- 172.21.15.19:0/102080412 learned_addr learned my addr 172.21.15.19:0/102080412 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:49.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.373+0000 7fdc62800700 1 --2- 172.21.15.19:0/102080412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567b853fa90 0x5567b8545e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:49.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.373+0000 7fdc61e00700 1 -- 172.21.15.19:0/102080412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567b853fa90 msgr2=0x5567b8545e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:49.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.373+0000 7fdc61e00700 1 --2- 172.21.15.19:0/102080412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567b853fa90 0x5567b8545e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:49.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.373+0000 7fdc61e00700 1 -- 172.21.15.19:0/102080412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b8549b60 msgr2=0x5567b8555a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:49.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.373+0000 7fdc61e00700 1 --2- 172.21.15.19:0/102080412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b8549b60 0x5567b8555a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:49.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.373+0000 7fdc61e00700 1 -- 172.21.15.19:0/102080412 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567b853ac00 con 0x5567b85403e0 2024-05-07T02:33:49.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.373+0000 7fdc62800700 1 --2- 172.21.15.19:0/102080412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b8549b60 0x5567b8555a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:33:49.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.373+0000 7fdc61e00700 1 --2- 172.21.15.19:0/102080412 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567b85403e0 0x5567b85471e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x5567b853bcc0 tx=0x7fdc5400b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd17229c7f69ec7b server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:49.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.373+0000 7fdc4ea00700 1 -- 172.21.15.19:0/102080412 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc54003cc0 con 0x5567b85403e0 2024-05-07T02:33:49.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.374+0000 7fdc4ea00700 1 -- 172.21.15.19:0/102080412 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc54026070 con 0x5567b85403e0 2024-05-07T02:33:49.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.374+0000 7fdc4ea00700 1 -- 172.21.15.19:0/102080412 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc54007c50 con 0x5567b85403e0 2024-05-07T02:33:49.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.374+0000 7fdc77df5b80 1 -- 172.21.15.19:0/102080412 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567b85403e0 msgr2=0x5567b85471e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:49.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.374+0000 7fdc77df5b80 1 --2- 172.21.15.19:0/102080412 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567b85403e0 0x5567b85471e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x5567b853bcc0 tx=0x7fdc5400b040 comp rx=0 tx=0).stop 2024-05-07T02:33:49.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.374+0000 7fdc77df5b80 1 -- 172.21.15.19:0/102080412 shutdown_connections 2024-05-07T02:33:49.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.374+0000 7fdc77df5b80 1 --2- 172.21.15.19:0/102080412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b8549b60 0x5567b8555a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:49.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.374+0000 7fdc77df5b80 1 --2- 172.21.15.19:0/102080412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567b853fa90 0x5567b8545e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:49.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.374+0000 7fdc77df5b80 1 --2- 172.21.15.19:0/102080412 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567b85403e0 0x5567b85471e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:49.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.374+0000 7fdc77df5b80 1 -- 172.21.15.19:0/102080412 >> 172.21.15.19:0/102080412 conn(0x5567b83c1e40 msgr2=0x5567b8546ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:49.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.375+0000 7fdc77df5b80 1 -- 172.21.15.19:0/102080412 shutdown_connections 2024-05-07T02:33:49.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.375+0000 7fdc77df5b80 1 -- 172.21.15.19:0/102080412 wait complete. 2024-05-07T02:33:49.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.375+0000 7fdc77df5b80 1 -- start start 2024-05-07T02:33:49.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.375+0000 7fdc77df5b80 1 Processor -- start 2024-05-07T02:33:49.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.376+0000 7fdc77df5b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567b853fa90 0x5567b854cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:49.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.376+0000 7fdc77df5b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b85403e0 0x5567b854d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:49.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.376+0000 7fdc77df5b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567b8549b60 0x5567b85c2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:49.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.376+0000 7fdc77df5b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdc641b3a40 con 0x5567b85403e0 2024-05-07T02:33:49.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.376+0000 7fdc77df5b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdc641b3370 con 0x5567b853fa90 2024-05-07T02:33:49.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.376+0000 7fdc77df5b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdc641b0570 con 0x5567b8549b60 2024-05-07T02:33:49.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.376+0000 7fdc77df5b80 10 client.0 _ll_register_callbacks cb 0x5567b8408c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:33:49.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.376+0000 7fdc62800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567b853fa90 0x5567b854cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:49.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.376+0000 7fdc61e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b85403e0 0x5567b854d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:49.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.376+0000 7fdc62800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567b853fa90 0x5567b854cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:32816/0 (socket says 172.21.15.19:32816) 2024-05-07T02:33:49.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.376+0000 7fdc62800700 1 -- 172.21.15.19:0/140509434 learned_addr learned my addr 172.21.15.19:0/140509434 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:49.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.376+0000 7fdc62800700 1 --2- 172.21.15.19:0/140509434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567b8549b60 0x5567b85c2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:49.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.377+0000 7fdc62800700 1 -- 172.21.15.19:0/140509434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567b8549b60 msgr2=0x5567b85c2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:49.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.377+0000 7fdc62800700 1 --2- 172.21.15.19:0/140509434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567b8549b60 0x5567b85c2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:49.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.377+0000 7fdc62800700 1 -- 172.21.15.19:0/140509434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b85403e0 msgr2=0x5567b854d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:49.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.377+0000 7fdc62800700 1 --2- 172.21.15.19:0/140509434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b85403e0 0x5567b854d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:49.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.377+0000 7fdc62800700 1 -- 172.21.15.19:0/140509434 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567b85c2660 con 0x5567b853fa90 2024-05-07T02:33:49.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.377+0000 7fdc62800700 1 --2- 172.21.15.19:0/140509434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567b853fa90 0x5567b854cd60 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c013600 tx=0x7fdc5c010770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99218fbc70436a9b server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:49.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.377+0000 7fdc43e00700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:33:49.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.377+0000 7fdc43e00700 1 -- 172.21.15.19:0/140509434 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c01fe00 con 0x5567b853fa90 2024-05-07T02:33:49.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.377+0000 7fdc77df5b80 1 -- 172.21.15.19:0/140509434 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc54026070 con 0x5567b853fa90 2024-05-07T02:33:49.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.377+0000 7fdc43e00700 1 -- 172.21.15.19:0/140509434 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fdc5c0050b0 con 0x5567b853fa90 2024-05-07T02:33:49.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.378+0000 7fdc43e00700 1 -- 172.21.15.19:0/140509434 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fdc5c0052b0 con 0x5567b853fa90 2024-05-07T02:33:49.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.378+0000 7fdc77df5b80 10 client.25679 fetch_fsmap learned FSMap version 34 2024-05-07T02:33:49.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.378+0000 7fdc77df5b80 1 -- 172.21.15.19:0/140509434 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5567b85c3740 con 0x5567b853fa90 2024-05-07T02:33:49.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.378+0000 7fdc43e00700 1 -- 172.21.15.19:0/140509434 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc54026070 con 0x5567b853fa90 2024-05-07T02:33:49.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.378+0000 7fdc43e00700 1 -- 172.21.15.19:0/140509434 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7fdc5c010850 con 0x5567b853fa90 2024-05-07T02:33:49.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.378+0000 7fdc77df5b80 10 client.25679 fetch_fsmap finished waiting for FSMap version 34 2024-05-07T02:33:49.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.378+0000 7fdc77df5b80 10 client.25679 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:33:49.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.378+0000 7fdc77df5b80 20 client.25679 populate_metadata read hostname 'smithi019' 2024-05-07T02:33:49.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.378+0000 7fdc77df5b80 1 --2- 172.21.15.19:0/140509434 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5567b85c3dc0 0x5567b85c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:49.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.378+0000 7fdc77df5b80 4 client.25679 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:33:49.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.378+0000 7fdc77df5b80 1 -- 172.21.15.19:0/140509434 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5567b846f400 con 0x5567b85c3dc0 2024-05-07T02:33:49.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.379+0000 7fdc61e00700 1 --2- 172.21.15.19:0/140509434 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5567b85c3dc0 0x5567b85c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:49.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.379+0000 7fdc61e00700 1 --2- 172.21.15.19:0/140509434 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5567b85c3dc0 0x5567b85c61e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfe67922689e7d81 server_cookie=7fd3ecd5e575c339 in_seq=0 out_seq=0 2024-05-07T02:33:49.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.379+0000 7fdc43e00700 10 client.25679 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:33:49.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.379+0000 7fdc43e00700 1 -- 172.21.15.19:0/140509434 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5567b846f400 con 0x5567b85c3dc0 2024-05-07T02:33:49.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.379+0000 7fdc43e00700 10 client.25679 handle_command_reply: tid=0 2024-05-07T02:33:49.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.379+0000 7fdc77df5b80 1 client.25679 shutdown 2024-05-07T02:33:49.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.380+0000 7fdc77df5b80 1 -- 172.21.15.19:0/140509434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567b853fa90 msgr2=0x5567b854cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:49.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.380+0000 7fdc77df5b80 1 --2- 172.21.15.19:0/140509434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567b853fa90 0x5567b854cd60 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c013600 tx=0x7fdc5c010770 comp rx=0 tx=0).stop 2024-05-07T02:33:49.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.380+0000 7fdc77df5b80 1 -- 172.21.15.19:0/140509434 shutdown_connections 2024-05-07T02:33:49.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.380+0000 7fdc77df5b80 1 --2- 172.21.15.19:0/140509434 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5567b85c3dc0 0x5567b85c61e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:49.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.380+0000 7fdc77df5b80 1 --2- 172.21.15.19:0/140509434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5567b8549b60 0x5567b85c2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:49.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.380+0000 7fdc77df5b80 1 --2- 172.21.15.19:0/140509434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5567b853fa90 0x5567b854cd60 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:49.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.380+0000 7fdc77df5b80 1 --2- 172.21.15.19:0/140509434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b85403e0 0x5567b854d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:49.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.380+0000 7fdc43e00700 0 client.25679 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:33:49.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.380+0000 7fdc77df5b80 1 -- 172.21.15.19:0/140509434 >> 172.21.15.19:0/140509434 conn(0x5567b83c1e40 msgr2=0x5567b8545b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:49.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.380+0000 7fdc77df5b80 1 -- 172.21.15.19:0/140509434 shutdown_connections 2024-05-07T02:33:49.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.380+0000 7fdc77df5b80 1 -- 172.21.15.19:0/140509434 wait complete. 2024-05-07T02:33:49.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.380+0000 7fdc77df5b80 20 client.25679 trim_cache size 0 max 16384 2024-05-07T02:33:49.384 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:33:49.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.381+0000 7fdc63200700 1 -- 172.21.15.19:0/1830072372 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fdc480b2a70 msgr2=0x7fdc480b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:49.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.381+0000 7fdc63200700 1 --2- 172.21.15.19:0/1830072372 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fdc480b2a70 0x7fdc480b4f80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdc500e0ae0 tx=0x7fdc50009f50 comp rx=0 tx=0).stop 2024-05-07T02:33:49.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.381+0000 7fdc63200700 1 -- 172.21.15.19:0/1830072372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc64105c40 msgr2=0x7fdc64103520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:49.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.381+0000 7fdc63200700 1 --2- 172.21.15.19:0/1830072372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc64105c40 0x7fdc64103520 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000c010 tx=0x7fdc5000fa00 comp rx=0 tx=0).stop 2024-05-07T02:33:49.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.381+0000 7fdc63200700 1 -- 172.21.15.19:0/1830072372 shutdown_connections 2024-05-07T02:33:49.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.381+0000 7fdc63200700 1 --2- 172.21.15.19:0/1830072372 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fdc480b2a70 0x7fdc480b4f80 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:49.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.381+0000 7fdc63200700 1 --2- 172.21.15.19:0/1830072372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc6410e720 0x7fdc64103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:49.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.381+0000 7fdc63200700 1 --2- 172.21.15.19:0/1830072372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc641052f0 0x7fdc64102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:49.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.381+0000 7fdc63200700 1 --2- 172.21.15.19:0/1830072372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc64105c40 0x7fdc64103520 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:49.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.381+0000 7fdc63200700 1 -- 172.21.15.19:0/1830072372 >> 172.21.15.19:0/1830072372 conn(0x7fdc64078ac0 msgr2=0x7fdc640fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:49.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.381+0000 7fdc63200700 1 -- 172.21.15.19:0/1830072372 shutdown_connections 2024-05-07T02:33:49.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:49.381+0000 7fdc63200700 1 -- 172.21.15.19:0/1830072372 wait complete. 2024-05-07T02:33:49.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:48 smithi027 ceph-mon[34398]: pgmap v62: 129 pgs: 129 active+clean; 34 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 166 KiB/s wr, 59 op/s 2024-05-07T02:33:49.726 INFO:tasks.workunit.client.0.smithi019.stdout: 1 35581 4.68 MB/sec warmup 45 sec latency 2740.450 ms 2024-05-07T02:33:50.125 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:49 smithi019 ceph-mon[29781]: pgmap v63: 129 pgs: 129 active+clean; 34 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 166 KiB/s wr, 59 op/s 2024-05-07T02:33:50.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:49 smithi064 ceph-mon[34699]: pgmap v63: 129 pgs: 129 active+clean; 34 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 166 KiB/s wr, 59 op/s 2024-05-07T02:33:50.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:49 smithi027 ceph-mon[34398]: pgmap v63: 129 pgs: 129 active+clean; 34 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 166 KiB/s wr, 59 op/s 2024-05-07T02:33:50.726 INFO:tasks.workunit.client.0.smithi019.stdout: 1 35648 4.58 MB/sec warmup 46 sec latency 3500.390 ms 2024-05-07T02:33:51.392 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-07T02:33:51.393 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-07T02:33:51.726 INFO:tasks.workunit.client.0.smithi019.stdout: 1 35648 4.49 MB/sec warmup 47 sec latency 1146.731 ms 2024-05-07T02:33:51.859 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:33:52.726 INFO:tasks.workunit.client.0.smithi019.stdout: 1 35648 4.39 MB/sec warmup 48 sec latency 2146.818 ms 2024-05-07T02:33:53.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:52 smithi064 ceph-mon[34699]: pgmap v64: 129 pgs: 129 active+clean; 34 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 254 KiB/s wr, 83 op/s 2024-05-07T02:33:53.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:52 smithi019 ceph-mon[29781]: pgmap v64: 129 pgs: 129 active+clean; 34 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 254 KiB/s wr, 83 op/s 2024-05-07T02:33:53.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:52 smithi027 ceph-mon[34398]: pgmap v64: 129 pgs: 129 active+clean; 34 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 254 KiB/s wr, 83 op/s 2024-05-07T02:33:53.726 INFO:tasks.workunit.client.0.smithi019.stdout: 1 35648 4.30 MB/sec warmup 49 sec latency 3146.970 ms 2024-05-07T02:33:54.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.423+0000 7f274e000700 1 -- 172.21.15.19:0/945138741 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27481045f0 msgr2=0x7f2748104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:54.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.423+0000 7f274e000700 1 --2- 172.21.15.19:0/945138741 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27481045f0 0x7f2748104a00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f273800b0d0 tx=0x7f273800b3e0 comp rx=0 tx=0).stop 2024-05-07T02:33:54.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.424+0000 7f274e000700 1 -- 172.21.15.19:0/945138741 shutdown_connections 2024-05-07T02:33:54.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.424+0000 7f274e000700 1 --2- 172.21.15.19:0/945138741 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2748109870 0x7f27481111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:54.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.424+0000 7f274e000700 1 --2- 172.21.15.19:0/945138741 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27481045f0 0x7f2748104a00 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:54.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.424+0000 7f274e000700 1 --2- 172.21.15.19:0/945138741 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2748108f20 0x7f2748109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:54.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.424+0000 7f274e000700 1 -- 172.21.15.19:0/945138741 >> 172.21.15.19:0/945138741 conn(0x7f2748100040 msgr2=0x7f2748102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:54.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.424+0000 7f274e000700 1 -- 172.21.15.19:0/945138741 shutdown_connections 2024-05-07T02:33:54.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.424+0000 7f274e000700 1 -- 172.21.15.19:0/945138741 wait complete. 2024-05-07T02:33:54.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.425+0000 7f274e000700 1 Processor -- start 2024-05-07T02:33:54.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.425+0000 7f274e000700 1 -- start start 2024-05-07T02:33:54.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.426+0000 7f274e000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27481045f0 0x7f274819ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:54.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.426+0000 7f274e000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2748108f20 0x7f274819b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:54.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.426+0000 7f274e000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2748109870 0x7f274819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:54.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.426+0000 7f274e000700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f27481144a0 con 0x7f2748109870 2024-05-07T02:33:54.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.426+0000 7f274e000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2748114320 con 0x7f2748108f20 2024-05-07T02:33:54.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.426+0000 7f274e000700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2748114620 con 0x7f27481045f0 2024-05-07T02:33:54.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.426+0000 7f2747e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2748109870 0x7f274819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:54.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.426+0000 7f2747400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27481045f0 0x7f274819ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:54.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.426+0000 7f2747e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2748109870 0x7f274819f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48160/0 (socket says 172.21.15.19:48160) 2024-05-07T02:33:54.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.426+0000 7f2747e00700 1 -- 172.21.15.19:0/3008844542 learned_addr learned my addr 172.21.15.19:0/3008844542 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:54.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.426+0000 7f273ea00700 1 --2- 172.21.15.19:0/3008844542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2748108f20 0x7f274819b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:54.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.427+0000 7f2747e00700 1 -- 172.21.15.19:0/3008844542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27481045f0 msgr2=0x7f274819ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:54.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.427+0000 7f2747e00700 1 --2- 172.21.15.19:0/3008844542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27481045f0 0x7f274819ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:54.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.427+0000 7f2747e00700 1 -- 172.21.15.19:0/3008844542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2748108f20 msgr2=0x7f274819b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:54.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.427+0000 7f2747e00700 1 --2- 172.21.15.19:0/3008844542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2748108f20 0x7f274819b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:54.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.427+0000 7f2747e00700 1 -- 172.21.15.19:0/3008844542 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f274819fd40 con 0x7f2748109870 2024-05-07T02:33:54.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.427+0000 7f2747e00700 1 --2- 172.21.15.19:0/3008844542 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2748109870 0x7f274819f6a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f274000e940 tx=0x7f274000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:54.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.427+0000 7f2744c00700 1 -- 172.21.15.19:0/3008844542 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2740004a30 con 0x7f2748109870 2024-05-07T02:33:54.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.427+0000 7f2744c00700 1 -- 172.21.15.19:0/3008844542 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27400054d0 con 0x7f2748109870 2024-05-07T02:33:54.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.427+0000 7f2744c00700 1 -- 172.21.15.19:0/3008844542 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274000ccf0 con 0x7f2748109870 2024-05-07T02:33:54.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.427+0000 7f274e000700 1 -- 172.21.15.19:0/3008844542 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27481a0030 con 0x7f2748109870 2024-05-07T02:33:54.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.428+0000 7f274e000700 1 -- 172.21.15.19:0/3008844542 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27481a7950 con 0x7f2748109870 2024-05-07T02:33:54.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.431+0000 7f2744c00700 1 -- 172.21.15.19:0/3008844542 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2740005020 con 0x7f2748109870 2024-05-07T02:33:54.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.433+0000 7f275ca93b80 1 Processor -- start 2024-05-07T02:33:54.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.433+0000 7f275ca93b80 1 -- start start 2024-05-07T02:33:54.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.433+0000 7f2744c00700 1 --2- 172.21.15.19:0/3008844542 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f272c0b2a70 0x7f272c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:54.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.433+0000 7f2744c00700 1 -- 172.21.15.19:0/3008844542 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f27400e2a90 con 0x7f2748109870 2024-05-07T02:33:54.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.433+0000 7f273ea00700 1 --2- 172.21.15.19:0/3008844542 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f272c0b2a70 0x7f272c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:54.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.433+0000 7f275ca93b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55802dd0c4f0 0x55802dd0c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:54.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.433+0000 7f275ca93b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55802dd066a0 0x55802dd06ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:54.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.433+0000 7f275ca93b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55802dd06ff0 0x55802dd131d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:54.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.433+0000 7f275ca93b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55802dc28aa0 con 0x55802dd0c4f0 2024-05-07T02:33:54.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.433+0000 7f275ca93b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55802dbc67e0 con 0x55802dd06ff0 2024-05-07T02:33:54.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.433+0000 7f275ca93b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55802dc24a60 con 0x55802dd066a0 2024-05-07T02:33:54.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.434+0000 7f2747400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55802dd0c4f0 0x55802dd0c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:54.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.434+0000 7f2747400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55802dd0c4f0 0x55802dd0c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48162/0 (socket says 172.21.15.19:48162) 2024-05-07T02:33:54.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.434+0000 7f2747400700 1 -- 172.21.15.19:0/4054990381 learned_addr learned my addr 172.21.15.19:0/4054990381 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:54.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.434+0000 7f2747e00700 1 --2- 172.21.15.19:0/4054990381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55802dd06ff0 0x55802dd131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:54.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.434+0000 7f273ea00700 1 --2- 172.21.15.19:0/4054990381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55802dd066a0 0x55802dd06ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:54.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.434+0000 7f2747400700 1 -- 172.21.15.19:0/4054990381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55802dd066a0 msgr2=0x55802dd06ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:54.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.434+0000 7f2747400700 1 --2- 172.21.15.19:0/4054990381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55802dd066a0 0x55802dd06ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:54.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.434+0000 7f2747400700 1 -- 172.21.15.19:0/4054990381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55802dd06ff0 msgr2=0x55802dd131d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:54.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.434+0000 7f2747400700 1 --2- 172.21.15.19:0/4054990381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55802dd06ff0 0x55802dd131d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:54.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.434+0000 7f2747400700 1 -- 172.21.15.19:0/4054990381 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55802db75950 con 0x55802dd0c4f0 2024-05-07T02:33:54.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.434+0000 7f2747e00700 1 --2- 172.21.15.19:0/4054990381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55802dd06ff0 0x55802dd131d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:33:54.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.435+0000 7f2747400700 1 --2- 172.21.15.19:0/4054990381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55802dd0c4f0 0x55802dd0c900 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2738005bb0 tx=0x7f2738009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbf95cc37090af9c server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:54.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.435+0000 7f273ea00700 1 --2- 172.21.15.19:0/3008844542 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f272c0b2a70 0x7f272c0b4f80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2730009680 tx=0x7f2730009b60 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:54.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.435+0000 7f273cc00700 1 -- 172.21.15.19:0/4054990381 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27380092e0 con 0x55802dd0c4f0 2024-05-07T02:33:54.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.435+0000 7f273cc00700 1 -- 172.21.15.19:0/4054990381 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2738020070 con 0x55802dd0c4f0 2024-05-07T02:33:54.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.435+0000 7f273cc00700 1 -- 172.21.15.19:0/4054990381 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2738005290 con 0x55802dd0c4f0 2024-05-07T02:33:54.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.436+0000 7f275ca93b80 1 -- 172.21.15.19:0/4054990381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55802dd0c4f0 msgr2=0x55802dd0c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:54.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.436+0000 7f275ca93b80 1 --2- 172.21.15.19:0/4054990381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55802dd0c4f0 0x55802dd0c900 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2738005bb0 tx=0x7f2738009b80 comp rx=0 tx=0).stop 2024-05-07T02:33:54.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.436+0000 7f275ca93b80 1 -- 172.21.15.19:0/4054990381 shutdown_connections 2024-05-07T02:33:54.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.436+0000 7f275ca93b80 1 --2- 172.21.15.19:0/4054990381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55802dd06ff0 0x55802dd131d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:54.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.436+0000 7f275ca93b80 1 --2- 172.21.15.19:0/4054990381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55802dd0c4f0 0x55802dd0c900 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:54.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.436+0000 7f275ca93b80 1 --2- 172.21.15.19:0/4054990381 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55802dd066a0 0x55802dd06ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:54.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.436+0000 7f275ca93b80 1 -- 172.21.15.19:0/4054990381 >> 172.21.15.19:0/4054990381 conn(0x55802db88ec0 msgr2=0x55802dd01400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:54.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.436+0000 7f275ca93b80 1 -- 172.21.15.19:0/4054990381 shutdown_connections 2024-05-07T02:33:54.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.437+0000 7f275ca93b80 1 -- 172.21.15.19:0/4054990381 wait complete. 2024-05-07T02:33:54.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.437+0000 7f275ca93b80 1 -- start start 2024-05-07T02:33:54.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.437+0000 7f275ca93b80 1 Processor -- start 2024-05-07T02:33:54.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.438+0000 7f275ca93b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55802dd066a0 0x55802dd10660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:54.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.438+0000 7f275ca93b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55802dd06ff0 0x55802dd10ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:54.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.438+0000 7f275ca93b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55802dd0c4f0 0x55802dd110e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:54.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.438+0000 7f275ca93b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55802dd13860 con 0x55802dd066a0 2024-05-07T02:33:54.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.438+0000 7f275ca93b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55802db8eba0 con 0x55802dd0c4f0 2024-05-07T02:33:54.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.438+0000 7f275ca93b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55802dd139e0 con 0x55802dd06ff0 2024-05-07T02:33:54.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.438+0000 7f275ca93b80 10 client.0 _ll_register_callbacks cb 0x55802dbbc400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:33:54.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.438+0000 7f2747400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55802dd066a0 0x55802dd10660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:54.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.438+0000 7f2747400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55802dd066a0 0x55802dd10660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48166/0 (socket says 172.21.15.19:48166) 2024-05-07T02:33:54.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.438+0000 7f2747400700 1 -- 172.21.15.19:0/2486463760 learned_addr learned my addr 172.21.15.19:0/2486463760 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:54.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.439+0000 7f273ea00700 1 --2- 172.21.15.19:0/2486463760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55802dd06ff0 0x55802dd10ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:54.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.439+0000 7f2747e00700 1 --2- 172.21.15.19:0/2486463760 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55802dd0c4f0 0x55802dd110e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:54.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.439+0000 7f2747400700 1 -- 172.21.15.19:0/2486463760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55802dd06ff0 msgr2=0x55802dd10ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:54.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.439+0000 7f2747400700 1 --2- 172.21.15.19:0/2486463760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55802dd06ff0 0x55802dd10ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:54.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.439+0000 7f2747400700 1 -- 172.21.15.19:0/2486463760 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55802dd0c4f0 msgr2=0x55802dd110e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:54.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.439+0000 7f2747400700 1 --2- 172.21.15.19:0/2486463760 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55802dd0c4f0 0x55802dd110e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:54.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.439+0000 7f2747400700 1 -- 172.21.15.19:0/2486463760 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55802dbee4a0 con 0x55802dd066a0 2024-05-07T02:33:54.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.439+0000 7f273ea00700 1 --2- 172.21.15.19:0/2486463760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55802dd06ff0 0x55802dd10ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:33:54.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.439+0000 7f2747e00700 1 --2- 172.21.15.19:0/2486463760 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55802dd0c4f0 0x55802dd110e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:33:54.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.439+0000 7f2747400700 1 --2- 172.21.15.19:0/2486463760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55802dd066a0 0x55802dd10660 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2738000f80 tx=0x7f2738003a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5605f29568dbfe8d server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:54.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.439+0000 7f2735600700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-07T02:33:54.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.440+0000 7f2735600700 1 -- 172.21.15.19:0/2486463760 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273800bcb0 con 0x55802dd066a0 2024-05-07T02:33:54.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.440+0000 7f2735600700 1 -- 172.21.15.19:0/2486463760 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f273800f040 con 0x55802dd066a0 2024-05-07T02:33:54.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.440+0000 7f275ca93b80 1 -- 172.21.15.19:0/2486463760 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2738020070 con 0x55802dd066a0 2024-05-07T02:33:54.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.440+0000 7f2735600700 1 -- 172.21.15.19:0/2486463760 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2738035050 con 0x55802dd066a0 2024-05-07T02:33:54.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.440+0000 7f275ca93b80 10 client.15537 fetch_fsmap learned FSMap version 34 2024-05-07T02:33:54.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.440+0000 7f275ca93b80 1 -- 172.21.15.19:0/2486463760 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55802dd22820 con 0x55802dd066a0 2024-05-07T02:33:54.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.440+0000 7f2735600700 1 -- 172.21.15.19:0/2486463760 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2738020070 con 0x55802dd066a0 2024-05-07T02:33:54.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.441+0000 7f2735600700 1 -- 172.21.15.19:0/2486463760 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f2738011070 con 0x55802dd066a0 2024-05-07T02:33:54.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.441+0000 7f275ca93b80 10 client.15537 fetch_fsmap finished waiting for FSMap version 34 2024-05-07T02:33:54.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.441+0000 7f275ca93b80 10 client.15537 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:33:54.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.441+0000 7f275ca93b80 20 client.15537 populate_metadata read hostname 'smithi019' 2024-05-07T02:33:54.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.441+0000 7f275ca93b80 1 --2- 172.21.15.19:0/2486463760 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55802dd24cf0 0x55802dd27110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:54.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.441+0000 7f275ca93b80 4 client.15537 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:33:54.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.441+0000 7f275ca93b80 1 -- 172.21.15.19:0/2486463760 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55802dcd3160 con 0x55802dd24cf0 2024-05-07T02:33:54.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.441+0000 7f273ea00700 1 --2- 172.21.15.19:0/2486463760 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55802dd24cf0 0x55802dd27110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:54.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.442+0000 7f273ea00700 1 --2- 172.21.15.19:0/2486463760 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55802dd24cf0 0x55802dd27110 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aec9e0eb5476470e server_cookie=dd1f18bad4a84a5c in_seq=0 out_seq=0 2024-05-07T02:33:54.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.442+0000 7f2735600700 10 client.15537 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:33:54.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.443+0000 7f2735600700 1 -- 172.21.15.19:0/2486463760 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55802dcd3160 con 0x55802dd24cf0 2024-05-07T02:33:54.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.443+0000 7f2735600700 10 client.15537 handle_command_reply: tid=0 2024-05-07T02:33:54.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.443+0000 7f275ca93b80 1 client.15537 shutdown 2024-05-07T02:33:54.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.443+0000 7f275ca93b80 1 -- 172.21.15.19:0/2486463760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55802dd066a0 msgr2=0x55802dd10660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:54.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.443+0000 7f275ca93b80 1 --2- 172.21.15.19:0/2486463760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55802dd066a0 0x55802dd10660 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2738000f80 tx=0x7f2738003a00 comp rx=0 tx=0).stop 2024-05-07T02:33:54.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.443+0000 7f275ca93b80 1 -- 172.21.15.19:0/2486463760 shutdown_connections 2024-05-07T02:33:54.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.443+0000 7f275ca93b80 1 --2- 172.21.15.19:0/2486463760 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55802dd24cf0 0x55802dd27110 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:54.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.443+0000 7f275ca93b80 1 --2- 172.21.15.19:0/2486463760 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55802dd0c4f0 0x55802dd110e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:54.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.443+0000 7f275ca93b80 1 --2- 172.21.15.19:0/2486463760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55802dd066a0 0x55802dd10660 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:54.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.443+0000 7f275ca93b80 1 --2- 172.21.15.19:0/2486463760 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55802dd06ff0 0x55802dd10ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:54.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.443+0000 7f275ca93b80 1 -- 172.21.15.19:0/2486463760 >> 172.21.15.19:0/2486463760 conn(0x55802db88ec0 msgr2=0x55802dd05690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:54.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.443+0000 7f2735600700 0 client.15537 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:33:54.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.443+0000 7f275ca93b80 1 -- 172.21.15.19:0/2486463760 shutdown_connections 2024-05-07T02:33:54.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.443+0000 7f275ca93b80 1 -- 172.21.15.19:0/2486463760 wait complete. 2024-05-07T02:33:54.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.443+0000 7f275ca93b80 20 client.15537 trim_cache size 0 max 16384 2024-05-07T02:33:54.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.467+0000 7f275ca93b80 1 Processor -- start 2024-05-07T02:33:54.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.467+0000 7f275ca93b80 1 -- start start 2024-05-07T02:33:54.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.467+0000 7f275ca93b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55802dd066a0 0x55802dd21110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:54.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.467+0000 7f275ca93b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55802dd06ff0 0x55802dd12740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:54.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.467+0000 7f275ca93b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55802dd0c4f0 0x55802dd10f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:54.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.467+0000 7f275ca93b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f27481ae3b0 con 0x55802dd06ff0 2024-05-07T02:33:54.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.467+0000 7f275ca93b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f27481affb0 con 0x55802dd066a0 2024-05-07T02:33:54.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.467+0000 7f275ca93b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f27481aea80 con 0x55802dd0c4f0 2024-05-07T02:33:54.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.468+0000 7f273ea00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55802dd06ff0 0x55802dd12740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:54.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.468+0000 7f2747400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55802dd066a0 0x55802dd21110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:54.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.468+0000 7f273ea00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55802dd06ff0 0x55802dd12740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48182/0 (socket says 172.21.15.19:48182) 2024-05-07T02:33:54.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.468+0000 7f273ea00700 1 -- 172.21.15.19:0/3079083326 learned_addr learned my addr 172.21.15.19:0/3079083326 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:54.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.468+0000 7f2747e00700 1 --2- 172.21.15.19:0/3079083326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55802dd0c4f0 0x55802dd10f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:54.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.468+0000 7f2747400700 1 -- 172.21.15.19:0/3079083326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55802dd0c4f0 msgr2=0x55802dd10f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:54.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.468+0000 7f2747400700 1 --2- 172.21.15.19:0/3079083326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55802dd0c4f0 0x55802dd10f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:54.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.468+0000 7f2747400700 1 -- 172.21.15.19:0/3079083326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55802dd06ff0 msgr2=0x55802dd12740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:54.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.468+0000 7f2747400700 1 --2- 172.21.15.19:0/3079083326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55802dd06ff0 0x55802dd12740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:54.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.468+0000 7f2747400700 1 -- 172.21.15.19:0/3079083326 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55802dd02ab0 con 0x55802dd066a0 2024-05-07T02:33:54.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.468+0000 7f273ea00700 1 --2- 172.21.15.19:0/3079083326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55802dd06ff0 0x55802dd12740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:33:54.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.468+0000 7f2747400700 1 --2- 172.21.15.19:0/3079083326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55802dd066a0 0x55802dd21110 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2738009e60 tx=0x7f2738007ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=199d71bc78b6b1ea server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:54.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.469+0000 7f2737e00700 1 -- 172.21.15.19:0/3079083326 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2738003a70 con 0x55802dd066a0 2024-05-07T02:33:54.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.469+0000 7f2737e00700 1 -- 172.21.15.19:0/3079083326 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2738020070 con 0x55802dd066a0 2024-05-07T02:33:54.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.469+0000 7f2737e00700 1 -- 172.21.15.19:0/3079083326 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2738005800 con 0x55802dd066a0 2024-05-07T02:33:54.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.469+0000 7f275ca93b80 1 -- 172.21.15.19:0/3079083326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55802dd066a0 msgr2=0x55802dd21110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:54.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.469+0000 7f275ca93b80 1 --2- 172.21.15.19:0/3079083326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55802dd066a0 0x55802dd21110 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2738009e60 tx=0x7f2738007ca0 comp rx=0 tx=0).stop 2024-05-07T02:33:54.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.470+0000 7f275ca93b80 1 -- 172.21.15.19:0/3079083326 shutdown_connections 2024-05-07T02:33:54.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.470+0000 7f275ca93b80 1 --2- 172.21.15.19:0/3079083326 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55802dd0c4f0 0x55802dd10f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:54.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.470+0000 7f275ca93b80 1 --2- 172.21.15.19:0/3079083326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55802dd066a0 0x55802dd21110 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:54.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.470+0000 7f275ca93b80 1 --2- 172.21.15.19:0/3079083326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55802dd06ff0 0x55802dd12740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:54.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.470+0000 7f275ca93b80 1 -- 172.21.15.19:0/3079083326 >> 172.21.15.19:0/3079083326 conn(0x55802db88ec0 msgr2=0x55802dcfb8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:54.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.470+0000 7f275ca93b80 1 -- 172.21.15.19:0/3079083326 shutdown_connections 2024-05-07T02:33:54.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.470+0000 7f275ca93b80 1 -- 172.21.15.19:0/3079083326 wait complete. 2024-05-07T02:33:54.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.470+0000 7f275ca93b80 1 -- start start 2024-05-07T02:33:54.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.470+0000 7f275ca93b80 1 Processor -- start 2024-05-07T02:33:54.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.471+0000 7f275ca93b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55802dd066a0 0x55802dd280a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:54.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.471+0000 7f275ca93b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55802dd06ff0 0x55802dd285e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:54.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.471+0000 7f275ca93b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55802dd0c4f0 0x55802dd8e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:54.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.471+0000 7f275ca93b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f27481af820 con 0x55802dd0c4f0 2024-05-07T02:33:54.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.471+0000 7f275ca93b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f27481af150 con 0x55802dd06ff0 2024-05-07T02:33:54.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.471+0000 7f275ca93b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f27481ac370 con 0x55802dd066a0 2024-05-07T02:33:54.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.471+0000 7f275ca93b80 10 client.0 _ll_register_callbacks cb 0x55802dbbc400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:33:54.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.471+0000 7f2747e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55802dd0c4f0 0x55802dd8e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:54.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.471+0000 7f2747e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55802dd0c4f0 0x55802dd8e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48190/0 (socket says 172.21.15.19:48190) 2024-05-07T02:33:54.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.471+0000 7f2747e00700 1 -- 172.21.15.19:0/206961349 learned_addr learned my addr 172.21.15.19:0/206961349 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:54.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.471+0000 7f273ea00700 1 --2- 172.21.15.19:0/206961349 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55802dd06ff0 0x55802dd285e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:54.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.471+0000 7f2747400700 1 --2- 172.21.15.19:0/206961349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55802dd066a0 0x55802dd280a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:54.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.472+0000 7f2747e00700 1 -- 172.21.15.19:0/206961349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55802dd066a0 msgr2=0x55802dd280a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:54.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.472+0000 7f2747e00700 1 --2- 172.21.15.19:0/206961349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55802dd066a0 0x55802dd280a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:54.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.472+0000 7f2747e00700 1 -- 172.21.15.19:0/206961349 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55802dd06ff0 msgr2=0x55802dd285e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:54.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.472+0000 7f2747e00700 1 --2- 172.21.15.19:0/206961349 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55802dd06ff0 0x55802dd285e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:54.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.472+0000 7f2747e00700 1 -- 172.21.15.19:0/206961349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55802dd8e780 con 0x55802dd0c4f0 2024-05-07T02:33:54.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.472+0000 7f2747e00700 1 --2- 172.21.15.19:0/206961349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55802dd0c4f0 0x55802dd8e240 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2740004800 tx=0x7f274000ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=954d2f12a7687789 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:54.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.472+0000 7f2735600700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-07T02:33:54.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.472+0000 7f2735600700 1 -- 172.21.15.19:0/206961349 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2740005310 con 0x55802dd0c4f0 2024-05-07T02:33:54.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.472+0000 7f2735600700 1 -- 172.21.15.19:0/206961349 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f27400a9be0 con 0x55802dd0c4f0 2024-05-07T02:33:54.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.472+0000 7f275ca93b80 1 -- 172.21.15.19:0/206961349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2738020070 con 0x55802dd0c4f0 2024-05-07T02:33:54.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.473+0000 7f2735600700 1 -- 172.21.15.19:0/206961349 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f27400b2120 con 0x55802dd0c4f0 2024-05-07T02:33:54.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.473+0000 7f275ca93b80 10 client.15543 fetch_fsmap learned FSMap version 34 2024-05-07T02:33:54.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.473+0000 7f275ca93b80 1 -- 172.21.15.19:0/206961349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55802dd8fa60 con 0x55802dd0c4f0 2024-05-07T02:33:54.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.473+0000 7f2735600700 1 -- 172.21.15.19:0/206961349 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2738020070 con 0x55802dd0c4f0 2024-05-07T02:33:54.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.473+0000 7f2735600700 1 -- 172.21.15.19:0/206961349 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f27400b2350 con 0x55802dd0c4f0 2024-05-07T02:33:54.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.473+0000 7f275ca93b80 10 client.15543 fetch_fsmap finished waiting for FSMap version 34 2024-05-07T02:33:54.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.473+0000 7f275ca93b80 10 client.15543 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:33:54.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.473+0000 7f275ca93b80 20 client.15543 populate_metadata read hostname 'smithi019' 2024-05-07T02:33:54.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.473+0000 7f275ca93b80 1 --2- 172.21.15.19:0/206961349 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55802dd90220 0x55802dd92640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:54.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.473+0000 7f275ca93b80 4 client.15543 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-07T02:33:54.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.473+0000 7f275ca93b80 1 -- 172.21.15.19:0/206961349 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55802dc36480 con 0x55802dd90220 2024-05-07T02:33:54.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.474+0000 7f2747400700 1 --2- 172.21.15.19:0/206961349 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55802dd90220 0x55802dd92640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:54.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.474+0000 7f2747400700 1 --2- 172.21.15.19:0/206961349 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55802dd90220 0x55802dd92640 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71e59daf6c12204e server_cookie=2c519a5e119569ee in_seq=0 out_seq=0 2024-05-07T02:33:54.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.474+0000 7f2735600700 10 client.15543 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:33:54.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.475+0000 7f2735600700 1 -- 172.21.15.19:0/206961349 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55802dc36480 con 0x55802dd90220 2024-05-07T02:33:54.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.475+0000 7f2735600700 10 client.15543 handle_command_reply: tid=0 2024-05-07T02:33:54.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.476+0000 7f275ca93b80 1 client.15543 shutdown 2024-05-07T02:33:54.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.476+0000 7f275ca93b80 1 -- 172.21.15.19:0/206961349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55802dd0c4f0 msgr2=0x55802dd8e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:54.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.476+0000 7f275ca93b80 1 --2- 172.21.15.19:0/206961349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55802dd0c4f0 0x55802dd8e240 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2740004800 tx=0x7f274000ce90 comp rx=0 tx=0).stop 2024-05-07T02:33:54.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.476+0000 7f275ca93b80 1 -- 172.21.15.19:0/206961349 shutdown_connections 2024-05-07T02:33:54.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.476+0000 7f275ca93b80 1 --2- 172.21.15.19:0/206961349 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55802dd90220 0x55802dd92640 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:54.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.476+0000 7f275ca93b80 1 --2- 172.21.15.19:0/206961349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55802dd0c4f0 0x55802dd8e240 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:54.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.476+0000 7f275ca93b80 1 --2- 172.21.15.19:0/206961349 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55802dd066a0 0x55802dd280a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:54.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.476+0000 7f275ca93b80 1 --2- 172.21.15.19:0/206961349 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55802dd06ff0 0x55802dd285e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:54.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.476+0000 7f275ca93b80 1 -- 172.21.15.19:0/206961349 >> 172.21.15.19:0/206961349 conn(0x55802db88ec0 msgr2=0x55802dd123e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:54.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.476+0000 7f2735600700 0 client.15543 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:33:54.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.476+0000 7f275ca93b80 1 -- 172.21.15.19:0/206961349 shutdown_connections 2024-05-07T02:33:54.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.476+0000 7f275ca93b80 1 -- 172.21.15.19:0/206961349 wait complete. 2024-05-07T02:33:54.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.476+0000 7f275ca93b80 20 client.15543 trim_cache size 0 max 16384 2024-05-07T02:33:54.478 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:33:54.478 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-07T02:33:54.478 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "ddc682a5-c92f-4cb1-b53a-4ce3a024c1e3", 2024-05-07T02:33:54.478 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-07T02:33:54.479 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:33:54.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.477+0000 7f274e000700 1 -- 172.21.15.19:0/3008844542 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f272c0b2a70 msgr2=0x7f272c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:54.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.477+0000 7f274e000700 1 --2- 172.21.15.19:0/3008844542 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f272c0b2a70 0x7f272c0b4f80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2730009680 tx=0x7f2730009b60 comp rx=0 tx=0).stop 2024-05-07T02:33:54.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.477+0000 7f274e000700 1 -- 172.21.15.19:0/3008844542 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2748109870 msgr2=0x7f274819f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:54.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.477+0000 7f274e000700 1 --2- 172.21.15.19:0/3008844542 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2748109870 0x7f274819f6a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f274000e940 tx=0x7f274000ec50 comp rx=0 tx=0).stop 2024-05-07T02:33:54.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.477+0000 7f274e000700 1 -- 172.21.15.19:0/3008844542 shutdown_connections 2024-05-07T02:33:54.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.477+0000 7f274e000700 1 --2- 172.21.15.19:0/3008844542 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f272c0b2a70 0x7f272c0b4f80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:54.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.477+0000 7f274e000700 1 --2- 172.21.15.19:0/3008844542 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2748109870 0x7f274819f6a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:54.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.477+0000 7f274e000700 1 --2- 172.21.15.19:0/3008844542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f27481045f0 0x7f274819ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:54.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.477+0000 7f274e000700 1 --2- 172.21.15.19:0/3008844542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2748108f20 0x7f274819b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:54.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.477+0000 7f274e000700 1 -- 172.21.15.19:0/3008844542 >> 172.21.15.19:0/3008844542 conn(0x7f2748100040 msgr2=0x7f2748108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:54.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.478+0000 7f274e000700 1 -- 172.21.15.19:0/3008844542 shutdown_connections 2024-05-07T02:33:54.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:54.478+0000 7f274e000700 1 -- 172.21.15.19:0/3008844542 wait complete. 2024-05-07T02:33:54.726 INFO:tasks.workunit.client.0.smithi019.stdout: 1 35667 4.22 MB/sec warmup 50 sec latency 3922.201 ms 2024-05-07T02:33:55.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:54 smithi019 ceph-mon[29781]: pgmap v65: 129 pgs: 129 active+clean; 34 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 117 KiB/s wr, 35 op/s 2024-05-07T02:33:55.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:54 smithi019 ceph-mon[29781]: scrub queued for path: / 2024-05-07T02:33:55.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:54 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:33:55.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:54 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:33:55.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:54 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:33:55.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:54 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:33:55.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:54 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:33:55.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:54 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:33:55.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:54 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:33:55.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:54 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:33:55.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:54 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:33:55.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:54 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:33:55.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:54 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:33:55.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:54 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:33:55.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:54 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:33:55.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:54 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:33:55.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:54 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:33:55.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:54 smithi064 ceph-mon[34699]: pgmap v65: 129 pgs: 129 active+clean; 34 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 117 KiB/s wr, 35 op/s 2024-05-07T02:33:55.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:54 smithi064 ceph-mon[34699]: scrub queued for path: / 2024-05-07T02:33:55.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:54 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:33:55.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:54 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:33:55.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:54 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:33:55.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:54 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:33:55.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:54 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:33:55.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:54 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:33:55.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:54 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:33:55.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:54 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:33:55.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:54 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:33:55.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:54 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:33:55.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:54 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:33:55.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:54 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:33:55.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:54 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:33:55.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:54 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:33:55.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:54 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:33:55.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:54 smithi027 ceph-mon[34398]: pgmap v65: 129 pgs: 129 active+clean; 34 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 117 KiB/s wr, 35 op/s 2024-05-07T02:33:55.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:54 smithi027 ceph-mon[34398]: scrub queued for path: / 2024-05-07T02:33:55.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:54 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:33:55.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:54 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:33:55.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:54 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:33:55.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:54 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:33:55.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:54 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:33:55.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:54 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:33:55.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:54 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:33:55.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:54 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:33:55.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:54 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:33:55.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:54 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:33:55.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:54 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:33:55.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:54 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:33:55.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:54 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:33:55.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:54 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:33:55.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:54 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:33:55.485 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-07T02:33:55.726 INFO:tasks.workunit.client.0.smithi019.stdout: 1 35772 4.15 MB/sec warmup 51 sec latency 954.465 ms 2024-05-07T02:33:56.022 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:33:56.726 INFO:tasks.workunit.client.0.smithi019.stdout: 1 36174 4.08 MB/sec warmup 52 sec latency 287.810 ms 2024-05-07T02:33:57.228 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:56 smithi019 ceph-mon[29781]: pgmap v66: 129 pgs: 129 active+clean; 35 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 135 KiB/s wr, 46 op/s 2024-05-07T02:33:57.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:56 smithi064 ceph-mon[34699]: pgmap v66: 129 pgs: 129 active+clean; 35 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 135 KiB/s wr, 46 op/s 2024-05-07T02:33:57.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:56 smithi027 ceph-mon[34398]: pgmap v66: 129 pgs: 129 active+clean; 35 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 135 KiB/s wr, 46 op/s 2024-05-07T02:33:57.726 INFO:tasks.workunit.client.0.smithi019.stdout: 1 36174 4.01 MB/sec warmup 53 sec latency 1287.896 ms 2024-05-07T02:33:58.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.592+0000 7fa876000700 1 -- 172.21.15.19:0/2333826482 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa87010e900 msgr2=0x7fa870110dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:58.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.592+0000 7fa876000700 1 --2- 172.21.15.19:0/2333826482 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa87010e900 0x7fa870110dc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa868009f90 tx=0x7fa86800b330 comp rx=0 tx=0).stop 2024-05-07T02:33:58.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.592+0000 7fa876000700 1 -- 172.21.15.19:0/2333826482 shutdown_connections 2024-05-07T02:33:58.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.592+0000 7fa876000700 1 --2- 172.21.15.19:0/2333826482 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa87010e900 0x7fa870110dc0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:58.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.592+0000 7fa876000700 1 --2- 172.21.15.19:0/2333826482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa870105520 0x7fa870105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:58.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.592+0000 7fa876000700 1 --2- 172.21.15.19:0/2333826482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa870105e70 0x7fa87010e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:58.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.592+0000 7fa876000700 1 -- 172.21.15.19:0/2333826482 >> 172.21.15.19:0/2333826482 conn(0x7fa8700fedc0 msgr2=0x7fa8701011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:58.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.592+0000 7fa876000700 1 -- 172.21.15.19:0/2333826482 shutdown_connections 2024-05-07T02:33:58.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.592+0000 7fa876000700 1 -- 172.21.15.19:0/2333826482 wait complete. 2024-05-07T02:33:58.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.593+0000 7fa876000700 1 Processor -- start 2024-05-07T02:33:58.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.594+0000 7fa876000700 1 -- start start 2024-05-07T02:33:58.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.594+0000 7fa876000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa870105520 0x7fa87019ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:58.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.594+0000 7fa876000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa870105e70 0x7fa87019b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:58.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.594+0000 7fa876000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa87010e900 0x7fa87019f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:58.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.594+0000 7fa876000700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa870114440 con 0x7fa870105e70 2024-05-07T02:33:58.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.594+0000 7fa876000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa8701142c0 con 0x7fa870105520 2024-05-07T02:33:58.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.594+0000 7fa876000700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa8701145c0 con 0x7fa87010e900 2024-05-07T02:33:58.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.594+0000 7fa86ea00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa870105e70 0x7fa87019b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:58.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.594+0000 7fa86ea00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa870105e70 0x7fa87019b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48198/0 (socket says 172.21.15.19:48198) 2024-05-07T02:33:58.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.595+0000 7fa86ea00700 1 -- 172.21.15.19:0/3365648843 learned_addr learned my addr 172.21.15.19:0/3365648843 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:58.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.595+0000 7fa86fe00700 1 --2- 172.21.15.19:0/3365648843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa87010e900 0x7fa87019f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:58.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.595+0000 7fa86f400700 1 --2- 172.21.15.19:0/3365648843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa870105520 0x7fa87019ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:58.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.595+0000 7fa86ea00700 1 -- 172.21.15.19:0/3365648843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa87010e900 msgr2=0x7fa87019f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:58.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.595+0000 7fa86ea00700 1 --2- 172.21.15.19:0/3365648843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa87010e900 0x7fa87019f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:58.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.595+0000 7fa86ea00700 1 -- 172.21.15.19:0/3365648843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa870105520 msgr2=0x7fa87019ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:58.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.595+0000 7fa86ea00700 1 --2- 172.21.15.19:0/3365648843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa870105520 0x7fa87019ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:58.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.595+0000 7fa86ea00700 1 -- 172.21.15.19:0/3365648843 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa87019fc00 con 0x7fa870105e70 2024-05-07T02:33:58.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.595+0000 7fa86fe00700 1 --2- 172.21.15.19:0/3365648843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa87010e900 0x7fa87019f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:33:58.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.596+0000 7fa86ea00700 1 --2- 172.21.15.19:0/3365648843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa870105e70 0x7fa87019b2d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa85c00ec10 tx=0x7fa85c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:58.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.596+0000 7fa867e00700 1 -- 172.21.15.19:0/3365648843 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa85c00cc90 con 0x7fa870105e70 2024-05-07T02:33:58.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.596+0000 7fa867e00700 1 -- 172.21.15.19:0/3365648843 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa85c00ce30 con 0x7fa870105e70 2024-05-07T02:33:58.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.596+0000 7fa876000700 1 -- 172.21.15.19:0/3365648843 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa87019fef0 con 0x7fa870105e70 2024-05-07T02:33:58.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.596+0000 7fa867e00700 1 -- 172.21.15.19:0/3365648843 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa85c010430 con 0x7fa870105e70 2024-05-07T02:33:58.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.596+0000 7fa876000700 1 -- 172.21.15.19:0/3365648843 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8701a7810 con 0x7fa870105e70 2024-05-07T02:33:58.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.600+0000 7fa867e00700 1 -- 172.21.15.19:0/3365648843 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa85c0105d0 con 0x7fa870105e70 2024-05-07T02:33:58.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.601+0000 7fa884b4db80 1 Processor -- start 2024-05-07T02:33:58.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.601+0000 7fa884b4db80 1 -- start start 2024-05-07T02:33:58.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.602+0000 7fa884b4db80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598656ae830 0x5598656aec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:58.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.602+0000 7fa884b4db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598656a8a30 0x5598656a8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:58.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.602+0000 7fa884b4db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598656a9380 0x5598656b55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:58.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.602+0000 7fa884b4db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5598655c6a20 con 0x5598656a9380 2024-05-07T02:33:58.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.602+0000 7fa884b4db80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559865564760 con 0x5598656ae830 2024-05-07T02:33:58.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.602+0000 7fa884b4db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5598655c29e0 con 0x5598656a8a30 2024-05-07T02:33:58.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.602+0000 7fa86fe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598656ae830 0x5598656aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:58.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.602+0000 7fa86fe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598656ae830 0x5598656aec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:32866/0 (socket says 172.21.15.19:32866) 2024-05-07T02:33:58.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.602+0000 7fa86fe00700 1 -- 172.21.15.19:0/3693955677 learned_addr learned my addr 172.21.15.19:0/3693955677 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:58.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.602+0000 7fa86f400700 1 --2- 172.21.15.19:0/3693955677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598656a8a30 0x5598656a8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:58.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.602+0000 7fa86ea00700 1 --2- 172.21.15.19:0/3693955677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598656a9380 0x5598656b55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:58.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.603+0000 7fa867e00700 1 --2- 172.21.15.19:0/3365648843 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fa8540b29f0 0x7fa8540b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:58.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.603+0000 7fa86fe00700 1 -- 172.21.15.19:0/3693955677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598656a8a30 msgr2=0x5598656a8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:58.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.603+0000 7fa86fe00700 1 --2- 172.21.15.19:0/3693955677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598656a8a30 0x5598656a8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:58.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.603+0000 7fa86fe00700 1 -- 172.21.15.19:0/3693955677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598656a9380 msgr2=0x5598656b55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:58.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.603+0000 7fa86fe00700 1 --2- 172.21.15.19:0/3693955677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598656a9380 0x5598656b55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:58.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.603+0000 7fa86fe00700 1 -- 172.21.15.19:0/3693955677 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598655138d0 con 0x5598656ae830 2024-05-07T02:33:58.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.603+0000 7fa867e00700 1 -- 172.21.15.19:0/3365648843 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fa85c0e2c60 con 0x7fa870105e70 2024-05-07T02:33:58.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.603+0000 7fa86fe00700 1 --2- 172.21.15.19:0/3365648843 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fa8540b29f0 0x7fa8540b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:58.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.603+0000 7fa86fe00700 1 --2- 172.21.15.19:0/3693955677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598656ae830 0x5598656aec40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa868004f70 tx=0x7fa868009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f9f3cc26020e2e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:58.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.604+0000 7fa864c00700 1 -- 172.21.15.19:0/3693955677 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86800bdf0 con 0x5598656ae830 2024-05-07T02:33:58.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.604+0000 7fa864c00700 1 -- 172.21.15.19:0/3693955677 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8680079b0 con 0x5598656ae830 2024-05-07T02:33:58.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.604+0000 7fa864c00700 1 -- 172.21.15.19:0/3693955677 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8680055a0 con 0x5598656ae830 2024-05-07T02:33:58.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.604+0000 7fa86fe00700 1 --2- 172.21.15.19:0/3365648843 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fa8540b29f0 0x7fa8540b4f00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa868029000 tx=0x7fa86802c000 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:58.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.605+0000 7fa884b4db80 1 -- 172.21.15.19:0/3693955677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598656ae830 msgr2=0x5598656aec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:58.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.605+0000 7fa884b4db80 1 --2- 172.21.15.19:0/3693955677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598656ae830 0x5598656aec40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa868004f70 tx=0x7fa868009b60 comp rx=0 tx=0).stop 2024-05-07T02:33:58.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.605+0000 7fa884b4db80 1 -- 172.21.15.19:0/3693955677 shutdown_connections 2024-05-07T02:33:58.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.605+0000 7fa884b4db80 1 --2- 172.21.15.19:0/3693955677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598656a9380 0x5598656b55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:58.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.605+0000 7fa884b4db80 1 --2- 172.21.15.19:0/3693955677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598656ae830 0x5598656aec40 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:58.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.605+0000 7fa884b4db80 1 --2- 172.21.15.19:0/3693955677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598656a8a30 0x5598656a8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:58.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.605+0000 7fa884b4db80 1 -- 172.21.15.19:0/3693955677 >> 172.21.15.19:0/3693955677 conn(0x559865526e40 msgr2=0x55986569f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:58.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.605+0000 7fa884b4db80 1 -- 172.21.15.19:0/3693955677 shutdown_connections 2024-05-07T02:33:58.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.605+0000 7fa884b4db80 1 -- 172.21.15.19:0/3693955677 wait complete. 2024-05-07T02:33:58.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.606+0000 7fa884b4db80 1 -- start start 2024-05-07T02:33:58.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.606+0000 7fa884b4db80 1 Processor -- start 2024-05-07T02:33:58.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.607+0000 7fa884b4db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598656a8a30 0x5598656b2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:58.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.607+0000 7fa884b4db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598656a9380 0x5598656b2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:58.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.607+0000 7fa884b4db80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598656ae830 0x5598656b3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:58.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.607+0000 7fa884b4db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5598656b5ba0 con 0x5598656a9380 2024-05-07T02:33:58.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.607+0000 7fa884b4db80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55986552cb20 con 0x5598656ae830 2024-05-07T02:33:58.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.607+0000 7fa884b4db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5598656b5d20 con 0x5598656a8a30 2024-05-07T02:33:58.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.607+0000 7fa884b4db80 10 client.0 _ll_register_callbacks cb 0x55986555a380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:33:58.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.607+0000 7fa86f400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598656a8a30 0x5598656b2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:58.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.607+0000 7fa86ea00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598656a9380 0x5598656b2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:58.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.607+0000 7fa86f400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598656a8a30 0x5598656b2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:45752/0 (socket says 172.21.15.19:45752) 2024-05-07T02:33:58.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.607+0000 7fa86f400700 1 -- 172.21.15.19:0/342904301 learned_addr learned my addr 172.21.15.19:0/342904301 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:58.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.607+0000 7fa86fe00700 1 --2- 172.21.15.19:0/342904301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598656ae830 0x5598656b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:58.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.608+0000 7fa86f400700 1 -- 172.21.15.19:0/342904301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598656ae830 msgr2=0x5598656b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:58.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.608+0000 7fa86f400700 1 --2- 172.21.15.19:0/342904301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598656ae830 0x5598656b3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:58.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.608+0000 7fa86f400700 1 -- 172.21.15.19:0/342904301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598656a9380 msgr2=0x5598656b2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:58.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.608+0000 7fa86f400700 1 --2- 172.21.15.19:0/342904301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598656a9380 0x5598656b2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:58.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.608+0000 7fa86f400700 1 -- 172.21.15.19:0/342904301 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55986558c420 con 0x5598656a8a30 2024-05-07T02:33:58.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.608+0000 7fa86f400700 1 --2- 172.21.15.19:0/342904301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598656a8a30 0x5598656b2a00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa860011830 tx=0x7fa860011b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8b2e6cd2fd54eac server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:58.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.608+0000 7fa859600700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:33:58.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.608+0000 7fa859600700 1 -- 172.21.15.19:0/342904301 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86000f750 con 0x5598656a8a30 2024-05-07T02:33:58.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.608+0000 7fa884b4db80 1 -- 172.21.15.19:0/342904301 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8680079b0 con 0x5598656a8a30 2024-05-07T02:33:58.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.609+0000 7fa859600700 1 -- 172.21.15.19:0/342904301 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fa86000f8f0 con 0x5598656a8a30 2024-05-07T02:33:58.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.609+0000 7fa859600700 1 -- 172.21.15.19:0/342904301 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa860027050 con 0x5598656a8a30 2024-05-07T02:33:58.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.609+0000 7fa884b4db80 10 client.25693 fetch_fsmap learned FSMap version 34 2024-05-07T02:33:58.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.609+0000 7fa884b4db80 1 -- 172.21.15.19:0/342904301 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5598656c0800 con 0x5598656a8a30 2024-05-07T02:33:58.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.609+0000 7fa859600700 1 -- 172.21.15.19:0/342904301 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8680079b0 con 0x5598656a8a30 2024-05-07T02:33:58.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.609+0000 7fa859600700 1 -- 172.21.15.19:0/342904301 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7fa860022030 con 0x5598656a8a30 2024-05-07T02:33:58.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.609+0000 7fa884b4db80 10 client.25693 fetch_fsmap finished waiting for FSMap version 34 2024-05-07T02:33:58.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.609+0000 7fa884b4db80 10 client.25693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:33:58.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.610+0000 7fa884b4db80 20 client.25693 populate_metadata read hostname 'smithi019' 2024-05-07T02:33:58.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.610+0000 7fa884b4db80 1 --2- 172.21.15.19:0/342904301 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5598656c2e30 0x5598656c5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:58.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.610+0000 7fa884b4db80 4 client.25693 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:33:58.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.610+0000 7fa884b4db80 1 -- 172.21.15.19:0/342904301 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598656710e0 con 0x5598656c2e30 2024-05-07T02:33:58.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.610+0000 7fa86ea00700 1 --2- 172.21.15.19:0/342904301 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5598656c2e30 0x5598656c5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:58.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.611+0000 7fa86ea00700 1 --2- 172.21.15.19:0/342904301 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5598656c2e30 0x5598656c5300 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3a5da266c95da28 server_cookie=7695de03f7effb0a in_seq=0 out_seq=0 2024-05-07T02:33:58.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.611+0000 7fa859600700 10 client.25693 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:33:58.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.611+0000 7fa859600700 1 -- 172.21.15.19:0/342904301 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5598656710e0 con 0x5598656c2e30 2024-05-07T02:33:58.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.611+0000 7fa859600700 10 client.25693 handle_command_reply: tid=0 2024-05-07T02:33:58.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.611+0000 7fa884b4db80 1 client.25693 shutdown 2024-05-07T02:33:58.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.612+0000 7fa884b4db80 1 -- 172.21.15.19:0/342904301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598656a8a30 msgr2=0x5598656b2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:58.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.612+0000 7fa884b4db80 1 --2- 172.21.15.19:0/342904301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598656a8a30 0x5598656b2a00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa860011830 tx=0x7fa860011b40 comp rx=0 tx=0).stop 2024-05-07T02:33:58.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.612+0000 7fa884b4db80 1 -- 172.21.15.19:0/342904301 shutdown_connections 2024-05-07T02:33:58.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.612+0000 7fa884b4db80 1 --2- 172.21.15.19:0/342904301 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5598656c2e30 0x5598656c5300 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:58.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.612+0000 7fa884b4db80 1 --2- 172.21.15.19:0/342904301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598656ae830 0x5598656b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:58.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.612+0000 7fa859600700 0 client.25693 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:33:58.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.612+0000 7fa884b4db80 1 --2- 172.21.15.19:0/342904301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598656a8a30 0x5598656b2a00 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:58.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.612+0000 7fa884b4db80 1 --2- 172.21.15.19:0/342904301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598656a9380 0x5598656b2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:58.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.612+0000 7fa884b4db80 1 -- 172.21.15.19:0/342904301 >> 172.21.15.19:0/342904301 conn(0x559865526e40 msgr2=0x5598656a3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:58.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.612+0000 7fa884b4db80 1 -- 172.21.15.19:0/342904301 shutdown_connections 2024-05-07T02:33:58.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.612+0000 7fa884b4db80 1 -- 172.21.15.19:0/342904301 wait complete. 2024-05-07T02:33:58.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.612+0000 7fa884b4db80 20 client.25693 trim_cache size 0 max 16384 2024-05-07T02:33:58.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.635+0000 7fa884b4db80 1 Processor -- start 2024-05-07T02:33:58.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.635+0000 7fa884b4db80 1 -- start start 2024-05-07T02:33:58.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.636+0000 7fa884b4db80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598656a8a30 0x5598656ad190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:58.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.636+0000 7fa884b4db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598656a9380 0x5598656b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:58.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.636+0000 7fa884b4db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598656ae830 0x5598656b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:58.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.636+0000 7fa884b4db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa8701ae340 con 0x5598656ae830 2024-05-07T02:33:58.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.636+0000 7fa884b4db80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa8701aff40 con 0x5598656a8a30 2024-05-07T02:33:58.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.636+0000 7fa884b4db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa8701aea10 con 0x5598656a9380 2024-05-07T02:33:58.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.636+0000 7fa86f400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598656a8a30 0x5598656ad190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:58.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.636+0000 7fa86f400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598656a8a30 0x5598656ad190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:32882/0 (socket says 172.21.15.19:32882) 2024-05-07T02:33:58.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.636+0000 7fa86f400700 1 -- 172.21.15.19:0/2326932396 learned_addr learned my addr 172.21.15.19:0/2326932396 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:58.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.636+0000 7fa86fe00700 1 --2- 172.21.15.19:0/2326932396 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598656ae830 0x5598656b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:58.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.636+0000 7fa86ea00700 1 --2- 172.21.15.19:0/2326932396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598656a9380 0x5598656b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:58.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.637+0000 7fa86f400700 1 -- 172.21.15.19:0/2326932396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598656a9380 msgr2=0x5598656b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:58.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.637+0000 7fa86f400700 1 --2- 172.21.15.19:0/2326932396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598656a9380 0x5598656b04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:58.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.637+0000 7fa86f400700 1 -- 172.21.15.19:0/2326932396 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598656ae830 msgr2=0x5598656b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:58.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.637+0000 7fa86f400700 1 --2- 172.21.15.19:0/2326932396 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598656ae830 0x5598656b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:58.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.637+0000 7fa86f400700 1 -- 172.21.15.19:0/2326932396 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55986569bb80 con 0x5598656a8a30 2024-05-07T02:33:58.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.637+0000 7fa86f400700 1 --2- 172.21.15.19:0/2326932396 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598656a8a30 0x5598656ad190 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa860009870 tx=0x7fa860019620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5006df6aa6516cf8 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:58.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.637+0000 7fa85be00700 1 -- 172.21.15.19:0/2326932396 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86000f750 con 0x5598656a8a30 2024-05-07T02:33:58.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.638+0000 7fa85be00700 1 -- 172.21.15.19:0/2326932396 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa86000fda0 con 0x5598656a8a30 2024-05-07T02:33:58.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.638+0000 7fa85be00700 1 -- 172.21.15.19:0/2326932396 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8600052c0 con 0x5598656a8a30 2024-05-07T02:33:58.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.638+0000 7fa884b4db80 1 -- 172.21.15.19:0/2326932396 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598656a8a30 msgr2=0x5598656ad190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:58.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.638+0000 7fa884b4db80 1 --2- 172.21.15.19:0/2326932396 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598656a8a30 0x5598656ad190 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa860009870 tx=0x7fa860019620 comp rx=0 tx=0).stop 2024-05-07T02:33:58.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.639+0000 7fa884b4db80 1 -- 172.21.15.19:0/2326932396 shutdown_connections 2024-05-07T02:33:58.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.639+0000 7fa884b4db80 1 --2- 172.21.15.19:0/2326932396 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598656ae830 0x5598656b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:58.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.639+0000 7fa884b4db80 1 --2- 172.21.15.19:0/2326932396 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598656a8a30 0x5598656ad190 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:58.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.639+0000 7fa884b4db80 1 --2- 172.21.15.19:0/2326932396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598656a9380 0x5598656b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:58.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.639+0000 7fa884b4db80 1 -- 172.21.15.19:0/2326932396 >> 172.21.15.19:0/2326932396 conn(0x559865526e40 msgr2=0x5598656991c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:58.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.639+0000 7fa884b4db80 1 -- 172.21.15.19:0/2326932396 shutdown_connections 2024-05-07T02:33:58.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.639+0000 7fa884b4db80 1 -- 172.21.15.19:0/2326932396 wait complete. 2024-05-07T02:33:58.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.639+0000 7fa884b4db80 1 -- start start 2024-05-07T02:33:58.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.640+0000 7fa884b4db80 1 Processor -- start 2024-05-07T02:33:58.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.640+0000 7fa884b4db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598656a8a30 0x5598656bb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:58.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.640+0000 7fa884b4db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598656a9380 0x5598656ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:58.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.640+0000 7fa884b4db80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598656ae830 0x5598656ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:58.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.640+0000 7fa884b4db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa8701af7b0 con 0x5598656a9380 2024-05-07T02:33:58.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.640+0000 7fa884b4db80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa8701af0e0 con 0x5598656ae830 2024-05-07T02:33:58.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.640+0000 7fa884b4db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa8701ac2e0 con 0x5598656a8a30 2024-05-07T02:33:58.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.640+0000 7fa884b4db80 10 client.0 _ll_register_callbacks cb 0x55986555a380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:33:58.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.641+0000 7fa86f400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598656a8a30 0x5598656bb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:58.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.641+0000 7fa86ea00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598656a9380 0x5598656ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:58.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.641+0000 7fa86f400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598656a8a30 0x5598656bb680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:45772/0 (socket says 172.21.15.19:45772) 2024-05-07T02:33:58.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.641+0000 7fa86ea00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598656a9380 0x5598656ca0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48218/0 (socket says 172.21.15.19:48218) 2024-05-07T02:33:58.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.641+0000 7fa86f400700 1 -- 172.21.15.19:0/3256451489 learned_addr learned my addr 172.21.15.19:0/3256451489 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:33:58.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.641+0000 7fa86fe00700 1 --2- 172.21.15.19:0/3256451489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598656ae830 0x5598656ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:58.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.641+0000 7fa86f400700 1 -- 172.21.15.19:0/3256451489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598656ae830 msgr2=0x5598656ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:58.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.641+0000 7fa86f400700 1 --2- 172.21.15.19:0/3256451489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598656ae830 0x5598656ca5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:58.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.641+0000 7fa86f400700 1 -- 172.21.15.19:0/3256451489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598656a9380 msgr2=0x5598656ca0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:33:58.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.641+0000 7fa86f400700 1 --2- 172.21.15.19:0/3256451489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598656a9380 0x5598656ca0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:58.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.641+0000 7fa86f400700 1 -- 172.21.15.19:0/3256451489 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598656cadc0 con 0x5598656a8a30 2024-05-07T02:33:58.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.641+0000 7fa86ea00700 1 --2- 172.21.15.19:0/3256451489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598656a9380 0x5598656ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:33:58.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.642+0000 7fa86f400700 1 --2- 172.21.15.19:0/3256451489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598656a8a30 0x5598656bb680 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa86001ae00 tx=0x7fa8600092b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a0ecece3c1bdffc server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:33:58.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.642+0000 7fa859600700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:33:58.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.642+0000 7fa859600700 1 -- 172.21.15.19:0/3256451489 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86001a700 con 0x5598656a8a30 2024-05-07T02:33:58.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.642+0000 7fa859600700 1 -- 172.21.15.19:0/3256451489 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fa860017040 con 0x5598656a8a30 2024-05-07T02:33:58.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.642+0000 7fa884b4db80 1 -- 172.21.15.19:0/3256451489 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa86000fda0 con 0x5598656a8a30 2024-05-07T02:33:58.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.642+0000 7fa859600700 1 -- 172.21.15.19:0/3256451489 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa86003f050 con 0x5598656a8a30 2024-05-07T02:33:58.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.642+0000 7fa884b4db80 10 client.25699 fetch_fsmap learned FSMap version 34 2024-05-07T02:33:58.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.642+0000 7fa884b4db80 1 -- 172.21.15.19:0/3256451489 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55986572d5e0 con 0x5598656a8a30 2024-05-07T02:33:58.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.643+0000 7fa859600700 1 -- 172.21.15.19:0/3256451489 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa86000fda0 con 0x5598656a8a30 2024-05-07T02:33:58.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.643+0000 7fa859600700 1 -- 172.21.15.19:0/3256451489 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7fa860022040 con 0x5598656a8a30 2024-05-07T02:33:58.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.643+0000 7fa884b4db80 10 client.25699 fetch_fsmap finished waiting for FSMap version 34 2024-05-07T02:33:58.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.643+0000 7fa884b4db80 10 client.25699 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:33:58.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.643+0000 7fa884b4db80 20 client.25699 populate_metadata read hostname 'smithi019' 2024-05-07T02:33:58.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.643+0000 7fa884b4db80 1 --2- 172.21.15.19:0/3256451489 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55986572dc00 0x559865730020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:33:58.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.643+0000 7fa884b4db80 4 client.25699 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-07T02:33:58.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.643+0000 7fa884b4db80 1 -- 172.21.15.19:0/3256451489 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5598655d4400 con 0x55986572dc00 2024-05-07T02:33:58.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.644+0000 7fa86ea00700 1 --2- 172.21.15.19:0/3256451489 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55986572dc00 0x559865730020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:33:58.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.644+0000 7fa86ea00700 1 --2- 172.21.15.19:0/3256451489 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55986572dc00 0x559865730020 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9bd18c4d4597831 server_cookie=f675a266701f3412 in_seq=0 out_seq=0 2024-05-07T02:33:58.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.644+0000 7fa859600700 10 client.25699 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:33:58.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.644+0000 7fa859600700 1 -- 172.21.15.19:0/3256451489 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5598655d4400 con 0x55986572dc00 2024-05-07T02:33:58.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.644+0000 7fa859600700 10 client.25699 handle_command_reply: tid=0 2024-05-07T02:33:58.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.644+0000 7fa884b4db80 1 client.25699 shutdown 2024-05-07T02:33:58.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.645+0000 7fa884b4db80 1 -- 172.21.15.19:0/3256451489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598656a8a30 msgr2=0x5598656bb680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:58.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.645+0000 7fa884b4db80 1 --2- 172.21.15.19:0/3256451489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598656a8a30 0x5598656bb680 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa86001ae00 tx=0x7fa8600092b0 comp rx=0 tx=0).stop 2024-05-07T02:33:58.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.645+0000 7fa884b4db80 1 -- 172.21.15.19:0/3256451489 shutdown_connections 2024-05-07T02:33:58.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.645+0000 7fa884b4db80 1 --2- 172.21.15.19:0/3256451489 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55986572dc00 0x559865730020 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:58.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.645+0000 7fa884b4db80 1 --2- 172.21.15.19:0/3256451489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598656ae830 0x5598656ca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:58.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.645+0000 7fa884b4db80 1 --2- 172.21.15.19:0/3256451489 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598656a8a30 0x5598656bb680 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:58.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.645+0000 7fa884b4db80 1 --2- 172.21.15.19:0/3256451489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598656a9380 0x5598656ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:58.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.645+0000 7fa859600700 0 client.25699 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:33:58.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.645+0000 7fa884b4db80 1 -- 172.21.15.19:0/3256451489 >> 172.21.15.19:0/3256451489 conn(0x559865526e40 msgr2=0x5598656a3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:58.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.645+0000 7fa884b4db80 1 -- 172.21.15.19:0/3256451489 shutdown_connections 2024-05-07T02:33:58.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.645+0000 7fa884b4db80 1 -- 172.21.15.19:0/3256451489 wait complete. 2024-05-07T02:33:58.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.645+0000 7fa884b4db80 20 client.25699 trim_cache size 0 max 16384 2024-05-07T02:33:58.648 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:33:58.648 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-07T02:33:58.648 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-05-07T02:33:58.648 INFO:teuthology.orchestra.run.smithi019.stdout: "ddc682a5-c92f-4cb1-b53a-4ce3a024c1e3": { 2024-05-07T02:33:58.648 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-05-07T02:33:58.648 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "ddc682a5-c92f-4cb1-b53a-4ce3a024c1e3", 2024-05-07T02:33:58.649 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-05-07T02:33:58.649 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:33:58.649 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:33:58.649 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:33:58.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.646+0000 7fa876000700 1 -- 172.21.15.19:0/3365648843 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fa8540b29f0 msgr2=0x7fa8540b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:58.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.646+0000 7fa876000700 1 --2- 172.21.15.19:0/3365648843 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fa8540b29f0 0x7fa8540b4f00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa868029000 tx=0x7fa86802c000 comp rx=0 tx=0).stop 2024-05-07T02:33:58.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.646+0000 7fa876000700 1 -- 172.21.15.19:0/3365648843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa870105e70 msgr2=0x7fa87019b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:33:58.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.646+0000 7fa876000700 1 --2- 172.21.15.19:0/3365648843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa870105e70 0x7fa87019b2d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa85c00ec10 tx=0x7fa85c00ef20 comp rx=0 tx=0).stop 2024-05-07T02:33:58.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.646+0000 7fa876000700 1 -- 172.21.15.19:0/3365648843 shutdown_connections 2024-05-07T02:33:58.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.646+0000 7fa876000700 1 --2- 172.21.15.19:0/3365648843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa87010e900 0x7fa87019f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:58.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.646+0000 7fa876000700 1 --2- 172.21.15.19:0/3365648843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa870105520 0x7fa87019ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:58.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.646+0000 7fa876000700 1 --2- 172.21.15.19:0/3365648843 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fa8540b29f0 0x7fa8540b4f00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:58.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.646+0000 7fa876000700 1 --2- 172.21.15.19:0/3365648843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa870105e70 0x7fa87019b2d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:33:58.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.646+0000 7fa876000700 1 -- 172.21.15.19:0/3365648843 >> 172.21.15.19:0/3365648843 conn(0x7fa8700fedc0 msgr2=0x7fa8700ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:33:58.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.646+0000 7fa876000700 1 -- 172.21.15.19:0/3365648843 shutdown_connections 2024-05-07T02:33:58.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:33:58.646+0000 7fa876000700 1 -- 172.21.15.19:0/3365648843 wait complete. 2024-05-07T02:33:58.726 INFO:tasks.workunit.client.0.smithi019.stdout: 1 36174 3.93 MB/sec warmup 54 sec latency 2288.025 ms 2024-05-07T02:33:59.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:33:58 smithi064 ceph-mon[34699]: pgmap v67: 129 pgs: 129 active+clean; 35 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 106 KiB/s wr, 35 op/s 2024-05-07T02:33:59.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:33:58 smithi019 ceph-mon[29781]: pgmap v67: 129 pgs: 129 active+clean; 35 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 106 KiB/s wr, 35 op/s 2024-05-07T02:33:59.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:33:58 smithi027 ceph-mon[34398]: pgmap v67: 129 pgs: 129 active+clean; 35 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 106 KiB/s wr, 35 op/s 2024-05-07T02:33:59.727 INFO:tasks.workunit.client.0.smithi019.stdout: 1 36174 3.86 MB/sec warmup 55 sec latency 3288.134 ms 2024-05-07T02:33:59.746 INFO:tasks.cephfs.filesystem:scrub status for tag:ddc682a5-c92f-4cb1-b53a-4ce3a024c1e3 - {'path': '/', 'tag': 'ddc682a5-c92f-4cb1-b53a-4ce3a024c1e3', 'options': 'recursive,force'} 2024-05-07T02:34:00.727 INFO:tasks.workunit.client.0.smithi019.stdout: 1 36341 3.80 MB/sec warmup 56 sec latency 4054.067 ms 2024-05-07T02:34:01.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:34:00 smithi064 ceph-mon[34699]: pgmap v68: 129 pgs: 129 active+clean; 35 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 106 KiB/s wr, 35 op/s 2024-05-07T02:34:01.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:34:00 smithi019 ceph-mon[29781]: pgmap v68: 129 pgs: 129 active+clean; 35 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 106 KiB/s wr, 35 op/s 2024-05-07T02:34:01.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:34:00 smithi027 ceph-mon[34398]: pgmap v68: 129 pgs: 129 active+clean; 35 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 106 KiB/s wr, 35 op/s 2024-05-07T02:34:01.727 INFO:tasks.workunit.client.0.smithi019.stdout: 1 36540 3.75 MB/sec warmup 57 sec latency 90.934 ms 2024-05-07T02:34:02.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:34:02 smithi064 ceph-mon[34699]: pgmap v69: 129 pgs: 129 active+clean; 35 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 216 KiB/s wr, 68 op/s 2024-05-07T02:34:02.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:34:02 smithi019 ceph-mon[29781]: pgmap v69: 129 pgs: 129 active+clean; 35 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 216 KiB/s wr, 68 op/s 2024-05-07T02:34:02.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:34:02 smithi027 ceph-mon[34398]: pgmap v69: 129 pgs: 129 active+clean; 35 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 216 KiB/s wr, 68 op/s 2024-05-07T02:34:02.727 INFO:tasks.workunit.client.0.smithi019.stdout: 1 37885 3.78 MB/sec warmup 58 sec latency 94.984 ms 2024-05-07T02:34:03.727 INFO:tasks.workunit.client.0.smithi019.stdout: 1 39197 3.90 MB/sec warmup 59 sec latency 14.484 ms 2024-05-07T02:34:03.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:34:03 smithi027 ceph-mon[34398]: scrub summary: idle 2024-05-07T02:34:04.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:34:03 smithi064 ceph-mon[34699]: scrub summary: idle 2024-05-07T02:34:04.136 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:34:03 smithi019 ceph-mon[29781]: scrub summary: idle 2024-05-07T02:34:04.727 INFO:tasks.workunit.client.0.smithi019.stdout: 1 39490 3.85 MB/sec warmup 60 sec latency 28.807 ms 2024-05-07T02:34:04.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:34:04 smithi027 ceph-mon[34398]: scrub summary: idle 2024-05-07T02:34:04.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:34:04 smithi027 ceph-mon[34398]: scrub summary: idle 2024-05-07T02:34:04.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:34:04 smithi027 ceph-mon[34398]: scrub summary: idle 2024-05-07T02:34:04.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:34:04 smithi027 ceph-mon[34398]: scrub summary: idle 2024-05-07T02:34:04.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:34:04 smithi027 ceph-mon[34398]: pgmap v70: 129 pgs: 129 active+clean; 35 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 128 KiB/s wr, 44 op/s 2024-05-07T02:34:05.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:34:04 smithi064 ceph-mon[34699]: scrub summary: idle 2024-05-07T02:34:05.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:34:04 smithi064 ceph-mon[34699]: scrub summary: idle 2024-05-07T02:34:05.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:34:04 smithi064 ceph-mon[34699]: scrub summary: idle 2024-05-07T02:34:05.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:34:04 smithi064 ceph-mon[34699]: scrub summary: idle 2024-05-07T02:34:05.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:34:04 smithi064 ceph-mon[34699]: pgmap v70: 129 pgs: 129 active+clean; 35 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 128 KiB/s wr, 44 op/s 2024-05-07T02:34:05.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:34:04 smithi019 ceph-mon[29781]: scrub summary: idle 2024-05-07T02:34:05.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:34:04 smithi019 ceph-mon[29781]: scrub summary: idle 2024-05-07T02:34:05.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:34:04 smithi019 ceph-mon[29781]: scrub summary: idle 2024-05-07T02:34:05.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:34:04 smithi019 ceph-mon[29781]: scrub summary: idle 2024-05-07T02:34:05.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:34:04 smithi019 ceph-mon[29781]: pgmap v70: 129 pgs: 129 active+clean; 35 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 128 KiB/s wr, 44 op/s 2024-05-07T02:34:05.727 INFO:tasks.workunit.client.0.smithi019.stdout: 1 39846 3.80 MB/sec warmup 61 sec latency 222.691 ms 2024-05-07T02:34:06.727 INFO:tasks.workunit.client.0.smithi019.stdout: 1 39846 3.74 MB/sec warmup 62 sec latency 1154.559 ms 2024-05-07T02:34:07.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:34:06 smithi064 ceph-mon[34699]: pgmap v71: 129 pgs: 129 active+clean; 37 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 392 KiB/s wr, 102 op/s 2024-05-07T02:34:07.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:34:06 smithi019 ceph-mon[29781]: pgmap v71: 129 pgs: 129 active+clean; 37 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 392 KiB/s wr, 102 op/s 2024-05-07T02:34:07.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:34:06 smithi027 ceph-mon[34398]: pgmap v71: 129 pgs: 129 active+clean; 37 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 392 KiB/s wr, 102 op/s 2024-05-07T02:34:07.727 INFO:tasks.workunit.client.0.smithi019.stdout: 1 39846 3.68 MB/sec warmup 63 sec latency 2154.655 ms 2024-05-07T02:34:08.728 INFO:tasks.workunit.client.0.smithi019.stdout: 1 39846 3.62 MB/sec warmup 64 sec latency 3154.759 ms 2024-05-07T02:34:09.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:34:08 smithi064 ceph-mon[34699]: pgmap v72: 129 pgs: 129 active+clean; 37 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 374 KiB/s wr, 91 op/s 2024-05-07T02:34:09.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:34:08 smithi019 ceph-mon[29781]: pgmap v72: 129 pgs: 129 active+clean; 37 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 374 KiB/s wr, 91 op/s 2024-05-07T02:34:09.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:34:08 smithi027 ceph-mon[34398]: pgmap v72: 129 pgs: 129 active+clean; 37 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 374 KiB/s wr, 91 op/s 2024-05-07T02:34:09.728 INFO:tasks.workunit.client.0.smithi019.stdout: 1 39846 3.56 MB/sec warmup 65 sec latency 4154.928 ms 2024-05-07T02:34:10.728 INFO:tasks.workunit.client.0.smithi019.stdout: 1 40102 3.53 MB/sec warmup 66 sec latency 4910.024 ms 2024-05-07T02:34:11.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:34:10 smithi064 ceph-mon[34699]: pgmap v73: 129 pgs: 129 active+clean; 37 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 374 KiB/s wr, 91 op/s 2024-05-07T02:34:11.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:34:10 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:34:11.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:34:10 smithi019 ceph-mon[29781]: pgmap v73: 129 pgs: 129 active+clean; 37 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 374 KiB/s wr, 91 op/s 2024-05-07T02:34:11.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:34:10 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:34:11.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:34:10 smithi027 ceph-mon[34398]: pgmap v73: 129 pgs: 129 active+clean; 37 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 374 KiB/s wr, 91 op/s 2024-05-07T02:34:11.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:34:10 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:34:11.728 INFO:tasks.workunit.client.0.smithi019.stdout: 1 41841 3.62 MB/sec warmup 67 sec latency 15.203 ms 2024-05-07T02:34:12.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:34:11 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:34:12.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:34:11 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:34:12.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:34:11 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:34:12.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:34:11 smithi064 ceph-mon[34699]: pgmap v74: 129 pgs: 129 active+clean; 37 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 428 KiB/s wr, 106 op/s 2024-05-07T02:34:12.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:34:11 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:34:12.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:34:11 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:34:12.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:34:11 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:34:12.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:34:11 smithi019 ceph-mon[29781]: pgmap v74: 129 pgs: 129 active+clean; 37 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 428 KiB/s wr, 106 op/s 2024-05-07T02:34:12.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:34:11 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:34:12.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:34:11 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:34:12.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:34:11 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:34:12.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:34:11 smithi027 ceph-mon[34398]: pgmap v74: 129 pgs: 129 active+clean; 37 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 428 KiB/s wr, 106 op/s 2024-05-07T02:34:12.728 INFO:tasks.workunit.client.0.smithi019.stdout: 1 42878 3.68 MB/sec warmup 68 sec latency 24.256 ms 2024-05-07T02:34:13.728 INFO:tasks.workunit.client.0.smithi019.stdout: 1 43190 3.63 MB/sec warmup 69 sec latency 21.818 ms 2024-05-07T02:34:14.728 INFO:tasks.workunit.client.0.smithi019.stdout: 1 43388 3.58 MB/sec warmup 70 sec latency 744.247 ms 2024-05-07T02:34:15.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:34:14 smithi064 ceph-mon[34699]: pgmap v75: 129 pgs: 129 active+clean; 37 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 318 KiB/s wr, 73 op/s 2024-05-07T02:34:15.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:34:14 smithi019 ceph-mon[29781]: pgmap v75: 129 pgs: 129 active+clean; 37 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 318 KiB/s wr, 73 op/s 2024-05-07T02:34:15.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:34:14 smithi027 ceph-mon[34398]: pgmap v75: 129 pgs: 129 active+clean; 37 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 318 KiB/s wr, 73 op/s 2024-05-07T02:34:15.728 INFO:tasks.workunit.client.0.smithi019.stdout: 1 43600 3.54 MB/sec warmup 71 sec latency 1502.888 ms 2024-05-07T02:34:16.729 INFO:tasks.workunit.client.0.smithi019.stdout: 1 45193 3.59 MB/sec warmup 72 sec latency 19.174 ms 2024-05-07T02:34:17.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:34:16 smithi064 ceph-mon[34699]: pgmap v76: 129 pgs: 129 active+clean; 39 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 537 KiB/s wr, 117 op/s 2024-05-07T02:34:17.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:34:16 smithi019 ceph-mon[29781]: pgmap v76: 129 pgs: 129 active+clean; 39 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 537 KiB/s wr, 117 op/s 2024-05-07T02:34:17.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:34:16 smithi027 ceph-mon[34398]: pgmap v76: 129 pgs: 129 active+clean; 39 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 537 KiB/s wr, 117 op/s 2024-05-07T02:34:17.729 INFO:tasks.workunit.client.0.smithi019.stdout: 1 46380 3.69 MB/sec warmup 73 sec latency 42.595 ms 2024-05-07T02:34:18.729 INFO:tasks.workunit.client.0.smithi019.stdout: 1 46693 3.64 MB/sec warmup 74 sec latency 24.447 ms 2024-05-07T02:34:19.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:34:18 smithi064 ceph-mon[34699]: pgmap v77: 129 pgs: 129 active+clean; 39 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 273 KiB/s wr, 58 op/s 2024-05-07T02:34:19.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:34:18 smithi019 ceph-mon[29781]: pgmap v77: 129 pgs: 129 active+clean; 39 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 273 KiB/s wr, 58 op/s 2024-05-07T02:34:19.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:34:18 smithi027 ceph-mon[34398]: pgmap v77: 129 pgs: 129 active+clean; 39 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 273 KiB/s wr, 58 op/s 2024-05-07T02:34:19.729 INFO:tasks.workunit.client.0.smithi019.stdout: 1 46814 3.60 MB/sec warmup 75 sec latency 867.148 ms 2024-05-07T02:34:20.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:34:20 smithi064 ceph-mon[34699]: pgmap v78: 129 pgs: 129 active+clean; 39 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 273 KiB/s wr, 58 op/s 2024-05-07T02:34:20.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:34:20 smithi019 ceph-mon[29781]: pgmap v78: 129 pgs: 129 active+clean; 39 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 273 KiB/s wr, 58 op/s 2024-05-07T02:34:20.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:34:20 smithi027 ceph-mon[34398]: pgmap v78: 129 pgs: 129 active+clean; 39 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 273 KiB/s wr, 58 op/s 2024-05-07T02:34:20.729 INFO:tasks.workunit.client.0.smithi019.stdout: 1 46870 3.55 MB/sec warmup 76 sec latency 1632.537 ms 2024-05-07T02:34:21.729 INFO:tasks.workunit.client.0.smithi019.stdout: 1 46870 3.50 MB/sec warmup 77 sec latency 1162.188 ms 2024-05-07T02:34:22.730 INFO:tasks.workunit.client.0.smithi019.stdout: 1 46870 3.46 MB/sec warmup 78 sec latency 2162.297 ms 2024-05-07T02:34:23.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:34:22 smithi064 ceph-mon[34699]: pgmap v79: 129 pgs: 129 active+clean; 42 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 735 KiB/s wr, 104 op/s 2024-05-07T02:34:23.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:34:22 smithi019 ceph-mon[29781]: pgmap v79: 129 pgs: 129 active+clean; 42 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 735 KiB/s wr, 104 op/s 2024-05-07T02:34:23.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:34:22 smithi027 ceph-mon[34398]: pgmap v79: 129 pgs: 129 active+clean; 42 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 735 KiB/s wr, 104 op/s 2024-05-07T02:34:23.730 INFO:tasks.workunit.client.0.smithi019.stdout: 1 46870 3.42 MB/sec warmup 79 sec latency 3162.419 ms 2024-05-07T02:34:24.730 INFO:tasks.workunit.client.0.smithi019.stdout: 1 46870 3.37 MB/sec warmup 80 sec latency 4162.554 ms 2024-05-07T02:34:25.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:34:24 smithi064 ceph-mon[34699]: pgmap v80: 129 pgs: 129 active+clean; 42 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 681 KiB/s wr, 89 op/s 2024-05-07T02:34:25.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:34:24 smithi019 ceph-mon[29781]: pgmap v80: 129 pgs: 129 active+clean; 42 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 681 KiB/s wr, 89 op/s 2024-05-07T02:34:25.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:34:24 smithi027 ceph-mon[34398]: pgmap v80: 129 pgs: 129 active+clean; 42 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 681 KiB/s wr, 89 op/s 2024-05-07T02:34:25.730 INFO:tasks.workunit.client.0.smithi019.stdout: 1 46938 3.33 MB/sec warmup 81 sec latency 4922.007 ms 2024-05-07T02:34:26.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:34:26 smithi064 ceph-mon[34699]: pgmap v81: 129 pgs: 129 active+clean; 42 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 716 KiB/s wr, 100 op/s 2024-05-07T02:34:26.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:34:26 smithi019 ceph-mon[29781]: pgmap v81: 129 pgs: 129 active+clean; 42 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 716 KiB/s wr, 100 op/s 2024-05-07T02:34:26.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:34:26 smithi027 ceph-mon[34398]: pgmap v81: 129 pgs: 129 active+clean; 42 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 716 KiB/s wr, 100 op/s 2024-05-07T02:34:26.730 INFO:tasks.workunit.client.0.smithi019.stdout: 1 46938 3.29 MB/sec warmup 82 sec latency 1161.119 ms 2024-05-07T02:34:27.730 INFO:tasks.workunit.client.0.smithi019.stdout: 1 46938 3.25 MB/sec warmup 83 sec latency 2161.237 ms 2024-05-07T02:34:28.731 INFO:tasks.workunit.client.0.smithi019.stdout: 1 46938 3.21 MB/sec warmup 84 sec latency 3161.407 ms 2024-05-07T02:34:29.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:34:28 smithi064 ceph-mon[34699]: pgmap v82: 129 pgs: 129 active+clean; 42 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 497 KiB/s wr, 56 op/s 2024-05-07T02:34:29.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:34:28 smithi019 ceph-mon[29781]: pgmap v82: 129 pgs: 129 active+clean; 42 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 497 KiB/s wr, 56 op/s 2024-05-07T02:34:29.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:34:28 smithi027 ceph-mon[34398]: pgmap v82: 129 pgs: 129 active+clean; 42 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 497 KiB/s wr, 56 op/s 2024-05-07T02:34:29.730 INFO:tasks.workunit.client.0.smithi019.stdout: 1 46938 3.18 MB/sec warmup 85 sec latency 4161.503 ms 2024-05-07T02:34:29.747 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-07T02:34:30.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:34:30 smithi019 ceph-mon[29781]: pgmap v83: 129 pgs: 129 active+clean; 42 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 497 KiB/s wr, 56 op/s 2024-05-07T02:34:30.231 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:34:30.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:34:30 smithi064 ceph-mon[34699]: pgmap v83: 129 pgs: 129 active+clean; 42 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 497 KiB/s wr, 56 op/s 2024-05-07T02:34:30.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:34:30 smithi027 ceph-mon[34398]: pgmap v83: 129 pgs: 129 active+clean; 42 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 497 KiB/s wr, 56 op/s 2024-05-07T02:34:30.731 INFO:tasks.workunit.client.0.smithi019.stdout: 1 47204 3.16 MB/sec warmup 86 sec latency 4918.065 ms 2024-05-07T02:34:31.731 INFO:tasks.workunit.client.0.smithi019.stdout: 1 48723 3.19 MB/sec warmup 87 sec latency 18.310 ms 2024-05-07T02:34:32.731 INFO:tasks.workunit.client.0.smithi019.stdout: 1 49963 3.28 MB/sec warmup 88 sec latency 21.865 ms 2024-05-07T02:34:33.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.076+0000 7f5974a00700 1 -- 172.21.15.19:0/3279920282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5970109870 msgr2=0x7f59701111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:33.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.076+0000 7f5974a00700 1 --2- 172.21.15.19:0/3279920282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5970109870 0x7f59701111d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f596800b0a0 tx=0x7f596800b3b0 comp rx=0 tx=0).stop 2024-05-07T02:34:33.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.076+0000 7f5974a00700 1 -- 172.21.15.19:0/3279920282 shutdown_connections 2024-05-07T02:34:33.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.076+0000 7f5974a00700 1 --2- 172.21.15.19:0/3279920282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5970109870 0x7f59701111d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:33.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.076+0000 7f5974a00700 1 --2- 172.21.15.19:0/3279920282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59701045f0 0x7f5970104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:33.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.076+0000 7f5974a00700 1 --2- 172.21.15.19:0/3279920282 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5970108f20 0x7f5970109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:33.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.076+0000 7f5974a00700 1 -- 172.21.15.19:0/3279920282 >> 172.21.15.19:0/3279920282 conn(0x7f5970100040 msgr2=0x7f5970102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:34:33.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.076+0000 7f5974a00700 1 -- 172.21.15.19:0/3279920282 shutdown_connections 2024-05-07T02:34:33.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.076+0000 7f5974a00700 1 -- 172.21.15.19:0/3279920282 wait complete. 2024-05-07T02:34:33.077 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:34:32 smithi019 ceph-mon[29781]: pgmap v84: 129 pgs: 129 active+clean; 43 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 573 KiB/s wr, 69 op/s 2024-05-07T02:34:33.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.077+0000 7f5974a00700 1 Processor -- start 2024-05-07T02:34:33.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.078+0000 7f5974a00700 1 -- start start 2024-05-07T02:34:33.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.078+0000 7f5974a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59701045f0 0x7f5970075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:33.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.078+0000 7f5974a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5970108f20 0x7f5970076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:33.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.078+0000 7f5974a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5970109870 0x7f5970076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:33.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.078+0000 7f5974a00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5970114510 con 0x7f5970108f20 2024-05-07T02:34:33.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.078+0000 7f5974a00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5970114390 con 0x7f5970109870 2024-05-07T02:34:33.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.078+0000 7f5974a00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5970114690 con 0x7f59701045f0 2024-05-07T02:34:33.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.078+0000 7f596de00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59701045f0 0x7f5970075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:33.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.078+0000 7f596de00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59701045f0 0x7f5970075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:35070/0 (socket says 172.21.15.19:35070) 2024-05-07T02:34:33.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.078+0000 7f596de00700 1 -- 172.21.15.19:0/4241240069 learned_addr learned my addr 172.21.15.19:0/4241240069 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:34:33.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.079+0000 7f596d400700 1 --2- 172.21.15.19:0/4241240069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5970108f20 0x7f5970076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:33.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.079+0000 7f596e800700 1 --2- 172.21.15.19:0/4241240069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5970109870 0x7f5970076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:33.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.079+0000 7f596de00700 1 -- 172.21.15.19:0/4241240069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5970109870 msgr2=0x7f5970076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:33.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.079+0000 7f596de00700 1 --2- 172.21.15.19:0/4241240069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5970109870 0x7f5970076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:33.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.079+0000 7f596de00700 1 -- 172.21.15.19:0/4241240069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5970108f20 msgr2=0x7f5970076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:33.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.079+0000 7f596de00700 1 --2- 172.21.15.19:0/4241240069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5970108f20 0x7f5970076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:33.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.079+0000 7f596de00700 1 -- 172.21.15.19:0/4241240069 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59701a91c0 con 0x7f59701045f0 2024-05-07T02:34:33.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.079+0000 7f596e800700 1 --2- 172.21.15.19:0/4241240069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5970109870 0x7f5970076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-07T02:34:33.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.079+0000 7f596de00700 1 --2- 172.21.15.19:0/4241240069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59701045f0 0x7f5970075ac0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f596000e9a0 tx=0x7f596000ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:34:33.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.080+0000 7f5966a00700 1 -- 172.21.15.19:0/4241240069 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596000cbd0 con 0x7f59701045f0 2024-05-07T02:34:33.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.080+0000 7f5966a00700 1 -- 172.21.15.19:0/4241240069 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f596000cd70 con 0x7f59701045f0 2024-05-07T02:34:33.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.080+0000 7f5974a00700 1 -- 172.21.15.19:0/4241240069 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59701a94b0 con 0x7f59701045f0 2024-05-07T02:34:33.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.080+0000 7f5966a00700 1 -- 172.21.15.19:0/4241240069 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5960018680 con 0x7f59701045f0 2024-05-07T02:34:33.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.080+0000 7f5974a00700 1 -- 172.21.15.19:0/4241240069 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59701a9a40 con 0x7f59701045f0 2024-05-07T02:34:33.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.084+0000 7f5966a00700 1 -- 172.21.15.19:0/4241240069 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5960010970 con 0x7f59701045f0 2024-05-07T02:34:33.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.086+0000 7f598361bb80 1 Processor -- start 2024-05-07T02:34:33.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.086+0000 7f598361bb80 1 -- start start 2024-05-07T02:34:33.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.086+0000 7f5966a00700 1 --2- 172.21.15.19:0/4241240069 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f59540b2a70 0x7f59540b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:33.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.086+0000 7f598361bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6cb388470 0x55c6cb388880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:33.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.086+0000 7f5966a00700 1 -- 172.21.15.19:0/4241240069 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f59600e2d20 con 0x7f59701045f0 2024-05-07T02:34:33.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.086+0000 7f598361bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6cb382620 0x55c6cb382a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:33.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.087+0000 7f596e800700 1 --2- 172.21.15.19:0/4241240069 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f59540b2a70 0x7f59540b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:33.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.087+0000 7f598361bb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6cb382f70 0x55c6cb38f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:33.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.087+0000 7f598361bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c6cb2a4a20 con 0x55c6cb382620 2024-05-07T02:34:33.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.087+0000 7f598361bb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c6cb242760 con 0x55c6cb382f70 2024-05-07T02:34:33.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.087+0000 7f598361bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c6cb2a09e0 con 0x55c6cb388470 2024-05-07T02:34:33.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.087+0000 7f596d400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6cb382620 0x55c6cb382a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:33.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.087+0000 7f596d400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6cb382620 0x55c6cb382a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59572/0 (socket says 172.21.15.19:59572) 2024-05-07T02:34:33.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.087+0000 7f596d400700 1 -- 172.21.15.19:0/2550658415 learned_addr learned my addr 172.21.15.19:0/2550658415 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:34:33.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.087+0000 7f596de00700 1 --2- 172.21.15.19:0/2550658415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6cb388470 0x55c6cb388880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:33.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.087+0000 7f596e800700 1 --2- 172.21.15.19:0/2550658415 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6cb382f70 0x55c6cb38f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:33.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.088+0000 7f596de00700 1 -- 172.21.15.19:0/2550658415 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6cb382f70 msgr2=0x55c6cb38f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:33.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.088+0000 7f596de00700 1 --2- 172.21.15.19:0/2550658415 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6cb382f70 0x55c6cb38f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:33.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.088+0000 7f596de00700 1 -- 172.21.15.19:0/2550658415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6cb382620 msgr2=0x55c6cb382a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:33.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.088+0000 7f596de00700 1 --2- 172.21.15.19:0/2550658415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6cb382620 0x55c6cb382a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:33.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.088+0000 7f596de00700 1 -- 172.21.15.19:0/2550658415 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6cb1f18d0 con 0x55c6cb388470 2024-05-07T02:34:33.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.088+0000 7f596d400700 1 --2- 172.21.15.19:0/2550658415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6cb382620 0x55c6cb382a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:34:33.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.088+0000 7f596e800700 1 --2- 172.21.15.19:0/4241240069 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f59540b2a70 0x7f59540b4f80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5968009e60 tx=0x7f5968009590 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:34:33.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.088+0000 7f596de00700 1 --2- 172.21.15.19:0/2550658415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6cb388470 0x55c6cb388880 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f59600e0080 tx=0x7f59600a9670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=815e0eec86d5a61b server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:34:33.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.089+0000 7f595b400700 1 -- 172.21.15.19:0/2550658415 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59600adfc0 con 0x55c6cb388470 2024-05-07T02:34:33.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.089+0000 7f595b400700 1 -- 172.21.15.19:0/2550658415 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59600b2a70 con 0x55c6cb388470 2024-05-07T02:34:33.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.089+0000 7f595b400700 1 -- 172.21.15.19:0/2550658415 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59600a57e0 con 0x55c6cb388470 2024-05-07T02:34:33.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.090+0000 7f598361bb80 1 -- 172.21.15.19:0/2550658415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6cb388470 msgr2=0x55c6cb388880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:33.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.090+0000 7f598361bb80 1 --2- 172.21.15.19:0/2550658415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6cb388470 0x55c6cb388880 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f59600e0080 tx=0x7f59600a9670 comp rx=0 tx=0).stop 2024-05-07T02:34:33.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.090+0000 7f598361bb80 1 -- 172.21.15.19:0/2550658415 shutdown_connections 2024-05-07T02:34:33.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.090+0000 7f598361bb80 1 --2- 172.21.15.19:0/2550658415 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6cb382f70 0x55c6cb38f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:33.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.090+0000 7f598361bb80 1 --2- 172.21.15.19:0/2550658415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6cb388470 0x55c6cb388880 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:33.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.090+0000 7f598361bb80 1 --2- 172.21.15.19:0/2550658415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6cb382620 0x55c6cb382a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:33.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.090+0000 7f598361bb80 1 -- 172.21.15.19:0/2550658415 >> 172.21.15.19:0/2550658415 conn(0x55c6cb204e40 msgr2=0x55c6cb37d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:34:33.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.090+0000 7f598361bb80 1 -- 172.21.15.19:0/2550658415 shutdown_connections 2024-05-07T02:34:33.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.090+0000 7f598361bb80 1 -- 172.21.15.19:0/2550658415 wait complete. 2024-05-07T02:34:33.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.091+0000 7f598361bb80 1 -- start start 2024-05-07T02:34:33.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.091+0000 7f598361bb80 1 Processor -- start 2024-05-07T02:34:33.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.092+0000 7f598361bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6cb382620 0x55c6cb38c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:33.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.092+0000 7f598361bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6cb382f70 0x55c6cb38cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:33.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.092+0000 7f598361bb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6cb388470 0x55c6cb38d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:33.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.092+0000 7f598361bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c6cb38f7e0 con 0x55c6cb382f70 2024-05-07T02:34:33.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.092+0000 7f598361bb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c6cb20ab20 con 0x55c6cb388470 2024-05-07T02:34:33.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.092+0000 7f598361bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c6cb38f960 con 0x55c6cb382620 2024-05-07T02:34:33.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.092+0000 7f598361bb80 10 client.0 _ll_register_callbacks cb 0x55c6cb238380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:34:33.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.092+0000 7f596de00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6cb382620 0x55c6cb38c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:33.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.093+0000 7f596de00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6cb382620 0x55c6cb38c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:35092/0 (socket says 172.21.15.19:35092) 2024-05-07T02:34:33.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.093+0000 7f596de00700 1 -- 172.21.15.19:0/2499433091 learned_addr learned my addr 172.21.15.19:0/2499433091 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:34:33.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.093+0000 7f596d400700 1 --2- 172.21.15.19:0/2499433091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6cb382f70 0x55c6cb38cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:33.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.093+0000 7f596e800700 1 --2- 172.21.15.19:0/2499433091 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6cb388470 0x55c6cb38d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:33.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.093+0000 7f596de00700 1 -- 172.21.15.19:0/2499433091 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6cb388470 msgr2=0x55c6cb38d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:33.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.093+0000 7f596de00700 1 --2- 172.21.15.19:0/2499433091 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6cb388470 0x55c6cb38d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:33.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.093+0000 7f596de00700 1 -- 172.21.15.19:0/2499433091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6cb382f70 msgr2=0x55c6cb38cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:33.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.093+0000 7f596de00700 1 --2- 172.21.15.19:0/2499433091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6cb382f70 0x55c6cb38cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:33.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.093+0000 7f596de00700 1 -- 172.21.15.19:0/2499433091 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6cb26a420 con 0x55c6cb382620 2024-05-07T02:34:33.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.093+0000 7f596de00700 1 --2- 172.21.15.19:0/2499433091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6cb382620 0x55c6cb38c6e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f59600e0080 tx=0x7f59600a9b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60278c4888395155 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:34:33.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.093+0000 7f594fe00700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:34:33.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.093+0000 7f594fe00700 1 -- 172.21.15.19:0/2499433091 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59600c4070 con 0x55c6cb382620 2024-05-07T02:34:33.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.094+0000 7f594fe00700 1 -- 172.21.15.19:0/2499433091 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f59600a8e70 con 0x55c6cb382620 2024-05-07T02:34:33.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.094+0000 7f598361bb80 1 -- 172.21.15.19:0/2499433091 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59600b2a70 con 0x55c6cb382620 2024-05-07T02:34:33.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.094+0000 7f594fe00700 1 -- 172.21.15.19:0/2499433091 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f59600adfc0 con 0x55c6cb382620 2024-05-07T02:34:33.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.094+0000 7f598361bb80 10 client.25711 fetch_fsmap learned FSMap version 34 2024-05-07T02:34:33.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.094+0000 7f598361bb80 1 -- 172.21.15.19:0/2499433091 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c6cb38f7e0 con 0x55c6cb382620 2024-05-07T02:34:33.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.094+0000 7f594fe00700 1 -- 172.21.15.19:0/2499433091 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59600b2a70 con 0x55c6cb382620 2024-05-07T02:34:33.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.094+0000 7f594fe00700 1 -- 172.21.15.19:0/2499433091 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f59600a88e0 con 0x55c6cb382620 2024-05-07T02:34:33.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.095+0000 7f598361bb80 10 client.25711 fetch_fsmap finished waiting for FSMap version 34 2024-05-07T02:34:33.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.095+0000 7f598361bb80 10 client.25711 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:34:33.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.095+0000 7f598361bb80 20 client.25711 populate_metadata read hostname 'smithi019' 2024-05-07T02:34:33.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.095+0000 7f598361bb80 1 --2- 172.21.15.19:0/2499433091 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55c6cb39c770 0x55c6cb39eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:33.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.095+0000 7f598361bb80 4 client.25711 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:34:33.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.095+0000 7f598361bb80 1 -- 172.21.15.19:0/2499433091 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6cb34f0e0 con 0x55c6cb39c770 2024-05-07T02:34:33.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.095+0000 7f596d400700 1 --2- 172.21.15.19:0/2499433091 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55c6cb39c770 0x55c6cb39eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:33.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.096+0000 7f596d400700 1 --2- 172.21.15.19:0/2499433091 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55c6cb39c770 0x55c6cb39eb90 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71db08acfec2c60a server_cookie=4845205a438ab8be in_seq=0 out_seq=0 2024-05-07T02:34:33.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.096+0000 7f594fe00700 10 client.25711 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:34:33.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.097+0000 7f594fe00700 1 -- 172.21.15.19:0/2499433091 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c6cb34f0e0 con 0x55c6cb39c770 2024-05-07T02:34:33.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.097+0000 7f594fe00700 10 client.25711 handle_command_reply: tid=0 2024-05-07T02:34:33.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.097+0000 7f598361bb80 1 client.25711 shutdown 2024-05-07T02:34:33.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.097+0000 7f598361bb80 1 -- 172.21.15.19:0/2499433091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6cb382620 msgr2=0x55c6cb38c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:33.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.097+0000 7f598361bb80 1 --2- 172.21.15.19:0/2499433091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6cb382620 0x55c6cb38c6e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f59600e0080 tx=0x7f59600a9b10 comp rx=0 tx=0).stop 2024-05-07T02:34:33.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.097+0000 7f598361bb80 1 -- 172.21.15.19:0/2499433091 shutdown_connections 2024-05-07T02:34:33.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.097+0000 7f598361bb80 1 --2- 172.21.15.19:0/2499433091 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55c6cb39c770 0x55c6cb39eb90 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:33.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.097+0000 7f598361bb80 1 --2- 172.21.15.19:0/2499433091 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6cb388470 0x55c6cb38d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:33.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.097+0000 7f598361bb80 1 --2- 172.21.15.19:0/2499433091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6cb382620 0x55c6cb38c6e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:33.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.097+0000 7f598361bb80 1 --2- 172.21.15.19:0/2499433091 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6cb382f70 0x55c6cb38cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:33.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.097+0000 7f598361bb80 1 -- 172.21.15.19:0/2499433091 >> 172.21.15.19:0/2499433091 conn(0x55c6cb204e40 msgr2=0x55c6cb381610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:34:33.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.097+0000 7f594fe00700 0 client.25711 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:34:33.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.097+0000 7f598361bb80 1 -- 172.21.15.19:0/2499433091 shutdown_connections 2024-05-07T02:34:33.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.098+0000 7f598361bb80 1 -- 172.21.15.19:0/2499433091 wait complete. 2024-05-07T02:34:33.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.098+0000 7f598361bb80 20 client.25711 trim_cache size 0 max 16384 2024-05-07T02:34:33.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.122+0000 7f598361bb80 1 Processor -- start 2024-05-07T02:34:33.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.122+0000 7f598361bb80 1 -- start start 2024-05-07T02:34:33.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.123+0000 7f598361bb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6cb382620 0x55c6cb386d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:33.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.123+0000 7f598361bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6cb382f70 0x55c6cb3778d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:33.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.123+0000 7f598361bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6cb388470 0x55c6cb3a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:33.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.123+0000 7f598361bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f59701b2800 con 0x55c6cb388470 2024-05-07T02:34:33.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.123+0000 7f598361bb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f59701b4400 con 0x55c6cb382620 2024-05-07T02:34:33.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.123+0000 7f598361bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f59701b2ed0 con 0x55c6cb382f70 2024-05-07T02:34:33.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.123+0000 7f596de00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6cb382620 0x55c6cb386d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:33.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.123+0000 7f596de00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6cb382620 0x55c6cb386d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:34978/0 (socket says 172.21.15.19:34978) 2024-05-07T02:34:33.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.123+0000 7f596de00700 1 -- 172.21.15.19:0/3738222602 learned_addr learned my addr 172.21.15.19:0/3738222602 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:34:33.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.123+0000 7f596d400700 1 --2- 172.21.15.19:0/3738222602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6cb382f70 0x55c6cb3778d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:33.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.123+0000 7f596de00700 1 -- 172.21.15.19:0/3738222602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6cb382f70 msgr2=0x55c6cb3778d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:33.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.123+0000 7f596de00700 1 --2- 172.21.15.19:0/3738222602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6cb382f70 0x55c6cb3778d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:33.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.123+0000 7f596de00700 1 -- 172.21.15.19:0/3738222602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6cb388470 msgr2=0x55c6cb3a17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:33.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.123+0000 7f596de00700 1 --2- 172.21.15.19:0/3738222602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6cb388470 0x55c6cb3a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:33.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.123+0000 7f596de00700 1 -- 172.21.15.19:0/3738222602 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6cb38c4b0 con 0x55c6cb382620 2024-05-07T02:34:33.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.124+0000 7f596de00700 1 --2- 172.21.15.19:0/3738222602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6cb382620 0x55c6cb386d90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f596000e970 tx=0x7f596000efa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=210258e33e2a7fc6 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:34:33.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.124+0000 7f595aa00700 1 -- 172.21.15.19:0/3738222602 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596008b030 con 0x55c6cb382620 2024-05-07T02:34:33.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.124+0000 7f595aa00700 1 -- 172.21.15.19:0/3738222602 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59600b2a70 con 0x55c6cb382620 2024-05-07T02:34:33.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.124+0000 7f595aa00700 1 -- 172.21.15.19:0/3738222602 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59600c4040 con 0x55c6cb382620 2024-05-07T02:34:33.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.125+0000 7f598361bb80 1 -- 172.21.15.19:0/3738222602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6cb382620 msgr2=0x55c6cb386d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:33.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.125+0000 7f598361bb80 1 --2- 172.21.15.19:0/3738222602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6cb382620 0x55c6cb386d90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f596000e970 tx=0x7f596000efa0 comp rx=0 tx=0).stop 2024-05-07T02:34:33.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.125+0000 7f598361bb80 1 -- 172.21.15.19:0/3738222602 shutdown_connections 2024-05-07T02:34:33.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.125+0000 7f598361bb80 1 --2- 172.21.15.19:0/3738222602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6cb388470 0x55c6cb3a17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:33.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.125+0000 7f598361bb80 1 --2- 172.21.15.19:0/3738222602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6cb382620 0x55c6cb386d90 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:33.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.125+0000 7f598361bb80 1 --2- 172.21.15.19:0/3738222602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6cb382f70 0x55c6cb3778d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:33.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.125+0000 7f598361bb80 1 -- 172.21.15.19:0/3738222602 >> 172.21.15.19:0/3738222602 conn(0x55c6cb204e40 msgr2=0x55c6cb3771c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:34:33.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.126+0000 7f598361bb80 1 -- 172.21.15.19:0/3738222602 shutdown_connections 2024-05-07T02:34:33.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.126+0000 7f598361bb80 1 -- 172.21.15.19:0/3738222602 wait complete. 2024-05-07T02:34:33.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.126+0000 7f598361bb80 1 -- start start 2024-05-07T02:34:33.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.126+0000 7f598361bb80 1 Processor -- start 2024-05-07T02:34:33.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.127+0000 7f598361bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6cb382620 0x55c6cb3a1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:33.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.127+0000 7f598361bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6cb382f70 0x55c6cb3a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:33.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.127+0000 7f598361bb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6cb388470 0x55c6cb39ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:33.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.127+0000 7f598361bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f59701b3c70 con 0x55c6cb382620 2024-05-07T02:34:33.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.127+0000 7f598361bb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f59701b35a0 con 0x55c6cb388470 2024-05-07T02:34:33.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.127+0000 7f598361bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f59701b07a0 con 0x55c6cb382f70 2024-05-07T02:34:33.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.127+0000 7f598361bb80 10 client.0 _ll_register_callbacks cb 0x55c6cb238380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:34:33.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.127+0000 7f596d400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6cb382f70 0x55c6cb3a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:33.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.127+0000 7f596d400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6cb382f70 0x55c6cb3a1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:35114/0 (socket says 172.21.15.19:35114) 2024-05-07T02:34:33.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.127+0000 7f596d400700 1 -- 172.21.15.19:0/2908013907 learned_addr learned my addr 172.21.15.19:0/2908013907 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:34:33.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.127+0000 7f596e800700 1 --2- 172.21.15.19:0/2908013907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6cb388470 0x55c6cb39ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:33.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.127+0000 7f596de00700 1 --2- 172.21.15.19:0/2908013907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6cb382620 0x55c6cb3a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:33.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.128+0000 7f596d400700 1 -- 172.21.15.19:0/2908013907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6cb388470 msgr2=0x55c6cb39ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:33.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.128+0000 7f596d400700 1 --2- 172.21.15.19:0/2908013907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6cb388470 0x55c6cb39ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:33.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.128+0000 7f596d400700 1 -- 172.21.15.19:0/2908013907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6cb382620 msgr2=0x55c6cb3a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:33.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.128+0000 7f596d400700 1 --2- 172.21.15.19:0/2908013907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6cb382620 0x55c6cb3a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:33.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.128+0000 7f596d400700 1 -- 172.21.15.19:0/2908013907 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6cb39f5f0 con 0x55c6cb382f70 2024-05-07T02:34:33.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.128+0000 7f596de00700 1 --2- 172.21.15.19:0/2908013907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6cb382620 0x55c6cb3a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:34:33.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.128+0000 7f596d400700 1 --2- 172.21.15.19:0/2908013907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6cb382f70 0x55c6cb3a1e60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x55c6cb37d9f0 tx=0x7f595c0045a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3d9781b6555e22f server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:34:33.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.128+0000 7f594fe00700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:34:33.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.128+0000 7f594fe00700 1 -- 172.21.15.19:0/2908013907 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c004a30 con 0x55c6cb382f70 2024-05-07T02:34:33.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.129+0000 7f598361bb80 1 -- 172.21.15.19:0/2908013907 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59600b2a70 con 0x55c6cb382f70 2024-05-07T02:34:33.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.129+0000 7f594fe00700 1 -- 172.21.15.19:0/2908013907 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f595c004bd0 con 0x55c6cb382f70 2024-05-07T02:34:33.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.129+0000 7f594fe00700 1 -- 172.21.15.19:0/2908013907 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f595c02e050 con 0x55c6cb382f70 2024-05-07T02:34:33.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.129+0000 7f598361bb80 10 client.25717 fetch_fsmap learned FSMap version 34 2024-05-07T02:34:33.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.129+0000 7f598361bb80 1 -- 172.21.15.19:0/2908013907 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c6cb406860 con 0x55c6cb382f70 2024-05-07T02:34:33.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.129+0000 7f594fe00700 1 -- 172.21.15.19:0/2908013907 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59600b2a70 con 0x55c6cb382f70 2024-05-07T02:34:33.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.130+0000 7f594fe00700 1 -- 172.21.15.19:0/2908013907 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f595c005220 con 0x55c6cb382f70 2024-05-07T02:34:33.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.130+0000 7f598361bb80 10 client.25717 fetch_fsmap finished waiting for FSMap version 34 2024-05-07T02:34:33.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.130+0000 7f598361bb80 10 client.25717 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:34:33.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.130+0000 7f598361bb80 20 client.25717 populate_metadata read hostname 'smithi019' 2024-05-07T02:34:33.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.130+0000 7f598361bb80 1 --2- 172.21.15.19:0/2908013907 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55c6cb407090 0x55c6cb4094b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:33.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.130+0000 7f598361bb80 4 client.25717 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-07T02:34:33.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.130+0000 7f598361bb80 1 -- 172.21.15.19:0/2908013907 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c6cb2b2400 con 0x55c6cb407090 2024-05-07T02:34:33.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.130+0000 7f596de00700 1 --2- 172.21.15.19:0/2908013907 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55c6cb407090 0x55c6cb4094b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:33.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.131+0000 7f596de00700 1 --2- 172.21.15.19:0/2908013907 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55c6cb407090 0x55c6cb4094b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2a3a9d5e33b64e2 server_cookie=cb5caacb3432a0d in_seq=0 out_seq=0 2024-05-07T02:34:33.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.131+0000 7f594fe00700 10 client.25717 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:34:33.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.131+0000 7f594fe00700 1 -- 172.21.15.19:0/2908013907 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c6cb2b2400 con 0x55c6cb407090 2024-05-07T02:34:33.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.131+0000 7f594fe00700 10 client.25717 handle_command_reply: tid=0 2024-05-07T02:34:33.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.131+0000 7f598361bb80 1 client.25717 shutdown 2024-05-07T02:34:33.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.131+0000 7f598361bb80 1 -- 172.21.15.19:0/2908013907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6cb382f70 msgr2=0x55c6cb3a1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:33.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.131+0000 7f598361bb80 1 --2- 172.21.15.19:0/2908013907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6cb382f70 0x55c6cb3a1e60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x55c6cb37d9f0 tx=0x7f595c0045a0 comp rx=0 tx=0).stop 2024-05-07T02:34:33.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.132+0000 7f598361bb80 1 -- 172.21.15.19:0/2908013907 shutdown_connections 2024-05-07T02:34:33.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.132+0000 7f598361bb80 1 --2- 172.21.15.19:0/2908013907 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55c6cb407090 0x55c6cb4094b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:33.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.132+0000 7f598361bb80 1 --2- 172.21.15.19:0/2908013907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c6cb388470 0x55c6cb39ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:33.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.132+0000 7f598361bb80 1 --2- 172.21.15.19:0/2908013907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c6cb382620 0x55c6cb3a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:33.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.132+0000 7f598361bb80 1 --2- 172.21.15.19:0/2908013907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c6cb382f70 0x55c6cb3a1e60 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:33.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.132+0000 7f598361bb80 1 -- 172.21.15.19:0/2908013907 >> 172.21.15.19:0/2908013907 conn(0x55c6cb204e40 msgr2=0x55c6cb38b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:34:33.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.132+0000 7f594fe00700 0 client.25717 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:34:33.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.132+0000 7f598361bb80 1 -- 172.21.15.19:0/2908013907 shutdown_connections 2024-05-07T02:34:33.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.132+0000 7f598361bb80 1 -- 172.21.15.19:0/2908013907 wait complete. 2024-05-07T02:34:33.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.133+0000 7f598361bb80 20 client.25717 trim_cache size 0 max 16384 2024-05-07T02:34:33.134 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:34:33.134 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-07T02:34:33.134 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-07T02:34:33.134 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:34:33.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.134+0000 7f5974a00700 1 -- 172.21.15.19:0/4241240069 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f59540b2a70 msgr2=0x7f59540b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:33.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.134+0000 7f5974a00700 1 --2- 172.21.15.19:0/4241240069 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f59540b2a70 0x7f59540b4f80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5968009e60 tx=0x7f5968009590 comp rx=0 tx=0).stop 2024-05-07T02:34:33.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.134+0000 7f5974a00700 1 -- 172.21.15.19:0/4241240069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59701045f0 msgr2=0x7f5970075ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:33.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.134+0000 7f5974a00700 1 --2- 172.21.15.19:0/4241240069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59701045f0 0x7f5970075ac0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f596000e9a0 tx=0x7f596000ed60 comp rx=0 tx=0).stop 2024-05-07T02:34:33.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.134+0000 7f5974a00700 1 -- 172.21.15.19:0/4241240069 shutdown_connections 2024-05-07T02:34:33.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.134+0000 7f5974a00700 1 --2- 172.21.15.19:0/4241240069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5970109870 0x7f5970076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:33.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.134+0000 7f5974a00700 1 --2- 172.21.15.19:0/4241240069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f59701045f0 0x7f5970075ac0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:33.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.134+0000 7f5974a00700 1 --2- 172.21.15.19:0/4241240069 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f59540b2a70 0x7f59540b4f80 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:33.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.134+0000 7f5974a00700 1 --2- 172.21.15.19:0/4241240069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5970108f20 0x7f5970076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:33.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.134+0000 7f5974a00700 1 -- 172.21.15.19:0/4241240069 >> 172.21.15.19:0/4241240069 conn(0x7f5970100040 msgr2=0x7f5970108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:34:33.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.135+0000 7f5974a00700 1 -- 172.21.15.19:0/4241240069 shutdown_connections 2024-05-07T02:34:33.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:33.135+0000 7f5974a00700 1 -- 172.21.15.19:0/4241240069 wait complete. 2024-05-07T02:34:33.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:34:32 smithi064 ceph-mon[34699]: pgmap v84: 129 pgs: 129 active+clean; 43 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 573 KiB/s wr, 69 op/s 2024-05-07T02:34:33.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:34:32 smithi027 ceph-mon[34398]: pgmap v84: 129 pgs: 129 active+clean; 43 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 573 KiB/s wr, 69 op/s 2024-05-07T02:34:33.732 INFO:tasks.workunit.client.0.smithi019.stdout: 1 50479 3.25 MB/sec warmup 89 sec latency 98.902 ms 2024-05-07T02:34:34.191 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-07T02:34:34.191 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-07T02:34:34.655 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:34:34.732 INFO:tasks.workunit.client.0.smithi019.stdout: 1 50479 3.21 MB/sec warmup 90 sec latency 1098.988 ms 2024-05-07T02:34:35.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:34:34 smithi019 ceph-mon[29781]: pgmap v85: 129 pgs: 129 active+clean; 43 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 111 KiB/s wr, 24 op/s 2024-05-07T02:34:35.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:34:34 smithi064 ceph-mon[34699]: pgmap v85: 129 pgs: 129 active+clean; 43 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 111 KiB/s wr, 24 op/s 2024-05-07T02:34:35.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:34:34 smithi027 ceph-mon[34398]: pgmap v85: 129 pgs: 129 active+clean; 43 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 111 KiB/s wr, 24 op/s 2024-05-07T02:34:35.732 INFO:tasks.workunit.client.0.smithi019.stdout: 1 50678 3.18 MB/sec warmup 91 sec latency 1855.485 ms 2024-05-07T02:34:36.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:34:36 smithi064 ceph-mon[34699]: pgmap v86: 129 pgs: 129 active+clean; 44 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 342 KiB/s wr, 71 op/s 2024-05-07T02:34:36.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:34:36 smithi019 ceph-mon[29781]: pgmap v86: 129 pgs: 129 active+clean; 44 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 342 KiB/s wr, 71 op/s 2024-05-07T02:34:36.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:34:36 smithi027 ceph-mon[34398]: pgmap v86: 129 pgs: 129 active+clean; 44 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 342 KiB/s wr, 71 op/s 2024-05-07T02:34:36.732 INFO:tasks.workunit.client.0.smithi019.stdout: 1 52272 3.22 MB/sec warmup 92 sec latency 16.267 ms 2024-05-07T02:34:37.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.449+0000 7f9fd5600700 1 -- 172.21.15.19:0/2836325674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fd0110ba0 msgr2=0x7f9fd0113000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:37.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.449+0000 7f9fd5600700 1 --2- 172.21.15.19:0/2836325674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fd0110ba0 0x7f9fd0113000 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9fc800b0a0 tx=0x7f9fc800b3b0 comp rx=0 tx=0).stop 2024-05-07T02:34:37.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.449+0000 7f9fd5600700 1 -- 172.21.15.19:0/2836325674 shutdown_connections 2024-05-07T02:34:37.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.449+0000 7f9fd5600700 1 --2- 172.21.15.19:0/2836325674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fd0110ba0 0x7f9fd0113000 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:37.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.449+0000 7f9fd5600700 1 --2- 172.21.15.19:0/2836325674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fd0075d30 0x7f9fd0076140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:37.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.449+0000 7f9fd5600700 1 --2- 172.21.15.19:0/2836325674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fd0076680 0x7f9fd0110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:37.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.449+0000 7f9fd5600700 1 -- 172.21.15.19:0/2836325674 >> 172.21.15.19:0/2836325674 conn(0x7f9fd0100040 msgr2=0x7f9fd0102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:34:37.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.449+0000 7f9fd5600700 1 -- 172.21.15.19:0/2836325674 shutdown_connections 2024-05-07T02:34:37.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.449+0000 7f9fd5600700 1 -- 172.21.15.19:0/2836325674 wait complete. 2024-05-07T02:34:37.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.450+0000 7f9fd5600700 1 Processor -- start 2024-05-07T02:34:37.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.450+0000 7f9fd5600700 1 -- start start 2024-05-07T02:34:37.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.451+0000 7f9fd5600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fd0075d30 0x7f9fd019d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:37.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.451+0000 7f9fd5600700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fd0076680 0x7f9fd019d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:37.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.451+0000 7f9fd5600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fd0110ba0 0x7f9fd01a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:37.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.451+0000 7f9fd5600700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd01166d0 con 0x7f9fd0075d30 2024-05-07T02:34:37.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.451+0000 7f9fd5600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd0116550 con 0x7f9fd0110ba0 2024-05-07T02:34:37.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.451+0000 7f9fd5600700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd0116850 con 0x7f9fd0076680 2024-05-07T02:34:37.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.452+0000 7f9fcf400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fd0110ba0 0x7f9fd01a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:37.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.452+0000 7f9fce000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fd0076680 0x7f9fd019d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:37.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.452+0000 7f9fcf400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fd0110ba0 0x7f9fd01a1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:34998/0 (socket says 172.21.15.19:34998) 2024-05-07T02:34:37.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.452+0000 7f9fcea00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fd0075d30 0x7f9fd019d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:37.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.452+0000 7f9fcf400700 1 -- 172.21.15.19:0/374392882 learned_addr learned my addr 172.21.15.19:0/374392882 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:34:37.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.452+0000 7f9fcf400700 1 -- 172.21.15.19:0/374392882 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fd0076680 msgr2=0x7f9fd019d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:37.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.452+0000 7f9fcf400700 1 --2- 172.21.15.19:0/374392882 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fd0076680 0x7f9fd019d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:37.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.452+0000 7f9fcf400700 1 -- 172.21.15.19:0/374392882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fd0075d30 msgr2=0x7f9fd019d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:37.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.452+0000 7f9fcf400700 1 --2- 172.21.15.19:0/374392882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fd0075d30 0x7f9fd019d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:37.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.452+0000 7f9fcf400700 1 -- 172.21.15.19:0/374392882 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fd01a1ef0 con 0x7f9fd0110ba0 2024-05-07T02:34:37.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.452+0000 7f9fce000700 1 --2- 172.21.15.19:0/374392882 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fd0076680 0x7f9fd019d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:34:37.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.453+0000 7f9fcf400700 1 --2- 172.21.15.19:0/374392882 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fd0110ba0 0x7f9fd01a1850 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9fc800b070 tx=0x7f9fc8004c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:34:37.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.453+0000 7f9fc7400700 1 -- 172.21.15.19:0/374392882 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc8021070 con 0x7f9fd0110ba0 2024-05-07T02:34:37.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.453+0000 7f9fc7400700 1 -- 172.21.15.19:0/374392882 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fc8007e30 con 0x7f9fd0110ba0 2024-05-07T02:34:37.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.453+0000 7f9fd5600700 1 -- 172.21.15.19:0/374392882 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fd01a2180 con 0x7f9fd0110ba0 2024-05-07T02:34:37.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.453+0000 7f9fc7400700 1 -- 172.21.15.19:0/374392882 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc80045e0 con 0x7f9fd0110ba0 2024-05-07T02:34:37.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.453+0000 7f9fd5600700 1 -- 172.21.15.19:0/374392882 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fd01a9b00 con 0x7f9fd0110ba0 2024-05-07T02:34:37.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.454+0000 7f9fd5600700 1 -- 172.21.15.19:0/374392882 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fd00772a0 con 0x7f9fd0110ba0 2024-05-07T02:34:37.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.457+0000 7f9fc7400700 1 -- 172.21.15.19:0/374392882 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9fc8052d10 con 0x7f9fd0110ba0 2024-05-07T02:34:37.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.459+0000 7f9fc7400700 1 --2- 172.21.15.19:0/374392882 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f9fb40b29f0 0x7f9fb40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:37.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.459+0000 7f9fc7400700 1 -- 172.21.15.19:0/374392882 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f9fc800f020 con 0x7f9fd0110ba0 2024-05-07T02:34:37.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.459+0000 7f9fcea00700 1 --2- 172.21.15.19:0/374392882 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f9fb40b29f0 0x7f9fb40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:37.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.460+0000 7f9fcea00700 1 --2- 172.21.15.19:0/374392882 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f9fb40b29f0 0x7f9fb40b4f00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0009c60 tx=0x7f9fc0009260 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:34:37.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.461+0000 7f9fc7400700 1 -- 172.21.15.19:0/374392882 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9fc802a020 con 0x7f9fd0110ba0 2024-05-07T02:34:37.732 INFO:tasks.workunit.client.0.smithi019.stdout: 1 53514 3.31 MB/sec warmup 93 sec latency 24.673 ms 2024-05-07T02:34:37.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.815+0000 7f9fd5600700 1 -- 172.21.15.19:0/374392882 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9fd019e380 con 0x7f9fd0110ba0 2024-05-07T02:34:37.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.816+0000 7f9fc7400700 1 -- 172.21.15.19:0/374392882 <== mon.2 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11931 (secure 0 0 0) 0x7f9fc80ab470 con 0x7f9fd0110ba0 2024-05-07T02:34:37.816 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:34:37.817 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1763670446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1763670446},{"type":"v1","addr":"172.21.15.19:6839","nonce":1763670446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/477529985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":477529985},{"type":"v1","addr":"172.21.15.64:6833","nonce":477529985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-07T02:26:14.960465+0000","modified":"2024-05-07T02:33:28.638165+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24539,"mds_2":24520,"mds_3":24515,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6833/2171873162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2171873162},{"type":"v1","addr":"172.21.15.27:6833","nonce":2171873162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/2379917839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2379917839},{"type":"v1","addr":"172.21.15.19:6835","nonce":2379917839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6837/2067515538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2067515538},{"type":"v1","addr":"172.21.15.19:6837","nonce":2067515538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6835/1000540600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1000540600},{"type":"v1","addr":"172.21.15.27:6835","nonce":1000540600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/1997318872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1997318872},{"type":"v1","addr":"172.21.15.64:6835","nonce":1997318872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":34,"addr":"172.21.15.64:6837/2585907777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2585907777},{"type":"v1","addr":"172.21.15.64:6837","nonce":2585907777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":33,"addr":"172.21.15.27:6837/2583138143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2583138143},{"type":"v1","addr":"172.21.15.27:6837","nonce":2583138143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":32,"addr":"172.21.15.64:6839/1583120004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":1583120004},{"type":"v1","addr":"172.21.15.64:6839","nonce":1583120004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":29,"addr":"172.21.15.19:6841/3905645246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3905645246},{"type":"v1","addr":"172.21.15.19:6841","nonce":3905645246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6839/3963268634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3963268634},{"type":"v1","addr":"172.21.15.27:6839","nonce":3963268634}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24551,24539,24520,24505,24515]},"id":1}]} 2024-05-07T02:34:37.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.820+0000 7f9fd5600700 1 -- 172.21.15.19:0/374392882 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f9fb40b29f0 msgr2=0x7f9fb40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:37.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.820+0000 7f9fd5600700 1 --2- 172.21.15.19:0/374392882 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f9fb40b29f0 0x7f9fb40b4f00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0009c60 tx=0x7f9fc0009260 comp rx=0 tx=0).stop 2024-05-07T02:34:37.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.820+0000 7f9fd5600700 1 -- 172.21.15.19:0/374392882 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fd0110ba0 msgr2=0x7f9fd01a1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:37.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.820+0000 7f9fd5600700 1 --2- 172.21.15.19:0/374392882 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fd0110ba0 0x7f9fd01a1850 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9fc800b070 tx=0x7f9fc8004c60 comp rx=0 tx=0).stop 2024-05-07T02:34:37.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.820+0000 7f9fd5600700 1 -- 172.21.15.19:0/374392882 shutdown_connections 2024-05-07T02:34:37.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.820+0000 7f9fd5600700 1 --2- 172.21.15.19:0/374392882 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fd0110ba0 0x7f9fd01a1850 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:37.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.820+0000 7f9fd5600700 1 --2- 172.21.15.19:0/374392882 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f9fb40b29f0 0x7f9fb40b4f00 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:37.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.820+0000 7f9fd5600700 1 --2- 172.21.15.19:0/374392882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fd0075d30 0x7f9fd019d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:37.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.820+0000 7f9fd5600700 1 --2- 172.21.15.19:0/374392882 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9fd0076680 0x7f9fd019d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:37.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.820+0000 7f9fd5600700 1 -- 172.21.15.19:0/374392882 >> 172.21.15.19:0/374392882 conn(0x7f9fd0100040 msgr2=0x7f9fd0100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:34:37.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.821+0000 7f9fd5600700 1 -- 172.21.15.19:0/374392882 shutdown_connections 2024-05-07T02:34:37.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:37.822+0000 7f9fd5600700 1 -- 172.21.15.19:0/374392882 wait complete. 2024-05-07T02:34:37.823 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 34 2024-05-07T02:34:38.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:34:37 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/374392882' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:34:38.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:34:37 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/374392882' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:34:38.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:34:37 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/374392882' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:34:38.653 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-07T02:34:38.732 INFO:tasks.workunit.client.0.smithi019.stdout: 1 53903 3.28 MB/sec warmup 94 sec latency 306.280 ms 2024-05-07T02:34:39.126 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:34:39.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:34:38 smithi019 ceph-mon[29781]: pgmap v87: 129 pgs: 129 active+clean; 44 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 307 KiB/s wr, 60 op/s 2024-05-07T02:34:39.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:34:38 smithi027 ceph-mon[34398]: pgmap v87: 129 pgs: 129 active+clean; 44 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 307 KiB/s wr, 60 op/s 2024-05-07T02:34:39.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:34:38 smithi064 ceph-mon[34699]: pgmap v87: 129 pgs: 129 active+clean; 44 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 307 KiB/s wr, 60 op/s 2024-05-07T02:34:39.732 INFO:tasks.workunit.client.0.smithi019.stdout: 1 53903 3.24 MB/sec warmup 95 sec latency 1306.398 ms 2024-05-07T02:34:40.732 INFO:tasks.workunit.client.0.smithi019.stdout: 1 54027 3.21 MB/sec warmup 96 sec latency 2071.460 ms 2024-05-07T02:34:41.251 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:34:40 smithi019 ceph-mon[29781]: pgmap v88: 129 pgs: 129 active+clean; 44 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 307 KiB/s wr, 60 op/s 2024-05-07T02:34:41.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:34:40 smithi064 ceph-mon[34699]: pgmap v88: 129 pgs: 129 active+clean; 44 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 307 KiB/s wr, 60 op/s 2024-05-07T02:34:41.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:34:40 smithi027 ceph-mon[34398]: pgmap v88: 129 pgs: 129 active+clean; 44 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 307 KiB/s wr, 60 op/s 2024-05-07T02:34:41.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.651+0000 7fa124600700 1 -- 172.21.15.19:0/3218353113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa11c115370 msgr2=0x7fa11c117790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:41.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.651+0000 7fa124600700 1 --2- 172.21.15.19:0/3218353113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa11c115370 0x7fa11c117790 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa11400b0a0 tx=0x7fa11400b3b0 comp rx=0 tx=0).stop 2024-05-07T02:34:41.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.651+0000 7fa124600700 1 -- 172.21.15.19:0/3218353113 shutdown_connections 2024-05-07T02:34:41.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.651+0000 7fa124600700 1 --2- 172.21.15.19:0/3218353113 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa11c115370 0x7fa11c117790 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:41.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.651+0000 7fa124600700 1 --2- 172.21.15.19:0/3218353113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa11c075b60 0x7fa11c075f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:41.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.651+0000 7fa124600700 1 --2- 172.21.15.19:0/3218353113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa11c0764b0 0x7fa11c114e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:41.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.651+0000 7fa124600700 1 -- 172.21.15.19:0/3218353113 >> 172.21.15.19:0/3218353113 conn(0x7fa11c100040 msgr2=0x7fa11c102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:34:41.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.651+0000 7fa124600700 1 -- 172.21.15.19:0/3218353113 shutdown_connections 2024-05-07T02:34:41.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.651+0000 7fa124600700 1 -- 172.21.15.19:0/3218353113 wait complete. 2024-05-07T02:34:41.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.653+0000 7fa124600700 1 Processor -- start 2024-05-07T02:34:41.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.653+0000 7fa124600700 1 -- start start 2024-05-07T02:34:41.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.653+0000 7fa124600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa11c075b60 0x7fa11c1a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:41.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.653+0000 7fa124600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa11c0764b0 0x7fa11c1a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:41.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.653+0000 7fa124600700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa11c115370 0x7fa11c1a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:41.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.653+0000 7fa124600700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa11c11aad0 con 0x7fa11c075b60 2024-05-07T02:34:41.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.653+0000 7fa124600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa11c11a950 con 0x7fa11c0764b0 2024-05-07T02:34:41.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.653+0000 7fa124600700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa11c11ac50 con 0x7fa11c115370 2024-05-07T02:34:41.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.654+0000 7fa121c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa11c075b60 0x7fa11c1a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:41.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.654+0000 7fa121c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa11c075b60 0x7fa11c1a13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46264/0 (socket says 172.21.15.19:46264) 2024-05-07T02:34:41.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.654+0000 7fa121c00700 1 -- 172.21.15.19:0/3332193153 learned_addr learned my addr 172.21.15.19:0/3332193153 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:34:41.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.654+0000 7fa121200700 1 --2- 172.21.15.19:0/3332193153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa11c0764b0 0x7fa11c1a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:41.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.654+0000 7fa122600700 1 --2- 172.21.15.19:0/3332193153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa11c115370 0x7fa11c1a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:41.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.654+0000 7fa121c00700 1 -- 172.21.15.19:0/3332193153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa11c115370 msgr2=0x7fa11c1a5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:41.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.654+0000 7fa121c00700 1 --2- 172.21.15.19:0/3332193153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa11c115370 0x7fa11c1a5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:41.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.654+0000 7fa121c00700 1 -- 172.21.15.19:0/3332193153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa11c0764b0 msgr2=0x7fa11c1a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:41.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.654+0000 7fa121c00700 1 --2- 172.21.15.19:0/3332193153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa11c0764b0 0x7fa11c1a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:41.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.654+0000 7fa121c00700 1 -- 172.21.15.19:0/3332193153 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa11c1a6290 con 0x7fa11c075b60 2024-05-07T02:34:41.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.654+0000 7fa121200700 1 --2- 172.21.15.19:0/3332193153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa11c0764b0 0x7fa11c1a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:34:41.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.654+0000 7fa122600700 1 --2- 172.21.15.19:0/3332193153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa11c115370 0x7fa11c1a5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:34:41.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.655+0000 7fa121c00700 1 --2- 172.21.15.19:0/3332193153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa11c075b60 0x7fa11c1a13c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa11000e9a0 tx=0x7fa11000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:34:41.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.655+0000 7fa11a000700 1 -- 172.21.15.19:0/3332193153 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11000ca10 con 0x7fa11c075b60 2024-05-07T02:34:41.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.655+0000 7fa124600700 1 -- 172.21.15.19:0/3332193153 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa11c1a6580 con 0x7fa11c075b60 2024-05-07T02:34:41.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.655+0000 7fa11a000700 1 -- 172.21.15.19:0/3332193153 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa11000cbb0 con 0x7fa11c075b60 2024-05-07T02:34:41.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.655+0000 7fa11a000700 1 -- 172.21.15.19:0/3332193153 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa110018680 con 0x7fa11c075b60 2024-05-07T02:34:41.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.655+0000 7fa124600700 1 -- 172.21.15.19:0/3332193153 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa11c1adea0 con 0x7fa11c075b60 2024-05-07T02:34:41.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.659+0000 7fa11a000700 1 -- 172.21.15.19:0/3332193153 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa110010430 con 0x7fa11c075b60 2024-05-07T02:34:41.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.660+0000 7fa13319cb80 1 Processor -- start 2024-05-07T02:34:41.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.661+0000 7fa13319cb80 1 -- start start 2024-05-07T02:34:41.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.661+0000 7fa11a000700 1 --2- 172.21.15.19:0/3332193153 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fa1040b29f0 0x7fa1040b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:41.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.661+0000 7fa13319cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c2f52470 0x55a8c2f52880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:41.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.661+0000 7fa13319cb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c2f4c620 0x55a8c2f4ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:41.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.661+0000 7fa13319cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c2f4cf70 0x55a8c2f59150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:41.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.661+0000 7fa13319cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a8c2e6ea20 con 0x55a8c2f52470 2024-05-07T02:34:41.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.661+0000 7fa13319cb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a8c2e0c760 con 0x55a8c2f4c620 2024-05-07T02:34:41.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.661+0000 7fa13319cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a8c2e6a9e0 con 0x55a8c2f4cf70 2024-05-07T02:34:41.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.661+0000 7fa11a000700 1 -- 172.21.15.19:0/3332193153 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fa1100e2b60 con 0x7fa11c075b60 2024-05-07T02:34:41.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.662+0000 7fa122600700 1 --2- 172.21.15.19:0/3332193153 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fa1040b29f0 0x7fa1040b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:41.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.662+0000 7fa121c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c2f52470 0x55a8c2f52880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:41.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.662+0000 7fa121200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c2f4c620 0x55a8c2f4ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:41.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.662+0000 7fa121c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c2f52470 0x55a8c2f52880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46278/0 (socket says 172.21.15.19:46278) 2024-05-07T02:34:41.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.662+0000 7fa121c00700 1 -- 172.21.15.19:0/967406542 learned_addr learned my addr 172.21.15.19:0/967406542 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:34:41.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.662+0000 7fa122600700 1 --2- 172.21.15.19:0/967406542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c2f4cf70 0x55a8c2f59150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:41.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.662+0000 7fa121200700 1 -- 172.21.15.19:0/967406542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c2f4cf70 msgr2=0x55a8c2f59150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:41.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.662+0000 7fa121200700 1 --2- 172.21.15.19:0/967406542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c2f4cf70 0x55a8c2f59150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:41.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.662+0000 7fa121200700 1 -- 172.21.15.19:0/967406542 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c2f52470 msgr2=0x55a8c2f52880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:41.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.662+0000 7fa121200700 1 --2- 172.21.15.19:0/967406542 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c2f52470 0x55a8c2f52880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:41.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.662+0000 7fa121200700 1 -- 172.21.15.19:0/967406542 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8c2dbb8d0 con 0x55a8c2f4c620 2024-05-07T02:34:41.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.663+0000 7fa121200700 1 --2- 172.21.15.19:0/967406542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c2f4c620 0x55a8c2f4ca30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa108009fb0 tx=0x7fa108011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b074e7a6d47b910 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:34:41.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.663+0000 7fa10ea00700 1 -- 172.21.15.19:0/967406542 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa108004230 con 0x55a8c2f4c620 2024-05-07T02:34:41.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.663+0000 7fa10ea00700 1 -- 172.21.15.19:0/967406542 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa108004ce0 con 0x55a8c2f4c620 2024-05-07T02:34:41.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.663+0000 7fa122600700 1 --2- 172.21.15.19:0/3332193153 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fa1040b29f0 0x7fa1040b4f00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa11400b780 tx=0x7fa11400bf60 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:34:41.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.663+0000 7fa10ea00700 1 -- 172.21.15.19:0/967406542 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa108003680 con 0x55a8c2f4c620 2024-05-07T02:34:41.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.664+0000 7fa13319cb80 1 -- 172.21.15.19:0/967406542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c2f4c620 msgr2=0x55a8c2f4ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:41.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.664+0000 7fa13319cb80 1 --2- 172.21.15.19:0/967406542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c2f4c620 0x55a8c2f4ca30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa108009fb0 tx=0x7fa108011720 comp rx=0 tx=0).stop 2024-05-07T02:34:41.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.664+0000 7fa13319cb80 1 -- 172.21.15.19:0/967406542 shutdown_connections 2024-05-07T02:34:41.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.664+0000 7fa13319cb80 1 --2- 172.21.15.19:0/967406542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c2f4cf70 0x55a8c2f59150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:41.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.664+0000 7fa13319cb80 1 --2- 172.21.15.19:0/967406542 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c2f52470 0x55a8c2f52880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:41.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.664+0000 7fa13319cb80 1 --2- 172.21.15.19:0/967406542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c2f4c620 0x55a8c2f4ca30 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:41.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.664+0000 7fa13319cb80 1 -- 172.21.15.19:0/967406542 >> 172.21.15.19:0/967406542 conn(0x55a8c2dcee40 msgr2=0x55a8c2f47380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:34:41.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.664+0000 7fa13319cb80 1 -- 172.21.15.19:0/967406542 shutdown_connections 2024-05-07T02:34:41.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.664+0000 7fa13319cb80 1 -- 172.21.15.19:0/967406542 wait complete. 2024-05-07T02:34:41.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.665+0000 7fa13319cb80 1 -- start start 2024-05-07T02:34:41.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.665+0000 7fa13319cb80 1 Processor -- start 2024-05-07T02:34:41.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.666+0000 7fa13319cb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c2f4c620 0x55a8c2f565a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:41.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.666+0000 7fa13319cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c2f4cf70 0x55a8c2f56ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:41.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.666+0000 7fa13319cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c2f52470 0x55a8c2f57020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:41.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.666+0000 7fa13319cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a8c2f59840 con 0x55a8c2f52470 2024-05-07T02:34:41.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.666+0000 7fa13319cb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a8c2dd4b20 con 0x55a8c2f4c620 2024-05-07T02:34:41.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.666+0000 7fa13319cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a8c2f599c0 con 0x55a8c2f4cf70 2024-05-07T02:34:41.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.666+0000 7fa13319cb80 10 client.0 _ll_register_callbacks cb 0x55a8c2e02380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:34:41.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.666+0000 7fa121c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c2f4c620 0x55a8c2f565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:41.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.666+0000 7fa121200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c2f4cf70 0x55a8c2f56ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:41.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.666+0000 7fa121c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c2f4c620 0x55a8c2f565a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:59260/0 (socket says 172.21.15.19:59260) 2024-05-07T02:34:41.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.666+0000 7fa121200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c2f4cf70 0x55a8c2f56ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:60718/0 (socket says 172.21.15.19:60718) 2024-05-07T02:34:41.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.666+0000 7fa121c00700 1 -- 172.21.15.19:0/2957210262 learned_addr learned my addr 172.21.15.19:0/2957210262 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:34:41.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.666+0000 7fa122600700 1 --2- 172.21.15.19:0/2957210262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c2f52470 0x55a8c2f57020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:41.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.667+0000 7fa121c00700 1 -- 172.21.15.19:0/2957210262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c2f4cf70 msgr2=0x55a8c2f56ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:41.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.667+0000 7fa121c00700 1 --2- 172.21.15.19:0/2957210262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c2f4cf70 0x55a8c2f56ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:41.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.667+0000 7fa121c00700 1 -- 172.21.15.19:0/2957210262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c2f52470 msgr2=0x55a8c2f57020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:41.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.667+0000 7fa121c00700 1 --2- 172.21.15.19:0/2957210262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c2f52470 0x55a8c2f57020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:41.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.667+0000 7fa121c00700 1 -- 172.21.15.19:0/2957210262 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8c2e34420 con 0x55a8c2f4c620 2024-05-07T02:34:41.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.667+0000 7fa121200700 1 --2- 172.21.15.19:0/2957210262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c2f4cf70 0x55a8c2f56ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:34:41.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.667+0000 7fa121c00700 1 --2- 172.21.15.19:0/2957210262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c2f4c620 0x55a8c2f565a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa1100098c0 tx=0x7fa1100a9f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ace590cb7376fd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:34:41.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.667+0000 7fa103400700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:34:41.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.667+0000 7fa103400700 1 -- 172.21.15.19:0/2957210262 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1100a5040 con 0x55a8c2f4c620 2024-05-07T02:34:41.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.667+0000 7fa103400700 1 -- 172.21.15.19:0/2957210262 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fa1100cc2d0 con 0x55a8c2f4c620 2024-05-07T02:34:41.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.667+0000 7fa13319cb80 1 -- 172.21.15.19:0/2957210262 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa108004ce0 con 0x55a8c2f4c620 2024-05-07T02:34:41.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.668+0000 7fa103400700 1 -- 172.21.15.19:0/2957210262 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa1100b21f0 con 0x55a8c2f4c620 2024-05-07T02:34:41.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.668+0000 7fa13319cb80 10 client.25748 fetch_fsmap learned FSMap version 34 2024-05-07T02:34:41.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.668+0000 7fa13319cb80 1 -- 172.21.15.19:0/2957210262 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a8c2f64490 con 0x55a8c2f4c620 2024-05-07T02:34:41.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.668+0000 7fa103400700 1 -- 172.21.15.19:0/2957210262 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa108004ce0 con 0x55a8c2f4c620 2024-05-07T02:34:41.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.668+0000 7fa103400700 1 -- 172.21.15.19:0/2957210262 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7fa110010720 con 0x55a8c2f4c620 2024-05-07T02:34:41.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.668+0000 7fa13319cb80 10 client.25748 fetch_fsmap finished waiting for FSMap version 34 2024-05-07T02:34:41.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.668+0000 7fa13319cb80 10 client.25748 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-07T02:34:41.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.668+0000 7fa13319cb80 20 client.25748 populate_metadata read hostname 'smithi019' 2024-05-07T02:34:41.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.669+0000 7fa13319cb80 1 --2- 172.21.15.19:0/2957210262 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x55a8c2f66ad0 0x55a8c2f68ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:41.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.669+0000 7fa13319cb80 4 client.25748 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:34:41.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.669+0000 7fa13319cb80 1 -- 172.21.15.19:0/2957210262 --> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8c2f190e0 con 0x55a8c2f66ad0 2024-05-07T02:34:41.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.669+0000 7fa121200700 1 --2- 172.21.15.19:0/2957210262 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x55a8c2f66ad0 0x55a8c2f68ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:41.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.670+0000 7fa121200700 1 --2- 172.21.15.19:0/2957210262 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x55a8c2f66ad0 0x55a8c2f68ef0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=93f1a9906da25891 server_cookie=c880fcf676bea0e9 in_seq=0 out_seq=0 2024-05-07T02:34:41.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.670+0000 7fa103400700 10 client.25748 ms_handle_connect on v2:172.21.15.64:6836/2585907777 2024-05-07T02:34:41.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.670+0000 7fa103400700 1 -- 172.21.15.19:0/2957210262 <== mds.4 v2:172.21.15.64:6836/2585907777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a8c2f190e0 con 0x55a8c2f66ad0 2024-05-07T02:34:41.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.670+0000 7fa103400700 10 client.25748 handle_command_reply: tid=0 2024-05-07T02:34:41.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.671+0000 7fa13319cb80 1 client.25748 shutdown 2024-05-07T02:34:41.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.671+0000 7fa13319cb80 1 -- 172.21.15.19:0/2957210262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c2f4c620 msgr2=0x55a8c2f565a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:41.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.671+0000 7fa13319cb80 1 --2- 172.21.15.19:0/2957210262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c2f4c620 0x55a8c2f565a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa1100098c0 tx=0x7fa1100a9f90 comp rx=0 tx=0).stop 2024-05-07T02:34:41.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.671+0000 7fa13319cb80 1 -- 172.21.15.19:0/2957210262 shutdown_connections 2024-05-07T02:34:41.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.671+0000 7fa13319cb80 1 --2- 172.21.15.19:0/2957210262 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x55a8c2f66ad0 0x55a8c2f68ef0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:41.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.671+0000 7fa13319cb80 1 --2- 172.21.15.19:0/2957210262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c2f52470 0x55a8c2f57020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:41.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.671+0000 7fa13319cb80 1 --2- 172.21.15.19:0/2957210262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c2f4c620 0x55a8c2f565a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:41.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.671+0000 7fa13319cb80 1 --2- 172.21.15.19:0/2957210262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c2f4cf70 0x55a8c2f56ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:41.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.671+0000 7fa13319cb80 1 -- 172.21.15.19:0/2957210262 >> 172.21.15.19:0/2957210262 conn(0x55a8c2dcee40 msgr2=0x55a8c2f4b610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:34:41.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.671+0000 7fa103400700 0 client.25748 ms_handle_reset on v2:172.21.15.64:6836/2585907777 2024-05-07T02:34:41.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.671+0000 7fa13319cb80 1 -- 172.21.15.19:0/2957210262 shutdown_connections 2024-05-07T02:34:41.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.671+0000 7fa13319cb80 1 -- 172.21.15.19:0/2957210262 wait complete. 2024-05-07T02:34:41.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.671+0000 7fa13319cb80 20 client.25748 trim_cache size 0 max 16384 2024-05-07T02:34:41.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.689+0000 7fa13319cb80 1 Processor -- start 2024-05-07T02:34:41.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.689+0000 7fa13319cb80 1 -- start start 2024-05-07T02:34:41.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.689+0000 7fa13319cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c2f4c620 0x55a8c2f47410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:41.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.689+0000 7fa13319cb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c2f4cf70 0x55a8c2f56c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:41.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.689+0000 7fa13319cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c2f52470 0x55a8c2f53eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:41.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.689+0000 7fa13319cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa11c1b6b80 con 0x55a8c2f52470 2024-05-07T02:34:41.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.689+0000 7fa13319cb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa11c1b8780 con 0x55a8c2f4cf70 2024-05-07T02:34:41.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.689+0000 7fa13319cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa11c1b7250 con 0x55a8c2f4c620 2024-05-07T02:34:41.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.690+0000 7fa121c00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c2f4c620 0x55a8c2f47410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:41.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.690+0000 7fa121c00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c2f4c620 0x55a8c2f47410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:60732/0 (socket says 172.21.15.19:60732) 2024-05-07T02:34:41.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.690+0000 7fa122600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c2f52470 0x55a8c2f53eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:41.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.690+0000 7fa121c00700 1 -- 172.21.15.19:0/3709313019 learned_addr learned my addr 172.21.15.19:0/3709313019 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:34:41.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.690+0000 7fa121200700 1 --2- 172.21.15.19:0/3709313019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c2f4cf70 0x55a8c2f56c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:41.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.690+0000 7fa121c00700 1 -- 172.21.15.19:0/3709313019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c2f4cf70 msgr2=0x55a8c2f56c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:41.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.690+0000 7fa121c00700 1 --2- 172.21.15.19:0/3709313019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c2f4cf70 0x55a8c2f56c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:41.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.690+0000 7fa121c00700 1 -- 172.21.15.19:0/3709313019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c2f52470 msgr2=0x55a8c2f53eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:41.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.690+0000 7fa121c00700 1 --2- 172.21.15.19:0/3709313019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c2f52470 0x55a8c2f53eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:41.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.690+0000 7fa121c00700 1 -- 172.21.15.19:0/3709313019 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8c2f56370 con 0x55a8c2f4c620 2024-05-07T02:34:41.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.690+0000 7fa121c00700 1 --2- 172.21.15.19:0/3709313019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c2f4c620 0x55a8c2f47410 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa110000c00 tx=0x7fa1100a5310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2db65bc607ad8b6c server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:34:41.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.691+0000 7fa10e000700 1 -- 172.21.15.19:0/3709313019 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1100aa150 con 0x55a8c2f4c620 2024-05-07T02:34:41.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.691+0000 7fa10e000700 1 -- 172.21.15.19:0/3709313019 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1100b2040 con 0x55a8c2f4c620 2024-05-07T02:34:41.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.691+0000 7fa10e000700 1 -- 172.21.15.19:0/3709313019 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11006f2b0 con 0x55a8c2f4c620 2024-05-07T02:34:41.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.692+0000 7fa13319cb80 1 -- 172.21.15.19:0/3709313019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c2f4c620 msgr2=0x55a8c2f47410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:41.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.692+0000 7fa13319cb80 1 --2- 172.21.15.19:0/3709313019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c2f4c620 0x55a8c2f47410 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa110000c00 tx=0x7fa1100a5310 comp rx=0 tx=0).stop 2024-05-07T02:34:41.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.692+0000 7fa13319cb80 1 -- 172.21.15.19:0/3709313019 shutdown_connections 2024-05-07T02:34:41.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.692+0000 7fa13319cb80 1 --2- 172.21.15.19:0/3709313019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c2f52470 0x55a8c2f53eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:41.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.692+0000 7fa13319cb80 1 --2- 172.21.15.19:0/3709313019 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c2f4c620 0x55a8c2f47410 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:41.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.692+0000 7fa13319cb80 1 --2- 172.21.15.19:0/3709313019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c2f4cf70 0x55a8c2f56c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:41.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.692+0000 7fa13319cb80 1 -- 172.21.15.19:0/3709313019 >> 172.21.15.19:0/3709313019 conn(0x55a8c2dcee40 msgr2=0x55a8c2f56560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:34:41.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.692+0000 7fa13319cb80 1 -- 172.21.15.19:0/3709313019 shutdown_connections 2024-05-07T02:34:41.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.692+0000 7fa13319cb80 1 -- 172.21.15.19:0/3709313019 wait complete. 2024-05-07T02:34:41.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.693+0000 7fa13319cb80 1 -- start start 2024-05-07T02:34:41.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.693+0000 7fa13319cb80 1 Processor -- start 2024-05-07T02:34:41.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.694+0000 7fa13319cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c2f4c620 0x55a8c2f5ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:41.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.694+0000 7fa13319cb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c2f4cf70 0x55a8c2f5f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:41.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.694+0000 7fa13319cb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c2f52470 0x55a8c2f6d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:41.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.694+0000 7fa13319cb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa11c1b7ff0 con 0x55a8c2f4cf70 2024-05-07T02:34:41.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.694+0000 7fa13319cb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa11c1b7920 con 0x55a8c2f52470 2024-05-07T02:34:41.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.694+0000 7fa13319cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa11c1b4b20 con 0x55a8c2f4c620 2024-05-07T02:34:41.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.694+0000 7fa13319cb80 10 client.0 _ll_register_callbacks cb 0x55a8c2e02380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:34:41.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.694+0000 7fa121c00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c2f4c620 0x55a8c2f5ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:41.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.694+0000 7fa121c00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c2f4c620 0x55a8c2f5ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:60734/0 (socket says 172.21.15.19:60734) 2024-05-07T02:34:41.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.694+0000 7fa121c00700 1 -- 172.21.15.19:0/2717107762 learned_addr learned my addr 172.21.15.19:0/2717107762 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:34:41.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.694+0000 7fa121200700 1 --2- 172.21.15.19:0/2717107762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c2f4cf70 0x55a8c2f5f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:41.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.694+0000 7fa122600700 1 --2- 172.21.15.19:0/2717107762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c2f52470 0x55a8c2f6d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:41.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.695+0000 7fa121c00700 1 -- 172.21.15.19:0/2717107762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c2f52470 msgr2=0x55a8c2f6d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:41.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.695+0000 7fa121c00700 1 --2- 172.21.15.19:0/2717107762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c2f52470 0x55a8c2f6d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:41.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.695+0000 7fa121c00700 1 -- 172.21.15.19:0/2717107762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c2f4cf70 msgr2=0x55a8c2f5f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:41.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.695+0000 7fa121c00700 1 --2- 172.21.15.19:0/2717107762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c2f4cf70 0x55a8c2f5f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:41.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.695+0000 7fa121c00700 1 -- 172.21.15.19:0/2717107762 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8c2f6f010 con 0x55a8c2f4c620 2024-05-07T02:34:41.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.695+0000 7fa121c00700 1 --2- 172.21.15.19:0/2717107762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c2f4c620 0x55a8c2f5ec80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa110010b20 tx=0x7fa1100a9f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=754b80368a70b3d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:34:41.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.695+0000 7fa103400700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:34:41.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.695+0000 7fa103400700 1 -- 172.21.15.19:0/2717107762 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa110086070 con 0x55a8c2f4c620 2024-05-07T02:34:41.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.695+0000 7fa103400700 1 -- 172.21.15.19:0/2717107762 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fa110010720 con 0x55a8c2f4c620 2024-05-07T02:34:41.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.695+0000 7fa13319cb80 1 -- 172.21.15.19:0/2717107762 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1100b2040 con 0x55a8c2f4c620 2024-05-07T02:34:41.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.695+0000 7fa103400700 1 -- 172.21.15.19:0/2717107762 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa1100ca070 con 0x55a8c2f4c620 2024-05-07T02:34:41.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.695+0000 7fa13319cb80 10 client.25741 fetch_fsmap learned FSMap version 34 2024-05-07T02:34:41.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.695+0000 7fa13319cb80 1 -- 172.21.15.19:0/2717107762 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a8c2fd0720 con 0x55a8c2f4c620 2024-05-07T02:34:41.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.696+0000 7fa103400700 1 -- 172.21.15.19:0/2717107762 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1100b2040 con 0x55a8c2f4c620 2024-05-07T02:34:41.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.696+0000 7fa103400700 1 -- 172.21.15.19:0/2717107762 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7fa110089030 con 0x55a8c2f4c620 2024-05-07T02:34:41.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.696+0000 7fa13319cb80 10 client.25741 fetch_fsmap finished waiting for FSMap version 34 2024-05-07T02:34:41.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.696+0000 7fa13319cb80 10 client.25741 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-07T02:34:41.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.696+0000 7fa13319cb80 20 client.25741 populate_metadata read hostname 'smithi019' 2024-05-07T02:34:41.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.696+0000 7fa13319cb80 1 --2- 172.21.15.19:0/2717107762 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x55a8c2fd0b40 0x55a8c2fd2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:41.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.696+0000 7fa13319cb80 4 client.25741 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:34:41.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.696+0000 7fa13319cb80 1 -- 172.21.15.19:0/2717107762 --> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a8c2e7c400 con 0x55a8c2fd0b40 2024-05-07T02:34:41.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.697+0000 7fa121200700 1 --2- 172.21.15.19:0/2717107762 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x55a8c2fd0b40 0x55a8c2fd2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:41.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.697+0000 7fa121200700 1 --2- 172.21.15.19:0/2717107762 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x55a8c2fd0b40 0x55a8c2fd2f60 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=83c8d8605ec9fe78 server_cookie=1e98601751382e54 in_seq=0 out_seq=0 2024-05-07T02:34:41.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.697+0000 7fa103400700 10 client.25741 ms_handle_connect on v2:172.21.15.64:6836/2585907777 2024-05-07T02:34:41.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.697+0000 7fa103400700 1 -- 172.21.15.19:0/2717107762 <== mds.4 v2:172.21.15.64:6836/2585907777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a8c2e7c400 con 0x55a8c2fd0b40 2024-05-07T02:34:41.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.697+0000 7fa103400700 10 client.25741 handle_command_reply: tid=0 2024-05-07T02:34:41.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.697+0000 7fa13319cb80 1 client.25741 shutdown 2024-05-07T02:34:41.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.698+0000 7fa13319cb80 1 -- 172.21.15.19:0/2717107762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c2f4c620 msgr2=0x55a8c2f5ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:41.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.698+0000 7fa13319cb80 1 --2- 172.21.15.19:0/2717107762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c2f4c620 0x55a8c2f5ec80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa110010b20 tx=0x7fa1100a9f90 comp rx=0 tx=0).stop 2024-05-07T02:34:41.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.698+0000 7fa13319cb80 1 -- 172.21.15.19:0/2717107762 shutdown_connections 2024-05-07T02:34:41.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.698+0000 7fa13319cb80 1 --2- 172.21.15.19:0/2717107762 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x55a8c2fd0b40 0x55a8c2fd2f60 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:41.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.698+0000 7fa13319cb80 1 --2- 172.21.15.19:0/2717107762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c2f52470 0x55a8c2f6d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:41.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.698+0000 7fa13319cb80 1 --2- 172.21.15.19:0/2717107762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c2f4c620 0x55a8c2f5ec80 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:41.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.698+0000 7fa13319cb80 1 --2- 172.21.15.19:0/2717107762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c2f4cf70 0x55a8c2f5f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:41.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.698+0000 7fa103400700 0 client.25741 ms_handle_reset on v2:172.21.15.64:6836/2585907777 2024-05-07T02:34:41.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.698+0000 7fa13319cb80 1 -- 172.21.15.19:0/2717107762 >> 172.21.15.19:0/2717107762 conn(0x55a8c2dcee40 msgr2=0x55a8c2f56560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:34:41.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.698+0000 7fa13319cb80 1 -- 172.21.15.19:0/2717107762 shutdown_connections 2024-05-07T02:34:41.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.698+0000 7fa13319cb80 1 -- 172.21.15.19:0/2717107762 wait complete. 2024-05-07T02:34:41.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.698+0000 7fa13319cb80 20 client.25741 trim_cache size 0 max 16384 2024-05-07T02:34:41.700 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:34:41.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.699+0000 7fa124600700 1 -- 172.21.15.19:0/3332193153 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fa1040b29f0 msgr2=0x7fa1040b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:41.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.699+0000 7fa124600700 1 --2- 172.21.15.19:0/3332193153 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fa1040b29f0 0x7fa1040b4f00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa11400b780 tx=0x7fa11400bf60 comp rx=0 tx=0).stop 2024-05-07T02:34:41.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.699+0000 7fa124600700 1 -- 172.21.15.19:0/3332193153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa11c075b60 msgr2=0x7fa11c1a13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:41.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.699+0000 7fa124600700 1 --2- 172.21.15.19:0/3332193153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa11c075b60 0x7fa11c1a13c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa11000e9a0 tx=0x7fa11000ecb0 comp rx=0 tx=0).stop 2024-05-07T02:34:41.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.699+0000 7fa124600700 1 -- 172.21.15.19:0/3332193153 shutdown_connections 2024-05-07T02:34:41.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.699+0000 7fa124600700 1 --2- 172.21.15.19:0/3332193153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa11c115370 0x7fa11c1a5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:41.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.699+0000 7fa124600700 1 --2- 172.21.15.19:0/3332193153 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fa1040b29f0 0x7fa1040b4f00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:41.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.699+0000 7fa124600700 1 --2- 172.21.15.19:0/3332193153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa11c075b60 0x7fa11c1a13c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:41.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.699+0000 7fa124600700 1 --2- 172.21.15.19:0/3332193153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa11c0764b0 0x7fa11c1a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:41.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.699+0000 7fa124600700 1 -- 172.21.15.19:0/3332193153 >> 172.21.15.19:0/3332193153 conn(0x7fa11c100040 msgr2=0x7fa11c1005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:34:41.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.699+0000 7fa124600700 1 -- 172.21.15.19:0/3332193153 shutdown_connections 2024-05-07T02:34:41.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:41.699+0000 7fa124600700 1 -- 172.21.15.19:0/3332193153 wait complete. 2024-05-07T02:34:41.732 INFO:tasks.workunit.client.0.smithi019.stdout: 1 54027 3.18 MB/sec warmup 97 sec latency 1062.764 ms 2024-05-07T02:34:42.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:34:42 smithi064 ceph-mon[34699]: pgmap v89: 129 pgs: 129 active+clean; 45 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 716 KiB/s wr, 104 op/s 2024-05-07T02:34:42.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:34:42 smithi019 ceph-mon[29781]: pgmap v89: 129 pgs: 129 active+clean; 45 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 716 KiB/s wr, 104 op/s 2024-05-07T02:34:42.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:34:42 smithi027 ceph-mon[34398]: pgmap v89: 129 pgs: 129 active+clean; 45 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 716 KiB/s wr, 104 op/s 2024-05-07T02:34:42.705 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-07T02:34:42.732 INFO:tasks.workunit.client.0.smithi019.stdout: 1 54027 3.14 MB/sec warmup 98 sec latency 2062.848 ms 2024-05-07T02:34:43.145 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:34:43.732 INFO:tasks.workunit.client.0.smithi019.stdout: 1 54027 3.11 MB/sec warmup 99 sec latency 3062.889 ms 2024-05-07T02:34:44.340 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:34:44 smithi019 ceph-mon[29781]: pgmap v90: 129 pgs: 129 active+clean; 45 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 640 KiB/s wr, 91 op/s 2024-05-07T02:34:44.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:34:44 smithi064 ceph-mon[34699]: pgmap v90: 129 pgs: 129 active+clean; 45 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 640 KiB/s wr, 91 op/s 2024-05-07T02:34:44.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:34:44 smithi027 ceph-mon[34398]: pgmap v90: 129 pgs: 129 active+clean; 45 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 640 KiB/s wr, 91 op/s 2024-05-07T02:34:44.732 INFO:tasks.workunit.client.0.smithi019.stdout: 1 54027 3.08 MB/sec warmup 100 sec latency 4062.967 ms 2024-05-07T02:34:45.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.608+0000 7f0b11c00700 1 -- 172.21.15.19:0/1703587970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b0c108d30 msgr2=0x7f0b0c109140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:45.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.608+0000 7f0b11c00700 1 --2- 172.21.15.19:0/1703587970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b0c108d30 0x7f0b0c109140 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0af8009920 tx=0x7f0af8009c30 comp rx=0 tx=0).stop 2024-05-07T02:34:45.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.608+0000 7f0b11c00700 1 -- 172.21.15.19:0/1703587970 shutdown_connections 2024-05-07T02:34:45.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.608+0000 7f0b11c00700 1 --2- 172.21.15.19:0/1703587970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b0c109680 0x7f0b0c110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:45.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.608+0000 7f0b11c00700 1 --2- 172.21.15.19:0/1703587970 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b0c104400 0x7f0b0c104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:45.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.608+0000 7f0b11c00700 1 --2- 172.21.15.19:0/1703587970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b0c108d30 0x7f0b0c109140 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:45.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.608+0000 7f0b11c00700 1 -- 172.21.15.19:0/1703587970 >> 172.21.15.19:0/1703587970 conn(0x7f0b0c079d10 msgr2=0x7f0b0c07a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:34:45.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.609+0000 7f0b11c00700 1 -- 172.21.15.19:0/1703587970 shutdown_connections 2024-05-07T02:34:45.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.609+0000 7f0b11c00700 1 -- 172.21.15.19:0/1703587970 wait complete. 2024-05-07T02:34:45.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.610+0000 7f0b11c00700 1 Processor -- start 2024-05-07T02:34:45.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.610+0000 7f0b11c00700 1 -- start start 2024-05-07T02:34:45.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.611+0000 7f0b11c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b0c104400 0x7f0b0c19cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:45.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.611+0000 7f0b11c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b0c108d30 0x7f0b0c19d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:45.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.611+0000 7f0b11c00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b0c109680 0x7f0b0c1a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:45.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.611+0000 7f0b11c00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c114290 con 0x7f0b0c104400 2024-05-07T02:34:45.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.611+0000 7f0b11c00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c114110 con 0x7f0b0c108d30 2024-05-07T02:34:45.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.611+0000 7f0b11c00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c114410 con 0x7f0b0c109680 2024-05-07T02:34:45.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.611+0000 7f0b0b400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b0c108d30 0x7f0b0c19d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:45.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.611+0000 7f0b0be00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b0c104400 0x7f0b0c19cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:45.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.611+0000 7f0b0b400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b0c108d30 0x7f0b0c19d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:59294/0 (socket says 172.21.15.19:59294) 2024-05-07T02:34:45.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.611+0000 7f0b0b400700 1 -- 172.21.15.19:0/2162009435 learned_addr learned my addr 172.21.15.19:0/2162009435 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:34:45.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.611+0000 7f0b11200700 1 --2- 172.21.15.19:0/2162009435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b0c109680 0x7f0b0c1a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:45.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.612+0000 7f0b0b400700 1 -- 172.21.15.19:0/2162009435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b0c109680 msgr2=0x7f0b0c1a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:45.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.612+0000 7f0b0b400700 1 --2- 172.21.15.19:0/2162009435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b0c109680 0x7f0b0c1a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:45.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.612+0000 7f0b0b400700 1 -- 172.21.15.19:0/2162009435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b0c104400 msgr2=0x7f0b0c19cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:45.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.612+0000 7f0b0b400700 1 --2- 172.21.15.19:0/2162009435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b0c104400 0x7f0b0c19cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:45.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.612+0000 7f0b0b400700 1 -- 172.21.15.19:0/2162009435 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b0c1a1dd0 con 0x7f0b0c108d30 2024-05-07T02:34:45.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.612+0000 7f0b0be00700 1 --2- 172.21.15.19:0/2162009435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b0c104400 0x7f0b0c19cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:34:45.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.612+0000 7f0b0b400700 1 --2- 172.21.15.19:0/2162009435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b0c108d30 0x7f0b0c19d4a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0af8000c00 tx=0x7f0af800fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:34:45.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.612+0000 7f0b08c00700 1 -- 172.21.15.19:0/2162009435 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af80051c0 con 0x7f0b0c108d30 2024-05-07T02:34:45.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.612+0000 7f0b08c00700 1 -- 172.21.15.19:0/2162009435 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0af800b560 con 0x7f0b0c108d30 2024-05-07T02:34:45.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.612+0000 7f0b11c00700 1 -- 172.21.15.19:0/2162009435 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b0c1a2060 con 0x7f0b0c108d30 2024-05-07T02:34:45.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.613+0000 7f0b08c00700 1 -- 172.21.15.19:0/2162009435 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af80039b0 con 0x7f0b0c108d30 2024-05-07T02:34:45.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.614+0000 7f0b11c00700 1 -- 172.21.15.19:0/2162009435 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b0c1a99e0 con 0x7f0b0c108d30 2024-05-07T02:34:45.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.617+0000 7f0b08c00700 1 -- 172.21.15.19:0/2162009435 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0af8003b50 con 0x7f0b0c108d30 2024-05-07T02:34:45.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.618+0000 7f0b08c00700 1 --2- 172.21.15.19:0/2162009435 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f0af00b2a70 0x7f0af00b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:45.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.618+0000 7f0b08c00700 1 -- 172.21.15.19:0/2162009435 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f0af80e38a0 con 0x7f0b0c108d30 2024-05-07T02:34:45.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.619+0000 7f0b0be00700 1 --2- 172.21.15.19:0/2162009435 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f0af00b2a70 0x7f0af00b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:45.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.619+0000 7f0b21ce1b80 1 Processor -- start 2024-05-07T02:34:45.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.619+0000 7f0b21ce1b80 1 -- start start 2024-05-07T02:34:45.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.619+0000 7f0b21ce1b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a2294dc40 0x561a2294e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:45.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.619+0000 7f0b21ce1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a2294e590 0x561a229576f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:45.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.620+0000 7f0b21ce1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a22957c30 0x561a2295a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:45.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.620+0000 7f0b21ce1b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561a2286fa20 con 0x561a22957c30 2024-05-07T02:34:45.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.620+0000 7f0b21ce1b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561a2280d760 con 0x561a2294dc40 2024-05-07T02:34:45.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.620+0000 7f0b21ce1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561a2286b9e0 con 0x561a2294e590 2024-05-07T02:34:45.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.620+0000 7f0b0be00700 1 --2- 172.21.15.19:0/2162009435 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f0af00b2a70 0x7f0af00b4f80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0afc005fd0 tx=0x7f0afc009310 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:34:45.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.620+0000 7f0b0b400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a2294e590 0x561a229576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:45.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.620+0000 7f0b0b400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a2294e590 0x561a229576f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:60778/0 (socket says 172.21.15.19:60778) 2024-05-07T02:34:45.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.620+0000 7f0b0b400700 1 -- 172.21.15.19:0/2963698068 learned_addr learned my addr 172.21.15.19:0/2963698068 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:34:45.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.620+0000 7f0b0be00700 1 --2- 172.21.15.19:0/2963698068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a2294dc40 0x561a2294e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:45.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.620+0000 7f0b11200700 1 --2- 172.21.15.19:0/2963698068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a22957c30 0x561a2295a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:45.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.620+0000 7f0b0b400700 1 -- 172.21.15.19:0/2963698068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a2294dc40 msgr2=0x561a2294e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:45.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.620+0000 7f0b0b400700 1 --2- 172.21.15.19:0/2963698068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a2294dc40 0x561a2294e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:45.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.620+0000 7f0b0b400700 1 -- 172.21.15.19:0/2963698068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a22957c30 msgr2=0x561a2295a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:45.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.620+0000 7f0b0b400700 1 --2- 172.21.15.19:0/2963698068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a22957c30 0x561a2295a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:45.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.620+0000 7f0b0b400700 1 -- 172.21.15.19:0/2963698068 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a227bc8d0 con 0x561a2294e590 2024-05-07T02:34:45.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.620+0000 7f0b0be00700 1 --2- 172.21.15.19:0/2963698068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a2294dc40 0x561a2294e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:34:45.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.620+0000 7f0b11200700 1 --2- 172.21.15.19:0/2963698068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a22957c30 0x561a2295a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-07T02:34:45.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.621+0000 7f0b0b400700 1 --2- 172.21.15.19:0/2963698068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a2294e590 0x561a229576f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0af80aa7b0 tx=0x7f0af800bd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd2811ebaec4c808 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:34:45.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.621+0000 7f0b01600700 1 -- 172.21.15.19:0/2963698068 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af80cd0f0 con 0x561a2294e590 2024-05-07T02:34:45.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.621+0000 7f0b01600700 1 -- 172.21.15.19:0/2963698068 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0af8003e40 con 0x561a2294e590 2024-05-07T02:34:45.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.621+0000 7f0b01600700 1 -- 172.21.15.19:0/2963698068 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af800b930 con 0x561a2294e590 2024-05-07T02:34:45.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.622+0000 7f0b21ce1b80 1 -- 172.21.15.19:0/2963698068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a2294e590 msgr2=0x561a229576f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:45.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.622+0000 7f0b21ce1b80 1 --2- 172.21.15.19:0/2963698068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a2294e590 0x561a229576f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0af80aa7b0 tx=0x7f0af800bd10 comp rx=0 tx=0).stop 2024-05-07T02:34:45.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.623+0000 7f0b21ce1b80 1 -- 172.21.15.19:0/2963698068 shutdown_connections 2024-05-07T02:34:45.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.623+0000 7f0b21ce1b80 1 --2- 172.21.15.19:0/2963698068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a22957c30 0x561a2295a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:45.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.623+0000 7f0b21ce1b80 1 --2- 172.21.15.19:0/2963698068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a2294dc40 0x561a2294e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:45.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.623+0000 7f0b21ce1b80 1 --2- 172.21.15.19:0/2963698068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a2294e590 0x561a229576f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:45.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.623+0000 7f0b21ce1b80 1 -- 172.21.15.19:0/2963698068 >> 172.21.15.19:0/2963698068 conn(0x561a227cfe40 msgr2=0x561a22948980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:34:45.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.623+0000 7f0b21ce1b80 1 -- 172.21.15.19:0/2963698068 shutdown_connections 2024-05-07T02:34:45.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.623+0000 7f0b21ce1b80 1 -- 172.21.15.19:0/2963698068 wait complete. 2024-05-07T02:34:45.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.624+0000 7f0b21ce1b80 1 -- start start 2024-05-07T02:34:45.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.624+0000 7f0b21ce1b80 1 Processor -- start 2024-05-07T02:34:45.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.625+0000 7f0b21ce1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a2294dc40 0x561a22954bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:45.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.625+0000 7f0b21ce1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a2294e590 0x561a22955130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:45.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.625+0000 7f0b21ce1b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a22957c30 0x561a22955670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:45.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.625+0000 7f0b21ce1b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561a2295a7c0 con 0x561a2294dc40 2024-05-07T02:34:45.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.625+0000 7f0b21ce1b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x561a227d5b20 con 0x561a22957c30 2024-05-07T02:34:45.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.625+0000 7f0b21ce1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561a2295a940 con 0x561a2294e590 2024-05-07T02:34:45.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.625+0000 7f0b21ce1b80 10 client.0 _ll_register_callbacks cb 0x561a22803380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:34:45.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.625+0000 7f0b0b400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a2294e590 0x561a22955130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:45.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.625+0000 7f0b0b400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a2294e590 0x561a22955130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:60794/0 (socket says 172.21.15.19:60794) 2024-05-07T02:34:45.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.625+0000 7f0b0b400700 1 -- 172.21.15.19:0/2680658635 learned_addr learned my addr 172.21.15.19:0/2680658635 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:34:45.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.625+0000 7f0b11200700 1 --2- 172.21.15.19:0/2680658635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a22957c30 0x561a22955670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:45.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.625+0000 7f0b0be00700 1 --2- 172.21.15.19:0/2680658635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a2294dc40 0x561a22954bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:45.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.625+0000 7f0b0b400700 1 -- 172.21.15.19:0/2680658635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a22957c30 msgr2=0x561a22955670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:45.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.625+0000 7f0b0b400700 1 --2- 172.21.15.19:0/2680658635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a22957c30 0x561a22955670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:45.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.626+0000 7f0b0b400700 1 -- 172.21.15.19:0/2680658635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a2294dc40 msgr2=0x561a22954bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:45.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.626+0000 7f0b0b400700 1 --2- 172.21.15.19:0/2680658635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a2294dc40 0x561a22954bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:45.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.626+0000 7f0b0b400700 1 -- 172.21.15.19:0/2680658635 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a22835420 con 0x561a2294e590 2024-05-07T02:34:45.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.626+0000 7f0b0b400700 1 --2- 172.21.15.19:0/2680658635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a2294e590 0x561a22955130 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0af80098f0 tx=0x7f0af80a6d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73acfe2a7612f6af server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:34:45.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.626+0000 7f0af6000700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:34:45.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.626+0000 7f0af6000700 1 -- 172.21.15.19:0/2680658635 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af80cd0f0 con 0x561a2294e590 2024-05-07T02:34:45.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.626+0000 7f0af6000700 1 -- 172.21.15.19:0/2680658635 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f0af800b710 con 0x561a2294e590 2024-05-07T02:34:45.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.626+0000 7f0b21ce1b80 1 -- 172.21.15.19:0/2680658635 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561a2291a0e0 con 0x561a2294e590 2024-05-07T02:34:45.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.626+0000 7f0b0be00700 1 --2- 172.21.15.19:0/2680658635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a2294dc40 0x561a22954bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:34:45.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.626+0000 7f0af6000700 1 -- 172.21.15.19:0/2680658635 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0af80aa820 con 0x561a2294e590 2024-05-07T02:34:45.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.626+0000 7f0b21ce1b80 10 client.25753 fetch_fsmap learned FSMap version 34 2024-05-07T02:34:45.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.627+0000 7f0b21ce1b80 1 -- 172.21.15.19:0/2680658635 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561a2295a7c0 con 0x561a2294e590 2024-05-07T02:34:45.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.627+0000 7f0af6000700 1 -- 172.21.15.19:0/2680658635 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561a2291a0e0 con 0x561a2294e590 2024-05-07T02:34:45.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.627+0000 7f0af6000700 1 -- 172.21.15.19:0/2680658635 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f0af80bfbe0 con 0x561a2294e590 2024-05-07T02:34:45.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.627+0000 7f0b21ce1b80 10 client.25753 fetch_fsmap finished waiting for FSMap version 34 2024-05-07T02:34:45.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.627+0000 7f0b21ce1b80 10 client.25753 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-05-07T02:34:45.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.627+0000 7f0b21ce1b80 20 client.25753 populate_metadata read hostname 'smithi019' 2024-05-07T02:34:45.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.627+0000 7f0b21ce1b80 1 --2- 172.21.15.19:0/2680658635 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x561a22967750 0x561a22969b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:45.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.627+0000 7f0b21ce1b80 4 client.25753 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:34:45.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.627+0000 7f0b21ce1b80 1 -- 172.21.15.19:0/2680658635 --> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a2287d400 con 0x561a22967750 2024-05-07T02:34:45.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.628+0000 7f0b0be00700 1 --2- 172.21.15.19:0/2680658635 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x561a22967750 0x561a22969b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:45.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.628+0000 7f0af6000700 10 client.25753 ms_handle_connect on v2:172.21.15.27:6836/2583138143 2024-05-07T02:34:45.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.628+0000 7f0b0be00700 1 --2- 172.21.15.19:0/2680658635 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x561a22967750 0x561a22969b70 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5fea36cb13bf624f server_cookie=7e36d71b0bef0a29 in_seq=0 out_seq=0 2024-05-07T02:34:45.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.629+0000 7f0af6000700 1 -- 172.21.15.19:0/2680658635 <== mds.3 v2:172.21.15.27:6836/2583138143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561a2287d400 con 0x561a22967750 2024-05-07T02:34:45.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.629+0000 7f0af6000700 10 client.25753 handle_command_reply: tid=0 2024-05-07T02:34:45.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.630+0000 7f0b21ce1b80 1 client.25753 shutdown 2024-05-07T02:34:45.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.630+0000 7f0b21ce1b80 1 -- 172.21.15.19:0/2680658635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a2294e590 msgr2=0x561a22955130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:45.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.630+0000 7f0b21ce1b80 1 --2- 172.21.15.19:0/2680658635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a2294e590 0x561a22955130 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0af80098f0 tx=0x7f0af80a6d60 comp rx=0 tx=0).stop 2024-05-07T02:34:45.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.630+0000 7f0b21ce1b80 1 -- 172.21.15.19:0/2680658635 shutdown_connections 2024-05-07T02:34:45.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.630+0000 7f0b21ce1b80 1 --2- 172.21.15.19:0/2680658635 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x561a22967750 0x561a22969b70 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:45.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.630+0000 7f0b21ce1b80 1 --2- 172.21.15.19:0/2680658635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a22957c30 0x561a22955670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:45.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.630+0000 7f0b21ce1b80 1 --2- 172.21.15.19:0/2680658635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a2294dc40 0x561a22954bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:45.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.630+0000 7f0b21ce1b80 1 --2- 172.21.15.19:0/2680658635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a2294e590 0x561a22955130 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:45.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.630+0000 7f0b21ce1b80 1 -- 172.21.15.19:0/2680658635 >> 172.21.15.19:0/2680658635 conn(0x561a227cfe40 msgr2=0x561a2294a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:34:45.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.630+0000 7f0af6000700 0 client.25753 ms_handle_reset on v2:172.21.15.27:6836/2583138143 2024-05-07T02:34:45.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.630+0000 7f0b21ce1b80 1 -- 172.21.15.19:0/2680658635 shutdown_connections 2024-05-07T02:34:45.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.630+0000 7f0b21ce1b80 1 -- 172.21.15.19:0/2680658635 wait complete. 2024-05-07T02:34:45.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.630+0000 7f0b21ce1b80 20 client.25753 trim_cache size 0 max 16384 2024-05-07T02:34:45.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.648+0000 7f0b21ce1b80 1 Processor -- start 2024-05-07T02:34:45.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.648+0000 7f0b21ce1b80 1 -- start start 2024-05-07T02:34:45.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.648+0000 7f0b21ce1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a2294dc40 0x561a22948a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:45.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.648+0000 7f0b21ce1b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a22957c30 0x561a22963860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:45.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.648+0000 7f0b21ce1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a22950540 0x561a22955160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:45.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.648+0000 7f0b21ce1b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c1b25b0 con 0x561a2294dc40 2024-05-07T02:34:45.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.648+0000 7f0b21ce1b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c1b41b0 con 0x561a22957c30 2024-05-07T02:34:45.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.648+0000 7f0b21ce1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c1b2c80 con 0x561a22950540 2024-05-07T02:34:45.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.649+0000 7f0b0b400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a22957c30 0x561a22963860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:45.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.649+0000 7f0b11200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a22950540 0x561a22955160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:45.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.649+0000 7f0b11200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a22950540 0x561a22955160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:60808/0 (socket says 172.21.15.19:60808) 2024-05-07T02:34:45.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.649+0000 7f0b0b400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a22957c30 0x561a22963860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:59312/0 (socket says 172.21.15.19:59312) 2024-05-07T02:34:45.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.649+0000 7f0b11200700 1 -- 172.21.15.19:0/177945501 learned_addr learned my addr 172.21.15.19:0/177945501 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:34:45.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.649+0000 7f0b0be00700 1 --2- 172.21.15.19:0/177945501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a2294dc40 0x561a22948a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:45.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.649+0000 7f0b0b400700 1 -- 172.21.15.19:0/177945501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a22950540 msgr2=0x561a22955160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:45.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.649+0000 7f0b0b400700 1 --2- 172.21.15.19:0/177945501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a22950540 0x561a22955160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:45.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.649+0000 7f0b0b400700 1 -- 172.21.15.19:0/177945501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a2294dc40 msgr2=0x561a22948a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:45.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.649+0000 7f0b0b400700 1 --2- 172.21.15.19:0/177945501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a2294dc40 0x561a22948a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:45.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.649+0000 7f0b0b400700 1 -- 172.21.15.19:0/177945501 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a22944b20 con 0x561a22957c30 2024-05-07T02:34:45.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.649+0000 7f0b0be00700 1 --2- 172.21.15.19:0/177945501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a2294dc40 0x561a22948a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:34:45.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.649+0000 7f0b11200700 1 --2- 172.21.15.19:0/177945501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a22950540 0x561a22955160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:34:45.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.650+0000 7f0b0b400700 1 --2- 172.21.15.19:0/177945501 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a22957c30 0x561a22963860 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x561a22955cb0 tx=0x7f0af800bd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=655bfcf05f079d96 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:34:45.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.650+0000 7f0b00c00700 1 -- 172.21.15.19:0/177945501 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af80cd0f0 con 0x561a22957c30 2024-05-07T02:34:45.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.650+0000 7f0b00c00700 1 -- 172.21.15.19:0/177945501 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0af80b35f0 con 0x561a22957c30 2024-05-07T02:34:45.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.650+0000 7f0b00c00700 1 -- 172.21.15.19:0/177945501 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af80e1090 con 0x561a22957c30 2024-05-07T02:34:45.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.651+0000 7f0b21ce1b80 1 -- 172.21.15.19:0/177945501 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a22957c30 msgr2=0x561a22963860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:45.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.651+0000 7f0b21ce1b80 1 --2- 172.21.15.19:0/177945501 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a22957c30 0x561a22963860 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x561a22955cb0 tx=0x7f0af800bd10 comp rx=0 tx=0).stop 2024-05-07T02:34:45.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.651+0000 7f0b21ce1b80 1 -- 172.21.15.19:0/177945501 shutdown_connections 2024-05-07T02:34:45.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.651+0000 7f0b21ce1b80 1 --2- 172.21.15.19:0/177945501 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a22950540 0x561a22955160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:45.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.651+0000 7f0b21ce1b80 1 --2- 172.21.15.19:0/177945501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a2294dc40 0x561a22948a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:45.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.651+0000 7f0b21ce1b80 1 --2- 172.21.15.19:0/177945501 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a22957c30 0x561a22963860 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:45.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.651+0000 7f0b21ce1b80 1 -- 172.21.15.19:0/177945501 >> 172.21.15.19:0/177945501 conn(0x561a227cfe40 msgr2=0x561a22948340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:34:45.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.652+0000 7f0b21ce1b80 1 -- 172.21.15.19:0/177945501 shutdown_connections 2024-05-07T02:34:45.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.652+0000 7f0b21ce1b80 1 -- 172.21.15.19:0/177945501 wait complete. 2024-05-07T02:34:45.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.652+0000 7f0b21ce1b80 1 -- start start 2024-05-07T02:34:45.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.652+0000 7f0b21ce1b80 1 Processor -- start 2024-05-07T02:34:45.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.653+0000 7f0b21ce1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a2294dc40 0x561a2295aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:45.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.653+0000 7f0b21ce1b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a22950540 0x561a2295b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:45.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.653+0000 7f0b21ce1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a22957c30 0x561a229cfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:45.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.653+0000 7f0b21ce1b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c1b3a20 con 0x561a22957c30 2024-05-07T02:34:45.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.653+0000 7f0b21ce1b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c1b3350 con 0x561a22950540 2024-05-07T02:34:45.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.653+0000 7f0b21ce1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c1b0550 con 0x561a2294dc40 2024-05-07T02:34:45.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.653+0000 7f0b21ce1b80 10 client.0 _ll_register_callbacks cb 0x561a22803380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:34:45.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.653+0000 7f0b0be00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a2294dc40 0x561a2295aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:45.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.653+0000 7f0b0be00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a2294dc40 0x561a2295aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:60820/0 (socket says 172.21.15.19:60820) 2024-05-07T02:34:45.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.653+0000 7f0b11200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a22957c30 0x561a229cfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:45.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.653+0000 7f0b0be00700 1 -- 172.21.15.19:0/2373078883 learned_addr learned my addr 172.21.15.19:0/2373078883 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:34:45.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.653+0000 7f0b0b400700 1 --2- 172.21.15.19:0/2373078883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a22950540 0x561a2295b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:45.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.654+0000 7f0b0be00700 1 -- 172.21.15.19:0/2373078883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a22950540 msgr2=0x561a2295b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:45.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.654+0000 7f0b0be00700 1 --2- 172.21.15.19:0/2373078883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a22950540 0x561a2295b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:45.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.654+0000 7f0b0be00700 1 -- 172.21.15.19:0/2373078883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a22957c30 msgr2=0x561a229cfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:45.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.654+0000 7f0b0be00700 1 --2- 172.21.15.19:0/2373078883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a22957c30 0x561a229cfeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:45.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.654+0000 7f0b0be00700 1 -- 172.21.15.19:0/2373078883 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a2295b9c0 con 0x561a2294dc40 2024-05-07T02:34:45.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.654+0000 7f0b0b400700 1 --2- 172.21.15.19:0/2373078883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a22950540 0x561a2295b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:34:45.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.654+0000 7f0b0be00700 1 --2- 172.21.15.19:0/2373078883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a2294dc40 0x561a2295aba0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0afc011a50 tx=0x7f0afc0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9772e6d19390488 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:34:45.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.654+0000 7f0af6000700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:34:45.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.654+0000 7f0af6000700 1 -- 172.21.15.19:0/2373078883 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0afc02e070 con 0x561a2294dc40 2024-05-07T02:34:45.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.654+0000 7f0b21ce1b80 1 -- 172.21.15.19:0/2373078883 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0af80b35f0 con 0x561a2294dc40 2024-05-07T02:34:45.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.654+0000 7f0af6000700 1 -- 172.21.15.19:0/2373078883 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f0afc029450 con 0x561a2294dc40 2024-05-07T02:34:45.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.655+0000 7f0af6000700 1 -- 172.21.15.19:0/2373078883 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0afc036050 con 0x561a2294dc40 2024-05-07T02:34:45.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.655+0000 7f0b21ce1b80 10 client.25759 fetch_fsmap learned FSMap version 34 2024-05-07T02:34:45.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.655+0000 7f0b21ce1b80 1 -- 172.21.15.19:0/2373078883 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561a229d14a0 con 0x561a2294dc40 2024-05-07T02:34:45.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.655+0000 7f0af6000700 1 -- 172.21.15.19:0/2373078883 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0af80b35f0 con 0x561a2294dc40 2024-05-07T02:34:45.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.655+0000 7f0af6000700 1 -- 172.21.15.19:0/2373078883 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f0afc031030 con 0x561a2294dc40 2024-05-07T02:34:45.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.655+0000 7f0b21ce1b80 10 client.25759 fetch_fsmap finished waiting for FSMap version 34 2024-05-07T02:34:45.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.655+0000 7f0b21ce1b80 10 client.25759 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-05-07T02:34:45.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.655+0000 7f0b21ce1b80 20 client.25759 populate_metadata read hostname 'smithi019' 2024-05-07T02:34:45.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.655+0000 7f0b21ce1b80 1 --2- 172.21.15.19:0/2373078883 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x561a229d1b90 0x561a229d3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:45.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.655+0000 7f0b21ce1b80 4 client.25759 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:34:45.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.655+0000 7f0b21ce1b80 1 -- 172.21.15.19:0/2373078883 --> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561a228774a0 con 0x561a229d1b90 2024-05-07T02:34:45.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.656+0000 7f0b0b400700 1 --2- 172.21.15.19:0/2373078883 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x561a229d1b90 0x561a229d3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:45.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.656+0000 7f0b0b400700 1 --2- 172.21.15.19:0/2373078883 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x561a229d1b90 0x561a229d3fb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f24ab57a4479efa server_cookie=3ad4be9adf092973 in_seq=0 out_seq=0 2024-05-07T02:34:45.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.656+0000 7f0af6000700 10 client.25759 ms_handle_connect on v2:172.21.15.27:6836/2583138143 2024-05-07T02:34:45.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.657+0000 7f0af6000700 1 -- 172.21.15.19:0/2373078883 <== mds.3 v2:172.21.15.27:6836/2583138143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561a228774a0 con 0x561a229d1b90 2024-05-07T02:34:45.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.657+0000 7f0af6000700 10 client.25759 handle_command_reply: tid=0 2024-05-07T02:34:45.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.657+0000 7f0b21ce1b80 1 client.25759 shutdown 2024-05-07T02:34:45.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.657+0000 7f0b21ce1b80 1 -- 172.21.15.19:0/2373078883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a2294dc40 msgr2=0x561a2295aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:45.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.657+0000 7f0b21ce1b80 1 --2- 172.21.15.19:0/2373078883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a2294dc40 0x561a2295aba0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0afc011a50 tx=0x7f0afc0047c0 comp rx=0 tx=0).stop 2024-05-07T02:34:45.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.657+0000 7f0b21ce1b80 1 -- 172.21.15.19:0/2373078883 shutdown_connections 2024-05-07T02:34:45.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.657+0000 7f0b21ce1b80 1 --2- 172.21.15.19:0/2373078883 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x561a229d1b90 0x561a229d3fb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:45.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.657+0000 7f0b21ce1b80 1 --2- 172.21.15.19:0/2373078883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a22957c30 0x561a229cfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:45.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.657+0000 7f0b21ce1b80 1 --2- 172.21.15.19:0/2373078883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561a2294dc40 0x561a2295aba0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:45.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.657+0000 7f0b21ce1b80 1 --2- 172.21.15.19:0/2373078883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x561a22950540 0x561a2295b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:45.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.657+0000 7f0b21ce1b80 1 -- 172.21.15.19:0/2373078883 >> 172.21.15.19:0/2373078883 conn(0x561a227cfe40 msgr2=0x561a2294c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:34:45.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.657+0000 7f0af6000700 0 client.25759 ms_handle_reset on v2:172.21.15.27:6836/2583138143 2024-05-07T02:34:45.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.657+0000 7f0b21ce1b80 1 -- 172.21.15.19:0/2373078883 shutdown_connections 2024-05-07T02:34:45.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.657+0000 7f0b21ce1b80 1 -- 172.21.15.19:0/2373078883 wait complete. 2024-05-07T02:34:45.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.658+0000 7f0b21ce1b80 20 client.25759 trim_cache size 0 max 16384 2024-05-07T02:34:45.660 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:34:45.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.658+0000 7f0b11c00700 1 -- 172.21.15.19:0/2162009435 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f0af00b2a70 msgr2=0x7f0af00b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:45.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.658+0000 7f0b11c00700 1 --2- 172.21.15.19:0/2162009435 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f0af00b2a70 0x7f0af00b4f80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0afc005fd0 tx=0x7f0afc009310 comp rx=0 tx=0).stop 2024-05-07T02:34:45.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.658+0000 7f0b11c00700 1 -- 172.21.15.19:0/2162009435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b0c108d30 msgr2=0x7f0b0c19d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:45.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.658+0000 7f0b11c00700 1 --2- 172.21.15.19:0/2162009435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b0c108d30 0x7f0b0c19d4a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0af8000c00 tx=0x7f0af800fa00 comp rx=0 tx=0).stop 2024-05-07T02:34:45.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.658+0000 7f0b11c00700 1 -- 172.21.15.19:0/2162009435 shutdown_connections 2024-05-07T02:34:45.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.658+0000 7f0b11c00700 1 --2- 172.21.15.19:0/2162009435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b0c109680 0x7f0b0c1a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:45.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.658+0000 7f0b11c00700 1 --2- 172.21.15.19:0/2162009435 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f0af00b2a70 0x7f0af00b4f80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:45.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.658+0000 7f0b11c00700 1 --2- 172.21.15.19:0/2162009435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b0c104400 0x7f0b0c19cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:45.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.658+0000 7f0b11c00700 1 --2- 172.21.15.19:0/2162009435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b0c108d30 0x7f0b0c19d4a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:45.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.658+0000 7f0b11c00700 1 -- 172.21.15.19:0/2162009435 >> 172.21.15.19:0/2162009435 conn(0x7f0b0c079d10 msgr2=0x7f0b0c110d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:34:45.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.659+0000 7f0b11c00700 1 -- 172.21.15.19:0/2162009435 shutdown_connections 2024-05-07T02:34:45.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:45.659+0000 7f0b11c00700 1 -- 172.21.15.19:0/2162009435 wait complete. 2024-05-07T02:34:45.732 INFO:tasks.workunit.client.0.smithi019.stdout: 1 54308 3.07 MB/sec warmup 101 sec latency 4819.175 ms 2024-05-07T02:34:46.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:34:46 smithi064 ceph-mon[34699]: pgmap v91: 129 pgs: 129 active+clean; 48 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 919 KiB/s wr, 111 op/s 2024-05-07T02:34:46.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:34:46 smithi019 ceph-mon[29781]: pgmap v91: 129 pgs: 129 active+clean; 48 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 919 KiB/s wr, 111 op/s 2024-05-07T02:34:46.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:34:46 smithi027 ceph-mon[34398]: pgmap v91: 129 pgs: 129 active+clean; 48 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 919 KiB/s wr, 111 op/s 2024-05-07T02:34:46.661 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-07T02:34:46.732 INFO:tasks.workunit.client.0.smithi019.stdout: 1 56015 3.13 MB/sec warmup 102 sec latency 17.913 ms 2024-05-07T02:34:47.184 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:34:47.732 INFO:tasks.workunit.client.0.smithi019.stdout: 1 57108 3.17 MB/sec warmup 103 sec latency 23.276 ms 2024-05-07T02:34:48.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:34:48 smithi064 ceph-mon[34699]: pgmap v92: 129 pgs: 129 active+clean; 48 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 689 KiB/s wr, 64 op/s 2024-05-07T02:34:48.375 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:34:48 smithi019 ceph-mon[29781]: pgmap v92: 129 pgs: 129 active+clean; 48 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 689 KiB/s wr, 64 op/s 2024-05-07T02:34:48.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:34:48 smithi027 ceph-mon[34398]: pgmap v92: 129 pgs: 129 active+clean; 48 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 689 KiB/s wr, 64 op/s 2024-05-07T02:34:48.733 INFO:tasks.workunit.client.0.smithi019.stdout: 1 57760 3.15 MB/sec warmup 104 sec latency 41.372 ms 2024-05-07T02:34:49.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.722+0000 7fbd13400700 1 -- 172.21.15.19:0/3243871314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd0c1034c0 msgr2=0x7fbd0c1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:49.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.722+0000 7fbd13400700 1 --2- 172.21.15.19:0/3243871314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd0c1034c0 0x7fbd0c1038d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbd00009920 tx=0x7fbd00009c30 comp rx=0 tx=0).stop 2024-05-07T02:34:49.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.722+0000 7fbd13400700 1 -- 172.21.15.19:0/3243871314 shutdown_connections 2024-05-07T02:34:49.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.722+0000 7fbd13400700 1 --2- 172.21.15.19:0/3243871314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd0c108740 0x7fbd0c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:49.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.722+0000 7fbd13400700 1 --2- 172.21.15.19:0/3243871314 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd0c1034c0 0x7fbd0c1038d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:49.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.722+0000 7fbd13400700 1 --2- 172.21.15.19:0/3243871314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd0c107df0 0x7fbd0c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:49.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.722+0000 7fbd13400700 1 -- 172.21.15.19:0/3243871314 >> 172.21.15.19:0/3243871314 conn(0x7fbd0c0fedc0 msgr2=0x7fbd0c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:34:49.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.722+0000 7fbd13400700 1 -- 172.21.15.19:0/3243871314 shutdown_connections 2024-05-07T02:34:49.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.722+0000 7fbd13400700 1 -- 172.21.15.19:0/3243871314 wait complete. 2024-05-07T02:34:49.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.724+0000 7fbd13400700 1 Processor -- start 2024-05-07T02:34:49.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.724+0000 7fbd13400700 1 -- start start 2024-05-07T02:34:49.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.725+0000 7fbd13400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd0c1034c0 0x7fbd0c1a1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:49.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.725+0000 7fbd13400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd0c107df0 0x7fbd0c1a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:49.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.725+0000 7fbd13400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd0c108740 0x7fbd0c1a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:49.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.725+0000 7fbd13400700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c113370 con 0x7fbd0c1034c0 2024-05-07T02:34:49.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.725+0000 7fbd13400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c1131f0 con 0x7fbd0c108740 2024-05-07T02:34:49.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.725+0000 7fbd13400700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c1134f0 con 0x7fbd0c107df0 2024-05-07T02:34:49.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.725+0000 7fbd0be00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd0c107df0 0x7fbd0c1a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:49.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.725+0000 7fbd11400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd0c108740 0x7fbd0c1a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:49.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.725+0000 7fbd0be00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd0c107df0 0x7fbd0c1a1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:59276/0 (socket says 172.21.15.19:59276) 2024-05-07T02:34:49.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.725+0000 7fbd0be00700 1 -- 172.21.15.19:0/1049807634 learned_addr learned my addr 172.21.15.19:0/1049807634 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:34:49.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.725+0000 7fbd10a00700 1 --2- 172.21.15.19:0/1049807634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd0c1034c0 0x7fbd0c1a1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:49.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.725+0000 7fbd11400700 1 -- 172.21.15.19:0/1049807634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd0c107df0 msgr2=0x7fbd0c1a1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:49.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.725+0000 7fbd11400700 1 --2- 172.21.15.19:0/1049807634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd0c107df0 0x7fbd0c1a1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:49.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.726+0000 7fbd11400700 1 -- 172.21.15.19:0/1049807634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd0c1034c0 msgr2=0x7fbd0c1a1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:49.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.726+0000 7fbd11400700 1 --2- 172.21.15.19:0/1049807634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd0c1034c0 0x7fbd0c1a1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:49.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.726+0000 7fbd11400700 1 -- 172.21.15.19:0/1049807634 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd0c1a6310 con 0x7fbd0c108740 2024-05-07T02:34:49.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.726+0000 7fbd11400700 1 --2- 172.21.15.19:0/1049807634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd0c108740 0x7fbd0c1a5c70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbd0400efc0 tx=0x7fbd0400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:34:49.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.726+0000 7fbd09600700 1 -- 172.21.15.19:0/1049807634 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd040092e0 con 0x7fbd0c108740 2024-05-07T02:34:49.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.726+0000 7fbd09600700 1 -- 172.21.15.19:0/1049807634 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd04004bb0 con 0x7fbd0c108740 2024-05-07T02:34:49.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.726+0000 7fbd13400700 1 -- 172.21.15.19:0/1049807634 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd0c1a65a0 con 0x7fbd0c108740 2024-05-07T02:34:49.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.726+0000 7fbd09600700 1 -- 172.21.15.19:0/1049807634 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd04004020 con 0x7fbd0c108740 2024-05-07T02:34:49.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.726+0000 7fbd13400700 1 -- 172.21.15.19:0/1049807634 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd0c1adf20 con 0x7fbd0c108740 2024-05-07T02:34:49.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.730+0000 7fbd09600700 1 -- 172.21.15.19:0/1049807634 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fbd0401b030 con 0x7fbd0c108740 2024-05-07T02:34:49.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.732+0000 7fbd21e60b80 1 Processor -- start 2024-05-07T02:34:49.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.732+0000 7fbd21e60b80 1 -- start start 2024-05-07T02:34:49.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.732+0000 7fbd09600700 1 --2- 172.21.15.19:0/1049807634 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fbcf40b2a70 0x7fbcf40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:49.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.732+0000 7fbd09600700 1 -- 172.21.15.19:0/1049807634 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fbd040e2420 con 0x7fbd0c108740 2024-05-07T02:34:49.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.732+0000 7fbd21e60b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639569e8470 0x5639569e8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:49.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.732+0000 7fbd21e60b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639569e2620 0x5639569e2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:49.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.732+0000 7fbd21e60b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639569e2f70 0x5639569ef150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:49.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.732+0000 7fbd0be00700 1 --2- 172.21.15.19:0/1049807634 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fbcf40b2a70 0x7fbcf40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:49.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.732+0000 7fbd21e60b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563956904a20 con 0x5639569e2620 2024-05-07T02:34:49.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.732+0000 7fbd21e60b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5639568a2760 con 0x5639569e8470 2024-05-07T02:34:49.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.732+0000 7fbd21e60b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5639569009e0 con 0x5639569e2f70 2024-05-07T02:34:49.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.733+0000 7fbd10a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639569e8470 0x5639569e8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:49.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.733+0000 7fbd10a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639569e8470 0x5639569e8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:36424/0 (socket says 172.21.15.19:36424) 2024-05-07T02:34:49.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.733+0000 7fbd10a00700 1 -- 172.21.15.19:0/3387439963 learned_addr learned my addr 172.21.15.19:0/3387439963 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:34:49.733 INFO:tasks.workunit.client.0.smithi019.stdout: 1 59387 3.19 MB/sec warmup 105 sec latency 24.186 ms 2024-05-07T02:34:49.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.733+0000 7fbd0be00700 1 --2- 172.21.15.19:0/3387439963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639569e2620 0x5639569e2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:49.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.733+0000 7fbd11400700 1 --2- 172.21.15.19:0/3387439963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639569e2f70 0x5639569ef150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:49.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.733+0000 7fbd10a00700 1 -- 172.21.15.19:0/3387439963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639569e2f70 msgr2=0x5639569ef150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:49.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.733+0000 7fbd10a00700 1 --2- 172.21.15.19:0/3387439963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639569e2f70 0x5639569ef150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:49.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.733+0000 7fbd10a00700 1 -- 172.21.15.19:0/3387439963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639569e2620 msgr2=0x5639569e2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:49.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.733+0000 7fbd10a00700 1 --2- 172.21.15.19:0/3387439963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639569e2620 0x5639569e2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:49.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.733+0000 7fbd10a00700 1 -- 172.21.15.19:0/3387439963 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639568518d0 con 0x5639569e8470 2024-05-07T02:34:49.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.734+0000 7fbd10a00700 1 --2- 172.21.15.19:0/3387439963 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639569e8470 0x5639569e8880 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbd00016000 tx=0x7fbd0000bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2ecd8c24b65a8fb server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:34:49.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.734+0000 7fbd0be00700 1 --2- 172.21.15.19:0/1049807634 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fbcf40b2a70 0x7fbcf40b4f80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c107dc0 tx=0x7fbcf8013980 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:34:49.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.734+0000 7fbcfe000700 1 -- 172.21.15.19:0/3387439963 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0000f460 con 0x5639569e8470 2024-05-07T02:34:49.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.734+0000 7fbcfe000700 1 -- 172.21.15.19:0/3387439963 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd00005040 con 0x5639569e8470 2024-05-07T02:34:49.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.734+0000 7fbcfe000700 1 -- 172.21.15.19:0/3387439963 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd000038e0 con 0x5639569e8470 2024-05-07T02:34:49.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.735+0000 7fbd21e60b80 1 -- 172.21.15.19:0/3387439963 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639569e8470 msgr2=0x5639569e8880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:49.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.735+0000 7fbd21e60b80 1 --2- 172.21.15.19:0/3387439963 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639569e8470 0x5639569e8880 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbd00016000 tx=0x7fbd0000bfa0 comp rx=0 tx=0).stop 2024-05-07T02:34:49.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.735+0000 7fbd21e60b80 1 -- 172.21.15.19:0/3387439963 shutdown_connections 2024-05-07T02:34:49.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.735+0000 7fbd21e60b80 1 --2- 172.21.15.19:0/3387439963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639569e2f70 0x5639569ef150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:49.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.735+0000 7fbd21e60b80 1 --2- 172.21.15.19:0/3387439963 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639569e8470 0x5639569e8880 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:49.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.735+0000 7fbd21e60b80 1 --2- 172.21.15.19:0/3387439963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639569e2620 0x5639569e2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:49.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.735+0000 7fbd21e60b80 1 -- 172.21.15.19:0/3387439963 >> 172.21.15.19:0/3387439963 conn(0x563956864e40 msgr2=0x5639569dd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:34:49.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.736+0000 7fbd21e60b80 1 -- 172.21.15.19:0/3387439963 shutdown_connections 2024-05-07T02:34:49.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.736+0000 7fbd21e60b80 1 -- 172.21.15.19:0/3387439963 wait complete. 2024-05-07T02:34:49.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.736+0000 7fbd21e60b80 1 -- start start 2024-05-07T02:34:49.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.737+0000 7fbd21e60b80 1 Processor -- start 2024-05-07T02:34:49.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.737+0000 7fbd21e60b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639569e2620 0x5639569ec6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:49.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.737+0000 7fbd21e60b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639569e2f70 0x5639569ecc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:49.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.737+0000 7fbd21e60b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639569e8470 0x5639569ed150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:49.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.737+0000 7fbd21e60b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5639569ef7e0 con 0x5639569e8470 2024-05-07T02:34:49.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.737+0000 7fbd21e60b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56395686ab20 con 0x5639569e2620 2024-05-07T02:34:49.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.737+0000 7fbd21e60b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5639569ef960 con 0x5639569e2f70 2024-05-07T02:34:49.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.738+0000 7fbd21e60b80 10 client.0 _ll_register_callbacks cb 0x563956898380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:34:49.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.738+0000 7fbd10a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639569e2620 0x5639569ec6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:49.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.738+0000 7fbd10a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639569e2620 0x5639569ec6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:36436/0 (socket says 172.21.15.19:36436) 2024-05-07T02:34:49.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.738+0000 7fbd10a00700 1 -- 172.21.15.19:0/793944422 learned_addr learned my addr 172.21.15.19:0/793944422 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:34:49.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.738+0000 7fbd0be00700 1 --2- 172.21.15.19:0/793944422 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639569e2f70 0x5639569ecc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:49.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.738+0000 7fbd11400700 1 --2- 172.21.15.19:0/793944422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639569e8470 0x5639569ed150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:49.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.738+0000 7fbd0be00700 1 -- 172.21.15.19:0/793944422 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639569e2620 msgr2=0x5639569ec6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:49.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.738+0000 7fbd0be00700 1 --2- 172.21.15.19:0/793944422 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639569e2620 0x5639569ec6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:49.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.738+0000 7fbd0be00700 1 -- 172.21.15.19:0/793944422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639569e8470 msgr2=0x5639569ed150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:49.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.738+0000 7fbd0be00700 1 --2- 172.21.15.19:0/793944422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639569e8470 0x5639569ed150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:49.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.738+0000 7fbd0be00700 1 -- 172.21.15.19:0/793944422 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639568ca420 con 0x5639569e2f70 2024-05-07T02:34:49.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.739+0000 7fbd0be00700 1 --2- 172.21.15.19:0/793944422 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639569e2f70 0x5639569ecc10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8009750 tx=0x7fbcf80047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58edec56f4b837f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:34:49.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.739+0000 7fbcf2a00700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:34:49.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.739+0000 7fbcf2a00700 1 -- 172.21.15.19:0/793944422 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf8024070 con 0x5639569e2f70 2024-05-07T02:34:49.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.739+0000 7fbcf2a00700 1 -- 172.21.15.19:0/793944422 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fbcf8020630 con 0x5639569e2f70 2024-05-07T02:34:49.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.740+0000 7fbd21e60b80 1 -- 172.21.15.19:0/793944422 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd00005040 con 0x5639569e2f70 2024-05-07T02:34:49.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.740+0000 7fbcf2a00700 1 -- 172.21.15.19:0/793944422 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbcf8020830 con 0x5639569e2f70 2024-05-07T02:34:49.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.740+0000 7fbd21e60b80 10 client.25771 fetch_fsmap learned FSMap version 34 2024-05-07T02:34:49.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.740+0000 7fbd21e60b80 1 -- 172.21.15.19:0/793944422 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5639569fa4b0 con 0x5639569e2f70 2024-05-07T02:34:49.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.740+0000 7fbcf2a00700 1 -- 172.21.15.19:0/793944422 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd00005040 con 0x5639569e2f70 2024-05-07T02:34:49.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.740+0000 7fbcf2a00700 1 -- 172.21.15.19:0/793944422 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7fbcf8031020 con 0x5639569e2f70 2024-05-07T02:34:49.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.741+0000 7fbd21e60b80 10 client.25771 fetch_fsmap finished waiting for FSMap version 34 2024-05-07T02:34:49.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.741+0000 7fbd21e60b80 10 client.25771 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-07T02:34:49.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.741+0000 7fbd21e60b80 20 client.25771 populate_metadata read hostname 'smithi019' 2024-05-07T02:34:49.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.741+0000 7fbd21e60b80 1 --2- 172.21.15.19:0/793944422 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x5639569fcb40 0x5639569fef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:49.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.741+0000 7fbd21e60b80 4 client.25771 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:34:49.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.741+0000 7fbd21e60b80 1 -- 172.21.15.19:0/793944422 --> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639569af0e0 con 0x5639569fcb40 2024-05-07T02:34:49.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.741+0000 7fbd10a00700 1 --2- 172.21.15.19:0/793944422 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x5639569fcb40 0x5639569fef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:49.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.742+0000 7fbd10a00700 1 --2- 172.21.15.19:0/793944422 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x5639569fcb40 0x5639569fef80 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1200353f4f4f36f9 server_cookie=a34b8154d704549d in_seq=0 out_seq=0 2024-05-07T02:34:49.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.742+0000 7fbcf2a00700 10 client.25771 ms_handle_connect on v2:172.21.15.64:6838/1583120004 2024-05-07T02:34:49.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.743+0000 7fbcf2a00700 1 -- 172.21.15.19:0/793944422 <== mds.2 v2:172.21.15.64:6838/1583120004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5639569af0e0 con 0x5639569fcb40 2024-05-07T02:34:49.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.743+0000 7fbcf2a00700 10 client.25771 handle_command_reply: tid=0 2024-05-07T02:34:49.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.743+0000 7fbd21e60b80 1 client.25771 shutdown 2024-05-07T02:34:49.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.743+0000 7fbd21e60b80 1 -- 172.21.15.19:0/793944422 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639569e2f70 msgr2=0x5639569ecc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:49.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.743+0000 7fbd21e60b80 1 --2- 172.21.15.19:0/793944422 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639569e2f70 0x5639569ecc10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8009750 tx=0x7fbcf80047c0 comp rx=0 tx=0).stop 2024-05-07T02:34:49.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.743+0000 7fbd21e60b80 1 -- 172.21.15.19:0/793944422 shutdown_connections 2024-05-07T02:34:49.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.743+0000 7fbd21e60b80 1 --2- 172.21.15.19:0/793944422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639569e8470 0x5639569ed150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:49.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.743+0000 7fbd21e60b80 1 --2- 172.21.15.19:0/793944422 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x5639569fcb40 0x5639569fef80 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:49.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.743+0000 7fbd21e60b80 1 --2- 172.21.15.19:0/793944422 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639569e2620 0x5639569ec6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:49.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.743+0000 7fbd21e60b80 1 --2- 172.21.15.19:0/793944422 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639569e2f70 0x5639569ecc10 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:49.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.743+0000 7fbcf2a00700 0 client.25771 ms_handle_reset on v2:172.21.15.64:6838/1583120004 2024-05-07T02:34:49.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.743+0000 7fbd21e60b80 1 -- 172.21.15.19:0/793944422 >> 172.21.15.19:0/793944422 conn(0x563956864e40 msgr2=0x5639569e1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:34:49.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.743+0000 7fbd21e60b80 1 -- 172.21.15.19:0/793944422 shutdown_connections 2024-05-07T02:34:49.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.744+0000 7fbd21e60b80 1 -- 172.21.15.19:0/793944422 wait complete. 2024-05-07T02:34:49.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.744+0000 7fbd21e60b80 20 client.25771 trim_cache size 0 max 16384 2024-05-07T02:34:49.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.761+0000 7fbd21e60b80 1 Processor -- start 2024-05-07T02:34:49.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.761+0000 7fbd21e60b80 1 -- start start 2024-05-07T02:34:49.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.762+0000 7fbd21e60b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639569e2620 0x5639569e1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:49.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.762+0000 7fbd21e60b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639569e2f70 0x5639569e9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:49.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.762+0000 7fbd21e60b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639569e8470 0x5639569ecc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:49.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.762+0000 7fbd21e60b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c1b6bd0 con 0x5639569e8470 2024-05-07T02:34:49.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.762+0000 7fbd21e60b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c1b87d0 con 0x5639569e2f70 2024-05-07T02:34:49.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.762+0000 7fbd21e60b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c1b72a0 con 0x5639569e2620 2024-05-07T02:34:49.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.762+0000 7fbd10a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639569e2620 0x5639569e1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:49.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.762+0000 7fbd10a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639569e2620 0x5639569e1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:59294/0 (socket says 172.21.15.19:59294) 2024-05-07T02:34:49.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.762+0000 7fbd10a00700 1 -- 172.21.15.19:0/3518374527 learned_addr learned my addr 172.21.15.19:0/3518374527 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:34:49.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.762+0000 7fbd11400700 1 --2- 172.21.15.19:0/3518374527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639569e8470 0x5639569ecc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:49.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.762+0000 7fbd0be00700 1 --2- 172.21.15.19:0/3518374527 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639569e2f70 0x5639569e9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:49.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.763+0000 7fbd10a00700 1 -- 172.21.15.19:0/3518374527 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639569e2f70 msgr2=0x5639569e9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:49.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.763+0000 7fbd10a00700 1 --2- 172.21.15.19:0/3518374527 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639569e2f70 0x5639569e9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:49.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.763+0000 7fbd10a00700 1 -- 172.21.15.19:0/3518374527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639569e8470 msgr2=0x5639569ecc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:49.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.763+0000 7fbd10a00700 1 --2- 172.21.15.19:0/3518374527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639569e8470 0x5639569ecc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:49.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.763+0000 7fbd10a00700 1 -- 172.21.15.19:0/3518374527 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639569ef4c0 con 0x5639569e2620 2024-05-07T02:34:49.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.763+0000 7fbd10a00700 1 --2- 172.21.15.19:0/3518374527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639569e2620 0x5639569e1d20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbd00005f40 tx=0x7fbd00005fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f31c6cadd1ff7d server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:34:49.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.763+0000 7fbcfd600700 1 -- 172.21.15.19:0/3518374527 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0000f460 con 0x5639569e2620 2024-05-07T02:34:49.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.763+0000 7fbcfd600700 1 -- 172.21.15.19:0/3518374527 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd00003680 con 0x5639569e2620 2024-05-07T02:34:49.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.763+0000 7fbcfd600700 1 -- 172.21.15.19:0/3518374527 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0000b7c0 con 0x5639569e2620 2024-05-07T02:34:49.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.764+0000 7fbd21e60b80 1 -- 172.21.15.19:0/3518374527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639569e2620 msgr2=0x5639569e1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:49.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.764+0000 7fbd21e60b80 1 --2- 172.21.15.19:0/3518374527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639569e2620 0x5639569e1d20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbd00005f40 tx=0x7fbd00005fa0 comp rx=0 tx=0).stop 2024-05-07T02:34:49.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.764+0000 7fbd21e60b80 1 -- 172.21.15.19:0/3518374527 shutdown_connections 2024-05-07T02:34:49.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.764+0000 7fbd21e60b80 1 --2- 172.21.15.19:0/3518374527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639569e8470 0x5639569ecc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:49.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.764+0000 7fbd21e60b80 1 --2- 172.21.15.19:0/3518374527 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639569e2620 0x5639569e1d20 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:49.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.764+0000 7fbd21e60b80 1 --2- 172.21.15.19:0/3518374527 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639569e2f70 0x5639569e9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:49.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.764+0000 7fbd21e60b80 1 -- 172.21.15.19:0/3518374527 >> 172.21.15.19:0/3518374527 conn(0x563956864e40 msgr2=0x5639569e1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:34:49.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.765+0000 7fbd21e60b80 1 -- 172.21.15.19:0/3518374527 shutdown_connections 2024-05-07T02:34:49.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.765+0000 7fbd21e60b80 1 -- 172.21.15.19:0/3518374527 wait complete. 2024-05-07T02:34:49.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.765+0000 7fbd21e60b80 1 -- start start 2024-05-07T02:34:49.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.765+0000 7fbd21e60b80 1 Processor -- start 2024-05-07T02:34:49.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.766+0000 7fbd21e60b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639569e2620 0x5639569f52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:49.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.766+0000 7fbd21e60b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639569e2f70 0x563956a03db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:49.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.766+0000 7fbd21e60b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639569e8470 0x563956a043e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:49.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.766+0000 7fbd21e60b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c1b8040 con 0x5639569e8470 2024-05-07T02:34:49.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.766+0000 7fbd21e60b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c1b7970 con 0x5639569e2f70 2024-05-07T02:34:49.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.766+0000 7fbd21e60b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c1b4b70 con 0x5639569e2620 2024-05-07T02:34:49.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.766+0000 7fbd21e60b80 10 client.0 _ll_register_callbacks cb 0x563956898380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:34:49.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.766+0000 7fbd0be00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639569e2f70 0x563956a03db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:49.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.766+0000 7fbd10a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639569e2620 0x5639569f52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:49.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.766+0000 7fbd0be00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639569e2f70 0x563956a03db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:36462/0 (socket says 172.21.15.19:36462) 2024-05-07T02:34:49.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.766+0000 7fbd10a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639569e2620 0x5639569f52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:59308/0 (socket says 172.21.15.19:59308) 2024-05-07T02:34:49.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.766+0000 7fbd0be00700 1 -- 172.21.15.19:0/3161946905 learned_addr learned my addr 172.21.15.19:0/3161946905 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:34:49.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.766+0000 7fbd11400700 1 --2- 172.21.15.19:0/3161946905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639569e8470 0x563956a043e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:49.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.767+0000 7fbd0be00700 1 -- 172.21.15.19:0/3161946905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639569e2620 msgr2=0x5639569f52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:49.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.767+0000 7fbd0be00700 1 --2- 172.21.15.19:0/3161946905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639569e2620 0x5639569f52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:49.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.767+0000 7fbd0be00700 1 -- 172.21.15.19:0/3161946905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639569e8470 msgr2=0x563956a043e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:49.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.767+0000 7fbd0be00700 1 --2- 172.21.15.19:0/3161946905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639569e8470 0x563956a043e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:49.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.767+0000 7fbd0be00700 1 -- 172.21.15.19:0/3161946905 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563956a04b30 con 0x5639569e2f70 2024-05-07T02:34:49.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.767+0000 7fbd0be00700 1 --2- 172.21.15.19:0/3161946905 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639569e2f70 0x563956a03db0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8029aa0 tx=0x7fbcf80206e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6eb33e5fb8bc9fe6 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:34:49.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.767+0000 7fbcf2a00700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:34:49.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.767+0000 7fbcf2a00700 1 -- 172.21.15.19:0/3161946905 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf8024040 con 0x5639569e2f70 2024-05-07T02:34:49.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.768+0000 7fbcf2a00700 1 -- 172.21.15.19:0/3161946905 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fbcf8029850 con 0x5639569e2f70 2024-05-07T02:34:49.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.768+0000 7fbd21e60b80 1 -- 172.21.15.19:0/3161946905 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd00003680 con 0x5639569e2f70 2024-05-07T02:34:49.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.768+0000 7fbcf2a00700 1 -- 172.21.15.19:0/3161946905 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbcf8040050 con 0x5639569e2f70 2024-05-07T02:34:49.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.768+0000 7fbd21e60b80 10 client.25790 fetch_fsmap learned FSMap version 34 2024-05-07T02:34:49.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.768+0000 7fbd21e60b80 1 -- 172.21.15.19:0/3161946905 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563956a67320 con 0x5639569e2f70 2024-05-07T02:34:49.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.768+0000 7fbcf2a00700 1 -- 172.21.15.19:0/3161946905 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd00003680 con 0x5639569e2f70 2024-05-07T02:34:49.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.769+0000 7fbcf2a00700 1 -- 172.21.15.19:0/3161946905 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7fbcf8031030 con 0x5639569e2f70 2024-05-07T02:34:49.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.769+0000 7fbd21e60b80 10 client.25790 fetch_fsmap finished waiting for FSMap version 34 2024-05-07T02:34:49.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.769+0000 7fbd21e60b80 10 client.25790 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-07T02:34:49.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.769+0000 7fbd21e60b80 20 client.25790 populate_metadata read hostname 'smithi019' 2024-05-07T02:34:49.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.769+0000 7fbd21e60b80 1 --2- 172.21.15.19:0/3161946905 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x563956a679f0 0x563956a69e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:49.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.769+0000 7fbd21e60b80 4 client.25790 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:34:49.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.769+0000 7fbd21e60b80 1 -- 172.21.15.19:0/3161946905 --> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563956912400 con 0x563956a679f0 2024-05-07T02:34:49.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.769+0000 7fbd10a00700 1 --2- 172.21.15.19:0/3161946905 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x563956a679f0 0x563956a69e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:49.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.770+0000 7fbd10a00700 1 --2- 172.21.15.19:0/3161946905 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x563956a679f0 0x563956a69e10 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dae6425d9a88ba33 server_cookie=478e8d7aa19be4e8 in_seq=0 out_seq=0 2024-05-07T02:34:49.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.770+0000 7fbcf2a00700 10 client.25790 ms_handle_connect on v2:172.21.15.64:6838/1583120004 2024-05-07T02:34:49.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.770+0000 7fbcf2a00700 1 -- 172.21.15.19:0/3161946905 <== mds.2 v2:172.21.15.64:6838/1583120004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563956912400 con 0x563956a679f0 2024-05-07T02:34:49.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.770+0000 7fbcf2a00700 10 client.25790 handle_command_reply: tid=0 2024-05-07T02:34:49.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.770+0000 7fbd21e60b80 1 client.25790 shutdown 2024-05-07T02:34:49.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.770+0000 7fbd21e60b80 1 -- 172.21.15.19:0/3161946905 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639569e2f70 msgr2=0x563956a03db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:49.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.770+0000 7fbd21e60b80 1 --2- 172.21.15.19:0/3161946905 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639569e2f70 0x563956a03db0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8029aa0 tx=0x7fbcf80206e0 comp rx=0 tx=0).stop 2024-05-07T02:34:49.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.770+0000 7fbd21e60b80 1 -- 172.21.15.19:0/3161946905 shutdown_connections 2024-05-07T02:34:49.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.770+0000 7fbd21e60b80 1 --2- 172.21.15.19:0/3161946905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639569e8470 0x563956a043e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:49.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.770+0000 7fbd21e60b80 1 --2- 172.21.15.19:0/3161946905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639569e2620 0x5639569f52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:49.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.770+0000 7fbd21e60b80 1 --2- 172.21.15.19:0/3161946905 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x563956a679f0 0x563956a69e10 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:49.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.771+0000 7fbd21e60b80 1 --2- 172.21.15.19:0/3161946905 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639569e2f70 0x563956a03db0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:49.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.771+0000 7fbcf2a00700 0 client.25790 ms_handle_reset on v2:172.21.15.64:6838/1583120004 2024-05-07T02:34:49.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.771+0000 7fbd21e60b80 1 -- 172.21.15.19:0/3161946905 >> 172.21.15.19:0/3161946905 conn(0x563956864e40 msgr2=0x5639569edb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:34:49.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.771+0000 7fbd21e60b80 1 -- 172.21.15.19:0/3161946905 shutdown_connections 2024-05-07T02:34:49.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.771+0000 7fbd21e60b80 1 -- 172.21.15.19:0/3161946905 wait complete. 2024-05-07T02:34:49.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.771+0000 7fbd21e60b80 20 client.25790 trim_cache size 0 max 16384 2024-05-07T02:34:49.773 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:34:49.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.772+0000 7fbd13400700 1 -- 172.21.15.19:0/1049807634 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fbcf40b2a70 msgr2=0x7fbcf40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:49.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.772+0000 7fbd13400700 1 --2- 172.21.15.19:0/1049807634 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fbcf40b2a70 0x7fbcf40b4f80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c107dc0 tx=0x7fbcf8013980 comp rx=0 tx=0).stop 2024-05-07T02:34:49.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.772+0000 7fbd13400700 1 -- 172.21.15.19:0/1049807634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd0c108740 msgr2=0x7fbd0c1a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:49.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.772+0000 7fbd13400700 1 --2- 172.21.15.19:0/1049807634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd0c108740 0x7fbd0c1a5c70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbd0400efc0 tx=0x7fbd0400c3a0 comp rx=0 tx=0).stop 2024-05-07T02:34:49.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.772+0000 7fbd13400700 1 -- 172.21.15.19:0/1049807634 shutdown_connections 2024-05-07T02:34:49.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.772+0000 7fbd13400700 1 --2- 172.21.15.19:0/1049807634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd0c108740 0x7fbd0c1a5c70 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:49.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.772+0000 7fbd13400700 1 --2- 172.21.15.19:0/1049807634 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fbcf40b2a70 0x7fbcf40b4f80 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:49.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.772+0000 7fbd13400700 1 --2- 172.21.15.19:0/1049807634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd0c1034c0 0x7fbd0c1a1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:49.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.772+0000 7fbd13400700 1 --2- 172.21.15.19:0/1049807634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbd0c107df0 0x7fbd0c1a1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:49.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.772+0000 7fbd13400700 1 -- 172.21.15.19:0/1049807634 >> 172.21.15.19:0/1049807634 conn(0x7fbd0c0fedc0 msgr2=0x7fbd0c06b610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:34:49.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.773+0000 7fbd13400700 1 -- 172.21.15.19:0/1049807634 shutdown_connections 2024-05-07T02:34:49.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:49.773+0000 7fbd13400700 1 -- 172.21.15.19:0/1049807634 wait complete. 2024-05-07T02:34:50.310 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:34:50 smithi019 ceph-mon[29781]: pgmap v93: 129 pgs: 129 active+clean; 48 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 689 KiB/s wr, 64 op/s 2024-05-07T02:34:50.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:34:50 smithi064 ceph-mon[34699]: pgmap v93: 129 pgs: 129 active+clean; 48 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 689 KiB/s wr, 64 op/s 2024-05-07T02:34:50.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:34:50 smithi027 ceph-mon[34398]: pgmap v93: 129 pgs: 129 active+clean; 48 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 689 KiB/s wr, 64 op/s 2024-05-07T02:34:50.733 INFO:tasks.workunit.client.0.smithi019.stdout: 1 60579 3.26 MB/sec warmup 106 sec latency 21.903 ms 2024-05-07T02:34:51.040 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-07T02:34:51.574 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:34:51.733 INFO:tasks.workunit.client.0.smithi019.stdout: 1 61050 3.24 MB/sec warmup 107 sec latency 23.047 ms 2024-05-07T02:34:52.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:34:52 smithi064 ceph-mon[34699]: pgmap v94: 129 pgs: 129 active+clean; 48 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 1.3 MiB/s wr, 141 op/s 2024-05-07T02:34:52.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:34:52 smithi019 ceph-mon[29781]: pgmap v94: 129 pgs: 129 active+clean; 48 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 1.3 MiB/s wr, 141 op/s 2024-05-07T02:34:52.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:34:52 smithi027 ceph-mon[34398]: pgmap v94: 129 pgs: 129 active+clean; 48 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 1.3 MiB/s wr, 141 op/s 2024-05-07T02:34:52.733 INFO:tasks.workunit.client.0.smithi019.stdout: 1 62665 3.27 MB/sec warmup 108 sec latency 14.409 ms 2024-05-07T02:34:53.734 INFO:tasks.workunit.client.0.smithi019.stdout: 1 63950 3.34 MB/sec warmup 109 sec latency 54.645 ms 2024-05-07T02:34:54.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.076+0000 7ff771e00700 1 -- 172.21.15.19:0/1125715627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff76c108d30 msgr2=0x7ff76c109140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:54.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.076+0000 7ff771e00700 1 --2- 172.21.15.19:0/1125715627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff76c108d30 0x7ff76c109140 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff758009990 tx=0x7ff758009ca0 comp rx=0 tx=0).stop 2024-05-07T02:34:54.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.076+0000 7ff771e00700 1 -- 172.21.15.19:0/1125715627 shutdown_connections 2024-05-07T02:34:54.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.076+0000 7ff771e00700 1 --2- 172.21.15.19:0/1125715627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff76c109680 0x7ff76c110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:54.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.076+0000 7ff771e00700 1 --2- 172.21.15.19:0/1125715627 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff76c104400 0x7ff76c104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:54.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.076+0000 7ff771e00700 1 --2- 172.21.15.19:0/1125715627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff76c108d30 0x7ff76c109140 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:54.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.076+0000 7ff771e00700 1 -- 172.21.15.19:0/1125715627 >> 172.21.15.19:0/1125715627 conn(0x7ff76c079d10 msgr2=0x7ff76c07a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:34:54.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.076+0000 7ff771e00700 1 -- 172.21.15.19:0/1125715627 shutdown_connections 2024-05-07T02:34:54.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.076+0000 7ff771e00700 1 -- 172.21.15.19:0/1125715627 wait complete. 2024-05-07T02:34:54.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.078+0000 7ff771e00700 1 Processor -- start 2024-05-07T02:34:54.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.078+0000 7ff771e00700 1 -- start start 2024-05-07T02:34:54.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.079+0000 7ff771e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff76c104400 0x7ff76c19ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:54.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.079+0000 7ff771e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff76c108d30 0x7ff76c19d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:54.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.079+0000 7ff771e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff76c109680 0x7ff76c1a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:54.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.079+0000 7ff771e00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff76c1142e0 con 0x7ff76c104400 2024-05-07T02:34:54.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.079+0000 7ff771e00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff76c114160 con 0x7ff76c109680 2024-05-07T02:34:54.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.079+0000 7ff771e00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff76c114460 con 0x7ff76c108d30 2024-05-07T02:34:54.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.079+0000 7ff76be00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff76c108d30 0x7ff76c19d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:54.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.079+0000 7ff770a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff76c104400 0x7ff76c19ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:54.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.079+0000 7ff76be00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff76c108d30 0x7ff76c19d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:59322/0 (socket says 172.21.15.19:59322) 2024-05-07T02:34:54.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.079+0000 7ff76be00700 1 -- 172.21.15.19:0/3806420950 learned_addr learned my addr 172.21.15.19:0/3806420950 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:34:54.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.079+0000 7ff771400700 1 --2- 172.21.15.19:0/3806420950 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff76c109680 0x7ff76c1a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:54.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.080+0000 7ff76be00700 1 -- 172.21.15.19:0/3806420950 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff76c109680 msgr2=0x7ff76c1a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:54.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.080+0000 7ff76be00700 1 --2- 172.21.15.19:0/3806420950 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff76c109680 0x7ff76c1a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:54.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.080+0000 7ff76be00700 1 -- 172.21.15.19:0/3806420950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff76c104400 msgr2=0x7ff76c19ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:54.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.080+0000 7ff76be00700 1 --2- 172.21.15.19:0/3806420950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff76c104400 0x7ff76c19ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:54.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.080+0000 7ff76be00700 1 -- 172.21.15.19:0/3806420950 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff76c1a1d60 con 0x7ff76c108d30 2024-05-07T02:34:54.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.080+0000 7ff76be00700 1 --2- 172.21.15.19:0/3806420950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff76c108d30 0x7ff76c19d430 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff758000c00 tx=0x7ff758009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:34:54.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.080+0000 7ff769600700 1 -- 172.21.15.19:0/3806420950 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff758026070 con 0x7ff76c108d30 2024-05-07T02:34:54.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.080+0000 7ff769600700 1 -- 172.21.15.19:0/3806420950 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff75800b610 con 0x7ff76c108d30 2024-05-07T02:34:54.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.080+0000 7ff771e00700 1 -- 172.21.15.19:0/3806420950 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff76c1a1ff0 con 0x7ff76c108d30 2024-05-07T02:34:54.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.081+0000 7ff769600700 1 -- 172.21.15.19:0/3806420950 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff758021e40 con 0x7ff76c108d30 2024-05-07T02:34:54.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.081+0000 7ff771e00700 1 -- 172.21.15.19:0/3806420950 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff76c1a9970 con 0x7ff76c108d30 2024-05-07T02:34:54.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.084+0000 7ff769600700 1 -- 172.21.15.19:0/3806420950 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff758029070 con 0x7ff76c108d30 2024-05-07T02:34:54.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.086+0000 7ff781e87b80 1 Processor -- start 2024-05-07T02:34:54.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.086+0000 7ff769600700 1 --2- 172.21.15.19:0/3806420950 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff7540b2a70 0x7ff7540b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:54.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.086+0000 7ff781e87b80 1 -- start start 2024-05-07T02:34:54.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.086+0000 7ff769600700 1 -- 172.21.15.19:0/3806420950 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7ff7580e3900 con 0x7ff76c108d30 2024-05-07T02:34:54.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.087+0000 7ff771400700 1 --2- 172.21.15.19:0/3806420950 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff7540b2a70 0x7ff7540b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:54.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.087+0000 7ff781e87b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598170f5a90 0x5598170f5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:54.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.087+0000 7ff781e87b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598170f63e0 0x5598170ff620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:54.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.087+0000 7ff781e87b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598170ffb60 0x559817101f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:54.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.087+0000 7ff781e87b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559817017a20 con 0x5598170f63e0 2024-05-07T02:34:54.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.087+0000 7ff781e87b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559816fb5760 con 0x5598170f5a90 2024-05-07T02:34:54.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.087+0000 7ff781e87b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5598170139e0 con 0x5598170ffb60 2024-05-07T02:34:54.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.087+0000 7ff770a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598170f5a90 0x5598170f5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:54.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.087+0000 7ff771400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598170ffb60 0x559817101f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:54.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.087+0000 7ff770a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598170f5a90 0x5598170f5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:36494/0 (socket says 172.21.15.19:36494) 2024-05-07T02:34:54.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.087+0000 7ff770a00700 1 -- 172.21.15.19:0/1257526764 learned_addr learned my addr 172.21.15.19:0/1257526764 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:34:54.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.087+0000 7ff76be00700 1 --2- 172.21.15.19:0/1257526764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598170f63e0 0x5598170ff620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:54.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.088+0000 7ff770a00700 1 -- 172.21.15.19:0/1257526764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598170ffb60 msgr2=0x559817101f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:54.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.088+0000 7ff770a00700 1 --2- 172.21.15.19:0/1257526764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598170ffb60 0x559817101f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:54.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.088+0000 7ff770a00700 1 -- 172.21.15.19:0/1257526764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598170f63e0 msgr2=0x5598170ff620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:54.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.088+0000 7ff770a00700 1 --2- 172.21.15.19:0/1257526764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598170f63e0 0x5598170ff620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:54.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.088+0000 7ff770a00700 1 -- 172.21.15.19:0/1257526764 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559816f648d0 con 0x5598170f5a90 2024-05-07T02:34:54.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.088+0000 7ff770a00700 1 --2- 172.21.15.19:0/1257526764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598170f5a90 0x5598170f5ea0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff760009d80 tx=0x7ff760011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc215a79b2f4078e server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:34:54.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.088+0000 7ff771400700 1 --2- 172.21.15.19:0/3806420950 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff7540b2a70 0x7ff7540b4f80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff76400f870 tx=0x7ff76400fc30 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:34:54.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.089+0000 7ff75e000700 1 -- 172.21.15.19:0/1257526764 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff760004230 con 0x5598170f5a90 2024-05-07T02:34:54.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.089+0000 7ff75e000700 1 -- 172.21.15.19:0/1257526764 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7600043d0 con 0x5598170f5a90 2024-05-07T02:34:54.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.089+0000 7ff75e000700 1 -- 172.21.15.19:0/1257526764 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff760019760 con 0x5598170f5a90 2024-05-07T02:34:54.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.090+0000 7ff781e87b80 1 -- 172.21.15.19:0/1257526764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598170f5a90 msgr2=0x5598170f5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:54.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.090+0000 7ff781e87b80 1 --2- 172.21.15.19:0/1257526764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598170f5a90 0x5598170f5ea0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff760009d80 tx=0x7ff760011720 comp rx=0 tx=0).stop 2024-05-07T02:34:54.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.090+0000 7ff781e87b80 1 -- 172.21.15.19:0/1257526764 shutdown_connections 2024-05-07T02:34:54.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.090+0000 7ff781e87b80 1 --2- 172.21.15.19:0/1257526764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598170ffb60 0x559817101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:54.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.090+0000 7ff781e87b80 1 --2- 172.21.15.19:0/1257526764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598170f5a90 0x5598170f5ea0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:54.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.090+0000 7ff781e87b80 1 --2- 172.21.15.19:0/1257526764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598170f63e0 0x5598170ff620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:54.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.090+0000 7ff781e87b80 1 -- 172.21.15.19:0/1257526764 >> 172.21.15.19:0/1257526764 conn(0x559816f77e40 msgr2=0x5598170f0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:34:54.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.090+0000 7ff781e87b80 1 -- 172.21.15.19:0/1257526764 shutdown_connections 2024-05-07T02:34:54.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.090+0000 7ff781e87b80 1 -- 172.21.15.19:0/1257526764 wait complete. 2024-05-07T02:34:54.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.091+0000 7ff781e87b80 1 -- start start 2024-05-07T02:34:54.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.091+0000 7ff781e87b80 1 Processor -- start 2024-05-07T02:34:54.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.092+0000 7ff781e87b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598170f5a90 0x5598170fca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:54.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.092+0000 7ff781e87b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598170f63e0 0x5598170fcfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:54.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.092+0000 7ff781e87b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598170ffb60 0x5598170fd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:54.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.092+0000 7ff781e87b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559817102960 con 0x5598170ffb60 2024-05-07T02:34:54.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.092+0000 7ff781e87b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x559816f7db20 con 0x5598170f5a90 2024-05-07T02:34:54.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.092+0000 7ff781e87b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559817102ae0 con 0x5598170f63e0 2024-05-07T02:34:54.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.092+0000 7ff781e87b80 10 client.0 _ll_register_callbacks cb 0x559816fbec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:34:54.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.092+0000 7ff770a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598170f5a90 0x5598170fca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:54.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.092+0000 7ff771400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598170ffb60 0x5598170fd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:54.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.092+0000 7ff770a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598170f5a90 0x5598170fca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:36500/0 (socket says 172.21.15.19:36500) 2024-05-07T02:34:54.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.092+0000 7ff770a00700 1 -- 172.21.15.19:0/2416014215 learned_addr learned my addr 172.21.15.19:0/2416014215 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:34:54.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.092+0000 7ff76be00700 1 --2- 172.21.15.19:0/2416014215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598170f63e0 0x5598170fcfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:54.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.093+0000 7ff771400700 1 -- 172.21.15.19:0/2416014215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598170f63e0 msgr2=0x5598170fcfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:54.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.093+0000 7ff771400700 1 --2- 172.21.15.19:0/2416014215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598170f63e0 0x5598170fcfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:54.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.093+0000 7ff771400700 1 -- 172.21.15.19:0/2416014215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598170f5a90 msgr2=0x5598170fca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:54.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.093+0000 7ff771400700 1 --2- 172.21.15.19:0/2416014215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598170f5a90 0x5598170fca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:54.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.093+0000 7ff771400700 1 -- 172.21.15.19:0/2416014215 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559816fdd420 con 0x5598170ffb60 2024-05-07T02:34:54.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.093+0000 7ff76be00700 1 --2- 172.21.15.19:0/2416014215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598170f63e0 0x5598170fcfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-07T02:34:54.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.093+0000 7ff771400700 1 --2- 172.21.15.19:0/2416014215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598170ffb60 0x5598170fd510 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff76400f650 tx=0x7ff764004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b074a4d83214f0a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:34:54.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.093+0000 7ff752a00700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-07T02:34:54.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.093+0000 7ff752a00700 1 -- 172.21.15.19:0/2416014215 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff764027030 con 0x5598170ffb60 2024-05-07T02:34:54.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.093+0000 7ff752a00700 1 -- 172.21.15.19:0/2416014215 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7ff764004bb0 con 0x5598170ffb60 2024-05-07T02:34:54.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.094+0000 7ff781e87b80 1 -- 172.21.15.19:0/2416014215 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7600043d0 con 0x5598170ffb60 2024-05-07T02:34:54.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.094+0000 7ff752a00700 1 -- 172.21.15.19:0/2416014215 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff764031050 con 0x5598170ffb60 2024-05-07T02:34:54.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.094+0000 7ff781e87b80 10 client.15642 fetch_fsmap learned FSMap version 34 2024-05-07T02:34:54.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.094+0000 7ff781e87b80 1 -- 172.21.15.19:0/2416014215 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55981710d550 con 0x5598170ffb60 2024-05-07T02:34:54.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.094+0000 7ff752a00700 1 -- 172.21.15.19:0/2416014215 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7600043d0 con 0x5598170ffb60 2024-05-07T02:34:54.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.094+0000 7ff752a00700 1 -- 172.21.15.19:0/2416014215 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7ff76402c8e0 con 0x5598170ffb60 2024-05-07T02:34:54.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.095+0000 7ff781e87b80 10 client.15642 fetch_fsmap finished waiting for FSMap version 34 2024-05-07T02:34:54.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.095+0000 7ff781e87b80 10 client.15642 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-07T02:34:54.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.095+0000 7ff781e87b80 20 client.15642 populate_metadata read hostname 'smithi019' 2024-05-07T02:34:54.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.095+0000 7ff781e87b80 1 --2- 172.21.15.19:0/2416014215 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x55981710fbe0 0x559817112040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:54.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.095+0000 7ff781e87b80 4 client.15642 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:34:54.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.095+0000 7ff781e87b80 1 -- 172.21.15.19:0/2416014215 --> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598170c22e0 con 0x55981710fbe0 2024-05-07T02:34:54.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.095+0000 7ff770a00700 1 --2- 172.21.15.19:0/2416014215 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x55981710fbe0 0x559817112040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:54.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.096+0000 7ff770a00700 1 --2- 172.21.15.19:0/2416014215 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x55981710fbe0 0x559817112040 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8518eab2395089d server_cookie=5f7075d122cdab8e in_seq=0 out_seq=0 2024-05-07T02:34:54.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.096+0000 7ff752a00700 10 client.15642 ms_handle_connect on v2:172.21.15.19:6840/3905645246 2024-05-07T02:34:54.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.097+0000 7ff752a00700 1 -- 172.21.15.19:0/2416014215 <== mds.1 v2:172.21.15.19:6840/3905645246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5598170c22e0 con 0x55981710fbe0 2024-05-07T02:34:54.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.097+0000 7ff752a00700 10 client.15642 handle_command_reply: tid=0 2024-05-07T02:34:54.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.097+0000 7ff781e87b80 1 client.15642 shutdown 2024-05-07T02:34:54.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.097+0000 7ff781e87b80 1 -- 172.21.15.19:0/2416014215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598170ffb60 msgr2=0x5598170fd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:54.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.097+0000 7ff781e87b80 1 --2- 172.21.15.19:0/2416014215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598170ffb60 0x5598170fd510 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff76400f650 tx=0x7ff764004b40 comp rx=0 tx=0).stop 2024-05-07T02:34:54.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.098+0000 7ff781e87b80 1 -- 172.21.15.19:0/2416014215 shutdown_connections 2024-05-07T02:34:54.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.098+0000 7ff781e87b80 1 --2- 172.21.15.19:0/2416014215 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x55981710fbe0 0x559817112040 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:54.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.098+0000 7ff781e87b80 1 --2- 172.21.15.19:0/2416014215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598170ffb60 0x5598170fd510 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:54.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.098+0000 7ff781e87b80 1 --2- 172.21.15.19:0/2416014215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598170f5a90 0x5598170fca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:54.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.098+0000 7ff781e87b80 1 --2- 172.21.15.19:0/2416014215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598170f63e0 0x5598170fcfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:54.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.098+0000 7ff752a00700 0 client.15642 ms_handle_reset on v2:172.21.15.19:6840/3905645246 2024-05-07T02:34:54.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.098+0000 7ff781e87b80 1 -- 172.21.15.19:0/2416014215 >> 172.21.15.19:0/2416014215 conn(0x559816f77e40 msgr2=0x5598170f2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:34:54.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.098+0000 7ff781e87b80 1 -- 172.21.15.19:0/2416014215 shutdown_connections 2024-05-07T02:34:54.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.099+0000 7ff781e87b80 1 -- 172.21.15.19:0/2416014215 wait complete. 2024-05-07T02:34:54.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.099+0000 7ff781e87b80 20 client.15642 trim_cache size 0 max 16384 2024-05-07T02:34:54.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.116+0000 7ff781e87b80 1 Processor -- start 2024-05-07T02:34:54.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.117+0000 7ff781e87b80 1 -- start start 2024-05-07T02:34:54.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.117+0000 7ff781e87b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559816f77e40 0x5598170f4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:54.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.117+0000 7ff781e87b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598170f5a90 0x55981710ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:54.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.117+0000 7ff781e87b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598170f63e0 0x559817117800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:54.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.117+0000 7ff781e87b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff76c1b2480 con 0x559816f77e40 2024-05-07T02:34:54.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.117+0000 7ff781e87b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff76c1b4080 con 0x5598170f5a90 2024-05-07T02:34:54.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.117+0000 7ff781e87b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff76c1b2b50 con 0x5598170f63e0 2024-05-07T02:34:54.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.118+0000 7ff76be00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598170f5a90 0x55981710ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:54.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.118+0000 7ff771400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598170f63e0 0x559817117800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:54.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.118+0000 7ff770a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559816f77e40 0x5598170f4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:54.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.118+0000 7ff76be00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598170f5a90 0x55981710ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:36506/0 (socket says 172.21.15.19:36506) 2024-05-07T02:34:54.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.118+0000 7ff76be00700 1 -- 172.21.15.19:0/3954901744 learned_addr learned my addr 172.21.15.19:0/3954901744 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:34:54.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.118+0000 7ff76be00700 1 -- 172.21.15.19:0/3954901744 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598170f63e0 msgr2=0x559817117800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:54.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.118+0000 7ff76be00700 1 --2- 172.21.15.19:0/3954901744 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598170f63e0 0x559817117800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:54.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.118+0000 7ff76be00700 1 -- 172.21.15.19:0/3954901744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559816f77e40 msgr2=0x5598170f4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:54.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.118+0000 7ff76be00700 1 --2- 172.21.15.19:0/3954901744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559816f77e40 0x5598170f4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:54.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.118+0000 7ff76be00700 1 -- 172.21.15.19:0/3954901744 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598170f6830 con 0x5598170f5a90 2024-05-07T02:34:54.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.118+0000 7ff771400700 1 --2- 172.21.15.19:0/3954901744 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598170f63e0 0x559817117800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:34:54.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.118+0000 7ff76be00700 1 --2- 172.21.15.19:0/3954901744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598170f5a90 0x55981710ba50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff7580aa580 tx=0x7ff758009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93a98b5bf154bf9a server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:34:54.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.119+0000 7ff75d600700 1 -- 172.21.15.19:0/3954901744 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7580aa390 con 0x5598170f5a90 2024-05-07T02:34:54.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.119+0000 7ff75d600700 1 -- 172.21.15.19:0/3954901744 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7580aaf80 con 0x5598170f5a90 2024-05-07T02:34:54.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.119+0000 7ff75d600700 1 -- 172.21.15.19:0/3954901744 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7580215c0 con 0x5598170f5a90 2024-05-07T02:34:54.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.120+0000 7ff781e87b80 1 -- 172.21.15.19:0/3954901744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598170f5a90 msgr2=0x55981710ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:54.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.120+0000 7ff781e87b80 1 --2- 172.21.15.19:0/3954901744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598170f5a90 0x55981710ba50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff7580aa580 tx=0x7ff758009310 comp rx=0 tx=0).stop 2024-05-07T02:34:54.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.120+0000 7ff781e87b80 1 -- 172.21.15.19:0/3954901744 shutdown_connections 2024-05-07T02:34:54.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.120+0000 7ff781e87b80 1 --2- 172.21.15.19:0/3954901744 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598170f63e0 0x559817117800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:54.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.120+0000 7ff781e87b80 1 --2- 172.21.15.19:0/3954901744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559816f77e40 0x5598170f4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:54.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.120+0000 7ff781e87b80 1 --2- 172.21.15.19:0/3954901744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5598170f5a90 0x55981710ba50 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:54.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.120+0000 7ff781e87b80 1 -- 172.21.15.19:0/3954901744 >> 172.21.15.19:0/3954901744 conn(0x559817102250 msgr2=0x5598170fd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:34:54.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.120+0000 7ff781e87b80 1 -- 172.21.15.19:0/3954901744 shutdown_connections 2024-05-07T02:34:54.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.120+0000 7ff781e87b80 1 -- 172.21.15.19:0/3954901744 wait complete. 2024-05-07T02:34:54.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.121+0000 7ff781e87b80 1 -- start start 2024-05-07T02:34:54.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.121+0000 7ff781e87b80 1 Processor -- start 2024-05-07T02:34:54.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.122+0000 7ff781e87b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559816f77e40 0x5598170f9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:54.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.122+0000 7ff781e87b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598170f5a90 0x5598170fa080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:54.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.122+0000 7ff781e87b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598170f63e0 0x5598170fa5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:54.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.122+0000 7ff781e87b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff76c1b38f0 con 0x5598170f63e0 2024-05-07T02:34:54.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.122+0000 7ff781e87b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff76c1b3220 con 0x559816f77e40 2024-05-07T02:34:54.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.122+0000 7ff781e87b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff76c1b0420 con 0x5598170f5a90 2024-05-07T02:34:54.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.122+0000 7ff781e87b80 10 client.0 _ll_register_callbacks cb 0x559816fbec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:34:54.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.122+0000 7ff770a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559816f77e40 0x5598170f9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:54.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.122+0000 7ff770a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559816f77e40 0x5598170f9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:36514/0 (socket says 172.21.15.19:36514) 2024-05-07T02:34:54.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.122+0000 7ff770a00700 1 -- 172.21.15.19:0/505907560 learned_addr learned my addr 172.21.15.19:0/505907560 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:34:54.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.122+0000 7ff76be00700 1 --2- 172.21.15.19:0/505907560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598170f5a90 0x5598170fa080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:54.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.122+0000 7ff770a00700 1 -- 172.21.15.19:0/505907560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598170f5a90 msgr2=0x5598170fa080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:54.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.123+0000 7ff770a00700 1 --2- 172.21.15.19:0/505907560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598170f5a90 0x5598170fa080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:54.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.123+0000 7ff770a00700 1 -- 172.21.15.19:0/505907560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598170f63e0 msgr2=0x5598170fa5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:54.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.123+0000 7ff770a00700 1 --2- 172.21.15.19:0/505907560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598170f63e0 0x5598170fa5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:54.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.123+0000 7ff770a00700 1 -- 172.21.15.19:0/505907560 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598171795c0 con 0x559816f77e40 2024-05-07T02:34:54.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.123+0000 7ff76be00700 1 --2- 172.21.15.19:0/505907560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598170f5a90 0x5598170fa080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:34:54.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.123+0000 7ff770a00700 1 --2- 172.21.15.19:0/505907560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559816f77e40 0x5598170f9b40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff760009cd0 tx=0x7ff760005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5faed1f81e27176 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:34:54.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.123+0000 7ff752a00700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:34:54.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.124+0000 7ff752a00700 1 -- 172.21.15.19:0/505907560 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff760003880 con 0x559816f77e40 2024-05-07T02:34:54.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.124+0000 7ff781e87b80 1 -- 172.21.15.19:0/505907560 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7580aaf80 con 0x559816f77e40 2024-05-07T02:34:54.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.124+0000 7ff752a00700 1 -- 172.21.15.19:0/505907560 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7ff760003a20 con 0x559816f77e40 2024-05-07T02:34:54.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.124+0000 7ff752a00700 1 -- 172.21.15.19:0/505907560 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff760039050 con 0x559816f77e40 2024-05-07T02:34:54.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.124+0000 7ff781e87b80 10 client.25808 fetch_fsmap learned FSMap version 34 2024-05-07T02:34:54.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.124+0000 7ff781e87b80 1 -- 172.21.15.19:0/505907560 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55981717a820 con 0x559816f77e40 2024-05-07T02:34:54.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.124+0000 7ff752a00700 1 -- 172.21.15.19:0/505907560 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7580aaf80 con 0x559816f77e40 2024-05-07T02:34:54.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.124+0000 7ff752a00700 1 -- 172.21.15.19:0/505907560 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7ff760019420 con 0x559816f77e40 2024-05-07T02:34:54.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.125+0000 7ff781e87b80 10 client.25808 fetch_fsmap finished waiting for FSMap version 34 2024-05-07T02:34:54.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.125+0000 7ff781e87b80 10 client.25808 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-07T02:34:54.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.125+0000 7ff781e87b80 20 client.25808 populate_metadata read hostname 'smithi019' 2024-05-07T02:34:54.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.125+0000 7ff781e87b80 1 --2- 172.21.15.19:0/505907560 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5598170ffb60 0x55981717cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:54.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.125+0000 7ff781e87b80 4 client.25808 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:34:54.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.125+0000 7ff781e87b80 1 -- 172.21.15.19:0/505907560 --> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559817025400 con 0x5598170ffb60 2024-05-07T02:34:54.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.125+0000 7ff76be00700 1 --2- 172.21.15.19:0/505907560 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5598170ffb60 0x55981717cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:54.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.126+0000 7ff76be00700 1 --2- 172.21.15.19:0/505907560 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5598170ffb60 0x55981717cd90 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9dca46bb1eacd219 server_cookie=47cbc64d87624b25 in_seq=0 out_seq=0 2024-05-07T02:34:54.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.126+0000 7ff752a00700 10 client.25808 ms_handle_connect on v2:172.21.15.19:6840/3905645246 2024-05-07T02:34:54.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.127+0000 7ff752a00700 1 -- 172.21.15.19:0/505907560 <== mds.1 v2:172.21.15.19:6840/3905645246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559817025400 con 0x5598170ffb60 2024-05-07T02:34:54.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.127+0000 7ff752a00700 10 client.25808 handle_command_reply: tid=0 2024-05-07T02:34:54.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.127+0000 7ff781e87b80 1 client.25808 shutdown 2024-05-07T02:34:54.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.127+0000 7ff781e87b80 1 -- 172.21.15.19:0/505907560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559816f77e40 msgr2=0x5598170f9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:54.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.127+0000 7ff781e87b80 1 --2- 172.21.15.19:0/505907560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559816f77e40 0x5598170f9b40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff760009cd0 tx=0x7ff760005e50 comp rx=0 tx=0).stop 2024-05-07T02:34:54.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.127+0000 7ff781e87b80 1 -- 172.21.15.19:0/505907560 shutdown_connections 2024-05-07T02:34:54.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.127+0000 7ff781e87b80 1 --2- 172.21.15.19:0/505907560 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5598170ffb60 0x55981717cd90 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:54.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.127+0000 7ff781e87b80 1 --2- 172.21.15.19:0/505907560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5598170f63e0 0x5598170fa5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:54.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.127+0000 7ff781e87b80 1 --2- 172.21.15.19:0/505907560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559816f77e40 0x5598170f9b40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:54.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.127+0000 7ff781e87b80 1 --2- 172.21.15.19:0/505907560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5598170f5a90 0x5598170fa080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:54.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.127+0000 7ff781e87b80 1 -- 172.21.15.19:0/505907560 >> 172.21.15.19:0/505907560 conn(0x559817102250 msgr2=0x5598170efbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:34:54.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.127+0000 7ff752a00700 0 client.25808 ms_handle_reset on v2:172.21.15.19:6840/3905645246 2024-05-07T02:34:54.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.128+0000 7ff781e87b80 1 -- 172.21.15.19:0/505907560 shutdown_connections 2024-05-07T02:34:54.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.128+0000 7ff781e87b80 1 -- 172.21.15.19:0/505907560 wait complete. 2024-05-07T02:34:54.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.128+0000 7ff781e87b80 20 client.25808 trim_cache size 0 max 16384 2024-05-07T02:34:54.129 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:34:54.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.129+0000 7ff771e00700 1 -- 172.21.15.19:0/3806420950 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff7540b2a70 msgr2=0x7ff7540b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:54.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.129+0000 7ff771e00700 1 --2- 172.21.15.19:0/3806420950 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff7540b2a70 0x7ff7540b4f80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff76400f870 tx=0x7ff76400fc30 comp rx=0 tx=0).stop 2024-05-07T02:34:54.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.129+0000 7ff771e00700 1 -- 172.21.15.19:0/3806420950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff76c108d30 msgr2=0x7ff76c19d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:54.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.129+0000 7ff771e00700 1 --2- 172.21.15.19:0/3806420950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff76c108d30 0x7ff76c19d430 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff758000c00 tx=0x7ff758009580 comp rx=0 tx=0).stop 2024-05-07T02:34:54.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.129+0000 7ff771e00700 1 -- 172.21.15.19:0/3806420950 shutdown_connections 2024-05-07T02:34:54.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.129+0000 7ff771e00700 1 --2- 172.21.15.19:0/3806420950 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff76c109680 0x7ff76c1a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:54.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.129+0000 7ff771e00700 1 --2- 172.21.15.19:0/3806420950 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff7540b2a70 0x7ff7540b4f80 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:54.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.129+0000 7ff771e00700 1 --2- 172.21.15.19:0/3806420950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff76c104400 0x7ff76c19ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:54.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.129+0000 7ff771e00700 1 --2- 172.21.15.19:0/3806420950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff76c108d30 0x7ff76c19d430 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:54.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.129+0000 7ff771e00700 1 -- 172.21.15.19:0/3806420950 >> 172.21.15.19:0/3806420950 conn(0x7ff76c079d10 msgr2=0x7ff76c10f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:34:54.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.129+0000 7ff771e00700 1 -- 172.21.15.19:0/3806420950 shutdown_connections 2024-05-07T02:34:54.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:54.129+0000 7ff771e00700 1 -- 172.21.15.19:0/3806420950 wait complete. 2024-05-07T02:34:54.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:34:54 smithi064 ceph-mon[34699]: pgmap v95: 129 pgs: 129 active+clean; 48 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 922 KiB/s wr, 97 op/s 2024-05-07T02:34:54.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:34:54 smithi019 ceph-mon[29781]: pgmap v95: 129 pgs: 129 active+clean; 48 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 922 KiB/s wr, 97 op/s 2024-05-07T02:34:54.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:34:54 smithi027 ceph-mon[34398]: pgmap v95: 129 pgs: 129 active+clean; 48 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 922 KiB/s wr, 97 op/s 2024-05-07T02:34:54.733 INFO:tasks.workunit.client.0.smithi019.stdout: 1 64348 3.32 MB/sec warmup 110 sec latency 22.011 ms 2024-05-07T02:34:55.137 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-07T02:34:55.619 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:34:55.733 INFO:tasks.workunit.client.0.smithi019.stdout: 1 65681 3.34 MB/sec warmup 111 sec latency 31.111 ms 2024-05-07T02:34:56.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:34:56 smithi064 ceph-mon[34699]: pgmap v96: 129 pgs: 129 active+clean; 51 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-05-07T02:34:56.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:34:56 smithi019 ceph-mon[29781]: pgmap v96: 129 pgs: 129 active+clean; 51 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-05-07T02:34:56.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:34:56 smithi027 ceph-mon[34398]: pgmap v96: 129 pgs: 129 active+clean; 51 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-05-07T02:34:56.734 INFO:tasks.workunit.client.0.smithi019.stdout: 1 67242 3.39 MB/sec warmup 112 sec latency 30.714 ms 2024-05-07T02:34:57.734 INFO:tasks.workunit.client.0.smithi019.stdout: 1 67884 3.40 MB/sec warmup 113 sec latency 23.113 ms 2024-05-07T02:34:58.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:34:58 smithi019 ceph-mon[29781]: pgmap v97: 129 pgs: 129 active+clean; 51 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 1004 KiB/s wr, 139 op/s 2024-05-07T02:34:58.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.158+0000 7f5059600700 1 -- 172.21.15.19:0/2601648185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5054106ec0 msgr2=0x7f5054113350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:58.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.158+0000 7f5059600700 1 --2- 172.21.15.19:0/2601648185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5054106ec0 0x7f5054113350 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f504c00b0a0 tx=0x7f504c00b3b0 comp rx=0 tx=0).stop 2024-05-07T02:34:58.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.158+0000 7f5059600700 1 -- 172.21.15.19:0/2601648185 shutdown_connections 2024-05-07T02:34:58.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.158+0000 7f5059600700 1 --2- 172.21.15.19:0/2601648185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5054106ec0 0x7f5054113350 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:58.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.158+0000 7f5059600700 1 --2- 172.21.15.19:0/2601648185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f505410c5b0 0x7f505410c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:58.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.158+0000 7f5059600700 1 --2- 172.21.15.19:0/2601648185 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5054106570 0x7f5054106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:58.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.158+0000 7f5059600700 1 -- 172.21.15.19:0/2601648185 >> 172.21.15.19:0/2601648185 conn(0x7f5054102090 msgr2=0x7f50541044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:34:58.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.158+0000 7f5059600700 1 -- 172.21.15.19:0/2601648185 shutdown_connections 2024-05-07T02:34:58.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.159+0000 7f5059600700 1 -- 172.21.15.19:0/2601648185 wait complete. 2024-05-07T02:34:58.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.160+0000 7f5059600700 1 Processor -- start 2024-05-07T02:34:58.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.160+0000 7f5059600700 1 -- start start 2024-05-07T02:34:58.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.161+0000 7f5059600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5054106570 0x7f505419d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:58.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.161+0000 7f5059600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5054106ec0 0x7f505419d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:58.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.161+0000 7f5059600700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f505410c5b0 0x7f50541a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:58.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.161+0000 7f5059600700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5054116730 con 0x7f5054106570 2024-05-07T02:34:58.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.161+0000 7f5059600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f50541165b0 con 0x7f5054106ec0 2024-05-07T02:34:58.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.161+0000 7f5059600700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f50541168b0 con 0x7f505410c5b0 2024-05-07T02:34:58.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.161+0000 7f5052000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5054106ec0 0x7f505419d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:58.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.161+0000 7f5053400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f505410c5b0 0x7f50541a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:58.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.161+0000 7f5052000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5054106ec0 0x7f505419d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:36544/0 (socket says 172.21.15.19:36544) 2024-05-07T02:34:58.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.161+0000 7f5052000700 1 -- 172.21.15.19:0/1714191132 learned_addr learned my addr 172.21.15.19:0/1714191132 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:34:58.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.161+0000 7f5053400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f505410c5b0 0x7f50541a1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:59362/0 (socket says 172.21.15.19:59362) 2024-05-07T02:34:58.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.161+0000 7f5052a00700 1 --2- 172.21.15.19:0/1714191132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5054106570 0x7f505419d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:58.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.161+0000 7f5053400700 1 -- 172.21.15.19:0/1714191132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5054106ec0 msgr2=0x7f505419d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:58.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.161+0000 7f5053400700 1 --2- 172.21.15.19:0/1714191132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5054106ec0 0x7f505419d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:58.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.161+0000 7f5053400700 1 -- 172.21.15.19:0/1714191132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5054106570 msgr2=0x7f505419d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:58.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.161+0000 7f5053400700 1 --2- 172.21.15.19:0/1714191132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5054106570 0x7f505419d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:58.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.162+0000 7f5053400700 1 -- 172.21.15.19:0/1714191132 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50541a1f10 con 0x7f505410c5b0 2024-05-07T02:34:58.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.162+0000 7f5052000700 1 --2- 172.21.15.19:0/1714191132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5054106ec0 0x7f505419d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:34:58.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.162+0000 7f5053400700 1 --2- 172.21.15.19:0/1714191132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f505410c5b0 0x7f50541a1870 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f504c00b070 tx=0x7f504c004c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:34:58.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.162+0000 7f504b400700 1 -- 172.21.15.19:0/1714191132 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504c021070 con 0x7f505410c5b0 2024-05-07T02:34:58.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.162+0000 7f504b400700 1 -- 172.21.15.19:0/1714191132 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f504c007e30 con 0x7f505410c5b0 2024-05-07T02:34:58.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.162+0000 7f504b400700 1 -- 172.21.15.19:0/1714191132 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504c0046f0 con 0x7f505410c5b0 2024-05-07T02:34:58.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.162+0000 7f5059600700 1 -- 172.21.15.19:0/1714191132 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50541a21a0 con 0x7f505410c5b0 2024-05-07T02:34:58.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.163+0000 7f5059600700 1 -- 172.21.15.19:0/1714191132 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50541a2690 con 0x7f505410c5b0 2024-05-07T02:34:58.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.169+0000 7f504b400700 1 -- 172.21.15.19:0/1714191132 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f504c007980 con 0x7f505410c5b0 2024-05-07T02:34:58.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.169+0000 7f50680a6b80 1 Processor -- start 2024-05-07T02:34:58.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.170+0000 7f50680a6b80 1 -- start start 2024-05-07T02:34:58.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.170+0000 7f50680a6b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610a1a6f3b0 0x5610a1a6f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:58.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.170+0000 7f50680a6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610a1a69470 0x5610a1a69880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:58.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.170+0000 7f50680a6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610a1a69dc0 0x5610a1a76040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:58.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.170+0000 7f50680a6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5610a198ba20 con 0x5610a1a69dc0 2024-05-07T02:34:58.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.170+0000 7f50680a6b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5610a1929760 con 0x5610a1a6f3b0 2024-05-07T02:34:58.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.170+0000 7f50680a6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5610a19879e0 con 0x5610a1a69470 2024-05-07T02:34:58.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.170+0000 7f5052a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610a1a69470 0x5610a1a69880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:58.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.170+0000 7f5052a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610a1a69470 0x5610a1a69880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:59372/0 (socket says 172.21.15.19:59372) 2024-05-07T02:34:58.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.170+0000 7f5052a00700 1 -- 172.21.15.19:0/3008191041 learned_addr learned my addr 172.21.15.19:0/3008191041 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:34:58.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.171+0000 7f5052000700 1 --2- 172.21.15.19:0/3008191041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610a1a6f3b0 0x5610a1a6f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:58.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.171+0000 7f504b400700 1 --2- 172.21.15.19:0/1714191132 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f50380b29f0 0x7f50380b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:58.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.171+0000 7f5052000700 1 --2- 172.21.15.19:0/3008191041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610a1a69dc0 0x5610a1a76040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:58.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.171+0000 7f504b400700 1 -- 172.21.15.19:0/1714191132 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f504c00f020 con 0x7f505410c5b0 2024-05-07T02:34:58.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.171+0000 7f5052a00700 1 -- 172.21.15.19:0/3008191041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610a1a6f3b0 msgr2=0x5610a1a6f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:58.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.171+0000 7f5052a00700 1 --2- 172.21.15.19:0/3008191041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610a1a6f3b0 0x5610a1a6f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:58.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.171+0000 7f5052a00700 1 -- 172.21.15.19:0/3008191041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610a1a69dc0 msgr2=0x5610a1a76040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:58.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.171+0000 7f5052a00700 1 --2- 172.21.15.19:0/3008191041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610a1a69dc0 0x5610a1a76040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:58.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.171+0000 7f5052a00700 1 -- 172.21.15.19:0/3008191041 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610a18d88d0 con 0x5610a1a69470 2024-05-07T02:34:58.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.171+0000 7f5053400700 1 --2- 172.21.15.19:0/1714191132 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f50380b29f0 0x7f50380b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:58.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.171+0000 7f5052000700 1 --2- 172.21.15.19:0/3008191041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610a1a69dc0 0x5610a1a76040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:34:58.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.171+0000 7f5052a00700 1 --2- 172.21.15.19:0/3008191041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610a1a69470 0x5610a1a69880 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f503c009c90 tx=0x7f503c0103d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1a3ded9146a28b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:34:58.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.172+0000 7f5043e00700 1 -- 172.21.15.19:0/3008191041 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503c010bf0 con 0x5610a1a69470 2024-05-07T02:34:58.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.172+0000 7f5043e00700 1 -- 172.21.15.19:0/3008191041 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f503c010d90 con 0x5610a1a69470 2024-05-07T02:34:58.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.172+0000 7f5043e00700 1 -- 172.21.15.19:0/3008191041 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503c0117b0 con 0x5610a1a69470 2024-05-07T02:34:58.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.173+0000 7f50680a6b80 1 -- 172.21.15.19:0/3008191041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610a1a69470 msgr2=0x5610a1a69880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:58.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.173+0000 7f50680a6b80 1 --2- 172.21.15.19:0/3008191041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610a1a69470 0x5610a1a69880 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f503c009c90 tx=0x7f503c0103d0 comp rx=0 tx=0).stop 2024-05-07T02:34:58.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.173+0000 7f5053400700 1 --2- 172.21.15.19:0/1714191132 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f50380b29f0 0x7f50380b4f00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f504c00b780 tx=0x7f504c009040 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:34:58.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.173+0000 7f50680a6b80 1 -- 172.21.15.19:0/3008191041 shutdown_connections 2024-05-07T02:34:58.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.173+0000 7f50680a6b80 1 --2- 172.21.15.19:0/3008191041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610a1a69dc0 0x5610a1a76040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:58.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.173+0000 7f50680a6b80 1 --2- 172.21.15.19:0/3008191041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610a1a6f3b0 0x5610a1a6f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:58.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.173+0000 7f50680a6b80 1 --2- 172.21.15.19:0/3008191041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610a1a69470 0x5610a1a69880 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:58.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.173+0000 7f50680a6b80 1 -- 172.21.15.19:0/3008191041 >> 172.21.15.19:0/3008191041 conn(0x5610a18ebe40 msgr2=0x5610a1a642f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:34:58.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.173+0000 7f50680a6b80 1 -- 172.21.15.19:0/3008191041 shutdown_connections 2024-05-07T02:34:58.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.173+0000 7f50680a6b80 1 -- 172.21.15.19:0/3008191041 wait complete. 2024-05-07T02:34:58.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.174+0000 7f50680a6b80 1 -- start start 2024-05-07T02:34:58.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.174+0000 7f50680a6b80 1 Processor -- start 2024-05-07T02:34:58.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.175+0000 7f50680a6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610a1a69470 0x5610a1a73520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:58.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.175+0000 7f50680a6b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610a1a69dc0 0x5610a1a73a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:58.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.175+0000 7f50680a6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610a1a6f3b0 0x5610a1a73fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:58.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.175+0000 7f50680a6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5610a1a767c0 con 0x5610a1a69470 2024-05-07T02:34:58.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.175+0000 7f50680a6b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5610a18f1b20 con 0x5610a1a69dc0 2024-05-07T02:34:58.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.175+0000 7f50680a6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5610a1a76940 con 0x5610a1a6f3b0 2024-05-07T02:34:58.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.175+0000 7f5052a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610a1a69dc0 0x5610a1a73a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:58.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.175+0000 7f50680a6b80 10 client.0 _ll_register_callbacks cb 0x5610a1932c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:34:58.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.175+0000 7f5052a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610a1a69dc0 0x5610a1a73a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:36560/0 (socket says 172.21.15.19:36560) 2024-05-07T02:34:58.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.175+0000 7f5052a00700 1 -- 172.21.15.19:0/1485529427 learned_addr learned my addr 172.21.15.19:0/1485529427 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:34:58.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.175+0000 7f5052000700 1 --2- 172.21.15.19:0/1485529427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610a1a69470 0x5610a1a73520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:58.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.175+0000 7f5052000700 1 --2- 172.21.15.19:0/1485529427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610a1a6f3b0 0x5610a1a73fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:58.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.176+0000 7f5052a00700 1 -- 172.21.15.19:0/1485529427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610a1a6f3b0 msgr2=0x5610a1a73fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:58.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.176+0000 7f5052a00700 1 --2- 172.21.15.19:0/1485529427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610a1a6f3b0 0x5610a1a73fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:58.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.176+0000 7f5052a00700 1 -- 172.21.15.19:0/1485529427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610a1a69470 msgr2=0x5610a1a73520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:58.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.176+0000 7f5052a00700 1 --2- 172.21.15.19:0/1485529427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610a1a69470 0x5610a1a73520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:58.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.176+0000 7f5052a00700 1 -- 172.21.15.19:0/1485529427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610a1951420 con 0x5610a1a69dc0 2024-05-07T02:34:58.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.176+0000 7f5052000700 1 --2- 172.21.15.19:0/1485529427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610a1a6f3b0 0x5610a1a73fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-07T02:34:58.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.176+0000 7f5052a00700 1 --2- 172.21.15.19:0/1485529427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610a1a69dc0 0x5610a1a73a60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f503c011010 tx=0x7f503c01c7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ad939cb0b5f4b86 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:34:58.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.176+0000 7f5040c00700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:34:58.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.176+0000 7f5040c00700 1 -- 172.21.15.19:0/1485529427 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503c01ca70 con 0x5610a1a69dc0 2024-05-07T02:34:58.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.176+0000 7f5040c00700 1 -- 172.21.15.19:0/1485529427 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f503c005280 con 0x5610a1a69dc0 2024-05-07T02:34:58.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.176+0000 7f50680a6b80 1 -- 172.21.15.19:0/1485529427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f503c010d90 con 0x5610a1a69dc0 2024-05-07T02:34:58.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.177+0000 7f5040c00700 1 -- 172.21.15.19:0/1485529427 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f503c0054a0 con 0x5610a1a69dc0 2024-05-07T02:34:58.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.177+0000 7f50680a6b80 10 client.25820 fetch_fsmap learned FSMap version 34 2024-05-07T02:34:58.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.178+0000 7f50680a6b80 1 -- 172.21.15.19:0/1485529427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5610a1a813c0 con 0x5610a1a69dc0 2024-05-07T02:34:58.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.178+0000 7f5040c00700 1 -- 172.21.15.19:0/1485529427 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f503c010d90 con 0x5610a1a69dc0 2024-05-07T02:34:58.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.178+0000 7f5040c00700 1 -- 172.21.15.19:0/1485529427 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f503c011560 con 0x5610a1a69dc0 2024-05-07T02:34:58.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.178+0000 7f50680a6b80 10 client.25820 fetch_fsmap finished waiting for FSMap version 34 2024-05-07T02:34:58.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.178+0000 7f50680a6b80 10 client.25820 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:34:58.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.178+0000 7f50680a6b80 20 client.25820 populate_metadata read hostname 'smithi019' 2024-05-07T02:34:58.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.178+0000 7f50680a6b80 1 --2- 172.21.15.19:0/1485529427 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5610a1a838f0 0x5610a1a85d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:58.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.178+0000 7f50680a6b80 4 client.25820 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:34:58.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.178+0000 7f50680a6b80 1 -- 172.21.15.19:0/1485529427 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610a1a362e0 con 0x5610a1a838f0 2024-05-07T02:34:58.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.179+0000 7f5052000700 1 --2- 172.21.15.19:0/1485529427 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5610a1a838f0 0x5610a1a85d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:58.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.179+0000 7f5040c00700 10 client.25820 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:34:58.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.179+0000 7f5052000700 1 --2- 172.21.15.19:0/1485529427 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5610a1a838f0 0x5610a1a85d10 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58530d5a87d88cdb server_cookie=b58a9db2e8c6f626 in_seq=0 out_seq=0 2024-05-07T02:34:58.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.180+0000 7f5040c00700 1 -- 172.21.15.19:0/1485529427 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5610a1a362e0 con 0x5610a1a838f0 2024-05-07T02:34:58.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.180+0000 7f5040c00700 10 client.25820 handle_command_reply: tid=0 2024-05-07T02:34:58.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.180+0000 7f50680a6b80 1 client.25820 shutdown 2024-05-07T02:34:58.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.180+0000 7f50680a6b80 1 -- 172.21.15.19:0/1485529427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610a1a69dc0 msgr2=0x5610a1a73a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:58.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.180+0000 7f50680a6b80 1 --2- 172.21.15.19:0/1485529427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610a1a69dc0 0x5610a1a73a60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f503c011010 tx=0x7f503c01c7d0 comp rx=0 tx=0).stop 2024-05-07T02:34:58.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.180+0000 7f50680a6b80 1 -- 172.21.15.19:0/1485529427 shutdown_connections 2024-05-07T02:34:58.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.180+0000 7f50680a6b80 1 --2- 172.21.15.19:0/1485529427 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5610a1a838f0 0x5610a1a85d10 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:58.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.180+0000 7f50680a6b80 1 --2- 172.21.15.19:0/1485529427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610a1a6f3b0 0x5610a1a73fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:58.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.181+0000 7f50680a6b80 1 --2- 172.21.15.19:0/1485529427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610a1a69470 0x5610a1a73520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:58.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.181+0000 7f50680a6b80 1 --2- 172.21.15.19:0/1485529427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610a1a69dc0 0x5610a1a73a60 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:58.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.181+0000 7f50680a6b80 1 -- 172.21.15.19:0/1485529427 >> 172.21.15.19:0/1485529427 conn(0x5610a18ebe40 msgr2=0x5610a1a68460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:34:58.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.181+0000 7f5040c00700 0 client.25820 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:34:58.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.181+0000 7f50680a6b80 1 -- 172.21.15.19:0/1485529427 shutdown_connections 2024-05-07T02:34:58.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.181+0000 7f50680a6b80 1 -- 172.21.15.19:0/1485529427 wait complete. 2024-05-07T02:34:58.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.181+0000 7f50680a6b80 20 client.25820 trim_cache size 0 max 16384 2024-05-07T02:34:58.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.200+0000 7f50680a6b80 1 Processor -- start 2024-05-07T02:34:58.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.200+0000 7f50680a6b80 1 -- start start 2024-05-07T02:34:58.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.201+0000 7f50680a6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610a1a69470 0x5610a1a68e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:58.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.201+0000 7f50680a6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610a1a69dc0 0x5610a1a7fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:58.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.201+0000 7f50680a6b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610a1a6f3b0 0x5610a1a72e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:58.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.201+0000 7f50680a6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f50541b27b0 con 0x5610a1a69dc0 2024-05-07T02:34:58.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.201+0000 7f50680a6b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f50541b43b0 con 0x5610a1a6f3b0 2024-05-07T02:34:58.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.201+0000 7f50680a6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f50541b2e80 con 0x5610a1a69470 2024-05-07T02:34:58.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.201+0000 7f5052000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610a1a69470 0x5610a1a68e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:58.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.201+0000 7f5052000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610a1a69470 0x5610a1a68e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:59402/0 (socket says 172.21.15.19:59402) 2024-05-07T02:34:58.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.201+0000 7f5052000700 1 -- 172.21.15.19:0/2551000227 learned_addr learned my addr 172.21.15.19:0/2551000227 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:34:58.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.201+0000 7f5052000700 1 --2- 172.21.15.19:0/2551000227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610a1a6f3b0 0x5610a1a72e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:58.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.202+0000 7f5052000700 1 -- 172.21.15.19:0/2551000227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610a1a6f3b0 msgr2=0x5610a1a72e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:58.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.202+0000 7f5052000700 1 --2- 172.21.15.19:0/2551000227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610a1a6f3b0 0x5610a1a72e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:58.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.202+0000 7f5052000700 1 -- 172.21.15.19:0/2551000227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610a1a69dc0 msgr2=0x5610a1a7fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:58.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.202+0000 7f5052000700 1 --2- 172.21.15.19:0/2551000227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610a1a69dc0 0x5610a1a7fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:58.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.202+0000 7f5052000700 1 -- 172.21.15.19:0/2551000227 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610a1a65f30 con 0x5610a1a69470 2024-05-07T02:34:58.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.202+0000 7f5052000700 1 --2- 172.21.15.19:0/2551000227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610a1a69470 0x5610a1a68e90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5044011430 tx=0x7f5044013040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=155b4fca663a0798 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:34:58.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.202+0000 7f5043400700 1 -- 172.21.15.19:0/2551000227 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5044003d00 con 0x5610a1a69470 2024-05-07T02:34:58.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.202+0000 7f5043400700 1 -- 172.21.15.19:0/2551000227 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5044003ea0 con 0x5610a1a69470 2024-05-07T02:34:58.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.202+0000 7f5043400700 1 -- 172.21.15.19:0/2551000227 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50440053f0 con 0x5610a1a69470 2024-05-07T02:34:58.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.203+0000 7f50680a6b80 1 -- 172.21.15.19:0/2551000227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610a1a69470 msgr2=0x5610a1a68e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:58.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.203+0000 7f50680a6b80 1 --2- 172.21.15.19:0/2551000227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610a1a69470 0x5610a1a68e90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5044011430 tx=0x7f5044013040 comp rx=0 tx=0).stop 2024-05-07T02:34:58.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.204+0000 7f50680a6b80 1 -- 172.21.15.19:0/2551000227 shutdown_connections 2024-05-07T02:34:58.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.204+0000 7f50680a6b80 1 --2- 172.21.15.19:0/2551000227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610a1a6f3b0 0x5610a1a72e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:58.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.204+0000 7f50680a6b80 1 --2- 172.21.15.19:0/2551000227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610a1a69470 0x5610a1a68e90 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:58.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.204+0000 7f50680a6b80 1 --2- 172.21.15.19:0/2551000227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610a1a69dc0 0x5610a1a7fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:58.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.204+0000 7f50680a6b80 1 -- 172.21.15.19:0/2551000227 >> 172.21.15.19:0/2551000227 conn(0x5610a18ebe40 msgr2=0x5610a1a5e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:34:58.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.204+0000 7f50680a6b80 1 -- 172.21.15.19:0/2551000227 shutdown_connections 2024-05-07T02:34:58.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.204+0000 7f50680a6b80 1 -- 172.21.15.19:0/2551000227 wait complete. 2024-05-07T02:34:58.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.205+0000 7f50680a6b80 1 -- start start 2024-05-07T02:34:58.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.205+0000 7f50680a6b80 1 Processor -- start 2024-05-07T02:34:58.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.206+0000 7f50680a6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610a1a69470 0x5610a1a85d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:58.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.206+0000 7f50680a6b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610a1a69dc0 0x5610a1a86260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:58.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.206+0000 7f50680a6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610a1a6f3b0 0x5610a1ae5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:58.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.206+0000 7f50680a6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f50541b3c20 con 0x5610a1a69470 2024-05-07T02:34:58.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.206+0000 7f50680a6b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f50541b3550 con 0x5610a1a69dc0 2024-05-07T02:34:58.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.206+0000 7f50680a6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f50541b0750 con 0x5610a1a6f3b0 2024-05-07T02:34:58.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.206+0000 7f50680a6b80 10 client.0 _ll_register_callbacks cb 0x5610a1932c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:34:58.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.206+0000 7f5052a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610a1a69dc0 0x5610a1a86260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:58.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.206+0000 7f5052a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610a1a69dc0 0x5610a1a86260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:36582/0 (socket says 172.21.15.19:36582) 2024-05-07T02:34:58.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.206+0000 7f5052a00700 1 -- 172.21.15.19:0/4158844204 learned_addr learned my addr 172.21.15.19:0/4158844204 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:34:58.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.206+0000 7f5052000700 1 --2- 172.21.15.19:0/4158844204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610a1a69470 0x5610a1a85d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:58.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.207+0000 7f5052000700 1 --2- 172.21.15.19:0/4158844204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610a1a6f3b0 0x5610a1ae5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:58.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.207+0000 7f5052a00700 1 -- 172.21.15.19:0/4158844204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610a1a6f3b0 msgr2=0x5610a1ae5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:58.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.207+0000 7f5052a00700 1 --2- 172.21.15.19:0/4158844204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610a1a6f3b0 0x5610a1ae5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:58.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.207+0000 7f5052a00700 1 -- 172.21.15.19:0/4158844204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610a1a69470 msgr2=0x5610a1a85d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:34:58.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.207+0000 7f5052a00700 1 --2- 172.21.15.19:0/4158844204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610a1a69470 0x5610a1a85d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:58.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.207+0000 7f5052a00700 1 -- 172.21.15.19:0/4158844204 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610a1ae6370 con 0x5610a1a69dc0 2024-05-07T02:34:58.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.207+0000 7f5052000700 1 --2- 172.21.15.19:0/4158844204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610a1a6f3b0 0x5610a1ae5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-07T02:34:58.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.207+0000 7f5052a00700 1 --2- 172.21.15.19:0/4158844204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610a1a69dc0 0x5610a1a86260 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f503c011010 tx=0x7f503c01cc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e12464f9b43495b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:34:58.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.207+0000 7f5040c00700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:34:58.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.207+0000 7f5040c00700 1 -- 172.21.15.19:0/4158844204 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503c0058e0 con 0x5610a1a69dc0 2024-05-07T02:34:58.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.208+0000 7f5040c00700 1 -- 172.21.15.19:0/4158844204 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f503c011ca0 con 0x5610a1a69dc0 2024-05-07T02:34:58.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.208+0000 7f50680a6b80 1 -- 172.21.15.19:0/4158844204 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5044003ea0 con 0x5610a1a69dc0 2024-05-07T02:34:58.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.208+0000 7f5040c00700 1 -- 172.21.15.19:0/4158844204 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f503c010d00 con 0x5610a1a69dc0 2024-05-07T02:34:58.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.208+0000 7f50680a6b80 10 client.25826 fetch_fsmap learned FSMap version 34 2024-05-07T02:34:58.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.208+0000 7f50680a6b80 1 -- 172.21.15.19:0/4158844204 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5610a1aed650 con 0x5610a1a69dc0 2024-05-07T02:34:58.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.209+0000 7f5040c00700 1 -- 172.21.15.19:0/4158844204 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5044003ea0 con 0x5610a1a69dc0 2024-05-07T02:34:58.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.209+0000 7f5040c00700 1 -- 172.21.15.19:0/4158844204 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 34) ==== 8378+0+0 (secure 0 0 0) 0x7f503c039450 con 0x5610a1a69dc0 2024-05-07T02:34:58.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.209+0000 7f50680a6b80 10 client.25826 fetch_fsmap finished waiting for FSMap version 34 2024-05-07T02:34:58.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.209+0000 7f50680a6b80 10 client.25826 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:34:58.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.209+0000 7f50680a6b80 20 client.25826 populate_metadata read hostname 'smithi019' 2024-05-07T02:34:58.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.209+0000 7f50680a6b80 1 --2- 172.21.15.19:0/4158844204 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5610a1aeddb0 0x5610a1af01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:34:58.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.209+0000 7f50680a6b80 4 client.25826 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:34:58.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.209+0000 7f50680a6b80 1 -- 172.21.15.19:0/4158844204 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5610a1999400 con 0x5610a1aeddb0 2024-05-07T02:34:58.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.211+0000 7f5052000700 1 --2- 172.21.15.19:0/4158844204 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5610a1aeddb0 0x5610a1af01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:34:58.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.211+0000 7f5040c00700 10 client.25826 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:34:58.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.211+0000 7f5052000700 1 --2- 172.21.15.19:0/4158844204 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5610a1aeddb0 0x5610a1af01d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee4328473d09c00e server_cookie=eb741bb579c1a69e in_seq=0 out_seq=0 2024-05-07T02:34:58.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.212+0000 7f5040c00700 1 -- 172.21.15.19:0/4158844204 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5610a1999400 con 0x5610a1aeddb0 2024-05-07T02:34:58.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.212+0000 7f5040c00700 10 client.25826 handle_command_reply: tid=0 2024-05-07T02:34:58.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.212+0000 7f50680a6b80 1 client.25826 shutdown 2024-05-07T02:34:58.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.212+0000 7f50680a6b80 1 -- 172.21.15.19:0/4158844204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610a1a69dc0 msgr2=0x5610a1a86260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:58.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.212+0000 7f50680a6b80 1 --2- 172.21.15.19:0/4158844204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610a1a69dc0 0x5610a1a86260 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f503c011010 tx=0x7f503c01cc60 comp rx=0 tx=0).stop 2024-05-07T02:34:58.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.212+0000 7f50680a6b80 1 -- 172.21.15.19:0/4158844204 shutdown_connections 2024-05-07T02:34:58.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.212+0000 7f50680a6b80 1 --2- 172.21.15.19:0/4158844204 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5610a1aeddb0 0x5610a1af01d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:58.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.212+0000 7f50680a6b80 1 --2- 172.21.15.19:0/4158844204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610a1a6f3b0 0x5610a1ae5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:58.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.212+0000 7f50680a6b80 1 --2- 172.21.15.19:0/4158844204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610a1a69470 0x5610a1a85d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:58.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.212+0000 7f50680a6b80 1 --2- 172.21.15.19:0/4158844204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610a1a69dc0 0x5610a1a86260 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:58.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.212+0000 7f50680a6b80 1 -- 172.21.15.19:0/4158844204 >> 172.21.15.19:0/4158844204 conn(0x5610a18ebe40 msgr2=0x5610a1a5e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:34:58.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.212+0000 7f5040c00700 0 client.25826 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:34:58.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.213+0000 7f50680a6b80 1 -- 172.21.15.19:0/4158844204 shutdown_connections 2024-05-07T02:34:58.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.213+0000 7f50680a6b80 1 -- 172.21.15.19:0/4158844204 wait complete. 2024-05-07T02:34:58.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.213+0000 7f50680a6b80 20 client.25826 trim_cache size 0 max 16384 2024-05-07T02:34:58.214 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:34:58.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.214+0000 7f5059600700 1 -- 172.21.15.19:0/1714191132 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f50380b29f0 msgr2=0x7f50380b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:58.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.214+0000 7f5059600700 1 --2- 172.21.15.19:0/1714191132 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f50380b29f0 0x7f50380b4f00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f504c00b780 tx=0x7f504c009040 comp rx=0 tx=0).stop 2024-05-07T02:34:58.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.214+0000 7f5059600700 1 -- 172.21.15.19:0/1714191132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f505410c5b0 msgr2=0x7f50541a1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:34:58.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.214+0000 7f5059600700 1 --2- 172.21.15.19:0/1714191132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f505410c5b0 0x7f50541a1870 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f504c00b070 tx=0x7f504c004c80 comp rx=0 tx=0).stop 2024-05-07T02:34:58.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.214+0000 7f5059600700 1 -- 172.21.15.19:0/1714191132 shutdown_connections 2024-05-07T02:34:58.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.214+0000 7f5059600700 1 --2- 172.21.15.19:0/1714191132 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f505410c5b0 0x7f50541a1870 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:58.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.214+0000 7f5059600700 1 --2- 172.21.15.19:0/1714191132 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f50380b29f0 0x7f50380b4f00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:58.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.214+0000 7f5059600700 1 --2- 172.21.15.19:0/1714191132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5054106570 0x7f505419d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:58.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.214+0000 7f5059600700 1 --2- 172.21.15.19:0/1714191132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5054106ec0 0x7f505419d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:34:58.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.214+0000 7f5059600700 1 -- 172.21.15.19:0/1714191132 >> 172.21.15.19:0/1714191132 conn(0x7f5054102090 msgr2=0x7f50541046b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:34:58.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.215+0000 7f5059600700 1 -- 172.21.15.19:0/1714191132 shutdown_connections 2024-05-07T02:34:58.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:34:58.215+0000 7f5059600700 1 -- 172.21.15.19:0/1714191132 wait complete. 2024-05-07T02:34:58.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:34:58 smithi064 ceph-mon[34699]: pgmap v97: 129 pgs: 129 active+clean; 51 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 1004 KiB/s wr, 139 op/s 2024-05-07T02:34:58.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:34:58 smithi027 ceph-mon[34398]: pgmap v97: 129 pgs: 129 active+clean; 51 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 1004 KiB/s wr, 139 op/s 2024-05-07T02:34:58.734 INFO:tasks.workunit.client.0.smithi019.stdout: 1 68979 3.42 MB/sec warmup 114 sec latency 46.667 ms 2024-05-07T02:34:59.734 INFO:tasks.workunit.client.0.smithi019.stdout: 1 70602 3.46 MB/sec warmup 115 sec latency 15.057 ms 2024-05-07T02:35:00.249 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-07T02:35:00.250 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-07T02:35:00.334 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:00 smithi019 ceph-mon[29781]: pgmap v98: 129 pgs: 129 active+clean; 51 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 1004 KiB/s wr, 139 op/s 2024-05-07T02:35:00.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:00 smithi064 ceph-mon[34699]: pgmap v98: 129 pgs: 129 active+clean; 51 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 1004 KiB/s wr, 139 op/s 2024-05-07T02:35:00.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:00 smithi027 ceph-mon[34398]: pgmap v98: 129 pgs: 129 active+clean; 51 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 1004 KiB/s wr, 139 op/s 2024-05-07T02:35:00.734 INFO:tasks.workunit.client.0.smithi019.stdout: 1 71383 3.48 MB/sec warmup 116 sec latency 26.956 ms 2024-05-07T02:35:00.776 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:35:01.734 INFO:tasks.workunit.client.0.smithi019.stdout: 1 71979 3.46 MB/sec warmup 117 sec latency 79.744 ms 2024-05-07T02:35:02.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:02 smithi064 ceph-mon[34699]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:35:02.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:02 smithi064 ceph-mon[34699]: pgmap v99: 129 pgs: 129 active+clean; 54 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 1.3 MiB/s wr, 216 op/s 2024-05-07T02:35:02.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:02 smithi019 ceph-mon[29781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:35:02.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:02 smithi019 ceph-mon[29781]: pgmap v99: 129 pgs: 129 active+clean; 54 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 1.3 MiB/s wr, 216 op/s 2024-05-07T02:35:02.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:02 smithi027 ceph-mon[34398]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:35:02.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:02 smithi027 ceph-mon[34398]: pgmap v99: 129 pgs: 129 active+clean; 54 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 1.3 MiB/s wr, 216 op/s 2024-05-07T02:35:02.735 INFO:tasks.workunit.client.0.smithi019.stdout: 1 72194 3.44 MB/sec warmup 118 sec latency 94.531 ms 2024-05-07T02:35:03.735 INFO:tasks.workunit.client.0.smithi019.stdout: 1 73746 3.48 MB/sec warmup 119 sec latency 135.985 ms 2024-05-07T02:35:03.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.772+0000 7fc86f200700 1 -- 172.21.15.19:0/2996869064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc870104400 msgr2=0x7fc870104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:03.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.772+0000 7fc86f200700 1 --2- 172.21.15.19:0/2996869064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc870104400 0x7fc870104810 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc8600099c0 tx=0x7fc860009cd0 comp rx=0 tx=0).stop 2024-05-07T02:35:03.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.773+0000 7fc86f200700 1 -- 172.21.15.19:0/2996869064 shutdown_connections 2024-05-07T02:35:03.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.773+0000 7fc86f200700 1 --2- 172.21.15.19:0/2996869064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc870109680 0x7fc870110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:03.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.773+0000 7fc86f200700 1 --2- 172.21.15.19:0/2996869064 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc870104400 0x7fc870104810 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:03.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.773+0000 7fc86f200700 1 --2- 172.21.15.19:0/2996869064 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc870108d30 0x7fc870109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:03.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.773+0000 7fc86f200700 1 -- 172.21.15.19:0/2996869064 >> 172.21.15.19:0/2996869064 conn(0x7fc870079d10 msgr2=0x7fc87007a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:35:03.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.773+0000 7fc86f200700 1 -- 172.21.15.19:0/2996869064 shutdown_connections 2024-05-07T02:35:03.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.773+0000 7fc86f200700 1 -- 172.21.15.19:0/2996869064 wait complete. 2024-05-07T02:35:03.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.774+0000 7fc86f200700 1 Processor -- start 2024-05-07T02:35:03.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.774+0000 7fc86f200700 1 -- start start 2024-05-07T02:35:03.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.775+0000 7fc86f200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc870104400 0x7fc870075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:03.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.775+0000 7fc86f200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc870108d30 0x7fc870076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:03.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.775+0000 7fc86f200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc870109680 0x7fc870076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:03.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.775+0000 7fc86f200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc870114320 con 0x7fc870109680 2024-05-07T02:35:03.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.775+0000 7fc86f200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc8701141a0 con 0x7fc870108d30 2024-05-07T02:35:03.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.775+0000 7fc86f200700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc8701144a0 con 0x7fc870104400 2024-05-07T02:35:03.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.775+0000 7fc86de00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc870104400 0x7fc870075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:03.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.775+0000 7fc86de00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc870104400 0x7fc870075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:46434/0 (socket says 172.21.15.19:46434) 2024-05-07T02:35:03.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.775+0000 7fc86de00700 1 -- 172.21.15.19:0/218126664 learned_addr learned my addr 172.21.15.19:0/218126664 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:35:03.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.775+0000 7fc865400700 1 --2- 172.21.15.19:0/218126664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc870108d30 0x7fc870076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:03.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.775+0000 7fc86e800700 1 --2- 172.21.15.19:0/218126664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc870109680 0x7fc870076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:03.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.776+0000 7fc86de00700 1 -- 172.21.15.19:0/218126664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc870108d30 msgr2=0x7fc870076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:03.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.776+0000 7fc86de00700 1 --2- 172.21.15.19:0/218126664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc870108d30 0x7fc870076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:03.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.776+0000 7fc86de00700 1 -- 172.21.15.19:0/218126664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc870109680 msgr2=0x7fc870076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:03.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.776+0000 7fc86de00700 1 --2- 172.21.15.19:0/218126664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc870109680 0x7fc870076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:03.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.776+0000 7fc86de00700 1 -- 172.21.15.19:0/218126664 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8701a9050 con 0x7fc870104400 2024-05-07T02:35:03.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.776+0000 7fc86e800700 1 --2- 172.21.15.19:0/218126664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc870109680 0x7fc870076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-07T02:35:03.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.776+0000 7fc865400700 1 --2- 172.21.15.19:0/218126664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc870108d30 0x7fc870076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:35:03.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.776+0000 7fc86de00700 1 --2- 172.21.15.19:0/218126664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc870104400 0x7fc870075ac0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc860006010 tx=0x7fc860005280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:35:03.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.776+0000 7fc867400700 1 -- 172.21.15.19:0/218126664 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc860026070 con 0x7fc870104400 2024-05-07T02:35:03.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.776+0000 7fc867400700 1 -- 172.21.15.19:0/218126664 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc860003c70 con 0x7fc870104400 2024-05-07T02:35:03.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.776+0000 7fc86f200700 1 -- 172.21.15.19:0/218126664 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8701a92e0 con 0x7fc870104400 2024-05-07T02:35:03.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.776+0000 7fc867400700 1 -- 172.21.15.19:0/218126664 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86000fb40 con 0x7fc870104400 2024-05-07T02:35:03.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.777+0000 7fc86f200700 1 -- 172.21.15.19:0/218126664 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8701a9730 con 0x7fc870104400 2024-05-07T02:35:03.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.780+0000 7fc867400700 1 -- 172.21.15.19:0/218126664 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc860029070 con 0x7fc870104400 2024-05-07T02:35:03.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.781+0000 7fc88359ab80 1 Processor -- start 2024-05-07T02:35:03.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.782+0000 7fc88359ab80 1 -- start start 2024-05-07T02:35:03.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.782+0000 7fc867400700 1 --2- 172.21.15.19:0/218126664 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc8540b29f0 0x7fc8540b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:03.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.782+0000 7fc88359ab80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56373bddacc0 0x56373bddb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:03.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.782+0000 7fc88359ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56373bddb610 0x56373bde4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:03.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.782+0000 7fc88359ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56373bde4cb0 0x56373bde70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:03.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.782+0000 7fc867400700 1 -- 172.21.15.19:0/218126664 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fc8600e3290 con 0x7fc870104400 2024-05-07T02:35:03.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.782+0000 7fc88359ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56373bcfcaa0 con 0x56373bde4cb0 2024-05-07T02:35:03.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.782+0000 7fc88359ab80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56373bc9a7e0 con 0x56373bddacc0 2024-05-07T02:35:03.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.782+0000 7fc88359ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56373bcf8a60 con 0x56373bddb610 2024-05-07T02:35:03.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.782+0000 7fc865400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56373bddb610 0x56373bde4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:03.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.782+0000 7fc865400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56373bddb610 0x56373bde4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:46442/0 (socket says 172.21.15.19:46442) 2024-05-07T02:35:03.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.782+0000 7fc86e800700 1 --2- 172.21.15.19:0/218126664 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc8540b29f0 0x7fc8540b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:03.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.782+0000 7fc865400700 1 -- 172.21.15.19:0/1783358608 learned_addr learned my addr 172.21.15.19:0/1783358608 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:35:03.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.783+0000 7fc86de00700 1 --2- 172.21.15.19:0/1783358608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56373bddacc0 0x56373bddb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:03.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.783+0000 7fc86e800700 1 --2- 172.21.15.19:0/1783358608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56373bde4cb0 0x56373bde70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:03.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.783+0000 7fc865400700 1 -- 172.21.15.19:0/1783358608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56373bddacc0 msgr2=0x56373bddb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:03.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.783+0000 7fc865400700 1 --2- 172.21.15.19:0/1783358608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56373bddacc0 0x56373bddb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:03.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.783+0000 7fc865400700 1 -- 172.21.15.19:0/1783358608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56373bde4cb0 msgr2=0x56373bde70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:03.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.783+0000 7fc865400700 1 --2- 172.21.15.19:0/1783358608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56373bde4cb0 0x56373bde70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:03.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.783+0000 7fc865400700 1 -- 172.21.15.19:0/1783358608 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56373bc49950 con 0x56373bddb610 2024-05-07T02:35:03.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.783+0000 7fc86de00700 1 --2- 172.21.15.19:0/1783358608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56373bddacc0 0x56373bddb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:35:03.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.783+0000 7fc865400700 1 --2- 172.21.15.19:0/1783358608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56373bddb610 0x56373bde4770 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc858011540 tx=0x7fc858011850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cea360035584a25 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:35:03.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.784+0000 7fc85f400700 1 -- 172.21.15.19:0/1783358608 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc858004a30 con 0x56373bddb610 2024-05-07T02:35:03.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.784+0000 7fc85f400700 1 -- 172.21.15.19:0/1783358608 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8580054d0 con 0x56373bddb610 2024-05-07T02:35:03.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.784+0000 7fc85f400700 1 -- 172.21.15.19:0/1783358608 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85800e930 con 0x56373bddb610 2024-05-07T02:35:03.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.784+0000 7fc86e800700 1 --2- 172.21.15.19:0/218126664 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc8540b29f0 0x7fc8540b4f00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc868009fd0 tx=0x7fc86800d040 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:35:03.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.785+0000 7fc88359ab80 1 -- 172.21.15.19:0/1783358608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56373bddb610 msgr2=0x56373bde4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:03.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.785+0000 7fc88359ab80 1 --2- 172.21.15.19:0/1783358608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56373bddb610 0x56373bde4770 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc858011540 tx=0x7fc858011850 comp rx=0 tx=0).stop 2024-05-07T02:35:03.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.785+0000 7fc88359ab80 1 -- 172.21.15.19:0/1783358608 shutdown_connections 2024-05-07T02:35:03.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.785+0000 7fc88359ab80 1 --2- 172.21.15.19:0/1783358608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56373bde4cb0 0x56373bde70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:03.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.785+0000 7fc88359ab80 1 --2- 172.21.15.19:0/1783358608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56373bddacc0 0x56373bddb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:03.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.785+0000 7fc88359ab80 1 --2- 172.21.15.19:0/1783358608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56373bddb610 0x56373bde4770 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:03.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.785+0000 7fc88359ab80 1 -- 172.21.15.19:0/1783358608 >> 172.21.15.19:0/1783358608 conn(0x56373bc5cec0 msgr2=0x56373bdd5a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:35:03.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.785+0000 7fc88359ab80 1 -- 172.21.15.19:0/1783358608 shutdown_connections 2024-05-07T02:35:03.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.785+0000 7fc88359ab80 1 -- 172.21.15.19:0/1783358608 wait complete. 2024-05-07T02:35:03.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.786+0000 7fc88359ab80 1 -- start start 2024-05-07T02:35:03.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.786+0000 7fc88359ab80 1 Processor -- start 2024-05-07T02:35:03.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.787+0000 7fc88359ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56373bddacc0 0x56373bde1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:03.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.787+0000 7fc88359ab80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56373bddb610 0x56373bde2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:03.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.787+0000 7fc88359ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56373bde4cb0 0x56373bde26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:03.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.787+0000 7fc88359ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56373bde7a80 con 0x56373bde4cb0 2024-05-07T02:35:03.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.787+0000 7fc88359ab80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56373bc62ba0 con 0x56373bddb610 2024-05-07T02:35:03.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.787+0000 7fc88359ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56373bde7c00 con 0x56373bddacc0 2024-05-07T02:35:03.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.787+0000 7fc88359ab80 10 client.0 _ll_register_callbacks cb 0x56373bc90400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:35:03.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.787+0000 7fc865400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56373bddb610 0x56373bde2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:03.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.787+0000 7fc86de00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56373bddacc0 0x56373bde1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:03.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.787+0000 7fc865400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56373bddb610 0x56373bde2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:38556/0 (socket says 172.21.15.19:38556) 2024-05-07T02:35:03.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.787+0000 7fc865400700 1 -- 172.21.15.19:0/2371823730 learned_addr learned my addr 172.21.15.19:0/2371823730 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:35:03.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.787+0000 7fc86e800700 1 --2- 172.21.15.19:0/2371823730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56373bde4cb0 0x56373bde26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:03.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.788+0000 7fc86de00700 1 -- 172.21.15.19:0/2371823730 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56373bddb610 msgr2=0x56373bde2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:03.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.788+0000 7fc86de00700 1 --2- 172.21.15.19:0/2371823730 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56373bddb610 0x56373bde2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:03.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.788+0000 7fc86de00700 1 -- 172.21.15.19:0/2371823730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56373bde4cb0 msgr2=0x56373bde26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:03.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.788+0000 7fc86de00700 1 --2- 172.21.15.19:0/2371823730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56373bde4cb0 0x56373bde26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:03.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.788+0000 7fc86de00700 1 -- 172.21.15.19:0/2371823730 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56373bcc24a0 con 0x56373bddacc0 2024-05-07T02:35:03.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.788+0000 7fc86de00700 1 --2- 172.21.15.19:0/2371823730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56373bddacc0 0x56373bde1c50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc8600e05f0 tx=0x7fc8600a9d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a031af96bb397683 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:35:03.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.788+0000 7fc853e00700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:35:03.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.788+0000 7fc853e00700 1 -- 172.21.15.19:0/2371823730 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8600aa910 con 0x56373bddacc0 2024-05-07T02:35:03.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.788+0000 7fc853e00700 1 -- 172.21.15.19:0/2371823730 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fc8600b2b50 con 0x56373bddacc0 2024-05-07T02:35:03.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.788+0000 7fc88359ab80 1 -- 172.21.15.19:0/2371823730 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8580054d0 con 0x56373bddacc0 2024-05-07T02:35:03.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.788+0000 7fc853e00700 1 -- 172.21.15.19:0/2371823730 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc8600aa210 con 0x56373bddacc0 2024-05-07T02:35:03.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.789+0000 7fc88359ab80 10 client.25816 fetch_fsmap learned FSMap version 35 2024-05-07T02:35:03.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.789+0000 7fc88359ab80 1 -- 172.21.15.19:0/2371823730 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56373bde7ae0 con 0x56373bddacc0 2024-05-07T02:35:03.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.789+0000 7fc853e00700 1 -- 172.21.15.19:0/2371823730 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8580054d0 con 0x56373bddacc0 2024-05-07T02:35:03.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.789+0000 7fc853e00700 1 -- 172.21.15.19:0/2371823730 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7fc86000fce0 con 0x56373bddacc0 2024-05-07T02:35:03.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.789+0000 7fc88359ab80 10 client.25816 fetch_fsmap finished waiting for FSMap version 35 2024-05-07T02:35:03.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.789+0000 7fc88359ab80 10 client.25816 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:35:03.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.789+0000 7fc88359ab80 20 client.25816 populate_metadata read hostname 'smithi019' 2024-05-07T02:35:03.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.789+0000 7fc88359ab80 1 --2- 172.21.15.19:0/2371823730 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x56373bdf4d70 0x56373bdf7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:03.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.789+0000 7fc88359ab80 4 client.25816 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:35:03.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.789+0000 7fc88359ab80 1 -- 172.21.15.19:0/2371823730 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56373bda7160 con 0x56373bdf4d70 2024-05-07T02:35:03.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.790+0000 7fc865400700 1 --2- 172.21.15.19:0/2371823730 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x56373bdf4d70 0x56373bdf7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:03.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.790+0000 7fc865400700 1 --2- 172.21.15.19:0/2371823730 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x56373bdf4d70 0x56373bdf7200 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f6d3e1cfc5e33dd server_cookie=8913ca5bc047a919 in_seq=0 out_seq=0 2024-05-07T02:35:03.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.790+0000 7fc853e00700 10 client.25816 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:35:03.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.791+0000 7fc853e00700 1 -- 172.21.15.19:0/2371823730 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56373bda7160 con 0x56373bdf4d70 2024-05-07T02:35:03.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.791+0000 7fc853e00700 10 client.25816 handle_command_reply: tid=0 2024-05-07T02:35:03.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.791+0000 7fc88359ab80 1 client.25816 shutdown 2024-05-07T02:35:03.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.791+0000 7fc88359ab80 1 -- 172.21.15.19:0/2371823730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56373bddacc0 msgr2=0x56373bde1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:03.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.791+0000 7fc88359ab80 1 --2- 172.21.15.19:0/2371823730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56373bddacc0 0x56373bde1c50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc8600e05f0 tx=0x7fc8600a9d90 comp rx=0 tx=0).stop 2024-05-07T02:35:03.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.791+0000 7fc88359ab80 1 -- 172.21.15.19:0/2371823730 shutdown_connections 2024-05-07T02:35:03.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.791+0000 7fc88359ab80 1 --2- 172.21.15.19:0/2371823730 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x56373bdf4d70 0x56373bdf7200 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:03.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.791+0000 7fc88359ab80 1 --2- 172.21.15.19:0/2371823730 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56373bde4cb0 0x56373bde26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:03.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.791+0000 7fc88359ab80 1 --2- 172.21.15.19:0/2371823730 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56373bddacc0 0x56373bde1c50 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:03.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.791+0000 7fc88359ab80 1 --2- 172.21.15.19:0/2371823730 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56373bddb610 0x56373bde2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:03.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.791+0000 7fc853e00700 0 client.25816 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:35:03.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.791+0000 7fc88359ab80 1 -- 172.21.15.19:0/2371823730 >> 172.21.15.19:0/2371823730 conn(0x56373bc5cec0 msgr2=0x56373bdd7680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:35:03.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.792+0000 7fc88359ab80 1 -- 172.21.15.19:0/2371823730 shutdown_connections 2024-05-07T02:35:03.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.792+0000 7fc88359ab80 1 -- 172.21.15.19:0/2371823730 wait complete. 2024-05-07T02:35:03.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.792+0000 7fc88359ab80 20 client.25816 trim_cache size 0 max 16384 2024-05-07T02:35:03.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.816+0000 7fc88359ab80 1 Processor -- start 2024-05-07T02:35:03.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.816+0000 7fc88359ab80 1 -- start start 2024-05-07T02:35:03.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.816+0000 7fc88359ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56373bddacc0 0x56373bde0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:03.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.816+0000 7fc88359ab80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56373bddb610 0x56373bde2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:03.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.816+0000 7fc88359ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56373bde4cb0 0x56373bdd9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:03.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.816+0000 7fc88359ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc8701b2490 con 0x56373bde4cb0 2024-05-07T02:35:03.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.816+0000 7fc88359ab80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc8701b40c0 con 0x56373bddb610 2024-05-07T02:35:03.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.816+0000 7fc88359ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc8701b2b60 con 0x56373bddacc0 2024-05-07T02:35:03.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.817+0000 7fc86e800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56373bde4cb0 0x56373bdd9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:03.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.817+0000 7fc86e800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56373bde4cb0 0x56373bdd9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50996/0 (socket says 172.21.15.19:50996) 2024-05-07T02:35:03.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.817+0000 7fc86de00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56373bddacc0 0x56373bde0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:03.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.817+0000 7fc86e800700 1 -- 172.21.15.19:0/826388123 learned_addr learned my addr 172.21.15.19:0/826388123 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:35:03.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.817+0000 7fc865400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56373bddb610 0x56373bde2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:03.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.817+0000 7fc865400700 1 -- 172.21.15.19:0/826388123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56373bddacc0 msgr2=0x56373bde0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:03.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.817+0000 7fc865400700 1 --2- 172.21.15.19:0/826388123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56373bddacc0 0x56373bde0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:03.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.817+0000 7fc865400700 1 -- 172.21.15.19:0/826388123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56373bde4cb0 msgr2=0x56373bdd9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:03.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.817+0000 7fc865400700 1 --2- 172.21.15.19:0/826388123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56373bde4cb0 0x56373bdd9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:03.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.817+0000 7fc865400700 1 -- 172.21.15.19:0/826388123 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56373bdd1ba0 con 0x56373bddb610 2024-05-07T02:35:03.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.817+0000 7fc86e800700 1 --2- 172.21.15.19:0/826388123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56373bde4cb0 0x56373bdd9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:35:03.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.818+0000 7fc865400700 1 --2- 172.21.15.19:0/826388123 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56373bddb610 0x56373bde2640 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x56373bdd5f60 tx=0x7fc85800e510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81ddf0049bfbff8e server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:35:03.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.818+0000 7fc85ea00700 1 -- 172.21.15.19:0/826388123 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc858005460 con 0x56373bddb610 2024-05-07T02:35:03.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.818+0000 7fc85ea00700 1 -- 172.21.15.19:0/826388123 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc858003b30 con 0x56373bddb610 2024-05-07T02:35:03.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.818+0000 7fc85ea00700 1 -- 172.21.15.19:0/826388123 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85802c360 con 0x56373bddb610 2024-05-07T02:35:03.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.819+0000 7fc88359ab80 1 -- 172.21.15.19:0/826388123 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56373bddb610 msgr2=0x56373bde2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:03.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.819+0000 7fc88359ab80 1 --2- 172.21.15.19:0/826388123 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56373bddb610 0x56373bde2640 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x56373bdd5f60 tx=0x7fc85800e510 comp rx=0 tx=0).stop 2024-05-07T02:35:03.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.819+0000 7fc88359ab80 1 -- 172.21.15.19:0/826388123 shutdown_connections 2024-05-07T02:35:03.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.819+0000 7fc88359ab80 1 --2- 172.21.15.19:0/826388123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56373bde4cb0 0x56373bdd9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:03.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.819+0000 7fc88359ab80 1 --2- 172.21.15.19:0/826388123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56373bddacc0 0x56373bde0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:03.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.819+0000 7fc88359ab80 1 --2- 172.21.15.19:0/826388123 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56373bddb610 0x56373bde2640 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:03.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.819+0000 7fc88359ab80 1 -- 172.21.15.19:0/826388123 >> 172.21.15.19:0/826388123 conn(0x56373bc5cec0 msgr2=0x56373bdd9690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:35:03.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.819+0000 7fc88359ab80 1 -- 172.21.15.19:0/826388123 shutdown_connections 2024-05-07T02:35:03.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.819+0000 7fc88359ab80 1 -- 172.21.15.19:0/826388123 wait complete. 2024-05-07T02:35:03.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.820+0000 7fc88359ab80 1 -- start start 2024-05-07T02:35:03.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.820+0000 7fc88359ab80 1 Processor -- start 2024-05-07T02:35:03.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.821+0000 7fc88359ab80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56373bddacc0 0x56373bded100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:03.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.821+0000 7fc88359ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56373bddb610 0x56373bde9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:03.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.821+0000 7fc88359ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56373bde4cb0 0x56373bde99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:03.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.821+0000 7fc88359ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc8701b3930 con 0x56373bde4cb0 2024-05-07T02:35:03.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.821+0000 7fc88359ab80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc8701b3260 con 0x56373bddacc0 2024-05-07T02:35:03.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.821+0000 7fc88359ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc8701b0430 con 0x56373bddb610 2024-05-07T02:35:03.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.821+0000 7fc88359ab80 10 client.0 _ll_register_callbacks cb 0x56373bc90400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:35:03.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.821+0000 7fc86de00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56373bddacc0 0x56373bded100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:03.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.821+0000 7fc865400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56373bddb610 0x56373bde9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:03.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.821+0000 7fc86de00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56373bddacc0 0x56373bded100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:38582/0 (socket says 172.21.15.19:38582) 2024-05-07T02:35:03.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.821+0000 7fc865400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56373bddb610 0x56373bde9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:46482/0 (socket says 172.21.15.19:46482) 2024-05-07T02:35:03.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.821+0000 7fc86de00700 1 -- 172.21.15.19:0/988912545 learned_addr learned my addr 172.21.15.19:0/988912545 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:35:03.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.821+0000 7fc86e800700 1 --2- 172.21.15.19:0/988912545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56373bde4cb0 0x56373bde99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:03.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.821+0000 7fc86de00700 1 -- 172.21.15.19:0/988912545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56373bddb610 msgr2=0x56373bde9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:03.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.821+0000 7fc86de00700 1 --2- 172.21.15.19:0/988912545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56373bddb610 0x56373bde9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:03.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.821+0000 7fc86de00700 1 -- 172.21.15.19:0/988912545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56373bde4cb0 msgr2=0x56373bde99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:03.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.822+0000 7fc86de00700 1 --2- 172.21.15.19:0/988912545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56373bde4cb0 0x56373bde99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:03.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.822+0000 7fc86de00700 1 -- 172.21.15.19:0/988912545 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56373bddb440 con 0x56373bddacc0 2024-05-07T02:35:03.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.822+0000 7fc86de00700 1 --2- 172.21.15.19:0/988912545 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56373bddacc0 0x56373bded100 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc8600a6290 tx=0x7fc860005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fb78e62f8ed29fc server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:35:03.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.822+0000 7fc853e00700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:35:03.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.822+0000 7fc853e00700 1 -- 172.21.15.19:0/988912545 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8600aa910 con 0x56373bddacc0 2024-05-07T02:35:03.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.822+0000 7fc853e00700 1 -- 172.21.15.19:0/988912545 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fc8600039d0 con 0x56373bddacc0 2024-05-07T02:35:03.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.822+0000 7fc88359ab80 1 -- 172.21.15.19:0/988912545 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc858003b30 con 0x56373bddacc0 2024-05-07T02:35:03.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.822+0000 7fc853e00700 1 -- 172.21.15.19:0/988912545 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc8600a97a0 con 0x56373bddacc0 2024-05-07T02:35:03.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.823+0000 7fc88359ab80 10 client.25844 fetch_fsmap learned FSMap version 35 2024-05-07T02:35:03.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.823+0000 7fc88359ab80 1 -- 172.21.15.19:0/988912545 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56373be68850 con 0x56373bddacc0 2024-05-07T02:35:03.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.823+0000 7fc853e00700 1 -- 172.21.15.19:0/988912545 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc858003b30 con 0x56373bddacc0 2024-05-07T02:35:03.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.823+0000 7fc853e00700 1 -- 172.21.15.19:0/988912545 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7fc86000fce0 con 0x56373bddacc0 2024-05-07T02:35:03.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.823+0000 7fc88359ab80 10 client.25844 fetch_fsmap finished waiting for FSMap version 35 2024-05-07T02:35:03.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.823+0000 7fc88359ab80 10 client.25844 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:35:03.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.823+0000 7fc88359ab80 20 client.25844 populate_metadata read hostname 'smithi019' 2024-05-07T02:35:03.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.823+0000 7fc88359ab80 1 --2- 172.21.15.19:0/988912545 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x56373be68e20 0x56373be6b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:03.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.823+0000 7fc88359ab80 4 client.25844 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-07T02:35:03.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.823+0000 7fc88359ab80 1 -- 172.21.15.19:0/988912545 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56373bd0a480 con 0x56373be68e20 2024-05-07T02:35:03.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.823+0000 7fc865400700 1 --2- 172.21.15.19:0/988912545 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x56373be68e20 0x56373be6b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:03.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.824+0000 7fc853e00700 10 client.25844 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:35:03.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.824+0000 7fc865400700 1 --2- 172.21.15.19:0/988912545 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x56373be68e20 0x56373be6b240 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d069d71b67aea6b8 server_cookie=2ed6ed9d85b9c06 in_seq=0 out_seq=0 2024-05-07T02:35:03.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.825+0000 7fc853e00700 1 -- 172.21.15.19:0/988912545 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56373bd0a480 con 0x56373be68e20 2024-05-07T02:35:03.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.825+0000 7fc853e00700 10 client.25844 handle_command_reply: tid=0 2024-05-07T02:35:03.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.825+0000 7fc88359ab80 1 client.25844 shutdown 2024-05-07T02:35:03.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.825+0000 7fc88359ab80 1 -- 172.21.15.19:0/988912545 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56373bddacc0 msgr2=0x56373bded100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:03.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.825+0000 7fc88359ab80 1 --2- 172.21.15.19:0/988912545 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56373bddacc0 0x56373bded100 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc8600a6290 tx=0x7fc860005850 comp rx=0 tx=0).stop 2024-05-07T02:35:03.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.826+0000 7fc88359ab80 1 -- 172.21.15.19:0/988912545 shutdown_connections 2024-05-07T02:35:03.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.826+0000 7fc88359ab80 1 --2- 172.21.15.19:0/988912545 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x56373be68e20 0x56373be6b240 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:03.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.826+0000 7fc88359ab80 1 --2- 172.21.15.19:0/988912545 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56373bde4cb0 0x56373bde99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:03.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.826+0000 7fc88359ab80 1 --2- 172.21.15.19:0/988912545 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56373bddacc0 0x56373bded100 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:03.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.826+0000 7fc88359ab80 1 --2- 172.21.15.19:0/988912545 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56373bddb610 0x56373bde9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:03.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.826+0000 7fc853e00700 0 client.25844 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:35:03.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.826+0000 7fc88359ab80 1 -- 172.21.15.19:0/988912545 >> 172.21.15.19:0/988912545 conn(0x56373bc5cec0 msgr2=0x56373bdd9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:35:03.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.826+0000 7fc88359ab80 1 -- 172.21.15.19:0/988912545 shutdown_connections 2024-05-07T02:35:03.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.826+0000 7fc88359ab80 1 -- 172.21.15.19:0/988912545 wait complete. 2024-05-07T02:35:03.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.826+0000 7fc88359ab80 20 client.25844 trim_cache size 0 max 16384 2024-05-07T02:35:03.827 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:35:03.827 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-07T02:35:03.828 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "3353218b-0f7c-420c-81fe-c6a0a709f368", 2024-05-07T02:35:03.828 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-07T02:35:03.828 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:35:03.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.827+0000 7fc86f200700 1 -- 172.21.15.19:0/218126664 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc8540b29f0 msgr2=0x7fc8540b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:03.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.827+0000 7fc86f200700 1 --2- 172.21.15.19:0/218126664 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc8540b29f0 0x7fc8540b4f00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc868009fd0 tx=0x7fc86800d040 comp rx=0 tx=0).stop 2024-05-07T02:35:03.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.827+0000 7fc86f200700 1 -- 172.21.15.19:0/218126664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc870104400 msgr2=0x7fc870075ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:03.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.827+0000 7fc86f200700 1 --2- 172.21.15.19:0/218126664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc870104400 0x7fc870075ac0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc860006010 tx=0x7fc860005280 comp rx=0 tx=0).stop 2024-05-07T02:35:03.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.827+0000 7fc86f200700 1 -- 172.21.15.19:0/218126664 shutdown_connections 2024-05-07T02:35:03.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.827+0000 7fc86f200700 1 --2- 172.21.15.19:0/218126664 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc8540b29f0 0x7fc8540b4f00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:03.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.827+0000 7fc86f200700 1 --2- 172.21.15.19:0/218126664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc870109680 0x7fc870076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:03.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.827+0000 7fc86f200700 1 --2- 172.21.15.19:0/218126664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc870104400 0x7fc870075ac0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:03.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.827+0000 7fc86f200700 1 --2- 172.21.15.19:0/218126664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc870108d30 0x7fc870076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:03.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.827+0000 7fc86f200700 1 -- 172.21.15.19:0/218126664 >> 172.21.15.19:0/218126664 conn(0x7fc870079d10 msgr2=0x7fc870102970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:35:03.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.828+0000 7fc86f200700 1 -- 172.21.15.19:0/218126664 shutdown_connections 2024-05-07T02:35:03.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:03.828+0000 7fc86f200700 1 -- 172.21.15.19:0/218126664 wait complete. 2024-05-07T02:35:04.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:04 smithi064 ceph-mon[34699]: pgmap v100: 129 pgs: 129 active+clean; 54 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 699 KiB/s wr, 139 op/s 2024-05-07T02:35:04.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:04 smithi019 ceph-mon[29781]: pgmap v100: 129 pgs: 129 active+clean; 54 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 699 KiB/s wr, 139 op/s 2024-05-07T02:35:04.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:04 smithi027 ceph-mon[34398]: pgmap v100: 129 pgs: 129 active+clean; 54 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 699 KiB/s wr, 139 op/s 2024-05-07T02:35:04.872 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-07T02:35:05.054 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:05 smithi019 ceph-mon[29781]: scrub queued for path: / 2024-05-07T02:35:05.349 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:35:05.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:05 smithi064 ceph-mon[34699]: scrub queued for path: / 2024-05-07T02:35:05.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:05 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:35:05.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:05 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:35:05.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:05 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:35:05.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:05 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:35:05.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:05 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:35:05.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:05 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:35:05.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:05 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:35:05.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:05 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:35:05.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:05 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:35:05.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:05 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:35:05.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:05 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:35:05.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:05 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:35:05.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:05 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:35:05.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:05 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:35:05.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:05 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:35:05.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:05 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:35:05.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:05 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:35:05.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:05 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:35:05.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:05 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:35:05.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:05 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:35:05.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:05 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:35:05.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:05 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:35:05.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:05 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:35:05.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:05 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:35:05.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:05 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:35:05.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:05 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:35:05.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:05 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:35:05.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:05 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:35:05.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:05 smithi027 ceph-mon[34398]: scrub queued for path: / 2024-05-07T02:35:05.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:05 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:35:05.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:05 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:35:05.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:05 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:35:05.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:05 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:35:05.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:05 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:35:05.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:05 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:35:05.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:05 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:35:05.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:05 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:35:05.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:05 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:35:05.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:05 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:35:05.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:05 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:35:05.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:05 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:35:05.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:05 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:35:05.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:05 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:35:05.735 INFO:tasks.workunit.client.0.smithi019.stdout: 1 73746 0.00 MB/sec execute 1 sec latency 2136.715 ms 2024-05-07T02:35:06.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:06 smithi064 ceph-mon[34699]: pgmap v101: 129 pgs: 129 active+clean; 56 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 1.2 MiB/s wr, 198 op/s 2024-05-07T02:35:06.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:06 smithi019 ceph-mon[29781]: pgmap v101: 129 pgs: 129 active+clean; 56 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 1.2 MiB/s wr, 198 op/s 2024-05-07T02:35:06.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:06 smithi027 ceph-mon[34398]: pgmap v101: 129 pgs: 129 active+clean; 56 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 1.2 MiB/s wr, 198 op/s 2024-05-07T02:35:06.736 INFO:tasks.workunit.client.0.smithi019.stdout: 1 73746 0.00 MB/sec execute 2 sec latency 3136.876 ms 2024-05-07T02:35:07.736 INFO:tasks.workunit.client.0.smithi019.stdout: 1 73746 0.00 MB/sec execute 3 sec latency 4136.960 ms 2024-05-07T02:35:07.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.976+0000 7f6179e00700 1 -- 172.21.15.19:0/2545715994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6174110970 msgr2=0x7f6174110d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:07.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.976+0000 7f6179e00700 1 --2- 172.21.15.19:0/2545715994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6174110970 0x7f6174110d80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6164009990 tx=0x7f6164009ca0 comp rx=0 tx=0).stop 2024-05-07T02:35:07.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.977+0000 7f6179e00700 1 -- 172.21.15.19:0/2545715994 shutdown_connections 2024-05-07T02:35:07.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.977+0000 7f6179e00700 1 --2- 172.21.15.19:0/2545715994 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f617410b2e0 0x7f6174117810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:07.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.977+0000 7f6179e00700 1 --2- 172.21.15.19:0/2545715994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6174110970 0x7f6174110d80 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:07.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.977+0000 7f6179e00700 1 --2- 172.21.15.19:0/2545715994 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f617410a990 0x7f617410ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:07.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.977+0000 7f6179e00700 1 -- 172.21.15.19:0/2545715994 >> 172.21.15.19:0/2545715994 conn(0x7f6174077c80 msgr2=0x7f61740780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:35:07.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.977+0000 7f6179e00700 1 -- 172.21.15.19:0/2545715994 shutdown_connections 2024-05-07T02:35:07.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.977+0000 7f6179e00700 1 -- 172.21.15.19:0/2545715994 wait complete. 2024-05-07T02:35:07.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.978+0000 7f6179e00700 1 Processor -- start 2024-05-07T02:35:07.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.978+0000 7f6179e00700 1 -- start start 2024-05-07T02:35:07.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.979+0000 7f6179e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f617410a990 0x7f61741a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:07.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.979+0000 7f6179e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f617410b2e0 0x7f61741a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:07.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.979+0000 7f6179e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6174110970 0x7f61741a5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:07.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.979+0000 7f6179e00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f617411aa70 con 0x7f617410a990 2024-05-07T02:35:07.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.979+0000 7f6179e00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f617411a8f0 con 0x7f617410b2e0 2024-05-07T02:35:07.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.979+0000 7f6179e00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f617411abf0 con 0x7f6174110970 2024-05-07T02:35:07.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.979+0000 7f6172a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f617410a990 0x7f61741a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:07.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.979+0000 7f6172000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f617410b2e0 0x7f61741a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:07.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.979+0000 7f6172000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f617410b2e0 0x7f61741a1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:38600/0 (socket says 172.21.15.19:38600) 2024-05-07T02:35:07.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.979+0000 7f6172a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f617410a990 0x7f61741a14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51024/0 (socket says 172.21.15.19:51024) 2024-05-07T02:35:07.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.979+0000 7f6172000700 1 -- 172.21.15.19:0/3284231455 learned_addr learned my addr 172.21.15.19:0/3284231455 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:35:07.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.979+0000 7f6173400700 1 --2- 172.21.15.19:0/3284231455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6174110970 0x7f61741a5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:07.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.980+0000 7f6172000700 1 -- 172.21.15.19:0/3284231455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6174110970 msgr2=0x7f61741a5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:07.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.980+0000 7f6172000700 1 --2- 172.21.15.19:0/3284231455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6174110970 0x7f61741a5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:07.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.980+0000 7f6172000700 1 -- 172.21.15.19:0/3284231455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f617410a990 msgr2=0x7f61741a14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:07.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.980+0000 7f6172000700 1 --2- 172.21.15.19:0/3284231455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f617410a990 0x7f61741a14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:07.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.980+0000 7f6172000700 1 -- 172.21.15.19:0/3284231455 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61741a6370 con 0x7f617410b2e0 2024-05-07T02:35:07.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.980+0000 7f6172000700 1 --2- 172.21.15.19:0/3284231455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f617410b2e0 0x7f61741a1a40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f616000e970 tx=0x7f616000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:35:07.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.980+0000 7f6178a00700 1 -- 172.21.15.19:0/3284231455 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6160004a30 con 0x7f617410b2e0 2024-05-07T02:35:07.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.980+0000 7f6178a00700 1 -- 172.21.15.19:0/3284231455 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f616000cc90 con 0x7f617410b2e0 2024-05-07T02:35:07.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.980+0000 7f6178a00700 1 -- 172.21.15.19:0/3284231455 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6160005610 con 0x7f617410b2e0 2024-05-07T02:35:07.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.980+0000 7f6179e00700 1 -- 172.21.15.19:0/3284231455 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61741a6660 con 0x7f617410b2e0 2024-05-07T02:35:07.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.981+0000 7f6179e00700 1 -- 172.21.15.19:0/3284231455 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61741adf80 con 0x7f617410b2e0 2024-05-07T02:35:07.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.984+0000 7f6178a00700 1 -- 172.21.15.19:0/3284231455 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f616000c7e0 con 0x7f617410b2e0 2024-05-07T02:35:07.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.985+0000 7f61888a2b80 1 Processor -- start 2024-05-07T02:35:07.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.985+0000 7f61888a2b80 1 -- start start 2024-05-07T02:35:07.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.986+0000 7f61888a2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55904ea8e3b0 0x55904ea8e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:07.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.986+0000 7f61888a2b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55904ea88470 0x55904ea88880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:07.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.986+0000 7f61888a2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55904ea88dc0 0x55904ea95040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:07.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.986+0000 7f61888a2b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55904e9aaa20 con 0x55904ea88dc0 2024-05-07T02:35:07.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.986+0000 7f61888a2b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55904e948760 con 0x55904ea88470 2024-05-07T02:35:07.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.986+0000 7f61888a2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55904e9a69e0 con 0x55904ea8e3b0 2024-05-07T02:35:07.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.986+0000 7f6173400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55904ea8e3b0 0x55904ea8e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:07.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.986+0000 7f6178a00700 1 --2- 172.21.15.19:0/3284231455 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f61580b29f0 0x7f61580b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:07.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.986+0000 7f6173400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55904ea8e3b0 0x55904ea8e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:46506/0 (socket says 172.21.15.19:46506) 2024-05-07T02:35:07.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.986+0000 7f6173400700 1 -- 172.21.15.19:0/1695068906 learned_addr learned my addr 172.21.15.19:0/1695068906 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:35:07.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.986+0000 7f6172a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55904ea88470 0x55904ea88880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:07.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.986+0000 7f6172000700 1 --2- 172.21.15.19:0/1695068906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55904ea88dc0 0x55904ea95040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:07.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.987+0000 7f6172a00700 1 -- 172.21.15.19:0/1695068906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55904ea8e3b0 msgr2=0x55904ea8e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:07.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.987+0000 7f6173400700 1 --2- 172.21.15.19:0/3284231455 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f61580b29f0 0x7f61580b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:07.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.987+0000 7f6172a00700 1 --2- 172.21.15.19:0/1695068906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55904ea8e3b0 0x55904ea8e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:07.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.987+0000 7f6172a00700 1 -- 172.21.15.19:0/1695068906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55904ea88dc0 msgr2=0x55904ea95040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:07.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.987+0000 7f6172a00700 1 --2- 172.21.15.19:0/1695068906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55904ea88dc0 0x55904ea95040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:07.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.987+0000 7f6172a00700 1 -- 172.21.15.19:0/1695068906 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55904e8f78d0 con 0x55904ea88470 2024-05-07T02:35:07.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.987+0000 7f6178a00700 1 -- 172.21.15.19:0/3284231455 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f61600e2b90 con 0x7f617410b2e0 2024-05-07T02:35:07.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.987+0000 7f6172a00700 1 --2- 172.21.15.19:0/1695068906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55904ea88470 0x55904ea88880 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f61640059d0 tx=0x7f61640096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d41fa4c2827891cc server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:35:07.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.988+0000 7f6168c00700 1 -- 172.21.15.19:0/1695068906 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6164027070 con 0x55904ea88470 2024-05-07T02:35:07.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.988+0000 7f6168c00700 1 -- 172.21.15.19:0/1695068906 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61640056b0 con 0x55904ea88470 2024-05-07T02:35:07.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.988+0000 7f6168c00700 1 -- 172.21.15.19:0/1695068906 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6164003c10 con 0x55904ea88470 2024-05-07T02:35:07.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.989+0000 7f61888a2b80 1 -- 172.21.15.19:0/1695068906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55904ea88470 msgr2=0x55904ea88880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:07.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.989+0000 7f6173400700 1 --2- 172.21.15.19:0/3284231455 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f61580b29f0 0x7f61580b4f00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x55904ea8de70 tx=0x7f616c004b40 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:35:07.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.989+0000 7f61888a2b80 1 --2- 172.21.15.19:0/1695068906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55904ea88470 0x55904ea88880 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f61640059d0 tx=0x7f61640096a0 comp rx=0 tx=0).stop 2024-05-07T02:35:07.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.989+0000 7f61888a2b80 1 -- 172.21.15.19:0/1695068906 shutdown_connections 2024-05-07T02:35:07.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.989+0000 7f61888a2b80 1 --2- 172.21.15.19:0/1695068906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55904ea88dc0 0x55904ea95040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:07.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.989+0000 7f61888a2b80 1 --2- 172.21.15.19:0/1695068906 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55904ea8e3b0 0x55904ea8e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:07.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.989+0000 7f61888a2b80 1 --2- 172.21.15.19:0/1695068906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55904ea88470 0x55904ea88880 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:07.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.989+0000 7f61888a2b80 1 -- 172.21.15.19:0/1695068906 >> 172.21.15.19:0/1695068906 conn(0x55904e90ae40 msgr2=0x55904ea832f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:35:07.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.989+0000 7f61888a2b80 1 -- 172.21.15.19:0/1695068906 shutdown_connections 2024-05-07T02:35:07.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.989+0000 7f61888a2b80 1 -- 172.21.15.19:0/1695068906 wait complete. 2024-05-07T02:35:07.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.989+0000 7f61888a2b80 1 -- start start 2024-05-07T02:35:07.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.990+0000 7f61888a2b80 1 Processor -- start 2024-05-07T02:35:07.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.990+0000 7f61888a2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55904ea88470 0x55904ea924c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:07.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.990+0000 7f61888a2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55904ea88dc0 0x55904ea92a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:07.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.990+0000 7f61888a2b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55904ea8e3b0 0x55904ea92f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:07.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.990+0000 7f61888a2b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55904ea95760 con 0x55904ea88470 2024-05-07T02:35:07.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.990+0000 7f61888a2b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55904e910b20 con 0x55904ea8e3b0 2024-05-07T02:35:07.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.990+0000 7f61888a2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55904ea958e0 con 0x55904ea88dc0 2024-05-07T02:35:07.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.990+0000 7f61888a2b80 10 client.0 _ll_register_callbacks cb 0x55904e951c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:35:07.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.991+0000 7f6172a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55904ea88470 0x55904ea924c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:07.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.991+0000 7f6173400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55904ea8e3b0 0x55904ea92f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:07.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.991+0000 7f6172a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55904ea88470 0x55904ea924c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51052/0 (socket says 172.21.15.19:51052) 2024-05-07T02:35:07.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.991+0000 7f6172000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55904ea88dc0 0x55904ea92a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:07.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.991+0000 7f6172a00700 1 -- 172.21.15.19:0/2802163952 learned_addr learned my addr 172.21.15.19:0/2802163952 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:35:07.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.991+0000 7f6172a00700 1 -- 172.21.15.19:0/2802163952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55904ea88dc0 msgr2=0x55904ea92a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:07.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.991+0000 7f6172a00700 1 --2- 172.21.15.19:0/2802163952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55904ea88dc0 0x55904ea92a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:07.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.991+0000 7f6172a00700 1 -- 172.21.15.19:0/2802163952 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55904ea8e3b0 msgr2=0x55904ea92f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:07.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.991+0000 7f6172a00700 1 --2- 172.21.15.19:0/2802163952 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55904ea8e3b0 0x55904ea92f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:07.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.991+0000 7f6172a00700 1 -- 172.21.15.19:0/2802163952 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55904e970420 con 0x55904ea88470 2024-05-07T02:35:07.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.991+0000 7f6172000700 1 --2- 172.21.15.19:0/2802163952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55904ea88dc0 0x55904ea92a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:35:07.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.992+0000 7f6172a00700 1 --2- 172.21.15.19:0/2802163952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55904ea88470 0x55904ea924c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f61640098e0 tx=0x7f6164003b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4829e6f2cd4c2a37 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:35:07.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.992+0000 7f615d600700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-07T02:35:07.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.992+0000 7f615d600700 1 -- 172.21.15.19:0/2802163952 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6164031060 con 0x55904ea88470 2024-05-07T02:35:07.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.992+0000 7f61888a2b80 1 -- 172.21.15.19:0/2802163952 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61640056b0 con 0x55904ea88470 2024-05-07T02:35:07.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.992+0000 7f615d600700 1 -- 172.21.15.19:0/2802163952 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f616400b720 con 0x55904ea88470 2024-05-07T02:35:07.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.992+0000 7f615d600700 1 -- 172.21.15.19:0/2802163952 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6164027040 con 0x55904ea88470 2024-05-07T02:35:07.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.992+0000 7f61888a2b80 10 client.15687 fetch_fsmap learned FSMap version 35 2024-05-07T02:35:07.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.992+0000 7f61888a2b80 1 -- 172.21.15.19:0/2802163952 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55904eaa0300 con 0x55904ea88470 2024-05-07T02:35:07.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.993+0000 7f615d600700 1 -- 172.21.15.19:0/2802163952 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61640056b0 con 0x55904ea88470 2024-05-07T02:35:07.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.993+0000 7f615d600700 1 -- 172.21.15.19:0/2802163952 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7f6164022860 con 0x55904ea88470 2024-05-07T02:35:07.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.993+0000 7f61888a2b80 10 client.15687 fetch_fsmap finished waiting for FSMap version 35 2024-05-07T02:35:07.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.993+0000 7f61888a2b80 10 client.15687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:35:07.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.993+0000 7f61888a2b80 20 client.15687 populate_metadata read hostname 'smithi019' 2024-05-07T02:35:07.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.993+0000 7f61888a2b80 1 --2- 172.21.15.19:0/2802163952 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55904eaa27d0 0x55904eaa4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:07.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.993+0000 7f61888a2b80 4 client.15687 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:35:07.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.993+0000 7f61888a2b80 1 -- 172.21.15.19:0/2802163952 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55904ea552e0 con 0x55904eaa27d0 2024-05-07T02:35:07.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.994+0000 7f6172000700 1 --2- 172.21.15.19:0/2802163952 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55904eaa27d0 0x55904eaa4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:07.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.994+0000 7f6172000700 1 --2- 172.21.15.19:0/2802163952 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55904eaa27d0 0x55904eaa4bf0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33a8ea58dec00104 server_cookie=41c9b553bc196eb7 in_seq=0 out_seq=0 2024-05-07T02:35:07.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.994+0000 7f615d600700 10 client.15687 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:35:07.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.995+0000 7f615d600700 1 -- 172.21.15.19:0/2802163952 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55904ea552e0 con 0x55904eaa27d0 2024-05-07T02:35:07.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.995+0000 7f615d600700 10 client.15687 handle_command_reply: tid=0 2024-05-07T02:35:07.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.995+0000 7f61888a2b80 1 client.15687 shutdown 2024-05-07T02:35:07.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.995+0000 7f61888a2b80 1 -- 172.21.15.19:0/2802163952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55904ea88470 msgr2=0x55904ea924c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:07.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.995+0000 7f61888a2b80 1 --2- 172.21.15.19:0/2802163952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55904ea88470 0x55904ea924c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f61640098e0 tx=0x7f6164003b70 comp rx=0 tx=0).stop 2024-05-07T02:35:07.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.995+0000 7f61888a2b80 1 -- 172.21.15.19:0/2802163952 shutdown_connections 2024-05-07T02:35:07.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.995+0000 7f61888a2b80 1 --2- 172.21.15.19:0/2802163952 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55904eaa27d0 0x55904eaa4bf0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:07.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.995+0000 7f61888a2b80 1 --2- 172.21.15.19:0/2802163952 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55904ea8e3b0 0x55904ea92f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:07.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.995+0000 7f61888a2b80 1 --2- 172.21.15.19:0/2802163952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55904ea88470 0x55904ea924c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:07.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.995+0000 7f61888a2b80 1 --2- 172.21.15.19:0/2802163952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55904ea88dc0 0x55904ea92a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:07.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.995+0000 7f615d600700 0 client.15687 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:35:07.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.995+0000 7f61888a2b80 1 -- 172.21.15.19:0/2802163952 >> 172.21.15.19:0/2802163952 conn(0x55904e90ae40 msgr2=0x55904ea87460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:35:07.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.996+0000 7f61888a2b80 1 -- 172.21.15.19:0/2802163952 shutdown_connections 2024-05-07T02:35:07.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.996+0000 7f61888a2b80 1 -- 172.21.15.19:0/2802163952 wait complete. 2024-05-07T02:35:08.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:07.996+0000 7f61888a2b80 20 client.15687 trim_cache size 0 max 16384 2024-05-07T02:35:08.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.019+0000 7f61888a2b80 1 Processor -- start 2024-05-07T02:35:08.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.019+0000 7f61888a2b80 1 -- start start 2024-05-07T02:35:08.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.019+0000 7f61888a2b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55904ea88470 0x55904ea917e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:08.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.019+0000 7f61888a2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55904ea88dc0 0x55904ea9eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:08.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.019+0000 7f61888a2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55904ea8e3b0 0x55904ea92d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:08.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.019+0000 7f61888a2b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f61741b6bc0 con 0x55904ea88dc0 2024-05-07T02:35:08.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.019+0000 7f61888a2b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f61741b87f0 con 0x55904ea88470 2024-05-07T02:35:08.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.019+0000 7f61888a2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f61741b7290 con 0x55904ea8e3b0 2024-05-07T02:35:08.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.020+0000 7f6173400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55904ea8e3b0 0x55904ea92d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:08.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.020+0000 7f6172000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55904ea88dc0 0x55904ea9eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:08.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.020+0000 7f6172a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55904ea88470 0x55904ea917e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:08.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.020+0000 7f6173400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55904ea8e3b0 0x55904ea92d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:46514/0 (socket says 172.21.15.19:46514) 2024-05-07T02:35:08.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.020+0000 7f6173400700 1 -- 172.21.15.19:0/2659045184 learned_addr learned my addr 172.21.15.19:0/2659045184 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:35:08.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.020+0000 7f6172a00700 1 -- 172.21.15.19:0/2659045184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55904ea8e3b0 msgr2=0x55904ea92d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:08.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.020+0000 7f6172a00700 1 --2- 172.21.15.19:0/2659045184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55904ea8e3b0 0x55904ea92d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:08.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.020+0000 7f6172a00700 1 -- 172.21.15.19:0/2659045184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55904ea88dc0 msgr2=0x55904ea9eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:08.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.020+0000 7f6172a00700 1 --2- 172.21.15.19:0/2659045184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55904ea88dc0 0x55904ea9eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:08.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.020+0000 7f6172a00700 1 -- 172.21.15.19:0/2659045184 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55904ea956a0 con 0x55904ea88470 2024-05-07T02:35:08.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.021+0000 7f6172a00700 1 --2- 172.21.15.19:0/2659045184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55904ea88470 0x55904ea917e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6164022e20 tx=0x7f6164003d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6f0abd229817d41 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:35:08.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.021+0000 7f615fe00700 1 -- 172.21.15.19:0/2659045184 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6164031040 con 0x55904ea88470 2024-05-07T02:35:08.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.021+0000 7f615fe00700 1 -- 172.21.15.19:0/2659045184 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6164022480 con 0x55904ea88470 2024-05-07T02:35:08.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.021+0000 7f615fe00700 1 -- 172.21.15.19:0/2659045184 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616400f9d0 con 0x55904ea88470 2024-05-07T02:35:08.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.022+0000 7f61888a2b80 1 -- 172.21.15.19:0/2659045184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55904ea88470 msgr2=0x55904ea917e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:08.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.022+0000 7f61888a2b80 1 --2- 172.21.15.19:0/2659045184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55904ea88470 0x55904ea917e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6164022e20 tx=0x7f6164003d10 comp rx=0 tx=0).stop 2024-05-07T02:35:08.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.022+0000 7f61888a2b80 1 -- 172.21.15.19:0/2659045184 shutdown_connections 2024-05-07T02:35:08.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.022+0000 7f61888a2b80 1 --2- 172.21.15.19:0/2659045184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55904ea8e3b0 0x55904ea92d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:08.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.022+0000 7f61888a2b80 1 --2- 172.21.15.19:0/2659045184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55904ea88470 0x55904ea917e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:08.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.022+0000 7f61888a2b80 1 --2- 172.21.15.19:0/2659045184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55904ea88dc0 0x55904ea9eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:08.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.022+0000 7f61888a2b80 1 -- 172.21.15.19:0/2659045184 >> 172.21.15.19:0/2659045184 conn(0x55904e90ae40 msgr2=0x55904ea93b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:35:08.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.022+0000 7f61888a2b80 1 -- 172.21.15.19:0/2659045184 shutdown_connections 2024-05-07T02:35:08.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.022+0000 7f61888a2b80 1 -- 172.21.15.19:0/2659045184 wait complete. 2024-05-07T02:35:08.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.023+0000 7f61888a2b80 1 -- start start 2024-05-07T02:35:08.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.023+0000 7f61888a2b80 1 Processor -- start 2024-05-07T02:35:08.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.023+0000 7f61888a2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55904ea88470 0x55904eaa7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:08.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.023+0000 7f61888a2b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55904ea88dc0 0x55904eaa8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:08.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.023+0000 7f61888a2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55904ea8e3b0 0x55904eaa4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:08.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.023+0000 7f61888a2b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f61741b8060 con 0x55904ea8e3b0 2024-05-07T02:35:08.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.023+0000 7f61888a2b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f61741b7990 con 0x55904ea88dc0 2024-05-07T02:35:08.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.023+0000 7f61888a2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f61741b4b60 con 0x55904ea88470 2024-05-07T02:35:08.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.023+0000 7f61888a2b80 10 client.0 _ll_register_callbacks cb 0x55904e951c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:35:08.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.024+0000 7f6172a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55904ea88470 0x55904eaa7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:08.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.024+0000 7f6173400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55904ea8e3b0 0x55904eaa4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:08.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.024+0000 7f6172a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55904ea88470 0x55904eaa7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:46530/0 (socket says 172.21.15.19:46530) 2024-05-07T02:35:08.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.024+0000 7f6172a00700 1 -- 172.21.15.19:0/1499446434 learned_addr learned my addr 172.21.15.19:0/1499446434 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:35:08.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.024+0000 7f6172000700 1 --2- 172.21.15.19:0/1499446434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55904ea88dc0 0x55904eaa8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:08.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.024+0000 7f6172a00700 1 -- 172.21.15.19:0/1499446434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55904ea88dc0 msgr2=0x55904eaa8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:08.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.024+0000 7f6172a00700 1 --2- 172.21.15.19:0/1499446434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55904ea88dc0 0x55904eaa8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:08.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.024+0000 7f6172a00700 1 -- 172.21.15.19:0/1499446434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55904ea8e3b0 msgr2=0x55904eaa4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:08.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.024+0000 7f6172a00700 1 --2- 172.21.15.19:0/1499446434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55904ea8e3b0 0x55904eaa4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:08.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.024+0000 7f6172a00700 1 -- 172.21.15.19:0/1499446434 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55904eaa5730 con 0x55904ea88470 2024-05-07T02:35:08.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.025+0000 7f6172a00700 1 --2- 172.21.15.19:0/1499446434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55904ea88470 0x55904eaa7b00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6164009990 tx=0x7f61640057e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99c1b675125f0614 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:35:08.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.025+0000 7f615d600700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:35:08.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.025+0000 7f615d600700 1 -- 172.21.15.19:0/1499446434 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6164031070 con 0x55904ea88470 2024-05-07T02:35:08.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.025+0000 7f61888a2b80 1 -- 172.21.15.19:0/1499446434 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6164022480 con 0x55904ea88470 2024-05-07T02:35:08.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.025+0000 7f615d600700 1 -- 172.21.15.19:0/1499446434 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f616400b920 con 0x55904ea88470 2024-05-07T02:35:08.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.025+0000 7f615d600700 1 -- 172.21.15.19:0/1499446434 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6164049050 con 0x55904ea88470 2024-05-07T02:35:08.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.025+0000 7f61888a2b80 10 client.25840 fetch_fsmap learned FSMap version 35 2024-05-07T02:35:08.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.025+0000 7f61888a2b80 1 -- 172.21.15.19:0/1499446434 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55904eaa5730 con 0x55904ea88470 2024-05-07T02:35:08.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.026+0000 7f615d600700 1 -- 172.21.15.19:0/1499446434 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6164022480 con 0x55904ea88470 2024-05-07T02:35:08.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.026+0000 7f615d600700 1 -- 172.21.15.19:0/1499446434 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 35) ==== 8382+0+0 (secure 0 0 0) 0x7f616400fb30 con 0x55904ea88470 2024-05-07T02:35:08.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.026+0000 7f61888a2b80 10 client.25840 fetch_fsmap finished waiting for FSMap version 35 2024-05-07T02:35:08.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.026+0000 7f61888a2b80 10 client.25840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:35:08.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.026+0000 7f61888a2b80 20 client.25840 populate_metadata read hostname 'smithi019' 2024-05-07T02:35:08.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.026+0000 7f61888a2b80 1 --2- 172.21.15.19:0/1499446434 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55904eb0c9c0 0x55904eb0ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:08.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.026+0000 7f61888a2b80 4 client.25840 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-07T02:35:08.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.026+0000 7f61888a2b80 1 -- 172.21.15.19:0/1499446434 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55904e9b8400 con 0x55904eb0c9c0 2024-05-07T02:35:08.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.026+0000 7f6172000700 1 --2- 172.21.15.19:0/1499446434 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55904eb0c9c0 0x55904eb0ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:08.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.027+0000 7f6172000700 1 --2- 172.21.15.19:0/1499446434 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55904eb0c9c0 0x55904eb0ede0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7279ef7dfcce7217 server_cookie=6ba456aeb5de1099 in_seq=0 out_seq=0 2024-05-07T02:35:08.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.027+0000 7f615d600700 10 client.25840 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:35:08.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.027+0000 7f615d600700 1 -- 172.21.15.19:0/1499446434 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x55904e9b8400 con 0x55904eb0c9c0 2024-05-07T02:35:08.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.027+0000 7f615d600700 10 client.25840 handle_command_reply: tid=0 2024-05-07T02:35:08.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.027+0000 7f61888a2b80 1 client.25840 shutdown 2024-05-07T02:35:08.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.027+0000 7f61888a2b80 1 -- 172.21.15.19:0/1499446434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55904ea88470 msgr2=0x55904eaa7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:08.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.027+0000 7f61888a2b80 1 --2- 172.21.15.19:0/1499446434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55904ea88470 0x55904eaa7b00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6164009990 tx=0x7f61640057e0 comp rx=0 tx=0).stop 2024-05-07T02:35:08.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.027+0000 7f61888a2b80 1 -- 172.21.15.19:0/1499446434 shutdown_connections 2024-05-07T02:35:08.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.027+0000 7f61888a2b80 1 --2- 172.21.15.19:0/1499446434 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55904eb0c9c0 0x55904eb0ede0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:08.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.027+0000 7f61888a2b80 1 --2- 172.21.15.19:0/1499446434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55904ea8e3b0 0x55904eaa4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:08.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.027+0000 7f61888a2b80 1 --2- 172.21.15.19:0/1499446434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55904ea88470 0x55904eaa7b00 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:08.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.027+0000 7f61888a2b80 1 --2- 172.21.15.19:0/1499446434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55904ea88dc0 0x55904eaa8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:08.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.027+0000 7f615d600700 0 client.25840 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:35:08.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.028+0000 7f61888a2b80 1 -- 172.21.15.19:0/1499446434 >> 172.21.15.19:0/1499446434 conn(0x55904e90ae40 msgr2=0x55904ea91540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:35:08.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.028+0000 7f61888a2b80 1 -- 172.21.15.19:0/1499446434 shutdown_connections 2024-05-07T02:35:08.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.028+0000 7f61888a2b80 1 -- 172.21.15.19:0/1499446434 wait complete. 2024-05-07T02:35:08.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.028+0000 7f61888a2b80 20 client.25840 trim_cache size 0 max 16384 2024-05-07T02:35:08.037 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:35:08.037 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "scrub active (1 inodes in the stack)", 2024-05-07T02:35:08.037 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-05-07T02:35:08.037 INFO:teuthology.orchestra.run.smithi019.stdout: "3353218b-0f7c-420c-81fe-c6a0a709f368": { 2024-05-07T02:35:08.037 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-05-07T02:35:08.037 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "3353218b-0f7c-420c-81fe-c6a0a709f368", 2024-05-07T02:35:08.037 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-05-07T02:35:08.037 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:35:08.038 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:35:08.038 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:35:08.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.029+0000 7f6179e00700 1 -- 172.21.15.19:0/3284231455 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f61580b29f0 msgr2=0x7f61580b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:08.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.029+0000 7f6179e00700 1 --2- 172.21.15.19:0/3284231455 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f61580b29f0 0x7f61580b4f00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x55904ea8de70 tx=0x7f616c004b40 comp rx=0 tx=0).stop 2024-05-07T02:35:08.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.029+0000 7f6179e00700 1 -- 172.21.15.19:0/3284231455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f617410b2e0 msgr2=0x7f61741a1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:08.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.029+0000 7f6179e00700 1 --2- 172.21.15.19:0/3284231455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f617410b2e0 0x7f61741a1a40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f616000e970 tx=0x7f616000ec80 comp rx=0 tx=0).stop 2024-05-07T02:35:08.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.029+0000 7f6179e00700 1 -- 172.21.15.19:0/3284231455 shutdown_connections 2024-05-07T02:35:08.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.029+0000 7f6179e00700 1 --2- 172.21.15.19:0/3284231455 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6174110970 0x7f61741a5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:08.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.029+0000 7f6179e00700 1 --2- 172.21.15.19:0/3284231455 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f61580b29f0 0x7f61580b4f00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:08.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.029+0000 7f6179e00700 1 --2- 172.21.15.19:0/3284231455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f617410a990 0x7f61741a14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:08.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.029+0000 7f6179e00700 1 --2- 172.21.15.19:0/3284231455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f617410b2e0 0x7f61741a1a40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:08.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.029+0000 7f6179e00700 1 -- 172.21.15.19:0/3284231455 >> 172.21.15.19:0/3284231455 conn(0x7f6174077c80 msgr2=0x7f617407bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:35:08.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.029+0000 7f6179e00700 1 -- 172.21.15.19:0/3284231455 shutdown_connections 2024-05-07T02:35:08.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:08.029+0000 7f6179e00700 1 -- 172.21.15.19:0/3284231455 wait complete. 2024-05-07T02:35:08.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:08 smithi064 ceph-mon[34699]: pgmap v102: 129 pgs: 129 active+clean; 56 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 843 KiB/s wr, 135 op/s 2024-05-07T02:35:08.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:08 smithi019 ceph-mon[29781]: pgmap v102: 129 pgs: 129 active+clean; 56 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 843 KiB/s wr, 135 op/s 2024-05-07T02:35:08.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:08 smithi027 ceph-mon[34398]: pgmap v102: 129 pgs: 129 active+clean; 56 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 843 KiB/s wr, 135 op/s 2024-05-07T02:35:08.736 INFO:tasks.workunit.client.0.smithi019.stdout: 1 73746 0.00 MB/sec execute 4 sec latency 5137.083 ms 2024-05-07T02:35:08.989 INFO:tasks.cephfs.filesystem:scrub status for tag:3353218b-0f7c-420c-81fe-c6a0a709f368 - {'path': '/', 'tag': '3353218b-0f7c-420c-81fe-c6a0a709f368', 'options': 'recursive,force'} 2024-05-07T02:35:09.736 INFO:tasks.workunit.client.0.smithi019.stdout: 1 73746 0.00 MB/sec execute 5 sec latency 6137.218 ms 2024-05-07T02:35:10.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:10 smithi064 ceph-mon[34699]: pgmap v103: 129 pgs: 129 active+clean; 56 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 843 KiB/s wr, 135 op/s 2024-05-07T02:35:10.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:10 smithi019 ceph-mon[29781]: pgmap v103: 129 pgs: 129 active+clean; 56 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 843 KiB/s wr, 135 op/s 2024-05-07T02:35:10.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:10 smithi027 ceph-mon[34398]: pgmap v103: 129 pgs: 129 active+clean; 56 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 843 KiB/s wr, 135 op/s 2024-05-07T02:35:10.736 INFO:tasks.workunit.client.0.smithi019.stdout: 1 73746 0.00 MB/sec execute 6 sec latency 7137.312 ms 2024-05-07T02:35:11.736 INFO:tasks.workunit.client.0.smithi019.stdout: 1 73746 0.00 MB/sec execute 7 sec latency 8137.456 ms 2024-05-07T02:35:12.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:12 smithi064 ceph-mon[34699]: pgmap v104: 129 pgs: 129 active+clean; 58 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 1.1 MiB/s wr, 148 op/s 2024-05-07T02:35:12.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:12 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:35:12.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:12 smithi019 ceph-mon[29781]: pgmap v104: 129 pgs: 129 active+clean; 58 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 1.1 MiB/s wr, 148 op/s 2024-05-07T02:35:12.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:12 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:35:12.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:12 smithi027 ceph-mon[34398]: pgmap v104: 129 pgs: 129 active+clean; 58 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 1.1 MiB/s wr, 148 op/s 2024-05-07T02:35:12.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:12 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:35:12.736 INFO:tasks.workunit.client.0.smithi019.stdout: 1 73746 0.00 MB/sec execute 8 sec latency 9137.545 ms 2024-05-07T02:35:13.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:13 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:35:13.648 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:13 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:35:13.649 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:13 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:35:13.736 INFO:tasks.workunit.client.0.smithi019.stdout: 1 73746 0.00 MB/sec execute 9 sec latency 10137.667 ms 2024-05-07T02:35:13.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:13 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:35:13.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:13 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:35:13.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:13 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:35:13.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:13 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:35:13.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:13 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:35:13.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:13 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:35:14.737 INFO:tasks.workunit.client.0.smithi019.stdout: 1 73746 0.00 MB/sec execute 10 sec latency 11137.768 ms 2024-05-07T02:35:14.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:14 smithi064 ceph-mon[34699]: pgmap v105: 129 pgs: 129 active+clean; 58 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 750 KiB/s wr, 71 op/s 2024-05-07T02:35:14.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:14 smithi019 ceph-mon[29781]: pgmap v105: 129 pgs: 129 active+clean; 58 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 750 KiB/s wr, 71 op/s 2024-05-07T02:35:14.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:14 smithi027 ceph-mon[34398]: pgmap v105: 129 pgs: 129 active+clean; 58 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 750 KiB/s wr, 71 op/s 2024-05-07T02:35:15.737 INFO:tasks.workunit.client.0.smithi019.stdout: 1 73746 0.00 MB/sec execute 11 sec latency 12137.907 ms 2024-05-07T02:35:16.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:16 smithi064 ceph-mon[34699]: pgmap v106: 129 pgs: 129 active+clean; 58 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 984 KiB/s wr, 83 op/s 2024-05-07T02:35:16.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:16 smithi019 ceph-mon[29781]: pgmap v106: 129 pgs: 129 active+clean; 58 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 984 KiB/s wr, 83 op/s 2024-05-07T02:35:16.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:16 smithi027 ceph-mon[34398]: pgmap v106: 129 pgs: 129 active+clean; 58 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 984 KiB/s wr, 83 op/s 2024-05-07T02:35:16.737 INFO:tasks.workunit.client.0.smithi019.stdout: 1 73746 0.00 MB/sec execute 12 sec latency 13138.054 ms 2024-05-07T02:35:17.737 INFO:tasks.workunit.client.0.smithi019.stdout: 1 73746 0.00 MB/sec execute 13 sec latency 14138.206 ms 2024-05-07T02:35:18.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:18 smithi064 ceph-mon[34699]: pgmap v107: 129 pgs: 129 active+clean; 58 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 479 KiB/s wr, 24 op/s 2024-05-07T02:35:18.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:18 smithi019 ceph-mon[29781]: pgmap v107: 129 pgs: 129 active+clean; 58 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 479 KiB/s wr, 24 op/s 2024-05-07T02:35:18.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:18 smithi027 ceph-mon[34398]: pgmap v107: 129 pgs: 129 active+clean; 58 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 479 KiB/s wr, 24 op/s 2024-05-07T02:35:18.737 INFO:tasks.workunit.client.0.smithi019.stdout: 1 73746 0.00 MB/sec execute 14 sec latency 15138.311 ms 2024-05-07T02:35:19.737 INFO:tasks.workunit.client.0.smithi019.stdout: 1 73746 0.00 MB/sec execute 15 sec latency 16138.467 ms 2024-05-07T02:35:20.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:20 smithi064 ceph-mon[34699]: pgmap v108: 129 pgs: 129 active+clean; 58 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 479 KiB/s wr, 24 op/s 2024-05-07T02:35:20.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:20 smithi019 ceph-mon[29781]: pgmap v108: 129 pgs: 129 active+clean; 58 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 479 KiB/s wr, 24 op/s 2024-05-07T02:35:20.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:20 smithi027 ceph-mon[34398]: pgmap v108: 129 pgs: 129 active+clean; 58 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 479 KiB/s wr, 24 op/s 2024-05-07T02:35:20.738 INFO:tasks.workunit.client.0.smithi019.stdout: 1 74005 0.10 MB/sec execute 16 sec latency 16895.772 ms 2024-05-07T02:35:21.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:21 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:35:21.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:21 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:35:21.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:21 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:35:21.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:21 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:35:21.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:21 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:35:21.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:21 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:35:21.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:21 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:35:21.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:21 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:35:21.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:21 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:35:21.737 INFO:tasks.workunit.client.0.smithi019.stdout: 1 74852 0.46 MB/sec execute 17 sec latency 24.932 ms 2024-05-07T02:35:22.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:22 smithi064 ceph-mon[34699]: pgmap v109: 129 pgs: 129 active+clean; 61 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 765 KiB/s wr, 43 op/s 2024-05-07T02:35:22.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:22 smithi019 ceph-mon[29781]: pgmap v109: 129 pgs: 129 active+clean; 61 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 765 KiB/s wr, 43 op/s 2024-05-07T02:35:22.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:22 smithi027 ceph-mon[34398]: pgmap v109: 129 pgs: 129 active+clean; 61 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 765 KiB/s wr, 43 op/s 2024-05-07T02:35:22.738 INFO:tasks.workunit.client.0.smithi019.stdout: 1 75333 0.47 MB/sec execute 18 sec latency 25.151 ms 2024-05-07T02:35:23.738 INFO:tasks.workunit.client.0.smithi019.stdout: 1 76747 0.73 MB/sec execute 19 sec latency 39.973 ms 2024-05-07T02:35:24.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:24 smithi064 ceph-mon[34699]: pgmap v110: 129 pgs: 129 active+clean; 61 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 519 KiB/s wr, 31 op/s 2024-05-07T02:35:24.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:24 smithi019 ceph-mon[29781]: pgmap v110: 129 pgs: 129 active+clean; 61 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 519 KiB/s wr, 31 op/s 2024-05-07T02:35:24.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:24 smithi027 ceph-mon[34398]: pgmap v110: 129 pgs: 129 active+clean; 61 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 519 KiB/s wr, 31 op/s 2024-05-07T02:35:24.738 INFO:tasks.workunit.client.0.smithi019.stdout: 1 77906 1.13 MB/sec execute 20 sec latency 27.562 ms 2024-05-07T02:35:25.738 INFO:tasks.workunit.client.0.smithi019.stdout: 1 78462 1.30 MB/sec execute 21 sec latency 31.014 ms 2024-05-07T02:35:26.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:26 smithi064 ceph-mon[34699]: pgmap v111: 129 pgs: 129 active+clean; 60 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 1.3 MiB/s wr, 108 op/s 2024-05-07T02:35:26.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:26 smithi019 ceph-mon[29781]: pgmap v111: 129 pgs: 129 active+clean; 60 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 1.3 MiB/s wr, 108 op/s 2024-05-07T02:35:26.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:26 smithi027 ceph-mon[34398]: pgmap v111: 129 pgs: 129 active+clean; 60 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 1.3 MiB/s wr, 108 op/s 2024-05-07T02:35:26.738 INFO:tasks.workunit.client.0.smithi019.stdout: 1 78912 1.26 MB/sec execute 22 sec latency 24.260 ms 2024-05-07T02:35:27.738 INFO:tasks.workunit.client.0.smithi019.stdout: 1 80535 1.50 MB/sec execute 23 sec latency 42.457 ms 2024-05-07T02:35:28.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:28 smithi064 ceph-mon[34699]: pgmap v112: 129 pgs: 129 active+clean; 60 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 1.1 MiB/s wr, 96 op/s 2024-05-07T02:35:28.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:28 smithi019 ceph-mon[29781]: pgmap v112: 129 pgs: 129 active+clean; 60 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 1.1 MiB/s wr, 96 op/s 2024-05-07T02:35:28.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:28 smithi027 ceph-mon[34398]: pgmap v112: 129 pgs: 129 active+clean; 60 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 1.1 MiB/s wr, 96 op/s 2024-05-07T02:35:28.738 INFO:tasks.workunit.client.0.smithi019.stdout: 1 81748 1.87 MB/sec execute 24 sec latency 23.554 ms 2024-05-07T02:35:29.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:29 smithi064 ceph-mon[34699]: scrub summary: idle 2024-05-07T02:35:29.361 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:29 smithi064 ceph-mon[34699]: scrub summary: idle 2024-05-07T02:35:29.361 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:29 smithi064 ceph-mon[34699]: scrub summary: idle 2024-05-07T02:35:29.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:29 smithi019 ceph-mon[29781]: scrub summary: idle 2024-05-07T02:35:29.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:29 smithi019 ceph-mon[29781]: scrub summary: idle 2024-05-07T02:35:29.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:29 smithi019 ceph-mon[29781]: scrub summary: idle 2024-05-07T02:35:29.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:29 smithi027 ceph-mon[34398]: scrub summary: idle 2024-05-07T02:35:29.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:29 smithi027 ceph-mon[34398]: scrub summary: idle 2024-05-07T02:35:29.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:29 smithi027 ceph-mon[34398]: scrub summary: idle 2024-05-07T02:35:29.738 INFO:tasks.workunit.client.0.smithi019.stdout: 1 82095 1.85 MB/sec execute 25 sec latency 57.847 ms 2024-05-07T02:35:30.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:30 smithi064 ceph-mon[34699]: scrub summary: idle 2024-05-07T02:35:30.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:30 smithi064 ceph-mon[34699]: scrub summary: idle 2024-05-07T02:35:30.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:30 smithi064 ceph-mon[34699]: pgmap v113: 129 pgs: 129 active+clean; 60 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 1.1 MiB/s wr, 96 op/s 2024-05-07T02:35:30.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:30 smithi019 ceph-mon[29781]: scrub summary: idle 2024-05-07T02:35:30.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:30 smithi019 ceph-mon[29781]: scrub summary: idle 2024-05-07T02:35:30.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:30 smithi019 ceph-mon[29781]: pgmap v113: 129 pgs: 129 active+clean; 60 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 1.1 MiB/s wr, 96 op/s 2024-05-07T02:35:30.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:30 smithi027 ceph-mon[34398]: scrub summary: idle 2024-05-07T02:35:30.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:30 smithi027 ceph-mon[34398]: scrub summary: idle 2024-05-07T02:35:30.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:30 smithi027 ceph-mon[34398]: pgmap v113: 129 pgs: 129 active+clean; 60 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 1.1 MiB/s wr, 96 op/s 2024-05-07T02:35:30.738 INFO:tasks.workunit.client.0.smithi019.stdout: 1 82778 1.86 MB/sec execute 26 sec latency 24.059 ms 2024-05-07T02:35:31.739 INFO:tasks.workunit.client.0.smithi019.stdout: 1 84244 2.02 MB/sec execute 27 sec latency 18.148 ms 2024-05-07T02:35:32.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:32 smithi064 ceph-mon[34699]: pgmap v114: 129 pgs: 129 active+clean; 64 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 1.7 MiB/s wr, 162 op/s 2024-05-07T02:35:32.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:32 smithi019 ceph-mon[29781]: pgmap v114: 129 pgs: 129 active+clean; 64 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 1.7 MiB/s wr, 162 op/s 2024-05-07T02:35:32.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:32 smithi027 ceph-mon[34398]: pgmap v114: 129 pgs: 129 active+clean; 64 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 1.7 MiB/s wr, 162 op/s 2024-05-07T02:35:32.739 INFO:tasks.workunit.client.0.smithi019.stdout: 1 85394 2.33 MB/sec execute 28 sec latency 24.556 ms 2024-05-07T02:35:33.739 INFO:tasks.workunit.client.0.smithi019.stdout: 1 85627 2.26 MB/sec execute 29 sec latency 31.971 ms 2024-05-07T02:35:34.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:34 smithi064 ceph-mon[34699]: pgmap v115: 129 pgs: 129 active+clean; 64 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 1.4 MiB/s wr, 143 op/s 2024-05-07T02:35:34.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:34 smithi019 ceph-mon[29781]: pgmap v115: 129 pgs: 129 active+clean; 64 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 1.4 MiB/s wr, 143 op/s 2024-05-07T02:35:34.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:34 smithi027 ceph-mon[34398]: pgmap v115: 129 pgs: 129 active+clean; 64 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 1.4 MiB/s wr, 143 op/s 2024-05-07T02:35:34.739 INFO:tasks.workunit.client.0.smithi019.stdout: 1 86355 2.25 MB/sec execute 30 sec latency 40.730 ms 2024-05-07T02:35:35.739 INFO:tasks.workunit.client.0.smithi019.stdout: 1 87896 2.47 MB/sec execute 31 sec latency 19.649 ms 2024-05-07T02:35:36.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:36 smithi064 ceph-mon[34699]: pgmap v116: 129 pgs: 129 active+clean; 65 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 992 KiB/s rd, 1.8 MiB/s wr, 209 op/s 2024-05-07T02:35:36.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:36 smithi019 ceph-mon[29781]: pgmap v116: 129 pgs: 129 active+clean; 65 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 992 KiB/s rd, 1.8 MiB/s wr, 209 op/s 2024-05-07T02:35:36.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:36 smithi027 ceph-mon[34398]: pgmap v116: 129 pgs: 129 active+clean; 65 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 992 KiB/s rd, 1.8 MiB/s wr, 209 op/s 2024-05-07T02:35:36.739 INFO:tasks.workunit.client.0.smithi019.stdout: 1 88934 2.63 MB/sec execute 32 sec latency 24.403 ms 2024-05-07T02:35:37.739 INFO:tasks.workunit.client.0.smithi019.stdout: 1 89168 2.56 MB/sec execute 33 sec latency 26.813 ms 2024-05-07T02:35:38.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:38 smithi064 ceph-mon[34699]: pgmap v117: 129 pgs: 129 active+clean; 65 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 1019 KiB/s wr, 132 op/s 2024-05-07T02:35:38.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:38 smithi019 ceph-mon[29781]: pgmap v117: 129 pgs: 129 active+clean; 65 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 1019 KiB/s wr, 132 op/s 2024-05-07T02:35:38.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:38 smithi027 ceph-mon[34398]: pgmap v117: 129 pgs: 129 active+clean; 65 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 1019 KiB/s wr, 132 op/s 2024-05-07T02:35:38.739 INFO:tasks.workunit.client.0.smithi019.stdout: 1 89764 2.54 MB/sec execute 34 sec latency 28.488 ms 2024-05-07T02:35:38.990 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-07T02:35:39.493 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:35:39.739 INFO:tasks.workunit.client.0.smithi019.stdout: 1 91485 2.73 MB/sec execute 35 sec latency 67.588 ms 2024-05-07T02:35:40.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:40 smithi064 ceph-mon[34699]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:35:40.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:40 smithi064 ceph-mon[34699]: pgmap v118: 129 pgs: 129 active+clean; 65 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 1019 KiB/s wr, 132 op/s 2024-05-07T02:35:40.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:40 smithi019 ceph-mon[29781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:35:40.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:40 smithi019 ceph-mon[29781]: pgmap v118: 129 pgs: 129 active+clean; 65 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 1019 KiB/s wr, 132 op/s 2024-05-07T02:35:40.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:40 smithi027 ceph-mon[34398]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:35:40.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:40 smithi027 ceph-mon[34398]: pgmap v118: 129 pgs: 129 active+clean; 65 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 1019 KiB/s wr, 132 op/s 2024-05-07T02:35:40.740 INFO:tasks.workunit.client.0.smithi019.stdout: 1 91485 2.66 MB/sec execute 36 sec latency 1067.682 ms 2024-05-07T02:35:41.740 INFO:tasks.workunit.client.0.smithi019.stdout: 1 91485 2.58 MB/sec execute 37 sec latency 2067.782 ms 2024-05-07T02:35:42.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.156+0000 7f92f5c00700 1 -- 172.21.15.19:0/4219563723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92f0105290 msgr2=0x7f92f01056a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:42.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.156+0000 7f92f5c00700 1 --2- 172.21.15.19:0/4219563723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92f0105290 0x7f92f01056a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f92e000b0a0 tx=0x7f92e000b3b0 comp rx=0 tx=0).stop 2024-05-07T02:35:42.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.156+0000 7f92f5c00700 1 -- 172.21.15.19:0/4219563723 shutdown_connections 2024-05-07T02:35:42.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.156+0000 7f92f5c00700 1 --2- 172.21.15.19:0/4219563723 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92f010e700 0x7f92f0110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:42.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.156+0000 7f92f5c00700 1 --2- 172.21.15.19:0/4219563723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92f0105290 0x7f92f01056a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:42.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.156+0000 7f92f5c00700 1 --2- 172.21.15.19:0/4219563723 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92f0105be0 0x7f92f010e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:42.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.156+0000 7f92f5c00700 1 -- 172.21.15.19:0/4219563723 >> 172.21.15.19:0/4219563723 conn(0x7f92f0078ac0 msgr2=0x7f92f0078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:35:42.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.156+0000 7f92f5c00700 1 -- 172.21.15.19:0/4219563723 shutdown_connections 2024-05-07T02:35:42.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.156+0000 7f92f5c00700 1 -- 172.21.15.19:0/4219563723 wait complete. 2024-05-07T02:35:42.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.157+0000 7f92f5c00700 1 Processor -- start 2024-05-07T02:35:42.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.157+0000 7f92f5c00700 1 -- start start 2024-05-07T02:35:42.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.157+0000 7f92f5c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92f0105290 0x7f92f019cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:42.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.157+0000 7f92f5c00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92f0105be0 0x7f92f019d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:42.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.157+0000 7f92f5c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92f010e700 0x7f92f01a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:42.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.157+0000 7f92f5c00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f92f01142d0 con 0x7f92f010e700 2024-05-07T02:35:42.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.157+0000 7f92f5c00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f92f0114150 con 0x7f92f0105290 2024-05-07T02:35:42.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.157+0000 7f92f5c00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f92f0114450 con 0x7f92f0105be0 2024-05-07T02:35:42.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.157+0000 7f92f5200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92f010e700 0x7f92f01a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:42.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.157+0000 7f92efe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92f0105290 0x7f92f019cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:42.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.157+0000 7f92f5200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92f010e700 0x7f92f01a16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39994/0 (socket says 172.21.15.19:39994) 2024-05-07T02:35:42.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.157+0000 7f92efe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92f0105290 0x7f92f019cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:49496/0 (socket says 172.21.15.19:49496) 2024-05-07T02:35:42.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.157+0000 7f92f5200700 1 -- 172.21.15.19:0/800206246 learned_addr learned my addr 172.21.15.19:0/800206246 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:35:42.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.157+0000 7f92e7400700 1 --2- 172.21.15.19:0/800206246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92f0105be0 0x7f92f019d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:42.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.158+0000 7f92f5200700 1 -- 172.21.15.19:0/800206246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92f0105be0 msgr2=0x7f92f019d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:42.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.158+0000 7f92f5200700 1 --2- 172.21.15.19:0/800206246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92f0105be0 0x7f92f019d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:42.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.158+0000 7f92f5200700 1 -- 172.21.15.19:0/800206246 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92f0105290 msgr2=0x7f92f019cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:42.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.158+0000 7f92f5200700 1 --2- 172.21.15.19:0/800206246 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92f0105290 0x7f92f019cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:42.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.158+0000 7f92f5200700 1 -- 172.21.15.19:0/800206246 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f01a1d70 con 0x7f92f010e700 2024-05-07T02:35:42.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.158+0000 7f92e7400700 1 --2- 172.21.15.19:0/800206246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92f0105be0 0x7f92f019d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:35:42.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.158+0000 7f92efe00700 1 --2- 172.21.15.19:0/800206246 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92f0105290 0x7f92f019cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:35:42.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.158+0000 7f92f5200700 1 --2- 172.21.15.19:0/800206246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92f010e700 0x7f92f01a16d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f92e800e940 tx=0x7f92e800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:35:42.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.159+0000 7f92ed600700 1 -- 172.21.15.19:0/800206246 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e8004a30 con 0x7f92f010e700 2024-05-07T02:35:42.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.159+0000 7f92f5c00700 1 -- 172.21.15.19:0/800206246 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92f01a2060 con 0x7f92f010e700 2024-05-07T02:35:42.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.159+0000 7f92ed600700 1 -- 172.21.15.19:0/800206246 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92e80054d0 con 0x7f92f010e700 2024-05-07T02:35:42.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.159+0000 7f92ed600700 1 -- 172.21.15.19:0/800206246 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e800ccf0 con 0x7f92f010e700 2024-05-07T02:35:42.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.159+0000 7f92f5c00700 1 -- 172.21.15.19:0/800206246 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92f01a9980 con 0x7f92f010e700 2024-05-07T02:35:42.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.163+0000 7f92ed600700 1 -- 172.21.15.19:0/800206246 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f92e8005020 con 0x7f92f010e700 2024-05-07T02:35:42.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.164+0000 7f9305c8bb80 1 Processor -- start 2024-05-07T02:35:42.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.164+0000 7f9305c8bb80 1 -- start start 2024-05-07T02:35:42.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.165+0000 7f9305c8bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a58c136a90 0x55a58c136ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:42.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.165+0000 7f92ed600700 1 --2- 172.21.15.19:0/800206246 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f92d40b29f0 0x7f92d40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:42.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.165+0000 7f9305c8bb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a58c1373e0 0x55a58c140620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:42.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.165+0000 7f9305c8bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a58c140b60 0x55a58c142f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:42.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.165+0000 7f9305c8bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a58c058a20 con 0x55a58c140b60 2024-05-07T02:35:42.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.165+0000 7f9305c8bb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a58bff6760 con 0x55a58c1373e0 2024-05-07T02:35:42.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.165+0000 7f9305c8bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a58c0549e0 con 0x55a58c136a90 2024-05-07T02:35:42.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.165+0000 7f92ed600700 1 -- 172.21.15.19:0/800206246 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f92e80e2b00 con 0x7f92f010e700 2024-05-07T02:35:42.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.165+0000 7f92efe00700 1 --2- 172.21.15.19:0/800206246 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f92d40b29f0 0x7f92d40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:42.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.165+0000 7f92e7400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a58c136a90 0x55a58c136ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:42.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.166+0000 7f92e7400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a58c136a90 0x55a58c136ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:42678/0 (socket says 172.21.15.19:42678) 2024-05-07T02:35:42.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.166+0000 7f92e7400700 1 -- 172.21.15.19:0/2984208689 learned_addr learned my addr 172.21.15.19:0/2984208689 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:35:42.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.166+0000 7f92e7400700 1 --2- 172.21.15.19:0/2984208689 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a58c1373e0 0x55a58c140620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:42.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.166+0000 7f92f5200700 1 --2- 172.21.15.19:0/2984208689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a58c140b60 0x55a58c142f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:42.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.166+0000 7f92e7400700 1 -- 172.21.15.19:0/2984208689 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a58c1373e0 msgr2=0x55a58c140620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:42.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.166+0000 7f92e7400700 1 --2- 172.21.15.19:0/2984208689 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a58c1373e0 0x55a58c140620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:42.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.166+0000 7f92e7400700 1 -- 172.21.15.19:0/2984208689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a58c140b60 msgr2=0x55a58c142f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:42.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.166+0000 7f92e7400700 1 --2- 172.21.15.19:0/2984208689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a58c140b60 0x55a58c142f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:42.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.166+0000 7f92e7400700 1 -- 172.21.15.19:0/2984208689 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a58bfa58d0 con 0x55a58c136a90 2024-05-07T02:35:42.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.166+0000 7f92e7400700 1 --2- 172.21.15.19:0/2984208689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a58c136a90 0x55a58c136ea0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f92d8011c70 tx=0x7f92d8009c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b8dd65bcbb2d193 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:35:42.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.166+0000 7f92efe00700 1 --2- 172.21.15.19:0/800206246 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f92d40b29f0 0x7f92d40b4f00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f92e00062a0 tx=0x7f92e0006210 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:35:42.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.167+0000 7f92e5600700 1 -- 172.21.15.19:0/2984208689 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d800f6d0 con 0x55a58c136a90 2024-05-07T02:35:42.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.167+0000 7f92e5600700 1 -- 172.21.15.19:0/2984208689 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92d800fd20 con 0x55a58c136a90 2024-05-07T02:35:42.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.167+0000 7f92e5600700 1 -- 172.21.15.19:0/2984208689 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d801fb90 con 0x55a58c136a90 2024-05-07T02:35:42.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.168+0000 7f9305c8bb80 1 -- 172.21.15.19:0/2984208689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a58c136a90 msgr2=0x55a58c136ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:42.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.168+0000 7f9305c8bb80 1 --2- 172.21.15.19:0/2984208689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a58c136a90 0x55a58c136ea0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f92d8011c70 tx=0x7f92d8009c10 comp rx=0 tx=0).stop 2024-05-07T02:35:42.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.168+0000 7f9305c8bb80 1 -- 172.21.15.19:0/2984208689 shutdown_connections 2024-05-07T02:35:42.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.168+0000 7f9305c8bb80 1 --2- 172.21.15.19:0/2984208689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a58c140b60 0x55a58c142f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:42.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.168+0000 7f9305c8bb80 1 --2- 172.21.15.19:0/2984208689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a58c136a90 0x55a58c136ea0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:42.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.168+0000 7f9305c8bb80 1 --2- 172.21.15.19:0/2984208689 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a58c1373e0 0x55a58c140620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:42.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.168+0000 7f9305c8bb80 1 -- 172.21.15.19:0/2984208689 >> 172.21.15.19:0/2984208689 conn(0x55a58bfb8e40 msgr2=0x55a58c131880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:35:42.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.168+0000 7f9305c8bb80 1 -- 172.21.15.19:0/2984208689 shutdown_connections 2024-05-07T02:35:42.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.168+0000 7f9305c8bb80 1 -- 172.21.15.19:0/2984208689 wait complete. 2024-05-07T02:35:42.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.169+0000 7f9305c8bb80 1 -- start start 2024-05-07T02:35:42.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.169+0000 7f9305c8bb80 1 Processor -- start 2024-05-07T02:35:42.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.169+0000 7f9305c8bb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a58c136a90 0x55a58c13dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:42.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.170+0000 7f9305c8bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a58c1373e0 0x55a58c13e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:42.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.170+0000 7f9305c8bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a58c140b60 0x55a58c13e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:42.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.170+0000 7f9305c8bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a58c143960 con 0x55a58c1373e0 2024-05-07T02:35:42.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.170+0000 7f9305c8bb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a58bfbeb20 con 0x55a58c136a90 2024-05-07T02:35:42.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.170+0000 7f9305c8bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a58c143ae0 con 0x55a58c140b60 2024-05-07T02:35:42.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.170+0000 7f9305c8bb80 10 client.0 _ll_register_callbacks cb 0x55a58bfffc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:35:42.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.170+0000 7f92efe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a58c136a90 0x55a58c13dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:42.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.170+0000 7f92e7400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a58c1373e0 0x55a58c13e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:42.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.170+0000 7f92f5200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a58c140b60 0x55a58c13e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:42.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.170+0000 7f92efe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a58c136a90 0x55a58c13dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:49508/0 (socket says 172.21.15.19:49508) 2024-05-07T02:35:42.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.170+0000 7f92e7400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a58c1373e0 0x55a58c13e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40010/0 (socket says 172.21.15.19:40010) 2024-05-07T02:35:42.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.170+0000 7f92f5200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a58c140b60 0x55a58c13e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:42688/0 (socket says 172.21.15.19:42688) 2024-05-07T02:35:42.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.170+0000 7f92efe00700 1 -- 172.21.15.19:0/4089281601 learned_addr learned my addr 172.21.15.19:0/4089281601 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:35:42.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.170+0000 7f92f5200700 1 -- 172.21.15.19:0/4089281601 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a58c136a90 msgr2=0x55a58c13dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:42.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.170+0000 7f92f5200700 1 --2- 172.21.15.19:0/4089281601 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a58c136a90 0x55a58c13dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:42.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.171+0000 7f92f5200700 1 -- 172.21.15.19:0/4089281601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a58c1373e0 msgr2=0x55a58c13e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:42.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.171+0000 7f92f5200700 1 --2- 172.21.15.19:0/4089281601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a58c1373e0 0x55a58c13e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:42.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.171+0000 7f92f5200700 1 -- 172.21.15.19:0/4089281601 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a58c01e420 con 0x55a58c140b60 2024-05-07T02:35:42.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.171+0000 7f92efe00700 1 --2- 172.21.15.19:0/4089281601 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a58c136a90 0x55a58c13dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:35:42.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.171+0000 7f92f5200700 1 --2- 172.21.15.19:0/4089281601 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a58c140b60 0x55a58c13e630 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f92e80dfe60 tx=0x7f92e80a91e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c74541e00ad7293 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:35:42.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.171+0000 7f92de000700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:35:42.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.171+0000 7f92de000700 1 -- 172.21.15.19:0/4089281601 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e80b2190 con 0x55a58c140b60 2024-05-07T02:35:42.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.171+0000 7f92de000700 1 -- 172.21.15.19:0/4089281601 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f92e806fe20 con 0x55a58c140b60 2024-05-07T02:35:42.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.171+0000 7f9305c8bb80 1 -- 172.21.15.19:0/4089281601 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92d800fd20 con 0x55a58c140b60 2024-05-07T02:35:42.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.171+0000 7f92de000700 1 -- 172.21.15.19:0/4089281601 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f92e8016540 con 0x55a58c140b60 2024-05-07T02:35:42.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.172+0000 7f9305c8bb80 10 client.25852 fetch_fsmap learned FSMap version 37 2024-05-07T02:35:42.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.172+0000 7f9305c8bb80 1 -- 172.21.15.19:0/4089281601 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a58c14e690 con 0x55a58c140b60 2024-05-07T02:35:42.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.172+0000 7f92de000700 1 -- 172.21.15.19:0/4089281601 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92d800fd20 con 0x55a58c140b60 2024-05-07T02:35:42.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.172+0000 7f92de000700 1 -- 172.21.15.19:0/4089281601 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7f92e80a9c50 con 0x55a58c140b60 2024-05-07T02:35:42.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.172+0000 7f9305c8bb80 10 client.25852 fetch_fsmap finished waiting for FSMap version 37 2024-05-07T02:35:42.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.172+0000 7f9305c8bb80 10 client.25852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:35:42.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.172+0000 7f9305c8bb80 20 client.25852 populate_metadata read hostname 'smithi019' 2024-05-07T02:35:42.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.172+0000 7f9305c8bb80 1 --2- 172.21.15.19:0/4089281601 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55a58c150cd0 0x55a58c1530f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:42.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.172+0000 7f9305c8bb80 4 client.25852 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:35:42.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.172+0000 7f9305c8bb80 1 -- 172.21.15.19:0/4089281601 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a58c1032e0 con 0x55a58c150cd0 2024-05-07T02:35:42.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.173+0000 7f92efe00700 1 --2- 172.21.15.19:0/4089281601 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55a58c150cd0 0x55a58c1530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:42.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.173+0000 7f92efe00700 1 --2- 172.21.15.19:0/4089281601 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55a58c150cd0 0x55a58c1530f0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ced6a6f1a189487 server_cookie=d2691f571d6ffc77 in_seq=0 out_seq=0 2024-05-07T02:35:42.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.173+0000 7f92de000700 10 client.25852 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:35:42.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.174+0000 7f92de000700 1 -- 172.21.15.19:0/4089281601 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a58c1032e0 con 0x55a58c150cd0 2024-05-07T02:35:42.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.174+0000 7f92de000700 10 client.25852 handle_command_reply: tid=0 2024-05-07T02:35:42.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.174+0000 7f9305c8bb80 1 client.25852 shutdown 2024-05-07T02:35:42.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.175+0000 7f9305c8bb80 1 -- 172.21.15.19:0/4089281601 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a58c140b60 msgr2=0x55a58c13e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:42.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.175+0000 7f9305c8bb80 1 --2- 172.21.15.19:0/4089281601 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a58c140b60 0x55a58c13e630 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f92e80dfe60 tx=0x7f92e80a91e0 comp rx=0 tx=0).stop 2024-05-07T02:35:42.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.175+0000 7f9305c8bb80 1 -- 172.21.15.19:0/4089281601 shutdown_connections 2024-05-07T02:35:42.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.175+0000 7f9305c8bb80 1 --2- 172.21.15.19:0/4089281601 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55a58c150cd0 0x55a58c1530f0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:42.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.175+0000 7f9305c8bb80 1 --2- 172.21.15.19:0/4089281601 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a58c140b60 0x55a58c13e630 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:42.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.175+0000 7f9305c8bb80 1 --2- 172.21.15.19:0/4089281601 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a58c136a90 0x55a58c13dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:42.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.175+0000 7f9305c8bb80 1 --2- 172.21.15.19:0/4089281601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a58c1373e0 0x55a58c13e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:42.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.175+0000 7f9305c8bb80 1 -- 172.21.15.19:0/4089281601 >> 172.21.15.19:0/4089281601 conn(0x55a58bfb8e40 msgr2=0x55a58c133450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:35:42.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.175+0000 7f92de000700 0 client.25852 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:35:42.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.175+0000 7f9305c8bb80 1 -- 172.21.15.19:0/4089281601 shutdown_connections 2024-05-07T02:35:42.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.175+0000 7f9305c8bb80 1 -- 172.21.15.19:0/4089281601 wait complete. 2024-05-07T02:35:42.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.175+0000 7f9305c8bb80 20 client.25852 trim_cache size 0 max 16384 2024-05-07T02:35:42.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.198+0000 7f9305c8bb80 1 Processor -- start 2024-05-07T02:35:42.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.198+0000 7f9305c8bb80 1 -- start start 2024-05-07T02:35:42.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.198+0000 7f9305c8bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a58c136a90 0x55a58c14ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:42.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.198+0000 7f9305c8bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a58c1373e0 0x55a58c13e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:42.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.198+0000 7f9305c8bb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a58c140b60 0x55a58c158920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:42.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.198+0000 7f9305c8bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f92f01b26d0 con 0x55a58c136a90 2024-05-07T02:35:42.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.198+0000 7f9305c8bb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f92f01b4300 con 0x55a58c140b60 2024-05-07T02:35:42.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.198+0000 7f9305c8bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f92f01b2da0 con 0x55a58c1373e0 2024-05-07T02:35:42.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.199+0000 7f92efe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a58c136a90 0x55a58c14ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:42.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.199+0000 7f92e7400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a58c1373e0 0x55a58c13e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:42.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.199+0000 7f92f5200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a58c140b60 0x55a58c158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:42.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.199+0000 7f92e7400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a58c1373e0 0x55a58c13e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:42696/0 (socket says 172.21.15.19:42696) 2024-05-07T02:35:42.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.199+0000 7f92f5200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a58c140b60 0x55a58c158920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:49518/0 (socket says 172.21.15.19:49518) 2024-05-07T02:35:42.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.199+0000 7f92e7400700 1 -- 172.21.15.19:0/1342142358 learned_addr learned my addr 172.21.15.19:0/1342142358 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:35:42.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.199+0000 7f92e7400700 1 -- 172.21.15.19:0/1342142358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a58c140b60 msgr2=0x55a58c158920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:42.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.199+0000 7f92e7400700 1 --2- 172.21.15.19:0/1342142358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a58c140b60 0x55a58c158920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:42.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.199+0000 7f92e7400700 1 -- 172.21.15.19:0/1342142358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a58c136a90 msgr2=0x55a58c14ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:42.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.199+0000 7f92e7400700 1 --2- 172.21.15.19:0/1342142358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a58c136a90 0x55a58c14ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:42.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.199+0000 7f92e7400700 1 -- 172.21.15.19:0/1342142358 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a58c12d580 con 0x55a58c1373e0 2024-05-07T02:35:42.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.199+0000 7f92e7400700 1 --2- 172.21.15.19:0/1342142358 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a58c1373e0 0x55a58c13e120 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f92d80097a0 tx=0x7f92d8009730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef1bfd1ff0a252a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:35:42.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.200+0000 7f92e4c00700 1 -- 172.21.15.19:0/1342142358 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d80050b0 con 0x55a58c1373e0 2024-05-07T02:35:42.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.200+0000 7f92e4c00700 1 -- 172.21.15.19:0/1342142358 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92d8003680 con 0x55a58c1373e0 2024-05-07T02:35:42.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.200+0000 7f92e4c00700 1 -- 172.21.15.19:0/1342142358 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d800f8e0 con 0x55a58c1373e0 2024-05-07T02:35:42.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.201+0000 7f9305c8bb80 1 -- 172.21.15.19:0/1342142358 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a58c1373e0 msgr2=0x55a58c13e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:42.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.201+0000 7f9305c8bb80 1 --2- 172.21.15.19:0/1342142358 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a58c1373e0 0x55a58c13e120 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f92d80097a0 tx=0x7f92d8009730 comp rx=0 tx=0).stop 2024-05-07T02:35:42.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.201+0000 7f9305c8bb80 1 -- 172.21.15.19:0/1342142358 shutdown_connections 2024-05-07T02:35:42.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.201+0000 7f9305c8bb80 1 --2- 172.21.15.19:0/1342142358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a58c140b60 0x55a58c158920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:42.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.201+0000 7f9305c8bb80 1 --2- 172.21.15.19:0/1342142358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a58c136a90 0x55a58c14ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:42.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.201+0000 7f9305c8bb80 1 --2- 172.21.15.19:0/1342142358 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a58c1373e0 0x55a58c13e120 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:42.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.201+0000 7f9305c8bb80 1 -- 172.21.15.19:0/1342142358 >> 172.21.15.19:0/1342142358 conn(0x55a58bfb8e40 msgr2=0x55a58c131620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:35:42.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.201+0000 7f9305c8bb80 1 -- 172.21.15.19:0/1342142358 shutdown_connections 2024-05-07T02:35:42.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.201+0000 7f9305c8bb80 1 -- 172.21.15.19:0/1342142358 wait complete. 2024-05-07T02:35:42.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.203+0000 7f9305c8bb80 1 -- start start 2024-05-07T02:35:42.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.203+0000 7f9305c8bb80 1 Processor -- start 2024-05-07T02:35:42.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.203+0000 7f9305c8bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a58c136a90 0x55a58c148c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:42.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.203+0000 7f9305c8bb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a58c1373e0 0x55a58c149170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:42.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.203+0000 7f9305c8bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a58c140b60 0x55a58c144160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:42.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.203+0000 7f9305c8bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f92f01b3ba0 con 0x55a58c140b60 2024-05-07T02:35:42.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.203+0000 7f9305c8bb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f92f01b34d0 con 0x55a58c1373e0 2024-05-07T02:35:42.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.203+0000 7f9305c8bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f92f01b0670 con 0x55a58c136a90 2024-05-07T02:35:42.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.203+0000 7f9305c8bb80 10 client.0 _ll_register_callbacks cb 0x55a58bfffc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:35:42.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.203+0000 7f92e7400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a58c1373e0 0x55a58c149170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:42.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.203+0000 7f92f5200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a58c140b60 0x55a58c144160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:42.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.203+0000 7f92efe00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a58c136a90 0x55a58c148c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:42.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.203+0000 7f92e7400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a58c1373e0 0x55a58c149170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:49534/0 (socket says 172.21.15.19:49534) 2024-05-07T02:35:42.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.203+0000 7f92e7400700 1 -- 172.21.15.19:0/2726696763 learned_addr learned my addr 172.21.15.19:0/2726696763 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:35:42.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.203+0000 7f92e7400700 1 -- 172.21.15.19:0/2726696763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a58c136a90 msgr2=0x55a58c148c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:42.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.203+0000 7f92e7400700 1 --2- 172.21.15.19:0/2726696763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a58c136a90 0x55a58c148c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:42.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.203+0000 7f92e7400700 1 -- 172.21.15.19:0/2726696763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a58c140b60 msgr2=0x55a58c144160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:42.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.203+0000 7f92e7400700 1 --2- 172.21.15.19:0/2726696763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a58c140b60 0x55a58c144160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:42.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.203+0000 7f92e7400700 1 -- 172.21.15.19:0/2726696763 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a58c144820 con 0x55a58c1373e0 2024-05-07T02:35:42.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.204+0000 7f92e7400700 1 --2- 172.21.15.19:0/2726696763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a58c1373e0 0x55a58c149170 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f92d8011c70 tx=0x7f92d8009c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60aba1aac558a630 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:35:42.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.204+0000 7f92de000700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:35:42.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.204+0000 7f92de000700 1 -- 172.21.15.19:0/2726696763 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d8003ac0 con 0x55a58c1373e0 2024-05-07T02:35:42.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.204+0000 7f9305c8bb80 1 -- 172.21.15.19:0/2726696763 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92d8003680 con 0x55a58c1373e0 2024-05-07T02:35:42.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.204+0000 7f92de000700 1 -- 172.21.15.19:0/2726696763 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f92d8003c60 con 0x55a58c1373e0 2024-05-07T02:35:42.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.204+0000 7f92de000700 1 -- 172.21.15.19:0/2726696763 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f92d8040050 con 0x55a58c1373e0 2024-05-07T02:35:42.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.204+0000 7f9305c8bb80 10 client.25880 fetch_fsmap learned FSMap version 37 2024-05-07T02:35:42.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.204+0000 7f9305c8bb80 1 -- 172.21.15.19:0/2726696763 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a58c1badc0 con 0x55a58c1373e0 2024-05-07T02:35:42.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.205+0000 7f92de000700 1 -- 172.21.15.19:0/2726696763 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92d8003680 con 0x55a58c1373e0 2024-05-07T02:35:42.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.205+0000 7f92de000700 1 -- 172.21.15.19:0/2726696763 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7f92d803b030 con 0x55a58c1373e0 2024-05-07T02:35:42.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.205+0000 7f9305c8bb80 10 client.25880 fetch_fsmap finished waiting for FSMap version 37 2024-05-07T02:35:42.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.205+0000 7f9305c8bb80 10 client.25880 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:35:42.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.205+0000 7f9305c8bb80 20 client.25880 populate_metadata read hostname 'smithi019' 2024-05-07T02:35:42.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.205+0000 7f9305c8bb80 1 --2- 172.21.15.19:0/2726696763 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55a58c1bb2b0 0x55a58c1bd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:42.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.205+0000 7f9305c8bb80 4 client.25880 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-07T02:35:42.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.205+0000 7f9305c8bb80 1 -- 172.21.15.19:0/2726696763 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a58c066400 con 0x55a58c1bb2b0 2024-05-07T02:35:42.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.206+0000 7f92efe00700 1 --2- 172.21.15.19:0/2726696763 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55a58c1bb2b0 0x55a58c1bd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:42.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.206+0000 7f92efe00700 1 --2- 172.21.15.19:0/2726696763 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55a58c1bb2b0 0x55a58c1bd6d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75d187fb4728c24c server_cookie=5f824344c4152983 in_seq=0 out_seq=0 2024-05-07T02:35:42.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.206+0000 7f92de000700 10 client.25880 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:35:42.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.206+0000 7f92de000700 1 -- 172.21.15.19:0/2726696763 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a58c066400 con 0x55a58c1bb2b0 2024-05-07T02:35:42.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.206+0000 7f92de000700 10 client.25880 handle_command_reply: tid=0 2024-05-07T02:35:42.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.206+0000 7f9305c8bb80 1 client.25880 shutdown 2024-05-07T02:35:42.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.207+0000 7f9305c8bb80 1 -- 172.21.15.19:0/2726696763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a58c1373e0 msgr2=0x55a58c149170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:42.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.207+0000 7f9305c8bb80 1 --2- 172.21.15.19:0/2726696763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a58c1373e0 0x55a58c149170 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f92d8011c70 tx=0x7f92d8009c40 comp rx=0 tx=0).stop 2024-05-07T02:35:42.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.207+0000 7f9305c8bb80 1 -- 172.21.15.19:0/2726696763 shutdown_connections 2024-05-07T02:35:42.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.207+0000 7f9305c8bb80 1 --2- 172.21.15.19:0/2726696763 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55a58c1bb2b0 0x55a58c1bd6d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:42.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.207+0000 7f9305c8bb80 1 --2- 172.21.15.19:0/2726696763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a58c140b60 0x55a58c144160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:42.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.207+0000 7f9305c8bb80 1 --2- 172.21.15.19:0/2726696763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a58c136a90 0x55a58c148c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:42.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.207+0000 7f9305c8bb80 1 --2- 172.21.15.19:0/2726696763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a58c1373e0 0x55a58c149170 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:42.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.207+0000 7f9305c8bb80 1 -- 172.21.15.19:0/2726696763 >> 172.21.15.19:0/2726696763 conn(0x55a58bfb8e40 msgr2=0x55a58c13f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:35:42.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.207+0000 7f92de000700 0 client.25880 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:35:42.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.207+0000 7f9305c8bb80 1 -- 172.21.15.19:0/2726696763 shutdown_connections 2024-05-07T02:35:42.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.207+0000 7f9305c8bb80 1 -- 172.21.15.19:0/2726696763 wait complete. 2024-05-07T02:35:42.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.207+0000 7f9305c8bb80 20 client.25880 trim_cache size 0 max 16384 2024-05-07T02:35:42.210 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:35:42.210 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-07T02:35:42.210 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-07T02:35:42.210 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:35:42.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.208+0000 7f92f5c00700 1 -- 172.21.15.19:0/800206246 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f92d40b29f0 msgr2=0x7f92d40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:42.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.208+0000 7f92f5c00700 1 --2- 172.21.15.19:0/800206246 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f92d40b29f0 0x7f92d40b4f00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f92e00062a0 tx=0x7f92e0006210 comp rx=0 tx=0).stop 2024-05-07T02:35:42.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.208+0000 7f92f5c00700 1 -- 172.21.15.19:0/800206246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92f010e700 msgr2=0x7f92f01a16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:42.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.208+0000 7f92f5c00700 1 --2- 172.21.15.19:0/800206246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92f010e700 0x7f92f01a16d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f92e800e940 tx=0x7f92e800ec50 comp rx=0 tx=0).stop 2024-05-07T02:35:42.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.208+0000 7f92f5c00700 1 -- 172.21.15.19:0/800206246 shutdown_connections 2024-05-07T02:35:42.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.208+0000 7f92f5c00700 1 --2- 172.21.15.19:0/800206246 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f92d40b29f0 0x7f92d40b4f00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:42.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.208+0000 7f92f5c00700 1 --2- 172.21.15.19:0/800206246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92f010e700 0x7f92f01a16d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:42.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.208+0000 7f92f5c00700 1 --2- 172.21.15.19:0/800206246 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92f0105290 0x7f92f019cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:42.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.208+0000 7f92f5c00700 1 --2- 172.21.15.19:0/800206246 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92f0105be0 0x7f92f019d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:42.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.208+0000 7f92f5c00700 1 -- 172.21.15.19:0/800206246 >> 172.21.15.19:0/800206246 conn(0x7f92f0078ac0 msgr2=0x7f92f0102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:35:42.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.208+0000 7f92f5c00700 1 -- 172.21.15.19:0/800206246 shutdown_connections 2024-05-07T02:35:42.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:42.208+0000 7f92f5c00700 1 -- 172.21.15.19:0/800206246 wait complete. 2024-05-07T02:35:42.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:42 smithi064 ceph-mon[34699]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:35:42.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:42 smithi064 ceph-mon[34699]: pgmap v119: 129 pgs: 129 active+clean; 67 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 1.2 MiB/s wr, 176 op/s 2024-05-07T02:35:42.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:42 smithi019 ceph-mon[29781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:35:42.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:42 smithi019 ceph-mon[29781]: pgmap v119: 129 pgs: 129 active+clean; 67 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 1.2 MiB/s wr, 176 op/s 2024-05-07T02:35:42.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:42 smithi027 ceph-mon[34398]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:35:42.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:42 smithi027 ceph-mon[34398]: pgmap v119: 129 pgs: 129 active+clean; 67 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 1.2 MiB/s wr, 176 op/s 2024-05-07T02:35:42.740 INFO:tasks.workunit.client.0.smithi019.stdout: 1 91485 2.52 MB/sec execute 38 sec latency 3067.897 ms 2024-05-07T02:35:43.230 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-07T02:35:43.230 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-07T02:35:43.700 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:35:43.740 INFO:tasks.workunit.client.0.smithi019.stdout: 1 91485 2.45 MB/sec execute 39 sec latency 4067.965 ms 2024-05-07T02:35:44.740 INFO:tasks.workunit.client.0.smithi019.stdout: 1 91655 2.43 MB/sec execute 40 sec latency 4916.754 ms 2024-05-07T02:35:45.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:44 smithi064 ceph-mon[34699]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:35:45.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:44 smithi064 ceph-mon[34699]: pgmap v120: 129 pgs: 129 active+clean; 67 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 632 KiB/s wr, 110 op/s 2024-05-07T02:35:45.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:44 smithi019 ceph-mon[29781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:35:45.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:44 smithi019 ceph-mon[29781]: pgmap v120: 129 pgs: 129 active+clean; 67 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 632 KiB/s wr, 110 op/s 2024-05-07T02:35:45.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:44 smithi027 ceph-mon[34398]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:35:45.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:44 smithi027 ceph-mon[34398]: pgmap v120: 129 pgs: 129 active+clean; 67 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 632 KiB/s wr, 110 op/s 2024-05-07T02:35:45.740 INFO:tasks.workunit.client.0.smithi019.stdout: 1 91747 2.37 MB/sec execute 41 sec latency 887.716 ms 2024-05-07T02:35:46.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.245+0000 7f6a84c00700 1 -- 172.21.15.19:0/2292671516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a80110970 msgr2=0x7f6a80110d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:46.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.245+0000 7f6a84c00700 1 --2- 172.21.15.19:0/2292671516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a80110970 0x7f6a80110d80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6a70009990 tx=0x7f6a70009ca0 comp rx=0 tx=0).stop 2024-05-07T02:35:46.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.246+0000 7f6a84c00700 1 -- 172.21.15.19:0/2292671516 shutdown_connections 2024-05-07T02:35:46.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.246+0000 7f6a84c00700 1 --2- 172.21.15.19:0/2292671516 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a8010b2e0 0x7f6a80117810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:46.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.246+0000 7f6a84c00700 1 --2- 172.21.15.19:0/2292671516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a80110970 0x7f6a80110d80 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:46.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.246+0000 7f6a84c00700 1 --2- 172.21.15.19:0/2292671516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a8010a990 0x7f6a8010ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:46.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.246+0000 7f6a84c00700 1 -- 172.21.15.19:0/2292671516 >> 172.21.15.19:0/2292671516 conn(0x7f6a80077c80 msgr2=0x7f6a800780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:35:46.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.246+0000 7f6a84c00700 1 -- 172.21.15.19:0/2292671516 shutdown_connections 2024-05-07T02:35:46.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.246+0000 7f6a84c00700 1 -- 172.21.15.19:0/2292671516 wait complete. 2024-05-07T02:35:46.247 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:46 smithi019 ceph-mon[29781]: pgmap v121: 129 pgs: 129 active+clean; 67 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 656 KiB/s wr, 120 op/s 2024-05-07T02:35:46.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.247+0000 7f6a84c00700 1 Processor -- start 2024-05-07T02:35:46.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.247+0000 7f6a84c00700 1 -- start start 2024-05-07T02:35:46.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.248+0000 7f6a84c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a8010a990 0x7f6a801a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:46.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.248+0000 7f6a84c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a8010b2e0 0x7f6a801a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:46.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.248+0000 7f6a84c00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a80110970 0x7f6a801a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:46.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.248+0000 7f6a84c00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8011aa70 con 0x7f6a8010a990 2024-05-07T02:35:46.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.248+0000 7f6a84c00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8011a8f0 con 0x7f6a8010b2e0 2024-05-07T02:35:46.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.248+0000 7f6a84c00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8011abf0 con 0x7f6a80110970 2024-05-07T02:35:46.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.248+0000 7f6a7de00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a8010a990 0x7f6a801a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:46.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.248+0000 7f6a7de00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a8010a990 0x7f6a801a1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40058/0 (socket says 172.21.15.19:40058) 2024-05-07T02:35:46.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.248+0000 7f6a7de00700 1 -- 172.21.15.19:0/2197615869 learned_addr learned my addr 172.21.15.19:0/2197615869 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:35:46.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.248+0000 7f6a7d400700 1 --2- 172.21.15.19:0/2197615869 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a8010b2e0 0x7f6a801a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:46.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.248+0000 7f6a7e800700 1 --2- 172.21.15.19:0/2197615869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a80110970 0x7f6a801a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:46.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.249+0000 7f6a7d400700 1 -- 172.21.15.19:0/2197615869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a80110970 msgr2=0x7f6a801a5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:46.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.249+0000 7f6a7d400700 1 --2- 172.21.15.19:0/2197615869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a80110970 0x7f6a801a5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:46.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.249+0000 7f6a7d400700 1 -- 172.21.15.19:0/2197615869 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a8010a990 msgr2=0x7f6a801a1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:46.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.249+0000 7f6a7d400700 1 --2- 172.21.15.19:0/2197615869 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a8010a990 0x7f6a801a1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:46.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.249+0000 7f6a7d400700 1 -- 172.21.15.19:0/2197615869 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a801a6330 con 0x7f6a8010b2e0 2024-05-07T02:35:46.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.249+0000 7f6a7de00700 1 --2- 172.21.15.19:0/2197615869 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a8010a990 0x7f6a801a1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:35:46.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.249+0000 7f6a7d400700 1 --2- 172.21.15.19:0/2197615869 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a8010b2e0 0x7f6a801a1a00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c00e9c0 tx=0x7f6a6c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:35:46.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.249+0000 7f6a76a00700 1 -- 172.21.15.19:0/2197615869 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c00c9c0 con 0x7f6a8010b2e0 2024-05-07T02:35:46.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.249+0000 7f6a76a00700 1 -- 172.21.15.19:0/2197615869 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a6c00cb60 con 0x7f6a8010b2e0 2024-05-07T02:35:46.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.249+0000 7f6a76a00700 1 -- 172.21.15.19:0/2197615869 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c018680 con 0x7f6a8010b2e0 2024-05-07T02:35:46.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.249+0000 7f6a84c00700 1 -- 172.21.15.19:0/2197615869 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a801a6620 con 0x7f6a8010b2e0 2024-05-07T02:35:46.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.250+0000 7f6a84c00700 1 -- 172.21.15.19:0/2197615869 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a801adf40 con 0x7f6a8010b2e0 2024-05-07T02:35:46.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.251+0000 7f6a84c00700 1 -- 172.21.15.19:0/2197615869 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a8010c080 con 0x7f6a8010b2e0 2024-05-07T02:35:46.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.253+0000 7f6a76a00700 1 -- 172.21.15.19:0/2197615869 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f6a6c010430 con 0x7f6a8010b2e0 2024-05-07T02:35:46.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.255+0000 7f6a76a00700 1 --2- 172.21.15.19:0/2197615869 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f6a640b29f0 0x7f6a640b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:46.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.255+0000 7f6a76a00700 1 -- 172.21.15.19:0/2197615869 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f6a6c0e2ee0 con 0x7f6a8010b2e0 2024-05-07T02:35:46.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.255+0000 7f6a7de00700 1 --2- 172.21.15.19:0/2197615869 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f6a640b29f0 0x7f6a640b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:46.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.256+0000 7f6a7de00700 1 --2- 172.21.15.19:0/2197615869 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f6a640b29f0 0x7f6a640b4f00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6a70006010 tx=0x7f6a70005960 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:35:46.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.257+0000 7f6a76a00700 1 -- 172.21.15.19:0/2197615869 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6a6c0aa4d0 con 0x7f6a8010b2e0 2024-05-07T02:35:46.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:46 smithi064 ceph-mon[34699]: pgmap v121: 129 pgs: 129 active+clean; 67 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 656 KiB/s wr, 120 op/s 2024-05-07T02:35:46.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:46 smithi027 ceph-mon[34398]: pgmap v121: 129 pgs: 129 active+clean; 67 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 656 KiB/s wr, 120 op/s 2024-05-07T02:35:46.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.544+0000 7f6a84c00700 1 -- 172.21.15.19:0/2197615869 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a8002e5f0 con 0x7f6a8010b2e0 2024-05-07T02:35:46.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.545+0000 7f6a76a00700 1 -- 172.21.15.19:0/2197615869 <== mon.2 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11930 (secure 0 0 0) 0x7f6a6c0a9dd0 con 0x7f6a8010b2e0 2024-05-07T02:35:46.546 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:35:46.546 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1763670446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1763670446},{"type":"v1","addr":"172.21.15.19:6839","nonce":1763670446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/477529985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":477529985},{"type":"v1","addr":"172.21.15.64:6833","nonce":477529985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-07T02:26:14.960465+0000","modified":"2024-05-07T02:35:43.643320+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24539,"mds_2":24520,"mds_3":24515,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6833/2171873162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2171873162},{"type":"v1","addr":"172.21.15.27:6833","nonce":2171873162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/2379917839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2379917839},{"type":"v1","addr":"172.21.15.19:6835","nonce":2379917839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6837/2067515538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2067515538},{"type":"v1","addr":"172.21.15.19:6837","nonce":2067515538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6835/1000540600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1000540600},{"type":"v1","addr":"172.21.15.27:6835","nonce":1000540600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/1997318872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1997318872},{"type":"v1","addr":"172.21.15.64:6835","nonce":1997318872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":34,"addr":"172.21.15.64:6837/2585907777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2585907777},{"type":"v1","addr":"172.21.15.64:6837","nonce":2585907777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":33,"addr":"172.21.15.27:6837/2583138143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2583138143},{"type":"v1","addr":"172.21.15.27:6837","nonce":2583138143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":32,"addr":"172.21.15.64:6839/1583120004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":1583120004},{"type":"v1","addr":"172.21.15.64:6839","nonce":1583120004}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":29,"addr":"172.21.15.19:6841/3905645246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3905645246},{"type":"v1","addr":"172.21.15.19:6841","nonce":3905645246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6839/3963268634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3963268634},{"type":"v1","addr":"172.21.15.27:6839","nonce":3963268634}]},"join_fscid":-1,"export_targets":[2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24551,24539,24520,24505,24515]},"id":1}]} 2024-05-07T02:35:46.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.549+0000 7f6a84c00700 1 -- 172.21.15.19:0/2197615869 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f6a640b29f0 msgr2=0x7f6a640b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:46.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.549+0000 7f6a84c00700 1 --2- 172.21.15.19:0/2197615869 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f6a640b29f0 0x7f6a640b4f00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6a70006010 tx=0x7f6a70005960 comp rx=0 tx=0).stop 2024-05-07T02:35:46.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.549+0000 7f6a84c00700 1 -- 172.21.15.19:0/2197615869 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a8010b2e0 msgr2=0x7f6a801a1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:46.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.549+0000 7f6a84c00700 1 --2- 172.21.15.19:0/2197615869 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a8010b2e0 0x7f6a801a1a00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c00e9c0 tx=0x7f6a6c00ecd0 comp rx=0 tx=0).stop 2024-05-07T02:35:46.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.549+0000 7f6a84c00700 1 -- 172.21.15.19:0/2197615869 shutdown_connections 2024-05-07T02:35:46.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.549+0000 7f6a84c00700 1 --2- 172.21.15.19:0/2197615869 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6a80110970 0x7f6a801a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:46.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.549+0000 7f6a84c00700 1 --2- 172.21.15.19:0/2197615869 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f6a640b29f0 0x7f6a640b4f00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:46.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.549+0000 7f6a84c00700 1 --2- 172.21.15.19:0/2197615869 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a8010a990 0x7f6a801a1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:46.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.549+0000 7f6a84c00700 1 --2- 172.21.15.19:0/2197615869 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a8010b2e0 0x7f6a801a1a00 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:46.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.549+0000 7f6a84c00700 1 -- 172.21.15.19:0/2197615869 >> 172.21.15.19:0/2197615869 conn(0x7f6a80077c80 msgr2=0x7f6a8007bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:35:46.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.549+0000 7f6a84c00700 1 -- 172.21.15.19:0/2197615869 shutdown_connections 2024-05-07T02:35:46.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:46.549+0000 7f6a84c00700 1 -- 172.21.15.19:0/2197615869 wait complete. 2024-05-07T02:35:46.551 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 38 2024-05-07T02:35:46.740 INFO:tasks.workunit.client.0.smithi019.stdout: 1 91747 2.31 MB/sec execute 42 sec latency 1887.845 ms 2024-05-07T02:35:47.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:47 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/2197615869' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:35:47.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:47 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/2197615869' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:35:47.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:47 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/2197615869' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:35:47.661 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-07T02:35:47.740 INFO:tasks.workunit.client.0.smithi019.stdout: 1 91747 2.26 MB/sec execute 43 sec latency 2887.963 ms 2024-05-07T02:35:48.141 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:35:48.352 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:48 smithi019 ceph-mon[29781]: pgmap v122: 129 pgs: 129 active+clean; 67 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 254 KiB/s wr, 54 op/s 2024-05-07T02:35:48.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:48 smithi064 ceph-mon[34699]: pgmap v122: 129 pgs: 129 active+clean; 67 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 254 KiB/s wr, 54 op/s 2024-05-07T02:35:48.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:48 smithi027 ceph-mon[34398]: pgmap v122: 129 pgs: 129 active+clean; 67 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 254 KiB/s wr, 54 op/s 2024-05-07T02:35:48.740 INFO:tasks.workunit.client.0.smithi019.stdout: 1 91747 2.21 MB/sec execute 44 sec latency 3888.017 ms 2024-05-07T02:35:49.741 INFO:tasks.workunit.client.0.smithi019.stdout: 1 91932 2.18 MB/sec execute 45 sec latency 4543.143 ms 2024-05-07T02:35:50.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:50 smithi064 ceph-mon[34699]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:35:50.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:50 smithi064 ceph-mon[34699]: pgmap v123: 129 pgs: 129 active+clean; 67 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 254 KiB/s wr, 54 op/s 2024-05-07T02:35:50.382 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:50 smithi019 ceph-mon[29781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:35:50.382 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:50 smithi019 ceph-mon[29781]: pgmap v123: 129 pgs: 129 active+clean; 67 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 254 KiB/s wr, 54 op/s 2024-05-07T02:35:50.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:50 smithi027 ceph-mon[34398]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:35:50.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:50 smithi027 ceph-mon[34398]: pgmap v123: 129 pgs: 129 active+clean; 67 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 254 KiB/s wr, 54 op/s 2024-05-07T02:35:50.741 INFO:tasks.workunit.client.0.smithi019.stdout: 1 91932 2.13 MB/sec execute 46 sec latency 1116.378 ms 2024-05-07T02:35:50.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.805+0000 7fd4c5800700 1 -- 172.21.15.19:0/3725757281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4c0105be0 msgr2=0x7fd4c010e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:50.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.805+0000 7fd4c5800700 1 --2- 172.21.15.19:0/3725757281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4c0105be0 0x7fd4c010e130 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac0098f0 tx=0x7fd4ac009c00 comp rx=0 tx=0).stop 2024-05-07T02:35:50.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.806+0000 7fd4c5800700 1 -- 172.21.15.19:0/3725757281 shutdown_connections 2024-05-07T02:35:50.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.806+0000 7fd4c5800700 1 --2- 172.21.15.19:0/3725757281 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4c010e700 0x7fd4c0110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:50.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.806+0000 7fd4c5800700 1 --2- 172.21.15.19:0/3725757281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4c0105290 0x7fd4c01056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:50.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.806+0000 7fd4c5800700 1 --2- 172.21.15.19:0/3725757281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4c0105be0 0x7fd4c010e130 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:50.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.806+0000 7fd4c5800700 1 -- 172.21.15.19:0/3725757281 >> 172.21.15.19:0/3725757281 conn(0x7fd4c0078b50 msgr2=0x7fd4c0078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:35:50.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.806+0000 7fd4c5800700 1 -- 172.21.15.19:0/3725757281 shutdown_connections 2024-05-07T02:35:50.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.806+0000 7fd4c5800700 1 -- 172.21.15.19:0/3725757281 wait complete. 2024-05-07T02:35:50.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.807+0000 7fd4c5800700 1 Processor -- start 2024-05-07T02:35:50.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.808+0000 7fd4c5800700 1 -- start start 2024-05-07T02:35:50.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.808+0000 7fd4c5800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4c0105290 0x7fd4c019cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:50.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.808+0000 7fd4c5800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4c0105be0 0x7fd4c019d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:50.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.808+0000 7fd4c5800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4c010e700 0x7fd4c01a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:50.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.808+0000 7fd4c5800700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c01142b0 con 0x7fd4c0105290 2024-05-07T02:35:50.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.808+0000 7fd4c5800700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c0114130 con 0x7fd4c010e700 2024-05-07T02:35:50.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.808+0000 7fd4c5800700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c0114430 con 0x7fd4c0105be0 2024-05-07T02:35:50.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.809+0000 7fd4bfe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4c0105290 0x7fd4c019cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:50.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.809+0000 7fd4bfe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4c0105290 0x7fd4c019cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52262/0 (socket says 172.21.15.19:52262) 2024-05-07T02:35:50.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.809+0000 7fd4bfe00700 1 -- 172.21.15.19:0/3143228841 learned_addr learned my addr 172.21.15.19:0/3143228841 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:35:50.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.809+0000 7fd4c4e00700 1 --2- 172.21.15.19:0/3143228841 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4c010e700 0x7fd4c01a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:50.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.809+0000 7fd4bf400700 1 --2- 172.21.15.19:0/3143228841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4c0105be0 0x7fd4c019d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:50.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.809+0000 7fd4bf400700 1 -- 172.21.15.19:0/3143228841 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4c010e700 msgr2=0x7fd4c01a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:50.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.809+0000 7fd4bf400700 1 --2- 172.21.15.19:0/3143228841 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4c010e700 0x7fd4c01a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:50.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.809+0000 7fd4bf400700 1 -- 172.21.15.19:0/3143228841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4c0105290 msgr2=0x7fd4c019cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:50.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.809+0000 7fd4bf400700 1 --2- 172.21.15.19:0/3143228841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4c0105290 0x7fd4c019cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:50.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.809+0000 7fd4bf400700 1 -- 172.21.15.19:0/3143228841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4c01a1d70 con 0x7fd4c0105be0 2024-05-07T02:35:50.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.810+0000 7fd4bf400700 1 --2- 172.21.15.19:0/3143228841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4c0105be0 0x7fd4c019d440 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac00c010 tx=0x7fd4ac00fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:35:50.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.810+0000 7fd4bcc00700 1 -- 172.21.15.19:0/3143228841 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ac0050c0 con 0x7fd4c0105be0 2024-05-07T02:35:50.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.810+0000 7fd4bcc00700 1 -- 172.21.15.19:0/3143228841 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4ac00b560 con 0x7fd4c0105be0 2024-05-07T02:35:50.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.810+0000 7fd4c5800700 1 -- 172.21.15.19:0/3143228841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4c01a2000 con 0x7fd4c0105be0 2024-05-07T02:35:50.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.810+0000 7fd4bcc00700 1 -- 172.21.15.19:0/3143228841 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ac003a00 con 0x7fd4c0105be0 2024-05-07T02:35:50.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.810+0000 7fd4c5800700 1 -- 172.21.15.19:0/3143228841 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4c01a9980 con 0x7fd4c0105be0 2024-05-07T02:35:50.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.814+0000 7fd4bcc00700 1 -- 172.21.15.19:0/3143228841 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd4ac00b710 con 0x7fd4c0105be0 2024-05-07T02:35:50.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.816+0000 7fd4d5907b80 1 Processor -- start 2024-05-07T02:35:50.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.816+0000 7fd4d5907b80 1 -- start start 2024-05-07T02:35:50.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.817+0000 7fd4d5907b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562190348a90 0x562190348ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:50.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.817+0000 7fd4d5907b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5621903493e0 0x562190352620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:50.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.817+0000 7fd4d5907b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562190352b60 0x562190354f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:50.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.817+0000 7fd4bcc00700 1 --2- 172.21.15.19:0/3143228841 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fd4a40b2a70 0x7fd4a40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:50.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.817+0000 7fd4d5907b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56219026aa20 con 0x5621903493e0 2024-05-07T02:35:50.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.817+0000 7fd4d5907b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562190208760 con 0x562190352b60 2024-05-07T02:35:50.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.817+0000 7fd4d5907b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5621902669e0 con 0x562190348a90 2024-05-07T02:35:50.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.817+0000 7fd4bcc00700 1 -- 172.21.15.19:0/3143228841 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fd4ac0e3750 con 0x7fd4c0105be0 2024-05-07T02:35:50.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.817+0000 7fd4c4e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562190348a90 0x562190348ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:50.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.817+0000 7fd4c4e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562190348a90 0x562190348ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:47836/0 (socket says 172.21.15.19:47836) 2024-05-07T02:35:50.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.817+0000 7fd4c4e00700 1 -- 172.21.15.19:0/2365964059 learned_addr learned my addr 172.21.15.19:0/2365964059 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:35:50.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.817+0000 7fd4bfe00700 1 --2- 172.21.15.19:0/2365964059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5621903493e0 0x562190352620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:50.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.817+0000 7fd4c4e00700 1 --2- 172.21.15.19:0/3143228841 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fd4a40b2a70 0x7fd4a40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:50.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.817+0000 7fd4bf400700 1 --2- 172.21.15.19:0/2365964059 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562190352b60 0x562190354f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:50.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.818+0000 7fd4c4e00700 1 -- 172.21.15.19:0/2365964059 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562190352b60 msgr2=0x562190354f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:50.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.818+0000 7fd4c4e00700 1 --2- 172.21.15.19:0/2365964059 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562190352b60 0x562190354f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:50.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.818+0000 7fd4c4e00700 1 -- 172.21.15.19:0/2365964059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5621903493e0 msgr2=0x562190352620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:50.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.818+0000 7fd4c4e00700 1 --2- 172.21.15.19:0/2365964059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5621903493e0 0x562190352620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:50.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.818+0000 7fd4c4e00700 1 -- 172.21.15.19:0/2365964059 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621901b78d0 con 0x562190348a90 2024-05-07T02:35:50.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.818+0000 7fd4c4e00700 1 --2- 172.21.15.19:0/2365964059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562190348a90 0x562190348ea0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd4b80089f0 tx=0x7fd4b8008d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96d55de548bbfb1a server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:35:50.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.819+0000 7fd4c4e00700 1 --2- 172.21.15.19:0/3143228841 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fd4a40b2a70 0x7fd4a40b4f80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8008fd0 tx=0x7fd4b801e000 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:35:50.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.819+0000 7fd4b5600700 1 -- 172.21.15.19:0/2365964059 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b80094c0 con 0x562190348a90 2024-05-07T02:35:50.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.819+0000 7fd4b5600700 1 -- 172.21.15.19:0/2365964059 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4b801ab00 con 0x562190348a90 2024-05-07T02:35:50.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.819+0000 7fd4b5600700 1 -- 172.21.15.19:0/2365964059 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b8023a90 con 0x562190348a90 2024-05-07T02:35:50.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.820+0000 7fd4d5907b80 1 -- 172.21.15.19:0/2365964059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562190348a90 msgr2=0x562190348ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:50.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.820+0000 7fd4d5907b80 1 --2- 172.21.15.19:0/2365964059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562190348a90 0x562190348ea0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd4b80089f0 tx=0x7fd4b8008d00 comp rx=0 tx=0).stop 2024-05-07T02:35:50.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.820+0000 7fd4d5907b80 1 -- 172.21.15.19:0/2365964059 shutdown_connections 2024-05-07T02:35:50.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.820+0000 7fd4d5907b80 1 --2- 172.21.15.19:0/2365964059 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562190352b60 0x562190354f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:50.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.820+0000 7fd4d5907b80 1 --2- 172.21.15.19:0/2365964059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562190348a90 0x562190348ea0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:50.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.820+0000 7fd4d5907b80 1 --2- 172.21.15.19:0/2365964059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5621903493e0 0x562190352620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:50.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.820+0000 7fd4d5907b80 1 -- 172.21.15.19:0/2365964059 >> 172.21.15.19:0/2365964059 conn(0x5621901cae40 msgr2=0x562190343880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:35:50.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.820+0000 7fd4d5907b80 1 -- 172.21.15.19:0/2365964059 shutdown_connections 2024-05-07T02:35:50.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.820+0000 7fd4d5907b80 1 -- 172.21.15.19:0/2365964059 wait complete. 2024-05-07T02:35:50.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.821+0000 7fd4d5907b80 1 -- start start 2024-05-07T02:35:50.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.821+0000 7fd4d5907b80 1 Processor -- start 2024-05-07T02:35:50.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.821+0000 7fd4d5907b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562190348a90 0x56219034fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:50.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.822+0000 7fd4d5907b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5621903493e0 0x5621903500b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:50.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.822+0000 7fd4d5907b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562190352b60 0x5621903505f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:50.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.822+0000 7fd4d5907b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5621903558a0 con 0x5621903493e0 2024-05-07T02:35:50.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.822+0000 7fd4d5907b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5621901d0b20 con 0x562190348a90 2024-05-07T02:35:50.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.822+0000 7fd4d5907b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562190355a20 con 0x562190352b60 2024-05-07T02:35:50.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.822+0000 7fd4d5907b80 10 client.0 _ll_register_callbacks cb 0x562190211c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:35:50.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.822+0000 7fd4bfe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562190348a90 0x56219034fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:50.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.822+0000 7fd4bf400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5621903493e0 0x5621903500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:50.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.822+0000 7fd4c4e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562190352b60 0x5621903505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:50.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.822+0000 7fd4bfe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562190348a90 0x56219034fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:48984/0 (socket says 172.21.15.19:48984) 2024-05-07T02:35:50.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.822+0000 7fd4bfe00700 1 -- 172.21.15.19:0/715775887 learned_addr learned my addr 172.21.15.19:0/715775887 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:35:50.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.822+0000 7fd4c4e00700 1 -- 172.21.15.19:0/715775887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562190348a90 msgr2=0x56219034fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:50.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.822+0000 7fd4c4e00700 1 --2- 172.21.15.19:0/715775887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562190348a90 0x56219034fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:50.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.822+0000 7fd4c4e00700 1 -- 172.21.15.19:0/715775887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5621903493e0 msgr2=0x5621903500b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:50.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.822+0000 7fd4c4e00700 1 --2- 172.21.15.19:0/715775887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5621903493e0 0x5621903500b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:50.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.822+0000 7fd4c4e00700 1 -- 172.21.15.19:0/715775887 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562190230420 con 0x562190352b60 2024-05-07T02:35:50.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.822+0000 7fd4bf400700 1 --2- 172.21.15.19:0/715775887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5621903493e0 0x5621903500b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:35:50.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.823+0000 7fd4c4e00700 1 --2- 172.21.15.19:0/715775887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562190352b60 0x5621903505f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8025040 tx=0x7fd4b801aa50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b40f348f544d65de server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:35:50.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.823+0000 7fd4aa000700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:35:50.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.823+0000 7fd4aa000700 1 -- 172.21.15.19:0/715775887 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b8033070 con 0x562190352b60 2024-05-07T02:35:50.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.823+0000 7fd4aa000700 1 -- 172.21.15.19:0/715775887 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fd4b8004c50 con 0x562190352b60 2024-05-07T02:35:50.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.823+0000 7fd4d5907b80 1 -- 172.21.15.19:0/715775887 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4b801ab00 con 0x562190352b60 2024-05-07T02:35:50.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.823+0000 7fd4aa000700 1 -- 172.21.15.19:0/715775887 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd4b803f050 con 0x562190352b60 2024-05-07T02:35:50.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.823+0000 7fd4d5907b80 10 client.25876 fetch_fsmap learned FSMap version 39 2024-05-07T02:35:50.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.823+0000 7fd4d5907b80 1 -- 172.21.15.19:0/715775887 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562190360500 con 0x562190352b60 2024-05-07T02:35:50.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.824+0000 7fd4aa000700 1 -- 172.21.15.19:0/715775887 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4b801ab00 con 0x562190352b60 2024-05-07T02:35:50.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.824+0000 7fd4aa000700 1 -- 172.21.15.19:0/715775887 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 39) ==== 8374+0+0 (secure 0 0 0) 0x7fd4b802c5f0 con 0x562190352b60 2024-05-07T02:35:50.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.824+0000 7fd4d5907b80 10 client.25876 fetch_fsmap finished waiting for FSMap version 39 2024-05-07T02:35:50.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.824+0000 7fd4d5907b80 10 client.25876 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-07T02:35:50.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.824+0000 7fd4d5907b80 20 client.25876 populate_metadata read hostname 'smithi019' 2024-05-07T02:35:50.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.824+0000 7fd4d5907b80 1 --2- 172.21.15.19:0/715775887 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x562190362a90 0x562190364eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:50.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.824+0000 7fd4d5907b80 4 client.25876 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:35:50.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.824+0000 7fd4d5907b80 1 -- 172.21.15.19:0/715775887 --> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621903152e0 con 0x562190362a90 2024-05-07T02:35:50.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.824+0000 7fd4bfe00700 1 --2- 172.21.15.19:0/715775887 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x562190362a90 0x562190364eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:50.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.825+0000 7fd4bfe00700 1 --2- 172.21.15.19:0/715775887 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x562190362a90 0x562190364eb0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5e8d12258511d6ec server_cookie=38999d6c77f6c438 in_seq=0 out_seq=0 2024-05-07T02:35:50.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.825+0000 7fd4aa000700 10 client.25876 ms_handle_connect on v2:172.21.15.64:6836/2585907777 2024-05-07T02:35:50.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.826+0000 7fd4aa000700 1 -- 172.21.15.19:0/715775887 <== mds.4 v2:172.21.15.64:6836/2585907777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5621903152e0 con 0x562190362a90 2024-05-07T02:35:50.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.826+0000 7fd4aa000700 10 client.25876 handle_command_reply: tid=0 2024-05-07T02:35:50.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.826+0000 7fd4d5907b80 1 client.25876 shutdown 2024-05-07T02:35:50.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.826+0000 7fd4d5907b80 1 -- 172.21.15.19:0/715775887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562190352b60 msgr2=0x5621903505f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:50.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.826+0000 7fd4d5907b80 1 --2- 172.21.15.19:0/715775887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562190352b60 0x5621903505f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8025040 tx=0x7fd4b801aa50 comp rx=0 tx=0).stop 2024-05-07T02:35:50.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.826+0000 7fd4d5907b80 1 -- 172.21.15.19:0/715775887 shutdown_connections 2024-05-07T02:35:50.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.826+0000 7fd4d5907b80 1 --2- 172.21.15.19:0/715775887 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x562190362a90 0x562190364eb0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:50.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.826+0000 7fd4d5907b80 1 --2- 172.21.15.19:0/715775887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562190352b60 0x5621903505f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:50.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.826+0000 7fd4d5907b80 1 --2- 172.21.15.19:0/715775887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562190348a90 0x56219034fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:50.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.826+0000 7fd4d5907b80 1 --2- 172.21.15.19:0/715775887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5621903493e0 0x5621903500b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:50.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.826+0000 7fd4aa000700 0 client.25876 ms_handle_reset on v2:172.21.15.64:6836/2585907777 2024-05-07T02:35:50.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.826+0000 7fd4d5907b80 1 -- 172.21.15.19:0/715775887 >> 172.21.15.19:0/715775887 conn(0x5621901cae40 msgr2=0x562190345450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:35:50.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.826+0000 7fd4d5907b80 1 -- 172.21.15.19:0/715775887 shutdown_connections 2024-05-07T02:35:50.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.826+0000 7fd4d5907b80 1 -- 172.21.15.19:0/715775887 wait complete. 2024-05-07T02:35:50.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.827+0000 7fd4d5907b80 20 client.25876 trim_cache size 0 max 16384 2024-05-07T02:35:50.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.843+0000 7fd4d5907b80 1 Processor -- start 2024-05-07T02:35:50.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.844+0000 7fd4d5907b80 1 -- start start 2024-05-07T02:35:50.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.844+0000 7fd4d5907b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562190348a90 0x562190351590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:50.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.844+0000 7fd4d5907b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5621903493e0 0x56219035ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:50.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.844+0000 7fd4d5907b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562190352b60 0x562190347b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:50.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.844+0000 7fd4d5907b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c01b2540 con 0x5621903493e0 2024-05-07T02:35:50.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.844+0000 7fd4d5907b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c01b4110 con 0x562190352b60 2024-05-07T02:35:50.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.844+0000 7fd4d5907b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c01b2c10 con 0x562190348a90 2024-05-07T02:35:50.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.845+0000 7fd4bf400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5621903493e0 0x56219035ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:50.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.845+0000 7fd4bfe00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562190348a90 0x562190351590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:50.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.845+0000 7fd4c4e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562190352b60 0x562190347b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:50.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.845+0000 7fd4bfe00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562190348a90 0x562190351590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:47850/0 (socket says 172.21.15.19:47850) 2024-05-07T02:35:50.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.845+0000 7fd4bfe00700 1 -- 172.21.15.19:0/2062583696 learned_addr learned my addr 172.21.15.19:0/2062583696 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:35:50.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.845+0000 7fd4c4e00700 1 -- 172.21.15.19:0/2062583696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562190348a90 msgr2=0x562190351590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:50.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.845+0000 7fd4c4e00700 1 --2- 172.21.15.19:0/2062583696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562190348a90 0x562190351590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:50.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.845+0000 7fd4c4e00700 1 -- 172.21.15.19:0/2062583696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5621903493e0 msgr2=0x56219035ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:50.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.845+0000 7fd4c4e00700 1 --2- 172.21.15.19:0/2062583696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5621903493e0 0x56219035ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:50.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.845+0000 7fd4c4e00700 1 -- 172.21.15.19:0/2062583696 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562190343c00 con 0x562190352b60 2024-05-07T02:35:50.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.845+0000 7fd4bfe00700 1 --2- 172.21.15.19:0/2062583696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562190348a90 0x562190351590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:35:50.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.845+0000 7fd4c4e00700 1 --2- 172.21.15.19:0/2062583696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562190352b60 0x562190347b70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8009e60 tx=0x7fd4b8006160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf1d4269b77441de server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:35:50.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.846+0000 7fd4b4c00700 1 -- 172.21.15.19:0/2062583696 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b803c070 con 0x562190352b60 2024-05-07T02:35:50.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.846+0000 7fd4b4c00700 1 -- 172.21.15.19:0/2062583696 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4b802ce30 con 0x562190352b60 2024-05-07T02:35:50.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.846+0000 7fd4b4c00700 1 -- 172.21.15.19:0/2062583696 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b8033040 con 0x562190352b60 2024-05-07T02:35:50.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.847+0000 7fd4d5907b80 1 -- 172.21.15.19:0/2062583696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562190352b60 msgr2=0x562190347b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:50.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.847+0000 7fd4d5907b80 1 --2- 172.21.15.19:0/2062583696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562190352b60 0x562190347b70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8009e60 tx=0x7fd4b8006160 comp rx=0 tx=0).stop 2024-05-07T02:35:50.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.847+0000 7fd4d5907b80 1 -- 172.21.15.19:0/2062583696 shutdown_connections 2024-05-07T02:35:50.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.847+0000 7fd4d5907b80 1 --2- 172.21.15.19:0/2062583696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562190352b60 0x562190347b70 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:50.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.847+0000 7fd4d5907b80 1 --2- 172.21.15.19:0/2062583696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562190348a90 0x562190351590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:50.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.847+0000 7fd4d5907b80 1 --2- 172.21.15.19:0/2062583696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5621903493e0 0x56219035ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:50.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.847+0000 7fd4d5907b80 1 -- 172.21.15.19:0/2062583696 >> 172.21.15.19:0/2062583696 conn(0x5621901cae40 msgr2=0x562190347460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:35:50.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.847+0000 7fd4d5907b80 1 -- 172.21.15.19:0/2062583696 shutdown_connections 2024-05-07T02:35:50.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.847+0000 7fd4d5907b80 1 -- 172.21.15.19:0/2062583696 wait complete. 2024-05-07T02:35:50.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.847+0000 7fd4d5907b80 1 -- start start 2024-05-07T02:35:50.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.848+0000 7fd4d5907b80 1 Processor -- start 2024-05-07T02:35:50.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.848+0000 7fd4d5907b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562190348a90 0x562190367210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:50.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.848+0000 7fd4d5907b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621903493e0 0x562190367750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:50.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.848+0000 7fd4d5907b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562190352b60 0x5621903cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:50.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.848+0000 7fd4d5907b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c01b3a10 con 0x562190352b60 2024-05-07T02:35:50.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.848+0000 7fd4d5907b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c01b3340 con 0x562190348a90 2024-05-07T02:35:50.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.848+0000 7fd4d5907b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c01b04e0 con 0x5621903493e0 2024-05-07T02:35:50.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.848+0000 7fd4d5907b80 10 client.0 _ll_register_callbacks cb 0x562190211c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:35:50.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.848+0000 7fd4c4e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562190352b60 0x5621903cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:50.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.849+0000 7fd4bfe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562190348a90 0x562190367210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:50.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.849+0000 7fd4c4e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562190352b60 0x5621903cafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52292/0 (socket says 172.21.15.19:52292) 2024-05-07T02:35:50.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.849+0000 7fd4bfe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562190348a90 0x562190367210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:49002/0 (socket says 172.21.15.19:49002) 2024-05-07T02:35:50.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.849+0000 7fd4c4e00700 1 -- 172.21.15.19:0/3418424507 learned_addr learned my addr 172.21.15.19:0/3418424507 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:35:50.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.849+0000 7fd4bf400700 1 --2- 172.21.15.19:0/3418424507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621903493e0 0x562190367750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:50.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.849+0000 7fd4bfe00700 1 -- 172.21.15.19:0/3418424507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621903493e0 msgr2=0x562190367750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:50.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.849+0000 7fd4bfe00700 1 --2- 172.21.15.19:0/3418424507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621903493e0 0x562190367750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:50.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.849+0000 7fd4bfe00700 1 -- 172.21.15.19:0/3418424507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562190352b60 msgr2=0x5621903cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:50.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.849+0000 7fd4bfe00700 1 --2- 172.21.15.19:0/3418424507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562190352b60 0x5621903cafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:50.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.849+0000 7fd4bfe00700 1 -- 172.21.15.19:0/3418424507 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621903cb520 con 0x562190348a90 2024-05-07T02:35:50.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.849+0000 7fd4c4e00700 1 --2- 172.21.15.19:0/3418424507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562190352b60 0x5621903cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:35:50.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.849+0000 7fd4bf400700 1 --2- 172.21.15.19:0/3418424507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621903493e0 0x562190367750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:35:50.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.849+0000 7fd4bfe00700 1 --2- 172.21.15.19:0/3418424507 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562190348a90 0x562190367210 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd4b0004790 tx=0x7fd4b0012a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25f9ebc25de31654 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:35:50.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.849+0000 7fd4aa000700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:35:50.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.850+0000 7fd4aa000700 1 -- 172.21.15.19:0/3418424507 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b0004a30 con 0x562190348a90 2024-05-07T02:35:50.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.850+0000 7fd4d5907b80 1 -- 172.21.15.19:0/3418424507 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4b802ce30 con 0x562190348a90 2024-05-07T02:35:50.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.850+0000 7fd4aa000700 1 -- 172.21.15.19:0/3418424507 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fd4b0004bd0 con 0x562190348a90 2024-05-07T02:35:50.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.850+0000 7fd4aa000700 1 -- 172.21.15.19:0/3418424507 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd4b0031050 con 0x562190348a90 2024-05-07T02:35:50.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.850+0000 7fd4d5907b80 10 client.25904 fetch_fsmap learned FSMap version 39 2024-05-07T02:35:50.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.850+0000 7fd4d5907b80 1 -- 172.21.15.19:0/3418424507 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5621903cc7b0 con 0x562190348a90 2024-05-07T02:35:50.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.850+0000 7fd4aa000700 1 -- 172.21.15.19:0/3418424507 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4b802ce30 con 0x562190348a90 2024-05-07T02:35:50.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.851+0000 7fd4aa000700 1 -- 172.21.15.19:0/3418424507 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 39) ==== 8374+0+0 (secure 0 0 0) 0x7fd4b00221e0 con 0x562190348a90 2024-05-07T02:35:50.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.851+0000 7fd4d5907b80 10 client.25904 fetch_fsmap finished waiting for FSMap version 39 2024-05-07T02:35:50.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.851+0000 7fd4d5907b80 10 client.25904 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-07T02:35:50.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.851+0000 7fd4d5907b80 20 client.25904 populate_metadata read hostname 'smithi019' 2024-05-07T02:35:50.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.851+0000 7fd4d5907b80 1 --2- 172.21.15.19:0/3418424507 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x5621903ccec0 0x5621903cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:50.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.851+0000 7fd4d5907b80 4 client.25904 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:35:50.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.851+0000 7fd4d5907b80 1 -- 172.21.15.19:0/3418424507 --> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562190278400 con 0x5621903ccec0 2024-05-07T02:35:50.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.851+0000 7fd4bf400700 1 --2- 172.21.15.19:0/3418424507 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x5621903ccec0 0x5621903cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:50.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.852+0000 7fd4bf400700 1 --2- 172.21.15.19:0/3418424507 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x5621903ccec0 0x5621903cf2e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ad21d085d97d9784 server_cookie=7406f1ee17d0780e in_seq=0 out_seq=0 2024-05-07T02:35:50.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.852+0000 7fd4aa000700 10 client.25904 ms_handle_connect on v2:172.21.15.64:6836/2585907777 2024-05-07T02:35:50.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.852+0000 7fd4aa000700 1 -- 172.21.15.19:0/3418424507 <== mds.4 v2:172.21.15.64:6836/2585907777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562190278400 con 0x5621903ccec0 2024-05-07T02:35:50.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.852+0000 7fd4aa000700 10 client.25904 handle_command_reply: tid=0 2024-05-07T02:35:50.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.852+0000 7fd4d5907b80 1 client.25904 shutdown 2024-05-07T02:35:50.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.852+0000 7fd4d5907b80 1 -- 172.21.15.19:0/3418424507 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562190348a90 msgr2=0x562190367210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:50.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.852+0000 7fd4d5907b80 1 --2- 172.21.15.19:0/3418424507 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562190348a90 0x562190367210 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd4b0004790 tx=0x7fd4b0012a40 comp rx=0 tx=0).stop 2024-05-07T02:35:50.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.852+0000 7fd4d5907b80 1 -- 172.21.15.19:0/3418424507 shutdown_connections 2024-05-07T02:35:50.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.852+0000 7fd4d5907b80 1 --2- 172.21.15.19:0/3418424507 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x5621903ccec0 0x5621903cf2e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:50.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.852+0000 7fd4d5907b80 1 --2- 172.21.15.19:0/3418424507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562190352b60 0x5621903cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:50.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.852+0000 7fd4d5907b80 1 --2- 172.21.15.19:0/3418424507 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562190348a90 0x562190367210 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:50.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.852+0000 7fd4aa000700 0 client.25904 ms_handle_reset on v2:172.21.15.64:6836/2585907777 2024-05-07T02:35:50.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.852+0000 7fd4d5907b80 1 --2- 172.21.15.19:0/3418424507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5621903493e0 0x562190367750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:50.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.852+0000 7fd4d5907b80 1 -- 172.21.15.19:0/3418424507 >> 172.21.15.19:0/3418424507 conn(0x5621901cae40 msgr2=0x562190342bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:35:50.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.852+0000 7fd4d5907b80 1 -- 172.21.15.19:0/3418424507 shutdown_connections 2024-05-07T02:35:50.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.852+0000 7fd4d5907b80 1 -- 172.21.15.19:0/3418424507 wait complete. 2024-05-07T02:35:50.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.853+0000 7fd4d5907b80 20 client.25904 trim_cache size 0 max 16384 2024-05-07T02:35:50.856 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:35:50.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.853+0000 7fd4c5800700 1 -- 172.21.15.19:0/3143228841 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fd4a40b2a70 msgr2=0x7fd4a40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:50.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.853+0000 7fd4c5800700 1 --2- 172.21.15.19:0/3143228841 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fd4a40b2a70 0x7fd4a40b4f80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8008fd0 tx=0x7fd4b801e000 comp rx=0 tx=0).stop 2024-05-07T02:35:50.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.853+0000 7fd4c5800700 1 -- 172.21.15.19:0/3143228841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4c0105be0 msgr2=0x7fd4c019d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:50.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.853+0000 7fd4c5800700 1 --2- 172.21.15.19:0/3143228841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4c0105be0 0x7fd4c019d440 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac00c010 tx=0x7fd4ac00fa00 comp rx=0 tx=0).stop 2024-05-07T02:35:50.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.853+0000 7fd4c5800700 1 -- 172.21.15.19:0/3143228841 shutdown_connections 2024-05-07T02:35:50.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.853+0000 7fd4c5800700 1 --2- 172.21.15.19:0/3143228841 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4c010e700 0x7fd4c01a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:50.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.853+0000 7fd4c5800700 1 --2- 172.21.15.19:0/3143228841 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fd4a40b2a70 0x7fd4a40b4f80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:50.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.853+0000 7fd4c5800700 1 --2- 172.21.15.19:0/3143228841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4c0105290 0x7fd4c019cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:50.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.853+0000 7fd4c5800700 1 --2- 172.21.15.19:0/3143228841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd4c0105be0 0x7fd4c019d440 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:50.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.854+0000 7fd4c5800700 1 -- 172.21.15.19:0/3143228841 >> 172.21.15.19:0/3143228841 conn(0x7fd4c0078b50 msgr2=0x7fd4c0101c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:35:50.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.854+0000 7fd4c5800700 1 -- 172.21.15.19:0/3143228841 shutdown_connections 2024-05-07T02:35:50.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:50.854+0000 7fd4c5800700 1 -- 172.21.15.19:0/3143228841 wait complete. 2024-05-07T02:35:51.741 INFO:tasks.workunit.client.0.smithi019.stdout: 1 91932 2.08 MB/sec execute 47 sec latency 2116.465 ms 2024-05-07T02:35:52.055 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-07T02:35:52.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:52 smithi064 ceph-mon[34699]: pgmap v124: 129 pgs: 129 active+clean; 78 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1.6 MiB/s wr, 100 op/s 2024-05-07T02:35:52.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:52 smithi019 ceph-mon[29781]: pgmap v124: 129 pgs: 129 active+clean; 78 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1.6 MiB/s wr, 100 op/s 2024-05-07T02:35:52.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:52 smithi027 ceph-mon[34398]: pgmap v124: 129 pgs: 129 active+clean; 78 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1.6 MiB/s wr, 100 op/s 2024-05-07T02:35:52.512 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:35:52.741 INFO:tasks.workunit.client.0.smithi019.stdout: 1 91932 2.04 MB/sec execute 48 sec latency 3116.552 ms 2024-05-07T02:35:53.741 INFO:tasks.workunit.client.0.smithi019.stdout: 1 91932 2.00 MB/sec execute 49 sec latency 4116.661 ms 2024-05-07T02:35:54.741 INFO:tasks.workunit.client.0.smithi019.stdout: 1 92100 1.98 MB/sec execute 50 sec latency 4969.450 ms 2024-05-07T02:35:54.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.978+0000 7fa2e9400700 1 -- 172.21.15.19:0/2717650032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2e4105e10 msgr2=0x7fa2e410e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:54.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.978+0000 7fa2e9400700 1 --2- 172.21.15.19:0/2717650032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2e4105e10 0x7fa2e410e360 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0009990 tx=0x7fa2d0009ca0 comp rx=0 tx=0).stop 2024-05-07T02:35:54.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.979+0000 7fa2e9400700 1 -- 172.21.15.19:0/2717650032 shutdown_connections 2024-05-07T02:35:54.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.979+0000 7fa2e9400700 1 --2- 172.21.15.19:0/2717650032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2e410e8f0 0x7fa2e4110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:54.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.979+0000 7fa2e9400700 1 --2- 172.21.15.19:0/2717650032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2e41054c0 0x7fa2e41058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:54.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.979+0000 7fa2e9400700 1 --2- 172.21.15.19:0/2717650032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2e4105e10 0x7fa2e410e360 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:54.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.979+0000 7fa2e9400700 1 -- 172.21.15.19:0/2717650032 >> 172.21.15.19:0/2717650032 conn(0x7fa2e40fedc0 msgr2=0x7fa2e41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:35:54.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.979+0000 7fa2e9400700 1 -- 172.21.15.19:0/2717650032 shutdown_connections 2024-05-07T02:35:54.980 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:54 smithi019 ceph-mon[29781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:35:54.980 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:54 smithi019 ceph-mon[29781]: pgmap v125: 129 pgs: 129 active+clean; 78 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 1.4 MiB/s wr, 56 op/s 2024-05-07T02:35:54.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.979+0000 7fa2e9400700 1 -- 172.21.15.19:0/2717650032 wait complete. 2024-05-07T02:35:54.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.980+0000 7fa2e9400700 1 Processor -- start 2024-05-07T02:35:54.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.981+0000 7fa2e9400700 1 -- start start 2024-05-07T02:35:54.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.981+0000 7fa2e9400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2e41054c0 0x7fa2e419d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:54.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.981+0000 7fa2e9400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2e4105e10 0x7fa2e419d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:54.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.981+0000 7fa2e9400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2e410e8f0 0x7fa2e41a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:54.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.981+0000 7fa2e9400700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e41144c0 con 0x7fa2e410e8f0 2024-05-07T02:35:54.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.981+0000 7fa2e9400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e4114340 con 0x7fa2e4105e10 2024-05-07T02:35:54.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.981+0000 7fa2e9400700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e4114640 con 0x7fa2e41054c0 2024-05-07T02:35:54.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.981+0000 7fa2e1e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2e4105e10 0x7fa2e419d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:54.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.981+0000 7fa2e1e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2e4105e10 0x7fa2e419d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:49012/0 (socket says 172.21.15.19:49012) 2024-05-07T02:35:54.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.981+0000 7fa2e1e00700 1 -- 172.21.15.19:0/3813686996 learned_addr learned my addr 172.21.15.19:0/3813686996 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:35:54.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.981+0000 7fa2e2800700 1 --2- 172.21.15.19:0/3813686996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2e41054c0 0x7fa2e419d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:54.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.982+0000 7fa2e3200700 1 --2- 172.21.15.19:0/3813686996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2e410e8f0 0x7fa2e41a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:54.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.982+0000 7fa2e1e00700 1 -- 172.21.15.19:0/3813686996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2e41054c0 msgr2=0x7fa2e419d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:54.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.982+0000 7fa2e1e00700 1 --2- 172.21.15.19:0/3813686996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2e41054c0 0x7fa2e419d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:54.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.982+0000 7fa2e1e00700 1 -- 172.21.15.19:0/3813686996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2e410e8f0 msgr2=0x7fa2e41a1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:54.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.982+0000 7fa2e1e00700 1 --2- 172.21.15.19:0/3813686996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2e410e8f0 0x7fa2e41a1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:54.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.982+0000 7fa2e1e00700 1 -- 172.21.15.19:0/3813686996 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2e41a1ee0 con 0x7fa2e4105e10 2024-05-07T02:35:54.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.982+0000 7fa2e3200700 1 --2- 172.21.15.19:0/3813686996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2e410e8f0 0x7fa2e41a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:35:54.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.982+0000 7fa2e2800700 1 --2- 172.21.15.19:0/3813686996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2e41054c0 0x7fa2e419d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:35:54.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.982+0000 7fa2e1e00700 1 --2- 172.21.15.19:0/3813686996 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2e4105e10 0x7fa2e419d5b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0005910 tx=0x7fa2d000b670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:35:54.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.983+0000 7fa2db400700 1 -- 172.21.15.19:0/3813686996 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0027070 con 0x7fa2e4105e10 2024-05-07T02:35:54.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.983+0000 7fa2db400700 1 -- 172.21.15.19:0/3813686996 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2d0022470 con 0x7fa2e4105e10 2024-05-07T02:35:54.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.983+0000 7fa2db400700 1 -- 172.21.15.19:0/3813686996 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0017650 con 0x7fa2e4105e10 2024-05-07T02:35:54.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.983+0000 7fa2e9400700 1 -- 172.21.15.19:0/3813686996 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2e41a2170 con 0x7fa2e4105e10 2024-05-07T02:35:54.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.983+0000 7fa2e9400700 1 -- 172.21.15.19:0/3813686996 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2e41a2600 con 0x7fa2e4105e10 2024-05-07T02:35:54.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.989+0000 7fa2db400700 1 -- 172.21.15.19:0/3813686996 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa2d0009d10 con 0x7fa2e4105e10 2024-05-07T02:35:54.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.990+0000 7fa2f7f0fb80 1 Processor -- start 2024-05-07T02:35:54.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.990+0000 7fa2f7f0fb80 1 -- start start 2024-05-07T02:35:54.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.991+0000 7fa2f7f0fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c4927470 0x55a8c4927880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:54.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.991+0000 7fa2f7f0fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c4921620 0x55a8c4921a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:54.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.991+0000 7fa2f7f0fb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c4921f70 0x55a8c492e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:54.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.991+0000 7fa2f7f0fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a8c4843a20 con 0x55a8c4927470 2024-05-07T02:35:54.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.991+0000 7fa2f7f0fb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a8c47e1760 con 0x55a8c4921f70 2024-05-07T02:35:54.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.991+0000 7fa2f7f0fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a8c483f9e0 con 0x55a8c4921620 2024-05-07T02:35:54.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.991+0000 7fa2db400700 1 --2- 172.21.15.19:0/3813686996 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fa2c80b2a70 0x7fa2c80b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:54.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.991+0000 7fa2e1e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c4921f70 0x55a8c492e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:54.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.991+0000 7fa2e1e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c4921f70 0x55a8c492e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:49020/0 (socket says 172.21.15.19:49020) 2024-05-07T02:35:54.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.991+0000 7fa2e1e00700 1 -- 172.21.15.19:0/1253395646 learned_addr learned my addr 172.21.15.19:0/1253395646 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:35:54.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.991+0000 7fa2e3200700 1 --2- 172.21.15.19:0/1253395646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c4927470 0x55a8c4927880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:54.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.991+0000 7fa2e2800700 1 --2- 172.21.15.19:0/1253395646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c4921620 0x55a8c4921a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:54.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.991+0000 7fa2e3200700 1 --2- 172.21.15.19:0/3813686996 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fa2c80b2a70 0x7fa2c80b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:54.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.991+0000 7fa2db400700 1 -- 172.21.15.19:0/3813686996 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fa2d00e3810 con 0x7fa2e4105e10 2024-05-07T02:35:54.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.992+0000 7fa2e1e00700 1 -- 172.21.15.19:0/1253395646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c4921620 msgr2=0x55a8c4921a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:54.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.992+0000 7fa2e1e00700 1 --2- 172.21.15.19:0/1253395646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c4921620 0x55a8c4921a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:54.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.992+0000 7fa2e1e00700 1 -- 172.21.15.19:0/1253395646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c4927470 msgr2=0x55a8c4927880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:54.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.992+0000 7fa2e1e00700 1 --2- 172.21.15.19:0/1253395646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c4927470 0x55a8c4927880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:54.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.992+0000 7fa2e1e00700 1 -- 172.21.15.19:0/1253395646 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8c47908d0 con 0x55a8c4921f70 2024-05-07T02:35:54.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.992+0000 7fa2e1e00700 1 --2- 172.21.15.19:0/1253395646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c4921f70 0x55a8c492e150 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa2d00a6780 tx=0x7fa2d0009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70cbb6c3596a9af5 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:35:54.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.992+0000 7fa2cfe00700 1 -- 172.21.15.19:0/1253395646 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d00aa210 con 0x55a8c4921f70 2024-05-07T02:35:54.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.992+0000 7fa2cfe00700 1 -- 172.21.15.19:0/1253395646 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2d00042a0 con 0x55a8c4921f70 2024-05-07T02:35:54.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.992+0000 7fa2cfe00700 1 -- 172.21.15.19:0/1253395646 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d00b3560 con 0x55a8c4921f70 2024-05-07T02:35:54.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.993+0000 7fa2f7f0fb80 1 -- 172.21.15.19:0/1253395646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c4921f70 msgr2=0x55a8c492e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:54.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.993+0000 7fa2f7f0fb80 1 --2- 172.21.15.19:0/1253395646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c4921f70 0x55a8c492e150 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa2d00a6780 tx=0x7fa2d0009310 comp rx=0 tx=0).stop 2024-05-07T02:35:54.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.993+0000 7fa2e3200700 1 --2- 172.21.15.19:0/3813686996 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fa2c80b2a70 0x7fa2c80b4f80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc009710 tx=0x7fa2dc009680 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:35:54.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.994+0000 7fa2f7f0fb80 1 -- 172.21.15.19:0/1253395646 shutdown_connections 2024-05-07T02:35:54.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.994+0000 7fa2f7f0fb80 1 --2- 172.21.15.19:0/1253395646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c4921f70 0x55a8c492e150 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:54.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.994+0000 7fa2f7f0fb80 1 --2- 172.21.15.19:0/1253395646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c4927470 0x55a8c4927880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:54.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.994+0000 7fa2f7f0fb80 1 --2- 172.21.15.19:0/1253395646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c4921620 0x55a8c4921a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:54.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.994+0000 7fa2f7f0fb80 1 -- 172.21.15.19:0/1253395646 >> 172.21.15.19:0/1253395646 conn(0x55a8c47a3e40 msgr2=0x55a8c491c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:35:54.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.994+0000 7fa2f7f0fb80 1 -- 172.21.15.19:0/1253395646 shutdown_connections 2024-05-07T02:35:54.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.994+0000 7fa2f7f0fb80 1 -- 172.21.15.19:0/1253395646 wait complete. 2024-05-07T02:35:54.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.994+0000 7fa2f7f0fb80 1 -- start start 2024-05-07T02:35:54.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.995+0000 7fa2f7f0fb80 1 Processor -- start 2024-05-07T02:35:54.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.995+0000 7fa2f7f0fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c4921620 0x55a8c492b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:54.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.995+0000 7fa2f7f0fb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c4921f70 0x55a8c492bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:54.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.995+0000 7fa2f7f0fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c4927470 0x55a8c492c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:54.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.995+0000 7fa2f7f0fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a8c492e690 con 0x55a8c4921620 2024-05-07T02:35:54.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.995+0000 7fa2f7f0fb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a8c47a9b20 con 0x55a8c4921f70 2024-05-07T02:35:54.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.995+0000 7fa2f7f0fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a8c492e810 con 0x55a8c4927470 2024-05-07T02:35:54.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.995+0000 7fa2f7f0fb80 10 client.0 _ll_register_callbacks cb 0x55a8c47d7380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:35:54.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.995+0000 7fa2e3200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c4927470 0x55a8c492c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:54.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.995+0000 7fa2e2800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c4921620 0x55a8c492b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:54.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.995+0000 7fa2e3200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c4927470 0x55a8c492c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:47896/0 (socket says 172.21.15.19:47896) 2024-05-07T02:35:54.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.996+0000 7fa2e3200700 1 -- 172.21.15.19:0/1383934156 learned_addr learned my addr 172.21.15.19:0/1383934156 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:35:54.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.996+0000 7fa2e1e00700 1 --2- 172.21.15.19:0/1383934156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c4921f70 0x55a8c492bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:54.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.996+0000 7fa2e3200700 1 -- 172.21.15.19:0/1383934156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c4921f70 msgr2=0x55a8c492bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:54.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.996+0000 7fa2e3200700 1 --2- 172.21.15.19:0/1383934156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c4921f70 0x55a8c492bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:54.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.996+0000 7fa2e3200700 1 -- 172.21.15.19:0/1383934156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c4921620 msgr2=0x55a8c492b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:54.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.996+0000 7fa2e3200700 1 --2- 172.21.15.19:0/1383934156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c4921620 0x55a8c492b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:54.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.996+0000 7fa2e3200700 1 -- 172.21.15.19:0/1383934156 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8c4809420 con 0x55a8c4927470 2024-05-07T02:35:54.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.996+0000 7fa2ccc00700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:35:54.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.996+0000 7fa2e3200700 1 --2- 172.21.15.19:0/1383934156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c4927470 0x55a8c492c060 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc0199b0 tx=0x7fa2dc004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a89f1d64aa0ba6b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:35:54.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.997+0000 7fa2ccc00700 1 -- 172.21.15.19:0/1383934156 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc029060 con 0x55a8c4927470 2024-05-07T02:35:54.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.997+0000 7fa2ccc00700 1 -- 172.21.15.19:0/1383934156 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fa2dc012040 con 0x55a8c4927470 2024-05-07T02:35:54.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.997+0000 7fa2f7f0fb80 1 -- 172.21.15.19:0/1383934156 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2d00042a0 con 0x55a8c4927470 2024-05-07T02:35:54.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.997+0000 7fa2ccc00700 1 -- 172.21.15.19:0/1383934156 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa2dc030050 con 0x55a8c4927470 2024-05-07T02:35:54.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.997+0000 7fa2f7f0fb80 10 client.25894 fetch_fsmap learned FSMap version 40 2024-05-07T02:35:54.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.997+0000 7fa2f7f0fb80 1 -- 172.21.15.19:0/1383934156 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a8c49391d0 con 0x55a8c4927470 2024-05-07T02:35:54.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.998+0000 7fa2ccc00700 1 -- 172.21.15.19:0/1383934156 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2d00042a0 con 0x55a8c4927470 2024-05-07T02:35:54.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.998+0000 7fa2ccc00700 1 -- 172.21.15.19:0/1383934156 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7fa2dc024690 con 0x55a8c4927470 2024-05-07T02:35:54.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.998+0000 7fa2f7f0fb80 10 client.25894 fetch_fsmap finished waiting for FSMap version 40 2024-05-07T02:35:54.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.998+0000 7fa2f7f0fb80 10 client.25894 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-05-07T02:35:54.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.998+0000 7fa2f7f0fb80 20 client.25894 populate_metadata read hostname 'smithi019' 2024-05-07T02:35:54.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.998+0000 7fa2f7f0fb80 1 --2- 172.21.15.19:0/1383934156 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x55a8c493b810 0x55a8c493dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:54.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.998+0000 7fa2f7f0fb80 4 client.25894 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:35:55.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.998+0000 7fa2f7f0fb80 1 -- 172.21.15.19:0/1383934156 --> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8c48ee0e0 con 0x55a8c493b810 2024-05-07T02:35:55.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.999+0000 7fa2e2800700 1 --2- 172.21.15.19:0/1383934156 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x55a8c493b810 0x55a8c493dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:55.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.999+0000 7fa2e2800700 1 --2- 172.21.15.19:0/1383934156 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x55a8c493b810 0x55a8c493dc70 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=357b7a8ba56788fb server_cookie=7f7e2e406c262b2a in_seq=0 out_seq=0 2024-05-07T02:35:55.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:54.999+0000 7fa2ccc00700 10 client.25894 ms_handle_connect on v2:172.21.15.27:6836/2583138143 2024-05-07T02:35:55.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.000+0000 7fa2ccc00700 1 -- 172.21.15.19:0/1383934156 <== mds.3 v2:172.21.15.27:6836/2583138143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a8c48ee0e0 con 0x55a8c493b810 2024-05-07T02:35:55.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.000+0000 7fa2ccc00700 10 client.25894 handle_command_reply: tid=0 2024-05-07T02:35:55.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.000+0000 7fa2f7f0fb80 1 client.25894 shutdown 2024-05-07T02:35:55.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.000+0000 7fa2f7f0fb80 1 -- 172.21.15.19:0/1383934156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c4927470 msgr2=0x55a8c492c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:55.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.000+0000 7fa2f7f0fb80 1 --2- 172.21.15.19:0/1383934156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c4927470 0x55a8c492c060 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc0199b0 tx=0x7fa2dc004ad0 comp rx=0 tx=0).stop 2024-05-07T02:35:55.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.001+0000 7fa2f7f0fb80 1 -- 172.21.15.19:0/1383934156 shutdown_connections 2024-05-07T02:35:55.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.001+0000 7fa2f7f0fb80 1 --2- 172.21.15.19:0/1383934156 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x55a8c493b810 0x55a8c493dc70 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:55.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.001+0000 7fa2f7f0fb80 1 --2- 172.21.15.19:0/1383934156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c4927470 0x55a8c492c060 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:55.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.001+0000 7fa2f7f0fb80 1 --2- 172.21.15.19:0/1383934156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c4921620 0x55a8c492b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:55.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.001+0000 7fa2f7f0fb80 1 --2- 172.21.15.19:0/1383934156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c4921f70 0x55a8c492bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:55.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.001+0000 7fa2f7f0fb80 1 -- 172.21.15.19:0/1383934156 >> 172.21.15.19:0/1383934156 conn(0x55a8c47a3e40 msgr2=0x55a8c4920610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:35:55.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.001+0000 7fa2ccc00700 0 client.25894 ms_handle_reset on v2:172.21.15.27:6836/2583138143 2024-05-07T02:35:55.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.001+0000 7fa2f7f0fb80 1 -- 172.21.15.19:0/1383934156 shutdown_connections 2024-05-07T02:35:55.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.001+0000 7fa2f7f0fb80 1 -- 172.21.15.19:0/1383934156 wait complete. 2024-05-07T02:35:55.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.001+0000 7fa2f7f0fb80 20 client.25894 trim_cache size 0 max 16384 2024-05-07T02:35:55.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.020+0000 7fa2f7f0fb80 1 Processor -- start 2024-05-07T02:35:55.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.020+0000 7fa2f7f0fb80 1 -- start start 2024-05-07T02:35:55.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.020+0000 7fa2f7f0fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c4921620 0x55a8c4925d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:55.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.020+0000 7fa2f7f0fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c4921f70 0x55a8c4928f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:55.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.020+0000 7fa2f7f0fb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c4927470 0x55a8c4937610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:55.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.020+0000 7fa2f7f0fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e41b2700 con 0x55a8c4921f70 2024-05-07T02:35:55.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.020+0000 7fa2f7f0fb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e41b42a0 con 0x55a8c4927470 2024-05-07T02:35:55.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.020+0000 7fa2f7f0fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e41b2dd0 con 0x55a8c4921620 2024-05-07T02:35:55.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.020+0000 7fa2e2800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c4921620 0x55a8c4925d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:55.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.020+0000 7fa2e2800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c4921620 0x55a8c4925d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:47906/0 (socket says 172.21.15.19:47906) 2024-05-07T02:35:55.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.020+0000 7fa2e2800700 1 -- 172.21.15.19:0/563965428 learned_addr learned my addr 172.21.15.19:0/563965428 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:35:55.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.021+0000 7fa2e3200700 1 --2- 172.21.15.19:0/563965428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c4927470 0x55a8c4937610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:55.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.021+0000 7fa2e1e00700 1 --2- 172.21.15.19:0/563965428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c4921f70 0x55a8c4928f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:55.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.021+0000 7fa2e2800700 1 -- 172.21.15.19:0/563965428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c4927470 msgr2=0x55a8c4937610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:55.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.021+0000 7fa2e2800700 1 --2- 172.21.15.19:0/563965428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c4927470 0x55a8c4937610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:55.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.021+0000 7fa2e2800700 1 -- 172.21.15.19:0/563965428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c4921f70 msgr2=0x55a8c4928f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:55.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.021+0000 7fa2e2800700 1 --2- 172.21.15.19:0/563965428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c4921f70 0x55a8c4928f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:55.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.021+0000 7fa2e2800700 1 -- 172.21.15.19:0/563965428 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8c4918b80 con 0x55a8c4921620 2024-05-07T02:35:55.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.021+0000 7fa2e2800700 1 --2- 172.21.15.19:0/563965428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c4921620 0x55a8c4925d90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4012ab0 tx=0x7fa2d4009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77d442c2c186d887 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:35:55.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.022+0000 7fa2cf400700 1 -- 172.21.15.19:0/563965428 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4026020 con 0x55a8c4921620 2024-05-07T02:35:55.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.022+0000 7fa2cf400700 1 -- 172.21.15.19:0/563965428 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2d4004ce0 con 0x55a8c4921620 2024-05-07T02:35:55.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.022+0000 7fa2cf400700 1 -- 172.21.15.19:0/563965428 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4005020 con 0x55a8c4921620 2024-05-07T02:35:55.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.023+0000 7fa2f7f0fb80 1 -- 172.21.15.19:0/563965428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c4921620 msgr2=0x55a8c4925d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:55.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.023+0000 7fa2f7f0fb80 1 --2- 172.21.15.19:0/563965428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c4921620 0x55a8c4925d90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4012ab0 tx=0x7fa2d4009260 comp rx=0 tx=0).stop 2024-05-07T02:35:55.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.023+0000 7fa2f7f0fb80 1 -- 172.21.15.19:0/563965428 shutdown_connections 2024-05-07T02:35:55.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.023+0000 7fa2f7f0fb80 1 --2- 172.21.15.19:0/563965428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c4927470 0x55a8c4937610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:55.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.023+0000 7fa2f7f0fb80 1 --2- 172.21.15.19:0/563965428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c4921620 0x55a8c4925d90 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:55.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.023+0000 7fa2f7f0fb80 1 --2- 172.21.15.19:0/563965428 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c4921f70 0x55a8c4928f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:55.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.023+0000 7fa2f7f0fb80 1 -- 172.21.15.19:0/563965428 >> 172.21.15.19:0/563965428 conn(0x55a8c47a3e40 msgr2=0x55a8c4929ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:35:55.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.023+0000 7fa2f7f0fb80 1 -- 172.21.15.19:0/563965428 shutdown_connections 2024-05-07T02:35:55.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.023+0000 7fa2f7f0fb80 1 -- 172.21.15.19:0/563965428 wait complete. 2024-05-07T02:35:55.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.024+0000 7fa2f7f0fb80 1 -- start start 2024-05-07T02:35:55.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.024+0000 7fa2f7f0fb80 1 Processor -- start 2024-05-07T02:35:55.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.025+0000 7fa2f7f0fb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c4921620 0x55a8c4943750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:55.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.025+0000 7fa2f7f0fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c4921f70 0x55a8c4933990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:55.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.025+0000 7fa2f7f0fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c4927470 0x55a8c4933ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:55.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.025+0000 7fa2f7f0fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e41b3bd0 con 0x55a8c4927470 2024-05-07T02:35:55.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.025+0000 7fa2f7f0fb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e41b3500 con 0x55a8c4921620 2024-05-07T02:35:55.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.025+0000 7fa2f7f0fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e41b06a0 con 0x55a8c4921f70 2024-05-07T02:35:55.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.025+0000 7fa2f7f0fb80 10 client.0 _ll_register_callbacks cb 0x55a8c47d7380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:35:55.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.025+0000 7fa2e2800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c4921620 0x55a8c4943750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:55.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.025+0000 7fa2e1e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c4921f70 0x55a8c4933990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:55.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.025+0000 7fa2e2800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c4921620 0x55a8c4943750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:49052/0 (socket says 172.21.15.19:49052) 2024-05-07T02:35:55.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.025+0000 7fa2e2800700 1 -- 172.21.15.19:0/1425894459 learned_addr learned my addr 172.21.15.19:0/1425894459 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:35:55.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.025+0000 7fa2e3200700 1 --2- 172.21.15.19:0/1425894459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c4927470 0x55a8c4933ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:55.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.025+0000 7fa2e1e00700 1 -- 172.21.15.19:0/1425894459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c4921620 msgr2=0x55a8c4943750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:55.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.025+0000 7fa2e1e00700 1 --2- 172.21.15.19:0/1425894459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c4921620 0x55a8c4943750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:55.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.025+0000 7fa2e1e00700 1 -- 172.21.15.19:0/1425894459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c4927470 msgr2=0x55a8c4933ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:55.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.025+0000 7fa2e1e00700 1 --2- 172.21.15.19:0/1425894459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c4927470 0x55a8c4933ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:55.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.025+0000 7fa2e1e00700 1 -- 172.21.15.19:0/1425894459 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8c4942460 con 0x55a8c4921f70 2024-05-07T02:35:55.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.025+0000 7fa2e2800700 1 --2- 172.21.15.19:0/1425894459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c4921620 0x55a8c4943750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:35:55.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.026+0000 7fa2e1e00700 1 --2- 172.21.15.19:0/1425894459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c4921f70 0x55a8c4933990 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0000c00 tx=0x7fa2d0005fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3bffafbb2f14d02 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:35:55.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.026+0000 7fa2ccc00700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:35:55.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.026+0000 7fa2ccc00700 1 -- 172.21.15.19:0/1425894459 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d00c3070 con 0x55a8c4921f70 2024-05-07T02:35:55.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.026+0000 7fa2ccc00700 1 -- 172.21.15.19:0/1425894459 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fa2d00920b0 con 0x55a8c4921f70 2024-05-07T02:35:55.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.026+0000 7fa2f7f0fb80 1 -- 172.21.15.19:0/1425894459 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2d4004ce0 con 0x55a8c4921f70 2024-05-07T02:35:55.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.026+0000 7fa2ccc00700 1 -- 172.21.15.19:0/1425894459 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa2d00e1000 con 0x55a8c4921f70 2024-05-07T02:35:55.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.026+0000 7fa2f7f0fb80 10 client.25900 fetch_fsmap learned FSMap version 40 2024-05-07T02:35:55.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.027+0000 7fa2f7f0fb80 1 -- 172.21.15.19:0/1425894459 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a8c49a50e0 con 0x55a8c4921f70 2024-05-07T02:35:55.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.027+0000 7fa2ccc00700 1 -- 172.21.15.19:0/1425894459 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2d4004ce0 con 0x55a8c4921f70 2024-05-07T02:35:55.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.027+0000 7fa2ccc00700 1 -- 172.21.15.19:0/1425894459 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7fa2d00b0240 con 0x55a8c4921f70 2024-05-07T02:35:55.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.027+0000 7fa2f7f0fb80 10 client.25900 fetch_fsmap finished waiting for FSMap version 40 2024-05-07T02:35:55.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.027+0000 7fa2f7f0fb80 10 client.25900 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-05-07T02:35:55.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.027+0000 7fa2f7f0fb80 20 client.25900 populate_metadata read hostname 'smithi019' 2024-05-07T02:35:55.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.027+0000 7fa2f7f0fb80 1 --2- 172.21.15.19:0/1425894459 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x55a8c49a54c0 0x55a8c49a78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:55.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.027+0000 7fa2f7f0fb80 4 client.25900 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:35:55.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.027+0000 7fa2f7f0fb80 1 -- 172.21.15.19:0/1425894459 --> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a8c4851400 con 0x55a8c49a54c0 2024-05-07T02:35:55.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.027+0000 7fa2e2800700 1 --2- 172.21.15.19:0/1425894459 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x55a8c49a54c0 0x55a8c49a78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:55.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.028+0000 7fa2e2800700 1 --2- 172.21.15.19:0/1425894459 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x55a8c49a54c0 0x55a8c49a78e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=638e32a80b084aed server_cookie=2b44d00cb85b38cb in_seq=0 out_seq=0 2024-05-07T02:35:55.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.028+0000 7fa2ccc00700 10 client.25900 ms_handle_connect on v2:172.21.15.27:6836/2583138143 2024-05-07T02:35:55.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.028+0000 7fa2ccc00700 1 -- 172.21.15.19:0/1425894459 <== mds.3 v2:172.21.15.27:6836/2583138143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a8c4851400 con 0x55a8c49a54c0 2024-05-07T02:35:55.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.028+0000 7fa2ccc00700 10 client.25900 handle_command_reply: tid=0 2024-05-07T02:35:55.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.028+0000 7fa2f7f0fb80 1 client.25900 shutdown 2024-05-07T02:35:55.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.029+0000 7fa2f7f0fb80 1 -- 172.21.15.19:0/1425894459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c4921f70 msgr2=0x55a8c4933990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:55.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.029+0000 7fa2f7f0fb80 1 --2- 172.21.15.19:0/1425894459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c4921f70 0x55a8c4933990 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0000c00 tx=0x7fa2d0005fb0 comp rx=0 tx=0).stop 2024-05-07T02:35:55.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.029+0000 7fa2f7f0fb80 1 -- 172.21.15.19:0/1425894459 shutdown_connections 2024-05-07T02:35:55.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.029+0000 7fa2f7f0fb80 1 --2- 172.21.15.19:0/1425894459 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x55a8c49a54c0 0x55a8c49a78e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:55.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.029+0000 7fa2f7f0fb80 1 --2- 172.21.15.19:0/1425894459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a8c4927470 0x55a8c4933ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:55.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.029+0000 7fa2f7f0fb80 1 --2- 172.21.15.19:0/1425894459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a8c4921620 0x55a8c4943750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:55.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.029+0000 7fa2f7f0fb80 1 --2- 172.21.15.19:0/1425894459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a8c4921f70 0x55a8c4933990 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:55.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.029+0000 7fa2ccc00700 0 client.25900 ms_handle_reset on v2:172.21.15.27:6836/2583138143 2024-05-07T02:35:55.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.029+0000 7fa2f7f0fb80 1 -- 172.21.15.19:0/1425894459 >> 172.21.15.19:0/1425894459 conn(0x55a8c47a3e40 msgr2=0x55a8c4929c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:35:55.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.029+0000 7fa2f7f0fb80 1 -- 172.21.15.19:0/1425894459 shutdown_connections 2024-05-07T02:35:55.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.029+0000 7fa2f7f0fb80 1 -- 172.21.15.19:0/1425894459 wait complete. 2024-05-07T02:35:55.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.029+0000 7fa2f7f0fb80 20 client.25900 trim_cache size 0 max 16384 2024-05-07T02:35:55.031 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:35:55.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.030+0000 7fa2e9400700 1 -- 172.21.15.19:0/3813686996 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fa2c80b2a70 msgr2=0x7fa2c80b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:55.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.030+0000 7fa2e9400700 1 --2- 172.21.15.19:0/3813686996 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fa2c80b2a70 0x7fa2c80b4f80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc009710 tx=0x7fa2dc009680 comp rx=0 tx=0).stop 2024-05-07T02:35:55.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.030+0000 7fa2e9400700 1 -- 172.21.15.19:0/3813686996 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2e4105e10 msgr2=0x7fa2e419d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:55.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.030+0000 7fa2e9400700 1 --2- 172.21.15.19:0/3813686996 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2e4105e10 0x7fa2e419d5b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0005910 tx=0x7fa2d000b670 comp rx=0 tx=0).stop 2024-05-07T02:35:55.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.030+0000 7fa2e9400700 1 -- 172.21.15.19:0/3813686996 shutdown_connections 2024-05-07T02:35:55.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.030+0000 7fa2e9400700 1 --2- 172.21.15.19:0/3813686996 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fa2c80b2a70 0x7fa2c80b4f80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:55.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.030+0000 7fa2e9400700 1 --2- 172.21.15.19:0/3813686996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2e410e8f0 0x7fa2e41a1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:55.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.030+0000 7fa2e9400700 1 --2- 172.21.15.19:0/3813686996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2e41054c0 0x7fa2e419d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:55.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.030+0000 7fa2e9400700 1 --2- 172.21.15.19:0/3813686996 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2e4105e10 0x7fa2e419d5b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:55.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.030+0000 7fa2e9400700 1 -- 172.21.15.19:0/3813686996 >> 172.21.15.19:0/3813686996 conn(0x7fa2e40fedc0 msgr2=0x7fa2e40ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:35:55.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.030+0000 7fa2e9400700 1 -- 172.21.15.19:0/3813686996 shutdown_connections 2024-05-07T02:35:55.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:55.030+0000 7fa2e9400700 1 -- 172.21.15.19:0/3813686996 wait complete. 2024-05-07T02:35:55.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:54 smithi064 ceph-mon[34699]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:35:55.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:54 smithi064 ceph-mon[34699]: pgmap v125: 129 pgs: 129 active+clean; 78 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 1.4 MiB/s wr, 56 op/s 2024-05-07T02:35:55.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:54 smithi027 ceph-mon[34398]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:35:55.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:54 smithi027 ceph-mon[34398]: pgmap v125: 129 pgs: 129 active+clean; 78 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 1.4 MiB/s wr, 56 op/s 2024-05-07T02:35:55.741 INFO:tasks.workunit.client.0.smithi019.stdout: 1 92718 2.03 MB/sec execute 51 sec latency 35.841 ms 2024-05-07T02:35:56.030 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-07T02:35:56.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:56 smithi064 ceph-mon[34699]: pgmap v126: 129 pgs: 129 active+clean; 68 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 1.5 MiB/s wr, 82 op/s 2024-05-07T02:35:56.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:56 smithi019 ceph-mon[29781]: pgmap v126: 129 pgs: 129 active+clean; 68 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 1.5 MiB/s wr, 82 op/s 2024-05-07T02:35:56.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:56 smithi027 ceph-mon[34398]: pgmap v126: 129 pgs: 129 active+clean; 68 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 1.5 MiB/s wr, 82 op/s 2024-05-07T02:35:56.522 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:35:56.741 INFO:tasks.workunit.client.0.smithi019.stdout: 1 93907 2.09 MB/sec execute 52 sec latency 17.714 ms 2024-05-07T02:35:57.742 INFO:tasks.workunit.client.0.smithi019.stdout: 1 95291 2.19 MB/sec execute 53 sec latency 94.207 ms 2024-05-07T02:35:58.144 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:35:58 smithi019 ceph-mon[29781]: pgmap v127: 129 pgs: 129 active+clean; 68 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 1.5 MiB/s wr, 71 op/s 2024-05-07T02:35:58.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:35:58 smithi064 ceph-mon[34699]: pgmap v127: 129 pgs: 129 active+clean; 68 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 1.5 MiB/s wr, 71 op/s 2024-05-07T02:35:58.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:35:58 smithi027 ceph-mon[34398]: pgmap v127: 129 pgs: 129 active+clean; 68 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 1.5 MiB/s wr, 71 op/s 2024-05-07T02:35:58.742 INFO:tasks.workunit.client.0.smithi019.stdout: 1 95291 2.15 MB/sec execute 54 sec latency 1094.301 ms 2024-05-07T02:35:58.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.860+0000 7f9002200700 1 -- 172.21.15.19:0/2256328167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ffc107df0 msgr2=0x7f8ffc108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:58.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.860+0000 7f9002200700 1 --2- 172.21.15.19:0/2256328167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ffc107df0 0x7f8ffc108200 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8fe80099c0 tx=0x7f8fe8009cd0 comp rx=0 tx=0).stop 2024-05-07T02:35:58.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.860+0000 7f9002200700 1 -- 172.21.15.19:0/2256328167 shutdown_connections 2024-05-07T02:35:58.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.860+0000 7f9002200700 1 --2- 172.21.15.19:0/2256328167 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ffc108740 0x7f8ffc1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:58.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.860+0000 7f9002200700 1 --2- 172.21.15.19:0/2256328167 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ffc1034c0 0x7f8ffc1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:58.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.860+0000 7f9002200700 1 --2- 172.21.15.19:0/2256328167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ffc107df0 0x7f8ffc108200 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:58.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.860+0000 7f9002200700 1 -- 172.21.15.19:0/2256328167 >> 172.21.15.19:0/2256328167 conn(0x7f8ffc0fedc0 msgr2=0x7f8ffc1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:35:58.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.860+0000 7f9002200700 1 -- 172.21.15.19:0/2256328167 shutdown_connections 2024-05-07T02:35:58.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.860+0000 7f9002200700 1 -- 172.21.15.19:0/2256328167 wait complete. 2024-05-07T02:35:58.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.861+0000 7f9002200700 1 Processor -- start 2024-05-07T02:35:58.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.861+0000 7f9002200700 1 -- start start 2024-05-07T02:35:58.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.862+0000 7f9002200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ffc1034c0 0x7f8ffc19cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:58.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.862+0000 7f9002200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ffc107df0 0x7f8ffc19d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:58.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.862+0000 7f9002200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ffc108740 0x7f8ffc1a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:58.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.862+0000 7f9002200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8ffc114480 con 0x7f8ffc107df0 2024-05-07T02:35:58.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.862+0000 7f9002200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8ffc114300 con 0x7f8ffc108740 2024-05-07T02:35:58.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.862+0000 7f9002200700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8ffc114600 con 0x7f8ffc1034c0 2024-05-07T02:35:58.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.862+0000 7f8ffbe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ffc108740 0x7f8ffc1a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:58.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.862+0000 7f8ffaa00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ffc107df0 0x7f8ffc19d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:58.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.862+0000 7f8ffbe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ffc108740 0x7f8ffc1a1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:53886/0 (socket says 172.21.15.19:53886) 2024-05-07T02:35:58.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.862+0000 7f8ffbe00700 1 -- 172.21.15.19:0/1351952392 learned_addr learned my addr 172.21.15.19:0/1351952392 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:35:58.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.862+0000 7f8ffb400700 1 --2- 172.21.15.19:0/1351952392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ffc1034c0 0x7f8ffc19cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:58.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.862+0000 7f8ffbe00700 1 -- 172.21.15.19:0/1351952392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ffc1034c0 msgr2=0x7f8ffc19cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:58.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.862+0000 7f8ffbe00700 1 --2- 172.21.15.19:0/1351952392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ffc1034c0 0x7f8ffc19cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:58.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.862+0000 7f8ffbe00700 1 -- 172.21.15.19:0/1351952392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ffc107df0 msgr2=0x7f8ffc19d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:58.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.862+0000 7f8ffbe00700 1 --2- 172.21.15.19:0/1351952392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ffc107df0 0x7f8ffc19d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:58.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.862+0000 7f8ffbe00700 1 -- 172.21.15.19:0/1351952392 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ffc1a1ec0 con 0x7f8ffc108740 2024-05-07T02:35:58.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.863+0000 7f8ffbe00700 1 --2- 172.21.15.19:0/1351952392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ffc108740 0x7f8ffc1a1820 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8ff400ef30 tx=0x7f8ff400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:35:58.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.863+0000 7f8ff3e00700 1 -- 172.21.15.19:0/1351952392 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff40092e0 con 0x7f8ffc108740 2024-05-07T02:35:58.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.863+0000 7f9002200700 1 -- 172.21.15.19:0/1351952392 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ffc1a2150 con 0x7f8ffc108740 2024-05-07T02:35:58.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.863+0000 7f8ff3e00700 1 -- 172.21.15.19:0/1351952392 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ff4004bb0 con 0x7f8ffc108740 2024-05-07T02:35:58.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.863+0000 7f8ff3e00700 1 -- 172.21.15.19:0/1351952392 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff4004020 con 0x7f8ffc108740 2024-05-07T02:35:58.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.863+0000 7f9002200700 1 -- 172.21.15.19:0/1351952392 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ffc1a2660 con 0x7f8ffc108740 2024-05-07T02:35:58.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.867+0000 7f8ff3e00700 1 -- 172.21.15.19:0/1351952392 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8ff401b030 con 0x7f8ffc108740 2024-05-07T02:35:58.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.868+0000 7f9010cf2b80 1 Processor -- start 2024-05-07T02:35:58.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.868+0000 7f9010cf2b80 1 -- start start 2024-05-07T02:35:58.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.868+0000 7f9010cf2b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2a239f470 0x55e2a239f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:58.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.868+0000 7f9010cf2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2a2399620 0x55e2a2399a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:58.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.869+0000 7f9010cf2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a2399f70 0x55e2a23a6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:58.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.869+0000 7f9010cf2b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e2a22bba20 con 0x55e2a2399f70 2024-05-07T02:35:58.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.869+0000 7f9010cf2b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55e2a2259760 con 0x55e2a239f470 2024-05-07T02:35:58.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.869+0000 7f9010cf2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e2a22b79e0 con 0x55e2a2399620 2024-05-07T02:35:58.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.869+0000 7f8ff3e00700 1 --2- 172.21.15.19:0/1351952392 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f8fe00b29f0 0x7f8fe00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:58.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.869+0000 7f8ffaa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2a239f470 0x55e2a239f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:58.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.869+0000 7f8ffaa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2a239f470 0x55e2a239f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:53900/0 (socket says 172.21.15.19:53900) 2024-05-07T02:35:58.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.869+0000 7f8ffaa00700 1 -- 172.21.15.19:0/738130299 learned_addr learned my addr 172.21.15.19:0/738130299 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:35:58.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.869+0000 7f8ffaa00700 1 --2- 172.21.15.19:0/738130299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a2399f70 0x55e2a23a6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:58.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.869+0000 7f8ffb400700 1 --2- 172.21.15.19:0/738130299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2a2399620 0x55e2a2399a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:58.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.869+0000 7f8ffbe00700 1 --2- 172.21.15.19:0/1351952392 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f8fe00b29f0 0x7f8fe00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:58.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.869+0000 7f8ff3e00700 1 -- 172.21.15.19:0/1351952392 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f8ff40e23f0 con 0x7f8ffc108740 2024-05-07T02:35:58.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.870+0000 7f8ffaa00700 1 -- 172.21.15.19:0/738130299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2a2399620 msgr2=0x55e2a2399a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:58.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.870+0000 7f8ffaa00700 1 --2- 172.21.15.19:0/738130299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2a2399620 0x55e2a2399a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:58.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.870+0000 7f8ffaa00700 1 -- 172.21.15.19:0/738130299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a2399f70 msgr2=0x55e2a23a6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:58.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.870+0000 7f8ffaa00700 1 --2- 172.21.15.19:0/738130299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a2399f70 0x55e2a23a6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:58.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.870+0000 7f8ffaa00700 1 -- 172.21.15.19:0/738130299 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2a22088d0 con 0x55e2a239f470 2024-05-07T02:35:58.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.870+0000 7f8ffaa00700 1 --2- 172.21.15.19:0/738130299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2a239f470 0x55e2a239f880 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8009fd0 tx=0x7f8fe8009230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81d182a2bc6b3b2d server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:35:58.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.870+0000 7f8ff0c00700 1 -- 172.21.15.19:0/738130299 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8005460 con 0x55e2a239f470 2024-05-07T02:35:58.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.870+0000 7f8ffbe00700 1 --2- 172.21.15.19:0/1351952392 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f8fe00b29f0 0x7f8fe00b4f00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8ff40df860 tx=0x7f8ff4009480 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:35:58.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.870+0000 7f8ff0c00700 1 -- 172.21.15.19:0/738130299 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fe8005600 con 0x55e2a239f470 2024-05-07T02:35:58.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.870+0000 7f8ff0c00700 1 -- 172.21.15.19:0/738130299 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe800bd90 con 0x55e2a239f470 2024-05-07T02:35:58.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.871+0000 7f9010cf2b80 1 -- 172.21.15.19:0/738130299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2a239f470 msgr2=0x55e2a239f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:58.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.871+0000 7f9010cf2b80 1 --2- 172.21.15.19:0/738130299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2a239f470 0x55e2a239f880 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8009fd0 tx=0x7f8fe8009230 comp rx=0 tx=0).stop 2024-05-07T02:35:58.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.871+0000 7f9010cf2b80 1 -- 172.21.15.19:0/738130299 shutdown_connections 2024-05-07T02:35:58.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.871+0000 7f9010cf2b80 1 --2- 172.21.15.19:0/738130299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a2399f70 0x55e2a23a6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:58.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.871+0000 7f9010cf2b80 1 --2- 172.21.15.19:0/738130299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2a239f470 0x55e2a239f880 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:58.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.871+0000 7f9010cf2b80 1 --2- 172.21.15.19:0/738130299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2a2399620 0x55e2a2399a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:58.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.871+0000 7f9010cf2b80 1 -- 172.21.15.19:0/738130299 >> 172.21.15.19:0/738130299 conn(0x55e2a221be40 msgr2=0x55e2a2394380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:35:58.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.872+0000 7f9010cf2b80 1 -- 172.21.15.19:0/738130299 shutdown_connections 2024-05-07T02:35:58.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.872+0000 7f9010cf2b80 1 -- 172.21.15.19:0/738130299 wait complete. 2024-05-07T02:35:58.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.872+0000 7f9010cf2b80 1 -- start start 2024-05-07T02:35:58.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.872+0000 7f9010cf2b80 1 Processor -- start 2024-05-07T02:35:58.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.873+0000 7f9010cf2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2a2399620 0x55e2a23a3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:58.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.873+0000 7f9010cf2b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2a2399f70 0x55e2a23a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:58.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.873+0000 7f9010cf2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a239f470 0x55e2a23a40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:58.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.873+0000 7f9010cf2b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e2a23a67e0 con 0x55e2a239f470 2024-05-07T02:35:58.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.873+0000 7f9010cf2b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55e2a2221b20 con 0x55e2a2399f70 2024-05-07T02:35:58.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.873+0000 7f9010cf2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e2a23a6960 con 0x55e2a2399620 2024-05-07T02:35:58.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.873+0000 7f9010cf2b80 10 client.0 _ll_register_callbacks cb 0x55e2a224f380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:35:58.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.873+0000 7f8ffaa00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2a2399620 0x55e2a23a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:58.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.873+0000 7f8ffb400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2a2399f70 0x55e2a23a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:58.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.874+0000 7f8ffaa00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2a2399620 0x55e2a23a3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:34266/0 (socket says 172.21.15.19:34266) 2024-05-07T02:35:58.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.874+0000 7f8ffaa00700 1 -- 172.21.15.19:0/4003294533 learned_addr learned my addr 172.21.15.19:0/4003294533 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:35:58.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.874+0000 7f8ffb400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2a2399f70 0x55e2a23a3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:53916/0 (socket says 172.21.15.19:53916) 2024-05-07T02:35:58.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.874+0000 7f8ffaa00700 1 --2- 172.21.15.19:0/4003294533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a239f470 0x55e2a23a40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:58.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.874+0000 7f8ffb400700 1 -- 172.21.15.19:0/4003294533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2a2399620 msgr2=0x55e2a23a3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:58.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.874+0000 7f8ffb400700 1 --2- 172.21.15.19:0/4003294533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2a2399620 0x55e2a23a3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:58.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.874+0000 7f8ffb400700 1 -- 172.21.15.19:0/4003294533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a239f470 msgr2=0x55e2a23a40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:58.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.874+0000 7f8ffb400700 1 --2- 172.21.15.19:0/4003294533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a239f470 0x55e2a23a40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:58.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.874+0000 7f8ffb400700 1 -- 172.21.15.19:0/4003294533 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2a2281420 con 0x55e2a2399f70 2024-05-07T02:35:58.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.874+0000 7f8ffb400700 1 --2- 172.21.15.19:0/4003294533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2a2399f70 0x55e2a23a3bb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8fec012770 tx=0x7f8fec012a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ee158a73d83c56f server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:35:58.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.875+0000 7f8fe5600700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:35:58.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.875+0000 7f8fe5600700 1 -- 172.21.15.19:0/4003294533 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec00f480 con 0x55e2a2399f70 2024-05-07T02:35:58.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.875+0000 7f9010cf2b80 1 -- 172.21.15.19:0/4003294533 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fe8005600 con 0x55e2a2399f70 2024-05-07T02:35:58.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.875+0000 7f8fe5600700 1 -- 172.21.15.19:0/4003294533 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f8fec00f620 con 0x55e2a2399f70 2024-05-07T02:35:58.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.875+0000 7f8fe5600700 1 -- 172.21.15.19:0/4003294533 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f8fec00f820 con 0x55e2a2399f70 2024-05-07T02:35:58.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.875+0000 7f9010cf2b80 10 client.25934 fetch_fsmap learned FSMap version 40 2024-05-07T02:35:58.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.875+0000 7f9010cf2b80 1 -- 172.21.15.19:0/4003294533 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55e2a23b1510 con 0x55e2a2399f70 2024-05-07T02:35:58.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.876+0000 7f8fe5600700 1 -- 172.21.15.19:0/4003294533 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fe8005600 con 0x55e2a2399f70 2024-05-07T02:35:58.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.876+0000 7f8fe5600700 1 -- 172.21.15.19:0/4003294533 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f8fec0109a0 con 0x55e2a2399f70 2024-05-07T02:35:58.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.876+0000 7f9010cf2b80 10 client.25934 fetch_fsmap finished waiting for FSMap version 40 2024-05-07T02:35:58.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.876+0000 7f9010cf2b80 10 client.25934 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-07T02:35:58.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.876+0000 7f9010cf2b80 20 client.25934 populate_metadata read hostname 'smithi019' 2024-05-07T02:35:58.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.876+0000 7f9010cf2b80 1 --2- 172.21.15.19:0/4003294533 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55e2a23b3bf0 0x55e2a23b6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:58.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.876+0000 7f9010cf2b80 4 client.25934 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:35:58.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.876+0000 7f9010cf2b80 1 -- 172.21.15.19:0/4003294533 --> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2a23660e0 con 0x55e2a23b3bf0 2024-05-07T02:35:58.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.876+0000 7f8ffaa00700 1 --2- 172.21.15.19:0/4003294533 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55e2a23b3bf0 0x55e2a23b6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:58.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.877+0000 7f8ffaa00700 1 --2- 172.21.15.19:0/4003294533 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55e2a23b3bf0 0x55e2a23b6010 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=909312e28d2fbcd0 server_cookie=38f6ad13b1e00183 in_seq=0 out_seq=0 2024-05-07T02:35:58.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.877+0000 7f8fe5600700 10 client.25934 ms_handle_connect on v2:172.21.15.64:6838/1583120004 2024-05-07T02:35:58.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.878+0000 7f8fe5600700 1 -- 172.21.15.19:0/4003294533 <== mds.2 v2:172.21.15.64:6838/1583120004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e2a23660e0 con 0x55e2a23b3bf0 2024-05-07T02:35:58.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.878+0000 7f8fe5600700 10 client.25934 handle_command_reply: tid=0 2024-05-07T02:35:58.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.878+0000 7f9010cf2b80 1 client.25934 shutdown 2024-05-07T02:35:58.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.878+0000 7f9010cf2b80 1 -- 172.21.15.19:0/4003294533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2a2399f70 msgr2=0x55e2a23a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:58.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.878+0000 7f9010cf2b80 1 --2- 172.21.15.19:0/4003294533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2a2399f70 0x55e2a23a3bb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8fec012770 tx=0x7f8fec012a80 comp rx=0 tx=0).stop 2024-05-07T02:35:58.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.878+0000 7f9010cf2b80 1 -- 172.21.15.19:0/4003294533 shutdown_connections 2024-05-07T02:35:58.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.878+0000 7f9010cf2b80 1 --2- 172.21.15.19:0/4003294533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a239f470 0x55e2a23a40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:58.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.878+0000 7f9010cf2b80 1 --2- 172.21.15.19:0/4003294533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2a2399620 0x55e2a23a3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:58.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.878+0000 7f9010cf2b80 1 --2- 172.21.15.19:0/4003294533 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55e2a23b3bf0 0x55e2a23b6010 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:58.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.878+0000 7f9010cf2b80 1 --2- 172.21.15.19:0/4003294533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2a2399f70 0x55e2a23a3bb0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:58.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.878+0000 7f9010cf2b80 1 -- 172.21.15.19:0/4003294533 >> 172.21.15.19:0/4003294533 conn(0x55e2a221be40 msgr2=0x55e2a2398610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:35:58.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.878+0000 7f8fe5600700 0 client.25934 ms_handle_reset on v2:172.21.15.64:6838/1583120004 2024-05-07T02:35:58.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.878+0000 7f9010cf2b80 1 -- 172.21.15.19:0/4003294533 shutdown_connections 2024-05-07T02:35:58.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.879+0000 7f9010cf2b80 1 -- 172.21.15.19:0/4003294533 wait complete. 2024-05-07T02:35:58.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.879+0000 7f9010cf2b80 20 client.25934 trim_cache size 0 max 16384 2024-05-07T02:35:58.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.895+0000 7f9010cf2b80 1 Processor -- start 2024-05-07T02:35:58.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.895+0000 7f9010cf2b80 1 -- start start 2024-05-07T02:35:58.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.895+0000 7f9010cf2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a2399620 0x55e2a238e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:58.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.895+0000 7f9010cf2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2a2399f70 0x55e2a23af8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:58.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.895+0000 7f9010cf2b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2a239f470 0x55e2a23a3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:58.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.895+0000 7f9010cf2b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8ffc1b25a0 con 0x55e2a2399620 2024-05-07T02:35:58.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.895+0000 7f9010cf2b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8ffc1b4140 con 0x55e2a239f470 2024-05-07T02:35:58.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.895+0000 7f9010cf2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8ffc1b2c70 con 0x55e2a2399f70 2024-05-07T02:35:58.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.896+0000 7f8ffaa00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a2399620 0x55e2a238e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:58.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.896+0000 7f8ffaa00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a2399620 0x55e2a238e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58734/0 (socket says 172.21.15.19:58734) 2024-05-07T02:35:58.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.896+0000 7f8ffaa00700 1 -- 172.21.15.19:0/771794734 learned_addr learned my addr 172.21.15.19:0/771794734 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:35:58.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.896+0000 7f8ffb400700 1 --2- 172.21.15.19:0/771794734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2a2399f70 0x55e2a23af8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:58.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.896+0000 7f8ffaa00700 1 --2- 172.21.15.19:0/771794734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2a239f470 0x55e2a23a3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:58.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.896+0000 7f8ffaa00700 1 -- 172.21.15.19:0/771794734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2a2399f70 msgr2=0x55e2a23af8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:58.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.896+0000 7f8ffaa00700 1 --2- 172.21.15.19:0/771794734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2a2399f70 0x55e2a23af8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:58.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.896+0000 7f8ffaa00700 1 -- 172.21.15.19:0/771794734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2a239f470 msgr2=0x55e2a23a3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:58.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.896+0000 7f8ffaa00700 1 --2- 172.21.15.19:0/771794734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2a239f470 0x55e2a23a3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:58.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.896+0000 7f8ffaa00700 1 -- 172.21.15.19:0/771794734 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2a2390b80 con 0x55e2a2399620 2024-05-07T02:35:58.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.896+0000 7f8ffaa00700 1 --2- 172.21.15.19:0/771794734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a2399620 0x55e2a238e8d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8005850 tx=0x7f8fe8009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a0a89d08b933cc9 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:35:58.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.897+0000 7f8fe7e00700 1 -- 172.21.15.19:0/771794734 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe803d070 con 0x55e2a2399620 2024-05-07T02:35:58.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.897+0000 7f8fe7e00700 1 -- 172.21.15.19:0/771794734 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fe8005600 con 0x55e2a2399620 2024-05-07T02:35:58.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.897+0000 7f8fe7e00700 1 -- 172.21.15.19:0/771794734 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8038630 con 0x55e2a2399620 2024-05-07T02:35:58.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.898+0000 7f9010cf2b80 1 -- 172.21.15.19:0/771794734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a2399620 msgr2=0x55e2a238e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:58.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.898+0000 7f9010cf2b80 1 --2- 172.21.15.19:0/771794734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a2399620 0x55e2a238e8d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8005850 tx=0x7f8fe8009cd0 comp rx=0 tx=0).stop 2024-05-07T02:35:58.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.898+0000 7f9010cf2b80 1 -- 172.21.15.19:0/771794734 shutdown_connections 2024-05-07T02:35:58.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.898+0000 7f9010cf2b80 1 --2- 172.21.15.19:0/771794734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2a239f470 0x55e2a23a3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:58.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.898+0000 7f9010cf2b80 1 --2- 172.21.15.19:0/771794734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a2399620 0x55e2a238e8d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:58.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.898+0000 7f9010cf2b80 1 --2- 172.21.15.19:0/771794734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2a2399f70 0x55e2a23af8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:58.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.898+0000 7f9010cf2b80 1 -- 172.21.15.19:0/771794734 >> 172.21.15.19:0/771794734 conn(0x55e2a221be40 msgr2=0x55e2a238e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:35:58.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.898+0000 7f9010cf2b80 1 -- 172.21.15.19:0/771794734 shutdown_connections 2024-05-07T02:35:58.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.898+0000 7f9010cf2b80 1 -- 172.21.15.19:0/771794734 wait complete. 2024-05-07T02:35:58.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.898+0000 7f9010cf2b80 1 -- start start 2024-05-07T02:35:58.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.899+0000 7f9010cf2b80 1 Processor -- start 2024-05-07T02:35:58.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.899+0000 7f9010cf2b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2a2399620 0x55e2a2416770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:58.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.899+0000 7f9010cf2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a2399f70 0x55e2a2416cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:58.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.899+0000 7f9010cf2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2a239f470 0x55e2a23a71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:58.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.899+0000 7f9010cf2b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8ffc1b3a70 con 0x55e2a2399f70 2024-05-07T02:35:58.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.899+0000 7f9010cf2b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8ffc1b33a0 con 0x55e2a2399620 2024-05-07T02:35:58.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.899+0000 7f9010cf2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8ffc1b0540 con 0x55e2a239f470 2024-05-07T02:35:58.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.899+0000 7f9010cf2b80 10 client.0 _ll_register_callbacks cb 0x55e2a224f380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:35:58.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.900+0000 7f8ffb400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a2399f70 0x55e2a2416cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:58.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.900+0000 7f8ffaa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2a2399620 0x55e2a2416770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:58.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.900+0000 7f8ffb400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a2399f70 0x55e2a2416cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58736/0 (socket says 172.21.15.19:58736) 2024-05-07T02:35:58.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.900+0000 7f8ffb400700 1 -- 172.21.15.19:0/2306452116 learned_addr learned my addr 172.21.15.19:0/2306452116 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:35:58.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.900+0000 7f8ffaa00700 1 --2- 172.21.15.19:0/2306452116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2a239f470 0x55e2a23a71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:58.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.900+0000 7f8ffb400700 1 -- 172.21.15.19:0/2306452116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2a239f470 msgr2=0x55e2a23a71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:58.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.900+0000 7f8ffb400700 1 --2- 172.21.15.19:0/2306452116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2a239f470 0x55e2a23a71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:58.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.900+0000 7f8ffb400700 1 -- 172.21.15.19:0/2306452116 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2a2399620 msgr2=0x55e2a2416770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:35:58.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.900+0000 7f8ffb400700 1 --2- 172.21.15.19:0/2306452116 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2a2399620 0x55e2a2416770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:58.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.900+0000 7f8ffb400700 1 -- 172.21.15.19:0/2306452116 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2a23a77f0 con 0x55e2a2399f70 2024-05-07T02:35:58.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.900+0000 7f8ffaa00700 1 --2- 172.21.15.19:0/2306452116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2a239f470 0x55e2a23a71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:35:58.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.900+0000 7f8ffb400700 1 --2- 172.21.15.19:0/2306452116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a2399f70 0x55e2a2416cb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x55e2a2394910 tx=0x7f8fec00fbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9127d95f59fe0fb server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:35:58.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.900+0000 7f8fe5600700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-07T02:35:58.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.901+0000 7f8fe5600700 1 -- 172.21.15.19:0/2306452116 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec030060 con 0x55e2a2399f70 2024-05-07T02:35:58.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.901+0000 7f8fe5600700 1 -- 172.21.15.19:0/2306452116 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f8fec005040 con 0x55e2a2399f70 2024-05-07T02:35:58.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.901+0000 7f9010cf2b80 1 -- 172.21.15.19:0/2306452116 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fe8005600 con 0x55e2a2399f70 2024-05-07T02:35:58.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.901+0000 7f8fe5600700 1 -- 172.21.15.19:0/2306452116 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f8fec005dc0 con 0x55e2a2399f70 2024-05-07T02:35:58.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.901+0000 7f9010cf2b80 10 client.15771 fetch_fsmap learned FSMap version 40 2024-05-07T02:35:58.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.901+0000 7f9010cf2b80 1 -- 172.21.15.19:0/2306452116 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55e2a241ddd0 con 0x55e2a2399f70 2024-05-07T02:35:58.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.901+0000 7f8fe5600700 1 -- 172.21.15.19:0/2306452116 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fe8005600 con 0x55e2a2399f70 2024-05-07T02:35:58.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.902+0000 7f8fe5600700 1 -- 172.21.15.19:0/2306452116 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f8fec003680 con 0x55e2a2399f70 2024-05-07T02:35:58.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.902+0000 7f9010cf2b80 10 client.15771 fetch_fsmap finished waiting for FSMap version 40 2024-05-07T02:35:58.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.902+0000 7f9010cf2b80 10 client.15771 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-07T02:35:58.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.902+0000 7f9010cf2b80 20 client.15771 populate_metadata read hostname 'smithi019' 2024-05-07T02:35:58.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.902+0000 7f9010cf2b80 1 --2- 172.21.15.19:0/2306452116 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55e2a241e520 0x55e2a2420940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:35:58.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.902+0000 7f9010cf2b80 4 client.15771 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:35:58.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.902+0000 7f9010cf2b80 1 -- 172.21.15.19:0/2306452116 --> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e2a22c9400 con 0x55e2a241e520 2024-05-07T02:35:58.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.902+0000 7f8ffaa00700 1 --2- 172.21.15.19:0/2306452116 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55e2a241e520 0x55e2a2420940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:35:58.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.903+0000 7f8ffaa00700 1 --2- 172.21.15.19:0/2306452116 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55e2a241e520 0x55e2a2420940 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=37c356bb13d97f8d server_cookie=a7092341f5b2b64 in_seq=0 out_seq=0 2024-05-07T02:35:58.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.903+0000 7f8fe5600700 10 client.15771 ms_handle_connect on v2:172.21.15.64:6838/1583120004 2024-05-07T02:35:58.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.903+0000 7f8fe5600700 1 -- 172.21.15.19:0/2306452116 <== mds.2 v2:172.21.15.64:6838/1583120004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e2a22c9400 con 0x55e2a241e520 2024-05-07T02:35:58.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.903+0000 7f8fe5600700 10 client.15771 handle_command_reply: tid=0 2024-05-07T02:35:58.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.903+0000 7f9010cf2b80 1 client.15771 shutdown 2024-05-07T02:35:58.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.903+0000 7f9010cf2b80 1 -- 172.21.15.19:0/2306452116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a2399f70 msgr2=0x55e2a2416cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:58.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.903+0000 7f9010cf2b80 1 --2- 172.21.15.19:0/2306452116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a2399f70 0x55e2a2416cb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x55e2a2394910 tx=0x7f8fec00fbd0 comp rx=0 tx=0).stop 2024-05-07T02:35:58.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.903+0000 7f9010cf2b80 1 -- 172.21.15.19:0/2306452116 shutdown_connections 2024-05-07T02:35:58.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.903+0000 7f9010cf2b80 1 --2- 172.21.15.19:0/2306452116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2a239f470 0x55e2a23a71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:58.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.903+0000 7f9010cf2b80 1 --2- 172.21.15.19:0/2306452116 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55e2a241e520 0x55e2a2420940 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:58.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.903+0000 7f9010cf2b80 1 --2- 172.21.15.19:0/2306452116 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2a2399620 0x55e2a2416770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:58.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.903+0000 7f9010cf2b80 1 --2- 172.21.15.19:0/2306452116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2a2399f70 0x55e2a2416cb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:58.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.903+0000 7f9010cf2b80 1 -- 172.21.15.19:0/2306452116 >> 172.21.15.19:0/2306452116 conn(0x55e2a221be40 msgr2=0x55e2a23a4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:35:58.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.903+0000 7f8fe5600700 0 client.15771 ms_handle_reset on v2:172.21.15.64:6838/1583120004 2024-05-07T02:35:58.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.904+0000 7f9010cf2b80 1 -- 172.21.15.19:0/2306452116 shutdown_connections 2024-05-07T02:35:58.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.904+0000 7f9010cf2b80 1 -- 172.21.15.19:0/2306452116 wait complete. 2024-05-07T02:35:58.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.904+0000 7f9010cf2b80 20 client.15771 trim_cache size 0 max 16384 2024-05-07T02:35:58.907 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:35:58.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.905+0000 7f9002200700 1 -- 172.21.15.19:0/1351952392 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f8fe00b29f0 msgr2=0x7f8fe00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:58.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.905+0000 7f9002200700 1 --2- 172.21.15.19:0/1351952392 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f8fe00b29f0 0x7f8fe00b4f00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8ff40df860 tx=0x7f8ff4009480 comp rx=0 tx=0).stop 2024-05-07T02:35:58.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.905+0000 7f9002200700 1 -- 172.21.15.19:0/1351952392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ffc108740 msgr2=0x7f8ffc1a1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:35:58.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.905+0000 7f9002200700 1 --2- 172.21.15.19:0/1351952392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ffc108740 0x7f8ffc1a1820 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8ff400ef30 tx=0x7f8ff400c3a0 comp rx=0 tx=0).stop 2024-05-07T02:35:58.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.905+0000 7f9002200700 1 -- 172.21.15.19:0/1351952392 shutdown_connections 2024-05-07T02:35:58.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.905+0000 7f9002200700 1 --2- 172.21.15.19:0/1351952392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ffc108740 0x7f8ffc1a1820 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:58.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.905+0000 7f9002200700 1 --2- 172.21.15.19:0/1351952392 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8ffc1034c0 0x7f8ffc19cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:58.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.905+0000 7f9002200700 1 --2- 172.21.15.19:0/1351952392 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f8fe00b29f0 0x7f8fe00b4f00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:58.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.905+0000 7f9002200700 1 --2- 172.21.15.19:0/1351952392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ffc107df0 0x7f8ffc19d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:35:58.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.905+0000 7f9002200700 1 -- 172.21.15.19:0/1351952392 >> 172.21.15.19:0/1351952392 conn(0x7f8ffc0fedc0 msgr2=0x7f8ffc107610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:35:58.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.905+0000 7f9002200700 1 -- 172.21.15.19:0/1351952392 shutdown_connections 2024-05-07T02:35:58.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:35:58.905+0000 7f9002200700 1 -- 172.21.15.19:0/1351952392 wait complete. 2024-05-07T02:35:59.742 INFO:tasks.workunit.client.0.smithi019.stdout: 1 95608 2.15 MB/sec execute 55 sec latency 1943.942 ms 2024-05-07T02:36:00.107 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-07T02:36:00.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:00 smithi064 ceph-mon[34699]: pgmap v128: 129 pgs: 129 active+clean; 68 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 1.5 MiB/s wr, 71 op/s 2024-05-07T02:36:00.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:00 smithi019 ceph-mon[29781]: pgmap v128: 129 pgs: 129 active+clean; 68 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 1.5 MiB/s wr, 71 op/s 2024-05-07T02:36:00.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:00 smithi027 ceph-mon[34398]: pgmap v128: 129 pgs: 129 active+clean; 68 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 1.5 MiB/s wr, 71 op/s 2024-05-07T02:36:00.618 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:36:00.742 INFO:tasks.workunit.client.0.smithi019.stdout: 1 96260 2.19 MB/sec execute 56 sec latency 17.943 ms 2024-05-07T02:36:01.743 INFO:tasks.workunit.client.0.smithi019.stdout: 1 96832 2.18 MB/sec execute 57 sec latency 70.597 ms 2024-05-07T02:36:02.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:02 smithi064 ceph-mon[34699]: pgmap v129: 129 pgs: 129 active+clean; 70 MiB data, 742 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 1.7 MiB/s wr, 115 op/s 2024-05-07T02:36:02.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:02 smithi019 ceph-mon[29781]: pgmap v129: 129 pgs: 129 active+clean; 70 MiB data, 742 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 1.7 MiB/s wr, 115 op/s 2024-05-07T02:36:02.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:02 smithi027 ceph-mon[34398]: pgmap v129: 129 pgs: 129 active+clean; 70 MiB data, 742 MiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 1.7 MiB/s wr, 115 op/s 2024-05-07T02:36:02.742 INFO:tasks.workunit.client.0.smithi019.stdout: 1 97226 2.20 MB/sec execute 58 sec latency 107.584 ms 2024-05-07T02:36:03.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.239+0000 7f1d9ee00700 1 -- 172.21.15.19:0/98448745 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d981034c0 msgr2=0x7f1d981038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:03.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.239+0000 7f1d9ee00700 1 --2- 172.21.15.19:0/98448745 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d981034c0 0x7f1d981038d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1d88009990 tx=0x7f1d88009ca0 comp rx=0 tx=0).stop 2024-05-07T02:36:03.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.239+0000 7f1d9ee00700 1 -- 172.21.15.19:0/98448745 shutdown_connections 2024-05-07T02:36:03.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.239+0000 7f1d9ee00700 1 --2- 172.21.15.19:0/98448745 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d98108740 0x7f1d981100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:03.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.239+0000 7f1d9ee00700 1 --2- 172.21.15.19:0/98448745 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d981034c0 0x7f1d981038d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:03.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.239+0000 7f1d9ee00700 1 --2- 172.21.15.19:0/98448745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d98107df0 0x7f1d98108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:03.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.239+0000 7f1d9ee00700 1 -- 172.21.15.19:0/98448745 >> 172.21.15.19:0/98448745 conn(0x7f1d980fedc0 msgr2=0x7f1d981011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:36:03.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.240+0000 7f1d9ee00700 1 -- 172.21.15.19:0/98448745 shutdown_connections 2024-05-07T02:36:03.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.240+0000 7f1d9ee00700 1 -- 172.21.15.19:0/98448745 wait complete. 2024-05-07T02:36:03.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.242+0000 7f1d9ee00700 1 Processor -- start 2024-05-07T02:36:03.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.242+0000 7f1d9ee00700 1 -- start start 2024-05-07T02:36:03.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.242+0000 7f1d9ee00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d981034c0 0x7f1d9819d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:03.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.242+0000 7f1d9ee00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d98107df0 0x7f1d9819d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:03.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.243+0000 7f1d9ee00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d98108740 0x7f1d981a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:03.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.243+0000 7f1d9ee00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1d98114510 con 0x7f1d981034c0 2024-05-07T02:36:03.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.243+0000 7f1d9ee00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1d98114390 con 0x7f1d98107df0 2024-05-07T02:36:03.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.243+0000 7f1d9ee00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1d98114690 con 0x7f1d98108740 2024-05-07T02:36:03.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.243+0000 7f1d97e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d981034c0 0x7f1d9819d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:03.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.243+0000 7f1d8fe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d98107df0 0x7f1d9819d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:03.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.243+0000 7f1d97e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d981034c0 0x7f1d9819d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58746/0 (socket says 172.21.15.19:58746) 2024-05-07T02:36:03.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.243+0000 7f1d97e00700 1 -- 172.21.15.19:0/374115335 learned_addr learned my addr 172.21.15.19:0/374115335 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:36:03.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.243+0000 7f1d9ce00700 1 --2- 172.21.15.19:0/374115335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d98108740 0x7f1d981a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:03.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.244+0000 7f1d97e00700 1 -- 172.21.15.19:0/374115335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d98108740 msgr2=0x7f1d981a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:03.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.244+0000 7f1d97e00700 1 --2- 172.21.15.19:0/374115335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d98108740 0x7f1d981a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:03.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.244+0000 7f1d97e00700 1 -- 172.21.15.19:0/374115335 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d98107df0 msgr2=0x7f1d9819d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:03.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.244+0000 7f1d97e00700 1 --2- 172.21.15.19:0/374115335 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d98107df0 0x7f1d9819d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:03.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.244+0000 7f1d97e00700 1 -- 172.21.15.19:0/374115335 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d981a1f90 con 0x7f1d981034c0 2024-05-07T02:36:03.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.244+0000 7f1d8fe00700 1 --2- 172.21.15.19:0/374115335 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d98107df0 0x7f1d9819d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:36:03.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.244+0000 7f1d97e00700 1 --2- 172.21.15.19:0/374115335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d981034c0 0x7f1d9819d0c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1d88000c00 tx=0x7f1d880057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:36:03.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.245+0000 7f1d95600700 1 -- 172.21.15.19:0/374115335 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d88027070 con 0x7f1d981034c0 2024-05-07T02:36:03.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.245+0000 7f1d9ee00700 1 -- 172.21.15.19:0/374115335 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d981a2220 con 0x7f1d981034c0 2024-05-07T02:36:03.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.245+0000 7f1d95600700 1 -- 172.21.15.19:0/374115335 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d8800b610 con 0x7f1d981034c0 2024-05-07T02:36:03.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.245+0000 7f1d95600700 1 -- 172.21.15.19:0/374115335 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d880173f0 con 0x7f1d981034c0 2024-05-07T02:36:03.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.245+0000 7f1d9ee00700 1 -- 172.21.15.19:0/374115335 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d981a2700 con 0x7f1d981034c0 2024-05-07T02:36:03.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.249+0000 7f1d95600700 1 -- 172.21.15.19:0/374115335 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1d8802a070 con 0x7f1d981034c0 2024-05-07T02:36:03.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.251+0000 7f1dad8f1b80 1 Processor -- start 2024-05-07T02:36:03.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.251+0000 7f1d95600700 1 --2- 172.21.15.19:0/374115335 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f1d7c0b29f0 0x7f1d7c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:03.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.251+0000 7f1dad8f1b80 1 -- start start 2024-05-07T02:36:03.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.251+0000 7f1d95600700 1 -- 172.21.15.19:0/374115335 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f1d880e38f0 con 0x7f1d981034c0 2024-05-07T02:36:03.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.251+0000 7f1d9ce00700 1 --2- 172.21.15.19:0/374115335 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f1d7c0b29f0 0x7f1d7c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:03.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.252+0000 7f1dad8f1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f7bcb73b0 0x563f7bcb77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:03.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.252+0000 7f1dad8f1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f7bcb1470 0x563f7bcb1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:03.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.252+0000 7f1dad8f1b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563f7bcb1dc0 0x563f7bcbe040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:03.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.252+0000 7f1dad8f1b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563f7bbd3a20 con 0x563f7bcb73b0 2024-05-07T02:36:03.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.252+0000 7f1dad8f1b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x563f7bb71760 con 0x563f7bcb1dc0 2024-05-07T02:36:03.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.252+0000 7f1dad8f1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563f7bbcf9e0 con 0x563f7bcb1470 2024-05-07T02:36:03.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.252+0000 7f1d8fe00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f7bcb1470 0x563f7bcb1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:03.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.252+0000 7f1d8fe00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f7bcb1470 0x563f7bcb1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:34320/0 (socket says 172.21.15.19:34320) 2024-05-07T02:36:03.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.252+0000 7f1d8fe00700 1 -- 172.21.15.19:0/982858934 learned_addr learned my addr 172.21.15.19:0/982858934 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:36:03.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.252+0000 7f1d97e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f7bcb73b0 0x563f7bcb77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:03.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.252+0000 7f1d9ce00700 1 --2- 172.21.15.19:0/982858934 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563f7bcb1dc0 0x563f7bcbe040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:03.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.252+0000 7f1d8fe00700 1 -- 172.21.15.19:0/982858934 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563f7bcb1dc0 msgr2=0x563f7bcbe040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:36:03.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.252+0000 7f1d8fe00700 1 --2- 172.21.15.19:0/982858934 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563f7bcb1dc0 0x563f7bcbe040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:03.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.252+0000 7f1d9ce00700 1 --2- 172.21.15.19:0/374115335 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f1d7c0b29f0 0x7f1d7c0b4f00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1d9000f520 tx=0x7f1d9000b040 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:36:03.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.252+0000 7f1d8fe00700 1 -- 172.21.15.19:0/982858934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f7bcb73b0 msgr2=0x563f7bcb77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:36:03.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.252+0000 7f1d8fe00700 1 --2- 172.21.15.19:0/982858934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f7bcb73b0 0x563f7bcb77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:03.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.253+0000 7f1d8fe00700 1 -- 172.21.15.19:0/982858934 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f7bb208d0 con 0x563f7bcb1470 2024-05-07T02:36:03.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.253+0000 7f1d8fe00700 1 --2- 172.21.15.19:0/982858934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f7bcb1470 0x563f7bcb1880 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1d80011540 tx=0x7f1d80011850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc630a1f42d22c19 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:36:03.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.253+0000 7f1d8d600700 1 -- 172.21.15.19:0/982858934 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d80004a30 con 0x563f7bcb1470 2024-05-07T02:36:03.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.253+0000 7f1d8d600700 1 -- 172.21.15.19:0/982858934 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d800054d0 con 0x563f7bcb1470 2024-05-07T02:36:03.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.253+0000 7f1d8d600700 1 -- 172.21.15.19:0/982858934 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d8000e860 con 0x563f7bcb1470 2024-05-07T02:36:03.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.254+0000 7f1dad8f1b80 1 -- 172.21.15.19:0/982858934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f7bcb1470 msgr2=0x563f7bcb1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:03.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.254+0000 7f1dad8f1b80 1 --2- 172.21.15.19:0/982858934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f7bcb1470 0x563f7bcb1880 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1d80011540 tx=0x7f1d80011850 comp rx=0 tx=0).stop 2024-05-07T02:36:03.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.254+0000 7f1dad8f1b80 1 -- 172.21.15.19:0/982858934 shutdown_connections 2024-05-07T02:36:03.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.254+0000 7f1dad8f1b80 1 --2- 172.21.15.19:0/982858934 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563f7bcb1dc0 0x563f7bcbe040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:03.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.254+0000 7f1dad8f1b80 1 --2- 172.21.15.19:0/982858934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f7bcb73b0 0x563f7bcb77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:03.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.254+0000 7f1dad8f1b80 1 --2- 172.21.15.19:0/982858934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f7bcb1470 0x563f7bcb1880 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:03.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.254+0000 7f1dad8f1b80 1 -- 172.21.15.19:0/982858934 >> 172.21.15.19:0/982858934 conn(0x563f7bb33e40 msgr2=0x563f7bcac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:36:03.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.255+0000 7f1dad8f1b80 1 -- 172.21.15.19:0/982858934 shutdown_connections 2024-05-07T02:36:03.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.255+0000 7f1dad8f1b80 1 -- 172.21.15.19:0/982858934 wait complete. 2024-05-07T02:36:03.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.255+0000 7f1dad8f1b80 1 -- start start 2024-05-07T02:36:03.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.255+0000 7f1dad8f1b80 1 Processor -- start 2024-05-07T02:36:03.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.256+0000 7f1dad8f1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f7bcb1470 0x563f7bcbb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:03.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.256+0000 7f1dad8f1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f7bcb1dc0 0x563f7bcbba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:03.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.256+0000 7f1dad8f1b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563f7bcb73b0 0x563f7bcbbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:03.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.256+0000 7f1dad8f1b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563f7bcbe7c0 con 0x563f7bcb1470 2024-05-07T02:36:03.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.256+0000 7f1dad8f1b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x563f7bb39b20 con 0x563f7bcb73b0 2024-05-07T02:36:03.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.256+0000 7f1dad8f1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563f7bcbe940 con 0x563f7bcb1dc0 2024-05-07T02:36:03.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.256+0000 7f1dad8f1b80 10 client.0 _ll_register_callbacks cb 0x563f7bb7ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:36:03.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.256+0000 7f1d9ce00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563f7bcb73b0 0x563f7bcbbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:03.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.256+0000 7f1d9ce00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563f7bcb73b0 0x563f7bcbbfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:53956/0 (socket says 172.21.15.19:53956) 2024-05-07T02:36:03.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.256+0000 7f1d9ce00700 1 -- 172.21.15.19:0/4168874955 learned_addr learned my addr 172.21.15.19:0/4168874955 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:36:03.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.256+0000 7f1d97e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f7bcb1470 0x563f7bcbb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:03.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.256+0000 7f1d8fe00700 1 --2- 172.21.15.19:0/4168874955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f7bcb1dc0 0x563f7bcbba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:03.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.257+0000 7f1d9ce00700 1 -- 172.21.15.19:0/4168874955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f7bcb1dc0 msgr2=0x563f7bcbba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:36:03.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.257+0000 7f1d9ce00700 1 --2- 172.21.15.19:0/4168874955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f7bcb1dc0 0x563f7bcbba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:03.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.257+0000 7f1d9ce00700 1 -- 172.21.15.19:0/4168874955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f7bcb1470 msgr2=0x563f7bcbb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:36:03.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.257+0000 7f1d9ce00700 1 --2- 172.21.15.19:0/4168874955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f7bcb1470 0x563f7bcbb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:03.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.257+0000 7f1d9ce00700 1 -- 172.21.15.19:0/4168874955 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f7bb99420 con 0x563f7bcb73b0 2024-05-07T02:36:03.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.257+0000 7f1d9ce00700 1 --2- 172.21.15.19:0/4168874955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563f7bcb73b0 0x563f7bcbbfd0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1d90008ac0 tx=0x7f1d90004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2df6f74429ebf81a server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:36:03.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.257+0000 7f1d86000700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:36:03.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.257+0000 7f1d86000700 1 -- 172.21.15.19:0/4168874955 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d90026020 con 0x563f7bcb73b0 2024-05-07T02:36:03.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.258+0000 7f1dad8f1b80 1 -- 172.21.15.19:0/4168874955 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d800054d0 con 0x563f7bcb73b0 2024-05-07T02:36:03.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.258+0000 7f1d86000700 1 -- 172.21.15.19:0/4168874955 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f1d90017ad0 con 0x563f7bcb73b0 2024-05-07T02:36:03.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.258+0000 7f1d86000700 1 -- 172.21.15.19:0/4168874955 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f1d90031050 con 0x563f7bcb73b0 2024-05-07T02:36:03.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.258+0000 7f1dad8f1b80 10 client.25952 fetch_fsmap learned FSMap version 40 2024-05-07T02:36:03.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.258+0000 7f1dad8f1b80 1 -- 172.21.15.19:0/4168874955 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x563f7bcc94a0 con 0x563f7bcb73b0 2024-05-07T02:36:03.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.258+0000 7f1d86000700 1 -- 172.21.15.19:0/4168874955 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d800054d0 con 0x563f7bcb73b0 2024-05-07T02:36:03.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.259+0000 7f1d86000700 1 -- 172.21.15.19:0/4168874955 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f1d90015410 con 0x563f7bcb73b0 2024-05-07T02:36:03.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.259+0000 7f1dad8f1b80 10 client.25952 fetch_fsmap finished waiting for FSMap version 40 2024-05-07T02:36:03.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.259+0000 7f1dad8f1b80 10 client.25952 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-07T02:36:03.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.259+0000 7f1dad8f1b80 20 client.25952 populate_metadata read hostname 'smithi019' 2024-05-07T02:36:03.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.259+0000 7f1dad8f1b80 1 --2- 172.21.15.19:0/4168874955 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x563f7bccbd50 0x563f7bcce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:03.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.259+0000 7f1dad8f1b80 4 client.25952 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:36:03.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.259+0000 7f1dad8f1b80 1 -- 172.21.15.19:0/4168874955 --> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f7bc7e2e0 con 0x563f7bccbd50 2024-05-07T02:36:03.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.259+0000 7f1d97e00700 1 --2- 172.21.15.19:0/4168874955 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x563f7bccbd50 0x563f7bcce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:03.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.260+0000 7f1d97e00700 1 --2- 172.21.15.19:0/4168874955 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x563f7bccbd50 0x563f7bcce170 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bb6b53ce18ebb2a1 server_cookie=e0cde682587d220 in_seq=0 out_seq=0 2024-05-07T02:36:03.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.260+0000 7f1d86000700 10 client.25952 ms_handle_connect on v2:172.21.15.19:6840/3905645246 2024-05-07T02:36:03.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.261+0000 7f1d86000700 1 -- 172.21.15.19:0/4168874955 <== mds.1 v2:172.21.15.19:6840/3905645246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563f7bc7e2e0 con 0x563f7bccbd50 2024-05-07T02:36:03.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.261+0000 7f1d86000700 10 client.25952 handle_command_reply: tid=0 2024-05-07T02:36:03.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.261+0000 7f1dad8f1b80 1 client.25952 shutdown 2024-05-07T02:36:03.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.261+0000 7f1dad8f1b80 1 -- 172.21.15.19:0/4168874955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563f7bcb73b0 msgr2=0x563f7bcbbfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:03.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.261+0000 7f1dad8f1b80 1 --2- 172.21.15.19:0/4168874955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563f7bcb73b0 0x563f7bcbbfd0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1d90008ac0 tx=0x7f1d90004ad0 comp rx=0 tx=0).stop 2024-05-07T02:36:03.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.261+0000 7f1dad8f1b80 1 -- 172.21.15.19:0/4168874955 shutdown_connections 2024-05-07T02:36:03.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.261+0000 7f1dad8f1b80 1 --2- 172.21.15.19:0/4168874955 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x563f7bccbd50 0x563f7bcce170 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:03.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.261+0000 7f1dad8f1b80 1 --2- 172.21.15.19:0/4168874955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563f7bcb73b0 0x563f7bcbbfd0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:03.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.261+0000 7f1dad8f1b80 1 --2- 172.21.15.19:0/4168874955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f7bcb1470 0x563f7bcbb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:03.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.261+0000 7f1dad8f1b80 1 --2- 172.21.15.19:0/4168874955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f7bcb1dc0 0x563f7bcbba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:03.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.261+0000 7f1d86000700 0 client.25952 ms_handle_reset on v2:172.21.15.19:6840/3905645246 2024-05-07T02:36:03.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.261+0000 7f1dad8f1b80 1 -- 172.21.15.19:0/4168874955 >> 172.21.15.19:0/4168874955 conn(0x563f7bb33e40 msgr2=0x563f7bcb0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:36:03.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.262+0000 7f1dad8f1b80 1 -- 172.21.15.19:0/4168874955 shutdown_connections 2024-05-07T02:36:03.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.262+0000 7f1dad8f1b80 1 -- 172.21.15.19:0/4168874955 wait complete. 2024-05-07T02:36:03.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.262+0000 7f1dad8f1b80 20 client.25952 trim_cache size 0 max 16384 2024-05-07T02:36:03.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.280+0000 7f1dad8f1b80 1 Processor -- start 2024-05-07T02:36:03.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.280+0000 7f1dad8f1b80 1 -- start start 2024-05-07T02:36:03.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.280+0000 7f1dad8f1b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563f7bcb1470 0x563f7bcc7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:03.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.281+0000 7f1dad8f1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f7bcb1dc0 0x563f7bca6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:03.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.281+0000 7f1dad8f1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f7bcb73b0 0x563f7bcb8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:03.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.281+0000 7f1dad8f1b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1d981b26a0 con 0x563f7bcb1dc0 2024-05-07T02:36:03.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.281+0000 7f1dad8f1b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1d981b4240 con 0x563f7bcb1470 2024-05-07T02:36:03.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.281+0000 7f1dad8f1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1d981b2d70 con 0x563f7bcb73b0 2024-05-07T02:36:03.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.281+0000 7f1d9ce00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f7bcb73b0 0x563f7bcb8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:03.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.281+0000 7f1d9ce00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f7bcb73b0 0x563f7bcb8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:34334/0 (socket says 172.21.15.19:34334) 2024-05-07T02:36:03.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.281+0000 7f1d9ce00700 1 -- 172.21.15.19:0/589145808 learned_addr learned my addr 172.21.15.19:0/589145808 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:36:03.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.281+0000 7f1d8fe00700 1 --2- 172.21.15.19:0/589145808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f7bcb1dc0 0x563f7bca6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:03.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.281+0000 7f1d97e00700 1 --2- 172.21.15.19:0/589145808 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563f7bcb1470 0x563f7bcc7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:03.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.281+0000 7f1d9ce00700 1 -- 172.21.15.19:0/589145808 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563f7bcb1470 msgr2=0x563f7bcc7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:36:03.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.281+0000 7f1d9ce00700 1 --2- 172.21.15.19:0/589145808 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563f7bcb1470 0x563f7bcc7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:03.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.281+0000 7f1d9ce00700 1 -- 172.21.15.19:0/589145808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f7bcb1dc0 msgr2=0x563f7bca6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:36:03.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.281+0000 7f1d9ce00700 1 --2- 172.21.15.19:0/589145808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f7bcb1dc0 0x563f7bca6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:03.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.281+0000 7f1d9ce00700 1 -- 172.21.15.19:0/589145808 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f7bcb21d0 con 0x563f7bcb73b0 2024-05-07T02:36:03.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.282+0000 7f1d9ce00700 1 --2- 172.21.15.19:0/589145808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f7bcb73b0 0x563f7bcb8e50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1d90015830 tx=0x7f1d90009460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99306c1e8f5544fe server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:36:03.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.282+0000 7f1d8cc00700 1 -- 172.21.15.19:0/589145808 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d90026040 con 0x563f7bcb73b0 2024-05-07T02:36:03.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.282+0000 7f1d8cc00700 1 -- 172.21.15.19:0/589145808 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d90015610 con 0x563f7bcb73b0 2024-05-07T02:36:03.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.282+0000 7f1d8cc00700 1 -- 172.21.15.19:0/589145808 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9001b070 con 0x563f7bcb73b0 2024-05-07T02:36:03.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.283+0000 7f1dad8f1b80 1 -- 172.21.15.19:0/589145808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f7bcb73b0 msgr2=0x563f7bcb8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:03.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.283+0000 7f1dad8f1b80 1 --2- 172.21.15.19:0/589145808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f7bcb73b0 0x563f7bcb8e50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1d90015830 tx=0x7f1d90009460 comp rx=0 tx=0).stop 2024-05-07T02:36:03.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.283+0000 7f1dad8f1b80 1 -- 172.21.15.19:0/589145808 shutdown_connections 2024-05-07T02:36:03.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.283+0000 7f1dad8f1b80 1 --2- 172.21.15.19:0/589145808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f7bcb73b0 0x563f7bcb8e50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:03.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.283+0000 7f1dad8f1b80 1 --2- 172.21.15.19:0/589145808 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563f7bcb1470 0x563f7bcc7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:03.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.283+0000 7f1dad8f1b80 1 --2- 172.21.15.19:0/589145808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f7bcb1dc0 0x563f7bca6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:03.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.283+0000 7f1dad8f1b80 1 -- 172.21.15.19:0/589145808 >> 172.21.15.19:0/589145808 conn(0x563f7bb33e40 msgr2=0x563f7bca6120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:36:03.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.283+0000 7f1dad8f1b80 1 -- 172.21.15.19:0/589145808 shutdown_connections 2024-05-07T02:36:03.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.283+0000 7f1dad8f1b80 1 -- 172.21.15.19:0/589145808 wait complete. 2024-05-07T02:36:03.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.284+0000 7f1dad8f1b80 1 -- start start 2024-05-07T02:36:03.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.284+0000 7f1dad8f1b80 1 Processor -- start 2024-05-07T02:36:03.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.285+0000 7f1dad8f1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f7bcb1470 0x563f7bcd18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:03.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.285+0000 7f1dad8f1b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563f7bcb1dc0 0x563f7bccec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:03.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.285+0000 7f1dad8f1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f7bcb73b0 0x563f7bccf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:03.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.285+0000 7f1dad8f1b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1d981b3b70 con 0x563f7bcb73b0 2024-05-07T02:36:03.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.285+0000 7f1dad8f1b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1d981b34a0 con 0x563f7bcb1dc0 2024-05-07T02:36:03.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.285+0000 7f1dad8f1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1d981b0640 con 0x563f7bcb1470 2024-05-07T02:36:03.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.285+0000 7f1dad8f1b80 10 client.0 _ll_register_callbacks cb 0x563f7bb7ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:36:03.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.285+0000 7f1d97e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f7bcb1470 0x563f7bcd18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:03.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.285+0000 7f1d8fe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563f7bcb1dc0 0x563f7bccec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:03.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.285+0000 7f1d97e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f7bcb1470 0x563f7bcd18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:34344/0 (socket says 172.21.15.19:34344) 2024-05-07T02:36:03.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.285+0000 7f1d97e00700 1 -- 172.21.15.19:0/1111623361 learned_addr learned my addr 172.21.15.19:0/1111623361 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:36:03.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.285+0000 7f1d9ce00700 1 --2- 172.21.15.19:0/1111623361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f7bcb73b0 0x563f7bccf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:03.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.285+0000 7f1d97e00700 1 -- 172.21.15.19:0/1111623361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563f7bcb1dc0 msgr2=0x563f7bccec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:36:03.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.285+0000 7f1d97e00700 1 --2- 172.21.15.19:0/1111623361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563f7bcb1dc0 0x563f7bccec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:03.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.285+0000 7f1d97e00700 1 -- 172.21.15.19:0/1111623361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f7bcb73b0 msgr2=0x563f7bccf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:36:03.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.285+0000 7f1d97e00700 1 --2- 172.21.15.19:0/1111623361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f7bcb73b0 0x563f7bccf2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:03.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.285+0000 7f1d97e00700 1 -- 172.21.15.19:0/1111623361 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f7bd352a0 con 0x563f7bcb1470 2024-05-07T02:36:03.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.286+0000 7f1d97e00700 1 --2- 172.21.15.19:0/1111623361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f7bcb1470 0x563f7bcd18b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1d880e0cb0 tx=0x7f1d88025000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f5ecf32557210f server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:36:03.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.286+0000 7f1d86000700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:36:03.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.286+0000 7f1d86000700 1 -- 172.21.15.19:0/1111623361 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d880c4060 con 0x563f7bcb1470 2024-05-07T02:36:03.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.286+0000 7f1d86000700 1 -- 172.21.15.19:0/1111623361 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f1d88022600 con 0x563f7bcb1470 2024-05-07T02:36:03.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.286+0000 7f1dad8f1b80 1 -- 172.21.15.19:0/1111623361 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d90015610 con 0x563f7bcb1470 2024-05-07T02:36:03.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.286+0000 7f1d86000700 1 -- 172.21.15.19:0/1111623361 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f1d880aaf70 con 0x563f7bcb1470 2024-05-07T02:36:03.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.286+0000 7f1dad8f1b80 10 client.25936 fetch_fsmap learned FSMap version 40 2024-05-07T02:36:03.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.286+0000 7f1dad8f1b80 1 -- 172.21.15.19:0/1111623361 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x563f7bd363c0 con 0x563f7bcb1470 2024-05-07T02:36:03.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.286+0000 7f1d86000700 1 -- 172.21.15.19:0/1111623361 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d90015610 con 0x563f7bcb1470 2024-05-07T02:36:03.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.287+0000 7f1d86000700 1 -- 172.21.15.19:0/1111623361 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f1d880aaad0 con 0x563f7bcb1470 2024-05-07T02:36:03.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.287+0000 7f1dad8f1b80 10 client.25936 fetch_fsmap finished waiting for FSMap version 40 2024-05-07T02:36:03.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.287+0000 7f1dad8f1b80 10 client.25936 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-07T02:36:03.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.287+0000 7f1dad8f1b80 20 client.25936 populate_metadata read hostname 'smithi019' 2024-05-07T02:36:03.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.287+0000 7f1dad8f1b80 1 --2- 172.21.15.19:0/1111623361 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x563f7bd36a20 0x563f7bd38e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:03.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.287+0000 7f1dad8f1b80 4 client.25936 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:36:03.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.287+0000 7f1dad8f1b80 1 -- 172.21.15.19:0/1111623361 --> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563f7bbe1400 con 0x563f7bd36a20 2024-05-07T02:36:03.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.287+0000 7f1d8fe00700 1 --2- 172.21.15.19:0/1111623361 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x563f7bd36a20 0x563f7bd38e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:03.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.288+0000 7f1d8fe00700 1 --2- 172.21.15.19:0/1111623361 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x563f7bd36a20 0x563f7bd38e40 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6dbd122a78f4b898 server_cookie=85a80cbda11afbeb in_seq=0 out_seq=0 2024-05-07T02:36:03.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.288+0000 7f1d86000700 10 client.25936 ms_handle_connect on v2:172.21.15.19:6840/3905645246 2024-05-07T02:36:03.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.288+0000 7f1d86000700 1 -- 172.21.15.19:0/1111623361 <== mds.1 v2:172.21.15.19:6840/3905645246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563f7bbe1400 con 0x563f7bd36a20 2024-05-07T02:36:03.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.288+0000 7f1d86000700 10 client.25936 handle_command_reply: tid=0 2024-05-07T02:36:03.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.288+0000 7f1dad8f1b80 1 client.25936 shutdown 2024-05-07T02:36:03.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.288+0000 7f1dad8f1b80 1 -- 172.21.15.19:0/1111623361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f7bcb1470 msgr2=0x563f7bcd18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:03.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.288+0000 7f1dad8f1b80 1 --2- 172.21.15.19:0/1111623361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f7bcb1470 0x563f7bcd18b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1d880e0cb0 tx=0x7f1d88025000 comp rx=0 tx=0).stop 2024-05-07T02:36:03.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.289+0000 7f1dad8f1b80 1 -- 172.21.15.19:0/1111623361 shutdown_connections 2024-05-07T02:36:03.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.289+0000 7f1dad8f1b80 1 --2- 172.21.15.19:0/1111623361 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x563f7bd36a20 0x563f7bd38e40 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:03.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.289+0000 7f1dad8f1b80 1 --2- 172.21.15.19:0/1111623361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563f7bcb73b0 0x563f7bccf2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:03.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.289+0000 7f1dad8f1b80 1 --2- 172.21.15.19:0/1111623361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f7bcb1470 0x563f7bcd18b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:03.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.289+0000 7f1dad8f1b80 1 --2- 172.21.15.19:0/1111623361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x563f7bcb1dc0 0x563f7bccec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:03.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.289+0000 7f1d86000700 0 client.25936 ms_handle_reset on v2:172.21.15.19:6840/3905645246 2024-05-07T02:36:03.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.289+0000 7f1dad8f1b80 1 -- 172.21.15.19:0/1111623361 >> 172.21.15.19:0/1111623361 conn(0x563f7bb33e40 msgr2=0x563f7bcb0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:36:03.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.289+0000 7f1dad8f1b80 1 -- 172.21.15.19:0/1111623361 shutdown_connections 2024-05-07T02:36:03.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.289+0000 7f1dad8f1b80 1 -- 172.21.15.19:0/1111623361 wait complete. 2024-05-07T02:36:03.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.289+0000 7f1dad8f1b80 20 client.25936 trim_cache size 0 max 16384 2024-05-07T02:36:03.291 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:36:03.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.290+0000 7f1d9ee00700 1 -- 172.21.15.19:0/374115335 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f1d7c0b29f0 msgr2=0x7f1d7c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:03.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.290+0000 7f1d9ee00700 1 --2- 172.21.15.19:0/374115335 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f1d7c0b29f0 0x7f1d7c0b4f00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1d9000f520 tx=0x7f1d9000b040 comp rx=0 tx=0).stop 2024-05-07T02:36:03.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.290+0000 7f1d9ee00700 1 -- 172.21.15.19:0/374115335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d981034c0 msgr2=0x7f1d9819d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:03.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.290+0000 7f1d9ee00700 1 --2- 172.21.15.19:0/374115335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d981034c0 0x7f1d9819d0c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1d88000c00 tx=0x7f1d880057c0 comp rx=0 tx=0).stop 2024-05-07T02:36:03.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.290+0000 7f1d9ee00700 1 -- 172.21.15.19:0/374115335 shutdown_connections 2024-05-07T02:36:03.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.290+0000 7f1d9ee00700 1 --2- 172.21.15.19:0/374115335 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1d98108740 0x7f1d981a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:03.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.290+0000 7f1d9ee00700 1 --2- 172.21.15.19:0/374115335 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f1d7c0b29f0 0x7f1d7c0b4f00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:03.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.290+0000 7f1d9ee00700 1 --2- 172.21.15.19:0/374115335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d981034c0 0x7f1d9819d0c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:03.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.290+0000 7f1d9ee00700 1 --2- 172.21.15.19:0/374115335 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d98107df0 0x7f1d9819d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:03.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.290+0000 7f1d9ee00700 1 -- 172.21.15.19:0/374115335 >> 172.21.15.19:0/374115335 conn(0x7f1d980fedc0 msgr2=0x7f1d9810f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:36:03.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.290+0000 7f1d9ee00700 1 -- 172.21.15.19:0/374115335 shutdown_connections 2024-05-07T02:36:03.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:03.290+0000 7f1d9ee00700 1 -- 172.21.15.19:0/374115335 wait complete. 2024-05-07T02:36:03.743 INFO:tasks.workunit.client.0.smithi019.stdout: 1 98577 2.27 MB/sec execute 59 sec latency 247.668 ms 2024-05-07T02:36:04.346 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:04 smithi019 ceph-mon[29781]: pgmap v130: 129 pgs: 129 active+clean; 70 MiB data, 742 MiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 303 KiB/s wr, 69 op/s 2024-05-07T02:36:04.349 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-07T02:36:04.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:04 smithi064 ceph-mon[34699]: pgmap v130: 129 pgs: 129 active+clean; 70 MiB data, 742 MiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 303 KiB/s wr, 69 op/s 2024-05-07T02:36:04.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:04 smithi027 ceph-mon[34398]: pgmap v130: 129 pgs: 129 active+clean; 70 MiB data, 742 MiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 303 KiB/s wr, 69 op/s 2024-05-07T02:36:04.743 INFO:tasks.workunit.client.0.smithi019.stdout: 1 98797 2.26 MB/sec execute 60 sec latency 1095.639 ms 2024-05-07T02:36:04.833 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:36:05.743 INFO:tasks.workunit.client.0.smithi019.stdout: 1 99732 2.33 MB/sec execute 61 sec latency 22.200 ms 2024-05-07T02:36:06.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:06 smithi064 ceph-mon[34699]: pgmap v131: 129 pgs: 129 active+clean; 71 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 653 KiB/s wr, 146 op/s 2024-05-07T02:36:06.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:06 smithi019 ceph-mon[29781]: pgmap v131: 129 pgs: 129 active+clean; 71 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 653 KiB/s wr, 146 op/s 2024-05-07T02:36:06.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:06 smithi027 ceph-mon[34398]: pgmap v131: 129 pgs: 129 active+clean; 71 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 653 KiB/s wr, 146 op/s 2024-05-07T02:36:06.743 INFO:tasks.workunit.client.0.smithi019.stdout: 1 100341 2.30 MB/sec execute 62 sec latency 20.292 ms 2024-05-07T02:36:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.585+0000 7efcc0200700 1 -- 172.21.15.19:0/806548119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcb8076c10 msgr2=0x7efcb8077020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.585+0000 7efcc0200700 1 --2- 172.21.15.19:0/806548119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcb8076c10 0x7efcb8077020 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7efcac00b0a0 tx=0x7efcac00b3b0 comp rx=0 tx=0).stop 2024-05-07T02:36:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.585+0000 7efcc0200700 1 -- 172.21.15.19:0/806548119 shutdown_connections 2024-05-07T02:36:07.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.585+0000 7efcc0200700 1 --2- 172.21.15.19:0/806548119 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcb8110be0 0x7efcb8113000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:07.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.585+0000 7efcc0200700 1 --2- 172.21.15.19:0/806548119 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcb8076c10 0x7efcb8077020 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:07.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.585+0000 7efcc0200700 1 --2- 172.21.15.19:0/806548119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcb8077560 0x7efcb81106a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:07.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.585+0000 7efcc0200700 1 -- 172.21.15.19:0/806548119 >> 172.21.15.19:0/806548119 conn(0x7efcb80ffea0 msgr2=0x7efcb81022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:36:07.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.585+0000 7efcc0200700 1 -- 172.21.15.19:0/806548119 shutdown_connections 2024-05-07T02:36:07.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.585+0000 7efcc0200700 1 -- 172.21.15.19:0/806548119 wait complete. 2024-05-07T02:36:07.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.586+0000 7efcc0200700 1 Processor -- start 2024-05-07T02:36:07.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.587+0000 7efcc0200700 1 -- start start 2024-05-07T02:36:07.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.587+0000 7efcc0200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcb8077560 0x7efcb819d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:07.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.587+0000 7efcc0200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcb8110be0 0x7efcb819d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:07.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.587+0000 7efcc0200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcb81a19b0 0x7efcb81a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:07.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.587+0000 7efcc0200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efcb8116710 con 0x7efcb8110be0 2024-05-07T02:36:07.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.587+0000 7efcc0200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efcb8116590 con 0x7efcb81a19b0 2024-05-07T02:36:07.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.587+0000 7efcc0200700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7efcb8116890 con 0x7efcb8077560 2024-05-07T02:36:07.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.588+0000 7efcbe200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcb81a19b0 0x7efcb81a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:07.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.588+0000 7efcbce00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcb8110be0 0x7efcb819d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:07.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.588+0000 7efcbe200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcb81a19b0 0x7efcb81a1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:53982/0 (socket says 172.21.15.19:53982) 2024-05-07T02:36:07.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.588+0000 7efcbd800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcb8077560 0x7efcb819d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:07.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.588+0000 7efcbe200700 1 -- 172.21.15.19:0/2153943469 learned_addr learned my addr 172.21.15.19:0/2153943469 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:36:07.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.588+0000 7efcbd800700 1 -- 172.21.15.19:0/2153943469 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcb81a19b0 msgr2=0x7efcb81a1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:07.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.588+0000 7efcbd800700 1 --2- 172.21.15.19:0/2153943469 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcb81a19b0 0x7efcb81a1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:07.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.588+0000 7efcbd800700 1 -- 172.21.15.19:0/2153943469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcb8110be0 msgr2=0x7efcb819d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:07.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.588+0000 7efcbd800700 1 --2- 172.21.15.19:0/2153943469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcb8110be0 0x7efcb819d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:07.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.588+0000 7efcbd800700 1 -- 172.21.15.19:0/2153943469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcb81a2500 con 0x7efcb8077560 2024-05-07T02:36:07.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.589+0000 7efcbd800700 1 --2- 172.21.15.19:0/2153943469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcb8077560 0x7efcb819d240 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7efcac008000 tx=0x7efcac004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:36:07.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.589+0000 7efcb6000700 1 -- 172.21.15.19:0/2153943469 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcac022070 con 0x7efcb8077560 2024-05-07T02:36:07.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.589+0000 7efcb6000700 1 -- 172.21.15.19:0/2153943469 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcac007e30 con 0x7efcb8077560 2024-05-07T02:36:07.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.589+0000 7efcc0200700 1 -- 172.21.15.19:0/2153943469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcb81a2790 con 0x7efcb8077560 2024-05-07T02:36:07.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.589+0000 7efcb6000700 1 -- 172.21.15.19:0/2153943469 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcac0045d0 con 0x7efcb8077560 2024-05-07T02:36:07.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.589+0000 7efcc0200700 1 -- 172.21.15.19:0/2153943469 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcb81a9f40 con 0x7efcb8077560 2024-05-07T02:36:07.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.593+0000 7efcb6000700 1 -- 172.21.15.19:0/2153943469 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7efcac007980 con 0x7efcb8077560 2024-05-07T02:36:07.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.594+0000 7efccece0b80 1 Processor -- start 2024-05-07T02:36:07.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.594+0000 7efccece0b80 1 -- start start 2024-05-07T02:36:07.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.595+0000 7efccece0b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560a269fc3b0 0x560a269fc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:07.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.595+0000 7efccece0b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a269f6470 0x560a269f6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:07.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.595+0000 7efccece0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a269f6dc0 0x560a26a03040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:07.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.595+0000 7efccece0b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560a26918a20 con 0x560a269f6dc0 2024-05-07T02:36:07.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.595+0000 7efccece0b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x560a268b6760 con 0x560a269fc3b0 2024-05-07T02:36:07.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.595+0000 7efccece0b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560a269149e0 con 0x560a269f6470 2024-05-07T02:36:07.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.595+0000 7efcbe200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560a269fc3b0 0x560a269fc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:07.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.595+0000 7efcbe200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560a269fc3b0 0x560a269fc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:53988/0 (socket says 172.21.15.19:53988) 2024-05-07T02:36:07.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.595+0000 7efcbe200700 1 -- 172.21.15.19:0/83430299 learned_addr learned my addr 172.21.15.19:0/83430299 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:36:07.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.595+0000 7efcbd800700 1 --2- 172.21.15.19:0/83430299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a269f6470 0x560a269f6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:07.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.595+0000 7efcbce00700 1 --2- 172.21.15.19:0/83430299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a269f6dc0 0x560a26a03040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:07.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.596+0000 7efcbe200700 1 -- 172.21.15.19:0/83430299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a269f6470 msgr2=0x560a269f6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:36:07.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.596+0000 7efcbe200700 1 --2- 172.21.15.19:0/83430299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a269f6470 0x560a269f6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:07.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.596+0000 7efcbe200700 1 -- 172.21.15.19:0/83430299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a269f6dc0 msgr2=0x560a26a03040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:36:07.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.596+0000 7efcb6000700 1 --2- 172.21.15.19:0/2153943469 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7efca00b2a70 0x7efca00b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:07.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.596+0000 7efcbe200700 1 --2- 172.21.15.19:0/83430299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a269f6dc0 0x560a26a03040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:07.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.596+0000 7efcbe200700 1 -- 172.21.15.19:0/83430299 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a268658d0 con 0x560a269fc3b0 2024-05-07T02:36:07.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.596+0000 7efcb6000700 1 -- 172.21.15.19:0/2153943469 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7efcac00f020 con 0x7efcb8077560 2024-05-07T02:36:07.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.596+0000 7efcbe200700 1 --2- 172.21.15.19:0/2153943469 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7efca00b2a70 0x7efca00b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:07.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.596+0000 7efcbe200700 1 --2- 172.21.15.19:0/83430299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560a269fc3b0 0x560a269fc7c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efcb0009d30 tx=0x7efcb0011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e59cc52218351f7d server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:36:07.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.597+0000 7efcaaa00700 1 -- 172.21.15.19:0/83430299 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb0019910 con 0x560a269fc3b0 2024-05-07T02:36:07.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.597+0000 7efcaaa00700 1 -- 172.21.15.19:0/83430299 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcb001bd40 con 0x560a269fc3b0 2024-05-07T02:36:07.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.597+0000 7efcaaa00700 1 -- 172.21.15.19:0/83430299 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb000eb10 con 0x560a269fc3b0 2024-05-07T02:36:07.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.598+0000 7efccece0b80 1 -- 172.21.15.19:0/83430299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560a269fc3b0 msgr2=0x560a269fc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:07.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.598+0000 7efccece0b80 1 --2- 172.21.15.19:0/83430299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560a269fc3b0 0x560a269fc7c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efcb0009d30 tx=0x7efcb0011720 comp rx=0 tx=0).stop 2024-05-07T02:36:07.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.598+0000 7efcbe200700 1 --2- 172.21.15.19:0/2153943469 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7efca00b2a70 0x7efca00b4f80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efcb0020000 tx=0x7efcb0021000 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:36:07.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.598+0000 7efccece0b80 1 -- 172.21.15.19:0/83430299 shutdown_connections 2024-05-07T02:36:07.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.598+0000 7efccece0b80 1 --2- 172.21.15.19:0/83430299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a269f6dc0 0x560a26a03040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:07.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.598+0000 7efccece0b80 1 --2- 172.21.15.19:0/83430299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560a269fc3b0 0x560a269fc7c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:07.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.598+0000 7efccece0b80 1 --2- 172.21.15.19:0/83430299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a269f6470 0x560a269f6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:07.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.598+0000 7efccece0b80 1 -- 172.21.15.19:0/83430299 >> 172.21.15.19:0/83430299 conn(0x560a26878e40 msgr2=0x560a269f12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:36:07.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.598+0000 7efccece0b80 1 -- 172.21.15.19:0/83430299 shutdown_connections 2024-05-07T02:36:07.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.598+0000 7efccece0b80 1 -- 172.21.15.19:0/83430299 wait complete. 2024-05-07T02:36:07.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.599+0000 7efccece0b80 1 -- start start 2024-05-07T02:36:07.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.599+0000 7efccece0b80 1 Processor -- start 2024-05-07T02:36:07.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.600+0000 7efccece0b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a269f6470 0x560a26a00590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:07.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.600+0000 7efccece0b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560a269f6dc0 0x560a26a00ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:07.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.600+0000 7efccece0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a269fc3b0 0x560a26a01010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:07.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.600+0000 7efccece0b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560a26a037c0 con 0x560a269fc3b0 2024-05-07T02:36:07.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.600+0000 7efccece0b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x560a2687eb20 con 0x560a269f6dc0 2024-05-07T02:36:07.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.600+0000 7efccece0b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560a26a03940 con 0x560a269f6470 2024-05-07T02:36:07.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.600+0000 7efccece0b80 10 client.0 _ll_register_callbacks cb 0x560a268bfc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:36:07.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.600+0000 7efcbce00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560a269f6dc0 0x560a26a00ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:07.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.600+0000 7efcbd800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a269f6470 0x560a26a00590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:07.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.600+0000 7efcbce00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560a269f6dc0 0x560a26a00ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:53998/0 (socket says 172.21.15.19:53998) 2024-05-07T02:36:07.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.600+0000 7efcbce00700 1 -- 172.21.15.19:0/3056064708 learned_addr learned my addr 172.21.15.19:0/3056064708 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:36:07.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.601+0000 7efcbe200700 1 --2- 172.21.15.19:0/3056064708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a269fc3b0 0x560a26a01010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:07.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.601+0000 7efcbce00700 1 -- 172.21.15.19:0/3056064708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a269f6470 msgr2=0x560a26a00590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:36:07.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.601+0000 7efcbce00700 1 --2- 172.21.15.19:0/3056064708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a269f6470 0x560a26a00590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:07.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.601+0000 7efcbce00700 1 -- 172.21.15.19:0/3056064708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a269fc3b0 msgr2=0x560a26a01010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:36:07.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.601+0000 7efcbce00700 1 --2- 172.21.15.19:0/3056064708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a269fc3b0 0x560a26a01010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:07.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.601+0000 7efcbce00700 1 -- 172.21.15.19:0/3056064708 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a268de420 con 0x560a269f6dc0 2024-05-07T02:36:07.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.601+0000 7efcbce00700 1 --2- 172.21.15.19:0/3056064708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560a269f6dc0 0x560a26a00ad0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7efca40116d0 tx=0x7efca40119e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3add045c9d1f9990 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:36:07.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.601+0000 7efc9f400700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:36:07.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.602+0000 7efc9f400700 1 -- 172.21.15.19:0/3056064708 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca401f060 con 0x560a269f6dc0 2024-05-07T02:36:07.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.602+0000 7efccece0b80 1 -- 172.21.15.19:0/3056064708 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcb001bd40 con 0x560a269f6dc0 2024-05-07T02:36:07.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.602+0000 7efc9f400700 1 -- 172.21.15.19:0/3056064708 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7efca4004260 con 0x560a269f6dc0 2024-05-07T02:36:07.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.602+0000 7efc9f400700 1 -- 172.21.15.19:0/3056064708 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7efca4026050 con 0x560a269f6dc0 2024-05-07T02:36:07.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.602+0000 7efccece0b80 10 client.25970 fetch_fsmap learned FSMap version 40 2024-05-07T02:36:07.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.602+0000 7efccece0b80 1 -- 172.21.15.19:0/3056064708 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x560a26a0e4f0 con 0x560a269f6dc0 2024-05-07T02:36:07.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.602+0000 7efc9f400700 1 -- 172.21.15.19:0/3056064708 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcb001bd40 con 0x560a269f6dc0 2024-05-07T02:36:07.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.603+0000 7efc9f400700 1 -- 172.21.15.19:0/3056064708 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7efca400f5f0 con 0x560a269f6dc0 2024-05-07T02:36:07.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.603+0000 7efccece0b80 10 client.25970 fetch_fsmap finished waiting for FSMap version 40 2024-05-07T02:36:07.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.603+0000 7efccece0b80 10 client.25970 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:36:07.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.603+0000 7efccece0b80 20 client.25970 populate_metadata read hostname 'smithi019' 2024-05-07T02:36:07.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.603+0000 7efccece0b80 1 --2- 172.21.15.19:0/3056064708 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x560a26a10b80 0x560a26a12fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:07.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.603+0000 7efccece0b80 4 client.25970 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:36:07.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.603+0000 7efccece0b80 1 -- 172.21.15.19:0/3056064708 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a269c32e0 con 0x560a26a10b80 2024-05-07T02:36:07.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.603+0000 7efcbd800700 1 --2- 172.21.15.19:0/3056064708 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x560a26a10b80 0x560a26a12fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:07.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.604+0000 7efc9f400700 10 client.25970 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:36:07.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.604+0000 7efcbd800700 1 --2- 172.21.15.19:0/3056064708 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x560a26a10b80 0x560a26a12fa0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca50d301575a22f3 server_cookie=fa8a7ad248e50b7a in_seq=0 out_seq=0 2024-05-07T02:36:07.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.604+0000 7efc9f400700 1 -- 172.21.15.19:0/3056064708 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560a269c32e0 con 0x560a26a10b80 2024-05-07T02:36:07.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.604+0000 7efc9f400700 10 client.25970 handle_command_reply: tid=0 2024-05-07T02:36:07.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.605+0000 7efccece0b80 1 client.25970 shutdown 2024-05-07T02:36:07.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.605+0000 7efccece0b80 1 -- 172.21.15.19:0/3056064708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560a269f6dc0 msgr2=0x560a26a00ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:07.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.605+0000 7efccece0b80 1 --2- 172.21.15.19:0/3056064708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560a269f6dc0 0x560a26a00ad0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7efca40116d0 tx=0x7efca40119e0 comp rx=0 tx=0).stop 2024-05-07T02:36:07.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.605+0000 7efccece0b80 1 -- 172.21.15.19:0/3056064708 shutdown_connections 2024-05-07T02:36:07.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.605+0000 7efccece0b80 1 --2- 172.21.15.19:0/3056064708 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x560a26a10b80 0x560a26a12fa0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:07.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.605+0000 7efccece0b80 1 --2- 172.21.15.19:0/3056064708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a269fc3b0 0x560a26a01010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:07.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.605+0000 7efc9f400700 0 client.25970 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:36:07.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.605+0000 7efccece0b80 1 --2- 172.21.15.19:0/3056064708 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a269f6470 0x560a26a00590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:07.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.605+0000 7efccece0b80 1 --2- 172.21.15.19:0/3056064708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560a269f6dc0 0x560a26a00ad0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:07.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.605+0000 7efccece0b80 1 -- 172.21.15.19:0/3056064708 >> 172.21.15.19:0/3056064708 conn(0x560a26878e40 msgr2=0x560a269f5460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:36:07.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.605+0000 7efccece0b80 1 -- 172.21.15.19:0/3056064708 shutdown_connections 2024-05-07T02:36:07.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.605+0000 7efccece0b80 1 -- 172.21.15.19:0/3056064708 wait complete. 2024-05-07T02:36:07.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.605+0000 7efccece0b80 20 client.25970 trim_cache size 0 max 16384 2024-05-07T02:36:07.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.623+0000 7efccece0b80 1 Processor -- start 2024-05-07T02:36:07.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.623+0000 7efccece0b80 1 -- start start 2024-05-07T02:36:07.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.623+0000 7efccece0b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560a269f6470 0x560a269f1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:07.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.624+0000 7efccece0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a269f6dc0 0x560a26a00e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:07.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.624+0000 7efccece0b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a269fc3b0 0x560a269fe000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:07.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.624+0000 7efccece0b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efcb81b2d80 con 0x560a269f6dc0 2024-05-07T02:36:07.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.624+0000 7efccece0b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efcb81b4920 con 0x560a269f6470 2024-05-07T02:36:07.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.624+0000 7efccece0b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7efcb81b3450 con 0x560a269fc3b0 2024-05-07T02:36:07.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.624+0000 7efcbd800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560a269f6470 0x560a269f1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:07.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.624+0000 7efcbd800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560a269f6470 0x560a269f1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:54008/0 (socket says 172.21.15.19:54008) 2024-05-07T02:36:07.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.624+0000 7efcbd800700 1 -- 172.21.15.19:0/2653615249 learned_addr learned my addr 172.21.15.19:0/2653615249 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:36:07.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.624+0000 7efcbce00700 1 --2- 172.21.15.19:0/2653615249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a269f6dc0 0x560a26a00e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:07.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.624+0000 7efcbe200700 1 --2- 172.21.15.19:0/2653615249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a269fc3b0 0x560a269fe000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:07.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.624+0000 7efcbd800700 1 -- 172.21.15.19:0/2653615249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a269fc3b0 msgr2=0x560a269fe000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:36:07.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.624+0000 7efcbd800700 1 --2- 172.21.15.19:0/2653615249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a269fc3b0 0x560a269fe000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:07.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.624+0000 7efcbd800700 1 -- 172.21.15.19:0/2653615249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a269f6dc0 msgr2=0x560a26a00e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:36:07.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.624+0000 7efcbd800700 1 --2- 172.21.15.19:0/2653615249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a269f6dc0 0x560a26a00e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:07.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.624+0000 7efcbd800700 1 -- 172.21.15.19:0/2653615249 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a269feb20 con 0x560a269f6470 2024-05-07T02:36:07.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.624+0000 7efcbce00700 1 --2- 172.21.15.19:0/2653615249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a269f6dc0 0x560a26a00e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:36:07.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.624+0000 7efcbe200700 1 --2- 172.21.15.19:0/2653615249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a269fc3b0 0x560a269fe000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-07T02:36:07.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.625+0000 7efcbd800700 1 --2- 172.21.15.19:0/2653615249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560a269f6470 0x560a269f1360 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7efcac0e05b0 tx=0x7efcac00b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=774be9d1d499cf51 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:36:07.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.625+0000 7efcaa000700 1 -- 172.21.15.19:0/2653615249 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcac01f040 con 0x560a269f6470 2024-05-07T02:36:07.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.625+0000 7efcaa000700 1 -- 172.21.15.19:0/2653615249 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcac0075c0 con 0x560a269f6470 2024-05-07T02:36:07.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.625+0000 7efcaa000700 1 -- 172.21.15.19:0/2653615249 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcac0aa870 con 0x560a269f6470 2024-05-07T02:36:07.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.626+0000 7efccece0b80 1 -- 172.21.15.19:0/2653615249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560a269f6470 msgr2=0x560a269f1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:07.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.626+0000 7efccece0b80 1 --2- 172.21.15.19:0/2653615249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560a269f6470 0x560a269f1360 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7efcac0e05b0 tx=0x7efcac00b3b0 comp rx=0 tx=0).stop 2024-05-07T02:36:07.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.626+0000 7efccece0b80 1 -- 172.21.15.19:0/2653615249 shutdown_connections 2024-05-07T02:36:07.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.626+0000 7efccece0b80 1 --2- 172.21.15.19:0/2653615249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a269fc3b0 0x560a269fe000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:07.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.626+0000 7efccece0b80 1 --2- 172.21.15.19:0/2653615249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560a269f6470 0x560a269f1360 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:07.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.626+0000 7efccece0b80 1 --2- 172.21.15.19:0/2653615249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a269f6dc0 0x560a26a00e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:07.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.626+0000 7efccece0b80 1 -- 172.21.15.19:0/2653615249 >> 172.21.15.19:0/2653615249 conn(0x560a26878e40 msgr2=0x560a269f5460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:36:07.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.626+0000 7efccece0b80 1 -- 172.21.15.19:0/2653615249 shutdown_connections 2024-05-07T02:36:07.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.626+0000 7efccece0b80 1 -- 172.21.15.19:0/2653615249 wait complete. 2024-05-07T02:36:07.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.627+0000 7efccece0b80 1 -- start start 2024-05-07T02:36:07.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.627+0000 7efccece0b80 1 Processor -- start 2024-05-07T02:36:07.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.628+0000 7efccece0b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a269f6470 0x560a26a092f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:07.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.628+0000 7efccece0b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560a269f6dc0 0x560a26a17e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:07.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.628+0000 7efccece0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a269fc3b0 0x560a26a18460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:07.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.628+0000 7efccece0b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efcb81b4250 con 0x560a269fc3b0 2024-05-07T02:36:07.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.628+0000 7efccece0b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7efcb81b3b80 con 0x560a269f6dc0 2024-05-07T02:36:07.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.628+0000 7efccece0b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7efcb81b0d20 con 0x560a269f6470 2024-05-07T02:36:07.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.628+0000 7efccece0b80 10 client.0 _ll_register_callbacks cb 0x560a268bfc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:36:07.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.628+0000 7efcbce00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560a269f6dc0 0x560a26a17e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:07.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.628+0000 7efcbd800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a269f6470 0x560a26a092f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:07.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.628+0000 7efcbce00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560a269f6dc0 0x560a26a17e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:54018/0 (socket says 172.21.15.19:54018) 2024-05-07T02:36:07.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.628+0000 7efcbce00700 1 -- 172.21.15.19:0/1484099504 learned_addr learned my addr 172.21.15.19:0/1484099504 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:36:07.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.628+0000 7efcbd800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a269f6470 0x560a26a092f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:34412/0 (socket says 172.21.15.19:34412) 2024-05-07T02:36:07.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.628+0000 7efcbe200700 1 --2- 172.21.15.19:0/1484099504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a269fc3b0 0x560a26a18460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:07.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.628+0000 7efcbce00700 1 -- 172.21.15.19:0/1484099504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a269f6470 msgr2=0x560a26a092f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:36:07.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.628+0000 7efcbce00700 1 --2- 172.21.15.19:0/1484099504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a269f6470 0x560a26a092f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:07.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.628+0000 7efcbce00700 1 -- 172.21.15.19:0/1484099504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a269fc3b0 msgr2=0x560a26a18460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:36:07.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.628+0000 7efcbce00700 1 --2- 172.21.15.19:0/1484099504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a269fc3b0 0x560a26a18460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:07.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.628+0000 7efcbce00700 1 -- 172.21.15.19:0/1484099504 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a26a18be0 con 0x560a269f6dc0 2024-05-07T02:36:07.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.628+0000 7efcbd800700 1 --2- 172.21.15.19:0/1484099504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a269f6470 0x560a26a092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:36:07.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.629+0000 7efcbce00700 1 --2- 172.21.15.19:0/1484099504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560a269f6dc0 0x560a26a17e30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7efca4000c00 tx=0x7efca400feb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=617af55a1919a640 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:36:07.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.629+0000 7efc9f400700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:36:07.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.629+0000 7efc9f400700 1 -- 172.21.15.19:0/1484099504 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca402e020 con 0x560a269f6dc0 2024-05-07T02:36:07.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.629+0000 7efccece0b80 1 -- 172.21.15.19:0/1484099504 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcac0075c0 con 0x560a269f6dc0 2024-05-07T02:36:07.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.629+0000 7efc9f400700 1 -- 172.21.15.19:0/1484099504 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7efca400f6d0 con 0x560a269f6dc0 2024-05-07T02:36:07.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.629+0000 7efc9f400700 1 -- 172.21.15.19:0/1484099504 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7efca40052e0 con 0x560a269f6dc0 2024-05-07T02:36:07.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.629+0000 7efccece0b80 10 client.25976 fetch_fsmap learned FSMap version 40 2024-05-07T02:36:07.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.629+0000 7efccece0b80 1 -- 172.21.15.19:0/1484099504 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x560a26a83ab0 con 0x560a269f6dc0 2024-05-07T02:36:07.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.630+0000 7efc9f400700 1 -- 172.21.15.19:0/1484099504 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcac0075c0 con 0x560a269f6dc0 2024-05-07T02:36:07.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.630+0000 7efc9f400700 1 -- 172.21.15.19:0/1484099504 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7efca401b450 con 0x560a269f6dc0 2024-05-07T02:36:07.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.630+0000 7efccece0b80 10 client.25976 fetch_fsmap finished waiting for FSMap version 40 2024-05-07T02:36:07.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.630+0000 7efccece0b80 10 client.25976 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:36:07.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.630+0000 7efccece0b80 20 client.25976 populate_metadata read hostname 'smithi019' 2024-05-07T02:36:07.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.630+0000 7efccece0b80 1 --2- 172.21.15.19:0/1484099504 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x560a26a84180 0x560a26a865a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:07.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.630+0000 7efccece0b80 4 client.25976 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:36:07.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.630+0000 7efccece0b80 1 -- 172.21.15.19:0/1484099504 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560a26926400 con 0x560a26a84180 2024-05-07T02:36:07.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.630+0000 7efcbd800700 1 --2- 172.21.15.19:0/1484099504 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x560a26a84180 0x560a26a865a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:07.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.631+0000 7efcbd800700 1 --2- 172.21.15.19:0/1484099504 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x560a26a84180 0x560a26a865a0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b87a19ffcea4833 server_cookie=77a2a1864dac4757 in_seq=0 out_seq=0 2024-05-07T02:36:07.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.631+0000 7efc9f400700 10 client.25976 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:36:07.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.631+0000 7efc9f400700 1 -- 172.21.15.19:0/1484099504 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560a26926400 con 0x560a26a84180 2024-05-07T02:36:07.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.631+0000 7efc9f400700 10 client.25976 handle_command_reply: tid=0 2024-05-07T02:36:07.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.631+0000 7efccece0b80 1 client.25976 shutdown 2024-05-07T02:36:07.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.631+0000 7efccece0b80 1 -- 172.21.15.19:0/1484099504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560a269f6dc0 msgr2=0x560a26a17e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:07.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.631+0000 7efccece0b80 1 --2- 172.21.15.19:0/1484099504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560a269f6dc0 0x560a26a17e30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7efca4000c00 tx=0x7efca400feb0 comp rx=0 tx=0).stop 2024-05-07T02:36:07.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.631+0000 7efccece0b80 1 -- 172.21.15.19:0/1484099504 shutdown_connections 2024-05-07T02:36:07.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.631+0000 7efccece0b80 1 --2- 172.21.15.19:0/1484099504 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x560a26a84180 0x560a26a865a0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:07.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.631+0000 7efccece0b80 1 --2- 172.21.15.19:0/1484099504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560a269fc3b0 0x560a26a18460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:07.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.631+0000 7efccece0b80 1 --2- 172.21.15.19:0/1484099504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560a269f6470 0x560a26a092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:07.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.631+0000 7efccece0b80 1 --2- 172.21.15.19:0/1484099504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560a269f6dc0 0x560a26a17e30 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:07.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.631+0000 7efccece0b80 1 -- 172.21.15.19:0/1484099504 >> 172.21.15.19:0/1484099504 conn(0x560a26878e40 msgr2=0x560a269eb8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:36:07.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.631+0000 7efc9f400700 0 client.25976 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:36:07.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.632+0000 7efccece0b80 1 -- 172.21.15.19:0/1484099504 shutdown_connections 2024-05-07T02:36:07.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.632+0000 7efccece0b80 1 -- 172.21.15.19:0/1484099504 wait complete. 2024-05-07T02:36:07.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.632+0000 7efccece0b80 20 client.25976 trim_cache size 0 max 16384 2024-05-07T02:36:07.635 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:36:07.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.632+0000 7efcc0200700 1 -- 172.21.15.19:0/2153943469 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7efca00b2a70 msgr2=0x7efca00b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:07.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.632+0000 7efcc0200700 1 --2- 172.21.15.19:0/2153943469 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7efca00b2a70 0x7efca00b4f80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efcb0020000 tx=0x7efcb0021000 comp rx=0 tx=0).stop 2024-05-07T02:36:07.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.632+0000 7efcc0200700 1 -- 172.21.15.19:0/2153943469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcb8077560 msgr2=0x7efcb819d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:07.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.632+0000 7efcc0200700 1 --2- 172.21.15.19:0/2153943469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcb8077560 0x7efcb819d240 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7efcac008000 tx=0x7efcac004d00 comp rx=0 tx=0).stop 2024-05-07T02:36:07.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.633+0000 7efcc0200700 1 -- 172.21.15.19:0/2153943469 shutdown_connections 2024-05-07T02:36:07.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.633+0000 7efcc0200700 1 --2- 172.21.15.19:0/2153943469 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcb81a19b0 0x7efcb81a1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:07.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.633+0000 7efcc0200700 1 --2- 172.21.15.19:0/2153943469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efcb8077560 0x7efcb819d240 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:07.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.633+0000 7efcc0200700 1 --2- 172.21.15.19:0/2153943469 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7efca00b2a70 0x7efca00b4f80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:07.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.633+0000 7efcc0200700 1 --2- 172.21.15.19:0/2153943469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcb8110be0 0x7efcb819d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:07.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.633+0000 7efcc0200700 1 -- 172.21.15.19:0/2153943469 >> 172.21.15.19:0/2153943469 conn(0x7efcb80ffea0 msgr2=0x7efcb81005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:36:07.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.633+0000 7efcc0200700 1 -- 172.21.15.19:0/2153943469 shutdown_connections 2024-05-07T02:36:07.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:07.633+0000 7efcc0200700 1 -- 172.21.15.19:0/2153943469 wait complete. 2024-05-07T02:36:07.743 INFO:tasks.workunit.client.0.smithi019.stdout: 1 102236 2.43 MB/sec execute 63 sec latency 18.372 ms 2024-05-07T02:36:08.246 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:08 smithi019 ceph-mon[29781]: pgmap v132: 129 pgs: 129 active+clean; 71 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 580 KiB/s wr, 120 op/s 2024-05-07T02:36:08.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:08 smithi064 ceph-mon[34699]: pgmap v132: 129 pgs: 129 active+clean; 71 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 580 KiB/s wr, 120 op/s 2024-05-07T02:36:08.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:08 smithi027 ceph-mon[34398]: pgmap v132: 129 pgs: 129 active+clean; 71 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 580 KiB/s wr, 120 op/s 2024-05-07T02:36:08.743 INFO:tasks.workunit.client.0.smithi019.stdout: 1 103176 2.51 MB/sec execute 64 sec latency 27.935 ms 2024-05-07T02:36:09.559 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-07T02:36:09.559 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-07T02:36:09.743 INFO:tasks.workunit.client.0.smithi019.stdout: 1 103589 2.48 MB/sec execute 65 sec latency 23.253 ms 2024-05-07T02:36:10.067 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:36:10.347 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:10 smithi019 ceph-mon[29781]: pgmap v133: 129 pgs: 129 active+clean; 71 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 580 KiB/s wr, 120 op/s 2024-05-07T02:36:10.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:10 smithi064 ceph-mon[34699]: pgmap v133: 129 pgs: 129 active+clean; 71 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 580 KiB/s wr, 120 op/s 2024-05-07T02:36:10.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:10 smithi027 ceph-mon[34398]: pgmap v133: 129 pgs: 129 active+clean; 71 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 580 KiB/s wr, 120 op/s 2024-05-07T02:36:10.744 INFO:tasks.workunit.client.0.smithi019.stdout: 1 105341 2.55 MB/sec execute 66 sec latency 16.226 ms 2024-05-07T02:36:11.744 INFO:tasks.workunit.client.0.smithi019.stdout: 1 106593 2.67 MB/sec execute 67 sec latency 21.073 ms 2024-05-07T02:36:12.339 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:12 smithi019 ceph-mon[29781]: pgmap v134: 129 pgs: 129 active+clean; 75 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 988 KiB/s wr, 183 op/s 2024-05-07T02:36:12.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:12 smithi064 ceph-mon[34699]: pgmap v134: 129 pgs: 129 active+clean; 75 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 988 KiB/s wr, 183 op/s 2024-05-07T02:36:12.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:12 smithi027 ceph-mon[34398]: pgmap v134: 129 pgs: 129 active+clean; 75 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 988 KiB/s wr, 183 op/s 2024-05-07T02:36:12.744 INFO:tasks.workunit.client.0.smithi019.stdout: 1 107004 2.65 MB/sec execute 68 sec latency 22.236 ms 2024-05-07T02:36:12.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.819+0000 7fcdcec00700 1 -- 172.21.15.19:0/1069948344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdc8106570 msgr2=0x7fcdc8106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:12.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.819+0000 7fcdcec00700 1 --2- 172.21.15.19:0/1069948344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdc8106570 0x7fcdc8106980 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4009990 tx=0x7fcdb4009ca0 comp rx=0 tx=0).stop 2024-05-07T02:36:12.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.819+0000 7fcdcec00700 1 -- 172.21.15.19:0/1069948344 shutdown_connections 2024-05-07T02:36:12.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.819+0000 7fcdcec00700 1 --2- 172.21.15.19:0/1069948344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdc8106ec0 0x7fcdc8113350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:12.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.819+0000 7fcdcec00700 1 --2- 172.21.15.19:0/1069948344 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdc810c5b0 0x7fcdc810c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:12.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.819+0000 7fcdcec00700 1 --2- 172.21.15.19:0/1069948344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdc8106570 0x7fcdc8106980 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:12.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.819+0000 7fcdcec00700 1 -- 172.21.15.19:0/1069948344 >> 172.21.15.19:0/1069948344 conn(0x7fcdc8102090 msgr2=0x7fcdc81044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:36:12.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.819+0000 7fcdcec00700 1 -- 172.21.15.19:0/1069948344 shutdown_connections 2024-05-07T02:36:12.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.819+0000 7fcdcec00700 1 -- 172.21.15.19:0/1069948344 wait complete. 2024-05-07T02:36:12.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.820+0000 7fcdcec00700 1 Processor -- start 2024-05-07T02:36:12.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.820+0000 7fcdcec00700 1 -- start start 2024-05-07T02:36:12.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.821+0000 7fcdcec00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdc8106570 0x7fcdc819d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:12.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.821+0000 7fcdcec00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdc8106ec0 0x7fcdc819d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:12.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.821+0000 7fcdcec00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdc810c5b0 0x7fcdc81a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:12.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.821+0000 7fcdcec00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc81166e0 con 0x7fcdc8106ec0 2024-05-07T02:36:12.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.821+0000 7fcdcec00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc8116560 con 0x7fcdc810c5b0 2024-05-07T02:36:12.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.821+0000 7fcdcec00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc8116860 con 0x7fcdc8106570 2024-05-07T02:36:12.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.821+0000 7fcdccc00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdc810c5b0 0x7fcdc81a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:12.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.821+0000 7fcdccc00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdc810c5b0 0x7fcdc81a18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:42320/0 (socket says 172.21.15.19:42320) 2024-05-07T02:36:12.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.821+0000 7fcdccc00700 1 -- 172.21.15.19:0/3626995961 learned_addr learned my addr 172.21.15.19:0/3626995961 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:36:12.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.821+0000 7fcdc7400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdc8106ec0 0x7fcdc819d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:12.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.821+0000 7fcdc7e00700 1 --2- 172.21.15.19:0/3626995961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdc8106570 0x7fcdc819d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:12.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.822+0000 7fcdccc00700 1 -- 172.21.15.19:0/3626995961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdc8106570 msgr2=0x7fcdc819d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:12.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.822+0000 7fcdccc00700 1 --2- 172.21.15.19:0/3626995961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdc8106570 0x7fcdc819d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:12.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.822+0000 7fcdccc00700 1 -- 172.21.15.19:0/3626995961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdc8106ec0 msgr2=0x7fcdc819d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:12.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.822+0000 7fcdccc00700 1 --2- 172.21.15.19:0/3626995961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdc8106ec0 0x7fcdc819d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:12.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.822+0000 7fcdccc00700 1 -- 172.21.15.19:0/3626995961 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdc81a1f40 con 0x7fcdc810c5b0 2024-05-07T02:36:12.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.822+0000 7fcdccc00700 1 --2- 172.21.15.19:0/3626995961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdc810c5b0 0x7fcdc81a18a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0007ee0 tx=0x7fcdc000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:36:12.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.822+0000 7fcdc4c00700 1 -- 172.21.15.19:0/3626995961 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc000dd20 con 0x7fcdc810c5b0 2024-05-07T02:36:12.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.822+0000 7fcdc4c00700 1 -- 172.21.15.19:0/3626995961 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdc0004020 con 0x7fcdc810c5b0 2024-05-07T02:36:12.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.822+0000 7fcdc4c00700 1 -- 172.21.15.19:0/3626995961 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0015660 con 0x7fcdc810c5b0 2024-05-07T02:36:12.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.822+0000 7fcdcec00700 1 -- 172.21.15.19:0/3626995961 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdc81a2230 con 0x7fcdc810c5b0 2024-05-07T02:36:12.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.823+0000 7fcdcec00700 1 -- 172.21.15.19:0/3626995961 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdc81a9b50 con 0x7fcdc810c5b0 2024-05-07T02:36:12.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.827+0000 7fcdc4c00700 1 -- 172.21.15.19:0/3626995961 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fcdc00041d0 con 0x7fcdc810c5b0 2024-05-07T02:36:12.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.828+0000 7fcddd6cbb80 1 Processor -- start 2024-05-07T02:36:12.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.828+0000 7fcddd6cbb80 1 -- start start 2024-05-07T02:36:12.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.829+0000 7fcddd6cbb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55963464e430 0x55963464e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:12.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.829+0000 7fcddd6cbb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596346484f0 0x559634648900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:12.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.829+0000 7fcddd6cbb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559634648e40 0x5596346550c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:12.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.829+0000 7fcddd6cbb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55963456aaa0 con 0x559634648e40 2024-05-07T02:36:12.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.829+0000 7fcddd6cbb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5596345087e0 con 0x55963464e430 2024-05-07T02:36:12.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.829+0000 7fcddd6cbb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559634566a60 con 0x5596346484f0 2024-05-07T02:36:12.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.829+0000 7fcdc7400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55963464e430 0x55963464e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:12.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.829+0000 7fcdc4c00700 1 --2- 172.21.15.19:0/3626995961 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fcdac0b2a70 0x7fcdac0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:12.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.829+0000 7fcdc7400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55963464e430 0x55963464e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:42326/0 (socket says 172.21.15.19:42326) 2024-05-07T02:36:12.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.829+0000 7fcdc7400700 1 -- 172.21.15.19:0/1094967010 learned_addr learned my addr 172.21.15.19:0/1094967010 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:36:12.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.829+0000 7fcdc7400700 1 --2- 172.21.15.19:0/1094967010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559634648e40 0x5596346550c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:12.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.829+0000 7fcdc7e00700 1 --2- 172.21.15.19:0/1094967010 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596346484f0 0x559634648900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:12.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.830+0000 7fcdc4c00700 1 -- 172.21.15.19:0/3626995961 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fcdc00e2400 con 0x7fcdc810c5b0 2024-05-07T02:36:12.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.830+0000 7fcdc7400700 1 -- 172.21.15.19:0/1094967010 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596346484f0 msgr2=0x559634648900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:36:12.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.830+0000 7fcdc7400700 1 --2- 172.21.15.19:0/1094967010 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596346484f0 0x559634648900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:12.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.830+0000 7fcdc7400700 1 -- 172.21.15.19:0/1094967010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559634648e40 msgr2=0x5596346550c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:36:12.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.830+0000 7fcdc7400700 1 --2- 172.21.15.19:0/1094967010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559634648e40 0x5596346550c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:12.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.830+0000 7fcdc7400700 1 -- 172.21.15.19:0/1094967010 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596344b7950 con 0x55963464e430 2024-05-07T02:36:12.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.830+0000 7fcdc7e00700 1 --2- 172.21.15.19:0/1094967010 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596346484f0 0x559634648900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:36:12.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.830+0000 7fcdccc00700 1 --2- 172.21.15.19:0/3626995961 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fcdac0b2a70 0x7fcdac0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:12.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.830+0000 7fcdc7400700 1 --2- 172.21.15.19:0/1094967010 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55963464e430 0x55963464e840 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4009990 tx=0x7fcdb401a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82be7597413c944d server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:36:12.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.831+0000 7fcdbd600700 1 -- 172.21.15.19:0/1094967010 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb400bcc0 con 0x55963464e430 2024-05-07T02:36:12.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.831+0000 7fcdbd600700 1 -- 172.21.15.19:0/1094967010 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdb400be60 con 0x55963464e430 2024-05-07T02:36:12.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.831+0000 7fcdbd600700 1 -- 172.21.15.19:0/1094967010 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb400f460 con 0x55963464e430 2024-05-07T02:36:12.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.832+0000 7fcddd6cbb80 1 -- 172.21.15.19:0/1094967010 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55963464e430 msgr2=0x55963464e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:12.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.832+0000 7fcddd6cbb80 1 --2- 172.21.15.19:0/1094967010 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55963464e430 0x55963464e840 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4009990 tx=0x7fcdb401a040 comp rx=0 tx=0).stop 2024-05-07T02:36:12.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.832+0000 7fcdccc00700 1 --2- 172.21.15.19:0/3626995961 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fcdac0b2a70 0x7fcdac0b4f80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcdc00a5370 tx=0x7fcdc0004b40 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:36:12.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.832+0000 7fcddd6cbb80 1 -- 172.21.15.19:0/1094967010 shutdown_connections 2024-05-07T02:36:12.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.832+0000 7fcddd6cbb80 1 --2- 172.21.15.19:0/1094967010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559634648e40 0x5596346550c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:12.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.832+0000 7fcddd6cbb80 1 --2- 172.21.15.19:0/1094967010 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55963464e430 0x55963464e840 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:12.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.832+0000 7fcddd6cbb80 1 --2- 172.21.15.19:0/1094967010 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5596346484f0 0x559634648900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:12.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.832+0000 7fcddd6cbb80 1 -- 172.21.15.19:0/1094967010 >> 172.21.15.19:0/1094967010 conn(0x5596344caec0 msgr2=0x559634643370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:36:12.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.832+0000 7fcddd6cbb80 1 -- 172.21.15.19:0/1094967010 shutdown_connections 2024-05-07T02:36:12.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.832+0000 7fcddd6cbb80 1 -- 172.21.15.19:0/1094967010 wait complete. 2024-05-07T02:36:12.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.833+0000 7fcddd6cbb80 1 -- start start 2024-05-07T02:36:12.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.834+0000 7fcddd6cbb80 1 Processor -- start 2024-05-07T02:36:12.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.834+0000 7fcddd6cbb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596346484f0 0x559634652630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:12.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.834+0000 7fcddd6cbb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559634648e40 0x559634652b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:12.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.834+0000 7fcddd6cbb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55963464e430 0x5596346530b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:12.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.834+0000 7fcddd6cbb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5596346557e0 con 0x5596346484f0 2024-05-07T02:36:12.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.834+0000 7fcddd6cbb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5596344d0ba0 con 0x559634648e40 2024-05-07T02:36:12.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.834+0000 7fcddd6cbb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559634655960 con 0x55963464e430 2024-05-07T02:36:12.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.834+0000 7fcddd6cbb80 10 client.0 _ll_register_callbacks cb 0x559634511cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:36:12.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.835+0000 7fcdc7e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559634648e40 0x559634652b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:12.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.835+0000 7fcdc7e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559634648e40 0x559634652b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:42334/0 (socket says 172.21.15.19:42334) 2024-05-07T02:36:12.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.835+0000 7fcdc7e00700 1 -- 172.21.15.19:0/3352464610 learned_addr learned my addr 172.21.15.19:0/3352464610 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:36:12.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.835+0000 7fcdc7400700 1 --2- 172.21.15.19:0/3352464610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596346484f0 0x559634652630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:12.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.835+0000 7fcdc7400700 1 --2- 172.21.15.19:0/3352464610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55963464e430 0x5596346530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:12.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.835+0000 7fcdc7e00700 1 -- 172.21.15.19:0/3352464610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55963464e430 msgr2=0x5596346530b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:36:12.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.835+0000 7fcdc7e00700 1 --2- 172.21.15.19:0/3352464610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55963464e430 0x5596346530b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:12.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.835+0000 7fcdc7e00700 1 -- 172.21.15.19:0/3352464610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596346484f0 msgr2=0x559634652630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:36:12.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.835+0000 7fcdc7e00700 1 --2- 172.21.15.19:0/3352464610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596346484f0 0x559634652630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:12.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.835+0000 7fcdc7e00700 1 -- 172.21.15.19:0/3352464610 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596345304a0 con 0x559634648e40 2024-05-07T02:36:12.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.835+0000 7fcdc7e00700 1 --2- 172.21.15.19:0/3352464610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559634648e40 0x559634652b70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8011720 tx=0x7fcdb8011a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e65149062bf3a48 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:36:12.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.836+0000 7fcdb2000700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:36:12.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.836+0000 7fcdb2000700 1 -- 172.21.15.19:0/3352464610 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb801e020 con 0x559634648e40 2024-05-07T02:36:12.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.836+0000 7fcdb2000700 1 -- 172.21.15.19:0/3352464610 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fcdb8004830 con 0x559634648e40 2024-05-07T02:36:12.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.836+0000 7fcddd6cbb80 1 -- 172.21.15.19:0/3352464610 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdb400be60 con 0x559634648e40 2024-05-07T02:36:12.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.836+0000 7fcdb2000700 1 -- 172.21.15.19:0/3352464610 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fcdb8026050 con 0x559634648e40 2024-05-07T02:36:12.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.836+0000 7fcddd6cbb80 10 client.25988 fetch_fsmap learned FSMap version 40 2024-05-07T02:36:12.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.836+0000 7fcddd6cbb80 1 -- 172.21.15.19:0/3352464610 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x559634660530 con 0x559634648e40 2024-05-07T02:36:12.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.837+0000 7fcdb2000700 1 -- 172.21.15.19:0/3352464610 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdb400be60 con 0x559634648e40 2024-05-07T02:36:12.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.837+0000 7fcdb2000700 1 -- 172.21.15.19:0/3352464610 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7fcdb8019070 con 0x559634648e40 2024-05-07T02:36:12.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.837+0000 7fcddd6cbb80 10 client.25988 fetch_fsmap finished waiting for FSMap version 40 2024-05-07T02:36:12.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.837+0000 7fcddd6cbb80 10 client.25988 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:36:12.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.837+0000 7fcddd6cbb80 20 client.25988 populate_metadata read hostname 'smithi019' 2024-05-07T02:36:12.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.837+0000 7fcddd6cbb80 1 --2- 172.21.15.19:0/3352464610 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x559634662b70 0x559634664fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:12.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.837+0000 7fcddd6cbb80 4 client.25988 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:36:12.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.837+0000 7fcddd6cbb80 1 -- 172.21.15.19:0/3352464610 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559634615360 con 0x559634662b70 2024-05-07T02:36:12.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.838+0000 7fcdc7400700 1 --2- 172.21.15.19:0/3352464610 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x559634662b70 0x559634664fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:12.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.838+0000 7fcdc7400700 1 --2- 172.21.15.19:0/3352464610 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x559634662b70 0x559634664fb0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99311bcf696fa129 server_cookie=a0b79e53eb1b30ae in_seq=0 out_seq=0 2024-05-07T02:36:12.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.838+0000 7fcdb2000700 10 client.25988 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:36:12.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.839+0000 7fcdb2000700 1 -- 172.21.15.19:0/3352464610 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559634615360 con 0x559634662b70 2024-05-07T02:36:12.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.839+0000 7fcdb2000700 10 client.25988 handle_command_reply: tid=0 2024-05-07T02:36:12.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.839+0000 7fcddd6cbb80 1 client.25988 shutdown 2024-05-07T02:36:12.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.840+0000 7fcddd6cbb80 1 -- 172.21.15.19:0/3352464610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559634648e40 msgr2=0x559634652b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:12.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.840+0000 7fcddd6cbb80 1 --2- 172.21.15.19:0/3352464610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559634648e40 0x559634652b70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8011720 tx=0x7fcdb8011a30 comp rx=0 tx=0).stop 2024-05-07T02:36:12.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.840+0000 7fcddd6cbb80 1 -- 172.21.15.19:0/3352464610 shutdown_connections 2024-05-07T02:36:12.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.840+0000 7fcddd6cbb80 1 --2- 172.21.15.19:0/3352464610 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x559634662b70 0x559634664fb0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:12.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.840+0000 7fcddd6cbb80 1 --2- 172.21.15.19:0/3352464610 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55963464e430 0x5596346530b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:12.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.840+0000 7fcddd6cbb80 1 --2- 172.21.15.19:0/3352464610 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596346484f0 0x559634652630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:12.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.840+0000 7fcddd6cbb80 1 --2- 172.21.15.19:0/3352464610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x559634648e40 0x559634652b70 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:12.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.840+0000 7fcddd6cbb80 1 -- 172.21.15.19:0/3352464610 >> 172.21.15.19:0/3352464610 conn(0x5596344caec0 msgr2=0x5596346474e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:36:12.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.840+0000 7fcdb2000700 0 client.25988 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:36:12.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.840+0000 7fcddd6cbb80 1 -- 172.21.15.19:0/3352464610 shutdown_connections 2024-05-07T02:36:12.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.840+0000 7fcddd6cbb80 1 -- 172.21.15.19:0/3352464610 wait complete. 2024-05-07T02:36:12.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.840+0000 7fcddd6cbb80 20 client.25988 trim_cache size 0 max 16384 2024-05-07T02:36:12.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.865+0000 7fcddd6cbb80 1 Processor -- start 2024-05-07T02:36:12.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.865+0000 7fcddd6cbb80 1 -- start start 2024-05-07T02:36:12.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.865+0000 7fcddd6cbb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5596346484f0 0x5596346543f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:12.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.865+0000 7fcddd6cbb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559634648e40 0x55963466a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:12.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.865+0000 7fcddd6cbb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55963464e430 0x55963466acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:12.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.865+0000 7fcddd6cbb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc81b26c0 con 0x55963464e430 2024-05-07T02:36:12.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.865+0000 7fcddd6cbb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc81b4260 con 0x5596346484f0 2024-05-07T02:36:12.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.865+0000 7fcddd6cbb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc81b2d90 con 0x559634648e40 2024-05-07T02:36:12.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.866+0000 7fcdc7e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559634648e40 0x55963466a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:12.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.866+0000 7fcdc7400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5596346484f0 0x5596346543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:12.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.866+0000 7fcdc7e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559634648e40 0x55963466a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:50522/0 (socket says 172.21.15.19:50522) 2024-05-07T02:36:12.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.866+0000 7fcdc7400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5596346484f0 0x5596346543f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:42336/0 (socket says 172.21.15.19:42336) 2024-05-07T02:36:12.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.866+0000 7fcdc7e00700 1 -- 172.21.15.19:0/1544540145 learned_addr learned my addr 172.21.15.19:0/1544540145 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:36:12.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.866+0000 7fcdc7400700 1 --2- 172.21.15.19:0/1544540145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55963464e430 0x55963466acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:12.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.866+0000 7fcdc7e00700 1 -- 172.21.15.19:0/1544540145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5596346484f0 msgr2=0x5596346543f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:36:12.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.866+0000 7fcdc7e00700 1 --2- 172.21.15.19:0/1544540145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5596346484f0 0x5596346543f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:12.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.866+0000 7fcdc7e00700 1 -- 172.21.15.19:0/1544540145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55963464e430 msgr2=0x55963466acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:36:12.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.867+0000 7fcdc7e00700 1 --2- 172.21.15.19:0/1544540145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55963464e430 0x55963466acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:12.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.867+0000 7fcdc7e00700 1 -- 172.21.15.19:0/1544540145 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559634655430 con 0x559634648e40 2024-05-07T02:36:12.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.867+0000 7fcdc7400700 1 --2- 172.21.15.19:0/1544540145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55963464e430 0x55963466acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:36:12.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.867+0000 7fcdc7e00700 1 --2- 172.21.15.19:0/1544540145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559634648e40 0x55963466a770 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8001eb0 tx=0x7fcdb80092d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=651622ac8c25fb4c server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:36:12.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.868+0000 7fcdbcc00700 1 -- 172.21.15.19:0/1544540145 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb802f070 con 0x559634648e40 2024-05-07T02:36:12.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.868+0000 7fcdbcc00700 1 -- 172.21.15.19:0/1544540145 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdb801ab50 con 0x559634648e40 2024-05-07T02:36:12.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.868+0000 7fcdbcc00700 1 -- 172.21.15.19:0/1544540145 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb801e040 con 0x559634648e40 2024-05-07T02:36:12.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.869+0000 7fcddd6cbb80 1 -- 172.21.15.19:0/1544540145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559634648e40 msgr2=0x55963466a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:12.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.869+0000 7fcddd6cbb80 1 --2- 172.21.15.19:0/1544540145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559634648e40 0x55963466a770 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8001eb0 tx=0x7fcdb80092d0 comp rx=0 tx=0).stop 2024-05-07T02:36:12.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.869+0000 7fcddd6cbb80 1 -- 172.21.15.19:0/1544540145 shutdown_connections 2024-05-07T02:36:12.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.869+0000 7fcddd6cbb80 1 --2- 172.21.15.19:0/1544540145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55963464e430 0x55963466acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:12.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.869+0000 7fcddd6cbb80 1 --2- 172.21.15.19:0/1544540145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5596346484f0 0x5596346543f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:12.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.869+0000 7fcddd6cbb80 1 --2- 172.21.15.19:0/1544540145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559634648e40 0x55963466a770 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:12.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.869+0000 7fcddd6cbb80 1 -- 172.21.15.19:0/1544540145 >> 172.21.15.19:0/1544540145 conn(0x5596344caec0 msgr2=0x559634653ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:36:12.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.869+0000 7fcddd6cbb80 1 -- 172.21.15.19:0/1544540145 shutdown_connections 2024-05-07T02:36:12.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.869+0000 7fcddd6cbb80 1 -- 172.21.15.19:0/1544540145 wait complete. 2024-05-07T02:36:12.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.870+0000 7fcddd6cbb80 1 -- start start 2024-05-07T02:36:12.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.870+0000 7fcddd6cbb80 1 Processor -- start 2024-05-07T02:36:12.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.871+0000 7fcddd6cbb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5596346484f0 0x559634656fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:12.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.871+0000 7fcddd6cbb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559634648e40 0x5596346574e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:12.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.871+0000 7fcddd6cbb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55963464e430 0x5596346cc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:12.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.871+0000 7fcddd6cbb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc81b3b90 con 0x55963464e430 2024-05-07T02:36:12.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.871+0000 7fcddd6cbb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc81b34c0 con 0x5596346484f0 2024-05-07T02:36:12.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.871+0000 7fcddd6cbb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc81b0660 con 0x559634648e40 2024-05-07T02:36:12.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.871+0000 7fcddd6cbb80 10 client.0 _ll_register_callbacks cb 0x559634511cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:36:12.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.872+0000 7fcdc7400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5596346484f0 0x559634656fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:12.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.872+0000 7fcdc7400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5596346484f0 0x559634656fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:42338/0 (socket says 172.21.15.19:42338) 2024-05-07T02:36:12.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.872+0000 7fcdc7400700 1 -- 172.21.15.19:0/3375433067 learned_addr learned my addr 172.21.15.19:0/3375433067 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:36:12.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.872+0000 7fcdc7400700 1 --2- 172.21.15.19:0/3375433067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55963464e430 0x5596346cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:12.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.872+0000 7fcdc7400700 1 -- 172.21.15.19:0/3375433067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559634648e40 msgr2=0x5596346574e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-07T02:36:12.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.872+0000 7fcdc7400700 1 --2- 172.21.15.19:0/3375433067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559634648e40 0x5596346574e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:12.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.873+0000 7fcdc7400700 1 -- 172.21.15.19:0/3375433067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55963464e430 msgr2=0x5596346cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:36:12.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.873+0000 7fcdc7400700 1 --2- 172.21.15.19:0/3375433067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55963464e430 0x5596346cc320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:12.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.873+0000 7fcdc7400700 1 -- 172.21.15.19:0/3375433067 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596346cc8b0 con 0x5596346484f0 2024-05-07T02:36:12.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.873+0000 7fcdb2000700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:36:12.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.873+0000 7fcdc7400700 1 --2- 172.21.15.19:0/3375433067 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5596346484f0 0x559634656fa0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcdb400b6c0 tx=0x7fcdb4037f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60c6cce0344f8443 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:36:12.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.874+0000 7fcdb2000700 1 -- 172.21.15.19:0/3375433067 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb402c040 con 0x5596346484f0 2024-05-07T02:36:12.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.874+0000 7fcdb2000700 1 -- 172.21.15.19:0/3375433067 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fcdb4003680 con 0x5596346484f0 2024-05-07T02:36:12.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.874+0000 7fcddd6cbb80 1 -- 172.21.15.19:0/3375433067 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdb801ab50 con 0x5596346484f0 2024-05-07T02:36:12.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.874+0000 7fcdb2000700 1 -- 172.21.15.19:0/3375433067 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fcdb4003880 con 0x5596346484f0 2024-05-07T02:36:12.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.874+0000 7fcddd6cbb80 10 client.25994 fetch_fsmap learned FSMap version 40 2024-05-07T02:36:12.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.875+0000 7fcddd6cbb80 1 -- 172.21.15.19:0/3375433067 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5596346cd9e0 con 0x5596346484f0 2024-05-07T02:36:12.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.876+0000 7fcdb2000700 1 -- 172.21.15.19:0/3375433067 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdb801ab50 con 0x5596346484f0 2024-05-07T02:36:12.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.876+0000 7fcdb2000700 1 -- 172.21.15.19:0/3375433067 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7fcdb4032070 con 0x5596346484f0 2024-05-07T02:36:12.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.876+0000 7fcddd6cbb80 10 client.25994 fetch_fsmap finished waiting for FSMap version 40 2024-05-07T02:36:12.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.876+0000 7fcddd6cbb80 10 client.25994 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:36:12.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.876+0000 7fcddd6cbb80 20 client.25994 populate_metadata read hostname 'smithi019' 2024-05-07T02:36:12.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.876+0000 7fcddd6cbb80 1 --2- 172.21.15.19:0/3375433067 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5596346cdd20 0x5596346d0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:12.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.876+0000 7fcddd6cbb80 4 client.25994 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-07T02:36:12.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.876+0000 7fcddd6cbb80 1 -- 172.21.15.19:0/3375433067 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559634578480 con 0x5596346cdd20 2024-05-07T02:36:12.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.877+0000 7fcdc7e00700 1 --2- 172.21.15.19:0/3375433067 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5596346cdd20 0x5596346d0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:12.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.877+0000 7fcdc7e00700 1 --2- 172.21.15.19:0/3375433067 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5596346cdd20 0x5596346d0140 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c68ba27b588544b3 server_cookie=5404d876c42cd42e in_seq=0 out_seq=0 2024-05-07T02:36:12.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.877+0000 7fcdb2000700 10 client.25994 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:36:12.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.878+0000 7fcdb2000700 1 -- 172.21.15.19:0/3375433067 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559634578480 con 0x5596346cdd20 2024-05-07T02:36:12.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.878+0000 7fcdb2000700 10 client.25994 handle_command_reply: tid=0 2024-05-07T02:36:12.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.878+0000 7fcddd6cbb80 1 client.25994 shutdown 2024-05-07T02:36:12.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.879+0000 7fcddd6cbb80 1 -- 172.21.15.19:0/3375433067 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5596346484f0 msgr2=0x559634656fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:12.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.879+0000 7fcddd6cbb80 1 --2- 172.21.15.19:0/3375433067 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5596346484f0 0x559634656fa0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcdb400b6c0 tx=0x7fcdb4037f90 comp rx=0 tx=0).stop 2024-05-07T02:36:12.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.879+0000 7fcddd6cbb80 1 -- 172.21.15.19:0/3375433067 shutdown_connections 2024-05-07T02:36:12.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.879+0000 7fcddd6cbb80 1 --2- 172.21.15.19:0/3375433067 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5596346cdd20 0x5596346d0140 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:12.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.879+0000 7fcddd6cbb80 1 --2- 172.21.15.19:0/3375433067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55963464e430 0x5596346cc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:12.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.879+0000 7fcddd6cbb80 1 --2- 172.21.15.19:0/3375433067 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5596346484f0 0x559634656fa0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:12.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.879+0000 7fcddd6cbb80 1 --2- 172.21.15.19:0/3375433067 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559634648e40 0x5596346574e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:12.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.879+0000 7fcddd6cbb80 1 -- 172.21.15.19:0/3375433067 >> 172.21.15.19:0/3375433067 conn(0x5596344caec0 msgr2=0x55963463d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:36:12.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.879+0000 7fcdb2000700 0 client.25994 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:36:12.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.880+0000 7fcddd6cbb80 1 -- 172.21.15.19:0/3375433067 shutdown_connections 2024-05-07T02:36:12.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.880+0000 7fcddd6cbb80 1 -- 172.21.15.19:0/3375433067 wait complete. 2024-05-07T02:36:12.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.880+0000 7fcddd6cbb80 20 client.25994 trim_cache size 0 max 16384 2024-05-07T02:36:12.882 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:36:12.882 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-07T02:36:12.883 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "981dc9e9-c23a-4c11-a45e-73069346138a", 2024-05-07T02:36:12.883 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-07T02:36:12.883 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:36:12.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.881+0000 7fcdcec00700 1 -- 172.21.15.19:0/3626995961 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fcdac0b2a70 msgr2=0x7fcdac0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:12.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.881+0000 7fcdcec00700 1 --2- 172.21.15.19:0/3626995961 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fcdac0b2a70 0x7fcdac0b4f80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcdc00a5370 tx=0x7fcdc0004b40 comp rx=0 tx=0).stop 2024-05-07T02:36:12.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.881+0000 7fcdcec00700 1 -- 172.21.15.19:0/3626995961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdc810c5b0 msgr2=0x7fcdc81a18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:12.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.881+0000 7fcdcec00700 1 --2- 172.21.15.19:0/3626995961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdc810c5b0 0x7fcdc81a18a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0007ee0 tx=0x7fcdc000d3b0 comp rx=0 tx=0).stop 2024-05-07T02:36:12.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.882+0000 7fcdcec00700 1 -- 172.21.15.19:0/3626995961 shutdown_connections 2024-05-07T02:36:12.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.882+0000 7fcdcec00700 1 --2- 172.21.15.19:0/3626995961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdc810c5b0 0x7fcdc81a18a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:12.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.882+0000 7fcdcec00700 1 --2- 172.21.15.19:0/3626995961 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdc8106570 0x7fcdc819d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:12.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.882+0000 7fcdcec00700 1 --2- 172.21.15.19:0/3626995961 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fcdac0b2a70 0x7fcdac0b4f80 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:12.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.882+0000 7fcdcec00700 1 --2- 172.21.15.19:0/3626995961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcdc8106ec0 0x7fcdc819d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:12.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.882+0000 7fcdcec00700 1 -- 172.21.15.19:0/3626995961 >> 172.21.15.19:0/3626995961 conn(0x7fcdc8102090 msgr2=0x7fcdc8104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:36:12.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.882+0000 7fcdcec00700 1 -- 172.21.15.19:0/3626995961 shutdown_connections 2024-05-07T02:36:12.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:12.882+0000 7fcdcec00700 1 -- 172.21.15.19:0/3626995961 wait complete. 2024-05-07T02:36:13.744 INFO:tasks.workunit.client.0.smithi019.stdout: 1 108365 2.69 MB/sec execute 69 sec latency 29.752 ms 2024-05-07T02:36:13.850 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-07T02:36:13.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:13 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:36:13.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:13 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:36:13.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:13 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:36:13.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:13 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:36:13.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:13 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:36:13.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:13 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:36:13.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:13 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:36:13.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:13 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:36:13.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:13 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:36:13.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:13 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:36:13.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:13 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:36:13.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:13 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:36:13.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:13 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:36:13.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:13 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:36:13.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:13 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:36:13.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:13 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:36:13.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:13 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:36:13.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:13 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:36:14.300 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:36:14.744 INFO:tasks.workunit.client.0.smithi019.stdout: 1 110115 2.83 MB/sec execute 70 sec latency 19.609 ms 2024-05-07T02:36:14.808 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:14 smithi019 ceph-mon[29781]: scrub queued for path: / 2024-05-07T02:36:14.808 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:14 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:36:14.809 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:14 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:36:14.809 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:14 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:36:14.809 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:14 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:36:14.809 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:14 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:36:14.809 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:14 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:36:14.809 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:14 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:36:14.809 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:14 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:36:14.809 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:14 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:36:14.809 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:14 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:36:14.809 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:14 smithi019 ceph-mon[29781]: pgmap v135: 129 pgs: 129 active+clean; 75 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 758 KiB/s wr, 138 op/s 2024-05-07T02:36:14.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:14 smithi027 ceph-mon[34398]: scrub queued for path: / 2024-05-07T02:36:14.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:14 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:36:14.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:14 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:36:14.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:14 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:36:14.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:14 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:36:14.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:14 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:36:14.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:14 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:36:14.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:14 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:36:14.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:14 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:36:14.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:14 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:36:14.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:14 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:36:14.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:14 smithi027 ceph-mon[34398]: pgmap v135: 129 pgs: 129 active+clean; 75 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 758 KiB/s wr, 138 op/s 2024-05-07T02:36:15.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:14 smithi064 ceph-mon[34699]: scrub queued for path: / 2024-05-07T02:36:15.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:14 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:36:15.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:14 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:36:15.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:14 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:36:15.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:14 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:36:15.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:14 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:36:15.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:14 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:36:15.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:14 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:36:15.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:14 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:36:15.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:14 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:36:15.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:14 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:36:15.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:14 smithi064 ceph-mon[34699]: pgmap v135: 129 pgs: 129 active+clean; 75 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 758 KiB/s wr, 138 op/s 2024-05-07T02:36:15.744 INFO:tasks.workunit.client.0.smithi019.stdout: 1 110546 2.81 MB/sec execute 71 sec latency 21.055 ms 2024-05-07T02:36:15.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:15 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:36:15.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:15 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:36:15.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:15 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:36:15.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:15 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:36:15.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:15 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:36:15.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:15 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:36:16.010 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:15 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:36:16.010 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:15 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:36:16.010 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:15 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:36:16.745 INFO:tasks.workunit.client.0.smithi019.stdout: 1 111843 2.84 MB/sec execute 72 sec latency 20.865 ms 2024-05-07T02:36:16.894 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:16 smithi019 ceph-mon[29781]: pgmap v136: 129 pgs: 129 active+clean; 81 MiB data, 834 MiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 1.3 MiB/s wr, 205 op/s 2024-05-07T02:36:16.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.910+0000 7f06c4200700 1 -- 172.21.15.19:0/863302086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06bc110c00 msgr2=0x7f06bc113020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:16.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.910+0000 7f06c4200700 1 --2- 172.21.15.19:0/863302086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06bc110c00 0x7f06bc113020 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f06b400b0a0 tx=0x7f06b400b3b0 comp rx=0 tx=0).stop 2024-05-07T02:36:16.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.910+0000 7f06c4200700 1 -- 172.21.15.19:0/863302086 shutdown_connections 2024-05-07T02:36:16.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.910+0000 7f06c4200700 1 --2- 172.21.15.19:0/863302086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06bc110c00 0x7f06bc113020 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:16.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.910+0000 7f06c4200700 1 --2- 172.21.15.19:0/863302086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06bc075b00 0x7f06bc075f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:16.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.910+0000 7f06c4200700 1 --2- 172.21.15.19:0/863302086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06bc076450 0x7f06bc1106c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:16.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.910+0000 7f06c4200700 1 -- 172.21.15.19:0/863302086 >> 172.21.15.19:0/863302086 conn(0x7f06bc100040 msgr2=0x7f06bc102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:36:16.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.911+0000 7f06c4200700 1 -- 172.21.15.19:0/863302086 shutdown_connections 2024-05-07T02:36:16.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.911+0000 7f06c4200700 1 -- 172.21.15.19:0/863302086 wait complete. 2024-05-07T02:36:16.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.912+0000 7f06c4200700 1 Processor -- start 2024-05-07T02:36:16.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.913+0000 7f06c4200700 1 -- start start 2024-05-07T02:36:16.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.913+0000 7f06c4200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06bc075b00 0x7f06bc10da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:16.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.913+0000 7f06c4200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06bc076450 0x7f06bc10dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:16.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.913+0000 7f06c4200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06bc110c00 0x7f06bc10e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:16.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.913+0000 7f06c4200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f06bc116730 con 0x7f06bc075b00 2024-05-07T02:36:16.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.913+0000 7f06c4200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f06bc1165b0 con 0x7f06bc076450 2024-05-07T02:36:16.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.913+0000 7f06c4200700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f06bc1168b0 con 0x7f06bc110c00 2024-05-07T02:36:16.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.913+0000 7f06c0e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06bc076450 0x7f06bc10dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:16.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.913+0000 7f06c0e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06bc076450 0x7f06bc10dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:42364/0 (socket says 172.21.15.19:42364) 2024-05-07T02:36:16.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.913+0000 7f06c0e00700 1 -- 172.21.15.19:0/2716995830 learned_addr learned my addr 172.21.15.19:0/2716995830 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:36:16.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.914+0000 7f06c1800700 1 --2- 172.21.15.19:0/2716995830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06bc075b00 0x7f06bc10da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:16.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.914+0000 7f06c2200700 1 --2- 172.21.15.19:0/2716995830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06bc110c00 0x7f06bc10e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:16.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.914+0000 7f06c0e00700 1 -- 172.21.15.19:0/2716995830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06bc110c00 msgr2=0x7f06bc10e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:16.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.914+0000 7f06c0e00700 1 --2- 172.21.15.19:0/2716995830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06bc110c00 0x7f06bc10e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:16.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.914+0000 7f06c0e00700 1 -- 172.21.15.19:0/2716995830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06bc075b00 msgr2=0x7f06bc10da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:16.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.914+0000 7f06c0e00700 1 --2- 172.21.15.19:0/2716995830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06bc075b00 0x7f06bc10da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:16.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.914+0000 7f06c0e00700 1 -- 172.21.15.19:0/2716995830 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06bc1ad5a0 con 0x7f06bc076450 2024-05-07T02:36:16.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.914+0000 7f06c0e00700 1 --2- 172.21.15.19:0/2716995830 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06bc076450 0x7f06bc10dfc0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f06a800e9f0 tx=0x7f06a800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:36:16.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.915+0000 7f06ba000700 1 -- 172.21.15.19:0/2716995830 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a8004a30 con 0x7f06bc076450 2024-05-07T02:36:16.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.915+0000 7f06ba000700 1 -- 172.21.15.19:0/2716995830 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06a800cc70 con 0x7f06bc076450 2024-05-07T02:36:16.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.915+0000 7f06c4200700 1 -- 172.21.15.19:0/2716995830 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06bc1ad890 con 0x7f06bc076450 2024-05-07T02:36:16.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.915+0000 7f06ba000700 1 -- 172.21.15.19:0/2716995830 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a8005530 con 0x7f06bc076450 2024-05-07T02:36:16.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.915+0000 7f06c4200700 1 -- 172.21.15.19:0/2716995830 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06bc1addd0 con 0x7f06bc076450 2024-05-07T02:36:16.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.919+0000 7f06ba000700 1 -- 172.21.15.19:0/2716995830 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f06a8004bd0 con 0x7f06bc076450 2024-05-07T02:36:16.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.920+0000 7f06d2c7ab80 1 Processor -- start 2024-05-07T02:36:16.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.920+0000 7f06d2c7ab80 1 -- start start 2024-05-07T02:36:16.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.920+0000 7f06ba000700 1 --2- 172.21.15.19:0/2716995830 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f06a40b2a70 0x7f06a40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:16.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.921+0000 7f06ba000700 1 -- 172.21.15.19:0/2716995830 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f06a80e2b30 con 0x7f06bc076450 2024-05-07T02:36:16.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.921+0000 7f06d2c7ab80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adcc4bf470 0x55adcc4bf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:16.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.921+0000 7f06c2200700 1 --2- 172.21.15.19:0/2716995830 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f06a40b2a70 0x7f06a40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:16.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.921+0000 7f06d2c7ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55adcc4b9620 0x55adcc4b9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:16.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.921+0000 7f06d2c7ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adcc4b9f70 0x55adcc4c6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:16.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.921+0000 7f06d2c7ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55adcc3dba20 con 0x55adcc4b9f70 2024-05-07T02:36:16.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.921+0000 7f06d2c7ab80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55adcc379760 con 0x55adcc4bf470 2024-05-07T02:36:16.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.921+0000 7f06d2c7ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55adcc3d79e0 con 0x55adcc4b9620 2024-05-07T02:36:16.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.921+0000 7f06c1800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adcc4bf470 0x55adcc4bf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:16.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.921+0000 7f06c1800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adcc4bf470 0x55adcc4bf880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:42378/0 (socket says 172.21.15.19:42378) 2024-05-07T02:36:16.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.921+0000 7f06c1800700 1 -- 172.21.15.19:0/2847228131 learned_addr learned my addr 172.21.15.19:0/2847228131 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:36:16.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.921+0000 7f06c0e00700 1 --2- 172.21.15.19:0/2847228131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55adcc4b9620 0x55adcc4b9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:16.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.921+0000 7f06c2200700 1 --2- 172.21.15.19:0/2847228131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adcc4b9f70 0x55adcc4c6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:16.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.922+0000 7f06c1800700 1 -- 172.21.15.19:0/2847228131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55adcc4b9620 msgr2=0x55adcc4b9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:36:16.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.922+0000 7f06c1800700 1 --2- 172.21.15.19:0/2847228131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55adcc4b9620 0x55adcc4b9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:16.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.922+0000 7f06c1800700 1 -- 172.21.15.19:0/2847228131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adcc4b9f70 msgr2=0x55adcc4c6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:36:16.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.922+0000 7f06c1800700 1 --2- 172.21.15.19:0/2847228131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adcc4b9f70 0x55adcc4c6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:16.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.922+0000 7f06c1800700 1 -- 172.21.15.19:0/2847228131 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adcc3288d0 con 0x55adcc4bf470 2024-05-07T02:36:16.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.922+0000 7f06c0e00700 1 --2- 172.21.15.19:0/2847228131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55adcc4b9620 0x55adcc4b9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:36:16.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.922+0000 7f06c1800700 1 --2- 172.21.15.19:0/2847228131 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adcc4bf470 0x55adcc4bf880 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f06b0009d80 tx=0x7f06b0011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d73f329d8f42e99 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:36:16.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.922+0000 7f06c2200700 1 --2- 172.21.15.19:0/2716995830 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f06a40b2a70 0x7f06a40b4f80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f06b4009fd0 tx=0x7f06b400bb90 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:36:16.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.922+0000 7f06aea00700 1 -- 172.21.15.19:0/2847228131 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b0004230 con 0x55adcc4bf470 2024-05-07T02:36:16.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.922+0000 7f06aea00700 1 -- 172.21.15.19:0/2847228131 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06b00043d0 con 0x55adcc4bf470 2024-05-07T02:36:16.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.922+0000 7f06aea00700 1 -- 172.21.15.19:0/2847228131 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b00198c0 con 0x55adcc4bf470 2024-05-07T02:36:16.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.924+0000 7f06d2c7ab80 1 -- 172.21.15.19:0/2847228131 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adcc4bf470 msgr2=0x55adcc4bf880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:16.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.924+0000 7f06d2c7ab80 1 --2- 172.21.15.19:0/2847228131 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adcc4bf470 0x55adcc4bf880 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f06b0009d80 tx=0x7f06b0011720 comp rx=0 tx=0).stop 2024-05-07T02:36:16.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.924+0000 7f06d2c7ab80 1 -- 172.21.15.19:0/2847228131 shutdown_connections 2024-05-07T02:36:16.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.924+0000 7f06d2c7ab80 1 --2- 172.21.15.19:0/2847228131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adcc4b9f70 0x55adcc4c6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:16.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.924+0000 7f06d2c7ab80 1 --2- 172.21.15.19:0/2847228131 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adcc4bf470 0x55adcc4bf880 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:16.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.924+0000 7f06d2c7ab80 1 --2- 172.21.15.19:0/2847228131 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55adcc4b9620 0x55adcc4b9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:16.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.924+0000 7f06d2c7ab80 1 -- 172.21.15.19:0/2847228131 >> 172.21.15.19:0/2847228131 conn(0x55adcc33be40 msgr2=0x55adcc4b4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:36:16.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.924+0000 7f06d2c7ab80 1 -- 172.21.15.19:0/2847228131 shutdown_connections 2024-05-07T02:36:16.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.924+0000 7f06d2c7ab80 1 -- 172.21.15.19:0/2847228131 wait complete. 2024-05-07T02:36:16.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.925+0000 7f06d2c7ab80 1 -- start start 2024-05-07T02:36:16.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.925+0000 7f06d2c7ab80 1 Processor -- start 2024-05-07T02:36:16.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.926+0000 7f06d2c7ab80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adcc4b9620 0x55adcc4c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:16.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.926+0000 7f06d2c7ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adcc4b9f70 0x55adcc4c3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:16.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.926+0000 7f06d2c7ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55adcc4bf470 0x55adcc4c4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:16.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.926+0000 7f06d2c7ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55adcc4c6840 con 0x55adcc4b9f70 2024-05-07T02:36:16.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.926+0000 7f06d2c7ab80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55adcc341b20 con 0x55adcc4b9620 2024-05-07T02:36:16.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.926+0000 7f06d2c7ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55adcc4c69c0 con 0x55adcc4bf470 2024-05-07T02:36:16.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.926+0000 7f06c1800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adcc4b9620 0x55adcc4c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:16.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.926+0000 7f06d2c7ab80 10 client.0 _ll_register_callbacks cb 0x55adcc36f380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:36:16.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.926+0000 7f06c1800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adcc4b9620 0x55adcc4c36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:42392/0 (socket says 172.21.15.19:42392) 2024-05-07T02:36:16.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.926+0000 7f06c1800700 1 -- 172.21.15.19:0/423767509 learned_addr learned my addr 172.21.15.19:0/423767509 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:36:16.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.926+0000 7f06c0e00700 1 --2- 172.21.15.19:0/423767509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adcc4b9f70 0x55adcc4c3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:16.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.926+0000 7f06c2200700 1 --2- 172.21.15.19:0/423767509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55adcc4bf470 0x55adcc4c4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:16.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.927+0000 7f06c1800700 1 -- 172.21.15.19:0/423767509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55adcc4bf470 msgr2=0x55adcc4c4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:36:16.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.927+0000 7f06c1800700 1 --2- 172.21.15.19:0/423767509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55adcc4bf470 0x55adcc4c4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:16.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.927+0000 7f06c1800700 1 -- 172.21.15.19:0/423767509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adcc4b9f70 msgr2=0x55adcc4c3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:36:16.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.927+0000 7f06c1800700 1 --2- 172.21.15.19:0/423767509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adcc4b9f70 0x55adcc4c3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:16.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.927+0000 7f06c1800700 1 -- 172.21.15.19:0/423767509 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adcc3a1420 con 0x55adcc4b9620 2024-05-07T02:36:16.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.927+0000 7f06c1800700 1 --2- 172.21.15.19:0/423767509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adcc4b9620 0x55adcc4c36e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f06b00097c0 tx=0x7f06b0005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1727d0bc1152e3bf server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:36:16.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.927+0000 7f06a3400700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:36:16.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.928+0000 7f06a3400700 1 -- 172.21.15.19:0/423767509 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b0027070 con 0x55adcc4b9620 2024-05-07T02:36:16.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.928+0000 7f06d2c7ab80 1 -- 172.21.15.19:0/423767509 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06b00043d0 con 0x55adcc4b9620 2024-05-07T02:36:16.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.928+0000 7f06a3400700 1 -- 172.21.15.19:0/423767509 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f06b0003680 con 0x55adcc4b9620 2024-05-07T02:36:16.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.928+0000 7f06a3400700 1 -- 172.21.15.19:0/423767509 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f06b0033050 con 0x55adcc4b9620 2024-05-07T02:36:16.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.928+0000 7f06d2c7ab80 10 client.26006 fetch_fsmap learned FSMap version 40 2024-05-07T02:36:16.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.928+0000 7f06d2c7ab80 1 -- 172.21.15.19:0/423767509 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55adcc4d14f0 con 0x55adcc4b9620 2024-05-07T02:36:16.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.928+0000 7f06a3400700 1 -- 172.21.15.19:0/423767509 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06b00043d0 con 0x55adcc4b9620 2024-05-07T02:36:16.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.929+0000 7f06a3400700 1 -- 172.21.15.19:0/423767509 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f06b00194c0 con 0x55adcc4b9620 2024-05-07T02:36:16.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.929+0000 7f06d2c7ab80 10 client.26006 fetch_fsmap finished waiting for FSMap version 40 2024-05-07T02:36:16.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.929+0000 7f06d2c7ab80 10 client.26006 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:36:16.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.929+0000 7f06d2c7ab80 20 client.26006 populate_metadata read hostname 'smithi019' 2024-05-07T02:36:16.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.929+0000 7f06d2c7ab80 1 --2- 172.21.15.19:0/423767509 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55adcc4d3a80 0x55adcc4d5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:16.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.929+0000 7f06d2c7ab80 4 client.26006 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:36:16.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.929+0000 7f06d2c7ab80 1 -- 172.21.15.19:0/423767509 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55adcc4860e0 con 0x55adcc4d3a80 2024-05-07T02:36:16.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.929+0000 7f06c0e00700 1 --2- 172.21.15.19:0/423767509 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55adcc4d3a80 0x55adcc4d5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:16.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.930+0000 7f06c0e00700 1 --2- 172.21.15.19:0/423767509 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55adcc4d3a80 0x55adcc4d5ea0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7cacd7cc7c9db62 server_cookie=1787fd4de3ee7c56 in_seq=0 out_seq=0 2024-05-07T02:36:16.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.930+0000 7f06a3400700 10 client.26006 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:36:16.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.930+0000 7f06a3400700 1 -- 172.21.15.19:0/423767509 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55adcc4860e0 con 0x55adcc4d3a80 2024-05-07T02:36:16.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.930+0000 7f06a3400700 10 client.26006 handle_command_reply: tid=0 2024-05-07T02:36:16.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.930+0000 7f06d2c7ab80 1 client.26006 shutdown 2024-05-07T02:36:16.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.931+0000 7f06d2c7ab80 1 -- 172.21.15.19:0/423767509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adcc4b9620 msgr2=0x55adcc4c36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:16.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.931+0000 7f06d2c7ab80 1 --2- 172.21.15.19:0/423767509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adcc4b9620 0x55adcc4c36e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f06b00097c0 tx=0x7f06b0005270 comp rx=0 tx=0).stop 2024-05-07T02:36:16.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.931+0000 7f06d2c7ab80 1 -- 172.21.15.19:0/423767509 shutdown_connections 2024-05-07T02:36:16.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.931+0000 7f06d2c7ab80 1 --2- 172.21.15.19:0/423767509 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55adcc4d3a80 0x55adcc4d5ea0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:16.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.931+0000 7f06d2c7ab80 1 --2- 172.21.15.19:0/423767509 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55adcc4bf470 0x55adcc4c4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:16.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.931+0000 7f06d2c7ab80 1 --2- 172.21.15.19:0/423767509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adcc4b9620 0x55adcc4c36e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:16.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.931+0000 7f06a3400700 0 client.26006 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:36:16.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.931+0000 7f06d2c7ab80 1 --2- 172.21.15.19:0/423767509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adcc4b9f70 0x55adcc4c3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:16.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.931+0000 7f06d2c7ab80 1 -- 172.21.15.19:0/423767509 >> 172.21.15.19:0/423767509 conn(0x55adcc33be40 msgr2=0x55adcc4b8610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:36:16.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.931+0000 7f06d2c7ab80 1 -- 172.21.15.19:0/423767509 shutdown_connections 2024-05-07T02:36:16.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.931+0000 7f06d2c7ab80 1 -- 172.21.15.19:0/423767509 wait complete. 2024-05-07T02:36:16.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.931+0000 7f06d2c7ab80 20 client.26006 trim_cache size 0 max 16384 2024-05-07T02:36:16.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.955+0000 7f06d2c7ab80 1 Processor -- start 2024-05-07T02:36:16.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.956+0000 7f06d2c7ab80 1 -- start start 2024-05-07T02:36:16.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.956+0000 7f06d2c7ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adcc4b9620 0x55adcc4b9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:16.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.956+0000 7f06d2c7ab80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adcc4b9f70 0x55adcc4bdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:16.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.956+0000 7f06d2c7ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55adcc4bf470 0x55adcc4c0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:16.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.956+0000 7f06d2c7ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f06bc1b6ce0 con 0x55adcc4b9620 2024-05-07T02:36:16.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.956+0000 7f06d2c7ab80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f06bc1b8880 con 0x55adcc4b9f70 2024-05-07T02:36:16.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.956+0000 7f06d2c7ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f06bc1b73b0 con 0x55adcc4bf470 2024-05-07T02:36:16.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:16 smithi027 ceph-mon[34398]: pgmap v136: 129 pgs: 129 active+clean; 81 MiB data, 834 MiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 1.3 MiB/s wr, 205 op/s 2024-05-07T02:36:16.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.956+0000 7f06c2200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55adcc4bf470 0x55adcc4c0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:16.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.957+0000 7f06c1800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adcc4b9620 0x55adcc4b9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:16.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.957+0000 7f06c2200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55adcc4bf470 0x55adcc4c0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:50578/0 (socket says 172.21.15.19:50578) 2024-05-07T02:36:16.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.957+0000 7f06c2200700 1 -- 172.21.15.19:0/753471394 learned_addr learned my addr 172.21.15.19:0/753471394 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:36:16.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.957+0000 7f06c0e00700 1 --2- 172.21.15.19:0/753471394 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adcc4b9f70 0x55adcc4bdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:16.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.957+0000 7f06c2200700 1 -- 172.21.15.19:0/753471394 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adcc4b9f70 msgr2=0x55adcc4bdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:36:16.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.957+0000 7f06c2200700 1 --2- 172.21.15.19:0/753471394 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adcc4b9f70 0x55adcc4bdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:16.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.957+0000 7f06c2200700 1 -- 172.21.15.19:0/753471394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adcc4b9620 msgr2=0x55adcc4b9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:36:16.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.957+0000 7f06c2200700 1 --2- 172.21.15.19:0/753471394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adcc4b9620 0x55adcc4b9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:16.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.957+0000 7f06c2200700 1 -- 172.21.15.19:0/753471394 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adcc4bfc70 con 0x55adcc4bf470 2024-05-07T02:36:16.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.957+0000 7f06c0e00700 1 --2- 172.21.15.19:0/753471394 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adcc4b9f70 0x55adcc4bdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:36:16.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.957+0000 7f06c2200700 1 --2- 172.21.15.19:0/753471394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55adcc4bf470 0x55adcc4c0e70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f06bc076ae0 tx=0x7f06b4024000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97af1e24f50e0cd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:36:16.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.958+0000 7f06ae000700 1 -- 172.21.15.19:0/753471394 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b40075c0 con 0x55adcc4bf470 2024-05-07T02:36:16.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.958+0000 7f06ae000700 1 -- 172.21.15.19:0/753471394 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06b4007c10 con 0x55adcc4bf470 2024-05-07T02:36:16.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.958+0000 7f06ae000700 1 -- 172.21.15.19:0/753471394 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b401fa80 con 0x55adcc4bf470 2024-05-07T02:36:16.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.959+0000 7f06d2c7ab80 1 -- 172.21.15.19:0/753471394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55adcc4bf470 msgr2=0x55adcc4c0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:16.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.959+0000 7f06d2c7ab80 1 --2- 172.21.15.19:0/753471394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55adcc4bf470 0x55adcc4c0e70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f06bc076ae0 tx=0x7f06b4024000 comp rx=0 tx=0).stop 2024-05-07T02:36:16.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.959+0000 7f06d2c7ab80 1 -- 172.21.15.19:0/753471394 shutdown_connections 2024-05-07T02:36:16.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.959+0000 7f06d2c7ab80 1 --2- 172.21.15.19:0/753471394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55adcc4bf470 0x55adcc4c0e70 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:16.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.959+0000 7f06d2c7ab80 1 --2- 172.21.15.19:0/753471394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adcc4b9620 0x55adcc4b9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:16.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.959+0000 7f06d2c7ab80 1 --2- 172.21.15.19:0/753471394 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adcc4b9f70 0x55adcc4bdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:16.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.959+0000 7f06d2c7ab80 1 -- 172.21.15.19:0/753471394 >> 172.21.15.19:0/753471394 conn(0x55adcc33be40 msgr2=0x55adcc4c2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:36:16.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.959+0000 7f06d2c7ab80 1 -- 172.21.15.19:0/753471394 shutdown_connections 2024-05-07T02:36:16.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.959+0000 7f06d2c7ab80 1 -- 172.21.15.19:0/753471394 wait complete. 2024-05-07T02:36:16.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.960+0000 7f06d2c7ab80 1 -- start start 2024-05-07T02:36:16.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.960+0000 7f06d2c7ab80 1 Processor -- start 2024-05-07T02:36:16.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.960+0000 7f06d2c7ab80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adcc4b9620 0x55adcc4db070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:16.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.960+0000 7f06d2c7ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adcc4b9f70 0x55adcc4db5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:16.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.961+0000 7f06d2c7ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55adcc4bf470 0x55adcc53d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:16.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.961+0000 7f06d2c7ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f06bc1b81b0 con 0x55adcc4b9f70 2024-05-07T02:36:16.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.961+0000 7f06d2c7ab80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f06bc1b7ae0 con 0x55adcc4b9620 2024-05-07T02:36:16.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.961+0000 7f06d2c7ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f06bc1b4c80 con 0x55adcc4bf470 2024-05-07T02:36:16.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.961+0000 7f06d2c7ab80 10 client.0 _ll_register_callbacks cb 0x55adcc36f380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:36:16.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.961+0000 7f06c1800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adcc4b9620 0x55adcc4db070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:16.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.961+0000 7f06c1800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adcc4b9620 0x55adcc4db070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:42412/0 (socket says 172.21.15.19:42412) 2024-05-07T02:36:16.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.961+0000 7f06c1800700 1 -- 172.21.15.19:0/250079383 learned_addr learned my addr 172.21.15.19:0/250079383 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:36:16.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.961+0000 7f06c2200700 1 --2- 172.21.15.19:0/250079383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55adcc4bf470 0x55adcc53d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:16.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.961+0000 7f06c0e00700 1 --2- 172.21.15.19:0/250079383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adcc4b9f70 0x55adcc4db5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:16.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.961+0000 7f06c1800700 1 -- 172.21.15.19:0/250079383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55adcc4bf470 msgr2=0x55adcc53d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:36:16.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.961+0000 7f06c1800700 1 --2- 172.21.15.19:0/250079383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55adcc4bf470 0x55adcc53d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:16.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.961+0000 7f06c1800700 1 -- 172.21.15.19:0/250079383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adcc4b9f70 msgr2=0x55adcc4db5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:36:16.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.961+0000 7f06c1800700 1 --2- 172.21.15.19:0/250079383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adcc4b9f70 0x55adcc4db5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:16.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.961+0000 7f06c1800700 1 -- 172.21.15.19:0/250079383 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adcc4dbe40 con 0x55adcc4b9620 2024-05-07T02:36:16.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.961+0000 7f06c0e00700 1 --2- 172.21.15.19:0/250079383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adcc4b9f70 0x55adcc4db5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-07T02:36:16.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.962+0000 7f06c1800700 1 --2- 172.21.15.19:0/250079383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adcc4b9620 0x55adcc4db070 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f06b00097c0 tx=0x7f06b0009f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3128304546523a50 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:36:16.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.962+0000 7f06a3400700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:36:16.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.962+0000 7f06a3400700 1 -- 172.21.15.19:0/250079383 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b003b020 con 0x55adcc4b9620 2024-05-07T02:36:16.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.962+0000 7f06d2c7ab80 1 -- 172.21.15.19:0/250079383 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06b4007c10 con 0x55adcc4b9620 2024-05-07T02:36:16.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.962+0000 7f06a3400700 1 -- 172.21.15.19:0/250079383 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f06b0019620 con 0x55adcc4b9620 2024-05-07T02:36:16.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.962+0000 7f06a3400700 1 -- 172.21.15.19:0/250079383 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f06b0046050 con 0x55adcc4b9620 2024-05-07T02:36:16.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.962+0000 7f06d2c7ab80 10 client.26012 fetch_fsmap learned FSMap version 40 2024-05-07T02:36:16.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.962+0000 7f06d2c7ab80 1 -- 172.21.15.19:0/250079383 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55adcc53e5f0 con 0x55adcc4b9620 2024-05-07T02:36:16.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.962+0000 7f06a3400700 1 -- 172.21.15.19:0/250079383 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06b4007c10 con 0x55adcc4b9620 2024-05-07T02:36:16.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.963+0000 7f06a3400700 1 -- 172.21.15.19:0/250079383 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f06b0004640 con 0x55adcc4b9620 2024-05-07T02:36:16.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.963+0000 7f06d2c7ab80 10 client.26012 fetch_fsmap finished waiting for FSMap version 40 2024-05-07T02:36:16.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.963+0000 7f06d2c7ab80 10 client.26012 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:36:16.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.963+0000 7f06d2c7ab80 20 client.26012 populate_metadata read hostname 'smithi019' 2024-05-07T02:36:16.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.963+0000 7f06d2c7ab80 1 --2- 172.21.15.19:0/250079383 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55adcc53ecc0 0x55adcc5410e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:16.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.963+0000 7f06d2c7ab80 4 client.26012 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-07T02:36:16.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.963+0000 7f06d2c7ab80 1 -- 172.21.15.19:0/250079383 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55adcc3e9400 con 0x55adcc53ecc0 2024-05-07T02:36:16.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.963+0000 7f06c0e00700 1 --2- 172.21.15.19:0/250079383 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55adcc53ecc0 0x55adcc5410e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:16.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.964+0000 7f06c0e00700 1 --2- 172.21.15.19:0/250079383 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55adcc53ecc0 0x55adcc5410e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d40e4241980a1570 server_cookie=a55bb7b33b86698c in_seq=0 out_seq=0 2024-05-07T02:36:16.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.964+0000 7f06a3400700 10 client.26012 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:36:16.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.964+0000 7f06a3400700 1 -- 172.21.15.19:0/250079383 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55adcc3e9400 con 0x55adcc53ecc0 2024-05-07T02:36:16.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.964+0000 7f06a3400700 10 client.26012 handle_command_reply: tid=0 2024-05-07T02:36:16.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.964+0000 7f06d2c7ab80 1 client.26012 shutdown 2024-05-07T02:36:16.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.964+0000 7f06d2c7ab80 1 -- 172.21.15.19:0/250079383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adcc4b9620 msgr2=0x55adcc4db070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:16.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.964+0000 7f06d2c7ab80 1 --2- 172.21.15.19:0/250079383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adcc4b9620 0x55adcc4db070 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f06b00097c0 tx=0x7f06b0009f80 comp rx=0 tx=0).stop 2024-05-07T02:36:16.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.964+0000 7f06d2c7ab80 1 -- 172.21.15.19:0/250079383 shutdown_connections 2024-05-07T02:36:16.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.964+0000 7f06d2c7ab80 1 --2- 172.21.15.19:0/250079383 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55adcc53ecc0 0x55adcc5410e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:16.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.964+0000 7f06d2c7ab80 1 --2- 172.21.15.19:0/250079383 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55adcc4bf470 0x55adcc53d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:16.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.964+0000 7f06d2c7ab80 1 --2- 172.21.15.19:0/250079383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55adcc4b9620 0x55adcc4db070 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:16.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.964+0000 7f06d2c7ab80 1 --2- 172.21.15.19:0/250079383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55adcc4b9f70 0x55adcc4db5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:16.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.964+0000 7f06a3400700 0 client.26012 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:36:16.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.964+0000 7f06d2c7ab80 1 -- 172.21.15.19:0/250079383 >> 172.21.15.19:0/250079383 conn(0x55adcc33be40 msgr2=0x55adcc4c2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:36:16.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.965+0000 7f06d2c7ab80 1 -- 172.21.15.19:0/250079383 shutdown_connections 2024-05-07T02:36:16.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.965+0000 7f06d2c7ab80 1 -- 172.21.15.19:0/250079383 wait complete. 2024-05-07T02:36:16.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.965+0000 7f06d2c7ab80 20 client.26012 trim_cache size 0 max 16384 2024-05-07T02:36:16.967 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:36:16.968 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-07T02:36:16.968 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-05-07T02:36:16.968 INFO:teuthology.orchestra.run.smithi019.stdout: "981dc9e9-c23a-4c11-a45e-73069346138a": { 2024-05-07T02:36:16.968 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-05-07T02:36:16.968 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "981dc9e9-c23a-4c11-a45e-73069346138a", 2024-05-07T02:36:16.968 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-05-07T02:36:16.968 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:36:16.968 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:36:16.968 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:36:16.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.965+0000 7f06c4200700 1 -- 172.21.15.19:0/2716995830 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f06a40b2a70 msgr2=0x7f06a40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:16.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.965+0000 7f06c4200700 1 --2- 172.21.15.19:0/2716995830 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f06a40b2a70 0x7f06a40b4f80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f06b4009fd0 tx=0x7f06b400bb90 comp rx=0 tx=0).stop 2024-05-07T02:36:16.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.965+0000 7f06c4200700 1 -- 172.21.15.19:0/2716995830 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06bc076450 msgr2=0x7f06bc10dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:16.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.965+0000 7f06c4200700 1 --2- 172.21.15.19:0/2716995830 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06bc076450 0x7f06bc10dfc0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f06a800e9f0 tx=0x7f06a800ed00 comp rx=0 tx=0).stop 2024-05-07T02:36:16.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.966+0000 7f06c4200700 1 -- 172.21.15.19:0/2716995830 shutdown_connections 2024-05-07T02:36:16.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.966+0000 7f06c4200700 1 --2- 172.21.15.19:0/2716995830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f06bc110c00 0x7f06bc10e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:16.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.966+0000 7f06c4200700 1 --2- 172.21.15.19:0/2716995830 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f06a40b2a70 0x7f06a40b4f80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:16.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.966+0000 7f06c4200700 1 --2- 172.21.15.19:0/2716995830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06bc075b00 0x7f06bc10da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:16.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.966+0000 7f06c4200700 1 --2- 172.21.15.19:0/2716995830 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06bc076450 0x7f06bc10dfc0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:16.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.966+0000 7f06c4200700 1 -- 172.21.15.19:0/2716995830 >> 172.21.15.19:0/2716995830 conn(0x7f06bc100040 msgr2=0x7f06bc106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:36:16.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.966+0000 7f06c4200700 1 -- 172.21.15.19:0/2716995830 shutdown_connections 2024-05-07T02:36:16.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:16.966+0000 7f06c4200700 1 -- 172.21.15.19:0/2716995830 wait complete. 2024-05-07T02:36:17.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:16 smithi064 ceph-mon[34699]: pgmap v136: 129 pgs: 129 active+clean; 81 MiB data, 834 MiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 1.3 MiB/s wr, 205 op/s 2024-05-07T02:36:17.745 INFO:tasks.workunit.client.0.smithi019.stdout: 1 113367 2.93 MB/sec execute 73 sec latency 50.758 ms 2024-05-07T02:36:18.138 INFO:tasks.cephfs.filesystem:scrub status for tag:981dc9e9-c23a-4c11-a45e-73069346138a - {'path': '/', 'tag': '981dc9e9-c23a-4c11-a45e-73069346138a', 'options': 'recursive,force'} 2024-05-07T02:36:18.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:18 smithi064 ceph-mon[34699]: pgmap v137: 129 pgs: 129 active+clean; 81 MiB data, 834 MiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 1003 KiB/s wr, 129 op/s 2024-05-07T02:36:18.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:18 smithi019 ceph-mon[29781]: pgmap v137: 129 pgs: 129 active+clean; 81 MiB data, 834 MiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 1003 KiB/s wr, 129 op/s 2024-05-07T02:36:18.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:18 smithi027 ceph-mon[34398]: pgmap v137: 129 pgs: 129 active+clean; 81 MiB data, 834 MiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 1003 KiB/s wr, 129 op/s 2024-05-07T02:36:18.745 INFO:tasks.workunit.client.0.smithi019.stdout: 1 113919 2.95 MB/sec execute 74 sec latency 30.815 ms 2024-05-07T02:36:19.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:19 smithi064 ceph-mon[34699]: scrub summary: idle 2024-05-07T02:36:19.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:19 smithi064 ceph-mon[34699]: scrub summary: idle 2024-05-07T02:36:19.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:19 smithi064 ceph-mon[34699]: scrub summary: idle 2024-05-07T02:36:19.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:19 smithi019 ceph-mon[29781]: scrub summary: idle 2024-05-07T02:36:19.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:19 smithi019 ceph-mon[29781]: scrub summary: idle 2024-05-07T02:36:19.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:19 smithi019 ceph-mon[29781]: scrub summary: idle 2024-05-07T02:36:19.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:19 smithi027 ceph-mon[34398]: scrub summary: idle 2024-05-07T02:36:19.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:19 smithi027 ceph-mon[34398]: scrub summary: idle 2024-05-07T02:36:19.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:19 smithi027 ceph-mon[34398]: scrub summary: idle 2024-05-07T02:36:19.745 INFO:tasks.workunit.client.0.smithi019.stdout: 1 114533 2.93 MB/sec execute 75 sec latency 22.401 ms 2024-05-07T02:36:20.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:20 smithi064 ceph-mon[34699]: scrub summary: idle 2024-05-07T02:36:20.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:20 smithi064 ceph-mon[34699]: scrub summary: idle 2024-05-07T02:36:20.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:20 smithi064 ceph-mon[34699]: pgmap v138: 129 pgs: 129 active+clean; 81 MiB data, 834 MiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 1003 KiB/s wr, 129 op/s 2024-05-07T02:36:20.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:20 smithi019 ceph-mon[29781]: scrub summary: idle 2024-05-07T02:36:20.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:20 smithi019 ceph-mon[29781]: scrub summary: idle 2024-05-07T02:36:20.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:20 smithi019 ceph-mon[29781]: pgmap v138: 129 pgs: 129 active+clean; 81 MiB data, 834 MiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 1003 KiB/s wr, 129 op/s 2024-05-07T02:36:20.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:20 smithi027 ceph-mon[34398]: scrub summary: idle 2024-05-07T02:36:20.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:20 smithi027 ceph-mon[34398]: scrub summary: idle 2024-05-07T02:36:20.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:20 smithi027 ceph-mon[34398]: pgmap v138: 129 pgs: 129 active+clean; 81 MiB data, 834 MiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 1003 KiB/s wr, 129 op/s 2024-05-07T02:36:20.745 INFO:tasks.workunit.client.0.smithi019.stdout: 1 116302 3.02 MB/sec execute 76 sec latency 15.033 ms 2024-05-07T02:36:21.746 INFO:tasks.workunit.client.0.smithi019.stdout: 1 117358 3.08 MB/sec execute 77 sec latency 19.601 ms 2024-05-07T02:36:22.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:22 smithi064 ceph-mon[34699]: pgmap v139: 129 pgs: 129 active+clean; 85 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 1.4 MiB/s wr, 195 op/s 2024-05-07T02:36:22.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:22 smithi019 ceph-mon[29781]: pgmap v139: 129 pgs: 129 active+clean; 85 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 1.4 MiB/s wr, 195 op/s 2024-05-07T02:36:22.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:22 smithi027 ceph-mon[34398]: pgmap v139: 129 pgs: 129 active+clean; 85 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 1.4 MiB/s wr, 195 op/s 2024-05-07T02:36:22.746 INFO:tasks.workunit.client.0.smithi019.stdout: 1 117766 3.05 MB/sec execute 78 sec latency 22.350 ms 2024-05-07T02:36:23.746 INFO:tasks.workunit.client.0.smithi019.stdout: 1 119293 3.08 MB/sec execute 79 sec latency 22.771 ms 2024-05-07T02:36:24.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:24 smithi064 ceph-mon[34699]: pgmap v140: 129 pgs: 129 active+clean; 85 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 1007 KiB/s wr, 132 op/s 2024-05-07T02:36:24.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:24 smithi019 ceph-mon[29781]: pgmap v140: 129 pgs: 129 active+clean; 85 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 1007 KiB/s wr, 132 op/s 2024-05-07T02:36:24.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:24 smithi027 ceph-mon[34398]: pgmap v140: 129 pgs: 129 active+clean; 85 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 1007 KiB/s wr, 132 op/s 2024-05-07T02:36:24.746 INFO:tasks.workunit.client.0.smithi019.stdout: 1 120642 3.19 MB/sec execute 80 sec latency 16.286 ms 2024-05-07T02:36:25.746 INFO:tasks.workunit.client.0.smithi019.stdout: 1 121075 3.17 MB/sec execute 81 sec latency 32.905 ms 2024-05-07T02:36:26.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:26 smithi064 ceph-mon[34699]: pgmap v141: 129 pgs: 129 active+clean; 89 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 193 op/s 2024-05-07T02:36:26.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:26 smithi019 ceph-mon[29781]: pgmap v141: 129 pgs: 129 active+clean; 89 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 193 op/s 2024-05-07T02:36:26.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:26 smithi027 ceph-mon[34398]: pgmap v141: 129 pgs: 129 active+clean; 89 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 193 op/s 2024-05-07T02:36:26.746 INFO:tasks.workunit.client.0.smithi019.stdout: 1 121854 3.16 MB/sec execute 82 sec latency 19.585 ms 2024-05-07T02:36:27.746 INFO:tasks.workunit.client.0.smithi019.stdout: 1 123605 3.24 MB/sec execute 83 sec latency 16.107 ms 2024-05-07T02:36:28.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:28 smithi064 ceph-mon[34699]: pgmap v142: 129 pgs: 129 active+clean; 89 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 849 KiB/s wr, 125 op/s 2024-05-07T02:36:28.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:28 smithi019 ceph-mon[29781]: pgmap v142: 129 pgs: 129 active+clean; 89 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 849 KiB/s wr, 125 op/s 2024-05-07T02:36:28.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:28 smithi027 ceph-mon[34398]: pgmap v142: 129 pgs: 129 active+clean; 89 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 849 KiB/s wr, 125 op/s 2024-05-07T02:36:28.746 INFO:tasks.workunit.client.0.smithi019.stdout: 1 124489 3.28 MB/sec execute 84 sec latency 24.401 ms 2024-05-07T02:36:29.747 INFO:tasks.workunit.client.0.smithi019.stdout: 1 124992 3.25 MB/sec execute 85 sec latency 27.692 ms 2024-05-07T02:36:30.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:30 smithi064 ceph-mon[34699]: pgmap v143: 129 pgs: 129 active+clean; 89 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 849 KiB/s wr, 126 op/s 2024-05-07T02:36:30.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:30 smithi019 ceph-mon[29781]: pgmap v143: 129 pgs: 129 active+clean; 89 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 849 KiB/s wr, 126 op/s 2024-05-07T02:36:30.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:30 smithi027 ceph-mon[34398]: pgmap v143: 129 pgs: 129 active+clean; 89 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 849 KiB/s wr, 126 op/s 2024-05-07T02:36:30.747 INFO:tasks.workunit.client.0.smithi019.stdout: 1 126622 3.29 MB/sec execute 86 sec latency 19.622 ms 2024-05-07T02:36:31.747 INFO:tasks.workunit.client.0.smithi019.stdout: 1 127887 3.37 MB/sec execute 87 sec latency 17.313 ms 2024-05-07T02:36:32.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:32 smithi064 ceph-mon[34699]: pgmap v144: 129 pgs: 129 active+clean; 92 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 194 op/s 2024-05-07T02:36:32.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:32 smithi019 ceph-mon[29781]: pgmap v144: 129 pgs: 129 active+clean; 92 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 194 op/s 2024-05-07T02:36:32.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:32 smithi027 ceph-mon[34398]: pgmap v144: 129 pgs: 129 active+clean; 92 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 194 op/s 2024-05-07T02:36:32.747 INFO:tasks.workunit.client.0.smithi019.stdout: 1 128185 3.35 MB/sec execute 88 sec latency 24.411 ms 2024-05-07T02:36:33.747 INFO:tasks.workunit.client.0.smithi019.stdout: 1 128949 3.34 MB/sec execute 89 sec latency 17.645 ms 2024-05-07T02:36:34.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:34 smithi064 ceph-mon[34699]: pgmap v145: 129 pgs: 129 active+clean; 92 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 864 KiB/s wr, 128 op/s 2024-05-07T02:36:34.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:34 smithi019 ceph-mon[29781]: pgmap v145: 129 pgs: 129 active+clean; 92 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 864 KiB/s wr, 128 op/s 2024-05-07T02:36:34.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:34 smithi027 ceph-mon[34398]: pgmap v145: 129 pgs: 129 active+clean; 92 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 864 KiB/s wr, 128 op/s 2024-05-07T02:36:34.747 INFO:tasks.workunit.client.0.smithi019.stdout: 1 130719 3.42 MB/sec execute 90 sec latency 13.652 ms 2024-05-07T02:36:35.747 INFO:tasks.workunit.client.0.smithi019.stdout: 1 131584 3.45 MB/sec execute 91 sec latency 22.248 ms 2024-05-07T02:36:36.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:36 smithi064 ceph-mon[34699]: pgmap v146: 129 pgs: 129 active+clean; 98 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 196 op/s 2024-05-07T02:36:36.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:36 smithi019 ceph-mon[29781]: pgmap v146: 129 pgs: 129 active+clean; 98 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 196 op/s 2024-05-07T02:36:36.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:36 smithi027 ceph-mon[34398]: pgmap v146: 129 pgs: 129 active+clean; 98 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 196 op/s 2024-05-07T02:36:36.747 INFO:tasks.workunit.client.0.smithi019.stdout: 1 132160 3.42 MB/sec execute 92 sec latency 38.105 ms 2024-05-07T02:36:37.747 INFO:tasks.workunit.client.0.smithi019.stdout: 1 133695 3.45 MB/sec execute 93 sec latency 18.017 ms 2024-05-07T02:36:38.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:38 smithi064 ceph-mon[34699]: pgmap v147: 129 pgs: 129 active+clean; 98 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 992 KiB/s wr, 135 op/s 2024-05-07T02:36:38.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:38 smithi019 ceph-mon[29781]: pgmap v147: 129 pgs: 129 active+clean; 98 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 992 KiB/s wr, 135 op/s 2024-05-07T02:36:38.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:38 smithi027 ceph-mon[34398]: pgmap v147: 129 pgs: 129 active+clean; 98 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 992 KiB/s wr, 135 op/s 2024-05-07T02:36:38.748 INFO:tasks.workunit.client.0.smithi019.stdout: 1 134908 3.53 MB/sec execute 94 sec latency 17.433 ms 2024-05-07T02:36:39.748 INFO:tasks.workunit.client.0.smithi019.stdout: 1 135237 3.51 MB/sec execute 95 sec latency 30.255 ms 2024-05-07T02:36:40.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:40 smithi064 ceph-mon[34699]: pgmap v148: 129 pgs: 129 active+clean; 98 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 992 KiB/s wr, 135 op/s 2024-05-07T02:36:40.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:40 smithi019 ceph-mon[29781]: pgmap v148: 129 pgs: 129 active+clean; 98 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 992 KiB/s wr, 135 op/s 2024-05-07T02:36:40.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:40 smithi027 ceph-mon[34398]: pgmap v148: 129 pgs: 129 active+clean; 98 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 992 KiB/s wr, 135 op/s 2024-05-07T02:36:40.748 INFO:tasks.workunit.client.0.smithi019.stdout: 1 135858 3.49 MB/sec execute 96 sec latency 26.734 ms 2024-05-07T02:36:41.748 INFO:tasks.workunit.client.0.smithi019.stdout: 1 137276 3.51 MB/sec execute 97 sec latency 41.297 ms 2024-05-07T02:36:42.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:42 smithi064 ceph-mon[34699]: pgmap v149: 129 pgs: 129 active+clean; 98 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 1.3 MiB/s wr, 201 op/s 2024-05-07T02:36:42.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:42 smithi019 ceph-mon[29781]: pgmap v149: 129 pgs: 129 active+clean; 98 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 1.3 MiB/s wr, 201 op/s 2024-05-07T02:36:42.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:42 smithi027 ceph-mon[34398]: pgmap v149: 129 pgs: 129 active+clean; 98 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 1.3 MiB/s wr, 201 op/s 2024-05-07T02:36:42.748 INFO:tasks.workunit.client.0.smithi019.stdout: 1 138505 3.58 MB/sec execute 98 sec latency 18.174 ms 2024-05-07T02:36:43.748 INFO:tasks.workunit.client.0.smithi019.stdout: 1 138815 3.56 MB/sec execute 99 sec latency 25.051 ms 2024-05-07T02:36:44.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:44 smithi064 ceph-mon[34699]: pgmap v150: 129 pgs: 129 active+clean; 98 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 936 KiB/s wr, 132 op/s 2024-05-07T02:36:44.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:44 smithi019 ceph-mon[29781]: pgmap v150: 129 pgs: 129 active+clean; 98 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 936 KiB/s wr, 132 op/s 2024-05-07T02:36:44.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:44 smithi027 ceph-mon[34398]: pgmap v150: 129 pgs: 129 active+clean; 98 MiB data, 865 MiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 936 KiB/s wr, 132 op/s 2024-05-07T02:36:44.748 INFO:tasks.workunit.client.0.smithi019.stdout: 1 139522 3.54 MB/sec execute 100 sec latency 21.194 ms 2024-05-07T02:36:45.748 INFO:tasks.workunit.client.0.smithi019.stdout: 1 140938 3.57 MB/sec execute 101 sec latency 22.387 ms 2024-05-07T02:36:46.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:46 smithi064 ceph-mon[34699]: pgmap v151: 129 pgs: 129 active+clean; 100 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 207 op/s 2024-05-07T02:36:46.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:46 smithi019 ceph-mon[29781]: pgmap v151: 129 pgs: 129 active+clean; 100 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 207 op/s 2024-05-07T02:36:46.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:46 smithi027 ceph-mon[34398]: pgmap v151: 129 pgs: 129 active+clean; 100 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 207 op/s 2024-05-07T02:36:46.749 INFO:tasks.workunit.client.0.smithi019.stdout: 1 142127 3.64 MB/sec execute 102 sec latency 34.564 ms 2024-05-07T02:36:47.749 INFO:tasks.workunit.client.0.smithi019.stdout: 1 142417 3.61 MB/sec execute 103 sec latency 20.664 ms 2024-05-07T02:36:48.140 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-07T02:36:48.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:48 smithi064 ceph-mon[34699]: pgmap v152: 129 pgs: 129 active+clean; 100 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 139 op/s 2024-05-07T02:36:48.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:48 smithi019 ceph-mon[29781]: pgmap v152: 129 pgs: 129 active+clean; 100 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 139 op/s 2024-05-07T02:36:48.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:48 smithi027 ceph-mon[34398]: pgmap v152: 129 pgs: 129 active+clean; 100 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 139 op/s 2024-05-07T02:36:48.634 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:36:48.748 INFO:tasks.workunit.client.0.smithi019.stdout: 1 143570 3.62 MB/sec execute 104 sec latency 16.394 ms 2024-05-07T02:36:49.749 INFO:tasks.workunit.client.0.smithi019.stdout: 1 145083 3.66 MB/sec execute 105 sec latency 17.418 ms 2024-05-07T02:36:50.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:50 smithi064 ceph-mon[34699]: pgmap v153: 129 pgs: 129 active+clean; 100 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 139 op/s 2024-05-07T02:36:50.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:50 smithi019 ceph-mon[29781]: pgmap v153: 129 pgs: 129 active+clean; 100 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 139 op/s 2024-05-07T02:36:50.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:50 smithi027 ceph-mon[34398]: pgmap v153: 129 pgs: 129 active+clean; 100 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 139 op/s 2024-05-07T02:36:50.749 INFO:tasks.workunit.client.0.smithi019.stdout: 1 145837 3.68 MB/sec execute 106 sec latency 21.229 ms 2024-05-07T02:36:51.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.436+0000 7f8042200700 1 -- 172.21.15.19:0/189744836 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f803c110c70 msgr2=0x7f803c113090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:51.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.436+0000 7f8042200700 1 --2- 172.21.15.19:0/189744836 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f803c110c70 0x7f803c113090 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f803400b0a0 tx=0x7f803400b3b0 comp rx=0 tx=0).stop 2024-05-07T02:36:51.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.436+0000 7f8042200700 1 -- 172.21.15.19:0/189744836 shutdown_connections 2024-05-07T02:36:51.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.436+0000 7f8042200700 1 --2- 172.21.15.19:0/189744836 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f803c110c70 0x7f803c113090 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:51.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.436+0000 7f8042200700 1 --2- 172.21.15.19:0/189744836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f803c1030c0 0x7f803c1034d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:51.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.436+0000 7f8042200700 1 --2- 172.21.15.19:0/189744836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f803c103a10 0x7f803c110730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:51.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.436+0000 7f8042200700 1 -- 172.21.15.19:0/189744836 >> 172.21.15.19:0/189744836 conn(0x7f803c0fedc0 msgr2=0x7f803c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:36:51.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.436+0000 7f8042200700 1 -- 172.21.15.19:0/189744836 shutdown_connections 2024-05-07T02:36:51.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.437+0000 7f8042200700 1 -- 172.21.15.19:0/189744836 wait complete. 2024-05-07T02:36:51.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.438+0000 7f8042200700 1 Processor -- start 2024-05-07T02:36:51.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.438+0000 7f8042200700 1 -- start start 2024-05-07T02:36:51.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.439+0000 7f8042200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f803c1030c0 0x7f803c19cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:51.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.439+0000 7f8042200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f803c103a10 0x7f803c19d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:51.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.439+0000 7f8042200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f803c110c70 0x7f803c1a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:51.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.439+0000 7f8042200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f803c1166d0 con 0x7f803c1030c0 2024-05-07T02:36:51.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.439+0000 7f8042200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f803c116550 con 0x7f803c103a10 2024-05-07T02:36:51.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.439+0000 7f8042200700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f803c116850 con 0x7f803c110c70 2024-05-07T02:36:51.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.439+0000 7f803be00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f803c110c70 0x7f803c1a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:51.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.439+0000 7f803b400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f803c1030c0 0x7f803c19cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:51.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.439+0000 7f803be00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f803c110c70 0x7f803c1a17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:43008/0 (socket says 172.21.15.19:43008) 2024-05-07T02:36:51.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.439+0000 7f803be00700 1 -- 172.21.15.19:0/874008259 learned_addr learned my addr 172.21.15.19:0/874008259 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:36:51.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.439+0000 7f803aa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f803c103a10 0x7f803c19d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:51.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.439+0000 7f803be00700 1 -- 172.21.15.19:0/874008259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f803c103a10 msgr2=0x7f803c19d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:51.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.439+0000 7f803be00700 1 --2- 172.21.15.19:0/874008259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f803c103a10 0x7f803c19d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:51.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.439+0000 7f803be00700 1 -- 172.21.15.19:0/874008259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f803c1030c0 msgr2=0x7f803c19cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:51.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.439+0000 7f803be00700 1 --2- 172.21.15.19:0/874008259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f803c1030c0 0x7f803c19cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:51.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.439+0000 7f803be00700 1 -- 172.21.15.19:0/874008259 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f803c1a1e50 con 0x7f803c110c70 2024-05-07T02:36:51.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.440+0000 7f803be00700 1 --2- 172.21.15.19:0/874008259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f803c110c70 0x7f803c1a17b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f803400b070 tx=0x7f8034009c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:36:51.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.440+0000 7f8033e00700 1 -- 172.21.15.19:0/874008259 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8034021070 con 0x7f803c110c70 2024-05-07T02:36:51.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.440+0000 7f8033e00700 1 -- 172.21.15.19:0/874008259 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8034004e40 con 0x7f803c110c70 2024-05-07T02:36:51.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.440+0000 7f8042200700 1 -- 172.21.15.19:0/874008259 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f803c1a20e0 con 0x7f803c110c70 2024-05-07T02:36:51.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.440+0000 7f8033e00700 1 -- 172.21.15.19:0/874008259 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8034004630 con 0x7f803c110c70 2024-05-07T02:36:51.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.440+0000 7f8042200700 1 -- 172.21.15.19:0/874008259 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f803c1a9a60 con 0x7f803c110c70 2024-05-07T02:36:51.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.444+0000 7f8033e00700 1 -- 172.21.15.19:0/874008259 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f803401d450 con 0x7f803c110c70 2024-05-07T02:36:51.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.446+0000 7f8050e01b80 1 Processor -- start 2024-05-07T02:36:51.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.446+0000 7f8050e01b80 1 -- start start 2024-05-07T02:36:51.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.446+0000 7f8050e01b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5f3553470 0x55b5f3553880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:51.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.446+0000 7f8033e00700 1 --2- 172.21.15.19:0/874008259 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f80200b2a70 0x7f80200b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:51.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.446+0000 7f8050e01b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5f354d620 0x55b5f354da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:51.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.446+0000 7f8050e01b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5f354df70 0x55b5f355a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:51.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.446+0000 7f8050e01b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b5f346fa20 con 0x55b5f354df70 2024-05-07T02:36:51.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.446+0000 7f8050e01b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b5f340d760 con 0x55b5f354d620 2024-05-07T02:36:51.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.446+0000 7f8050e01b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b5f346b9e0 con 0x55b5f3553470 2024-05-07T02:36:51.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.447+0000 7f803aa00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5f3553470 0x55b5f3553880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:51.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.447+0000 7f803aa00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5f3553470 0x55b5f3553880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:43012/0 (socket says 172.21.15.19:43012) 2024-05-07T02:36:51.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.447+0000 7f803aa00700 1 -- 172.21.15.19:0/3905406843 learned_addr learned my addr 172.21.15.19:0/3905406843 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:36:51.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.447+0000 7f803aa00700 1 --2- 172.21.15.19:0/3905406843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5f354d620 0x55b5f354da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:51.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.447+0000 7f8033e00700 1 -- 172.21.15.19:0/874008259 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f803400f020 con 0x7f803c110c70 2024-05-07T02:36:51.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.447+0000 7f803aa00700 1 -- 172.21.15.19:0/3905406843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5f354d620 msgr2=0x55b5f354da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:36:51.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.447+0000 7f803aa00700 1 --2- 172.21.15.19:0/3905406843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5f354d620 0x55b5f354da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:51.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.447+0000 7f803aa00700 1 -- 172.21.15.19:0/3905406843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5f354df70 msgr2=0x55b5f355a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:36:51.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.447+0000 7f803be00700 1 --2- 172.21.15.19:0/3905406843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5f354df70 0x55b5f355a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:51.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.447+0000 7f803aa00700 1 --2- 172.21.15.19:0/3905406843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5f354df70 0x55b5f355a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:51.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.447+0000 7f803aa00700 1 -- 172.21.15.19:0/3905406843 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5f33bc8d0 con 0x55b5f3553470 2024-05-07T02:36:51.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.447+0000 7f803be00700 1 --2- 172.21.15.19:0/3905406843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5f354df70 0x55b5f355a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-07T02:36:51.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.448+0000 7f803b400700 1 --2- 172.21.15.19:0/874008259 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f80200b2a70 0x7f80200b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:51.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.448+0000 7f803aa00700 1 --2- 172.21.15.19:0/3905406843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5f3553470 0x55b5f3553880 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8028005950 tx=0x7f80280058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7129242dc369bb9e server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:36:51.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.448+0000 7f8030c00700 1 -- 172.21.15.19:0/3905406843 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8028013e20 con 0x55b5f3553470 2024-05-07T02:36:51.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.448+0000 7f8030c00700 1 -- 172.21.15.19:0/3905406843 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8028004ce0 con 0x55b5f3553470 2024-05-07T02:36:51.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.448+0000 7f8030c00700 1 -- 172.21.15.19:0/3905406843 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8028010420 con 0x55b5f3553470 2024-05-07T02:36:51.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.449+0000 7f8050e01b80 1 -- 172.21.15.19:0/3905406843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5f3553470 msgr2=0x55b5f3553880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:51.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.450+0000 7f8050e01b80 1 --2- 172.21.15.19:0/3905406843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5f3553470 0x55b5f3553880 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8028005950 tx=0x7f80280058e0 comp rx=0 tx=0).stop 2024-05-07T02:36:51.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.449+0000 7f803b400700 1 --2- 172.21.15.19:0/874008259 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f80200b2a70 0x7f80200b4f80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f802c009680 tx=0x7f802c009260 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:36:51.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.450+0000 7f8050e01b80 1 -- 172.21.15.19:0/3905406843 shutdown_connections 2024-05-07T02:36:51.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.450+0000 7f8050e01b80 1 --2- 172.21.15.19:0/3905406843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5f354df70 0x55b5f355a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:51.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.450+0000 7f8050e01b80 1 --2- 172.21.15.19:0/3905406843 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5f3553470 0x55b5f3553880 secure :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8028005950 tx=0x7f80280058e0 comp rx=0 tx=0).stop 2024-05-07T02:36:51.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.450+0000 7f8050e01b80 1 --2- 172.21.15.19:0/3905406843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5f354d620 0x55b5f354da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:51.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.450+0000 7f8050e01b80 1 -- 172.21.15.19:0/3905406843 >> 172.21.15.19:0/3905406843 conn(0x55b5f33cfe40 msgr2=0x55b5f3548380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:36:51.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.450+0000 7f8050e01b80 1 -- 172.21.15.19:0/3905406843 shutdown_connections 2024-05-07T02:36:51.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.450+0000 7f8050e01b80 1 -- 172.21.15.19:0/3905406843 wait complete. 2024-05-07T02:36:51.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.451+0000 7f8050e01b80 1 -- start start 2024-05-07T02:36:51.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.451+0000 7f8050e01b80 1 Processor -- start 2024-05-07T02:36:51.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.452+0000 7f8050e01b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5f354d620 0x55b5f35577c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:51.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.452+0000 7f8050e01b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5f354df70 0x55b5f3557d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:51.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.452+0000 7f8050e01b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5f3558240 0x55b5f3563d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:51.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.452+0000 7f8050e01b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b5f355a890 con 0x55b5f3558240 2024-05-07T02:36:51.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.452+0000 7f8050e01b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b5f33d5b20 con 0x55b5f354df70 2024-05-07T02:36:51.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.452+0000 7f8050e01b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b5f355aa10 con 0x55b5f354d620 2024-05-07T02:36:51.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.452+0000 7f8050e01b80 10 client.0 _ll_register_callbacks cb 0x55b5f3403380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:36:51.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.453+0000 7f803b400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5f354d620 0x55b5f35577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:51.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.453+0000 7f803b400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5f354d620 0x55b5f35577c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:43026/0 (socket says 172.21.15.19:43026) 2024-05-07T02:36:51.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.453+0000 7f803b400700 1 -- 172.21.15.19:0/1804828327 learned_addr learned my addr 172.21.15.19:0/1804828327 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:36:51.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.453+0000 7f803be00700 1 --2- 172.21.15.19:0/1804828327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5f3558240 0x55b5f3563d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:51.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.453+0000 7f803aa00700 1 --2- 172.21.15.19:0/1804828327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5f354df70 0x55b5f3557d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:51.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.453+0000 7f803b400700 1 -- 172.21.15.19:0/1804828327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5f354df70 msgr2=0x55b5f3557d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:36:51.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.453+0000 7f803b400700 1 --2- 172.21.15.19:0/1804828327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5f354df70 0x55b5f3557d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:51.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.453+0000 7f803b400700 1 -- 172.21.15.19:0/1804828327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5f3558240 msgr2=0x55b5f3563d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:36:51.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.453+0000 7f803b400700 1 --2- 172.21.15.19:0/1804828327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5f3558240 0x55b5f3563d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:51.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.453+0000 7f803b400700 1 -- 172.21.15.19:0/1804828327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5f3435420 con 0x55b5f354d620 2024-05-07T02:36:51.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.453+0000 7f803be00700 1 --2- 172.21.15.19:0/1804828327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5f3558240 0x55b5f3563d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:36:51.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.454+0000 7f803b400700 1 --2- 172.21.15.19:0/1804828327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5f354d620 0x55b5f35577c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f802c009a50 tx=0x7f802c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b853bd422226d67 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:36:51.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.454+0000 7f8025600700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:36:51.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.454+0000 7f8025600700 1 -- 172.21.15.19:0/1804828327 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802c00e9c0 con 0x55b5f354d620 2024-05-07T02:36:51.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.454+0000 7f8050e01b80 1 -- 172.21.15.19:0/1804828327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8028004ce0 con 0x55b5f354d620 2024-05-07T02:36:51.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.454+0000 7f8025600700 1 -- 172.21.15.19:0/1804828327 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f802c00eb60 con 0x55b5f354d620 2024-05-07T02:36:51.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.455+0000 7f8025600700 1 -- 172.21.15.19:0/1804828327 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f802c00ed60 con 0x55b5f354d620 2024-05-07T02:36:51.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.455+0000 7f8050e01b80 10 client.25996 fetch_fsmap learned FSMap version 40 2024-05-07T02:36:51.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.455+0000 7f8050e01b80 1 -- 172.21.15.19:0/1804828327 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55b5f3565b20 con 0x55b5f354d620 2024-05-07T02:36:51.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.455+0000 7f8025600700 1 -- 172.21.15.19:0/1804828327 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8028004ce0 con 0x55b5f354d620 2024-05-07T02:36:51.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.455+0000 7f8025600700 1 -- 172.21.15.19:0/1804828327 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f802c026400 con 0x55b5f354d620 2024-05-07T02:36:51.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.455+0000 7f8050e01b80 10 client.25996 fetch_fsmap finished waiting for FSMap version 40 2024-05-07T02:36:51.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.455+0000 7f8050e01b80 10 client.25996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:36:51.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.455+0000 7f8050e01b80 20 client.25996 populate_metadata read hostname 'smithi019' 2024-05-07T02:36:51.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.455+0000 7f8050e01b80 1 --2- 172.21.15.19:0/1804828327 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55b5f35680b0 0x55b5f356a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:51.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.455+0000 7f8050e01b80 4 client.25996 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:36:51.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.455+0000 7f8050e01b80 1 -- 172.21.15.19:0/1804828327 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5f351a0e0 con 0x55b5f35680b0 2024-05-07T02:36:51.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.456+0000 7f803aa00700 1 --2- 172.21.15.19:0/1804828327 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55b5f35680b0 0x55b5f356a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:51.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.456+0000 7f803aa00700 1 --2- 172.21.15.19:0/1804828327 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55b5f35680b0 0x55b5f356a4d0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f45cffdab9d2d319 server_cookie=3af0c053bc03540a in_seq=0 out_seq=0 2024-05-07T02:36:51.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.456+0000 7f8025600700 10 client.25996 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:36:51.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.457+0000 7f8025600700 1 -- 172.21.15.19:0/1804828327 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b5f351a0e0 con 0x55b5f35680b0 2024-05-07T02:36:51.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.457+0000 7f8025600700 10 client.25996 handle_command_reply: tid=0 2024-05-07T02:36:51.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.457+0000 7f8050e01b80 1 client.25996 shutdown 2024-05-07T02:36:51.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.457+0000 7f8050e01b80 1 -- 172.21.15.19:0/1804828327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5f354d620 msgr2=0x55b5f35577c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:51.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.457+0000 7f8050e01b80 1 --2- 172.21.15.19:0/1804828327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5f354d620 0x55b5f35577c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f802c009a50 tx=0x7f802c0047c0 comp rx=0 tx=0).stop 2024-05-07T02:36:51.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.457+0000 7f8050e01b80 1 -- 172.21.15.19:0/1804828327 shutdown_connections 2024-05-07T02:36:51.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.457+0000 7f8050e01b80 1 --2- 172.21.15.19:0/1804828327 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55b5f35680b0 0x55b5f356a4d0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:51.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.457+0000 7f8025600700 0 client.25996 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:36:51.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.457+0000 7f8050e01b80 1 --2- 172.21.15.19:0/1804828327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5f3558240 0x55b5f3563d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:51.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.457+0000 7f8050e01b80 1 --2- 172.21.15.19:0/1804828327 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5f354d620 0x55b5f35577c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:51.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.458+0000 7f8050e01b80 1 --2- 172.21.15.19:0/1804828327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5f354df70 0x55b5f3557d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:51.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.458+0000 7f8050e01b80 1 -- 172.21.15.19:0/1804828327 >> 172.21.15.19:0/1804828327 conn(0x55b5f33cfe40 msgr2=0x55b5f354c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:36:51.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.458+0000 7f8050e01b80 1 -- 172.21.15.19:0/1804828327 shutdown_connections 2024-05-07T02:36:51.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.458+0000 7f8050e01b80 1 -- 172.21.15.19:0/1804828327 wait complete. 2024-05-07T02:36:51.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.458+0000 7f8050e01b80 20 client.25996 trim_cache size 0 max 16384 2024-05-07T02:36:51.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.488+0000 7f8050e01b80 1 Processor -- start 2024-05-07T02:36:51.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.489+0000 7f8050e01b80 1 -- start start 2024-05-07T02:36:51.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.489+0000 7f8050e01b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5f354d620 0x55b5f3548410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:51.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.489+0000 7f8050e01b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5f354df70 0x55b5f35428d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:51.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.489+0000 7f8050e01b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5f3558240 0x55b5f3563920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:51.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.489+0000 7f8050e01b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f803c1b2630 con 0x55b5f354d620 2024-05-07T02:36:51.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.489+0000 7f8050e01b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f803c1b41d0 con 0x55b5f354df70 2024-05-07T02:36:51.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.489+0000 7f8050e01b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f803c1b2d00 con 0x55b5f3558240 2024-05-07T02:36:51.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.489+0000 7f803aa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5f354df70 0x55b5f35428d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:51.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.489+0000 7f803aa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5f354df70 0x55b5f35428d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:42340/0 (socket says 172.21.15.19:42340) 2024-05-07T02:36:51.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.489+0000 7f803aa00700 1 -- 172.21.15.19:0/4286218139 learned_addr learned my addr 172.21.15.19:0/4286218139 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:36:51.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.489+0000 7f803b400700 1 --2- 172.21.15.19:0/4286218139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5f354d620 0x55b5f3548410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:51.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.490+0000 7f803be00700 1 --2- 172.21.15.19:0/4286218139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5f3558240 0x55b5f3563920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:51.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.490+0000 7f803aa00700 1 -- 172.21.15.19:0/4286218139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5f3558240 msgr2=0x55b5f3563920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:36:51.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.490+0000 7f803aa00700 1 --2- 172.21.15.19:0/4286218139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5f3558240 0x55b5f3563920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:51.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.490+0000 7f803aa00700 1 -- 172.21.15.19:0/4286218139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5f354d620 msgr2=0x55b5f3548410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:36:51.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.490+0000 7f803aa00700 1 --2- 172.21.15.19:0/4286218139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5f354d620 0x55b5f3548410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:51.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.490+0000 7f803aa00700 1 -- 172.21.15.19:0/4286218139 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5f354a060 con 0x55b5f354df70 2024-05-07T02:36:51.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.490+0000 7f803aa00700 1 --2- 172.21.15.19:0/4286218139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5f354df70 0x55b5f35428d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x55b5f3552ff0 tx=0x7f8028009fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd51825eaf49bff8 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:36:51.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.491+0000 7f8027e00700 1 -- 172.21.15.19:0/4286218139 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80280056a0 con 0x55b5f354df70 2024-05-07T02:36:51.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.491+0000 7f8027e00700 1 -- 172.21.15.19:0/4286218139 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8028017040 con 0x55b5f354df70 2024-05-07T02:36:51.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.491+0000 7f8027e00700 1 -- 172.21.15.19:0/4286218139 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8028029690 con 0x55b5f354df70 2024-05-07T02:36:51.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.492+0000 7f8050e01b80 1 -- 172.21.15.19:0/4286218139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5f354df70 msgr2=0x55b5f35428d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:51.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.492+0000 7f8050e01b80 1 --2- 172.21.15.19:0/4286218139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5f354df70 0x55b5f35428d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x55b5f3552ff0 tx=0x7f8028009fd0 comp rx=0 tx=0).stop 2024-05-07T02:36:51.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.492+0000 7f8050e01b80 1 -- 172.21.15.19:0/4286218139 shutdown_connections 2024-05-07T02:36:51.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.492+0000 7f8050e01b80 1 --2- 172.21.15.19:0/4286218139 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5f3558240 0x55b5f3563920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:51.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.492+0000 7f8050e01b80 1 --2- 172.21.15.19:0/4286218139 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5f354d620 0x55b5f3548410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:51.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.492+0000 7f8050e01b80 1 --2- 172.21.15.19:0/4286218139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5f354df70 0x55b5f35428d0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:51.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.492+0000 7f8050e01b80 1 -- 172.21.15.19:0/4286218139 >> 172.21.15.19:0/4286218139 conn(0x55b5f33cfe40 msgr2=0x55b5f35421c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:36:51.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.492+0000 7f8050e01b80 1 -- 172.21.15.19:0/4286218139 shutdown_connections 2024-05-07T02:36:51.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.492+0000 7f8050e01b80 1 -- 172.21.15.19:0/4286218139 wait complete. 2024-05-07T02:36:51.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.493+0000 7f8050e01b80 1 -- start start 2024-05-07T02:36:51.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.493+0000 7f8050e01b80 1 Processor -- start 2024-05-07T02:36:51.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.493+0000 7f8050e01b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5f354d620 0x55b5f356f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:51.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.493+0000 7f8050e01b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5f354df70 0x55b5f356f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:51.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.493+0000 7f8050e01b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5f3558240 0x55b5f356fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:51.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.494+0000 7f8050e01b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f803c1b3b00 con 0x55b5f354df70 2024-05-07T02:36:51.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.494+0000 7f8050e01b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f803c1b3430 con 0x55b5f354d620 2024-05-07T02:36:51.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.494+0000 7f8050e01b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f803c1b05d0 con 0x55b5f3558240 2024-05-07T02:36:51.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.494+0000 7f8050e01b80 10 client.0 _ll_register_callbacks cb 0x55b5f3403380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:36:51.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.494+0000 7f803b400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5f354d620 0x55b5f356f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:51.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.494+0000 7f803aa00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5f354df70 0x55b5f356f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:51.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.494+0000 7f803b400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5f354d620 0x55b5f356f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:42350/0 (socket says 172.21.15.19:42350) 2024-05-07T02:36:51.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.494+0000 7f803b400700 1 -- 172.21.15.19:0/282584099 learned_addr learned my addr 172.21.15.19:0/282584099 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:36:51.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.494+0000 7f803be00700 1 --2- 172.21.15.19:0/282584099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5f3558240 0x55b5f356fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:51.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.494+0000 7f803b400700 1 -- 172.21.15.19:0/282584099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5f3558240 msgr2=0x55b5f356fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:36:51.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.494+0000 7f803b400700 1 --2- 172.21.15.19:0/282584099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5f3558240 0x55b5f356fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:51.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.494+0000 7f803b400700 1 -- 172.21.15.19:0/282584099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5f354df70 msgr2=0x55b5f356f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:36:51.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.494+0000 7f803b400700 1 --2- 172.21.15.19:0/282584099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5f354df70 0x55b5f356f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:51.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.494+0000 7f803b400700 1 -- 172.21.15.19:0/282584099 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5f35d1820 con 0x55b5f354d620 2024-05-07T02:36:51.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.495+0000 7f803b400700 1 --2- 172.21.15.19:0/282584099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5f354d620 0x55b5f356f3b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f802c009870 tx=0x7f802c003f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43a1ef906a673b98 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:36:51.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.495+0000 7f8025600700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:36:51.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.495+0000 7f8025600700 1 -- 172.21.15.19:0/282584099 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802c026e60 con 0x55b5f354d620 2024-05-07T02:36:51.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.495+0000 7f8050e01b80 1 -- 172.21.15.19:0/282584099 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8028017040 con 0x55b5f354d620 2024-05-07T02:36:51.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.495+0000 7f8025600700 1 -- 172.21.15.19:0/282584099 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f802c00e890 con 0x55b5f354d620 2024-05-07T02:36:51.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.496+0000 7f8025600700 1 -- 172.21.15.19:0/282584099 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f802c00ea90 con 0x55b5f354d620 2024-05-07T02:36:51.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.496+0000 7f8050e01b80 10 client.26030 fetch_fsmap learned FSMap version 40 2024-05-07T02:36:51.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.496+0000 7f8050e01b80 1 -- 172.21.15.19:0/282584099 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55b5f35d2990 con 0x55b5f354d620 2024-05-07T02:36:51.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.497+0000 7f8025600700 1 -- 172.21.15.19:0/282584099 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8028017040 con 0x55b5f354d620 2024-05-07T02:36:51.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.497+0000 7f8025600700 1 -- 172.21.15.19:0/282584099 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f802c034430 con 0x55b5f354d620 2024-05-07T02:36:51.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.497+0000 7f8050e01b80 10 client.26030 fetch_fsmap finished waiting for FSMap version 40 2024-05-07T02:36:51.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.497+0000 7f8050e01b80 10 client.26030 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:36:51.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.497+0000 7f8050e01b80 20 client.26030 populate_metadata read hostname 'smithi019' 2024-05-07T02:36:51.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.497+0000 7f8050e01b80 1 --2- 172.21.15.19:0/282584099 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55b5f35d30a0 0x55b5f35d54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:51.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.497+0000 7f8050e01b80 4 client.26030 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-07T02:36:51.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.497+0000 7f8050e01b80 1 -- 172.21.15.19:0/282584099 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b5f347d400 con 0x55b5f35d30a0 2024-05-07T02:36:51.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.497+0000 7f803aa00700 1 --2- 172.21.15.19:0/282584099 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55b5f35d30a0 0x55b5f35d54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:51.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.498+0000 7f803aa00700 1 --2- 172.21.15.19:0/282584099 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55b5f35d30a0 0x55b5f35d54c0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf84b4bb44db0257 server_cookie=c098481118e8ddf5 in_seq=0 out_seq=0 2024-05-07T02:36:51.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.498+0000 7f8025600700 10 client.26030 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:36:51.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.498+0000 7f8025600700 1 -- 172.21.15.19:0/282584099 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b5f347d400 con 0x55b5f35d30a0 2024-05-07T02:36:51.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.498+0000 7f8025600700 10 client.26030 handle_command_reply: tid=0 2024-05-07T02:36:51.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.498+0000 7f8050e01b80 1 client.26030 shutdown 2024-05-07T02:36:51.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.499+0000 7f8050e01b80 1 -- 172.21.15.19:0/282584099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5f354d620 msgr2=0x55b5f356f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:51.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.499+0000 7f8050e01b80 1 --2- 172.21.15.19:0/282584099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5f354d620 0x55b5f356f3b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f802c009870 tx=0x7f802c003f50 comp rx=0 tx=0).stop 2024-05-07T02:36:51.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.499+0000 7f8050e01b80 1 -- 172.21.15.19:0/282584099 shutdown_connections 2024-05-07T02:36:51.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.499+0000 7f8050e01b80 1 --2- 172.21.15.19:0/282584099 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55b5f35d30a0 0x55b5f35d54c0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:51.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.499+0000 7f8050e01b80 1 --2- 172.21.15.19:0/282584099 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5f3558240 0x55b5f356fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:51.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.499+0000 7f8050e01b80 1 --2- 172.21.15.19:0/282584099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5f354d620 0x55b5f356f3b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:51.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.499+0000 7f8025600700 0 client.26030 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:36:51.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.499+0000 7f8050e01b80 1 --2- 172.21.15.19:0/282584099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5f354df70 0x55b5f356f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:51.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.499+0000 7f8050e01b80 1 -- 172.21.15.19:0/282584099 >> 172.21.15.19:0/282584099 conn(0x55b5f33cfe40 msgr2=0x55b5f3555cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:36:51.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.499+0000 7f8050e01b80 1 -- 172.21.15.19:0/282584099 shutdown_connections 2024-05-07T02:36:51.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.499+0000 7f8050e01b80 1 -- 172.21.15.19:0/282584099 wait complete. 2024-05-07T02:36:51.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.500+0000 7f8050e01b80 20 client.26030 trim_cache size 0 max 16384 2024-05-07T02:36:51.500 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:36:51.501 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-07T02:36:51.501 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-07T02:36:51.501 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:36:51.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.500+0000 7f8042200700 1 -- 172.21.15.19:0/874008259 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f80200b2a70 msgr2=0x7f80200b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:51.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.500+0000 7f8042200700 1 --2- 172.21.15.19:0/874008259 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f80200b2a70 0x7f80200b4f80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f802c009680 tx=0x7f802c009260 comp rx=0 tx=0).stop 2024-05-07T02:36:51.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.500+0000 7f8042200700 1 -- 172.21.15.19:0/874008259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f803c110c70 msgr2=0x7f803c1a17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:51.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.500+0000 7f8042200700 1 --2- 172.21.15.19:0/874008259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f803c110c70 0x7f803c1a17b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f803400b070 tx=0x7f8034009c90 comp rx=0 tx=0).stop 2024-05-07T02:36:51.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.501+0000 7f8042200700 1 -- 172.21.15.19:0/874008259 shutdown_connections 2024-05-07T02:36:51.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.501+0000 7f8042200700 1 --2- 172.21.15.19:0/874008259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f803c110c70 0x7f803c1a17b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:51.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.501+0000 7f8042200700 1 --2- 172.21.15.19:0/874008259 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f80200b2a70 0x7f80200b4f80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:51.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.501+0000 7f8042200700 1 --2- 172.21.15.19:0/874008259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f803c1030c0 0x7f803c19cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:51.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.501+0000 7f8042200700 1 --2- 172.21.15.19:0/874008259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f803c103a10 0x7f803c19d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:51.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.501+0000 7f8042200700 1 -- 172.21.15.19:0/874008259 >> 172.21.15.19:0/874008259 conn(0x7f803c0fedc0 msgr2=0x7f803c1126e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:36:51.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.501+0000 7f8042200700 1 -- 172.21.15.19:0/874008259 shutdown_connections 2024-05-07T02:36:51.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:51.501+0000 7f8042200700 1 -- 172.21.15.19:0/874008259 wait complete. 2024-05-07T02:36:51.749 INFO:tasks.workunit.client.0.smithi019.stdout: 1 146528 3.67 MB/sec execute 107 sec latency 34.655 ms 2024-05-07T02:36:52.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:52 smithi064 ceph-mon[34699]: pgmap v154: 129 pgs: 129 active+clean; 103 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 212 op/s 2024-05-07T02:36:52.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:52 smithi019 ceph-mon[29781]: pgmap v154: 129 pgs: 129 active+clean; 103 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 212 op/s 2024-05-07T02:36:52.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:52 smithi027 ceph-mon[34398]: pgmap v154: 129 pgs: 129 active+clean; 103 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 212 op/s 2024-05-07T02:36:52.461 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-07T02:36:52.461 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-07T02:36:52.749 INFO:tasks.workunit.client.0.smithi019.stdout: 1 147958 3.69 MB/sec execute 108 sec latency 45.744 ms 2024-05-07T02:36:52.961 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:36:53.749 INFO:tasks.workunit.client.0.smithi019.stdout: 1 149226 3.76 MB/sec execute 109 sec latency 35.713 ms 2024-05-07T02:36:54.301 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:54 smithi019 ceph-mon[29781]: pgmap v155: 129 pgs: 129 active+clean; 103 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-05-07T02:36:54.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:54 smithi064 ceph-mon[34699]: pgmap v155: 129 pgs: 129 active+clean; 103 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-05-07T02:36:54.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:54 smithi027 ceph-mon[34398]: pgmap v155: 129 pgs: 129 active+clean; 103 MiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-05-07T02:36:54.749 INFO:tasks.workunit.client.0.smithi019.stdout: 1 149511 3.72 MB/sec execute 110 sec latency 31.057 ms 2024-05-07T02:36:55.749 INFO:tasks.workunit.client.0.smithi019.stdout: 1 150493 3.74 MB/sec execute 111 sec latency 18.098 ms 2024-05-07T02:36:56.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.352+0000 7f7553800700 1 -- 172.21.15.19:0/1415968554 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f754c1045f0 msgr2=0x7f754c104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:56.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.352+0000 7f7553800700 1 --2- 172.21.15.19:0/1415968554 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f754c1045f0 0x7f754c104a00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7540009990 tx=0x7f7540009ca0 comp rx=0 tx=0).stop 2024-05-07T02:36:56.354 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:56 smithi019 ceph-mon[29781]: pgmap v156: 129 pgs: 129 active+clean; 106 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 225 op/s 2024-05-07T02:36:56.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.352+0000 7f7553800700 1 -- 172.21.15.19:0/1415968554 shutdown_connections 2024-05-07T02:36:56.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.352+0000 7f7553800700 1 --2- 172.21.15.19:0/1415968554 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f754c109870 0x7f754c1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:56.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.352+0000 7f7553800700 1 --2- 172.21.15.19:0/1415968554 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f754c1045f0 0x7f754c104a00 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:56.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.352+0000 7f7553800700 1 --2- 172.21.15.19:0/1415968554 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f754c108f20 0x7f754c109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:56.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.352+0000 7f7553800700 1 -- 172.21.15.19:0/1415968554 >> 172.21.15.19:0/1415968554 conn(0x7f754c0fffe0 msgr2=0x7f754c102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:36:56.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.352+0000 7f7553800700 1 -- 172.21.15.19:0/1415968554 shutdown_connections 2024-05-07T02:36:56.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.352+0000 7f7553800700 1 -- 172.21.15.19:0/1415968554 wait complete. 2024-05-07T02:36:56.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.353+0000 7f7553800700 1 Processor -- start 2024-05-07T02:36:56.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.354+0000 7f7553800700 1 -- start start 2024-05-07T02:36:56.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.354+0000 7f7553800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f754c1045f0 0x7f754c1a21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:56.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.354+0000 7f7553800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f754c108f20 0x7f754c1a2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:56.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.354+0000 7f7553800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f754c109870 0x7f754c1ad250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:56.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.354+0000 7f7553800700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f754c1144f0 con 0x7f754c1045f0 2024-05-07T02:36:56.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.354+0000 7f7553800700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f754c114370 con 0x7f754c108f20 2024-05-07T02:36:56.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.354+0000 7f7553800700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f754c114670 con 0x7f754c109870 2024-05-07T02:36:56.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.354+0000 7f754be00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f754c108f20 0x7f754c1a2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:56.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.354+0000 7f7550e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f754c1045f0 0x7f754c1a21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:56.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.354+0000 7f754be00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f754c108f20 0x7f754c1a2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:42368/0 (socket says 172.21.15.19:42368) 2024-05-07T02:36:56.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.355+0000 7f754be00700 1 -- 172.21.15.19:0/4047428574 learned_addr learned my addr 172.21.15.19:0/4047428574 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:36:56.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.355+0000 7f7551800700 1 --2- 172.21.15.19:0/4047428574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f754c109870 0x7f754c1ad250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:56.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.355+0000 7f7551800700 1 -- 172.21.15.19:0/4047428574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f754c108f20 msgr2=0x7f754c1a2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:56.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.355+0000 7f7551800700 1 --2- 172.21.15.19:0/4047428574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f754c108f20 0x7f754c1a2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:56.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.355+0000 7f7551800700 1 -- 172.21.15.19:0/4047428574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f754c1045f0 msgr2=0x7f754c1a21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:56.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.355+0000 7f7551800700 1 --2- 172.21.15.19:0/4047428574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f754c1045f0 0x7f754c1a21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:56.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.355+0000 7f7551800700 1 -- 172.21.15.19:0/4047428574 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f754c1ad790 con 0x7f754c109870 2024-05-07T02:36:56.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.355+0000 7f7550e00700 1 --2- 172.21.15.19:0/4047428574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f754c1045f0 0x7f754c1a21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:36:56.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.355+0000 7f754be00700 1 --2- 172.21.15.19:0/4047428574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f754c108f20 0x7f754c1a2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:36:56.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.356+0000 7f7551800700 1 --2- 172.21.15.19:0/4047428574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f754c109870 0x7f754c1ad250 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f754400efc0 tx=0x7f754400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:36:56.362 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:56 smithi064 ceph-mon[34699]: pgmap v156: 129 pgs: 129 active+clean; 106 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 225 op/s 2024-05-07T02:36:56.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.356+0000 7f7549600700 1 -- 172.21.15.19:0/4047428574 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75440092e0 con 0x7f754c109870 2024-05-07T02:36:56.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.356+0000 7f7549600700 1 -- 172.21.15.19:0/4047428574 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7544004bb0 con 0x7f754c109870 2024-05-07T02:36:56.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.356+0000 7f7553800700 1 -- 172.21.15.19:0/4047428574 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f754c1ad9c0 con 0x7f754c109870 2024-05-07T02:36:56.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.356+0000 7f7549600700 1 -- 172.21.15.19:0/4047428574 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7544004020 con 0x7f754c109870 2024-05-07T02:36:56.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.356+0000 7f7553800700 1 -- 172.21.15.19:0/4047428574 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f754c1adf00 con 0x7f754c109870 2024-05-07T02:36:56.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.358+0000 7f7553800700 1 -- 172.21.15.19:0/4047428574 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f754c0512c0 con 0x7f754c109870 2024-05-07T02:36:56.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.360+0000 7f7549600700 1 -- 172.21.15.19:0/4047428574 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f754401b030 con 0x7f754c109870 2024-05-07T02:36:56.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.361+0000 7f7549600700 1 --2- 172.21.15.19:0/4047428574 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f75340b2a70 0x7f75340b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:36:56.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.362+0000 7f7549600700 1 -- 172.21.15.19:0/4047428574 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f75440e23f0 con 0x7f754c109870 2024-05-07T02:36:56.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.362+0000 7f7550e00700 1 --2- 172.21.15.19:0/4047428574 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f75340b2a70 0x7f75340b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:36:56.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.363+0000 7f7550e00700 1 --2- 172.21.15.19:0/4047428574 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f75340b2a70 0x7f75340b4f80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f754000b790 tx=0x7f754000b260 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:36:56.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.366+0000 7f7549600700 1 -- 172.21.15.19:0/4047428574 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7544022020 con 0x7f754c109870 2024-05-07T02:36:56.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:56 smithi027 ceph-mon[34398]: pgmap v156: 129 pgs: 129 active+clean; 106 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 225 op/s 2024-05-07T02:36:56.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.723+0000 7f7553800700 1 -- 172.21.15.19:0/4047428574 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f754c1ae0d0 con 0x7f754c109870 2024-05-07T02:36:56.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.724+0000 7f7549600700 1 -- 172.21.15.19:0/4047428574 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11927 (secure 0 0 0) 0x7f7544010040 con 0x7f754c109870 2024-05-07T02:36:56.726 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:36:56.726 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1763670446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1763670446},{"type":"v1","addr":"172.21.15.19:6839","nonce":1763670446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/477529985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":477529985},{"type":"v1","addr":"172.21.15.64:6833","nonce":477529985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-07T02:26:14.960465+0000","modified":"2024-05-07T02:35:53.643705+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24539,"mds_2":24520,"mds_3":24515,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6833/2171873162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2171873162},{"type":"v1","addr":"172.21.15.27:6833","nonce":2171873162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/2379917839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2379917839},{"type":"v1","addr":"172.21.15.19:6835","nonce":2379917839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6837/2067515538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2067515538},{"type":"v1","addr":"172.21.15.19:6837","nonce":2067515538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6835/1000540600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1000540600},{"type":"v1","addr":"172.21.15.27:6835","nonce":1000540600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/1997318872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1997318872},{"type":"v1","addr":"172.21.15.64:6835","nonce":1997318872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":34,"addr":"172.21.15.64:6837/2585907777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2585907777},{"type":"v1","addr":"172.21.15.64:6837","nonce":2585907777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":33,"addr":"172.21.15.27:6837/2583138143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2583138143},{"type":"v1","addr":"172.21.15.27:6837","nonce":2583138143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":32,"addr":"172.21.15.64:6839/1583120004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":1583120004},{"type":"v1","addr":"172.21.15.64:6839","nonce":1583120004}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":29,"addr":"172.21.15.19:6841/3905645246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3905645246},{"type":"v1","addr":"172.21.15.19:6841","nonce":3905645246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6839/3963268634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3963268634},{"type":"v1","addr":"172.21.15.27:6839","nonce":3963268634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24539,24551,24520,24515,24505]},"id":1}]} 2024-05-07T02:36:56.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.728+0000 7f7553800700 1 -- 172.21.15.19:0/4047428574 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f75340b2a70 msgr2=0x7f75340b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:56.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.728+0000 7f7553800700 1 --2- 172.21.15.19:0/4047428574 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f75340b2a70 0x7f75340b4f80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f754000b790 tx=0x7f754000b260 comp rx=0 tx=0).stop 2024-05-07T02:36:56.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.728+0000 7f7553800700 1 -- 172.21.15.19:0/4047428574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f754c109870 msgr2=0x7f754c1ad250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:36:56.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.728+0000 7f7553800700 1 --2- 172.21.15.19:0/4047428574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f754c109870 0x7f754c1ad250 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f754400efc0 tx=0x7f754400c3a0 comp rx=0 tx=0).stop 2024-05-07T02:36:56.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.728+0000 7f7553800700 1 -- 172.21.15.19:0/4047428574 shutdown_connections 2024-05-07T02:36:56.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.728+0000 7f7553800700 1 --2- 172.21.15.19:0/4047428574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f754c109870 0x7f754c1ad250 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:56.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.728+0000 7f7553800700 1 --2- 172.21.15.19:0/4047428574 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f75340b2a70 0x7f75340b4f80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:56.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.728+0000 7f7553800700 1 --2- 172.21.15.19:0/4047428574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f754c1045f0 0x7f754c1a21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:56.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.728+0000 7f7553800700 1 --2- 172.21.15.19:0/4047428574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f754c108f20 0x7f754c1a2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:36:56.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.728+0000 7f7553800700 1 -- 172.21.15.19:0/4047428574 >> 172.21.15.19:0/4047428574 conn(0x7f754c0fffe0 msgr2=0x7f754c0766a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:36:56.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.728+0000 7f7553800700 1 -- 172.21.15.19:0/4047428574 shutdown_connections 2024-05-07T02:36:56.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:36:56.728+0000 7f7553800700 1 -- 172.21.15.19:0/4047428574 wait complete. 2024-05-07T02:36:56.730 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 40 2024-05-07T02:36:56.749 INFO:tasks.workunit.client.0.smithi019.stdout: 1 151983 3.77 MB/sec execute 112 sec latency 44.427 ms 2024-05-07T02:36:57.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:57 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/4047428574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:36:57.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:57 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/4047428574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:36:57.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:57 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/4047428574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:36:57.750 INFO:tasks.workunit.client.0.smithi019.stdout: 1 152843 3.79 MB/sec execute 113 sec latency 42.214 ms 2024-05-07T02:36:57.853 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-07T02:36:58.347 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:36:58.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:36:58 smithi064 ceph-mon[34699]: pgmap v157: 129 pgs: 129 active+clean; 106 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 151 op/s 2024-05-07T02:36:58.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:36:58 smithi019 ceph-mon[29781]: pgmap v157: 129 pgs: 129 active+clean; 106 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 151 op/s 2024-05-07T02:36:58.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:36:58 smithi027 ceph-mon[34398]: pgmap v157: 129 pgs: 129 active+clean; 106 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 151 op/s 2024-05-07T02:36:58.750 INFO:tasks.workunit.client.0.smithi019.stdout: 1 153463 3.77 MB/sec execute 114 sec latency 21.865 ms 2024-05-07T02:36:59.750 INFO:tasks.workunit.client.0.smithi019.stdout: 1 154979 3.79 MB/sec execute 115 sec latency 16.250 ms 2024-05-07T02:37:00.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:00 smithi064 ceph-mon[34699]: pgmap v158: 129 pgs: 129 active+clean; 106 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 151 op/s 2024-05-07T02:37:00.373 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:00 smithi019 ceph-mon[29781]: pgmap v158: 129 pgs: 129 active+clean; 106 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 151 op/s 2024-05-07T02:37:00.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:00 smithi027 ceph-mon[34398]: pgmap v158: 129 pgs: 129 active+clean; 106 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 151 op/s 2024-05-07T02:37:00.750 INFO:tasks.workunit.client.0.smithi019.stdout: 1 155791 3.81 MB/sec execute 116 sec latency 79.621 ms 2024-05-07T02:37:01.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.033+0000 7f105a600700 1 -- 172.21.15.19:0/1488424811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1054108f20 msgr2=0x7f1054109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:01.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.033+0000 7f105a600700 1 --2- 172.21.15.19:0/1488424811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1054108f20 0x7f1054109330 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1040009990 tx=0x7f1040009ca0 comp rx=0 tx=0).stop 2024-05-07T02:37:01.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.033+0000 7f105a600700 1 -- 172.21.15.19:0/1488424811 shutdown_connections 2024-05-07T02:37:01.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.033+0000 7f105a600700 1 --2- 172.21.15.19:0/1488424811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1054109870 0x7f10541111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:01.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.033+0000 7f105a600700 1 --2- 172.21.15.19:0/1488424811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f10541045f0 0x7f1054104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:01.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.033+0000 7f105a600700 1 --2- 172.21.15.19:0/1488424811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1054108f20 0x7f1054109330 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:01.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.033+0000 7f105a600700 1 -- 172.21.15.19:0/1488424811 >> 172.21.15.19:0/1488424811 conn(0x7f1054100040 msgr2=0x7f1054102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:37:01.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.033+0000 7f105a600700 1 -- 172.21.15.19:0/1488424811 shutdown_connections 2024-05-07T02:37:01.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.033+0000 7f105a600700 1 -- 172.21.15.19:0/1488424811 wait complete. 2024-05-07T02:37:01.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.035+0000 7f105a600700 1 Processor -- start 2024-05-07T02:37:01.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.035+0000 7f105a600700 1 -- start start 2024-05-07T02:37:01.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.036+0000 7f105a600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10541045f0 0x7f10541a21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:01.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.036+0000 7f105a600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1054108f20 0x7f10541a2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:01.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.036+0000 7f105a600700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1054109870 0x7f10541ad230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:01.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.036+0000 7f105a600700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f10541144d0 con 0x7f10541045f0 2024-05-07T02:37:01.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.036+0000 7f105a600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1054114350 con 0x7f1054108f20 2024-05-07T02:37:01.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.036+0000 7f105a600700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1054114650 con 0x7f1054109870 2024-05-07T02:37:01.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.036+0000 7f1052a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1054108f20 0x7f10541a2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:01.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.036+0000 7f1053e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1054109870 0x7f10541ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:01.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.036+0000 7f1052a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1054108f20 0x7f10541a2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:52318/0 (socket says 172.21.15.19:52318) 2024-05-07T02:37:01.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.036+0000 7f1053e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1054109870 0x7f10541ad230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:41798/0 (socket says 172.21.15.19:41798) 2024-05-07T02:37:01.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.036+0000 7f1052a00700 1 -- 172.21.15.19:0/1217790506 learned_addr learned my addr 172.21.15.19:0/1217790506 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:37:01.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.036+0000 7f1053400700 1 --2- 172.21.15.19:0/1217790506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10541045f0 0x7f10541a21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:01.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.037+0000 7f1053e00700 1 -- 172.21.15.19:0/1217790506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1054108f20 msgr2=0x7f10541a2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:01.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.037+0000 7f1053e00700 1 --2- 172.21.15.19:0/1217790506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1054108f20 0x7f10541a2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:01.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.037+0000 7f1053e00700 1 -- 172.21.15.19:0/1217790506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10541045f0 msgr2=0x7f10541a21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:01.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.037+0000 7f1053e00700 1 --2- 172.21.15.19:0/1217790506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10541045f0 0x7f10541a21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:01.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.037+0000 7f1053e00700 1 -- 172.21.15.19:0/1217790506 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10541ad770 con 0x7f1054109870 2024-05-07T02:37:01.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.037+0000 7f1053400700 1 --2- 172.21.15.19:0/1217790506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10541045f0 0x7f10541a21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:37:01.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.037+0000 7f1053e00700 1 --2- 172.21.15.19:0/1217790506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1054109870 0x7f10541ad230 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f104c00efc0 tx=0x7f104c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:37:01.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.038+0000 7f104be00700 1 -- 172.21.15.19:0/1217790506 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104c0092e0 con 0x7f1054109870 2024-05-07T02:37:01.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.038+0000 7f104be00700 1 -- 172.21.15.19:0/1217790506 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f104c004bb0 con 0x7f1054109870 2024-05-07T02:37:01.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.038+0000 7f104be00700 1 -- 172.21.15.19:0/1217790506 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104c004280 con 0x7f1054109870 2024-05-07T02:37:01.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.038+0000 7f105a600700 1 -- 172.21.15.19:0/1217790506 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10541ad9a0 con 0x7f1054109870 2024-05-07T02:37:01.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.038+0000 7f105a600700 1 -- 172.21.15.19:0/1217790506 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10541ade90 con 0x7f1054109870 2024-05-07T02:37:01.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.043+0000 7f104be00700 1 -- 172.21.15.19:0/1217790506 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f104c01b030 con 0x7f1054109870 2024-05-07T02:37:01.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.044+0000 7f1069220b80 1 Processor -- start 2024-05-07T02:37:01.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.044+0000 7f1069220b80 1 -- start start 2024-05-07T02:37:01.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.045+0000 7f1069220b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c871483b0 0x562c871487c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:01.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.045+0000 7f1069220b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c87142470 0x562c87142880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:01.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.045+0000 7f1069220b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562c87142dc0 0x562c8714f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:01.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.045+0000 7f1069220b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562c87064a20 con 0x562c87142dc0 2024-05-07T02:37:01.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.045+0000 7f1069220b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562c87002760 con 0x562c87142470 2024-05-07T02:37:01.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.045+0000 7f1069220b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562c870609e0 con 0x562c871483b0 2024-05-07T02:37:01.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.045+0000 7f1053400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c87142470 0x562c87142880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:01.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.046+0000 7f1052a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c871483b0 0x562c871487c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:01.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.046+0000 7f1052a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c871483b0 0x562c871487c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:41810/0 (socket says 172.21.15.19:41810) 2024-05-07T02:37:01.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.046+0000 7f1052a00700 1 -- 172.21.15.19:0/1677761977 learned_addr learned my addr 172.21.15.19:0/1677761977 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:37:01.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.046+0000 7f1052a00700 1 --2- 172.21.15.19:0/1677761977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562c87142dc0 0x562c8714f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:01.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.046+0000 7f1053400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c87142470 0x562c87142880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:52320/0 (socket says 172.21.15.19:52320) 2024-05-07T02:37:01.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.046+0000 7f1052a00700 1 -- 172.21.15.19:0/1677761977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c87142470 msgr2=0x562c87142880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:01.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.046+0000 7f1052a00700 1 --2- 172.21.15.19:0/1677761977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c87142470 0x562c87142880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:01.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.046+0000 7f1052a00700 1 -- 172.21.15.19:0/1677761977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562c87142dc0 msgr2=0x562c8714f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:01.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.046+0000 7f1052a00700 1 --2- 172.21.15.19:0/1677761977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562c87142dc0 0x562c8714f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:01.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.046+0000 7f1052a00700 1 -- 172.21.15.19:0/1677761977 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c86fb18d0 con 0x562c871483b0 2024-05-07T02:37:01.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.046+0000 7f104be00700 1 --2- 172.21.15.19:0/1217790506 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f10380b2b40 0x7f10380b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:01.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.047+0000 7f104be00700 1 -- 172.21.15.19:0/1217790506 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f104c0e22a0 con 0x7f1054109870 2024-05-07T02:37:01.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.047+0000 7f1053400700 1 --2- 172.21.15.19:0/1217790506 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f10380b2b40 0x7f10380b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:01.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.047+0000 7f1052a00700 1 --2- 172.21.15.19:0/1677761977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c871483b0 0x562c871487c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1040005910 tx=0x7f104001a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6d48a493c36b08a server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:37:01.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.047+0000 7f1048c00700 1 -- 172.21.15.19:0/1677761977 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104000bd40 con 0x562c871483b0 2024-05-07T02:37:01.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.048+0000 7f1048c00700 1 -- 172.21.15.19:0/1677761977 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1040003680 con 0x562c871483b0 2024-05-07T02:37:01.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.048+0000 7f1048c00700 1 -- 172.21.15.19:0/1677761977 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104000f6c0 con 0x562c871483b0 2024-05-07T02:37:01.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.049+0000 7f1069220b80 1 -- 172.21.15.19:0/1677761977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c871483b0 msgr2=0x562c871487c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:01.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.049+0000 7f1053400700 1 --2- 172.21.15.19:0/1217790506 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f10380b2b40 0x7f10380b5050 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x562c8713ea40 tx=0x7f1044009260 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:37:01.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.049+0000 7f1069220b80 1 --2- 172.21.15.19:0/1677761977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c871483b0 0x562c871487c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1040005910 tx=0x7f104001a040 comp rx=0 tx=0).stop 2024-05-07T02:37:01.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.049+0000 7f1069220b80 1 -- 172.21.15.19:0/1677761977 shutdown_connections 2024-05-07T02:37:01.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.049+0000 7f1069220b80 1 --2- 172.21.15.19:0/1677761977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562c87142dc0 0x562c8714f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:01.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.049+0000 7f1069220b80 1 --2- 172.21.15.19:0/1677761977 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c871483b0 0x562c871487c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:01.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.049+0000 7f1069220b80 1 --2- 172.21.15.19:0/1677761977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c87142470 0x562c87142880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:01.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.049+0000 7f1069220b80 1 -- 172.21.15.19:0/1677761977 >> 172.21.15.19:0/1677761977 conn(0x562c86fc4e40 msgr2=0x562c8713d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:37:01.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.049+0000 7f1069220b80 1 -- 172.21.15.19:0/1677761977 shutdown_connections 2024-05-07T02:37:01.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.049+0000 7f1069220b80 1 -- 172.21.15.19:0/1677761977 wait complete. 2024-05-07T02:37:01.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.050+0000 7f1069220b80 1 -- start start 2024-05-07T02:37:01.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.050+0000 7f1069220b80 1 Processor -- start 2024-05-07T02:37:01.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.051+0000 7f1069220b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c87142470 0x562c8714c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:01.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.051+0000 7f1069220b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c87142dc0 0x562c8714ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:01.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.051+0000 7f1069220b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562c871483b0 0x562c8714cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:01.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.051+0000 7f1069220b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562c8714f760 con 0x562c871483b0 2024-05-07T02:37:01.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.051+0000 7f1069220b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562c86fcab20 con 0x562c87142dc0 2024-05-07T02:37:01.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.051+0000 7f1069220b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562c8714f8e0 con 0x562c87142470 2024-05-07T02:37:01.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.052+0000 7f1069220b80 10 client.0 _ll_register_callbacks cb 0x562c8700bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:37:01.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.052+0000 7f1053400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c87142470 0x562c8714c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:01.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.052+0000 7f1053400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c87142470 0x562c8714c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:41826/0 (socket says 172.21.15.19:41826) 2024-05-07T02:37:01.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.052+0000 7f1053400700 1 -- 172.21.15.19:0/1998974337 learned_addr learned my addr 172.21.15.19:0/1998974337 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:37:01.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.052+0000 7f1052a00700 1 --2- 172.21.15.19:0/1998974337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c87142dc0 0x562c8714ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:01.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.052+0000 7f1053400700 1 -- 172.21.15.19:0/1998974337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c87142dc0 msgr2=0x562c8714ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:01.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.052+0000 7f1053400700 1 --2- 172.21.15.19:0/1998974337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c87142dc0 0x562c8714ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:01.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.052+0000 7f1053400700 1 -- 172.21.15.19:0/1998974337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562c871483b0 msgr2=0x562c8714cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:01.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.052+0000 7f1053400700 1 --2- 172.21.15.19:0/1998974337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562c871483b0 0x562c8714cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:01.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.052+0000 7f1053400700 1 -- 172.21.15.19:0/1998974337 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c8702a420 con 0x562c87142470 2024-05-07T02:37:01.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.053+0000 7f1053400700 1 --2- 172.21.15.19:0/1998974337 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c87142470 0x562c8714c500 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f10440209e0 tx=0x7f10440047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58ae494551e12027 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:37:01.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.053+0000 7f103d600700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:37:01.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.053+0000 7f103d600700 1 -- 172.21.15.19:0/1998974337 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104402d060 con 0x562c87142470 2024-05-07T02:37:01.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.053+0000 7f103d600700 1 -- 172.21.15.19:0/1998974337 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f1044018030 con 0x562c87142470 2024-05-07T02:37:01.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.053+0000 7f1069220b80 1 -- 172.21.15.19:0/1998974337 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1040003680 con 0x562c87142470 2024-05-07T02:37:01.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.054+0000 7f103d600700 1 -- 172.21.15.19:0/1998974337 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f1044020e60 con 0x562c87142470 2024-05-07T02:37:01.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.054+0000 7f1069220b80 10 client.26020 fetch_fsmap learned FSMap version 40 2024-05-07T02:37:01.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.054+0000 7f1069220b80 1 -- 172.21.15.19:0/1998974337 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x562c8715a430 con 0x562c87142470 2024-05-07T02:37:01.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.054+0000 7f103d600700 1 -- 172.21.15.19:0/1998974337 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1040003680 con 0x562c87142470 2024-05-07T02:37:01.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.055+0000 7f103d600700 1 -- 172.21.15.19:0/1998974337 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f104401d8e0 con 0x562c87142470 2024-05-07T02:37:01.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.055+0000 7f1069220b80 10 client.26020 fetch_fsmap finished waiting for FSMap version 40 2024-05-07T02:37:01.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.055+0000 7f1069220b80 10 client.26020 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-07T02:37:01.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.055+0000 7f1069220b80 20 client.26020 populate_metadata read hostname 'smithi019' 2024-05-07T02:37:01.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.055+0000 7f1069220b80 1 --2- 172.21.15.19:0/1998974337 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x562c8715cac0 0x562c8715efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:01.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.055+0000 7f1069220b80 4 client.26020 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:37:01.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.055+0000 7f1069220b80 1 -- 172.21.15.19:0/1998974337 --> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c8710f2e0 con 0x562c8715cac0 2024-05-07T02:37:01.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.055+0000 7f1052a00700 1 --2- 172.21.15.19:0/1998974337 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x562c8715cac0 0x562c8715efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:01.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.056+0000 7f1052a00700 1 --2- 172.21.15.19:0/1998974337 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x562c8715cac0 0x562c8715efb0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=52f9d005d1aa8f8 server_cookie=bf07dee7bf59f834 in_seq=0 out_seq=0 2024-05-07T02:37:01.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.056+0000 7f103d600700 10 client.26020 ms_handle_connect on v2:172.21.15.64:6836/2585907777 2024-05-07T02:37:01.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.057+0000 7f103d600700 1 -- 172.21.15.19:0/1998974337 <== mds.4 v2:172.21.15.64:6836/2585907777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562c8710f2e0 con 0x562c8715cac0 2024-05-07T02:37:01.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.057+0000 7f103d600700 10 client.26020 handle_command_reply: tid=0 2024-05-07T02:37:01.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.057+0000 7f1069220b80 1 client.26020 shutdown 2024-05-07T02:37:01.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.057+0000 7f1069220b80 1 -- 172.21.15.19:0/1998974337 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c87142470 msgr2=0x562c8714c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:01.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.057+0000 7f1069220b80 1 --2- 172.21.15.19:0/1998974337 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c87142470 0x562c8714c500 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f10440209e0 tx=0x7f10440047c0 comp rx=0 tx=0).stop 2024-05-07T02:37:01.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.057+0000 7f1069220b80 1 -- 172.21.15.19:0/1998974337 shutdown_connections 2024-05-07T02:37:01.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.057+0000 7f1069220b80 1 --2- 172.21.15.19:0/1998974337 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x562c8715cac0 0x562c8715efb0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:01.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.057+0000 7f1069220b80 1 --2- 172.21.15.19:0/1998974337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562c871483b0 0x562c8714cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:01.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.057+0000 7f1069220b80 1 --2- 172.21.15.19:0/1998974337 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c87142470 0x562c8714c500 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:01.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.057+0000 7f1069220b80 1 --2- 172.21.15.19:0/1998974337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c87142dc0 0x562c8714ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:01.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.057+0000 7f1069220b80 1 -- 172.21.15.19:0/1998974337 >> 172.21.15.19:0/1998974337 conn(0x562c86fc4e40 msgr2=0x562c87141460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:37:01.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.057+0000 7f103d600700 0 client.26020 ms_handle_reset on v2:172.21.15.64:6836/2585907777 2024-05-07T02:37:01.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.058+0000 7f1069220b80 1 -- 172.21.15.19:0/1998974337 shutdown_connections 2024-05-07T02:37:01.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.058+0000 7f1069220b80 1 -- 172.21.15.19:0/1998974337 wait complete. 2024-05-07T02:37:01.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.058+0000 7f1069220b80 20 client.26020 trim_cache size 0 max 16384 2024-05-07T02:37:01.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.079+0000 7f1069220b80 1 Processor -- start 2024-05-07T02:37:01.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.079+0000 7f1069220b80 1 -- start start 2024-05-07T02:37:01.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.080+0000 7f1069220b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c86fc4e40 0x562c8714cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:01.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.080+0000 7f1069220b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562c87142470 0x562c87158880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:01.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.080+0000 7f1069220b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c87142dc0 0x562c87158dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:01.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.080+0000 7f1069220b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f103400ed20 con 0x562c87142470 2024-05-07T02:37:01.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.080+0000 7f1069220b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f10340108c0 con 0x562c86fc4e40 2024-05-07T02:37:01.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.080+0000 7f1069220b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f103400f3f0 con 0x562c87142dc0 2024-05-07T02:37:01.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.080+0000 7f1053e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c87142dc0 0x562c87158dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:01.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.080+0000 7f1053400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c86fc4e40 0x562c8714cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:01.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.080+0000 7f1053e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c87142dc0 0x562c87158dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:41834/0 (socket says 172.21.15.19:41834) 2024-05-07T02:37:01.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.080+0000 7f1053400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c86fc4e40 0x562c8714cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:52344/0 (socket says 172.21.15.19:52344) 2024-05-07T02:37:01.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.080+0000 7f1053e00700 1 -- 172.21.15.19:0/1458679722 learned_addr learned my addr 172.21.15.19:0/1458679722 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:37:01.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.081+0000 7f1052a00700 1 --2- 172.21.15.19:0/1458679722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562c87142470 0x562c87158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:01.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.081+0000 7f1052a00700 1 -- 172.21.15.19:0/1458679722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c87142dc0 msgr2=0x562c87158dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:01.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.081+0000 7f1052a00700 1 --2- 172.21.15.19:0/1458679722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c87142dc0 0x562c87158dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:01.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.081+0000 7f1052a00700 1 -- 172.21.15.19:0/1458679722 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c86fc4e40 msgr2=0x562c8714cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:01.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.081+0000 7f1052a00700 1 --2- 172.21.15.19:0/1458679722 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c86fc4e40 0x562c8714cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:01.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.081+0000 7f1052a00700 1 -- 172.21.15.19:0/1458679722 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c871431d0 con 0x562c87142470 2024-05-07T02:37:01.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.082+0000 7f1052a00700 1 --2- 172.21.15.19:0/1458679722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562c87142470 0x562c87158880 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f10400098e0 tx=0x7f1040005940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=960ac6cfe2dc4d93 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:37:01.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.082+0000 7f103fe00700 1 -- 172.21.15.19:0/1458679722 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1040039070 con 0x562c87142470 2024-05-07T02:37:01.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.082+0000 7f103fe00700 1 -- 172.21.15.19:0/1458679722 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10400095b0 con 0x562c87142470 2024-05-07T02:37:01.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.082+0000 7f103fe00700 1 -- 172.21.15.19:0/1458679722 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104000f460 con 0x562c87142470 2024-05-07T02:37:01.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.083+0000 7f1069220b80 1 -- 172.21.15.19:0/1458679722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562c87142470 msgr2=0x562c87158880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:01.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.083+0000 7f1069220b80 1 --2- 172.21.15.19:0/1458679722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562c87142470 0x562c87158880 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f10400098e0 tx=0x7f1040005940 comp rx=0 tx=0).stop 2024-05-07T02:37:01.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.084+0000 7f1069220b80 1 -- 172.21.15.19:0/1458679722 shutdown_connections 2024-05-07T02:37:01.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.084+0000 7f1069220b80 1 --2- 172.21.15.19:0/1458679722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c87142dc0 0x562c87158dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:01.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.084+0000 7f1069220b80 1 --2- 172.21.15.19:0/1458679722 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c86fc4e40 0x562c8714cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:01.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.084+0000 7f1069220b80 1 --2- 172.21.15.19:0/1458679722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562c87142470 0x562c87158880 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:01.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.084+0000 7f1069220b80 1 -- 172.21.15.19:0/1458679722 >> 172.21.15.19:0/1458679722 conn(0x562c8714c090 msgr2=0x562c87137790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:37:01.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.084+0000 7f1069220b80 1 -- 172.21.15.19:0/1458679722 shutdown_connections 2024-05-07T02:37:01.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.084+0000 7f1069220b80 1 -- 172.21.15.19:0/1458679722 wait complete. 2024-05-07T02:37:01.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.085+0000 7f1069220b80 1 -- start start 2024-05-07T02:37:01.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.085+0000 7f1069220b80 1 Processor -- start 2024-05-07T02:37:01.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.086+0000 7f1069220b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562c86fc4e40 0x562c871552f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:01.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.086+0000 7f1069220b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c87142470 0x562c871491c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:01.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.086+0000 7f1069220b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c87142dc0 0x562c871497f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:01.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.086+0000 7f1069220b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f10340101f0 con 0x562c86fc4e40 2024-05-07T02:37:01.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.086+0000 7f1069220b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f103400fb20 con 0x562c87142470 2024-05-07T02:37:01.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.086+0000 7f1069220b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f103400ccc0 con 0x562c87142dc0 2024-05-07T02:37:01.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.086+0000 7f1069220b80 10 client.0 _ll_register_callbacks cb 0x562c8700bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:37:01.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.086+0000 7f1052a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c87142470 0x562c871491c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:01.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.086+0000 7f1052a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c87142470 0x562c871491c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:52356/0 (socket says 172.21.15.19:52356) 2024-05-07T02:37:01.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.086+0000 7f1052a00700 1 -- 172.21.15.19:0/1691228346 learned_addr learned my addr 172.21.15.19:0/1691228346 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:37:01.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.086+0000 7f1053400700 1 --2- 172.21.15.19:0/1691228346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562c86fc4e40 0x562c871552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:01.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.086+0000 7f1053e00700 1 --2- 172.21.15.19:0/1691228346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c87142dc0 0x562c871497f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:01.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.087+0000 7f1052a00700 1 -- 172.21.15.19:0/1691228346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c87142dc0 msgr2=0x562c871497f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:01.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.087+0000 7f1052a00700 1 --2- 172.21.15.19:0/1691228346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c87142dc0 0x562c871497f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:01.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.087+0000 7f1052a00700 1 -- 172.21.15.19:0/1691228346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562c86fc4e40 msgr2=0x562c871552f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:01.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.087+0000 7f1052a00700 1 --2- 172.21.15.19:0/1691228346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562c86fc4e40 0x562c871552f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:01.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.087+0000 7f1052a00700 1 -- 172.21.15.19:0/1691228346 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c87149f90 con 0x562c87142470 2024-05-07T02:37:01.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.087+0000 7f1052a00700 1 --2- 172.21.15.19:0/1691228346 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c87142470 0x562c871491c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1040003710 tx=0x7f1040003740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d18ad4aa1cf7de91 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:37:01.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.087+0000 7f103d600700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:37:01.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.087+0000 7f103d600700 1 -- 172.21.15.19:0/1691228346 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1040039070 con 0x562c87142470 2024-05-07T02:37:01.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.087+0000 7f103d600700 1 -- 172.21.15.19:0/1691228346 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f1040003ae0 con 0x562c87142470 2024-05-07T02:37:01.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.087+0000 7f1069220b80 1 -- 172.21.15.19:0/1691228346 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10400095b0 con 0x562c87142470 2024-05-07T02:37:01.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.088+0000 7f103d600700 1 -- 172.21.15.19:0/1691228346 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f104000be50 con 0x562c87142470 2024-05-07T02:37:01.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.088+0000 7f1069220b80 10 client.26054 fetch_fsmap learned FSMap version 40 2024-05-07T02:37:01.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.088+0000 7f1069220b80 1 -- 172.21.15.19:0/1691228346 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x562c871c73a0 con 0x562c87142470 2024-05-07T02:37:01.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.088+0000 7f103d600700 1 -- 172.21.15.19:0/1691228346 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10400095b0 con 0x562c87142470 2024-05-07T02:37:01.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.089+0000 7f103d600700 1 -- 172.21.15.19:0/1691228346 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f10400347d0 con 0x562c87142470 2024-05-07T02:37:01.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.089+0000 7f1069220b80 10 client.26054 fetch_fsmap finished waiting for FSMap version 40 2024-05-07T02:37:01.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.089+0000 7f1069220b80 10 client.26054 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-07T02:37:01.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.089+0000 7f1069220b80 20 client.26054 populate_metadata read hostname 'smithi019' 2024-05-07T02:37:01.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.089+0000 7f1069220b80 1 --2- 172.21.15.19:0/1691228346 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x562c871483b0 0x562c871c9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:01.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.089+0000 7f1069220b80 4 client.26054 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:37:01.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.089+0000 7f1069220b80 1 -- 172.21.15.19:0/1691228346 --> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562c87072400 con 0x562c871483b0 2024-05-07T02:37:01.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.089+0000 7f1053400700 1 --2- 172.21.15.19:0/1691228346 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x562c871483b0 0x562c871c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:01.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.090+0000 7f1053400700 1 --2- 172.21.15.19:0/1691228346 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x562c871483b0 0x562c871c9580 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c0a556284ae4b5a8 server_cookie=627e8c723a749c3a in_seq=0 out_seq=0 2024-05-07T02:37:01.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.090+0000 7f103d600700 10 client.26054 ms_handle_connect on v2:172.21.15.64:6836/2585907777 2024-05-07T02:37:01.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.090+0000 7f103d600700 1 -- 172.21.15.19:0/1691228346 <== mds.4 v2:172.21.15.64:6836/2585907777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562c87072400 con 0x562c871483b0 2024-05-07T02:37:01.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.090+0000 7f103d600700 10 client.26054 handle_command_reply: tid=0 2024-05-07T02:37:01.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.090+0000 7f1069220b80 1 client.26054 shutdown 2024-05-07T02:37:01.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.090+0000 7f1069220b80 1 -- 172.21.15.19:0/1691228346 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c87142470 msgr2=0x562c871491c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:01.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.090+0000 7f1069220b80 1 --2- 172.21.15.19:0/1691228346 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c87142470 0x562c871491c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1040003710 tx=0x7f1040003740 comp rx=0 tx=0).stop 2024-05-07T02:37:01.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.090+0000 7f1069220b80 1 -- 172.21.15.19:0/1691228346 shutdown_connections 2024-05-07T02:37:01.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.090+0000 7f1069220b80 1 --2- 172.21.15.19:0/1691228346 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x562c871483b0 0x562c871c9580 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:01.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.090+0000 7f1069220b80 1 --2- 172.21.15.19:0/1691228346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562c87142dc0 0x562c871497f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:01.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.090+0000 7f1069220b80 1 --2- 172.21.15.19:0/1691228346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562c86fc4e40 0x562c871552f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:01.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.090+0000 7f1069220b80 1 --2- 172.21.15.19:0/1691228346 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562c87142470 0x562c871491c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:01.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.091+0000 7f1069220b80 1 -- 172.21.15.19:0/1691228346 >> 172.21.15.19:0/1691228346 conn(0x562c8714c090 msgr2=0x562c87137790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:37:01.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.091+0000 7f103d600700 0 client.26054 ms_handle_reset on v2:172.21.15.64:6836/2585907777 2024-05-07T02:37:01.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.091+0000 7f1069220b80 1 -- 172.21.15.19:0/1691228346 shutdown_connections 2024-05-07T02:37:01.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.091+0000 7f1069220b80 1 -- 172.21.15.19:0/1691228346 wait complete. 2024-05-07T02:37:01.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.091+0000 7f1069220b80 20 client.26054 trim_cache size 0 max 16384 2024-05-07T02:37:01.094 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:37:01.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.092+0000 7f105a600700 1 -- 172.21.15.19:0/1217790506 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f10380b2b40 msgr2=0x7f10380b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:01.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.092+0000 7f105a600700 1 --2- 172.21.15.19:0/1217790506 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f10380b2b40 0x7f10380b5050 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x562c8713ea40 tx=0x7f1044009260 comp rx=0 tx=0).stop 2024-05-07T02:37:01.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.092+0000 7f105a600700 1 -- 172.21.15.19:0/1217790506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1054109870 msgr2=0x7f10541ad230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:01.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.092+0000 7f105a600700 1 --2- 172.21.15.19:0/1217790506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1054109870 0x7f10541ad230 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f104c00efc0 tx=0x7f104c00c3a0 comp rx=0 tx=0).stop 2024-05-07T02:37:01.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.092+0000 7f105a600700 1 -- 172.21.15.19:0/1217790506 shutdown_connections 2024-05-07T02:37:01.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.092+0000 7f105a600700 1 --2- 172.21.15.19:0/1217790506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1054109870 0x7f10541ad230 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:01.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.092+0000 7f105a600700 1 --2- 172.21.15.19:0/1217790506 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f10380b2b40 0x7f10380b5050 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:01.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.092+0000 7f105a600700 1 --2- 172.21.15.19:0/1217790506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10541045f0 0x7f10541a21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:01.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.092+0000 7f105a600700 1 --2- 172.21.15.19:0/1217790506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1054108f20 0x7f10541a2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:01.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.092+0000 7f105a600700 1 -- 172.21.15.19:0/1217790506 >> 172.21.15.19:0/1217790506 conn(0x7f1054100040 msgr2=0x7f105410a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:37:01.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.093+0000 7f105a600700 1 -- 172.21.15.19:0/1217790506 shutdown_connections 2024-05-07T02:37:01.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:01.093+0000 7f105a600700 1 -- 172.21.15.19:0/1217790506 wait complete. 2024-05-07T02:37:01.751 INFO:tasks.workunit.client.0.smithi019.stdout: 1 156242 3.82 MB/sec execute 117 sec latency 83.958 ms 2024-05-07T02:37:02.498 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-07T02:37:02.751 INFO:tasks.workunit.client.0.smithi019.stdout: 1 156744 3.80 MB/sec execute 118 sec latency 31.098 ms 2024-05-07T02:37:02.957 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:37:03.312 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:03 smithi019 ceph-mon[29781]: pgmap v159: 129 pgs: 129 active+clean; 109 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 243 op/s 2024-05-07T02:37:03.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:03 smithi064 ceph-mon[34699]: pgmap v159: 129 pgs: 129 active+clean; 109 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 243 op/s 2024-05-07T02:37:03.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:03 smithi027 ceph-mon[34398]: pgmap v159: 129 pgs: 129 active+clean; 109 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 243 op/s 2024-05-07T02:37:03.751 INFO:tasks.workunit.client.0.smithi019.stdout: 1 158177 3.81 MB/sec execute 119 sec latency 16.727 ms 2024-05-07T02:37:04.751 INFO:tasks.workunit.client.0.smithi019.stdout: 1 159467 3.85 MB/sec execute 120 sec latency 19.105 ms 2024-05-07T02:37:05.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:05 smithi064 ceph-mon[34699]: pgmap v160: 129 pgs: 129 active+clean; 109 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 170 op/s 2024-05-07T02:37:05.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:05 smithi019 ceph-mon[29781]: pgmap v160: 129 pgs: 129 active+clean; 109 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 170 op/s 2024-05-07T02:37:05.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:05 smithi027 ceph-mon[34398]: pgmap v160: 129 pgs: 129 active+clean; 109 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 170 op/s 2024-05-07T02:37:05.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.695+0000 7f8be0000700 1 -- 172.21.15.19:0/2758307579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bd8110c00 msgr2=0x7f8bd8113020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:05.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.695+0000 7f8be0000700 1 --2- 172.21.15.19:0/2758307579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bd8110c00 0x7f8bd8113020 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8bd000b0a0 tx=0x7f8bd000b3b0 comp rx=0 tx=0).stop 2024-05-07T02:37:05.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.695+0000 7f8be0000700 1 -- 172.21.15.19:0/2758307579 shutdown_connections 2024-05-07T02:37:05.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.695+0000 7f8be0000700 1 --2- 172.21.15.19:0/2758307579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bd8110c00 0x7f8bd8113020 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:05.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.695+0000 7f8be0000700 1 --2- 172.21.15.19:0/2758307579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bd8075b00 0x7f8bd8075f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:05.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.695+0000 7f8be0000700 1 --2- 172.21.15.19:0/2758307579 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bd8076450 0x7f8bd81106c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:05.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.695+0000 7f8be0000700 1 -- 172.21.15.19:0/2758307579 >> 172.21.15.19:0/2758307579 conn(0x7f8bd8100040 msgr2=0x7f8bd8102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:37:05.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.695+0000 7f8be0000700 1 -- 172.21.15.19:0/2758307579 shutdown_connections 2024-05-07T02:37:05.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.695+0000 7f8be0000700 1 -- 172.21.15.19:0/2758307579 wait complete. 2024-05-07T02:37:05.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.697+0000 7f8be0000700 1 Processor -- start 2024-05-07T02:37:05.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.697+0000 7f8be0000700 1 -- start start 2024-05-07T02:37:05.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.697+0000 7f8be0000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bd8075b00 0x7f8bd819d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:05.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.697+0000 7f8be0000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bd8076450 0x7f8bd819d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:05.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.697+0000 7f8be0000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bd8110c00 0x7f8bd81a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:05.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.697+0000 7f8be0000700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd8116710 con 0x7f8bd8110c00 2024-05-07T02:37:05.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.697+0000 7f8be0000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd8116590 con 0x7f8bd8076450 2024-05-07T02:37:05.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.697+0000 7f8be0000700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd8116890 con 0x7f8bd8075b00 2024-05-07T02:37:05.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.698+0000 7f8bdd600700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bd8075b00 0x7f8bd819d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:05.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.698+0000 7f8bdd600700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bd8075b00 0x7f8bd819d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:41848/0 (socket says 172.21.15.19:41848) 2024-05-07T02:37:05.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.698+0000 7f8bdcc00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bd8076450 0x7f8bd819d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:05.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.698+0000 7f8bdd600700 1 -- 172.21.15.19:0/879749184 learned_addr learned my addr 172.21.15.19:0/879749184 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:37:05.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.698+0000 7f8bde000700 1 --2- 172.21.15.19:0/879749184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bd8110c00 0x7f8bd81a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:05.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.698+0000 7f8bdcc00700 1 -- 172.21.15.19:0/879749184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bd8075b00 msgr2=0x7f8bd819d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:05.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.698+0000 7f8bdcc00700 1 --2- 172.21.15.19:0/879749184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bd8075b00 0x7f8bd819d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:05.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.698+0000 7f8bdcc00700 1 -- 172.21.15.19:0/879749184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bd8110c00 msgr2=0x7f8bd81a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:05.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.698+0000 7f8bdcc00700 1 --2- 172.21.15.19:0/879749184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bd8110c00 0x7f8bd81a1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:05.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.698+0000 7f8bdcc00700 1 -- 172.21.15.19:0/879749184 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd81a1f10 con 0x7f8bd8076450 2024-05-07T02:37:05.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.698+0000 7f8bdd600700 1 --2- 172.21.15.19:0/879749184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bd8075b00 0x7f8bd819d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:37:05.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.699+0000 7f8bdcc00700 1 --2- 172.21.15.19:0/879749184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bd8076450 0x7f8bd819d5e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8bc400e9c0 tx=0x7f8bc400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:37:05.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.699+0000 7f8bd6000700 1 -- 172.21.15.19:0/879749184 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc400cb70 con 0x7f8bd8076450 2024-05-07T02:37:05.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.699+0000 7f8be0000700 1 -- 172.21.15.19:0/879749184 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bd81a2200 con 0x7f8bd8076450 2024-05-07T02:37:05.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.699+0000 7f8bd6000700 1 -- 172.21.15.19:0/879749184 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bc40044d0 con 0x7f8bd8076450 2024-05-07T02:37:05.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.699+0000 7f8bd6000700 1 -- 172.21.15.19:0/879749184 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4003dd0 con 0x7f8bd8076450 2024-05-07T02:37:05.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.699+0000 7f8be0000700 1 -- 172.21.15.19:0/879749184 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bd81a9b20 con 0x7f8bd8076450 2024-05-07T02:37:05.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.703+0000 7f8bd6000700 1 -- 172.21.15.19:0/879749184 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8bc4004000 con 0x7f8bd8076450 2024-05-07T02:37:05.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.705+0000 7f8beeb17b80 1 Processor -- start 2024-05-07T02:37:05.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.705+0000 7f8beeb17b80 1 -- start start 2024-05-07T02:37:05.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.706+0000 7f8beeb17b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639ee09f3b0 0x5639ee09f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:05.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.706+0000 7f8beeb17b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639ee099470 0x5639ee099880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:05.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.706+0000 7f8beeb17b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639ee099dc0 0x5639ee0a6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:05.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.706+0000 7f8beeb17b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5639edfbba20 con 0x5639ee09f3b0 2024-05-07T02:37:05.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.706+0000 7f8bd6000700 1 --2- 172.21.15.19:0/879749184 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f8bc00b29f0 0x7f8bc00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:05.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.706+0000 7f8beeb17b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5639edf59760 con 0x5639ee099470 2024-05-07T02:37:05.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.706+0000 7f8beeb17b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5639edfb79e0 con 0x5639ee099dc0 2024-05-07T02:37:05.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.706+0000 7f8bd6000700 1 -- 172.21.15.19:0/879749184 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f8bc40e2b70 con 0x7f8bd8076450 2024-05-07T02:37:05.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.706+0000 7f8bde000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639ee09f3b0 0x5639ee09f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:05.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.706+0000 7f8bdcc00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639ee099dc0 0x5639ee0a6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:05.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.706+0000 7f8bde000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639ee09f3b0 0x5639ee09f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37434/0 (socket says 172.21.15.19:37434) 2024-05-07T02:37:05.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.706+0000 7f8bde000700 1 -- 172.21.15.19:0/1172303655 learned_addr learned my addr 172.21.15.19:0/1172303655 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:37:05.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.706+0000 7f8bdd600700 1 --2- 172.21.15.19:0/1172303655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639ee099470 0x5639ee099880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:05.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.706+0000 7f8bde000700 1 --2- 172.21.15.19:0/879749184 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f8bc00b29f0 0x7f8bc00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:05.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.707+0000 7f8bdd600700 1 -- 172.21.15.19:0/1172303655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639ee099dc0 msgr2=0x5639ee0a6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:05.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.707+0000 7f8bdd600700 1 --2- 172.21.15.19:0/1172303655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639ee099dc0 0x5639ee0a6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:05.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.707+0000 7f8bdd600700 1 -- 172.21.15.19:0/1172303655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639ee09f3b0 msgr2=0x5639ee09f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:05.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.707+0000 7f8bdd600700 1 --2- 172.21.15.19:0/1172303655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639ee09f3b0 0x5639ee09f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:05.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.707+0000 7f8bdd600700 1 -- 172.21.15.19:0/1172303655 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639edf088d0 con 0x5639ee099470 2024-05-07T02:37:05.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.707+0000 7f8bdd600700 1 --2- 172.21.15.19:0/1172303655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639ee099470 0x5639ee099880 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc011570 tx=0x7f8bcc011880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36e2da6d7359e7c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:37:05.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.707+0000 7f8bcaa00700 1 -- 172.21.15.19:0/1172303655 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc004230 con 0x5639ee099470 2024-05-07T02:37:05.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.708+0000 7f8bcaa00700 1 -- 172.21.15.19:0/1172303655 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bcc0043d0 con 0x5639ee099470 2024-05-07T02:37:05.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.708+0000 7f8bcaa00700 1 -- 172.21.15.19:0/1172303655 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc019950 con 0x5639ee099470 2024-05-07T02:37:05.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.708+0000 7f8beeb17b80 1 -- 172.21.15.19:0/1172303655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639ee099470 msgr2=0x5639ee099880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:05.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.708+0000 7f8beeb17b80 1 --2- 172.21.15.19:0/1172303655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639ee099470 0x5639ee099880 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc011570 tx=0x7f8bcc011880 comp rx=0 tx=0).stop 2024-05-07T02:37:05.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.708+0000 7f8bde000700 1 --2- 172.21.15.19:0/879749184 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f8bc00b29f0 0x7f8bc00b4f00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x5639ee09ee70 tx=0x7f8bd00099e0 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:37:05.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.709+0000 7f8beeb17b80 1 -- 172.21.15.19:0/1172303655 shutdown_connections 2024-05-07T02:37:05.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.709+0000 7f8beeb17b80 1 --2- 172.21.15.19:0/1172303655 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639ee099dc0 0x5639ee0a6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:05.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.709+0000 7f8beeb17b80 1 --2- 172.21.15.19:0/1172303655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639ee09f3b0 0x5639ee09f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:05.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.709+0000 7f8beeb17b80 1 --2- 172.21.15.19:0/1172303655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639ee099470 0x5639ee099880 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:05.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.709+0000 7f8beeb17b80 1 -- 172.21.15.19:0/1172303655 >> 172.21.15.19:0/1172303655 conn(0x5639edf1be40 msgr2=0x5639ee0942f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:37:05.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.709+0000 7f8beeb17b80 1 -- 172.21.15.19:0/1172303655 shutdown_connections 2024-05-07T02:37:05.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.709+0000 7f8beeb17b80 1 -- 172.21.15.19:0/1172303655 wait complete. 2024-05-07T02:37:05.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.710+0000 7f8beeb17b80 1 -- start start 2024-05-07T02:37:05.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.710+0000 7f8beeb17b80 1 Processor -- start 2024-05-07T02:37:05.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.711+0000 7f8beeb17b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639ee099470 0x5639ee0a3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:05.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.711+0000 7f8beeb17b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639ee099dc0 0x5639ee0a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:05.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.711+0000 7f8beeb17b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639ee09f3b0 0x5639ee0a4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:05.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.711+0000 7f8beeb17b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5639ee0aefd0 con 0x5639ee099470 2024-05-07T02:37:05.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.711+0000 7f8beeb17b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5639edf21b20 con 0x5639ee09f3b0 2024-05-07T02:37:05.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.711+0000 7f8beeb17b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5639ee0af150 con 0x5639ee099dc0 2024-05-07T02:37:05.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.711+0000 7f8beeb17b80 10 client.0 _ll_register_callbacks cb 0x5639edf62c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:37:05.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.711+0000 7f8bdd600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639ee099470 0x5639ee0a3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:05.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.711+0000 7f8bdd600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639ee099470 0x5639ee0a3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37438/0 (socket says 172.21.15.19:37438) 2024-05-07T02:37:05.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.711+0000 7f8bdd600700 1 -- 172.21.15.19:0/790736904 learned_addr learned my addr 172.21.15.19:0/790736904 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:37:05.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.711+0000 7f8bdcc00700 1 --2- 172.21.15.19:0/790736904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639ee099dc0 0x5639ee0a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:05.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.711+0000 7f8bde000700 1 --2- 172.21.15.19:0/790736904 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639ee09f3b0 0x5639ee0a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:05.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.712+0000 7f8bdd600700 1 -- 172.21.15.19:0/790736904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639ee099dc0 msgr2=0x5639ee0a3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:05.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.712+0000 7f8bdd600700 1 --2- 172.21.15.19:0/790736904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639ee099dc0 0x5639ee0a3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:05.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.712+0000 7f8bdd600700 1 -- 172.21.15.19:0/790736904 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639ee09f3b0 msgr2=0x5639ee0a4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:05.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.712+0000 7f8bdd600700 1 --2- 172.21.15.19:0/790736904 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639ee09f3b0 0x5639ee0a4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:05.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.712+0000 7f8bdd600700 1 -- 172.21.15.19:0/790736904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639edf81420 con 0x5639ee099470 2024-05-07T02:37:05.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.712+0000 7f8bde000700 1 --2- 172.21.15.19:0/790736904 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639ee09f3b0 0x5639ee0a4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:37:05.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.712+0000 7f8bdd600700 1 --2- 172.21.15.19:0/790736904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639ee099470 0x5639ee0a3580 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc0096b0 tx=0x7f8bcc005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fd84f8bc53cfaf8 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:37:05.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.712+0000 7f8bbf400700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-07T02:37:05.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.713+0000 7f8bbf400700 1 -- 172.21.15.19:0/790736904 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc004950 con 0x5639ee099470 2024-05-07T02:37:05.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.713+0000 7f8bbf400700 1 -- 172.21.15.19:0/790736904 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f8bcc004af0 con 0x5639ee099470 2024-05-07T02:37:05.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.713+0000 7f8beeb17b80 1 -- 172.21.15.19:0/790736904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bcc0043d0 con 0x5639ee099470 2024-05-07T02:37:05.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.713+0000 7f8bbf400700 1 -- 172.21.15.19:0/790736904 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f8bcc033050 con 0x5639ee099470 2024-05-07T02:37:05.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.713+0000 7f8beeb17b80 10 client.15888 fetch_fsmap learned FSMap version 40 2024-05-07T02:37:05.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.713+0000 7f8beeb17b80 1 -- 172.21.15.19:0/790736904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5639ee0b9c30 con 0x5639ee099470 2024-05-07T02:37:05.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.714+0000 7f8bbf400700 1 -- 172.21.15.19:0/790736904 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bcc0043d0 con 0x5639ee099470 2024-05-07T02:37:05.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.714+0000 7f8bbf400700 1 -- 172.21.15.19:0/790736904 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f8bcc0195d0 con 0x5639ee099470 2024-05-07T02:37:05.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.714+0000 7f8beeb17b80 10 client.15888 fetch_fsmap finished waiting for FSMap version 40 2024-05-07T02:37:05.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.714+0000 7f8beeb17b80 10 client.15888 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-05-07T02:37:05.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.714+0000 7f8beeb17b80 20 client.15888 populate_metadata read hostname 'smithi019' 2024-05-07T02:37:05.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.714+0000 7f8beeb17b80 1 --2- 172.21.15.19:0/790736904 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x5639ee0bc210 0x5639ee0be630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:05.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.714+0000 7f8beeb17b80 4 client.15888 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:37:05.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.714+0000 7f8beeb17b80 1 -- 172.21.15.19:0/790736904 --> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639ee0662e0 con 0x5639ee0bc210 2024-05-07T02:37:05.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.715+0000 7f8bdcc00700 1 --2- 172.21.15.19:0/790736904 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x5639ee0bc210 0x5639ee0be630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:05.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.715+0000 7f8bdcc00700 1 --2- 172.21.15.19:0/790736904 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x5639ee0bc210 0x5639ee0be630 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2fdc68351f09f2b6 server_cookie=9375a0465106caae in_seq=0 out_seq=0 2024-05-07T02:37:05.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.715+0000 7f8bbf400700 10 client.15888 ms_handle_connect on v2:172.21.15.27:6836/2583138143 2024-05-07T02:37:05.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.716+0000 7f8bbf400700 1 -- 172.21.15.19:0/790736904 <== mds.3 v2:172.21.15.27:6836/2583138143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5639ee0662e0 con 0x5639ee0bc210 2024-05-07T02:37:05.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.716+0000 7f8bbf400700 10 client.15888 handle_command_reply: tid=0 2024-05-07T02:37:05.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.716+0000 7f8beeb17b80 1 client.15888 shutdown 2024-05-07T02:37:05.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.716+0000 7f8beeb17b80 1 -- 172.21.15.19:0/790736904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639ee099470 msgr2=0x5639ee0a3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:05.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.716+0000 7f8beeb17b80 1 --2- 172.21.15.19:0/790736904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639ee099470 0x5639ee0a3580 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc0096b0 tx=0x7f8bcc005270 comp rx=0 tx=0).stop 2024-05-07T02:37:05.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.716+0000 7f8beeb17b80 1 -- 172.21.15.19:0/790736904 shutdown_connections 2024-05-07T02:37:05.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.716+0000 7f8beeb17b80 1 --2- 172.21.15.19:0/790736904 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x5639ee0bc210 0x5639ee0be630 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:05.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.716+0000 7f8beeb17b80 1 --2- 172.21.15.19:0/790736904 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639ee09f3b0 0x5639ee0a4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:05.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.716+0000 7f8beeb17b80 1 --2- 172.21.15.19:0/790736904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639ee099470 0x5639ee0a3580 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:05.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.716+0000 7f8beeb17b80 1 --2- 172.21.15.19:0/790736904 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639ee099dc0 0x5639ee0a3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:05.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.716+0000 7f8beeb17b80 1 -- 172.21.15.19:0/790736904 >> 172.21.15.19:0/790736904 conn(0x5639edf1be40 msgr2=0x5639ee098460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:37:05.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.716+0000 7f8bbf400700 0 client.15888 ms_handle_reset on v2:172.21.15.27:6836/2583138143 2024-05-07T02:37:05.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.717+0000 7f8beeb17b80 1 -- 172.21.15.19:0/790736904 shutdown_connections 2024-05-07T02:37:05.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.717+0000 7f8beeb17b80 1 -- 172.21.15.19:0/790736904 wait complete. 2024-05-07T02:37:05.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.717+0000 7f8beeb17b80 20 client.15888 trim_cache size 0 max 16384 2024-05-07T02:37:05.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.738+0000 7f8beeb17b80 1 Processor -- start 2024-05-07T02:37:05.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.738+0000 7f8beeb17b80 1 -- start start 2024-05-07T02:37:05.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.738+0000 7f8beeb17b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639ee099470 0x5639ee09dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:05.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.738+0000 7f8beeb17b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639ee099dc0 0x5639ee0a28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:05.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.738+0000 7f8beeb17b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639ee09f3b0 0x5639ee0a3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:05.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.738+0000 7f8beeb17b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd81ae450 con 0x5639ee099470 2024-05-07T02:37:05.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.738+0000 7f8beeb17b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd81afff0 con 0x5639ee099dc0 2024-05-07T02:37:05.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.738+0000 7f8beeb17b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd81aeb20 con 0x5639ee09f3b0 2024-05-07T02:37:05.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.739+0000 7f8bde000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639ee09f3b0 0x5639ee0a3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:05.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.739+0000 7f8bde000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639ee09f3b0 0x5639ee0a3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:41880/0 (socket says 172.21.15.19:41880) 2024-05-07T02:37:05.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.739+0000 7f8bde000700 1 -- 172.21.15.19:0/2926628817 learned_addr learned my addr 172.21.15.19:0/2926628817 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:37:05.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.739+0000 7f8bdd600700 1 --2- 172.21.15.19:0/2926628817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639ee099470 0x5639ee09dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:05.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.739+0000 7f8bdcc00700 1 --2- 172.21.15.19:0/2926628817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639ee099dc0 0x5639ee0a28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:05.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.739+0000 7f8bde000700 1 -- 172.21.15.19:0/2926628817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639ee099dc0 msgr2=0x5639ee0a28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:05.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.739+0000 7f8bde000700 1 --2- 172.21.15.19:0/2926628817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639ee099dc0 0x5639ee0a28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:05.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.739+0000 7f8bde000700 1 -- 172.21.15.19:0/2926628817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639ee099470 msgr2=0x5639ee09dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:05.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.739+0000 7f8bde000700 1 --2- 172.21.15.19:0/2926628817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639ee099470 0x5639ee09dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:05.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.739+0000 7f8bde000700 1 -- 172.21.15.19:0/2926628817 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639ee095240 con 0x5639ee09f3b0 2024-05-07T02:37:05.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.739+0000 7f8bdd600700 1 --2- 172.21.15.19:0/2926628817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639ee099470 0x5639ee09dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:37:05.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.739+0000 7f8bdcc00700 1 --2- 172.21.15.19:0/2926628817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639ee099dc0 0x5639ee0a28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:37:05.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.740+0000 7f8bde000700 1 --2- 172.21.15.19:0/2926628817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639ee09f3b0 0x5639ee0a3e40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0009e60 tx=0x7f8bd0019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed18e32945736a30 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:37:05.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.740+0000 7f8bca000700 1 -- 172.21.15.19:0/2926628817 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0030070 con 0x5639ee09f3b0 2024-05-07T02:37:05.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.740+0000 7f8bca000700 1 -- 172.21.15.19:0/2926628817 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bd0026040 con 0x5639ee09f3b0 2024-05-07T02:37:05.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.740+0000 7f8bca000700 1 -- 172.21.15.19:0/2926628817 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd002bae0 con 0x5639ee09f3b0 2024-05-07T02:37:05.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.741+0000 7f8beeb17b80 1 -- 172.21.15.19:0/2926628817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639ee09f3b0 msgr2=0x5639ee0a3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:05.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.741+0000 7f8beeb17b80 1 --2- 172.21.15.19:0/2926628817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639ee09f3b0 0x5639ee0a3e40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0009e60 tx=0x7f8bd0019000 comp rx=0 tx=0).stop 2024-05-07T02:37:05.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.742+0000 7f8beeb17b80 1 -- 172.21.15.19:0/2926628817 shutdown_connections 2024-05-07T02:37:05.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.742+0000 7f8beeb17b80 1 --2- 172.21.15.19:0/2926628817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639ee09f3b0 0x5639ee0a3e40 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:05.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.742+0000 7f8beeb17b80 1 --2- 172.21.15.19:0/2926628817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639ee099470 0x5639ee09dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:05.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.742+0000 7f8beeb17b80 1 --2- 172.21.15.19:0/2926628817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639ee099dc0 0x5639ee0a28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:05.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.742+0000 7f8beeb17b80 1 -- 172.21.15.19:0/2926628817 >> 172.21.15.19:0/2926628817 conn(0x5639edf1be40 msgr2=0x5639ee08e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:37:05.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.742+0000 7f8beeb17b80 1 -- 172.21.15.19:0/2926628817 shutdown_connections 2024-05-07T02:37:05.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.742+0000 7f8beeb17b80 1 -- 172.21.15.19:0/2926628817 wait complete. 2024-05-07T02:37:05.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.742+0000 7f8beeb17b80 1 -- start start 2024-05-07T02:37:05.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.743+0000 7f8beeb17b80 1 Processor -- start 2024-05-07T02:37:05.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.743+0000 7f8beeb17b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639ee099470 0x5639ee0b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:05.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.743+0000 7f8beeb17b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639ee099dc0 0x5639ee0b4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:05.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.743+0000 7f8beeb17b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639ee09f3b0 0x5639ee0c2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:05.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.743+0000 7f8beeb17b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd81af920 con 0x5639ee099470 2024-05-07T02:37:05.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.743+0000 7f8beeb17b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd81af250 con 0x5639ee09f3b0 2024-05-07T02:37:05.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.743+0000 7f8beeb17b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd81ac410 con 0x5639ee099dc0 2024-05-07T02:37:05.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.743+0000 7f8beeb17b80 10 client.0 _ll_register_callbacks cb 0x5639edf62c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:37:05.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.743+0000 7f8bdd600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639ee099470 0x5639ee0b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:05.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.743+0000 7f8bde000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639ee09f3b0 0x5639ee0c2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:05.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.744+0000 7f8bde000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639ee09f3b0 0x5639ee0c2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:52428/0 (socket says 172.21.15.19:52428) 2024-05-07T02:37:05.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.744+0000 7f8bdcc00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639ee099dc0 0x5639ee0b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:05.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.744+0000 7f8bde000700 1 -- 172.21.15.19:0/1658773048 learned_addr learned my addr 172.21.15.19:0/1658773048 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:37:05.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.744+0000 7f8bdcc00700 1 -- 172.21.15.19:0/1658773048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639ee09f3b0 msgr2=0x5639ee0c2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:05.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.744+0000 7f8bdcc00700 1 --2- 172.21.15.19:0/1658773048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639ee09f3b0 0x5639ee0c2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:05.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.744+0000 7f8bdcc00700 1 -- 172.21.15.19:0/1658773048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639ee099470 msgr2=0x5639ee0b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:05.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.744+0000 7f8bdcc00700 1 --2- 172.21.15.19:0/1658773048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639ee099470 0x5639ee0b43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:05.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.744+0000 7f8bdcc00700 1 -- 172.21.15.19:0/1658773048 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639ee0c3160 con 0x5639ee099dc0 2024-05-07T02:37:05.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.744+0000 7f8bdcc00700 1 --2- 172.21.15.19:0/1658773048 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639ee099dc0 0x5639ee0b4920 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8bc40dfed0 tx=0x7f8bc400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38ae2d1d836fe077 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:37:05.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.744+0000 7f8bbf400700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:37:05.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.744+0000 7f8bbf400700 1 -- 172.21.15.19:0/1658773048 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc40a9a60 con 0x5639ee099dc0 2024-05-07T02:37:05.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.745+0000 7f8bbf400700 1 -- 172.21.15.19:0/1658773048 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f8bc4009400 con 0x5639ee099dc0 2024-05-07T02:37:05.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.745+0000 7f8beeb17b80 1 -- 172.21.15.19:0/1658773048 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bd0026040 con 0x5639ee099dc0 2024-05-07T02:37:05.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.745+0000 7f8bbf400700 1 -- 172.21.15.19:0/1658773048 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f8bc40ade10 con 0x5639ee099dc0 2024-05-07T02:37:05.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.745+0000 7f8beeb17b80 10 client.26044 fetch_fsmap learned FSMap version 40 2024-05-07T02:37:05.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.745+0000 7f8beeb17b80 1 -- 172.21.15.19:0/1658773048 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5639ee125e60 con 0x5639ee099dc0 2024-05-07T02:37:05.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.745+0000 7f8bbf400700 1 -- 172.21.15.19:0/1658773048 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bd0026040 con 0x5639ee099dc0 2024-05-07T02:37:05.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.745+0000 7f8bbf400700 1 -- 172.21.15.19:0/1658773048 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f8bc401e2e0 con 0x5639ee099dc0 2024-05-07T02:37:05.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.746+0000 7f8beeb17b80 10 client.26044 fetch_fsmap finished waiting for FSMap version 40 2024-05-07T02:37:05.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.746+0000 7f8beeb17b80 10 client.26044 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-05-07T02:37:05.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.746+0000 7f8beeb17b80 20 client.26044 populate_metadata read hostname 'smithi019' 2024-05-07T02:37:05.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.746+0000 7f8beeb17b80 1 --2- 172.21.15.19:0/1658773048 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x5639ee126590 0x5639ee1289b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:05.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.746+0000 7f8beeb17b80 4 client.26044 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:37:05.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.746+0000 7f8beeb17b80 1 -- 172.21.15.19:0/1658773048 --> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5639edfc9400 con 0x5639ee126590 2024-05-07T02:37:05.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.746+0000 7f8bdd600700 1 --2- 172.21.15.19:0/1658773048 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x5639ee126590 0x5639ee1289b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:05.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.747+0000 7f8bdd600700 1 --2- 172.21.15.19:0/1658773048 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x5639ee126590 0x5639ee1289b0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b8caaf352245263 server_cookie=cab86836f6a5a554 in_seq=0 out_seq=0 2024-05-07T02:37:05.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.747+0000 7f8bbf400700 10 client.26044 ms_handle_connect on v2:172.21.15.27:6836/2583138143 2024-05-07T02:37:05.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.747+0000 7f8bbf400700 1 -- 172.21.15.19:0/1658773048 <== mds.3 v2:172.21.15.27:6836/2583138143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5639edfc9400 con 0x5639ee126590 2024-05-07T02:37:05.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.747+0000 7f8bbf400700 10 client.26044 handle_command_reply: tid=0 2024-05-07T02:37:05.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.747+0000 7f8beeb17b80 1 client.26044 shutdown 2024-05-07T02:37:05.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.747+0000 7f8beeb17b80 1 -- 172.21.15.19:0/1658773048 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639ee099dc0 msgr2=0x5639ee0b4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:05.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.747+0000 7f8beeb17b80 1 --2- 172.21.15.19:0/1658773048 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639ee099dc0 0x5639ee0b4920 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8bc40dfed0 tx=0x7f8bc400ef10 comp rx=0 tx=0).stop 2024-05-07T02:37:05.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.747+0000 7f8beeb17b80 1 -- 172.21.15.19:0/1658773048 shutdown_connections 2024-05-07T02:37:05.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.747+0000 7f8beeb17b80 1 --2- 172.21.15.19:0/1658773048 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x5639ee126590 0x5639ee1289b0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:05.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.747+0000 7f8beeb17b80 1 --2- 172.21.15.19:0/1658773048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639ee09f3b0 0x5639ee0c2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:05.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.747+0000 7f8beeb17b80 1 --2- 172.21.15.19:0/1658773048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639ee099470 0x5639ee0b43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:05.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.747+0000 7f8beeb17b80 1 --2- 172.21.15.19:0/1658773048 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5639ee099dc0 0x5639ee0b4920 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:05.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.747+0000 7f8bbf400700 0 client.26044 ms_handle_reset on v2:172.21.15.27:6836/2583138143 2024-05-07T02:37:05.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.747+0000 7f8beeb17b80 1 -- 172.21.15.19:0/1658773048 >> 172.21.15.19:0/1658773048 conn(0x5639edf1be40 msgr2=0x5639ee08e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:37:05.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.748+0000 7f8beeb17b80 1 -- 172.21.15.19:0/1658773048 shutdown_connections 2024-05-07T02:37:05.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.748+0000 7f8beeb17b80 1 -- 172.21.15.19:0/1658773048 wait complete. 2024-05-07T02:37:05.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.748+0000 7f8beeb17b80 20 client.26044 trim_cache size 0 max 16384 2024-05-07T02:37:05.751 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:37:05.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.749+0000 7f8be0000700 1 -- 172.21.15.19:0/879749184 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f8bc00b29f0 msgr2=0x7f8bc00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:05.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.749+0000 7f8be0000700 1 --2- 172.21.15.19:0/879749184 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f8bc00b29f0 0x7f8bc00b4f00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x5639ee09ee70 tx=0x7f8bd00099e0 comp rx=0 tx=0).stop 2024-05-07T02:37:05.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.749+0000 7f8be0000700 1 -- 172.21.15.19:0/879749184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bd8076450 msgr2=0x7f8bd819d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:05.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.749+0000 7f8be0000700 1 --2- 172.21.15.19:0/879749184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bd8076450 0x7f8bd819d5e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8bc400e9c0 tx=0x7f8bc400ecd0 comp rx=0 tx=0).stop 2024-05-07T02:37:05.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.749+0000 7f8be0000700 1 -- 172.21.15.19:0/879749184 shutdown_connections 2024-05-07T02:37:05.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.749+0000 7f8be0000700 1 --2- 172.21.15.19:0/879749184 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f8bc00b29f0 0x7f8bc00b4f00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:05.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.749+0000 7f8be0000700 1 --2- 172.21.15.19:0/879749184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bd8110c00 0x7f8bd81a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:05.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.749+0000 7f8be0000700 1 --2- 172.21.15.19:0/879749184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8bd8075b00 0x7f8bd819d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:05.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.749+0000 7f8be0000700 1 --2- 172.21.15.19:0/879749184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bd8076450 0x7f8bd819d5e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:05.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.749+0000 7f8be0000700 1 -- 172.21.15.19:0/879749184 >> 172.21.15.19:0/879749184 conn(0x7f8bd8100040 msgr2=0x7f8bd8101e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:37:05.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.749+0000 7f8be0000700 1 -- 172.21.15.19:0/879749184 shutdown_connections 2024-05-07T02:37:05.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:05.749+0000 7f8be0000700 1 -- 172.21.15.19:0/879749184 wait complete. 2024-05-07T02:37:05.753 INFO:tasks.workunit.client.0.smithi019.stdout: 1 160064 3.86 MB/sec execute 121 sec latency 27.304 ms 2024-05-07T02:37:06.751 INFO:tasks.workunit.client.0.smithi019.stdout: 1 160827 3.84 MB/sec execute 122 sec latency 57.450 ms 2024-05-07T02:37:06.908 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-07T02:37:07.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:07 smithi064 ceph-mon[34699]: pgmap v161: 129 pgs: 129 active+clean; 112 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 255 op/s 2024-05-07T02:37:07.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:07 smithi019 ceph-mon[29781]: pgmap v161: 129 pgs: 129 active+clean; 112 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 255 op/s 2024-05-07T02:37:07.426 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:37:07.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:07 smithi027 ceph-mon[34398]: pgmap v161: 129 pgs: 129 active+clean; 112 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 255 op/s 2024-05-07T02:37:07.751 INFO:tasks.workunit.client.0.smithi019.stdout: 1 162623 3.90 MB/sec execute 123 sec latency 14.480 ms 2024-05-07T02:37:08.751 INFO:tasks.workunit.client.0.smithi019.stdout: 1 163511 3.92 MB/sec execute 124 sec latency 20.010 ms 2024-05-07T02:37:09.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:09 smithi064 ceph-mon[34699]: pgmap v162: 129 pgs: 129 active+clean; 112 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 177 op/s 2024-05-07T02:37:09.366 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:09 smithi019 ceph-mon[29781]: pgmap v162: 129 pgs: 129 active+clean; 112 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 177 op/s 2024-05-07T02:37:09.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:09 smithi027 ceph-mon[34398]: pgmap v162: 129 pgs: 129 active+clean; 112 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 177 op/s 2024-05-07T02:37:09.752 INFO:tasks.workunit.client.0.smithi019.stdout: 1 164070 3.89 MB/sec execute 125 sec latency 19.244 ms 2024-05-07T02:37:10.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.016+0000 7f656be00700 1 -- 172.21.15.19:0/274477465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6564105e70 msgr2=0x7f656410e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:10.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.016+0000 7f656be00700 1 --2- 172.21.15.19:0/274477465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6564105e70 0x7f656410e3c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6550009990 tx=0x7f6550009ca0 comp rx=0 tx=0).stop 2024-05-07T02:37:10.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.016+0000 7f656be00700 1 -- 172.21.15.19:0/274477465 shutdown_connections 2024-05-07T02:37:10.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.016+0000 7f656be00700 1 --2- 172.21.15.19:0/274477465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f656410e900 0x7f6564110dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:10.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.016+0000 7f656be00700 1 --2- 172.21.15.19:0/274477465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6564105520 0x7f6564105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:10.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.016+0000 7f656be00700 1 --2- 172.21.15.19:0/274477465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6564105e70 0x7f656410e3c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:10.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.016+0000 7f656be00700 1 -- 172.21.15.19:0/274477465 >> 172.21.15.19:0/274477465 conn(0x7f65640fedc0 msgr2=0x7f65641011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:37:10.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.016+0000 7f656be00700 1 -- 172.21.15.19:0/274477465 shutdown_connections 2024-05-07T02:37:10.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.016+0000 7f656be00700 1 -- 172.21.15.19:0/274477465 wait complete. 2024-05-07T02:37:10.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.018+0000 7f656be00700 1 Processor -- start 2024-05-07T02:37:10.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.018+0000 7f656be00700 1 -- start start 2024-05-07T02:37:10.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.019+0000 7f656be00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6564105520 0x7f656419d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:10.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.019+0000 7f656be00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6564105e70 0x7f656419d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:10.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.019+0000 7f656be00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f656410e900 0x7f65641a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:10.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.019+0000 7f656be00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6564114490 con 0x7f6564105e70 2024-05-07T02:37:10.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.019+0000 7f656be00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f6564114310 con 0x7f6564105520 2024-05-07T02:37:10.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.019+0000 7f656be00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6564114610 con 0x7f656410e900 2024-05-07T02:37:10.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.019+0000 7f6569400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6564105520 0x7f656419d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:10.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.019+0000 7f6569400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6564105520 0x7f656419d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:55518/0 (socket says 172.21.15.19:55518) 2024-05-07T02:37:10.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.019+0000 7f6569400700 1 -- 172.21.15.19:0/1237279280 learned_addr learned my addr 172.21.15.19:0/1237279280 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:37:10.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.019+0000 7f6569e00700 1 --2- 172.21.15.19:0/1237279280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f656410e900 0x7f65641a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:10.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.019+0000 7f6568a00700 1 --2- 172.21.15.19:0/1237279280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6564105e70 0x7f656419d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:10.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.020+0000 7f6569400700 1 -- 172.21.15.19:0/1237279280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f656410e900 msgr2=0x7f65641a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:10.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.020+0000 7f6569400700 1 --2- 172.21.15.19:0/1237279280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f656410e900 0x7f65641a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:10.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.020+0000 7f6569400700 1 -- 172.21.15.19:0/1237279280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6564105e70 msgr2=0x7f656419d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:10.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.020+0000 7f6569400700 1 --2- 172.21.15.19:0/1237279280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6564105e70 0x7f656419d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:10.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.020+0000 7f6569400700 1 -- 172.21.15.19:0/1237279280 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65641a1ef0 con 0x7f6564105520 2024-05-07T02:37:10.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.020+0000 7f6568a00700 1 --2- 172.21.15.19:0/1237279280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6564105e70 0x7f656419d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:37:10.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.020+0000 7f6569400700 1 --2- 172.21.15.19:0/1237279280 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6564105520 0x7f656419d020 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f655800e9a0 tx=0x7f655800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:37:10.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.020+0000 7f6562000700 1 -- 172.21.15.19:0/1237279280 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655800cbd0 con 0x7f6564105520 2024-05-07T02:37:10.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.020+0000 7f6562000700 1 -- 172.21.15.19:0/1237279280 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f655800cd70 con 0x7f6564105520 2024-05-07T02:37:10.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.020+0000 7f656be00700 1 -- 172.21.15.19:0/1237279280 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65641a21e0 con 0x7f6564105520 2024-05-07T02:37:10.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.020+0000 7f6562000700 1 -- 172.21.15.19:0/1237279280 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6558018680 con 0x7f6564105520 2024-05-07T02:37:10.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.021+0000 7f656be00700 1 -- 172.21.15.19:0/1237279280 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65641a9b00 con 0x7f6564105520 2024-05-07T02:37:10.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.025+0000 7f6562000700 1 -- 172.21.15.19:0/1237279280 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f6558010430 con 0x7f6564105520 2024-05-07T02:37:10.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.028+0000 7f657a9deb80 1 Processor -- start 2024-05-07T02:37:10.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.028+0000 7f657a9deb80 1 -- start start 2024-05-07T02:37:10.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.028+0000 7f6562000700 1 --2- 172.21.15.19:0/1237279280 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f654c0b2a70 0x7f654c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:10.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.028+0000 7f6562000700 1 -- 172.21.15.19:0/1237279280 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f65580e2d70 con 0x7f6564105520 2024-05-07T02:37:10.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.028+0000 7f6569e00700 1 --2- 172.21.15.19:0/1237279280 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f654c0b2a70 0x7f654c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:10.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.028+0000 7f657a9deb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d79a2e53b0 0x55d79a2e57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:10.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.029+0000 7f657a9deb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d79a2df470 0x55d79a2df880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:10.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.029+0000 7f657a9deb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d79a2dfdc0 0x55d79a2ec040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:10.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.029+0000 7f657a9deb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d79a201a20 con 0x55d79a2df470 2024-05-07T02:37:10.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.029+0000 7f657a9deb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d79a19f760 con 0x55d79a2dfdc0 2024-05-07T02:37:10.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.029+0000 7f657a9deb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d79a1fd9e0 con 0x55d79a2e53b0 2024-05-07T02:37:10.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.029+0000 7f6569400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d79a2e53b0 0x55d79a2e57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:10.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.029+0000 7f6569400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d79a2e53b0 0x55d79a2e57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:58374/0 (socket says 172.21.15.19:58374) 2024-05-07T02:37:10.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.029+0000 7f6569e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d79a2dfdc0 0x55d79a2ec040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:10.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.029+0000 7f6569400700 1 -- 172.21.15.19:0/3075969530 learned_addr learned my addr 172.21.15.19:0/3075969530 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:37:10.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.029+0000 7f6568a00700 1 --2- 172.21.15.19:0/3075969530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d79a2df470 0x55d79a2df880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:10.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.030+0000 7f6569400700 1 -- 172.21.15.19:0/3075969530 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d79a2dfdc0 msgr2=0x55d79a2ec040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:10.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.030+0000 7f6569400700 1 --2- 172.21.15.19:0/3075969530 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d79a2dfdc0 0x55d79a2ec040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:10.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.030+0000 7f6569400700 1 -- 172.21.15.19:0/3075969530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d79a2df470 msgr2=0x55d79a2df880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:10.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.030+0000 7f6569400700 1 --2- 172.21.15.19:0/3075969530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d79a2df470 0x55d79a2df880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:10.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.030+0000 7f6569400700 1 -- 172.21.15.19:0/3075969530 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d79a14e8d0 con 0x55d79a2e53b0 2024-05-07T02:37:10.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.030+0000 7f6569e00700 1 --2- 172.21.15.19:0/1237279280 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f654c0b2a70 0x7f654c0b4f80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f655c00f870 tx=0x7f655c00fc30 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:37:10.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.030+0000 7f6569400700 1 --2- 172.21.15.19:0/3075969530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d79a2e53b0 0x55d79a2e57c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f65580a5ce0 tx=0x7f65580b2400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=513bce4395d783ef server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:37:10.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.030+0000 7f6556a00700 1 -- 172.21.15.19:0/3075969530 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65580aa360 con 0x55d79a2e53b0 2024-05-07T02:37:10.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.030+0000 7f6556a00700 1 -- 172.21.15.19:0/3075969530 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6558091040 con 0x55d79a2e53b0 2024-05-07T02:37:10.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.030+0000 7f6556a00700 1 -- 172.21.15.19:0/3075969530 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6558010720 con 0x55d79a2e53b0 2024-05-07T02:37:10.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.031+0000 7f657a9deb80 1 -- 172.21.15.19:0/3075969530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d79a2e53b0 msgr2=0x55d79a2e57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:10.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.031+0000 7f657a9deb80 1 --2- 172.21.15.19:0/3075969530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d79a2e53b0 0x55d79a2e57c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f65580a5ce0 tx=0x7f65580b2400 comp rx=0 tx=0).stop 2024-05-07T02:37:10.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.032+0000 7f657a9deb80 1 -- 172.21.15.19:0/3075969530 shutdown_connections 2024-05-07T02:37:10.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.032+0000 7f657a9deb80 1 --2- 172.21.15.19:0/3075969530 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d79a2dfdc0 0x55d79a2ec040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:10.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.032+0000 7f657a9deb80 1 --2- 172.21.15.19:0/3075969530 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d79a2e53b0 0x55d79a2e57c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:10.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.032+0000 7f657a9deb80 1 --2- 172.21.15.19:0/3075969530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d79a2df470 0x55d79a2df880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:10.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.032+0000 7f657a9deb80 1 -- 172.21.15.19:0/3075969530 >> 172.21.15.19:0/3075969530 conn(0x55d79a161e40 msgr2=0x55d79a2da2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:37:10.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.032+0000 7f657a9deb80 1 -- 172.21.15.19:0/3075969530 shutdown_connections 2024-05-07T02:37:10.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.032+0000 7f657a9deb80 1 -- 172.21.15.19:0/3075969530 wait complete. 2024-05-07T02:37:10.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.033+0000 7f657a9deb80 1 -- start start 2024-05-07T02:37:10.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.033+0000 7f657a9deb80 1 Processor -- start 2024-05-07T02:37:10.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.034+0000 7f657a9deb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d79a2df470 0x55d79a2e9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:10.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.034+0000 7f657a9deb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d79a2dfdc0 0x55d79a2e9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:10.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.034+0000 7f657a9deb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d79a2e53b0 0x55d79a2ea010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:10.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.034+0000 7f657a9deb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d79a2ec760 con 0x55d79a2e53b0 2024-05-07T02:37:10.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.034+0000 7f657a9deb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d79a167b20 con 0x55d79a2dfdc0 2024-05-07T02:37:10.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.034+0000 7f657a9deb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d79a2ec8e0 con 0x55d79a2df470 2024-05-07T02:37:10.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.034+0000 7f657a9deb80 10 client.0 _ll_register_callbacks cb 0x55d79a1a8c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:37:10.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.034+0000 7f6568a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d79a2dfdc0 0x55d79a2e9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:10.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.034+0000 7f6568a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d79a2dfdc0 0x55d79a2e9ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:55526/0 (socket says 172.21.15.19:55526) 2024-05-07T02:37:10.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.034+0000 7f6568a00700 1 -- 172.21.15.19:0/1869599293 learned_addr learned my addr 172.21.15.19:0/1869599293 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:37:10.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.034+0000 7f6569400700 1 --2- 172.21.15.19:0/1869599293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d79a2df470 0x55d79a2e9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:10.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.034+0000 7f6568a00700 1 -- 172.21.15.19:0/1869599293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d79a2df470 msgr2=0x55d79a2e9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:10.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.034+0000 7f6568a00700 1 --2- 172.21.15.19:0/1869599293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d79a2df470 0x55d79a2e9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:10.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.034+0000 7f6568a00700 1 -- 172.21.15.19:0/1869599293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d79a2e53b0 msgr2=0x55d79a2ea010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:10.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.034+0000 7f6568a00700 1 --2- 172.21.15.19:0/1869599293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d79a2e53b0 0x55d79a2ea010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:10.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.034+0000 7f6568a00700 1 -- 172.21.15.19:0/1869599293 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d79a1c7420 con 0x55d79a2dfdc0 2024-05-07T02:37:10.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.035+0000 7f6568a00700 1 --2- 172.21.15.19:0/1869599293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d79a2dfdc0 0x55d79a2e9ad0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6550000c00 tx=0x7f65500056c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a593fb81967caef7 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:37:10.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.035+0000 7f654b400700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:37:10.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.035+0000 7f654b400700 1 -- 172.21.15.19:0/1869599293 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6550027060 con 0x55d79a2dfdc0 2024-05-07T02:37:10.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.035+0000 7f657a9deb80 1 -- 172.21.15.19:0/1869599293 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6558091040 con 0x55d79a2dfdc0 2024-05-07T02:37:10.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.035+0000 7f654b400700 1 -- 172.21.15.19:0/1869599293 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f6550003680 con 0x55d79a2dfdc0 2024-05-07T02:37:10.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.035+0000 7f654b400700 1 -- 172.21.15.19:0/1869599293 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f655002e050 con 0x55d79a2dfdc0 2024-05-07T02:37:10.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.036+0000 7f657a9deb80 10 client.26084 fetch_fsmap learned FSMap version 40 2024-05-07T02:37:10.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.036+0000 7f657a9deb80 1 -- 172.21.15.19:0/1869599293 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d79a2f7460 con 0x55d79a2dfdc0 2024-05-07T02:37:10.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.036+0000 7f654b400700 1 -- 172.21.15.19:0/1869599293 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6558091040 con 0x55d79a2dfdc0 2024-05-07T02:37:10.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.036+0000 7f654b400700 1 -- 172.21.15.19:0/1869599293 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f6550021030 con 0x55d79a2dfdc0 2024-05-07T02:37:10.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.036+0000 7f657a9deb80 10 client.26084 fetch_fsmap finished waiting for FSMap version 40 2024-05-07T02:37:10.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.036+0000 7f657a9deb80 10 client.26084 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-07T02:37:10.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.036+0000 7f657a9deb80 20 client.26084 populate_metadata read hostname 'smithi019' 2024-05-07T02:37:10.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.036+0000 7f657a9deb80 1 --2- 172.21.15.19:0/1869599293 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55d79a2f9a40 0x55d79a2fbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:10.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.036+0000 7f657a9deb80 4 client.26084 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:37:10.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.036+0000 7f657a9deb80 1 -- 172.21.15.19:0/1869599293 --> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d79a2ac2e0 con 0x55d79a2f9a40 2024-05-07T02:37:10.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.037+0000 7f6569400700 1 --2- 172.21.15.19:0/1869599293 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55d79a2f9a40 0x55d79a2fbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:10.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.037+0000 7f6569400700 1 --2- 172.21.15.19:0/1869599293 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55d79a2f9a40 0x55d79a2fbf10 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a5cac40caef1c85d server_cookie=77121d2e302cfd53 in_seq=0 out_seq=0 2024-05-07T02:37:10.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.037+0000 7f654b400700 10 client.26084 ms_handle_connect on v2:172.21.15.64:6838/1583120004 2024-05-07T02:37:10.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.038+0000 7f654b400700 1 -- 172.21.15.19:0/1869599293 <== mds.2 v2:172.21.15.64:6838/1583120004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d79a2ac2e0 con 0x55d79a2f9a40 2024-05-07T02:37:10.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.038+0000 7f654b400700 10 client.26084 handle_command_reply: tid=0 2024-05-07T02:37:10.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.038+0000 7f657a9deb80 1 client.26084 shutdown 2024-05-07T02:37:10.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.038+0000 7f657a9deb80 1 -- 172.21.15.19:0/1869599293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d79a2dfdc0 msgr2=0x55d79a2e9ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:10.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.038+0000 7f657a9deb80 1 --2- 172.21.15.19:0/1869599293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d79a2dfdc0 0x55d79a2e9ad0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6550000c00 tx=0x7f65500056c0 comp rx=0 tx=0).stop 2024-05-07T02:37:10.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.039+0000 7f657a9deb80 1 -- 172.21.15.19:0/1869599293 shutdown_connections 2024-05-07T02:37:10.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.039+0000 7f657a9deb80 1 --2- 172.21.15.19:0/1869599293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d79a2e53b0 0x55d79a2ea010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:10.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.039+0000 7f657a9deb80 1 --2- 172.21.15.19:0/1869599293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d79a2df470 0x55d79a2e9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:10.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.039+0000 7f657a9deb80 1 --2- 172.21.15.19:0/1869599293 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55d79a2f9a40 0x55d79a2fbf10 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:10.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.039+0000 7f657a9deb80 1 --2- 172.21.15.19:0/1869599293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d79a2dfdc0 0x55d79a2e9ad0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:10.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.039+0000 7f657a9deb80 1 -- 172.21.15.19:0/1869599293 >> 172.21.15.19:0/1869599293 conn(0x55d79a161e40 msgr2=0x55d79a2de460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:37:10.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.039+0000 7f654b400700 0 client.26084 ms_handle_reset on v2:172.21.15.64:6838/1583120004 2024-05-07T02:37:10.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.039+0000 7f657a9deb80 1 -- 172.21.15.19:0/1869599293 shutdown_connections 2024-05-07T02:37:10.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.039+0000 7f657a9deb80 1 -- 172.21.15.19:0/1869599293 wait complete. 2024-05-07T02:37:10.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.039+0000 7f657a9deb80 20 client.26084 trim_cache size 0 max 16384 2024-05-07T02:37:10.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.059+0000 7f657a9deb80 1 Processor -- start 2024-05-07T02:37:10.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.059+0000 7f657a9deb80 1 -- start start 2024-05-07T02:37:10.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.060+0000 7f657a9deb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d79a2df470 0x55d79a2e8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:10.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.060+0000 7f657a9deb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d79a2dfdc0 0x55d79a2d4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:10.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.060+0000 7f657a9deb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d79a2e53b0 0x55d79a2f5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:10.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.060+0000 7f657a9deb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f65641b2750 con 0x55d79a2dfdc0 2024-05-07T02:37:10.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.060+0000 7f657a9deb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f65641b42f0 con 0x55d79a2e53b0 2024-05-07T02:37:10.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.060+0000 7f657a9deb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f65641b2e20 con 0x55d79a2df470 2024-05-07T02:37:10.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.060+0000 7f6569e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d79a2e53b0 0x55d79a2f5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:10.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.060+0000 7f6568a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d79a2dfdc0 0x55d79a2d4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:10.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.060+0000 7f6569e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d79a2e53b0 0x55d79a2f5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:55532/0 (socket says 172.21.15.19:55532) 2024-05-07T02:37:10.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.060+0000 7f6569e00700 1 -- 172.21.15.19:0/1924264249 learned_addr learned my addr 172.21.15.19:0/1924264249 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:37:10.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.060+0000 7f6569400700 1 --2- 172.21.15.19:0/1924264249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d79a2df470 0x55d79a2e8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:10.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.061+0000 7f6569e00700 1 -- 172.21.15.19:0/1924264249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d79a2df470 msgr2=0x55d79a2e8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:10.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.061+0000 7f6569e00700 1 --2- 172.21.15.19:0/1924264249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d79a2df470 0x55d79a2e8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:10.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.061+0000 7f6569e00700 1 -- 172.21.15.19:0/1924264249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d79a2dfdc0 msgr2=0x55d79a2d4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:10.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.061+0000 7f6569e00700 1 --2- 172.21.15.19:0/1924264249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d79a2dfdc0 0x55d79a2d4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:10.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.061+0000 7f6569e00700 1 -- 172.21.15.19:0/1924264249 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d79a2da810 con 0x55d79a2e53b0 2024-05-07T02:37:10.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.061+0000 7f6569e00700 1 --2- 172.21.15.19:0/1924264249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d79a2e53b0 0x55d79a2f5820 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f655c0098f0 tx=0x7f655c01b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a28d636956d8464 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:37:10.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.061+0000 7f6556000700 1 -- 172.21.15.19:0/1924264249 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655c018c60 con 0x55d79a2e53b0 2024-05-07T02:37:10.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.062+0000 7f6556000700 1 -- 172.21.15.19:0/1924264249 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f655c018e00 con 0x55d79a2e53b0 2024-05-07T02:37:10.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.062+0000 7f6556000700 1 -- 172.21.15.19:0/1924264249 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655c0047d0 con 0x55d79a2e53b0 2024-05-07T02:37:10.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.063+0000 7f657a9deb80 1 -- 172.21.15.19:0/1924264249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d79a2e53b0 msgr2=0x55d79a2f5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:10.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.063+0000 7f657a9deb80 1 --2- 172.21.15.19:0/1924264249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d79a2e53b0 0x55d79a2f5820 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f655c0098f0 tx=0x7f655c01b000 comp rx=0 tx=0).stop 2024-05-07T02:37:10.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.063+0000 7f657a9deb80 1 -- 172.21.15.19:0/1924264249 shutdown_connections 2024-05-07T02:37:10.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.063+0000 7f657a9deb80 1 --2- 172.21.15.19:0/1924264249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d79a2e53b0 0x55d79a2f5820 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:10.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.063+0000 7f657a9deb80 1 --2- 172.21.15.19:0/1924264249 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d79a2df470 0x55d79a2e8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:10.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.063+0000 7f657a9deb80 1 --2- 172.21.15.19:0/1924264249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d79a2dfdc0 0x55d79a2d4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:10.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.063+0000 7f657a9deb80 1 -- 172.21.15.19:0/1924264249 >> 172.21.15.19:0/1924264249 conn(0x55d79a161e40 msgr2=0x55d79a2d4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:37:10.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.063+0000 7f657a9deb80 1 -- 172.21.15.19:0/1924264249 shutdown_connections 2024-05-07T02:37:10.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.063+0000 7f657a9deb80 1 -- 172.21.15.19:0/1924264249 wait complete. 2024-05-07T02:37:10.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.063+0000 7f657a9deb80 1 -- start start 2024-05-07T02:37:10.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.064+0000 7f657a9deb80 1 Processor -- start 2024-05-07T02:37:10.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.064+0000 7f657a9deb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d79a2df470 0x55d79a2ecf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:10.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.064+0000 7f657a9deb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d79a2dfdc0 0x55d79a2ed4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:10.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.064+0000 7f657a9deb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d79a2e53b0 0x55d79a35c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:10.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.064+0000 7f657a9deb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f65641b3c20 con 0x55d79a2df470 2024-05-07T02:37:10.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.064+0000 7f657a9deb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f65641b3550 con 0x55d79a2dfdc0 2024-05-07T02:37:10.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.064+0000 7f657a9deb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f65641b06f0 con 0x55d79a2e53b0 2024-05-07T02:37:10.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.064+0000 7f657a9deb80 10 client.0 _ll_register_callbacks cb 0x55d79a1a8c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:37:10.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.065+0000 7f6569400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d79a2df470 0x55d79a2ecf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:10.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.065+0000 7f6568a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d79a2dfdc0 0x55d79a2ed4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:10.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.065+0000 7f6568a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d79a2dfdc0 0x55d79a2ed4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:55546/0 (socket says 172.21.15.19:55546) 2024-05-07T02:37:10.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.065+0000 7f6568a00700 1 -- 172.21.15.19:0/3571394453 learned_addr learned my addr 172.21.15.19:0/3571394453 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:37:10.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.065+0000 7f6569400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d79a2df470 0x55d79a2ecf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42290/0 (socket says 172.21.15.19:42290) 2024-05-07T02:37:10.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.065+0000 7f6569e00700 1 --2- 172.21.15.19:0/3571394453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d79a2e53b0 0x55d79a35c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:10.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.065+0000 7f6568a00700 1 -- 172.21.15.19:0/3571394453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d79a2e53b0 msgr2=0x55d79a35c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:10.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.065+0000 7f6568a00700 1 --2- 172.21.15.19:0/3571394453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d79a2e53b0 0x55d79a35c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:10.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.065+0000 7f6568a00700 1 -- 172.21.15.19:0/3571394453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d79a2df470 msgr2=0x55d79a2ecf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:10.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.065+0000 7f6568a00700 1 --2- 172.21.15.19:0/3571394453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d79a2df470 0x55d79a2ecf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:10.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.065+0000 7f6568a00700 1 -- 172.21.15.19:0/3571394453 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d79a35c800 con 0x55d79a2dfdc0 2024-05-07T02:37:10.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.065+0000 7f6569400700 1 --2- 172.21.15.19:0/3571394453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d79a2df470 0x55d79a2ecf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:37:10.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.066+0000 7f6568a00700 1 --2- 172.21.15.19:0/3571394453 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d79a2dfdc0 0x55d79a2ed4c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6550009ce0 tx=0x7f655000bfc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1ad78128099a4e server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:37:10.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.066+0000 7f654b400700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:37:10.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.066+0000 7f654b400700 1 -- 172.21.15.19:0/3571394453 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6550036020 con 0x55d79a2dfdc0 2024-05-07T02:37:10.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.066+0000 7f657a9deb80 1 -- 172.21.15.19:0/3571394453 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f655c018e00 con 0x55d79a2dfdc0 2024-05-07T02:37:10.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.066+0000 7f654b400700 1 -- 172.21.15.19:0/3571394453 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f655000fb60 con 0x55d79a2dfdc0 2024-05-07T02:37:10.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.066+0000 7f654b400700 1 -- 172.21.15.19:0/3571394453 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f6550027070 con 0x55d79a2dfdc0 2024-05-07T02:37:10.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.066+0000 7f657a9deb80 10 client.26090 fetch_fsmap learned FSMap version 40 2024-05-07T02:37:10.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.066+0000 7f657a9deb80 1 -- 172.21.15.19:0/3571394453 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d79a363b10 con 0x55d79a2dfdc0 2024-05-07T02:37:10.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.067+0000 7f654b400700 1 -- 172.21.15.19:0/3571394453 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f655c018e00 con 0x55d79a2dfdc0 2024-05-07T02:37:10.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.067+0000 7f654b400700 1 -- 172.21.15.19:0/3571394453 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f6550021070 con 0x55d79a2dfdc0 2024-05-07T02:37:10.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.067+0000 7f657a9deb80 10 client.26090 fetch_fsmap finished waiting for FSMap version 40 2024-05-07T02:37:10.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.067+0000 7f657a9deb80 10 client.26090 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-07T02:37:10.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.067+0000 7f657a9deb80 20 client.26090 populate_metadata read hostname 'smithi019' 2024-05-07T02:37:10.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.067+0000 7f657a9deb80 1 --2- 172.21.15.19:0/3571394453 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55d79a364240 0x55d79a366660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:10.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.067+0000 7f657a9deb80 4 client.26090 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:37:10.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.067+0000 7f657a9deb80 1 -- 172.21.15.19:0/3571394453 --> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d79a20f400 con 0x55d79a364240 2024-05-07T02:37:10.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.067+0000 7f6569400700 1 --2- 172.21.15.19:0/3571394453 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55d79a364240 0x55d79a366660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:10.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.068+0000 7f6569400700 1 --2- 172.21.15.19:0/3571394453 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55d79a364240 0x55d79a366660 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d64e1953698c78f3 server_cookie=43e1b90620eed9f6 in_seq=0 out_seq=0 2024-05-07T02:37:10.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.068+0000 7f654b400700 10 client.26090 ms_handle_connect on v2:172.21.15.64:6838/1583120004 2024-05-07T02:37:10.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.068+0000 7f654b400700 1 -- 172.21.15.19:0/3571394453 <== mds.2 v2:172.21.15.64:6838/1583120004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d79a20f400 con 0x55d79a364240 2024-05-07T02:37:10.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.068+0000 7f654b400700 10 client.26090 handle_command_reply: tid=0 2024-05-07T02:37:10.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.068+0000 7f657a9deb80 1 client.26090 shutdown 2024-05-07T02:37:10.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.068+0000 7f657a9deb80 1 -- 172.21.15.19:0/3571394453 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d79a2dfdc0 msgr2=0x55d79a2ed4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:10.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.068+0000 7f657a9deb80 1 --2- 172.21.15.19:0/3571394453 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d79a2dfdc0 0x55d79a2ed4c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6550009ce0 tx=0x7f655000bfc0 comp rx=0 tx=0).stop 2024-05-07T02:37:10.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.068+0000 7f657a9deb80 1 -- 172.21.15.19:0/3571394453 shutdown_connections 2024-05-07T02:37:10.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.068+0000 7f657a9deb80 1 --2- 172.21.15.19:0/3571394453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d79a2e53b0 0x55d79a35c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:10.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.068+0000 7f657a9deb80 1 --2- 172.21.15.19:0/3571394453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d79a2df470 0x55d79a2ecf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:10.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.068+0000 7f657a9deb80 1 --2- 172.21.15.19:0/3571394453 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55d79a364240 0x55d79a366660 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:10.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.068+0000 7f657a9deb80 1 --2- 172.21.15.19:0/3571394453 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d79a2dfdc0 0x55d79a2ed4c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:10.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.068+0000 7f654b400700 0 client.26090 ms_handle_reset on v2:172.21.15.64:6838/1583120004 2024-05-07T02:37:10.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.068+0000 7f657a9deb80 1 -- 172.21.15.19:0/3571394453 >> 172.21.15.19:0/3571394453 conn(0x55d79a161e40 msgr2=0x55d79a2de460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:37:10.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.069+0000 7f657a9deb80 1 -- 172.21.15.19:0/3571394453 shutdown_connections 2024-05-07T02:37:10.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.069+0000 7f657a9deb80 1 -- 172.21.15.19:0/3571394453 wait complete. 2024-05-07T02:37:10.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.069+0000 7f657a9deb80 20 client.26090 trim_cache size 0 max 16384 2024-05-07T02:37:10.072 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:37:10.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.069+0000 7f656be00700 1 -- 172.21.15.19:0/1237279280 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f654c0b2a70 msgr2=0x7f654c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:10.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.069+0000 7f656be00700 1 --2- 172.21.15.19:0/1237279280 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f654c0b2a70 0x7f654c0b4f80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f655c00f870 tx=0x7f655c00fc30 comp rx=0 tx=0).stop 2024-05-07T02:37:10.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.069+0000 7f656be00700 1 -- 172.21.15.19:0/1237279280 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6564105520 msgr2=0x7f656419d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:10.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.069+0000 7f656be00700 1 --2- 172.21.15.19:0/1237279280 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6564105520 0x7f656419d020 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f655800e9a0 tx=0x7f655800ecb0 comp rx=0 tx=0).stop 2024-05-07T02:37:10.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.070+0000 7f656be00700 1 -- 172.21.15.19:0/1237279280 shutdown_connections 2024-05-07T02:37:10.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.070+0000 7f656be00700 1 --2- 172.21.15.19:0/1237279280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f656410e900 0x7f65641a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:10.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.070+0000 7f656be00700 1 --2- 172.21.15.19:0/1237279280 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6564105520 0x7f656419d020 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:10.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.070+0000 7f656be00700 1 --2- 172.21.15.19:0/1237279280 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f654c0b2a70 0x7f654c0b4f80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:10.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.070+0000 7f656be00700 1 --2- 172.21.15.19:0/1237279280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6564105e70 0x7f656419d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:10.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.070+0000 7f656be00700 1 -- 172.21.15.19:0/1237279280 >> 172.21.15.19:0/1237279280 conn(0x7f65640fedc0 msgr2=0x7f6564103140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:37:10.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.070+0000 7f656be00700 1 -- 172.21.15.19:0/1237279280 shutdown_connections 2024-05-07T02:37:10.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:10.070+0000 7f656be00700 1 -- 172.21.15.19:0/1237279280 wait complete. 2024-05-07T02:37:10.752 INFO:tasks.workunit.client.0.smithi019.stdout: 1 165572 3.91 MB/sec execute 126 sec latency 29.307 ms 2024-05-07T02:37:11.102 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-07T02:37:11.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:11 smithi064 ceph-mon[34699]: pgmap v163: 129 pgs: 129 active+clean; 112 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 177 op/s 2024-05-07T02:37:11.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:11 smithi019 ceph-mon[29781]: pgmap v163: 129 pgs: 129 active+clean; 112 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 177 op/s 2024-05-07T02:37:11.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:11 smithi027 ceph-mon[34398]: pgmap v163: 129 pgs: 129 active+clean; 112 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 177 op/s 2024-05-07T02:37:11.583 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:37:11.752 INFO:tasks.workunit.client.0.smithi019.stdout: 1 166800 3.97 MB/sec execute 127 sec latency 57.306 ms 2024-05-07T02:37:12.752 INFO:tasks.workunit.client.0.smithi019.stdout: 1 167236 3.95 MB/sec execute 128 sec latency 31.495 ms 2024-05-07T02:37:13.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:13 smithi064 ceph-mon[34699]: pgmap v164: 129 pgs: 129 active+clean; 116 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 254 op/s 2024-05-07T02:37:13.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:13 smithi019 ceph-mon[29781]: pgmap v164: 129 pgs: 129 active+clean; 116 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 254 op/s 2024-05-07T02:37:13.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:13 smithi027 ceph-mon[34398]: pgmap v164: 129 pgs: 129 active+clean; 116 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 254 op/s 2024-05-07T02:37:13.752 INFO:tasks.workunit.client.0.smithi019.stdout: 1 168238 3.96 MB/sec execute 129 sec latency 27.502 ms 2024-05-07T02:37:14.752 INFO:tasks.workunit.client.0.smithi019.stdout: 1 169656 3.98 MB/sec execute 130 sec latency 24.529 ms 2024-05-07T02:37:14.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.782+0000 7f7a41e00700 1 -- 172.21.15.19:0/1923497646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a3c109870 msgr2=0x7f7a3c1111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:14.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.782+0000 7f7a41e00700 1 --2- 172.21.15.19:0/1923497646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a3c109870 0x7f7a3c1111d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7a3400b0d0 tx=0x7f7a3400b3e0 comp rx=0 tx=0).stop 2024-05-07T02:37:14.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.782+0000 7f7a41e00700 1 -- 172.21.15.19:0/1923497646 shutdown_connections 2024-05-07T02:37:14.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.782+0000 7f7a41e00700 1 --2- 172.21.15.19:0/1923497646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a3c109870 0x7f7a3c1111d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:14.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.782+0000 7f7a41e00700 1 --2- 172.21.15.19:0/1923497646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a3c1045f0 0x7f7a3c104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:14.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.782+0000 7f7a41e00700 1 --2- 172.21.15.19:0/1923497646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a3c108f20 0x7f7a3c109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:14.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.782+0000 7f7a41e00700 1 -- 172.21.15.19:0/1923497646 >> 172.21.15.19:0/1923497646 conn(0x7f7a3c100040 msgr2=0x7f7a3c102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:37:14.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.782+0000 7f7a41e00700 1 -- 172.21.15.19:0/1923497646 shutdown_connections 2024-05-07T02:37:14.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.782+0000 7f7a41e00700 1 -- 172.21.15.19:0/1923497646 wait complete. 2024-05-07T02:37:14.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.784+0000 7f7a41e00700 1 Processor -- start 2024-05-07T02:37:14.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.784+0000 7f7a41e00700 1 -- start start 2024-05-07T02:37:14.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.784+0000 7f7a41e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a3c1045f0 0x7f7a3c075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:14.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.784+0000 7f7a41e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a3c108f20 0x7f7a3c076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:14.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.784+0000 7f7a41e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a3c109870 0x7f7a3c076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:14.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.784+0000 7f7a41e00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3c1144f0 con 0x7f7a3c1045f0 2024-05-07T02:37:14.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.784+0000 7f7a41e00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3c114370 con 0x7f7a3c109870 2024-05-07T02:37:14.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.784+0000 7f7a41e00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3c114670 con 0x7f7a3c108f20 2024-05-07T02:37:14.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.785+0000 7f7a3aa00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a3c1045f0 0x7f7a3c075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:14.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.785+0000 7f7a3b400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a3c109870 0x7f7a3c076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:14.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.785+0000 7f7a32000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a3c108f20 0x7f7a3c076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:14.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.785+0000 7f7a3aa00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a3c1045f0 0x7f7a3c075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42310/0 (socket says 172.21.15.19:42310) 2024-05-07T02:37:14.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.785+0000 7f7a3aa00700 1 -- 172.21.15.19:0/2224808783 learned_addr learned my addr 172.21.15.19:0/2224808783 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:37:14.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.785+0000 7f7a32000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a3c108f20 0x7f7a3c076000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:58414/0 (socket says 172.21.15.19:58414) 2024-05-07T02:37:14.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.785+0000 7f7a3b400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a3c109870 0x7f7a3c076540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:55568/0 (socket says 172.21.15.19:55568) 2024-05-07T02:37:14.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.785+0000 7f7a3b400700 1 -- 172.21.15.19:0/2224808783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a3c108f20 msgr2=0x7f7a3c076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:14.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.785+0000 7f7a3b400700 1 --2- 172.21.15.19:0/2224808783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a3c108f20 0x7f7a3c076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:14.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.785+0000 7f7a3b400700 1 -- 172.21.15.19:0/2224808783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a3c1045f0 msgr2=0x7f7a3c075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:14.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.785+0000 7f7a3b400700 1 --2- 172.21.15.19:0/2224808783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a3c1045f0 0x7f7a3c075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:14.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.785+0000 7f7a3b400700 1 -- 172.21.15.19:0/2224808783 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a3c1a9240 con 0x7f7a3c109870 2024-05-07T02:37:14.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.786+0000 7f7a3b400700 1 --2- 172.21.15.19:0/2224808783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a3c109870 0x7f7a3c076540 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7a3400b770 tx=0x7f7a34007980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:37:14.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.786+0000 7f7a40a00700 1 -- 172.21.15.19:0/2224808783 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3401f070 con 0x7f7a3c109870 2024-05-07T02:37:14.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.786+0000 7f7a40a00700 1 -- 172.21.15.19:0/2224808783 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a34007dc0 con 0x7f7a3c109870 2024-05-07T02:37:14.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.786+0000 7f7a41e00700 1 -- 172.21.15.19:0/2224808783 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a3c1a94d0 con 0x7f7a3c109870 2024-05-07T02:37:14.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.786+0000 7f7a40a00700 1 -- 172.21.15.19:0/2224808783 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3401b430 con 0x7f7a3c109870 2024-05-07T02:37:14.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.786+0000 7f7a41e00700 1 -- 172.21.15.19:0/2224808783 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a3c1a9980 con 0x7f7a3c109870 2024-05-07T02:37:14.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.790+0000 7f7a40a00700 1 -- 172.21.15.19:0/2224808783 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7a34022070 con 0x7f7a3c109870 2024-05-07T02:37:14.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.791+0000 7f7a508d7b80 1 Processor -- start 2024-05-07T02:37:14.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.792+0000 7f7a508d7b80 1 -- start start 2024-05-07T02:37:14.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.792+0000 7f7a40a00700 1 --2- 172.21.15.19:0/2224808783 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f7a200b2a70 0x7f7a200b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:14.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.792+0000 7f7a508d7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2163c53b0 0x55e2163c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:14.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.792+0000 7f7a508d7b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2163bf470 0x55e2163bf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:14.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.792+0000 7f7a508d7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2163bfdc0 0x55e2163cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:14.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.792+0000 7f7a508d7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e2162e1a20 con 0x55e2163c53b0 2024-05-07T02:37:14.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.792+0000 7f7a508d7b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55e21627f760 con 0x55e2163bf470 2024-05-07T02:37:14.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.792+0000 7f7a508d7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e2162dd9e0 con 0x55e2163bfdc0 2024-05-07T02:37:14.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.792+0000 7f7a40a00700 1 -- 172.21.15.19:0/2224808783 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f7a3400f020 con 0x7f7a3c109870 2024-05-07T02:37:14.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.792+0000 7f7a3aa00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2163c53b0 0x55e2163c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:14.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.793+0000 7f7a32000700 1 --2- 172.21.15.19:0/2224808783 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f7a200b2a70 0x7f7a200b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:14.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.793+0000 7f7a3aa00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2163c53b0 0x55e2163c57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42320/0 (socket says 172.21.15.19:42320) 2024-05-07T02:37:14.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.793+0000 7f7a3aa00700 1 -- 172.21.15.19:0/427989957 learned_addr learned my addr 172.21.15.19:0/427989957 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:37:14.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.793+0000 7f7a3b400700 1 --2- 172.21.15.19:0/427989957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2163bfdc0 0x55e2163cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:14.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.793+0000 7f7a32000700 1 --2- 172.21.15.19:0/427989957 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2163bf470 0x55e2163bf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:14.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.793+0000 7f7a3b400700 1 -- 172.21.15.19:0/427989957 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2163bf470 msgr2=0x55e2163bf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:14.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.793+0000 7f7a3b400700 1 --2- 172.21.15.19:0/427989957 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2163bf470 0x55e2163bf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:14.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.793+0000 7f7a3b400700 1 -- 172.21.15.19:0/427989957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2163c53b0 msgr2=0x55e2163c57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:14.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.793+0000 7f7a3b400700 1 --2- 172.21.15.19:0/427989957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2163c53b0 0x55e2163c57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:14.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.793+0000 7f7a3b400700 1 -- 172.21.15.19:0/427989957 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e21622e8d0 con 0x55e2163bfdc0 2024-05-07T02:37:14.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.793+0000 7f7a32000700 1 --2- 172.21.15.19:0/427989957 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2163bf470 0x55e2163bf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:37:14.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.794+0000 7f7a3b400700 1 --2- 172.21.15.19:0/427989957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2163bfdc0 0x55e2163cc040 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7a340e0c70 tx=0x7f7a340aa660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8450d9181e785c34 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:37:14.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.794+0000 7f7a30c00700 1 -- 172.21.15.19:0/427989957 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a340aa000 con 0x55e2163bfdc0 2024-05-07T02:37:14.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.794+0000 7f7a30c00700 1 -- 172.21.15.19:0/427989957 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a340aa3a0 con 0x55e2163bfdc0 2024-05-07T02:37:14.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.794+0000 7f7a30c00700 1 -- 172.21.15.19:0/427989957 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a34010da0 con 0x55e2163bfdc0 2024-05-07T02:37:14.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.795+0000 7f7a508d7b80 1 -- 172.21.15.19:0/427989957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2163bfdc0 msgr2=0x55e2163cc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:14.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.795+0000 7f7a32000700 1 --2- 172.21.15.19:0/2224808783 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f7a200b2a70 0x7f7a200b4f80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x55e2163bba40 tx=0x7f7a24009850 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:37:14.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.795+0000 7f7a508d7b80 1 --2- 172.21.15.19:0/427989957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2163bfdc0 0x55e2163cc040 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7a340e0c70 tx=0x7f7a340aa660 comp rx=0 tx=0).stop 2024-05-07T02:37:14.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.795+0000 7f7a508d7b80 1 -- 172.21.15.19:0/427989957 shutdown_connections 2024-05-07T02:37:14.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.795+0000 7f7a508d7b80 1 --2- 172.21.15.19:0/427989957 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2163bfdc0 0x55e2163cc040 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:14.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.795+0000 7f7a508d7b80 1 --2- 172.21.15.19:0/427989957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2163c53b0 0x55e2163c57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:14.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.795+0000 7f7a508d7b80 1 --2- 172.21.15.19:0/427989957 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2163bf470 0x55e2163bf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:14.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.795+0000 7f7a508d7b80 1 -- 172.21.15.19:0/427989957 >> 172.21.15.19:0/427989957 conn(0x55e216241e40 msgr2=0x55e2163ba2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:37:14.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.795+0000 7f7a508d7b80 1 -- 172.21.15.19:0/427989957 shutdown_connections 2024-05-07T02:37:14.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.795+0000 7f7a508d7b80 1 -- 172.21.15.19:0/427989957 wait complete. 2024-05-07T02:37:14.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.796+0000 7f7a508d7b80 1 -- start start 2024-05-07T02:37:14.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.796+0000 7f7a508d7b80 1 Processor -- start 2024-05-07T02:37:14.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.797+0000 7f7a508d7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2163bf470 0x55e2163c9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:14.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.797+0000 7f7a508d7b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2163bfdc0 0x55e2163c9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:14.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.797+0000 7f7a508d7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2163c53b0 0x55e2163c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:14.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.797+0000 7f7a508d7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e2163cc580 con 0x55e2163bf470 2024-05-07T02:37:14.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.797+0000 7f7a508d7b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55e216247b20 con 0x55e2163bfdc0 2024-05-07T02:37:14.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.797+0000 7f7a508d7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e2163cc700 con 0x55e2163c53b0 2024-05-07T02:37:14.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.797+0000 7f7a508d7b80 10 client.0 _ll_register_callbacks cb 0x55e216288c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:37:14.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.797+0000 7f7a3b400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2163c53b0 0x55e2163c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:14.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.797+0000 7f7a32000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2163bfdc0 0x55e2163c9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:14.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.797+0000 7f7a32000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2163bfdc0 0x55e2163c9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:55588/0 (socket says 172.21.15.19:55588) 2024-05-07T02:37:14.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.797+0000 7f7a3b400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2163c53b0 0x55e2163c9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:58426/0 (socket says 172.21.15.19:58426) 2024-05-07T02:37:14.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.797+0000 7f7a32000700 1 -- 172.21.15.19:0/3435153714 learned_addr learned my addr 172.21.15.19:0/3435153714 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:37:14.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.798+0000 7f7a3aa00700 1 --2- 172.21.15.19:0/3435153714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2163bf470 0x55e2163c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:14.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.798+0000 7f7a32000700 1 -- 172.21.15.19:0/3435153714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2163c53b0 msgr2=0x55e2163c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:14.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.798+0000 7f7a32000700 1 --2- 172.21.15.19:0/3435153714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2163c53b0 0x55e2163c9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:14.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.798+0000 7f7a32000700 1 -- 172.21.15.19:0/3435153714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2163bf470 msgr2=0x55e2163c9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:14.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.798+0000 7f7a32000700 1 --2- 172.21.15.19:0/3435153714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2163bf470 0x55e2163c9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:14.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.798+0000 7f7a32000700 1 -- 172.21.15.19:0/3435153714 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2162a7420 con 0x55e2163bfdc0 2024-05-07T02:37:14.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.798+0000 7f7a3aa00700 1 --2- 172.21.15.19:0/3435153714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2163bf470 0x55e2163c9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:37:14.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.798+0000 7f7a32000700 1 --2- 172.21.15.19:0/3435153714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2163bfdc0 0x55e2163c9a90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7a24011470 tx=0x7f7a24009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4652d67bb7b480ee server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:37:14.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.798+0000 7f7a29600700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:37:14.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.799+0000 7f7a29600700 1 -- 172.21.15.19:0/3435153714 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a24025070 con 0x55e2163bfdc0 2024-05-07T02:37:14.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.799+0000 7f7a29600700 1 -- 172.21.15.19:0/3435153714 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f7a24004050 con 0x55e2163bfdc0 2024-05-07T02:37:14.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.799+0000 7f7a508d7b80 1 -- 172.21.15.19:0/3435153714 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a340aa3a0 con 0x55e2163bfdc0 2024-05-07T02:37:14.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.799+0000 7f7a29600700 1 -- 172.21.15.19:0/3435153714 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f7a24030050 con 0x55e2163bfdc0 2024-05-07T02:37:14.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.799+0000 7f7a508d7b80 10 client.26102 fetch_fsmap learned FSMap version 40 2024-05-07T02:37:14.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.799+0000 7f7a508d7b80 1 -- 172.21.15.19:0/3435153714 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55e2163d7150 con 0x55e2163bfdc0 2024-05-07T02:37:14.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.800+0000 7f7a29600700 1 -- 172.21.15.19:0/3435153714 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a340aa3a0 con 0x55e2163bfdc0 2024-05-07T02:37:14.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.800+0000 7f7a29600700 1 -- 172.21.15.19:0/3435153714 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f7a2401e070 con 0x55e2163bfdc0 2024-05-07T02:37:14.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.800+0000 7f7a508d7b80 10 client.26102 fetch_fsmap finished waiting for FSMap version 40 2024-05-07T02:37:14.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.800+0000 7f7a508d7b80 10 client.26102 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-07T02:37:14.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.800+0000 7f7a508d7b80 20 client.26102 populate_metadata read hostname 'smithi019' 2024-05-07T02:37:14.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.800+0000 7f7a508d7b80 1 --2- 172.21.15.19:0/3435153714 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x55e2163d96e0 0x55e2163dbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:14.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.800+0000 7f7a508d7b80 4 client.26102 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:37:14.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.800+0000 7f7a508d7b80 1 -- 172.21.15.19:0/3435153714 --> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e21638c2e0 con 0x55e2163d96e0 2024-05-07T02:37:14.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.800+0000 7f7a3aa00700 1 --2- 172.21.15.19:0/3435153714 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x55e2163d96e0 0x55e2163dbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:14.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.801+0000 7f7a3aa00700 1 --2- 172.21.15.19:0/3435153714 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x55e2163d96e0 0x55e2163dbb40 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2376f745a0ecb244 server_cookie=d6f94fe237f56bf9 in_seq=0 out_seq=0 2024-05-07T02:37:14.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.801+0000 7f7a29600700 10 client.26102 ms_handle_connect on v2:172.21.15.19:6840/3905645246 2024-05-07T02:37:14.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.802+0000 7f7a29600700 1 -- 172.21.15.19:0/3435153714 <== mds.1 v2:172.21.15.19:6840/3905645246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e21638c2e0 con 0x55e2163d96e0 2024-05-07T02:37:14.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.802+0000 7f7a29600700 10 client.26102 handle_command_reply: tid=0 2024-05-07T02:37:14.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.802+0000 7f7a508d7b80 1 client.26102 shutdown 2024-05-07T02:37:14.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.802+0000 7f7a508d7b80 1 -- 172.21.15.19:0/3435153714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2163bfdc0 msgr2=0x55e2163c9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:14.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.802+0000 7f7a508d7b80 1 --2- 172.21.15.19:0/3435153714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2163bfdc0 0x55e2163c9a90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7a24011470 tx=0x7f7a24009260 comp rx=0 tx=0).stop 2024-05-07T02:37:14.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.802+0000 7f7a508d7b80 1 -- 172.21.15.19:0/3435153714 shutdown_connections 2024-05-07T02:37:14.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.802+0000 7f7a508d7b80 1 --2- 172.21.15.19:0/3435153714 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x55e2163d96e0 0x55e2163dbb40 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:14.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.802+0000 7f7a508d7b80 1 --2- 172.21.15.19:0/3435153714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2163c53b0 0x55e2163c9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:14.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.802+0000 7f7a508d7b80 1 --2- 172.21.15.19:0/3435153714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2163bf470 0x55e2163c9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:14.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.802+0000 7f7a508d7b80 1 --2- 172.21.15.19:0/3435153714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2163bfdc0 0x55e2163c9a90 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:14.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.802+0000 7f7a508d7b80 1 -- 172.21.15.19:0/3435153714 >> 172.21.15.19:0/3435153714 conn(0x55e216241e40 msgr2=0x55e2163be460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:37:14.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.803+0000 7f7a29600700 0 client.26102 ms_handle_reset on v2:172.21.15.19:6840/3905645246 2024-05-07T02:37:14.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.803+0000 7f7a508d7b80 1 -- 172.21.15.19:0/3435153714 shutdown_connections 2024-05-07T02:37:14.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.803+0000 7f7a508d7b80 1 -- 172.21.15.19:0/3435153714 wait complete. 2024-05-07T02:37:14.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.803+0000 7f7a508d7b80 20 client.26102 trim_cache size 0 max 16384 2024-05-07T02:37:14.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.822+0000 7f7a508d7b80 1 Processor -- start 2024-05-07T02:37:14.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.822+0000 7f7a508d7b80 1 -- start start 2024-05-07T02:37:14.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.822+0000 7f7a508d7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2163bf470 0x55e2163ba4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:14.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.822+0000 7f7a508d7b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2163bfdc0 0x55e2163c6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:14.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.822+0000 7f7a508d7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2163c53b0 0x55e2163d55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:14.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.822+0000 7f7a508d7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3c1b2760 con 0x55e2163bf470 2024-05-07T02:37:14.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.822+0000 7f7a508d7b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3c1b4300 con 0x55e2163bfdc0 2024-05-07T02:37:14.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.822+0000 7f7a508d7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3c1b2e30 con 0x55e2163c53b0 2024-05-07T02:37:14.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.823+0000 7f7a32000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2163bfdc0 0x55e2163c6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:14.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.823+0000 7f7a3b400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2163c53b0 0x55e2163d55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:14.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.823+0000 7f7a32000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2163bfdc0 0x55e2163c6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:55596/0 (socket says 172.21.15.19:55596) 2024-05-07T02:37:14.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.823+0000 7f7a32000700 1 -- 172.21.15.19:0/1121749230 learned_addr learned my addr 172.21.15.19:0/1121749230 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:37:14.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.823+0000 7f7a3aa00700 1 --2- 172.21.15.19:0/1121749230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2163bf470 0x55e2163ba4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:14.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.823+0000 7f7a32000700 1 -- 172.21.15.19:0/1121749230 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2163c53b0 msgr2=0x55e2163d55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:14.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.823+0000 7f7a32000700 1 --2- 172.21.15.19:0/1121749230 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2163c53b0 0x55e2163d55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:14.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.823+0000 7f7a32000700 1 -- 172.21.15.19:0/1121749230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2163bf470 msgr2=0x55e2163ba4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:14.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.823+0000 7f7a32000700 1 --2- 172.21.15.19:0/1121749230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2163bf470 0x55e2163ba4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:14.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.823+0000 7f7a32000700 1 -- 172.21.15.19:0/1121749230 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2163bbaa0 con 0x55e2163bfdc0 2024-05-07T02:37:14.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.823+0000 7f7a32000700 1 --2- 172.21.15.19:0/1121749230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2163bfdc0 0x55e2163c6f10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7a24005240 tx=0x7f7a24009cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6bc37cb82f90223 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:37:14.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.824+0000 7f7a2be00700 1 -- 172.21.15.19:0/1121749230 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2402d070 con 0x55e2163bfdc0 2024-05-07T02:37:14.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.824+0000 7f7a2be00700 1 -- 172.21.15.19:0/1121749230 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a240044f0 con 0x55e2163bfdc0 2024-05-07T02:37:14.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.824+0000 7f7a2be00700 1 -- 172.21.15.19:0/1121749230 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a24025070 con 0x55e2163bfdc0 2024-05-07T02:37:14.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.825+0000 7f7a508d7b80 1 -- 172.21.15.19:0/1121749230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2163bfdc0 msgr2=0x55e2163c6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:14.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.825+0000 7f7a508d7b80 1 --2- 172.21.15.19:0/1121749230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2163bfdc0 0x55e2163c6f10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7a24005240 tx=0x7f7a24009cf0 comp rx=0 tx=0).stop 2024-05-07T02:37:14.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.825+0000 7f7a508d7b80 1 -- 172.21.15.19:0/1121749230 shutdown_connections 2024-05-07T02:37:14.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.825+0000 7f7a508d7b80 1 --2- 172.21.15.19:0/1121749230 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2163c53b0 0x55e2163d55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:14.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.825+0000 7f7a508d7b80 1 --2- 172.21.15.19:0/1121749230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2163bf470 0x55e2163ba4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:14.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.825+0000 7f7a508d7b80 1 --2- 172.21.15.19:0/1121749230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2163bfdc0 0x55e2163c6f10 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:14.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.825+0000 7f7a508d7b80 1 -- 172.21.15.19:0/1121749230 >> 172.21.15.19:0/1121749230 conn(0x55e216241e40 msgr2=0x55e2163b9d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:37:14.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.826+0000 7f7a508d7b80 1 -- 172.21.15.19:0/1121749230 shutdown_connections 2024-05-07T02:37:14.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.826+0000 7f7a508d7b80 1 -- 172.21.15.19:0/1121749230 wait complete. 2024-05-07T02:37:14.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.826+0000 7f7a508d7b80 1 -- start start 2024-05-07T02:37:14.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.826+0000 7f7a508d7b80 1 Processor -- start 2024-05-07T02:37:14.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.827+0000 7f7a508d7b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2163bf470 0x55e2163d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:14.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.827+0000 7f7a508d7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2163bfdc0 0x55e2163d1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:14.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.827+0000 7f7a508d7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2163c53b0 0x55e2163e02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:14.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.827+0000 7f7a508d7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3c1b3c30 con 0x55e2163bfdc0 2024-05-07T02:37:14.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.827+0000 7f7a508d7b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3c1b3560 con 0x55e2163bf470 2024-05-07T02:37:14.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.827+0000 7f7a508d7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3c1b0700 con 0x55e2163c53b0 2024-05-07T02:37:14.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.827+0000 7f7a508d7b80 10 client.0 _ll_register_callbacks cb 0x55e216288c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:37:14.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.827+0000 7f7a3aa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2163bf470 0x55e2163d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:14.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.827+0000 7f7a3aa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2163bf470 0x55e2163d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:55610/0 (socket says 172.21.15.19:55610) 2024-05-07T02:37:14.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.827+0000 7f7a3aa00700 1 -- 172.21.15.19:0/1814558817 learned_addr learned my addr 172.21.15.19:0/1814558817 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:37:14.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.827+0000 7f7a32000700 1 --2- 172.21.15.19:0/1814558817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2163bfdc0 0x55e2163d1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:14.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.827+0000 7f7a3b400700 1 --2- 172.21.15.19:0/1814558817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2163c53b0 0x55e2163e02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:14.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.828+0000 7f7a3aa00700 1 -- 172.21.15.19:0/1814558817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2163c53b0 msgr2=0x55e2163e02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:14.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.828+0000 7f7a3aa00700 1 --2- 172.21.15.19:0/1814558817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2163c53b0 0x55e2163e02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:14.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.828+0000 7f7a3aa00700 1 -- 172.21.15.19:0/1814558817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2163bfdc0 msgr2=0x55e2163d1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:14.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.828+0000 7f7a3aa00700 1 --2- 172.21.15.19:0/1814558817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2163bfdc0 0x55e2163d1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:14.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.828+0000 7f7a3aa00700 1 -- 172.21.15.19:0/1814558817 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2163e0930 con 0x55e2163bf470 2024-05-07T02:37:14.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.828+0000 7f7a3aa00700 1 --2- 172.21.15.19:0/1814558817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2163bf470 0x55e2163d1950 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x55e2163c8dd0 tx=0x7f7a2c010b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d9d7b3b6596dacb server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:37:14.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.828+0000 7f7a29600700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:37:14.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.828+0000 7f7a29600700 1 -- 172.21.15.19:0/1814558817 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c004220 con 0x55e2163bf470 2024-05-07T02:37:14.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.829+0000 7f7a508d7b80 1 -- 172.21.15.19:0/1814558817 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a240044f0 con 0x55e2163bf470 2024-05-07T02:37:14.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.829+0000 7f7a29600700 1 -- 172.21.15.19:0/1814558817 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f7a2c0043c0 con 0x55e2163bf470 2024-05-07T02:37:14.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.829+0000 7f7a29600700 1 -- 172.21.15.19:0/1814558817 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f7a2c02e050 con 0x55e2163bf470 2024-05-07T02:37:14.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.829+0000 7f7a508d7b80 10 client.26108 fetch_fsmap learned FSMap version 40 2024-05-07T02:37:14.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.829+0000 7f7a508d7b80 1 -- 172.21.15.19:0/1814558817 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55e2164437b0 con 0x55e2163bf470 2024-05-07T02:37:14.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.829+0000 7f7a29600700 1 -- 172.21.15.19:0/1814558817 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a240044f0 con 0x55e2163bf470 2024-05-07T02:37:14.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.829+0000 7f7a29600700 1 -- 172.21.15.19:0/1814558817 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f7a2c003a40 con 0x55e2163bf470 2024-05-07T02:37:14.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.829+0000 7f7a508d7b80 10 client.26108 fetch_fsmap finished waiting for FSMap version 40 2024-05-07T02:37:14.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.829+0000 7f7a508d7b80 10 client.26108 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-07T02:37:14.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.829+0000 7f7a508d7b80 20 client.26108 populate_metadata read hostname 'smithi019' 2024-05-07T02:37:14.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.829+0000 7f7a508d7b80 1 --2- 172.21.15.19:0/1814558817 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x55e216443ef0 0x55e216446310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:14.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.829+0000 7f7a508d7b80 4 client.26108 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:37:14.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.830+0000 7f7a508d7b80 1 -- 172.21.15.19:0/1814558817 --> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e2163c4c10 con 0x55e216443ef0 2024-05-07T02:37:14.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.830+0000 7f7a32000700 1 --2- 172.21.15.19:0/1814558817 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x55e216443ef0 0x55e216446310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:14.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.831+0000 7f7a32000700 1 --2- 172.21.15.19:0/1814558817 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x55e216443ef0 0x55e216446310 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ffcafeab51033897 server_cookie=1e908225485f3058 in_seq=0 out_seq=0 2024-05-07T02:37:14.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.831+0000 7f7a29600700 10 client.26108 ms_handle_connect on v2:172.21.15.19:6840/3905645246 2024-05-07T02:37:14.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.831+0000 7f7a29600700 1 -- 172.21.15.19:0/1814558817 <== mds.1 v2:172.21.15.19:6840/3905645246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e2163c4c10 con 0x55e216443ef0 2024-05-07T02:37:14.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.831+0000 7f7a29600700 10 client.26108 handle_command_reply: tid=0 2024-05-07T02:37:14.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.831+0000 7f7a508d7b80 1 client.26108 shutdown 2024-05-07T02:37:14.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.831+0000 7f7a508d7b80 1 -- 172.21.15.19:0/1814558817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2163bf470 msgr2=0x55e2163d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:14.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.831+0000 7f7a508d7b80 1 --2- 172.21.15.19:0/1814558817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2163bf470 0x55e2163d1950 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x55e2163c8dd0 tx=0x7f7a2c010b20 comp rx=0 tx=0).stop 2024-05-07T02:37:14.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.831+0000 7f7a508d7b80 1 -- 172.21.15.19:0/1814558817 shutdown_connections 2024-05-07T02:37:14.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.831+0000 7f7a508d7b80 1 --2- 172.21.15.19:0/1814558817 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x55e216443ef0 0x55e216446310 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:14.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.831+0000 7f7a508d7b80 1 --2- 172.21.15.19:0/1814558817 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e2163c53b0 0x55e2163e02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:14.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.831+0000 7f7a508d7b80 1 --2- 172.21.15.19:0/1814558817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e2163bf470 0x55e2163d1950 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:14.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.831+0000 7f7a508d7b80 1 --2- 172.21.15.19:0/1814558817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2163bfdc0 0x55e2163d1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:14.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.831+0000 7f7a508d7b80 1 -- 172.21.15.19:0/1814558817 >> 172.21.15.19:0/1814558817 conn(0x55e216241e40 msgr2=0x55e2163b4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:37:14.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.831+0000 7f7a29600700 0 client.26108 ms_handle_reset on v2:172.21.15.19:6840/3905645246 2024-05-07T02:37:14.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.831+0000 7f7a508d7b80 1 -- 172.21.15.19:0/1814558817 shutdown_connections 2024-05-07T02:37:14.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.832+0000 7f7a508d7b80 1 -- 172.21.15.19:0/1814558817 wait complete. 2024-05-07T02:37:14.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.832+0000 7f7a508d7b80 20 client.26108 trim_cache size 0 max 16384 2024-05-07T02:37:14.835 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:37:14.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.833+0000 7f7a41e00700 1 -- 172.21.15.19:0/2224808783 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f7a200b2a70 msgr2=0x7f7a200b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:14.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.833+0000 7f7a41e00700 1 --2- 172.21.15.19:0/2224808783 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f7a200b2a70 0x7f7a200b4f80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x55e2163bba40 tx=0x7f7a24009850 comp rx=0 tx=0).stop 2024-05-07T02:37:14.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.833+0000 7f7a41e00700 1 -- 172.21.15.19:0/2224808783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a3c109870 msgr2=0x7f7a3c076540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:14.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.833+0000 7f7a41e00700 1 --2- 172.21.15.19:0/2224808783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a3c109870 0x7f7a3c076540 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7a3400b770 tx=0x7f7a34007980 comp rx=0 tx=0).stop 2024-05-07T02:37:14.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.833+0000 7f7a41e00700 1 -- 172.21.15.19:0/2224808783 shutdown_connections 2024-05-07T02:37:14.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.833+0000 7f7a41e00700 1 --2- 172.21.15.19:0/2224808783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a3c109870 0x7f7a3c076540 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:14.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.833+0000 7f7a41e00700 1 --2- 172.21.15.19:0/2224808783 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f7a200b2a70 0x7f7a200b4f80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:14.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.833+0000 7f7a41e00700 1 --2- 172.21.15.19:0/2224808783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a3c1045f0 0x7f7a3c075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:14.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.833+0000 7f7a41e00700 1 --2- 172.21.15.19:0/2224808783 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a3c108f20 0x7f7a3c076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:14.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.833+0000 7f7a41e00700 1 -- 172.21.15.19:0/2224808783 >> 172.21.15.19:0/2224808783 conn(0x7f7a3c100040 msgr2=0x7f7a3c108660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:37:14.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.833+0000 7f7a41e00700 1 -- 172.21.15.19:0/2224808783 shutdown_connections 2024-05-07T02:37:14.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:14.833+0000 7f7a41e00700 1 -- 172.21.15.19:0/2224808783 wait complete. 2024-05-07T02:37:15.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:15 smithi019 ceph-mon[29781]: pgmap v165: 129 pgs: 129 active+clean; 116 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 161 op/s 2024-05-07T02:37:15.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:15 smithi027 ceph-mon[34398]: pgmap v165: 129 pgs: 129 active+clean; 116 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 161 op/s 2024-05-07T02:37:15.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:15 smithi064 ceph-mon[34699]: pgmap v165: 129 pgs: 129 active+clean; 116 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 161 op/s 2024-05-07T02:37:15.747 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-07T02:37:15.752 INFO:tasks.workunit.client.0.smithi019.stdout: 1 170579 4.00 MB/sec execute 131 sec latency 22.970 ms 2024-05-07T02:37:16.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:16 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:37:16.239 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:37:16.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:16 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:37:16.510 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:16 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:37:16.755 INFO:tasks.workunit.client.0.smithi019.stdout: 1 171224 3.98 MB/sec execute 132 sec latency 18.773 ms 2024-05-07T02:37:17.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:17 smithi019 ceph-mon[29781]: pgmap v166: 129 pgs: 129 active+clean; 120 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 241 op/s 2024-05-07T02:37:17.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:17 smithi027 ceph-mon[34398]: pgmap v166: 129 pgs: 129 active+clean; 120 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 241 op/s 2024-05-07T02:37:17.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:17 smithi064 ceph-mon[34699]: pgmap v166: 129 pgs: 129 active+clean; 120 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 241 op/s 2024-05-07T02:37:17.754 INFO:tasks.workunit.client.0.smithi019.stdout: 1 172715 4.00 MB/sec execute 133 sec latency 38.222 ms 2024-05-07T02:37:18.255 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:18 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:37:18.255 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:18 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:37:18.255 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:18 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:37:18.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:18 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:37:18.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:18 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:37:18.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:18 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:37:18.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:18 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:37:18.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:18 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:37:18.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:18 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:37:18.755 INFO:tasks.workunit.client.0.smithi019.stdout: 1 173908 4.04 MB/sec execute 134 sec latency 19.336 ms 2024-05-07T02:37:19.448 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:19 smithi019 ceph-mon[29781]: pgmap v167: 129 pgs: 129 active+clean; 120 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-05-07T02:37:19.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:19 smithi027 ceph-mon[34398]: pgmap v167: 129 pgs: 129 active+clean; 120 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-05-07T02:37:19.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:19 smithi064 ceph-mon[34699]: pgmap v167: 129 pgs: 129 active+clean; 120 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-05-07T02:37:19.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.737+0000 7f554b200700 1 -- 172.21.15.19:0/2979563907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5544104400 msgr2=0x7f5544104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:19.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.737+0000 7f554b200700 1 --2- 172.21.15.19:0/2979563907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5544104400 0x7f5544104810 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f55380098f0 tx=0x7f5538009c00 comp rx=0 tx=0).stop 2024-05-07T02:37:19.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.737+0000 7f554b200700 1 -- 172.21.15.19:0/2979563907 shutdown_connections 2024-05-07T02:37:19.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.737+0000 7f554b200700 1 --2- 172.21.15.19:0/2979563907 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5544109680 0x7f5544110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:19.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.737+0000 7f554b200700 1 --2- 172.21.15.19:0/2979563907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5544104400 0x7f5544104810 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:19.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.737+0000 7f554b200700 1 --2- 172.21.15.19:0/2979563907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5544108d30 0x7f5544109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:19.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.737+0000 7f554b200700 1 -- 172.21.15.19:0/2979563907 >> 172.21.15.19:0/2979563907 conn(0x7f5544079d10 msgr2=0x7f554407a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:37:19.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.737+0000 7f554b200700 1 -- 172.21.15.19:0/2979563907 shutdown_connections 2024-05-07T02:37:19.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.738+0000 7f554b200700 1 -- 172.21.15.19:0/2979563907 wait complete. 2024-05-07T02:37:19.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.739+0000 7f554b200700 1 Processor -- start 2024-05-07T02:37:19.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.739+0000 7f554b200700 1 -- start start 2024-05-07T02:37:19.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.739+0000 7f554b200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5544104400 0x7f554419cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:19.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.739+0000 7f554b200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5544108d30 0x7f554419d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:19.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.740+0000 7f554b200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5544109680 0x7f55441a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:19.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.740+0000 7f554b200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f55441142b0 con 0x7f5544108d30 2024-05-07T02:37:19.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.740+0000 7f554b200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5544114130 con 0x7f5544104400 2024-05-07T02:37:19.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.740+0000 7f554b200700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5544114430 con 0x7f5544109680 2024-05-07T02:37:19.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.740+0000 7f5549e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5544104400 0x7f554419cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:19.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.740+0000 7f554a800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5544109680 0x7f55441a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:19.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.740+0000 7f5549e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5544104400 0x7f554419cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:45266/0 (socket says 172.21.15.19:45266) 2024-05-07T02:37:19.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.740+0000 7f5549e00700 1 -- 172.21.15.19:0/2944098101 learned_addr learned my addr 172.21.15.19:0/2944098101 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:37:19.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.740+0000 7f5549e00700 1 -- 172.21.15.19:0/2944098101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5544109680 msgr2=0x7f55441a17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:19.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.740+0000 7f5549e00700 1 --2- 172.21.15.19:0/2944098101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5544109680 0x7f55441a17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:19.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.740+0000 7f5549e00700 1 -- 172.21.15.19:0/2944098101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5544108d30 msgr2=0x7f554419d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:19.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.740+0000 7f5549e00700 1 --2- 172.21.15.19:0/2944098101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5544108d30 0x7f554419d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:19.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.740+0000 7f5549e00700 1 -- 172.21.15.19:0/2944098101 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55441a1e70 con 0x7f5544104400 2024-05-07T02:37:19.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.741+0000 7f5549e00700 1 --2- 172.21.15.19:0/2944098101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5544104400 0x7f554419cfa0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5538009230 tx=0x7f553800f9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:37:19.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.741+0000 7f553ea00700 1 -- 172.21.15.19:0/2944098101 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5538027070 con 0x7f5544104400 2024-05-07T02:37:19.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.741+0000 7f553ea00700 1 -- 172.21.15.19:0/2944098101 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55380055f0 con 0x7f5544104400 2024-05-07T02:37:19.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.741+0000 7f554b200700 1 -- 172.21.15.19:0/2944098101 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55441a2100 con 0x7f5544104400 2024-05-07T02:37:19.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.741+0000 7f553ea00700 1 -- 172.21.15.19:0/2944098101 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5538004930 con 0x7f5544104400 2024-05-07T02:37:19.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.741+0000 7f554b200700 1 -- 172.21.15.19:0/2944098101 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55441a25e0 con 0x7f5544104400 2024-05-07T02:37:19.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.745+0000 7f553ea00700 1 -- 172.21.15.19:0/2944098101 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f553802a070 con 0x7f5544104400 2024-05-07T02:37:19.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.746+0000 7f555b400b80 1 Processor -- start 2024-05-07T02:37:19.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.747+0000 7f553ea00700 1 --2- 172.21.15.19:0/2944098101 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f552c0b29f0 0x7f552c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:19.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.747+0000 7f555b400b80 1 -- start start 2024-05-07T02:37:19.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.747+0000 7f553ea00700 1 -- 172.21.15.19:0/2944098101 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f55380e3410 con 0x7f5544104400 2024-05-07T02:37:19.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.747+0000 7f555b400b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565448d70a90 0x565448d70ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:19.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.747+0000 7f555b400b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565448d713e0 0x565448d7a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:19.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.747+0000 7f555b400b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565448d7ab60 0x565448d7cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:19.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.747+0000 7f555b400b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x565448c92a20 con 0x565448d713e0 2024-05-07T02:37:19.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.747+0000 7f555b400b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x565448c30760 con 0x565448d70a90 2024-05-07T02:37:19.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.747+0000 7f555b400b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x565448c8e9e0 con 0x565448d7ab60 2024-05-07T02:37:19.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.747+0000 7f554a800700 1 --2- 172.21.15.19:0/2944098101 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f552c0b29f0 0x7f552c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:19.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.748+0000 7f5549e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565448d70a90 0x565448d70ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:19.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.748+0000 7f5549e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565448d70a90 0x565448d70ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:45276/0 (socket says 172.21.15.19:45276) 2024-05-07T02:37:19.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.748+0000 7f5549e00700 1 -- 172.21.15.19:0/2563171036 learned_addr learned my addr 172.21.15.19:0/2563171036 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:37:19.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.748+0000 7f5549400700 1 --2- 172.21.15.19:0/2563171036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565448d713e0 0x565448d7a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:19.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.748+0000 7f554a800700 1 --2- 172.21.15.19:0/2563171036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565448d7ab60 0x565448d7cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:19.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.749+0000 7f5549400700 1 -- 172.21.15.19:0/2563171036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565448d7ab60 msgr2=0x565448d7cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:19.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.749+0000 7f5549400700 1 --2- 172.21.15.19:0/2563171036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565448d7ab60 0x565448d7cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:19.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.749+0000 7f5549400700 1 -- 172.21.15.19:0/2563171036 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565448d70a90 msgr2=0x565448d70ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:19.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.749+0000 7f5549400700 1 --2- 172.21.15.19:0/2563171036 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565448d70a90 0x565448d70ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:19.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.749+0000 7f5549400700 1 -- 172.21.15.19:0/2563171036 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565448bdf8d0 con 0x565448d713e0 2024-05-07T02:37:19.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.749+0000 7f554a800700 1 --2- 172.21.15.19:0/2944098101 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f552c0b29f0 0x7f552c0b4f00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5540013de0 tx=0x7f5540013460 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:37:19.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.749+0000 7f5549400700 1 --2- 172.21.15.19:0/2563171036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565448d713e0 0x565448d7a620 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f554406c3c0 tx=0x7f5534009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72960b14670c6f35 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:37:19.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.750+0000 7f5533400700 1 -- 172.21.15.19:0/2563171036 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5534004230 con 0x565448d713e0 2024-05-07T02:37:19.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.750+0000 7f5533400700 1 -- 172.21.15.19:0/2563171036 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5534004ce0 con 0x565448d713e0 2024-05-07T02:37:19.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.750+0000 7f5533400700 1 -- 172.21.15.19:0/2563171036 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55340056a0 con 0x565448d713e0 2024-05-07T02:37:19.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.751+0000 7f555b400b80 1 -- 172.21.15.19:0/2563171036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565448d713e0 msgr2=0x565448d7a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:19.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.751+0000 7f555b400b80 1 --2- 172.21.15.19:0/2563171036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565448d713e0 0x565448d7a620 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f554406c3c0 tx=0x7f5534009e60 comp rx=0 tx=0).stop 2024-05-07T02:37:19.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.751+0000 7f555b400b80 1 -- 172.21.15.19:0/2563171036 shutdown_connections 2024-05-07T02:37:19.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.751+0000 7f555b400b80 1 --2- 172.21.15.19:0/2563171036 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565448d7ab60 0x565448d7cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:19.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.751+0000 7f555b400b80 1 --2- 172.21.15.19:0/2563171036 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565448d70a90 0x565448d70ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:19.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.751+0000 7f555b400b80 1 --2- 172.21.15.19:0/2563171036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565448d713e0 0x565448d7a620 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:19.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.751+0000 7f555b400b80 1 -- 172.21.15.19:0/2563171036 >> 172.21.15.19:0/2563171036 conn(0x565448bf2e40 msgr2=0x565448d6b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:37:19.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.751+0000 7f555b400b80 1 -- 172.21.15.19:0/2563171036 shutdown_connections 2024-05-07T02:37:19.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.752+0000 7f555b400b80 1 -- 172.21.15.19:0/2563171036 wait complete. 2024-05-07T02:37:19.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.753+0000 7f555b400b80 1 -- start start 2024-05-07T02:37:19.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.753+0000 7f555b400b80 1 Processor -- start 2024-05-07T02:37:19.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.754+0000 7f555b400b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565448d70a90 0x565448d77a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:19.754 INFO:tasks.workunit.client.0.smithi019.stdout: 1 174229 4.03 MB/sec execute 135 sec latency 52.513 ms 2024-05-07T02:37:19.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.754+0000 7f555b400b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565448d713e0 0x565448d77f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:19.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.754+0000 7f555b400b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565448d7ab60 0x565448d784d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:19.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.754+0000 7f555b400b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x565448d7d960 con 0x565448d7ab60 2024-05-07T02:37:19.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.754+0000 7f555b400b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x565448bf8b20 con 0x565448d70a90 2024-05-07T02:37:19.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.754+0000 7f555b400b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x565448d7dae0 con 0x565448d713e0 2024-05-07T02:37:19.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.754+0000 7f555b400b80 10 client.0 _ll_register_callbacks cb 0x565448c39c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:37:19.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.754+0000 7f5549400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565448d713e0 0x565448d77f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:19.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.754+0000 7f5549400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565448d713e0 0x565448d77f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:45464/0 (socket says 172.21.15.19:45464) 2024-05-07T02:37:19.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.754+0000 7f5549400700 1 -- 172.21.15.19:0/4121628005 learned_addr learned my addr 172.21.15.19:0/4121628005 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:37:19.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.754+0000 7f554a800700 1 --2- 172.21.15.19:0/4121628005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565448d7ab60 0x565448d784d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:19.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.754+0000 7f5549e00700 1 --2- 172.21.15.19:0/4121628005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565448d70a90 0x565448d77a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:19.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.754+0000 7f5549400700 1 -- 172.21.15.19:0/4121628005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565448d70a90 msgr2=0x565448d77a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:19.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.754+0000 7f5549400700 1 --2- 172.21.15.19:0/4121628005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565448d70a90 0x565448d77a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:19.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.754+0000 7f5549400700 1 -- 172.21.15.19:0/4121628005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565448d7ab60 msgr2=0x565448d784d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:19.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.755+0000 7f5549400700 1 --2- 172.21.15.19:0/4121628005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565448d7ab60 0x565448d784d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:19.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.755+0000 7f5549400700 1 -- 172.21.15.19:0/4121628005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565448c58420 con 0x565448d713e0 2024-05-07T02:37:19.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.755+0000 7f554a800700 1 --2- 172.21.15.19:0/4121628005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565448d7ab60 0x565448d784d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:37:19.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.755+0000 7f5549400700 1 --2- 172.21.15.19:0/4121628005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565448d713e0 0x565448d77f90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5534005590 tx=0x7f5534005670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea4b354136016a40 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:37:19.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.755+0000 7f5527e00700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:37:19.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.756+0000 7f5527e00700 1 -- 172.21.15.19:0/4121628005 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5534029060 con 0x565448d713e0 2024-05-07T02:37:19.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.756+0000 7f5527e00700 1 -- 172.21.15.19:0/4121628005 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f553400eb60 con 0x565448d713e0 2024-05-07T02:37:19.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.756+0000 7f555b400b80 1 -- 172.21.15.19:0/4121628005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5534004ce0 con 0x565448d713e0 2024-05-07T02:37:19.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.756+0000 7f5527e00700 1 -- 172.21.15.19:0/4121628005 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f553400ed60 con 0x565448d713e0 2024-05-07T02:37:19.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.756+0000 7f555b400b80 10 client.26092 fetch_fsmap learned FSMap version 40 2024-05-07T02:37:19.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.756+0000 7f555b400b80 1 -- 172.21.15.19:0/4121628005 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x565448d884e0 con 0x565448d713e0 2024-05-07T02:37:19.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.757+0000 7f5527e00700 1 -- 172.21.15.19:0/4121628005 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5534004ce0 con 0x565448d713e0 2024-05-07T02:37:19.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.757+0000 7f5527e00700 1 -- 172.21.15.19:0/4121628005 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f5534010430 con 0x565448d713e0 2024-05-07T02:37:19.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.757+0000 7f555b400b80 10 client.26092 fetch_fsmap finished waiting for FSMap version 40 2024-05-07T02:37:19.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.757+0000 7f555b400b80 10 client.26092 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:37:19.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.757+0000 7f555b400b80 20 client.26092 populate_metadata read hostname 'smithi019' 2024-05-07T02:37:19.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.757+0000 7f555b400b80 1 --2- 172.21.15.19:0/4121628005 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x565448d8aa60 0x565448d8ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:19.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.757+0000 7f555b400b80 4 client.26092 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:37:19.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.757+0000 7f555b400b80 1 -- 172.21.15.19:0/4121628005 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565448d3d2e0 con 0x565448d8aa60 2024-05-07T02:37:19.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.757+0000 7f5549e00700 1 --2- 172.21.15.19:0/4121628005 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x565448d8aa60 0x565448d8ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:19.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.758+0000 7f5549e00700 1 --2- 172.21.15.19:0/4121628005 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x565448d8aa60 0x565448d8ce80 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58b24a24df7c42a server_cookie=2ac47680656eb1d0 in_seq=0 out_seq=0 2024-05-07T02:37:19.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.758+0000 7f5527e00700 10 client.26092 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:37:19.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.759+0000 7f5527e00700 1 -- 172.21.15.19:0/4121628005 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x565448d3d2e0 con 0x565448d8aa60 2024-05-07T02:37:19.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.759+0000 7f5527e00700 10 client.26092 handle_command_reply: tid=0 2024-05-07T02:37:19.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.759+0000 7f555b400b80 1 client.26092 shutdown 2024-05-07T02:37:19.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.759+0000 7f555b400b80 1 -- 172.21.15.19:0/4121628005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565448d713e0 msgr2=0x565448d77f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:19.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.759+0000 7f555b400b80 1 --2- 172.21.15.19:0/4121628005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565448d713e0 0x565448d77f90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5534005590 tx=0x7f5534005670 comp rx=0 tx=0).stop 2024-05-07T02:37:19.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.759+0000 7f555b400b80 1 -- 172.21.15.19:0/4121628005 shutdown_connections 2024-05-07T02:37:19.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.759+0000 7f555b400b80 1 --2- 172.21.15.19:0/4121628005 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x565448d8aa60 0x565448d8ce80 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:19.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.759+0000 7f555b400b80 1 --2- 172.21.15.19:0/4121628005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565448d7ab60 0x565448d784d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:19.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.759+0000 7f555b400b80 1 --2- 172.21.15.19:0/4121628005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565448d70a90 0x565448d77a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:19.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.759+0000 7f555b400b80 1 --2- 172.21.15.19:0/4121628005 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565448d713e0 0x565448d77f90 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:19.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.759+0000 7f555b400b80 1 -- 172.21.15.19:0/4121628005 >> 172.21.15.19:0/4121628005 conn(0x565448bf2e40 msgr2=0x565448d6d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:37:19.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.759+0000 7f5527e00700 0 client.26092 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:37:19.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.760+0000 7f555b400b80 1 -- 172.21.15.19:0/4121628005 shutdown_connections 2024-05-07T02:37:19.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.760+0000 7f555b400b80 1 -- 172.21.15.19:0/4121628005 wait complete. 2024-05-07T02:37:19.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.760+0000 7f555b400b80 20 client.26092 trim_cache size 0 max 16384 2024-05-07T02:37:19.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.778+0000 7f555b400b80 1 Processor -- start 2024-05-07T02:37:19.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.778+0000 7f555b400b80 1 -- start start 2024-05-07T02:37:19.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.778+0000 7f555b400b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565448d70a90 0x565448d6b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:19.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.779+0000 7f555b400b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565448d713e0 0x565448d86db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:19.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.779+0000 7f555b400b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565448d7ab60 0x565448d6fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:19.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.779+0000 7f555b400b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f55441b26a0 con 0x565448d7ab60 2024-05-07T02:37:19.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.779+0000 7f555b400b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f55441b4240 con 0x565448d70a90 2024-05-07T02:37:19.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.779+0000 7f555b400b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f55441b2d70 con 0x565448d713e0 2024-05-07T02:37:19.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.779+0000 7f5549400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565448d713e0 0x565448d86db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:19.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.779+0000 7f5549e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565448d70a90 0x565448d6b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:19.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.779+0000 7f5549e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565448d70a90 0x565448d6b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:45298/0 (socket says 172.21.15.19:45298) 2024-05-07T02:37:19.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.779+0000 7f5549400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565448d713e0 0x565448d86db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:45468/0 (socket says 172.21.15.19:45468) 2024-05-07T02:37:19.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.779+0000 7f5549e00700 1 -- 172.21.15.19:0/2999733519 learned_addr learned my addr 172.21.15.19:0/2999733519 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:37:19.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.779+0000 7f554a800700 1 --2- 172.21.15.19:0/2999733519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565448d7ab60 0x565448d6fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:19.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.779+0000 7f5549400700 1 -- 172.21.15.19:0/2999733519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565448d70a90 msgr2=0x565448d6b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:19.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.779+0000 7f5549400700 1 --2- 172.21.15.19:0/2999733519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565448d70a90 0x565448d6b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:19.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.779+0000 7f5549400700 1 -- 172.21.15.19:0/2999733519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565448d7ab60 msgr2=0x565448d6fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:19.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.779+0000 7f5549400700 1 --2- 172.21.15.19:0/2999733519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565448d7ab60 0x565448d6fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:19.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.780+0000 7f5549400700 1 -- 172.21.15.19:0/2999733519 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565448d7dad0 con 0x565448d713e0 2024-05-07T02:37:19.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.780+0000 7f5549400700 1 --2- 172.21.15.19:0/2999733519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565448d713e0 0x565448d86db0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5534009b50 tx=0x7f55340092d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff13a7b0ca81e3f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:37:19.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.780+0000 7f5532a00700 1 -- 172.21.15.19:0/2999733519 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5534029070 con 0x565448d713e0 2024-05-07T02:37:19.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.781+0000 7f5532a00700 1 -- 172.21.15.19:0/2999733519 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5534010630 con 0x565448d713e0 2024-05-07T02:37:19.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.781+0000 7f5532a00700 1 -- 172.21.15.19:0/2999733519 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5534033080 con 0x565448d713e0 2024-05-07T02:37:19.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.782+0000 7f555b400b80 1 -- 172.21.15.19:0/2999733519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565448d713e0 msgr2=0x565448d86db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:19.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.782+0000 7f555b400b80 1 --2- 172.21.15.19:0/2999733519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565448d713e0 0x565448d86db0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5534009b50 tx=0x7f55340092d0 comp rx=0 tx=0).stop 2024-05-07T02:37:19.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.782+0000 7f555b400b80 1 -- 172.21.15.19:0/2999733519 shutdown_connections 2024-05-07T02:37:19.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.782+0000 7f555b400b80 1 --2- 172.21.15.19:0/2999733519 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565448d7ab60 0x565448d6fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:19.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.782+0000 7f555b400b80 1 --2- 172.21.15.19:0/2999733519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565448d70a90 0x565448d6b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:19.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.782+0000 7f555b400b80 1 --2- 172.21.15.19:0/2999733519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565448d713e0 0x565448d86db0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:19.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.782+0000 7f555b400b80 1 -- 172.21.15.19:0/2999733519 >> 172.21.15.19:0/2999733519 conn(0x565448bf2e40 msgr2=0x565448d6f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:37:19.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.782+0000 7f555b400b80 1 -- 172.21.15.19:0/2999733519 shutdown_connections 2024-05-07T02:37:19.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.782+0000 7f555b400b80 1 -- 172.21.15.19:0/2999733519 wait complete. 2024-05-07T02:37:19.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.782+0000 7f555b400b80 1 -- start start 2024-05-07T02:37:19.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.783+0000 7f555b400b80 1 Processor -- start 2024-05-07T02:37:19.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.783+0000 7f555b400b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565448d70a90 0x565448d8c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:19.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.783+0000 7f555b400b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565448d713e0 0x565448d8cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:19.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.783+0000 7f555b400b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565448d7ab60 0x565448d8d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:19.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.783+0000 7f555b400b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f55441b3b70 con 0x565448d70a90 2024-05-07T02:37:19.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.783+0000 7f555b400b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f55441b34a0 con 0x565448d713e0 2024-05-07T02:37:19.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.783+0000 7f555b400b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f55441b0640 con 0x565448d7ab60 2024-05-07T02:37:19.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.783+0000 7f555b400b80 10 client.0 _ll_register_callbacks cb 0x565448c39c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:37:19.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.784+0000 7f5549400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565448d713e0 0x565448d8cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:19.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.784+0000 7f5549400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565448d713e0 0x565448d8cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:45314/0 (socket says 172.21.15.19:45314) 2024-05-07T02:37:19.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.784+0000 7f5549400700 1 -- 172.21.15.19:0/4077655426 learned_addr learned my addr 172.21.15.19:0/4077655426 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:37:19.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.784+0000 7f5549e00700 1 --2- 172.21.15.19:0/4077655426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565448d70a90 0x565448d8c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:19.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.784+0000 7f5549400700 1 -- 172.21.15.19:0/4077655426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565448d7ab60 msgr2=0x565448d8d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:19.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.784+0000 7f5549400700 1 --2- 172.21.15.19:0/4077655426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565448d7ab60 0x565448d8d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:19.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.784+0000 7f5549400700 1 -- 172.21.15.19:0/4077655426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565448d70a90 msgr2=0x565448d8c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:19.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.784+0000 7f5549400700 1 --2- 172.21.15.19:0/4077655426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565448d70a90 0x565448d8c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:19.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.784+0000 7f5549400700 1 -- 172.21.15.19:0/4077655426 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565448df2fa0 con 0x565448d713e0 2024-05-07T02:37:19.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.784+0000 7f5549e00700 1 --2- 172.21.15.19:0/4077655426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565448d70a90 0x565448d8c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:37:19.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.785+0000 7f5549400700 1 --2- 172.21.15.19:0/4077655426 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565448d713e0 0x565448d8cbb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f55340056b0 tx=0x7f553400eb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e04d65257124e3e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:37:19.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.785+0000 7f5527e00700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:37:19.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.785+0000 7f5527e00700 1 -- 172.21.15.19:0/4077655426 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553401c030 con 0x565448d713e0 2024-05-07T02:37:19.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.785+0000 7f555b400b80 1 -- 172.21.15.19:0/4077655426 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5534010630 con 0x565448d713e0 2024-05-07T02:37:19.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.785+0000 7f5527e00700 1 -- 172.21.15.19:0/4077655426 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f553403bc40 con 0x565448d713e0 2024-05-07T02:37:19.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.785+0000 7f5527e00700 1 -- 172.21.15.19:0/4077655426 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f553404a050 con 0x565448d713e0 2024-05-07T02:37:19.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.785+0000 7f555b400b80 10 client.26126 fetch_fsmap learned FSMap version 40 2024-05-07T02:37:19.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.785+0000 7f555b400b80 1 -- 172.21.15.19:0/4077655426 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x565448df4190 con 0x565448d713e0 2024-05-07T02:37:19.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.786+0000 7f5527e00700 1 -- 172.21.15.19:0/4077655426 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5534010630 con 0x565448d713e0 2024-05-07T02:37:19.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.786+0000 7f5527e00700 1 -- 172.21.15.19:0/4077655426 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f5534004030 con 0x565448d713e0 2024-05-07T02:37:19.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.786+0000 7f555b400b80 10 client.26126 fetch_fsmap finished waiting for FSMap version 40 2024-05-07T02:37:19.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.786+0000 7f555b400b80 10 client.26126 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:37:19.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.786+0000 7f555b400b80 20 client.26126 populate_metadata read hostname 'smithi019' 2024-05-07T02:37:19.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.786+0000 7f555b400b80 1 --2- 172.21.15.19:0/4077655426 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x565448df4800 0x565448df6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:19.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.786+0000 7f555b400b80 4 client.26126 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:37:19.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.786+0000 7f555b400b80 1 -- 172.21.15.19:0/4077655426 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565448ca0400 con 0x565448df4800 2024-05-07T02:37:19.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.787+0000 7f5549e00700 1 --2- 172.21.15.19:0/4077655426 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x565448df4800 0x565448df6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:19.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.788+0000 7f5549e00700 1 --2- 172.21.15.19:0/4077655426 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x565448df4800 0x565448df6c20 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8477f7b22e7002e0 server_cookie=d40d3bb7116551e6 in_seq=0 out_seq=0 2024-05-07T02:37:19.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.788+0000 7f5527e00700 10 client.26126 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:37:19.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.788+0000 7f5527e00700 1 -- 172.21.15.19:0/4077655426 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565448ca0400 con 0x565448df4800 2024-05-07T02:37:19.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.788+0000 7f5527e00700 10 client.26126 handle_command_reply: tid=0 2024-05-07T02:37:19.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.788+0000 7f555b400b80 1 client.26126 shutdown 2024-05-07T02:37:19.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.788+0000 7f555b400b80 1 -- 172.21.15.19:0/4077655426 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565448d713e0 msgr2=0x565448d8cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:19.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.788+0000 7f555b400b80 1 --2- 172.21.15.19:0/4077655426 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565448d713e0 0x565448d8cbb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f55340056b0 tx=0x7f553400eb90 comp rx=0 tx=0).stop 2024-05-07T02:37:19.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.788+0000 7f555b400b80 1 -- 172.21.15.19:0/4077655426 shutdown_connections 2024-05-07T02:37:19.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.788+0000 7f555b400b80 1 --2- 172.21.15.19:0/4077655426 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x565448df4800 0x565448df6c20 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:19.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.788+0000 7f555b400b80 1 --2- 172.21.15.19:0/4077655426 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565448d7ab60 0x565448d8d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:19.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.788+0000 7f555b400b80 1 --2- 172.21.15.19:0/4077655426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565448d70a90 0x565448d8c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:19.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.788+0000 7f555b400b80 1 --2- 172.21.15.19:0/4077655426 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565448d713e0 0x565448d8cbb0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:19.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.788+0000 7f555b400b80 1 -- 172.21.15.19:0/4077655426 >> 172.21.15.19:0/4077655426 conn(0x565448bf2e40 msgr2=0x565448d723b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:37:19.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.788+0000 7f5527e00700 0 client.26126 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:37:19.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.788+0000 7f555b400b80 1 -- 172.21.15.19:0/4077655426 shutdown_connections 2024-05-07T02:37:19.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.789+0000 7f555b400b80 1 -- 172.21.15.19:0/4077655426 wait complete. 2024-05-07T02:37:19.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.789+0000 7f555b400b80 20 client.26126 trim_cache size 0 max 16384 2024-05-07T02:37:19.792 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:37:19.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.790+0000 7f554b200700 1 -- 172.21.15.19:0/2944098101 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f552c0b29f0 msgr2=0x7f552c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:19.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.790+0000 7f554b200700 1 --2- 172.21.15.19:0/2944098101 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f552c0b29f0 0x7f552c0b4f00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5540013de0 tx=0x7f5540013460 comp rx=0 tx=0).stop 2024-05-07T02:37:19.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.790+0000 7f554b200700 1 -- 172.21.15.19:0/2944098101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5544104400 msgr2=0x7f554419cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:19.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.790+0000 7f554b200700 1 --2- 172.21.15.19:0/2944098101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5544104400 0x7f554419cfa0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5538009230 tx=0x7f553800f9b0 comp rx=0 tx=0).stop 2024-05-07T02:37:19.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.790+0000 7f554b200700 1 -- 172.21.15.19:0/2944098101 shutdown_connections 2024-05-07T02:37:19.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.790+0000 7f554b200700 1 --2- 172.21.15.19:0/2944098101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5544109680 0x7f55441a17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:19.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.790+0000 7f554b200700 1 --2- 172.21.15.19:0/2944098101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5544104400 0x7f554419cfa0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:19.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.790+0000 7f554b200700 1 --2- 172.21.15.19:0/2944098101 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f552c0b29f0 0x7f552c0b4f00 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:19.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.790+0000 7f554b200700 1 --2- 172.21.15.19:0/2944098101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5544108d30 0x7f554419d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:19.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.790+0000 7f554b200700 1 -- 172.21.15.19:0/2944098101 >> 172.21.15.19:0/2944098101 conn(0x7f5544079d10 msgr2=0x7f5544102820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:37:19.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.790+0000 7f554b200700 1 -- 172.21.15.19:0/2944098101 shutdown_connections 2024-05-07T02:37:19.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:19.791+0000 7f554b200700 1 -- 172.21.15.19:0/2944098101 wait complete. 2024-05-07T02:37:20.755 INFO:tasks.workunit.client.0.smithi019.stdout: 1 175002 4.01 MB/sec execute 136 sec latency 19.694 ms 2024-05-07T02:37:21.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:21 smithi027 ceph-mon[34398]: pgmap v168: 129 pgs: 129 active+clean; 120 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-05-07T02:37:21.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:21 smithi064 ceph-mon[34699]: pgmap v168: 129 pgs: 129 active+clean; 120 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-05-07T02:37:21.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:21 smithi019 ceph-mon[29781]: pgmap v168: 129 pgs: 129 active+clean; 120 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-05-07T02:37:21.735 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-07T02:37:21.735 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-07T02:37:21.755 INFO:tasks.workunit.client.0.smithi019.stdout: 1 176398 4.03 MB/sec execute 137 sec latency 15.315 ms 2024-05-07T02:37:22.256 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:37:22.756 INFO:tasks.workunit.client.0.smithi019.stdout: 1 177619 4.07 MB/sec execute 138 sec latency 28.559 ms 2024-05-07T02:37:23.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:23 smithi027 ceph-mon[34398]: pgmap v169: 129 pgs: 129 active+clean; 123 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 231 op/s 2024-05-07T02:37:23.459 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:23 smithi019 ceph-mon[29781]: pgmap v169: 129 pgs: 129 active+clean; 123 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 231 op/s 2024-05-07T02:37:23.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:23 smithi064 ceph-mon[34699]: pgmap v169: 129 pgs: 129 active+clean; 123 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 231 op/s 2024-05-07T02:37:23.755 INFO:tasks.workunit.client.0.smithi019.stdout: 1 178166 4.05 MB/sec execute 139 sec latency 26.507 ms 2024-05-07T02:37:24.755 INFO:tasks.workunit.client.0.smithi019.stdout: 1 179768 4.07 MB/sec execute 140 sec latency 17.069 ms 2024-05-07T02:37:24.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.933+0000 7f79a9800700 1 -- 172.21.15.19:0/1994303557 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79a4109870 msgr2=0x7f79a41111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:24.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.933+0000 7f79a9800700 1 --2- 172.21.15.19:0/1994303557 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79a4109870 0x7f79a41111d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f799c009990 tx=0x7f799c009ca0 comp rx=0 tx=0).stop 2024-05-07T02:37:24.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.934+0000 7f79a9800700 1 -- 172.21.15.19:0/1994303557 shutdown_connections 2024-05-07T02:37:24.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.934+0000 7f79a9800700 1 --2- 172.21.15.19:0/1994303557 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79a4109870 0x7f79a41111d0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:24.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.934+0000 7f79a9800700 1 --2- 172.21.15.19:0/1994303557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79a41045f0 0x7f79a4104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:24.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.934+0000 7f79a9800700 1 --2- 172.21.15.19:0/1994303557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79a4108f20 0x7f79a4109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:24.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.934+0000 7f79a9800700 1 -- 172.21.15.19:0/1994303557 >> 172.21.15.19:0/1994303557 conn(0x7f79a4100040 msgr2=0x7f79a4102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:37:24.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.934+0000 7f79a9800700 1 -- 172.21.15.19:0/1994303557 shutdown_connections 2024-05-07T02:37:24.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.934+0000 7f79a9800700 1 -- 172.21.15.19:0/1994303557 wait complete. 2024-05-07T02:37:24.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.935+0000 7f79a9800700 1 Processor -- start 2024-05-07T02:37:24.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.935+0000 7f79a9800700 1 -- start start 2024-05-07T02:37:24.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.936+0000 7f79a9800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79a41045f0 0x7f79a419aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:24.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.936+0000 7f79a9800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79a4108f20 0x7f79a419b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:24.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.936+0000 7f79a9800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79a4109870 0x7f79a419f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:24.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.936+0000 7f79a9800700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f79a41144f0 con 0x7f79a41045f0 2024-05-07T02:37:24.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.936+0000 7f79a9800700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f79a4114370 con 0x7f79a4109870 2024-05-07T02:37:24.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.936+0000 7f79a9800700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f79a4114670 con 0x7f79a4108f20 2024-05-07T02:37:24.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.936+0000 7f799a000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79a4108f20 0x7f79a419b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:24.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.936+0000 7f79a2a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79a41045f0 0x7f79a419aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:24.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.936+0000 7f799a000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79a4108f20 0x7f79a419b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:45504/0 (socket says 172.21.15.19:45504) 2024-05-07T02:37:24.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.936+0000 7f799a000700 1 -- 172.21.15.19:0/350493608 learned_addr learned my addr 172.21.15.19:0/350493608 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:37:24.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.936+0000 7f79a3400700 1 --2- 172.21.15.19:0/350493608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79a4109870 0x7f79a419f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:24.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.937+0000 7f799a000700 1 -- 172.21.15.19:0/350493608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79a4109870 msgr2=0x7f79a419f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:24.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.937+0000 7f799a000700 1 --2- 172.21.15.19:0/350493608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79a4109870 0x7f79a419f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:24.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.937+0000 7f799a000700 1 -- 172.21.15.19:0/350493608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79a41045f0 msgr2=0x7f79a419aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:24.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.937+0000 7f799a000700 1 --2- 172.21.15.19:0/350493608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79a41045f0 0x7f79a419aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:24.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.937+0000 7f799a000700 1 -- 172.21.15.19:0/350493608 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a419fd90 con 0x7f79a4108f20 2024-05-07T02:37:24.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.937+0000 7f79a3400700 1 --2- 172.21.15.19:0/350493608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79a4109870 0x7f79a419f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:37:24.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.937+0000 7f799a000700 1 --2- 172.21.15.19:0/350493608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79a4108f20 0x7f79a419b460 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f798c00e970 tx=0x7f798c00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:37:24.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.937+0000 7f799be00700 1 -- 172.21.15.19:0/350493608 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c00cbd0 con 0x7f79a4108f20 2024-05-07T02:37:24.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.937+0000 7f79a9800700 1 -- 172.21.15.19:0/350493608 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79a41a0080 con 0x7f79a4108f20 2024-05-07T02:37:24.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.937+0000 7f799be00700 1 -- 172.21.15.19:0/350493608 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f798c00cd70 con 0x7f79a4108f20 2024-05-07T02:37:24.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.938+0000 7f799be00700 1 -- 172.21.15.19:0/350493608 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c005680 con 0x7f79a4108f20 2024-05-07T02:37:24.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.938+0000 7f79a9800700 1 -- 172.21.15.19:0/350493608 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79a41a79a0 con 0x7f79a4108f20 2024-05-07T02:37:24.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.942+0000 7f799be00700 1 -- 172.21.15.19:0/350493608 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f798c004750 con 0x7f79a4108f20 2024-05-07T02:37:24.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.943+0000 7f79b8334b80 1 Processor -- start 2024-05-07T02:37:24.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.943+0000 7f79b8334b80 1 -- start start 2024-05-07T02:37:24.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.944+0000 7f79b8334b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ac0bb24f0 0x555ac0bb2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:24.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.944+0000 7f79b8334b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac0bac6a0 0x555ac0bacab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:24.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.944+0000 7f79b8334b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac0bacff0 0x555ac0bb91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:24.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.944+0000 7f79b8334b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555ac0aceaa0 con 0x555ac0bacff0 2024-05-07T02:37:24.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.944+0000 7f79b8334b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x555ac0a6c7e0 con 0x555ac0bb24f0 2024-05-07T02:37:24.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.944+0000 7f79b8334b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555ac0acaa60 con 0x555ac0bac6a0 2024-05-07T02:37:24.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.944+0000 7f79a3400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ac0bb24f0 0x555ac0bb2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:24.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.944+0000 7f79a3400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ac0bb24f0 0x555ac0bb2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:45334/0 (socket says 172.21.15.19:45334) 2024-05-07T02:37:24.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.944+0000 7f79a3400700 1 -- 172.21.15.19:0/2421134872 learned_addr learned my addr 172.21.15.19:0/2421134872 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:37:24.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.944+0000 7f79a2a00700 1 --2- 172.21.15.19:0/2421134872 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac0bac6a0 0x555ac0bacab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:24.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.944+0000 7f799a000700 1 --2- 172.21.15.19:0/2421134872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac0bacff0 0x555ac0bb91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:24.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.944+0000 7f79a3400700 1 -- 172.21.15.19:0/2421134872 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac0bac6a0 msgr2=0x555ac0bacab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:24.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.944+0000 7f79a3400700 1 --2- 172.21.15.19:0/2421134872 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac0bac6a0 0x555ac0bacab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:24.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.944+0000 7f79a3400700 1 -- 172.21.15.19:0/2421134872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac0bacff0 msgr2=0x555ac0bb91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:24.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.944+0000 7f79a3400700 1 --2- 172.21.15.19:0/2421134872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac0bacff0 0x555ac0bb91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:24.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.945+0000 7f79a3400700 1 -- 172.21.15.19:0/2421134872 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ac0a1b950 con 0x555ac0bb24f0 2024-05-07T02:37:24.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.945+0000 7f799be00700 1 --2- 172.21.15.19:0/350493608 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f79840b29f0 0x7f79840b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:24.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.945+0000 7f79a3400700 1 --2- 172.21.15.19:0/2421134872 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ac0bb24f0 0x555ac0bb2900 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f799c00b2e0 tx=0x7f799c0096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6dbe38f31022880e server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:37:24.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.945+0000 7f799be00700 1 -- 172.21.15.19:0/350493608 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f798c0e2c00 con 0x7f79a4108f20 2024-05-07T02:37:24.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.945+0000 7f79a2a00700 1 --2- 172.21.15.19:0/350493608 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f79840b29f0 0x7f79840b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:24.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.945+0000 7f7993e00700 1 -- 172.21.15.19:0/2421134872 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c00b610 con 0x555ac0bb24f0 2024-05-07T02:37:24.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.945+0000 7f7993e00700 1 -- 172.21.15.19:0/2421134872 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f799c00bca0 con 0x555ac0bb24f0 2024-05-07T02:37:24.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.945+0000 7f7993e00700 1 -- 172.21.15.19:0/2421134872 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c003c40 con 0x555ac0bb24f0 2024-05-07T02:37:24.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.946+0000 7f79b8334b80 1 -- 172.21.15.19:0/2421134872 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ac0bb24f0 msgr2=0x555ac0bb2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:24.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.946+0000 7f79b8334b80 1 --2- 172.21.15.19:0/2421134872 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ac0bb24f0 0x555ac0bb2900 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f799c00b2e0 tx=0x7f799c0096a0 comp rx=0 tx=0).stop 2024-05-07T02:37:24.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.946+0000 7f79a2a00700 1 --2- 172.21.15.19:0/350493608 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f79840b29f0 0x7f79840b4f00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x555ac0ba8ed0 tx=0x7f799400b040 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:37:24.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.947+0000 7f79b8334b80 1 -- 172.21.15.19:0/2421134872 shutdown_connections 2024-05-07T02:37:24.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.947+0000 7f79b8334b80 1 --2- 172.21.15.19:0/2421134872 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac0bacff0 0x555ac0bb91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:24.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.947+0000 7f79b8334b80 1 --2- 172.21.15.19:0/2421134872 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ac0bb24f0 0x555ac0bb2900 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:24.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.947+0000 7f79b8334b80 1 --2- 172.21.15.19:0/2421134872 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac0bac6a0 0x555ac0bacab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:24.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.947+0000 7f79b8334b80 1 -- 172.21.15.19:0/2421134872 >> 172.21.15.19:0/2421134872 conn(0x555ac0a2eec0 msgr2=0x555ac0ba7400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:37:24.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.947+0000 7f79b8334b80 1 -- 172.21.15.19:0/2421134872 shutdown_connections 2024-05-07T02:37:24.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.947+0000 7f79b8334b80 1 -- 172.21.15.19:0/2421134872 wait complete. 2024-05-07T02:37:24.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.948+0000 7f79b8334b80 1 -- start start 2024-05-07T02:37:24.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.948+0000 7f79b8334b80 1 Processor -- start 2024-05-07T02:37:24.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.949+0000 7f79b8334b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ac0bac6a0 0x555ac0bb6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:24.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.949+0000 7f79b8334b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac0bacff0 0x555ac0bb6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:24.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.949+0000 7f79b8334b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac0bb24f0 0x555ac0bb71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:24.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.949+0000 7f79b8334b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555ac0bb9860 con 0x555ac0bacff0 2024-05-07T02:37:24.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.949+0000 7f79b8334b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x555ac0a34ba0 con 0x555ac0bac6a0 2024-05-07T02:37:24.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.949+0000 7f79b8334b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555ac0bb99e0 con 0x555ac0bb24f0 2024-05-07T02:37:24.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.949+0000 7f79b8334b80 10 client.0 _ll_register_callbacks cb 0x555ac0a62400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:37:24.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.949+0000 7f79a2a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ac0bac6a0 0x555ac0bb6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:24.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.949+0000 7f79a2a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ac0bac6a0 0x555ac0bb6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:45344/0 (socket says 172.21.15.19:45344) 2024-05-07T02:37:24.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.949+0000 7f79a2a00700 1 -- 172.21.15.19:0/3224572542 learned_addr learned my addr 172.21.15.19:0/3224572542 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:37:24.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.949+0000 7f79a3400700 1 --2- 172.21.15.19:0/3224572542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac0bb24f0 0x555ac0bb71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:24.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.949+0000 7f799a000700 1 --2- 172.21.15.19:0/3224572542 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac0bacff0 0x555ac0bb6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:24.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.949+0000 7f79a2a00700 1 -- 172.21.15.19:0/3224572542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac0bb24f0 msgr2=0x555ac0bb71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:24.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.949+0000 7f79a2a00700 1 --2- 172.21.15.19:0/3224572542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac0bb24f0 0x555ac0bb71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:24.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.949+0000 7f79a2a00700 1 -- 172.21.15.19:0/3224572542 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac0bacff0 msgr2=0x555ac0bb6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:24.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.949+0000 7f79a2a00700 1 --2- 172.21.15.19:0/3224572542 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac0bacff0 0x555ac0bb6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:24.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.949+0000 7f79a2a00700 1 -- 172.21.15.19:0/3224572542 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ac0a944a0 con 0x555ac0bac6a0 2024-05-07T02:37:24.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.949+0000 7f79a3400700 1 --2- 172.21.15.19:0/3224572542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac0bb24f0 0x555ac0bb71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:37:24.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.950+0000 7f79a2a00700 1 --2- 172.21.15.19:0/3224572542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ac0bac6a0 0x555ac0bb6750 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f79940042c0 tx=0x7f7994004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=583f1263c8a5de06 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:37:24.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.950+0000 7f7990c00700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:37:24.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.950+0000 7f7990c00700 1 -- 172.21.15.19:0/3224572542 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7994027070 con 0x555ac0bac6a0 2024-05-07T02:37:24.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.950+0000 7f7990c00700 1 -- 172.21.15.19:0/3224572542 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f7994023870 con 0x555ac0bac6a0 2024-05-07T02:37:24.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.950+0000 7f79b8334b80 1 -- 172.21.15.19:0/3224572542 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f799c00bca0 con 0x555ac0bac6a0 2024-05-07T02:37:24.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.951+0000 7f7990c00700 1 -- 172.21.15.19:0/3224572542 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f7994033050 con 0x555ac0bac6a0 2024-05-07T02:37:24.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.951+0000 7f79b8334b80 10 client.26138 fetch_fsmap learned FSMap version 40 2024-05-07T02:37:24.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.951+0000 7f79b8334b80 1 -- 172.21.15.19:0/3224572542 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x555ac0bc4510 con 0x555ac0bac6a0 2024-05-07T02:37:24.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.951+0000 7f7990c00700 1 -- 172.21.15.19:0/3224572542 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f799c00bca0 con 0x555ac0bac6a0 2024-05-07T02:37:24.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.951+0000 7f7990c00700 1 -- 172.21.15.19:0/3224572542 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f7994014c20 con 0x555ac0bac6a0 2024-05-07T02:37:24.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.951+0000 7f79b8334b80 10 client.26138 fetch_fsmap finished waiting for FSMap version 40 2024-05-07T02:37:24.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.951+0000 7f79b8334b80 10 client.26138 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:37:24.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.951+0000 7f79b8334b80 20 client.26138 populate_metadata read hostname 'smithi019' 2024-05-07T02:37:24.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.951+0000 7f79b8334b80 1 --2- 172.21.15.19:0/3224572542 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x555ac0bc6b50 0x555ac0bc8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:24.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.951+0000 7f79b8334b80 4 client.26138 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:37:24.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.951+0000 7f79b8334b80 1 -- 172.21.15.19:0/3224572542 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ac0b79160 con 0x555ac0bc6b50 2024-05-07T02:37:24.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.952+0000 7f799a000700 1 --2- 172.21.15.19:0/3224572542 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x555ac0bc6b50 0x555ac0bc8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:24.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.952+0000 7f799a000700 1 --2- 172.21.15.19:0/3224572542 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x555ac0bc6b50 0x555ac0bc8fe0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed9a5d8cac81b5f8 server_cookie=f6f82867606c73e2 in_seq=0 out_seq=0 2024-05-07T02:37:24.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.952+0000 7f7990c00700 10 client.26138 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:37:24.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.953+0000 7f7990c00700 1 -- 172.21.15.19:0/3224572542 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555ac0b79160 con 0x555ac0bc6b50 2024-05-07T02:37:24.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.953+0000 7f7990c00700 10 client.26138 handle_command_reply: tid=0 2024-05-07T02:37:24.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.953+0000 7f79b8334b80 1 client.26138 shutdown 2024-05-07T02:37:24.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.953+0000 7f79b8334b80 1 -- 172.21.15.19:0/3224572542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ac0bac6a0 msgr2=0x555ac0bb6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:24.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.953+0000 7f79b8334b80 1 --2- 172.21.15.19:0/3224572542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ac0bac6a0 0x555ac0bb6750 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f79940042c0 tx=0x7f7994004b40 comp rx=0 tx=0).stop 2024-05-07T02:37:24.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.953+0000 7f79b8334b80 1 -- 172.21.15.19:0/3224572542 shutdown_connections 2024-05-07T02:37:24.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.954+0000 7f79b8334b80 1 --2- 172.21.15.19:0/3224572542 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x555ac0bc6b50 0x555ac0bc8fe0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:24.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.954+0000 7f79b8334b80 1 --2- 172.21.15.19:0/3224572542 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac0bb24f0 0x555ac0bb71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:24.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.954+0000 7f79b8334b80 1 --2- 172.21.15.19:0/3224572542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ac0bac6a0 0x555ac0bb6750 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:24.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.954+0000 7f79b8334b80 1 --2- 172.21.15.19:0/3224572542 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac0bacff0 0x555ac0bb6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:24.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.954+0000 7f7990c00700 0 client.26138 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:37:24.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.954+0000 7f79b8334b80 1 -- 172.21.15.19:0/3224572542 >> 172.21.15.19:0/3224572542 conn(0x555ac0a2eec0 msgr2=0x555ac0bab690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:37:24.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.954+0000 7f79b8334b80 1 -- 172.21.15.19:0/3224572542 shutdown_connections 2024-05-07T02:37:24.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.954+0000 7f79b8334b80 1 -- 172.21.15.19:0/3224572542 wait complete. 2024-05-07T02:37:24.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.954+0000 7f79b8334b80 20 client.26138 trim_cache size 0 max 16384 2024-05-07T02:37:24.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.982+0000 7f79b8334b80 1 Processor -- start 2024-05-07T02:37:24.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.982+0000 7f79b8334b80 1 -- start start 2024-05-07T02:37:24.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.983+0000 7f79b8334b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac0bac6a0 0x555ac0bb7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:24.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.983+0000 7f79b8334b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ac0bacff0 0x555ac0bb5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:24.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.983+0000 7f79b8334b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac0bb24f0 0x555ac0ba1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:24.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.983+0000 7f79b8334b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f79a41b05c0 con 0x555ac0bb24f0 2024-05-07T02:37:24.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.983+0000 7f79b8334b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f79a41b2160 con 0x555ac0bacff0 2024-05-07T02:37:24.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.983+0000 7f79b8334b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f79a41b0c90 con 0x555ac0bac6a0 2024-05-07T02:37:24.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.983+0000 7f79a2a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac0bac6a0 0x555ac0bb7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:24.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.983+0000 7f799a000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ac0bacff0 0x555ac0bb5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:24.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.983+0000 7f79a2a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac0bac6a0 0x555ac0bb7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:45528/0 (socket says 172.21.15.19:45528) 2024-05-07T02:37:24.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.983+0000 7f79a2a00700 1 -- 172.21.15.19:0/536720722 learned_addr learned my addr 172.21.15.19:0/536720722 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:37:24.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.983+0000 7f79a3400700 1 --2- 172.21.15.19:0/536720722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac0bb24f0 0x555ac0ba1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:24.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.984+0000 7f799a000700 1 -- 172.21.15.19:0/536720722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac0bac6a0 msgr2=0x555ac0bb7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:24.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.984+0000 7f799a000700 1 --2- 172.21.15.19:0/536720722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac0bac6a0 0x555ac0bb7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:24.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.984+0000 7f799a000700 1 -- 172.21.15.19:0/536720722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac0bb24f0 msgr2=0x555ac0ba1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:24.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.984+0000 7f799a000700 1 --2- 172.21.15.19:0/536720722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac0bb24f0 0x555ac0ba1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:24.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.984+0000 7f799a000700 1 -- 172.21.15.19:0/536720722 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ac0ba3c00 con 0x555ac0bacff0 2024-05-07T02:37:24.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.984+0000 7f79a2a00700 1 --2- 172.21.15.19:0/536720722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac0bac6a0 0x555ac0bb7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:37:24.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.984+0000 7f799a000700 1 --2- 172.21.15.19:0/536720722 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ac0bacff0 0x555ac0bb5900 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f798c0dff30 tx=0x7f798c0cc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1b9ebdb83734a09 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:37:24.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.984+0000 7f7993400700 1 -- 172.21.15.19:0/536720722 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c0a9260 con 0x555ac0bacff0 2024-05-07T02:37:24.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.984+0000 7f7993400700 1 -- 172.21.15.19:0/536720722 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f798c08ae50 con 0x555ac0bacff0 2024-05-07T02:37:24.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.984+0000 7f7993400700 1 -- 172.21.15.19:0/536720722 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c0a5280 con 0x555ac0bacff0 2024-05-07T02:37:24.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.985+0000 7f79b8334b80 1 -- 172.21.15.19:0/536720722 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ac0bacff0 msgr2=0x555ac0bb5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:24.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.985+0000 7f79b8334b80 1 --2- 172.21.15.19:0/536720722 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ac0bacff0 0x555ac0bb5900 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f798c0dff30 tx=0x7f798c0cc040 comp rx=0 tx=0).stop 2024-05-07T02:37:24.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.985+0000 7f79b8334b80 1 -- 172.21.15.19:0/536720722 shutdown_connections 2024-05-07T02:37:24.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.985+0000 7f79b8334b80 1 --2- 172.21.15.19:0/536720722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac0bb24f0 0x555ac0ba1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:24.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.985+0000 7f79b8334b80 1 --2- 172.21.15.19:0/536720722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac0bac6a0 0x555ac0bb7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:24.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.985+0000 7f79b8334b80 1 --2- 172.21.15.19:0/536720722 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ac0bacff0 0x555ac0bb5900 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:24.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.985+0000 7f79b8334b80 1 -- 172.21.15.19:0/536720722 >> 172.21.15.19:0/536720722 conn(0x555ac0a2eec0 msgr2=0x555ac0ba1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:37:24.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.986+0000 7f79b8334b80 1 -- 172.21.15.19:0/536720722 shutdown_connections 2024-05-07T02:37:24.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.986+0000 7f79b8334b80 1 -- 172.21.15.19:0/536720722 wait complete. 2024-05-07T02:37:24.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.986+0000 7f79b8334b80 1 -- start start 2024-05-07T02:37:24.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.986+0000 7f79b8334b80 1 Processor -- start 2024-05-07T02:37:24.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.987+0000 7f79b8334b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ac0bac6a0 0x555ac0bba860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:24.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.987+0000 7f79b8334b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac0bacff0 0x555ac0bbada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:24.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.987+0000 7f79b8334b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac0bb24f0 0x555ac0bbb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:24.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.987+0000 7f79b8334b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f79a41b1a90 con 0x555ac0bb24f0 2024-05-07T02:37:24.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.987+0000 7f79b8334b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f79a41b13c0 con 0x555ac0bac6a0 2024-05-07T02:37:24.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.987+0000 7f79b8334b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f79a41ae560 con 0x555ac0bacff0 2024-05-07T02:37:24.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.987+0000 7f79b8334b80 10 client.0 _ll_register_callbacks cb 0x555ac0a62400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:37:24.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.987+0000 7f79a2a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ac0bac6a0 0x555ac0bba860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:24.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.987+0000 7f799a000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac0bacff0 0x555ac0bbada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:24.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.987+0000 7f79a2a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ac0bac6a0 0x555ac0bba860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:45358/0 (socket says 172.21.15.19:45358) 2024-05-07T02:37:24.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.987+0000 7f799a000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac0bacff0 0x555ac0bbada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:45532/0 (socket says 172.21.15.19:45532) 2024-05-07T02:37:24.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.987+0000 7f79a2a00700 1 -- 172.21.15.19:0/3263117811 learned_addr learned my addr 172.21.15.19:0/3263117811 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:37:24.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.987+0000 7f79a3400700 1 --2- 172.21.15.19:0/3263117811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac0bb24f0 0x555ac0bbb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:24.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.988+0000 7f79a2a00700 1 -- 172.21.15.19:0/3263117811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac0bacff0 msgr2=0x555ac0bbada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:24.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.988+0000 7f79a2a00700 1 --2- 172.21.15.19:0/3263117811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac0bacff0 0x555ac0bbada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:24.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.988+0000 7f79a2a00700 1 -- 172.21.15.19:0/3263117811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac0bb24f0 msgr2=0x555ac0bbb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:24.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.988+0000 7f79a2a00700 1 --2- 172.21.15.19:0/3263117811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac0bb24f0 0x555ac0bbb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:24.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.988+0000 7f79a2a00700 1 -- 172.21.15.19:0/3263117811 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ac0c30350 con 0x555ac0bac6a0 2024-05-07T02:37:24.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.988+0000 7f799a000700 1 --2- 172.21.15.19:0/3263117811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac0bacff0 0x555ac0bbada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:37:24.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.988+0000 7f79a2a00700 1 --2- 172.21.15.19:0/3263117811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ac0bac6a0 0x555ac0bba860 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7994015010 tx=0x7f7994004a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54fb35a39d118987 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:37:24.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.988+0000 7f7990c00700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:37:24.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.988+0000 7f7990c00700 1 -- 172.21.15.19:0/3263117811 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799403d020 con 0x555ac0bac6a0 2024-05-07T02:37:24.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.988+0000 7f79b8334b80 1 -- 172.21.15.19:0/3263117811 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f798c08ae50 con 0x555ac0bac6a0 2024-05-07T02:37:24.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.988+0000 7f7990c00700 1 -- 172.21.15.19:0/3263117811 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f7994016dd0 con 0x555ac0bac6a0 2024-05-07T02:37:24.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.989+0000 7f7990c00700 1 -- 172.21.15.19:0/3263117811 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f7994027040 con 0x555ac0bac6a0 2024-05-07T02:37:24.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.989+0000 7f79b8334b80 10 client.26144 fetch_fsmap learned FSMap version 40 2024-05-07T02:37:24.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.989+0000 7f79b8334b80 1 -- 172.21.15.19:0/3263117811 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x555ac0c315c0 con 0x555ac0bac6a0 2024-05-07T02:37:24.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.989+0000 7f7990c00700 1 -- 172.21.15.19:0/3263117811 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f798c08ae50 con 0x555ac0bac6a0 2024-05-07T02:37:24.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.989+0000 7f7990c00700 1 -- 172.21.15.19:0/3263117811 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f7994036040 con 0x555ac0bac6a0 2024-05-07T02:37:24.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.989+0000 7f79b8334b80 10 client.26144 fetch_fsmap finished waiting for FSMap version 40 2024-05-07T02:37:24.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.989+0000 7f79b8334b80 10 client.26144 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:37:24.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.989+0000 7f79b8334b80 20 client.26144 populate_metadata read hostname 'smithi019' 2024-05-07T02:37:24.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.989+0000 7f79b8334b80 1 --2- 172.21.15.19:0/3263117811 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x555ac0c31d20 0x555ac0c34140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:24.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.989+0000 7f79b8334b80 4 client.26144 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-07T02:37:24.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.989+0000 7f79b8334b80 1 -- 172.21.15.19:0/3263117811 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555ac0adc480 con 0x555ac0c31d20 2024-05-07T02:37:24.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.990+0000 7f799a000700 1 --2- 172.21.15.19:0/3263117811 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x555ac0c31d20 0x555ac0c34140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:24.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.990+0000 7f799a000700 1 --2- 172.21.15.19:0/3263117811 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x555ac0c31d20 0x555ac0c34140 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44517f4073ae16d2 server_cookie=835b34e1a46efff7 in_seq=0 out_seq=0 2024-05-07T02:37:24.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.990+0000 7f7990c00700 10 client.26144 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:37:24.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.992+0000 7f7990c00700 1 -- 172.21.15.19:0/3263117811 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555ac0adc480 con 0x555ac0c31d20 2024-05-07T02:37:24.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.992+0000 7f7990c00700 10 client.26144 handle_command_reply: tid=0 2024-05-07T02:37:24.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.992+0000 7f79b8334b80 1 client.26144 shutdown 2024-05-07T02:37:24.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.992+0000 7f79b8334b80 1 -- 172.21.15.19:0/3263117811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ac0bac6a0 msgr2=0x555ac0bba860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:24.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.992+0000 7f79b8334b80 1 --2- 172.21.15.19:0/3263117811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ac0bac6a0 0x555ac0bba860 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7994015010 tx=0x7f7994004a50 comp rx=0 tx=0).stop 2024-05-07T02:37:24.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.992+0000 7f79b8334b80 1 -- 172.21.15.19:0/3263117811 shutdown_connections 2024-05-07T02:37:24.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.992+0000 7f79b8334b80 1 --2- 172.21.15.19:0/3263117811 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x555ac0c31d20 0x555ac0c34140 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:24.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.992+0000 7f79b8334b80 1 --2- 172.21.15.19:0/3263117811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac0bb24f0 0x555ac0bbb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:24.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.992+0000 7f79b8334b80 1 --2- 172.21.15.19:0/3263117811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555ac0bac6a0 0x555ac0bba860 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:24.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.992+0000 7f79b8334b80 1 --2- 172.21.15.19:0/3263117811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555ac0bacff0 0x555ac0bbada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:24.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.992+0000 7f7990c00700 0 client.26144 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:37:24.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.992+0000 7f79b8334b80 1 -- 172.21.15.19:0/3263117811 >> 172.21.15.19:0/3263117811 conn(0x555ac0a2eec0 msgr2=0x555ac0bb6e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:37:24.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.992+0000 7f79b8334b80 1 -- 172.21.15.19:0/3263117811 shutdown_connections 2024-05-07T02:37:24.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.992+0000 7f79b8334b80 1 -- 172.21.15.19:0/3263117811 wait complete. 2024-05-07T02:37:24.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.993+0000 7f79b8334b80 20 client.26144 trim_cache size 0 max 16384 2024-05-07T02:37:24.995 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:37:24.995 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-07T02:37:24.995 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "9a7bb2fc-c87b-4549-8559-ac79c77de966", 2024-05-07T02:37:24.995 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-07T02:37:24.995 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:37:24.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.993+0000 7f79a9800700 1 -- 172.21.15.19:0/350493608 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f79840b29f0 msgr2=0x7f79840b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:24.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.993+0000 7f79a9800700 1 --2- 172.21.15.19:0/350493608 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f79840b29f0 0x7f79840b4f00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x555ac0ba8ed0 tx=0x7f799400b040 comp rx=0 tx=0).stop 2024-05-07T02:37:24.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.993+0000 7f79a9800700 1 -- 172.21.15.19:0/350493608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79a4108f20 msgr2=0x7f79a419b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:24.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.993+0000 7f79a9800700 1 --2- 172.21.15.19:0/350493608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79a4108f20 0x7f79a419b460 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f798c00e970 tx=0x7f798c00ec80 comp rx=0 tx=0).stop 2024-05-07T02:37:24.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.994+0000 7f79a9800700 1 -- 172.21.15.19:0/350493608 shutdown_connections 2024-05-07T02:37:24.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.994+0000 7f79a9800700 1 --2- 172.21.15.19:0/350493608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79a4109870 0x7f79a419f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:24.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.994+0000 7f79a9800700 1 --2- 172.21.15.19:0/350493608 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f79840b29f0 0x7f79840b4f00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:24.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.994+0000 7f79a9800700 1 --2- 172.21.15.19:0/350493608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79a41045f0 0x7f79a419aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:24.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.994+0000 7f79a9800700 1 --2- 172.21.15.19:0/350493608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f79a4108f20 0x7f79a419b460 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:24.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.994+0000 7f79a9800700 1 -- 172.21.15.19:0/350493608 >> 172.21.15.19:0/350493608 conn(0x7f79a4100040 msgr2=0x7f79a410a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:37:24.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.994+0000 7f79a9800700 1 -- 172.21.15.19:0/350493608 shutdown_connections 2024-05-07T02:37:24.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:24.994+0000 7f79a9800700 1 -- 172.21.15.19:0/350493608 wait complete. 2024-05-07T02:37:25.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:25 smithi027 ceph-mon[34398]: pgmap v170: 129 pgs: 129 active+clean; 123 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 154 op/s 2024-05-07T02:37:25.460 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:25 smithi019 ceph-mon[29781]: pgmap v170: 129 pgs: 129 active+clean; 123 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 154 op/s 2024-05-07T02:37:25.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:25 smithi064 ceph-mon[34699]: pgmap v170: 129 pgs: 129 active+clean; 123 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 154 op/s 2024-05-07T02:37:25.755 INFO:tasks.workunit.client.0.smithi019.stdout: 1 181114 4.12 MB/sec execute 141 sec latency 13.927 ms 2024-05-07T02:37:25.891 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-07T02:37:26.422 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:37:26.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:26 smithi027 ceph-mon[34398]: scrub queued for path: / 2024-05-07T02:37:26.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:26 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:37:26.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:26 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:37:26.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:26 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:37:26.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:26 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:37:26.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:26 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:37:26.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:26 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:37:26.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:26 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:37:26.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:26 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:37:26.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:26 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:37:26.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:26 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:37:26.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:26 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:37:26.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:26 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:37:26.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:26 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:37:26.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:26 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:37:26.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:26 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:37:26.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:26 smithi019 ceph-mon[29781]: scrub queued for path: / 2024-05-07T02:37:26.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:26 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:37:26.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:26 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:37:26.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:26 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:37:26.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:26 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:37:26.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:26 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:37:26.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:26 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:37:26.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:26 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:37:26.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:26 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:37:26.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:26 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:37:26.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:26 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:37:26.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:26 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:37:26.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:26 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:37:26.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:26 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:37:26.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:26 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:37:26.572 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:26 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:37:26.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:26 smithi064 ceph-mon[34699]: scrub queued for path: / 2024-05-07T02:37:26.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:26 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:37:26.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:26 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:37:26.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:26 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:37:26.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:26 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:37:26.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:26 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:37:26.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:26 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:37:26.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:26 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:37:26.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:26 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:37:26.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:26 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:37:26.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:26 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:37:26.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:26 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:37:26.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:26 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:37:26.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:26 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:37:26.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:26 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:37:26.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:26 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:37:26.756 INFO:tasks.workunit.client.0.smithi019.stdout: 1 181601 4.10 MB/sec execute 142 sec latency 22.430 ms 2024-05-07T02:37:27.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:27 smithi019 ceph-mon[29781]: pgmap v171: 129 pgs: 129 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 235 op/s 2024-05-07T02:37:27.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:27 smithi027 ceph-mon[34398]: pgmap v171: 129 pgs: 129 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 235 op/s 2024-05-07T02:37:27.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:27 smithi064 ceph-mon[34699]: pgmap v171: 129 pgs: 129 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 235 op/s 2024-05-07T02:37:27.756 INFO:tasks.workunit.client.0.smithi019.stdout: 1 182773 4.11 MB/sec execute 143 sec latency 51.485 ms 2024-05-07T02:37:28.757 INFO:tasks.workunit.client.0.smithi019.stdout: 1 184302 4.14 MB/sec execute 144 sec latency 18.495 ms 2024-05-07T02:37:28.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.803+0000 7fc901600700 1 -- 172.21.15.19:0/1225925188 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8fc1191a0 msgr2=0x7fc8fc11b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:28.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.803+0000 7fc901600700 1 --2- 172.21.15.19:0/1225925188 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8fc1191a0 0x7fc8fc11b600 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400b0a0 tx=0x7fc8f400b3b0 comp rx=0 tx=0).stop 2024-05-07T02:37:28.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.804+0000 7fc901600700 1 -- 172.21.15.19:0/1225925188 shutdown_connections 2024-05-07T02:37:28.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.804+0000 7fc901600700 1 --2- 172.21.15.19:0/1225925188 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8fc1191a0 0x7fc8fc11b600 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:28.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.804+0000 7fc901600700 1 --2- 172.21.15.19:0/1225925188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8fc06d4e0 0x7fc8fc06d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:28.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.804+0000 7fc901600700 1 --2- 172.21.15.19:0/1225925188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8fc06de30 0x7fc8fc118c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:28.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.804+0000 7fc901600700 1 -- 172.21.15.19:0/1225925188 >> 172.21.15.19:0/1225925188 conn(0x7fc8fc0704c0 msgr2=0x7fc8fc0708d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:37:28.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.804+0000 7fc901600700 1 -- 172.21.15.19:0/1225925188 shutdown_connections 2024-05-07T02:37:28.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.804+0000 7fc901600700 1 -- 172.21.15.19:0/1225925188 wait complete. 2024-05-07T02:37:28.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.805+0000 7fc901600700 1 Processor -- start 2024-05-07T02:37:28.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.805+0000 7fc901600700 1 -- start start 2024-05-07T02:37:28.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.806+0000 7fc901600700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8fc06d4e0 0x7fc8fc10c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:28.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.806+0000 7fc901600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8fc06de30 0x7fc8fc10ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:28.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.806+0000 7fc901600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8fc1191a0 0x7fc8fc10d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:28.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.806+0000 7fc901600700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc8fc11ecd0 con 0x7fc8fc06de30 2024-05-07T02:37:28.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.806+0000 7fc901600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc8fc11eb50 con 0x7fc8fc1191a0 2024-05-07T02:37:28.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.806+0000 7fc901600700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc8fc11ee50 con 0x7fc8fc06d4e0 2024-05-07T02:37:28.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.806+0000 7fc900c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8fc1191a0 0x7fc8fc10d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:28.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.806+0000 7fc900c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8fc1191a0 0x7fc8fc10d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:45376/0 (socket says 172.21.15.19:45376) 2024-05-07T02:37:28.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.806+0000 7fc8fbe00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8fc06d4e0 0x7fc8fc10c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:28.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.806+0000 7fc900c00700 1 -- 172.21.15.19:0/331863614 learned_addr learned my addr 172.21.15.19:0/331863614 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:37:28.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.806+0000 7fc8f3e00700 1 --2- 172.21.15.19:0/331863614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8fc06de30 0x7fc8fc10ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:28.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.807+0000 7fc8fbe00700 1 -- 172.21.15.19:0/331863614 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8fc1191a0 msgr2=0x7fc8fc10d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:28.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.807+0000 7fc8fbe00700 1 --2- 172.21.15.19:0/331863614 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8fc1191a0 0x7fc8fc10d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:28.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.807+0000 7fc8fbe00700 1 -- 172.21.15.19:0/331863614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8fc06de30 msgr2=0x7fc8fc10ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:28.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.807+0000 7fc8fbe00700 1 --2- 172.21.15.19:0/331863614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8fc06de30 0x7fc8fc10ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:28.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.807+0000 7fc8fbe00700 1 -- 172.21.15.19:0/331863614 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8fc1b5bc0 con 0x7fc8fc06d4e0 2024-05-07T02:37:28.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.807+0000 7fc900c00700 1 --2- 172.21.15.19:0/331863614 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8fc1191a0 0x7fc8fc10d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:37:28.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.807+0000 7fc8fbe00700 1 --2- 172.21.15.19:0/331863614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8fc06d4e0 0x7fc8fc10c8c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec00e9f0 tx=0x7fc8ec00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:37:28.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.808+0000 7fc8f9600700 1 -- 172.21.15.19:0/331863614 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec00c9c0 con 0x7fc8fc06d4e0 2024-05-07T02:37:28.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.808+0000 7fc8f9600700 1 -- 172.21.15.19:0/331863614 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8ec00cb60 con 0x7fc8fc06d4e0 2024-05-07T02:37:28.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.808+0000 7fc8f9600700 1 -- 172.21.15.19:0/331863614 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec018680 con 0x7fc8fc06d4e0 2024-05-07T02:37:28.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.808+0000 7fc901600700 1 -- 172.21.15.19:0/331863614 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8fc1b5e50 con 0x7fc8fc06d4e0 2024-05-07T02:37:28.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.808+0000 7fc901600700 1 -- 172.21.15.19:0/331863614 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8fc1b6250 con 0x7fc8fc06d4e0 2024-05-07T02:37:28.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.812+0000 7fc8f9600700 1 -- 172.21.15.19:0/331863614 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc8ec010430 con 0x7fc8fc06d4e0 2024-05-07T02:37:28.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.813+0000 7fc911777b80 1 Processor -- start 2024-05-07T02:37:28.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.814+0000 7fc911777b80 1 -- start start 2024-05-07T02:37:28.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.814+0000 7fc911777b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4d1388c40 0x55c4d1389050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:28.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.814+0000 7fc911777b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4d1389590 0x55c4d13926f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:28.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.814+0000 7fc911777b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c4d1392c30 0x55c4d1395050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:28.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.814+0000 7fc911777b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c4d12aaa20 con 0x55c4d1388c40 2024-05-07T02:37:28.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.814+0000 7fc911777b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c4d1248760 con 0x55c4d1392c30 2024-05-07T02:37:28.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.814+0000 7fc911777b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c4d12a69e0 con 0x55c4d1389590 2024-05-07T02:37:28.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.814+0000 7fc8fbe00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4d1389590 0x55c4d13926f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:28.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.815+0000 7fc900c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4d1388c40 0x55c4d1389050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:28.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.815+0000 7fc8f3e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c4d1392c30 0x55c4d1395050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:28.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.815+0000 7fc8fbe00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4d1389590 0x55c4d13926f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:45552/0 (socket says 172.21.15.19:45552) 2024-05-07T02:37:28.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.815+0000 7fc8fbe00700 1 -- 172.21.15.19:0/110795485 learned_addr learned my addr 172.21.15.19:0/110795485 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:37:28.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.815+0000 7fc8f9600700 1 --2- 172.21.15.19:0/331863614 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc8e00b2a70 0x7fc8e00b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:28.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.815+0000 7fc8f9600700 1 -- 172.21.15.19:0/331863614 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fc8ec0e2a70 con 0x7fc8fc06d4e0 2024-05-07T02:37:28.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.815+0000 7fc8fbe00700 1 -- 172.21.15.19:0/110795485 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c4d1392c30 msgr2=0x55c4d1395050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:28.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.815+0000 7fc8fbe00700 1 --2- 172.21.15.19:0/110795485 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c4d1392c30 0x55c4d1395050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:28.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.815+0000 7fc8fbe00700 1 -- 172.21.15.19:0/110795485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4d1388c40 msgr2=0x55c4d1389050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:28.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.815+0000 7fc8fbe00700 1 --2- 172.21.15.19:0/110795485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4d1388c40 0x55c4d1389050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:28.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.815+0000 7fc8fbe00700 1 -- 172.21.15.19:0/110795485 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4d11f78d0 con 0x55c4d1389590 2024-05-07T02:37:28.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.815+0000 7fc8f3e00700 1 --2- 172.21.15.19:0/110795485 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c4d1392c30 0x55c4d1395050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:37:28.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.816+0000 7fc900c00700 1 --2- 172.21.15.19:0/331863614 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc8e00b2a70 0x7fc8e00b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:28.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.816+0000 7fc8fbe00700 1 --2- 172.21.15.19:0/110795485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4d1389590 0x55c4d13926f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec0a6bd0 tx=0x7fc8ec009790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc7a1b5af74e47d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:37:28.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.816+0000 7fc8f1600700 1 -- 172.21.15.19:0/110795485 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec0e2e60 con 0x55c4d1389590 2024-05-07T02:37:28.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.816+0000 7fc8f1600700 1 -- 172.21.15.19:0/110795485 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8ec0b20f0 con 0x55c4d1389590 2024-05-07T02:37:28.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.816+0000 7fc8f1600700 1 -- 172.21.15.19:0/110795485 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec0a9e20 con 0x55c4d1389590 2024-05-07T02:37:28.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.817+0000 7fc911777b80 1 -- 172.21.15.19:0/110795485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4d1389590 msgr2=0x55c4d13926f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:28.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.817+0000 7fc911777b80 1 --2- 172.21.15.19:0/110795485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4d1389590 0x55c4d13926f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec0a6bd0 tx=0x7fc8ec009790 comp rx=0 tx=0).stop 2024-05-07T02:37:28.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.817+0000 7fc900c00700 1 --2- 172.21.15.19:0/331863614 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc8e00b2a70 0x7fc8e00b4f80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x55c4d137e190 tx=0x7fc8f401e040 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:37:28.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.817+0000 7fc911777b80 1 -- 172.21.15.19:0/110795485 shutdown_connections 2024-05-07T02:37:28.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.817+0000 7fc911777b80 1 --2- 172.21.15.19:0/110795485 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c4d1392c30 0x55c4d1395050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:28.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.817+0000 7fc911777b80 1 --2- 172.21.15.19:0/110795485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4d1388c40 0x55c4d1389050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:28.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.817+0000 7fc911777b80 1 --2- 172.21.15.19:0/110795485 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4d1389590 0x55c4d13926f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:28.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.817+0000 7fc911777b80 1 -- 172.21.15.19:0/110795485 >> 172.21.15.19:0/110795485 conn(0x55c4d120ae40 msgr2=0x55c4d1383980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:37:28.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.817+0000 7fc911777b80 1 -- 172.21.15.19:0/110795485 shutdown_connections 2024-05-07T02:37:28.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.817+0000 7fc911777b80 1 -- 172.21.15.19:0/110795485 wait complete. 2024-05-07T02:37:28.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.818+0000 7fc911777b80 1 -- start start 2024-05-07T02:37:28.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.818+0000 7fc911777b80 1 Processor -- start 2024-05-07T02:37:28.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.819+0000 7fc911777b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c4d1388c40 0x55c4d138fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:28.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.819+0000 7fc911777b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4d1389590 0x55c4d13900e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:28.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.819+0000 7fc911777b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4d1392c30 0x55c4d1390620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:28.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.819+0000 7fc911777b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c4d1395940 con 0x55c4d1389590 2024-05-07T02:37:28.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.819+0000 7fc911777b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c4d1210b20 con 0x55c4d1388c40 2024-05-07T02:37:28.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.819+0000 7fc911777b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c4d1395ac0 con 0x55c4d1392c30 2024-05-07T02:37:28.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.819+0000 7fc911777b80 10 client.0 _ll_register_callbacks cb 0x55c4d123e380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:37:28.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.819+0000 7fc8f3e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4d1389590 0x55c4d13900e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:28.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.819+0000 7fc900c00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4d1392c30 0x55c4d1390620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:28.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.819+0000 7fc8fbe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c4d1388c40 0x55c4d138fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:28.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.820+0000 7fc8f3e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4d1389590 0x55c4d13900e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39378/0 (socket says 172.21.15.19:39378) 2024-05-07T02:37:28.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.820+0000 7fc8f3e00700 1 -- 172.21.15.19:0/731716557 learned_addr learned my addr 172.21.15.19:0/731716557 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:37:28.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.819+0000 7fc8fbe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c4d1388c40 0x55c4d138fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:45382/0 (socket says 172.21.15.19:45382) 2024-05-07T02:37:28.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.820+0000 7fc900c00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4d1392c30 0x55c4d1390620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:45562/0 (socket says 172.21.15.19:45562) 2024-05-07T02:37:28.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.820+0000 7fc900c00700 1 -- 172.21.15.19:0/731716557 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c4d1388c40 msgr2=0x55c4d138fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:28.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.820+0000 7fc900c00700 1 --2- 172.21.15.19:0/731716557 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c4d1388c40 0x55c4d138fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:28.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.820+0000 7fc900c00700 1 -- 172.21.15.19:0/731716557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4d1389590 msgr2=0x55c4d13900e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:28.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.820+0000 7fc900c00700 1 --2- 172.21.15.19:0/731716557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4d1389590 0x55c4d13900e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:28.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.820+0000 7fc900c00700 1 -- 172.21.15.19:0/731716557 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4d1270420 con 0x55c4d1392c30 2024-05-07T02:37:28.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.820+0000 7fc8fbe00700 1 --2- 172.21.15.19:0/731716557 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c4d1388c40 0x55c4d138fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:37:28.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.820+0000 7fc8f3e00700 1 --2- 172.21.15.19:0/731716557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4d1389590 0x55c4d13900e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:37:28.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.820+0000 7fc900c00700 1 --2- 172.21.15.19:0/731716557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4d1392c30 0x55c4d1390620 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400b780 tx=0x7fc8f4009880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a3de5a3c269b72a server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:37:28.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.820+0000 7fc8ea000700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:37:28.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.821+0000 7fc8ea000700 1 -- 172.21.15.19:0/731716557 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4025670 con 0x55c4d1392c30 2024-05-07T02:37:28.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.821+0000 7fc911777b80 1 -- 172.21.15.19:0/731716557 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8ec0b20f0 con 0x55c4d1392c30 2024-05-07T02:37:28.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.821+0000 7fc8ea000700 1 -- 172.21.15.19:0/731716557 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fc8f4025810 con 0x55c4d1392c30 2024-05-07T02:37:28.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.821+0000 7fc8ea000700 1 -- 172.21.15.19:0/731716557 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc8f4025a10 con 0x55c4d1392c30 2024-05-07T02:37:28.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.821+0000 7fc911777b80 10 client.26125 fetch_fsmap learned FSMap version 40 2024-05-07T02:37:28.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.821+0000 7fc911777b80 1 -- 172.21.15.19:0/731716557 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55c4d13a05c0 con 0x55c4d1392c30 2024-05-07T02:37:28.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.821+0000 7fc8ea000700 1 -- 172.21.15.19:0/731716557 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8ec0b20f0 con 0x55c4d1392c30 2024-05-07T02:37:28.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.822+0000 7fc8ea000700 1 -- 172.21.15.19:0/731716557 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7fc8f402e070 con 0x55c4d1392c30 2024-05-07T02:37:28.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.822+0000 7fc911777b80 10 client.26125 fetch_fsmap finished waiting for FSMap version 40 2024-05-07T02:37:28.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.822+0000 7fc911777b80 10 client.26125 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:37:28.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.822+0000 7fc911777b80 20 client.26125 populate_metadata read hostname 'smithi019' 2024-05-07T02:37:28.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.822+0000 7fc911777b80 1 --2- 172.21.15.19:0/731716557 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55c4d13a2b50 0x55c4d13a4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:28.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.822+0000 7fc911777b80 4 client.26125 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:37:28.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.822+0000 7fc911777b80 1 -- 172.21.15.19:0/731716557 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4d13550e0 con 0x55c4d13a2b50 2024-05-07T02:37:28.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.822+0000 7fc8fbe00700 1 --2- 172.21.15.19:0/731716557 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55c4d13a2b50 0x55c4d13a4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:28.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.823+0000 7fc8fbe00700 1 --2- 172.21.15.19:0/731716557 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55c4d13a2b50 0x55c4d13a4f90 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac33160f6b110cce server_cookie=f2cd9788e3fcb3cb in_seq=0 out_seq=0 2024-05-07T02:37:28.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.823+0000 7fc8ea000700 10 client.26125 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:37:28.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.824+0000 7fc8ea000700 1 -- 172.21.15.19:0/731716557 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c4d13550e0 con 0x55c4d13a2b50 2024-05-07T02:37:28.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.824+0000 7fc8ea000700 10 client.26125 handle_command_reply: tid=0 2024-05-07T02:37:28.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.824+0000 7fc911777b80 1 client.26125 shutdown 2024-05-07T02:37:28.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.824+0000 7fc911777b80 1 -- 172.21.15.19:0/731716557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4d1392c30 msgr2=0x55c4d1390620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:28.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.824+0000 7fc911777b80 1 --2- 172.21.15.19:0/731716557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4d1392c30 0x55c4d1390620 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400b780 tx=0x7fc8f4009880 comp rx=0 tx=0).stop 2024-05-07T02:37:28.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.824+0000 7fc911777b80 1 -- 172.21.15.19:0/731716557 shutdown_connections 2024-05-07T02:37:28.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.824+0000 7fc911777b80 1 --2- 172.21.15.19:0/731716557 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55c4d13a2b50 0x55c4d13a4f90 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:28.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.824+0000 7fc911777b80 1 --2- 172.21.15.19:0/731716557 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4d1392c30 0x55c4d1390620 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:28.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.824+0000 7fc911777b80 1 --2- 172.21.15.19:0/731716557 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c4d1388c40 0x55c4d138fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:28.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.824+0000 7fc911777b80 1 --2- 172.21.15.19:0/731716557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4d1389590 0x55c4d13900e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:28.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.824+0000 7fc8ea000700 0 client.26125 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:37:28.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.824+0000 7fc911777b80 1 -- 172.21.15.19:0/731716557 >> 172.21.15.19:0/731716557 conn(0x55c4d120ae40 msgr2=0x55c4d1385600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:37:28.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.825+0000 7fc911777b80 1 -- 172.21.15.19:0/731716557 shutdown_connections 2024-05-07T02:37:28.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.825+0000 7fc911777b80 1 -- 172.21.15.19:0/731716557 wait complete. 2024-05-07T02:37:28.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.825+0000 7fc911777b80 20 client.26125 trim_cache size 0 max 16384 2024-05-07T02:37:28.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.848+0000 7fc911777b80 1 Processor -- start 2024-05-07T02:37:28.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.848+0000 7fc911777b80 1 -- start start 2024-05-07T02:37:28.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.849+0000 7fc911777b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4d120ae40 0x55c4d1390460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:28.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.849+0000 7fc911777b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c4d1388c40 0x55c4d1387d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:28.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.849+0000 7fc911777b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4d1389590 0x55c4d139ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:28.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.849+0000 7fc911777b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc8fc1bf2c0 con 0x55c4d120ae40 2024-05-07T02:37:28.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.849+0000 7fc911777b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc8fc1c0e60 con 0x55c4d1388c40 2024-05-07T02:37:28.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.849+0000 7fc911777b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc8fc1bf990 con 0x55c4d1389590 2024-05-07T02:37:28.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.849+0000 7fc8f3e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c4d1388c40 0x55c4d1387d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:28.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.849+0000 7fc8fbe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4d120ae40 0x55c4d1390460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:28.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.849+0000 7fc8f3e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c4d1388c40 0x55c4d1387d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:35844/0 (socket says 172.21.15.19:35844) 2024-05-07T02:37:28.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.849+0000 7fc8f3e00700 1 -- 172.21.15.19:0/587966986 learned_addr learned my addr 172.21.15.19:0/587966986 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:37:28.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.849+0000 7fc900c00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4d1389590 0x55c4d139ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:28.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.850+0000 7fc900c00700 1 -- 172.21.15.19:0/587966986 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c4d1388c40 msgr2=0x55c4d1387d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:28.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.850+0000 7fc900c00700 1 --2- 172.21.15.19:0/587966986 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c4d1388c40 0x55c4d1387d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:28.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.850+0000 7fc900c00700 1 -- 172.21.15.19:0/587966986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4d120ae40 msgr2=0x55c4d1390460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:28.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.850+0000 7fc900c00700 1 --2- 172.21.15.19:0/587966986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4d120ae40 0x55c4d1390460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:28.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.850+0000 7fc900c00700 1 -- 172.21.15.19:0/587966986 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4d137f730 con 0x55c4d1389590 2024-05-07T02:37:28.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.850+0000 7fc8f3e00700 1 --2- 172.21.15.19:0/587966986 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c4d1388c40 0x55c4d1387d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:37:28.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.850+0000 7fc900c00700 1 --2- 172.21.15.19:0/587966986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4d1389590 0x55c4d139ea30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4030bc0 tx=0x7fc8f40099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b9eda9fa5cc24c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:37:28.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.850+0000 7fc8f0c00700 1 -- 172.21.15.19:0/587966986 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4025db0 con 0x55c4d1389590 2024-05-07T02:37:28.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.850+0000 7fc8f0c00700 1 -- 172.21.15.19:0/587966986 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8f4030440 con 0x55c4d1389590 2024-05-07T02:37:28.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.850+0000 7fc8f0c00700 1 -- 172.21.15.19:0/587966986 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f40054d0 con 0x55c4d1389590 2024-05-07T02:37:28.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.851+0000 7fc911777b80 1 -- 172.21.15.19:0/587966986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4d1389590 msgr2=0x55c4d139ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:28.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.851+0000 7fc911777b80 1 --2- 172.21.15.19:0/587966986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4d1389590 0x55c4d139ea30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4030bc0 tx=0x7fc8f40099e0 comp rx=0 tx=0).stop 2024-05-07T02:37:28.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.851+0000 7fc911777b80 1 -- 172.21.15.19:0/587966986 shutdown_connections 2024-05-07T02:37:28.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.851+0000 7fc911777b80 1 --2- 172.21.15.19:0/587966986 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4d1389590 0x55c4d139ea30 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:28.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.851+0000 7fc911777b80 1 --2- 172.21.15.19:0/587966986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4d120ae40 0x55c4d1390460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:28.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.852+0000 7fc911777b80 1 --2- 172.21.15.19:0/587966986 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c4d1388c40 0x55c4d1387d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:28.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.852+0000 7fc911777b80 1 -- 172.21.15.19:0/587966986 >> 172.21.15.19:0/587966986 conn(0x55c4d13953b0 msgr2=0x55c4d1387610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:37:28.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.852+0000 7fc911777b80 1 -- 172.21.15.19:0/587966986 shutdown_connections 2024-05-07T02:37:28.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.852+0000 7fc911777b80 1 -- 172.21.15.19:0/587966986 wait complete. 2024-05-07T02:37:28.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.852+0000 7fc911777b80 1 -- start start 2024-05-07T02:37:28.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.853+0000 7fc911777b80 1 Processor -- start 2024-05-07T02:37:28.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.853+0000 7fc911777b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4d120ae40 0x55c4d139afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:28.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.853+0000 7fc911777b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4d1388c40 0x55c4d1396330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:28.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.853+0000 7fc911777b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c4d1389590 0x55c4d1396870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:28.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.853+0000 7fc911777b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc8fc1c0790 con 0x55c4d1388c40 2024-05-07T02:37:28.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.853+0000 7fc911777b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc8fc1c00c0 con 0x55c4d1389590 2024-05-07T02:37:28.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.853+0000 7fc911777b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc8fc1bd260 con 0x55c4d120ae40 2024-05-07T02:37:28.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.853+0000 7fc911777b80 10 client.0 _ll_register_callbacks cb 0x55c4d123e380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:37:28.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.854+0000 7fc900c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c4d1389590 0x55c4d1396870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:28.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.854+0000 7fc8fbe00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4d120ae40 0x55c4d139afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:28.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.854+0000 7fc900c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c4d1389590 0x55c4d1396870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:35856/0 (socket says 172.21.15.19:35856) 2024-05-07T02:37:28.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.854+0000 7fc8fbe00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4d120ae40 0x55c4d139afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:57436/0 (socket says 172.21.15.19:57436) 2024-05-07T02:37:28.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.854+0000 7fc900c00700 1 -- 172.21.15.19:0/3499875702 learned_addr learned my addr 172.21.15.19:0/3499875702 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:37:28.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.854+0000 7fc8f3e00700 1 --2- 172.21.15.19:0/3499875702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4d1388c40 0x55c4d1396330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:28.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.854+0000 7fc900c00700 1 -- 172.21.15.19:0/3499875702 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4d120ae40 msgr2=0x55c4d139afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:28.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.854+0000 7fc900c00700 1 --2- 172.21.15.19:0/3499875702 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4d120ae40 0x55c4d139afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:28.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.854+0000 7fc900c00700 1 -- 172.21.15.19:0/3499875702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4d1388c40 msgr2=0x55c4d1396330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:37:28.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.854+0000 7fc900c00700 1 --2- 172.21.15.19:0/3499875702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4d1388c40 0x55c4d1396330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:28.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.854+0000 7fc900c00700 1 -- 172.21.15.19:0/3499875702 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4d1396db0 con 0x55c4d1389590 2024-05-07T02:37:28.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.854+0000 7fc8f3e00700 1 --2- 172.21.15.19:0/3499875702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4d1388c40 0x55c4d1396330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:37:28.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.854+0000 7fc8fbe00700 1 --2- 172.21.15.19:0/3499875702 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4d120ae40 0x55c4d139afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:37:28.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.855+0000 7fc900c00700 1 --2- 172.21.15.19:0/3499875702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c4d1389590 0x55c4d1396870 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc8f40259e0 tx=0x7fc8f4030820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4a021ad27880979 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:37:28.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.855+0000 7fc8ea000700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:37:28.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.855+0000 7fc8ea000700 1 -- 172.21.15.19:0/3499875702 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f404c020 con 0x55c4d1389590 2024-05-07T02:37:28.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.855+0000 7fc911777b80 1 -- 172.21.15.19:0/3499875702 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8f4030440 con 0x55c4d1389590 2024-05-07T02:37:28.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.855+0000 7fc8ea000700 1 -- 172.21.15.19:0/3499875702 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fc8f4005150 con 0x55c4d1389590 2024-05-07T02:37:28.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.855+0000 7fc8ea000700 1 -- 172.21.15.19:0/3499875702 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc8f4057050 con 0x55c4d1389590 2024-05-07T02:37:28.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.855+0000 7fc911777b80 10 client.26162 fetch_fsmap learned FSMap version 40 2024-05-07T02:37:28.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.855+0000 7fc911777b80 1 -- 172.21.15.19:0/3499875702 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55c4d140ced0 con 0x55c4d1389590 2024-05-07T02:37:28.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.856+0000 7fc8ea000700 1 -- 172.21.15.19:0/3499875702 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8f4030440 con 0x55c4d1389590 2024-05-07T02:37:28.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.856+0000 7fc8ea000700 1 -- 172.21.15.19:0/3499875702 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7fc8f402e040 con 0x55c4d1389590 2024-05-07T02:37:28.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.856+0000 7fc911777b80 10 client.26162 fetch_fsmap finished waiting for FSMap version 40 2024-05-07T02:37:28.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.856+0000 7fc911777b80 10 client.26162 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:37:28.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.856+0000 7fc911777b80 20 client.26162 populate_metadata read hostname 'smithi019' 2024-05-07T02:37:28.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.856+0000 7fc911777b80 1 --2- 172.21.15.19:0/3499875702 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55c4d1392c30 0x55c4d140f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:37:28.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.856+0000 7fc911777b80 4 client.26162 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-07T02:37:28.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.856+0000 7fc911777b80 1 -- 172.21.15.19:0/3499875702 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c4d12b8400 con 0x55c4d1392c30 2024-05-07T02:37:28.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.856+0000 7fc8fbe00700 1 --2- 172.21.15.19:0/3499875702 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55c4d1392c30 0x55c4d140f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:37:28.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.857+0000 7fc8fbe00700 1 --2- 172.21.15.19:0/3499875702 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55c4d1392c30 0x55c4d140f5c0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6a7a411956e30b3 server_cookie=b06d513ff02ffea9 in_seq=0 out_seq=0 2024-05-07T02:37:28.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.857+0000 7fc8ea000700 10 client.26162 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:37:28.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.857+0000 7fc8ea000700 1 -- 172.21.15.19:0/3499875702 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c4d12b8400 con 0x55c4d1392c30 2024-05-07T02:37:28.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.857+0000 7fc8ea000700 10 client.26162 handle_command_reply: tid=0 2024-05-07T02:37:28.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.857+0000 7fc911777b80 1 client.26162 shutdown 2024-05-07T02:37:28.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.857+0000 7fc911777b80 1 -- 172.21.15.19:0/3499875702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c4d1389590 msgr2=0x55c4d1396870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:28.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.857+0000 7fc911777b80 1 --2- 172.21.15.19:0/3499875702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c4d1389590 0x55c4d1396870 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc8f40259e0 tx=0x7fc8f4030820 comp rx=0 tx=0).stop 2024-05-07T02:37:28.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.857+0000 7fc911777b80 1 -- 172.21.15.19:0/3499875702 shutdown_connections 2024-05-07T02:37:28.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.857+0000 7fc911777b80 1 --2- 172.21.15.19:0/3499875702 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55c4d1392c30 0x55c4d140f5c0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:28.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.857+0000 7fc911777b80 1 --2- 172.21.15.19:0/3499875702 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c4d1389590 0x55c4d1396870 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:28.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.857+0000 7fc911777b80 1 --2- 172.21.15.19:0/3499875702 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4d120ae40 0x55c4d139afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:28.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.857+0000 7fc911777b80 1 --2- 172.21.15.19:0/3499875702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c4d1388c40 0x55c4d1396330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:28.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.857+0000 7fc8ea000700 0 client.26162 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:37:28.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.858+0000 7fc911777b80 1 -- 172.21.15.19:0/3499875702 >> 172.21.15.19:0/3499875702 conn(0x55c4d13953b0 msgr2=0x55c4d13911e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:37:28.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.858+0000 7fc911777b80 1 -- 172.21.15.19:0/3499875702 shutdown_connections 2024-05-07T02:37:28.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.858+0000 7fc911777b80 1 -- 172.21.15.19:0/3499875702 wait complete. 2024-05-07T02:37:28.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.858+0000 7fc911777b80 20 client.26162 trim_cache size 0 max 16384 2024-05-07T02:37:28.860 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:37:28.860 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-07T02:37:28.860 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-05-07T02:37:28.860 INFO:teuthology.orchestra.run.smithi019.stdout: "9a7bb2fc-c87b-4549-8559-ac79c77de966": { 2024-05-07T02:37:28.860 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-05-07T02:37:28.860 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "9a7bb2fc-c87b-4549-8559-ac79c77de966", 2024-05-07T02:37:28.860 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-05-07T02:37:28.860 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:37:28.861 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:37:28.861 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:37:28.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.859+0000 7fc901600700 1 -- 172.21.15.19:0/331863614 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc8e00b2a70 msgr2=0x7fc8e00b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:28.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.859+0000 7fc901600700 1 --2- 172.21.15.19:0/331863614 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc8e00b2a70 0x7fc8e00b4f80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x55c4d137e190 tx=0x7fc8f401e040 comp rx=0 tx=0).stop 2024-05-07T02:37:28.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.859+0000 7fc901600700 1 -- 172.21.15.19:0/331863614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8fc06d4e0 msgr2=0x7fc8fc10c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:37:28.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.859+0000 7fc901600700 1 --2- 172.21.15.19:0/331863614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8fc06d4e0 0x7fc8fc10c8c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec00e9f0 tx=0x7fc8ec00ed00 comp rx=0 tx=0).stop 2024-05-07T02:37:28.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.859+0000 7fc901600700 1 -- 172.21.15.19:0/331863614 shutdown_connections 2024-05-07T02:37:28.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.859+0000 7fc901600700 1 --2- 172.21.15.19:0/331863614 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8fc1191a0 0x7fc8fc10d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:28.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.859+0000 7fc901600700 1 --2- 172.21.15.19:0/331863614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc8fc06d4e0 0x7fc8fc10c8c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:28.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.859+0000 7fc901600700 1 --2- 172.21.15.19:0/331863614 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc8e00b2a70 0x7fc8e00b4f80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:28.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.859+0000 7fc901600700 1 --2- 172.21.15.19:0/331863614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8fc06de30 0x7fc8fc10ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:37:28.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.859+0000 7fc901600700 1 -- 172.21.15.19:0/331863614 >> 172.21.15.19:0/331863614 conn(0x7fc8fc0704c0 msgr2=0x7fc8fc119d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:37:28.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.859+0000 7fc901600700 1 -- 172.21.15.19:0/331863614 shutdown_connections 2024-05-07T02:37:28.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:37:28.859+0000 7fc901600700 1 -- 172.21.15.19:0/331863614 wait complete. 2024-05-07T02:37:29.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:29 smithi027 ceph-mon[34398]: pgmap v172: 129 pgs: 129 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 156 op/s 2024-05-07T02:37:29.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:29 smithi064 ceph-mon[34699]: pgmap v172: 129 pgs: 129 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 156 op/s 2024-05-07T02:37:29.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:29 smithi019 ceph-mon[29781]: pgmap v172: 129 pgs: 129 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 156 op/s 2024-05-07T02:37:29.756 INFO:tasks.workunit.client.0.smithi019.stdout: 1 184857 4.14 MB/sec execute 145 sec latency 28.432 ms 2024-05-07T02:37:29.876 INFO:tasks.cephfs.filesystem:scrub status for tag:9a7bb2fc-c87b-4549-8559-ac79c77de966 - {'path': '/', 'tag': '9a7bb2fc-c87b-4549-8559-ac79c77de966', 'options': 'recursive,force'} 2024-05-07T02:37:30.757 INFO:tasks.workunit.client.0.smithi019.stdout: 1 185370 4.12 MB/sec execute 146 sec latency 24.526 ms 2024-05-07T02:37:31.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:31 smithi027 ceph-mon[34398]: pgmap v173: 129 pgs: 129 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 156 op/s 2024-05-07T02:37:31.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:31 smithi064 ceph-mon[34699]: pgmap v173: 129 pgs: 129 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 156 op/s 2024-05-07T02:37:31.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:31 smithi019 ceph-mon[29781]: pgmap v173: 129 pgs: 129 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 156 op/s 2024-05-07T02:37:31.757 INFO:tasks.workunit.client.0.smithi019.stdout: 1 186668 4.13 MB/sec execute 147 sec latency 19.919 ms 2024-05-07T02:37:32.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:32 smithi064 ceph-mon[34699]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:37:32.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:32 smithi019 ceph-mon[29781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:37:32.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:32 smithi027 ceph-mon[34398]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:37:32.757 INFO:tasks.workunit.client.0.smithi019.stdout: 1 187750 4.16 MB/sec execute 148 sec latency 67.077 ms 2024-05-07T02:37:33.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:33 smithi019 ceph-mon[29781]: pgmap v174: 129 pgs: 129 active+clean; 130 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 235 op/s 2024-05-07T02:37:33.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:33 smithi064 ceph-mon[34699]: pgmap v174: 129 pgs: 129 active+clean; 130 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 235 op/s 2024-05-07T02:37:33.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:33 smithi027 ceph-mon[34398]: pgmap v174: 129 pgs: 129 active+clean; 130 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 235 op/s 2024-05-07T02:37:33.757 INFO:tasks.workunit.client.0.smithi019.stdout: 1 188399 4.16 MB/sec execute 149 sec latency 22.975 ms 2024-05-07T02:37:34.507 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:34 smithi019 ceph-mon[29781]: scrub summary: idle 2024-05-07T02:37:34.507 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:34 smithi019 ceph-mon[29781]: scrub summary: idle 2024-05-07T02:37:34.507 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:34 smithi019 ceph-mon[29781]: scrub summary: idle 2024-05-07T02:37:34.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:34 smithi064 ceph-mon[34699]: scrub summary: idle 2024-05-07T02:37:34.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:34 smithi064 ceph-mon[34699]: scrub summary: idle 2024-05-07T02:37:34.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:34 smithi064 ceph-mon[34699]: scrub summary: idle 2024-05-07T02:37:34.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:34 smithi027 ceph-mon[34398]: scrub summary: idle 2024-05-07T02:37:34.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:34 smithi027 ceph-mon[34398]: scrub summary: idle 2024-05-07T02:37:34.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:34 smithi027 ceph-mon[34398]: scrub summary: idle 2024-05-07T02:37:34.757 INFO:tasks.workunit.client.0.smithi019.stdout: 1 188985 4.14 MB/sec execute 150 sec latency 23.714 ms 2024-05-07T02:37:35.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:35 smithi064 ceph-mon[34699]: scrub summary: idle 2024-05-07T02:37:35.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:35 smithi064 ceph-mon[34699]: scrub summary: idle 2024-05-07T02:37:35.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:35 smithi064 ceph-mon[34699]: pgmap v175: 129 pgs: 129 active+clean; 130 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 160 op/s 2024-05-07T02:37:35.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:35 smithi019 ceph-mon[29781]: scrub summary: idle 2024-05-07T02:37:35.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:35 smithi019 ceph-mon[29781]: scrub summary: idle 2024-05-07T02:37:35.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:35 smithi019 ceph-mon[29781]: pgmap v175: 129 pgs: 129 active+clean; 130 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 160 op/s 2024-05-07T02:37:35.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:35 smithi027 ceph-mon[34398]: scrub summary: idle 2024-05-07T02:37:35.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:35 smithi027 ceph-mon[34398]: scrub summary: idle 2024-05-07T02:37:35.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:35 smithi027 ceph-mon[34398]: pgmap v175: 129 pgs: 129 active+clean; 130 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 160 op/s 2024-05-07T02:37:35.757 INFO:tasks.workunit.client.0.smithi019.stdout: 1 190397 4.15 MB/sec execute 151 sec latency 24.588 ms 2024-05-07T02:37:36.757 INFO:tasks.workunit.client.0.smithi019.stdout: 1 191556 4.19 MB/sec execute 152 sec latency 16.943 ms 2024-05-07T02:37:37.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:37 smithi064 ceph-mon[34699]: pgmap v176: 129 pgs: 129 active+clean; 133 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 228 op/s 2024-05-07T02:37:37.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:37 smithi019 ceph-mon[29781]: pgmap v176: 129 pgs: 129 active+clean; 133 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 228 op/s 2024-05-07T02:37:37.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:37 smithi027 ceph-mon[34398]: pgmap v176: 129 pgs: 129 active+clean; 133 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 228 op/s 2024-05-07T02:37:37.757 INFO:tasks.workunit.client.0.smithi019.stdout: 1 191990 4.18 MB/sec execute 153 sec latency 36.780 ms 2024-05-07T02:37:38.757 INFO:tasks.workunit.client.0.smithi019.stdout: 1 192682 4.16 MB/sec execute 154 sec latency 26.243 ms 2024-05-07T02:37:39.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:39 smithi064 ceph-mon[34699]: pgmap v177: 129 pgs: 129 active+clean; 133 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 147 op/s 2024-05-07T02:37:39.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:39 smithi019 ceph-mon[29781]: pgmap v177: 129 pgs: 129 active+clean; 133 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 147 op/s 2024-05-07T02:37:39.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:39 smithi027 ceph-mon[34398]: pgmap v177: 129 pgs: 129 active+clean; 133 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 147 op/s 2024-05-07T02:37:39.758 INFO:tasks.workunit.client.0.smithi019.stdout: 1 193839 4.17 MB/sec execute 155 sec latency 20.292 ms 2024-05-07T02:37:40.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:40 smithi064 ceph-mon[34699]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:37:40.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:40 smithi019 ceph-mon[29781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:37:40.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:40 smithi027 ceph-mon[34398]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:37:40.758 INFO:tasks.workunit.client.0.smithi019.stdout: 1 194924 4.19 MB/sec execute 156 sec latency 19.665 ms 2024-05-07T02:37:41.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:41 smithi064 ceph-mon[34699]: pgmap v178: 129 pgs: 129 active+clean; 133 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 147 op/s 2024-05-07T02:37:41.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:41 smithi019 ceph-mon[29781]: pgmap v178: 129 pgs: 129 active+clean; 133 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 147 op/s 2024-05-07T02:37:41.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:41 smithi027 ceph-mon[34398]: pgmap v178: 129 pgs: 129 active+clean; 133 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 147 op/s 2024-05-07T02:37:41.758 INFO:tasks.workunit.client.0.smithi019.stdout: 1 195481 4.19 MB/sec execute 157 sec latency 25.881 ms 2024-05-07T02:37:42.758 INFO:tasks.workunit.client.0.smithi019.stdout: 1 195902 4.17 MB/sec execute 158 sec latency 57.357 ms 2024-05-07T02:37:43.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:43 smithi064 ceph-mon[34699]: pgmap v179: 129 pgs: 129 active+clean; 136 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 219 op/s 2024-05-07T02:37:43.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:43 smithi019 ceph-mon[29781]: pgmap v179: 129 pgs: 129 active+clean; 136 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 219 op/s 2024-05-07T02:37:43.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:43 smithi027 ceph-mon[34398]: pgmap v179: 129 pgs: 129 active+clean; 136 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 219 op/s 2024-05-07T02:37:43.758 INFO:tasks.workunit.client.0.smithi019.stdout: 1 197052 4.17 MB/sec execute 159 sec latency 23.606 ms 2024-05-07T02:37:44.758 INFO:tasks.workunit.client.0.smithi019.stdout: 1 198301 4.20 MB/sec execute 160 sec latency 20.562 ms 2024-05-07T02:37:45.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:45 smithi064 ceph-mon[34699]: pgmap v180: 129 pgs: 129 active+clean; 136 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-05-07T02:37:45.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:45 smithi019 ceph-mon[29781]: pgmap v180: 129 pgs: 129 active+clean; 136 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-05-07T02:37:45.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:45 smithi027 ceph-mon[34398]: pgmap v180: 129 pgs: 129 active+clean; 136 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-05-07T02:37:45.758 INFO:tasks.workunit.client.0.smithi019.stdout: 1 198966 4.20 MB/sec execute 161 sec latency 23.333 ms 2024-05-07T02:37:46.760 INFO:tasks.workunit.client.0.smithi019.stdout: 1 199444 4.18 MB/sec execute 162 sec latency 34.158 ms 2024-05-07T02:37:47.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:47 smithi064 ceph-mon[34699]: pgmap v181: 129 pgs: 129 active+clean; 138 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 207 op/s 2024-05-07T02:37:47.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:47 smithi064 ceph-mon[34699]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:37:47.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:47 smithi019 ceph-mon[29781]: pgmap v181: 129 pgs: 129 active+clean; 138 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 207 op/s 2024-05-07T02:37:47.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:47 smithi019 ceph-mon[29781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:37:47.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:47 smithi027 ceph-mon[34398]: pgmap v181: 129 pgs: 129 active+clean; 138 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 207 op/s 2024-05-07T02:37:47.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:47 smithi027 ceph-mon[34398]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:37:47.759 INFO:tasks.workunit.client.0.smithi019.stdout: 1 200558 4.19 MB/sec execute 163 sec latency 30.838 ms 2024-05-07T02:37:48.759 INFO:tasks.workunit.client.0.smithi019.stdout: 1 201846 4.21 MB/sec execute 164 sec latency 22.045 ms 2024-05-07T02:37:49.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:49 smithi064 ceph-mon[34699]: pgmap v182: 129 pgs: 129 active+clean; 138 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-05-07T02:37:49.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:49 smithi019 ceph-mon[29781]: pgmap v182: 129 pgs: 129 active+clean; 138 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-05-07T02:37:49.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:49 smithi027 ceph-mon[34398]: pgmap v182: 129 pgs: 129 active+clean; 138 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-05-07T02:37:49.759 INFO:tasks.workunit.client.0.smithi019.stdout: 1 202494 4.22 MB/sec execute 165 sec latency 25.138 ms 2024-05-07T02:37:50.759 INFO:tasks.workunit.client.0.smithi019.stdout: 1 202788 4.20 MB/sec execute 166 sec latency 28.490 ms 2024-05-07T02:37:51.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:51 smithi064 ceph-mon[34699]: pgmap v183: 129 pgs: 129 active+clean; 138 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-05-07T02:37:51.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:51 smithi019 ceph-mon[29781]: pgmap v183: 129 pgs: 129 active+clean; 138 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-05-07T02:37:51.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:51 smithi027 ceph-mon[34398]: pgmap v183: 129 pgs: 129 active+clean; 138 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-05-07T02:37:51.759 INFO:tasks.workunit.client.0.smithi019.stdout: 1 203705 4.20 MB/sec execute 167 sec latency 24.447 ms 2024-05-07T02:37:52.759 INFO:tasks.workunit.client.0.smithi019.stdout: 1 204947 4.21 MB/sec execute 168 sec latency 80.892 ms 2024-05-07T02:37:53.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:53 smithi064 ceph-mon[34699]: pgmap v184: 129 pgs: 129 active+clean; 140 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 205 op/s 2024-05-07T02:37:53.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:53 smithi019 ceph-mon[29781]: pgmap v184: 129 pgs: 129 active+clean; 140 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 205 op/s 2024-05-07T02:37:53.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:53 smithi027 ceph-mon[34398]: pgmap v184: 129 pgs: 129 active+clean; 140 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 205 op/s 2024-05-07T02:37:53.759 INFO:tasks.workunit.client.0.smithi019.stdout: 1 205941 4.23 MB/sec execute 169 sec latency 64.017 ms 2024-05-07T02:37:54.760 INFO:tasks.workunit.client.0.smithi019.stdout: 1 206183 4.21 MB/sec execute 170 sec latency 36.821 ms 2024-05-07T02:37:55.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:55 smithi019 ceph-mon[29781]: pgmap v185: 129 pgs: 129 active+clean; 140 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 134 op/s 2024-05-07T02:37:55.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:55 smithi019 ceph-mon[29781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:37:55.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:55 smithi027 ceph-mon[34398]: pgmap v185: 129 pgs: 129 active+clean; 140 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 134 op/s 2024-05-07T02:37:55.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:55 smithi027 ceph-mon[34398]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:37:55.760 INFO:tasks.workunit.client.0.smithi019.stdout: 1 206802 4.20 MB/sec execute 171 sec latency 23.208 ms 2024-05-07T02:37:55.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:55 smithi064 ceph-mon[34699]: pgmap v185: 129 pgs: 129 active+clean; 140 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 134 op/s 2024-05-07T02:37:55.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:55 smithi064 ceph-mon[34699]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:37:56.760 INFO:tasks.workunit.client.0.smithi019.stdout: 1 207421 4.19 MB/sec execute 172 sec latency 278.149 ms 2024-05-07T02:37:57.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:57 smithi019 ceph-mon[29781]: pgmap v186: 129 pgs: 129 active+clean; 143 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 206 op/s 2024-05-07T02:37:57.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:57 smithi027 ceph-mon[34398]: pgmap v186: 129 pgs: 129 active+clean; 143 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 206 op/s 2024-05-07T02:37:57.760 INFO:tasks.workunit.client.0.smithi019.stdout: 1 207421 4.17 MB/sec execute 173 sec latency 1278.356 ms 2024-05-07T02:37:57.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:57 smithi064 ceph-mon[34699]: pgmap v186: 129 pgs: 129 active+clean; 143 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 206 op/s 2024-05-07T02:37:58.760 INFO:tasks.workunit.client.0.smithi019.stdout: 1 207421 4.14 MB/sec execute 174 sec latency 2278.602 ms 2024-05-07T02:37:59.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:37:59 smithi019 ceph-mon[29781]: pgmap v187: 129 pgs: 129 active+clean; 143 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-05-07T02:37:59.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:37:59 smithi027 ceph-mon[34398]: pgmap v187: 129 pgs: 129 active+clean; 143 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-05-07T02:37:59.760 INFO:tasks.workunit.client.0.smithi019.stdout: 1 207421 4.12 MB/sec execute 175 sec latency 3278.723 ms 2024-05-07T02:37:59.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:37:59 smithi064 ceph-mon[34699]: pgmap v187: 129 pgs: 129 active+clean; 143 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-05-07T02:37:59.877 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-07T02:38:00.373 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:38:02.291 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:01 smithi064 ceph-mon[34699]: pgmap v188: 129 pgs: 129 active+clean; 143 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-05-07T02:38:02.291 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:01 smithi027 ceph-mon[34398]: pgmap v188: 129 pgs: 129 active+clean; 143 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-05-07T02:38:02.292 INFO:tasks.workunit.client.0.smithi019.stdout: 1 207708 4.10 MB/sec execute 176 sec latency 4012.681 ms 2024-05-07T02:38:02.292 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:01 smithi019 ceph-mon[29781]: pgmap v188: 129 pgs: 129 active+clean; 143 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-05-07T02:38:02.292 INFO:tasks.workunit.client.0.smithi019.stdout: 1 208016 4.08 MB/sec execute 177 sec latency 101.265 ms 2024-05-07T02:38:02.761 INFO:tasks.workunit.client.0.smithi019.stdout: 1 208657 4.09 MB/sec execute 178 sec latency 99.351 ms 2024-05-07T02:38:03.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.033+0000 7fe31c400700 1 -- 172.21.15.19:0/490783991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe314103250 msgr2=0x7fe314103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:03.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.033+0000 7fe31c400700 1 --2- 172.21.15.19:0/490783991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe314103250 0x7fe314103660 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe3080098f0 tx=0x7fe308009c00 comp rx=0 tx=0).stop 2024-05-07T02:38:03.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.033+0000 7fe31c400700 1 -- 172.21.15.19:0/490783991 shutdown_connections 2024-05-07T02:38:03.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.033+0000 7fe31c400700 1 --2- 172.21.15.19:0/490783991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3141086a0 0x7fe314108b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:03.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.033+0000 7fe31c400700 1 --2- 172.21.15.19:0/490783991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe314103250 0x7fe314103660 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:03.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.033+0000 7fe31c400700 1 --2- 172.21.15.19:0/490783991 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe314103ba0 0x7fe3141080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:03.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.034+0000 7fe31c400700 1 -- 172.21.15.19:0/490783991 >> 172.21.15.19:0/490783991 conn(0x7fe3140fedc0 msgr2=0x7fe3141011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:03.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.034+0000 7fe31c400700 1 -- 172.21.15.19:0/490783991 shutdown_connections 2024-05-07T02:38:03.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.034+0000 7fe31c400700 1 -- 172.21.15.19:0/490783991 wait complete. 2024-05-07T02:38:03.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.035+0000 7fe31c400700 1 Processor -- start 2024-05-07T02:38:03.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.036+0000 7fe31c400700 1 -- start start 2024-05-07T02:38:03.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.036+0000 7fe31c400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe314103250 0x7fe3141a1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:03.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.036+0000 7fe31c400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe314103ba0 0x7fe3141a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:03.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.036+0000 7fe31c400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3141086a0 0x7fe3141a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:03.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.036+0000 7fe31c400700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe31411aa70 con 0x7fe314103ba0 2024-05-07T02:38:03.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.036+0000 7fe31c400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe31411a8f0 con 0x7fe3141086a0 2024-05-07T02:38:03.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.036+0000 7fe31c400700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe31411abf0 con 0x7fe314103250 2024-05-07T02:38:03.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.037+0000 7fe319a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe314103250 0x7fe3141a1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:03.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.037+0000 7fe319000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe314103ba0 0x7fe3141a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:03.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.037+0000 7fe319a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe314103250 0x7fe3141a1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:46406/0 (socket says 172.21.15.19:46406) 2024-05-07T02:38:03.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.037+0000 7fe319a00700 1 -- 172.21.15.19:0/2392484389 learned_addr learned my addr 172.21.15.19:0/2392484389 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:38:03.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.037+0000 7fe31a400700 1 --2- 172.21.15.19:0/2392484389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3141086a0 0x7fe3141a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:03.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.037+0000 7fe319a00700 1 -- 172.21.15.19:0/2392484389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3141086a0 msgr2=0x7fe3141a5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:03.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.037+0000 7fe319a00700 1 --2- 172.21.15.19:0/2392484389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3141086a0 0x7fe3141a5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:03.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.037+0000 7fe319a00700 1 -- 172.21.15.19:0/2392484389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe314103ba0 msgr2=0x7fe3141a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:03.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.037+0000 7fe319a00700 1 --2- 172.21.15.19:0/2392484389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe314103ba0 0x7fe3141a19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:03.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.037+0000 7fe319a00700 1 -- 172.21.15.19:0/2392484389 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3141a6310 con 0x7fe314103250 2024-05-07T02:38:03.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.037+0000 7fe31a400700 1 --2- 172.21.15.19:0/2392484389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3141086a0 0x7fe3141a5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:38:03.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.037+0000 7fe319000700 1 --2- 172.21.15.19:0/2392484389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe314103ba0 0x7fe3141a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:38:03.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.038+0000 7fe319a00700 1 --2- 172.21.15.19:0/2392484389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe314103250 0x7fe3141a1440 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe308009230 tx=0x7fe3080050f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:03.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.038+0000 7fe312000700 1 -- 172.21.15.19:0/2392484389 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3080052c0 con 0x7fe314103250 2024-05-07T02:38:03.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.038+0000 7fe31c400700 1 -- 172.21.15.19:0/2392484389 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3141a65a0 con 0x7fe314103250 2024-05-07T02:38:03.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.038+0000 7fe312000700 1 -- 172.21.15.19:0/2392484389 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe30800ba10 con 0x7fe314103250 2024-05-07T02:38:03.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.038+0000 7fe312000700 1 -- 172.21.15.19:0/2392484389 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe308003c00 con 0x7fe314103250 2024-05-07T02:38:03.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.038+0000 7fe31c400700 1 -- 172.21.15.19:0/2392484389 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3141a6a80 con 0x7fe314103250 2024-05-07T02:38:03.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.044+0000 7fe312000700 1 -- 172.21.15.19:0/2392484389 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe30802c440 con 0x7fe314103250 2024-05-07T02:38:03.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.044+0000 7fe32ae6ab80 1 Processor -- start 2024-05-07T02:38:03.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.045+0000 7fe32ae6ab80 1 -- start start 2024-05-07T02:38:03.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.045+0000 7fe32ae6ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610d23c03b0 0x5610d23c07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:03.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.045+0000 7fe32ae6ab80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610d23ba470 0x5610d23ba880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:03.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.045+0000 7fe32ae6ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d23badc0 0x5610d23c7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:03.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.045+0000 7fe32ae6ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5610d22dca20 con 0x5610d23badc0 2024-05-07T02:38:03.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.045+0000 7fe32ae6ab80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5610d227a760 con 0x5610d23ba470 2024-05-07T02:38:03.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.045+0000 7fe32ae6ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5610d22d89e0 con 0x5610d23c03b0 2024-05-07T02:38:03.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.045+0000 7fe319a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610d23ba470 0x5610d23ba880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:03.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.045+0000 7fe31a400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610d23c03b0 0x5610d23c07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:03.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.046+0000 7fe319a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610d23ba470 0x5610d23ba880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:45740/0 (socket says 172.21.15.19:45740) 2024-05-07T02:38:03.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.046+0000 7fe319a00700 1 -- 172.21.15.19:0/1805989571 learned_addr learned my addr 172.21.15.19:0/1805989571 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:38:03.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.046+0000 7fe319a00700 1 -- 172.21.15.19:0/1805989571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610d23c03b0 msgr2=0x5610d23c07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:03.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.046+0000 7fe319a00700 1 --2- 172.21.15.19:0/1805989571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610d23c03b0 0x5610d23c07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:03.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.046+0000 7fe319a00700 1 -- 172.21.15.19:0/1805989571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d23badc0 msgr2=0x5610d23c7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:03.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.046+0000 7fe319a00700 1 --2- 172.21.15.19:0/1805989571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d23badc0 0x5610d23c7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:03.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.046+0000 7fe319a00700 1 -- 172.21.15.19:0/1805989571 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610d22298d0 con 0x5610d23ba470 2024-05-07T02:38:03.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.047+0000 7fe319a00700 1 --2- 172.21.15.19:0/1805989571 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610d23ba470 0x5610d23ba880 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe3080e0eb0 tx=0x7fe3080e7bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3608862bfb200771 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:03.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.047+0000 7fe306a00700 1 -- 172.21.15.19:0/1805989571 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3080ee8b0 con 0x5610d23ba470 2024-05-07T02:38:03.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.047+0000 7fe306a00700 1 -- 172.21.15.19:0/1805989571 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3080ef480 con 0x5610d23ba470 2024-05-07T02:38:03.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.047+0000 7fe312000700 1 --2- 172.21.15.19:0/2392484389 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fe2fc0b29f0 0x7fe2fc0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:03.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.047+0000 7fe306a00700 1 -- 172.21.15.19:0/1805989571 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3080ec590 con 0x5610d23ba470 2024-05-07T02:38:03.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.047+0000 7fe312000700 1 -- 172.21.15.19:0/2392484389 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fe3080e3c10 con 0x7fe314103250 2024-05-07T02:38:03.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.048+0000 7fe31a400700 1 --2- 172.21.15.19:0/2392484389 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fe2fc0b29f0 0x7fe2fc0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:03.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.049+0000 7fe32ae6ab80 1 -- 172.21.15.19:0/1805989571 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610d23ba470 msgr2=0x5610d23ba880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:03.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.049+0000 7fe32ae6ab80 1 --2- 172.21.15.19:0/1805989571 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610d23ba470 0x5610d23ba880 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe3080e0eb0 tx=0x7fe3080e7bf0 comp rx=0 tx=0).stop 2024-05-07T02:38:03.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.049+0000 7fe31a400700 1 --2- 172.21.15.19:0/2392484389 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fe2fc0b29f0 0x7fe2fc0b4f00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x5610d23bfe70 tx=0x7fe30c009040 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:03.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.049+0000 7fe32ae6ab80 1 -- 172.21.15.19:0/1805989571 shutdown_connections 2024-05-07T02:38:03.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.049+0000 7fe32ae6ab80 1 --2- 172.21.15.19:0/1805989571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d23badc0 0x5610d23c7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:03.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.049+0000 7fe32ae6ab80 1 --2- 172.21.15.19:0/1805989571 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610d23c03b0 0x5610d23c07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:03.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.049+0000 7fe32ae6ab80 1 --2- 172.21.15.19:0/1805989571 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610d23ba470 0x5610d23ba880 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:03.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.049+0000 7fe32ae6ab80 1 -- 172.21.15.19:0/1805989571 >> 172.21.15.19:0/1805989571 conn(0x5610d223ce40 msgr2=0x5610d23b52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:03.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.050+0000 7fe32ae6ab80 1 -- 172.21.15.19:0/1805989571 shutdown_connections 2024-05-07T02:38:03.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.050+0000 7fe32ae6ab80 1 -- 172.21.15.19:0/1805989571 wait complete. 2024-05-07T02:38:03.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.050+0000 7fe32ae6ab80 1 -- start start 2024-05-07T02:38:03.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.051+0000 7fe32ae6ab80 1 Processor -- start 2024-05-07T02:38:03.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.051+0000 7fe32ae6ab80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610d23ba470 0x5610d23c4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:03.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.051+0000 7fe32ae6ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d23badc0 0x5610d23c4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:03.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.051+0000 7fe32ae6ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610d23c03b0 0x5610d23c50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:03.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.051+0000 7fe32ae6ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5610d23c7580 con 0x5610d23badc0 2024-05-07T02:38:03.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.051+0000 7fe32ae6ab80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5610d2242b20 con 0x5610d23ba470 2024-05-07T02:38:03.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.051+0000 7fe32ae6ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5610d23c7700 con 0x5610d23c03b0 2024-05-07T02:38:03.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.052+0000 7fe32ae6ab80 10 client.0 _ll_register_callbacks cb 0x5610d2283c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:38:03.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.052+0000 7fe319a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610d23ba470 0x5610d23c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:03.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.052+0000 7fe319a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610d23ba470 0x5610d23c4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:45750/0 (socket says 172.21.15.19:45750) 2024-05-07T02:38:03.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.052+0000 7fe319a00700 1 -- 172.21.15.19:0/2978747908 learned_addr learned my addr 172.21.15.19:0/2978747908 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:38:03.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.052+0000 7fe31a400700 1 --2- 172.21.15.19:0/2978747908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610d23c03b0 0x5610d23c50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:03.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.052+0000 7fe319000700 1 --2- 172.21.15.19:0/2978747908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d23badc0 0x5610d23c4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:03.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.052+0000 7fe319a00700 1 -- 172.21.15.19:0/2978747908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610d23c03b0 msgr2=0x5610d23c50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:03.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.052+0000 7fe319a00700 1 --2- 172.21.15.19:0/2978747908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610d23c03b0 0x5610d23c50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:03.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.052+0000 7fe319a00700 1 -- 172.21.15.19:0/2978747908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d23badc0 msgr2=0x5610d23c4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:03.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.053+0000 7fe319a00700 1 --2- 172.21.15.19:0/2978747908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d23badc0 0x5610d23c4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:03.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.053+0000 7fe319a00700 1 -- 172.21.15.19:0/2978747908 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610d22a2420 con 0x5610d23ba470 2024-05-07T02:38:03.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.053+0000 7fe31a400700 1 --2- 172.21.15.19:0/2978747908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610d23c03b0 0x5610d23c50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-07T02:38:03.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.053+0000 7fe319000700 1 --2- 172.21.15.19:0/2978747908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d23badc0 0x5610d23c4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-07T02:38:03.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.054+0000 7fe319a00700 1 --2- 172.21.15.19:0/2978747908 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610d23ba470 0x5610d23c4640 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe3080aab20 tx=0x7fe308009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15cde7422adc38be server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:03.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.054+0000 7fe2fb400700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:38:03.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.055+0000 7fe2fb400700 1 -- 172.21.15.19:0/2978747908 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3080a6a50 con 0x5610d23ba470 2024-05-07T02:38:03.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.055+0000 7fe2fb400700 1 -- 172.21.15.19:0/2978747908 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fe3080054f0 con 0x5610d23ba470 2024-05-07T02:38:03.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.055+0000 7fe32ae6ab80 1 -- 172.21.15.19:0/2978747908 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3080ef480 con 0x5610d23ba470 2024-05-07T02:38:03.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.055+0000 7fe2fb400700 1 -- 172.21.15.19:0/2978747908 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fe3080aa300 con 0x5610d23ba470 2024-05-07T02:38:03.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.055+0000 7fe32ae6ab80 10 client.26174 fetch_fsmap learned FSMap version 44 2024-05-07T02:38:03.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.055+0000 7fe32ae6ab80 1 -- 172.21.15.19:0/2978747908 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5610d23c7580 con 0x5610d23ba470 2024-05-07T02:38:03.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.055+0000 7fe2fb400700 1 -- 172.21.15.19:0/2978747908 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3080ef480 con 0x5610d23ba470 2024-05-07T02:38:03.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.056+0000 7fe2fb400700 1 -- 172.21.15.19:0/2978747908 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 44) ==== 8378+0+0 (secure 0 0 0) 0x7fe3080ec050 con 0x5610d23ba470 2024-05-07T02:38:03.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.056+0000 7fe32ae6ab80 10 client.26174 fetch_fsmap finished waiting for FSMap version 44 2024-05-07T02:38:03.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.056+0000 7fe32ae6ab80 10 client.26174 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:38:03.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.056+0000 7fe32ae6ab80 20 client.26174 populate_metadata read hostname 'smithi019' 2024-05-07T02:38:03.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.056+0000 7fe32ae6ab80 1 --2- 172.21.15.19:0/2978747908 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5610d23d45b0 0x5610d23d6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:03.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.056+0000 7fe32ae6ab80 4 client.26174 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:38:03.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.056+0000 7fe32ae6ab80 1 -- 172.21.15.19:0/2978747908 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610d23872e0 con 0x5610d23d45b0 2024-05-07T02:38:03.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.057+0000 7fe319000700 1 --2- 172.21.15.19:0/2978747908 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5610d23d45b0 0x5610d23d6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:03.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.057+0000 7fe319000700 1 --2- 172.21.15.19:0/2978747908 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5610d23d45b0 0x5610d23d6a40 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a27bcabcd089c3d8 server_cookie=9c2eba1b8ec04a32 in_seq=0 out_seq=0 2024-05-07T02:38:03.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.057+0000 7fe2fb400700 10 client.26174 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:38:03.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.058+0000 7fe2fb400700 1 -- 172.21.15.19:0/2978747908 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5610d23872e0 con 0x5610d23d45b0 2024-05-07T02:38:03.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.058+0000 7fe2fb400700 10 client.26174 handle_command_reply: tid=0 2024-05-07T02:38:03.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.059+0000 7fe32ae6ab80 1 client.26174 shutdown 2024-05-07T02:38:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.059+0000 7fe32ae6ab80 1 -- 172.21.15.19:0/2978747908 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610d23ba470 msgr2=0x5610d23c4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.059+0000 7fe32ae6ab80 1 --2- 172.21.15.19:0/2978747908 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610d23ba470 0x5610d23c4640 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe3080aab20 tx=0x7fe308009260 comp rx=0 tx=0).stop 2024-05-07T02:38:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.059+0000 7fe32ae6ab80 1 -- 172.21.15.19:0/2978747908 shutdown_connections 2024-05-07T02:38:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.059+0000 7fe32ae6ab80 1 --2- 172.21.15.19:0/2978747908 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5610d23d45b0 0x5610d23d6a40 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.059+0000 7fe32ae6ab80 1 --2- 172.21.15.19:0/2978747908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610d23c03b0 0x5610d23c50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.059+0000 7fe32ae6ab80 1 --2- 172.21.15.19:0/2978747908 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610d23ba470 0x5610d23c4640 secure :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe3080aab20 tx=0x7fe308009260 comp rx=0 tx=0).stop 2024-05-07T02:38:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.059+0000 7fe32ae6ab80 1 --2- 172.21.15.19:0/2978747908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d23badc0 0x5610d23c4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.059+0000 7fe32ae6ab80 1 -- 172.21.15.19:0/2978747908 >> 172.21.15.19:0/2978747908 conn(0x5610d223ce40 msgr2=0x5610d23b9460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:03.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.059+0000 7fe2fb400700 0 client.26174 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:38:03.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.060+0000 7fe32ae6ab80 1 -- 172.21.15.19:0/2978747908 shutdown_connections 2024-05-07T02:38:03.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.060+0000 7fe32ae6ab80 1 -- 172.21.15.19:0/2978747908 wait complete. 2024-05-07T02:38:03.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.060+0000 7fe32ae6ab80 20 client.26174 trim_cache size 0 max 16384 2024-05-07T02:38:03.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.103+0000 7fe32ae6ab80 1 Processor -- start 2024-05-07T02:38:03.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.103+0000 7fe32ae6ab80 1 -- start start 2024-05-07T02:38:03.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.103+0000 7fe32ae6ab80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610d23badc0 0x5610d23becd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:03.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.103+0000 7fe32ae6ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d23c03b0 0x5610d23c3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:03.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.104+0000 7fe32ae6ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610d23afbf0 0x5610d23c4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:03.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.104+0000 7fe32ae6ab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe3141b6a80 con 0x5610d23c03b0 2024-05-07T02:38:03.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.104+0000 7fe32ae6ab80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe3141b8680 con 0x5610d23badc0 2024-05-07T02:38:03.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.104+0000 7fe32ae6ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe3141b7150 con 0x5610d23afbf0 2024-05-07T02:38:03.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.104+0000 7fe319000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d23c03b0 0x5610d23c3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:03.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.105+0000 7fe319000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d23c03b0 0x5610d23c3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36656/0 (socket says 172.21.15.19:36656) 2024-05-07T02:38:03.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.105+0000 7fe319a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610d23badc0 0x5610d23becd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:03.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.105+0000 7fe319000700 1 -- 172.21.15.19:0/967361184 learned_addr learned my addr 172.21.15.19:0/967361184 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:38:03.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.105+0000 7fe31a400700 1 --2- 172.21.15.19:0/967361184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610d23afbf0 0x5610d23c4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:03.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.105+0000 7fe319a00700 1 -- 172.21.15.19:0/967361184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610d23afbf0 msgr2=0x5610d23c4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:03.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.105+0000 7fe319a00700 1 --2- 172.21.15.19:0/967361184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610d23afbf0 0x5610d23c4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:03.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.105+0000 7fe319a00700 1 -- 172.21.15.19:0/967361184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d23c03b0 msgr2=0x5610d23c3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:03.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.105+0000 7fe319a00700 1 --2- 172.21.15.19:0/967361184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d23c03b0 0x5610d23c3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:03.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.105+0000 7fe319a00700 1 -- 172.21.15.19:0/967361184 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610d23c6e90 con 0x5610d23badc0 2024-05-07T02:38:03.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.107+0000 7fe319a00700 1 --2- 172.21.15.19:0/967361184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610d23badc0 0x5610d23becd0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe3080cd000 tx=0x7fe308003680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a423efd556200820 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:03.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.107+0000 7fe306000700 1 -- 172.21.15.19:0/967361184 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3080aa4e0 con 0x5610d23badc0 2024-05-07T02:38:03.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.107+0000 7fe306000700 1 -- 172.21.15.19:0/967361184 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3080b32a0 con 0x5610d23badc0 2024-05-07T02:38:03.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.107+0000 7fe306000700 1 -- 172.21.15.19:0/967361184 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3080aa960 con 0x5610d23badc0 2024-05-07T02:38:03.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.108+0000 7fe32ae6ab80 1 -- 172.21.15.19:0/967361184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610d23badc0 msgr2=0x5610d23becd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:03.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.108+0000 7fe32ae6ab80 1 --2- 172.21.15.19:0/967361184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610d23badc0 0x5610d23becd0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe3080cd000 tx=0x7fe308003680 comp rx=0 tx=0).stop 2024-05-07T02:38:03.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.109+0000 7fe32ae6ab80 1 -- 172.21.15.19:0/967361184 shutdown_connections 2024-05-07T02:38:03.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.109+0000 7fe32ae6ab80 1 --2- 172.21.15.19:0/967361184 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610d23afbf0 0x5610d23c4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:03.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.109+0000 7fe32ae6ab80 1 --2- 172.21.15.19:0/967361184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610d23badc0 0x5610d23becd0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:03.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.109+0000 7fe32ae6ab80 1 --2- 172.21.15.19:0/967361184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d23c03b0 0x5610d23c3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:03.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.109+0000 7fe32ae6ab80 1 -- 172.21.15.19:0/967361184 >> 172.21.15.19:0/967361184 conn(0x5610d223ce40 msgr2=0x5610d23c5ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:03.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.109+0000 7fe32ae6ab80 1 -- 172.21.15.19:0/967361184 shutdown_connections 2024-05-07T02:38:03.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.109+0000 7fe32ae6ab80 1 -- 172.21.15.19:0/967361184 wait complete. 2024-05-07T02:38:03.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.110+0000 7fe32ae6ab80 1 -- start start 2024-05-07T02:38:03.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.110+0000 7fe32ae6ab80 1 Processor -- start 2024-05-07T02:38:03.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.112+0000 7fe32ae6ab80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610d23badc0 0x5610d23cce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:03.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.112+0000 7fe32ae6ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610d23c03b0 0x5610d23db7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:03.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.112+0000 7fe32ae6ab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d23dbd10 0x5610d23dc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:03.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.112+0000 7fe319a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610d23badc0 0x5610d23cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:03.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.112+0000 7fe319a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610d23badc0 0x5610d23cce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:45762/0 (socket says 172.21.15.19:45762) 2024-05-07T02:38:03.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.112+0000 7fe319a00700 1 -- 172.21.15.19:0/3873266538 learned_addr learned my addr 172.21.15.19:0/3873266538 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:38:03.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.112+0000 7fe32ae6ab80 1 -- 172.21.15.19:0/3873266538 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe3141b7ef0 con 0x5610d23dbd10 2024-05-07T02:38:03.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.112+0000 7fe32ae6ab80 1 -- 172.21.15.19:0/3873266538 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fe3141b7820 con 0x5610d23badc0 2024-05-07T02:38:03.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.112+0000 7fe32ae6ab80 1 -- 172.21.15.19:0/3873266538 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe3141b4a20 con 0x5610d23c03b0 2024-05-07T02:38:03.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.112+0000 7fe32ae6ab80 10 client.0 _ll_register_callbacks cb 0x5610d2283c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:38:03.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.112+0000 7fe319000700 1 --2- 172.21.15.19:0/3873266538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610d23c03b0 0x5610d23db7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:03.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.113+0000 7fe31a400700 1 --2- 172.21.15.19:0/3873266538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d23dbd10 0x5610d23dc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:03.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.113+0000 7fe319a00700 1 -- 172.21.15.19:0/3873266538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610d23c03b0 msgr2=0x5610d23db7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:03.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.113+0000 7fe319a00700 1 --2- 172.21.15.19:0/3873266538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610d23c03b0 0x5610d23db7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:03.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.113+0000 7fe319a00700 1 -- 172.21.15.19:0/3873266538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d23dbd10 msgr2=0x5610d23dc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:03.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.113+0000 7fe319a00700 1 --2- 172.21.15.19:0/3873266538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d23dbd10 0x5610d23dc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:03.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.113+0000 7fe319a00700 1 -- 172.21.15.19:0/3873266538 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610d243ddb0 con 0x5610d23badc0 2024-05-07T02:38:03.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.113+0000 7fe31a400700 1 --2- 172.21.15.19:0/3873266538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d23dbd10 0x5610d23dc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-07T02:38:03.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.113+0000 7fe319a00700 1 --2- 172.21.15.19:0/3873266538 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610d23badc0 0x5610d23cce90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe3080ab2e0 tx=0x7fe3080ab3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d558d2361f41d0f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:03.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.114+0000 7fe2fb400700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:38:03.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.114+0000 7fe2fb400700 1 -- 172.21.15.19:0/3873266538 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3080aa960 con 0x5610d23badc0 2024-05-07T02:38:03.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.114+0000 7fe2fb400700 1 -- 172.21.15.19:0/3873266538 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fe3080e11f0 con 0x5610d23badc0 2024-05-07T02:38:03.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.114+0000 7fe32ae6ab80 1 -- 172.21.15.19:0/3873266538 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5610d22ea400 con 0x5610d23badc0 2024-05-07T02:38:03.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.114+0000 7fe2fb400700 1 -- 172.21.15.19:0/3873266538 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fe3080aabe0 con 0x5610d23badc0 2024-05-07T02:38:03.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.114+0000 7fe32ae6ab80 10 client.26180 fetch_fsmap learned FSMap version 44 2024-05-07T02:38:03.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.114+0000 7fe32ae6ab80 1 -- 172.21.15.19:0/3873266538 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5610d243ddb0 con 0x5610d23badc0 2024-05-07T02:38:03.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.115+0000 7fe2fb400700 1 -- 172.21.15.19:0/3873266538 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5610d22ea400 con 0x5610d23badc0 2024-05-07T02:38:03.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.115+0000 7fe2fb400700 1 -- 172.21.15.19:0/3873266538 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 44) ==== 8378+0+0 (secure 0 0 0) 0x7fe3080ec070 con 0x5610d23badc0 2024-05-07T02:38:03.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.115+0000 7fe32ae6ab80 10 client.26180 fetch_fsmap finished waiting for FSMap version 44 2024-05-07T02:38:03.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.115+0000 7fe32ae6ab80 10 client.26180 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:38:03.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.115+0000 7fe32ae6ab80 20 client.26180 populate_metadata read hostname 'smithi019' 2024-05-07T02:38:03.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.115+0000 7fe32ae6ab80 1 --2- 172.21.15.19:0/3873266538 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5610d243f370 0x5610d2441790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:03.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.115+0000 7fe32ae6ab80 4 client.26180 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-07T02:38:03.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.115+0000 7fe32ae6ab80 1 -- 172.21.15.19:0/3873266538 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5610d22e44a0 con 0x5610d243f370 2024-05-07T02:38:03.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.116+0000 7fe319000700 1 --2- 172.21.15.19:0/3873266538 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5610d243f370 0x5610d2441790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:03.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.116+0000 7fe319000700 1 --2- 172.21.15.19:0/3873266538 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5610d243f370 0x5610d2441790 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d751b43d0cdfd1d server_cookie=d69e34f97eda6941 in_seq=0 out_seq=0 2024-05-07T02:38:03.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.116+0000 7fe2fb400700 10 client.26180 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:38:03.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.117+0000 7fe2fb400700 1 -- 172.21.15.19:0/3873266538 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5610d22e44a0 con 0x5610d243f370 2024-05-07T02:38:03.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.117+0000 7fe2fb400700 10 client.26180 handle_command_reply: tid=0 2024-05-07T02:38:03.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.117+0000 7fe32ae6ab80 1 client.26180 shutdown 2024-05-07T02:38:03.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.118+0000 7fe32ae6ab80 1 -- 172.21.15.19:0/3873266538 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610d23badc0 msgr2=0x5610d23cce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:03.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.118+0000 7fe32ae6ab80 1 --2- 172.21.15.19:0/3873266538 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610d23badc0 0x5610d23cce90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe3080ab2e0 tx=0x7fe3080ab3d0 comp rx=0 tx=0).stop 2024-05-07T02:38:03.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.118+0000 7fe32ae6ab80 1 -- 172.21.15.19:0/3873266538 shutdown_connections 2024-05-07T02:38:03.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.118+0000 7fe32ae6ab80 1 --2- 172.21.15.19:0/3873266538 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5610d243f370 0x5610d2441790 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:03.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.118+0000 7fe32ae6ab80 1 --2- 172.21.15.19:0/3873266538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d23dbd10 0x5610d23dc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:03.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.118+0000 7fe32ae6ab80 1 --2- 172.21.15.19:0/3873266538 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5610d23badc0 0x5610d23cce90 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:03.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.118+0000 7fe32ae6ab80 1 --2- 172.21.15.19:0/3873266538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5610d23c03b0 0x5610d23db7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:03.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.118+0000 7fe32ae6ab80 1 -- 172.21.15.19:0/3873266538 >> 172.21.15.19:0/3873266538 conn(0x5610d223ce40 msgr2=0x5610d23c36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:03.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.118+0000 7fe2fb400700 0 client.26180 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:38:03.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.118+0000 7fe32ae6ab80 1 -- 172.21.15.19:0/3873266538 shutdown_connections 2024-05-07T02:38:03.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.118+0000 7fe32ae6ab80 1 -- 172.21.15.19:0/3873266538 wait complete. 2024-05-07T02:38:03.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.118+0000 7fe32ae6ab80 20 client.26180 trim_cache size 0 max 16384 2024-05-07T02:38:03.134 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:38:03.134 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-07T02:38:03.135 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-07T02:38:03.135 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:38:03.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.119+0000 7fe31c400700 1 -- 172.21.15.19:0/2392484389 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fe2fc0b29f0 msgr2=0x7fe2fc0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:03.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.119+0000 7fe31c400700 1 --2- 172.21.15.19:0/2392484389 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fe2fc0b29f0 0x7fe2fc0b4f00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x5610d23bfe70 tx=0x7fe30c009040 comp rx=0 tx=0).stop 2024-05-07T02:38:03.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.119+0000 7fe31c400700 1 -- 172.21.15.19:0/2392484389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe314103250 msgr2=0x7fe3141a1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:03.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.119+0000 7fe31c400700 1 --2- 172.21.15.19:0/2392484389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe314103250 0x7fe3141a1440 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe308009230 tx=0x7fe3080050f0 comp rx=0 tx=0).stop 2024-05-07T02:38:03.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.119+0000 7fe31c400700 1 -- 172.21.15.19:0/2392484389 shutdown_connections 2024-05-07T02:38:03.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.119+0000 7fe31c400700 1 --2- 172.21.15.19:0/2392484389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3141086a0 0x7fe3141a5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:03.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.119+0000 7fe31c400700 1 --2- 172.21.15.19:0/2392484389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe314103250 0x7fe3141a1440 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:03.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.119+0000 7fe31c400700 1 --2- 172.21.15.19:0/2392484389 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fe2fc0b29f0 0x7fe2fc0b4f00 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:03.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.120+0000 7fe31c400700 1 --2- 172.21.15.19:0/2392484389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe314103ba0 0x7fe3141a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:03.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.120+0000 7fe31c400700 1 -- 172.21.15.19:0/2392484389 >> 172.21.15.19:0/2392484389 conn(0x7fe3140fedc0 msgr2=0x7fe314106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:03.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.120+0000 7fe31c400700 1 -- 172.21.15.19:0/2392484389 shutdown_connections 2024-05-07T02:38:03.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:03.120+0000 7fe31c400700 1 -- 172.21.15.19:0/2392484389 wait complete. 2024-05-07T02:38:03.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:03 smithi019 ceph-mon[29781]: pgmap v189: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 186 op/s 2024-05-07T02:38:03.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:03 smithi027 ceph-mon[34398]: pgmap v189: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 186 op/s 2024-05-07T02:38:03.761 INFO:tasks.workunit.client.0.smithi019.stdout: 1 209577 4.10 MB/sec execute 179 sec latency 48.104 ms 2024-05-07T02:38:03.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:03 smithi064 ceph-mon[34699]: pgmap v189: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 186 op/s 2024-05-07T02:38:04.083 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-07T02:38:04.084 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-07T02:38:04.553 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:38:04.761 INFO:tasks.workunit.client.0.smithi019.stdout: 1 210186 4.09 MB/sec execute 180 sec latency 21.956 ms 2024-05-07T02:38:05.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:05 smithi019 ceph-mon[29781]: pgmap v190: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 120 op/s 2024-05-07T02:38:05.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:05 smithi027 ceph-mon[34398]: pgmap v190: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 120 op/s 2024-05-07T02:38:05.761 INFO:tasks.workunit.client.0.smithi019.stdout: 1 210970 4.09 MB/sec execute 181 sec latency 91.194 ms 2024-05-07T02:38:05.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:05 smithi064 ceph-mon[34699]: pgmap v190: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 120 op/s 2024-05-07T02:38:06.761 INFO:tasks.workunit.client.0.smithi019.stdout: 1 210970 4.07 MB/sec execute 182 sec latency 1091.322 ms 2024-05-07T02:38:07.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.331+0000 7fa3baa00700 1 -- 172.21.15.19:0/2802123670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b4105e60 msgr2=0x7fa3b410e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:07.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.331+0000 7fa3baa00700 1 --2- 172.21.15.19:0/2802123670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b4105e60 0x7fa3b410e320 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0009990 tx=0x7fa3a0009ca0 comp rx=0 tx=0).stop 2024-05-07T02:38:07.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.331+0000 7fa3baa00700 1 -- 172.21.15.19:0/2802123670 shutdown_connections 2024-05-07T02:38:07.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.331+0000 7fa3baa00700 1 --2- 172.21.15.19:0/2802123670 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3b410e940 0x7fa3b4110e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:07.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.331+0000 7fa3baa00700 1 --2- 172.21.15.19:0/2802123670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3b4105480 0x7fa3b4105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:07.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.331+0000 7fa3baa00700 1 --2- 172.21.15.19:0/2802123670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b4105e60 0x7fa3b410e320 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:07.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.331+0000 7fa3baa00700 1 -- 172.21.15.19:0/2802123670 >> 172.21.15.19:0/2802123670 conn(0x7fa3b40fef50 msgr2=0x7fa3b4101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:07.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.331+0000 7fa3baa00700 1 -- 172.21.15.19:0/2802123670 shutdown_connections 2024-05-07T02:38:07.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.331+0000 7fa3baa00700 1 -- 172.21.15.19:0/2802123670 wait complete. 2024-05-07T02:38:07.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.332+0000 7fa3baa00700 1 Processor -- start 2024-05-07T02:38:07.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.333+0000 7fa3baa00700 1 -- start start 2024-05-07T02:38:07.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.333+0000 7fa3baa00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b4105480 0x7fa3b4103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:07.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.333+0000 7fa3baa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3b4105e60 0x7fa3b41036b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:07.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.333+0000 7fa3baa00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3b410e940 0x7fa3b4103bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:07.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.333+0000 7fa3baa00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b4114480 con 0x7fa3b4105480 2024-05-07T02:38:07.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.333+0000 7fa3baa00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b4114300 con 0x7fa3b4105e60 2024-05-07T02:38:07.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.333+0000 7fa3baa00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b4114600 con 0x7fa3b410e940 2024-05-07T02:38:07.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.334+0000 7fa3b3e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b4105480 0x7fa3b4103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:07.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.334+0000 7fa3b3e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b4105480 0x7fa3b4103170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36680/0 (socket says 172.21.15.19:36680) 2024-05-07T02:38:07.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.334+0000 7fa3b3e00700 1 -- 172.21.15.19:0/2649700621 learned_addr learned my addr 172.21.15.19:0/2649700621 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:38:07.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.334+0000 7fa3b3400700 1 --2- 172.21.15.19:0/2649700621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3b4105e60 0x7fa3b41036b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:07.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.334+0000 7fa3b8a00700 1 --2- 172.21.15.19:0/2649700621 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3b410e940 0x7fa3b4103bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:07.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.334+0000 7fa3b3e00700 1 -- 172.21.15.19:0/2649700621 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3b410e940 msgr2=0x7fa3b4103bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:07.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.334+0000 7fa3b3e00700 1 --2- 172.21.15.19:0/2649700621 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3b410e940 0x7fa3b4103bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:07.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.334+0000 7fa3b3e00700 1 -- 172.21.15.19:0/2649700621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3b4105e60 msgr2=0x7fa3b41036b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:07.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.334+0000 7fa3b3e00700 1 --2- 172.21.15.19:0/2649700621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3b4105e60 0x7fa3b41036b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:07.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.334+0000 7fa3b3e00700 1 -- 172.21.15.19:0/2649700621 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3b41a91b0 con 0x7fa3b4105480 2024-05-07T02:38:07.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.335+0000 7fa3b3e00700 1 --2- 172.21.15.19:0/2649700621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b4105480 0x7fa3b4103170 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa3a400e9a0 tx=0x7fa3a400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:07.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.335+0000 7fa3b0c00700 1 -- 172.21.15.19:0/2649700621 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a400ca10 con 0x7fa3b4105480 2024-05-07T02:38:07.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.335+0000 7fa3b0c00700 1 -- 172.21.15.19:0/2649700621 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3a400cbb0 con 0x7fa3b4105480 2024-05-07T02:38:07.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.335+0000 7fa3baa00700 1 -- 172.21.15.19:0/2649700621 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3b41a94a0 con 0x7fa3b4105480 2024-05-07T02:38:07.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.335+0000 7fa3b0c00700 1 -- 172.21.15.19:0/2649700621 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a4018680 con 0x7fa3b4105480 2024-05-07T02:38:07.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.335+0000 7fa3baa00700 1 -- 172.21.15.19:0/2649700621 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3b41a99b0 con 0x7fa3b4105480 2024-05-07T02:38:07.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.336+0000 7fa3baa00700 1 -- 172.21.15.19:0/2649700621 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3b40512c0 con 0x7fa3b4105480 2024-05-07T02:38:07.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.341+0000 7fa3b0c00700 1 -- 172.21.15.19:0/2649700621 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa3a4010430 con 0x7fa3b4105480 2024-05-07T02:38:07.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.343+0000 7fa3b0c00700 1 --2- 172.21.15.19:0/2649700621 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fa3980b29f0 0x7fa3980b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:07.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.343+0000 7fa3b0c00700 1 -- 172.21.15.19:0/2649700621 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fa3a40e2ec0 con 0x7fa3b4105480 2024-05-07T02:38:07.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.343+0000 7fa3b3400700 1 --2- 172.21.15.19:0/2649700621 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fa3980b29f0 0x7fa3980b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:07.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.344+0000 7fa3b3400700 1 --2- 172.21.15.19:0/2649700621 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fa3980b29f0 0x7fa3980b4f00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa3a000b2e0 tx=0x7fa3a0005960 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:07.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.345+0000 7fa3b0c00700 1 -- 172.21.15.19:0/2649700621 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa3a40aa4b0 con 0x7fa3b4105480 2024-05-07T02:38:07.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.637+0000 7fa3baa00700 1 -- 172.21.15.19:0/2649700621 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3b402e5f0 con 0x7fa3b4105480 2024-05-07T02:38:07.638 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:07 smithi019 ceph-mon[29781]: pgmap v191: 129 pgs: 129 active+clean; 147 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 203 op/s 2024-05-07T02:38:07.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.640+0000 7fa3b0c00700 1 -- 172.21.15.19:0/2649700621 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11931 (secure 0 0 0) 0x7fa3a40a9db0 con 0x7fa3b4105480 2024-05-07T02:38:07.641 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:38:07.641 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1763670446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1763670446},{"type":"v1","addr":"172.21.15.19:6839","nonce":1763670446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/477529985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":477529985},{"type":"v1","addr":"172.21.15.64:6833","nonce":477529985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-07T02:26:14.960465+0000","modified":"2024-05-07T02:37:53.645389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24539,"mds_2":24520,"mds_3":24515,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6833/2171873162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2171873162},{"type":"v1","addr":"172.21.15.27:6833","nonce":2171873162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/2379917839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2379917839},{"type":"v1","addr":"172.21.15.19:6835","nonce":2379917839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6837/2067515538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2067515538},{"type":"v1","addr":"172.21.15.19:6837","nonce":2067515538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6835/1000540600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1000540600},{"type":"v1","addr":"172.21.15.27:6835","nonce":1000540600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/1997318872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1997318872},{"type":"v1","addr":"172.21.15.64:6835","nonce":1997318872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":34,"addr":"172.21.15.64:6837/2585907777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2585907777},{"type":"v1","addr":"172.21.15.64:6837","nonce":2585907777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":33,"addr":"172.21.15.27:6837/2583138143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2583138143},{"type":"v1","addr":"172.21.15.27:6837","nonce":2583138143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":32,"addr":"172.21.15.64:6839/1583120004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":1583120004},{"type":"v1","addr":"172.21.15.64:6839","nonce":1583120004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":29,"addr":"172.21.15.19:6841/3905645246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3905645246},{"type":"v1","addr":"172.21.15.19:6841","nonce":3905645246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6839/3963268634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3963268634},{"type":"v1","addr":"172.21.15.27:6839","nonce":3963268634}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24539,24551,24520,24515,24505]},"id":1}]} 2024-05-07T02:38:07.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.644+0000 7fa3baa00700 1 -- 172.21.15.19:0/2649700621 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fa3980b29f0 msgr2=0x7fa3980b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:07.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.644+0000 7fa3baa00700 1 --2- 172.21.15.19:0/2649700621 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fa3980b29f0 0x7fa3980b4f00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa3a000b2e0 tx=0x7fa3a0005960 comp rx=0 tx=0).stop 2024-05-07T02:38:07.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.644+0000 7fa3baa00700 1 -- 172.21.15.19:0/2649700621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b4105480 msgr2=0x7fa3b4103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:07.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.644+0000 7fa3baa00700 1 --2- 172.21.15.19:0/2649700621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b4105480 0x7fa3b4103170 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa3a400e9a0 tx=0x7fa3a400ecb0 comp rx=0 tx=0).stop 2024-05-07T02:38:07.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.644+0000 7fa3baa00700 1 -- 172.21.15.19:0/2649700621 shutdown_connections 2024-05-07T02:38:07.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.644+0000 7fa3baa00700 1 --2- 172.21.15.19:0/2649700621 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa3b410e940 0x7fa3b4103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:07.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.644+0000 7fa3baa00700 1 --2- 172.21.15.19:0/2649700621 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fa3980b29f0 0x7fa3980b4f00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:07.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.644+0000 7fa3baa00700 1 --2- 172.21.15.19:0/2649700621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3b4105480 0x7fa3b4103170 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:07.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.644+0000 7fa3baa00700 1 --2- 172.21.15.19:0/2649700621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3b4105e60 0x7fa3b41036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:07.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.644+0000 7fa3baa00700 1 -- 172.21.15.19:0/2649700621 >> 172.21.15.19:0/2649700621 conn(0x7fa3b40fef50 msgr2=0x7fa3b40ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:07.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.645+0000 7fa3baa00700 1 -- 172.21.15.19:0/2649700621 shutdown_connections 2024-05-07T02:38:07.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:07.645+0000 7fa3baa00700 1 -- 172.21.15.19:0/2649700621 wait complete. 2024-05-07T02:38:07.647 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 44 2024-05-07T02:38:07.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:07 smithi027 ceph-mon[34398]: pgmap v191: 129 pgs: 129 active+clean; 147 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 203 op/s 2024-05-07T02:38:07.761 INFO:tasks.workunit.client.0.smithi019.stdout: 1 211104 4.04 MB/sec execute 183 sec latency 1992.868 ms 2024-05-07T02:38:07.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:07 smithi064 ceph-mon[34699]: pgmap v191: 129 pgs: 129 active+clean; 147 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 203 op/s 2024-05-07T02:38:08.612 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-07T02:38:08.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:08 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/2649700621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:38:08.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:08 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/2649700621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:38:08.761 INFO:tasks.workunit.client.0.smithi019.stdout: 1 212250 4.06 MB/sec execute 184 sec latency 54.042 ms 2024-05-07T02:38:08.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:08 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/2649700621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:38:09.113 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:38:09.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:09 smithi019 ceph-mon[29781]: pgmap v192: 129 pgs: 129 active+clean; 147 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-05-07T02:38:09.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:09 smithi027 ceph-mon[34398]: pgmap v192: 129 pgs: 129 active+clean; 147 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-05-07T02:38:09.761 INFO:tasks.workunit.client.0.smithi019.stdout: 1 213173 4.07 MB/sec execute 185 sec latency 32.208 ms 2024-05-07T02:38:09.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:09 smithi064 ceph-mon[34699]: pgmap v192: 129 pgs: 129 active+clean; 147 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-05-07T02:38:10.762 INFO:tasks.workunit.client.0.smithi019.stdout: 1 213626 4.06 MB/sec execute 186 sec latency 29.237 ms 2024-05-07T02:38:11.481 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:11 smithi019 ceph-mon[29781]: pgmap v193: 129 pgs: 129 active+clean; 147 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-05-07T02:38:11.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:11 smithi027 ceph-mon[34398]: pgmap v193: 129 pgs: 129 active+clean; 147 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-05-07T02:38:11.762 INFO:tasks.workunit.client.0.smithi019.stdout: 1 214407 4.06 MB/sec execute 187 sec latency 23.848 ms 2024-05-07T02:38:11.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:11 smithi064 ceph-mon[34699]: pgmap v193: 129 pgs: 129 active+clean; 147 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-05-07T02:38:11.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.895+0000 7fc877a00700 1 -- 172.21.15.19:0/3067264636 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc870076490 msgr2=0x7fc870110680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:11.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.895+0000 7fc877a00700 1 --2- 172.21.15.19:0/3067264636 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc870076490 0x7fc870110680 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc85c009990 tx=0x7fc85c009ca0 comp rx=0 tx=0).stop 2024-05-07T02:38:11.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.895+0000 7fc877a00700 1 -- 172.21.15.19:0/3067264636 shutdown_connections 2024-05-07T02:38:11.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.895+0000 7fc877a00700 1 --2- 172.21.15.19:0/3067264636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc870110bc0 0x7fc870113000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:11.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.895+0000 7fc877a00700 1 --2- 172.21.15.19:0/3067264636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc870075b40 0x7fc870075f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:11.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.895+0000 7fc877a00700 1 --2- 172.21.15.19:0/3067264636 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc870076490 0x7fc870110680 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:11.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.895+0000 7fc877a00700 1 -- 172.21.15.19:0/3067264636 >> 172.21.15.19:0/3067264636 conn(0x7fc870100040 msgr2=0x7fc870102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:11.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.895+0000 7fc877a00700 1 -- 172.21.15.19:0/3067264636 shutdown_connections 2024-05-07T02:38:11.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.895+0000 7fc877a00700 1 -- 172.21.15.19:0/3067264636 wait complete. 2024-05-07T02:38:11.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.897+0000 7fc877a00700 1 Processor -- start 2024-05-07T02:38:11.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.897+0000 7fc877a00700 1 -- start start 2024-05-07T02:38:11.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.898+0000 7fc877a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc870075b40 0x7fc8701042e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:11.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.898+0000 7fc877a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc870076490 0x7fc870104820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:11.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.898+0000 7fc877a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc870110bc0 0x7fc870104d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:11.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.898+0000 7fc877a00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc8701166e0 con 0x7fc870076490 2024-05-07T02:38:11.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.898+0000 7fc877a00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc870116560 con 0x7fc870075b40 2024-05-07T02:38:11.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.898+0000 7fc877a00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc870116860 con 0x7fc870110bc0 2024-05-07T02:38:11.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.898+0000 7fc875000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc870075b40 0x7fc8701042e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:11.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.898+0000 7fc86fe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc870076490 0x7fc870104820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:11.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.898+0000 7fc875000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc870075b40 0x7fc8701042e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:59472/0 (socket says 172.21.15.19:59472) 2024-05-07T02:38:11.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.898+0000 7fc875000700 1 -- 172.21.15.19:0/2642591950 learned_addr learned my addr 172.21.15.19:0/2642591950 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:38:11.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.898+0000 7fc875a00700 1 --2- 172.21.15.19:0/2642591950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc870110bc0 0x7fc870104d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:11.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.899+0000 7fc875000700 1 -- 172.21.15.19:0/2642591950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc870110bc0 msgr2=0x7fc870104d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:11.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.899+0000 7fc875000700 1 --2- 172.21.15.19:0/2642591950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc870110bc0 0x7fc870104d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:11.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.899+0000 7fc875000700 1 -- 172.21.15.19:0/2642591950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc870076490 msgr2=0x7fc870104820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:11.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.899+0000 7fc875000700 1 --2- 172.21.15.19:0/2642591950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc870076490 0x7fc870104820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:11.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.899+0000 7fc875000700 1 -- 172.21.15.19:0/2642591950 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8701ad650 con 0x7fc870075b40 2024-05-07T02:38:11.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.899+0000 7fc875000700 1 --2- 172.21.15.19:0/2642591950 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc870075b40 0x7fc8701042e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc86400e9a0 tx=0x7fc86400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:11.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.899+0000 7fc86d600700 1 -- 172.21.15.19:0/2642591950 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86400c9c0 con 0x7fc870075b40 2024-05-07T02:38:11.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.900+0000 7fc86d600700 1 -- 172.21.15.19:0/2642591950 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc86400cb60 con 0x7fc870075b40 2024-05-07T02:38:11.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.900+0000 7fc86d600700 1 -- 172.21.15.19:0/2642591950 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc864018680 con 0x7fc870075b40 2024-05-07T02:38:11.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.900+0000 7fc877a00700 1 -- 172.21.15.19:0/2642591950 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8701ad8e0 con 0x7fc870075b40 2024-05-07T02:38:11.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.900+0000 7fc877a00700 1 -- 172.21.15.19:0/2642591950 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8701addd0 con 0x7fc870075b40 2024-05-07T02:38:11.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.904+0000 7fc86d600700 1 -- 172.21.15.19:0/2642591950 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc864010430 con 0x7fc870075b40 2024-05-07T02:38:11.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.906+0000 7fc88663eb80 1 Processor -- start 2024-05-07T02:38:11.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.906+0000 7fc88663eb80 1 -- start start 2024-05-07T02:38:11.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.906+0000 7fc86d600700 1 --2- 172.21.15.19:0/2642591950 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc8580b2a70 0x7fc8580b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:11.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.907+0000 7fc88663eb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558877b38470 0x558877b38880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:11.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.907+0000 7fc88663eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558877b32620 0x558877b32a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:11.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.907+0000 7fc88663eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558877b32f70 0x558877b3f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:11.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.907+0000 7fc88663eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558877a54a20 con 0x558877b32f70 2024-05-07T02:38:11.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.907+0000 7fc88663eb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5588779f2760 con 0x558877b38470 2024-05-07T02:38:11.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.907+0000 7fc88663eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558877a509e0 con 0x558877b32620 2024-05-07T02:38:11.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.907+0000 7fc86d600700 1 -- 172.21.15.19:0/2642591950 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fc8640e2c20 con 0x7fc870075b40 2024-05-07T02:38:11.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.907+0000 7fc86fe00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558877b32620 0x558877b32a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:11.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.907+0000 7fc86fe00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558877b32620 0x558877b32a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:38766/0 (socket says 172.21.15.19:38766) 2024-05-07T02:38:11.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.907+0000 7fc86fe00700 1 -- 172.21.15.19:0/1633392300 learned_addr learned my addr 172.21.15.19:0/1633392300 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:38:11.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.907+0000 7fc875000700 1 --2- 172.21.15.19:0/1633392300 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558877b38470 0x558877b38880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:11.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.907+0000 7fc875a00700 1 --2- 172.21.15.19:0/1633392300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558877b32f70 0x558877b3f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:11.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.907+0000 7fc86fe00700 1 -- 172.21.15.19:0/1633392300 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558877b38470 msgr2=0x558877b38880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:11.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.907+0000 7fc86fe00700 1 --2- 172.21.15.19:0/1633392300 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558877b38470 0x558877b38880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:11.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.907+0000 7fc86fe00700 1 -- 172.21.15.19:0/1633392300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558877b32f70 msgr2=0x558877b3f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:11.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.908+0000 7fc86fe00700 1 --2- 172.21.15.19:0/1633392300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558877b32f70 0x558877b3f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:11.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.908+0000 7fc86fe00700 1 -- 172.21.15.19:0/1633392300 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588779a18d0 con 0x558877b32620 2024-05-07T02:38:11.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.908+0000 7fc875a00700 1 --2- 172.21.15.19:0/1633392300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558877b32f70 0x558877b3f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-07T02:38:11.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.908+0000 7fc875a00700 1 --2- 172.21.15.19:0/2642591950 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc8580b2a70 0x7fc8580b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:11.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.908+0000 7fc86fe00700 1 --2- 172.21.15.19:0/1633392300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558877b32620 0x558877b32a30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc85c015000 tx=0x7fc85c0096a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=251304ebf304d98c server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:11.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.908+0000 7fc862000700 1 -- 172.21.15.19:0/1633392300 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85c027070 con 0x558877b32620 2024-05-07T02:38:11.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.909+0000 7fc862000700 1 -- 172.21.15.19:0/1633392300 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc85c00bbd0 con 0x558877b32620 2024-05-07T02:38:11.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.909+0000 7fc862000700 1 -- 172.21.15.19:0/1633392300 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85c003cf0 con 0x558877b32620 2024-05-07T02:38:11.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.910+0000 7fc88663eb80 1 -- 172.21.15.19:0/1633392300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558877b32620 msgr2=0x558877b32a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:11.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.910+0000 7fc875a00700 1 --2- 172.21.15.19:0/2642591950 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc8580b2a70 0x7fc8580b4f80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc868009fd0 tx=0x7fc86800b040 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:11.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.910+0000 7fc88663eb80 1 --2- 172.21.15.19:0/1633392300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558877b32620 0x558877b32a30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc85c015000 tx=0x7fc85c0096a0 comp rx=0 tx=0).stop 2024-05-07T02:38:11.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.910+0000 7fc88663eb80 1 -- 172.21.15.19:0/1633392300 shutdown_connections 2024-05-07T02:38:11.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.910+0000 7fc88663eb80 1 --2- 172.21.15.19:0/1633392300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558877b32f70 0x558877b3f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:11.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.910+0000 7fc88663eb80 1 --2- 172.21.15.19:0/1633392300 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558877b38470 0x558877b38880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:11.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.910+0000 7fc88663eb80 1 --2- 172.21.15.19:0/1633392300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558877b32620 0x558877b32a30 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:11.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.910+0000 7fc88663eb80 1 -- 172.21.15.19:0/1633392300 >> 172.21.15.19:0/1633392300 conn(0x5588779b4e40 msgr2=0x558877b2d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:11.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.910+0000 7fc88663eb80 1 -- 172.21.15.19:0/1633392300 shutdown_connections 2024-05-07T02:38:11.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.910+0000 7fc88663eb80 1 -- 172.21.15.19:0/1633392300 wait complete. 2024-05-07T02:38:11.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.911+0000 7fc88663eb80 1 -- start start 2024-05-07T02:38:11.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.911+0000 7fc88663eb80 1 Processor -- start 2024-05-07T02:38:11.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.912+0000 7fc88663eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558877b32620 0x558877b3c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:11.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.912+0000 7fc88663eb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558877b32f70 0x558877b3cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:11.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.912+0000 7fc88663eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558877b38470 0x558877b3d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:11.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.912+0000 7fc88663eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558877b3f7e0 con 0x558877b32620 2024-05-07T02:38:11.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.912+0000 7fc88663eb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5588779bab20 con 0x558877b32f70 2024-05-07T02:38:11.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.912+0000 7fc88663eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558877b3f960 con 0x558877b38470 2024-05-07T02:38:11.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.912+0000 7fc88663eb80 10 client.0 _ll_register_callbacks cb 0x5588779e8380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:38:11.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.912+0000 7fc875a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558877b38470 0x558877b3d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:11.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.912+0000 7fc875a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558877b38470 0x558877b3d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:38782/0 (socket says 172.21.15.19:38782) 2024-05-07T02:38:11.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.912+0000 7fc875a00700 1 -- 172.21.15.19:0/3294050648 learned_addr learned my addr 172.21.15.19:0/3294050648 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:38:11.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.912+0000 7fc86fe00700 1 --2- 172.21.15.19:0/3294050648 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558877b32f70 0x558877b3cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:11.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.913+0000 7fc875000700 1 --2- 172.21.15.19:0/3294050648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558877b32620 0x558877b3c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:11.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.913+0000 7fc86fe00700 1 -- 172.21.15.19:0/3294050648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558877b38470 msgr2=0x558877b3d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:11.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.913+0000 7fc86fe00700 1 --2- 172.21.15.19:0/3294050648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558877b38470 0x558877b3d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:11.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.913+0000 7fc86fe00700 1 -- 172.21.15.19:0/3294050648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558877b32620 msgr2=0x558877b3c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:11.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.913+0000 7fc86fe00700 1 --2- 172.21.15.19:0/3294050648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558877b32620 0x558877b3c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:11.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.913+0000 7fc86fe00700 1 -- 172.21.15.19:0/3294050648 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558877a1a420 con 0x558877b32f70 2024-05-07T02:38:11.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.913+0000 7fc86fe00700 1 --2- 172.21.15.19:0/3294050648 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558877b32f70 0x558877b3cb70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc85c0098e0 tx=0x7fc85c005b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bab9d0790f66fab3 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:11.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.913+0000 7fc856a00700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:38:11.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.914+0000 7fc856a00700 1 -- 172.21.15.19:0/3294050648 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85c027040 con 0x558877b32f70 2024-05-07T02:38:11.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.914+0000 7fc856a00700 1 -- 172.21.15.19:0/3294050648 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fc85c00b720 con 0x558877b32f70 2024-05-07T02:38:11.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.914+0000 7fc88663eb80 1 -- 172.21.15.19:0/3294050648 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc85c00bbd0 con 0x558877b32f70 2024-05-07T02:38:11.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.914+0000 7fc856a00700 1 -- 172.21.15.19:0/3294050648 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc85c0056e0 con 0x558877b32f70 2024-05-07T02:38:11.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.914+0000 7fc88663eb80 10 client.26198 fetch_fsmap learned FSMap version 44 2024-05-07T02:38:11.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.914+0000 7fc88663eb80 1 -- 172.21.15.19:0/3294050648 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x558877b4a350 con 0x558877b32f70 2024-05-07T02:38:11.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.915+0000 7fc856a00700 1 -- 172.21.15.19:0/3294050648 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc85c00bbd0 con 0x558877b32f70 2024-05-07T02:38:11.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.915+0000 7fc856a00700 1 -- 172.21.15.19:0/3294050648 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 44) ==== 8378+0+0 (secure 0 0 0) 0x7fc85c00f820 con 0x558877b32f70 2024-05-07T02:38:11.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.916+0000 7fc88663eb80 10 client.26198 fetch_fsmap finished waiting for FSMap version 44 2024-05-07T02:38:11.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.916+0000 7fc88663eb80 10 client.26198 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-07T02:38:11.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.916+0000 7fc88663eb80 20 client.26198 populate_metadata read hostname 'smithi019' 2024-05-07T02:38:11.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.916+0000 7fc88663eb80 1 --2- 172.21.15.19:0/3294050648 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x558877b4c820 0x558877b4ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:11.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.916+0000 7fc88663eb80 4 client.26198 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:38:11.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.916+0000 7fc88663eb80 1 -- 172.21.15.19:0/3294050648 --> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558877aff0e0 con 0x558877b4c820 2024-05-07T02:38:11.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.916+0000 7fc875000700 1 --2- 172.21.15.19:0/3294050648 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x558877b4c820 0x558877b4ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:11.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.917+0000 7fc875000700 1 --2- 172.21.15.19:0/3294050648 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x558877b4c820 0x558877b4ec40 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b77e4d73ac2788f1 server_cookie=a55714015a6b0213 in_seq=0 out_seq=0 2024-05-07T02:38:11.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.917+0000 7fc856a00700 10 client.26198 ms_handle_connect on v2:172.21.15.64:6836/2585907777 2024-05-07T02:38:11.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.917+0000 7fc856a00700 1 -- 172.21.15.19:0/3294050648 <== mds.4 v2:172.21.15.64:6836/2585907777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558877aff0e0 con 0x558877b4c820 2024-05-07T02:38:11.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.918+0000 7fc856a00700 10 client.26198 handle_command_reply: tid=0 2024-05-07T02:38:11.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.918+0000 7fc88663eb80 1 client.26198 shutdown 2024-05-07T02:38:11.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.918+0000 7fc88663eb80 1 -- 172.21.15.19:0/3294050648 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558877b32f70 msgr2=0x558877b3cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:11.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.918+0000 7fc88663eb80 1 --2- 172.21.15.19:0/3294050648 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558877b32f70 0x558877b3cb70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc85c0098e0 tx=0x7fc85c005b00 comp rx=0 tx=0).stop 2024-05-07T02:38:11.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.918+0000 7fc88663eb80 1 -- 172.21.15.19:0/3294050648 shutdown_connections 2024-05-07T02:38:11.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.918+0000 7fc88663eb80 1 --2- 172.21.15.19:0/3294050648 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x558877b4c820 0x558877b4ec40 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:11.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.918+0000 7fc88663eb80 1 --2- 172.21.15.19:0/3294050648 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558877b38470 0x558877b3d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:11.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.918+0000 7fc88663eb80 1 --2- 172.21.15.19:0/3294050648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558877b32620 0x558877b3c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:11.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.918+0000 7fc88663eb80 1 --2- 172.21.15.19:0/3294050648 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558877b32f70 0x558877b3cb70 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:11.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.918+0000 7fc88663eb80 1 -- 172.21.15.19:0/3294050648 >> 172.21.15.19:0/3294050648 conn(0x5588779b4e40 msgr2=0x558877b31610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:11.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.918+0000 7fc856a00700 0 client.26198 ms_handle_reset on v2:172.21.15.64:6836/2585907777 2024-05-07T02:38:11.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.919+0000 7fc88663eb80 1 -- 172.21.15.19:0/3294050648 shutdown_connections 2024-05-07T02:38:11.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.919+0000 7fc88663eb80 1 -- 172.21.15.19:0/3294050648 wait complete. 2024-05-07T02:38:11.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.919+0000 7fc88663eb80 20 client.26198 trim_cache size 0 max 16384 2024-05-07T02:38:11.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.937+0000 7fc88663eb80 1 Processor -- start 2024-05-07T02:38:11.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.937+0000 7fc88663eb80 1 -- start start 2024-05-07T02:38:11.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.938+0000 7fc88663eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558877b32620 0x558877b3b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:11.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.938+0000 7fc88663eb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558877b32f70 0x558877b3d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:11.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.938+0000 7fc88663eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558877b38470 0x558877b48c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:11.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.938+0000 7fc88663eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc8701b6bf0 con 0x558877b32620 2024-05-07T02:38:11.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.938+0000 7fc88663eb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc8701b87f0 con 0x558877b32f70 2024-05-07T02:38:11.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.938+0000 7fc88663eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc8701b72c0 con 0x558877b38470 2024-05-07T02:38:11.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.938+0000 7fc875a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558877b38470 0x558877b48c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:11.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.938+0000 7fc875a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558877b38470 0x558877b48c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:38792/0 (socket says 172.21.15.19:38792) 2024-05-07T02:38:11.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.938+0000 7fc875000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558877b32620 0x558877b3b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:11.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.938+0000 7fc86fe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558877b32f70 0x558877b3d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:11.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.938+0000 7fc875a00700 1 -- 172.21.15.19:0/1654177141 learned_addr learned my addr 172.21.15.19:0/1654177141 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:38:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.939+0000 7fc875a00700 1 -- 172.21.15.19:0/1654177141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558877b32f70 msgr2=0x558877b3d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.939+0000 7fc875a00700 1 --2- 172.21.15.19:0/1654177141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558877b32f70 0x558877b3d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.939+0000 7fc875a00700 1 -- 172.21.15.19:0/1654177141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558877b32620 msgr2=0x558877b3b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.939+0000 7fc875a00700 1 --2- 172.21.15.19:0/1654177141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558877b32620 0x558877b3b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.939+0000 7fc875a00700 1 -- 172.21.15.19:0/1654177141 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558877b2d890 con 0x558877b38470 2024-05-07T02:38:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.939+0000 7fc86fe00700 1 --2- 172.21.15.19:0/1654177141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558877b32f70 0x558877b3d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:38:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.939+0000 7fc875a00700 1 --2- 172.21.15.19:0/1654177141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558877b38470 0x558877b48c00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc868017810 tx=0x7fc868009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=186d9084209fee83 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.940+0000 7fc861600700 1 -- 172.21.15.19:0/1654177141 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc868022590 con 0x558877b38470 2024-05-07T02:38:11.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.940+0000 7fc861600700 1 -- 172.21.15.19:0/1654177141 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc868028070 con 0x558877b38470 2024-05-07T02:38:11.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.940+0000 7fc861600700 1 -- 172.21.15.19:0/1654177141 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc868023b00 con 0x558877b38470 2024-05-07T02:38:11.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.941+0000 7fc88663eb80 1 -- 172.21.15.19:0/1654177141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558877b38470 msgr2=0x558877b48c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:11.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.941+0000 7fc88663eb80 1 --2- 172.21.15.19:0/1654177141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558877b38470 0x558877b48c00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc868017810 tx=0x7fc868009920 comp rx=0 tx=0).stop 2024-05-07T02:38:11.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.941+0000 7fc88663eb80 1 -- 172.21.15.19:0/1654177141 shutdown_connections 2024-05-07T02:38:11.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.941+0000 7fc88663eb80 1 --2- 172.21.15.19:0/1654177141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558877b38470 0x558877b48c00 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:11.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.941+0000 7fc88663eb80 1 --2- 172.21.15.19:0/1654177141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558877b32620 0x558877b3b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:11.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.941+0000 7fc88663eb80 1 --2- 172.21.15.19:0/1654177141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558877b32f70 0x558877b3d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:11.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.941+0000 7fc88663eb80 1 -- 172.21.15.19:0/1654177141 >> 172.21.15.19:0/1654177141 conn(0x5588779b4e40 msgr2=0x558877b271c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:11.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.941+0000 7fc88663eb80 1 -- 172.21.15.19:0/1654177141 shutdown_connections 2024-05-07T02:38:11.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.941+0000 7fc88663eb80 1 -- 172.21.15.19:0/1654177141 wait complete. 2024-05-07T02:38:11.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.941+0000 7fc88663eb80 1 -- start start 2024-05-07T02:38:11.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.942+0000 7fc88663eb80 1 Processor -- start 2024-05-07T02:38:11.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.942+0000 7fc88663eb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558877b32620 0x558877b521c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:11.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.942+0000 7fc88663eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558877b32f70 0x558877b4f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:11.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.942+0000 7fc88663eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558877b38470 0x558877b4fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:11.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.942+0000 7fc88663eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc8701b8060 con 0x558877b32f70 2024-05-07T02:38:11.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.942+0000 7fc88663eb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc8701b7990 con 0x558877b32620 2024-05-07T02:38:11.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.942+0000 7fc88663eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc8701b4b90 con 0x558877b38470 2024-05-07T02:38:11.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.942+0000 7fc88663eb80 10 client.0 _ll_register_callbacks cb 0x5588779e8380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:38:11.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.943+0000 7fc875000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558877b32620 0x558877b521c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:11.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.943+0000 7fc875000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558877b32620 0x558877b521c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:59492/0 (socket says 172.21.15.19:59492) 2024-05-07T02:38:11.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.943+0000 7fc86fe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558877b32f70 0x558877b4f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:11.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.943+0000 7fc875000700 1 -- 172.21.15.19:0/79265101 learned_addr learned my addr 172.21.15.19:0/79265101 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:38:11.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.943+0000 7fc875a00700 1 --2- 172.21.15.19:0/79265101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558877b38470 0x558877b4fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:11.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.943+0000 7fc875000700 1 -- 172.21.15.19:0/79265101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558877b38470 msgr2=0x558877b4fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:11.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.943+0000 7fc875000700 1 --2- 172.21.15.19:0/79265101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558877b38470 0x558877b4fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:11.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.943+0000 7fc875000700 1 -- 172.21.15.19:0/79265101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558877b32f70 msgr2=0x558877b4f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:11.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.943+0000 7fc875000700 1 --2- 172.21.15.19:0/79265101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558877b32f70 0x558877b4f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:11.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.943+0000 7fc875000700 1 -- 172.21.15.19:0/79265101 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558877b50220 con 0x558877b32620 2024-05-07T02:38:11.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.943+0000 7fc875000700 1 --2- 172.21.15.19:0/79265101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558877b32620 0x558877b521c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc8640a5b90 tx=0x7fc8640a5a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a796d21e1f20cfc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:11.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.943+0000 7fc856a00700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:38:11.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.944+0000 7fc856a00700 1 -- 172.21.15.19:0/79265101 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8640aa210 con 0x558877b32620 2024-05-07T02:38:11.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.944+0000 7fc856a00700 1 -- 172.21.15.19:0/79265101 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fc8640a6100 con 0x558877b32620 2024-05-07T02:38:11.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.944+0000 7fc88663eb80 1 -- 172.21.15.19:0/79265101 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc868028070 con 0x558877b32620 2024-05-07T02:38:11.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.944+0000 7fc856a00700 1 -- 172.21.15.19:0/79265101 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc8640a9b10 con 0x558877b32620 2024-05-07T02:38:11.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.944+0000 7fc88663eb80 10 client.26204 fetch_fsmap learned FSMap version 44 2024-05-07T02:38:11.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.944+0000 7fc88663eb80 1 -- 172.21.15.19:0/79265101 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x558877bb6cc0 con 0x558877b32620 2024-05-07T02:38:11.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.944+0000 7fc856a00700 1 -- 172.21.15.19:0/79265101 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc868028070 con 0x558877b32620 2024-05-07T02:38:11.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.944+0000 7fc856a00700 1 -- 172.21.15.19:0/79265101 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 44) ==== 8378+0+0 (secure 0 0 0) 0x7fc8640b24e0 con 0x558877b32620 2024-05-07T02:38:11.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.944+0000 7fc88663eb80 10 client.26204 fetch_fsmap finished waiting for FSMap version 44 2024-05-07T02:38:11.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.944+0000 7fc88663eb80 10 client.26204 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-07T02:38:11.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.944+0000 7fc88663eb80 20 client.26204 populate_metadata read hostname 'smithi019' 2024-05-07T02:38:11.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.944+0000 7fc88663eb80 1 --2- 172.21.15.19:0/79265101 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x558877bb7370 0x558877bb9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:11.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.944+0000 7fc88663eb80 4 client.26204 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:38:11.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.944+0000 7fc88663eb80 1 -- 172.21.15.19:0/79265101 --> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558877a62400 con 0x558877bb7370 2024-05-07T02:38:11.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.945+0000 7fc86fe00700 1 --2- 172.21.15.19:0/79265101 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x558877bb7370 0x558877bb9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:11.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.945+0000 7fc86fe00700 1 --2- 172.21.15.19:0/79265101 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x558877bb7370 0x558877bb9790 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1f671d7ed7e6e593 server_cookie=14c4e504c21c1967 in_seq=0 out_seq=0 2024-05-07T02:38:11.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.945+0000 7fc856a00700 10 client.26204 ms_handle_connect on v2:172.21.15.64:6836/2585907777 2024-05-07T02:38:11.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.946+0000 7fc856a00700 1 -- 172.21.15.19:0/79265101 <== mds.4 v2:172.21.15.64:6836/2585907777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558877a62400 con 0x558877bb7370 2024-05-07T02:38:11.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.946+0000 7fc856a00700 10 client.26204 handle_command_reply: tid=0 2024-05-07T02:38:11.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.946+0000 7fc88663eb80 1 client.26204 shutdown 2024-05-07T02:38:11.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.946+0000 7fc88663eb80 1 -- 172.21.15.19:0/79265101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558877b32620 msgr2=0x558877b521c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:11.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.946+0000 7fc88663eb80 1 --2- 172.21.15.19:0/79265101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558877b32620 0x558877b521c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc8640a5b90 tx=0x7fc8640a5a80 comp rx=0 tx=0).stop 2024-05-07T02:38:11.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.946+0000 7fc88663eb80 1 -- 172.21.15.19:0/79265101 shutdown_connections 2024-05-07T02:38:11.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.946+0000 7fc88663eb80 1 --2- 172.21.15.19:0/79265101 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x558877bb7370 0x558877bb9790 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:11.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.946+0000 7fc88663eb80 1 --2- 172.21.15.19:0/79265101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558877b38470 0x558877b4fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:11.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.946+0000 7fc88663eb80 1 --2- 172.21.15.19:0/79265101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558877b32620 0x558877b521c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:11.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.946+0000 7fc88663eb80 1 --2- 172.21.15.19:0/79265101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558877b32f70 0x558877b4f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:11.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.946+0000 7fc88663eb80 1 -- 172.21.15.19:0/79265101 >> 172.21.15.19:0/79265101 conn(0x5588779b4e40 msgr2=0x558877b3cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:11.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.946+0000 7fc856a00700 0 client.26204 ms_handle_reset on v2:172.21.15.64:6836/2585907777 2024-05-07T02:38:11.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.946+0000 7fc88663eb80 1 -- 172.21.15.19:0/79265101 shutdown_connections 2024-05-07T02:38:11.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.946+0000 7fc88663eb80 1 -- 172.21.15.19:0/79265101 wait complete. 2024-05-07T02:38:11.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.946+0000 7fc88663eb80 20 client.26204 trim_cache size 0 max 16384 2024-05-07T02:38:11.949 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:38:11.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.947+0000 7fc877a00700 1 -- 172.21.15.19:0/2642591950 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc8580b2a70 msgr2=0x7fc8580b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:11.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.947+0000 7fc877a00700 1 --2- 172.21.15.19:0/2642591950 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc8580b2a70 0x7fc8580b4f80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc868009fd0 tx=0x7fc86800b040 comp rx=0 tx=0).stop 2024-05-07T02:38:11.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.947+0000 7fc877a00700 1 -- 172.21.15.19:0/2642591950 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc870075b40 msgr2=0x7fc8701042e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:11.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.947+0000 7fc877a00700 1 --2- 172.21.15.19:0/2642591950 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc870075b40 0x7fc8701042e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc86400e9a0 tx=0x7fc86400ecb0 comp rx=0 tx=0).stop 2024-05-07T02:38:11.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.947+0000 7fc877a00700 1 -- 172.21.15.19:0/2642591950 shutdown_connections 2024-05-07T02:38:11.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.947+0000 7fc877a00700 1 --2- 172.21.15.19:0/2642591950 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc870110bc0 0x7fc870104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:11.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.947+0000 7fc877a00700 1 --2- 172.21.15.19:0/2642591950 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc870075b40 0x7fc8701042e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:11.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.947+0000 7fc877a00700 1 --2- 172.21.15.19:0/2642591950 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc8580b2a70 0x7fc8580b4f80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:11.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.947+0000 7fc877a00700 1 --2- 172.21.15.19:0/2642591950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc870076490 0x7fc870104820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:11.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.947+0000 7fc877a00700 1 -- 172.21.15.19:0/2642591950 >> 172.21.15.19:0/2642591950 conn(0x7fc870100040 msgr2=0x7fc8701006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:11.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.948+0000 7fc877a00700 1 -- 172.21.15.19:0/2642591950 shutdown_connections 2024-05-07T02:38:11.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:11.948+0000 7fc877a00700 1 -- 172.21.15.19:0/2642591950 wait complete. 2024-05-07T02:38:12.762 INFO:tasks.workunit.client.0.smithi019.stdout: 1 214512 4.04 MB/sec execute 188 sec latency 859.159 ms 2024-05-07T02:38:12.979 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-07T02:38:13.477 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:38:13.680 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:13 smithi019 ceph-mon[29781]: pgmap v194: 129 pgs: 129 active+clean; 149 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 183 op/s 2024-05-07T02:38:13.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:13 smithi027 ceph-mon[34398]: pgmap v194: 129 pgs: 129 active+clean; 149 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 183 op/s 2024-05-07T02:38:13.762 INFO:tasks.workunit.client.0.smithi019.stdout: 1 214512 4.02 MB/sec execute 189 sec latency 1859.250 ms 2024-05-07T02:38:13.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:13 smithi064 ceph-mon[34699]: pgmap v194: 129 pgs: 129 active+clean; 149 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 183 op/s 2024-05-07T02:38:14.762 INFO:tasks.workunit.client.0.smithi019.stdout: 1 214722 4.00 MB/sec execute 190 sec latency 2697.244 ms 2024-05-07T02:38:15.685 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:15 smithi019 ceph-mon[29781]: pgmap v195: 129 pgs: 129 active+clean; 149 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-05-07T02:38:15.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:15 smithi027 ceph-mon[34398]: pgmap v195: 129 pgs: 129 active+clean; 149 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-05-07T02:38:15.762 INFO:tasks.workunit.client.0.smithi019.stdout: 1 215891 4.01 MB/sec execute 191 sec latency 28.489 ms 2024-05-07T02:38:15.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:15 smithi064 ceph-mon[34699]: pgmap v195: 129 pgs: 129 active+clean; 149 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-05-07T02:38:16.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.675+0000 7f69a1000700 1 -- 172.21.15.19:0/3465503628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f699c104400 msgr2=0x7f699c104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:16.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.675+0000 7f69a1000700 1 --2- 172.21.15.19:0/3465503628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f699c104400 0x7f699c104810 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f698c009920 tx=0x7f698c009c30 comp rx=0 tx=0).stop 2024-05-07T02:38:16.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.675+0000 7f69a1000700 1 -- 172.21.15.19:0/3465503628 shutdown_connections 2024-05-07T02:38:16.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.675+0000 7f69a1000700 1 --2- 172.21.15.19:0/3465503628 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f699c109680 0x7f699c110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:16.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.675+0000 7f69a1000700 1 --2- 172.21.15.19:0/3465503628 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f699c104400 0x7f699c104810 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:16.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.675+0000 7f69a1000700 1 --2- 172.21.15.19:0/3465503628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f699c108d30 0x7f699c109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:16.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.675+0000 7f69a1000700 1 -- 172.21.15.19:0/3465503628 >> 172.21.15.19:0/3465503628 conn(0x7f699c079d10 msgr2=0x7f699c07a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:16.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.676+0000 7f69a1000700 1 -- 172.21.15.19:0/3465503628 shutdown_connections 2024-05-07T02:38:16.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.676+0000 7f69a1000700 1 -- 172.21.15.19:0/3465503628 wait complete. 2024-05-07T02:38:16.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.678+0000 7f69a1000700 1 Processor -- start 2024-05-07T02:38:16.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.678+0000 7f69a1000700 1 -- start start 2024-05-07T02:38:16.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.678+0000 7f69a1000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f699c104400 0x7f699c19ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:16.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.678+0000 7f69a1000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f699c108d30 0x7f699c19b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:16.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.678+0000 7f69a1000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f699c109680 0x7f699c19f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:16.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.678+0000 7f69a1000700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f699c114300 con 0x7f699c109680 2024-05-07T02:38:16.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.678+0000 7f69a1000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f699c114180 con 0x7f699c104400 2024-05-07T02:38:16.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.678+0000 7f69a1000700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f699c114480 con 0x7f699c108d30 2024-05-07T02:38:16.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.679+0000 7f699aa00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f699c108d30 0x7f699c19b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:16.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.679+0000 7f699b400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f699c104400 0x7f699c19ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:16.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.679+0000 7f699aa00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f699c108d30 0x7f699c19b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:38822/0 (socket says 172.21.15.19:38822) 2024-05-07T02:38:16.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.679+0000 7f699aa00700 1 -- 172.21.15.19:0/817279211 learned_addr learned my addr 172.21.15.19:0/817279211 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:38:16.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.679+0000 7f699be00700 1 --2- 172.21.15.19:0/817279211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f699c109680 0x7f699c19f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:16.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.679+0000 7f699b400700 1 -- 172.21.15.19:0/817279211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f699c108d30 msgr2=0x7f699c19b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:16.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.679+0000 7f699b400700 1 --2- 172.21.15.19:0/817279211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f699c108d30 0x7f699c19b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:16.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.679+0000 7f699b400700 1 -- 172.21.15.19:0/817279211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f699c109680 msgr2=0x7f699c19f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:16.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.679+0000 7f699b400700 1 --2- 172.21.15.19:0/817279211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f699c109680 0x7f699c19f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:16.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.679+0000 7f699b400700 1 -- 172.21.15.19:0/817279211 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f699c19fc20 con 0x7f699c104400 2024-05-07T02:38:16.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.680+0000 7f699b400700 1 --2- 172.21.15.19:0/817279211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f699c104400 0x7f699c19ad50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f698c000c00 tx=0x7f698c00f950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:16.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.680+0000 7f6993e00700 1 -- 172.21.15.19:0/817279211 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c027070 con 0x7f699c104400 2024-05-07T02:38:16.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.680+0000 7f69a1000700 1 -- 172.21.15.19:0/817279211 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f699c19feb0 con 0x7f699c104400 2024-05-07T02:38:16.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.680+0000 7f6993e00700 1 -- 172.21.15.19:0/817279211 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f698c0054f0 con 0x7f699c104400 2024-05-07T02:38:16.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.680+0000 7f6993e00700 1 -- 172.21.15.19:0/817279211 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c004930 con 0x7f699c104400 2024-05-07T02:38:16.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.680+0000 7f69a1000700 1 -- 172.21.15.19:0/817279211 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f699c1a0390 con 0x7f699c104400 2024-05-07T02:38:16.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.684+0000 7f6993e00700 1 -- 172.21.15.19:0/817279211 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f698c02a070 con 0x7f699c104400 2024-05-07T02:38:16.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.686+0000 7f69b1086b80 1 Processor -- start 2024-05-07T02:38:16.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.686+0000 7f69b1086b80 1 -- start start 2024-05-07T02:38:16.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.687+0000 7f69b1086b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558dc3f0ec40 0x558dc3f0f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:16.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.687+0000 7f69b1086b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc3f0f590 0x558dc3f186f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:16.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.687+0000 7f69b1086b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558dc3f18c30 0x558dc3f1b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:16.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.687+0000 7f69b1086b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558dc3e30a20 con 0x558dc3f18c30 2024-05-07T02:38:16.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.687+0000 7f69b1086b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x558dc3dce760 con 0x558dc3f0ec40 2024-05-07T02:38:16.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.687+0000 7f69b1086b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558dc3e2c9e0 con 0x558dc3f0f590 2024-05-07T02:38:16.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.687+0000 7f6993e00700 1 --2- 172.21.15.19:0/817279211 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f69800b2a70 0x7f69800b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:16.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.687+0000 7f699be00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558dc3f0ec40 0x558dc3f0f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:16.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.687+0000 7f699be00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558dc3f0ec40 0x558dc3f0f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:59514/0 (socket says 172.21.15.19:59514) 2024-05-07T02:38:16.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.687+0000 7f699be00700 1 -- 172.21.15.19:0/839938268 learned_addr learned my addr 172.21.15.19:0/839938268 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:38:16.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.687+0000 7f699aa00700 1 --2- 172.21.15.19:0/839938268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558dc3f18c30 0x558dc3f1b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:16.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.687+0000 7f699b400700 1 --2- 172.21.15.19:0/839938268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc3f0f590 0x558dc3f186f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:16.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.688+0000 7f699be00700 1 --2- 172.21.15.19:0/817279211 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f69800b2a70 0x7f69800b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:16.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.688+0000 7f6993e00700 1 -- 172.21.15.19:0/817279211 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f698c0e3220 con 0x7f699c104400 2024-05-07T02:38:16.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.688+0000 7f699b400700 1 -- 172.21.15.19:0/839938268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558dc3f0ec40 msgr2=0x558dc3f0f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:16.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.688+0000 7f699b400700 1 --2- 172.21.15.19:0/839938268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558dc3f0ec40 0x558dc3f0f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:16.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.688+0000 7f699b400700 1 -- 172.21.15.19:0/839938268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558dc3f18c30 msgr2=0x558dc3f1b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:16.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.688+0000 7f699b400700 1 --2- 172.21.15.19:0/839938268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558dc3f18c30 0x558dc3f1b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:16.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.688+0000 7f699b400700 1 -- 172.21.15.19:0/839938268 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dc3d7d8d0 con 0x558dc3f0f590 2024-05-07T02:38:16.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.688+0000 7f699b400700 1 --2- 172.21.15.19:0/839938268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc3f0f590 0x558dc3f186f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f698c0e0580 tx=0x7f698c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe86d8bb13e66feb server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:16.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.689+0000 7f6990c00700 1 -- 172.21.15.19:0/839938268 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c0e0a10 con 0x558dc3f0f590 2024-05-07T02:38:16.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.689+0000 7f6990c00700 1 -- 172.21.15.19:0/839938268 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f698c0a9cb0 con 0x558dc3f0f590 2024-05-07T02:38:16.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.689+0000 7f6990c00700 1 -- 172.21.15.19:0/839938268 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c0ae0e0 con 0x558dc3f0f590 2024-05-07T02:38:16.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.690+0000 7f69b1086b80 1 -- 172.21.15.19:0/839938268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc3f0f590 msgr2=0x558dc3f186f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:16.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.690+0000 7f69b1086b80 1 --2- 172.21.15.19:0/839938268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc3f0f590 0x558dc3f186f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f698c0e0580 tx=0x7f698c009c30 comp rx=0 tx=0).stop 2024-05-07T02:38:16.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.690+0000 7f699be00700 1 --2- 172.21.15.19:0/817279211 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f69800b2a70 0x7f69800b4f80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x558dc3f04190 tx=0x7f6994004b40 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:16.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.690+0000 7f69b1086b80 1 -- 172.21.15.19:0/839938268 shutdown_connections 2024-05-07T02:38:16.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.690+0000 7f69b1086b80 1 --2- 172.21.15.19:0/839938268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558dc3f18c30 0x558dc3f1b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:16.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.690+0000 7f69b1086b80 1 --2- 172.21.15.19:0/839938268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558dc3f0ec40 0x558dc3f0f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:16.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.690+0000 7f69b1086b80 1 --2- 172.21.15.19:0/839938268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc3f0f590 0x558dc3f186f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:16.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.690+0000 7f69b1086b80 1 -- 172.21.15.19:0/839938268 >> 172.21.15.19:0/839938268 conn(0x558dc3d90e40 msgr2=0x558dc3f09980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:16.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.690+0000 7f69b1086b80 1 -- 172.21.15.19:0/839938268 shutdown_connections 2024-05-07T02:38:16.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.690+0000 7f69b1086b80 1 -- 172.21.15.19:0/839938268 wait complete. 2024-05-07T02:38:16.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.691+0000 7f69b1086b80 1 -- start start 2024-05-07T02:38:16.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.691+0000 7f69b1086b80 1 Processor -- start 2024-05-07T02:38:16.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.692+0000 7f69b1086b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc3f0ec40 0x558dc3f15c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:16.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.692+0000 7f69b1086b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558dc3f0f590 0x558dc3f16160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:16.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.692+0000 7f69b1086b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558dc3f18c30 0x558dc3f166a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:16.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.692+0000 7f69b1086b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558dc3f1b7c0 con 0x558dc3f18c30 2024-05-07T02:38:16.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.692+0000 7f69b1086b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x558dc3d96b20 con 0x558dc3f0f590 2024-05-07T02:38:16.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.692+0000 7f69b1086b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558dc3f1b940 con 0x558dc3f0ec40 2024-05-07T02:38:16.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.692+0000 7f69b1086b80 10 client.0 _ll_register_callbacks cb 0x558dc3dc4380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:38:16.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.693+0000 7f699aa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558dc3f0f590 0x558dc3f16160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:16.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.693+0000 7f699be00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558dc3f18c30 0x558dc3f166a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:16.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.693+0000 7f699aa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558dc3f0f590 0x558dc3f16160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:59526/0 (socket says 172.21.15.19:59526) 2024-05-07T02:38:16.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.693+0000 7f699aa00700 1 -- 172.21.15.19:0/1847851374 learned_addr learned my addr 172.21.15.19:0/1847851374 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:38:16.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.693+0000 7f699aa00700 1 -- 172.21.15.19:0/1847851374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc3f0ec40 msgr2=0x558dc3f15c20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-07T02:38:16.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.693+0000 7f699aa00700 1 --2- 172.21.15.19:0/1847851374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc3f0ec40 0x558dc3f15c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:16.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.693+0000 7f699aa00700 1 -- 172.21.15.19:0/1847851374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558dc3f18c30 msgr2=0x558dc3f166a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:16.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.693+0000 7f699aa00700 1 --2- 172.21.15.19:0/1847851374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558dc3f18c30 0x558dc3f166a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:16.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.693+0000 7f699aa00700 1 -- 172.21.15.19:0/1847851374 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dc3df6420 con 0x558dc3f0f590 2024-05-07T02:38:16.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.694+0000 7f699aa00700 1 --2- 172.21.15.19:0/1847851374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558dc3f0f590 0x558dc3f16160 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6988012840 tx=0x7f6988012c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d78cc251d6238f4b server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:16.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.694+0000 7f6985600700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:38:16.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.695+0000 7f6985600700 1 -- 172.21.15.19:0/1847851374 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6988004050 con 0x558dc3f0f590 2024-05-07T02:38:16.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.695+0000 7f6985600700 1 -- 172.21.15.19:0/1847851374 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f69880041f0 con 0x558dc3f0f590 2024-05-07T02:38:16.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.695+0000 7f69b1086b80 1 -- 172.21.15.19:0/1847851374 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f698c0a9cb0 con 0x558dc3f0f590 2024-05-07T02:38:16.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.695+0000 7f6985600700 1 -- 172.21.15.19:0/1847851374 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f6988025050 con 0x558dc3f0f590 2024-05-07T02:38:16.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.695+0000 7f69b1086b80 10 client.26216 fetch_fsmap learned FSMap version 44 2024-05-07T02:38:16.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.695+0000 7f69b1086b80 1 -- 172.21.15.19:0/1847851374 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x558dc3f26360 con 0x558dc3f0f590 2024-05-07T02:38:16.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.695+0000 7f6985600700 1 -- 172.21.15.19:0/1847851374 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f698c0a9cb0 con 0x558dc3f0f590 2024-05-07T02:38:16.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.696+0000 7f6985600700 1 -- 172.21.15.19:0/1847851374 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 44) ==== 8378+0+0 (secure 0 0 0) 0x7f698800f660 con 0x558dc3f0f590 2024-05-07T02:38:16.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.696+0000 7f69b1086b80 10 client.26216 fetch_fsmap finished waiting for FSMap version 44 2024-05-07T02:38:16.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.696+0000 7f69b1086b80 10 client.26216 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-05-07T02:38:16.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.696+0000 7f69b1086b80 20 client.26216 populate_metadata read hostname 'smithi019' 2024-05-07T02:38:16.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.696+0000 7f69b1086b80 1 --2- 172.21.15.19:0/1847851374 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x558dc3f289f0 0x558dc3f2ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:16.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.696+0000 7f69b1086b80 4 client.26216 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:38:16.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.696+0000 7f69b1086b80 1 -- 172.21.15.19:0/1847851374 --> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558dc3edb0e0 con 0x558dc3f289f0 2024-05-07T02:38:16.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.697+0000 7f699b400700 1 --2- 172.21.15.19:0/1847851374 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x558dc3f289f0 0x558dc3f2ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:16.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.698+0000 7f699b400700 1 --2- 172.21.15.19:0/1847851374 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x558dc3f289f0 0x558dc3f2ae90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6c38c6a6561256e2 server_cookie=9869de69bb6cf086 in_seq=0 out_seq=0 2024-05-07T02:38:16.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.698+0000 7f6985600700 10 client.26216 ms_handle_connect on v2:172.21.15.27:6836/2583138143 2024-05-07T02:38:16.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.699+0000 7f6985600700 1 -- 172.21.15.19:0/1847851374 <== mds.3 v2:172.21.15.27:6836/2583138143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558dc3edb0e0 con 0x558dc3f289f0 2024-05-07T02:38:16.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.699+0000 7f6985600700 10 client.26216 handle_command_reply: tid=0 2024-05-07T02:38:16.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.699+0000 7f69b1086b80 1 client.26216 shutdown 2024-05-07T02:38:16.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.699+0000 7f69b1086b80 1 -- 172.21.15.19:0/1847851374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558dc3f0f590 msgr2=0x558dc3f16160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:16.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.699+0000 7f69b1086b80 1 --2- 172.21.15.19:0/1847851374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558dc3f0f590 0x558dc3f16160 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6988012840 tx=0x7f6988012c00 comp rx=0 tx=0).stop 2024-05-07T02:38:16.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.699+0000 7f69b1086b80 1 -- 172.21.15.19:0/1847851374 shutdown_connections 2024-05-07T02:38:16.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.699+0000 7f69b1086b80 1 --2- 172.21.15.19:0/1847851374 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x558dc3f289f0 0x558dc3f2ae90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:16.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.699+0000 7f69b1086b80 1 --2- 172.21.15.19:0/1847851374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558dc3f18c30 0x558dc3f166a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:16.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.699+0000 7f69b1086b80 1 --2- 172.21.15.19:0/1847851374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc3f0ec40 0x558dc3f15c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:16.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.699+0000 7f69b1086b80 1 --2- 172.21.15.19:0/1847851374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558dc3f0f590 0x558dc3f16160 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:16.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.699+0000 7f69b1086b80 1 -- 172.21.15.19:0/1847851374 >> 172.21.15.19:0/1847851374 conn(0x558dc3d90e40 msgr2=0x558dc3f0b600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:16.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.699+0000 7f6985600700 0 client.26216 ms_handle_reset on v2:172.21.15.27:6836/2583138143 2024-05-07T02:38:16.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.699+0000 7f69b1086b80 1 -- 172.21.15.19:0/1847851374 shutdown_connections 2024-05-07T02:38:16.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.700+0000 7f69b1086b80 1 -- 172.21.15.19:0/1847851374 wait complete. 2024-05-07T02:38:16.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.700+0000 7f69b1086b80 20 client.26216 trim_cache size 0 max 16384 2024-05-07T02:38:16.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.719+0000 7f69b1086b80 1 Processor -- start 2024-05-07T02:38:16.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.719+0000 7f69b1086b80 1 -- start start 2024-05-07T02:38:16.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.720+0000 7f69b1086b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc3d90e40 0x558dc3f14b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:16.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.720+0000 7f69b1086b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558dc3f0ec40 0x558dc3f164e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:16.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.720+0000 7f69b1086b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558dc3f0f590 0x558dc3f179e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:16.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.720+0000 7f69b1086b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1ae190 con 0x558dc3f0f590 2024-05-07T02:38:16.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.720+0000 7f69b1086b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1afd90 con 0x558dc3f0ec40 2024-05-07T02:38:16.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.720+0000 7f69b1086b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1ae860 con 0x558dc3d90e40 2024-05-07T02:38:16.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.720+0000 7f699b400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc3d90e40 0x558dc3f14b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:16.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.720+0000 7f699b400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc3d90e40 0x558dc3f14b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:38842/0 (socket says 172.21.15.19:38842) 2024-05-07T02:38:16.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.720+0000 7f699b400700 1 -- 172.21.15.19:0/1747668338 learned_addr learned my addr 172.21.15.19:0/1747668338 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:38:16.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.720+0000 7f699aa00700 1 --2- 172.21.15.19:0/1747668338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558dc3f0ec40 0x558dc3f164e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:16.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.721+0000 7f699be00700 1 --2- 172.21.15.19:0/1747668338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558dc3f0f590 0x558dc3f179e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:16.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.721+0000 7f699b400700 1 -- 172.21.15.19:0/1747668338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558dc3f0ec40 msgr2=0x558dc3f164e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:16.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.721+0000 7f699b400700 1 --2- 172.21.15.19:0/1747668338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558dc3f0ec40 0x558dc3f164e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:16.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.721+0000 7f699b400700 1 -- 172.21.15.19:0/1747668338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558dc3f0f590 msgr2=0x558dc3f179e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:16.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.721+0000 7f699b400700 1 --2- 172.21.15.19:0/1747668338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558dc3f0f590 0x558dc3f179e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:16.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.721+0000 7f699b400700 1 -- 172.21.15.19:0/1747668338 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dc3f05730 con 0x558dc3d90e40 2024-05-07T02:38:16.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.721+0000 7f699be00700 1 --2- 172.21.15.19:0/1747668338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558dc3f0f590 0x558dc3f179e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-07T02:38:16.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.721+0000 7f699b400700 1 --2- 172.21.15.19:0/1747668338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc3d90e40 0x558dc3f14b80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f698c0a7410 tx=0x7f698c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e12f5519503e1d58 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:16.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.721+0000 7f6987e00700 1 -- 172.21.15.19:0/1747668338 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c0a6060 con 0x558dc3d90e40 2024-05-07T02:38:16.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.721+0000 7f6987e00700 1 -- 172.21.15.19:0/1747668338 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f698c0a9cb0 con 0x558dc3d90e40 2024-05-07T02:38:16.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.721+0000 7f6987e00700 1 -- 172.21.15.19:0/1747668338 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c00b780 con 0x558dc3d90e40 2024-05-07T02:38:16.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.722+0000 7f69b1086b80 1 -- 172.21.15.19:0/1747668338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc3d90e40 msgr2=0x558dc3f14b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:16.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.722+0000 7f69b1086b80 1 --2- 172.21.15.19:0/1747668338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc3d90e40 0x558dc3f14b80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f698c0a7410 tx=0x7f698c009c30 comp rx=0 tx=0).stop 2024-05-07T02:38:16.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.722+0000 7f69b1086b80 1 -- 172.21.15.19:0/1747668338 shutdown_connections 2024-05-07T02:38:16.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.722+0000 7f69b1086b80 1 --2- 172.21.15.19:0/1747668338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558dc3f0f590 0x558dc3f179e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:16.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.722+0000 7f69b1086b80 1 --2- 172.21.15.19:0/1747668338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc3d90e40 0x558dc3f14b80 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:16.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.722+0000 7f69b1086b80 1 --2- 172.21.15.19:0/1747668338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558dc3f0ec40 0x558dc3f164e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:16.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.722+0000 7f69b1086b80 1 -- 172.21.15.19:0/1747668338 >> 172.21.15.19:0/1747668338 conn(0x558dc3f02d10 msgr2=0x558dc3f172d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:16.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.723+0000 7f69b1086b80 1 -- 172.21.15.19:0/1747668338 shutdown_connections 2024-05-07T02:38:16.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.723+0000 7f69b1086b80 1 -- 172.21.15.19:0/1747668338 wait complete. 2024-05-07T02:38:16.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.723+0000 7f69b1086b80 1 -- start start 2024-05-07T02:38:16.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.724+0000 7f69b1086b80 1 Processor -- start 2024-05-07T02:38:16.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.724+0000 7f69b1086b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558dc3d90e40 0x558dc3f20f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:16.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.724+0000 7f69b1086b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc3f0ec40 0x558dc3f1c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:16.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.724+0000 7f69b1086b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558dc3f0f590 0x558dc3f1c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:16.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.724+0000 7f69b1086b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1af600 con 0x558dc3d90e40 2024-05-07T02:38:16.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.724+0000 7f69b1086b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1aef30 con 0x558dc3f0f590 2024-05-07T02:38:16.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.724+0000 7f69b1086b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1ac130 con 0x558dc3f0ec40 2024-05-07T02:38:16.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.724+0000 7f69b1086b80 10 client.0 _ll_register_callbacks cb 0x558dc3dc4380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:38:16.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.725+0000 7f699b400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558dc3d90e40 0x558dc3f20f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:16.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.725+0000 7f699b400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558dc3d90e40 0x558dc3f20f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35196/0 (socket says 172.21.15.19:35196) 2024-05-07T02:38:16.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.725+0000 7f699b400700 1 -- 172.21.15.19:0/2915707852 learned_addr learned my addr 172.21.15.19:0/2915707852 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:38:16.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.725+0000 7f699aa00700 1 --2- 172.21.15.19:0/2915707852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc3f0ec40 0x558dc3f1c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:16.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.725+0000 7f699b400700 1 -- 172.21.15.19:0/2915707852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc3f0ec40 msgr2=0x558dc3f1c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:16.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.725+0000 7f699b400700 1 --2- 172.21.15.19:0/2915707852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc3f0ec40 0x558dc3f1c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:16.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.725+0000 7f699b400700 1 -- 172.21.15.19:0/2915707852 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558dc3f0f590 msgr2=0x558dc3f1c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:16.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.725+0000 7f699be00700 1 --2- 172.21.15.19:0/2915707852 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558dc3f0f590 0x558dc3f1c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:16.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.725+0000 7f699b400700 1 --2- 172.21.15.19:0/2915707852 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558dc3f0f590 0x558dc3f1c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:16.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.725+0000 7f699b400700 1 -- 172.21.15.19:0/2915707852 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dc3f0a4c0 con 0x558dc3d90e40 2024-05-07T02:38:16.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.725+0000 7f699aa00700 1 --2- 172.21.15.19:0/2915707852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc3f0ec40 0x558dc3f1c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:38:16.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.725+0000 7f699be00700 1 --2- 172.21.15.19:0/2915707852 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558dc3f0f590 0x558dc3f1c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-07T02:38:16.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.726+0000 7f699b400700 1 --2- 172.21.15.19:0/2915707852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558dc3d90e40 0x558dc3f20f60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f698c0aa2e0 tx=0x7f698c0e0680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17be28f52beed82 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:16.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.726+0000 7f6985600700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-07T02:38:16.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.726+0000 7f6985600700 1 -- 172.21.15.19:0/2915707852 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c089060 con 0x558dc3d90e40 2024-05-07T02:38:16.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.726+0000 7f6985600700 1 -- 172.21.15.19:0/2915707852 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f698c00b560 con 0x558dc3d90e40 2024-05-07T02:38:16.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.726+0000 7f69b1086b80 1 -- 172.21.15.19:0/2915707852 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f698c0a9cb0 con 0x558dc3d90e40 2024-05-07T02:38:16.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.726+0000 7f6985600700 1 -- 172.21.15.19:0/2915707852 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f698c0c8070 con 0x558dc3d90e40 2024-05-07T02:38:16.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.727+0000 7f69b1086b80 10 client.16020 fetch_fsmap learned FSMap version 44 2024-05-07T02:38:16.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.727+0000 7f69b1086b80 1 -- 172.21.15.19:0/2915707852 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x558dc3f92ee0 con 0x558dc3d90e40 2024-05-07T02:38:16.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.727+0000 7f6985600700 1 -- 172.21.15.19:0/2915707852 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f698c0a9cb0 con 0x558dc3d90e40 2024-05-07T02:38:16.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.727+0000 7f6985600700 1 -- 172.21.15.19:0/2915707852 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 44) ==== 8378+0+0 (secure 0 0 0) 0x7f698c0aa400 con 0x558dc3d90e40 2024-05-07T02:38:16.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.727+0000 7f69b1086b80 10 client.16020 fetch_fsmap finished waiting for FSMap version 44 2024-05-07T02:38:16.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.727+0000 7f69b1086b80 10 client.16020 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-05-07T02:38:16.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.727+0000 7f69b1086b80 20 client.16020 populate_metadata read hostname 'smithi019' 2024-05-07T02:38:16.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.727+0000 7f69b1086b80 1 --2- 172.21.15.19:0/2915707852 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x558dc3f18c30 0x558dc3f951c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:16.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.727+0000 7f69b1086b80 4 client.16020 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:38:16.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.727+0000 7f69b1086b80 1 -- 172.21.15.19:0/2915707852 --> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558dc3e3e400 con 0x558dc3f18c30 2024-05-07T02:38:16.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.728+0000 7f699aa00700 1 --2- 172.21.15.19:0/2915707852 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x558dc3f18c30 0x558dc3f951c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:16.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.728+0000 7f699aa00700 1 --2- 172.21.15.19:0/2915707852 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x558dc3f18c30 0x558dc3f951c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=77f30fce8d5b0079 server_cookie=569fb8ce45be76a5 in_seq=0 out_seq=0 2024-05-07T02:38:16.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.728+0000 7f6985600700 10 client.16020 ms_handle_connect on v2:172.21.15.27:6836/2583138143 2024-05-07T02:38:16.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.729+0000 7f6985600700 1 -- 172.21.15.19:0/2915707852 <== mds.3 v2:172.21.15.27:6836/2583138143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558dc3e3e400 con 0x558dc3f18c30 2024-05-07T02:38:16.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.729+0000 7f6985600700 10 client.16020 handle_command_reply: tid=0 2024-05-07T02:38:16.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.729+0000 7f69b1086b80 1 client.16020 shutdown 2024-05-07T02:38:16.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.729+0000 7f69b1086b80 1 -- 172.21.15.19:0/2915707852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558dc3d90e40 msgr2=0x558dc3f20f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:16.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.729+0000 7f69b1086b80 1 --2- 172.21.15.19:0/2915707852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558dc3d90e40 0x558dc3f20f60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f698c0aa2e0 tx=0x7f698c0e0680 comp rx=0 tx=0).stop 2024-05-07T02:38:16.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.729+0000 7f69b1086b80 1 -- 172.21.15.19:0/2915707852 shutdown_connections 2024-05-07T02:38:16.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.729+0000 7f69b1086b80 1 --2- 172.21.15.19:0/2915707852 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x558dc3f18c30 0x558dc3f951c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:16.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.729+0000 7f69b1086b80 1 --2- 172.21.15.19:0/2915707852 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558dc3f0f590 0x558dc3f1c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:16.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.729+0000 7f69b1086b80 1 --2- 172.21.15.19:0/2915707852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558dc3d90e40 0x558dc3f20f60 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:16.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.729+0000 7f69b1086b80 1 --2- 172.21.15.19:0/2915707852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558dc3f0ec40 0x558dc3f1c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:16.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.729+0000 7f6985600700 0 client.16020 ms_handle_reset on v2:172.21.15.27:6836/2583138143 2024-05-07T02:38:16.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.729+0000 7f69b1086b80 1 -- 172.21.15.19:0/2915707852 >> 172.21.15.19:0/2915707852 conn(0x558dc3f02d10 msgr2=0x558dc3f17260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:16.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.729+0000 7f69b1086b80 1 -- 172.21.15.19:0/2915707852 shutdown_connections 2024-05-07T02:38:16.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.730+0000 7f69b1086b80 1 -- 172.21.15.19:0/2915707852 wait complete. 2024-05-07T02:38:16.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.730+0000 7f69b1086b80 20 client.16020 trim_cache size 0 max 16384 2024-05-07T02:38:16.734 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:38:16.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.731+0000 7f69a1000700 1 -- 172.21.15.19:0/817279211 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f69800b2a70 msgr2=0x7f69800b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:16.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.731+0000 7f69a1000700 1 --2- 172.21.15.19:0/817279211 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f69800b2a70 0x7f69800b4f80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x558dc3f04190 tx=0x7f6994004b40 comp rx=0 tx=0).stop 2024-05-07T02:38:16.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.731+0000 7f69a1000700 1 -- 172.21.15.19:0/817279211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f699c104400 msgr2=0x7f699c19ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:16.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.731+0000 7f69a1000700 1 --2- 172.21.15.19:0/817279211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f699c104400 0x7f699c19ad50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f698c000c00 tx=0x7f698c00f950 comp rx=0 tx=0).stop 2024-05-07T02:38:16.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.731+0000 7f69a1000700 1 -- 172.21.15.19:0/817279211 shutdown_connections 2024-05-07T02:38:16.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.731+0000 7f69a1000700 1 --2- 172.21.15.19:0/817279211 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f69800b2a70 0x7f69800b4f80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:16.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.731+0000 7f69a1000700 1 --2- 172.21.15.19:0/817279211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f699c109680 0x7f699c19f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:16.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.731+0000 7f69a1000700 1 --2- 172.21.15.19:0/817279211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f699c104400 0x7f699c19ad50 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:16.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.731+0000 7f69a1000700 1 --2- 172.21.15.19:0/817279211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f699c108d30 0x7f699c19b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:16.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.731+0000 7f69a1000700 1 -- 172.21.15.19:0/817279211 >> 172.21.15.19:0/817279211 conn(0x7f699c079d10 msgr2=0x7f699c101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:16.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.731+0000 7f69a1000700 1 -- 172.21.15.19:0/817279211 shutdown_connections 2024-05-07T02:38:16.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:16.731+0000 7f69a1000700 1 -- 172.21.15.19:0/817279211 wait complete. 2024-05-07T02:38:16.762 INFO:tasks.workunit.client.0.smithi019.stdout: 1 216802 4.03 MB/sec execute 192 sec latency 23.997 ms 2024-05-07T02:38:17.696 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:17 smithi019 ceph-mon[29781]: pgmap v196: 129 pgs: 129 active+clean; 150 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 183 op/s 2024-05-07T02:38:17.698 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-07T02:38:17.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:17 smithi027 ceph-mon[34398]: pgmap v196: 129 pgs: 129 active+clean; 150 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 183 op/s 2024-05-07T02:38:17.763 INFO:tasks.workunit.client.0.smithi019.stdout: 1 217472 4.01 MB/sec execute 193 sec latency 20.274 ms 2024-05-07T02:38:17.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:17 smithi064 ceph-mon[34699]: pgmap v196: 129 pgs: 129 active+clean; 150 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 183 op/s 2024-05-07T02:38:18.246 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:38:18.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:18 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:38:18.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:18 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:38:18.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:18 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:38:18.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:18 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:38:18.763 INFO:tasks.workunit.client.0.smithi019.stdout: 1 218114 4.01 MB/sec execute 194 sec latency 313.201 ms 2024-05-07T02:38:18.860 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:18 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:38:18.860 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:18 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:38:18.860 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:18 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:38:18.861 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:18 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:38:18.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:18 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:38:18.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:18 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:38:18.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:18 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:38:18.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:18 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:38:19.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:19 smithi027 ceph-mon[34398]: pgmap v197: 129 pgs: 129 active+clean; 150 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 971 KiB/s wr, 100 op/s 2024-05-07T02:38:19.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:19 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:38:19.763 INFO:tasks.workunit.client.0.smithi019.stdout: 1 218114 3.99 MB/sec execute 195 sec latency 1313.269 ms 2024-05-07T02:38:19.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:19 smithi064 ceph-mon[34699]: pgmap v197: 129 pgs: 129 active+clean; 150 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 971 KiB/s wr, 100 op/s 2024-05-07T02:38:19.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:19 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:38:19.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:19 smithi019 ceph-mon[29781]: pgmap v197: 129 pgs: 129 active+clean; 150 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 971 KiB/s wr, 100 op/s 2024-05-07T02:38:19.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:19 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:38:20.583 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:20 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:38:20.583 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:20 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:38:20.583 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:20 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:38:20.763 INFO:tasks.workunit.client.0.smithi019.stdout: 1 218562 3.97 MB/sec execute 196 sec latency 2041.204 ms 2024-05-07T02:38:20.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:20 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:38:20.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:20 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:38:20.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:20 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:38:20.956 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:20 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:38:20.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:20 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:38:20.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:20 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:38:21.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.247+0000 7f7a20800700 1 -- 172.21.15.19:0/3655204440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a18110970 msgr2=0x7f7a18110d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:21.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.247+0000 7f7a20800700 1 --2- 172.21.15.19:0/3655204440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a18110970 0x7f7a18110d80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c009990 tx=0x7f7a0c009ca0 comp rx=0 tx=0).stop 2024-05-07T02:38:21.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.247+0000 7f7a20800700 1 -- 172.21.15.19:0/3655204440 shutdown_connections 2024-05-07T02:38:21.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.247+0000 7f7a20800700 1 --2- 172.21.15.19:0/3655204440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a1810b2e0 0x7f7a18117810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:21.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.247+0000 7f7a20800700 1 --2- 172.21.15.19:0/3655204440 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a18110970 0x7f7a18110d80 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:21.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.247+0000 7f7a20800700 1 --2- 172.21.15.19:0/3655204440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a1810a990 0x7f7a1810ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:21.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.247+0000 7f7a20800700 1 -- 172.21.15.19:0/3655204440 >> 172.21.15.19:0/3655204440 conn(0x7f7a18077c80 msgr2=0x7f7a180780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:21.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.249+0000 7f7a20800700 1 -- 172.21.15.19:0/3655204440 shutdown_connections 2024-05-07T02:38:21.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.249+0000 7f7a20800700 1 -- 172.21.15.19:0/3655204440 wait complete. 2024-05-07T02:38:21.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.250+0000 7f7a20800700 1 Processor -- start 2024-05-07T02:38:21.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.250+0000 7f7a20800700 1 -- start start 2024-05-07T02:38:21.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.250+0000 7f7a20800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a1810a990 0x7f7a181a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:21.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.250+0000 7f7a20800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a1810b2e0 0x7f7a181a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:21.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.251+0000 7f7a20800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a18110970 0x7f7a181a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:21.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.251+0000 7f7a20800700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1811aa70 con 0x7f7a18110970 2024-05-07T02:38:21.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.251+0000 7f7a20800700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1811a8f0 con 0x7f7a1810a990 2024-05-07T02:38:21.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.251+0000 7f7a20800700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1811abf0 con 0x7f7a1810b2e0 2024-05-07T02:38:21.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.251+0000 7f7a1de00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a1810a990 0x7f7a181a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:21.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.251+0000 7f7a1d400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a1810b2e0 0x7f7a181a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:21.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.251+0000 7f7a1de00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a1810a990 0x7f7a181a1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:56102/0 (socket says 172.21.15.19:56102) 2024-05-07T02:38:21.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.251+0000 7f7a1de00700 1 -- 172.21.15.19:0/3316629798 learned_addr learned my addr 172.21.15.19:0/3316629798 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:38:21.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.251+0000 7f7a1e800700 1 --2- 172.21.15.19:0/3316629798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a18110970 0x7f7a181a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:21.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.251+0000 7f7a1de00700 1 -- 172.21.15.19:0/3316629798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a1810b2e0 msgr2=0x7f7a181a1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:21.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.251+0000 7f7a1de00700 1 --2- 172.21.15.19:0/3316629798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a1810b2e0 0x7f7a181a1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:21.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.251+0000 7f7a1de00700 1 -- 172.21.15.19:0/3316629798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a18110970 msgr2=0x7f7a181a5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:21.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.251+0000 7f7a1de00700 1 --2- 172.21.15.19:0/3316629798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a18110970 0x7f7a181a5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:21.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.251+0000 7f7a1de00700 1 -- 172.21.15.19:0/3316629798 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a181a6330 con 0x7f7a1810a990 2024-05-07T02:38:21.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.252+0000 7f7a1de00700 1 --2- 172.21.15.19:0/3316629798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a1810a990 0x7f7a181a1460 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c000c00 tx=0x7f7a0c009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:21.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.252+0000 7f7a12a00700 1 -- 172.21.15.19:0/3316629798 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c026070 con 0x7f7a1810a990 2024-05-07T02:38:21.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.252+0000 7f7a12a00700 1 -- 172.21.15.19:0/3316629798 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a0c00fb90 con 0x7f7a1810a990 2024-05-07T02:38:21.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.252+0000 7f7a20800700 1 -- 172.21.15.19:0/3316629798 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a181a65c0 con 0x7f7a1810a990 2024-05-07T02:38:21.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.252+0000 7f7a12a00700 1 -- 172.21.15.19:0/3316629798 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c0173f0 con 0x7f7a1810a990 2024-05-07T02:38:21.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.252+0000 7f7a20800700 1 -- 172.21.15.19:0/3316629798 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a181adf40 con 0x7f7a1810a990 2024-05-07T02:38:21.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.256+0000 7f7a12a00700 1 -- 172.21.15.19:0/3316629798 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7a0c029070 con 0x7f7a1810a990 2024-05-07T02:38:21.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.257+0000 7f7a12a00700 1 --2- 172.21.15.19:0/3316629798 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f7a000b2a70 0x7f7a000b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:21.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.257+0000 7f7a2f420b80 1 Processor -- start 2024-05-07T02:38:21.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.258+0000 7f7a12a00700 1 -- 172.21.15.19:0/3316629798 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f7a0c0e3680 con 0x7f7a1810a990 2024-05-07T02:38:21.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.258+0000 7f7a2f420b80 1 -- start start 2024-05-07T02:38:21.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.258+0000 7f7a1d400700 1 --2- 172.21.15.19:0/3316629798 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f7a000b2a70 0x7f7a000b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:21.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.258+0000 7f7a2f420b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d688663b0 0x555d688667c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:21.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.258+0000 7f7a2f420b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d68860470 0x555d68860880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:21.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.258+0000 7f7a2f420b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d68860dc0 0x555d6886d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:21.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.258+0000 7f7a2f420b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555d68782a20 con 0x555d68860dc0 2024-05-07T02:38:21.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.258+0000 7f7a2f420b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x555d68720760 con 0x555d68860470 2024-05-07T02:38:21.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.258+0000 7f7a2f420b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555d6877e9e0 con 0x555d688663b0 2024-05-07T02:38:21.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.259+0000 7f7a1de00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d688663b0 0x555d688667c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:21.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.259+0000 7f7a1de00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d688663b0 0x555d688667c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:36626/0 (socket says 172.21.15.19:36626) 2024-05-07T02:38:21.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.259+0000 7f7a1de00700 1 -- 172.21.15.19:0/2699507235 learned_addr learned my addr 172.21.15.19:0/2699507235 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:38:21.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.259+0000 7f7a1d400700 1 --2- 172.21.15.19:0/2699507235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d68860470 0x555d68860880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:21.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.259+0000 7f7a1e800700 1 --2- 172.21.15.19:0/2699507235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d68860dc0 0x555d6886d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:21.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.259+0000 7f7a1de00700 1 -- 172.21.15.19:0/2699507235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d68860470 msgr2=0x555d68860880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:21.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.259+0000 7f7a1de00700 1 --2- 172.21.15.19:0/2699507235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d68860470 0x555d68860880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:21.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.259+0000 7f7a1de00700 1 -- 172.21.15.19:0/2699507235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d68860dc0 msgr2=0x555d6886d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:21.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.259+0000 7f7a1d400700 1 --2- 172.21.15.19:0/3316629798 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f7a000b2a70 0x7f7a000b4f80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7a1807b060 tx=0x7f7a080106b0 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:21.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.259+0000 7f7a1de00700 1 --2- 172.21.15.19:0/2699507235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d68860dc0 0x555d6886d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:21.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.259+0000 7f7a1de00700 1 -- 172.21.15.19:0/2699507235 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d686cf8d0 con 0x555d688663b0 2024-05-07T02:38:21.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.259+0000 7f7a1e800700 1 --2- 172.21.15.19:0/2699507235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d68860dc0 0x555d6886d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:38:21.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.259+0000 7f7a1de00700 1 --2- 172.21.15.19:0/2699507235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d688663b0 0x555d688667c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c0a6680 tx=0x7f7a0c009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af652f127a58d3f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:21.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.260+0000 7f7a07400700 1 -- 172.21.15.19:0/2699507235 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c0ae9b0 con 0x555d688663b0 2024-05-07T02:38:21.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.260+0000 7f7a07400700 1 -- 172.21.15.19:0/2699507235 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a0c0b33d0 con 0x555d688663b0 2024-05-07T02:38:21.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.260+0000 7f7a07400700 1 -- 172.21.15.19:0/2699507235 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c0b2f40 con 0x555d688663b0 2024-05-07T02:38:21.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.261+0000 7f7a2f420b80 1 -- 172.21.15.19:0/2699507235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d688663b0 msgr2=0x555d688667c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:21.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.261+0000 7f7a2f420b80 1 --2- 172.21.15.19:0/2699507235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d688663b0 0x555d688667c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c0a6680 tx=0x7f7a0c009310 comp rx=0 tx=0).stop 2024-05-07T02:38:21.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.261+0000 7f7a2f420b80 1 -- 172.21.15.19:0/2699507235 shutdown_connections 2024-05-07T02:38:21.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.261+0000 7f7a2f420b80 1 --2- 172.21.15.19:0/2699507235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d68860dc0 0x555d6886d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:21.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.261+0000 7f7a2f420b80 1 --2- 172.21.15.19:0/2699507235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d688663b0 0x555d688667c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:21.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.261+0000 7f7a2f420b80 1 --2- 172.21.15.19:0/2699507235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d68860470 0x555d68860880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:21.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.261+0000 7f7a2f420b80 1 -- 172.21.15.19:0/2699507235 >> 172.21.15.19:0/2699507235 conn(0x555d686e2e40 msgr2=0x555d6885b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:21.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.261+0000 7f7a2f420b80 1 -- 172.21.15.19:0/2699507235 shutdown_connections 2024-05-07T02:38:21.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.261+0000 7f7a2f420b80 1 -- 172.21.15.19:0/2699507235 wait complete. 2024-05-07T02:38:21.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.261+0000 7f7a2f420b80 1 -- start start 2024-05-07T02:38:21.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.262+0000 7f7a2f420b80 1 Processor -- start 2024-05-07T02:38:21.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.262+0000 7f7a2f420b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d68860470 0x555d6886a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:21.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.262+0000 7f7a2f420b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d68860dc0 0x555d6886ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:21.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.262+0000 7f7a2f420b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d688663b0 0x555d6886b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:21.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.262+0000 7f7a2f420b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555d6886d580 con 0x555d688663b0 2024-05-07T02:38:21.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.262+0000 7f7a2f420b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x555d686e8b20 con 0x555d68860470 2024-05-07T02:38:21.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.262+0000 7f7a2f420b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555d6886d700 con 0x555d68860dc0 2024-05-07T02:38:21.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.262+0000 7f7a2f420b80 10 client.0 _ll_register_callbacks cb 0x555d68729c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:38:21.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.263+0000 7f7a1de00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d68860470 0x555d6886a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:21.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.263+0000 7f7a1de00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d68860470 0x555d6886a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:56122/0 (socket says 172.21.15.19:56122) 2024-05-07T02:38:21.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.263+0000 7f7a1de00700 1 -- 172.21.15.19:0/54021460 learned_addr learned my addr 172.21.15.19:0/54021460 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:38:21.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.263+0000 7f7a1d400700 1 --2- 172.21.15.19:0/54021460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d68860dc0 0x555d6886ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:21.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.263+0000 7f7a1e800700 1 --2- 172.21.15.19:0/54021460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d688663b0 0x555d6886b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:21.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.263+0000 7f7a1de00700 1 -- 172.21.15.19:0/54021460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d68860dc0 msgr2=0x555d6886ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:21.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.263+0000 7f7a1de00700 1 --2- 172.21.15.19:0/54021460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d68860dc0 0x555d6886ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:21.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.263+0000 7f7a1de00700 1 -- 172.21.15.19:0/54021460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d688663b0 msgr2=0x555d6886b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:21.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.263+0000 7f7a1de00700 1 --2- 172.21.15.19:0/54021460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d688663b0 0x555d6886b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:21.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.263+0000 7f7a1de00700 1 -- 172.21.15.19:0/54021460 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d68748420 con 0x555d68860470 2024-05-07T02:38:21.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.263+0000 7f7a1de00700 1 --2- 172.21.15.19:0/54021460 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d68860470 0x555d6886a5c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c0a6680 tx=0x7f7a0c0aab10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28147968900c3e1e server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:21.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.263+0000 7f79fbe00700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:38:21.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.264+0000 7f79fbe00700 1 -- 172.21.15.19:0/54021460 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c0c4e60 con 0x555d68860470 2024-05-07T02:38:21.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.264+0000 7f79fbe00700 1 -- 172.21.15.19:0/54021460 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f7a0c00fd40 con 0x555d68860470 2024-05-07T02:38:21.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.264+0000 7f7a2f420b80 1 -- 172.21.15.19:0/54021460 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a0c0b33d0 con 0x555d68860470 2024-05-07T02:38:21.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.264+0000 7f79fbe00700 1 -- 172.21.15.19:0/54021460 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f7a0c0c3070 con 0x555d68860470 2024-05-07T02:38:21.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.264+0000 7f7a2f420b80 10 client.26231 fetch_fsmap learned FSMap version 44 2024-05-07T02:38:21.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.264+0000 7f7a2f420b80 1 -- 172.21.15.19:0/54021460 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x555d6886d580 con 0x555d68860470 2024-05-07T02:38:21.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.264+0000 7f79fbe00700 1 -- 172.21.15.19:0/54021460 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a0c0b33d0 con 0x555d68860470 2024-05-07T02:38:21.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.264+0000 7f79fbe00700 1 -- 172.21.15.19:0/54021460 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 44) ==== 8378+0+0 (secure 0 0 0) 0x7f7a0c0a9860 con 0x555d68860470 2024-05-07T02:38:21.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.265+0000 7f7a2f420b80 10 client.26231 fetch_fsmap finished waiting for FSMap version 44 2024-05-07T02:38:21.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.265+0000 7f7a2f420b80 10 client.26231 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-07T02:38:21.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.265+0000 7f7a2f420b80 20 client.26231 populate_metadata read hostname 'smithi019' 2024-05-07T02:38:21.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.265+0000 7f7a2f420b80 1 --2- 172.21.15.19:0/54021460 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x555d6887a590 0x555d6887c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:21.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.265+0000 7f7a2f420b80 4 client.26231 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:38:21.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.265+0000 7f7a2f420b80 1 -- 172.21.15.19:0/54021460 --> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d6882d2e0 con 0x555d6887a590 2024-05-07T02:38:21.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.265+0000 7f7a1d400700 1 --2- 172.21.15.19:0/54021460 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x555d6887a590 0x555d6887c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:21.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.265+0000 7f7a1d400700 1 --2- 172.21.15.19:0/54021460 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x555d6887a590 0x555d6887c9e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=40e8d04f65e5ed03 server_cookie=2aa9eaee485661db in_seq=0 out_seq=0 2024-05-07T02:38:21.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.265+0000 7f79fbe00700 10 client.26231 ms_handle_connect on v2:172.21.15.64:6838/1583120004 2024-05-07T02:38:21.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.266+0000 7f79fbe00700 1 -- 172.21.15.19:0/54021460 <== mds.2 v2:172.21.15.64:6838/1583120004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555d6882d2e0 con 0x555d6887a590 2024-05-07T02:38:21.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.266+0000 7f79fbe00700 10 client.26231 handle_command_reply: tid=0 2024-05-07T02:38:21.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.266+0000 7f7a2f420b80 1 client.26231 shutdown 2024-05-07T02:38:21.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.267+0000 7f7a2f420b80 1 -- 172.21.15.19:0/54021460 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d68860470 msgr2=0x555d6886a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:21.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.267+0000 7f7a2f420b80 1 --2- 172.21.15.19:0/54021460 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d68860470 0x555d6886a5c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c0a6680 tx=0x7f7a0c0aab10 comp rx=0 tx=0).stop 2024-05-07T02:38:21.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.267+0000 7f7a2f420b80 1 -- 172.21.15.19:0/54021460 shutdown_connections 2024-05-07T02:38:21.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.267+0000 7f7a2f420b80 1 --2- 172.21.15.19:0/54021460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d688663b0 0x555d6886b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:21.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.267+0000 7f7a2f420b80 1 --2- 172.21.15.19:0/54021460 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x555d6887a590 0x555d6887c9e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:21.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.267+0000 7f7a2f420b80 1 --2- 172.21.15.19:0/54021460 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d68860470 0x555d6886a5c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:21.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.267+0000 7f7a2f420b80 1 --2- 172.21.15.19:0/54021460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d68860dc0 0x555d6886ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:21.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.267+0000 7f7a2f420b80 1 -- 172.21.15.19:0/54021460 >> 172.21.15.19:0/54021460 conn(0x555d686e2e40 msgr2=0x555d6885f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:21.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.267+0000 7f79fbe00700 0 client.26231 ms_handle_reset on v2:172.21.15.64:6838/1583120004 2024-05-07T02:38:21.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.267+0000 7f7a2f420b80 1 -- 172.21.15.19:0/54021460 shutdown_connections 2024-05-07T02:38:21.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.267+0000 7f7a2f420b80 1 -- 172.21.15.19:0/54021460 wait complete. 2024-05-07T02:38:21.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.267+0000 7f7a2f420b80 20 client.26231 trim_cache size 0 max 16384 2024-05-07T02:38:21.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.286+0000 7f7a2f420b80 1 Processor -- start 2024-05-07T02:38:21.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.286+0000 7f7a2f420b80 1 -- start start 2024-05-07T02:38:21.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.287+0000 7f7a2f420b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d68860470 0x555d68869900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:21.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.287+0000 7f7a2f420b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d68860dc0 0x555d68868000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:21.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.287+0000 7f7a2f420b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d688663b0 0x555d6885fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:21.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.287+0000 7f7a2f420b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7a181b6bc0 con 0x555d68860dc0 2024-05-07T02:38:21.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.287+0000 7f7a2f420b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7a181b87c0 con 0x555d68860470 2024-05-07T02:38:21.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.287+0000 7f7a2f420b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7a181b7290 con 0x555d688663b0 2024-05-07T02:38:21.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.287+0000 7f7a1de00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d68860470 0x555d68869900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:21.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.287+0000 7f7a1e800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d688663b0 0x555d6885fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:21.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.287+0000 7f7a1de00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d68860470 0x555d68869900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:56126/0 (socket says 172.21.15.19:56126) 2024-05-07T02:38:21.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.287+0000 7f7a1de00700 1 -- 172.21.15.19:0/4103238868 learned_addr learned my addr 172.21.15.19:0/4103238868 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:38:21.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.287+0000 7f7a1d400700 1 --2- 172.21.15.19:0/4103238868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d68860dc0 0x555d68868000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:21.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.288+0000 7f7a1de00700 1 -- 172.21.15.19:0/4103238868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d688663b0 msgr2=0x555d6885fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:21.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.288+0000 7f7a1de00700 1 --2- 172.21.15.19:0/4103238868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d688663b0 0x555d6885fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:21.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.288+0000 7f7a1de00700 1 -- 172.21.15.19:0/4103238868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d68860dc0 msgr2=0x555d68868000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:21.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.288+0000 7f7a1de00700 1 --2- 172.21.15.19:0/4103238868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d68860dc0 0x555d68868000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:21.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.288+0000 7f7a1de00700 1 -- 172.21.15.19:0/4103238868 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d6886ce60 con 0x555d68860470 2024-05-07T02:38:21.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.288+0000 7f7a1d400700 1 --2- 172.21.15.19:0/4103238868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d68860dc0 0x555d68868000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:38:21.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.288+0000 7f7a1de00700 1 --2- 172.21.15.19:0/4103238868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d68860470 0x555d68869900 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c0aa300 tx=0x7f7a0c0aa500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=242a2581056082af server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:21.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.288+0000 7f7a06a00700 1 -- 172.21.15.19:0/4103238868 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c0b3150 con 0x555d68860470 2024-05-07T02:38:21.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.288+0000 7f7a06a00700 1 -- 172.21.15.19:0/4103238868 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a0c0b33d0 con 0x555d68860470 2024-05-07T02:38:21.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.288+0000 7f7a06a00700 1 -- 172.21.15.19:0/4103238868 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c005110 con 0x555d68860470 2024-05-07T02:38:21.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.289+0000 7f7a2f420b80 1 -- 172.21.15.19:0/4103238868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d68860470 msgr2=0x555d68869900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:21.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.289+0000 7f7a2f420b80 1 --2- 172.21.15.19:0/4103238868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d68860470 0x555d68869900 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c0aa300 tx=0x7f7a0c0aa500 comp rx=0 tx=0).stop 2024-05-07T02:38:21.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.289+0000 7f7a2f420b80 1 -- 172.21.15.19:0/4103238868 shutdown_connections 2024-05-07T02:38:21.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.289+0000 7f7a2f420b80 1 --2- 172.21.15.19:0/4103238868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d688663b0 0x555d6885fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:21.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.289+0000 7f7a2f420b80 1 --2- 172.21.15.19:0/4103238868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d68860470 0x555d68869900 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:21.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.289+0000 7f7a2f420b80 1 --2- 172.21.15.19:0/4103238868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d68860dc0 0x555d68868000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:21.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.289+0000 7f7a2f420b80 1 -- 172.21.15.19:0/4103238868 >> 172.21.15.19:0/4103238868 conn(0x555d686e2e40 msgr2=0x555d6885f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:21.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.290+0000 7f7a2f420b80 1 -- 172.21.15.19:0/4103238868 shutdown_connections 2024-05-07T02:38:21.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.290+0000 7f7a2f420b80 1 -- 172.21.15.19:0/4103238868 wait complete. 2024-05-07T02:38:21.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.290+0000 7f7a2f420b80 1 -- start start 2024-05-07T02:38:21.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.290+0000 7f7a2f420b80 1 Processor -- start 2024-05-07T02:38:21.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.291+0000 7f7a2f420b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d68860470 0x555d68872ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:21.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.291+0000 7f7a2f420b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d68860dc0 0x555d68875d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:21.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.291+0000 7f7a2f420b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d688663b0 0x555d688762a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:21.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.291+0000 7f7a2f420b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7a181b8030 con 0x555d688663b0 2024-05-07T02:38:21.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.291+0000 7f7a2f420b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f7a181b7960 con 0x555d68860470 2024-05-07T02:38:21.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.291+0000 7f7a2f420b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7a181b4b60 con 0x555d68860dc0 2024-05-07T02:38:21.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.291+0000 7f7a2f420b80 10 client.0 _ll_register_callbacks cb 0x555d68729c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:38:21.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.291+0000 7f7a1d400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d68860dc0 0x555d68875d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:21.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.291+0000 7f7a1de00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d68860470 0x555d68872ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:21.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.291+0000 7f7a1d400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d68860dc0 0x555d68875d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:36648/0 (socket says 172.21.15.19:36648) 2024-05-07T02:38:21.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.291+0000 7f7a1d400700 1 -- 172.21.15.19:0/1784686796 learned_addr learned my addr 172.21.15.19:0/1784686796 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:38:21.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.291+0000 7f7a1de00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d68860470 0x555d68872ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:56128/0 (socket says 172.21.15.19:56128) 2024-05-07T02:38:21.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.292+0000 7f7a1e800700 1 --2- 172.21.15.19:0/1784686796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d688663b0 0x555d688762a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:21.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.292+0000 7f7a1d400700 1 -- 172.21.15.19:0/1784686796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d68860470 msgr2=0x555d68872ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:21.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.292+0000 7f7a1d400700 1 --2- 172.21.15.19:0/1784686796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d68860470 0x555d68872ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:21.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.292+0000 7f7a1d400700 1 -- 172.21.15.19:0/1784686796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d688663b0 msgr2=0x555d688762a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:21.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.292+0000 7f7a1d400700 1 --2- 172.21.15.19:0/1784686796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d688663b0 0x555d688762a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:21.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.292+0000 7f7a1d400700 1 -- 172.21.15.19:0/1784686796 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d68876a20 con 0x555d68860dc0 2024-05-07T02:38:21.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.292+0000 7f7a1e800700 1 --2- 172.21.15.19:0/1784686796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d688663b0 0x555d688762a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:38:21.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.292+0000 7f7a1d400700 1 --2- 172.21.15.19:0/1784686796 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d68860dc0 0x555d68875d60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x555d6885ca40 tx=0x7f7a080047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c36420af37ae587f server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:21.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.292+0000 7f79fbe00700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:38:21.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.292+0000 7f79fbe00700 1 -- 172.21.15.19:0/1784686796 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a08004a30 con 0x555d68860dc0 2024-05-07T02:38:21.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.292+0000 7f7a2f420b80 1 -- 172.21.15.19:0/1784686796 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a0c0b33d0 con 0x555d68860dc0 2024-05-07T02:38:21.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.292+0000 7f79fbe00700 1 -- 172.21.15.19:0/1784686796 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f7a08004bd0 con 0x555d68860dc0 2024-05-07T02:38:21.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.293+0000 7f79fbe00700 1 -- 172.21.15.19:0/1784686796 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f7a08036050 con 0x555d68860dc0 2024-05-07T02:38:21.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.293+0000 7f7a2f420b80 10 client.26200 fetch_fsmap learned FSMap version 44 2024-05-07T02:38:21.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.293+0000 7f7a2f420b80 1 -- 172.21.15.19:0/1784686796 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x555d688e4d00 con 0x555d68860dc0 2024-05-07T02:38:21.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.293+0000 7f79fbe00700 1 -- 172.21.15.19:0/1784686796 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a0c0b33d0 con 0x555d68860dc0 2024-05-07T02:38:21.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.293+0000 7f79fbe00700 1 -- 172.21.15.19:0/1784686796 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 44) ==== 8378+0+0 (secure 0 0 0) 0x7f7a08005430 con 0x555d68860dc0 2024-05-07T02:38:21.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.293+0000 7f7a2f420b80 10 client.26200 fetch_fsmap finished waiting for FSMap version 44 2024-05-07T02:38:21.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.293+0000 7f7a2f420b80 10 client.26200 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-07T02:38:21.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.294+0000 7f7a2f420b80 20 client.26200 populate_metadata read hostname 'smithi019' 2024-05-07T02:38:21.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.294+0000 7f7a2f420b80 1 --2- 172.21.15.19:0/1784686796 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x555d688e5440 0x555d688e7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:21.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.294+0000 7f7a2f420b80 4 client.26200 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:38:21.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.294+0000 7f7a2f420b80 1 -- 172.21.15.19:0/1784686796 --> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555d68790400 con 0x555d688e5440 2024-05-07T02:38:21.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.294+0000 7f7a1de00700 1 --2- 172.21.15.19:0/1784686796 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x555d688e5440 0x555d688e7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:21.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.294+0000 7f7a1de00700 1 --2- 172.21.15.19:0/1784686796 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x555d688e5440 0x555d688e7860 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5d5298e43e1f4ae server_cookie=4f661548306a6ab in_seq=0 out_seq=0 2024-05-07T02:38:21.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.294+0000 7f79fbe00700 10 client.26200 ms_handle_connect on v2:172.21.15.64:6838/1583120004 2024-05-07T02:38:21.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.295+0000 7f79fbe00700 1 -- 172.21.15.19:0/1784686796 <== mds.2 v2:172.21.15.64:6838/1583120004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555d68790400 con 0x555d688e5440 2024-05-07T02:38:21.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.295+0000 7f79fbe00700 10 client.26200 handle_command_reply: tid=0 2024-05-07T02:38:21.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.295+0000 7f7a2f420b80 1 client.26200 shutdown 2024-05-07T02:38:21.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.295+0000 7f7a2f420b80 1 -- 172.21.15.19:0/1784686796 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d68860dc0 msgr2=0x555d68875d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:21.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.295+0000 7f7a2f420b80 1 --2- 172.21.15.19:0/1784686796 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d68860dc0 0x555d68875d60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x555d6885ca40 tx=0x7f7a080047c0 comp rx=0 tx=0).stop 2024-05-07T02:38:21.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.295+0000 7f7a2f420b80 1 -- 172.21.15.19:0/1784686796 shutdown_connections 2024-05-07T02:38:21.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.295+0000 7f7a2f420b80 1 --2- 172.21.15.19:0/1784686796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d688663b0 0x555d688762a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:21.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.295+0000 7f7a2f420b80 1 --2- 172.21.15.19:0/1784686796 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x555d688e5440 0x555d688e7860 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:21.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.295+0000 7f7a2f420b80 1 --2- 172.21.15.19:0/1784686796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555d68860470 0x555d68872ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:21.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.295+0000 7f7a2f420b80 1 --2- 172.21.15.19:0/1784686796 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555d68860dc0 0x555d68875d60 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:21.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.295+0000 7f79fbe00700 0 client.26200 ms_handle_reset on v2:172.21.15.64:6838/1583120004 2024-05-07T02:38:21.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.295+0000 7f7a2f420b80 1 -- 172.21.15.19:0/1784686796 >> 172.21.15.19:0/1784686796 conn(0x555d686e2e40 msgr2=0x555d6885fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:21.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.295+0000 7f7a2f420b80 1 -- 172.21.15.19:0/1784686796 shutdown_connections 2024-05-07T02:38:21.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.295+0000 7f7a2f420b80 1 -- 172.21.15.19:0/1784686796 wait complete. 2024-05-07T02:38:21.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.296+0000 7f7a2f420b80 20 client.26200 trim_cache size 0 max 16384 2024-05-07T02:38:21.298 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:38:21.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.296+0000 7f7a20800700 1 -- 172.21.15.19:0/3316629798 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f7a000b2a70 msgr2=0x7f7a000b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:21.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.296+0000 7f7a20800700 1 --2- 172.21.15.19:0/3316629798 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f7a000b2a70 0x7f7a000b4f80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7a1807b060 tx=0x7f7a080106b0 comp rx=0 tx=0).stop 2024-05-07T02:38:21.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.296+0000 7f7a20800700 1 -- 172.21.15.19:0/3316629798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a1810a990 msgr2=0x7f7a181a1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:21.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.296+0000 7f7a20800700 1 --2- 172.21.15.19:0/3316629798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a1810a990 0x7f7a181a1460 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c000c00 tx=0x7f7a0c009510 comp rx=0 tx=0).stop 2024-05-07T02:38:21.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.296+0000 7f7a20800700 1 -- 172.21.15.19:0/3316629798 shutdown_connections 2024-05-07T02:38:21.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.296+0000 7f7a20800700 1 --2- 172.21.15.19:0/3316629798 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f7a000b2a70 0x7f7a000b4f80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:21.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.296+0000 7f7a20800700 1 --2- 172.21.15.19:0/3316629798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a18110970 0x7f7a181a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:21.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.296+0000 7f7a20800700 1 --2- 172.21.15.19:0/3316629798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a1810a990 0x7f7a181a1460 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:21.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.296+0000 7f7a20800700 1 --2- 172.21.15.19:0/3316629798 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7a1810b2e0 0x7f7a181a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:21.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.296+0000 7f7a20800700 1 -- 172.21.15.19:0/3316629798 >> 172.21.15.19:0/3316629798 conn(0x7f7a18077c80 msgr2=0x7f7a1807bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:21.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.297+0000 7f7a20800700 1 -- 172.21.15.19:0/3316629798 shutdown_connections 2024-05-07T02:38:21.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:21.297+0000 7f7a20800700 1 -- 172.21.15.19:0/3316629798 wait complete. 2024-05-07T02:38:21.763 INFO:tasks.workunit.client.0.smithi019.stdout: 1 219973 4.01 MB/sec execute 197 sec latency 22.171 ms 2024-05-07T02:38:22.339 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:22 smithi019 ceph-mon[29781]: pgmap v198: 129 pgs: 129 active+clean; 150 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 971 KiB/s wr, 100 op/s 2024-05-07T02:38:22.343 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-07T02:38:22.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:22 smithi064 ceph-mon[34699]: pgmap v198: 129 pgs: 129 active+clean; 150 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 971 KiB/s wr, 100 op/s 2024-05-07T02:38:22.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:22 smithi027 ceph-mon[34398]: pgmap v198: 129 pgs: 129 active+clean; 150 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 971 KiB/s wr, 100 op/s 2024-05-07T02:38:22.763 INFO:tasks.workunit.client.0.smithi019.stdout: 1 220403 4.00 MB/sec execute 198 sec latency 32.610 ms 2024-05-07T02:38:22.828 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:38:23.763 INFO:tasks.workunit.client.0.smithi019.stdout: 1 221177 3.99 MB/sec execute 199 sec latency 44.979 ms 2024-05-07T02:38:24.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:24 smithi064 ceph-mon[34699]: pgmap v199: 129 pgs: 129 active+clean; 153 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 150 op/s 2024-05-07T02:38:24.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:24 smithi019 ceph-mon[29781]: pgmap v199: 129 pgs: 129 active+clean; 153 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 150 op/s 2024-05-07T02:38:24.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:24 smithi027 ceph-mon[34398]: pgmap v199: 129 pgs: 129 active+clean; 153 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 150 op/s 2024-05-07T02:38:24.763 INFO:tasks.workunit.client.0.smithi019.stdout: 1 221657 3.99 MB/sec execute 200 sec latency 542.092 ms 2024-05-07T02:38:25.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.274+0000 7fc7bcc00700 1 -- 172.21.15.19:0/566882685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7b81034c0 msgr2=0x7fc7b81038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:25.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.274+0000 7fc7bcc00700 1 --2- 172.21.15.19:0/566882685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7b81034c0 0x7fc7b81038d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc7a80098f0 tx=0x7fc7a8009c00 comp rx=0 tx=0).stop 2024-05-07T02:38:25.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.274+0000 7fc7bcc00700 1 -- 172.21.15.19:0/566882685 shutdown_connections 2024-05-07T02:38:25.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.274+0000 7fc7bcc00700 1 --2- 172.21.15.19:0/566882685 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7b8108740 0x7fc7b81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:25.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.274+0000 7fc7bcc00700 1 --2- 172.21.15.19:0/566882685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7b81034c0 0x7fc7b81038d0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:25.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.274+0000 7fc7bcc00700 1 --2- 172.21.15.19:0/566882685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7b8107df0 0x7fc7b8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:25.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.274+0000 7fc7bcc00700 1 -- 172.21.15.19:0/566882685 >> 172.21.15.19:0/566882685 conn(0x7fc7b80fedc0 msgr2=0x7fc7b81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:25.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.274+0000 7fc7bcc00700 1 -- 172.21.15.19:0/566882685 shutdown_connections 2024-05-07T02:38:25.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.274+0000 7fc7bcc00700 1 -- 172.21.15.19:0/566882685 wait complete. 2024-05-07T02:38:25.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.275+0000 7fc7bcc00700 1 Processor -- start 2024-05-07T02:38:25.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.275+0000 7fc7bcc00700 1 -- start start 2024-05-07T02:38:25.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.276+0000 7fc7bcc00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7b8107df0 0x7fc7b819b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:25.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.276+0000 7fc7bcc00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7b8108740 0x7fc7b819b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:25.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.276+0000 7fc7bcc00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7b819f790 0x7fc7b819fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:25.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.276+0000 7fc7bcc00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b81133c0 con 0x7fc7b8107df0 2024-05-07T02:38:25.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.276+0000 7fc7bcc00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b8113240 con 0x7fc7b819f790 2024-05-07T02:38:25.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.276+0000 7fc7bcc00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b8113540 con 0x7fc7b8108740 2024-05-07T02:38:25.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.276+0000 7fc7b5400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7b8108740 0x7fc7b819b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:25.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.276+0000 7fc7b5e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7b8107df0 0x7fc7b819b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:25.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.276+0000 7fc7b6800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7b819f790 0x7fc7b819fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:25.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.276+0000 7fc7b5e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7b8107df0 0x7fc7b819b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35522/0 (socket says 172.21.15.19:35522) 2024-05-07T02:38:25.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.276+0000 7fc7b6800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7b819f790 0x7fc7b819fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:56140/0 (socket says 172.21.15.19:56140) 2024-05-07T02:38:25.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.276+0000 7fc7b5400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7b8108740 0x7fc7b819b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:36666/0 (socket says 172.21.15.19:36666) 2024-05-07T02:38:25.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.276+0000 7fc7b5e00700 1 -- 172.21.15.19:0/1196091011 learned_addr learned my addr 172.21.15.19:0/1196091011 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:38:25.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.277+0000 7fc7b6800700 1 -- 172.21.15.19:0/1196091011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7b8108740 msgr2=0x7fc7b819b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:25.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.277+0000 7fc7b6800700 1 --2- 172.21.15.19:0/1196091011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7b8108740 0x7fc7b819b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:25.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.277+0000 7fc7b6800700 1 -- 172.21.15.19:0/1196091011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7b8107df0 msgr2=0x7fc7b819b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:25.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.277+0000 7fc7b6800700 1 --2- 172.21.15.19:0/1196091011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7b8107df0 0x7fc7b819b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:25.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.277+0000 7fc7b6800700 1 -- 172.21.15.19:0/1196091011 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b81a02e0 con 0x7fc7b819f790 2024-05-07T02:38:25.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.277+0000 7fc7b5400700 1 --2- 172.21.15.19:0/1196091011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7b8108740 0x7fc7b819b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:38:25.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.277+0000 7fc7b5e00700 1 --2- 172.21.15.19:0/1196091011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7b8107df0 0x7fc7b819b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:38:25.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.277+0000 7fc7b6800700 1 --2- 172.21.15.19:0/1196091011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7b819f790 0x7fc7b819fba0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc7b000efc0 tx=0x7fc7b000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:25.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.278+0000 7fc7aea00700 1 -- 172.21.15.19:0/1196091011 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b00092e0 con 0x7fc7b819f790 2024-05-07T02:38:25.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.278+0000 7fc7aea00700 1 -- 172.21.15.19:0/1196091011 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7b0004bb0 con 0x7fc7b819f790 2024-05-07T02:38:25.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.278+0000 7fc7bcc00700 1 -- 172.21.15.19:0/1196091011 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7b81a0570 con 0x7fc7b819f790 2024-05-07T02:38:25.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.278+0000 7fc7aea00700 1 -- 172.21.15.19:0/1196091011 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0004020 con 0x7fc7b819f790 2024-05-07T02:38:25.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.278+0000 7fc7bcc00700 1 -- 172.21.15.19:0/1196091011 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7b81a7d00 con 0x7fc7b819f790 2024-05-07T02:38:25.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.284+0000 7fc7aea00700 1 -- 172.21.15.19:0/1196091011 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc7b001b030 con 0x7fc7b819f790 2024-05-07T02:38:25.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.285+0000 7fc7cb7d6b80 1 Processor -- start 2024-05-07T02:38:25.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.285+0000 7fc7cb7d6b80 1 -- start start 2024-05-07T02:38:25.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.285+0000 7fc7cb7d6b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55776f0ba750 0x55776f0bab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:25.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.285+0000 7fc7cb7d6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55776f0b4880 0x55776f0b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:25.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.285+0000 7fc7cb7d6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55776f0b51d0 0x55776f0c14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:25.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.285+0000 7fc7cb7d6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55776efd2a20 con 0x55776f0b4880 2024-05-07T02:38:25.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.285+0000 7fc7cb7d6b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55776ef70760 con 0x55776f0ba750 2024-05-07T02:38:25.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.285+0000 7fc7cb7d6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55776efce9e0 con 0x55776f0b51d0 2024-05-07T02:38:25.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.286+0000 7fc7b5400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55776f0ba750 0x55776f0bab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:25.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.286+0000 7fc7b5400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55776f0ba750 0x55776f0bab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:56142/0 (socket says 172.21.15.19:56142) 2024-05-07T02:38:25.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.286+0000 7fc7b5400700 1 -- 172.21.15.19:0/3383877218 learned_addr learned my addr 172.21.15.19:0/3383877218 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:38:25.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.286+0000 7fc7aea00700 1 --2- 172.21.15.19:0/1196091011 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc79c0b29f0 0x7fc79c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:25.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.286+0000 7fc7b5e00700 1 --2- 172.21.15.19:0/3383877218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55776f0b4880 0x55776f0b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:25.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.286+0000 7fc7b5400700 1 --2- 172.21.15.19:0/3383877218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55776f0b51d0 0x55776f0c14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:25.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.286+0000 7fc7aea00700 1 -- 172.21.15.19:0/1196091011 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fc7b00e2470 con 0x7fc7b819f790 2024-05-07T02:38:25.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.286+0000 7fc7b5400700 1 -- 172.21.15.19:0/3383877218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55776f0b51d0 msgr2=0x55776f0c14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:25.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.286+0000 7fc7b5400700 1 --2- 172.21.15.19:0/3383877218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55776f0b51d0 0x55776f0c14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:25.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.286+0000 7fc7b5400700 1 -- 172.21.15.19:0/3383877218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55776f0b4880 msgr2=0x55776f0b4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:25.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.286+0000 7fc7b5400700 1 --2- 172.21.15.19:0/3383877218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55776f0b4880 0x55776f0b4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:25.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.286+0000 7fc7b5400700 1 -- 172.21.15.19:0/3383877218 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55776ef1f8d0 con 0x55776f0ba750 2024-05-07T02:38:25.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.286+0000 7fc7b6800700 1 --2- 172.21.15.19:0/1196091011 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc79c0b29f0 0x7fc79c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:25.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.287+0000 7fc7b5400700 1 --2- 172.21.15.19:0/3383877218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55776f0ba750 0x55776f0bab60 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4009890 tx=0x7fc7a4009820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efa58c10fc8af7de server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:25.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.287+0000 7fc7a3400700 1 -- 172.21.15.19:0/3383877218 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a4004030 con 0x55776f0ba750 2024-05-07T02:38:25.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.287+0000 7fc7a3400700 1 -- 172.21.15.19:0/3383877218 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7a4004830 con 0x55776f0ba750 2024-05-07T02:38:25.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.287+0000 7fc7a3400700 1 -- 172.21.15.19:0/3383877218 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a40052b0 con 0x55776f0ba750 2024-05-07T02:38:25.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.288+0000 7fc7cb7d6b80 1 -- 172.21.15.19:0/3383877218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55776f0ba750 msgr2=0x55776f0bab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:25.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.288+0000 7fc7cb7d6b80 1 --2- 172.21.15.19:0/3383877218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55776f0ba750 0x55776f0bab60 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4009890 tx=0x7fc7a4009820 comp rx=0 tx=0).stop 2024-05-07T02:38:25.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.288+0000 7fc7b6800700 1 --2- 172.21.15.19:0/1196091011 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc79c0b29f0 0x7fc79c0b4f00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc7b00e2710 tx=0x7fc7b0009040 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:25.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.288+0000 7fc7cb7d6b80 1 -- 172.21.15.19:0/3383877218 shutdown_connections 2024-05-07T02:38:25.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.288+0000 7fc7cb7d6b80 1 --2- 172.21.15.19:0/3383877218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55776f0b51d0 0x55776f0c14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:25.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.288+0000 7fc7cb7d6b80 1 --2- 172.21.15.19:0/3383877218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55776f0ba750 0x55776f0bab60 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:25.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.288+0000 7fc7cb7d6b80 1 --2- 172.21.15.19:0/3383877218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55776f0b4880 0x55776f0b4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:25.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.288+0000 7fc7cb7d6b80 1 -- 172.21.15.19:0/3383877218 >> 172.21.15.19:0/3383877218 conn(0x55776ef32e40 msgr2=0x55776f0ab2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:25.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.288+0000 7fc7cb7d6b80 1 -- 172.21.15.19:0/3383877218 shutdown_connections 2024-05-07T02:38:25.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.288+0000 7fc7cb7d6b80 1 -- 172.21.15.19:0/3383877218 wait complete. 2024-05-07T02:38:25.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.289+0000 7fc7cb7d6b80 1 -- start start 2024-05-07T02:38:25.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.289+0000 7fc7cb7d6b80 1 Processor -- start 2024-05-07T02:38:25.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.290+0000 7fc7cb7d6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55776f0b4880 0x55776f0be9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:25.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.290+0000 7fc7cb7d6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55776f0b51d0 0x55776f0bef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:25.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.290+0000 7fc7cb7d6b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55776f0ba750 0x55776f0bf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:25.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.290+0000 7fc7cb7d6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55776f0c1ae0 con 0x55776f0b51d0 2024-05-07T02:38:25.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.290+0000 7fc7cb7d6b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55776ef38b20 con 0x55776f0ba750 2024-05-07T02:38:25.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.290+0000 7fc7cb7d6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55776f0c1c60 con 0x55776f0b4880 2024-05-07T02:38:25.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.290+0000 7fc7cb7d6b80 10 client.0 _ll_register_callbacks cb 0x55776ef79c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:38:25.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.290+0000 7fc7b5e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55776f0b51d0 0x55776f0bef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:25.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.290+0000 7fc7b5400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55776f0b4880 0x55776f0be9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:25.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.290+0000 7fc7b5e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55776f0b51d0 0x55776f0bef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35542/0 (socket says 172.21.15.19:35542) 2024-05-07T02:38:25.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.290+0000 7fc7b5e00700 1 -- 172.21.15.19:0/3968902678 learned_addr learned my addr 172.21.15.19:0/3968902678 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:38:25.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.290+0000 7fc7b5400700 1 --2- 172.21.15.19:0/3968902678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55776f0ba750 0x55776f0bf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:25.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.291+0000 7fc7b5e00700 1 -- 172.21.15.19:0/3968902678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55776f0b4880 msgr2=0x55776f0be9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:25.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.291+0000 7fc7b5e00700 1 --2- 172.21.15.19:0/3968902678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55776f0b4880 0x55776f0be9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:25.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.291+0000 7fc7b5e00700 1 -- 172.21.15.19:0/3968902678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55776f0ba750 msgr2=0x55776f0bf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:25.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.291+0000 7fc7b5e00700 1 --2- 172.21.15.19:0/3968902678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55776f0ba750 0x55776f0bf440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:25.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.291+0000 7fc7b5e00700 1 -- 172.21.15.19:0/3968902678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55776ef98420 con 0x55776f0b51d0 2024-05-07T02:38:25.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.291+0000 7fc7b5e00700 1 --2- 172.21.15.19:0/3968902678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55776f0b51d0 0x55776f0bef00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8009230 tx=0x7fc7a8009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d896c1b3c0b6057 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:25.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.291+0000 7fc797e00700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-07T02:38:25.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.291+0000 7fc797e00700 1 -- 172.21.15.19:0/3968902678 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a8005450 con 0x55776f0b51d0 2024-05-07T02:38:25.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.292+0000 7fc797e00700 1 -- 172.21.15.19:0/3968902678 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fc7a80055f0 con 0x55776f0b51d0 2024-05-07T02:38:25.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.292+0000 7fc7cb7d6b80 1 -- 172.21.15.19:0/3968902678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7a4004830 con 0x55776f0b51d0 2024-05-07T02:38:25.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.292+0000 7fc797e00700 1 -- 172.21.15.19:0/3968902678 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc7a8003680 con 0x55776f0b51d0 2024-05-07T02:38:25.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.292+0000 7fc7cb7d6b80 10 client.16050 fetch_fsmap learned FSMap version 44 2024-05-07T02:38:25.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.292+0000 7fc7cb7d6b80 1 -- 172.21.15.19:0/3968902678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55776f0cc6e0 con 0x55776f0b51d0 2024-05-07T02:38:25.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.292+0000 7fc797e00700 1 -- 172.21.15.19:0/3968902678 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7a4004830 con 0x55776f0b51d0 2024-05-07T02:38:25.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.292+0000 7fc797e00700 1 -- 172.21.15.19:0/3968902678 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 44) ==== 8378+0+0 (secure 0 0 0) 0x7fc7a800b8e0 con 0x55776f0b51d0 2024-05-07T02:38:25.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.292+0000 7fc7cb7d6b80 10 client.16050 fetch_fsmap finished waiting for FSMap version 44 2024-05-07T02:38:25.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.293+0000 7fc7cb7d6b80 10 client.16050 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-07T02:38:25.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.293+0000 7fc7cb7d6b80 20 client.16050 populate_metadata read hostname 'smithi019' 2024-05-07T02:38:25.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.293+0000 7fc7cb7d6b80 1 --2- 172.21.15.19:0/3968902678 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x55776f0cec10 0x55776f0d1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:25.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.293+0000 7fc7cb7d6b80 4 client.16050 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:38:25.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.293+0000 7fc7cb7d6b80 1 -- 172.21.15.19:0/3968902678 --> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55776f07d2e0 con 0x55776f0cec10 2024-05-07T02:38:25.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.293+0000 7fc7b5400700 1 --2- 172.21.15.19:0/3968902678 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x55776f0cec10 0x55776f0d1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:25.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.294+0000 7fc7b5400700 1 --2- 172.21.15.19:0/3968902678 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x55776f0cec10 0x55776f0d1080 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2eb20a521abfa4e6 server_cookie=e24cfde3d0308886 in_seq=0 out_seq=0 2024-05-07T02:38:25.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.294+0000 7fc797e00700 10 client.16050 ms_handle_connect on v2:172.21.15.19:6840/3905645246 2024-05-07T02:38:25.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.294+0000 7fc797e00700 1 -- 172.21.15.19:0/3968902678 <== mds.1 v2:172.21.15.19:6840/3905645246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55776f07d2e0 con 0x55776f0cec10 2024-05-07T02:38:25.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.294+0000 7fc797e00700 10 client.16050 handle_command_reply: tid=0 2024-05-07T02:38:25.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.295+0000 7fc7cb7d6b80 1 client.16050 shutdown 2024-05-07T02:38:25.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.295+0000 7fc7cb7d6b80 1 -- 172.21.15.19:0/3968902678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55776f0b51d0 msgr2=0x55776f0bef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:25.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.295+0000 7fc7cb7d6b80 1 --2- 172.21.15.19:0/3968902678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55776f0b51d0 0x55776f0bef00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8009230 tx=0x7fc7a8009440 comp rx=0 tx=0).stop 2024-05-07T02:38:25.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.295+0000 7fc7cb7d6b80 1 -- 172.21.15.19:0/3968902678 shutdown_connections 2024-05-07T02:38:25.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.295+0000 7fc7cb7d6b80 1 --2- 172.21.15.19:0/3968902678 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x55776f0cec10 0x55776f0d1080 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:25.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.295+0000 7fc7cb7d6b80 1 --2- 172.21.15.19:0/3968902678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55776f0ba750 0x55776f0bf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:25.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.295+0000 7fc7cb7d6b80 1 --2- 172.21.15.19:0/3968902678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55776f0b4880 0x55776f0be9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:25.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.295+0000 7fc7cb7d6b80 1 --2- 172.21.15.19:0/3968902678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55776f0b51d0 0x55776f0bef00 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:25.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.295+0000 7fc7cb7d6b80 1 -- 172.21.15.19:0/3968902678 >> 172.21.15.19:0/3968902678 conn(0x55776ef32e40 msgr2=0x55776f0af460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:25.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.295+0000 7fc797e00700 0 client.16050 ms_handle_reset on v2:172.21.15.19:6840/3905645246 2024-05-07T02:38:25.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.295+0000 7fc7cb7d6b80 1 -- 172.21.15.19:0/3968902678 shutdown_connections 2024-05-07T02:38:25.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.295+0000 7fc7cb7d6b80 1 -- 172.21.15.19:0/3968902678 wait complete. 2024-05-07T02:38:25.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.296+0000 7fc7cb7d6b80 20 client.16050 trim_cache size 0 max 16384 2024-05-07T02:38:25.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.312+0000 7fc7cb7d6b80 1 Processor -- start 2024-05-07T02:38:25.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.312+0000 7fc7cb7d6b80 1 -- start start 2024-05-07T02:38:25.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.313+0000 7fc7cb7d6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55776f0b4880 0x55776f0bc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:25.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.313+0000 7fc7cb7d6b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55776f0b51d0 0x55776f0cab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:25.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.313+0000 7fc7cb7d6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55776f0ba750 0x55776f0d6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:25.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.313+0000 7fc7cb7d6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b81ae970 con 0x55776f0b4880 2024-05-07T02:38:25.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.313+0000 7fc7cb7d6b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b81b0570 con 0x55776f0b51d0 2024-05-07T02:38:25.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.313+0000 7fc7cb7d6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b81af040 con 0x55776f0ba750 2024-05-07T02:38:25.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.313+0000 7fc7b5400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55776f0b4880 0x55776f0bc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:25.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.313+0000 7fc7b5400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55776f0b4880 0x55776f0bc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35550/0 (socket says 172.21.15.19:35550) 2024-05-07T02:38:25.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.313+0000 7fc7b5400700 1 -- 172.21.15.19:0/3706723504 learned_addr learned my addr 172.21.15.19:0/3706723504 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:38:25.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.313+0000 7fc7b5e00700 1 --2- 172.21.15.19:0/3706723504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55776f0b51d0 0x55776f0cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:25.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.313+0000 7fc7b5400700 1 --2- 172.21.15.19:0/3706723504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55776f0ba750 0x55776f0d6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:25.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.314+0000 7fc7b5400700 1 -- 172.21.15.19:0/3706723504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55776f0ba750 msgr2=0x55776f0d6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:25.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.314+0000 7fc7b5400700 1 --2- 172.21.15.19:0/3706723504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55776f0ba750 0x55776f0d6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:25.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.314+0000 7fc7b5400700 1 -- 172.21.15.19:0/3706723504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55776f0b51d0 msgr2=0x55776f0cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:25.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.314+0000 7fc7b5400700 1 --2- 172.21.15.19:0/3706723504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55776f0b51d0 0x55776f0cab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:25.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.314+0000 7fc7b5400700 1 -- 172.21.15.19:0/3706723504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55776f0bfa80 con 0x55776f0b4880 2024-05-07T02:38:25.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.314+0000 7fc7b5e00700 1 --2- 172.21.15.19:0/3706723504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55776f0b51d0 0x55776f0cab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:38:25.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.314+0000 7fc7b5400700 1 --2- 172.21.15.19:0/3706723504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55776f0b4880 0x55776f0bc2f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc7a400f6a0 tx=0x7fc7a4011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b40e313d4f4f7a29 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:25.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.314+0000 7fc7a2a00700 1 -- 172.21.15.19:0/3706723504 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a4028040 con 0x55776f0b4880 2024-05-07T02:38:25.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.314+0000 7fc7a2a00700 1 -- 172.21.15.19:0/3706723504 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7a4004830 con 0x55776f0b4880 2024-05-07T02:38:25.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.314+0000 7fc7a2a00700 1 -- 172.21.15.19:0/3706723504 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a4032910 con 0x55776f0b4880 2024-05-07T02:38:25.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.315+0000 7fc7cb7d6b80 1 -- 172.21.15.19:0/3706723504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55776f0b4880 msgr2=0x55776f0bc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:25.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.315+0000 7fc7cb7d6b80 1 --2- 172.21.15.19:0/3706723504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55776f0b4880 0x55776f0bc2f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc7a400f6a0 tx=0x7fc7a4011440 comp rx=0 tx=0).stop 2024-05-07T02:38:25.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.315+0000 7fc7cb7d6b80 1 -- 172.21.15.19:0/3706723504 shutdown_connections 2024-05-07T02:38:25.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.315+0000 7fc7cb7d6b80 1 --2- 172.21.15.19:0/3706723504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55776f0ba750 0x55776f0d6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:25.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.315+0000 7fc7cb7d6b80 1 --2- 172.21.15.19:0/3706723504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55776f0b4880 0x55776f0bc2f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:25.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.315+0000 7fc7cb7d6b80 1 --2- 172.21.15.19:0/3706723504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55776f0b51d0 0x55776f0cab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:25.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.315+0000 7fc7cb7d6b80 1 -- 172.21.15.19:0/3706723504 >> 172.21.15.19:0/3706723504 conn(0x55776ef32e40 msgr2=0x55776f0af460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:25.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.315+0000 7fc7cb7d6b80 1 -- 172.21.15.19:0/3706723504 shutdown_connections 2024-05-07T02:38:25.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.315+0000 7fc7cb7d6b80 1 -- 172.21.15.19:0/3706723504 wait complete. 2024-05-07T02:38:25.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.316+0000 7fc7cb7d6b80 1 -- start start 2024-05-07T02:38:25.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.316+0000 7fc7cb7d6b80 1 Processor -- start 2024-05-07T02:38:25.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.317+0000 7fc7cb7d6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55776f0b4880 0x55776f0c7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:25.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.317+0000 7fc7cb7d6b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55776f0b51d0 0x55776f0b3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:25.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.317+0000 7fc7cb7d6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55776f0ba750 0x55776f0b3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:25.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.317+0000 7fc7cb7d6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b81afde0 con 0x55776f0ba750 2024-05-07T02:38:25.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.317+0000 7fc7cb7d6b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b81af710 con 0x55776f0b51d0 2024-05-07T02:38:25.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.317+0000 7fc7cb7d6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b81ac910 con 0x55776f0b4880 2024-05-07T02:38:25.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.317+0000 7fc7cb7d6b80 10 client.0 _ll_register_callbacks cb 0x55776ef79c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:38:25.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.317+0000 7fc7b5e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55776f0b51d0 0x55776f0b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:25.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.317+0000 7fc7b5400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55776f0ba750 0x55776f0b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:25.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.317+0000 7fc7b5e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55776f0b51d0 0x55776f0b3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:56168/0 (socket says 172.21.15.19:56168) 2024-05-07T02:38:25.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.317+0000 7fc7b5e00700 1 -- 172.21.15.19:0/512357988 learned_addr learned my addr 172.21.15.19:0/512357988 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:38:25.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.317+0000 7fc7b5400700 1 --2- 172.21.15.19:0/512357988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55776f0b4880 0x55776f0c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:25.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.318+0000 7fc7b5e00700 1 -- 172.21.15.19:0/512357988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55776f0b4880 msgr2=0x55776f0c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:25.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.318+0000 7fc7b5e00700 1 --2- 172.21.15.19:0/512357988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55776f0b4880 0x55776f0c7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:25.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.318+0000 7fc7b5e00700 1 -- 172.21.15.19:0/512357988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55776f0ba750 msgr2=0x55776f0b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:25.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.318+0000 7fc7b5e00700 1 --2- 172.21.15.19:0/512357988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55776f0ba750 0x55776f0b3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:25.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.318+0000 7fc7b5e00700 1 -- 172.21.15.19:0/512357988 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55776f0b3bc0 con 0x55776f0b51d0 2024-05-07T02:38:25.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.318+0000 7fc7b5e00700 1 --2- 172.21.15.19:0/512357988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55776f0b51d0 0x55776f0b3010 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc7a800bd00 tx=0x7fc7a8005500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74d683750b489ee9 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:25.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.318+0000 7fc797e00700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:38:25.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.318+0000 7fc797e00700 1 -- 172.21.15.19:0/512357988 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a803b070 con 0x55776f0b51d0 2024-05-07T02:38:25.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.318+0000 7fc797e00700 1 -- 172.21.15.19:0/512357988 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fc7a8023b80 con 0x55776f0b51d0 2024-05-07T02:38:25.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.318+0000 7fc7cb7d6b80 1 -- 172.21.15.19:0/512357988 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7a4004830 con 0x55776f0b51d0 2024-05-07T02:38:25.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.319+0000 7fc797e00700 1 -- 172.21.15.19:0/512357988 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc7a80058e0 con 0x55776f0b51d0 2024-05-07T02:38:25.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.319+0000 7fc7cb7d6b80 10 client.26255 fetch_fsmap learned FSMap version 44 2024-05-07T02:38:25.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.319+0000 7fc7cb7d6b80 1 -- 172.21.15.19:0/512357988 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55776f0c5ce0 con 0x55776f0b51d0 2024-05-07T02:38:25.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.319+0000 7fc797e00700 1 -- 172.21.15.19:0/512357988 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7a4004830 con 0x55776f0b51d0 2024-05-07T02:38:25.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.319+0000 7fc797e00700 1 -- 172.21.15.19:0/512357988 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 44) ==== 8378+0+0 (secure 0 0 0) 0x7fc7a8003820 con 0x55776f0b51d0 2024-05-07T02:38:25.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.319+0000 7fc7cb7d6b80 10 client.26255 fetch_fsmap finished waiting for FSMap version 44 2024-05-07T02:38:25.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.319+0000 7fc7cb7d6b80 10 client.26255 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-07T02:38:25.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.319+0000 7fc7cb7d6b80 20 client.26255 populate_metadata read hostname 'smithi019' 2024-05-07T02:38:25.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.319+0000 7fc7cb7d6b80 1 --2- 172.21.15.19:0/512357988 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x55776f13e110 0x55776f140530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:25.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.319+0000 7fc7cb7d6b80 4 client.26255 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:38:25.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.319+0000 7fc7cb7d6b80 1 -- 172.21.15.19:0/512357988 --> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55776efe0400 con 0x55776f13e110 2024-05-07T02:38:25.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.320+0000 7fc7b5400700 1 --2- 172.21.15.19:0/512357988 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x55776f13e110 0x55776f140530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:25.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.320+0000 7fc7b5400700 1 --2- 172.21.15.19:0/512357988 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x55776f13e110 0x55776f140530 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f73e0922b69f5cdd server_cookie=da6940947e7c18f3 in_seq=0 out_seq=0 2024-05-07T02:38:25.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.320+0000 7fc797e00700 10 client.26255 ms_handle_connect on v2:172.21.15.19:6840/3905645246 2024-05-07T02:38:25.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.321+0000 7fc797e00700 1 -- 172.21.15.19:0/512357988 <== mds.1 v2:172.21.15.19:6840/3905645246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55776efe0400 con 0x55776f13e110 2024-05-07T02:38:25.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.321+0000 7fc797e00700 10 client.26255 handle_command_reply: tid=0 2024-05-07T02:38:25.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.321+0000 7fc7cb7d6b80 1 client.26255 shutdown 2024-05-07T02:38:25.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.321+0000 7fc7cb7d6b80 1 -- 172.21.15.19:0/512357988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55776f0b51d0 msgr2=0x55776f0b3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:25.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.321+0000 7fc7cb7d6b80 1 --2- 172.21.15.19:0/512357988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55776f0b51d0 0x55776f0b3010 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc7a800bd00 tx=0x7fc7a8005500 comp rx=0 tx=0).stop 2024-05-07T02:38:25.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.321+0000 7fc7cb7d6b80 1 -- 172.21.15.19:0/512357988 shutdown_connections 2024-05-07T02:38:25.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.321+0000 7fc7cb7d6b80 1 --2- 172.21.15.19:0/512357988 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x55776f13e110 0x55776f140530 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:25.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.321+0000 7fc7cb7d6b80 1 --2- 172.21.15.19:0/512357988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55776f0ba750 0x55776f0b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:25.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.321+0000 7fc7cb7d6b80 1 --2- 172.21.15.19:0/512357988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55776f0b4880 0x55776f0c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:25.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.321+0000 7fc797e00700 0 client.26255 ms_handle_reset on v2:172.21.15.19:6840/3905645246 2024-05-07T02:38:25.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.321+0000 7fc7cb7d6b80 1 --2- 172.21.15.19:0/512357988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55776f0b51d0 0x55776f0b3010 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:25.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.321+0000 7fc7cb7d6b80 1 -- 172.21.15.19:0/512357988 >> 172.21.15.19:0/512357988 conn(0x55776ef32e40 msgr2=0x55776f0a58b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:25.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.321+0000 7fc7cb7d6b80 1 -- 172.21.15.19:0/512357988 shutdown_connections 2024-05-07T02:38:25.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.321+0000 7fc7cb7d6b80 1 -- 172.21.15.19:0/512357988 wait complete. 2024-05-07T02:38:25.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.322+0000 7fc7cb7d6b80 20 client.26255 trim_cache size 0 max 16384 2024-05-07T02:38:25.324 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:38:25.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.322+0000 7fc7bcc00700 1 -- 172.21.15.19:0/1196091011 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc79c0b29f0 msgr2=0x7fc79c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:25.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.322+0000 7fc7bcc00700 1 --2- 172.21.15.19:0/1196091011 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc79c0b29f0 0x7fc79c0b4f00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc7b00e2710 tx=0x7fc7b0009040 comp rx=0 tx=0).stop 2024-05-07T02:38:25.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.322+0000 7fc7bcc00700 1 -- 172.21.15.19:0/1196091011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7b819f790 msgr2=0x7fc7b819fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:25.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.322+0000 7fc7bcc00700 1 --2- 172.21.15.19:0/1196091011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7b819f790 0x7fc7b819fba0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc7b000efc0 tx=0x7fc7b000c3a0 comp rx=0 tx=0).stop 2024-05-07T02:38:25.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.322+0000 7fc7bcc00700 1 -- 172.21.15.19:0/1196091011 shutdown_connections 2024-05-07T02:38:25.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.322+0000 7fc7bcc00700 1 --2- 172.21.15.19:0/1196091011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7b819f790 0x7fc7b819fba0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:25.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.322+0000 7fc7bcc00700 1 --2- 172.21.15.19:0/1196091011 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc79c0b29f0 0x7fc79c0b4f00 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:25.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.322+0000 7fc7bcc00700 1 --2- 172.21.15.19:0/1196091011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7b8107df0 0x7fc7b819b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:25.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.322+0000 7fc7bcc00700 1 --2- 172.21.15.19:0/1196091011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7b8108740 0x7fc7b819b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:25.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.322+0000 7fc7bcc00700 1 -- 172.21.15.19:0/1196091011 >> 172.21.15.19:0/1196091011 conn(0x7fc7b80fedc0 msgr2=0x7fc7b806b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:25.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.323+0000 7fc7bcc00700 1 -- 172.21.15.19:0/1196091011 shutdown_connections 2024-05-07T02:38:25.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:25.323+0000 7fc7bcc00700 1 -- 172.21.15.19:0/1196091011 wait complete. 2024-05-07T02:38:25.764 INFO:tasks.workunit.client.0.smithi019.stdout: 1 222006 3.97 MB/sec execute 201 sec latency 1276.841 ms 2024-05-07T02:38:26.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:26 smithi019 ceph-mon[29781]: pgmap v200: 129 pgs: 129 active+clean; 153 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 986 KiB/s wr, 99 op/s 2024-05-07T02:38:26.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:26 smithi064 ceph-mon[34699]: pgmap v200: 129 pgs: 129 active+clean; 153 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 986 KiB/s wr, 99 op/s 2024-05-07T02:38:26.395 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-07T02:38:26.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:26 smithi027 ceph-mon[34398]: pgmap v200: 129 pgs: 129 active+clean; 153 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 986 KiB/s wr, 99 op/s 2024-05-07T02:38:26.763 INFO:tasks.workunit.client.0.smithi019.stdout: 1 223084 3.99 MB/sec execute 202 sec latency 72.029 ms 2024-05-07T02:38:26.901 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:38:27.311 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:27 smithi019 ceph-mon[29781]: pgmap v201: 129 pgs: 129 active+clean; 154 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 163 op/s 2024-05-07T02:38:27.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:27 smithi064 ceph-mon[34699]: pgmap v201: 129 pgs: 129 active+clean; 154 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 163 op/s 2024-05-07T02:38:27.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:27 smithi027 ceph-mon[34398]: pgmap v201: 129 pgs: 129 active+clean; 154 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 163 op/s 2024-05-07T02:38:27.763 INFO:tasks.workunit.client.0.smithi019.stdout: 1 223838 3.99 MB/sec execute 203 sec latency 23.864 ms 2024-05-07T02:38:28.763 INFO:tasks.workunit.client.0.smithi019.stdout: 1 224294 3.98 MB/sec execute 204 sec latency 27.765 ms 2024-05-07T02:38:29.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:29 smithi064 ceph-mon[34699]: pgmap v202: 129 pgs: 129 active+clean; 154 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 1.2 MiB/s wr, 114 op/s 2024-05-07T02:38:29.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:29 smithi019 ceph-mon[29781]: pgmap v202: 129 pgs: 129 active+clean; 154 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 1.2 MiB/s wr, 114 op/s 2024-05-07T02:38:29.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:29 smithi027 ceph-mon[34398]: pgmap v202: 129 pgs: 129 active+clean; 154 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 1.2 MiB/s wr, 114 op/s 2024-05-07T02:38:29.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.486+0000 7fce79c00700 1 -- 172.21.15.19:0/1788426886 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce7410e8f0 msgr2=0x7fce74110dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:29.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.486+0000 7fce79c00700 1 --2- 172.21.15.19:0/1788426886 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce7410e8f0 0x7fce74110dd0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fce6c00b0a0 tx=0x7fce6c00b3b0 comp rx=0 tx=0).stop 2024-05-07T02:38:29.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.486+0000 7fce79c00700 1 -- 172.21.15.19:0/1788426886 shutdown_connections 2024-05-07T02:38:29.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.486+0000 7fce79c00700 1 --2- 172.21.15.19:0/1788426886 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce7410e8f0 0x7fce74110dd0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:29.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.487+0000 7fce79c00700 1 --2- 172.21.15.19:0/1788426886 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce74105480 0x7fce74105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:29.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.487+0000 7fce79c00700 1 --2- 172.21.15.19:0/1788426886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce74105dd0 0x7fce7410e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:29.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.487+0000 7fce79c00700 1 -- 172.21.15.19:0/1788426886 >> 172.21.15.19:0/1788426886 conn(0x7fce740fef30 msgr2=0x7fce74101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:29.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.487+0000 7fce79c00700 1 -- 172.21.15.19:0/1788426886 shutdown_connections 2024-05-07T02:38:29.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.487+0000 7fce79c00700 1 -- 172.21.15.19:0/1788426886 wait complete. 2024-05-07T02:38:29.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.488+0000 7fce79c00700 1 Processor -- start 2024-05-07T02:38:29.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.489+0000 7fce79c00700 1 -- start start 2024-05-07T02:38:29.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.489+0000 7fce79c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce74105480 0x7fce7419ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:29.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.489+0000 7fce79c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce74105dd0 0x7fce7419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:29.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.489+0000 7fce79c00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce7410e8f0 0x7fce7419f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:29.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.489+0000 7fce79c00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fce74114450 con 0x7fce74105dd0 2024-05-07T02:38:29.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.489+0000 7fce79c00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fce741142d0 con 0x7fce74105480 2024-05-07T02:38:29.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.489+0000 7fce79c00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fce741145d0 con 0x7fce7410e8f0 2024-05-07T02:38:29.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.490+0000 7fce72a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce74105480 0x7fce7419ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:29.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.490+0000 7fce72a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce74105480 0x7fce7419ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:36560/0 (socket says 172.21.15.19:36560) 2024-05-07T02:38:29.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.490+0000 7fce73400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce7410e8f0 0x7fce7419f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:29.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.490+0000 7fce72a00700 1 -- 172.21.15.19:0/4047272299 learned_addr learned my addr 172.21.15.19:0/4047272299 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:38:29.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.490+0000 7fce72000700 1 --2- 172.21.15.19:0/4047272299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce74105dd0 0x7fce7419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:29.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.490+0000 7fce73400700 1 -- 172.21.15.19:0/4047272299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce74105480 msgr2=0x7fce7419ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:29.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.490+0000 7fce73400700 1 --2- 172.21.15.19:0/4047272299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce74105480 0x7fce7419ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:29.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.490+0000 7fce73400700 1 -- 172.21.15.19:0/4047272299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce74105dd0 msgr2=0x7fce7419b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:29.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.490+0000 7fce73400700 1 --2- 172.21.15.19:0/4047272299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce74105dd0 0x7fce7419b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:29.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.490+0000 7fce73400700 1 -- 172.21.15.19:0/4047272299 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce7419fc70 con 0x7fce7410e8f0 2024-05-07T02:38:29.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.490+0000 7fce72a00700 1 --2- 172.21.15.19:0/4047272299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce74105480 0x7fce7419ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:38:29.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.490+0000 7fce73400700 1 --2- 172.21.15.19:0/4047272299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce7410e8f0 0x7fce7419f5d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fce6c00b070 tx=0x7fce6c004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:29.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.491+0000 7fce6b400700 1 -- 172.21.15.19:0/4047272299 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6c021070 con 0x7fce7410e8f0 2024-05-07T02:38:29.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.491+0000 7fce6b400700 1 -- 172.21.15.19:0/4047272299 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce6c007e30 con 0x7fce7410e8f0 2024-05-07T02:38:29.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.491+0000 7fce6b400700 1 -- 172.21.15.19:0/4047272299 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6c004630 con 0x7fce7410e8f0 2024-05-07T02:38:29.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.491+0000 7fce79c00700 1 -- 172.21.15.19:0/4047272299 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce7419ff00 con 0x7fce7410e8f0 2024-05-07T02:38:29.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.491+0000 7fce79c00700 1 -- 172.21.15.19:0/4047272299 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce741030a0 con 0x7fce7410e8f0 2024-05-07T02:38:29.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.495+0000 7fce6b400700 1 -- 172.21.15.19:0/4047272299 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fce6c003c90 con 0x7fce7410e8f0 2024-05-07T02:38:29.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.496+0000 7fce6b400700 1 --2- 172.21.15.19:0/4047272299 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fce580b29f0 0x7fce580b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:29.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.496+0000 7fce886eab80 1 Processor -- start 2024-05-07T02:38:29.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.496+0000 7fce6b400700 1 -- 172.21.15.19:0/4047272299 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fce6c00f020 con 0x7fce7410e8f0 2024-05-07T02:38:29.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.497+0000 7fce72a00700 1 --2- 172.21.15.19:0/4047272299 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fce580b29f0 0x7fce580b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:29.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.497+0000 7fce886eab80 1 -- start start 2024-05-07T02:38:29.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.497+0000 7fce886eab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5a3f61470 0x55b5a3f61880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:29.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.497+0000 7fce886eab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5a3f5b620 0x55b5a3f5ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:29.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.497+0000 7fce886eab80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5a3f5bf70 0x55b5a3f68150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:29.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.497+0000 7fce886eab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b5a3e7da20 con 0x55b5a3f5b620 2024-05-07T02:38:29.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.497+0000 7fce886eab80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b5a3e1b760 con 0x55b5a3f5bf70 2024-05-07T02:38:29.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.497+0000 7fce886eab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b5a3e799e0 con 0x55b5a3f61470 2024-05-07T02:38:29.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.498+0000 7fce72000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5a3f5b620 0x55b5a3f5ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:29.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.498+0000 7fce72000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5a3f5b620 0x55b5a3f5ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47394/0 (socket says 172.21.15.19:47394) 2024-05-07T02:38:29.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.498+0000 7fce72000700 1 -- 172.21.15.19:0/651017556 learned_addr learned my addr 172.21.15.19:0/651017556 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:38:29.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.498+0000 7fce72a00700 1 --2- 172.21.15.19:0/651017556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5a3f61470 0x55b5a3f61880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:29.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.498+0000 7fce73400700 1 --2- 172.21.15.19:0/651017556 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5a3f5bf70 0x55b5a3f68150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:29.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.498+0000 7fce72a00700 1 --2- 172.21.15.19:0/4047272299 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fce580b29f0 0x7fce580b4f00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fce64011630 tx=0x7fce64009260 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:29.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.498+0000 7fce72a00700 1 -- 172.21.15.19:0/651017556 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5a3f5bf70 msgr2=0x55b5a3f68150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:29.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.498+0000 7fce72a00700 1 --2- 172.21.15.19:0/651017556 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5a3f5bf70 0x55b5a3f68150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:29.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.498+0000 7fce72a00700 1 -- 172.21.15.19:0/651017556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5a3f5b620 msgr2=0x55b5a3f5ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:29.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.498+0000 7fce72a00700 1 --2- 172.21.15.19:0/651017556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5a3f5b620 0x55b5a3f5ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:29.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.499+0000 7fce72a00700 1 -- 172.21.15.19:0/651017556 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5a3dca8d0 con 0x55b5a3f61470 2024-05-07T02:38:29.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.499+0000 7fce72000700 1 --2- 172.21.15.19:0/651017556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5a3f5b620 0x55b5a3f5ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:38:29.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.499+0000 7fce72a00700 1 --2- 172.21.15.19:0/651017556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5a3f61470 0x55b5a3f61880 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fce64009fd0 tx=0x7fce64014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=484edb7ac11c71ad server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:29.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.499+0000 7fce5fe00700 1 -- 172.21.15.19:0/651017556 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce64004220 con 0x55b5a3f61470 2024-05-07T02:38:29.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.499+0000 7fce5fe00700 1 -- 172.21.15.19:0/651017556 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce640043c0 con 0x55b5a3f61470 2024-05-07T02:38:29.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.499+0000 7fce5fe00700 1 -- 172.21.15.19:0/651017556 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6401ba30 con 0x55b5a3f61470 2024-05-07T02:38:29.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.500+0000 7fce886eab80 1 -- 172.21.15.19:0/651017556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5a3f61470 msgr2=0x55b5a3f61880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:29.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.500+0000 7fce886eab80 1 --2- 172.21.15.19:0/651017556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5a3f61470 0x55b5a3f61880 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fce64009fd0 tx=0x7fce64014000 comp rx=0 tx=0).stop 2024-05-07T02:38:29.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.501+0000 7fce886eab80 1 -- 172.21.15.19:0/651017556 shutdown_connections 2024-05-07T02:38:29.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.501+0000 7fce886eab80 1 --2- 172.21.15.19:0/651017556 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5a3f5bf70 0x55b5a3f68150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:29.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.501+0000 7fce886eab80 1 --2- 172.21.15.19:0/651017556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5a3f61470 0x55b5a3f61880 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:29.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.501+0000 7fce886eab80 1 --2- 172.21.15.19:0/651017556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5a3f5b620 0x55b5a3f5ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:29.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.501+0000 7fce886eab80 1 -- 172.21.15.19:0/651017556 >> 172.21.15.19:0/651017556 conn(0x55b5a3ddde40 msgr2=0x55b5a3f56380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:29.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.501+0000 7fce886eab80 1 -- 172.21.15.19:0/651017556 shutdown_connections 2024-05-07T02:38:29.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.501+0000 7fce886eab80 1 -- 172.21.15.19:0/651017556 wait complete. 2024-05-07T02:38:29.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.501+0000 7fce886eab80 1 -- start start 2024-05-07T02:38:29.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.502+0000 7fce886eab80 1 Processor -- start 2024-05-07T02:38:29.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.502+0000 7fce886eab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5a3f5b620 0x55b5a3f656d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:29.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.502+0000 7fce886eab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5a3f5bf70 0x55b5a3f65c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:29.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.502+0000 7fce886eab80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5a3f61470 0x55b5a3f66150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:29.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.502+0000 7fce886eab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b5a3f68840 con 0x55b5a3f5bf70 2024-05-07T02:38:29.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.502+0000 7fce886eab80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b5a3de3b20 con 0x55b5a3f61470 2024-05-07T02:38:29.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.502+0000 7fce886eab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b5a3f689c0 con 0x55b5a3f5b620 2024-05-07T02:38:29.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.502+0000 7fce886eab80 10 client.0 _ll_register_callbacks cb 0x55b5a3e11380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:38:29.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.503+0000 7fce73400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5a3f61470 0x55b5a3f66150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:29.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.503+0000 7fce73400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5a3f61470 0x55b5a3f66150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:36576/0 (socket says 172.21.15.19:36576) 2024-05-07T02:38:29.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.503+0000 7fce73400700 1 -- 172.21.15.19:0/3532164880 learned_addr learned my addr 172.21.15.19:0/3532164880 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:38:29.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.503+0000 7fce72000700 1 --2- 172.21.15.19:0/3532164880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5a3f5bf70 0x55b5a3f65c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:29.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.503+0000 7fce72a00700 1 --2- 172.21.15.19:0/3532164880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5a3f5b620 0x55b5a3f656d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:29.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.503+0000 7fce73400700 1 -- 172.21.15.19:0/3532164880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5a3f5b620 msgr2=0x55b5a3f656d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:29.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.503+0000 7fce73400700 1 --2- 172.21.15.19:0/3532164880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5a3f5b620 0x55b5a3f656d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:29.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.503+0000 7fce73400700 1 -- 172.21.15.19:0/3532164880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5a3f5bf70 msgr2=0x55b5a3f65c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:29.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.503+0000 7fce73400700 1 --2- 172.21.15.19:0/3532164880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5a3f5bf70 0x55b5a3f65c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:29.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.503+0000 7fce73400700 1 -- 172.21.15.19:0/3532164880 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5a3e43420 con 0x55b5a3f61470 2024-05-07T02:38:29.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.503+0000 7fce73400700 1 --2- 172.21.15.19:0/3532164880 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5a3f61470 0x55b5a3f66150 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fce6c0a7f90 tx=0x7fce6c00b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af629a2aa76710e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:29.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.503+0000 7fce72a00700 1 --2- 172.21.15.19:0/3532164880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5a3f5b620 0x55b5a3f656d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:38:29.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.503+0000 7fce5cc00700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:38:29.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.504+0000 7fce5cc00700 1 -- 172.21.15.19:0/3532164880 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6c0aad20 con 0x55b5a3f61470 2024-05-07T02:38:29.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.504+0000 7fce5cc00700 1 -- 172.21.15.19:0/3532164880 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fce6c0c5310 con 0x55b5a3f61470 2024-05-07T02:38:29.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.504+0000 7fce886eab80 1 -- 172.21.15.19:0/3532164880 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce640043c0 con 0x55b5a3f61470 2024-05-07T02:38:29.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.504+0000 7fce5cc00700 1 -- 172.21.15.19:0/3532164880 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fce6c0af0d0 con 0x55b5a3f61470 2024-05-07T02:38:29.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.504+0000 7fce886eab80 10 client.26267 fetch_fsmap learned FSMap version 44 2024-05-07T02:38:29.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.504+0000 7fce886eab80 1 -- 172.21.15.19:0/3532164880 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55b5a3f73490 con 0x55b5a3f61470 2024-05-07T02:38:29.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.504+0000 7fce5cc00700 1 -- 172.21.15.19:0/3532164880 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce640043c0 con 0x55b5a3f61470 2024-05-07T02:38:29.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.504+0000 7fce5cc00700 1 -- 172.21.15.19:0/3532164880 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 44) ==== 8378+0+0 (secure 0 0 0) 0x7fce6c011070 con 0x55b5a3f61470 2024-05-07T02:38:29.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.505+0000 7fce886eab80 10 client.26267 fetch_fsmap finished waiting for FSMap version 44 2024-05-07T02:38:29.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.505+0000 7fce886eab80 10 client.26267 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:38:29.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.505+0000 7fce886eab80 20 client.26267 populate_metadata read hostname 'smithi019' 2024-05-07T02:38:29.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.505+0000 7fce886eab80 1 --2- 172.21.15.19:0/3532164880 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55b5a3f75b30 0x55b5a3f77fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:29.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.505+0000 7fce886eab80 4 client.26267 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:38:29.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.505+0000 7fce886eab80 1 -- 172.21.15.19:0/3532164880 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5a3f280e0 con 0x55b5a3f75b30 2024-05-07T02:38:29.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.505+0000 7fce72a00700 1 --2- 172.21.15.19:0/3532164880 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55b5a3f75b30 0x55b5a3f77fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:29.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.506+0000 7fce72a00700 1 --2- 172.21.15.19:0/3532164880 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55b5a3f75b30 0x55b5a3f77fc0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=524bef2b25577e71 server_cookie=8546bf213a4376ca in_seq=0 out_seq=0 2024-05-07T02:38:29.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.506+0000 7fce5cc00700 10 client.26267 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:38:29.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.506+0000 7fce5cc00700 1 -- 172.21.15.19:0/3532164880 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b5a3f280e0 con 0x55b5a3f75b30 2024-05-07T02:38:29.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.506+0000 7fce5cc00700 10 client.26267 handle_command_reply: tid=0 2024-05-07T02:38:29.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.506+0000 7fce886eab80 1 client.26267 shutdown 2024-05-07T02:38:29.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.507+0000 7fce886eab80 1 -- 172.21.15.19:0/3532164880 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5a3f61470 msgr2=0x55b5a3f66150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:29.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.507+0000 7fce886eab80 1 --2- 172.21.15.19:0/3532164880 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5a3f61470 0x55b5a3f66150 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fce6c0a7f90 tx=0x7fce6c00b3b0 comp rx=0 tx=0).stop 2024-05-07T02:38:29.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.507+0000 7fce886eab80 1 -- 172.21.15.19:0/3532164880 shutdown_connections 2024-05-07T02:38:29.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.507+0000 7fce886eab80 1 --2- 172.21.15.19:0/3532164880 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55b5a3f75b30 0x55b5a3f77fc0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:29.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.507+0000 7fce886eab80 1 --2- 172.21.15.19:0/3532164880 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5a3f61470 0x55b5a3f66150 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:29.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.507+0000 7fce886eab80 1 --2- 172.21.15.19:0/3532164880 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5a3f5b620 0x55b5a3f656d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:29.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.507+0000 7fce886eab80 1 --2- 172.21.15.19:0/3532164880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5a3f5bf70 0x55b5a3f65c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:29.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.507+0000 7fce886eab80 1 -- 172.21.15.19:0/3532164880 >> 172.21.15.19:0/3532164880 conn(0x55b5a3ddde40 msgr2=0x55b5a3f5a610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:29.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.507+0000 7fce5cc00700 0 client.26267 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:38:29.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.507+0000 7fce886eab80 1 -- 172.21.15.19:0/3532164880 shutdown_connections 2024-05-07T02:38:29.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.507+0000 7fce886eab80 1 -- 172.21.15.19:0/3532164880 wait complete. 2024-05-07T02:38:29.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.507+0000 7fce886eab80 20 client.26267 trim_cache size 0 max 16384 2024-05-07T02:38:29.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.526+0000 7fce886eab80 1 Processor -- start 2024-05-07T02:38:29.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.527+0000 7fce886eab80 1 -- start start 2024-05-07T02:38:29.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.527+0000 7fce886eab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5a3f5b620 0x55b5a3f660c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:29.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.527+0000 7fce886eab80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5a3f5bf70 0x55b5a3f5fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:29.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.527+0000 7fce886eab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5a3f61470 0x55b5a3f508d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:29.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.527+0000 7fce886eab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fce741ae260 con 0x55b5a3f5b620 2024-05-07T02:38:29.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.527+0000 7fce886eab80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fce741afe60 con 0x55b5a3f5bf70 2024-05-07T02:38:29.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.527+0000 7fce886eab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fce741ae930 con 0x55b5a3f61470 2024-05-07T02:38:29.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.527+0000 7fce73400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5a3f61470 0x55b5a3f508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:29.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.527+0000 7fce72a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5a3f5b620 0x55b5a3f660c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:29.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.528+0000 7fce72000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5a3f5bf70 0x55b5a3f5fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:29.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.528+0000 7fce73400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5a3f61470 0x55b5a3f508d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:41430/0 (socket says 172.21.15.19:41430) 2024-05-07T02:38:29.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.528+0000 7fce73400700 1 -- 172.21.15.19:0/4099854719 learned_addr learned my addr 172.21.15.19:0/4099854719 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:38:29.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.528+0000 7fce73400700 1 -- 172.21.15.19:0/4099854719 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5a3f5bf70 msgr2=0x55b5a3f5fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:29.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.528+0000 7fce73400700 1 --2- 172.21.15.19:0/4099854719 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5a3f5bf70 0x55b5a3f5fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:29.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.528+0000 7fce73400700 1 -- 172.21.15.19:0/4099854719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5a3f5b620 msgr2=0x55b5a3f660c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:29.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.528+0000 7fce73400700 1 --2- 172.21.15.19:0/4099854719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5a3f5b620 0x55b5a3f660c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:29.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.528+0000 7fce73400700 1 -- 172.21.15.19:0/4099854719 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5a3f52b80 con 0x55b5a3f61470 2024-05-07T02:38:29.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.528+0000 7fce73400700 1 --2- 172.21.15.19:0/4099854719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5a3f61470 0x55b5a3f508d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fce6c0aaa20 tx=0x7fce6c0a6d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15d20372c6793047 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:29.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.528+0000 7fce5f400700 1 -- 172.21.15.19:0/4099854719 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6c0b3af0 con 0x55b5a3f61470 2024-05-07T02:38:29.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.529+0000 7fce5f400700 1 -- 172.21.15.19:0/4099854719 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce6c0c5310 con 0x55b5a3f61470 2024-05-07T02:38:29.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.529+0000 7fce5f400700 1 -- 172.21.15.19:0/4099854719 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6c0b38c0 con 0x55b5a3f61470 2024-05-07T02:38:29.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.529+0000 7fce886eab80 1 -- 172.21.15.19:0/4099854719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5a3f61470 msgr2=0x55b5a3f508d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:29.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.529+0000 7fce886eab80 1 --2- 172.21.15.19:0/4099854719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5a3f61470 0x55b5a3f508d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fce6c0aaa20 tx=0x7fce6c0a6d10 comp rx=0 tx=0).stop 2024-05-07T02:38:29.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.530+0000 7fce886eab80 1 -- 172.21.15.19:0/4099854719 shutdown_connections 2024-05-07T02:38:29.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.530+0000 7fce886eab80 1 --2- 172.21.15.19:0/4099854719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5a3f61470 0x55b5a3f508d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:29.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.530+0000 7fce886eab80 1 --2- 172.21.15.19:0/4099854719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5a3f5b620 0x55b5a3f660c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:29.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.530+0000 7fce886eab80 1 --2- 172.21.15.19:0/4099854719 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5a3f5bf70 0x55b5a3f5fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:29.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.530+0000 7fce886eab80 1 -- 172.21.15.19:0/4099854719 >> 172.21.15.19:0/4099854719 conn(0x55b5a3ddde40 msgr2=0x55b5a3f501c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:29.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.530+0000 7fce886eab80 1 -- 172.21.15.19:0/4099854719 shutdown_connections 2024-05-07T02:38:29.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.530+0000 7fce886eab80 1 -- 172.21.15.19:0/4099854719 wait complete. 2024-05-07T02:38:29.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.530+0000 7fce886eab80 1 -- start start 2024-05-07T02:38:29.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.531+0000 7fce886eab80 1 Processor -- start 2024-05-07T02:38:29.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.531+0000 7fce886eab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5a3f5b620 0x55b5a3f699c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:29.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.531+0000 7fce886eab80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5a3f5bf70 0x55b5a3f69f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:29.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.531+0000 7fce886eab80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5a3f61470 0x55b5a3f6a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:29.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.531+0000 7fce886eab80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fce741af6d0 con 0x55b5a3f61470 2024-05-07T02:38:29.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.531+0000 7fce886eab80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fce741af000 con 0x55b5a3f5bf70 2024-05-07T02:38:29.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.531+0000 7fce886eab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fce741ac200 con 0x55b5a3f5b620 2024-05-07T02:38:29.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.531+0000 7fce886eab80 10 client.0 _ll_register_callbacks cb 0x55b5a3e11380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:38:29.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.532+0000 7fce72000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5a3f5bf70 0x55b5a3f69f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:29.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.532+0000 7fce72000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5a3f5bf70 0x55b5a3f69f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:36586/0 (socket says 172.21.15.19:36586) 2024-05-07T02:38:29.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.532+0000 7fce72000700 1 -- 172.21.15.19:0/3095311570 learned_addr learned my addr 172.21.15.19:0/3095311570 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:38:29.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.532+0000 7fce72a00700 1 --2- 172.21.15.19:0/3095311570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5a3f5b620 0x55b5a3f699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:29.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.532+0000 7fce73400700 1 --2- 172.21.15.19:0/3095311570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5a3f61470 0x55b5a3f6a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:29.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.532+0000 7fce72a00700 1 -- 172.21.15.19:0/3095311570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5a3f5bf70 msgr2=0x55b5a3f69f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:29.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.532+0000 7fce72a00700 1 --2- 172.21.15.19:0/3095311570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5a3f5bf70 0x55b5a3f69f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:29.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.532+0000 7fce72a00700 1 -- 172.21.15.19:0/3095311570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5a3f61470 msgr2=0x55b5a3f6a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:29.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.532+0000 7fce72a00700 1 --2- 172.21.15.19:0/3095311570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5a3f61470 0x55b5a3f6a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:29.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.532+0000 7fce72a00700 1 -- 172.21.15.19:0/3095311570 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5a3f57c80 con 0x55b5a3f5b620 2024-05-07T02:38:29.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.532+0000 7fce72000700 1 --2- 172.21.15.19:0/3095311570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5a3f5bf70 0x55b5a3f69f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:38:29.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.532+0000 7fce73400700 1 --2- 172.21.15.19:0/3095311570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5a3f61470 0x55b5a3f6a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:38:29.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.533+0000 7fce72a00700 1 --2- 172.21.15.19:0/3095311570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5a3f5b620 0x55b5a3f699c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fce6401b530 tx=0x7fce64005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd86536b0586ad86 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:29.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.533+0000 7fce5cc00700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:38:29.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.533+0000 7fce5cc00700 1 -- 172.21.15.19:0/3095311570 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce64042070 con 0x55b5a3f5b620 2024-05-07T02:38:29.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.533+0000 7fce5cc00700 1 -- 172.21.15.19:0/3095311570 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fce64004230 con 0x55b5a3f5b620 2024-05-07T02:38:29.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.533+0000 7fce886eab80 1 -- 172.21.15.19:0/3095311570 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce6c0c5310 con 0x55b5a3f5b620 2024-05-07T02:38:29.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.534+0000 7fce5cc00700 1 -- 172.21.15.19:0/3095311570 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fce64004430 con 0x55b5a3f5b620 2024-05-07T02:38:29.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.534+0000 7fce886eab80 10 client.26236 fetch_fsmap learned FSMap version 44 2024-05-07T02:38:29.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.534+0000 7fce886eab80 1 -- 172.21.15.19:0/3095311570 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55b5a3fe0500 con 0x55b5a3f5b620 2024-05-07T02:38:29.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.534+0000 7fce5cc00700 1 -- 172.21.15.19:0/3095311570 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce6c0c5310 con 0x55b5a3f5b620 2024-05-07T02:38:29.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.534+0000 7fce5cc00700 1 -- 172.21.15.19:0/3095311570 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 44) ==== 8378+0+0 (secure 0 0 0) 0x7fce64024030 con 0x55b5a3f5b620 2024-05-07T02:38:29.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.534+0000 7fce886eab80 10 client.26236 fetch_fsmap finished waiting for FSMap version 44 2024-05-07T02:38:29.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.534+0000 7fce886eab80 10 client.26236 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:38:29.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.534+0000 7fce886eab80 20 client.26236 populate_metadata read hostname 'smithi019' 2024-05-07T02:38:29.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.534+0000 7fce886eab80 1 --2- 172.21.15.19:0/3095311570 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55b5a3fe09d0 0x55b5a3fe2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:29.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.534+0000 7fce886eab80 4 client.26236 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:38:29.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.534+0000 7fce886eab80 1 -- 172.21.15.19:0/3095311570 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b5a3e8b400 con 0x55b5a3fe09d0 2024-05-07T02:38:29.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.535+0000 7fce72000700 1 --2- 172.21.15.19:0/3095311570 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55b5a3fe09d0 0x55b5a3fe2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:29.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.535+0000 7fce72000700 1 --2- 172.21.15.19:0/3095311570 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55b5a3fe09d0 0x55b5a3fe2df0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=806ed8ddd905d49 server_cookie=b9305a65463edcfd in_seq=0 out_seq=0 2024-05-07T02:38:29.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.535+0000 7fce5cc00700 10 client.26236 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:38:29.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.536+0000 7fce5cc00700 1 -- 172.21.15.19:0/3095311570 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b5a3e8b400 con 0x55b5a3fe09d0 2024-05-07T02:38:29.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.536+0000 7fce5cc00700 10 client.26236 handle_command_reply: tid=0 2024-05-07T02:38:29.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.536+0000 7fce886eab80 1 client.26236 shutdown 2024-05-07T02:38:29.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.536+0000 7fce886eab80 1 -- 172.21.15.19:0/3095311570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5a3f5b620 msgr2=0x55b5a3f699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:29.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.536+0000 7fce886eab80 1 --2- 172.21.15.19:0/3095311570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5a3f5b620 0x55b5a3f699c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fce6401b530 tx=0x7fce64005dc0 comp rx=0 tx=0).stop 2024-05-07T02:38:29.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.536+0000 7fce886eab80 1 -- 172.21.15.19:0/3095311570 shutdown_connections 2024-05-07T02:38:29.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.536+0000 7fce886eab80 1 --2- 172.21.15.19:0/3095311570 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55b5a3fe09d0 0x55b5a3fe2df0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:29.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.536+0000 7fce886eab80 1 --2- 172.21.15.19:0/3095311570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5a3f61470 0x55b5a3f6a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:29.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.536+0000 7fce886eab80 1 --2- 172.21.15.19:0/3095311570 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b5a3f5b620 0x55b5a3f699c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:29.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.536+0000 7fce886eab80 1 --2- 172.21.15.19:0/3095311570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b5a3f5bf70 0x55b5a3f69f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:29.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.536+0000 7fce5cc00700 0 client.26236 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:38:29.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.536+0000 7fce886eab80 1 -- 172.21.15.19:0/3095311570 >> 172.21.15.19:0/3095311570 conn(0x55b5a3ddde40 msgr2=0x55b5a3f65e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:29.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.536+0000 7fce886eab80 1 -- 172.21.15.19:0/3095311570 shutdown_connections 2024-05-07T02:38:29.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.536+0000 7fce886eab80 1 -- 172.21.15.19:0/3095311570 wait complete. 2024-05-07T02:38:29.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.536+0000 7fce886eab80 20 client.26236 trim_cache size 0 max 16384 2024-05-07T02:38:29.538 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:38:29.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.537+0000 7fce79c00700 1 -- 172.21.15.19:0/4047272299 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fce580b29f0 msgr2=0x7fce580b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:29.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.537+0000 7fce79c00700 1 --2- 172.21.15.19:0/4047272299 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fce580b29f0 0x7fce580b4f00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fce64011630 tx=0x7fce64009260 comp rx=0 tx=0).stop 2024-05-07T02:38:29.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.537+0000 7fce79c00700 1 -- 172.21.15.19:0/4047272299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce7410e8f0 msgr2=0x7fce7419f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:29.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.537+0000 7fce79c00700 1 --2- 172.21.15.19:0/4047272299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce7410e8f0 0x7fce7419f5d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fce6c00b070 tx=0x7fce6c004d00 comp rx=0 tx=0).stop 2024-05-07T02:38:29.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.537+0000 7fce79c00700 1 -- 172.21.15.19:0/4047272299 shutdown_connections 2024-05-07T02:38:29.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.537+0000 7fce79c00700 1 --2- 172.21.15.19:0/4047272299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce7410e8f0 0x7fce7419f5d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:29.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.537+0000 7fce79c00700 1 --2- 172.21.15.19:0/4047272299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce74105480 0x7fce7419ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:29.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.537+0000 7fce79c00700 1 --2- 172.21.15.19:0/4047272299 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fce580b29f0 0x7fce580b4f00 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:29.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.537+0000 7fce79c00700 1 --2- 172.21.15.19:0/4047272299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce74105dd0 0x7fce7419b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:29.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.537+0000 7fce79c00700 1 -- 172.21.15.19:0/4047272299 >> 172.21.15.19:0/4047272299 conn(0x7fce740fef30 msgr2=0x7fce740ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:29.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.538+0000 7fce79c00700 1 -- 172.21.15.19:0/4047272299 shutdown_connections 2024-05-07T02:38:29.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:29.538+0000 7fce79c00700 1 -- 172.21.15.19:0/4047272299 wait complete. 2024-05-07T02:38:29.764 INFO:tasks.workunit.client.0.smithi019.stdout: 1 225103 3.98 MB/sec execute 205 sec latency 28.942 ms 2024-05-07T02:38:30.764 INFO:tasks.workunit.client.0.smithi019.stdout: 1 225576 3.97 MB/sec execute 206 sec latency 489.181 ms 2024-05-07T02:38:31.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:31 smithi064 ceph-mon[34699]: pgmap v203: 129 pgs: 129 active+clean; 154 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 1.2 MiB/s wr, 114 op/s 2024-05-07T02:38:31.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:31 smithi019 ceph-mon[29781]: pgmap v203: 129 pgs: 129 active+clean; 154 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 1.2 MiB/s wr, 114 op/s 2024-05-07T02:38:31.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:31 smithi027 ceph-mon[34398]: pgmap v203: 129 pgs: 129 active+clean; 154 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 1.2 MiB/s wr, 114 op/s 2024-05-07T02:38:31.764 INFO:tasks.workunit.client.0.smithi019.stdout: 1 226805 3.99 MB/sec execute 207 sec latency 27.959 ms 2024-05-07T02:38:31.904 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-07T02:38:31.904 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-07T02:38:32.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:32 smithi019 ceph-mon[29781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:38:32.406 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:38:32.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:32 smithi027 ceph-mon[34398]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:38:32.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:32 smithi064 ceph-mon[34699]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:38:32.764 INFO:tasks.workunit.client.0.smithi019.stdout: 1 227474 3.99 MB/sec execute 208 sec latency 30.803 ms 2024-05-07T02:38:33.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:33 smithi019 ceph-mon[29781]: pgmap v204: 129 pgs: 129 active+clean; 157 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.6 MiB/s wr, 167 op/s 2024-05-07T02:38:33.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:33 smithi027 ceph-mon[34398]: pgmap v204: 129 pgs: 129 active+clean; 157 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.6 MiB/s wr, 167 op/s 2024-05-07T02:38:33.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:33 smithi064 ceph-mon[34699]: pgmap v204: 129 pgs: 129 active+clean; 157 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.6 MiB/s wr, 167 op/s 2024-05-07T02:38:33.764 INFO:tasks.workunit.client.0.smithi019.stdout: 1 228219 3.98 MB/sec execute 209 sec latency 29.601 ms 2024-05-07T02:38:34.764 INFO:tasks.workunit.client.0.smithi019.stdout: 1 228748 3.98 MB/sec execute 210 sec latency 523.624 ms 2024-05-07T02:38:34.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.939+0000 7fdc5ec00700 1 -- 172.21.15.19:0/614487081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc5810c5c0 msgr2=0x7fdc5810c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:34.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.939+0000 7fdc5ec00700 1 --2- 172.21.15.19:0/614487081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc5810c5c0 0x7fdc5810c9d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdc48009990 tx=0x7fdc48009ca0 comp rx=0 tx=0).stop 2024-05-07T02:38:34.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.940+0000 7fdc5ec00700 1 -- 172.21.15.19:0/614487081 shutdown_connections 2024-05-07T02:38:34.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.940+0000 7fdc5ec00700 1 --2- 172.21.15.19:0/614487081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc58106ed0 0x7fdc58113360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:34.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.940+0000 7fdc5ec00700 1 --2- 172.21.15.19:0/614487081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc5810c5c0 0x7fdc5810c9d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:34.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.940+0000 7fdc5ec00700 1 --2- 172.21.15.19:0/614487081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc58106580 0x7fdc58106990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:34.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.940+0000 7fdc5ec00700 1 -- 172.21.15.19:0/614487081 >> 172.21.15.19:0/614487081 conn(0x7fdc58102080 msgr2=0x7fdc581044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:34.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.940+0000 7fdc5ec00700 1 -- 172.21.15.19:0/614487081 shutdown_connections 2024-05-07T02:38:34.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.940+0000 7fdc5ec00700 1 -- 172.21.15.19:0/614487081 wait complete. 2024-05-07T02:38:34.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.941+0000 7fdc5ec00700 1 Processor -- start 2024-05-07T02:38:34.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.941+0000 7fdc5ec00700 1 -- start start 2024-05-07T02:38:34.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.942+0000 7fdc5ec00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc58106580 0x7fdc5819d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:34.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.942+0000 7fdc5ec00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc58106ed0 0x7fdc5819d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:34.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.942+0000 7fdc5ec00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc5810c5c0 0x7fdc581a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:34.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.942+0000 7fdc5ec00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdc581166d0 con 0x7fdc58106580 2024-05-07T02:38:34.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.942+0000 7fdc5ec00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdc58116550 con 0x7fdc58106ed0 2024-05-07T02:38:34.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.942+0000 7fdc5ec00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdc58116850 con 0x7fdc5810c5c0 2024-05-07T02:38:34.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.942+0000 7fdc57e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc58106580 0x7fdc5819d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:34.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.942+0000 7fdc57e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc58106580 0x7fdc5819d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47446/0 (socket says 172.21.15.19:47446) 2024-05-07T02:38:34.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.942+0000 7fdc57e00700 1 -- 172.21.15.19:0/2770379637 learned_addr learned my addr 172.21.15.19:0/2770379637 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:38:34.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.942+0000 7fdc57400700 1 --2- 172.21.15.19:0/2770379637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc58106ed0 0x7fdc5819d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:34.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.942+0000 7fdc5cc00700 1 --2- 172.21.15.19:0/2770379637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc5810c5c0 0x7fdc581a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:34.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.943+0000 7fdc5cc00700 1 -- 172.21.15.19:0/2770379637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc58106ed0 msgr2=0x7fdc5819d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:34.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.943+0000 7fdc5cc00700 1 --2- 172.21.15.19:0/2770379637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc58106ed0 0x7fdc5819d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:34.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.943+0000 7fdc5cc00700 1 -- 172.21.15.19:0/2770379637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc58106580 msgr2=0x7fdc5819d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:34.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.943+0000 7fdc5cc00700 1 --2- 172.21.15.19:0/2770379637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc58106580 0x7fdc5819d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:34.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.943+0000 7fdc5cc00700 1 -- 172.21.15.19:0/2770379637 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc581a1fb0 con 0x7fdc5810c5c0 2024-05-07T02:38:34.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.943+0000 7fdc57400700 1 --2- 172.21.15.19:0/2770379637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc58106ed0 0x7fdc5819d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:38:34.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.943+0000 7fdc57e00700 1 --2- 172.21.15.19:0/2770379637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc58106580 0x7fdc5819d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:38:34.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.943+0000 7fdc5cc00700 1 --2- 172.21.15.19:0/2770379637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc5810c5c0 0x7fdc581a1910 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000efc0 tx=0x7fdc5000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:34.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.943+0000 7fdc54c00700 1 -- 172.21.15.19:0/2770379637 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc500092e0 con 0x7fdc5810c5c0 2024-05-07T02:38:34.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.943+0000 7fdc54c00700 1 -- 172.21.15.19:0/2770379637 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc50004bb0 con 0x7fdc5810c5c0 2024-05-07T02:38:34.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.943+0000 7fdc5ec00700 1 -- 172.21.15.19:0/2770379637 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc581a2240 con 0x7fdc5810c5c0 2024-05-07T02:38:34.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.943+0000 7fdc54c00700 1 -- 172.21.15.19:0/2770379637 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc50004020 con 0x7fdc5810c5c0 2024-05-07T02:38:34.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.944+0000 7fdc5ec00700 1 -- 172.21.15.19:0/2770379637 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc581a2730 con 0x7fdc5810c5c0 2024-05-07T02:38:34.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.947+0000 7fdc54c00700 1 -- 172.21.15.19:0/2770379637 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fdc5001b030 con 0x7fdc5810c5c0 2024-05-07T02:38:34.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.948+0000 7fdc6d67eb80 1 Processor -- start 2024-05-07T02:38:34.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.948+0000 7fdc6d67eb80 1 -- start start 2024-05-07T02:38:34.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.949+0000 7fdc6d67eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558add6dc430 0x558add6dc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:34.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.949+0000 7fdc6d67eb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558add6d64f0 0x558add6d6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:34.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.949+0000 7fdc6d67eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558add6d6e40 0x558add6e30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:34.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.949+0000 7fdc6d67eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558add5f8aa0 con 0x558add6dc430 2024-05-07T02:38:34.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.949+0000 7fdc6d67eb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x558add5967e0 con 0x558add6d64f0 2024-05-07T02:38:34.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.949+0000 7fdc6d67eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558add5f4a60 con 0x558add6d6e40 2024-05-07T02:38:34.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.949+0000 7fdc54c00700 1 --2- 172.21.15.19:0/2770379637 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fdc3c0b29f0 0x7fdc3c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:34.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.949+0000 7fdc57400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558add6dc430 0x558add6dc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:34.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.949+0000 7fdc57400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558add6dc430 0x558add6dc840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47448/0 (socket says 172.21.15.19:47448) 2024-05-07T02:38:34.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.949+0000 7fdc57400700 1 -- 172.21.15.19:0/90284379 learned_addr learned my addr 172.21.15.19:0/90284379 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:38:34.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.949+0000 7fdc57e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558add6d64f0 0x558add6d6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:34.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.950+0000 7fdc54c00700 1 -- 172.21.15.19:0/2770379637 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fdc500e2470 con 0x7fdc5810c5c0 2024-05-07T02:38:34.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.950+0000 7fdc57e00700 1 -- 172.21.15.19:0/90284379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558add6d6e40 msgr2=0x558add6e30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:34.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.950+0000 7fdc57e00700 1 --2- 172.21.15.19:0/90284379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558add6d6e40 0x558add6e30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:34.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.950+0000 7fdc57e00700 1 -- 172.21.15.19:0/90284379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558add6dc430 msgr2=0x558add6dc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:34.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.950+0000 7fdc57e00700 1 --2- 172.21.15.19:0/90284379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558add6dc430 0x558add6dc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:34.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.950+0000 7fdc57e00700 1 -- 172.21.15.19:0/90284379 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558add545950 con 0x558add6d64f0 2024-05-07T02:38:34.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.950+0000 7fdc5cc00700 1 --2- 172.21.15.19:0/2770379637 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fdc3c0b29f0 0x7fdc3c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:34.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.950+0000 7fdc57e00700 1 --2- 172.21.15.19:0/90284379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558add6d64f0 0x558add6d6900 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fdc48006010 tx=0x7fdc48009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30e40e405537d598 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:34.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.951+0000 7fdc4d600700 1 -- 172.21.15.19:0/90284379 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc48004830 con 0x558add6d64f0 2024-05-07T02:38:34.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.951+0000 7fdc4d600700 1 -- 172.21.15.19:0/90284379 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc48005020 con 0x558add6d64f0 2024-05-07T02:38:34.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.951+0000 7fdc4d600700 1 -- 172.21.15.19:0/90284379 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4800f6c0 con 0x558add6d64f0 2024-05-07T02:38:34.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.952+0000 7fdc6d67eb80 1 -- 172.21.15.19:0/90284379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558add6d64f0 msgr2=0x558add6d6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:34.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.952+0000 7fdc6d67eb80 1 --2- 172.21.15.19:0/90284379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558add6d64f0 0x558add6d6900 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fdc48006010 tx=0x7fdc48009580 comp rx=0 tx=0).stop 2024-05-07T02:38:34.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.952+0000 7fdc5cc00700 1 --2- 172.21.15.19:0/2770379637 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fdc3c0b29f0 0x7fdc3c0b4f00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdc50009fb0 tx=0x7fdc50009040 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:34.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.952+0000 7fdc6d67eb80 1 -- 172.21.15.19:0/90284379 shutdown_connections 2024-05-07T02:38:34.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.952+0000 7fdc6d67eb80 1 --2- 172.21.15.19:0/90284379 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558add6d6e40 0x558add6e30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:34.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.952+0000 7fdc6d67eb80 1 --2- 172.21.15.19:0/90284379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558add6dc430 0x558add6dc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:34.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.952+0000 7fdc6d67eb80 1 --2- 172.21.15.19:0/90284379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558add6d64f0 0x558add6d6900 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:34.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.952+0000 7fdc6d67eb80 1 -- 172.21.15.19:0/90284379 >> 172.21.15.19:0/90284379 conn(0x558add558ec0 msgr2=0x558add6d1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:34.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.952+0000 7fdc6d67eb80 1 -- 172.21.15.19:0/90284379 shutdown_connections 2024-05-07T02:38:34.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.952+0000 7fdc6d67eb80 1 -- 172.21.15.19:0/90284379 wait complete. 2024-05-07T02:38:34.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.952+0000 7fdc6d67eb80 1 -- start start 2024-05-07T02:38:34.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.953+0000 7fdc6d67eb80 1 Processor -- start 2024-05-07T02:38:34.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.953+0000 7fdc6d67eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558add6d64f0 0x558add6e0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:34.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.953+0000 7fdc6d67eb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558add6d6e40 0x558add6e0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:34.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.953+0000 7fdc6d67eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558add6dc430 0x558add6e0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:34.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.953+0000 7fdc6d67eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558add6e3780 con 0x558add6d64f0 2024-05-07T02:38:34.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.953+0000 7fdc6d67eb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x558add55eba0 con 0x558add6d6e40 2024-05-07T02:38:34.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.953+0000 7fdc6d67eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558add6e3900 con 0x558add6dc430 2024-05-07T02:38:34.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.954+0000 7fdc6d67eb80 10 client.0 _ll_register_callbacks cb 0x558add59fcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:38:34.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.954+0000 7fdc57400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558add6d64f0 0x558add6e0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:34.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.954+0000 7fdc57400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558add6d64f0 0x558add6e0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47460/0 (socket says 172.21.15.19:47460) 2024-05-07T02:38:34.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.954+0000 7fdc57400700 1 -- 172.21.15.19:0/3184773218 learned_addr learned my addr 172.21.15.19:0/3184773218 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:38:34.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.954+0000 7fdc57e00700 1 --2- 172.21.15.19:0/3184773218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558add6d6e40 0x558add6e0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:34.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.954+0000 7fdc57400700 1 --2- 172.21.15.19:0/3184773218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558add6dc430 0x558add6e0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:34.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.954+0000 7fdc57400700 1 -- 172.21.15.19:0/3184773218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558add6dc430 msgr2=0x558add6e0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:34.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.954+0000 7fdc57400700 1 --2- 172.21.15.19:0/3184773218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558add6dc430 0x558add6e0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:34.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.954+0000 7fdc57400700 1 -- 172.21.15.19:0/3184773218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558add6d6e40 msgr2=0x558add6e0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:34.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.954+0000 7fdc57400700 1 --2- 172.21.15.19:0/3184773218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558add6d6e40 0x558add6e0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:34.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.954+0000 7fdc57400700 1 -- 172.21.15.19:0/3184773218 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558add5be4a0 con 0x558add6d64f0 2024-05-07T02:38:34.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.954+0000 7fdc57e00700 1 --2- 172.21.15.19:0/3184773218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558add6d6e40 0x558add6e0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:38:34.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.955+0000 7fdc57400700 1 --2- 172.21.15.19:0/3184773218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558add6d64f0 0x558add6e0510 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdc44012f60 tx=0x7fdc44009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9556e6aca367dd2 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:34.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.955+0000 7fdc42000700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-07T02:38:34.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.955+0000 7fdc42000700 1 -- 172.21.15.19:0/3184773218 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc44010430 con 0x558add6d64f0 2024-05-07T02:38:34.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.955+0000 7fdc42000700 1 -- 172.21.15.19:0/3184773218 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fdc440105d0 con 0x558add6d64f0 2024-05-07T02:38:34.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.955+0000 7fdc6d67eb80 1 -- 172.21.15.19:0/3184773218 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc48005020 con 0x558add6d64f0 2024-05-07T02:38:34.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.955+0000 7fdc42000700 1 -- 172.21.15.19:0/3184773218 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fdc44027050 con 0x558add6d64f0 2024-05-07T02:38:34.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.955+0000 7fdc6d67eb80 10 client.16086 fetch_fsmap learned FSMap version 45 2024-05-07T02:38:34.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.955+0000 7fdc6d67eb80 1 -- 172.21.15.19:0/3184773218 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x558add6ee350 con 0x558add6d64f0 2024-05-07T02:38:34.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.956+0000 7fdc42000700 1 -- 172.21.15.19:0/3184773218 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc48005020 con 0x558add6d64f0 2024-05-07T02:38:34.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.956+0000 7fdc42000700 1 -- 172.21.15.19:0/3184773218 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 45) ==== 8382+0+0 (secure 0 0 0) 0x7fdc4401f020 con 0x558add6d64f0 2024-05-07T02:38:34.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.956+0000 7fdc6d67eb80 10 client.16086 fetch_fsmap finished waiting for FSMap version 45 2024-05-07T02:38:34.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.956+0000 7fdc6d67eb80 10 client.16086 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:38:34.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.956+0000 7fdc6d67eb80 20 client.16086 populate_metadata read hostname 'smithi019' 2024-05-07T02:38:34.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.956+0000 7fdc6d67eb80 1 --2- 172.21.15.19:0/3184773218 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x558add6f09e0 0x558add6f2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:34.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.956+0000 7fdc6d67eb80 4 client.16086 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:38:34.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.956+0000 7fdc6d67eb80 1 -- 172.21.15.19:0/3184773218 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558add6a3360 con 0x558add6f09e0 2024-05-07T02:38:34.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.956+0000 7fdc57e00700 1 --2- 172.21.15.19:0/3184773218 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x558add6f09e0 0x558add6f2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:34.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.957+0000 7fdc57e00700 1 --2- 172.21.15.19:0/3184773218 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x558add6f09e0 0x558add6f2ed0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46f9fac8ef98a881 server_cookie=7414066e4d1626c1 in_seq=0 out_seq=0 2024-05-07T02:38:34.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.957+0000 7fdc42000700 10 client.16086 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:38:34.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.958+0000 7fdc42000700 1 -- 172.21.15.19:0/3184773218 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558add6a3360 con 0x558add6f09e0 2024-05-07T02:38:34.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.958+0000 7fdc42000700 10 client.16086 handle_command_reply: tid=0 2024-05-07T02:38:34.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.958+0000 7fdc6d67eb80 1 client.16086 shutdown 2024-05-07T02:38:34.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.958+0000 7fdc6d67eb80 1 -- 172.21.15.19:0/3184773218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558add6d64f0 msgr2=0x558add6e0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:34.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.958+0000 7fdc6d67eb80 1 --2- 172.21.15.19:0/3184773218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558add6d64f0 0x558add6e0510 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdc44012f60 tx=0x7fdc44009f30 comp rx=0 tx=0).stop 2024-05-07T02:38:34.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.958+0000 7fdc6d67eb80 1 -- 172.21.15.19:0/3184773218 shutdown_connections 2024-05-07T02:38:34.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.958+0000 7fdc6d67eb80 1 --2- 172.21.15.19:0/3184773218 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x558add6f09e0 0x558add6f2ed0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:34.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.958+0000 7fdc6d67eb80 1 --2- 172.21.15.19:0/3184773218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558add6dc430 0x558add6e0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:34.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.958+0000 7fdc6d67eb80 1 --2- 172.21.15.19:0/3184773218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558add6d64f0 0x558add6e0510 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:34.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.958+0000 7fdc6d67eb80 1 --2- 172.21.15.19:0/3184773218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558add6d6e40 0x558add6e0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:34.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.958+0000 7fdc42000700 0 client.16086 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:38:34.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.958+0000 7fdc6d67eb80 1 -- 172.21.15.19:0/3184773218 >> 172.21.15.19:0/3184773218 conn(0x558add558ec0 msgr2=0x558add6d54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:34.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.958+0000 7fdc6d67eb80 1 -- 172.21.15.19:0/3184773218 shutdown_connections 2024-05-07T02:38:34.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.958+0000 7fdc6d67eb80 1 -- 172.21.15.19:0/3184773218 wait complete. 2024-05-07T02:38:34.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.959+0000 7fdc6d67eb80 20 client.16086 trim_cache size 0 max 16384 2024-05-07T02:38:34.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.981+0000 7fdc6d67eb80 1 Processor -- start 2024-05-07T02:38:34.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.981+0000 7fdc6d67eb80 1 -- start start 2024-05-07T02:38:34.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.982+0000 7fdc6d67eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558add6d64f0 0x558add6e0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:34.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.982+0000 7fdc6d67eb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558add6d6e40 0x558add6df6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:34.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.982+0000 7fdc6d67eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558add6dc430 0x558add6d5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:34.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.982+0000 7fdc6d67eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdc581b27d0 con 0x558add6dc430 2024-05-07T02:38:34.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.982+0000 7fdc6d67eb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdc581b4400 con 0x558add6d6e40 2024-05-07T02:38:34.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.982+0000 7fdc6d67eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdc581b2ea0 con 0x558add6d64f0 2024-05-07T02:38:34.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.982+0000 7fdc57e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558add6d6e40 0x558add6df6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:34.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.982+0000 7fdc57e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558add6d6e40 0x558add6df6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:36628/0 (socket says 172.21.15.19:36628) 2024-05-07T02:38:34.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.982+0000 7fdc57e00700 1 -- 172.21.15.19:0/415203618 learned_addr learned my addr 172.21.15.19:0/415203618 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:38:34.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.982+0000 7fdc57400700 1 --2- 172.21.15.19:0/415203618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558add6dc430 0x558add6d5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:34.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.982+0000 7fdc57400700 1 --2- 172.21.15.19:0/415203618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558add6d64f0 0x558add6e0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:34.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.983+0000 7fdc57e00700 1 -- 172.21.15.19:0/415203618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558add6d64f0 msgr2=0x558add6e0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:34.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.983+0000 7fdc57e00700 1 --2- 172.21.15.19:0/415203618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558add6d64f0 0x558add6e0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:34.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.983+0000 7fdc57e00700 1 -- 172.21.15.19:0/415203618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558add6dc430 msgr2=0x558add6d5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:34.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.983+0000 7fdc57e00700 1 --2- 172.21.15.19:0/415203618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558add6dc430 0x558add6d5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:34.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.983+0000 7fdc57e00700 1 -- 172.21.15.19:0/415203618 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558add6dbe50 con 0x558add6d6e40 2024-05-07T02:38:34.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.983+0000 7fdc57400700 1 --2- 172.21.15.19:0/415203618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558add6dc430 0x558add6d5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:38:34.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.983+0000 7fdc57e00700 1 --2- 172.21.15.19:0/415203618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558add6d6e40 0x558add6df6c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fdc48009750 tx=0x7fdc48005810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=772948799206e218 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:34.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.984+0000 7fdc4cc00700 1 -- 172.21.15.19:0/415203618 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4800bdb0 con 0x558add6d6e40 2024-05-07T02:38:34.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.984+0000 7fdc4cc00700 1 -- 172.21.15.19:0/415203618 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc48004ce0 con 0x558add6d6e40 2024-05-07T02:38:34.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.984+0000 7fdc4cc00700 1 -- 172.21.15.19:0/415203618 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc48034810 con 0x558add6d6e40 2024-05-07T02:38:34.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.984+0000 7fdc6d67eb80 1 -- 172.21.15.19:0/415203618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558add6d6e40 msgr2=0x558add6df6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:34.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.984+0000 7fdc6d67eb80 1 --2- 172.21.15.19:0/415203618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558add6d6e40 0x558add6df6c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fdc48009750 tx=0x7fdc48005810 comp rx=0 tx=0).stop 2024-05-07T02:38:34.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.985+0000 7fdc6d67eb80 1 -- 172.21.15.19:0/415203618 shutdown_connections 2024-05-07T02:38:34.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.985+0000 7fdc6d67eb80 1 --2- 172.21.15.19:0/415203618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558add6dc430 0x558add6d5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:34.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.985+0000 7fdc6d67eb80 1 --2- 172.21.15.19:0/415203618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558add6d64f0 0x558add6e0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:34.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.985+0000 7fdc6d67eb80 1 --2- 172.21.15.19:0/415203618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558add6d6e40 0x558add6df6c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:34.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.985+0000 7fdc6d67eb80 1 -- 172.21.15.19:0/415203618 >> 172.21.15.19:0/415203618 conn(0x558add558ec0 msgr2=0x558add6d54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:34.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.985+0000 7fdc6d67eb80 1 -- 172.21.15.19:0/415203618 shutdown_connections 2024-05-07T02:38:34.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.985+0000 7fdc6d67eb80 1 -- 172.21.15.19:0/415203618 wait complete. 2024-05-07T02:38:34.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.985+0000 7fdc6d67eb80 1 -- start start 2024-05-07T02:38:34.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.986+0000 7fdc6d67eb80 1 Processor -- start 2024-05-07T02:38:34.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.986+0000 7fdc6d67eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558add6d64f0 0x558add6e8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:34.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.986+0000 7fdc6d67eb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558add6d6e40 0x558add6e90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:34.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.986+0000 7fdc6d67eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558add6dc430 0x558add6e5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:34.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.986+0000 7fdc6d67eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdc581b3c70 con 0x558add6dc430 2024-05-07T02:38:34.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.986+0000 7fdc6d67eb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdc581b3570 con 0x558add6d6e40 2024-05-07T02:38:34.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.986+0000 7fdc6d67eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdc581b0770 con 0x558add6d64f0 2024-05-07T02:38:34.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.986+0000 7fdc6d67eb80 10 client.0 _ll_register_callbacks cb 0x558add59fcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:38:34.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.986+0000 7fdc57400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558add6d64f0 0x558add6e8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:34.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.986+0000 7fdc57400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558add6d64f0 0x558add6e8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:41480/0 (socket says 172.21.15.19:41480) 2024-05-07T02:38:34.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.986+0000 7fdc57400700 1 -- 172.21.15.19:0/987548586 learned_addr learned my addr 172.21.15.19:0/987548586 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:38:34.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.987+0000 7fdc57400700 1 --2- 172.21.15.19:0/987548586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558add6dc430 0x558add6e5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:34.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.987+0000 7fdc57e00700 1 --2- 172.21.15.19:0/987548586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558add6d6e40 0x558add6e90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:34.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.987+0000 7fdc57400700 1 -- 172.21.15.19:0/987548586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558add6d6e40 msgr2=0x558add6e90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:34.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.987+0000 7fdc57400700 1 --2- 172.21.15.19:0/987548586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558add6d6e40 0x558add6e90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:34.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.987+0000 7fdc57400700 1 -- 172.21.15.19:0/987548586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558add6dc430 msgr2=0x558add6e5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:34.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.987+0000 7fdc57400700 1 --2- 172.21.15.19:0/987548586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558add6dc430 0x558add6e5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:34.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.987+0000 7fdc57400700 1 -- 172.21.15.19:0/987548586 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558add6e6200 con 0x558add6d64f0 2024-05-07T02:38:34.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.987+0000 7fdc57e00700 1 --2- 172.21.15.19:0/987548586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558add6d6e40 0x558add6e90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:38:34.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.987+0000 7fdc57400700 1 --2- 172.21.15.19:0/987548586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558add6d64f0 0x558add6e8b90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdc44012f30 tx=0x7fdc44012be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a1616fef827d12e server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:34.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.987+0000 7fdc42000700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:38:34.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.988+0000 7fdc42000700 1 -- 172.21.15.19:0/987548586 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc44012110 con 0x558add6d64f0 2024-05-07T02:38:34.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.988+0000 7fdc6d67eb80 1 -- 172.21.15.19:0/987548586 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc48004ce0 con 0x558add6d64f0 2024-05-07T02:38:34.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.988+0000 7fdc42000700 1 -- 172.21.15.19:0/987548586 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7fdc440122b0 con 0x558add6d64f0 2024-05-07T02:38:34.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.988+0000 7fdc42000700 1 -- 172.21.15.19:0/987548586 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fdc44004030 con 0x558add6d64f0 2024-05-07T02:38:34.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.988+0000 7fdc6d67eb80 10 client.26251 fetch_fsmap learned FSMap version 45 2024-05-07T02:38:34.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.988+0000 7fdc6d67eb80 1 -- 172.21.15.19:0/987548586 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x558add7642f0 con 0x558add6d64f0 2024-05-07T02:38:34.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.988+0000 7fdc42000700 1 -- 172.21.15.19:0/987548586 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc48004ce0 con 0x558add6d64f0 2024-05-07T02:38:34.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.989+0000 7fdc42000700 1 -- 172.21.15.19:0/987548586 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 45) ==== 8382+0+0 (secure 0 0 0) 0x7fdc4401f070 con 0x558add6d64f0 2024-05-07T02:38:34.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.989+0000 7fdc6d67eb80 10 client.26251 fetch_fsmap finished waiting for FSMap version 45 2024-05-07T02:38:34.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.989+0000 7fdc6d67eb80 10 client.26251 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:38:34.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.989+0000 7fdc6d67eb80 20 client.26251 populate_metadata read hostname 'smithi019' 2024-05-07T02:38:34.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.989+0000 7fdc6d67eb80 1 --2- 172.21.15.19:0/987548586 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x558add764a60 0x558add766e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:34.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.989+0000 7fdc6d67eb80 4 client.26251 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-07T02:38:34.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.989+0000 7fdc6d67eb80 1 -- 172.21.15.19:0/987548586 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558add606480 con 0x558add764a60 2024-05-07T02:38:34.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.989+0000 7fdc57e00700 1 --2- 172.21.15.19:0/987548586 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x558add764a60 0x558add766e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:34.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.990+0000 7fdc57e00700 1 --2- 172.21.15.19:0/987548586 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x558add764a60 0x558add766e80 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cf9a2541e3b8fe5 server_cookie=6bc9f7fc024959a7 in_seq=0 out_seq=0 2024-05-07T02:38:34.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.990+0000 7fdc42000700 10 client.26251 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:38:34.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.991+0000 7fdc42000700 1 -- 172.21.15.19:0/987548586 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558add606480 con 0x558add764a60 2024-05-07T02:38:34.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.991+0000 7fdc42000700 10 client.26251 handle_command_reply: tid=0 2024-05-07T02:38:34.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.991+0000 7fdc6d67eb80 1 client.26251 shutdown 2024-05-07T02:38:34.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.991+0000 7fdc6d67eb80 1 -- 172.21.15.19:0/987548586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558add6d64f0 msgr2=0x558add6e8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:34.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.991+0000 7fdc6d67eb80 1 --2- 172.21.15.19:0/987548586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558add6d64f0 0x558add6e8b90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdc44012f30 tx=0x7fdc44012be0 comp rx=0 tx=0).stop 2024-05-07T02:38:34.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.991+0000 7fdc6d67eb80 1 -- 172.21.15.19:0/987548586 shutdown_connections 2024-05-07T02:38:34.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.991+0000 7fdc6d67eb80 1 --2- 172.21.15.19:0/987548586 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x558add764a60 0x558add766e80 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:34.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.991+0000 7fdc6d67eb80 1 --2- 172.21.15.19:0/987548586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558add6dc430 0x558add6e5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:34.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.991+0000 7fdc6d67eb80 1 --2- 172.21.15.19:0/987548586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558add6d64f0 0x558add6e8b90 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:34.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.991+0000 7fdc6d67eb80 1 --2- 172.21.15.19:0/987548586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558add6d6e40 0x558add6e90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:34.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.991+0000 7fdc42000700 0 client.26251 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:38:34.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.991+0000 7fdc6d67eb80 1 -- 172.21.15.19:0/987548586 >> 172.21.15.19:0/987548586 conn(0x558add558ec0 msgr2=0x558add6d5c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:34.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.992+0000 7fdc6d67eb80 1 -- 172.21.15.19:0/987548586 shutdown_connections 2024-05-07T02:38:34.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.992+0000 7fdc6d67eb80 1 -- 172.21.15.19:0/987548586 wait complete. 2024-05-07T02:38:34.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.992+0000 7fdc6d67eb80 20 client.26251 trim_cache size 0 max 16384 2024-05-07T02:38:34.995 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:38:34.995 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-07T02:38:34.995 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "9c143ae1-dcc7-450b-906f-767b92107b58", 2024-05-07T02:38:34.995 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-07T02:38:34.995 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:38:34.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.993+0000 7fdc5ec00700 1 -- 172.21.15.19:0/2770379637 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fdc3c0b29f0 msgr2=0x7fdc3c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:34.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.993+0000 7fdc5ec00700 1 --2- 172.21.15.19:0/2770379637 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fdc3c0b29f0 0x7fdc3c0b4f00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdc50009fb0 tx=0x7fdc50009040 comp rx=0 tx=0).stop 2024-05-07T02:38:34.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.993+0000 7fdc5ec00700 1 -- 172.21.15.19:0/2770379637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc5810c5c0 msgr2=0x7fdc581a1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:34.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.993+0000 7fdc5ec00700 1 --2- 172.21.15.19:0/2770379637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc5810c5c0 0x7fdc581a1910 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000efc0 tx=0x7fdc5000c3a0 comp rx=0 tx=0).stop 2024-05-07T02:38:34.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.993+0000 7fdc5ec00700 1 -- 172.21.15.19:0/2770379637 shutdown_connections 2024-05-07T02:38:34.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.993+0000 7fdc5ec00700 1 --2- 172.21.15.19:0/2770379637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdc5810c5c0 0x7fdc581a1910 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:34.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.993+0000 7fdc5ec00700 1 --2- 172.21.15.19:0/2770379637 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fdc3c0b29f0 0x7fdc3c0b4f00 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:34.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.993+0000 7fdc5ec00700 1 --2- 172.21.15.19:0/2770379637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc58106580 0x7fdc5819d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:34.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.993+0000 7fdc5ec00700 1 --2- 172.21.15.19:0/2770379637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc58106ed0 0x7fdc5819d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:34.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.993+0000 7fdc5ec00700 1 -- 172.21.15.19:0/2770379637 >> 172.21.15.19:0/2770379637 conn(0x7fdc58102080 msgr2=0x7fdc581048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:34.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.993+0000 7fdc5ec00700 1 -- 172.21.15.19:0/2770379637 shutdown_connections 2024-05-07T02:38:34.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:34.993+0000 7fdc5ec00700 1 -- 172.21.15.19:0/2770379637 wait complete. 2024-05-07T02:38:35.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:35 smithi019 ceph-mon[29781]: pgmap v205: 129 pgs: 129 active+clean; 157 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 1.0 MiB/s wr, 116 op/s 2024-05-07T02:38:35.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:35 smithi027 ceph-mon[34398]: pgmap v205: 129 pgs: 129 active+clean; 157 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 1.0 MiB/s wr, 116 op/s 2024-05-07T02:38:35.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:35 smithi064 ceph-mon[34699]: pgmap v205: 129 pgs: 129 active+clean; 157 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 1.0 MiB/s wr, 116 op/s 2024-05-07T02:38:35.764 INFO:tasks.workunit.client.0.smithi019.stdout: 1 229646 3.97 MB/sec execute 211 sec latency 796.487 ms 2024-05-07T02:38:36.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:36 smithi019 ceph-mon[29781]: scrub queued for path: / 2024-05-07T02:38:36.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:36 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:38:36.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:36 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:38:36.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:36 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:38:36.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:36 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:38:36.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:36 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:38:36.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:36 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:38:36.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:36 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:38:36.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:36 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:38:36.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:36 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:38:36.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:36 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:38:36.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:36 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:38:36.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:36 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:38:36.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:36 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:38:36.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:36 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:38:36.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:36 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:38:36.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:36 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:38:36.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:36 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:38:36.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:36 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:38:36.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:36 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:38:36.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:36 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:38:36.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:36 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:38:36.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:36 smithi027 ceph-mon[34398]: scrub queued for path: / 2024-05-07T02:38:36.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:36 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:38:36.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:36 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:38:36.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:36 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:38:36.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:36 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:38:36.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:36 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:38:36.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:36 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:38:36.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:36 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:38:36.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:36 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:38:36.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:36 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:38:36.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:36 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:38:36.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:36 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:38:36.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:36 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:38:36.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:36 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:38:36.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:36 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:38:36.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:36 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:38:36.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:36 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:38:36.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:36 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:38:36.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:36 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:38:36.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:36 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:38:36.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:36 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:38:36.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:36 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:38:36.542 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-07T02:38:36.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:36 smithi064 ceph-mon[34699]: scrub queued for path: / 2024-05-07T02:38:36.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:36 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:38:36.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:36 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:38:36.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:36 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:38:36.610 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:36 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:38:36.610 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:36 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:38:36.610 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:36 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:38:36.610 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:36 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:38:36.610 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:36 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:38:36.610 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:36 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:38:36.610 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:36 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:38:36.611 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:36 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:38:36.611 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:36 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:38:36.611 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:36 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:38:36.611 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:36 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:38:36.611 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:36 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:38:36.611 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:36 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:38:36.611 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:36 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:38:36.611 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:36 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:38:36.612 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:36 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:38:36.612 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:36 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:38:36.612 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:36 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:38:36.765 INFO:tasks.workunit.client.0.smithi019.stdout: 1 230817 4.01 MB/sec execute 212 sec latency 52.170 ms 2024-05-07T02:38:37.049 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:38:37.436 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:37 smithi019 ceph-mon[29781]: pgmap v206: 129 pgs: 129 active+clean; 160 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 176 op/s 2024-05-07T02:38:37.436 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:37 smithi019 ceph-mon[29781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:38:37.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:37 smithi027 ceph-mon[34398]: pgmap v206: 129 pgs: 129 active+clean; 160 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 176 op/s 2024-05-07T02:38:37.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:37 smithi027 ceph-mon[34398]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:38:37.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:37 smithi064 ceph-mon[34699]: pgmap v206: 129 pgs: 129 active+clean; 160 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 176 op/s 2024-05-07T02:38:37.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:37 smithi064 ceph-mon[34699]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:38:37.765 INFO:tasks.workunit.client.0.smithi019.stdout: 1 231290 3.99 MB/sec execute 213 sec latency 22.647 ms 2024-05-07T02:38:38.765 INFO:tasks.workunit.client.0.smithi019.stdout: 1 232163 3.99 MB/sec execute 214 sec latency 48.790 ms 2024-05-07T02:38:39.292 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:39 smithi019 ceph-mon[29781]: pgmap v207: 129 pgs: 129 active+clean; 160 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 986 KiB/s wr, 112 op/s 2024-05-07T02:38:39.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:39 smithi027 ceph-mon[34398]: pgmap v207: 129 pgs: 129 active+clean; 160 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 986 KiB/s wr, 112 op/s 2024-05-07T02:38:39.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.553+0000 7f0d5ee00700 1 -- 172.21.15.19:0/3769887158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d58076680 msgr2=0x7f0d58110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:39.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.553+0000 7f0d5ee00700 1 --2- 172.21.15.19:0/3769887158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d58076680 0x7f0d58110470 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0d44009990 tx=0x7f0d44009ca0 comp rx=0 tx=0).stop 2024-05-07T02:38:39.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.553+0000 7f0d5ee00700 1 -- 172.21.15.19:0/3769887158 shutdown_connections 2024-05-07T02:38:39.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.553+0000 7f0d5ee00700 1 --2- 172.21.15.19:0/3769887158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d581109b0 0x7f0d58112e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:39.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.553+0000 7f0d5ee00700 1 --2- 172.21.15.19:0/3769887158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d58075d30 0x7f0d58076140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:39.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.553+0000 7f0d5ee00700 1 --2- 172.21.15.19:0/3769887158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d58076680 0x7f0d58110470 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:39.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.553+0000 7f0d5ee00700 1 -- 172.21.15.19:0/3769887158 >> 172.21.15.19:0/3769887158 conn(0x7f0d58079d10 msgr2=0x7f0d5807a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:39.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.553+0000 7f0d5ee00700 1 -- 172.21.15.19:0/3769887158 shutdown_connections 2024-05-07T02:38:39.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.554+0000 7f0d5ee00700 1 -- 172.21.15.19:0/3769887158 wait complete. 2024-05-07T02:38:39.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.555+0000 7f0d5ee00700 1 Processor -- start 2024-05-07T02:38:39.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.555+0000 7f0d5ee00700 1 -- start start 2024-05-07T02:38:39.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.555+0000 7f0d5ee00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d58075d30 0x7f0d5819ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:39.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.555+0000 7f0d5ee00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d58076680 0x7f0d5819d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:39.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.555+0000 7f0d5ee00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d581109b0 0x7f0d581a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:39.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.556+0000 7f0d5ee00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0d58116500 con 0x7f0d58075d30 2024-05-07T02:38:39.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.556+0000 7f0d5ee00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0d58116380 con 0x7f0d58076680 2024-05-07T02:38:39.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.556+0000 7f0d5ee00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0d58116680 con 0x7f0d581109b0 2024-05-07T02:38:39.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.556+0000 7f0d5da00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d58075d30 0x7f0d5819ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:39.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.556+0000 7f0d5da00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d58075d30 0x7f0d5819ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49468/0 (socket says 172.21.15.19:49468) 2024-05-07T02:38:39.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.556+0000 7f0d5da00700 1 -- 172.21.15.19:0/3138190328 learned_addr learned my addr 172.21.15.19:0/3138190328 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:38:39.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.556+0000 7f0d5d000700 1 --2- 172.21.15.19:0/3138190328 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d58076680 0x7f0d5819d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:39.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.556+0000 7f0d5e400700 1 --2- 172.21.15.19:0/3138190328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d581109b0 0x7f0d581a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:39.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.556+0000 7f0d5d000700 1 -- 172.21.15.19:0/3138190328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d581109b0 msgr2=0x7f0d581a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:39.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.556+0000 7f0d5d000700 1 --2- 172.21.15.19:0/3138190328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d581109b0 0x7f0d581a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:39.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.556+0000 7f0d5d000700 1 -- 172.21.15.19:0/3138190328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d58075d30 msgr2=0x7f0d5819ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:39.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.556+0000 7f0d5d000700 1 --2- 172.21.15.19:0/3138190328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d58075d30 0x7f0d5819ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:39.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.556+0000 7f0d5d000700 1 -- 172.21.15.19:0/3138190328 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d581a1d80 con 0x7f0d58076680 2024-05-07T02:38:39.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.556+0000 7f0d5da00700 1 --2- 172.21.15.19:0/3138190328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d58075d30 0x7f0d5819ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:38:39.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.557+0000 7f0d5d000700 1 --2- 172.21.15.19:0/3138190328 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d58076680 0x7f0d5819d450 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0d44000c00 tx=0x7f0d440047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:39.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.557+0000 7f0d56000700 1 -- 172.21.15.19:0/3138190328 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d44026070 con 0x7f0d58076680 2024-05-07T02:38:39.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.557+0000 7f0d56000700 1 -- 172.21.15.19:0/3138190328 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d44021410 con 0x7f0d58076680 2024-05-07T02:38:39.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.557+0000 7f0d5ee00700 1 -- 172.21.15.19:0/3138190328 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d581a2010 con 0x7f0d58076680 2024-05-07T02:38:39.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.557+0000 7f0d56000700 1 -- 172.21.15.19:0/3138190328 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d44017650 con 0x7f0d58076680 2024-05-07T02:38:39.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.557+0000 7f0d5ee00700 1 -- 172.21.15.19:0/3138190328 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d581a9990 con 0x7f0d58076680 2024-05-07T02:38:39.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.561+0000 7f0d56000700 1 -- 172.21.15.19:0/3138190328 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0d44029070 con 0x7f0d58076680 2024-05-07T02:38:39.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.562+0000 7f0d6ee46b80 1 Processor -- start 2024-05-07T02:38:39.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.562+0000 7f0d6ee46b80 1 -- start start 2024-05-07T02:38:39.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.563+0000 7f0d6ee46b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f3091a0a90 0x55f3091a0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:39.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.563+0000 7f0d6ee46b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3091a13e0 0x55f3091aa620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:39.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.563+0000 7f0d6ee46b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f3091aab60 0x55f3091acf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:39.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.563+0000 7f0d6ee46b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f3090c2a20 con 0x55f3091a13e0 2024-05-07T02:38:39.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.563+0000 7f0d6ee46b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f309060760 con 0x55f3091a0a90 2024-05-07T02:38:39.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.563+0000 7f0d6ee46b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f3090be9e0 con 0x55f3091aab60 2024-05-07T02:38:39.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.563+0000 7f0d56000700 1 --2- 172.21.15.19:0/3138190328 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f0d400b2a70 0x7f0d400b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:39.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.563+0000 7f0d5da00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3091a13e0 0x55f3091aa620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:39.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.563+0000 7f0d5d000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f3091aab60 0x55f3091acf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:39.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.563+0000 7f0d5e400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f3091a0a90 0x55f3091a0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:39.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.563+0000 7f0d5da00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3091a13e0 0x55f3091aa620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49478/0 (socket says 172.21.15.19:49478) 2024-05-07T02:38:39.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.563+0000 7f0d5da00700 1 -- 172.21.15.19:0/2659868247 learned_addr learned my addr 172.21.15.19:0/2659868247 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:38:39.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.563+0000 7f0d56000700 1 -- 172.21.15.19:0/3138190328 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f0d440e3650 con 0x7f0d58076680 2024-05-07T02:38:39.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.564+0000 7f0d5e400700 1 --2- 172.21.15.19:0/3138190328 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f0d400b2a70 0x7f0d400b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:39.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.564+0000 7f0d5d000700 1 -- 172.21.15.19:0/2659868247 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f3091a0a90 msgr2=0x55f3091a0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:39.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.564+0000 7f0d5d000700 1 --2- 172.21.15.19:0/2659868247 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f3091a0a90 0x55f3091a0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:39.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.564+0000 7f0d5d000700 1 -- 172.21.15.19:0/2659868247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3091a13e0 msgr2=0x55f3091aa620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:39.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.564+0000 7f0d5d000700 1 --2- 172.21.15.19:0/2659868247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3091a13e0 0x55f3091aa620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:39.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.564+0000 7f0d5d000700 1 -- 172.21.15.19:0/2659868247 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f30900f8d0 con 0x55f3091aab60 2024-05-07T02:38:39.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.564+0000 7f0d5d000700 1 --2- 172.21.15.19:0/2659868247 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f3091aab60 0x55f3091acf80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0d440e09b0 tx=0x7f0d44009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=363ee693de2d1db9 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:39.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.564+0000 7f0d4aa00700 1 -- 172.21.15.19:0/2659868247 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d440aa5d0 con 0x55f3091aab60 2024-05-07T02:38:39.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.564+0000 7f0d4aa00700 1 -- 172.21.15.19:0/2659868247 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d440ae980 con 0x55f3091aab60 2024-05-07T02:38:39.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.564+0000 7f0d4aa00700 1 -- 172.21.15.19:0/2659868247 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d44021a50 con 0x55f3091aab60 2024-05-07T02:38:39.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.565+0000 7f0d6ee46b80 1 -- 172.21.15.19:0/2659868247 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f3091aab60 msgr2=0x55f3091acf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:39.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.565+0000 7f0d6ee46b80 1 --2- 172.21.15.19:0/2659868247 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f3091aab60 0x55f3091acf80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0d440e09b0 tx=0x7f0d44009ca0 comp rx=0 tx=0).stop 2024-05-07T02:38:39.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.565+0000 7f0d5e400700 1 --2- 172.21.15.19:0/3138190328 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f0d400b2a70 0x7f0d400b4f80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x55f30919ce40 tx=0x7f0d5000b040 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:39.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.565+0000 7f0d6ee46b80 1 -- 172.21.15.19:0/2659868247 shutdown_connections 2024-05-07T02:38:39.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.565+0000 7f0d6ee46b80 1 --2- 172.21.15.19:0/2659868247 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f3091aab60 0x55f3091acf80 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:39.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.565+0000 7f0d6ee46b80 1 --2- 172.21.15.19:0/2659868247 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f3091a0a90 0x55f3091a0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:39.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.565+0000 7f0d6ee46b80 1 --2- 172.21.15.19:0/2659868247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3091a13e0 0x55f3091aa620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:39.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.565+0000 7f0d6ee46b80 1 -- 172.21.15.19:0/2659868247 >> 172.21.15.19:0/2659868247 conn(0x55f309022e40 msgr2=0x55f30919b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:39.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.565+0000 7f0d6ee46b80 1 -- 172.21.15.19:0/2659868247 shutdown_connections 2024-05-07T02:38:39.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.565+0000 7f0d6ee46b80 1 -- 172.21.15.19:0/2659868247 wait complete. 2024-05-07T02:38:39.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.566+0000 7f0d6ee46b80 1 -- start start 2024-05-07T02:38:39.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.566+0000 7f0d6ee46b80 1 Processor -- start 2024-05-07T02:38:39.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.567+0000 7f0d6ee46b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f3091a0a90 0x55f3091a7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:39.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.567+0000 7f0d6ee46b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f3091a13e0 0x55f3091a8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:39.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.567+0000 7f0d6ee46b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3091aab60 0x55f3091a8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:39.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.567+0000 7f0d6ee46b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f3091ad720 con 0x55f3091aab60 2024-05-07T02:38:39.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.567+0000 7f0d6ee46b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55f309028b20 con 0x55f3091a0a90 2024-05-07T02:38:39.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.567+0000 7f0d6ee46b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f3091ad8a0 con 0x55f3091a13e0 2024-05-07T02:38:39.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.567+0000 7f0d6ee46b80 10 client.0 _ll_register_callbacks cb 0x55f309069c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:38:39.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.567+0000 7f0d5d000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f3091a13e0 0x55f3091a8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:39.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.567+0000 7f0d5da00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f3091a0a90 0x55f3091a7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:39.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.567+0000 7f0d5e400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3091aab60 0x55f3091a8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:39.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.567+0000 7f0d5d000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f3091a13e0 0x55f3091a8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:42348/0 (socket says 172.21.15.19:42348) 2024-05-07T02:38:39.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.567+0000 7f0d5d000700 1 -- 172.21.15.19:0/412958352 learned_addr learned my addr 172.21.15.19:0/412958352 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:38:39.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.568+0000 7f0d5d000700 1 -- 172.21.15.19:0/412958352 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f3091a0a90 msgr2=0x55f3091a7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:39.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.568+0000 7f0d5d000700 1 --2- 172.21.15.19:0/412958352 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f3091a0a90 0x55f3091a7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:39.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.568+0000 7f0d5d000700 1 -- 172.21.15.19:0/412958352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3091aab60 msgr2=0x55f3091a8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:39.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.568+0000 7f0d5d000700 1 --2- 172.21.15.19:0/412958352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3091aab60 0x55f3091a8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:39.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.568+0000 7f0d5d000700 1 -- 172.21.15.19:0/412958352 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f309088420 con 0x55f3091a13e0 2024-05-07T02:38:39.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.568+0000 7f0d5da00700 1 --2- 172.21.15.19:0/412958352 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f3091a0a90 0x55f3091a7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:38:39.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.568+0000 7f0d5d000700 1 --2- 172.21.15.19:0/412958352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f3091a13e0 0x55f3091a8010 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0d440a9eb0 tx=0x7f0d440b3430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70ddb08011684104 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:39.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.568+0000 7f0d3f400700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:38:39.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.568+0000 7f0d3f400700 1 -- 172.21.15.19:0/412958352 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d440c3030 con 0x55f3091a13e0 2024-05-07T02:38:39.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.568+0000 7f0d3f400700 1 -- 172.21.15.19:0/412958352 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f0d440a6bc0 con 0x55f3091a13e0 2024-05-07T02:38:39.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.568+0000 7f0d6ee46b80 1 -- 172.21.15.19:0/412958352 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d440ae980 con 0x55f3091a13e0 2024-05-07T02:38:39.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.569+0000 7f0d3f400700 1 -- 172.21.15.19:0/412958352 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f0d440c4e30 con 0x55f3091a13e0 2024-05-07T02:38:39.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.569+0000 7f0d6ee46b80 10 client.26263 fetch_fsmap learned FSMap version 46 2024-05-07T02:38:39.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.569+0000 7f0d6ee46b80 1 -- 172.21.15.19:0/412958352 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55f3091ad720 con 0x55f3091a13e0 2024-05-07T02:38:39.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.569+0000 7f0d3f400700 1 -- 172.21.15.19:0/412958352 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d440ae980 con 0x55f3091a13e0 2024-05-07T02:38:39.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.569+0000 7f0d3f400700 1 -- 172.21.15.19:0/412958352 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 46) ==== 8386+0+0 (secure 0 0 0) 0x7f0d440aa830 con 0x55f3091a13e0 2024-05-07T02:38:39.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.569+0000 7f0d6ee46b80 10 client.26263 fetch_fsmap finished waiting for FSMap version 46 2024-05-07T02:38:39.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.569+0000 7f0d6ee46b80 10 client.26263 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:38:39.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.569+0000 7f0d6ee46b80 20 client.26263 populate_metadata read hostname 'smithi019' 2024-05-07T02:38:39.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.569+0000 7f0d6ee46b80 1 --2- 172.21.15.19:0/412958352 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55f3091ba520 0x55f3091bc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:39.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.569+0000 7f0d6ee46b80 4 client.26263 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:38:39.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.569+0000 7f0d6ee46b80 1 -- 172.21.15.19:0/412958352 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f30916d2e0 con 0x55f3091ba520 2024-05-07T02:38:39.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.570+0000 7f0d5da00700 1 --2- 172.21.15.19:0/412958352 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55f3091ba520 0x55f3091bc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:39.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.570+0000 7f0d5da00700 1 --2- 172.21.15.19:0/412958352 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55f3091ba520 0x55f3091bc940 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27a8818430fca822 server_cookie=1931db07cb7ec3cf in_seq=0 out_seq=0 2024-05-07T02:38:39.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.570+0000 7f0d3f400700 10 client.26263 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:38:39.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.571+0000 7f0d3f400700 1 -- 172.21.15.19:0/412958352 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f30916d2e0 con 0x55f3091ba520 2024-05-07T02:38:39.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.571+0000 7f0d3f400700 10 client.26263 handle_command_reply: tid=0 2024-05-07T02:38:39.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.571+0000 7f0d6ee46b80 1 client.26263 shutdown 2024-05-07T02:38:39.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.571+0000 7f0d6ee46b80 1 -- 172.21.15.19:0/412958352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f3091a13e0 msgr2=0x55f3091a8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:39.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.571+0000 7f0d6ee46b80 1 --2- 172.21.15.19:0/412958352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f3091a13e0 0x55f3091a8010 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0d440a9eb0 tx=0x7f0d440b3430 comp rx=0 tx=0).stop 2024-05-07T02:38:39.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.571+0000 7f0d6ee46b80 1 -- 172.21.15.19:0/412958352 shutdown_connections 2024-05-07T02:38:39.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.572+0000 7f0d6ee46b80 1 --2- 172.21.15.19:0/412958352 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55f3091ba520 0x55f3091bc940 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:39.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.572+0000 7f0d6ee46b80 1 --2- 172.21.15.19:0/412958352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3091aab60 0x55f3091a8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:39.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.572+0000 7f0d6ee46b80 1 --2- 172.21.15.19:0/412958352 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f3091a0a90 0x55f3091a7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:39.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.572+0000 7f0d6ee46b80 1 --2- 172.21.15.19:0/412958352 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f3091a13e0 0x55f3091a8010 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:39.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.572+0000 7f0d3f400700 0 client.26263 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:38:39.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.572+0000 7f0d6ee46b80 1 -- 172.21.15.19:0/412958352 >> 172.21.15.19:0/412958352 conn(0x55f309022e40 msgr2=0x55f30919d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:39.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.572+0000 7f0d6ee46b80 1 -- 172.21.15.19:0/412958352 shutdown_connections 2024-05-07T02:38:39.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.572+0000 7f0d6ee46b80 1 -- 172.21.15.19:0/412958352 wait complete. 2024-05-07T02:38:39.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.572+0000 7f0d6ee46b80 20 client.26263 trim_cache size 0 max 16384 2024-05-07T02:38:39.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.596+0000 7f0d6ee46b80 1 Processor -- start 2024-05-07T02:38:39.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.596+0000 7f0d6ee46b80 1 -- start start 2024-05-07T02:38:39.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.597+0000 7f0d6ee46b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f3091a0a90 0x55f3091a6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:39.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.597+0000 7f0d6ee46b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f3091a13e0 0x55f3091a9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:39.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.597+0000 7f0d6ee46b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3091aab60 0x55f30919b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:39.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.597+0000 7f0d6ee46b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0d581b25b0 con 0x55f3091aab60 2024-05-07T02:38:39.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.597+0000 7f0d6ee46b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0d581b4210 con 0x55f3091a13e0 2024-05-07T02:38:39.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.597+0000 7f0d6ee46b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0d581b2c80 con 0x55f3091a0a90 2024-05-07T02:38:39.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.597+0000 7f0d5da00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f3091a0a90 0x55f3091a6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:39.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.597+0000 7f0d5d000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f3091a13e0 0x55f3091a9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:39.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.597+0000 7f0d5da00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f3091a0a90 0x55f3091a6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:42362/0 (socket says 172.21.15.19:42362) 2024-05-07T02:38:39.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.597+0000 7f0d5da00700 1 -- 172.21.15.19:0/4085761211 learned_addr learned my addr 172.21.15.19:0/4085761211 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:38:39.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.597+0000 7f0d5e400700 1 --2- 172.21.15.19:0/4085761211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3091aab60 0x55f30919b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:39.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.597+0000 7f0d5d000700 1 -- 172.21.15.19:0/4085761211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f3091a0a90 msgr2=0x55f3091a6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:39.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.597+0000 7f0d5d000700 1 --2- 172.21.15.19:0/4085761211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f3091a0a90 0x55f3091a6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:39.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.598+0000 7f0d5d000700 1 -- 172.21.15.19:0/4085761211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3091aab60 msgr2=0x55f30919b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:39.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.598+0000 7f0d5d000700 1 --2- 172.21.15.19:0/4085761211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3091aab60 0x55f30919b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:39.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.598+0000 7f0d5d000700 1 -- 172.21.15.19:0/4085761211 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f30919fec0 con 0x55f3091a13e0 2024-05-07T02:38:39.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.598+0000 7f0d5da00700 1 --2- 172.21.15.19:0/4085761211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f3091a0a90 0x55f3091a6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:38:39.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.598+0000 7f0d5d000700 1 --2- 172.21.15.19:0/4085761211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f3091a13e0 0x55f3091a9f70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0d440098e0 tx=0x7f0d44009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4eaa6125ecf7de46 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:39.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.598+0000 7f0d4a000700 1 -- 172.21.15.19:0/4085761211 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d440aa0e0 con 0x55f3091a13e0 2024-05-07T02:38:39.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.598+0000 7f0d4a000700 1 -- 172.21.15.19:0/4085761211 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d440ae980 con 0x55f3091a13e0 2024-05-07T02:38:39.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.598+0000 7f0d4a000700 1 -- 172.21.15.19:0/4085761211 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d440218c0 con 0x55f3091a13e0 2024-05-07T02:38:39.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.599+0000 7f0d6ee46b80 1 -- 172.21.15.19:0/4085761211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f3091a13e0 msgr2=0x55f3091a9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:39.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.599+0000 7f0d6ee46b80 1 --2- 172.21.15.19:0/4085761211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f3091a13e0 0x55f3091a9f70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0d440098e0 tx=0x7f0d44009ca0 comp rx=0 tx=0).stop 2024-05-07T02:38:39.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.599+0000 7f0d6ee46b80 1 -- 172.21.15.19:0/4085761211 shutdown_connections 2024-05-07T02:38:39.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.599+0000 7f0d6ee46b80 1 --2- 172.21.15.19:0/4085761211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3091aab60 0x55f30919b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:39.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.599+0000 7f0d6ee46b80 1 --2- 172.21.15.19:0/4085761211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f3091a0a90 0x55f3091a6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:39.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.599+0000 7f0d6ee46b80 1 --2- 172.21.15.19:0/4085761211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f3091a13e0 0x55f3091a9f70 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:39.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.599+0000 7f0d6ee46b80 1 -- 172.21.15.19:0/4085761211 >> 172.21.15.19:0/4085761211 conn(0x55f309022e40 msgr2=0x55f3091a2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:39.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.599+0000 7f0d6ee46b80 1 -- 172.21.15.19:0/4085761211 shutdown_connections 2024-05-07T02:38:39.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.599+0000 7f0d6ee46b80 1 -- 172.21.15.19:0/4085761211 wait complete. 2024-05-07T02:38:39.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.600+0000 7f0d6ee46b80 1 -- start start 2024-05-07T02:38:39.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.600+0000 7f0d6ee46b80 1 Processor -- start 2024-05-07T02:38:39.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.601+0000 7f0d6ee46b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f3091a0a90 0x55f3091a4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:39.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.601+0000 7f0d6ee46b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f3091a13e0 0x55f3091a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:39.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.601+0000 7f0d6ee46b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3091aab60 0x55f3091a5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:39.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.601+0000 7f0d6ee46b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0d581b3a80 con 0x55f3091aab60 2024-05-07T02:38:39.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.601+0000 7f0d6ee46b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0d581b3350 con 0x55f3091a13e0 2024-05-07T02:38:39.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.601+0000 7f0d6ee46b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0d581b0550 con 0x55f3091a0a90 2024-05-07T02:38:39.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.601+0000 7f0d6ee46b80 10 client.0 _ll_register_callbacks cb 0x55f309069c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:38:39.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.601+0000 7f0d5da00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f3091a0a90 0x55f3091a4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:39.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.601+0000 7f0d5da00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f3091a0a90 0x55f3091a4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:42378/0 (socket says 172.21.15.19:42378) 2024-05-07T02:38:39.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.601+0000 7f0d5da00700 1 -- 172.21.15.19:0/3968831365 learned_addr learned my addr 172.21.15.19:0/3968831365 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:38:39.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.601+0000 7f0d5d000700 1 --2- 172.21.15.19:0/3968831365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f3091a13e0 0x55f3091a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:39.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.601+0000 7f0d5e400700 1 --2- 172.21.15.19:0/3968831365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3091aab60 0x55f3091a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:39.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.602+0000 7f0d5da00700 1 -- 172.21.15.19:0/3968831365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f3091a13e0 msgr2=0x55f3091a4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:39.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.602+0000 7f0d5da00700 1 --2- 172.21.15.19:0/3968831365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f3091a13e0 0x55f3091a4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:39.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.602+0000 7f0d5da00700 1 -- 172.21.15.19:0/3968831365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3091aab60 msgr2=0x55f3091a5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:38:39.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.602+0000 7f0d5da00700 1 --2- 172.21.15.19:0/3968831365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3091aab60 0x55f3091a5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:39.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.602+0000 7f0d5da00700 1 -- 172.21.15.19:0/3968831365 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f309223bd0 con 0x55f3091a0a90 2024-05-07T02:38:39.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.602+0000 7f0d5da00700 1 --2- 172.21.15.19:0/3968831365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f3091a0a90 0x55f3091a4980 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c009880 tx=0x7f0d4c0206e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb10e05855b183de server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:38:39.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.602+0000 7f0d3f400700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:38:39.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.602+0000 7f0d3f400700 1 -- 172.21.15.19:0/3968831365 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c020d40 con 0x55f3091a0a90 2024-05-07T02:38:39.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.602+0000 7f0d3f400700 1 -- 172.21.15.19:0/3968831365 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9437+0+0 (secure 0 0 0) 0x7f0d4c004830 con 0x55f3091a0a90 2024-05-07T02:38:39.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.602+0000 7f0d6ee46b80 1 -- 172.21.15.19:0/3968831365 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d440ae980 con 0x55f3091a0a90 2024-05-07T02:38:39.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.603+0000 7f0d3f400700 1 -- 172.21.15.19:0/3968831365 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f0d4c033050 con 0x55f3091a0a90 2024-05-07T02:38:39.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.603+0000 7f0d6ee46b80 10 client.26269 fetch_fsmap learned FSMap version 46 2024-05-07T02:38:39.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.603+0000 7f0d6ee46b80 1 -- 172.21.15.19:0/3968831365 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55f309224e50 con 0x55f3091a0a90 2024-05-07T02:38:39.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.603+0000 7f0d3f400700 1 -- 172.21.15.19:0/3968831365 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d440ae980 con 0x55f3091a0a90 2024-05-07T02:38:39.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.604+0000 7f0d3f400700 1 -- 172.21.15.19:0/3968831365 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 46) ==== 8386+0+0 (secure 0 0 0) 0x7f0d4c010430 con 0x55f3091a0a90 2024-05-07T02:38:39.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.604+0000 7f0d6ee46b80 10 client.26269 fetch_fsmap finished waiting for FSMap version 46 2024-05-07T02:38:39.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.604+0000 7f0d6ee46b80 10 client.26269 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:38:39.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.604+0000 7f0d6ee46b80 20 client.26269 populate_metadata read hostname 'smithi019' 2024-05-07T02:38:39.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.604+0000 7f0d6ee46b80 1 --2- 172.21.15.19:0/3968831365 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55f309225650 0x55f309227a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:38:39.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.604+0000 7f0d6ee46b80 4 client.26269 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-07T02:38:39.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.604+0000 7f0d6ee46b80 1 -- 172.21.15.19:0/3968831365 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f3090d0400 con 0x55f309225650 2024-05-07T02:38:39.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.604+0000 7f0d5d000700 1 --2- 172.21.15.19:0/3968831365 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55f309225650 0x55f309227a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:38:39.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.604+0000 7f0d5d000700 1 --2- 172.21.15.19:0/3968831365 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55f309225650 0x55f309227a70 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=840172ee663d55be server_cookie=f0896bc059e07003 in_seq=0 out_seq=0 2024-05-07T02:38:39.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.604+0000 7f0d3f400700 10 client.26269 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:38:39.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.605+0000 7f0d3f400700 1 -- 172.21.15.19:0/3968831365 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f3090d0400 con 0x55f309225650 2024-05-07T02:38:39.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.605+0000 7f0d3f400700 10 client.26269 handle_command_reply: tid=0 2024-05-07T02:38:39.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.605+0000 7f0d6ee46b80 1 client.26269 shutdown 2024-05-07T02:38:39.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.605+0000 7f0d6ee46b80 1 -- 172.21.15.19:0/3968831365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f3091a0a90 msgr2=0x55f3091a4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:39.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.605+0000 7f0d6ee46b80 1 --2- 172.21.15.19:0/3968831365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f3091a0a90 0x55f3091a4980 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c009880 tx=0x7f0d4c0206e0 comp rx=0 tx=0).stop 2024-05-07T02:38:39.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.605+0000 7f0d6ee46b80 1 -- 172.21.15.19:0/3968831365 shutdown_connections 2024-05-07T02:38:39.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.605+0000 7f0d6ee46b80 1 --2- 172.21.15.19:0/3968831365 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55f309225650 0x55f309227a70 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:39.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.605+0000 7f0d6ee46b80 1 --2- 172.21.15.19:0/3968831365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3091aab60 0x55f3091a5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:39.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.605+0000 7f0d6ee46b80 1 --2- 172.21.15.19:0/3968831365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f3091a0a90 0x55f3091a4980 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:39.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.605+0000 7f0d6ee46b80 1 --2- 172.21.15.19:0/3968831365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55f3091a13e0 0x55f3091a4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:39.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.605+0000 7f0d6ee46b80 1 -- 172.21.15.19:0/3968831365 >> 172.21.15.19:0/3968831365 conn(0x55f309022e40 msgr2=0x55f3091a2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:39.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.605+0000 7f0d3f400700 0 client.26269 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:38:39.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.605+0000 7f0d6ee46b80 1 -- 172.21.15.19:0/3968831365 shutdown_connections 2024-05-07T02:38:39.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.605+0000 7f0d6ee46b80 1 -- 172.21.15.19:0/3968831365 wait complete. 2024-05-07T02:38:39.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.605+0000 7f0d6ee46b80 20 client.26269 trim_cache size 0 max 16384 2024-05-07T02:38:39.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:39 smithi064 ceph-mon[34699]: pgmap v207: 129 pgs: 129 active+clean; 160 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 986 KiB/s wr, 112 op/s 2024-05-07T02:38:39.609 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:38:39.609 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-07T02:38:39.609 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-05-07T02:38:39.610 INFO:teuthology.orchestra.run.smithi019.stdout: "9c143ae1-dcc7-450b-906f-767b92107b58": { 2024-05-07T02:38:39.610 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-05-07T02:38:39.610 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "9c143ae1-dcc7-450b-906f-767b92107b58", 2024-05-07T02:38:39.610 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-05-07T02:38:39.610 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:38:39.610 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:38:39.610 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:38:39.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.606+0000 7f0d5ee00700 1 -- 172.21.15.19:0/3138190328 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f0d400b2a70 msgr2=0x7f0d400b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:39.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.606+0000 7f0d5ee00700 1 --2- 172.21.15.19:0/3138190328 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f0d400b2a70 0x7f0d400b4f80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x55f30919ce40 tx=0x7f0d5000b040 comp rx=0 tx=0).stop 2024-05-07T02:38:39.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.606+0000 7f0d5ee00700 1 -- 172.21.15.19:0/3138190328 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d58076680 msgr2=0x7f0d5819d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:38:39.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.606+0000 7f0d5ee00700 1 --2- 172.21.15.19:0/3138190328 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d58076680 0x7f0d5819d450 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0d44000c00 tx=0x7f0d440047b0 comp rx=0 tx=0).stop 2024-05-07T02:38:39.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.606+0000 7f0d5ee00700 1 -- 172.21.15.19:0/3138190328 shutdown_connections 2024-05-07T02:38:39.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.606+0000 7f0d5ee00700 1 --2- 172.21.15.19:0/3138190328 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d581109b0 0x7f0d581a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:39.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.606+0000 7f0d5ee00700 1 --2- 172.21.15.19:0/3138190328 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f0d400b2a70 0x7f0d400b4f80 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:39.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.606+0000 7f0d5ee00700 1 --2- 172.21.15.19:0/3138190328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d58075d30 0x7f0d5819ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:39.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.606+0000 7f0d5ee00700 1 --2- 172.21.15.19:0/3138190328 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d58076680 0x7f0d5819d450 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:38:39.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.606+0000 7f0d5ee00700 1 -- 172.21.15.19:0/3138190328 >> 172.21.15.19:0/3138190328 conn(0x7f0d58079d10 msgr2=0x7f0d58100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:38:39.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.607+0000 7f0d5ee00700 1 -- 172.21.15.19:0/3138190328 shutdown_connections 2024-05-07T02:38:39.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:38:39.607+0000 7f0d5ee00700 1 -- 172.21.15.19:0/3138190328 wait complete. 2024-05-07T02:38:39.765 INFO:tasks.workunit.client.0.smithi019.stdout: 1 232296 3.98 MB/sec execute 215 sec latency 926.556 ms 2024-05-07T02:38:40.765 INFO:tasks.workunit.client.0.smithi019.stdout: 1 232696 3.96 MB/sec execute 216 sec latency 1657.217 ms 2024-05-07T02:38:41.205 INFO:tasks.cephfs.filesystem:scrub status for tag:9c143ae1-dcc7-450b-906f-767b92107b58 - {'path': '/', 'tag': '9c143ae1-dcc7-450b-906f-767b92107b58', 'options': 'recursive,force'} 2024-05-07T02:38:41.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:41 smithi027 ceph-mon[34398]: pgmap v208: 129 pgs: 129 active+clean; 160 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 986 KiB/s wr, 112 op/s 2024-05-07T02:38:41.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:41 smithi064 ceph-mon[34699]: pgmap v208: 129 pgs: 129 active+clean; 160 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 986 KiB/s wr, 112 op/s 2024-05-07T02:38:41.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:41 smithi019 ceph-mon[29781]: pgmap v208: 129 pgs: 129 active+clean; 160 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 986 KiB/s wr, 112 op/s 2024-05-07T02:38:41.765 INFO:tasks.workunit.client.0.smithi019.stdout: 1 233963 3.98 MB/sec execute 217 sec latency 20.706 ms 2024-05-07T02:38:42.765 INFO:tasks.workunit.client.0.smithi019.stdout: 1 234538 3.98 MB/sec execute 218 sec latency 32.814 ms 2024-05-07T02:38:43.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:43 smithi027 ceph-mon[34398]: pgmap v209: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 1.4 MiB/s wr, 161 op/s 2024-05-07T02:38:43.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:43 smithi064 ceph-mon[34699]: pgmap v209: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 1.4 MiB/s wr, 161 op/s 2024-05-07T02:38:43.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:43 smithi019 ceph-mon[29781]: pgmap v209: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 1.4 MiB/s wr, 161 op/s 2024-05-07T02:38:43.765 INFO:tasks.workunit.client.0.smithi019.stdout: 1 234950 3.97 MB/sec execute 219 sec latency 21.864 ms 2024-05-07T02:38:44.765 INFO:tasks.workunit.client.0.smithi019.stdout: 1 235839 3.97 MB/sec execute 220 sec latency 229.436 ms 2024-05-07T02:38:45.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:45 smithi027 ceph-mon[34398]: pgmap v210: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 1.0 MiB/s wr, 109 op/s 2024-05-07T02:38:45.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:45 smithi064 ceph-mon[34699]: pgmap v210: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 1.0 MiB/s wr, 109 op/s 2024-05-07T02:38:45.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:45 smithi019 ceph-mon[29781]: pgmap v210: 129 pgs: 129 active+clean; 163 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 1.0 MiB/s wr, 109 op/s 2024-05-07T02:38:45.765 INFO:tasks.workunit.client.0.smithi019.stdout: 1 236238 3.96 MB/sec execute 221 sec latency 957.264 ms 2024-05-07T02:38:46.766 INFO:tasks.workunit.client.0.smithi019.stdout: 1 237510 3.98 MB/sec execute 222 sec latency 24.079 ms 2024-05-07T02:38:47.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:47 smithi027 ceph-mon[34398]: pgmap v211: 129 pgs: 129 active+clean; 165 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-05-07T02:38:47.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:47 smithi064 ceph-mon[34699]: pgmap v211: 129 pgs: 129 active+clean; 165 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-05-07T02:38:47.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:47 smithi019 ceph-mon[29781]: pgmap v211: 129 pgs: 129 active+clean; 165 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-05-07T02:38:47.766 INFO:tasks.workunit.client.0.smithi019.stdout: 1 238081 3.98 MB/sec execute 223 sec latency 24.807 ms 2024-05-07T02:38:48.766 INFO:tasks.workunit.client.0.smithi019.stdout: 1 238571 3.97 MB/sec execute 224 sec latency 25.287 ms 2024-05-07T02:38:49.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:49 smithi027 ceph-mon[34398]: pgmap v212: 129 pgs: 129 active+clean; 165 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 836 KiB/s wr, 105 op/s 2024-05-07T02:38:49.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:49 smithi027 ceph-mon[34398]: scrub summary: idle 2024-05-07T02:38:49.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:49 smithi027 ceph-mon[34398]: scrub summary: idle 2024-05-07T02:38:49.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:49 smithi027 ceph-mon[34398]: scrub summary: idle 2024-05-07T02:38:49.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:49 smithi064 ceph-mon[34699]: pgmap v212: 129 pgs: 129 active+clean; 165 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 836 KiB/s wr, 105 op/s 2024-05-07T02:38:49.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:49 smithi064 ceph-mon[34699]: scrub summary: idle 2024-05-07T02:38:49.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:49 smithi064 ceph-mon[34699]: scrub summary: idle 2024-05-07T02:38:49.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:49 smithi064 ceph-mon[34699]: scrub summary: idle 2024-05-07T02:38:49.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:49 smithi019 ceph-mon[29781]: pgmap v212: 129 pgs: 129 active+clean; 165 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 836 KiB/s wr, 105 op/s 2024-05-07T02:38:49.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:49 smithi019 ceph-mon[29781]: scrub summary: idle 2024-05-07T02:38:49.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:49 smithi019 ceph-mon[29781]: scrub summary: idle 2024-05-07T02:38:49.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:49 smithi019 ceph-mon[29781]: scrub summary: idle 2024-05-07T02:38:49.766 INFO:tasks.workunit.client.0.smithi019.stdout: 1 239388 3.97 MB/sec execute 225 sec latency 240.664 ms 2024-05-07T02:38:50.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:50 smithi027 ceph-mon[34398]: scrub summary: idle 2024-05-07T02:38:50.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:50 smithi027 ceph-mon[34398]: scrub summary: idle 2024-05-07T02:38:50.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:50 smithi064 ceph-mon[34699]: scrub summary: idle 2024-05-07T02:38:50.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:50 smithi064 ceph-mon[34699]: scrub summary: idle 2024-05-07T02:38:50.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:50 smithi019 ceph-mon[29781]: scrub summary: idle 2024-05-07T02:38:50.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:50 smithi019 ceph-mon[29781]: scrub summary: idle 2024-05-07T02:38:50.766 INFO:tasks.workunit.client.0.smithi019.stdout: 1 239837 3.95 MB/sec execute 226 sec latency 966.637 ms 2024-05-07T02:38:51.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:51 smithi027 ceph-mon[34398]: pgmap v213: 129 pgs: 129 active+clean; 165 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 836 KiB/s wr, 105 op/s 2024-05-07T02:38:51.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:51 smithi064 ceph-mon[34699]: pgmap v213: 129 pgs: 129 active+clean; 165 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 836 KiB/s wr, 105 op/s 2024-05-07T02:38:51.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:51 smithi019 ceph-mon[29781]: pgmap v213: 129 pgs: 129 active+clean; 165 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 836 KiB/s wr, 105 op/s 2024-05-07T02:38:51.766 INFO:tasks.workunit.client.0.smithi019.stdout: 1 241058 3.97 MB/sec execute 227 sec latency 18.984 ms 2024-05-07T02:38:52.767 INFO:tasks.workunit.client.0.smithi019.stdout: 1 241628 3.98 MB/sec execute 228 sec latency 27.261 ms 2024-05-07T02:38:53.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:53 smithi027 ceph-mon[34398]: pgmap v214: 129 pgs: 129 active+clean; 168 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 1.4 MiB/s wr, 164 op/s 2024-05-07T02:38:53.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:53 smithi064 ceph-mon[34699]: pgmap v214: 129 pgs: 129 active+clean; 168 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 1.4 MiB/s wr, 164 op/s 2024-05-07T02:38:53.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:53 smithi019 ceph-mon[29781]: pgmap v214: 129 pgs: 129 active+clean; 168 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 878 KiB/s rd, 1.4 MiB/s wr, 164 op/s 2024-05-07T02:38:53.767 INFO:tasks.workunit.client.0.smithi019.stdout: 1 242152 3.96 MB/sec execute 229 sec latency 21.893 ms 2024-05-07T02:38:54.769 INFO:tasks.workunit.client.0.smithi019.stdout: 1 242931 3.97 MB/sec execute 230 sec latency 268.325 ms 2024-05-07T02:38:55.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:55 smithi064 ceph-mon[34699]: pgmap v215: 129 pgs: 129 active+clean; 168 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 985 KiB/s wr, 115 op/s 2024-05-07T02:38:55.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:55 smithi019 ceph-mon[29781]: pgmap v215: 129 pgs: 129 active+clean; 168 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 985 KiB/s wr, 115 op/s 2024-05-07T02:38:55.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:55 smithi027 ceph-mon[34398]: pgmap v215: 129 pgs: 129 active+clean; 168 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 985 KiB/s wr, 115 op/s 2024-05-07T02:38:55.767 INFO:tasks.workunit.client.0.smithi019.stdout: 1 243381 3.95 MB/sec execute 231 sec latency 996.665 ms 2024-05-07T02:38:56.767 INFO:tasks.workunit.client.0.smithi019.stdout: 1 244597 3.97 MB/sec execute 232 sec latency 20.152 ms 2024-05-07T02:38:57.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:57 smithi064 ceph-mon[34699]: pgmap v216: 129 pgs: 129 active+clean; 170 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 1.5 MiB/s wr, 173 op/s 2024-05-07T02:38:57.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:57 smithi019 ceph-mon[29781]: pgmap v216: 129 pgs: 129 active+clean; 170 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 1.5 MiB/s wr, 173 op/s 2024-05-07T02:38:57.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:57 smithi027 ceph-mon[34398]: pgmap v216: 129 pgs: 129 active+clean; 170 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 1.5 MiB/s wr, 173 op/s 2024-05-07T02:38:57.767 INFO:tasks.workunit.client.0.smithi019.stdout: 1 245166 3.97 MB/sec execute 233 sec latency 31.788 ms 2024-05-07T02:38:58.767 INFO:tasks.workunit.client.0.smithi019.stdout: 1 245670 3.96 MB/sec execute 234 sec latency 22.578 ms 2024-05-07T02:38:59.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:38:59 smithi064 ceph-mon[34699]: pgmap v217: 129 pgs: 129 active+clean; 170 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 1.1 MiB/s wr, 116 op/s 2024-05-07T02:38:59.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:38:59 smithi019 ceph-mon[29781]: pgmap v217: 129 pgs: 129 active+clean; 170 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 1.1 MiB/s wr, 116 op/s 2024-05-07T02:38:59.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:38:59 smithi027 ceph-mon[34398]: pgmap v217: 129 pgs: 129 active+clean; 170 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 1.1 MiB/s wr, 116 op/s 2024-05-07T02:38:59.767 INFO:tasks.workunit.client.0.smithi019.stdout: 1 246478 3.96 MB/sec execute 235 sec latency 238.398 ms 2024-05-07T02:39:00.768 INFO:tasks.workunit.client.0.smithi019.stdout: 1 246764 3.95 MB/sec execute 236 sec latency 963.184 ms 2024-05-07T02:39:01.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:01 smithi064 ceph-mon[34699]: pgmap v218: 129 pgs: 129 active+clean; 170 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 1.1 MiB/s wr, 116 op/s 2024-05-07T02:39:01.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:01 smithi019 ceph-mon[29781]: pgmap v218: 129 pgs: 129 active+clean; 170 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 1.1 MiB/s wr, 116 op/s 2024-05-07T02:39:01.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:01 smithi027 ceph-mon[34398]: pgmap v218: 129 pgs: 129 active+clean; 170 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 1.1 MiB/s wr, 116 op/s 2024-05-07T02:39:01.853 INFO:tasks.workunit.client.0.smithi019.stdout: 1 246764 3.93 MB/sec execute 237 sec latency 1083.704 ms 2024-05-07T02:39:02.768 INFO:tasks.workunit.client.0.smithi019.stdout: 1 246875 3.91 MB/sec execute 238 sec latency 1980.549 ms 2024-05-07T02:39:03.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:03 smithi064 ceph-mon[34699]: pgmap v219: 129 pgs: 129 active+clean; 172 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 882 KiB/s rd, 1.6 MiB/s wr, 171 op/s 2024-05-07T02:39:03.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:03 smithi019 ceph-mon[29781]: pgmap v219: 129 pgs: 129 active+clean; 172 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 882 KiB/s rd, 1.6 MiB/s wr, 171 op/s 2024-05-07T02:39:03.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:03 smithi027 ceph-mon[34398]: pgmap v219: 129 pgs: 129 active+clean; 172 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 882 KiB/s rd, 1.6 MiB/s wr, 171 op/s 2024-05-07T02:39:03.768 INFO:tasks.workunit.client.0.smithi019.stdout: 1 247075 3.90 MB/sec execute 239 sec latency 100.412 ms 2024-05-07T02:39:04.768 INFO:tasks.workunit.client.0.smithi019.stdout: 1 247800 3.91 MB/sec execute 240 sec latency 97.949 ms 2024-05-07T02:39:05.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:05 smithi064 ceph-mon[34699]: pgmap v220: 129 pgs: 129 active+clean; 172 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 1.0 MiB/s wr, 112 op/s 2024-05-07T02:39:05.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:05 smithi064 ceph-mon[34699]: osdmap e91: 12 total, 12 up, 12 in 2024-05-07T02:39:05.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:05 smithi019 ceph-mon[29781]: pgmap v220: 129 pgs: 129 active+clean; 172 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 1.0 MiB/s wr, 112 op/s 2024-05-07T02:39:05.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:05 smithi019 ceph-mon[29781]: osdmap e91: 12 total, 12 up, 12 in 2024-05-07T02:39:05.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:05 smithi027 ceph-mon[34398]: pgmap v220: 129 pgs: 129 active+clean; 172 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 1.0 MiB/s wr, 112 op/s 2024-05-07T02:39:05.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:05 smithi027 ceph-mon[34398]: osdmap e91: 12 total, 12 up, 12 in 2024-05-07T02:39:05.768 INFO:tasks.workunit.client.0.smithi019.stdout: 1 248641 3.92 MB/sec execute 241 sec latency 25.916 ms 2024-05-07T02:39:06.768 INFO:tasks.workunit.client.0.smithi019.stdout: 1 249011 3.91 MB/sec execute 242 sec latency 28.982 ms 2024-05-07T02:39:07.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:07 smithi064 ceph-mon[34699]: pgmap v222: 129 pgs: 129 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 1.3 MiB/s wr, 158 op/s 2024-05-07T02:39:07.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:07 smithi019 ceph-mon[29781]: pgmap v222: 129 pgs: 129 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 1.3 MiB/s wr, 158 op/s 2024-05-07T02:39:07.647 INFO:tasks.workunit.client.0.smithi019.stdout:[249854] write failed on handle 14006 (Resource temporarily unavailable) 2024-05-07T02:39:07.647 INFO:tasks.workunit.client.0.smithi019.stdout:Child failed with status 1 2024-05-07T02:39:07.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-07T02:39:07.651 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-05-07T02:39:07.651 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-07T02:39:07.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:07 smithi027 ceph-mon[34398]: pgmap v222: 129 pgs: 129 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 1.3 MiB/s wr, 158 op/s 2024-05-07T02:39:08.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:08 smithi064 ceph-mon[34699]: osdmap e92: 12 total, 12 up, 12 in 2024-05-07T02:39:08.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:08 smithi019 ceph-mon[29781]: osdmap e92: 12 total, 12 up, 12 in 2024-05-07T02:39:08.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:08 smithi027 ceph-mon[34398]: osdmap e92: 12 total, 12 up, 12 in 2024-05-07T02:39:08.841 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_batrick_ceph_366c9130c497dee9e73178650cf710ed21269b18/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_batrick_ceph_366c9130c497dee9e73178650cf710ed21269b18/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_batrick_ceph_366c9130c497dee9e73178650cf710ed21269b18/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi019 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=366c9130c497dee9e73178650cf710ed21269b18 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-05-07T02:39:09.076 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=c25603c46a8b4141989ee4e8e030368c Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_batrick_ceph_366c9130c497dee9e73178650cf710ed21269b18/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_batrick_ceph_366c9130c497dee9e73178650cf710ed21269b18/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_batrick_ceph_366c9130c497dee9e73178650cf710ed21269b18/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi019 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=366c9130c497dee9e73178650cf710ed21269b18 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-05-07T02:39:09.079 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-05-07T02:39:09.089 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-05-07T02:39:09.089 INFO:teuthology.task.exec:Executing custom commands... 2024-05-07T02:39:09.089 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-05-07T02:39:09.089 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-05-07T02:39:09.175 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-05-07T02:39:09.608 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:09 smithi027 ceph-mon[34398]: pgmap v224: 129 pgs: 129 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 1.7 MiB/s wr, 197 op/s 2024-05-07T02:39:09.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:09 smithi064 ceph-mon[34699]: pgmap v224: 129 pgs: 129 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 1.7 MiB/s wr, 197 op/s 2024-05-07T02:39:09.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:09 smithi019 ceph-mon[29781]: pgmap v224: 129 pgs: 129 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 1.7 MiB/s wr, 197 op/s 2024-05-07T02:39:11.205 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-07T02:39:11.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:11 smithi064 ceph-mon[34699]: pgmap v225: 129 pgs: 129 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 937 KiB/s wr, 115 op/s 2024-05-07T02:39:11.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:11 smithi019 ceph-mon[29781]: pgmap v225: 129 pgs: 129 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 937 KiB/s wr, 115 op/s 2024-05-07T02:39:11.684 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:39:11.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:11 smithi027 ceph-mon[34398]: pgmap v225: 129 pgs: 129 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 937 KiB/s wr, 115 op/s 2024-05-07T02:39:13.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:13 smithi064 ceph-mon[34699]: pgmap v226: 129 pgs: 129 active+clean; 178 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 1.5 MiB/s wr, 165 op/s 2024-05-07T02:39:13.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:13 smithi019 ceph-mon[29781]: pgmap v226: 129 pgs: 129 active+clean; 178 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 1.5 MiB/s wr, 165 op/s 2024-05-07T02:39:13.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:13 smithi027 ceph-mon[34398]: pgmap v226: 129 pgs: 129 active+clean; 178 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 1.5 MiB/s wr, 165 op/s 2024-05-07T02:39:14.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.406+0000 7fb7a7000700 1 -- 172.21.15.19:0/3907675779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7a0107c10 msgr2=0x7fb7a0108020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:14.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.406+0000 7fb7a7000700 1 --2- 172.21.15.19:0/3907675779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7a0107c10 0x7fb7a0108020 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb78c009920 tx=0x7fb78c009c30 comp rx=0 tx=0).stop 2024-05-07T02:39:14.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.406+0000 7fb7a7000700 1 -- 172.21.15.19:0/3907675779 shutdown_connections 2024-05-07T02:39:14.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.406+0000 7fb7a7000700 1 --2- 172.21.15.19:0/3907675779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7a0108560 0x7fb7a010fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:14.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.406+0000 7fb7a7000700 1 --2- 172.21.15.19:0/3907675779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7a01032e0 0x7fb7a01036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:14.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.406+0000 7fb7a7000700 1 --2- 172.21.15.19:0/3907675779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7a0107c10 0x7fb7a0108020 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:14.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.406+0000 7fb7a7000700 1 -- 172.21.15.19:0/3907675779 >> 172.21.15.19:0/3907675779 conn(0x7fb7a0078ac0 msgr2=0x7fb7a0078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:14.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.406+0000 7fb7a7000700 1 -- 172.21.15.19:0/3907675779 shutdown_connections 2024-05-07T02:39:14.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.406+0000 7fb7a7000700 1 -- 172.21.15.19:0/3907675779 wait complete. 2024-05-07T02:39:14.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.407+0000 7fb7a7000700 1 Processor -- start 2024-05-07T02:39:14.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.407+0000 7fb7a7000700 1 -- start start 2024-05-07T02:39:14.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.408+0000 7fb7a7000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7a01032e0 0x7fb7a019cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:14.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.408+0000 7fb7a7000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7a0107c10 0x7fb7a019d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:14.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.408+0000 7fb7a7000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7a0108560 0x7fb7a01a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:14.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.408+0000 7fb7a7000700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a01131c0 con 0x7fb7a0107c10 2024-05-07T02:39:14.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.408+0000 7fb7a7000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a0113040 con 0x7fb7a01032e0 2024-05-07T02:39:14.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.408+0000 7fb7a7000700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a0113340 con 0x7fb7a0108560 2024-05-07T02:39:14.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.408+0000 7fb7a5c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7a01032e0 0x7fb7a019cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:14.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.408+0000 7fb7a5c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7a01032e0 0x7fb7a019cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:46738/0 (socket says 172.21.15.19:46738) 2024-05-07T02:39:14.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.408+0000 7fb7a5c00700 1 -- 172.21.15.19:0/1580086689 learned_addr learned my addr 172.21.15.19:0/1580086689 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:39:14.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.408+0000 7fb7a5200700 1 --2- 172.21.15.19:0/1580086689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7a0107c10 0x7fb7a019d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:14.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.409+0000 7fb7a6600700 1 --2- 172.21.15.19:0/1580086689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7a0108560 0x7fb7a01a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:14.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.409+0000 7fb7a5c00700 1 -- 172.21.15.19:0/1580086689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7a0108560 msgr2=0x7fb7a01a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:14.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.409+0000 7fb7a5c00700 1 --2- 172.21.15.19:0/1580086689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7a0108560 0x7fb7a01a17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:14.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.409+0000 7fb7a5c00700 1 -- 172.21.15.19:0/1580086689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7a0107c10 msgr2=0x7fb7a019d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:14.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.409+0000 7fb7a5c00700 1 --2- 172.21.15.19:0/1580086689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7a0107c10 0x7fb7a019d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:14.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.409+0000 7fb7a5c00700 1 -- 172.21.15.19:0/1580086689 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7a01a1e50 con 0x7fb7a01032e0 2024-05-07T02:39:14.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.409+0000 7fb7a5c00700 1 --2- 172.21.15.19:0/1580086689 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7a01032e0 0x7fb7a019cf80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb79400e9c0 tx=0x7fb79400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:14.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.409+0000 7fb79e000700 1 -- 172.21.15.19:0/1580086689 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb794004a30 con 0x7fb7a01032e0 2024-05-07T02:39:14.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.410+0000 7fb7a7000700 1 -- 172.21.15.19:0/1580086689 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7a01a2140 con 0x7fb7a01032e0 2024-05-07T02:39:14.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.410+0000 7fb79e000700 1 -- 172.21.15.19:0/1580086689 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb79400cc70 con 0x7fb7a01032e0 2024-05-07T02:39:14.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.410+0000 7fb7a7000700 1 -- 172.21.15.19:0/1580086689 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7a01a9a60 con 0x7fb7a01032e0 2024-05-07T02:39:14.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.410+0000 7fb79e000700 1 -- 172.21.15.19:0/1580086689 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7940054a0 con 0x7fb7a01032e0 2024-05-07T02:39:14.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.414+0000 7fb79e000700 1 -- 172.21.15.19:0/1580086689 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb794021440 con 0x7fb7a01032e0 2024-05-07T02:39:14.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.415+0000 7fb7b7227b80 1 Processor -- start 2024-05-07T02:39:14.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.415+0000 7fb7b7227b80 1 -- start start 2024-05-07T02:39:14.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.415+0000 7fb79e000700 1 --2- 172.21.15.19:0/1580086689 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fb7880b2a70 0x7fb7880b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:14.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.416+0000 7fb79e000700 1 -- 172.21.15.19:0/1580086689 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9525+0+0 (secure 0 0 0) 0x7fb7940e2a90 con 0x7fb7a01032e0 2024-05-07T02:39:14.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.416+0000 7fb7a6600700 1 --2- 172.21.15.19:0/1580086689 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fb7880b2a70 0x7fb7880b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:14.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.416+0000 7fb7b7227b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bd59a2a90 0x555bd59a2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:14.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.416+0000 7fb7b7227b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bd59a33e0 0x555bd59ac620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:14.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.416+0000 7fb7b7227b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555bd59acb60 0x555bd59aef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:14.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.416+0000 7fb7b7227b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555bd58c4a20 con 0x555bd59acb60 2024-05-07T02:39:14.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.416+0000 7fb7b7227b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x555bd5862760 con 0x555bd59a33e0 2024-05-07T02:39:14.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.416+0000 7fb7b7227b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555bd58c09e0 con 0x555bd59a2a90 2024-05-07T02:39:14.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.416+0000 7fb7a5c00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bd59a2a90 0x555bd59a2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:14.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.416+0000 7fb7a6600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555bd59acb60 0x555bd59aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:14.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.416+0000 7fb7a5c00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bd59a2a90 0x555bd59a2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:39044/0 (socket says 172.21.15.19:39044) 2024-05-07T02:39:14.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.416+0000 7fb7a5c00700 1 -- 172.21.15.19:0/2881948156 learned_addr learned my addr 172.21.15.19:0/2881948156 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:39:14.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.416+0000 7fb7a5200700 1 --2- 172.21.15.19:0/2881948156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bd59a33e0 0x555bd59ac620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:14.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.417+0000 7fb7a5c00700 1 -- 172.21.15.19:0/2881948156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bd59a33e0 msgr2=0x555bd59ac620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:14.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.417+0000 7fb7a5c00700 1 --2- 172.21.15.19:0/2881948156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bd59a33e0 0x555bd59ac620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:14.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.417+0000 7fb7a5c00700 1 -- 172.21.15.19:0/2881948156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555bd59acb60 msgr2=0x555bd59aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:14.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.417+0000 7fb7a5c00700 1 --2- 172.21.15.19:0/2881948156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555bd59acb60 0x555bd59aef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:14.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.417+0000 7fb7a5c00700 1 -- 172.21.15.19:0/2881948156 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bd58118d0 con 0x555bd59a2a90 2024-05-07T02:39:14.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.417+0000 7fb7a5c00700 1 --2- 172.21.15.19:0/2881948156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bd59a2a90 0x555bd59a2ea0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb7940a59a0 tx=0x7fb7940a5130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13ae60edec0224a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:14.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.417+0000 7fb792a00700 1 -- 172.21.15.19:0/2881948156 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7940c3070 con 0x555bd59a2a90 2024-05-07T02:39:14.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.418+0000 7fb7a6600700 1 --2- 172.21.15.19:0/1580086689 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fb7880b2a70 0x7fb7880b4f80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb798013de0 tx=0x7fb798013460 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:14.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.418+0000 7fb792a00700 1 -- 172.21.15.19:0/2881948156 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb79400ce20 con 0x555bd59a2a90 2024-05-07T02:39:14.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.418+0000 7fb792a00700 1 -- 172.21.15.19:0/2881948156 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7940cc0d0 con 0x555bd59a2a90 2024-05-07T02:39:14.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.419+0000 7fb7b7227b80 1 -- 172.21.15.19:0/2881948156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bd59a2a90 msgr2=0x555bd59a2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:14.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.419+0000 7fb7b7227b80 1 --2- 172.21.15.19:0/2881948156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bd59a2a90 0x555bd59a2ea0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb7940a59a0 tx=0x7fb7940a5130 comp rx=0 tx=0).stop 2024-05-07T02:39:14.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.419+0000 7fb7b7227b80 1 -- 172.21.15.19:0/2881948156 shutdown_connections 2024-05-07T02:39:14.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.419+0000 7fb7b7227b80 1 --2- 172.21.15.19:0/2881948156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555bd59acb60 0x555bd59aef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:14.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.419+0000 7fb7b7227b80 1 --2- 172.21.15.19:0/2881948156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bd59a2a90 0x555bd59a2ea0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:14.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.419+0000 7fb7b7227b80 1 --2- 172.21.15.19:0/2881948156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bd59a33e0 0x555bd59ac620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:14.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.419+0000 7fb7b7227b80 1 -- 172.21.15.19:0/2881948156 >> 172.21.15.19:0/2881948156 conn(0x555bd5824e40 msgr2=0x555bd599d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:14.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.419+0000 7fb7b7227b80 1 -- 172.21.15.19:0/2881948156 shutdown_connections 2024-05-07T02:39:14.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.419+0000 7fb7b7227b80 1 -- 172.21.15.19:0/2881948156 wait complete. 2024-05-07T02:39:14.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.420+0000 7fb7b7227b80 1 -- start start 2024-05-07T02:39:14.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.420+0000 7fb7b7227b80 1 Processor -- start 2024-05-07T02:39:14.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.421+0000 7fb7b7227b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bd59a2a90 0x555bd59a9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:14.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.421+0000 7fb7b7227b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bd59a33e0 0x555bd59aa060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:14.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.421+0000 7fb7b7227b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555bd59acb60 0x555bd59aa5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:14.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.421+0000 7fb7b7227b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555bd59af960 con 0x555bd59acb60 2024-05-07T02:39:14.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.421+0000 7fb7b7227b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x555bd582ab20 con 0x555bd59a2a90 2024-05-07T02:39:14.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.421+0000 7fb7b7227b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555bd59afae0 con 0x555bd59a33e0 2024-05-07T02:39:14.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.421+0000 7fb7b7227b80 10 client.0 _ll_register_callbacks cb 0x555bd586bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:39:14.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.421+0000 7fb7a5c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bd59a2a90 0x555bd59a9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:14.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.421+0000 7fb7a5200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bd59a33e0 0x555bd59aa060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:14.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.421+0000 7fb7a5c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bd59a2a90 0x555bd59a9b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:46762/0 (socket says 172.21.15.19:46762) 2024-05-07T02:39:14.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.421+0000 7fb7a5c00700 1 -- 172.21.15.19:0/4024549138 learned_addr learned my addr 172.21.15.19:0/4024549138 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:39:14.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.421+0000 7fb7a6600700 1 --2- 172.21.15.19:0/4024549138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555bd59acb60 0x555bd59aa5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:14.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.422+0000 7fb7a5200700 1 -- 172.21.15.19:0/4024549138 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bd59a2a90 msgr2=0x555bd59a9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:14.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.422+0000 7fb7a5200700 1 --2- 172.21.15.19:0/4024549138 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bd59a2a90 0x555bd59a9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:14.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.422+0000 7fb7a5200700 1 -- 172.21.15.19:0/4024549138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555bd59acb60 msgr2=0x555bd59aa5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:14.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.422+0000 7fb7a5200700 1 --2- 172.21.15.19:0/4024549138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555bd59acb60 0x555bd59aa5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:14.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.422+0000 7fb7a5200700 1 -- 172.21.15.19:0/4024549138 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bd588a420 con 0x555bd59a33e0 2024-05-07T02:39:14.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.422+0000 7fb7a5c00700 1 --2- 172.21.15.19:0/4024549138 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bd59a2a90 0x555bd59a9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:39:14.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.422+0000 7fb7a5200700 1 --2- 172.21.15.19:0/4024549138 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bd59a33e0 0x555bd59aa060 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb78c000c00 tx=0x7fb78c00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a1f49e3efdd6121 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:14.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.422+0000 7fb787400700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:39:14.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.423+0000 7fb787400700 1 -- 172.21.15.19:0/4024549138 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78c005240 con 0x555bd59a33e0 2024-05-07T02:39:14.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.423+0000 7fb787400700 1 -- 172.21.15.19:0/4024549138 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9525+0+0 (secure 0 0 0) 0x7fb78c0053e0 con 0x555bd59a33e0 2024-05-07T02:39:14.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.423+0000 7fb7b7227b80 1 -- 172.21.15.19:0/4024549138 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb79400ce20 con 0x555bd59a33e0 2024-05-07T02:39:14.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.423+0000 7fb787400700 1 -- 172.21.15.19:0/4024549138 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fb78c031050 con 0x555bd59a33e0 2024-05-07T02:39:14.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.423+0000 7fb7b7227b80 10 client.26281 fetch_fsmap learned FSMap version 46 2024-05-07T02:39:14.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.423+0000 7fb7b7227b80 1 -- 172.21.15.19:0/4024549138 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x555bd59ba5b0 con 0x555bd59a33e0 2024-05-07T02:39:14.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.423+0000 7fb787400700 1 -- 172.21.15.19:0/4024549138 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb79400ce20 con 0x555bd59a33e0 2024-05-07T02:39:14.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.424+0000 7fb787400700 1 -- 172.21.15.19:0/4024549138 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 46) ==== 8386+0+0 (secure 0 0 0) 0x7fb78c00b560 con 0x555bd59a33e0 2024-05-07T02:39:14.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.424+0000 7fb7b7227b80 10 client.26281 fetch_fsmap finished waiting for FSMap version 46 2024-05-07T02:39:14.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.424+0000 7fb7b7227b80 10 client.26281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:39:14.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.424+0000 7fb7b7227b80 20 client.26281 populate_metadata read hostname 'smithi019' 2024-05-07T02:39:14.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.424+0000 7fb7b7227b80 1 --2- 172.21.15.19:0/4024549138 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x555bd59bccc0 0x555bd59bf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:14.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.424+0000 7fb7b7227b80 4 client.26281 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:39:14.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.424+0000 7fb7b7227b80 1 -- 172.21.15.19:0/4024549138 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555bd596f2e0 con 0x555bd59bccc0 2024-05-07T02:39:14.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.424+0000 7fb7a5c00700 1 --2- 172.21.15.19:0/4024549138 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x555bd59bccc0 0x555bd59bf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:14.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.425+0000 7fb787400700 10 client.26281 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:39:14.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.425+0000 7fb7a5c00700 1 --2- 172.21.15.19:0/4024549138 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x555bd59bccc0 0x555bd59bf110 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a216fc389dcaba6 server_cookie=f92f9cece514753f in_seq=0 out_seq=0 2024-05-07T02:39:14.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.426+0000 7fb787400700 1 -- 172.21.15.19:0/4024549138 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555bd596f2e0 con 0x555bd59bccc0 2024-05-07T02:39:14.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.426+0000 7fb787400700 10 client.26281 handle_command_reply: tid=0 2024-05-07T02:39:14.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.426+0000 7fb7b7227b80 1 client.26281 shutdown 2024-05-07T02:39:14.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.426+0000 7fb7b7227b80 1 -- 172.21.15.19:0/4024549138 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bd59a33e0 msgr2=0x555bd59aa060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:14.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.426+0000 7fb7b7227b80 1 --2- 172.21.15.19:0/4024549138 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bd59a33e0 0x555bd59aa060 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb78c000c00 tx=0x7fb78c00f690 comp rx=0 tx=0).stop 2024-05-07T02:39:14.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.426+0000 7fb7b7227b80 1 -- 172.21.15.19:0/4024549138 shutdown_connections 2024-05-07T02:39:14.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.426+0000 7fb7b7227b80 1 --2- 172.21.15.19:0/4024549138 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x555bd59bccc0 0x555bd59bf110 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:14.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.426+0000 7fb7b7227b80 1 --2- 172.21.15.19:0/4024549138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555bd59acb60 0x555bd59aa5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:14.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.426+0000 7fb7b7227b80 1 --2- 172.21.15.19:0/4024549138 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bd59a2a90 0x555bd59a9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:14.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.426+0000 7fb7b7227b80 1 --2- 172.21.15.19:0/4024549138 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bd59a33e0 0x555bd59aa060 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:14.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.426+0000 7fb787400700 0 client.26281 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:39:14.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.426+0000 7fb7b7227b80 1 -- 172.21.15.19:0/4024549138 >> 172.21.15.19:0/4024549138 conn(0x555bd5824e40 msgr2=0x555bd599f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:14.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.426+0000 7fb7b7227b80 1 -- 172.21.15.19:0/4024549138 shutdown_connections 2024-05-07T02:39:14.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.426+0000 7fb7b7227b80 1 -- 172.21.15.19:0/4024549138 wait complete. 2024-05-07T02:39:14.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.427+0000 7fb7b7227b80 20 client.26281 trim_cache size 0 max 16384 2024-05-07T02:39:14.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.450+0000 7fb7b7227b80 1 Processor -- start 2024-05-07T02:39:14.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.450+0000 7fb7b7227b80 1 -- start start 2024-05-07T02:39:14.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.450+0000 7fb7b7227b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bd5824e40 0x555bd59b8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:14.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.450+0000 7fb7b7227b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555bd59a2a90 0x555bd59c48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:14.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.450+0000 7fb7b7227b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bd59a33e0 0x555bd59c4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:14.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.450+0000 7fb7b7227b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb78000ee50 con 0x555bd59a2a90 2024-05-07T02:39:14.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.450+0000 7fb7b7227b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb780010ab0 con 0x555bd59a33e0 2024-05-07T02:39:14.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.450+0000 7fb7b7227b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb78000f520 con 0x555bd5824e40 2024-05-07T02:39:14.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.451+0000 7fb7a5c00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bd5824e40 0x555bd59b8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:14.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.451+0000 7fb7a5200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555bd59a2a90 0x555bd59c48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:14.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.451+0000 7fb7a5c00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bd5824e40 0x555bd59b8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:39070/0 (socket says 172.21.15.19:39070) 2024-05-07T02:39:14.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.451+0000 7fb7a5200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555bd59a2a90 0x555bd59c48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59244/0 (socket says 172.21.15.19:59244) 2024-05-07T02:39:14.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.451+0000 7fb7a5c00700 1 -- 172.21.15.19:0/3809965437 learned_addr learned my addr 172.21.15.19:0/3809965437 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:39:14.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.451+0000 7fb7a6600700 1 --2- 172.21.15.19:0/3809965437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bd59a33e0 0x555bd59c4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:14.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.451+0000 7fb7a5c00700 1 -- 172.21.15.19:0/3809965437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bd59a33e0 msgr2=0x555bd59c4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:14.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.451+0000 7fb7a5c00700 1 --2- 172.21.15.19:0/3809965437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bd59a33e0 0x555bd59c4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:14.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.451+0000 7fb7a5c00700 1 -- 172.21.15.19:0/3809965437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555bd59a2a90 msgr2=0x555bd59c48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:14.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.451+0000 7fb7a5c00700 1 --2- 172.21.15.19:0/3809965437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555bd59a2a90 0x555bd59c48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:14.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.451+0000 7fb7a5c00700 1 -- 172.21.15.19:0/3809965437 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bd59ac3c0 con 0x555bd5824e40 2024-05-07T02:39:14.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.452+0000 7fb7a5c00700 1 --2- 172.21.15.19:0/3809965437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bd5824e40 0x555bd59b8a20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb7940095c0 tx=0x7fb7940a5040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c29ca02267521c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:14.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.452+0000 7fb792000700 1 -- 172.21.15.19:0/3809965437 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7940c3040 con 0x555bd5824e40 2024-05-07T02:39:14.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.452+0000 7fb792000700 1 -- 172.21.15.19:0/3809965437 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7940a9920 con 0x555bd5824e40 2024-05-07T02:39:14.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.452+0000 7fb792000700 1 -- 172.21.15.19:0/3809965437 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7940a9de0 con 0x555bd5824e40 2024-05-07T02:39:14.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.453+0000 7fb7b7227b80 1 -- 172.21.15.19:0/3809965437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bd5824e40 msgr2=0x555bd59b8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:14.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.453+0000 7fb7b7227b80 1 --2- 172.21.15.19:0/3809965437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bd5824e40 0x555bd59b8a20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb7940095c0 tx=0x7fb7940a5040 comp rx=0 tx=0).stop 2024-05-07T02:39:14.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.453+0000 7fb7b7227b80 1 -- 172.21.15.19:0/3809965437 shutdown_connections 2024-05-07T02:39:14.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.453+0000 7fb7b7227b80 1 --2- 172.21.15.19:0/3809965437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bd59a33e0 0x555bd59c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:14.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.453+0000 7fb7b7227b80 1 --2- 172.21.15.19:0/3809965437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bd5824e40 0x555bd59b8a20 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:14.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.453+0000 7fb7b7227b80 1 --2- 172.21.15.19:0/3809965437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555bd59a2a90 0x555bd59c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:14.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.453+0000 7fb7b7227b80 1 -- 172.21.15.19:0/3809965437 >> 172.21.15.19:0/3809965437 conn(0x555bd59af250 msgr2=0x555bd59aae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:14.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.453+0000 7fb7b7227b80 1 -- 172.21.15.19:0/3809965437 shutdown_connections 2024-05-07T02:39:14.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.453+0000 7fb7b7227b80 1 -- 172.21.15.19:0/3809965437 wait complete. 2024-05-07T02:39:14.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.454+0000 7fb7b7227b80 1 -- start start 2024-05-07T02:39:14.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.454+0000 7fb7b7227b80 1 Processor -- start 2024-05-07T02:39:14.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.455+0000 7fb7b7227b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bd5824e40 0x555bd59b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:14.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.455+0000 7fb7b7227b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555bd59a2a90 0x555bd59b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:14.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.455+0000 7fb7b7227b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bd59a33e0 0x555bd59b14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:14.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.455+0000 7fb7b7227b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb780010320 con 0x555bd59a2a90 2024-05-07T02:39:14.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.455+0000 7fb7b7227b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fb78000fbf0 con 0x555bd59a33e0 2024-05-07T02:39:14.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.455+0000 7fb7b7227b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb78000ce10 con 0x555bd5824e40 2024-05-07T02:39:14.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.455+0000 7fb7b7227b80 10 client.0 _ll_register_callbacks cb 0x555bd586bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:39:14.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.455+0000 7fb7a5c00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bd5824e40 0x555bd59b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:14.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.455+0000 7fb7a5200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555bd59a2a90 0x555bd59b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:14.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.455+0000 7fb7a6600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bd59a33e0 0x555bd59b14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:14.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.455+0000 7fb7a5c00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bd5824e40 0x555bd59b0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:39074/0 (socket says 172.21.15.19:39074) 2024-05-07T02:39:14.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.455+0000 7fb7a5c00700 1 -- 172.21.15.19:0/664666818 learned_addr learned my addr 172.21.15.19:0/664666818 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:39:14.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.455+0000 7fb7a5c00700 1 -- 172.21.15.19:0/664666818 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bd59a33e0 msgr2=0x555bd59b14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:14.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.455+0000 7fb7a5c00700 1 --2- 172.21.15.19:0/664666818 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bd59a33e0 0x555bd59b14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:14.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.455+0000 7fb7a5c00700 1 -- 172.21.15.19:0/664666818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555bd59a2a90 msgr2=0x555bd59b0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:14.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.455+0000 7fb7a5c00700 1 --2- 172.21.15.19:0/664666818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555bd59a2a90 0x555bd59b0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:14.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.455+0000 7fb7a5c00700 1 -- 172.21.15.19:0/664666818 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bd5a264e0 con 0x555bd5824e40 2024-05-07T02:39:14.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.456+0000 7fb7a5c00700 1 --2- 172.21.15.19:0/664666818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bd5824e40 0x555bd59b0a50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb7940a9620 tx=0x7fb7940a9820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8779785651dd716 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:14.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.456+0000 7fb787400700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:39:14.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.456+0000 7fb787400700 1 -- 172.21.15.19:0/664666818 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb794088070 con 0x555bd5824e40 2024-05-07T02:39:14.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.456+0000 7fb787400700 1 -- 172.21.15.19:0/664666818 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9525+0+0 (secure 0 0 0) 0x7fb7940a9b80 con 0x555bd5824e40 2024-05-07T02:39:14.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.456+0000 7fb7b7227b80 1 -- 172.21.15.19:0/664666818 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7940a9920 con 0x555bd5824e40 2024-05-07T02:39:14.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.456+0000 7fb787400700 1 -- 172.21.15.19:0/664666818 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fb7940b7050 con 0x555bd5824e40 2024-05-07T02:39:14.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.456+0000 7fb7b7227b80 10 client.26287 fetch_fsmap learned FSMap version 46 2024-05-07T02:39:14.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.456+0000 7fb7b7227b80 1 -- 172.21.15.19:0/664666818 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x555bd5a27750 con 0x555bd5824e40 2024-05-07T02:39:14.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.457+0000 7fb787400700 1 -- 172.21.15.19:0/664666818 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7940a9920 con 0x555bd5824e40 2024-05-07T02:39:14.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.457+0000 7fb787400700 1 -- 172.21.15.19:0/664666818 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 46) ==== 8386+0+0 (secure 0 0 0) 0x7fb7940c7040 con 0x555bd5824e40 2024-05-07T02:39:14.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.457+0000 7fb7b7227b80 10 client.26287 fetch_fsmap finished waiting for FSMap version 46 2024-05-07T02:39:14.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.457+0000 7fb7b7227b80 10 client.26287 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:39:14.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.457+0000 7fb7b7227b80 20 client.26287 populate_metadata read hostname 'smithi019' 2024-05-07T02:39:14.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.457+0000 7fb7b7227b80 1 --2- 172.21.15.19:0/664666818 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x555bd59acb60 0x555bd5a299c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:14.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.457+0000 7fb7b7227b80 4 client.26287 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-07T02:39:14.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.457+0000 7fb7b7227b80 1 -- 172.21.15.19:0/664666818 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555bd58d2400 con 0x555bd59acb60 2024-05-07T02:39:14.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.457+0000 7fb7a5200700 1 --2- 172.21.15.19:0/664666818 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x555bd59acb60 0x555bd5a299c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:14.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.458+0000 7fb7a5200700 1 --2- 172.21.15.19:0/664666818 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x555bd59acb60 0x555bd5a299c0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae53c81fc3a2b79c server_cookie=9471bfd21e0c4c83 in_seq=0 out_seq=0 2024-05-07T02:39:14.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.458+0000 7fb787400700 10 client.26287 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:39:14.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.458+0000 7fb787400700 1 -- 172.21.15.19:0/664666818 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555bd58d2400 con 0x555bd59acb60 2024-05-07T02:39:14.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.458+0000 7fb787400700 10 client.26287 handle_command_reply: tid=0 2024-05-07T02:39:14.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.458+0000 7fb7b7227b80 1 client.26287 shutdown 2024-05-07T02:39:14.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.458+0000 7fb7b7227b80 1 -- 172.21.15.19:0/664666818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bd5824e40 msgr2=0x555bd59b0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:14.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.458+0000 7fb7b7227b80 1 --2- 172.21.15.19:0/664666818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bd5824e40 0x555bd59b0a50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb7940a9620 tx=0x7fb7940a9820 comp rx=0 tx=0).stop 2024-05-07T02:39:14.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.458+0000 7fb7b7227b80 1 -- 172.21.15.19:0/664666818 shutdown_connections 2024-05-07T02:39:14.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.458+0000 7fb7b7227b80 1 --2- 172.21.15.19:0/664666818 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x555bd59acb60 0x555bd5a299c0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:14.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.458+0000 7fb7b7227b80 1 --2- 172.21.15.19:0/664666818 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555bd59a33e0 0x555bd59b14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:14.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.459+0000 7fb7b7227b80 1 --2- 172.21.15.19:0/664666818 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bd5824e40 0x555bd59b0a50 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:14.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.459+0000 7fb7b7227b80 1 --2- 172.21.15.19:0/664666818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555bd59a2a90 0x555bd59b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:14.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.459+0000 7fb7b7227b80 1 -- 172.21.15.19:0/664666818 >> 172.21.15.19:0/664666818 conn(0x555bd59af250 msgr2=0x555bd59a4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:14.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.459+0000 7fb787400700 0 client.26287 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:39:14.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.459+0000 7fb7b7227b80 1 -- 172.21.15.19:0/664666818 shutdown_connections 2024-05-07T02:39:14.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.459+0000 7fb7b7227b80 1 -- 172.21.15.19:0/664666818 wait complete. 2024-05-07T02:39:14.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.459+0000 7fb7b7227b80 20 client.26287 trim_cache size 0 max 16384 2024-05-07T02:39:14.466 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:39:14.467 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-07T02:39:14.467 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-07T02:39:14.467 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:39:14.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.460+0000 7fb7a7000700 1 -- 172.21.15.19:0/1580086689 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fb7880b2a70 msgr2=0x7fb7880b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:14.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.460+0000 7fb7a7000700 1 --2- 172.21.15.19:0/1580086689 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fb7880b2a70 0x7fb7880b4f80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb798013de0 tx=0x7fb798013460 comp rx=0 tx=0).stop 2024-05-07T02:39:14.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.460+0000 7fb7a7000700 1 -- 172.21.15.19:0/1580086689 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7a01032e0 msgr2=0x7fb7a019cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:14.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.460+0000 7fb7a7000700 1 --2- 172.21.15.19:0/1580086689 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7a01032e0 0x7fb7a019cf80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb79400e9c0 tx=0x7fb79400ecd0 comp rx=0 tx=0).stop 2024-05-07T02:39:14.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.460+0000 7fb7a7000700 1 -- 172.21.15.19:0/1580086689 shutdown_connections 2024-05-07T02:39:14.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.460+0000 7fb7a7000700 1 --2- 172.21.15.19:0/1580086689 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7a0108560 0x7fb7a01a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:14.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.460+0000 7fb7a7000700 1 --2- 172.21.15.19:0/1580086689 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7a01032e0 0x7fb7a019cf80 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:14.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.460+0000 7fb7a7000700 1 --2- 172.21.15.19:0/1580086689 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fb7880b2a70 0x7fb7880b4f80 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:14.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.460+0000 7fb7a7000700 1 --2- 172.21.15.19:0/1580086689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7a0107c10 0x7fb7a019d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:14.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.460+0000 7fb7a7000700 1 -- 172.21.15.19:0/1580086689 >> 172.21.15.19:0/1580086689 conn(0x7fb7a0078ac0 msgr2=0x7fb7a010eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:14.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.460+0000 7fb7a7000700 1 -- 172.21.15.19:0/1580086689 shutdown_connections 2024-05-07T02:39:14.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:14.460+0000 7fb7a7000700 1 -- 172.21.15.19:0/1580086689 wait complete. 2024-05-07T02:39:15.347 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-07T02:39:15.348 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-07T02:39:15.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:15 smithi064 ceph-mon[34699]: pgmap v227: 129 pgs: 129 active+clean; 178 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-05-07T02:39:15.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:15 smithi019 ceph-mon[29781]: pgmap v227: 129 pgs: 129 active+clean; 178 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-05-07T02:39:15.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:15 smithi027 ceph-mon[34398]: pgmap v227: 129 pgs: 129 active+clean; 178 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 1.2 MiB/s wr, 134 op/s 2024-05-07T02:39:15.795 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:39:17.585 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:17 smithi019 ceph-mon[29781]: pgmap v228: 129 pgs: 129 active+clean; 177 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 911 KiB/s wr, 63 op/s 2024-05-07T02:39:17.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:17 smithi064 ceph-mon[34699]: pgmap v228: 129 pgs: 129 active+clean; 177 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 911 KiB/s wr, 63 op/s 2024-05-07T02:39:17.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:17 smithi027 ceph-mon[34398]: pgmap v228: 129 pgs: 129 active+clean; 177 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 911 KiB/s wr, 63 op/s 2024-05-07T02:39:18.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.280+0000 7faeb8400700 1 -- 172.21.15.19:0/3350785621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faeb0109870 msgr2=0x7faeb01111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:18.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.280+0000 7faeb8400700 1 --2- 172.21.15.19:0/3350785621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faeb0109870 0x7faeb01111d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7faea80099c0 tx=0x7faea8009cd0 comp rx=0 tx=0).stop 2024-05-07T02:39:18.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.280+0000 7faeb8400700 1 -- 172.21.15.19:0/3350785621 shutdown_connections 2024-05-07T02:39:18.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.280+0000 7faeb8400700 1 --2- 172.21.15.19:0/3350785621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faeb0109870 0x7faeb01111d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:18.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.280+0000 7faeb8400700 1 --2- 172.21.15.19:0/3350785621 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faeb01045f0 0x7faeb0104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:18.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.280+0000 7faeb8400700 1 --2- 172.21.15.19:0/3350785621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faeb0108f20 0x7faeb0109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:18.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.280+0000 7faeb8400700 1 -- 172.21.15.19:0/3350785621 >> 172.21.15.19:0/3350785621 conn(0x7faeb0100040 msgr2=0x7faeb0102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:18.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.280+0000 7faeb8400700 1 -- 172.21.15.19:0/3350785621 shutdown_connections 2024-05-07T02:39:18.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.280+0000 7faeb8400700 1 -- 172.21.15.19:0/3350785621 wait complete. 2024-05-07T02:39:18.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.281+0000 7faeb8400700 1 Processor -- start 2024-05-07T02:39:18.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.282+0000 7faeb8400700 1 -- start start 2024-05-07T02:39:18.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.282+0000 7faeb8400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faeb01045f0 0x7faeb0075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:18.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.282+0000 7faeb8400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faeb0108f20 0x7faeb0076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:18.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.282+0000 7faeb8400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faeb0109870 0x7faeb0076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:18.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.282+0000 7faeb8400700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faeb01144a0 con 0x7faeb0108f20 2024-05-07T02:39:18.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.282+0000 7faeb8400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7faeb0114320 con 0x7faeb01045f0 2024-05-07T02:39:18.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.282+0000 7faeb8400700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7faeb0114620 con 0x7faeb0109870 2024-05-07T02:39:18.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.282+0000 7faeb5a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faeb01045f0 0x7faeb0075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:18.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.282+0000 7faeb5000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faeb0108f20 0x7faeb0076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:18.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.282+0000 7faeb5a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faeb01045f0 0x7faeb0075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:46794/0 (socket says 172.21.15.19:46794) 2024-05-07T02:39:18.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.282+0000 7faeb5a00700 1 -- 172.21.15.19:0/3211290206 learned_addr learned my addr 172.21.15.19:0/3211290206 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:39:18.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.283+0000 7faeb6400700 1 --2- 172.21.15.19:0/3211290206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faeb0109870 0x7faeb0076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:18.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.283+0000 7faeb5a00700 1 -- 172.21.15.19:0/3211290206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faeb0109870 msgr2=0x7faeb0076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:18.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.283+0000 7faeb5a00700 1 --2- 172.21.15.19:0/3211290206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faeb0109870 0x7faeb0076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:18.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.283+0000 7faeb5a00700 1 -- 172.21.15.19:0/3211290206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faeb0108f20 msgr2=0x7faeb0076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:18.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.283+0000 7faeb5a00700 1 --2- 172.21.15.19:0/3211290206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faeb0108f20 0x7faeb0076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:18.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.283+0000 7faeb5a00700 1 -- 172.21.15.19:0/3211290206 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeb01a91b0 con 0x7faeb01045f0 2024-05-07T02:39:18.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.283+0000 7faeb5a00700 1 --2- 172.21.15.19:0/3211290206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faeb01045f0 0x7faeb0075ac0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7faea400ef10 tx=0x7faea400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:18.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.283+0000 7faeae000700 1 -- 172.21.15.19:0/3211290206 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea400c9c0 con 0x7faeb01045f0 2024-05-07T02:39:18.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.284+0000 7faeae000700 1 -- 172.21.15.19:0/3211290206 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faea4019070 con 0x7faeb01045f0 2024-05-07T02:39:18.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.284+0000 7faeae000700 1 -- 172.21.15.19:0/3211290206 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea4014480 con 0x7faeb01045f0 2024-05-07T02:39:18.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.284+0000 7faeb8400700 1 -- 172.21.15.19:0/3211290206 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faeb01a9440 con 0x7faeb01045f0 2024-05-07T02:39:18.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.284+0000 7faeb8400700 1 -- 172.21.15.19:0/3211290206 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faeb01a9980 con 0x7faeb01045f0 2024-05-07T02:39:18.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.285+0000 7faeb8400700 1 -- 172.21.15.19:0/3211290206 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faeb00512c0 con 0x7faeb01045f0 2024-05-07T02:39:18.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.288+0000 7faeae000700 1 -- 172.21.15.19:0/3211290206 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7faea4012050 con 0x7faeb01045f0 2024-05-07T02:39:18.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.289+0000 7faeae000700 1 --2- 172.21.15.19:0/3211290206 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fae980b29f0 0x7fae980b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:18.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.289+0000 7faeae000700 1 -- 172.21.15.19:0/3211290206 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9525+0+0 (secure 0 0 0) 0x7faea40e3060 con 0x7faeb01045f0 2024-05-07T02:39:18.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.289+0000 7faeb5000700 1 --2- 172.21.15.19:0/3211290206 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fae980b29f0 0x7fae980b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:18.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.290+0000 7faeb5000700 1 --2- 172.21.15.19:0/3211290206 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fae980b29f0 0x7fae980b4f00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7faeb01077f0 tx=0x7fae9c009260 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:18.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.292+0000 7faeae000700 1 -- 172.21.15.19:0/3211290206 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faea4010040 con 0x7faeb01045f0 2024-05-07T02:39:18.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.580+0000 7faeb8400700 1 -- 172.21.15.19:0/3211290206 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faeb002e7b0 con 0x7faeb01045f0 2024-05-07T02:39:18.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.581+0000 7faeae000700 1 -- 172.21.15.19:0/3211290206 <== mon.2 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11934 (secure 0 0 0) 0x7faea40aa5f0 con 0x7faeb01045f0 2024-05-07T02:39:18.583 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:39:18.583 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1763670446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1763670446},{"type":"v1","addr":"172.21.15.19:6839","nonce":1763670446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/477529985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":477529985},{"type":"v1","addr":"172.21.15.64:6833","nonce":477529985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-07T02:26:14.960465+0000","modified":"2024-05-07T02:38:35.528409+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24539,"mds_2":24520,"mds_3":24515,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6833/2171873162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2171873162},{"type":"v1","addr":"172.21.15.27:6833","nonce":2171873162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/2379917839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2379917839},{"type":"v1","addr":"172.21.15.19:6835","nonce":2379917839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6837/2067515538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2067515538},{"type":"v1","addr":"172.21.15.19:6837","nonce":2067515538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6835/1000540600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1000540600},{"type":"v1","addr":"172.21.15.27:6835","nonce":1000540600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/1997318872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1997318872},{"type":"v1","addr":"172.21.15.64:6835","nonce":1997318872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":34,"addr":"172.21.15.64:6837/2585907777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2585907777},{"type":"v1","addr":"172.21.15.64:6837","nonce":2585907777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":33,"addr":"172.21.15.27:6837/2583138143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2583138143},{"type":"v1","addr":"172.21.15.27:6837","nonce":2583138143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":32,"addr":"172.21.15.64:6839/1583120004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":1583120004},{"type":"v1","addr":"172.21.15.64:6839","nonce":1583120004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":29,"addr":"172.21.15.19:6841/3905645246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3905645246},{"type":"v1","addr":"172.21.15.19:6841","nonce":3905645246}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6839/3963268634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3963268634},{"type":"v1","addr":"172.21.15.27:6839","nonce":3963268634}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24551,24539,24520,24505,24515]},"id":1}]} 2024-05-07T02:39:18.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.585+0000 7faeb8400700 1 -- 172.21.15.19:0/3211290206 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fae980b29f0 msgr2=0x7fae980b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:18.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.585+0000 7faeb8400700 1 --2- 172.21.15.19:0/3211290206 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fae980b29f0 0x7fae980b4f00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7faeb01077f0 tx=0x7fae9c009260 comp rx=0 tx=0).stop 2024-05-07T02:39:18.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.585+0000 7faeb8400700 1 -- 172.21.15.19:0/3211290206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faeb01045f0 msgr2=0x7faeb0075ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:18.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.585+0000 7faeb8400700 1 --2- 172.21.15.19:0/3211290206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faeb01045f0 0x7faeb0075ac0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7faea400ef10 tx=0x7faea400c3a0 comp rx=0 tx=0).stop 2024-05-07T02:39:18.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.585+0000 7faeb8400700 1 -- 172.21.15.19:0/3211290206 shutdown_connections 2024-05-07T02:39:18.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.585+0000 7faeb8400700 1 --2- 172.21.15.19:0/3211290206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faeb0109870 0x7faeb0076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:18.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.585+0000 7faeb8400700 1 --2- 172.21.15.19:0/3211290206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faeb01045f0 0x7faeb0075ac0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:18.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.585+0000 7faeb8400700 1 --2- 172.21.15.19:0/3211290206 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fae980b29f0 0x7fae980b4f00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:18.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.585+0000 7faeb8400700 1 --2- 172.21.15.19:0/3211290206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faeb0108f20 0x7faeb0076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:18.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.585+0000 7faeb8400700 1 -- 172.21.15.19:0/3211290206 >> 172.21.15.19:0/3211290206 conn(0x7faeb0100040 msgr2=0x7faeb0108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:18.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.585+0000 7faeb8400700 1 -- 172.21.15.19:0/3211290206 shutdown_connections 2024-05-07T02:39:18.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:18.585+0000 7faeb8400700 1 -- 172.21.15.19:0/3211290206 wait complete. 2024-05-07T02:39:18.587 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 46 2024-05-07T02:39:19.442 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-07T02:39:19.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:19 smithi064 ceph-mon[34699]: pgmap v229: 129 pgs: 129 active+clean; 177 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 840 KiB/s wr, 58 op/s 2024-05-07T02:39:19.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:19 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/3211290206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:39:19.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:19 smithi019 ceph-mon[29781]: pgmap v229: 129 pgs: 129 active+clean; 177 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 840 KiB/s wr, 58 op/s 2024-05-07T02:39:19.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:19 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/3211290206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:39:19.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:19 smithi027 ceph-mon[34398]: pgmap v229: 129 pgs: 129 active+clean; 177 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 840 KiB/s wr, 58 op/s 2024-05-07T02:39:19.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:19 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/3211290206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:39:19.920 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:39:20.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:20 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:39:20.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:20 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:39:20.706 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:20 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:39:22.003 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:21 smithi019 ceph-mon[29781]: pgmap v230: 129 pgs: 129 active+clean; 177 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 759 KiB/s wr, 52 op/s 2024-05-07T02:39:22.003 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:21 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:39:22.004 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:21 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:39:22.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:21 smithi027 ceph-mon[34398]: pgmap v230: 129 pgs: 129 active+clean; 177 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 759 KiB/s wr, 52 op/s 2024-05-07T02:39:22.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:21 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:39:22.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:21 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:39:22.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.257+0000 7f58e9c00700 1 -- 172.21.15.19:0/2035616895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58e4108f10 msgr2=0x7f58e4109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:22.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.257+0000 7f58e9c00700 1 --2- 172.21.15.19:0/2035616895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58e4108f10 0x7f58e4109320 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f58cc0099c0 tx=0x7f58cc009cd0 comp rx=0 tx=0).stop 2024-05-07T02:39:22.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.258+0000 7f58e9c00700 1 -- 172.21.15.19:0/2035616895 shutdown_connections 2024-05-07T02:39:22.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.258+0000 7f58e9c00700 1 --2- 172.21.15.19:0/2035616895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58e4109860 0x7f58e41111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:22.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.258+0000 7f58e9c00700 1 --2- 172.21.15.19:0/2035616895 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58e41045e0 0x7f58e41049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:22.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.258+0000 7f58e9c00700 1 --2- 172.21.15.19:0/2035616895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58e4108f10 0x7f58e4109320 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:22.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.258+0000 7f58e9c00700 1 -- 172.21.15.19:0/2035616895 >> 172.21.15.19:0/2035616895 conn(0x7f58e40ffe70 msgr2=0x7f58e41022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:22.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.258+0000 7f58e9c00700 1 -- 172.21.15.19:0/2035616895 shutdown_connections 2024-05-07T02:39:22.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.258+0000 7f58e9c00700 1 -- 172.21.15.19:0/2035616895 wait complete. 2024-05-07T02:39:22.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.259+0000 7f58e9c00700 1 Processor -- start 2024-05-07T02:39:22.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.259+0000 7f58e9c00700 1 -- start start 2024-05-07T02:39:22.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.260+0000 7f58e9c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58e41045e0 0x7f58e419d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:22.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.260+0000 7f58e9c00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58e4108f10 0x7f58e419d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:22.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.260+0000 7f58e9c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58e4109860 0x7f58e41a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:22.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.260+0000 7f58e9c00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f58e41144e0 con 0x7f58e4109860 2024-05-07T02:39:22.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.260+0000 7f58e9c00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f58e4114360 con 0x7f58e41045e0 2024-05-07T02:39:22.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.260+0000 7f58e9c00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f58e4114660 con 0x7f58e4108f10 2024-05-07T02:39:22.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.260+0000 7f58e2a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58e41045e0 0x7f58e419d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:22.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.260+0000 7f58e3400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58e4109860 0x7f58e41a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:22.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.260+0000 7f58e2a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58e41045e0 0x7f58e419d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:60232/0 (socket says 172.21.15.19:60232) 2024-05-07T02:39:22.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.260+0000 7f58e2a00700 1 -- 172.21.15.19:0/2745549674 learned_addr learned my addr 172.21.15.19:0/2745549674 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:39:22.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.260+0000 7f58da000700 1 --2- 172.21.15.19:0/2745549674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58e4108f10 0x7f58e419d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:22.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.261+0000 7f58e3400700 1 -- 172.21.15.19:0/2745549674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58e4108f10 msgr2=0x7f58e419d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:22.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.261+0000 7f58e3400700 1 --2- 172.21.15.19:0/2745549674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58e4108f10 0x7f58e419d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:22.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.261+0000 7f58e3400700 1 -- 172.21.15.19:0/2745549674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58e41045e0 msgr2=0x7f58e419d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:22.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.261+0000 7f58e3400700 1 --2- 172.21.15.19:0/2745549674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58e41045e0 0x7f58e419d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:22.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.261+0000 7f58e3400700 1 -- 172.21.15.19:0/2745549674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e41a1f80 con 0x7f58e4109860 2024-05-07T02:39:22.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.261+0000 7f58e3400700 1 --2- 172.21.15.19:0/2745549674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58e4109860 0x7f58e41a18e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f58dc00e970 tx=0x7f58dc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:22.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.261+0000 7f58dbe00700 1 -- 172.21.15.19:0/2745549674 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58dc004220 con 0x7f58e4109860 2024-05-07T02:39:22.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.262+0000 7f58e9c00700 1 -- 172.21.15.19:0/2745549674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58e41a2270 con 0x7f58e4109860 2024-05-07T02:39:22.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.262+0000 7f58dbe00700 1 -- 172.21.15.19:0/2745549674 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58dc0043c0 con 0x7f58e4109860 2024-05-07T02:39:22.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.262+0000 7f58dbe00700 1 -- 172.21.15.19:0/2745549674 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58dc010700 con 0x7f58e4109860 2024-05-07T02:39:22.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.262+0000 7f58e9c00700 1 -- 172.21.15.19:0/2745549674 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58e41a9b90 con 0x7f58e4109860 2024-05-07T02:39:22.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.266+0000 7f58dbe00700 1 -- 172.21.15.19:0/2745549674 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f58dc0108a0 con 0x7f58e4109860 2024-05-07T02:39:22.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.267+0000 7f58f882bb80 1 Processor -- start 2024-05-07T02:39:22.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.267+0000 7f58f882bb80 1 -- start start 2024-05-07T02:39:22.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.267+0000 7f58f882bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593b9b623b0 0x5593b9b627c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:22.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.267+0000 7f58f882bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593b9b5c470 0x5593b9b5c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:22.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.267+0000 7f58f882bb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5593b9b5cdc0 0x5593b9b69040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:22.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.267+0000 7f58f882bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5593b9a7ea20 con 0x5593b9b5c470 2024-05-07T02:39:22.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.267+0000 7f58f882bb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5593b9a1c760 con 0x5593b9b5cdc0 2024-05-07T02:39:22.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.267+0000 7f58f882bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5593b9a7a9e0 con 0x5593b9b623b0 2024-05-07T02:39:22.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.268+0000 7f58e2a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593b9b5c470 0x5593b9b5c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:22.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.268+0000 7f58da000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593b9b623b0 0x5593b9b627c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:22.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.268+0000 7f58e2a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593b9b5c470 0x5593b9b5c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36174/0 (socket says 172.21.15.19:36174) 2024-05-07T02:39:22.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.268+0000 7f58da000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593b9b623b0 0x5593b9b627c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:43206/0 (socket says 172.21.15.19:43206) 2024-05-07T02:39:22.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.268+0000 7f58e2a00700 1 -- 172.21.15.19:0/3169142781 learned_addr learned my addr 172.21.15.19:0/3169142781 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:39:22.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.268+0000 7f58da000700 1 --2- 172.21.15.19:0/3169142781 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5593b9b5cdc0 0x5593b9b69040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:22.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.268+0000 7f58dbe00700 1 --2- 172.21.15.19:0/2745549674 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f58c80b29f0 0x7f58c80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:22.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.268+0000 7f58da000700 1 -- 172.21.15.19:0/3169142781 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5593b9b5cdc0 msgr2=0x5593b9b69040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:22.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.268+0000 7f58da000700 1 --2- 172.21.15.19:0/3169142781 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5593b9b5cdc0 0x5593b9b69040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:22.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.268+0000 7f58da000700 1 -- 172.21.15.19:0/3169142781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593b9b5c470 msgr2=0x5593b9b5c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:22.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.268+0000 7f58da000700 1 --2- 172.21.15.19:0/3169142781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593b9b5c470 0x5593b9b5c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:22.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.268+0000 7f58da000700 1 -- 172.21.15.19:0/3169142781 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593b99cb8d0 con 0x5593b9b623b0 2024-05-07T02:39:22.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.268+0000 7f58dbe00700 1 -- 172.21.15.19:0/2745549674 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9525+0+0 (secure 0 0 0) 0x7f58dc0e2800 con 0x7f58e4109860 2024-05-07T02:39:22.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.268+0000 7f58e3400700 1 --2- 172.21.15.19:0/2745549674 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f58c80b29f0 0x7f58c80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:22.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.269+0000 7f58da000700 1 --2- 172.21.15.19:0/3169142781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593b9b623b0 0x5593b9b627c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f58cc009990 tx=0x7f58cc0058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92ba99d9e6ad6911 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:22.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.269+0000 7f58d3e00700 1 -- 172.21.15.19:0/3169142781 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc0053f0 con 0x5593b9b623b0 2024-05-07T02:39:22.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.269+0000 7f58d3e00700 1 -- 172.21.15.19:0/3169142781 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58cc005590 con 0x5593b9b623b0 2024-05-07T02:39:22.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.269+0000 7f58d3e00700 1 -- 172.21.15.19:0/3169142781 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc00bc60 con 0x5593b9b623b0 2024-05-07T02:39:22.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.270+0000 7f58f882bb80 1 -- 172.21.15.19:0/3169142781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593b9b623b0 msgr2=0x5593b9b627c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:22.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.270+0000 7f58f882bb80 1 --2- 172.21.15.19:0/3169142781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593b9b623b0 0x5593b9b627c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f58cc009990 tx=0x7f58cc0058e0 comp rx=0 tx=0).stop 2024-05-07T02:39:22.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.270+0000 7f58e3400700 1 --2- 172.21.15.19:0/2745549674 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f58c80b29f0 0x7f58c80b4f00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f58dc0a9620 tx=0x7f58dc003fb0 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:22.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.270+0000 7f58f882bb80 1 -- 172.21.15.19:0/3169142781 shutdown_connections 2024-05-07T02:39:22.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.270+0000 7f58f882bb80 1 --2- 172.21.15.19:0/3169142781 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5593b9b5cdc0 0x5593b9b69040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:22.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.270+0000 7f58f882bb80 1 --2- 172.21.15.19:0/3169142781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593b9b623b0 0x5593b9b627c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:22.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.270+0000 7f58f882bb80 1 --2- 172.21.15.19:0/3169142781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593b9b5c470 0x5593b9b5c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:22.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.270+0000 7f58f882bb80 1 -- 172.21.15.19:0/3169142781 >> 172.21.15.19:0/3169142781 conn(0x5593b99dee40 msgr2=0x5593b9b572f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:22.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.270+0000 7f58f882bb80 1 -- 172.21.15.19:0/3169142781 shutdown_connections 2024-05-07T02:39:22.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.270+0000 7f58f882bb80 1 -- 172.21.15.19:0/3169142781 wait complete. 2024-05-07T02:39:22.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.271+0000 7f58f882bb80 1 -- start start 2024-05-07T02:39:22.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.271+0000 7f58f882bb80 1 Processor -- start 2024-05-07T02:39:22.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.272+0000 7f58f882bb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5593b9b5c470 0x5593b9b66590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:22.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.272+0000 7f58f882bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593b9b5cdc0 0x5593b9b66ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:22.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.272+0000 7f58f882bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593b9b623b0 0x5593b9b67010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:22.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.272+0000 7f58f882bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5593b9b69760 con 0x5593b9b5cdc0 2024-05-07T02:39:22.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.272+0000 7f58f882bb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5593b99e4b20 con 0x5593b9b5c470 2024-05-07T02:39:22.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.272+0000 7f58f882bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5593b9b698e0 con 0x5593b9b623b0 2024-05-07T02:39:22.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.272+0000 7f58f882bb80 10 client.0 _ll_register_callbacks cb 0x5593b9a25c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:39:22.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.272+0000 7f58e2a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593b9b5cdc0 0x5593b9b66ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:22.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.272+0000 7f58da000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593b9b623b0 0x5593b9b67010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:22.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.272+0000 7f58da000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593b9b623b0 0x5593b9b67010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:43218/0 (socket says 172.21.15.19:43218) 2024-05-07T02:39:22.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.272+0000 7f58da000700 1 -- 172.21.15.19:0/2356105980 learned_addr learned my addr 172.21.15.19:0/2356105980 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:39:22.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.272+0000 7f58da000700 1 --2- 172.21.15.19:0/2356105980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5593b9b5c470 0x5593b9b66590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:22.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.273+0000 7f58e2a00700 1 -- 172.21.15.19:0/2356105980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593b9b623b0 msgr2=0x5593b9b67010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:22.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.273+0000 7f58e2a00700 1 --2- 172.21.15.19:0/2356105980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593b9b623b0 0x5593b9b67010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:22.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.273+0000 7f58e2a00700 1 -- 172.21.15.19:0/2356105980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5593b9b5c470 msgr2=0x5593b9b66590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:22.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.273+0000 7f58e2a00700 1 --2- 172.21.15.19:0/2356105980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5593b9b5c470 0x5593b9b66590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:22.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.273+0000 7f58e2a00700 1 -- 172.21.15.19:0/2356105980 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593b9a44420 con 0x5593b9b5cdc0 2024-05-07T02:39:22.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.273+0000 7f58da000700 1 --2- 172.21.15.19:0/2356105980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5593b9b5c470 0x5593b9b66590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:39:22.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.273+0000 7f58e2a00700 1 --2- 172.21.15.19:0/2356105980 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593b9b5cdc0 0x5593b9b66ad0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f58d4009fd0 tx=0x7f58d4014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b45c7e1ff7499b72 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:22.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.273+0000 7f58d0c00700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-07T02:39:22.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.273+0000 7f58d0c00700 1 -- 172.21.15.19:0/2356105980 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d4014b30 con 0x5593b9b5cdc0 2024-05-07T02:39:22.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.273+0000 7f58f882bb80 1 -- 172.21.15.19:0/2356105980 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58cc005590 con 0x5593b9b5cdc0 2024-05-07T02:39:22.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.273+0000 7f58d0c00700 1 -- 172.21.15.19:0/2356105980 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9525+0+0 (secure 0 0 0) 0x7f58d4014cd0 con 0x5593b9b5cdc0 2024-05-07T02:39:22.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.274+0000 7f58d0c00700 1 -- 172.21.15.19:0/2356105980 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f58d4003a00 con 0x5593b9b5cdc0 2024-05-07T02:39:22.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.274+0000 7f58f882bb80 10 client.16146 fetch_fsmap learned FSMap version 46 2024-05-07T02:39:22.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.274+0000 7f58f882bb80 1 -- 172.21.15.19:0/2356105980 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5593b9b74490 con 0x5593b9b5cdc0 2024-05-07T02:39:22.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.274+0000 7f58d0c00700 1 -- 172.21.15.19:0/2356105980 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58cc005590 con 0x5593b9b5cdc0 2024-05-07T02:39:22.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.274+0000 7f58d0c00700 1 -- 172.21.15.19:0/2356105980 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 46) ==== 8386+0+0 (secure 0 0 0) 0x7f58d401e550 con 0x5593b9b5cdc0 2024-05-07T02:39:22.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.274+0000 7f58f882bb80 10 client.16146 fetch_fsmap finished waiting for FSMap version 46 2024-05-07T02:39:22.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.274+0000 7f58f882bb80 10 client.16146 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-07T02:39:22.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.274+0000 7f58f882bb80 20 client.16146 populate_metadata read hostname 'smithi019' 2024-05-07T02:39:22.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.274+0000 7f58f882bb80 1 --2- 172.21.15.19:0/2356105980 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x5593b9b76ad0 0x5593b9b78f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:22.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.274+0000 7f58f882bb80 4 client.16146 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:39:22.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.274+0000 7f58f882bb80 1 -- 172.21.15.19:0/2356105980 --> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593b9b292e0 con 0x5593b9b76ad0 2024-05-07T02:39:22.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.275+0000 7f58da000700 1 --2- 172.21.15.19:0/2356105980 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x5593b9b76ad0 0x5593b9b78f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:22.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.276+0000 7f58da000700 1 --2- 172.21.15.19:0/2356105980 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x5593b9b76ad0 0x5593b9b78f30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=144cdff134d1a6c9 server_cookie=62d1661937c7fbd5 in_seq=0 out_seq=0 2024-05-07T02:39:22.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.276+0000 7f58d0c00700 10 client.16146 ms_handle_connect on v2:172.21.15.64:6836/2585907777 2024-05-07T02:39:22.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.277+0000 7f58d0c00700 1 -- 172.21.15.19:0/2356105980 <== mds.4 v2:172.21.15.64:6836/2585907777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5593b9b292e0 con 0x5593b9b76ad0 2024-05-07T02:39:22.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.277+0000 7f58d0c00700 10 client.16146 handle_command_reply: tid=0 2024-05-07T02:39:22.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.277+0000 7f58f882bb80 1 client.16146 shutdown 2024-05-07T02:39:22.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.277+0000 7f58f882bb80 1 -- 172.21.15.19:0/2356105980 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593b9b5cdc0 msgr2=0x5593b9b66ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:22.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.277+0000 7f58f882bb80 1 --2- 172.21.15.19:0/2356105980 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593b9b5cdc0 0x5593b9b66ad0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f58d4009fd0 tx=0x7f58d4014600 comp rx=0 tx=0).stop 2024-05-07T02:39:22.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.277+0000 7f58f882bb80 1 -- 172.21.15.19:0/2356105980 shutdown_connections 2024-05-07T02:39:22.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.277+0000 7f58f882bb80 1 --2- 172.21.15.19:0/2356105980 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x5593b9b76ad0 0x5593b9b78f30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:22.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.277+0000 7f58f882bb80 1 --2- 172.21.15.19:0/2356105980 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593b9b623b0 0x5593b9b67010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:22.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.277+0000 7f58f882bb80 1 --2- 172.21.15.19:0/2356105980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5593b9b5c470 0x5593b9b66590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:22.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.277+0000 7f58f882bb80 1 --2- 172.21.15.19:0/2356105980 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593b9b5cdc0 0x5593b9b66ad0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:22.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.277+0000 7f58d0c00700 0 client.16146 ms_handle_reset on v2:172.21.15.64:6836/2585907777 2024-05-07T02:39:22.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.277+0000 7f58f882bb80 1 -- 172.21.15.19:0/2356105980 >> 172.21.15.19:0/2356105980 conn(0x5593b99dee40 msgr2=0x5593b9b5b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:22.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.277+0000 7f58f882bb80 1 -- 172.21.15.19:0/2356105980 shutdown_connections 2024-05-07T02:39:22.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.277+0000 7f58f882bb80 1 -- 172.21.15.19:0/2356105980 wait complete. 2024-05-07T02:39:22.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.278+0000 7f58f882bb80 20 client.16146 trim_cache size 0 max 16384 2024-05-07T02:39:22.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.294+0000 7f58f882bb80 1 Processor -- start 2024-05-07T02:39:22.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.294+0000 7f58f882bb80 1 -- start start 2024-05-07T02:39:22.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.295+0000 7f58f882bb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5593b9b5c470 0x5593b9b63fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:22.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.295+0000 7f58f882bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593b9b5cdc0 0x5593b9b727f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:22.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.295+0000 7f58f882bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593b9b623b0 0x5593b9b72d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:22.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.295+0000 7f58f882bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f58e41b28a0 con 0x5593b9b623b0 2024-05-07T02:39:22.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.295+0000 7f58f882bb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f58e41b4500 con 0x5593b9b5c470 2024-05-07T02:39:22.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.295+0000 7f58f882bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f58e41b2f70 con 0x5593b9b5cdc0 2024-05-07T02:39:22.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.295+0000 7f58da000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593b9b623b0 0x5593b9b72d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:22.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.295+0000 7f58e2a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593b9b5cdc0 0x5593b9b727f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:22.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.295+0000 7f58e2a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593b9b5cdc0 0x5593b9b727f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:43220/0 (socket says 172.21.15.19:43220) 2024-05-07T02:39:22.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.295+0000 7f58e2a00700 1 -- 172.21.15.19:0/2197698398 learned_addr learned my addr 172.21.15.19:0/2197698398 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:39:22.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.295+0000 7f58da000700 1 --2- 172.21.15.19:0/2197698398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5593b9b5c470 0x5593b9b63fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:22.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.296+0000 7f58e2a00700 1 -- 172.21.15.19:0/2197698398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5593b9b5c470 msgr2=0x5593b9b63fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:22.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.296+0000 7f58e2a00700 1 --2- 172.21.15.19:0/2197698398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5593b9b5c470 0x5593b9b63fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:22.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.296+0000 7f58e2a00700 1 -- 172.21.15.19:0/2197698398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593b9b623b0 msgr2=0x5593b9b72d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:22.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.296+0000 7f58e2a00700 1 --2- 172.21.15.19:0/2197698398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593b9b623b0 0x5593b9b72d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:22.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.296+0000 7f58e2a00700 1 -- 172.21.15.19:0/2197698398 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593b9b736d0 con 0x5593b9b5cdc0 2024-05-07T02:39:22.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.296+0000 7f58e2a00700 1 --2- 172.21.15.19:0/2197698398 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593b9b5cdc0 0x5593b9b727f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f58d401e9f0 tx=0x7f58d40093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38031853c658c109 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:22.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.296+0000 7f58d3400700 1 -- 172.21.15.19:0/2197698398 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d402f020 con 0x5593b9b5cdc0 2024-05-07T02:39:22.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.296+0000 7f58d3400700 1 -- 172.21.15.19:0/2197698398 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58d4005be0 con 0x5593b9b5cdc0 2024-05-07T02:39:22.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.296+0000 7f58d3400700 1 -- 172.21.15.19:0/2197698398 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d4014e50 con 0x5593b9b5cdc0 2024-05-07T02:39:22.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.297+0000 7f58f882bb80 1 -- 172.21.15.19:0/2197698398 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593b9b5cdc0 msgr2=0x5593b9b727f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:22.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.297+0000 7f58f882bb80 1 --2- 172.21.15.19:0/2197698398 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593b9b5cdc0 0x5593b9b727f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f58d401e9f0 tx=0x7f58d40093e0 comp rx=0 tx=0).stop 2024-05-07T02:39:22.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.297+0000 7f58f882bb80 1 -- 172.21.15.19:0/2197698398 shutdown_connections 2024-05-07T02:39:22.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.297+0000 7f58f882bb80 1 --2- 172.21.15.19:0/2197698398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593b9b623b0 0x5593b9b72d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:22.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.297+0000 7f58f882bb80 1 --2- 172.21.15.19:0/2197698398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5593b9b5c470 0x5593b9b63fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:22.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.297+0000 7f58f882bb80 1 --2- 172.21.15.19:0/2197698398 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593b9b5cdc0 0x5593b9b727f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:22.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.297+0000 7f58f882bb80 1 -- 172.21.15.19:0/2197698398 >> 172.21.15.19:0/2197698398 conn(0x5593b99dee40 msgr2=0x5593b9b65b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:22.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.297+0000 7f58f882bb80 1 -- 172.21.15.19:0/2197698398 shutdown_connections 2024-05-07T02:39:22.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.297+0000 7f58f882bb80 1 -- 172.21.15.19:0/2197698398 wait complete. 2024-05-07T02:39:22.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.298+0000 7f58f882bb80 1 -- start start 2024-05-07T02:39:22.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.298+0000 7f58f882bb80 1 Processor -- start 2024-05-07T02:39:22.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.299+0000 7f58f882bb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5593b9b5c470 0x5593b9b6eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:22.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.299+0000 7f58f882bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593b9b5cdc0 0x5593b9b6f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:22.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.299+0000 7f58f882bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593b9b623b0 0x5593b9b7d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:22.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.299+0000 7f58f882bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f58e41b3d70 con 0x5593b9b5cdc0 2024-05-07T02:39:22.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.299+0000 7f58f882bb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f58e41b3640 con 0x5593b9b5c470 2024-05-07T02:39:22.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.299+0000 7f58f882bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f58e41b0860 con 0x5593b9b623b0 2024-05-07T02:39:22.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.299+0000 7f58f882bb80 10 client.0 _ll_register_callbacks cb 0x5593b9a25c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:39:22.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.299+0000 7f58da000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5593b9b5c470 0x5593b9b6eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:22.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.299+0000 7f58e2a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593b9b5cdc0 0x5593b9b6f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:22.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.299+0000 7f58da000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5593b9b5c470 0x5593b9b6eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:60266/0 (socket says 172.21.15.19:60266) 2024-05-07T02:39:22.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.299+0000 7f58da000700 1 -- 172.21.15.19:0/1807069701 learned_addr learned my addr 172.21.15.19:0/1807069701 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:39:22.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.299+0000 7f58da000700 1 --2- 172.21.15.19:0/1807069701 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593b9b623b0 0x5593b9b7d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:22.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.300+0000 7f58da000700 1 -- 172.21.15.19:0/1807069701 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593b9b623b0 msgr2=0x5593b9b7d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:22.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.300+0000 7f58da000700 1 --2- 172.21.15.19:0/1807069701 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593b9b623b0 0x5593b9b7d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:22.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.300+0000 7f58da000700 1 -- 172.21.15.19:0/1807069701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593b9b5cdc0 msgr2=0x5593b9b6f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:22.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.300+0000 7f58da000700 1 --2- 172.21.15.19:0/1807069701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593b9b5cdc0 0x5593b9b6f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:22.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.300+0000 7f58da000700 1 -- 172.21.15.19:0/1807069701 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593b9b7dcb0 con 0x5593b9b5c470 2024-05-07T02:39:22.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.300+0000 7f58da000700 1 --2- 172.21.15.19:0/1807069701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5593b9b5c470 0x5593b9b6eb70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f58cc00b580 tx=0x7f58cc0058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b31aad226c62a6e server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:22.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.300+0000 7f58d0c00700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:39:22.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.300+0000 7f58d0c00700 1 -- 172.21.15.19:0/1807069701 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc003b80 con 0x5593b9b5c470 2024-05-07T02:39:22.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.300+0000 7f58f882bb80 1 -- 172.21.15.19:0/1807069701 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58d4005be0 con 0x5593b9b5c470 2024-05-07T02:39:22.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.300+0000 7f58d0c00700 1 -- 172.21.15.19:0/1807069701 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9525+0+0 (secure 0 0 0) 0x7f58cc003d20 con 0x5593b9b5c470 2024-05-07T02:39:22.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.301+0000 7f58d0c00700 1 -- 172.21.15.19:0/1807069701 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f58cc04b050 con 0x5593b9b5c470 2024-05-07T02:39:22.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.301+0000 7f58f882bb80 10 client.26351 fetch_fsmap learned FSMap version 46 2024-05-07T02:39:22.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.301+0000 7f58f882bb80 1 -- 172.21.15.19:0/1807069701 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5593b9be0920 con 0x5593b9b5c470 2024-05-07T02:39:22.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.301+0000 7f58d0c00700 1 -- 172.21.15.19:0/1807069701 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58d4005be0 con 0x5593b9b5c470 2024-05-07T02:39:22.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.301+0000 7f58d0c00700 1 -- 172.21.15.19:0/1807069701 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 46) ==== 8386+0+0 (secure 0 0 0) 0x7f58cc021040 con 0x5593b9b5c470 2024-05-07T02:39:22.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.301+0000 7f58f882bb80 10 client.26351 fetch_fsmap finished waiting for FSMap version 46 2024-05-07T02:39:22.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.301+0000 7f58f882bb80 10 client.26351 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-07T02:39:22.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.301+0000 7f58f882bb80 20 client.26351 populate_metadata read hostname 'smithi019' 2024-05-07T02:39:22.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.301+0000 7f58f882bb80 1 --2- 172.21.15.19:0/1807069701 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x5593b9be0bb0 0x5593b9be2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:22.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.301+0000 7f58f882bb80 4 client.26351 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:39:22.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.301+0000 7f58f882bb80 1 -- 172.21.15.19:0/1807069701 --> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5593b9a8c400 con 0x5593b9be0bb0 2024-05-07T02:39:22.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.302+0000 7f58e2a00700 1 --2- 172.21.15.19:0/1807069701 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x5593b9be0bb0 0x5593b9be2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:22.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.302+0000 7f58e2a00700 1 --2- 172.21.15.19:0/1807069701 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x5593b9be0bb0 0x5593b9be2fd0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f65119c2b3a3c89 server_cookie=4c6d60b3101ae1bc in_seq=0 out_seq=0 2024-05-07T02:39:22.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.302+0000 7f58d0c00700 10 client.26351 ms_handle_connect on v2:172.21.15.64:6836/2585907777 2024-05-07T02:39:22.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.302+0000 7f58d0c00700 1 -- 172.21.15.19:0/1807069701 <== mds.4 v2:172.21.15.64:6836/2585907777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5593b9a8c400 con 0x5593b9be0bb0 2024-05-07T02:39:22.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.302+0000 7f58d0c00700 10 client.26351 handle_command_reply: tid=0 2024-05-07T02:39:22.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.303+0000 7f58f882bb80 1 client.26351 shutdown 2024-05-07T02:39:22.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.303+0000 7f58f882bb80 1 -- 172.21.15.19:0/1807069701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5593b9b5c470 msgr2=0x5593b9b6eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:22.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.303+0000 7f58f882bb80 1 --2- 172.21.15.19:0/1807069701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5593b9b5c470 0x5593b9b6eb70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f58cc00b580 tx=0x7f58cc0058e0 comp rx=0 tx=0).stop 2024-05-07T02:39:22.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.303+0000 7f58f882bb80 1 -- 172.21.15.19:0/1807069701 shutdown_connections 2024-05-07T02:39:22.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.303+0000 7f58f882bb80 1 --2- 172.21.15.19:0/1807069701 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x5593b9be0bb0 0x5593b9be2fd0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:22.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.303+0000 7f58f882bb80 1 --2- 172.21.15.19:0/1807069701 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5593b9b623b0 0x5593b9b7d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:22.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.303+0000 7f58f882bb80 1 --2- 172.21.15.19:0/1807069701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5593b9b5c470 0x5593b9b6eb70 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:22.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.303+0000 7f58d0c00700 0 client.26351 ms_handle_reset on v2:172.21.15.64:6836/2585907777 2024-05-07T02:39:22.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.303+0000 7f58f882bb80 1 --2- 172.21.15.19:0/1807069701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5593b9b5cdc0 0x5593b9b6f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:22.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.303+0000 7f58f882bb80 1 -- 172.21.15.19:0/1807069701 >> 172.21.15.19:0/1807069701 conn(0x5593b99dee40 msgr2=0x5593b9b51120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:22.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.303+0000 7f58f882bb80 1 -- 172.21.15.19:0/1807069701 shutdown_connections 2024-05-07T02:39:22.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.303+0000 7f58f882bb80 1 -- 172.21.15.19:0/1807069701 wait complete. 2024-05-07T02:39:22.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.303+0000 7f58f882bb80 20 client.26351 trim_cache size 0 max 16384 2024-05-07T02:39:22.322 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:39:22.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.304+0000 7f58e9c00700 1 -- 172.21.15.19:0/2745549674 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f58c80b29f0 msgr2=0x7f58c80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:22.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.304+0000 7f58e9c00700 1 --2- 172.21.15.19:0/2745549674 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f58c80b29f0 0x7f58c80b4f00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f58dc0a9620 tx=0x7f58dc003fb0 comp rx=0 tx=0).stop 2024-05-07T02:39:22.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.304+0000 7f58e9c00700 1 -- 172.21.15.19:0/2745549674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58e4109860 msgr2=0x7f58e41a18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:22.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.304+0000 7f58e9c00700 1 --2- 172.21.15.19:0/2745549674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58e4109860 0x7f58e41a18e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f58dc00e970 tx=0x7f58dc00ec80 comp rx=0 tx=0).stop 2024-05-07T02:39:22.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.304+0000 7f58e9c00700 1 -- 172.21.15.19:0/2745549674 shutdown_connections 2024-05-07T02:39:22.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.304+0000 7f58e9c00700 1 --2- 172.21.15.19:0/2745549674 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f58c80b29f0 0x7f58c80b4f00 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:22.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.304+0000 7f58e9c00700 1 --2- 172.21.15.19:0/2745549674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58e4109860 0x7f58e41a18e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:22.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.304+0000 7f58e9c00700 1 --2- 172.21.15.19:0/2745549674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58e41045e0 0x7f58e419d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:22.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.304+0000 7f58e9c00700 1 --2- 172.21.15.19:0/2745549674 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f58e4108f10 0x7f58e419d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:22.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.304+0000 7f58e9c00700 1 -- 172.21.15.19:0/2745549674 >> 172.21.15.19:0/2745549674 conn(0x7f58e40ffe70 msgr2=0x7f58e410a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:22.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.305+0000 7f58e9c00700 1 -- 172.21.15.19:0/2745549674 shutdown_connections 2024-05-07T02:39:22.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:22.305+0000 7f58e9c00700 1 -- 172.21.15.19:0/2745549674 wait complete. 2024-05-07T02:39:22.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:21 smithi064 ceph-mon[34699]: pgmap v230: 129 pgs: 129 active+clean; 177 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 759 KiB/s wr, 52 op/s 2024-05-07T02:39:22.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:21 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:39:22.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:21 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:39:23.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:22 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:39:23.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:22 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:39:23.111 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:22 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:39:23.184 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-07T02:39:23.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:22 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:39:23.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:22 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:39:23.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:22 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:39:23.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:22 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:39:23.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:22 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:39:23.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:22 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:39:23.628 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:39:24.132 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:23 smithi019 ceph-mon[29781]: pgmap v231: 129 pgs: 129 active+clean; 179 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1017 KiB/s wr, 69 op/s 2024-05-07T02:39:24.206 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:23 smithi027 ceph-mon[34398]: pgmap v231: 129 pgs: 129 active+clean; 179 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1017 KiB/s wr, 69 op/s 2024-05-07T02:39:24.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:23 smithi064 ceph-mon[34699]: pgmap v231: 129 pgs: 129 active+clean; 179 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1017 KiB/s wr, 69 op/s 2024-05-07T02:39:26.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.054+0000 7f300be00700 1 -- 172.21.15.19:0/2650118829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f300c109680 msgr2=0x7f300c110fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:26.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.054+0000 7f300be00700 1 --2- 172.21.15.19:0/2650118829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f300c109680 0x7f300c110fe0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f30040099c0 tx=0x7f3004009cd0 comp rx=0 tx=0).stop 2024-05-07T02:39:26.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.054+0000 7f300be00700 1 -- 172.21.15.19:0/2650118829 shutdown_connections 2024-05-07T02:39:26.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.054+0000 7f300be00700 1 --2- 172.21.15.19:0/2650118829 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f300c109680 0x7f300c110fe0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:26.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.054+0000 7f300be00700 1 --2- 172.21.15.19:0/2650118829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f300c104400 0x7f300c104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:26.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.054+0000 7f300be00700 1 --2- 172.21.15.19:0/2650118829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f300c108d30 0x7f300c109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:26.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.054+0000 7f300be00700 1 -- 172.21.15.19:0/2650118829 >> 172.21.15.19:0/2650118829 conn(0x7f300c079d10 msgr2=0x7f300c07a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:26.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.054+0000 7f300be00700 1 -- 172.21.15.19:0/2650118829 shutdown_connections 2024-05-07T02:39:26.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.054+0000 7f300be00700 1 -- 172.21.15.19:0/2650118829 wait complete. 2024-05-07T02:39:26.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.055+0000 7f300be00700 1 Processor -- start 2024-05-07T02:39:26.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.056+0000 7f300be00700 1 -- start start 2024-05-07T02:39:26.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.056+0000 7f300be00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f300c104400 0x7f300c1a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:26.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.056+0000 7f300be00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f300c108d30 0x7f300c1a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:26.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.056+0000 7f300be00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f300c109680 0x7f300c19aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:26.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.056+0000 7f300be00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f300c1142b0 con 0x7f300c108d30 2024-05-07T02:39:26.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.056+0000 7f300be00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f300c114130 con 0x7f300c104400 2024-05-07T02:39:26.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.056+0000 7f300be00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f300c114430 con 0x7f300c109680 2024-05-07T02:39:26.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.057+0000 7f3002000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f300c108d30 0x7f300c1a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:26.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.057+0000 7f300aa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f300c104400 0x7f300c1a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:26.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.057+0000 7f300b400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f300c109680 0x7f300c19aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:26.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.057+0000 7f300aa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f300c104400 0x7f300c1a0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:60292/0 (socket says 172.21.15.19:60292) 2024-05-07T02:39:26.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.057+0000 7f300aa00700 1 -- 172.21.15.19:0/756841766 learned_addr learned my addr 172.21.15.19:0/756841766 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:39:26.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.057+0000 7f300b400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f300c109680 0x7f300c19aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:43248/0 (socket says 172.21.15.19:43248) 2024-05-07T02:39:26.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.057+0000 7f300b400700 1 -- 172.21.15.19:0/756841766 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f300c104400 msgr2=0x7f300c1a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:26.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.057+0000 7f300b400700 1 --2- 172.21.15.19:0/756841766 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f300c104400 0x7f300c1a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:26.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.057+0000 7f300b400700 1 -- 172.21.15.19:0/756841766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f300c108d30 msgr2=0x7f300c1a0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:26.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.057+0000 7f300b400700 1 --2- 172.21.15.19:0/756841766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f300c108d30 0x7f300c1a0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:26.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.057+0000 7f300b400700 1 -- 172.21.15.19:0/756841766 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f300c19b160 con 0x7f300c109680 2024-05-07T02:39:26.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.057+0000 7f300aa00700 1 --2- 172.21.15.19:0/756841766 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f300c104400 0x7f300c1a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:39:26.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.058+0000 7f300b400700 1 --2- 172.21.15.19:0/756841766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f300c109680 0x7f300c19aab0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3004009990 tx=0x7f3004004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:26.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.058+0000 7f3003e00700 1 -- 172.21.15.19:0/756841766 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3004009e60 con 0x7f300c109680 2024-05-07T02:39:26.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.058+0000 7f3003e00700 1 -- 172.21.15.19:0/756841766 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3004003c70 con 0x7f300c109680 2024-05-07T02:39:26.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.058+0000 7f3003e00700 1 -- 172.21.15.19:0/756841766 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30040046b0 con 0x7f300c109680 2024-05-07T02:39:26.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.058+0000 7f300be00700 1 -- 172.21.15.19:0/756841766 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f300c19b3f0 con 0x7f300c109680 2024-05-07T02:39:26.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.058+0000 7f300be00700 1 -- 172.21.15.19:0/756841766 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f300c19b820 con 0x7f300c109680 2024-05-07T02:39:26.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.062+0000 7f3003e00700 1 -- 172.21.15.19:0/756841766 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3004028060 con 0x7f300c109680 2024-05-07T02:39:26.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.063+0000 7f302046db80 1 Processor -- start 2024-05-07T02:39:26.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.063+0000 7f302046db80 1 -- start start 2024-05-07T02:39:26.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.063+0000 7f302046db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee798f3c40 0x55ee798f4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:26.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.064+0000 7f302046db80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee798f4590 0x55ee798fd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:26.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.064+0000 7f3003e00700 1 --2- 172.21.15.19:0/756841766 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f2ff00b29f0 0x7f2ff00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:26.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.064+0000 7f302046db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee798fdc30 0x55ee79900050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:26.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.064+0000 7f302046db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ee79815a20 con 0x55ee798fdc30 2024-05-07T02:39:26.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.064+0000 7f302046db80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ee797b3760 con 0x55ee798f4590 2024-05-07T02:39:26.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.064+0000 7f302046db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ee798119e0 con 0x55ee798f3c40 2024-05-07T02:39:26.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.064+0000 7f3003e00700 1 -- 172.21.15.19:0/756841766 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9525+0+0 (secure 0 0 0) 0x7f30040e2fe0 con 0x7f300c109680 2024-05-07T02:39:26.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.064+0000 7f3002000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee798f3c40 0x55ee798f4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:26.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.064+0000 7f300aa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee798f4590 0x55ee798fd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:26.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.064+0000 7f300aa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee798f4590 0x55ee798fd6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:60300/0 (socket says 172.21.15.19:60300) 2024-05-07T02:39:26.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.064+0000 7f300aa00700 1 -- 172.21.15.19:0/3712997241 learned_addr learned my addr 172.21.15.19:0/3712997241 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:39:26.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.064+0000 7f3002000700 1 --2- 172.21.15.19:0/756841766 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f2ff00b29f0 0x7f2ff00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:26.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.064+0000 7f300b400700 1 --2- 172.21.15.19:0/3712997241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee798fdc30 0x55ee79900050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:26.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.064+0000 7f300aa00700 1 -- 172.21.15.19:0/3712997241 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee798f3c40 msgr2=0x55ee798f4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:26.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.064+0000 7f300aa00700 1 --2- 172.21.15.19:0/3712997241 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee798f3c40 0x55ee798f4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:26.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.064+0000 7f300aa00700 1 -- 172.21.15.19:0/3712997241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee798fdc30 msgr2=0x55ee79900050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:26.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.065+0000 7f300aa00700 1 --2- 172.21.15.19:0/3712997241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee798fdc30 0x55ee79900050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:26.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.065+0000 7f300aa00700 1 -- 172.21.15.19:0/3712997241 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee797628d0 con 0x55ee798f4590 2024-05-07T02:39:26.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.065+0000 7f3002000700 1 --2- 172.21.15.19:0/3712997241 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee798f3c40 0x55ee798f4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:39:26.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.065+0000 7f300b400700 1 --2- 172.21.15.19:0/3712997241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee798fdc30 0x55ee79900050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:39:26.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.065+0000 7f300aa00700 1 --2- 172.21.15.19:0/3712997241 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee798f4590 0x55ee798fd6f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc009480 tx=0x7f2ffc009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9dee5fe7a1fdde9b server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:26.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.065+0000 7f3002000700 1 --2- 172.21.15.19:0/756841766 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f2ff00b29f0 0x7f2ff00b4f00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2ff40097c0 tx=0x7f2ff4009260 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:26.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.065+0000 7f2ffbe00700 1 -- 172.21.15.19:0/3712997241 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc019070 con 0x55ee798f4590 2024-05-07T02:39:26.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.065+0000 7f2ffbe00700 1 -- 172.21.15.19:0/3712997241 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ffc00edb0 con 0x55ee798f4590 2024-05-07T02:39:26.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.065+0000 7f2ffbe00700 1 -- 172.21.15.19:0/3712997241 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc015da0 con 0x55ee798f4590 2024-05-07T02:39:26.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.066+0000 7f302046db80 1 -- 172.21.15.19:0/3712997241 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee798f4590 msgr2=0x55ee798fd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:26.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.066+0000 7f302046db80 1 --2- 172.21.15.19:0/3712997241 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee798f4590 0x55ee798fd6f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc009480 tx=0x7f2ffc009920 comp rx=0 tx=0).stop 2024-05-07T02:39:26.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.066+0000 7f302046db80 1 -- 172.21.15.19:0/3712997241 shutdown_connections 2024-05-07T02:39:26.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.066+0000 7f302046db80 1 --2- 172.21.15.19:0/3712997241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee798fdc30 0x55ee79900050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:26.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.066+0000 7f302046db80 1 --2- 172.21.15.19:0/3712997241 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee798f3c40 0x55ee798f4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:26.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.066+0000 7f302046db80 1 --2- 172.21.15.19:0/3712997241 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee798f4590 0x55ee798fd6f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:26.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.066+0000 7f302046db80 1 -- 172.21.15.19:0/3712997241 >> 172.21.15.19:0/3712997241 conn(0x55ee79775e40 msgr2=0x55ee798ee980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:26.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.066+0000 7f302046db80 1 -- 172.21.15.19:0/3712997241 shutdown_connections 2024-05-07T02:39:26.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.067+0000 7f302046db80 1 -- 172.21.15.19:0/3712997241 wait complete. 2024-05-07T02:39:26.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.067+0000 7f302046db80 1 -- start start 2024-05-07T02:39:26.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.067+0000 7f302046db80 1 Processor -- start 2024-05-07T02:39:26.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.068+0000 7f302046db80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee798f3c40 0x55ee798fab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:26.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.068+0000 7f302046db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee798f4590 0x55ee798fb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:26.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.068+0000 7f302046db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee798fdc30 0x55ee798fb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:26.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.068+0000 7f302046db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ee79900a00 con 0x55ee798fdc30 2024-05-07T02:39:26.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.068+0000 7f302046db80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ee7977bb20 con 0x55ee798f3c40 2024-05-07T02:39:26.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.068+0000 7f302046db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ee79900b80 con 0x55ee798f4590 2024-05-07T02:39:26.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.068+0000 7f302046db80 10 client.0 _ll_register_callbacks cb 0x55ee797a9380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:39:26.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.068+0000 7f300aa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee798f3c40 0x55ee798fab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:26.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.068+0000 7f300b400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee798fdc30 0x55ee798fb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:26.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.068+0000 7f300aa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee798f3c40 0x55ee798fab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:60314/0 (socket says 172.21.15.19:60314) 2024-05-07T02:39:26.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.068+0000 7f300aa00700 1 -- 172.21.15.19:0/2707965305 learned_addr learned my addr 172.21.15.19:0/2707965305 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:39:26.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.068+0000 7f3002000700 1 --2- 172.21.15.19:0/2707965305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee798f4590 0x55ee798fb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:26.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.069+0000 7f300aa00700 1 -- 172.21.15.19:0/2707965305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee798f4590 msgr2=0x55ee798fb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:26.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.069+0000 7f300aa00700 1 --2- 172.21.15.19:0/2707965305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee798f4590 0x55ee798fb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:26.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.069+0000 7f300aa00700 1 -- 172.21.15.19:0/2707965305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee798fdc30 msgr2=0x55ee798fb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:26.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.069+0000 7f300aa00700 1 --2- 172.21.15.19:0/2707965305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee798fdc30 0x55ee798fb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:26.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.069+0000 7f300aa00700 1 -- 172.21.15.19:0/2707965305 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee797db420 con 0x55ee798f3c40 2024-05-07T02:39:26.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.069+0000 7f3002000700 1 --2- 172.21.15.19:0/2707965305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee798f4590 0x55ee798fb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:39:26.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.069+0000 7f300aa00700 1 --2- 172.21.15.19:0/2707965305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee798f3c40 0x55ee798fab60 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc009c10 tx=0x7f2ffc009750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44031512bcaddb35 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:26.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.069+0000 7f2ff8c00700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:39:26.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.069+0000 7f2ff8c00700 1 -- 172.21.15.19:0/2707965305 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc019040 con 0x55ee798f3c40 2024-05-07T02:39:26.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.070+0000 7f302046db80 1 -- 172.21.15.19:0/2707965305 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ffc00edb0 con 0x55ee798f3c40 2024-05-07T02:39:26.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.070+0000 7f2ff8c00700 1 -- 172.21.15.19:0/2707965305 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9525+0+0 (secure 0 0 0) 0x7f2ffc004cd0 con 0x55ee798f3c40 2024-05-07T02:39:26.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.070+0000 7f2ff8c00700 1 -- 172.21.15.19:0/2707965305 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f2ffc030050 con 0x55ee798f3c40 2024-05-07T02:39:26.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.070+0000 7f302046db80 10 client.26363 fetch_fsmap learned FSMap version 46 2024-05-07T02:39:26.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.070+0000 7f302046db80 1 -- 172.21.15.19:0/2707965305 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55ee7990b590 con 0x55ee798f3c40 2024-05-07T02:39:26.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.070+0000 7f2ff8c00700 1 -- 172.21.15.19:0/2707965305 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ffc00edb0 con 0x55ee798f3c40 2024-05-07T02:39:26.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.070+0000 7f2ff8c00700 1 -- 172.21.15.19:0/2707965305 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 46) ==== 8386+0+0 (secure 0 0 0) 0x7f2ffc027ac0 con 0x55ee798f3c40 2024-05-07T02:39:26.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.070+0000 7f302046db80 10 client.26363 fetch_fsmap finished waiting for FSMap version 46 2024-05-07T02:39:26.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.070+0000 7f302046db80 10 client.26363 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-05-07T02:39:26.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.070+0000 7f302046db80 20 client.26363 populate_metadata read hostname 'smithi019' 2024-05-07T02:39:26.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.070+0000 7f302046db80 1 --2- 172.21.15.19:0/2707965305 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x55ee7990db10 0x55ee7990ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:26.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.070+0000 7f302046db80 4 client.26363 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:39:26.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.070+0000 7f302046db80 1 -- 172.21.15.19:0/2707965305 --> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee798c00e0 con 0x55ee7990db10 2024-05-07T02:39:26.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.071+0000 7f3002000700 1 --2- 172.21.15.19:0/2707965305 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x55ee7990db10 0x55ee7990ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:26.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.072+0000 7f2ff8c00700 10 client.26363 ms_handle_connect on v2:172.21.15.27:6836/2583138143 2024-05-07T02:39:26.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.072+0000 7f3002000700 1 --2- 172.21.15.19:0/2707965305 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x55ee7990db10 0x55ee7990ff30 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=67ae0cd588e2a8cf server_cookie=4c0f56aa55c9a167 in_seq=0 out_seq=0 2024-05-07T02:39:26.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.073+0000 7f2ff8c00700 1 -- 172.21.15.19:0/2707965305 <== mds.3 v2:172.21.15.27:6836/2583138143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ee798c00e0 con 0x55ee7990db10 2024-05-07T02:39:26.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.073+0000 7f2ff8c00700 10 client.26363 handle_command_reply: tid=0 2024-05-07T02:39:26.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.073+0000 7f302046db80 1 client.26363 shutdown 2024-05-07T02:39:26.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.073+0000 7f302046db80 1 -- 172.21.15.19:0/2707965305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee798f3c40 msgr2=0x55ee798fab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:26.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.073+0000 7f302046db80 1 --2- 172.21.15.19:0/2707965305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee798f3c40 0x55ee798fab60 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc009c10 tx=0x7f2ffc009750 comp rx=0 tx=0).stop 2024-05-07T02:39:26.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.073+0000 7f302046db80 1 -- 172.21.15.19:0/2707965305 shutdown_connections 2024-05-07T02:39:26.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.073+0000 7f302046db80 1 --2- 172.21.15.19:0/2707965305 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x55ee7990db10 0x55ee7990ff30 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:26.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.073+0000 7f302046db80 1 --2- 172.21.15.19:0/2707965305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee798fdc30 0x55ee798fb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:26.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.073+0000 7f302046db80 1 --2- 172.21.15.19:0/2707965305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee798f3c40 0x55ee798fab60 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:26.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.073+0000 7f302046db80 1 --2- 172.21.15.19:0/2707965305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee798f4590 0x55ee798fb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:26.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.073+0000 7f2ff8c00700 0 client.26363 ms_handle_reset on v2:172.21.15.27:6836/2583138143 2024-05-07T02:39:26.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.073+0000 7f302046db80 1 -- 172.21.15.19:0/2707965305 >> 172.21.15.19:0/2707965305 conn(0x55ee79775e40 msgr2=0x55ee798f0600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:26.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.073+0000 7f302046db80 1 -- 172.21.15.19:0/2707965305 shutdown_connections 2024-05-07T02:39:26.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.073+0000 7f302046db80 1 -- 172.21.15.19:0/2707965305 wait complete. 2024-05-07T02:39:26.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.073+0000 7f302046db80 20 client.26363 trim_cache size 0 max 16384 2024-05-07T02:39:26.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.091+0000 7f302046db80 1 Processor -- start 2024-05-07T02:39:26.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.091+0000 7f302046db80 1 -- start start 2024-05-07T02:39:26.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.091+0000 7f302046db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee798f3c40 0x55ee79909e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:26.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.091+0000 7f302046db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee798f4590 0x55ee798f3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:26.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.091+0000 7f302046db80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee798fdc30 0x55ee798fb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:26.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.091+0000 7f302046db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f300c1ae2f0 con 0x55ee798f3c40 2024-05-07T02:39:26.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.091+0000 7f302046db80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f300c1aff50 con 0x55ee798fdc30 2024-05-07T02:39:26.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.091+0000 7f302046db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f300c1ae9c0 con 0x55ee798f4590 2024-05-07T02:39:26.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.092+0000 7f300b400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee798fdc30 0x55ee798fb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:26.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.092+0000 7f300b400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee798fdc30 0x55ee798fb420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:60320/0 (socket says 172.21.15.19:60320) 2024-05-07T02:39:26.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.092+0000 7f300b400700 1 -- 172.21.15.19:0/1205476364 learned_addr learned my addr 172.21.15.19:0/1205476364 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:39:26.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.092+0000 7f300aa00700 1 --2- 172.21.15.19:0/1205476364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee798f3c40 0x55ee79909e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:26.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.092+0000 7f3002000700 1 --2- 172.21.15.19:0/1205476364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee798f4590 0x55ee798f3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:26.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.092+0000 7f300b400700 1 -- 172.21.15.19:0/1205476364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee798f4590 msgr2=0x55ee798f3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:26.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.092+0000 7f300b400700 1 --2- 172.21.15.19:0/1205476364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee798f4590 0x55ee798f3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:26.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.092+0000 7f300b400700 1 -- 172.21.15.19:0/1205476364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee798f3c40 msgr2=0x55ee79909e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:26.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.092+0000 7f300b400700 1 --2- 172.21.15.19:0/1205476364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee798f3c40 0x55ee79909e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:26.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.092+0000 7f300b400700 1 -- 172.21.15.19:0/1205476364 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee798ea730 con 0x55ee798fdc30 2024-05-07T02:39:26.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.092+0000 7f300b400700 1 --2- 172.21.15.19:0/1205476364 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee798fdc30 0x55ee798fb420 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f30040a9e90 tx=0x7f30040a9e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2952d360541be76f server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:26.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.093+0000 7f2ffb400700 1 -- 172.21.15.19:0/1205476364 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3004005150 con 0x55ee798fdc30 2024-05-07T02:39:26.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.093+0000 7f2ffb400700 1 -- 172.21.15.19:0/1205476364 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30040095c0 con 0x55ee798fdc30 2024-05-07T02:39:26.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.093+0000 7f2ffb400700 1 -- 172.21.15.19:0/1205476364 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30040ae2b0 con 0x55ee798fdc30 2024-05-07T02:39:26.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.094+0000 7f302046db80 1 -- 172.21.15.19:0/1205476364 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee798fdc30 msgr2=0x55ee798fb420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:26.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.094+0000 7f302046db80 1 --2- 172.21.15.19:0/1205476364 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee798fdc30 0x55ee798fb420 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f30040a9e90 tx=0x7f30040a9e00 comp rx=0 tx=0).stop 2024-05-07T02:39:26.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.094+0000 7f302046db80 1 -- 172.21.15.19:0/1205476364 shutdown_connections 2024-05-07T02:39:26.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.094+0000 7f302046db80 1 --2- 172.21.15.19:0/1205476364 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee798fdc30 0x55ee798fb420 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:26.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.094+0000 7f302046db80 1 --2- 172.21.15.19:0/1205476364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee798f3c40 0x55ee79909e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:26.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.094+0000 7f302046db80 1 --2- 172.21.15.19:0/1205476364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee798f4590 0x55ee798f3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:26.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.094+0000 7f302046db80 1 -- 172.21.15.19:0/1205476364 >> 172.21.15.19:0/1205476364 conn(0x55ee79775e40 msgr2=0x55ee798f5530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:26.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.094+0000 7f302046db80 1 -- 172.21.15.19:0/1205476364 shutdown_connections 2024-05-07T02:39:26.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.094+0000 7f302046db80 1 -- 172.21.15.19:0/1205476364 wait complete. 2024-05-07T02:39:26.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.095+0000 7f302046db80 1 -- start start 2024-05-07T02:39:26.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.095+0000 7f302046db80 1 Processor -- start 2024-05-07T02:39:26.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.096+0000 7f302046db80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee798f3c40 0x55ee799133c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:26.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.096+0000 7f302046db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee798f4590 0x55ee79910790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:26.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.096+0000 7f302046db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee798fdc30 0x55ee79910cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:26.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.096+0000 7f302046db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f300c1af7c0 con 0x55ee798fdc30 2024-05-07T02:39:26.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.096+0000 7f302046db80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f300c1af090 con 0x55ee798f3c40 2024-05-07T02:39:26.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.096+0000 7f302046db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f300c1ac2b0 con 0x55ee798f4590 2024-05-07T02:39:26.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.096+0000 7f302046db80 10 client.0 _ll_register_callbacks cb 0x55ee797a9380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:39:26.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.096+0000 7f300aa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee798f3c40 0x55ee799133c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:26.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.096+0000 7f300aa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee798f3c40 0x55ee799133c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:60332/0 (socket says 172.21.15.19:60332) 2024-05-07T02:39:26.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.096+0000 7f300aa00700 1 -- 172.21.15.19:0/4048189699 learned_addr learned my addr 172.21.15.19:0/4048189699 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:39:26.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.096+0000 7f3002000700 1 --2- 172.21.15.19:0/4048189699 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee798f4590 0x55ee79910790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:26.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.096+0000 7f300b400700 1 --2- 172.21.15.19:0/4048189699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee798fdc30 0x55ee79910cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:26.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.096+0000 7f300aa00700 1 -- 172.21.15.19:0/4048189699 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee798f4590 msgr2=0x55ee79910790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:26.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.096+0000 7f300aa00700 1 --2- 172.21.15.19:0/4048189699 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee798f4590 0x55ee79910790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:26.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.096+0000 7f300aa00700 1 -- 172.21.15.19:0/4048189699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee798fdc30 msgr2=0x55ee79910cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:26.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.097+0000 7f300aa00700 1 --2- 172.21.15.19:0/4048189699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee798fdc30 0x55ee79910cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:26.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.097+0000 7f300aa00700 1 -- 172.21.15.19:0/4048189699 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee79911340 con 0x55ee798f3c40 2024-05-07T02:39:26.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.097+0000 7f300b400700 1 --2- 172.21.15.19:0/4048189699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee798fdc30 0x55ee79910cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:39:26.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.097+0000 7f3002000700 1 --2- 172.21.15.19:0/4048189699 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee798f4590 0x55ee79910790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:39:26.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.097+0000 7f300aa00700 1 --2- 172.21.15.19:0/4048189699 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee798f3c40 0x55ee799133c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc002230 tx=0x7f2ffc015e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=315610fccd93160a server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:26.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.097+0000 7f2ff8c00700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:39:26.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.098+0000 7f2ff8c00700 1 -- 172.21.15.19:0/4048189699 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc019040 con 0x55ee798f3c40 2024-05-07T02:39:26.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.098+0000 7f2ff8c00700 1 -- 172.21.15.19:0/4048189699 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9525+0+0 (secure 0 0 0) 0x7f2ffc005c60 con 0x55ee798f3c40 2024-05-07T02:39:26.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.098+0000 7f302046db80 1 -- 172.21.15.19:0/4048189699 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ee79823400 con 0x55ee798f3c40 2024-05-07T02:39:26.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.099+0000 7f2ff8c00700 1 -- 172.21.15.19:0/4048189699 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f2ffc0052e0 con 0x55ee798f3c40 2024-05-07T02:39:26.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.099+0000 7f302046db80 10 client.26369 fetch_fsmap learned FSMap version 46 2024-05-07T02:39:26.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.099+0000 7f302046db80 1 -- 172.21.15.19:0/4048189699 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55ee79977e60 con 0x55ee798f3c40 2024-05-07T02:39:26.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.100+0000 7f2ff8c00700 1 -- 172.21.15.19:0/4048189699 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ee79823400 con 0x55ee798f3c40 2024-05-07T02:39:26.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.100+0000 7f2ff8c00700 1 -- 172.21.15.19:0/4048189699 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 46) ==== 8386+0+0 (secure 0 0 0) 0x7f2ffc00e700 con 0x55ee798f3c40 2024-05-07T02:39:26.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.100+0000 7f302046db80 10 client.26369 fetch_fsmap finished waiting for FSMap version 46 2024-05-07T02:39:26.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.100+0000 7f302046db80 10 client.26369 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-05-07T02:39:26.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.100+0000 7f302046db80 20 client.26369 populate_metadata read hostname 'smithi019' 2024-05-07T02:39:26.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.100+0000 7f302046db80 1 --2- 172.21.15.19:0/4048189699 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x55ee799784e0 0x55ee7997a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:26.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.100+0000 7f302046db80 4 client.26369 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:39:26.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.100+0000 7f302046db80 1 -- 172.21.15.19:0/4048189699 --> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ee7981d4a0 con 0x55ee799784e0 2024-05-07T02:39:26.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.100+0000 7f3002000700 1 --2- 172.21.15.19:0/4048189699 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x55ee799784e0 0x55ee7997a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:26.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.103+0000 7f3002000700 1 --2- 172.21.15.19:0/4048189699 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x55ee799784e0 0x55ee7997a900 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5e5eefdb5c300f59 server_cookie=a60483d8ccc89d90 in_seq=0 out_seq=0 2024-05-07T02:39:26.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.103+0000 7f2ff8c00700 10 client.26369 ms_handle_connect on v2:172.21.15.27:6836/2583138143 2024-05-07T02:39:26.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.104+0000 7f2ff8c00700 1 -- 172.21.15.19:0/4048189699 <== mds.3 v2:172.21.15.27:6836/2583138143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ee7981d4a0 con 0x55ee799784e0 2024-05-07T02:39:26.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.104+0000 7f2ff8c00700 10 client.26369 handle_command_reply: tid=0 2024-05-07T02:39:26.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.104+0000 7f302046db80 1 client.26369 shutdown 2024-05-07T02:39:26.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.104+0000 7f302046db80 1 -- 172.21.15.19:0/4048189699 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee798f3c40 msgr2=0x55ee799133c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:26.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.104+0000 7f302046db80 1 --2- 172.21.15.19:0/4048189699 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee798f3c40 0x55ee799133c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc002230 tx=0x7f2ffc015e10 comp rx=0 tx=0).stop 2024-05-07T02:39:26.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.105+0000 7f302046db80 1 -- 172.21.15.19:0/4048189699 shutdown_connections 2024-05-07T02:39:26.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.105+0000 7f302046db80 1 --2- 172.21.15.19:0/4048189699 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x55ee799784e0 0x55ee7997a900 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:26.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.105+0000 7f302046db80 1 --2- 172.21.15.19:0/4048189699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ee798fdc30 0x55ee79910cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:26.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.105+0000 7f302046db80 1 --2- 172.21.15.19:0/4048189699 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ee798f3c40 0x55ee799133c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:26.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.105+0000 7f2ff8c00700 0 client.26369 ms_handle_reset on v2:172.21.15.27:6836/2583138143 2024-05-07T02:39:26.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.105+0000 7f302046db80 1 --2- 172.21.15.19:0/4048189699 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee798f4590 0x55ee79910790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:26.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.105+0000 7f302046db80 1 -- 172.21.15.19:0/4048189699 >> 172.21.15.19:0/4048189699 conn(0x55ee79775e40 msgr2=0x55ee798f2da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:26.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.106+0000 7f302046db80 1 -- 172.21.15.19:0/4048189699 shutdown_connections 2024-05-07T02:39:26.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.107+0000 7f302046db80 1 -- 172.21.15.19:0/4048189699 wait complete. 2024-05-07T02:39:26.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.107+0000 7f302046db80 20 client.26369 trim_cache size 0 max 16384 2024-05-07T02:39:26.109 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:39:26.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.109+0000 7f300be00700 1 -- 172.21.15.19:0/756841766 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f2ff00b29f0 msgr2=0x7f2ff00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:26.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.109+0000 7f300be00700 1 --2- 172.21.15.19:0/756841766 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f2ff00b29f0 0x7f2ff00b4f00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2ff40097c0 tx=0x7f2ff4009260 comp rx=0 tx=0).stop 2024-05-07T02:39:26.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.109+0000 7f300be00700 1 -- 172.21.15.19:0/756841766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f300c109680 msgr2=0x7f300c19aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:26.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.109+0000 7f300be00700 1 --2- 172.21.15.19:0/756841766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f300c109680 0x7f300c19aab0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3004009990 tx=0x7f3004004270 comp rx=0 tx=0).stop 2024-05-07T02:39:26.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.110+0000 7f300be00700 1 -- 172.21.15.19:0/756841766 shutdown_connections 2024-05-07T02:39:26.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.110+0000 7f300be00700 1 --2- 172.21.15.19:0/756841766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f300c109680 0x7f300c19aab0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:26.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.110+0000 7f300be00700 1 --2- 172.21.15.19:0/756841766 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f300c104400 0x7f300c1a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:26.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.110+0000 7f300be00700 1 --2- 172.21.15.19:0/756841766 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f2ff00b29f0 0x7f2ff00b4f00 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:26.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.110+0000 7f300be00700 1 --2- 172.21.15.19:0/756841766 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f300c108d30 0x7f300c1a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:26.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.110+0000 7f300be00700 1 -- 172.21.15.19:0/756841766 >> 172.21.15.19:0/756841766 conn(0x7f300c079d10 msgr2=0x7f300c101dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:26.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.112+0000 7f300be00700 1 -- 172.21.15.19:0/756841766 shutdown_connections 2024-05-07T02:39:26.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:26.112+0000 7f300be00700 1 -- 172.21.15.19:0/756841766 wait complete. 2024-05-07T02:39:26.195 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:25 smithi019 ceph-mon[29781]: pgmap v232: 129 pgs: 129 active+clean; 179 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 608 KiB/s wr, 35 op/s 2024-05-07T02:39:26.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:25 smithi064 ceph-mon[34699]: pgmap v232: 129 pgs: 129 active+clean; 179 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 608 KiB/s wr, 35 op/s 2024-05-07T02:39:26.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:25 smithi027 ceph-mon[34398]: pgmap v232: 129 pgs: 129 active+clean; 179 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 608 KiB/s wr, 35 op/s 2024-05-07T02:39:27.065 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-07T02:39:27.527 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:39:28.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:27 smithi064 ceph-mon[34699]: pgmap v233: 129 pgs: 129 active+clean; 178 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 889 KiB/s wr, 55 op/s 2024-05-07T02:39:28.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:27 smithi019 ceph-mon[29781]: pgmap v233: 129 pgs: 129 active+clean; 178 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 889 KiB/s wr, 55 op/s 2024-05-07T02:39:28.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:27 smithi027 ceph-mon[34398]: pgmap v233: 129 pgs: 129 active+clean; 178 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 889 KiB/s wr, 55 op/s 2024-05-07T02:39:29.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.975+0000 7fd374e00700 1 -- 172.21.15.19:0/111698773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd370075b00 msgr2=0x7fd370075f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:29.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.975+0000 7fd374e00700 1 --2- 172.21.15.19:0/111698773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd370075b00 0x7fd370075f10 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd3600098f0 tx=0x7fd360009c00 comp rx=0 tx=0).stop 2024-05-07T02:39:29.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.975+0000 7fd374e00700 1 -- 172.21.15.19:0/111698773 shutdown_connections 2024-05-07T02:39:29.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.975+0000 7fd374e00700 1 --2- 172.21.15.19:0/111698773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd370110c20 0x7fd370113040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:29.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.975+0000 7fd374e00700 1 --2- 172.21.15.19:0/111698773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd370075b00 0x7fd370075f10 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:29.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.975+0000 7fd374e00700 1 --2- 172.21.15.19:0/111698773 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd370076450 0x7fd3701106e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:29.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.975+0000 7fd374e00700 1 -- 172.21.15.19:0/111698773 >> 172.21.15.19:0/111698773 conn(0x7fd37007af70 msgr2=0x7fd37007b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:29.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.975+0000 7fd374e00700 1 -- 172.21.15.19:0/111698773 shutdown_connections 2024-05-07T02:39:29.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.975+0000 7fd374e00700 1 -- 172.21.15.19:0/111698773 wait complete. 2024-05-07T02:39:29.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.977+0000 7fd374e00700 1 Processor -- start 2024-05-07T02:39:29.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.977+0000 7fd374e00700 1 -- start start 2024-05-07T02:39:29.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.977+0000 7fd374e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd370075b00 0x7fd37019d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:29.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.977+0000 7fd374e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd370076450 0x7fd37019d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:29.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.977+0000 7fd374e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd370110c20 0x7fd3701a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:29.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.977+0000 7fd374e00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd3701166e0 con 0x7fd370075b00 2024-05-07T02:39:29.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.977+0000 7fd374e00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd370116560 con 0x7fd370110c20 2024-05-07T02:39:29.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.977+0000 7fd374e00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd370116860 con 0x7fd370076450 2024-05-07T02:39:29.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.978+0000 7fd36d400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd370076450 0x7fd37019d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:29.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.978+0000 7fd36d400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd370076450 0x7fd37019d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:45016/0 (socket says 172.21.15.19:45016) 2024-05-07T02:39:29.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.978+0000 7fd36d400700 1 -- 172.21.15.19:0/3828921189 learned_addr learned my addr 172.21.15.19:0/3828921189 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:39:29.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.978+0000 7fd36de00700 1 --2- 172.21.15.19:0/3828921189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd370075b00 0x7fd37019d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:29.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.978+0000 7fd36e800700 1 --2- 172.21.15.19:0/3828921189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd370110c20 0x7fd3701a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:29.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.978+0000 7fd36d400700 1 -- 172.21.15.19:0/3828921189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd370110c20 msgr2=0x7fd3701a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:29.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.978+0000 7fd36d400700 1 --2- 172.21.15.19:0/3828921189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd370110c20 0x7fd3701a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:29.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.978+0000 7fd36d400700 1 -- 172.21.15.19:0/3828921189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd370075b00 msgr2=0x7fd37019d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:29.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.978+0000 7fd36d400700 1 --2- 172.21.15.19:0/3828921189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd370075b00 0x7fd37019d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:29.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.978+0000 7fd36d400700 1 -- 172.21.15.19:0/3828921189 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3701a1fe0 con 0x7fd370076450 2024-05-07T02:39:29.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.978+0000 7fd36d400700 1 --2- 172.21.15.19:0/3828921189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd370076450 0x7fd37019d6b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd35c00e9a0 tx=0x7fd35c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:29.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.979+0000 7fd366a00700 1 -- 172.21.15.19:0/3828921189 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c00ca10 con 0x7fd370076450 2024-05-07T02:39:29.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.979+0000 7fd374e00700 1 -- 172.21.15.19:0/3828921189 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3701a22d0 con 0x7fd370076450 2024-05-07T02:39:29.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.979+0000 7fd366a00700 1 -- 172.21.15.19:0/3828921189 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd35c00cbb0 con 0x7fd370076450 2024-05-07T02:39:29.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.979+0000 7fd366a00700 1 -- 172.21.15.19:0/3828921189 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c018680 con 0x7fd370076450 2024-05-07T02:39:29.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.979+0000 7fd374e00700 1 -- 172.21.15.19:0/3828921189 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3701a9bf0 con 0x7fd370076450 2024-05-07T02:39:29.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.983+0000 7fd366a00700 1 -- 172.21.15.19:0/3828921189 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd35c010430 con 0x7fd370076450 2024-05-07T02:39:29.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.984+0000 7fd3839cfb80 1 Processor -- start 2024-05-07T02:39:29.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.984+0000 7fd3839cfb80 1 -- start start 2024-05-07T02:39:29.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.984+0000 7fd366a00700 1 --2- 172.21.15.19:0/3828921189 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fd3540b29f0 0x7fd3540b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:29.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.985+0000 7fd366a00700 1 -- 172.21.15.19:0/3828921189 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9525+0+0 (secure 0 0 0) 0x7fd35c0e2bc0 con 0x7fd370076450 2024-05-07T02:39:29.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.985+0000 7fd3839cfb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562cfa597470 0x562cfa597880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:29.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.985+0000 7fd36e800700 1 --2- 172.21.15.19:0/3828921189 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fd3540b29f0 0x7fd3540b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:29.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.985+0000 7fd3839cfb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562cfa591620 0x562cfa591a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:29.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.985+0000 7fd3839cfb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cfa591f70 0x562cfa59e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:29.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.985+0000 7fd3839cfb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562cfa4b3a20 con 0x562cfa591f70 2024-05-07T02:39:29.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.985+0000 7fd3839cfb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562cfa451760 con 0x562cfa597470 2024-05-07T02:39:29.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.985+0000 7fd3839cfb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562cfa4af9e0 con 0x562cfa591620 2024-05-07T02:39:29.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.985+0000 7fd36de00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562cfa597470 0x562cfa597880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:29.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.985+0000 7fd36de00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562cfa597470 0x562cfa597880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:56222/0 (socket says 172.21.15.19:56222) 2024-05-07T02:39:29.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.985+0000 7fd36de00700 1 -- 172.21.15.19:0/1942210253 learned_addr learned my addr 172.21.15.19:0/1942210253 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:39:29.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.985+0000 7fd36d400700 1 --2- 172.21.15.19:0/1942210253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562cfa591620 0x562cfa591a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:29.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.985+0000 7fd36e800700 1 --2- 172.21.15.19:0/1942210253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cfa591f70 0x562cfa59e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:29.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.986+0000 7fd36de00700 1 -- 172.21.15.19:0/1942210253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562cfa591620 msgr2=0x562cfa591a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:29.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.986+0000 7fd36de00700 1 --2- 172.21.15.19:0/1942210253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562cfa591620 0x562cfa591a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:29.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.986+0000 7fd36de00700 1 -- 172.21.15.19:0/1942210253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cfa591f70 msgr2=0x562cfa59e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:29.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.986+0000 7fd36de00700 1 --2- 172.21.15.19:0/1942210253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cfa591f70 0x562cfa59e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:29.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.986+0000 7fd36de00700 1 -- 172.21.15.19:0/1942210253 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cfa4008d0 con 0x562cfa597470 2024-05-07T02:39:29.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.986+0000 7fd36e800700 1 --2- 172.21.15.19:0/3828921189 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fd3540b29f0 0x7fd3540b4f00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd36800f870 tx=0x7fd36800fc30 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:29.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.986+0000 7fd36de00700 1 --2- 172.21.15.19:0/1942210253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562cfa597470 0x562cfa597880 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd3600098f0 tx=0x7fd36000bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a6f7a75cfe53bf server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:29.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.986+0000 7fd35b400700 1 -- 172.21.15.19:0/1942210253 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd360026070 con 0x562cfa597470 2024-05-07T02:39:29.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.986+0000 7fd35b400700 1 -- 172.21.15.19:0/1942210253 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd36000f910 con 0x562cfa597470 2024-05-07T02:39:29.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.987+0000 7fd35b400700 1 -- 172.21.15.19:0/1942210253 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd360003680 con 0x562cfa597470 2024-05-07T02:39:29.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.987+0000 7fd3839cfb80 1 -- 172.21.15.19:0/1942210253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562cfa597470 msgr2=0x562cfa597880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:29.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.987+0000 7fd3839cfb80 1 --2- 172.21.15.19:0/1942210253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562cfa597470 0x562cfa597880 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd3600098f0 tx=0x7fd36000bf60 comp rx=0 tx=0).stop 2024-05-07T02:39:29.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.988+0000 7fd3839cfb80 1 -- 172.21.15.19:0/1942210253 shutdown_connections 2024-05-07T02:39:29.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.988+0000 7fd3839cfb80 1 --2- 172.21.15.19:0/1942210253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cfa591f70 0x562cfa59e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:29.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.988+0000 7fd3839cfb80 1 --2- 172.21.15.19:0/1942210253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562cfa597470 0x562cfa597880 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:29.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.988+0000 7fd3839cfb80 1 --2- 172.21.15.19:0/1942210253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562cfa591620 0x562cfa591a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:29.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.988+0000 7fd3839cfb80 1 -- 172.21.15.19:0/1942210253 >> 172.21.15.19:0/1942210253 conn(0x562cfa413e40 msgr2=0x562cfa58c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:29.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.988+0000 7fd3839cfb80 1 -- 172.21.15.19:0/1942210253 shutdown_connections 2024-05-07T02:39:29.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.988+0000 7fd3839cfb80 1 -- 172.21.15.19:0/1942210253 wait complete. 2024-05-07T02:39:29.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.988+0000 7fd3839cfb80 1 -- start start 2024-05-07T02:39:29.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.989+0000 7fd3839cfb80 1 Processor -- start 2024-05-07T02:39:29.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.989+0000 7fd3839cfb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cfa591620 0x562cfa59b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:29.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.989+0000 7fd3839cfb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562cfa591f70 0x562cfa59bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:29.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.989+0000 7fd3839cfb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562cfa597470 0x562cfa59c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:29.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.989+0000 7fd3839cfb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562cfa59e7e0 con 0x562cfa591620 2024-05-07T02:39:29.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.989+0000 7fd3839cfb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562cfa419b20 con 0x562cfa591f70 2024-05-07T02:39:29.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.989+0000 7fd3839cfb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562cfa59e960 con 0x562cfa597470 2024-05-07T02:39:29.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.989+0000 7fd3839cfb80 10 client.0 _ll_register_callbacks cb 0x562cfa447380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:39:29.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.990+0000 7fd36de00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cfa591620 0x562cfa59b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:29.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.990+0000 7fd36d400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562cfa591f70 0x562cfa59bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:29.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.990+0000 7fd36d400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562cfa591f70 0x562cfa59bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:56232/0 (socket says 172.21.15.19:56232) 2024-05-07T02:39:29.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.990+0000 7fd36d400700 1 -- 172.21.15.19:0/3028139825 learned_addr learned my addr 172.21.15.19:0/3028139825 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:39:29.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.990+0000 7fd36e800700 1 --2- 172.21.15.19:0/3028139825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562cfa597470 0x562cfa59c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:29.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.990+0000 7fd36d400700 1 -- 172.21.15.19:0/3028139825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562cfa597470 msgr2=0x562cfa59c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:29.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.990+0000 7fd36d400700 1 --2- 172.21.15.19:0/3028139825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562cfa597470 0x562cfa59c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:29.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.990+0000 7fd36d400700 1 -- 172.21.15.19:0/3028139825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cfa591620 msgr2=0x562cfa59b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:29.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.990+0000 7fd36d400700 1 --2- 172.21.15.19:0/3028139825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cfa591620 0x562cfa59b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:29.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.990+0000 7fd36d400700 1 -- 172.21.15.19:0/3028139825 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cfa479420 con 0x562cfa591f70 2024-05-07T02:39:29.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.990+0000 7fd36d400700 1 --2- 172.21.15.19:0/3028139825 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562cfa591f70 0x562cfa59bb40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd35c00e970 tx=0x7fd35c0a9f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b118aad8d099e8a server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:29.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.990+0000 7fd34fe00700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:39:29.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.990+0000 7fd34fe00700 1 -- 172.21.15.19:0/3028139825 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c0a5040 con 0x562cfa591f70 2024-05-07T02:39:29.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.991+0000 7fd3839cfb80 1 -- 172.21.15.19:0/3028139825 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd36000f910 con 0x562cfa591f70 2024-05-07T02:39:29.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.991+0000 7fd34fe00700 1 -- 172.21.15.19:0/3028139825 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9525+0+0 (secure 0 0 0) 0x7fd35c0cc2d0 con 0x562cfa591f70 2024-05-07T02:39:29.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.991+0000 7fd34fe00700 1 -- 172.21.15.19:0/3028139825 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fd35c0a93a0 con 0x562cfa591f70 2024-05-07T02:39:29.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.991+0000 7fd3839cfb80 10 client.26381 fetch_fsmap learned FSMap version 46 2024-05-07T02:39:29.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.991+0000 7fd3839cfb80 1 -- 172.21.15.19:0/3028139825 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x562cfa5a9460 con 0x562cfa591f70 2024-05-07T02:39:29.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.991+0000 7fd34fe00700 1 -- 172.21.15.19:0/3028139825 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd36000f910 con 0x562cfa591f70 2024-05-07T02:39:29.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.991+0000 7fd34fe00700 1 -- 172.21.15.19:0/3028139825 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 46) ==== 8386+0+0 (secure 0 0 0) 0x7fd35c010720 con 0x562cfa591f70 2024-05-07T02:39:29.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.991+0000 7fd3839cfb80 10 client.26381 fetch_fsmap finished waiting for FSMap version 46 2024-05-07T02:39:29.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.991+0000 7fd3839cfb80 10 client.26381 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-07T02:39:29.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.991+0000 7fd3839cfb80 20 client.26381 populate_metadata read hostname 'smithi019' 2024-05-07T02:39:29.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.991+0000 7fd3839cfb80 1 --2- 172.21.15.19:0/3028139825 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x562cfa5abaa0 0x562cfa5adec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:29.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.992+0000 7fd3839cfb80 4 client.26381 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:39:29.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.992+0000 7fd3839cfb80 1 -- 172.21.15.19:0/3028139825 --> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562cfa55e0e0 con 0x562cfa5abaa0 2024-05-07T02:39:29.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.992+0000 7fd36de00700 1 --2- 172.21.15.19:0/3028139825 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x562cfa5abaa0 0x562cfa5adec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:29.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.992+0000 7fd36de00700 1 --2- 172.21.15.19:0/3028139825 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x562cfa5abaa0 0x562cfa5adec0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=82b7ff36f0cafae1 server_cookie=f5c084f80db525e7 in_seq=0 out_seq=0 2024-05-07T02:39:29.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.992+0000 7fd34fe00700 10 client.26381 ms_handle_connect on v2:172.21.15.64:6838/1583120004 2024-05-07T02:39:29.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.993+0000 7fd34fe00700 1 -- 172.21.15.19:0/3028139825 <== mds.2 v2:172.21.15.64:6838/1583120004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562cfa55e0e0 con 0x562cfa5abaa0 2024-05-07T02:39:29.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.993+0000 7fd34fe00700 10 client.26381 handle_command_reply: tid=0 2024-05-07T02:39:29.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.993+0000 7fd3839cfb80 1 client.26381 shutdown 2024-05-07T02:39:29.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.994+0000 7fd3839cfb80 1 -- 172.21.15.19:0/3028139825 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562cfa591f70 msgr2=0x562cfa59bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:29.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.994+0000 7fd3839cfb80 1 --2- 172.21.15.19:0/3028139825 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562cfa591f70 0x562cfa59bb40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd35c00e970 tx=0x7fd35c0a9f90 comp rx=0 tx=0).stop 2024-05-07T02:39:29.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.994+0000 7fd3839cfb80 1 -- 172.21.15.19:0/3028139825 shutdown_connections 2024-05-07T02:39:29.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.994+0000 7fd3839cfb80 1 --2- 172.21.15.19:0/3028139825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562cfa597470 0x562cfa59c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:29.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.994+0000 7fd3839cfb80 1 --2- 172.21.15.19:0/3028139825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cfa591620 0x562cfa59b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:29.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.994+0000 7fd3839cfb80 1 --2- 172.21.15.19:0/3028139825 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x562cfa5abaa0 0x562cfa5adec0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:29.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.994+0000 7fd3839cfb80 1 --2- 172.21.15.19:0/3028139825 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562cfa591f70 0x562cfa59bb40 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:30.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.994+0000 7fd3839cfb80 1 -- 172.21.15.19:0/3028139825 >> 172.21.15.19:0/3028139825 conn(0x562cfa413e40 msgr2=0x562cfa590610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:30.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.994+0000 7fd34fe00700 0 client.26381 ms_handle_reset on v2:172.21.15.64:6838/1583120004 2024-05-07T02:39:30.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.994+0000 7fd3839cfb80 1 -- 172.21.15.19:0/3028139825 shutdown_connections 2024-05-07T02:39:30.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.994+0000 7fd3839cfb80 1 -- 172.21.15.19:0/3028139825 wait complete. 2024-05-07T02:39:30.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:29.994+0000 7fd3839cfb80 20 client.26381 trim_cache size 0 max 16384 2024-05-07T02:39:30.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.011+0000 7fd3839cfb80 1 Processor -- start 2024-05-07T02:39:30.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.011+0000 7fd3839cfb80 1 -- start start 2024-05-07T02:39:30.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.011+0000 7fd3839cfb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cfa591620 0x562cfa59bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:30.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.011+0000 7fd3839cfb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562cfa591f70 0x562cfa595d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:30.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.011+0000 7fd3839cfb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562cfa597470 0x562cfa5868d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:30.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.011+0000 7fd3839cfb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd3701b2970 con 0x562cfa591620 2024-05-07T02:39:30.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.011+0000 7fd3839cfb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd3701b45d0 con 0x562cfa597470 2024-05-07T02:39:30.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.011+0000 7fd3839cfb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd3701b3040 con 0x562cfa591f70 2024-05-07T02:39:30.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.012+0000 7fd36de00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cfa591620 0x562cfa59bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:30.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.012+0000 7fd36de00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cfa591620 0x562cfa59bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39554/0 (socket says 172.21.15.19:39554) 2024-05-07T02:39:30.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.012+0000 7fd36de00700 1 -- 172.21.15.19:0/1308946887 learned_addr learned my addr 172.21.15.19:0/1308946887 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:39:30.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.012+0000 7fd36d400700 1 --2- 172.21.15.19:0/1308946887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562cfa591f70 0x562cfa595d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:30.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.012+0000 7fd36e800700 1 --2- 172.21.15.19:0/1308946887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562cfa597470 0x562cfa5868d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:30.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.012+0000 7fd36e800700 1 -- 172.21.15.19:0/1308946887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562cfa591f70 msgr2=0x562cfa595d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:30.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.012+0000 7fd36e800700 1 --2- 172.21.15.19:0/1308946887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562cfa591f70 0x562cfa595d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:30.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.012+0000 7fd36e800700 1 -- 172.21.15.19:0/1308946887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cfa591620 msgr2=0x562cfa59bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:30.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.012+0000 7fd36e800700 1 --2- 172.21.15.19:0/1308946887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cfa591620 0x562cfa59bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:30.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.012+0000 7fd36e800700 1 -- 172.21.15.19:0/1308946887 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cfa5a86a0 con 0x562cfa597470 2024-05-07T02:39:30.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.012+0000 7fd36d400700 1 --2- 172.21.15.19:0/1308946887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562cfa591f70 0x562cfa595d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:39:30.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.012+0000 7fd36de00700 1 --2- 172.21.15.19:0/1308946887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cfa591620 0x562cfa59bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:39:30.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.013+0000 7fd36e800700 1 --2- 172.21.15.19:0/1308946887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562cfa597470 0x562cfa5868d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd368018640 tx=0x7fd36801f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a7e3c7821713c4a server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:30.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.013+0000 7fd35aa00700 1 -- 172.21.15.19:0/1308946887 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd368018de0 con 0x562cfa597470 2024-05-07T02:39:30.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.013+0000 7fd35aa00700 1 -- 172.21.15.19:0/1308946887 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd368004020 con 0x562cfa597470 2024-05-07T02:39:30.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.013+0000 7fd35aa00700 1 -- 172.21.15.19:0/1308946887 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd368008830 con 0x562cfa597470 2024-05-07T02:39:30.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.014+0000 7fd3839cfb80 1 -- 172.21.15.19:0/1308946887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562cfa597470 msgr2=0x562cfa5868d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:30.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.014+0000 7fd3839cfb80 1 --2- 172.21.15.19:0/1308946887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562cfa597470 0x562cfa5868d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd368018640 tx=0x7fd36801f000 comp rx=0 tx=0).stop 2024-05-07T02:39:30.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.014+0000 7fd3839cfb80 1 -- 172.21.15.19:0/1308946887 shutdown_connections 2024-05-07T02:39:30.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.014+0000 7fd3839cfb80 1 --2- 172.21.15.19:0/1308946887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562cfa597470 0x562cfa5868d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:30.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.014+0000 7fd3839cfb80 1 --2- 172.21.15.19:0/1308946887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cfa591620 0x562cfa59bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:30.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.014+0000 7fd3839cfb80 1 --2- 172.21.15.19:0/1308946887 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562cfa591f70 0x562cfa595d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:30.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.014+0000 7fd3839cfb80 1 -- 172.21.15.19:0/1308946887 >> 172.21.15.19:0/1308946887 conn(0x562cfa413e40 msgr2=0x562cfa5861c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:30.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.014+0000 7fd3839cfb80 1 -- 172.21.15.19:0/1308946887 shutdown_connections 2024-05-07T02:39:30.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.014+0000 7fd3839cfb80 1 -- 172.21.15.19:0/1308946887 wait complete. 2024-05-07T02:39:30.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.015+0000 7fd3839cfb80 1 -- start start 2024-05-07T02:39:30.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.015+0000 7fd3839cfb80 1 Processor -- start 2024-05-07T02:39:30.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.016+0000 7fd3839cfb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562cfa591620 0x562cfa5a3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:30.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.016+0000 7fd3839cfb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cfa591f70 0x562cfa5b26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:30.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.016+0000 7fd3839cfb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562cfa597470 0x562cfa5b2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:30.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.016+0000 7fd3839cfb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd3701b3e40 con 0x562cfa591f70 2024-05-07T02:39:30.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.016+0000 7fd3839cfb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd3701b3710 con 0x562cfa591620 2024-05-07T02:39:30.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.016+0000 7fd3839cfb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd3701b0930 con 0x562cfa597470 2024-05-07T02:39:30.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.016+0000 7fd3839cfb80 10 client.0 _ll_register_callbacks cb 0x562cfa447380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:39:30.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.016+0000 7fd36de00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562cfa591620 0x562cfa5a3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:30.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.016+0000 7fd36de00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562cfa591620 0x562cfa5a3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:56262/0 (socket says 172.21.15.19:56262) 2024-05-07T02:39:30.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.016+0000 7fd36de00700 1 -- 172.21.15.19:0/1242483503 learned_addr learned my addr 172.21.15.19:0/1242483503 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:39:30.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.016+0000 7fd36e800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562cfa597470 0x562cfa5b2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:30.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.016+0000 7fd36d400700 1 --2- 172.21.15.19:0/1242483503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cfa591f70 0x562cfa5b26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:30.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.016+0000 7fd36de00700 1 -- 172.21.15.19:0/1242483503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562cfa597470 msgr2=0x562cfa5b2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:30.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.016+0000 7fd36de00700 1 --2- 172.21.15.19:0/1242483503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562cfa597470 0x562cfa5b2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:30.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.016+0000 7fd36de00700 1 -- 172.21.15.19:0/1242483503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cfa591f70 msgr2=0x562cfa5b26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:30.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.016+0000 7fd36de00700 1 --2- 172.21.15.19:0/1242483503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cfa591f70 0x562cfa5b26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:30.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.016+0000 7fd36de00700 1 -- 172.21.15.19:0/1242483503 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cfa5a45d0 con 0x562cfa591620 2024-05-07T02:39:30.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.017+0000 7fd36d400700 1 --2- 172.21.15.19:0/1242483503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cfa591f70 0x562cfa5b26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:39:30.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.017+0000 7fd36de00700 1 --2- 172.21.15.19:0/1242483503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562cfa591620 0x562cfa5a3df0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x562cfa596ff0 tx=0x7fd360005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56dd6b3969e0cad2 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:30.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.017+0000 7fd34fe00700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:39:30.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.017+0000 7fd34fe00700 1 -- 172.21.15.19:0/1242483503 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd360038060 con 0x562cfa591620 2024-05-07T02:39:30.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.017+0000 7fd34fe00700 1 -- 172.21.15.19:0/1242483503 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9525+0+0 (secure 0 0 0) 0x7fd360005610 con 0x562cfa591620 2024-05-07T02:39:30.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.017+0000 7fd3839cfb80 1 -- 172.21.15.19:0/1242483503 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd368004020 con 0x562cfa591620 2024-05-07T02:39:30.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.017+0000 7fd34fe00700 1 -- 172.21.15.19:0/1242483503 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fd36003f050 con 0x562cfa591620 2024-05-07T02:39:30.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.017+0000 7fd3839cfb80 10 client.26387 fetch_fsmap learned FSMap version 46 2024-05-07T02:39:30.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.018+0000 7fd3839cfb80 1 -- 172.21.15.19:0/1242483503 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x562cfa615c10 con 0x562cfa591620 2024-05-07T02:39:30.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.018+0000 7fd34fe00700 1 -- 172.21.15.19:0/1242483503 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd368004020 con 0x562cfa591620 2024-05-07T02:39:30.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.018+0000 7fd34fe00700 1 -- 172.21.15.19:0/1242483503 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 46) ==== 8386+0+0 (secure 0 0 0) 0x7fd360020070 con 0x562cfa591620 2024-05-07T02:39:30.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.018+0000 7fd3839cfb80 10 client.26387 fetch_fsmap finished waiting for FSMap version 46 2024-05-07T02:39:30.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.018+0000 7fd3839cfb80 10 client.26387 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-07T02:39:30.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.018+0000 7fd3839cfb80 20 client.26387 populate_metadata read hostname 'smithi019' 2024-05-07T02:39:30.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.018+0000 7fd3839cfb80 1 --2- 172.21.15.19:0/1242483503 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x562cfa615ef0 0x562cfa618310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:30.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.018+0000 7fd3839cfb80 4 client.26387 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:39:30.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.018+0000 7fd3839cfb80 1 -- 172.21.15.19:0/1242483503 --> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562cfa4c1400 con 0x562cfa615ef0 2024-05-07T02:39:30.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.019+0000 7fd36d400700 1 --2- 172.21.15.19:0/1242483503 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x562cfa615ef0 0x562cfa618310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:30.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.019+0000 7fd36d400700 1 --2- 172.21.15.19:0/1242483503 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x562cfa615ef0 0x562cfa618310 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=78d2f16f5e879ce9 server_cookie=847295b66bbdbc02 in_seq=0 out_seq=0 2024-05-07T02:39:30.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.019+0000 7fd34fe00700 10 client.26387 ms_handle_connect on v2:172.21.15.64:6838/1583120004 2024-05-07T02:39:30.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.019+0000 7fd34fe00700 1 -- 172.21.15.19:0/1242483503 <== mds.2 v2:172.21.15.64:6838/1583120004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562cfa4c1400 con 0x562cfa615ef0 2024-05-07T02:39:30.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.019+0000 7fd34fe00700 10 client.26387 handle_command_reply: tid=0 2024-05-07T02:39:30.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.019+0000 7fd3839cfb80 1 client.26387 shutdown 2024-05-07T02:39:30.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.020+0000 7fd3839cfb80 1 -- 172.21.15.19:0/1242483503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562cfa591620 msgr2=0x562cfa5a3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:30.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.020+0000 7fd3839cfb80 1 --2- 172.21.15.19:0/1242483503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562cfa591620 0x562cfa5a3df0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x562cfa596ff0 tx=0x7fd360005a20 comp rx=0 tx=0).stop 2024-05-07T02:39:30.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.020+0000 7fd3839cfb80 1 -- 172.21.15.19:0/1242483503 shutdown_connections 2024-05-07T02:39:30.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.020+0000 7fd3839cfb80 1 --2- 172.21.15.19:0/1242483503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562cfa597470 0x562cfa5b2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:30.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.020+0000 7fd3839cfb80 1 --2- 172.21.15.19:0/1242483503 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x562cfa615ef0 0x562cfa618310 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:30.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.020+0000 7fd3839cfb80 1 --2- 172.21.15.19:0/1242483503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562cfa591620 0x562cfa5a3df0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:30.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.020+0000 7fd3839cfb80 1 --2- 172.21.15.19:0/1242483503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562cfa591f70 0x562cfa5b26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:30.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.020+0000 7fd34fe00700 0 client.26387 ms_handle_reset on v2:172.21.15.64:6838/1583120004 2024-05-07T02:39:30.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.020+0000 7fd3839cfb80 1 -- 172.21.15.19:0/1242483503 >> 172.21.15.19:0/1242483503 conn(0x562cfa413e40 msgr2=0x562cfa59bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:30.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.020+0000 7fd3839cfb80 1 -- 172.21.15.19:0/1242483503 shutdown_connections 2024-05-07T02:39:30.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.020+0000 7fd3839cfb80 1 -- 172.21.15.19:0/1242483503 wait complete. 2024-05-07T02:39:30.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.020+0000 7fd3839cfb80 20 client.26387 trim_cache size 0 max 16384 2024-05-07T02:39:30.027 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:39:30.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.021+0000 7fd374e00700 1 -- 172.21.15.19:0/3828921189 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fd3540b29f0 msgr2=0x7fd3540b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:30.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.021+0000 7fd374e00700 1 --2- 172.21.15.19:0/3828921189 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fd3540b29f0 0x7fd3540b4f00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd36800f870 tx=0x7fd36800fc30 comp rx=0 tx=0).stop 2024-05-07T02:39:30.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.021+0000 7fd374e00700 1 -- 172.21.15.19:0/3828921189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd370076450 msgr2=0x7fd37019d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:30.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.021+0000 7fd374e00700 1 --2- 172.21.15.19:0/3828921189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd370076450 0x7fd37019d6b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd35c00e9a0 tx=0x7fd35c00ecb0 comp rx=0 tx=0).stop 2024-05-07T02:39:30.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.021+0000 7fd374e00700 1 -- 172.21.15.19:0/3828921189 shutdown_connections 2024-05-07T02:39:30.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.021+0000 7fd374e00700 1 --2- 172.21.15.19:0/3828921189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd370110c20 0x7fd3701a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:30.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.021+0000 7fd374e00700 1 --2- 172.21.15.19:0/3828921189 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fd3540b29f0 0x7fd3540b4f00 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:30.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.021+0000 7fd374e00700 1 --2- 172.21.15.19:0/3828921189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd370075b00 0x7fd37019d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:30.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.021+0000 7fd374e00700 1 --2- 172.21.15.19:0/3828921189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd370076450 0x7fd37019d6b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:30.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.021+0000 7fd374e00700 1 -- 172.21.15.19:0/3828921189 >> 172.21.15.19:0/3828921189 conn(0x7fd37007af70 msgr2=0x7fd3701011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:30.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.021+0000 7fd374e00700 1 -- 172.21.15.19:0/3828921189 shutdown_connections 2024-05-07T02:39:30.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:30.021+0000 7fd374e00700 1 -- 172.21.15.19:0/3828921189 wait complete. 2024-05-07T02:39:30.029 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:29 smithi019 ceph-mon[29781]: pgmap v234: 129 pgs: 129 active+clean; 178 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 540 KiB/s wr, 35 op/s 2024-05-07T02:39:30.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:29 smithi064 ceph-mon[34699]: pgmap v234: 129 pgs: 129 active+clean; 178 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 540 KiB/s wr, 35 op/s 2024-05-07T02:39:30.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:29 smithi027 ceph-mon[34398]: pgmap v234: 129 pgs: 129 active+clean; 178 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 540 KiB/s wr, 35 op/s 2024-05-07T02:39:31.049 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-07T02:39:31.533 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:39:32.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:31 smithi064 ceph-mon[34699]: pgmap v235: 129 pgs: 129 active+clean; 178 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 540 KiB/s wr, 35 op/s 2024-05-07T02:39:32.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:31 smithi019 ceph-mon[29781]: pgmap v235: 129 pgs: 129 active+clean; 178 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 540 KiB/s wr, 35 op/s 2024-05-07T02:39:32.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:31 smithi027 ceph-mon[34398]: pgmap v235: 129 pgs: 129 active+clean; 178 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 540 KiB/s wr, 35 op/s 2024-05-07T02:39:33.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.939+0000 7ff08e400700 1 -- 172.21.15.19:0/4041583186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff088109860 msgr2=0x7ff0881111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:33.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.939+0000 7ff08e400700 1 --2- 172.21.15.19:0/4041583186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff088109860 0x7ff0881111c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff08000b0d0 tx=0x7ff08000b3e0 comp rx=0 tx=0).stop 2024-05-07T02:39:33.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.939+0000 7ff08e400700 1 -- 172.21.15.19:0/4041583186 shutdown_connections 2024-05-07T02:39:33.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.939+0000 7ff08e400700 1 --2- 172.21.15.19:0/4041583186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff088109860 0x7ff0881111c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:33.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.939+0000 7ff08e400700 1 --2- 172.21.15.19:0/4041583186 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0881045e0 0x7ff0881049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:33.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.939+0000 7ff08e400700 1 --2- 172.21.15.19:0/4041583186 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff088108f10 0x7ff088109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:33.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.939+0000 7ff08e400700 1 -- 172.21.15.19:0/4041583186 >> 172.21.15.19:0/4041583186 conn(0x7ff0880ffef0 msgr2=0x7ff088102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:33.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.939+0000 7ff08e400700 1 -- 172.21.15.19:0/4041583186 shutdown_connections 2024-05-07T02:39:33.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.939+0000 7ff08e400700 1 -- 172.21.15.19:0/4041583186 wait complete. 2024-05-07T02:39:33.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.941+0000 7ff08e400700 1 Processor -- start 2024-05-07T02:39:33.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.941+0000 7ff08e400700 1 -- start start 2024-05-07T02:39:33.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.941+0000 7ff08e400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0881045e0 0x7ff08819d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:33.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.941+0000 7ff08e400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff088108f10 0x7ff08819d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:33.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.942+0000 7ff08e400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff088109860 0x7ff0881a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:33.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.942+0000 7ff08e400700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff0881144e0 con 0x7ff0881045e0 2024-05-07T02:39:33.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.942+0000 7ff08e400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff088114360 con 0x7ff088108f10 2024-05-07T02:39:33.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.942+0000 7ff08e400700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff088114660 con 0x7ff088109860 2024-05-07T02:39:33.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.942+0000 7ff087400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0881045e0 0x7ff08819d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:33.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.942+0000 7ff087400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0881045e0 0x7ff08819d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39580/0 (socket says 172.21.15.19:39580) 2024-05-07T02:39:33.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.942+0000 7ff087400700 1 -- 172.21.15.19:0/2689546062 learned_addr learned my addr 172.21.15.19:0/2689546062 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:39:33.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.942+0000 7ff086a00700 1 --2- 172.21.15.19:0/2689546062 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff088108f10 0x7ff08819d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:33.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.942+0000 7ff087e00700 1 --2- 172.21.15.19:0/2689546062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff088109860 0x7ff0881a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:33.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.942+0000 7ff087400700 1 -- 172.21.15.19:0/2689546062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff088109860 msgr2=0x7ff0881a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:33.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.942+0000 7ff087400700 1 --2- 172.21.15.19:0/2689546062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff088109860 0x7ff0881a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:33.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.942+0000 7ff087400700 1 -- 172.21.15.19:0/2689546062 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff088108f10 msgr2=0x7ff08819d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:33.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.942+0000 7ff087400700 1 --2- 172.21.15.19:0/2689546062 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff088108f10 0x7ff08819d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:33.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.942+0000 7ff087400700 1 -- 172.21.15.19:0/2689546062 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0881a1f00 con 0x7ff0881045e0 2024-05-07T02:39:33.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.943+0000 7ff087400700 1 --2- 172.21.15.19:0/2689546062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0881045e0 0x7ff08819d030 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff07800e9a0 tx=0x7ff07800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:33.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.943+0000 7ff07fe00700 1 -- 172.21.15.19:0/2689546062 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07800ca10 con 0x7ff0881045e0 2024-05-07T02:39:33.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.943+0000 7ff07fe00700 1 -- 172.21.15.19:0/2689546062 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff07800cbb0 con 0x7ff0881045e0 2024-05-07T02:39:33.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.943+0000 7ff08e400700 1 -- 172.21.15.19:0/2689546062 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0881a21f0 con 0x7ff0881045e0 2024-05-07T02:39:33.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.943+0000 7ff07fe00700 1 -- 172.21.15.19:0/2689546062 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff078018680 con 0x7ff0881045e0 2024-05-07T02:39:33.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.943+0000 7ff08e400700 1 -- 172.21.15.19:0/2689546062 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0881a9b10 con 0x7ff0881045e0 2024-05-07T02:39:33.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.947+0000 7ff07fe00700 1 -- 172.21.15.19:0/2689546062 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff078010430 con 0x7ff0881045e0 2024-05-07T02:39:33.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.949+0000 7ff09cfc2b80 1 Processor -- start 2024-05-07T02:39:33.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.949+0000 7ff09cfc2b80 1 -- start start 2024-05-07T02:39:33.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.949+0000 7ff09cfc2b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5616b2aad3b0 0x5616b2aad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:33.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.949+0000 7ff09cfc2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5616b2aa7470 0x5616b2aa7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:33.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.949+0000 7ff07fe00700 1 --2- 172.21.15.19:0/2689546062 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff06c0b29f0 0x7ff06c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:33.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.949+0000 7ff09cfc2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616b2aa7dc0 0x5616b2ab4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:33.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.949+0000 7ff09cfc2b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5616b29c9a20 con 0x5616b2aa7dc0 2024-05-07T02:39:33.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.949+0000 7ff09cfc2b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5616b2967760 con 0x5616b2aad3b0 2024-05-07T02:39:33.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.949+0000 7ff09cfc2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5616b29c59e0 con 0x5616b2aa7470 2024-05-07T02:39:33.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.950+0000 7ff087e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5616b2aad3b0 0x5616b2aad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:33.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.950+0000 7ff07fe00700 1 -- 172.21.15.19:0/2689546062 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9525+0+0 (secure 0 0 0) 0x7ff0780e2bc0 con 0x7ff0881045e0 2024-05-07T02:39:33.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.950+0000 7ff087400700 1 --2- 172.21.15.19:0/2689546062 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff06c0b29f0 0x7ff06c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:33.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.950+0000 7ff087e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616b2aa7dc0 0x5616b2ab4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:33.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.950+0000 7ff087e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616b2aa7dc0 0x5616b2ab4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39582/0 (socket says 172.21.15.19:39582) 2024-05-07T02:39:33.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.950+0000 7ff087e00700 1 -- 172.21.15.19:0/1061926611 learned_addr learned my addr 172.21.15.19:0/1061926611 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:39:33.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.950+0000 7ff086a00700 1 --2- 172.21.15.19:0/1061926611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5616b2aa7470 0x5616b2aa7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:33.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.950+0000 7ff087e00700 1 -- 172.21.15.19:0/1061926611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5616b2aa7470 msgr2=0x5616b2aa7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:33.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.950+0000 7ff087e00700 1 --2- 172.21.15.19:0/1061926611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5616b2aa7470 0x5616b2aa7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:33.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.950+0000 7ff087e00700 1 -- 172.21.15.19:0/1061926611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5616b2aad3b0 msgr2=0x5616b2aad7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:33.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.950+0000 7ff087e00700 1 --2- 172.21.15.19:0/1061926611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5616b2aad3b0 0x5616b2aad7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:33.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.951+0000 7ff087e00700 1 -- 172.21.15.19:0/1061926611 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616b29168d0 con 0x5616b2aa7dc0 2024-05-07T02:39:33.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.951+0000 7ff087e00700 1 --2- 172.21.15.19:0/1061926611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616b2aa7dc0 0x5616b2ab4040 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff080009900 tx=0x7ff08000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99a0135fd23222e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:33.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.951+0000 7ff087400700 1 --2- 172.21.15.19:0/2689546062 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff06c0b29f0 0x7ff06c0b4f00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff0780dff20 tx=0x7ff078010c10 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:33.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.952+0000 7ff07cc00700 1 -- 172.21.15.19:0/1061926611 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff080003c00 con 0x5616b2aa7dc0 2024-05-07T02:39:33.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.952+0000 7ff07cc00700 1 -- 172.21.15.19:0/1061926611 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff080003da0 con 0x5616b2aa7dc0 2024-05-07T02:39:33.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.952+0000 7ff07cc00700 1 -- 172.21.15.19:0/1061926611 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08001c6a0 con 0x5616b2aa7dc0 2024-05-07T02:39:33.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.953+0000 7ff09cfc2b80 1 -- 172.21.15.19:0/1061926611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616b2aa7dc0 msgr2=0x5616b2ab4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:33.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.953+0000 7ff09cfc2b80 1 --2- 172.21.15.19:0/1061926611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616b2aa7dc0 0x5616b2ab4040 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff080009900 tx=0x7ff08000b040 comp rx=0 tx=0).stop 2024-05-07T02:39:33.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.953+0000 7ff09cfc2b80 1 -- 172.21.15.19:0/1061926611 shutdown_connections 2024-05-07T02:39:33.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.953+0000 7ff09cfc2b80 1 --2- 172.21.15.19:0/1061926611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616b2aa7dc0 0x5616b2ab4040 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:33.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.953+0000 7ff09cfc2b80 1 --2- 172.21.15.19:0/1061926611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5616b2aad3b0 0x5616b2aad7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:33.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.953+0000 7ff09cfc2b80 1 --2- 172.21.15.19:0/1061926611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5616b2aa7470 0x5616b2aa7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:33.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.953+0000 7ff09cfc2b80 1 -- 172.21.15.19:0/1061926611 >> 172.21.15.19:0/1061926611 conn(0x5616b2929e40 msgr2=0x5616b2aa22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:33.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.953+0000 7ff09cfc2b80 1 -- 172.21.15.19:0/1061926611 shutdown_connections 2024-05-07T02:39:33.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.953+0000 7ff09cfc2b80 1 -- 172.21.15.19:0/1061926611 wait complete. 2024-05-07T02:39:33.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.953+0000 7ff09cfc2b80 1 -- start start 2024-05-07T02:39:33.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.954+0000 7ff09cfc2b80 1 Processor -- start 2024-05-07T02:39:33.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.954+0000 7ff09cfc2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616b2aa7470 0x5616b2ab1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:33.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.954+0000 7ff09cfc2b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5616b2aa7dc0 0x5616b2ab1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:33.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.954+0000 7ff09cfc2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5616b2aad3b0 0x5616b2ab1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:33.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.954+0000 7ff09cfc2b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5616b2ab4760 con 0x5616b2aa7470 2024-05-07T02:39:33.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.954+0000 7ff09cfc2b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5616b292fb20 con 0x5616b2aa7dc0 2024-05-07T02:39:33.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.954+0000 7ff09cfc2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5616b2ab48e0 con 0x5616b2aad3b0 2024-05-07T02:39:33.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.954+0000 7ff09cfc2b80 10 client.0 _ll_register_callbacks cb 0x5616b2970c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:39:33.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.955+0000 7ff086a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5616b2aa7dc0 0x5616b2ab1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:33.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.955+0000 7ff086a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5616b2aa7dc0 0x5616b2ab1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:56312/0 (socket says 172.21.15.19:56312) 2024-05-07T02:39:33.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.955+0000 7ff086a00700 1 -- 172.21.15.19:0/1017177550 learned_addr learned my addr 172.21.15.19:0/1017177550 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:39:33.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.955+0000 7ff087e00700 1 --2- 172.21.15.19:0/1017177550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616b2aa7470 0x5616b2ab1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:33.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.955+0000 7ff087e00700 1 --2- 172.21.15.19:0/1017177550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5616b2aad3b0 0x5616b2ab1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:33.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.955+0000 7ff086a00700 1 -- 172.21.15.19:0/1017177550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5616b2aad3b0 msgr2=0x5616b2ab1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:33.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.955+0000 7ff086a00700 1 --2- 172.21.15.19:0/1017177550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5616b2aad3b0 0x5616b2ab1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:33.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.955+0000 7ff086a00700 1 -- 172.21.15.19:0/1017177550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616b2aa7470 msgr2=0x5616b2ab1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:33.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.955+0000 7ff086a00700 1 --2- 172.21.15.19:0/1017177550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616b2aa7470 0x5616b2ab1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:33.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.955+0000 7ff086a00700 1 -- 172.21.15.19:0/1017177550 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616b298f420 con 0x5616b2aa7dc0 2024-05-07T02:39:33.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.955+0000 7ff087e00700 1 --2- 172.21.15.19:0/1017177550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616b2aa7470 0x5616b2ab1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:39:33.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.956+0000 7ff086a00700 1 --2- 172.21.15.19:0/1017177550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5616b2aa7dc0 0x5616b2ab1a60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff074012700 tx=0x7ff074012ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0224391127e107c server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:33.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.956+0000 7ff071600700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:39:33.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.956+0000 7ff071600700 1 -- 172.21.15.19:0/1017177550 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff074004830 con 0x5616b2aa7dc0 2024-05-07T02:39:33.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.956+0000 7ff071600700 1 -- 172.21.15.19:0/1017177550 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9525+0+0 (secure 0 0 0) 0x7ff0740049d0 con 0x5616b2aa7dc0 2024-05-07T02:39:33.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.956+0000 7ff09cfc2b80 1 -- 172.21.15.19:0/1017177550 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff080003da0 con 0x5616b2aa7dc0 2024-05-07T02:39:33.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.956+0000 7ff071600700 1 -- 172.21.15.19:0/1017177550 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7ff074025050 con 0x5616b2aa7dc0 2024-05-07T02:39:33.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.956+0000 7ff09cfc2b80 10 client.26399 fetch_fsmap learned FSMap version 46 2024-05-07T02:39:33.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.956+0000 7ff09cfc2b80 1 -- 172.21.15.19:0/1017177550 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5616b2abf440 con 0x5616b2aa7dc0 2024-05-07T02:39:33.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.957+0000 7ff071600700 1 -- 172.21.15.19:0/1017177550 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff080003da0 con 0x5616b2aa7dc0 2024-05-07T02:39:33.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.957+0000 7ff071600700 1 -- 172.21.15.19:0/1017177550 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 46) ==== 8386+0+0 (secure 0 0 0) 0x7ff07401c7a0 con 0x5616b2aa7dc0 2024-05-07T02:39:33.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.957+0000 7ff09cfc2b80 10 client.26399 fetch_fsmap finished waiting for FSMap version 46 2024-05-07T02:39:33.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.957+0000 7ff09cfc2b80 10 client.26399 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-07T02:39:33.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.957+0000 7ff09cfc2b80 20 client.26399 populate_metadata read hostname 'smithi019' 2024-05-07T02:39:33.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.957+0000 7ff09cfc2b80 1 --2- 172.21.15.19:0/1017177550 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5616b2ac1a80 0x5616b2ac3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:33.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.957+0000 7ff09cfc2b80 4 client.26399 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:39:33.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.957+0000 7ff09cfc2b80 1 -- 172.21.15.19:0/1017177550 --> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616b2a742e0 con 0x5616b2ac1a80 2024-05-07T02:39:33.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.958+0000 7ff087e00700 1 --2- 172.21.15.19:0/1017177550 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5616b2ac1a80 0x5616b2ac3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:33.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.958+0000 7ff087e00700 1 --2- 172.21.15.19:0/1017177550 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5616b2ac1a80 0x5616b2ac3ec0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a5ec4a2d60452f64 server_cookie=fd61bdf44278dc38 in_seq=0 out_seq=0 2024-05-07T02:39:33.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.958+0000 7ff071600700 10 client.26399 ms_handle_connect on v2:172.21.15.19:6840/3905645246 2024-05-07T02:39:33.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.959+0000 7ff071600700 1 -- 172.21.15.19:0/1017177550 <== mds.1 v2:172.21.15.19:6840/3905645246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5616b2a742e0 con 0x5616b2ac1a80 2024-05-07T02:39:33.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.959+0000 7ff071600700 10 client.26399 handle_command_reply: tid=0 2024-05-07T02:39:33.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.960+0000 7ff09cfc2b80 1 client.26399 shutdown 2024-05-07T02:39:33.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.960+0000 7ff09cfc2b80 1 -- 172.21.15.19:0/1017177550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5616b2aa7dc0 msgr2=0x5616b2ab1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:33.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.960+0000 7ff09cfc2b80 1 --2- 172.21.15.19:0/1017177550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5616b2aa7dc0 0x5616b2ab1a60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff074012700 tx=0x7ff074012ac0 comp rx=0 tx=0).stop 2024-05-07T02:39:33.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.960+0000 7ff09cfc2b80 1 -- 172.21.15.19:0/1017177550 shutdown_connections 2024-05-07T02:39:33.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.960+0000 7ff09cfc2b80 1 --2- 172.21.15.19:0/1017177550 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5616b2ac1a80 0x5616b2ac3ec0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:33.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.960+0000 7ff09cfc2b80 1 --2- 172.21.15.19:0/1017177550 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5616b2aad3b0 0x5616b2ab1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:33.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.960+0000 7ff09cfc2b80 1 --2- 172.21.15.19:0/1017177550 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616b2aa7470 0x5616b2ab1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:33.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.960+0000 7ff09cfc2b80 1 --2- 172.21.15.19:0/1017177550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5616b2aa7dc0 0x5616b2ab1a60 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:33.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.960+0000 7ff09cfc2b80 1 -- 172.21.15.19:0/1017177550 >> 172.21.15.19:0/1017177550 conn(0x5616b2929e40 msgr2=0x5616b2aa6460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:33.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.960+0000 7ff071600700 0 client.26399 ms_handle_reset on v2:172.21.15.19:6840/3905645246 2024-05-07T02:39:33.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.960+0000 7ff09cfc2b80 1 -- 172.21.15.19:0/1017177550 shutdown_connections 2024-05-07T02:39:33.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.960+0000 7ff09cfc2b80 1 -- 172.21.15.19:0/1017177550 wait complete. 2024-05-07T02:39:33.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.960+0000 7ff09cfc2b80 20 client.26399 trim_cache size 0 max 16384 2024-05-07T02:39:33.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.978+0000 7ff09cfc2b80 1 Processor -- start 2024-05-07T02:39:33.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.978+0000 7ff09cfc2b80 1 -- start start 2024-05-07T02:39:33.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.979+0000 7ff09cfc2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616b2aa7470 0x5616b2aaee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:33.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.979+0000 7ff09cfc2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5616b2aa7dc0 0x5616b2abd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:33.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.979+0000 7ff09cfc2b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5616b2aad3b0 0x5616b2a9c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:33.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.979+0000 7ff09cfc2b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff0881b2970 con 0x5616b2aa7470 2024-05-07T02:39:33.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.979+0000 7ff09cfc2b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff0881b45d0 con 0x5616b2aad3b0 2024-05-07T02:39:33.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.979+0000 7ff09cfc2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff0881b3040 con 0x5616b2aa7dc0 2024-05-07T02:39:33.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.979+0000 7ff086a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5616b2aa7dc0 0x5616b2abd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:33.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.979+0000 7ff086a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5616b2aa7dc0 0x5616b2abd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:45092/0 (socket says 172.21.15.19:45092) 2024-05-07T02:39:33.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.979+0000 7ff086a00700 1 -- 172.21.15.19:0/3477372836 learned_addr learned my addr 172.21.15.19:0/3477372836 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:39:33.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.979+0000 7ff087e00700 1 --2- 172.21.15.19:0/3477372836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616b2aa7470 0x5616b2aaee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:33.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.980+0000 7ff087e00700 1 --2- 172.21.15.19:0/3477372836 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5616b2aad3b0 0x5616b2a9c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:33.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.980+0000 7ff086a00700 1 -- 172.21.15.19:0/3477372836 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5616b2aad3b0 msgr2=0x5616b2a9c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:33.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.980+0000 7ff086a00700 1 --2- 172.21.15.19:0/3477372836 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5616b2aad3b0 0x5616b2a9c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:33.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.980+0000 7ff086a00700 1 -- 172.21.15.19:0/3477372836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616b2aa7470 msgr2=0x5616b2aaee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:33.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.980+0000 7ff086a00700 1 --2- 172.21.15.19:0/3477372836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616b2aa7470 0x5616b2aaee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:33.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.980+0000 7ff086a00700 1 -- 172.21.15.19:0/3477372836 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616b2aafa90 con 0x5616b2aa7dc0 2024-05-07T02:39:33.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.980+0000 7ff087e00700 1 --2- 172.21.15.19:0/3477372836 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5616b2aad3b0 0x5616b2a9c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:39:33.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.980+0000 7ff086a00700 1 --2- 172.21.15.19:0/3477372836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5616b2aa7dc0 0x5616b2abd870 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff074005a80 tx=0x7ff07401c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d0ab94bc1ed8e8a server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:33.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.980+0000 7ff073e00700 1 -- 172.21.15.19:0/3477372836 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07401c660 con 0x5616b2aa7dc0 2024-05-07T02:39:33.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.980+0000 7ff073e00700 1 -- 172.21.15.19:0/3477372836 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff07401c800 con 0x5616b2aa7dc0 2024-05-07T02:39:33.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.980+0000 7ff073e00700 1 -- 172.21.15.19:0/3477372836 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff074003990 con 0x5616b2aa7dc0 2024-05-07T02:39:33.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.981+0000 7ff09cfc2b80 1 -- 172.21.15.19:0/3477372836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5616b2aa7dc0 msgr2=0x5616b2abd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:33.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.981+0000 7ff09cfc2b80 1 --2- 172.21.15.19:0/3477372836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5616b2aa7dc0 0x5616b2abd870 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff074005a80 tx=0x7ff07401c470 comp rx=0 tx=0).stop 2024-05-07T02:39:33.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.981+0000 7ff09cfc2b80 1 -- 172.21.15.19:0/3477372836 shutdown_connections 2024-05-07T02:39:33.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.981+0000 7ff09cfc2b80 1 --2- 172.21.15.19:0/3477372836 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5616b2aad3b0 0x5616b2a9c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:33.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.981+0000 7ff09cfc2b80 1 --2- 172.21.15.19:0/3477372836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616b2aa7470 0x5616b2aaee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:33.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.981+0000 7ff09cfc2b80 1 --2- 172.21.15.19:0/3477372836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5616b2aa7dc0 0x5616b2abd870 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:33.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.981+0000 7ff09cfc2b80 1 -- 172.21.15.19:0/3477372836 >> 172.21.15.19:0/3477372836 conn(0x5616b2929e40 msgr2=0x5616b2aafe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:33.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.982+0000 7ff09cfc2b80 1 -- 172.21.15.19:0/3477372836 shutdown_connections 2024-05-07T02:39:33.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.982+0000 7ff09cfc2b80 1 -- 172.21.15.19:0/3477372836 wait complete. 2024-05-07T02:39:33.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.982+0000 7ff09cfc2b80 1 -- start start 2024-05-07T02:39:33.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.982+0000 7ff09cfc2b80 1 Processor -- start 2024-05-07T02:39:33.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.983+0000 7ff09cfc2b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5616b2aa7470 0x5616b2ab9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:33.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.983+0000 7ff09cfc2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5616b2aa7dc0 0x5616b2ac8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:33.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.983+0000 7ff09cfc2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616b2aad3b0 0x5616b2ac8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:33.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.983+0000 7ff09cfc2b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff0881b3e40 con 0x5616b2aad3b0 2024-05-07T02:39:33.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.983+0000 7ff09cfc2b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff0881b3710 con 0x5616b2aa7470 2024-05-07T02:39:33.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.983+0000 7ff09cfc2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff0881b0930 con 0x5616b2aa7dc0 2024-05-07T02:39:33.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.983+0000 7ff09cfc2b80 10 client.0 _ll_register_callbacks cb 0x5616b2970c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:39:33.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.983+0000 7ff087e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5616b2aa7470 0x5616b2ab9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:33.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.983+0000 7ff086a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5616b2aa7dc0 0x5616b2ac8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:33.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.984+0000 7ff086a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5616b2aa7dc0 0x5616b2ac8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:45096/0 (socket says 172.21.15.19:45096) 2024-05-07T02:39:33.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.984+0000 7ff086a00700 1 -- 172.21.15.19:0/3445432452 learned_addr learned my addr 172.21.15.19:0/3445432452 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:39:33.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.984+0000 7ff087e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5616b2aa7470 0x5616b2ab9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:56344/0 (socket says 172.21.15.19:56344) 2024-05-07T02:39:33.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.984+0000 7ff087e00700 1 --2- 172.21.15.19:0/3445432452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616b2aad3b0 0x5616b2ac8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:33.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.984+0000 7ff086a00700 1 -- 172.21.15.19:0/3445432452 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5616b2aa7470 msgr2=0x5616b2ab9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:33.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.984+0000 7ff086a00700 1 --2- 172.21.15.19:0/3445432452 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5616b2aa7470 0x5616b2ab9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:33.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.984+0000 7ff086a00700 1 -- 172.21.15.19:0/3445432452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616b2aad3b0 msgr2=0x5616b2ac8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:33.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.984+0000 7ff086a00700 1 --2- 172.21.15.19:0/3445432452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616b2aad3b0 0x5616b2ac8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:33.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.984+0000 7ff086a00700 1 -- 172.21.15.19:0/3445432452 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616b2aba5b0 con 0x5616b2aa7dc0 2024-05-07T02:39:33.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.984+0000 7ff086a00700 1 --2- 172.21.15.19:0/3445432452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5616b2aa7dc0 0x5616b2ac8740 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff074014040 tx=0x7ff074004a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ce9f1d3b25fd301 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:33.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.984+0000 7ff071600700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:39:33.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.985+0000 7ff071600700 1 -- 172.21.15.19:0/3445432452 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07401c660 con 0x5616b2aa7dc0 2024-05-07T02:39:33.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.985+0000 7ff09cfc2b80 1 -- 172.21.15.19:0/3445432452 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff07401c800 con 0x5616b2aa7dc0 2024-05-07T02:39:33.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.985+0000 7ff071600700 1 -- 172.21.15.19:0/3445432452 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9525+0+0 (secure 0 0 0) 0x7ff074018040 con 0x5616b2aa7dc0 2024-05-07T02:39:33.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.985+0000 7ff071600700 1 -- 172.21.15.19:0/3445432452 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7ff07403c050 con 0x5616b2aa7dc0 2024-05-07T02:39:33.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.985+0000 7ff09cfc2b80 10 client.26365 fetch_fsmap learned FSMap version 46 2024-05-07T02:39:33.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.985+0000 7ff09cfc2b80 1 -- 172.21.15.19:0/3445432452 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5616b2b2bd00 con 0x5616b2aa7dc0 2024-05-07T02:39:33.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.985+0000 7ff071600700 1 -- 172.21.15.19:0/3445432452 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff07401c800 con 0x5616b2aa7dc0 2024-05-07T02:39:33.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.985+0000 7ff071600700 1 -- 172.21.15.19:0/3445432452 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 46) ==== 8386+0+0 (secure 0 0 0) 0x7ff07400f5d0 con 0x5616b2aa7dc0 2024-05-07T02:39:33.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.986+0000 7ff09cfc2b80 10 client.26365 fetch_fsmap finished waiting for FSMap version 46 2024-05-07T02:39:33.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.986+0000 7ff09cfc2b80 10 client.26365 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-07T02:39:33.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.986+0000 7ff09cfc2b80 20 client.26365 populate_metadata read hostname 'smithi019' 2024-05-07T02:39:33.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.986+0000 7ff09cfc2b80 1 --2- 172.21.15.19:0/3445432452 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5616b2b2c340 0x5616b2b2e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:33.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.986+0000 7ff09cfc2b80 4 client.26365 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:39:33.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.986+0000 7ff09cfc2b80 1 -- 172.21.15.19:0/3445432452 --> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5616b29d7400 con 0x5616b2b2c340 2024-05-07T02:39:33.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.986+0000 7ff087e00700 1 --2- 172.21.15.19:0/3445432452 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5616b2b2c340 0x5616b2b2e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:33.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.987+0000 7ff087e00700 1 --2- 172.21.15.19:0/3445432452 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5616b2b2c340 0x5616b2b2e760 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b479e8f40d74f8d8 server_cookie=5648868d0dd26adc in_seq=0 out_seq=0 2024-05-07T02:39:33.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.987+0000 7ff071600700 10 client.26365 ms_handle_connect on v2:172.21.15.19:6840/3905645246 2024-05-07T02:39:33.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.987+0000 7ff071600700 1 -- 172.21.15.19:0/3445432452 <== mds.1 v2:172.21.15.19:6840/3905645246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5616b29d7400 con 0x5616b2b2c340 2024-05-07T02:39:33.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.987+0000 7ff071600700 10 client.26365 handle_command_reply: tid=0 2024-05-07T02:39:33.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.987+0000 7ff09cfc2b80 1 client.26365 shutdown 2024-05-07T02:39:33.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.987+0000 7ff09cfc2b80 1 -- 172.21.15.19:0/3445432452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5616b2aa7dc0 msgr2=0x5616b2ac8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:33.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.987+0000 7ff09cfc2b80 1 --2- 172.21.15.19:0/3445432452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5616b2aa7dc0 0x5616b2ac8740 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff074014040 tx=0x7ff074004a70 comp rx=0 tx=0).stop 2024-05-07T02:39:33.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.987+0000 7ff09cfc2b80 1 -- 172.21.15.19:0/3445432452 shutdown_connections 2024-05-07T02:39:33.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.987+0000 7ff09cfc2b80 1 --2- 172.21.15.19:0/3445432452 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5616b2b2c340 0x5616b2b2e760 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:33.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.987+0000 7ff09cfc2b80 1 --2- 172.21.15.19:0/3445432452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616b2aad3b0 0x5616b2ac8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:33.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.987+0000 7ff09cfc2b80 1 --2- 172.21.15.19:0/3445432452 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5616b2aa7470 0x5616b2ab9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:33.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.987+0000 7ff09cfc2b80 1 --2- 172.21.15.19:0/3445432452 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5616b2aa7dc0 0x5616b2ac8740 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:33.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.987+0000 7ff071600700 0 client.26365 ms_handle_reset on v2:172.21.15.19:6840/3905645246 2024-05-07T02:39:33.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.987+0000 7ff09cfc2b80 1 -- 172.21.15.19:0/3445432452 >> 172.21.15.19:0/3445432452 conn(0x5616b2929e40 msgr2=0x5616b2a9c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:33.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.987+0000 7ff09cfc2b80 1 -- 172.21.15.19:0/3445432452 shutdown_connections 2024-05-07T02:39:33.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.987+0000 7ff09cfc2b80 1 -- 172.21.15.19:0/3445432452 wait complete. 2024-05-07T02:39:33.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.988+0000 7ff09cfc2b80 20 client.26365 trim_cache size 0 max 16384 2024-05-07T02:39:33.994 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:39:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.988+0000 7ff08e400700 1 -- 172.21.15.19:0/2689546062 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff06c0b29f0 msgr2=0x7ff06c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.988+0000 7ff08e400700 1 --2- 172.21.15.19:0/2689546062 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff06c0b29f0 0x7ff06c0b4f00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff0780dff20 tx=0x7ff078010c10 comp rx=0 tx=0).stop 2024-05-07T02:39:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.988+0000 7ff08e400700 1 -- 172.21.15.19:0/2689546062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0881045e0 msgr2=0x7ff08819d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.988+0000 7ff08e400700 1 --2- 172.21.15.19:0/2689546062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0881045e0 0x7ff08819d030 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff07800e9a0 tx=0x7ff07800ecb0 comp rx=0 tx=0).stop 2024-05-07T02:39:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.989+0000 7ff08e400700 1 -- 172.21.15.19:0/2689546062 shutdown_connections 2024-05-07T02:39:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.989+0000 7ff08e400700 1 --2- 172.21.15.19:0/2689546062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff088109860 0x7ff0881a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.989+0000 7ff08e400700 1 --2- 172.21.15.19:0/2689546062 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff06c0b29f0 0x7ff06c0b4f00 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.989+0000 7ff08e400700 1 --2- 172.21.15.19:0/2689546062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0881045e0 0x7ff08819d030 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:33.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.989+0000 7ff08e400700 1 --2- 172.21.15.19:0/2689546062 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff088108f10 0x7ff08819d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:33.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.989+0000 7ff08e400700 1 -- 172.21.15.19:0/2689546062 >> 172.21.15.19:0/2689546062 conn(0x7ff0880ffef0 msgr2=0x7ff088110e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:33.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.989+0000 7ff08e400700 1 -- 172.21.15.19:0/2689546062 shutdown_connections 2024-05-07T02:39:33.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:33.989+0000 7ff08e400700 1 -- 172.21.15.19:0/2689546062 wait complete. 2024-05-07T02:39:34.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:33 smithi064 ceph-mon[34699]: pgmap v236: 129 pgs: 129 active+clean; 178 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 805 KiB/s wr, 52 op/s 2024-05-07T02:39:34.391 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:33 smithi019 ceph-mon[29781]: pgmap v236: 129 pgs: 129 active+clean; 178 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 805 KiB/s wr, 52 op/s 2024-05-07T02:39:34.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:33 smithi027 ceph-mon[34398]: pgmap v236: 129 pgs: 129 active+clean; 178 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 805 KiB/s wr, 52 op/s 2024-05-07T02:39:34.932 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-07T02:39:35.408 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:39:36.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:35 smithi064 ceph-mon[34699]: pgmap v237: 129 pgs: 129 active+clean; 178 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 547 KiB/s wr, 36 op/s 2024-05-07T02:39:36.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:35 smithi019 ceph-mon[29781]: pgmap v237: 129 pgs: 129 active+clean; 178 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 547 KiB/s wr, 36 op/s 2024-05-07T02:39:36.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:35 smithi027 ceph-mon[34398]: pgmap v237: 129 pgs: 129 active+clean; 178 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 547 KiB/s wr, 36 op/s 2024-05-07T02:39:38.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:37 smithi019 ceph-mon[29781]: pgmap v238: 129 pgs: 129 active+clean; 177 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 762 KiB/s wr, 50 op/s 2024-05-07T02:39:38.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.196+0000 7f821c800700 1 -- 172.21.15.19:0/3556232361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8214106570 msgr2=0x7f8214106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:38.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.197+0000 7f821c800700 1 --2- 172.21.15.19:0/3556232361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8214106570 0x7f8214106980 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8204009990 tx=0x7f8204009ca0 comp rx=0 tx=0).stop 2024-05-07T02:39:38.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.197+0000 7f821c800700 1 -- 172.21.15.19:0/3556232361 shutdown_connections 2024-05-07T02:39:38.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.197+0000 7f821c800700 1 --2- 172.21.15.19:0/3556232361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8214106ec0 0x7f8214113350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:38.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.197+0000 7f821c800700 1 --2- 172.21.15.19:0/3556232361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f821410c5b0 0x7f821410c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:38.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.197+0000 7f821c800700 1 --2- 172.21.15.19:0/3556232361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8214106570 0x7f8214106980 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:38.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.197+0000 7f821c800700 1 -- 172.21.15.19:0/3556232361 >> 172.21.15.19:0/3556232361 conn(0x7f8214102090 msgr2=0x7f82141044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:38.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.197+0000 7f821c800700 1 -- 172.21.15.19:0/3556232361 shutdown_connections 2024-05-07T02:39:38.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.197+0000 7f821c800700 1 -- 172.21.15.19:0/3556232361 wait complete. 2024-05-07T02:39:38.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.198+0000 7f821c800700 1 Processor -- start 2024-05-07T02:39:38.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.198+0000 7f821c800700 1 -- start start 2024-05-07T02:39:38.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.199+0000 7f821c800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8214106570 0x7f821419d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:38.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.199+0000 7f821c800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8214106ec0 0x7f821419d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:38.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.199+0000 7f821c800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f821410c5b0 0x7f82141a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:38.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.199+0000 7f821c800700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8214116700 con 0x7f8214106ec0 2024-05-07T02:39:38.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.199+0000 7f821c800700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8214116580 con 0x7f8214106570 2024-05-07T02:39:38.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.199+0000 7f821c800700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8214116880 con 0x7f821410c5b0 2024-05-07T02:39:38.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.199+0000 7f8219e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8214106570 0x7f821419d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:38.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.199+0000 7f8219e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8214106570 0x7f821419d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:56350/0 (socket says 172.21.15.19:56350) 2024-05-07T02:39:38.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.199+0000 7f8219e00700 1 -- 172.21.15.19:0/2377231663 learned_addr learned my addr 172.21.15.19:0/2377231663 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:39:38.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.199+0000 7f8219400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8214106ec0 0x7f821419d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:38.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.199+0000 7f821a800700 1 --2- 172.21.15.19:0/2377231663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f821410c5b0 0x7f82141a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:38.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.199+0000 7f8219e00700 1 -- 172.21.15.19:0/2377231663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f821410c5b0 msgr2=0x7f82141a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:38.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.199+0000 7f8219e00700 1 --2- 172.21.15.19:0/2377231663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f821410c5b0 0x7f82141a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:38.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.199+0000 7f8219e00700 1 -- 172.21.15.19:0/2377231663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8214106ec0 msgr2=0x7f821419d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:38.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.199+0000 7f8219e00700 1 --2- 172.21.15.19:0/2377231663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8214106ec0 0x7f821419d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:38.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.199+0000 7f8219e00700 1 -- 172.21.15.19:0/2377231663 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82141a1f80 con 0x7f8214106570 2024-05-07T02:39:38.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.200+0000 7f8219e00700 1 --2- 172.21.15.19:0/2377231663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8214106570 0x7f821419d0b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f820800e9f0 tx=0x7f820800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:38.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.200+0000 7f820ea00700 1 -- 172.21.15.19:0/2377231663 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8208004a30 con 0x7f8214106570 2024-05-07T02:39:38.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.200+0000 7f821c800700 1 -- 172.21.15.19:0/2377231663 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82141a2270 con 0x7f8214106570 2024-05-07T02:39:38.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.200+0000 7f820ea00700 1 -- 172.21.15.19:0/2377231663 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f820800cc70 con 0x7f8214106570 2024-05-07T02:39:38.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.200+0000 7f820ea00700 1 -- 172.21.15.19:0/2377231663 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8208005530 con 0x7f8214106570 2024-05-07T02:39:38.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.200+0000 7f821c800700 1 -- 172.21.15.19:0/2377231663 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82141a9b90 con 0x7f8214106570 2024-05-07T02:39:38.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.204+0000 7f820ea00700 1 -- 172.21.15.19:0/2377231663 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8208004bd0 con 0x7f8214106570 2024-05-07T02:39:38.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.206+0000 7f822b2cbb80 1 Processor -- start 2024-05-07T02:39:38.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.206+0000 7f822b2cbb80 1 -- start start 2024-05-07T02:39:38.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.207+0000 7f820ea00700 1 --2- 172.21.15.19:0/2377231663 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f81fc0b29f0 0x7f81fc0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:38.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.207+0000 7f822b2cbb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d221673b0 0x562d221677c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:38.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.207+0000 7f822b2cbb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d22161470 0x562d22161880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:38.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.207+0000 7f822b2cbb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d22161dc0 0x562d2216e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:38.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.207+0000 7f822b2cbb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562d22083a20 con 0x562d22161470 2024-05-07T02:39:38.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.207+0000 7f822b2cbb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562d22021760 con 0x562d221673b0 2024-05-07T02:39:38.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.207+0000 7f822b2cbb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562d2207f9e0 con 0x562d22161dc0 2024-05-07T02:39:38.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.207+0000 7f820ea00700 1 -- 172.21.15.19:0/2377231663 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9525+0+0 (secure 0 0 0) 0x7f82080e2c80 con 0x7f8214106570 2024-05-07T02:39:38.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.207+0000 7f821a800700 1 --2- 172.21.15.19:0/2377231663 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f81fc0b29f0 0x7f81fc0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:38.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.208+0000 7f8219400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d22161470 0x562d22161880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:38.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.208+0000 7f8219400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d22161470 0x562d22161880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39652/0 (socket says 172.21.15.19:39652) 2024-05-07T02:39:38.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.208+0000 7f8219400700 1 -- 172.21.15.19:0/2929496828 learned_addr learned my addr 172.21.15.19:0/2929496828 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:39:38.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.208+0000 7f8219e00700 1 --2- 172.21.15.19:0/2929496828 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d221673b0 0x562d221677c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:38.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.208+0000 7f821a800700 1 --2- 172.21.15.19:0/2929496828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d22161dc0 0x562d2216e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:38.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.208+0000 7f8219400700 1 -- 172.21.15.19:0/2929496828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d22161dc0 msgr2=0x562d2216e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:38.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.208+0000 7f8219400700 1 --2- 172.21.15.19:0/2929496828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d22161dc0 0x562d2216e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:38.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.208+0000 7f8219400700 1 -- 172.21.15.19:0/2929496828 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d221673b0 msgr2=0x562d221677c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:38.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.208+0000 7f8219400700 1 --2- 172.21.15.19:0/2929496828 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d221673b0 0x562d221677c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:38.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.208+0000 7f8219400700 1 -- 172.21.15.19:0/2929496828 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d21fd08d0 con 0x562d22161470 2024-05-07T02:39:38.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.209+0000 7f821a800700 1 --2- 172.21.15.19:0/2377231663 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f81fc0b29f0 0x7f81fc0b4f00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8210009ec0 tx=0x7f8210013250 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:38.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.209+0000 7f8219400700 1 --2- 172.21.15.19:0/2929496828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d22161470 0x562d22161880 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8204009990 tx=0x7f82040096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd89778121e69f8b server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:38.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.209+0000 7f8203400700 1 -- 172.21.15.19:0/2929496828 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8204027070 con 0x562d22161470 2024-05-07T02:39:38.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.209+0000 7f8203400700 1 -- 172.21.15.19:0/2929496828 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82040056b0 con 0x562d22161470 2024-05-07T02:39:38.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.209+0000 7f8203400700 1 -- 172.21.15.19:0/2929496828 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8204003a40 con 0x562d22161470 2024-05-07T02:39:38.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.210+0000 7f822b2cbb80 1 -- 172.21.15.19:0/2929496828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d22161470 msgr2=0x562d22161880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:38.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.210+0000 7f822b2cbb80 1 --2- 172.21.15.19:0/2929496828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d22161470 0x562d22161880 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8204009990 tx=0x7f82040096a0 comp rx=0 tx=0).stop 2024-05-07T02:39:38.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.210+0000 7f822b2cbb80 1 -- 172.21.15.19:0/2929496828 shutdown_connections 2024-05-07T02:39:38.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.210+0000 7f822b2cbb80 1 --2- 172.21.15.19:0/2929496828 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d22161dc0 0x562d2216e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:38.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.210+0000 7f822b2cbb80 1 --2- 172.21.15.19:0/2929496828 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d221673b0 0x562d221677c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:38.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.210+0000 7f822b2cbb80 1 --2- 172.21.15.19:0/2929496828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d22161470 0x562d22161880 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:38.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.210+0000 7f822b2cbb80 1 -- 172.21.15.19:0/2929496828 >> 172.21.15.19:0/2929496828 conn(0x562d21fe3e40 msgr2=0x562d2215c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:38.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.210+0000 7f822b2cbb80 1 -- 172.21.15.19:0/2929496828 shutdown_connections 2024-05-07T02:39:38.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.210+0000 7f822b2cbb80 1 -- 172.21.15.19:0/2929496828 wait complete. 2024-05-07T02:39:38.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.211+0000 7f822b2cbb80 1 -- start start 2024-05-07T02:39:38.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.211+0000 7f822b2cbb80 1 Processor -- start 2024-05-07T02:39:38.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.212+0000 7f822b2cbb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d22161470 0x562d2216b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:38.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.212+0000 7f822b2cbb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d22161dc0 0x562d2216ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:38.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.212+0000 7f822b2cbb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d221673b0 0x562d2216bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:38.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.212+0000 7f822b2cbb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562d2216e760 con 0x562d22161470 2024-05-07T02:39:38.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.212+0000 7f822b2cbb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562d21fe9b20 con 0x562d22161dc0 2024-05-07T02:39:38.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.212+0000 7f822b2cbb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562d2216e8e0 con 0x562d221673b0 2024-05-07T02:39:38.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.212+0000 7f822b2cbb80 10 client.0 _ll_register_callbacks cb 0x562d2202ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:39:38.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.212+0000 7f8219e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d22161470 0x562d2216b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:38.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.212+0000 7f8219400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d22161dc0 0x562d2216ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:38.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.212+0000 7f8219e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d22161470 0x562d2216b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39658/0 (socket says 172.21.15.19:39658) 2024-05-07T02:39:38.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.212+0000 7f8219400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d22161dc0 0x562d2216ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:56360/0 (socket says 172.21.15.19:56360) 2024-05-07T02:39:38.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.212+0000 7f8219e00700 1 -- 172.21.15.19:0/169774028 learned_addr learned my addr 172.21.15.19:0/169774028 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:39:38.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.212+0000 7f821a800700 1 --2- 172.21.15.19:0/169774028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d221673b0 0x562d2216bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:38.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.213+0000 7f8219400700 1 -- 172.21.15.19:0/169774028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d221673b0 msgr2=0x562d2216bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:38.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.213+0000 7f8219400700 1 --2- 172.21.15.19:0/169774028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d221673b0 0x562d2216bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:38.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.213+0000 7f8219400700 1 -- 172.21.15.19:0/169774028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d22161470 msgr2=0x562d2216b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:38.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.213+0000 7f8219400700 1 --2- 172.21.15.19:0/169774028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d22161470 0x562d2216b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:38.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.213+0000 7f8219400700 1 -- 172.21.15.19:0/169774028 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d22049420 con 0x562d22161dc0 2024-05-07T02:39:38.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.213+0000 7f8219e00700 1 --2- 172.21.15.19:0/169774028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d22161470 0x562d2216b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:39:38.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.213+0000 7f8219400700 1 --2- 172.21.15.19:0/169774028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d22161dc0 0x562d2216ba60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8204025040 tx=0x7f820400b9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7af4af82c0c5379 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:38.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.213+0000 7f81f7e00700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:39:38.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.214+0000 7f81f7e00700 1 -- 172.21.15.19:0/169774028 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8204027040 con 0x562d22161dc0 2024-05-07T02:39:38.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.214+0000 7f822b2cbb80 1 -- 172.21.15.19:0/169774028 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82040056b0 con 0x562d22161dc0 2024-05-07T02:39:38.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.214+0000 7f81f7e00700 1 -- 172.21.15.19:0/169774028 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9525+0+0 (secure 0 0 0) 0x7f820400bbc0 con 0x562d22161dc0 2024-05-07T02:39:38.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.214+0000 7f81f7e00700 1 -- 172.21.15.19:0/169774028 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f8204003680 con 0x562d22161dc0 2024-05-07T02:39:38.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.214+0000 7f822b2cbb80 10 client.26417 fetch_fsmap learned FSMap version 46 2024-05-07T02:39:38.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.214+0000 7f822b2cbb80 1 -- 172.21.15.19:0/169774028 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x562d22179360 con 0x562d22161dc0 2024-05-07T02:39:38.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.214+0000 7f81f7e00700 1 -- 172.21.15.19:0/169774028 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82040056b0 con 0x562d22161dc0 2024-05-07T02:39:38.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.214+0000 7f81f7e00700 1 -- 172.21.15.19:0/169774028 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 46) ==== 8386+0+0 (secure 0 0 0) 0x7f82040227b0 con 0x562d22161dc0 2024-05-07T02:39:38.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.215+0000 7f822b2cbb80 10 client.26417 fetch_fsmap finished waiting for FSMap version 46 2024-05-07T02:39:38.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.215+0000 7f822b2cbb80 10 client.26417 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:39:38.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.215+0000 7f822b2cbb80 20 client.26417 populate_metadata read hostname 'smithi019' 2024-05-07T02:39:38.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.215+0000 7f822b2cbb80 1 --2- 172.21.15.19:0/169774028 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x562d2217b830 0x562d2217dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:38.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.215+0000 7f822b2cbb80 4 client.26417 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:39:38.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.215+0000 7f822b2cbb80 1 -- 172.21.15.19:0/169774028 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d2212e2e0 con 0x562d2217b830 2024-05-07T02:39:38.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.215+0000 7f8219e00700 1 --2- 172.21.15.19:0/169774028 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x562d2217b830 0x562d2217dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:38.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.216+0000 7f8219e00700 1 --2- 172.21.15.19:0/169774028 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x562d2217b830 0x562d2217dc50 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dee68814665f14b server_cookie=2317e0ee5d017617 in_seq=0 out_seq=0 2024-05-07T02:39:38.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.216+0000 7f81f7e00700 10 client.26417 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:39:38.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.216+0000 7f81f7e00700 1 -- 172.21.15.19:0/169774028 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562d2212e2e0 con 0x562d2217b830 2024-05-07T02:39:38.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.216+0000 7f81f7e00700 10 client.26417 handle_command_reply: tid=0 2024-05-07T02:39:38.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.216+0000 7f822b2cbb80 1 client.26417 shutdown 2024-05-07T02:39:38.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.217+0000 7f822b2cbb80 1 -- 172.21.15.19:0/169774028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d22161dc0 msgr2=0x562d2216ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:38.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.217+0000 7f822b2cbb80 1 --2- 172.21.15.19:0/169774028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d22161dc0 0x562d2216ba60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8204025040 tx=0x7f820400b9d0 comp rx=0 tx=0).stop 2024-05-07T02:39:38.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.217+0000 7f822b2cbb80 1 -- 172.21.15.19:0/169774028 shutdown_connections 2024-05-07T02:39:38.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.217+0000 7f822b2cbb80 1 --2- 172.21.15.19:0/169774028 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x562d2217b830 0x562d2217dc50 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:38.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.217+0000 7f822b2cbb80 1 --2- 172.21.15.19:0/169774028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d221673b0 0x562d2216bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:38.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.217+0000 7f822b2cbb80 1 --2- 172.21.15.19:0/169774028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d22161470 0x562d2216b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:38.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.217+0000 7f822b2cbb80 1 --2- 172.21.15.19:0/169774028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d22161dc0 0x562d2216ba60 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:38.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.217+0000 7f81f7e00700 0 client.26417 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:39:38.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.217+0000 7f822b2cbb80 1 -- 172.21.15.19:0/169774028 >> 172.21.15.19:0/169774028 conn(0x562d21fe3e40 msgr2=0x562d22160460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:38.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.217+0000 7f822b2cbb80 1 -- 172.21.15.19:0/169774028 shutdown_connections 2024-05-07T02:39:38.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.217+0000 7f822b2cbb80 1 -- 172.21.15.19:0/169774028 wait complete. 2024-05-07T02:39:38.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.217+0000 7f822b2cbb80 20 client.26417 trim_cache size 0 max 16384 2024-05-07T02:39:38.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.235+0000 7f822b2cbb80 1 Processor -- start 2024-05-07T02:39:38.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.235+0000 7f822b2cbb80 1 -- start start 2024-05-07T02:39:38.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.236+0000 7f822b2cbb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d22161470 0x562d22177b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:38.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.236+0000 7f822b2cbb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d22161dc0 0x562d22165cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:38.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.236+0000 7f822b2cbb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d221673b0 0x562d22156830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:38.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.236+0000 7f822b2cbb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f82141b2880 con 0x562d22161470 2024-05-07T02:39:38.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.236+0000 7f822b2cbb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f82141b44e0 con 0x562d22161dc0 2024-05-07T02:39:38.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.236+0000 7f822b2cbb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f82141b2f50 con 0x562d221673b0 2024-05-07T02:39:38.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.236+0000 7f8219400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d22161dc0 0x562d22165cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:38.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.236+0000 7f8219e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d22161470 0x562d22177b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:38.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.236+0000 7f8219400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d22161dc0 0x562d22165cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:56370/0 (socket says 172.21.15.19:56370) 2024-05-07T02:39:38.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.236+0000 7f8219400700 1 -- 172.21.15.19:0/2945667395 learned_addr learned my addr 172.21.15.19:0/2945667395 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:39:38.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.236+0000 7f821a800700 1 --2- 172.21.15.19:0/2945667395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d221673b0 0x562d22156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:38.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.237+0000 7f8219400700 1 -- 172.21.15.19:0/2945667395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d221673b0 msgr2=0x562d22156830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:38.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.237+0000 7f8219400700 1 --2- 172.21.15.19:0/2945667395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d221673b0 0x562d22156830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:38.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.237+0000 7f8219400700 1 -- 172.21.15.19:0/2945667395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d22161470 msgr2=0x562d22177b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:38.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.237+0000 7f8219400700 1 --2- 172.21.15.19:0/2945667395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d22161470 0x562d22177b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:38.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.237+0000 7f8219400700 1 -- 172.21.15.19:0/2945667395 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d2216ddc0 con 0x562d22161dc0 2024-05-07T02:39:38.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.237+0000 7f8219e00700 1 --2- 172.21.15.19:0/2945667395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d22161470 0x562d22177b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:39:38.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.237+0000 7f8219400700 1 --2- 172.21.15.19:0/2945667395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d22161dc0 0x562d22165cd0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f82040098e0 tx=0x7f820400f460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e49b29137ae296c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:38.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.237+0000 7f8202a00700 1 -- 172.21.15.19:0/2945667395 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82040055b0 con 0x562d22161dc0 2024-05-07T02:39:38.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.237+0000 7f8202a00700 1 -- 172.21.15.19:0/2945667395 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8204022480 con 0x562d22161dc0 2024-05-07T02:39:38.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.237+0000 7f8202a00700 1 -- 172.21.15.19:0/2945667395 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8204003b80 con 0x562d22161dc0 2024-05-07T02:39:38.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.238+0000 7f822b2cbb80 1 -- 172.21.15.19:0/2945667395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d22161dc0 msgr2=0x562d22165cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:38.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.238+0000 7f822b2cbb80 1 --2- 172.21.15.19:0/2945667395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d22161dc0 0x562d22165cd0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f82040098e0 tx=0x7f820400f460 comp rx=0 tx=0).stop 2024-05-07T02:39:38.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.238+0000 7f822b2cbb80 1 -- 172.21.15.19:0/2945667395 shutdown_connections 2024-05-07T02:39:38.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.238+0000 7f822b2cbb80 1 --2- 172.21.15.19:0/2945667395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d221673b0 0x562d22156830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:38.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.238+0000 7f822b2cbb80 1 --2- 172.21.15.19:0/2945667395 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d22161470 0x562d22177b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:38.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.238+0000 7f822b2cbb80 1 --2- 172.21.15.19:0/2945667395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d22161dc0 0x562d22165cd0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:38.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.238+0000 7f822b2cbb80 1 -- 172.21.15.19:0/2945667395 >> 172.21.15.19:0/2945667395 conn(0x562d21fe3e40 msgr2=0x562d22156120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:38.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.239+0000 7f822b2cbb80 1 -- 172.21.15.19:0/2945667395 shutdown_connections 2024-05-07T02:39:38.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.239+0000 7f822b2cbb80 1 -- 172.21.15.19:0/2945667395 wait complete. 2024-05-07T02:39:38.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.239+0000 7f822b2cbb80 1 -- start start 2024-05-07T02:39:38.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.239+0000 7f822b2cbb80 1 Processor -- start 2024-05-07T02:39:38.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.240+0000 7f822b2cbb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d22161470 0x562d22180f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:38.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.240+0000 7f822b2cbb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d22161dc0 0x562d2217e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:38.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.240+0000 7f822b2cbb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d221673b0 0x562d2217e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:38.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.240+0000 7f822b2cbb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f82141b3d50 con 0x562d22161470 2024-05-07T02:39:38.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.240+0000 7f822b2cbb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f82141b3620 con 0x562d221673b0 2024-05-07T02:39:38.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.240+0000 7f822b2cbb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f82141b0840 con 0x562d22161dc0 2024-05-07T02:39:38.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.240+0000 7f822b2cbb80 10 client.0 _ll_register_callbacks cb 0x562d2202ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:39:38.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.240+0000 7f8219e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d22161470 0x562d22180f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:38.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.240+0000 7f8219400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d22161dc0 0x562d2217e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:38.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.240+0000 7f8219400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d22161dc0 0x562d2217e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:45148/0 (socket says 172.21.15.19:45148) 2024-05-07T02:39:38.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.240+0000 7f8219400700 1 -- 172.21.15.19:0/1742818793 learned_addr learned my addr 172.21.15.19:0/1742818793 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:39:38.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.240+0000 7f8219e00700 1 --2- 172.21.15.19:0/1742818793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d22161470 0x562d22180f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39682/0 (socket says 172.21.15.19:39682) 2024-05-07T02:39:38.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.241+0000 7f821a800700 1 --2- 172.21.15.19:0/1742818793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d221673b0 0x562d2217e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:38.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.241+0000 7f8219400700 1 -- 172.21.15.19:0/1742818793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d221673b0 msgr2=0x562d2217e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:38.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.241+0000 7f8219400700 1 --2- 172.21.15.19:0/1742818793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d221673b0 0x562d2217e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:38.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.241+0000 7f8219400700 1 -- 172.21.15.19:0/1742818793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d22161470 msgr2=0x562d22180f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:38.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.241+0000 7f8219400700 1 --2- 172.21.15.19:0/1742818793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d22161470 0x562d22180f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:38.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.241+0000 7f8219400700 1 -- 172.21.15.19:0/1742818793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d2217ed50 con 0x562d22161dc0 2024-05-07T02:39:38.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.241+0000 7f8219400700 1 --2- 172.21.15.19:0/1742818793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d22161dc0 0x562d2217e2d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8204009ce0 tx=0x7f8204022860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8af2f6d77e8dfadd server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:38.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.241+0000 7f81f7e00700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:39:38.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.242+0000 7f81f7e00700 1 -- 172.21.15.19:0/1742818793 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820400bdc0 con 0x562d22161dc0 2024-05-07T02:39:38.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.242+0000 7f822b2cbb80 1 -- 172.21.15.19:0/1742818793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8204022480 con 0x562d22161dc0 2024-05-07T02:39:38.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.242+0000 7f81f7e00700 1 -- 172.21.15.19:0/1742818793 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9525+0+0 (secure 0 0 0) 0x7f8204003820 con 0x562d22161dc0 2024-05-07T02:39:38.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.242+0000 7f81f7e00700 1 -- 172.21.15.19:0/1742818793 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f820404c050 con 0x562d22161dc0 2024-05-07T02:39:38.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.242+0000 7f822b2cbb80 10 client.26383 fetch_fsmap learned FSMap version 46 2024-05-07T02:39:38.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.242+0000 7f822b2cbb80 1 -- 172.21.15.19:0/1742818793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x562d221e59f0 con 0x562d22161dc0 2024-05-07T02:39:38.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.242+0000 7f81f7e00700 1 -- 172.21.15.19:0/1742818793 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8204022480 con 0x562d22161dc0 2024-05-07T02:39:38.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.243+0000 7f81f7e00700 1 -- 172.21.15.19:0/1742818793 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 46) ==== 8386+0+0 (secure 0 0 0) 0x7f820403a070 con 0x562d22161dc0 2024-05-07T02:39:38.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.243+0000 7f822b2cbb80 10 client.26383 fetch_fsmap finished waiting for FSMap version 46 2024-05-07T02:39:38.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.243+0000 7f822b2cbb80 10 client.26383 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:39:38.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.243+0000 7f822b2cbb80 20 client.26383 populate_metadata read hostname 'smithi019' 2024-05-07T02:39:38.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.243+0000 7f822b2cbb80 1 --2- 172.21.15.19:0/1742818793 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x562d221e5ea0 0x562d221e82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:38.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.243+0000 7f822b2cbb80 4 client.26383 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:39:38.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.243+0000 7f822b2cbb80 1 -- 172.21.15.19:0/1742818793 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562d22091400 con 0x562d221e5ea0 2024-05-07T02:39:38.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.243+0000 7f8219e00700 1 --2- 172.21.15.19:0/1742818793 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x562d221e5ea0 0x562d221e82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:38.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.244+0000 7f8219e00700 1 --2- 172.21.15.19:0/1742818793 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x562d221e5ea0 0x562d221e82c0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f715084d2f57de7c server_cookie=cfbb82cb00251540 in_seq=0 out_seq=0 2024-05-07T02:39:38.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.244+0000 7f81f7e00700 10 client.26383 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:39:38.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.244+0000 7f81f7e00700 1 -- 172.21.15.19:0/1742818793 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562d22091400 con 0x562d221e5ea0 2024-05-07T02:39:38.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.244+0000 7f81f7e00700 10 client.26383 handle_command_reply: tid=0 2024-05-07T02:39:38.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.244+0000 7f822b2cbb80 1 client.26383 shutdown 2024-05-07T02:39:38.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.244+0000 7f822b2cbb80 1 -- 172.21.15.19:0/1742818793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d22161dc0 msgr2=0x562d2217e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:38.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.244+0000 7f822b2cbb80 1 --2- 172.21.15.19:0/1742818793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d22161dc0 0x562d2217e2d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8204009ce0 tx=0x7f8204022860 comp rx=0 tx=0).stop 2024-05-07T02:39:38.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.244+0000 7f822b2cbb80 1 -- 172.21.15.19:0/1742818793 shutdown_connections 2024-05-07T02:39:38.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.244+0000 7f822b2cbb80 1 --2- 172.21.15.19:0/1742818793 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x562d221e5ea0 0x562d221e82c0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:38.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.244+0000 7f822b2cbb80 1 --2- 172.21.15.19:0/1742818793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562d221673b0 0x562d2217e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:38.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.244+0000 7f822b2cbb80 1 --2- 172.21.15.19:0/1742818793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562d22161470 0x562d22180f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:38.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.244+0000 7f822b2cbb80 1 --2- 172.21.15.19:0/1742818793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562d22161dc0 0x562d2217e2d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:38.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.244+0000 7f81f7e00700 0 client.26383 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:39:38.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.244+0000 7f822b2cbb80 1 -- 172.21.15.19:0/1742818793 >> 172.21.15.19:0/1742818793 conn(0x562d21fe3e40 msgr2=0x562d2216cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:38.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.244+0000 7f822b2cbb80 1 -- 172.21.15.19:0/1742818793 shutdown_connections 2024-05-07T02:39:38.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.245+0000 7f822b2cbb80 1 -- 172.21.15.19:0/1742818793 wait complete. 2024-05-07T02:39:38.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.245+0000 7f822b2cbb80 20 client.26383 trim_cache size 0 max 16384 2024-05-07T02:39:38.252 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:39:38.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.245+0000 7f821c800700 1 -- 172.21.15.19:0/2377231663 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f81fc0b29f0 msgr2=0x7f81fc0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:38.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.245+0000 7f821c800700 1 --2- 172.21.15.19:0/2377231663 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f81fc0b29f0 0x7f81fc0b4f00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8210009ec0 tx=0x7f8210013250 comp rx=0 tx=0).stop 2024-05-07T02:39:38.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.245+0000 7f821c800700 1 -- 172.21.15.19:0/2377231663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8214106570 msgr2=0x7f821419d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:38.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.245+0000 7f821c800700 1 --2- 172.21.15.19:0/2377231663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8214106570 0x7f821419d0b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f820800e9f0 tx=0x7f820800ed00 comp rx=0 tx=0).stop 2024-05-07T02:39:38.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.246+0000 7f821c800700 1 -- 172.21.15.19:0/2377231663 shutdown_connections 2024-05-07T02:39:38.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.246+0000 7f821c800700 1 --2- 172.21.15.19:0/2377231663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f821410c5b0 0x7f82141a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:38.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.246+0000 7f821c800700 1 --2- 172.21.15.19:0/2377231663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8214106570 0x7f821419d0b0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:38.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.246+0000 7f821c800700 1 --2- 172.21.15.19:0/2377231663 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f81fc0b29f0 0x7f81fc0b4f00 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:38.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.246+0000 7f821c800700 1 --2- 172.21.15.19:0/2377231663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8214106ec0 0x7f821419d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:38.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.246+0000 7f821c800700 1 -- 172.21.15.19:0/2377231663 >> 172.21.15.19:0/2377231663 conn(0x7f8214102090 msgr2=0x7f8214104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:38.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.246+0000 7f821c800700 1 -- 172.21.15.19:0/2377231663 shutdown_connections 2024-05-07T02:39:38.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:38.246+0000 7f821c800700 1 -- 172.21.15.19:0/2377231663 wait complete. 2024-05-07T02:39:38.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:37 smithi064 ceph-mon[34699]: pgmap v238: 129 pgs: 129 active+clean; 177 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 762 KiB/s wr, 50 op/s 2024-05-07T02:39:38.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:37 smithi027 ceph-mon[34398]: pgmap v238: 129 pgs: 129 active+clean; 177 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 762 KiB/s wr, 50 op/s 2024-05-07T02:39:40.259 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:39 smithi027 ceph-mon[34398]: pgmap v239: 129 pgs: 129 active+clean; 177 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 480 KiB/s wr, 30 op/s 2024-05-07T02:39:40.318 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-07T02:39:40.318 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-07T02:39:40.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:39 smithi064 ceph-mon[34699]: pgmap v239: 129 pgs: 129 active+clean; 177 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 480 KiB/s wr, 30 op/s 2024-05-07T02:39:40.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:39 smithi019 ceph-mon[29781]: pgmap v239: 129 pgs: 129 active+clean; 177 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 480 KiB/s wr, 30 op/s 2024-05-07T02:39:40.816 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:39:42.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:42 smithi019 ceph-mon[29781]: pgmap v240: 129 pgs: 129 active+clean; 177 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 480 KiB/s wr, 30 op/s 2024-05-07T02:39:42.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:42 smithi064 ceph-mon[34699]: pgmap v240: 129 pgs: 129 active+clean; 177 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 480 KiB/s wr, 30 op/s 2024-05-07T02:39:42.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:42 smithi027 ceph-mon[34398]: pgmap v240: 129 pgs: 129 active+clean; 177 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 480 KiB/s wr, 30 op/s 2024-05-07T02:39:43.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.397+0000 7ff0eaa00700 1 -- 172.21.15.19:0/465005663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0e4108f20 msgr2=0x7ff0e4109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:43.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.397+0000 7ff0eaa00700 1 --2- 172.21.15.19:0/465005663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0e4108f20 0x7ff0e4109330 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0009990 tx=0x7ff0d0009ca0 comp rx=0 tx=0).stop 2024-05-07T02:39:43.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.397+0000 7ff0eaa00700 1 -- 172.21.15.19:0/465005663 shutdown_connections 2024-05-07T02:39:43.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.397+0000 7ff0eaa00700 1 --2- 172.21.15.19:0/465005663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0e4109870 0x7ff0e41111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:43.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.397+0000 7ff0eaa00700 1 --2- 172.21.15.19:0/465005663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0e41045f0 0x7ff0e4104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:43.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.397+0000 7ff0eaa00700 1 --2- 172.21.15.19:0/465005663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0e4108f20 0x7ff0e4109330 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:43.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.397+0000 7ff0eaa00700 1 -- 172.21.15.19:0/465005663 >> 172.21.15.19:0/465005663 conn(0x7ff0e4100040 msgr2=0x7ff0e4102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:43.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.398+0000 7ff0eaa00700 1 -- 172.21.15.19:0/465005663 shutdown_connections 2024-05-07T02:39:43.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.398+0000 7ff0eaa00700 1 -- 172.21.15.19:0/465005663 wait complete. 2024-05-07T02:39:43.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.399+0000 7ff0eaa00700 1 Processor -- start 2024-05-07T02:39:43.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.399+0000 7ff0eaa00700 1 -- start start 2024-05-07T02:39:43.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.399+0000 7ff0eaa00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0e41045f0 0x7ff0e419aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:43.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.399+0000 7ff0eaa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0e4108f20 0x7ff0e419b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:43.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.399+0000 7ff0eaa00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0e4109870 0x7ff0e419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:43.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.399+0000 7ff0eaa00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e4114480 con 0x7ff0e41045f0 2024-05-07T02:39:43.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.399+0000 7ff0eaa00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e4114300 con 0x7ff0e4108f20 2024-05-07T02:39:43.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.400+0000 7ff0eaa00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e4114600 con 0x7ff0e4109870 2024-05-07T02:39:43.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.400+0000 7ff0e3e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0e41045f0 0x7ff0e419aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:43.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.400+0000 7ff0e3e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0e41045f0 0x7ff0e419aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58754/0 (socket says 172.21.15.19:58754) 2024-05-07T02:39:43.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.400+0000 7ff0e3e00700 1 -- 172.21.15.19:0/601291360 learned_addr learned my addr 172.21.15.19:0/601291360 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:39:43.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.400+0000 7ff0e3400700 1 --2- 172.21.15.19:0/601291360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0e4108f20 0x7ff0e419b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:43.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.400+0000 7ff0e8a00700 1 --2- 172.21.15.19:0/601291360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0e4109870 0x7ff0e419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:43.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.400+0000 7ff0e3e00700 1 -- 172.21.15.19:0/601291360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0e4109870 msgr2=0x7ff0e419f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:43.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.400+0000 7ff0e3e00700 1 --2- 172.21.15.19:0/601291360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0e4109870 0x7ff0e419f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:43.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.400+0000 7ff0e3e00700 1 -- 172.21.15.19:0/601291360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0e4108f20 msgr2=0x7ff0e419b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:43.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.400+0000 7ff0e3e00700 1 --2- 172.21.15.19:0/601291360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0e4108f20 0x7ff0e419b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:43.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.400+0000 7ff0e3e00700 1 -- 172.21.15.19:0/601291360 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0e419fdc0 con 0x7ff0e41045f0 2024-05-07T02:39:43.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.401+0000 7ff0e3e00700 1 --2- 172.21.15.19:0/601291360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0e41045f0 0x7ff0e419aef0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff0d400e9a0 tx=0x7ff0d400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:43.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.401+0000 7ff0e0c00700 1 -- 172.21.15.19:0/601291360 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d400c9c0 con 0x7ff0e41045f0 2024-05-07T02:39:43.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.401+0000 7ff0e0c00700 1 -- 172.21.15.19:0/601291360 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0d400cb60 con 0x7ff0e41045f0 2024-05-07T02:39:43.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.401+0000 7ff0e0c00700 1 -- 172.21.15.19:0/601291360 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d4018680 con 0x7ff0e41045f0 2024-05-07T02:39:43.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.401+0000 7ff0eaa00700 1 -- 172.21.15.19:0/601291360 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0e41a00b0 con 0x7ff0e41045f0 2024-05-07T02:39:43.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.401+0000 7ff0eaa00700 1 -- 172.21.15.19:0/601291360 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0e40759f0 con 0x7ff0e41045f0 2024-05-07T02:39:43.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.406+0000 7ff0e0c00700 1 -- 172.21.15.19:0/601291360 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff0d4010430 con 0x7ff0e41045f0 2024-05-07T02:39:43.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.406+0000 7ff0f94c7b80 1 Processor -- start 2024-05-07T02:39:43.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.407+0000 7ff0f94c7b80 1 -- start start 2024-05-07T02:39:43.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.407+0000 7ff0f94c7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565371c3f4f0 0x565371c3f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:43.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.407+0000 7ff0f94c7b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565371c396a0 0x565371c39ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:43.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.407+0000 7ff0f94c7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565371c39ff0 0x565371c461d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:43.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.407+0000 7ff0f94c7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x565371b5baa0 con 0x565371c3f4f0 2024-05-07T02:39:43.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.407+0000 7ff0f94c7b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x565371af97e0 con 0x565371c396a0 2024-05-07T02:39:43.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.407+0000 7ff0f94c7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x565371b57a60 con 0x565371c39ff0 2024-05-07T02:39:43.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.408+0000 7ff0e0c00700 1 --2- 172.21.15.19:0/601291360 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff0c80b29f0 0x7ff0c80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:43.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.408+0000 7ff0e8a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565371c3f4f0 0x565371c3f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:43.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.408+0000 7ff0e3400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565371c39ff0 0x565371c461d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:43.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.408+0000 7ff0e8a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565371c3f4f0 0x565371c3f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58768/0 (socket says 172.21.15.19:58768) 2024-05-07T02:39:43.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.408+0000 7ff0e8a00700 1 -- 172.21.15.19:0/1237711431 learned_addr learned my addr 172.21.15.19:0/1237711431 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:39:43.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.408+0000 7ff0e0c00700 1 -- 172.21.15.19:0/601291360 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9525+0+0 (secure 0 0 0) 0x7ff0d40e2d00 con 0x7ff0e41045f0 2024-05-07T02:39:43.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.408+0000 7ff0e3e00700 1 --2- 172.21.15.19:0/1237711431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565371c396a0 0x565371c39ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:43.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.408+0000 7ff0e3400700 1 -- 172.21.15.19:0/1237711431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565371c396a0 msgr2=0x565371c39ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:43.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.408+0000 7ff0e3400700 1 --2- 172.21.15.19:0/1237711431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565371c396a0 0x565371c39ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:43.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.408+0000 7ff0e8a00700 1 --2- 172.21.15.19:0/601291360 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff0c80b29f0 0x7ff0c80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:43.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.408+0000 7ff0e3400700 1 -- 172.21.15.19:0/1237711431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565371c3f4f0 msgr2=0x565371c3f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:43.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.408+0000 7ff0e3400700 1 --2- 172.21.15.19:0/1237711431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565371c3f4f0 0x565371c3f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:43.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.408+0000 7ff0e3400700 1 -- 172.21.15.19:0/1237711431 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565371aa8950 con 0x565371c39ff0 2024-05-07T02:39:43.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.408+0000 7ff0e3e00700 1 --2- 172.21.15.19:0/1237711431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565371c396a0 0x565371c39ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-07T02:39:43.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.409+0000 7ff0e3400700 1 --2- 172.21.15.19:0/1237711431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565371c39ff0 0x565371c461d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff0d00098e0 tx=0x7ff0d0009570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea7464e8554ba5b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:43.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.409+0000 7ff0d9600700 1 -- 172.21.15.19:0/1237711431 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d0027070 con 0x565371c39ff0 2024-05-07T02:39:43.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.409+0000 7ff0d9600700 1 -- 172.21.15.19:0/1237711431 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0d00056b0 con 0x565371c39ff0 2024-05-07T02:39:43.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.409+0000 7ff0d9600700 1 -- 172.21.15.19:0/1237711431 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d0003b10 con 0x565371c39ff0 2024-05-07T02:39:43.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.410+0000 7ff0f94c7b80 1 -- 172.21.15.19:0/1237711431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565371c39ff0 msgr2=0x565371c461d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:43.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.410+0000 7ff0e8a00700 1 --2- 172.21.15.19:0/601291360 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff0c80b29f0 0x7ff0c80b4f00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x565371c3f070 tx=0x7ff0dc004b40 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:43.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.410+0000 7ff0f94c7b80 1 --2- 172.21.15.19:0/1237711431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565371c39ff0 0x565371c461d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff0d00098e0 tx=0x7ff0d0009570 comp rx=0 tx=0).stop 2024-05-07T02:39:43.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.410+0000 7ff0f94c7b80 1 -- 172.21.15.19:0/1237711431 shutdown_connections 2024-05-07T02:39:43.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.410+0000 7ff0f94c7b80 1 --2- 172.21.15.19:0/1237711431 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565371c39ff0 0x565371c461d0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:43.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.410+0000 7ff0f94c7b80 1 --2- 172.21.15.19:0/1237711431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565371c3f4f0 0x565371c3f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:43.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.410+0000 7ff0f94c7b80 1 --2- 172.21.15.19:0/1237711431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565371c396a0 0x565371c39ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:43.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.410+0000 7ff0f94c7b80 1 -- 172.21.15.19:0/1237711431 >> 172.21.15.19:0/1237711431 conn(0x565371abbec0 msgr2=0x565371c34400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:43.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.410+0000 7ff0f94c7b80 1 -- 172.21.15.19:0/1237711431 shutdown_connections 2024-05-07T02:39:43.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.410+0000 7ff0f94c7b80 1 -- 172.21.15.19:0/1237711431 wait complete. 2024-05-07T02:39:43.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.411+0000 7ff0f94c7b80 1 -- start start 2024-05-07T02:39:43.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.412+0000 7ff0f94c7b80 1 Processor -- start 2024-05-07T02:39:43.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.412+0000 7ff0f94c7b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565371c396a0 0x565371c43660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:43.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.412+0000 7ff0f94c7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565371c39ff0 0x565371c43ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:43.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.412+0000 7ff0f94c7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565371c3f4f0 0x565371c440e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:43.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.412+0000 7ff0f94c7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x565371c46860 con 0x565371c3f4f0 2024-05-07T02:39:43.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.412+0000 7ff0f94c7b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x565371ac1ba0 con 0x565371c396a0 2024-05-07T02:39:43.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.412+0000 7ff0f94c7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x565371c469e0 con 0x565371c39ff0 2024-05-07T02:39:43.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.412+0000 7ff0f94c7b80 10 client.0 _ll_register_callbacks cb 0x565371aef400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:39:43.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.412+0000 7ff0e3e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565371c396a0 0x565371c43660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:43.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.412+0000 7ff0e8a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565371c3f4f0 0x565371c440e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:43.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.412+0000 7ff0e3e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565371c396a0 0x565371c43660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:58520/0 (socket says 172.21.15.19:58520) 2024-05-07T02:39:43.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.413+0000 7ff0e3e00700 1 -- 172.21.15.19:0/678590620 learned_addr learned my addr 172.21.15.19:0/678590620 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:39:43.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.413+0000 7ff0e3400700 1 --2- 172.21.15.19:0/678590620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565371c39ff0 0x565371c43ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:43.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.413+0000 7ff0e3e00700 1 -- 172.21.15.19:0/678590620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565371c39ff0 msgr2=0x565371c43ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:43.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.413+0000 7ff0e3e00700 1 --2- 172.21.15.19:0/678590620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565371c39ff0 0x565371c43ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:43.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.413+0000 7ff0e3e00700 1 -- 172.21.15.19:0/678590620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565371c3f4f0 msgr2=0x565371c440e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:43.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.413+0000 7ff0e3e00700 1 --2- 172.21.15.19:0/678590620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565371c3f4f0 0x565371c440e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:43.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.413+0000 7ff0e3e00700 1 -- 172.21.15.19:0/678590620 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565371b214a0 con 0x565371c396a0 2024-05-07T02:39:43.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.413+0000 7ff0e3e00700 1 --2- 172.21.15.19:0/678590620 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565371c396a0 0x565371c43660 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff0d400c5b0 tx=0x7ff0d40e0560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd01d7091394d450 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:43.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.413+0000 7ff0ce000700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:39:43.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.414+0000 7ff0ce000700 1 -- 172.21.15.19:0/678590620 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d40aa290 con 0x565371c396a0 2024-05-07T02:39:43.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.414+0000 7ff0f94c7b80 1 -- 172.21.15.19:0/678590620 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0d00056b0 con 0x565371c396a0 2024-05-07T02:39:43.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.414+0000 7ff0ce000700 1 -- 172.21.15.19:0/678590620 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9525+0+0 (secure 0 0 0) 0x7ff0d40b2770 con 0x565371c396a0 2024-05-07T02:39:43.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.414+0000 7ff0ce000700 1 -- 172.21.15.19:0/678590620 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7ff0d40a9b90 con 0x565371c396a0 2024-05-07T02:39:43.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.414+0000 7ff0f94c7b80 10 client.26432 fetch_fsmap learned FSMap version 46 2024-05-07T02:39:43.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.414+0000 7ff0f94c7b80 1 -- 172.21.15.19:0/678590620 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x565371c513d0 con 0x565371c396a0 2024-05-07T02:39:43.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.414+0000 7ff0ce000700 1 -- 172.21.15.19:0/678590620 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0d00056b0 con 0x565371c396a0 2024-05-07T02:39:43.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.414+0000 7ff0ce000700 1 -- 172.21.15.19:0/678590620 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 46) ==== 8386+0+0 (secure 0 0 0) 0x7ff0d40a5500 con 0x565371c396a0 2024-05-07T02:39:43.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.415+0000 7ff0f94c7b80 10 client.26432 fetch_fsmap finished waiting for FSMap version 46 2024-05-07T02:39:43.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.415+0000 7ff0f94c7b80 10 client.26432 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:39:43.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.415+0000 7ff0f94c7b80 20 client.26432 populate_metadata read hostname 'smithi019' 2024-05-07T02:39:43.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.415+0000 7ff0f94c7b80 1 --2- 172.21.15.19:0/678590620 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x565371c538f0 0x565371c55dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:43.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.415+0000 7ff0f94c7b80 4 client.26432 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:39:43.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.415+0000 7ff0f94c7b80 1 -- 172.21.15.19:0/678590620 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565371c06160 con 0x565371c538f0 2024-05-07T02:39:43.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.415+0000 7ff0e3400700 1 --2- 172.21.15.19:0/678590620 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x565371c538f0 0x565371c55dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:43.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.416+0000 7ff0e3400700 1 --2- 172.21.15.19:0/678590620 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x565371c538f0 0x565371c55dc0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3c81ccb1e979d67 server_cookie=317c5b04f0869481 in_seq=0 out_seq=0 2024-05-07T02:39:43.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.416+0000 7ff0ce000700 10 client.26432 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:39:43.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.416+0000 7ff0ce000700 1 -- 172.21.15.19:0/678590620 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x565371c06160 con 0x565371c538f0 2024-05-07T02:39:43.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.416+0000 7ff0ce000700 10 client.26432 handle_command_reply: tid=0 2024-05-07T02:39:43.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.417+0000 7ff0f94c7b80 1 client.26432 shutdown 2024-05-07T02:39:43.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.417+0000 7ff0f94c7b80 1 -- 172.21.15.19:0/678590620 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565371c396a0 msgr2=0x565371c43660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:43.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.417+0000 7ff0f94c7b80 1 --2- 172.21.15.19:0/678590620 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565371c396a0 0x565371c43660 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff0d400c5b0 tx=0x7ff0d40e0560 comp rx=0 tx=0).stop 2024-05-07T02:39:43.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.417+0000 7ff0f94c7b80 1 -- 172.21.15.19:0/678590620 shutdown_connections 2024-05-07T02:39:43.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.417+0000 7ff0f94c7b80 1 --2- 172.21.15.19:0/678590620 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x565371c538f0 0x565371c55dc0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:43.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.417+0000 7ff0f94c7b80 1 --2- 172.21.15.19:0/678590620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565371c3f4f0 0x565371c440e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:43.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.417+0000 7ff0ce000700 0 client.26432 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:39:43.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.417+0000 7ff0f94c7b80 1 --2- 172.21.15.19:0/678590620 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565371c396a0 0x565371c43660 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:43.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.417+0000 7ff0f94c7b80 1 --2- 172.21.15.19:0/678590620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565371c39ff0 0x565371c43ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:43.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.417+0000 7ff0f94c7b80 1 -- 172.21.15.19:0/678590620 >> 172.21.15.19:0/678590620 conn(0x565371abbec0 msgr2=0x565371c38690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:43.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.417+0000 7ff0f94c7b80 1 -- 172.21.15.19:0/678590620 shutdown_connections 2024-05-07T02:39:43.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.417+0000 7ff0f94c7b80 1 -- 172.21.15.19:0/678590620 wait complete. 2024-05-07T02:39:43.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.417+0000 7ff0f94c7b80 20 client.26432 trim_cache size 0 max 16384 2024-05-07T02:39:43.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.440+0000 7ff0f94c7b80 1 Processor -- start 2024-05-07T02:39:43.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.440+0000 7ff0f94c7b80 1 -- start start 2024-05-07T02:39:43.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.440+0000 7ff0f94c7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565371c396a0 0x565371c2ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:43.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.440+0000 7ff0f94c7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565371c39ff0 0x565371c3de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:43.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.440+0000 7ff0f94c7b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565371c3f4f0 0x565371c40f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:43.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.440+0000 7ff0f94c7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e41b2960 con 0x565371c396a0 2024-05-07T02:39:43.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.440+0000 7ff0f94c7b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e41b45c0 con 0x565371c3f4f0 2024-05-07T02:39:43.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.440+0000 7ff0f94c7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e41b3030 con 0x565371c39ff0 2024-05-07T02:39:43.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.441+0000 7ff0e3e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565371c396a0 0x565371c2ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:43.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.441+0000 7ff0e3e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565371c396a0 0x565371c2ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58792/0 (socket says 172.21.15.19:58792) 2024-05-07T02:39:43.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.441+0000 7ff0e3400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565371c39ff0 0x565371c3de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:43.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.441+0000 7ff0e3e00700 1 -- 172.21.15.19:0/661472292 learned_addr learned my addr 172.21.15.19:0/661472292 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:39:43.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.441+0000 7ff0e8a00700 1 --2- 172.21.15.19:0/661472292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565371c3f4f0 0x565371c40f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:43.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.441+0000 7ff0e3400700 1 -- 172.21.15.19:0/661472292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565371c3f4f0 msgr2=0x565371c40f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:43.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.441+0000 7ff0e3400700 1 --2- 172.21.15.19:0/661472292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565371c3f4f0 0x565371c40f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:43.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.441+0000 7ff0e3400700 1 -- 172.21.15.19:0/661472292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565371c396a0 msgr2=0x565371c2ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:43.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.441+0000 7ff0e3400700 1 --2- 172.21.15.19:0/661472292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565371c396a0 0x565371c2ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:43.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.441+0000 7ff0e3400700 1 -- 172.21.15.19:0/661472292 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565371c30c00 con 0x565371c39ff0 2024-05-07T02:39:43.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.442+0000 7ff0e3e00700 1 --2- 172.21.15.19:0/661472292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565371c396a0 0x565371c2ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:39:43.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.442+0000 7ff0e3400700 1 --2- 172.21.15.19:0/661472292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565371c39ff0 0x565371c3de10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0000c00 tx=0x7ff0d001a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=781ec02775398560 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:43.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.442+0000 7ff0d8c00700 1 -- 172.21.15.19:0/661472292 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d0027040 con 0x565371c39ff0 2024-05-07T02:39:43.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.442+0000 7ff0d8c00700 1 -- 172.21.15.19:0/661472292 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0d00036e0 con 0x565371c39ff0 2024-05-07T02:39:43.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.442+0000 7ff0d8c00700 1 -- 172.21.15.19:0/661472292 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d00323e0 con 0x565371c39ff0 2024-05-07T02:39:43.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.443+0000 7ff0f94c7b80 1 -- 172.21.15.19:0/661472292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565371c39ff0 msgr2=0x565371c3de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:43.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.443+0000 7ff0f94c7b80 1 --2- 172.21.15.19:0/661472292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565371c39ff0 0x565371c3de10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0000c00 tx=0x7ff0d001a040 comp rx=0 tx=0).stop 2024-05-07T02:39:43.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.443+0000 7ff0f94c7b80 1 -- 172.21.15.19:0/661472292 shutdown_connections 2024-05-07T02:39:43.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.443+0000 7ff0f94c7b80 1 --2- 172.21.15.19:0/661472292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565371c3f4f0 0x565371c40f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:43.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.443+0000 7ff0f94c7b80 1 --2- 172.21.15.19:0/661472292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565371c396a0 0x565371c2ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:43.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.443+0000 7ff0f94c7b80 1 --2- 172.21.15.19:0/661472292 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565371c39ff0 0x565371c3de10 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:43.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.443+0000 7ff0f94c7b80 1 -- 172.21.15.19:0/661472292 >> 172.21.15.19:0/661472292 conn(0x565371abbec0 msgr2=0x565371c43c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:43.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.443+0000 7ff0f94c7b80 1 -- 172.21.15.19:0/661472292 shutdown_connections 2024-05-07T02:39:43.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.443+0000 7ff0f94c7b80 1 -- 172.21.15.19:0/661472292 wait complete. 2024-05-07T02:39:43.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.444+0000 7ff0f94c7b80 1 -- start start 2024-05-07T02:39:43.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.444+0000 7ff0f94c7b80 1 Processor -- start 2024-05-07T02:39:43.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.444+0000 7ff0f94c7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565371c396a0 0x565371c48c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:43.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.444+0000 7ff0f94c7b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565371c39ff0 0x565371c49180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:43.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.445+0000 7ff0f94c7b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565371c3f4f0 0x565371c496c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:43.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.445+0000 7ff0f94c7b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e41b3e30 con 0x565371c39ff0 2024-05-07T02:39:43.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.445+0000 7ff0f94c7b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e41b3700 con 0x565371c3f4f0 2024-05-07T02:39:43.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.445+0000 7ff0f94c7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e41b0940 con 0x565371c396a0 2024-05-07T02:39:43.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.445+0000 7ff0f94c7b80 10 client.0 _ll_register_callbacks cb 0x565371aef400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:39:43.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.445+0000 7ff0e3e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565371c396a0 0x565371c48c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:43.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.445+0000 7ff0e3e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565371c396a0 0x565371c48c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:36426/0 (socket says 172.21.15.19:36426) 2024-05-07T02:39:43.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.445+0000 7ff0e3400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565371c39ff0 0x565371c49180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:43.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.445+0000 7ff0e3e00700 1 -- 172.21.15.19:0/1688169468 learned_addr learned my addr 172.21.15.19:0/1688169468 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:39:43.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.445+0000 7ff0e8a00700 1 --2- 172.21.15.19:0/1688169468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565371c3f4f0 0x565371c496c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:43.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.445+0000 7ff0e3e00700 1 -- 172.21.15.19:0/1688169468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565371c3f4f0 msgr2=0x565371c496c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:43.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.445+0000 7ff0e3e00700 1 --2- 172.21.15.19:0/1688169468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565371c3f4f0 0x565371c496c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:43.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.445+0000 7ff0e3e00700 1 -- 172.21.15.19:0/1688169468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565371c39ff0 msgr2=0x565371c49180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:43.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.445+0000 7ff0e3e00700 1 --2- 172.21.15.19:0/1688169468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565371c39ff0 0x565371c49180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:43.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.445+0000 7ff0e3e00700 1 -- 172.21.15.19:0/1688169468 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565371c4c6a0 con 0x565371c396a0 2024-05-07T02:39:43.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.445+0000 7ff0e8a00700 1 --2- 172.21.15.19:0/1688169468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565371c3f4f0 0x565371c496c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:39:43.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.446+0000 7ff0e3e00700 1 --2- 172.21.15.19:0/1688169468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565371c396a0 0x565371c48c40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff0d400c5b0 tx=0x7ff0d40a6030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=392b96ccd4c53c33 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:43.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.446+0000 7ff0ce000700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:39:43.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.446+0000 7ff0ce000700 1 -- 172.21.15.19:0/1688169468 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d40aa290 con 0x565371c396a0 2024-05-07T02:39:43.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.446+0000 7ff0f94c7b80 1 -- 172.21.15.19:0/1688169468 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0d00036e0 con 0x565371c396a0 2024-05-07T02:39:43.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.446+0000 7ff0ce000700 1 -- 172.21.15.19:0/1688169468 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9525+0+0 (secure 0 0 0) 0x7ff0d40a5500 con 0x565371c396a0 2024-05-07T02:39:43.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.446+0000 7ff0ce000700 1 -- 172.21.15.19:0/1688169468 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7ff0d40a5700 con 0x565371c396a0 2024-05-07T02:39:43.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.446+0000 7ff0f94c7b80 10 client.26401 fetch_fsmap learned FSMap version 46 2024-05-07T02:39:43.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.446+0000 7ff0f94c7b80 1 -- 172.21.15.19:0/1688169468 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x565371cbf1d0 con 0x565371c396a0 2024-05-07T02:39:43.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.447+0000 7ff0ce000700 1 -- 172.21.15.19:0/1688169468 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0d00036e0 con 0x565371c396a0 2024-05-07T02:39:43.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.447+0000 7ff0ce000700 1 -- 172.21.15.19:0/1688169468 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 46) ==== 8386+0+0 (secure 0 0 0) 0x7ff0d4086020 con 0x565371c396a0 2024-05-07T02:39:43.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.447+0000 7ff0f94c7b80 10 client.26401 fetch_fsmap finished waiting for FSMap version 46 2024-05-07T02:39:43.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.447+0000 7ff0f94c7b80 10 client.26401 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:39:43.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.447+0000 7ff0f94c7b80 20 client.26401 populate_metadata read hostname 'smithi019' 2024-05-07T02:39:43.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.447+0000 7ff0f94c7b80 1 --2- 172.21.15.19:0/1688169468 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x565371c538f0 0x565371cc1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:43.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.447+0000 7ff0f94c7b80 4 client.26401 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-07T02:39:43.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.447+0000 7ff0f94c7b80 1 -- 172.21.15.19:0/1688169468 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x565371b69480 con 0x565371c538f0 2024-05-07T02:39:43.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.447+0000 7ff0e3400700 1 --2- 172.21.15.19:0/1688169468 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x565371c538f0 0x565371cc1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:43.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.448+0000 7ff0e3400700 1 --2- 172.21.15.19:0/1688169468 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x565371c538f0 0x565371cc1a10 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faf30d92ea37f90a server_cookie=363f76f6283941b3 in_seq=0 out_seq=0 2024-05-07T02:39:43.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.448+0000 7ff0ce000700 10 client.26401 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:39:43.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.449+0000 7ff0ce000700 1 -- 172.21.15.19:0/1688169468 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x565371b69480 con 0x565371c538f0 2024-05-07T02:39:43.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.449+0000 7ff0ce000700 10 client.26401 handle_command_reply: tid=0 2024-05-07T02:39:43.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.449+0000 7ff0f94c7b80 1 client.26401 shutdown 2024-05-07T02:39:43.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.450+0000 7ff0f94c7b80 1 -- 172.21.15.19:0/1688169468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565371c396a0 msgr2=0x565371c48c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:43.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.450+0000 7ff0f94c7b80 1 --2- 172.21.15.19:0/1688169468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565371c396a0 0x565371c48c40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff0d400c5b0 tx=0x7ff0d40a6030 comp rx=0 tx=0).stop 2024-05-07T02:39:43.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.450+0000 7ff0f94c7b80 1 -- 172.21.15.19:0/1688169468 shutdown_connections 2024-05-07T02:39:43.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.450+0000 7ff0f94c7b80 1 --2- 172.21.15.19:0/1688169468 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x565371c538f0 0x565371cc1a10 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:43.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.450+0000 7ff0f94c7b80 1 --2- 172.21.15.19:0/1688169468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x565371c3f4f0 0x565371c496c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:43.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.450+0000 7ff0f94c7b80 1 --2- 172.21.15.19:0/1688169468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565371c396a0 0x565371c48c40 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:43.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.450+0000 7ff0f94c7b80 1 --2- 172.21.15.19:0/1688169468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565371c39ff0 0x565371c49180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:43.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.450+0000 7ff0f94c7b80 1 -- 172.21.15.19:0/1688169468 >> 172.21.15.19:0/1688169468 conn(0x565371abbec0 msgr2=0x565371c38690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:43.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.450+0000 7ff0ce000700 0 client.26401 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:39:43.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.450+0000 7ff0f94c7b80 1 -- 172.21.15.19:0/1688169468 shutdown_connections 2024-05-07T02:39:43.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.450+0000 7ff0f94c7b80 1 -- 172.21.15.19:0/1688169468 wait complete. 2024-05-07T02:39:43.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.450+0000 7ff0f94c7b80 20 client.26401 trim_cache size 0 max 16384 2024-05-07T02:39:43.456 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:39:43.457 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-07T02:39:43.457 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "de7896a7-2478-4f61-a1bb-f0e8975503a1", 2024-05-07T02:39:43.457 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-07T02:39:43.457 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:39:43.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.451+0000 7ff0eaa00700 1 -- 172.21.15.19:0/601291360 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff0c80b29f0 msgr2=0x7ff0c80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:43.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.451+0000 7ff0eaa00700 1 --2- 172.21.15.19:0/601291360 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff0c80b29f0 0x7ff0c80b4f00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x565371c3f070 tx=0x7ff0dc004b40 comp rx=0 tx=0).stop 2024-05-07T02:39:43.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.451+0000 7ff0eaa00700 1 -- 172.21.15.19:0/601291360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0e41045f0 msgr2=0x7ff0e419aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:43.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.451+0000 7ff0eaa00700 1 --2- 172.21.15.19:0/601291360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0e41045f0 0x7ff0e419aef0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff0d400e9a0 tx=0x7ff0d400ecb0 comp rx=0 tx=0).stop 2024-05-07T02:39:43.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.451+0000 7ff0eaa00700 1 -- 172.21.15.19:0/601291360 shutdown_connections 2024-05-07T02:39:43.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.451+0000 7ff0eaa00700 1 --2- 172.21.15.19:0/601291360 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0e4109870 0x7ff0e419f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:43.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.451+0000 7ff0eaa00700 1 --2- 172.21.15.19:0/601291360 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff0c80b29f0 0x7ff0c80b4f00 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:43.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.451+0000 7ff0eaa00700 1 --2- 172.21.15.19:0/601291360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0e41045f0 0x7ff0e419aef0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:43.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.451+0000 7ff0eaa00700 1 --2- 172.21.15.19:0/601291360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0e4108f20 0x7ff0e419b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:43.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.451+0000 7ff0eaa00700 1 -- 172.21.15.19:0/601291360 >> 172.21.15.19:0/601291360 conn(0x7ff0e4100040 msgr2=0x7ff0e410a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:43.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.451+0000 7ff0eaa00700 1 -- 172.21.15.19:0/601291360 shutdown_connections 2024-05-07T02:39:43.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:43.452+0000 7ff0eaa00700 1 -- 172.21.15.19:0/601291360 wait complete. 2024-05-07T02:39:44.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:44 smithi019 ceph-mon[29781]: pgmap v241: 129 pgs: 129 active+clean; 176 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 549 KiB/s wr, 41 op/s 2024-05-07T02:39:44.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:44 smithi019 ceph-mon[29781]: scrub queued for path: / 2024-05-07T02:39:44.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:44 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:39:44.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:44 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:39:44.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:44 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:39:44.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:44 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:39:44.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:44 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:39:44.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:44 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:39:44.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:44 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:39:44.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:44 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:39:44.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:44 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:39:44.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:44 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:39:44.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:44 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:39:44.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:44 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:39:44.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:44 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:39:44.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:44 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:39:44.149 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:44 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:39:44.149 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:44 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:39:44.149 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:44 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:39:44.149 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:44 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:39:44.149 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:44 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:39:44.149 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:44 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:39:44.149 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:44 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:39:44.149 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:44 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:39:44.149 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:44 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:39:44.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:44 smithi064 ceph-mon[34699]: pgmap v241: 129 pgs: 129 active+clean; 176 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 549 KiB/s wr, 41 op/s 2024-05-07T02:39:44.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:44 smithi064 ceph-mon[34699]: scrub queued for path: / 2024-05-07T02:39:44.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:44 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:39:44.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:44 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:39:44.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:44 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:39:44.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:44 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:39:44.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:44 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:39:44.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:44 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:39:44.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:44 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:39:44.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:44 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:39:44.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:44 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:39:44.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:44 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:39:44.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:44 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:39:44.361 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:44 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:39:44.361 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:44 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:39:44.361 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:44 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:39:44.361 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:44 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:39:44.361 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:44 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:39:44.361 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:44 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:39:44.361 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:44 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:39:44.361 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:44 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:39:44.361 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:44 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:39:44.362 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:44 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:39:44.362 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:44 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:39:44.362 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:44 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:39:44.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:44 smithi027 ceph-mon[34398]: pgmap v241: 129 pgs: 129 active+clean; 176 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 549 KiB/s wr, 41 op/s 2024-05-07T02:39:44.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:44 smithi027 ceph-mon[34398]: scrub queued for path: / 2024-05-07T02:39:44.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:44 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:39:44.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:44 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:39:44.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:44 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:39:44.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:44 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:39:44.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:44 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:39:44.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:44 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:39:44.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:44 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:39:44.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:44 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:39:44.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:44 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:39:44.459 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:44 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:39:44.459 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:44 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:39:44.459 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:44 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:39:44.459 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:44 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:39:44.459 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:44 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:39:44.459 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:44 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:39:44.459 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:44 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:39:44.459 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:44 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:39:44.459 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:44 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:39:44.459 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:44 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:39:44.460 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:44 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:39:44.460 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:44 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:39:44.460 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:44 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:39:44.460 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:44 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:39:44.475 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-07T02:39:44.965 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:39:46.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:46 smithi064 ceph-mon[34699]: pgmap v242: 129 pgs: 129 active+clean; 176 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 284 KiB/s wr, 25 op/s 2024-05-07T02:39:46.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:46 smithi019 ceph-mon[29781]: pgmap v242: 129 pgs: 129 active+clean; 176 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 284 KiB/s wr, 25 op/s 2024-05-07T02:39:46.456 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:46 smithi027 ceph-mon[34398]: pgmap v242: 129 pgs: 129 active+clean; 176 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 284 KiB/s wr, 25 op/s 2024-05-07T02:39:47.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.673+0000 7f2c26e00700 1 -- 172.21.15.19:0/1002333492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c2010f6c0 msgr2=0x7f2c2010fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:47.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.673+0000 7f2c26e00700 1 --2- 172.21.15.19:0/1002333492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c2010f6c0 0x7f2c2010fad0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2c14009990 tx=0x7f2c14009ca0 comp rx=0 tx=0).stop 2024-05-07T02:39:47.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.673+0000 7f2c26e00700 1 -- 172.21.15.19:0/1002333492 shutdown_connections 2024-05-07T02:39:47.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.673+0000 7f2c26e00700 1 --2- 172.21.15.19:0/1002333492 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c2010b110 0x7f2c20117620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:47.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.673+0000 7f2c26e00700 1 --2- 172.21.15.19:0/1002333492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c2010f6c0 0x7f2c2010fad0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:47.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.673+0000 7f2c26e00700 1 --2- 172.21.15.19:0/1002333492 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c2010a7c0 0x7f2c2010abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:47.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.673+0000 7f2c26e00700 1 -- 172.21.15.19:0/1002333492 >> 172.21.15.19:0/1002333492 conn(0x7f2c20076b00 msgr2=0x7f2c20076f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:47.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.674+0000 7f2c26e00700 1 -- 172.21.15.19:0/1002333492 shutdown_connections 2024-05-07T02:39:47.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.674+0000 7f2c26e00700 1 -- 172.21.15.19:0/1002333492 wait complete. 2024-05-07T02:39:47.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.675+0000 7f2c26e00700 1 Processor -- start 2024-05-07T02:39:47.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.675+0000 7f2c26e00700 1 -- start start 2024-05-07T02:39:47.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.675+0000 7f2c26e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c2010a7c0 0x7f2c201a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:47.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.675+0000 7f2c26e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c2010b110 0x7f2c201a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:47.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.675+0000 7f2c26e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c2010f6c0 0x7f2c201a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:47.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.675+0000 7f2c26e00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2011a920 con 0x7f2c2010b110 2024-05-07T02:39:47.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.675+0000 7f2c26e00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2011a7a0 con 0x7f2c2010f6c0 2024-05-07T02:39:47.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.675+0000 7f2c26e00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2011aaa0 con 0x7f2c2010a7c0 2024-05-07T02:39:47.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.676+0000 7f2c25000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c2010b110 0x7f2c201a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:47.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.676+0000 7f2c25000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c2010b110 0x7f2c201a1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58806/0 (socket says 172.21.15.19:58806) 2024-05-07T02:39:47.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.676+0000 7f2c25000700 1 -- 172.21.15.19:0/2988236709 learned_addr learned my addr 172.21.15.19:0/2988236709 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:39:47.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.676+0000 7f2c25a00700 1 --2- 172.21.15.19:0/2988236709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c2010a7c0 0x7f2c201a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:47.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.676+0000 7f2c26400700 1 --2- 172.21.15.19:0/2988236709 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c2010f6c0 0x7f2c201a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:47.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.676+0000 7f2c25a00700 1 -- 172.21.15.19:0/2988236709 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c2010f6c0 msgr2=0x7f2c201a5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:47.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.676+0000 7f2c25a00700 1 --2- 172.21.15.19:0/2988236709 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c2010f6c0 0x7f2c201a5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:47.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.676+0000 7f2c25a00700 1 -- 172.21.15.19:0/2988236709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c2010b110 msgr2=0x7f2c201a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:47.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.676+0000 7f2c25a00700 1 --2- 172.21.15.19:0/2988236709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c2010b110 0x7f2c201a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:47.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.676+0000 7f2c25a00700 1 -- 172.21.15.19:0/2988236709 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c201a6140 con 0x7f2c2010a7c0 2024-05-07T02:39:47.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.676+0000 7f2c25000700 1 --2- 172.21.15.19:0/2988236709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c2010b110 0x7f2c201a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:39:47.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.677+0000 7f2c25a00700 1 --2- 172.21.15.19:0/2988236709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c2010a7c0 0x7f2c201a1270 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2c14000c00 tx=0x7f2c140047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:47.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.677+0000 7f2c1e000700 1 -- 172.21.15.19:0/2988236709 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c14026070 con 0x7f2c2010a7c0 2024-05-07T02:39:47.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.677+0000 7f2c26e00700 1 -- 172.21.15.19:0/2988236709 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c201a63d0 con 0x7f2c2010a7c0 2024-05-07T02:39:47.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.677+0000 7f2c1e000700 1 -- 172.21.15.19:0/2988236709 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c14021410 con 0x7f2c2010a7c0 2024-05-07T02:39:47.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.677+0000 7f2c1e000700 1 -- 172.21.15.19:0/2988236709 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c14017650 con 0x7f2c2010a7c0 2024-05-07T02:39:47.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.677+0000 7f2c26e00700 1 -- 172.21.15.19:0/2988236709 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c201add50 con 0x7f2c2010a7c0 2024-05-07T02:39:47.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.681+0000 7f2c1e000700 1 -- 172.21.15.19:0/2988236709 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2c14029070 con 0x7f2c2010a7c0 2024-05-07T02:39:47.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.682+0000 7f2c36fd6b80 1 Processor -- start 2024-05-07T02:39:47.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.682+0000 7f2c36fd6b80 1 -- start start 2024-05-07T02:39:47.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.682+0000 7f2c36fd6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf396eaa90 0x55cf396eaea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:47.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.682+0000 7f2c36fd6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf396eb3e0 0x55cf396f4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:47.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.682+0000 7f2c36fd6b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf396f4b60 0x55cf396f6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:47.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.682+0000 7f2c36fd6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55cf3960ca20 con 0x55cf396eb3e0 2024-05-07T02:39:47.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.682+0000 7f2c36fd6b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55cf395aa760 con 0x55cf396f4b60 2024-05-07T02:39:47.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.682+0000 7f2c36fd6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55cf396089e0 con 0x55cf396eaa90 2024-05-07T02:39:47.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.683+0000 7f2c25000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf396f4b60 0x55cf396f6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:47.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.683+0000 7f2c25000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf396f4b60 0x55cf396f6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:58562/0 (socket says 172.21.15.19:58562) 2024-05-07T02:39:47.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.683+0000 7f2c25000700 1 -- 172.21.15.19:0/1242458898 learned_addr learned my addr 172.21.15.19:0/1242458898 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:39:47.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.683+0000 7f2c25a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf396eb3e0 0x55cf396f4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:47.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.683+0000 7f2c26400700 1 --2- 172.21.15.19:0/1242458898 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf396eaa90 0x55cf396eaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:47.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.683+0000 7f2c1e000700 1 --2- 172.21.15.19:0/2988236709 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f2c080b2a70 0x7f2c080b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:47.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.683+0000 7f2c25000700 1 -- 172.21.15.19:0/1242458898 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf396eaa90 msgr2=0x55cf396eaea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:47.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.683+0000 7f2c25000700 1 --2- 172.21.15.19:0/1242458898 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf396eaa90 0x55cf396eaea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:47.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.683+0000 7f2c25000700 1 -- 172.21.15.19:0/1242458898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf396eb3e0 msgr2=0x55cf396f4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:47.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.683+0000 7f2c25000700 1 --2- 172.21.15.19:0/1242458898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf396eb3e0 0x55cf396f4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:47.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.683+0000 7f2c25000700 1 -- 172.21.15.19:0/1242458898 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf395598d0 con 0x55cf396f4b60 2024-05-07T02:39:47.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.683+0000 7f2c1e000700 1 -- 172.21.15.19:0/2988236709 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9525+0+0 (secure 0 0 0) 0x7f2c140e36b0 con 0x7f2c2010a7c0 2024-05-07T02:39:47.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.683+0000 7f2c26400700 1 --2- 172.21.15.19:0/2988236709 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f2c080b2a70 0x7f2c080b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:47.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.684+0000 7f2c25000700 1 --2- 172.21.15.19:0/1242458898 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf396f4b60 0x55cf396f6f80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c011680 tx=0x7f2c0c011990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a1e5d6f96997a5c server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:47.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.684+0000 7f2c12a00700 1 -- 172.21.15.19:0/1242458898 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c01e070 con 0x55cf396f4b60 2024-05-07T02:39:47.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.684+0000 7f2c12a00700 1 -- 172.21.15.19:0/1242458898 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c0c00fb30 con 0x55cf396f4b60 2024-05-07T02:39:47.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.684+0000 7f2c26400700 1 --2- 172.21.15.19:0/2988236709 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f2c080b2a70 0x7f2c080b4f80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2c18004a60 tx=0x7f2c18009040 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:47.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.684+0000 7f2c12a00700 1 -- 172.21.15.19:0/1242458898 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c019be0 con 0x55cf396f4b60 2024-05-07T02:39:47.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.685+0000 7f2c36fd6b80 1 -- 172.21.15.19:0/1242458898 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf396f4b60 msgr2=0x55cf396f6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:47.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.685+0000 7f2c36fd6b80 1 --2- 172.21.15.19:0/1242458898 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf396f4b60 0x55cf396f6f80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c011680 tx=0x7f2c0c011990 comp rx=0 tx=0).stop 2024-05-07T02:39:47.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.685+0000 7f2c36fd6b80 1 -- 172.21.15.19:0/1242458898 shutdown_connections 2024-05-07T02:39:47.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.685+0000 7f2c36fd6b80 1 --2- 172.21.15.19:0/1242458898 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf396f4b60 0x55cf396f6f80 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:47.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.685+0000 7f2c36fd6b80 1 --2- 172.21.15.19:0/1242458898 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf396eaa90 0x55cf396eaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:47.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.685+0000 7f2c36fd6b80 1 --2- 172.21.15.19:0/1242458898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf396eb3e0 0x55cf396f4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:47.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.685+0000 7f2c36fd6b80 1 -- 172.21.15.19:0/1242458898 >> 172.21.15.19:0/1242458898 conn(0x55cf3956ce40 msgr2=0x55cf396e5880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:47.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.685+0000 7f2c36fd6b80 1 -- 172.21.15.19:0/1242458898 shutdown_connections 2024-05-07T02:39:47.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.686+0000 7f2c36fd6b80 1 -- 172.21.15.19:0/1242458898 wait complete. 2024-05-07T02:39:47.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.686+0000 7f2c36fd6b80 1 -- start start 2024-05-07T02:39:47.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.686+0000 7f2c36fd6b80 1 Processor -- start 2024-05-07T02:39:47.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.687+0000 7f2c36fd6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf396eaa90 0x55cf396f1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:47.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.687+0000 7f2c36fd6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf396eb3e0 0x55cf396f1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:47.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.687+0000 7f2c36fd6b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf396f4b60 0x55cf396f2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:47.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.687+0000 7f2c36fd6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55cf396f7960 con 0x55cf396eb3e0 2024-05-07T02:39:47.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.687+0000 7f2c36fd6b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55cf39572b20 con 0x55cf396f4b60 2024-05-07T02:39:47.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.687+0000 7f2c36fd6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55cf396f7ae0 con 0x55cf396eaa90 2024-05-07T02:39:47.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.687+0000 7f2c36fd6b80 10 client.0 _ll_register_callbacks cb 0x55cf395b3c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:39:47.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.687+0000 7f2c25a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf396eaa90 0x55cf396f1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:47.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.687+0000 7f2c25000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf396eb3e0 0x55cf396f1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:47.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.687+0000 7f2c25a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf396eaa90 0x55cf396f1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:36466/0 (socket says 172.21.15.19:36466) 2024-05-07T02:39:47.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.687+0000 7f2c25a00700 1 -- 172.21.15.19:0/1391867203 learned_addr learned my addr 172.21.15.19:0/1391867203 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:39:47.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.687+0000 7f2c26400700 1 --2- 172.21.15.19:0/1391867203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf396f4b60 0x55cf396f2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:47.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.688+0000 7f2c25a00700 1 -- 172.21.15.19:0/1391867203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf396f4b60 msgr2=0x55cf396f2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:47.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.688+0000 7f2c25a00700 1 --2- 172.21.15.19:0/1391867203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf396f4b60 0x55cf396f2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:47.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.688+0000 7f2c25a00700 1 -- 172.21.15.19:0/1391867203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf396eb3e0 msgr2=0x55cf396f1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:47.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.688+0000 7f2c25a00700 1 --2- 172.21.15.19:0/1391867203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf396eb3e0 0x55cf396f1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:47.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.688+0000 7f2c25a00700 1 -- 172.21.15.19:0/1391867203 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf395d2420 con 0x55cf396eaa90 2024-05-07T02:39:47.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.688+0000 7f2c25a00700 1 --2- 172.21.15.19:0/1391867203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf396eaa90 0x55cf396f1a80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2c140a6650 tx=0x7f2c14005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1152dea569e35fc3 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:47.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.688+0000 7f2c07400700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:39:47.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.688+0000 7f2c07400700 1 -- 172.21.15.19:0/1391867203 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c140a9f20 con 0x55cf396eaa90 2024-05-07T02:39:47.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.689+0000 7f2c07400700 1 -- 172.21.15.19:0/1391867203 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9525+0+0 (secure 0 0 0) 0x7f2c140e0c30 con 0x55cf396eaa90 2024-05-07T02:39:47.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.689+0000 7f2c36fd6b80 1 -- 172.21.15.19:0/1391867203 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c0c00fb30 con 0x55cf396eaa90 2024-05-07T02:39:47.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.689+0000 7f2c07400700 1 -- 172.21.15.19:0/1391867203 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f2c140a9d40 con 0x55cf396eaa90 2024-05-07T02:39:47.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.689+0000 7f2c36fd6b80 10 client.26413 fetch_fsmap learned FSMap version 46 2024-05-07T02:39:47.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.689+0000 7f2c36fd6b80 1 -- 172.21.15.19:0/1391867203 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55cf396f79c0 con 0x55cf396eaa90 2024-05-07T02:39:47.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.689+0000 7f2c07400700 1 -- 172.21.15.19:0/1391867203 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c0c00fb30 con 0x55cf396eaa90 2024-05-07T02:39:47.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.689+0000 7f2c07400700 1 -- 172.21.15.19:0/1391867203 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 46) ==== 8386+0+0 (secure 0 0 0) 0x7f2c140e5be0 con 0x55cf396eaa90 2024-05-07T02:39:47.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.689+0000 7f2c36fd6b80 10 client.26413 fetch_fsmap finished waiting for FSMap version 46 2024-05-07T02:39:47.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.689+0000 7f2c36fd6b80 10 client.26413 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:39:47.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.689+0000 7f2c36fd6b80 20 client.26413 populate_metadata read hostname 'smithi019' 2024-05-07T02:39:47.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.690+0000 7f2c36fd6b80 1 --2- 172.21.15.19:0/1391867203 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55cf39704b00 0x55cf39706f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:47.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.690+0000 7f2c36fd6b80 4 client.26413 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:39:47.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.690+0000 7f2c36fd6b80 1 -- 172.21.15.19:0/1391867203 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf396b72e0 con 0x55cf39704b00 2024-05-07T02:39:47.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.690+0000 7f2c25000700 1 --2- 172.21.15.19:0/1391867203 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55cf39704b00 0x55cf39706f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:47.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.690+0000 7f2c25000700 1 --2- 172.21.15.19:0/1391867203 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55cf39704b00 0x55cf39706f90 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dde8f0385845cdf2 server_cookie=a3a412fdc327ead2 in_seq=0 out_seq=0 2024-05-07T02:39:47.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.690+0000 7f2c07400700 10 client.26413 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:39:47.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.691+0000 7f2c07400700 1 -- 172.21.15.19:0/1391867203 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cf396b72e0 con 0x55cf39704b00 2024-05-07T02:39:47.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.691+0000 7f2c07400700 10 client.26413 handle_command_reply: tid=0 2024-05-07T02:39:47.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.692+0000 7f2c36fd6b80 1 client.26413 shutdown 2024-05-07T02:39:47.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.692+0000 7f2c36fd6b80 1 -- 172.21.15.19:0/1391867203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf396eaa90 msgr2=0x55cf396f1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:47.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.692+0000 7f2c36fd6b80 1 --2- 172.21.15.19:0/1391867203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf396eaa90 0x55cf396f1a80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2c140a6650 tx=0x7f2c14005e80 comp rx=0 tx=0).stop 2024-05-07T02:39:47.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.692+0000 7f2c36fd6b80 1 -- 172.21.15.19:0/1391867203 shutdown_connections 2024-05-07T02:39:47.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.692+0000 7f2c36fd6b80 1 --2- 172.21.15.19:0/1391867203 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55cf39704b00 0x55cf39706f90 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:47.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.692+0000 7f2c36fd6b80 1 --2- 172.21.15.19:0/1391867203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf396f4b60 0x55cf396f2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:47.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.692+0000 7f2c36fd6b80 1 --2- 172.21.15.19:0/1391867203 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf396eaa90 0x55cf396f1a80 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:47.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.692+0000 7f2c07400700 0 client.26413 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:39:47.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.692+0000 7f2c36fd6b80 1 --2- 172.21.15.19:0/1391867203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf396eb3e0 0x55cf396f1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:47.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.692+0000 7f2c36fd6b80 1 -- 172.21.15.19:0/1391867203 >> 172.21.15.19:0/1391867203 conn(0x55cf3956ce40 msgr2=0x55cf396e7450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:47.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.692+0000 7f2c36fd6b80 1 -- 172.21.15.19:0/1391867203 shutdown_connections 2024-05-07T02:39:47.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.692+0000 7f2c36fd6b80 1 -- 172.21.15.19:0/1391867203 wait complete. 2024-05-07T02:39:47.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.692+0000 7f2c36fd6b80 20 client.26413 trim_cache size 0 max 16384 2024-05-07T02:39:47.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.714+0000 7f2c36fd6b80 1 Processor -- start 2024-05-07T02:39:47.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.714+0000 7f2c36fd6b80 1 -- start start 2024-05-07T02:39:47.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.714+0000 7f2c36fd6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf3956ce40 0x55cf396e9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:47.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.715+0000 7f2c36fd6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf396eaa90 0x55cf397009f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:47.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.715+0000 7f2c36fd6b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf396eb3e0 0x55cf3970c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:47.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.715+0000 7f2c36fd6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2c201b68b0 con 0x55cf396eaa90 2024-05-07T02:39:47.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.715+0000 7f2c36fd6b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2c201b8510 con 0x55cf396eb3e0 2024-05-07T02:39:47.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.715+0000 7f2c36fd6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2c201b6f80 con 0x55cf3956ce40 2024-05-07T02:39:47.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.715+0000 7f2c25000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf396eaa90 0x55cf397009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:47.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.715+0000 7f2c25a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf3956ce40 0x55cf396e9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:47.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.715+0000 7f2c25a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf3956ce40 0x55cf396e9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:36482/0 (socket says 172.21.15.19:36482) 2024-05-07T02:39:47.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.715+0000 7f2c25a00700 1 -- 172.21.15.19:0/3907263813 learned_addr learned my addr 172.21.15.19:0/3907263813 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:39:47.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.715+0000 7f2c25000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf396eaa90 0x55cf397009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58834/0 (socket says 172.21.15.19:58834) 2024-05-07T02:39:47.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.715+0000 7f2c26400700 1 --2- 172.21.15.19:0/3907263813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf396eb3e0 0x55cf3970c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:47.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.715+0000 7f2c25000700 1 -- 172.21.15.19:0/3907263813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf3956ce40 msgr2=0x55cf396e9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:47.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.715+0000 7f2c25000700 1 --2- 172.21.15.19:0/3907263813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf3956ce40 0x55cf396e9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:47.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.715+0000 7f2c25000700 1 -- 172.21.15.19:0/3907263813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf396eb3e0 msgr2=0x55cf3970c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:47.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.715+0000 7f2c25000700 1 --2- 172.21.15.19:0/3907263813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf396eb3e0 0x55cf3970c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:47.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.715+0000 7f2c25000700 1 -- 172.21.15.19:0/3907263813 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf396eb210 con 0x55cf396eaa90 2024-05-07T02:39:47.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.715+0000 7f2c25a00700 1 --2- 172.21.15.19:0/3907263813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf3956ce40 0x55cf396e9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:39:47.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.716+0000 7f2c25000700 1 --2- 172.21.15.19:0/3907263813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf396eaa90 0x55cf397009f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c019760 tx=0x7f2c0c009d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84399bfbb37b1c5f server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:47.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.716+0000 7f2c12000700 1 -- 172.21.15.19:0/3907263813 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c02f070 con 0x55cf396eaa90 2024-05-07T02:39:47.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.716+0000 7f2c12000700 1 -- 172.21.15.19:0/3907263813 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c0c01f040 con 0x55cf396eaa90 2024-05-07T02:39:47.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.716+0000 7f2c12000700 1 -- 172.21.15.19:0/3907263813 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c01e040 con 0x55cf396eaa90 2024-05-07T02:39:47.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.717+0000 7f2c36fd6b80 1 -- 172.21.15.19:0/3907263813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf396eaa90 msgr2=0x55cf397009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:47.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.717+0000 7f2c36fd6b80 1 --2- 172.21.15.19:0/3907263813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf396eaa90 0x55cf397009f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c019760 tx=0x7f2c0c009d00 comp rx=0 tx=0).stop 2024-05-07T02:39:47.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.717+0000 7f2c36fd6b80 1 -- 172.21.15.19:0/3907263813 shutdown_connections 2024-05-07T02:39:47.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.717+0000 7f2c36fd6b80 1 --2- 172.21.15.19:0/3907263813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf396eb3e0 0x55cf3970c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:47.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.717+0000 7f2c36fd6b80 1 --2- 172.21.15.19:0/3907263813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf3956ce40 0x55cf396e9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:47.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.717+0000 7f2c36fd6b80 1 --2- 172.21.15.19:0/3907263813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf396eaa90 0x55cf397009f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:47.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.717+0000 7f2c36fd6b80 1 -- 172.21.15.19:0/3907263813 >> 172.21.15.19:0/3907263813 conn(0x55cf396f7250 msgr2=0x55cf396e9460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:47.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.717+0000 7f2c36fd6b80 1 -- 172.21.15.19:0/3907263813 shutdown_connections 2024-05-07T02:39:47.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.717+0000 7f2c36fd6b80 1 -- 172.21.15.19:0/3907263813 wait complete. 2024-05-07T02:39:47.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.718+0000 7f2c36fd6b80 1 -- start start 2024-05-07T02:39:47.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.718+0000 7f2c36fd6b80 1 Processor -- start 2024-05-07T02:39:47.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.718+0000 7f2c36fd6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf3956ce40 0x55cf396fd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:47.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.718+0000 7f2c36fd6b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf396eaa90 0x55cf396f8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:47.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.718+0000 7f2c36fd6b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf396eb3e0 0x55cf396f8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:47.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.718+0000 7f2c36fd6b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2c201b7d80 con 0x55cf396eaa90 2024-05-07T02:39:47.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.718+0000 7f2c36fd6b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2c201b7650 con 0x55cf396eb3e0 2024-05-07T02:39:47.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.718+0000 7f2c36fd6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2c201b4870 con 0x55cf3956ce40 2024-05-07T02:39:47.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.718+0000 7f2c36fd6b80 10 client.0 _ll_register_callbacks cb 0x55cf395b3c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:39:47.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.719+0000 7f2c25000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf396eaa90 0x55cf396f8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:47.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.719+0000 7f2c25000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf396eaa90 0x55cf396f8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58836/0 (socket says 172.21.15.19:58836) 2024-05-07T02:39:47.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.719+0000 7f2c25000700 1 -- 172.21.15.19:0/2267835273 learned_addr learned my addr 172.21.15.19:0/2267835273 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:39:47.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.719+0000 7f2c25a00700 1 --2- 172.21.15.19:0/2267835273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf3956ce40 0x55cf396fd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:47.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.719+0000 7f2c26400700 1 --2- 172.21.15.19:0/2267835273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf396eb3e0 0x55cf396f8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:47.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.719+0000 7f2c25000700 1 -- 172.21.15.19:0/2267835273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf3956ce40 msgr2=0x55cf396fd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:47.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.719+0000 7f2c25000700 1 --2- 172.21.15.19:0/2267835273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf3956ce40 0x55cf396fd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:47.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.719+0000 7f2c25000700 1 -- 172.21.15.19:0/2267835273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf396eb3e0 msgr2=0x55cf396f8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:39:47.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.719+0000 7f2c25000700 1 --2- 172.21.15.19:0/2267835273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf396eb3e0 0x55cf396f8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:47.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.719+0000 7f2c25000700 1 -- 172.21.15.19:0/2267835273 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf396f9580 con 0x55cf396eaa90 2024-05-07T02:39:47.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.719+0000 7f2c25a00700 1 --2- 172.21.15.19:0/2267835273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf3956ce40 0x55cf396fd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:39:47.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.720+0000 7f2c25000700 1 --2- 172.21.15.19:0/2267835273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf396eaa90 0x55cf396f8790 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c01c040 tx=0x7f2c0c0195c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c6b4058f92346c server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:39:47.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.720+0000 7f2c07400700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-07T02:39:47.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.720+0000 7f2c07400700 1 -- 172.21.15.19:0/2267835273 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c03a070 con 0x55cf396eaa90 2024-05-07T02:39:47.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.720+0000 7f2c07400700 1 -- 172.21.15.19:0/2267835273 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9525+0+0 (secure 0 0 0) 0x7f2c0c019830 con 0x55cf396eaa90 2024-05-07T02:39:47.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.720+0000 7f2c36fd6b80 1 -- 172.21.15.19:0/2267835273 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c0c01f040 con 0x55cf396eaa90 2024-05-07T02:39:47.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.720+0000 7f2c07400700 1 -- 172.21.15.19:0/2267835273 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f2c0c046050 con 0x55cf396eaa90 2024-05-07T02:39:47.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.720+0000 7f2c36fd6b80 10 client.16260 fetch_fsmap learned FSMap version 46 2024-05-07T02:39:47.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.720+0000 7f2c36fd6b80 1 -- 172.21.15.19:0/2267835273 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55cf3976f390 con 0x55cf396eaa90 2024-05-07T02:39:47.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.721+0000 7f2c07400700 1 -- 172.21.15.19:0/2267835273 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c0c01f040 con 0x55cf396eaa90 2024-05-07T02:39:47.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.721+0000 7f2c07400700 1 -- 172.21.15.19:0/2267835273 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 46) ==== 8386+0+0 (secure 0 0 0) 0x7f2c0c00f9a0 con 0x55cf396eaa90 2024-05-07T02:39:47.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.721+0000 7f2c36fd6b80 10 client.16260 fetch_fsmap finished waiting for FSMap version 46 2024-05-07T02:39:47.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.721+0000 7f2c36fd6b80 10 client.16260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:39:47.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.721+0000 7f2c36fd6b80 20 client.16260 populate_metadata read hostname 'smithi019' 2024-05-07T02:39:47.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.721+0000 7f2c36fd6b80 1 --2- 172.21.15.19:0/2267835273 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55cf396f4b60 0x55cf397718e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:39:47.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.721+0000 7f2c36fd6b80 4 client.16260 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-07T02:39:47.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.721+0000 7f2c36fd6b80 1 -- 172.21.15.19:0/2267835273 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cf3961a400 con 0x55cf396f4b60 2024-05-07T02:39:47.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.721+0000 7f2c25a00700 1 --2- 172.21.15.19:0/2267835273 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55cf396f4b60 0x55cf397718e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:39:47.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.722+0000 7f2c25a00700 1 --2- 172.21.15.19:0/2267835273 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55cf396f4b60 0x55cf397718e0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66e756bea044405d server_cookie=14917e13e5930488 in_seq=0 out_seq=0 2024-05-07T02:39:47.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.722+0000 7f2c07400700 10 client.16260 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:39:47.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.722+0000 7f2c07400700 1 -- 172.21.15.19:0/2267835273 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55cf3961a400 con 0x55cf396f4b60 2024-05-07T02:39:47.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.722+0000 7f2c07400700 10 client.16260 handle_command_reply: tid=0 2024-05-07T02:39:47.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.722+0000 7f2c36fd6b80 1 client.16260 shutdown 2024-05-07T02:39:47.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.722+0000 7f2c36fd6b80 1 -- 172.21.15.19:0/2267835273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf396eaa90 msgr2=0x55cf396f8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:47.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.722+0000 7f2c36fd6b80 1 --2- 172.21.15.19:0/2267835273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf396eaa90 0x55cf396f8790 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c01c040 tx=0x7f2c0c0195c0 comp rx=0 tx=0).stop 2024-05-07T02:39:47.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.722+0000 7f2c36fd6b80 1 -- 172.21.15.19:0/2267835273 shutdown_connections 2024-05-07T02:39:47.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.722+0000 7f2c36fd6b80 1 --2- 172.21.15.19:0/2267835273 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55cf396f4b60 0x55cf397718e0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:47.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.722+0000 7f2c36fd6b80 1 --2- 172.21.15.19:0/2267835273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55cf396eb3e0 0x55cf396f8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:47.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.722+0000 7f2c36fd6b80 1 --2- 172.21.15.19:0/2267835273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55cf3956ce40 0x55cf396fd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:47.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.722+0000 7f2c36fd6b80 1 --2- 172.21.15.19:0/2267835273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cf396eaa90 0x55cf396f8790 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:47.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.722+0000 7f2c36fd6b80 1 -- 172.21.15.19:0/2267835273 >> 172.21.15.19:0/2267835273 conn(0x55cf396f7250 msgr2=0x55cf396f1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:47.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.722+0000 7f2c07400700 0 client.16260 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:39:47.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.723+0000 7f2c36fd6b80 1 -- 172.21.15.19:0/2267835273 shutdown_connections 2024-05-07T02:39:47.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.723+0000 7f2c36fd6b80 1 -- 172.21.15.19:0/2267835273 wait complete. 2024-05-07T02:39:47.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.723+0000 7f2c36fd6b80 20 client.16260 trim_cache size 0 max 16384 2024-05-07T02:39:47.730 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:39:47.731 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-07T02:39:47.731 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-05-07T02:39:47.731 INFO:teuthology.orchestra.run.smithi019.stdout: "de7896a7-2478-4f61-a1bb-f0e8975503a1": { 2024-05-07T02:39:47.731 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-05-07T02:39:47.731 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "de7896a7-2478-4f61-a1bb-f0e8975503a1", 2024-05-07T02:39:47.731 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-05-07T02:39:47.731 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:39:47.731 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:39:47.731 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:39:47.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.724+0000 7f2c26e00700 1 -- 172.21.15.19:0/2988236709 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f2c080b2a70 msgr2=0x7f2c080b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:47.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.724+0000 7f2c26e00700 1 --2- 172.21.15.19:0/2988236709 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f2c080b2a70 0x7f2c080b4f80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2c18004a60 tx=0x7f2c18009040 comp rx=0 tx=0).stop 2024-05-07T02:39:47.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.724+0000 7f2c26e00700 1 -- 172.21.15.19:0/2988236709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c2010a7c0 msgr2=0x7f2c201a1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:39:47.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.724+0000 7f2c26e00700 1 --2- 172.21.15.19:0/2988236709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c2010a7c0 0x7f2c201a1270 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2c14000c00 tx=0x7f2c140047b0 comp rx=0 tx=0).stop 2024-05-07T02:39:47.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.724+0000 7f2c26e00700 1 -- 172.21.15.19:0/2988236709 shutdown_connections 2024-05-07T02:39:47.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.724+0000 7f2c26e00700 1 --2- 172.21.15.19:0/2988236709 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c2010f6c0 0x7f2c201a5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:47.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.724+0000 7f2c26e00700 1 --2- 172.21.15.19:0/2988236709 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c2010a7c0 0x7f2c201a1270 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:47.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.724+0000 7f2c26e00700 1 --2- 172.21.15.19:0/2988236709 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f2c080b2a70 0x7f2c080b4f80 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:47.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.724+0000 7f2c26e00700 1 --2- 172.21.15.19:0/2988236709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c2010b110 0x7f2c201a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:39:47.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.724+0000 7f2c26e00700 1 -- 172.21.15.19:0/2988236709 >> 172.21.15.19:0/2988236709 conn(0x7f2c20076b00 msgr2=0x7f2c201088c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:39:47.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.724+0000 7f2c26e00700 1 -- 172.21.15.19:0/2988236709 shutdown_connections 2024-05-07T02:39:47.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:39:47.724+0000 7f2c26e00700 1 -- 172.21.15.19:0/2988236709 wait complete. 2024-05-07T02:39:48.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:48 smithi064 ceph-mon[34699]: pgmap v243: 129 pgs: 129 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 297 KiB/s wr, 34 op/s 2024-05-07T02:39:48.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:48 smithi019 ceph-mon[29781]: pgmap v243: 129 pgs: 129 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 297 KiB/s wr, 34 op/s 2024-05-07T02:39:48.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:48 smithi027 ceph-mon[34398]: pgmap v243: 129 pgs: 129 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 297 KiB/s wr, 34 op/s 2024-05-07T02:39:48.769 INFO:tasks.cephfs.filesystem:scrub status for tag:de7896a7-2478-4f61-a1bb-f0e8975503a1 - {'path': '/', 'tag': 'de7896a7-2478-4f61-a1bb-f0e8975503a1', 'options': 'recursive,force'} 2024-05-07T02:39:49.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:49 smithi064 ceph-mon[34699]: scrub summary: idle 2024-05-07T02:39:49.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:49 smithi064 ceph-mon[34699]: scrub summary: idle 2024-05-07T02:39:49.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:49 smithi064 ceph-mon[34699]: scrub summary: idle 2024-05-07T02:39:49.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:49 smithi019 ceph-mon[29781]: scrub summary: idle 2024-05-07T02:39:49.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:49 smithi019 ceph-mon[29781]: scrub summary: idle 2024-05-07T02:39:49.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:49 smithi019 ceph-mon[29781]: scrub summary: idle 2024-05-07T02:39:49.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:49 smithi027 ceph-mon[34398]: scrub summary: idle 2024-05-07T02:39:49.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:49 smithi027 ceph-mon[34398]: scrub summary: idle 2024-05-07T02:39:49.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:49 smithi027 ceph-mon[34398]: scrub summary: idle 2024-05-07T02:39:50.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:50 smithi064 ceph-mon[34699]: pgmap v244: 129 pgs: 129 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 82 KiB/s wr, 20 op/s 2024-05-07T02:39:50.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:50 smithi064 ceph-mon[34699]: scrub summary: idle 2024-05-07T02:39:50.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:50 smithi064 ceph-mon[34699]: scrub summary: idle 2024-05-07T02:39:50.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:50 smithi019 ceph-mon[29781]: pgmap v244: 129 pgs: 129 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 82 KiB/s wr, 20 op/s 2024-05-07T02:39:50.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:50 smithi019 ceph-mon[29781]: scrub summary: idle 2024-05-07T02:39:50.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:50 smithi019 ceph-mon[29781]: scrub summary: idle 2024-05-07T02:39:50.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:50 smithi027 ceph-mon[34398]: pgmap v244: 129 pgs: 129 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 82 KiB/s wr, 20 op/s 2024-05-07T02:39:50.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:50 smithi027 ceph-mon[34398]: scrub summary: idle 2024-05-07T02:39:50.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:50 smithi027 ceph-mon[34398]: scrub summary: idle 2024-05-07T02:39:52.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:52 smithi064 ceph-mon[34699]: pgmap v245: 129 pgs: 129 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 82 KiB/s wr, 20 op/s 2024-05-07T02:39:52.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:52 smithi019 ceph-mon[29781]: pgmap v245: 129 pgs: 129 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 82 KiB/s wr, 20 op/s 2024-05-07T02:39:52.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:52 smithi027 ceph-mon[34398]: pgmap v245: 129 pgs: 129 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 82 KiB/s wr, 20 op/s 2024-05-07T02:39:54.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:54 smithi064 ceph-mon[34699]: pgmap v246: 129 pgs: 129 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 97 KiB/s wr, 26 op/s 2024-05-07T02:39:54.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:54 smithi019 ceph-mon[29781]: pgmap v246: 129 pgs: 129 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 97 KiB/s wr, 26 op/s 2024-05-07T02:39:54.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:54 smithi027 ceph-mon[34398]: pgmap v246: 129 pgs: 129 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 97 KiB/s wr, 26 op/s 2024-05-07T02:39:56.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:56 smithi064 ceph-mon[34699]: pgmap v247: 129 pgs: 129 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 15 op/s 2024-05-07T02:39:56.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:56 smithi019 ceph-mon[29781]: pgmap v247: 129 pgs: 129 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 15 op/s 2024-05-07T02:39:56.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:56 smithi027 ceph-mon[34398]: pgmap v247: 129 pgs: 129 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 15 op/s 2024-05-07T02:39:58.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:39:58 smithi064 ceph-mon[34699]: pgmap v248: 129 pgs: 129 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 21 op/s 2024-05-07T02:39:58.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:39:58 smithi019 ceph-mon[29781]: pgmap v248: 129 pgs: 129 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 21 op/s 2024-05-07T02:39:58.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:39:58 smithi027 ceph-mon[34398]: pgmap v248: 129 pgs: 129 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 36 KiB/s wr, 21 op/s 2024-05-07T02:40:00.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:00 smithi064 ceph-mon[34699]: pgmap v249: 129 pgs: 129 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-05-07T02:40:00.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:00 smithi064 ceph-mon[34699]: overall HEALTH_OK 2024-05-07T02:40:00.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:00 smithi019 ceph-mon[29781]: pgmap v249: 129 pgs: 129 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-05-07T02:40:00.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:00 smithi019 ceph-mon[29781]: overall HEALTH_OK 2024-05-07T02:40:00.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:00 smithi027 ceph-mon[34398]: pgmap v249: 129 pgs: 129 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-05-07T02:40:00.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:00 smithi027 ceph-mon[34398]: overall HEALTH_OK 2024-05-07T02:40:02.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:02 smithi064 ceph-mon[34699]: pgmap v250: 129 pgs: 129 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-05-07T02:40:02.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:02 smithi019 ceph-mon[29781]: pgmap v250: 129 pgs: 129 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-05-07T02:40:02.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:02 smithi027 ceph-mon[34398]: pgmap v250: 129 pgs: 129 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-05-07T02:40:03.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:03 smithi064 ceph-mon[34699]: pgmap v251: 129 pgs: 129 active+clean; 176 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 49 KiB/s wr, 27 op/s 2024-05-07T02:40:03.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:03 smithi019 ceph-mon[29781]: pgmap v251: 129 pgs: 129 active+clean; 176 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 49 KiB/s wr, 27 op/s 2024-05-07T02:40:03.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:03 smithi027 ceph-mon[34398]: pgmap v251: 129 pgs: 129 active+clean; 176 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 49 KiB/s wr, 27 op/s 2024-05-07T02:40:05.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:05 smithi027 ceph-mon[34398]: pgmap v252: 129 pgs: 129 active+clean; 176 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 34 KiB/s wr, 21 op/s 2024-05-07T02:40:05.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:05 smithi064 ceph-mon[34699]: pgmap v252: 129 pgs: 129 active+clean; 176 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 34 KiB/s wr, 21 op/s 2024-05-07T02:40:05.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:05 smithi019 ceph-mon[29781]: pgmap v252: 129 pgs: 129 active+clean; 176 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 34 KiB/s wr, 21 op/s 2024-05-07T02:40:07.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:07 smithi027 ceph-mon[34398]: pgmap v253: 129 pgs: 129 active+clean; 176 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 59 KiB/s wr, 32 op/s 2024-05-07T02:40:07.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:07 smithi064 ceph-mon[34699]: pgmap v253: 129 pgs: 129 active+clean; 176 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 59 KiB/s wr, 32 op/s 2024-05-07T02:40:07.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:07 smithi019 ceph-mon[29781]: pgmap v253: 129 pgs: 129 active+clean; 176 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 59 KiB/s wr, 32 op/s 2024-05-07T02:40:09.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:09 smithi027 ceph-mon[34398]: pgmap v254: 129 pgs: 129 active+clean; 176 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 51 KiB/s wr, 26 op/s 2024-05-07T02:40:09.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:09 smithi064 ceph-mon[34699]: pgmap v254: 129 pgs: 129 active+clean; 176 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 51 KiB/s wr, 26 op/s 2024-05-07T02:40:09.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:09 smithi019 ceph-mon[29781]: pgmap v254: 129 pgs: 129 active+clean; 176 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 51 KiB/s wr, 26 op/s 2024-05-07T02:40:10.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:10 smithi064 ceph-mon[34699]: osdmap e93: 12 total, 12 up, 12 in 2024-05-07T02:40:10.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:10 smithi019 ceph-mon[29781]: osdmap e93: 12 total, 12 up, 12 in 2024-05-07T02:40:10.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:10 smithi027 ceph-mon[34398]: osdmap e93: 12 total, 12 up, 12 in 2024-05-07T02:40:11.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:11 smithi064 ceph-mon[34699]: pgmap v256: 129 pgs: 129 active+clean; 176 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 62 KiB/s wr, 31 op/s 2024-05-07T02:40:11.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:11 smithi019 ceph-mon[29781]: pgmap v256: 129 pgs: 129 active+clean; 176 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 62 KiB/s wr, 31 op/s 2024-05-07T02:40:11.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:11 smithi027 ceph-mon[34398]: pgmap v256: 129 pgs: 129 active+clean; 176 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 62 KiB/s wr, 31 op/s 2024-05-07T02:40:13.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:13 smithi064 ceph-mon[34699]: pgmap v257: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 30 KiB/s wr, 18 op/s 2024-05-07T02:40:13.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:13 smithi064 ceph-mon[34699]: osdmap e94: 12 total, 12 up, 12 in 2024-05-07T02:40:13.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:13 smithi019 ceph-mon[29781]: pgmap v257: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 30 KiB/s wr, 18 op/s 2024-05-07T02:40:13.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:13 smithi019 ceph-mon[29781]: osdmap e94: 12 total, 12 up, 12 in 2024-05-07T02:40:13.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:13 smithi027 ceph-mon[34398]: pgmap v257: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 30 KiB/s wr, 18 op/s 2024-05-07T02:40:13.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:13 smithi027 ceph-mon[34398]: osdmap e94: 12 total, 12 up, 12 in 2024-05-07T02:40:15.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:15 smithi064 ceph-mon[34699]: pgmap v259: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-05-07T02:40:15.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:15 smithi064 ceph-mon[34699]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:40:15.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:15 smithi019 ceph-mon[29781]: pgmap v259: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-05-07T02:40:15.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:15 smithi019 ceph-mon[29781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:40:15.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:15 smithi027 ceph-mon[34398]: pgmap v259: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-05-07T02:40:15.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:15 smithi027 ceph-mon[34398]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:40:17.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:17 smithi064 ceph-mon[34699]: pgmap v260: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 3.1 KiB/s wr, 12 op/s 2024-05-07T02:40:17.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:17 smithi019 ceph-mon[29781]: pgmap v260: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 3.1 KiB/s wr, 12 op/s 2024-05-07T02:40:17.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:17 smithi027 ceph-mon[34398]: pgmap v260: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 3.1 KiB/s wr, 12 op/s 2024-05-07T02:40:18.770 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-07T02:40:19.250 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:40:19.571 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:19 smithi019 ceph-mon[29781]: pgmap v261: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-05-07T02:40:19.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:19 smithi064 ceph-mon[34699]: pgmap v261: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-05-07T02:40:19.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:19 smithi027 ceph-mon[34398]: pgmap v261: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-05-07T02:40:20.585 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:20 smithi019 ceph-mon[29781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:40:20.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:20 smithi064 ceph-mon[34699]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:40:20.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:20 smithi027 ceph-mon[34398]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:40:21.590 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:21 smithi019 ceph-mon[29781]: pgmap v262: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-05-07T02:40:21.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:21 smithi064 ceph-mon[34699]: pgmap v262: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-05-07T02:40:21.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:21 smithi027 ceph-mon[34398]: pgmap v262: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-05-07T02:40:22.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.045+0000 7f67bbe00700 1 -- 172.21.15.19:0/2738100059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67bc108560 msgr2=0x7f67bc10fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:22.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.045+0000 7f67bbe00700 1 --2- 172.21.15.19:0/2738100059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67bc108560 0x7f67bc10fec0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f67b400b0d0 tx=0x7f67b400b3e0 comp rx=0 tx=0).stop 2024-05-07T02:40:22.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.045+0000 7f67bbe00700 1 -- 172.21.15.19:0/2738100059 shutdown_connections 2024-05-07T02:40:22.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.045+0000 7f67bbe00700 1 --2- 172.21.15.19:0/2738100059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67bc108560 0x7f67bc10fec0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:22.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.045+0000 7f67bbe00700 1 --2- 172.21.15.19:0/2738100059 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67bc1032e0 0x7f67bc1036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:22.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.045+0000 7f67bbe00700 1 --2- 172.21.15.19:0/2738100059 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67bc107c10 0x7f67bc108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:22.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.045+0000 7f67bbe00700 1 -- 172.21.15.19:0/2738100059 >> 172.21.15.19:0/2738100059 conn(0x7f67bc078ac0 msgr2=0x7f67bc078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:22.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.046+0000 7f67bbe00700 1 -- 172.21.15.19:0/2738100059 shutdown_connections 2024-05-07T02:40:22.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.046+0000 7f67bbe00700 1 -- 172.21.15.19:0/2738100059 wait complete. 2024-05-07T02:40:22.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.047+0000 7f67bbe00700 1 Processor -- start 2024-05-07T02:40:22.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.047+0000 7f67bbe00700 1 -- start start 2024-05-07T02:40:22.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.048+0000 7f67bbe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67bc1032e0 0x7f67bc19ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:22.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.048+0000 7f67bbe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67bc107c10 0x7f67bc19d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:22.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.048+0000 7f67bbe00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67bc108560 0x7f67bc1a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:22.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.048+0000 7f67bbe00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc1131e0 con 0x7f67bc1032e0 2024-05-07T02:40:22.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.048+0000 7f67bbe00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc113060 con 0x7f67bc107c10 2024-05-07T02:40:22.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.048+0000 7f67bbe00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc113360 con 0x7f67bc108560 2024-05-07T02:40:22.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.048+0000 7f67bb400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67bc108560 0x7f67bc1a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:22.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.048+0000 7f67baa00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67bc1032e0 0x7f67bc19ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:22.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.048+0000 7f67bb400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67bc108560 0x7f67bc1a1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:49978/0 (socket says 172.21.15.19:49978) 2024-05-07T02:40:22.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.048+0000 7f67bb400700 1 -- 172.21.15.19:0/2037649864 learned_addr learned my addr 172.21.15.19:0/2037649864 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:40:22.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.048+0000 7f67b2000700 1 --2- 172.21.15.19:0/2037649864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67bc107c10 0x7f67bc19d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:22.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.049+0000 7f67bb400700 1 -- 172.21.15.19:0/2037649864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67bc107c10 msgr2=0x7f67bc19d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:22.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.049+0000 7f67bb400700 1 --2- 172.21.15.19:0/2037649864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67bc107c10 0x7f67bc19d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:22.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.049+0000 7f67bb400700 1 -- 172.21.15.19:0/2037649864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67bc1032e0 msgr2=0x7f67bc19ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:22.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.049+0000 7f67bb400700 1 --2- 172.21.15.19:0/2037649864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67bc1032e0 0x7f67bc19ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:22.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.049+0000 7f67bb400700 1 -- 172.21.15.19:0/2037649864 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67bc1a1d20 con 0x7f67bc108560 2024-05-07T02:40:22.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.049+0000 7f67bb400700 1 --2- 172.21.15.19:0/2037649864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67bc108560 0x7f67bc1a1680 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f67b40079a0 tx=0x7f67b40079d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:22.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.049+0000 7f67b3e00700 1 -- 172.21.15.19:0/2037649864 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b401f070 con 0x7f67bc108560 2024-05-07T02:40:22.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.049+0000 7f67b3e00700 1 -- 172.21.15.19:0/2037649864 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67b4003c00 con 0x7f67bc108560 2024-05-07T02:40:22.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.049+0000 7f67bbe00700 1 -- 172.21.15.19:0/2037649864 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67bc1a1fb0 con 0x7f67bc108560 2024-05-07T02:40:22.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.049+0000 7f67b3e00700 1 -- 172.21.15.19:0/2037649864 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b401b430 con 0x7f67bc108560 2024-05-07T02:40:22.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.050+0000 7f67bbe00700 1 -- 172.21.15.19:0/2037649864 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67bc1a2490 con 0x7f67bc108560 2024-05-07T02:40:22.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.053+0000 7f67b3e00700 1 -- 172.21.15.19:0/2037649864 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f67b4022070 con 0x7f67bc108560 2024-05-07T02:40:22.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.055+0000 7f67d0427b80 1 Processor -- start 2024-05-07T02:40:22.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.055+0000 7f67d0427b80 1 -- start start 2024-05-07T02:40:22.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.055+0000 7f67d0427b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c493be5c40 0x55c493be6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:22.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.056+0000 7f67d0427b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c493be6590 0x55c493bef6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:22.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.056+0000 7f67d0427b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c493befc30 0x55c493bf2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:22.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.056+0000 7f67d0427b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c493b07a20 con 0x55c493be6590 2024-05-07T02:40:22.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.056+0000 7f67d0427b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c493aa5760 con 0x55c493befc30 2024-05-07T02:40:22.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.056+0000 7f67d0427b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c493b039e0 con 0x55c493be5c40 2024-05-07T02:40:22.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.056+0000 7f67baa00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c493be6590 0x55c493bef6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:22.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.056+0000 7f67b2000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c493be5c40 0x55c493be6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:22.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.056+0000 7f67baa00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c493be6590 0x55c493bef6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46752/0 (socket says 172.21.15.19:46752) 2024-05-07T02:40:22.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.056+0000 7f67baa00700 1 -- 172.21.15.19:0/2827884657 learned_addr learned my addr 172.21.15.19:0/2827884657 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:40:22.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.056+0000 7f67b3e00700 1 --2- 172.21.15.19:0/2037649864 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f679c0b29f0 0x7f679c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:22.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.056+0000 7f67b2000700 1 --2- 172.21.15.19:0/2827884657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c493befc30 0x55c493bf2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:22.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.056+0000 7f67bb400700 1 --2- 172.21.15.19:0/2037649864 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f679c0b29f0 0x7f679c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:22.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.056+0000 7f67b3e00700 1 -- 172.21.15.19:0/2037649864 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9525+0+0 (secure 0 0 0) 0x7f67b400f020 con 0x7f67bc108560 2024-05-07T02:40:22.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.057+0000 7f67b2000700 1 -- 172.21.15.19:0/2827884657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c493be5c40 msgr2=0x55c493be6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:22.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.057+0000 7f67b2000700 1 --2- 172.21.15.19:0/2827884657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c493be5c40 0x55c493be6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:22.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.057+0000 7f67b2000700 1 -- 172.21.15.19:0/2827884657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c493be6590 msgr2=0x55c493bef6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:22.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.057+0000 7f67b2000700 1 --2- 172.21.15.19:0/2827884657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c493be6590 0x55c493bef6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:22.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.057+0000 7f67b2000700 1 -- 172.21.15.19:0/2827884657 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c493a548d0 con 0x55c493befc30 2024-05-07T02:40:22.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.057+0000 7f67baa00700 1 --2- 172.21.15.19:0/2827884657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c493be6590 0x55c493bef6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:40:22.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.057+0000 7f67b2000700 1 --2- 172.21.15.19:0/2827884657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c493befc30 0x55c493bf2050 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f67bc0fffc0 tx=0x7f67a4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2cf1974a9e7c45f server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:22.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.057+0000 7f67bb400700 1 --2- 172.21.15.19:0/2037649864 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f679c0b29f0 0x7f679c0b4f00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f67b40a6a00 tx=0x7f67b40099e0 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:22.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.057+0000 7f67abe00700 1 -- 172.21.15.19:0/2827884657 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a4004830 con 0x55c493befc30 2024-05-07T02:40:22.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.057+0000 7f67abe00700 1 -- 172.21.15.19:0/2827884657 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67a4005020 con 0x55c493befc30 2024-05-07T02:40:22.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.057+0000 7f67abe00700 1 -- 172.21.15.19:0/2827884657 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a4010420 con 0x55c493befc30 2024-05-07T02:40:22.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.058+0000 7f67d0427b80 1 -- 172.21.15.19:0/2827884657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c493befc30 msgr2=0x55c493bf2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:22.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.058+0000 7f67d0427b80 1 --2- 172.21.15.19:0/2827884657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c493befc30 0x55c493bf2050 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f67bc0fffc0 tx=0x7f67a4009260 comp rx=0 tx=0).stop 2024-05-07T02:40:22.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.058+0000 7f67d0427b80 1 -- 172.21.15.19:0/2827884657 shutdown_connections 2024-05-07T02:40:22.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.059+0000 7f67d0427b80 1 --2- 172.21.15.19:0/2827884657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c493befc30 0x55c493bf2050 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:22.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.059+0000 7f67d0427b80 1 --2- 172.21.15.19:0/2827884657 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c493be5c40 0x55c493be6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:22.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.059+0000 7f67d0427b80 1 --2- 172.21.15.19:0/2827884657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c493be6590 0x55c493bef6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:22.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.059+0000 7f67d0427b80 1 -- 172.21.15.19:0/2827884657 >> 172.21.15.19:0/2827884657 conn(0x55c493a67e40 msgr2=0x55c493be0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:22.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.059+0000 7f67d0427b80 1 -- 172.21.15.19:0/2827884657 shutdown_connections 2024-05-07T02:40:22.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.059+0000 7f67d0427b80 1 -- 172.21.15.19:0/2827884657 wait complete. 2024-05-07T02:40:22.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.059+0000 7f67d0427b80 1 -- start start 2024-05-07T02:40:22.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.060+0000 7f67d0427b80 1 Processor -- start 2024-05-07T02:40:22.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.060+0000 7f67d0427b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c493be5c40 0x55c493becc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:22.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.060+0000 7f67d0427b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c493be6590 0x55c493bed1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:22.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.060+0000 7f67d0427b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c493befc30 0x55c493bed6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:22.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.060+0000 7f67d0427b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c493bf2a00 con 0x55c493be5c40 2024-05-07T02:40:22.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.060+0000 7f67d0427b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55c493a6db20 con 0x55c493be6590 2024-05-07T02:40:22.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.060+0000 7f67d0427b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c493bf2b80 con 0x55c493befc30 2024-05-07T02:40:22.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.060+0000 7f67d0427b80 10 client.0 _ll_register_callbacks cb 0x55c493a9b380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:40:22.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.061+0000 7f67b2000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c493be5c40 0x55c493becc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:22.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.061+0000 7f67b2000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c493be5c40 0x55c493becc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46768/0 (socket says 172.21.15.19:46768) 2024-05-07T02:40:22.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.061+0000 7f67b2000700 1 -- 172.21.15.19:0/249237086 learned_addr learned my addr 172.21.15.19:0/249237086 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:40:22.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.061+0000 7f67baa00700 1 --2- 172.21.15.19:0/249237086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c493be6590 0x55c493bed1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:22.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.061+0000 7f67b2000700 1 --2- 172.21.15.19:0/249237086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c493befc30 0x55c493bed6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:22.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.061+0000 7f67b2000700 1 -- 172.21.15.19:0/249237086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c493befc30 msgr2=0x55c493bed6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:22.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.061+0000 7f67b2000700 1 --2- 172.21.15.19:0/249237086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c493befc30 0x55c493bed6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:22.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.061+0000 7f67b2000700 1 -- 172.21.15.19:0/249237086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c493be6590 msgr2=0x55c493bed1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:22.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.061+0000 7f67b2000700 1 --2- 172.21.15.19:0/249237086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c493be6590 0x55c493bed1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:22.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.061+0000 7f67b2000700 1 -- 172.21.15.19:0/249237086 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c493acd420 con 0x55c493be5c40 2024-05-07T02:40:22.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.062+0000 7f67b2000700 1 --2- 172.21.15.19:0/249237086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c493be5c40 0x55c493becc60 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f67a4013540 tx=0x7f67a4009d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3841996fd4a6ffa server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:22.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.062+0000 7f67a8c00700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-07T02:40:22.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.062+0000 7f67a8c00700 1 -- 172.21.15.19:0/249237086 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a4024070 con 0x55c493be5c40 2024-05-07T02:40:22.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.062+0000 7f67d0427b80 1 -- 172.21.15.19:0/249237086 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67a4005020 con 0x55c493be5c40 2024-05-07T02:40:22.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.062+0000 7f67a8c00700 1 -- 172.21.15.19:0/249237086 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9525+0+0 (secure 0 0 0) 0x7f67a4004d50 con 0x55c493be5c40 2024-05-07T02:40:22.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.062+0000 7f67a8c00700 1 -- 172.21.15.19:0/249237086 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f67a4036050 con 0x55c493be5c40 2024-05-07T02:40:22.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.062+0000 7f67d0427b80 10 client.16272 fetch_fsmap learned FSMap version 48 2024-05-07T02:40:22.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.062+0000 7f67d0427b80 1 -- 172.21.15.19:0/249237086 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55c493bfd6a0 con 0x55c493be5c40 2024-05-07T02:40:22.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.063+0000 7f67a8c00700 1 -- 172.21.15.19:0/249237086 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67a4005020 con 0x55c493be5c40 2024-05-07T02:40:22.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.063+0000 7f67a8c00700 1 -- 172.21.15.19:0/249237086 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f67a4010420 con 0x55c493be5c40 2024-05-07T02:40:22.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.063+0000 7f67d0427b80 10 client.16272 fetch_fsmap finished waiting for FSMap version 48 2024-05-07T02:40:22.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.063+0000 7f67d0427b80 10 client.16272 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:40:22.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.063+0000 7f67d0427b80 20 client.16272 populate_metadata read hostname 'smithi019' 2024-05-07T02:40:22.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.063+0000 7f67d0427b80 1 --2- 172.21.15.19:0/249237086 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55c493bffb70 0x55c493c01f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:22.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.063+0000 7f67d0427b80 4 client.16272 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:40:22.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.063+0000 7f67d0427b80 1 -- 172.21.15.19:0/249237086 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c493bb20e0 con 0x55c493bffb70 2024-05-07T02:40:22.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.064+0000 7f67baa00700 1 --2- 172.21.15.19:0/249237086 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55c493bffb70 0x55c493c01f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:22.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.064+0000 7f67baa00700 1 --2- 172.21.15.19:0/249237086 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55c493bffb70 0x55c493c01f90 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92fd5c2f236b14e7 server_cookie=103bc7b7c87b357e in_seq=0 out_seq=0 2024-05-07T02:40:22.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.064+0000 7f67a8c00700 10 client.16272 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:40:22.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.065+0000 7f67a8c00700 1 -- 172.21.15.19:0/249237086 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c493bb20e0 con 0x55c493bffb70 2024-05-07T02:40:22.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.065+0000 7f67a8c00700 10 client.16272 handle_command_reply: tid=0 2024-05-07T02:40:22.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.065+0000 7f67d0427b80 1 client.16272 shutdown 2024-05-07T02:40:22.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.065+0000 7f67d0427b80 1 -- 172.21.15.19:0/249237086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c493be5c40 msgr2=0x55c493becc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:22.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.065+0000 7f67d0427b80 1 --2- 172.21.15.19:0/249237086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c493be5c40 0x55c493becc60 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f67a4013540 tx=0x7f67a4009d60 comp rx=0 tx=0).stop 2024-05-07T02:40:22.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.065+0000 7f67d0427b80 1 -- 172.21.15.19:0/249237086 shutdown_connections 2024-05-07T02:40:22.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.065+0000 7f67d0427b80 1 --2- 172.21.15.19:0/249237086 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55c493bffb70 0x55c493c01f90 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:22.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.066+0000 7f67d0427b80 1 --2- 172.21.15.19:0/249237086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c493befc30 0x55c493bed6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:22.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.066+0000 7f67d0427b80 1 --2- 172.21.15.19:0/249237086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c493be5c40 0x55c493becc60 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:22.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.066+0000 7f67a8c00700 0 client.16272 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:40:22.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.066+0000 7f67d0427b80 1 --2- 172.21.15.19:0/249237086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c493be6590 0x55c493bed1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:22.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.066+0000 7f67d0427b80 1 -- 172.21.15.19:0/249237086 >> 172.21.15.19:0/249237086 conn(0x55c493a67e40 msgr2=0x55c493be2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:22.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.066+0000 7f67d0427b80 1 -- 172.21.15.19:0/249237086 shutdown_connections 2024-05-07T02:40:22.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.066+0000 7f67d0427b80 1 -- 172.21.15.19:0/249237086 wait complete. 2024-05-07T02:40:22.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.066+0000 7f67d0427b80 20 client.16272 trim_cache size 0 max 16384 2024-05-07T02:40:22.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.089+0000 7f67d0427b80 1 Processor -- start 2024-05-07T02:40:22.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.089+0000 7f67d0427b80 1 -- start start 2024-05-07T02:40:22.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.089+0000 7f67d0427b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c493be5c40 0x55c493be0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:22.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.089+0000 7f67d0427b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c493be6590 0x55c493be5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:22.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.089+0000 7f67d0427b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c493befc30 0x55c493bed330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:22.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.089+0000 7f67d0427b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc1b2660 con 0x55c493be6590 2024-05-07T02:40:22.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.089+0000 7f67d0427b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc1b4200 con 0x55c493befc30 2024-05-07T02:40:22.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.089+0000 7f67d0427b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc1b2d30 con 0x55c493be5c40 2024-05-07T02:40:22.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.090+0000 7f67baa00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c493be6590 0x55c493be5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:22.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.090+0000 7f67baa00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c493be6590 0x55c493be5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46770/0 (socket says 172.21.15.19:46770) 2024-05-07T02:40:22.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.090+0000 7f67baa00700 1 -- 172.21.15.19:0/4014792208 learned_addr learned my addr 172.21.15.19:0/4014792208 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:40:22.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.090+0000 7f67b2000700 1 --2- 172.21.15.19:0/4014792208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c493be5c40 0x55c493be0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:22.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.090+0000 7f67b2000700 1 --2- 172.21.15.19:0/4014792208 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c493befc30 0x55c493bed330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:22.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.090+0000 7f67baa00700 1 -- 172.21.15.19:0/4014792208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c493be5c40 msgr2=0x55c493be0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:22.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.090+0000 7f67baa00700 1 --2- 172.21.15.19:0/4014792208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c493be5c40 0x55c493be0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:22.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.090+0000 7f67baa00700 1 -- 172.21.15.19:0/4014792208 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c493befc30 msgr2=0x55c493bed330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:22.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.090+0000 7f67baa00700 1 --2- 172.21.15.19:0/4014792208 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c493befc30 0x55c493bed330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:22.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.090+0000 7f67baa00700 1 -- 172.21.15.19:0/4014792208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c493bdc730 con 0x55c493be6590 2024-05-07T02:40:22.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.090+0000 7f67baa00700 1 --2- 172.21.15.19:0/4014792208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c493be6590 0x55c493be5040 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f67ac009e00 tx=0x7f67ac009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5d153b78c5bd0d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:22.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.091+0000 7f67ab400700 1 -- 172.21.15.19:0/4014792208 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ac0200d0 con 0x55c493be6590 2024-05-07T02:40:22.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.091+0000 7f67ab400700 1 -- 172.21.15.19:0/4014792208 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67ac024020 con 0x55c493be6590 2024-05-07T02:40:22.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.091+0000 7f67ab400700 1 -- 172.21.15.19:0/4014792208 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ac01e7a0 con 0x55c493be6590 2024-05-07T02:40:22.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.092+0000 7f67d0427b80 1 -- 172.21.15.19:0/4014792208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c493be6590 msgr2=0x55c493be5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:22.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.092+0000 7f67d0427b80 1 --2- 172.21.15.19:0/4014792208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c493be6590 0x55c493be5040 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f67ac009e00 tx=0x7f67ac009260 comp rx=0 tx=0).stop 2024-05-07T02:40:22.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.092+0000 7f67d0427b80 1 -- 172.21.15.19:0/4014792208 shutdown_connections 2024-05-07T02:40:22.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.092+0000 7f67d0427b80 1 --2- 172.21.15.19:0/4014792208 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c493befc30 0x55c493bed330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:22.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.092+0000 7f67d0427b80 1 --2- 172.21.15.19:0/4014792208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c493be5c40 0x55c493be0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:22.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.092+0000 7f67d0427b80 1 --2- 172.21.15.19:0/4014792208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c493be6590 0x55c493be5040 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:22.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.092+0000 7f67d0427b80 1 -- 172.21.15.19:0/4014792208 >> 172.21.15.19:0/4014792208 conn(0x55c493a67e40 msgr2=0x55c493becc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:22.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.092+0000 7f67d0427b80 1 -- 172.21.15.19:0/4014792208 shutdown_connections 2024-05-07T02:40:22.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.092+0000 7f67d0427b80 1 -- 172.21.15.19:0/4014792208 wait complete. 2024-05-07T02:40:22.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.093+0000 7f67d0427b80 1 -- start start 2024-05-07T02:40:22.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.093+0000 7f67d0427b80 1 Processor -- start 2024-05-07T02:40:22.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.093+0000 7f67d0427b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c493be5c40 0x55c493bf8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:22.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.093+0000 7f67d0427b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c493be6590 0x55c493bf3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:22.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.094+0000 7f67d0427b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c493befc30 0x55c493bf3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:22.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.094+0000 7f67d0427b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc1b3b30 con 0x55c493be5c40 2024-05-07T02:40:22.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.094+0000 7f67d0427b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc1b3400 con 0x55c493befc30 2024-05-07T02:40:22.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.094+0000 7f67d0427b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc1b0620 con 0x55c493be6590 2024-05-07T02:40:22.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.094+0000 7f67d0427b80 10 client.0 _ll_register_callbacks cb 0x55c493a9b380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:40:22.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.094+0000 7f67b2000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c493be5c40 0x55c493bf8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:22.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.094+0000 7f67b2000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c493be5c40 0x55c493bf8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46782/0 (socket says 172.21.15.19:46782) 2024-05-07T02:40:22.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.094+0000 7f67b2000700 1 -- 172.21.15.19:0/594760728 learned_addr learned my addr 172.21.15.19:0/594760728 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:40:22.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.094+0000 7f67baa00700 1 --2- 172.21.15.19:0/594760728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c493be6590 0x55c493bf3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:22.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.094+0000 7f67b2000700 1 --2- 172.21.15.19:0/594760728 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c493befc30 0x55c493bf3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:22.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.094+0000 7f67b2000700 1 -- 172.21.15.19:0/594760728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c493be6590 msgr2=0x55c493bf3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:22.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.094+0000 7f67b2000700 1 --2- 172.21.15.19:0/594760728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c493be6590 0x55c493bf3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:22.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.094+0000 7f67b2000700 1 -- 172.21.15.19:0/594760728 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c493befc30 msgr2=0x55c493bf3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:22.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.094+0000 7f67b2000700 1 --2- 172.21.15.19:0/594760728 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c493befc30 0x55c493bf3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:22.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.094+0000 7f67b2000700 1 -- 172.21.15.19:0/594760728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c493bf4690 con 0x55c493be5c40 2024-05-07T02:40:22.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.095+0000 7f67b2000700 1 --2- 172.21.15.19:0/594760728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c493be5c40 0x55c493bf8510 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f67a4012010 tx=0x7f67a4010a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c1f0077be344ba8 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:22.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.095+0000 7f67a8c00700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-07T02:40:22.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.095+0000 7f67a8c00700 1 -- 172.21.15.19:0/594760728 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a4024050 con 0x55c493be5c40 2024-05-07T02:40:22.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.095+0000 7f67a8c00700 1 -- 172.21.15.19:0/594760728 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9525+0+0 (secure 0 0 0) 0x7f67a4010420 con 0x55c493be5c40 2024-05-07T02:40:22.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.095+0000 7f67d0427b80 1 -- 172.21.15.19:0/594760728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67ac024020 con 0x55c493be5c40 2024-05-07T02:40:22.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.095+0000 7f67a8c00700 1 -- 172.21.15.19:0/594760728 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f67a4017040 con 0x55c493be5c40 2024-05-07T02:40:22.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.095+0000 7f67d0427b80 10 client.16278 fetch_fsmap learned FSMap version 48 2024-05-07T02:40:22.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.095+0000 7f67d0427b80 1 -- 172.21.15.19:0/594760728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55c493c6a1e0 con 0x55c493be5c40 2024-05-07T02:40:22.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.096+0000 7f67a8c00700 1 -- 172.21.15.19:0/594760728 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67ac024020 con 0x55c493be5c40 2024-05-07T02:40:22.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.096+0000 7f67a8c00700 1 -- 172.21.15.19:0/594760728 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 48) ==== 8370+0+0 (secure 0 0 0) 0x7f67a4004390 con 0x55c493be5c40 2024-05-07T02:40:22.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.096+0000 7f67d0427b80 10 client.16278 fetch_fsmap finished waiting for FSMap version 48 2024-05-07T02:40:22.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.096+0000 7f67d0427b80 10 client.16278 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:40:22.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.096+0000 7f67d0427b80 20 client.16278 populate_metadata read hostname 'smithi019' 2024-05-07T02:40:22.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.096+0000 7f67d0427b80 1 --2- 172.21.15.19:0/594760728 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55c493c6a6a0 0x55c493c6cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:22.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.096+0000 7f67d0427b80 4 client.16278 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-07T02:40:22.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.096+0000 7f67d0427b80 1 -- 172.21.15.19:0/594760728 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c493b15400 con 0x55c493c6a6a0 2024-05-07T02:40:22.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.096+0000 7f67baa00700 1 --2- 172.21.15.19:0/594760728 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55c493c6a6a0 0x55c493c6cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:22.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.097+0000 7f67baa00700 1 --2- 172.21.15.19:0/594760728 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55c493c6a6a0 0x55c493c6cac0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=298ac10782e35094 server_cookie=6e03e28eca10636e in_seq=0 out_seq=0 2024-05-07T02:40:22.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.097+0000 7f67a8c00700 10 client.16278 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:40:22.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.097+0000 7f67a8c00700 1 -- 172.21.15.19:0/594760728 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c493b15400 con 0x55c493c6a6a0 2024-05-07T02:40:22.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.097+0000 7f67a8c00700 10 client.16278 handle_command_reply: tid=0 2024-05-07T02:40:22.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.097+0000 7f67d0427b80 1 client.16278 shutdown 2024-05-07T02:40:22.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.097+0000 7f67d0427b80 1 -- 172.21.15.19:0/594760728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c493be5c40 msgr2=0x55c493bf8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:22.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.097+0000 7f67d0427b80 1 --2- 172.21.15.19:0/594760728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c493be5c40 0x55c493bf8510 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f67a4012010 tx=0x7f67a4010a90 comp rx=0 tx=0).stop 2024-05-07T02:40:22.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.098+0000 7f67d0427b80 1 -- 172.21.15.19:0/594760728 shutdown_connections 2024-05-07T02:40:22.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.098+0000 7f67d0427b80 1 --2- 172.21.15.19:0/594760728 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55c493c6a6a0 0x55c493c6cac0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:22.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.098+0000 7f67d0427b80 1 --2- 172.21.15.19:0/594760728 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55c493befc30 0x55c493bf3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:22.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.098+0000 7f67d0427b80 1 --2- 172.21.15.19:0/594760728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c493be5c40 0x55c493bf8510 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:22.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.098+0000 7f67d0427b80 1 --2- 172.21.15.19:0/594760728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c493be6590 0x55c493bf3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:22.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.098+0000 7f67d0427b80 1 -- 172.21.15.19:0/594760728 >> 172.21.15.19:0/594760728 conn(0x55c493a67e40 msgr2=0x55c493be4da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:22.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.098+0000 7f67a8c00700 0 client.16278 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:40:22.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.098+0000 7f67d0427b80 1 -- 172.21.15.19:0/594760728 shutdown_connections 2024-05-07T02:40:22.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.098+0000 7f67d0427b80 1 -- 172.21.15.19:0/594760728 wait complete. 2024-05-07T02:40:22.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.098+0000 7f67d0427b80 20 client.16278 trim_cache size 0 max 16384 2024-05-07T02:40:22.109 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:40:22.109 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-07T02:40:22.109 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-07T02:40:22.109 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:40:22.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.099+0000 7f67bbe00700 1 -- 172.21.15.19:0/2037649864 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f679c0b29f0 msgr2=0x7f679c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:22.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.099+0000 7f67bbe00700 1 --2- 172.21.15.19:0/2037649864 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f679c0b29f0 0x7f679c0b4f00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f67b40a6a00 tx=0x7f67b40099e0 comp rx=0 tx=0).stop 2024-05-07T02:40:22.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.099+0000 7f67bbe00700 1 -- 172.21.15.19:0/2037649864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67bc108560 msgr2=0x7f67bc1a1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:22.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.099+0000 7f67bbe00700 1 --2- 172.21.15.19:0/2037649864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67bc108560 0x7f67bc1a1680 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f67b40079a0 tx=0x7f67b40079d0 comp rx=0 tx=0).stop 2024-05-07T02:40:22.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.099+0000 7f67bbe00700 1 -- 172.21.15.19:0/2037649864 shutdown_connections 2024-05-07T02:40:22.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.099+0000 7f67bbe00700 1 --2- 172.21.15.19:0/2037649864 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67bc108560 0x7f67bc1a1680 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:22.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.099+0000 7f67bbe00700 1 --2- 172.21.15.19:0/2037649864 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f679c0b29f0 0x7f679c0b4f00 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:22.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.099+0000 7f67bbe00700 1 --2- 172.21.15.19:0/2037649864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67bc1032e0 0x7f67bc19ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:22.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.099+0000 7f67bbe00700 1 --2- 172.21.15.19:0/2037649864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67bc107c10 0x7f67bc19d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:22.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.099+0000 7f67bbe00700 1 -- 172.21.15.19:0/2037649864 >> 172.21.15.19:0/2037649864 conn(0x7f67bc078ac0 msgr2=0x7f67bc1022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:22.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.099+0000 7f67bbe00700 1 -- 172.21.15.19:0/2037649864 shutdown_connections 2024-05-07T02:40:22.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:22.099+0000 7f67bbe00700 1 -- 172.21.15.19:0/2037649864 wait complete. 2024-05-07T02:40:22.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:22 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:40:22.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:22 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:40:22.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:22 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:40:22.957 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-07T02:40:22.958 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-07T02:40:23.389 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:40:23.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:23 smithi064 ceph-mon[34699]: pgmap v263: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-05-07T02:40:23.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:23 smithi019 ceph-mon[29781]: pgmap v263: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-05-07T02:40:23.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:23 smithi027 ceph-mon[34398]: pgmap v263: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-05-07T02:40:24.597 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:24 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:40:24.598 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:24 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:40:24.598 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:24 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:40:24.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:24 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:40:24.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:24 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:40:24.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:24 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:40:24.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:24 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:40:24.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:24 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:40:24.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:24 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:40:25.548 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:25 smithi019 ceph-mon[29781]: pgmap v264: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-05-07T02:40:25.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:25 smithi064 ceph-mon[34699]: pgmap v264: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-05-07T02:40:25.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:25.905+0000 7f198f000700 1 -- 172.21.15.19:0/1399568891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f198810c5c0 msgr2=0x7f198810c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:25.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:25.905+0000 7f198f000700 1 --2- 172.21.15.19:0/1399568891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f198810c5c0 0x7f198810c9d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1978009990 tx=0x7f1978009ca0 comp rx=0 tx=0).stop 2024-05-07T02:40:25.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:25.905+0000 7f198f000700 1 -- 172.21.15.19:0/1399568891 shutdown_connections 2024-05-07T02:40:25.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:25.905+0000 7f198f000700 1 --2- 172.21.15.19:0/1399568891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1988106ed0 0x7f1988113360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:25.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:25.905+0000 7f198f000700 1 --2- 172.21.15.19:0/1399568891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f198810c5c0 0x7f198810c9d0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:25.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:25.905+0000 7f198f000700 1 --2- 172.21.15.19:0/1399568891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1988106580 0x7f1988106990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:25.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:25.905+0000 7f198f000700 1 -- 172.21.15.19:0/1399568891 >> 172.21.15.19:0/1399568891 conn(0x7f1988102080 msgr2=0x7f19881044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:25.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:25.905+0000 7f198f000700 1 -- 172.21.15.19:0/1399568891 shutdown_connections 2024-05-07T02:40:25.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:25.905+0000 7f198f000700 1 -- 172.21.15.19:0/1399568891 wait complete. 2024-05-07T02:40:25.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:25.907+0000 7f198f000700 1 Processor -- start 2024-05-07T02:40:25.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:25.907+0000 7f198f000700 1 -- start start 2024-05-07T02:40:25.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:25.907+0000 7f198f000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1988106580 0x7f198819d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:25.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:25.907+0000 7f198f000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1988106ed0 0x7f198819d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:25.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:25.907+0000 7f198f000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f198810c5c0 0x7f19881a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:25.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:25.907+0000 7f198f000700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1988116720 con 0x7f1988106580 2024-05-07T02:40:25.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:25.907+0000 7f198f000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f19881165a0 con 0x7f1988106ed0 2024-05-07T02:40:25.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:25.907+0000 7f198f000700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f19881168a0 con 0x7f198810c5c0 2024-05-07T02:40:25.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:25.908+0000 7f1987400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1988106ed0 0x7f198819d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:25.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:25.908+0000 7f1987400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1988106ed0 0x7f198819d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:48764/0 (socket says 172.21.15.19:48764) 2024-05-07T02:40:25.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:25.908+0000 7f1987400700 1 -- 172.21.15.19:0/2281905435 learned_addr learned my addr 172.21.15.19:0/2281905435 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:40:25.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:25.908+0000 7f1987e00700 1 --2- 172.21.15.19:0/2281905435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1988106580 0x7f198819d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:25.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:25.908+0000 7f198d000700 1 --2- 172.21.15.19:0/2281905435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f198810c5c0 0x7f19881a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:25.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:25.908+0000 7f1987400700 1 -- 172.21.15.19:0/2281905435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f198810c5c0 msgr2=0x7f19881a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:25.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:25.908+0000 7f1987400700 1 --2- 172.21.15.19:0/2281905435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f198810c5c0 0x7f19881a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:25.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:25.908+0000 7f1987400700 1 -- 172.21.15.19:0/2281905435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1988106580 msgr2=0x7f198819d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:25.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:25.908+0000 7f1987400700 1 --2- 172.21.15.19:0/2281905435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1988106580 0x7f198819d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:25.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:25.908+0000 7f1987400700 1 -- 172.21.15.19:0/2281905435 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19881a1f20 con 0x7f1988106ed0 2024-05-07T02:40:25.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:25.909+0000 7f1987400700 1 --2- 172.21.15.19:0/2281905435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1988106ed0 0x7f198819d5f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f197400d950 tx=0x7f197400dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:25.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:25.909+0000 7f1984c00700 1 -- 172.21.15.19:0/2281905435 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1974016070 con 0x7f1988106ed0 2024-05-07T02:40:25.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:25.909+0000 7f1984c00700 1 -- 172.21.15.19:0/2281905435 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f197400de30 con 0x7f1988106ed0 2024-05-07T02:40:25.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:25.909+0000 7f198f000700 1 -- 172.21.15.19:0/2281905435 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19881a2210 con 0x7f1988106ed0 2024-05-07T02:40:25.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:25.909+0000 7f1984c00700 1 -- 172.21.15.19:0/2281905435 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19740053a0 con 0x7f1988106ed0 2024-05-07T02:40:25.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:25.909+0000 7f198f000700 1 -- 172.21.15.19:0/2281905435 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19881a9b30 con 0x7f1988106ed0 2024-05-07T02:40:25.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:25.910+0000 7f198f000700 1 -- 172.21.15.19:0/2281905435 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19880512c0 con 0x7f1988106ed0 2024-05-07T02:40:25.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:25.913+0000 7f1984c00700 1 -- 172.21.15.19:0/2281905435 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1974005540 con 0x7f1988106ed0 2024-05-07T02:40:25.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:25.915+0000 7f1984c00700 1 --2- 172.21.15.19:0/2281905435 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f196c0b2a70 0x7f196c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:25.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:25.915+0000 7f1984c00700 1 -- 172.21.15.19:0/2281905435 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9525+0+0 (secure 0 0 0) 0x7f19740e2790 con 0x7f1988106ed0 2024-05-07T02:40:25.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:25.915+0000 7f1987e00700 1 --2- 172.21.15.19:0/2281905435 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f196c0b2a70 0x7f196c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:25.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:25.916+0000 7f1987e00700 1 --2- 172.21.15.19:0/2281905435 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f196c0b2a70 0x7f196c0b4f80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f197800b2e0 tx=0x7f1978005fb0 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:25.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:25.917+0000 7f1984c00700 1 -- 172.21.15.19:0/2281905435 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f197401f070 con 0x7f1988106ed0 2024-05-07T02:40:25.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:25 smithi027 ceph-mon[34398]: pgmap v264: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-05-07T02:40:26.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:26.205+0000 7f198f000700 1 -- 172.21.15.19:0/2281905435 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f198802e5f0 con 0x7f1988106ed0 2024-05-07T02:40:26.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:26.206+0000 7f1984c00700 1 -- 172.21.15.19:0/2281905435 <== mon.2 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11927 (secure 0 0 0) 0x7f19740a9d20 con 0x7f1988106ed0 2024-05-07T02:40:26.208 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:40:26.208 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1763670446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1763670446},{"type":"v1","addr":"172.21.15.19:6839","nonce":1763670446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/477529985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":477529985},{"type":"v1","addr":"172.21.15.64:6833","nonce":477529985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-07T02:26:14.960465+0000","modified":"2024-05-07T02:40:18.648264+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24539,"mds_2":24520,"mds_3":24515,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6833/2171873162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2171873162},{"type":"v1","addr":"172.21.15.27:6833","nonce":2171873162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/2379917839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2379917839},{"type":"v1","addr":"172.21.15.19:6835","nonce":2379917839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6837/2067515538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2067515538},{"type":"v1","addr":"172.21.15.19:6837","nonce":2067515538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6835/1000540600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1000540600},{"type":"v1","addr":"172.21.15.27:6835","nonce":1000540600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/1997318872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1997318872},{"type":"v1","addr":"172.21.15.64:6835","nonce":1997318872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":34,"addr":"172.21.15.64:6837/2585907777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2585907777},{"type":"v1","addr":"172.21.15.64:6837","nonce":2585907777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":33,"addr":"172.21.15.27:6837/2583138143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2583138143},{"type":"v1","addr":"172.21.15.27:6837","nonce":2583138143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":32,"addr":"172.21.15.64:6839/1583120004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":1583120004},{"type":"v1","addr":"172.21.15.64:6839","nonce":1583120004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":29,"addr":"172.21.15.19:6841/3905645246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3905645246},{"type":"v1","addr":"172.21.15.19:6841","nonce":3905645246}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6839/3963268634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3963268634},{"type":"v1","addr":"172.21.15.27:6839","nonce":3963268634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24539,24551,24520,24515,24505]},"id":1}]} 2024-05-07T02:40:26.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:26.209+0000 7f198f000700 1 -- 172.21.15.19:0/2281905435 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f196c0b2a70 msgr2=0x7f196c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:26.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:26.209+0000 7f198f000700 1 --2- 172.21.15.19:0/2281905435 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f196c0b2a70 0x7f196c0b4f80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f197800b2e0 tx=0x7f1978005fb0 comp rx=0 tx=0).stop 2024-05-07T02:40:26.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:26.209+0000 7f198f000700 1 -- 172.21.15.19:0/2281905435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1988106ed0 msgr2=0x7f198819d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:26.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:26.209+0000 7f198f000700 1 --2- 172.21.15.19:0/2281905435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1988106ed0 0x7f198819d5f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f197400d950 tx=0x7f197400dd10 comp rx=0 tx=0).stop 2024-05-07T02:40:26.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:26.210+0000 7f198f000700 1 -- 172.21.15.19:0/2281905435 shutdown_connections 2024-05-07T02:40:26.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:26.210+0000 7f198f000700 1 --2- 172.21.15.19:0/2281905435 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f198810c5c0 0x7f19881a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:26.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:26.210+0000 7f198f000700 1 --2- 172.21.15.19:0/2281905435 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f196c0b2a70 0x7f196c0b4f80 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:26.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:26.210+0000 7f198f000700 1 --2- 172.21.15.19:0/2281905435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1988106580 0x7f198819d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:26.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:26.210+0000 7f198f000700 1 --2- 172.21.15.19:0/2281905435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1988106ed0 0x7f198819d5f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:26.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:26.210+0000 7f198f000700 1 -- 172.21.15.19:0/2281905435 >> 172.21.15.19:0/2281905435 conn(0x7f1988102080 msgr2=0x7f19881047d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:26.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:26.210+0000 7f198f000700 1 -- 172.21.15.19:0/2281905435 shutdown_connections 2024-05-07T02:40:26.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:26.210+0000 7f198f000700 1 -- 172.21.15.19:0/2281905435 wait complete. 2024-05-07T02:40:26.215 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 48 2024-05-07T02:40:26.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:26 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/2281905435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:40:26.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:26 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/2281905435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:40:26.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:26 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/2281905435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:40:27.076 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-07T02:40:27.546 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:40:27.732 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:27 smithi019 ceph-mon[29781]: pgmap v265: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-05-07T02:40:27.733 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:27 smithi019 ceph-mon[29781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:40:27.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:27 smithi064 ceph-mon[34699]: pgmap v265: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-05-07T02:40:27.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:27 smithi064 ceph-mon[34699]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:40:27.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:27 smithi027 ceph-mon[34398]: pgmap v265: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-05-07T02:40:27.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:27 smithi027 ceph-mon[34398]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:40:29.584 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:29 smithi019 ceph-mon[29781]: pgmap v266: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:40:29.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:29 smithi064 ceph-mon[34699]: pgmap v266: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:40:29.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:29 smithi027 ceph-mon[34398]: pgmap v266: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:40:30.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.042+0000 7fc699c00700 1 -- 172.21.15.19:0/3395912293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc69410b8a0 msgr2=0x7fc694113200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:30.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.042+0000 7fc699c00700 1 --2- 172.21.15.19:0/3395912293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc69410b8a0 0x7fc694113200 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc68c00b0d0 tx=0x7fc68c00b3e0 comp rx=0 tx=0).stop 2024-05-07T02:40:30.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.042+0000 7fc699c00700 1 -- 172.21.15.19:0/3395912293 shutdown_connections 2024-05-07T02:40:30.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.042+0000 7fc699c00700 1 --2- 172.21.15.19:0/3395912293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc69410b8a0 0x7fc694113200 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:30.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.042+0000 7fc699c00700 1 --2- 172.21.15.19:0/3395912293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc694106620 0x7fc694106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:30.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.042+0000 7fc699c00700 1 --2- 172.21.15.19:0/3395912293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc69410af50 0x7fc69410b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:30.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.042+0000 7fc699c00700 1 -- 172.21.15.19:0/3395912293 >> 172.21.15.19:0/3395912293 conn(0x7fc694102030 msgr2=0x7fc694104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:30.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.042+0000 7fc699c00700 1 -- 172.21.15.19:0/3395912293 shutdown_connections 2024-05-07T02:40:30.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.042+0000 7fc699c00700 1 -- 172.21.15.19:0/3395912293 wait complete. 2024-05-07T02:40:30.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.044+0000 7fc699c00700 1 Processor -- start 2024-05-07T02:40:30.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.044+0000 7fc699c00700 1 -- start start 2024-05-07T02:40:30.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.045+0000 7fc699c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc694106620 0x7fc69419cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:30.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.045+0000 7fc699c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc69410af50 0x7fc69419d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:30.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.045+0000 7fc699c00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc69410b8a0 0x7fc6941a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:30.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.045+0000 7fc699c00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc6941164d0 con 0x7fc69410af50 2024-05-07T02:40:30.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.045+0000 7fc699c00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc694116350 con 0x7fc694106620 2024-05-07T02:40:30.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.045+0000 7fc699c00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc694116650 con 0x7fc69410b8a0 2024-05-07T02:40:30.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.045+0000 7fc699200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc69410b8a0 0x7fc6941a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:30.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.045+0000 7fc68be00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc69410af50 0x7fc69419d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:30.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.045+0000 7fc699200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc69410b8a0 0x7fc6941a1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:57410/0 (socket says 172.21.15.19:57410) 2024-05-07T02:40:30.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.045+0000 7fc699200700 1 -- 172.21.15.19:0/19546021 learned_addr learned my addr 172.21.15.19:0/19546021 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:40:30.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.045+0000 7fc693e00700 1 --2- 172.21.15.19:0/19546021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc694106620 0x7fc69419cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:30.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.045+0000 7fc699200700 1 -- 172.21.15.19:0/19546021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc694106620 msgr2=0x7fc69419cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:30.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.046+0000 7fc699200700 1 --2- 172.21.15.19:0/19546021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc694106620 0x7fc69419cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:30.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.046+0000 7fc699200700 1 -- 172.21.15.19:0/19546021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc69410af50 msgr2=0x7fc69419d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:30.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.046+0000 7fc699200700 1 --2- 172.21.15.19:0/19546021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc69410af50 0x7fc69419d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:30.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.046+0000 7fc699200700 1 -- 172.21.15.19:0/19546021 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6941a1cb0 con 0x7fc69410b8a0 2024-05-07T02:40:30.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.046+0000 7fc699200700 1 --2- 172.21.15.19:0/19546021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc69410b8a0 0x7fc6941a1610 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc68c007a00 tx=0x7fc68c007a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:30.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.046+0000 7fc691600700 1 -- 172.21.15.19:0/19546021 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c01f070 con 0x7fc69410b8a0 2024-05-07T02:40:30.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.046+0000 7fc699c00700 1 -- 172.21.15.19:0/19546021 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6941a1f40 con 0x7fc69410b8a0 2024-05-07T02:40:30.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.047+0000 7fc699c00700 1 -- 172.21.15.19:0/19546021 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6941a98c0 con 0x7fc69410b8a0 2024-05-07T02:40:30.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.047+0000 7fc691600700 1 -- 172.21.15.19:0/19546021 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc68c004c50 con 0x7fc69410b8a0 2024-05-07T02:40:30.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.047+0000 7fc691600700 1 -- 172.21.15.19:0/19546021 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c01b430 con 0x7fc69410b8a0 2024-05-07T02:40:30.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.050+0000 7fc691600700 1 -- 172.21.15.19:0/19546021 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc68c022070 con 0x7fc69410b8a0 2024-05-07T02:40:30.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.052+0000 7fc6a9d4db80 1 Processor -- start 2024-05-07T02:40:30.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.052+0000 7fc6a9d4db80 1 -- start start 2024-05-07T02:40:30.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.052+0000 7fc691600700 1 --2- 172.21.15.19:0/19546021 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc6780b2a70 0x7fc6780b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:30.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.052+0000 7fc6a9d4db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564339356c40 0x564339357050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:30.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.052+0000 7fc6a9d4db80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564339357590 0x5643393606f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:30.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.052+0000 7fc691600700 1 -- 172.21.15.19:0/19546021 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9525+0+0 (secure 0 0 0) 0x7fc68c0e3180 con 0x7fc69410b8a0 2024-05-07T02:40:30.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.052+0000 7fc6a9d4db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564339360c30 0x564339363050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:30.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.052+0000 7fc6a9d4db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564339278a20 con 0x564339356c40 2024-05-07T02:40:30.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.052+0000 7fc6a9d4db80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x564339216760 con 0x564339357590 2024-05-07T02:40:30.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.052+0000 7fc6a9d4db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5643392749e0 con 0x564339360c30 2024-05-07T02:40:30.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.052+0000 7fc68be00700 1 --2- 172.21.15.19:0/19546021 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc6780b2a70 0x7fc6780b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:30.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.053+0000 7fc68be00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564339357590 0x5643393606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:30.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.053+0000 7fc68be00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564339357590 0x5643393606f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:38036/0 (socket says 172.21.15.19:38036) 2024-05-07T02:40:30.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.053+0000 7fc68be00700 1 -- 172.21.15.19:0/920244111 learned_addr learned my addr 172.21.15.19:0/920244111 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:40:30.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.053+0000 7fc693e00700 1 --2- 172.21.15.19:0/920244111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564339356c40 0x564339357050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:30.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.053+0000 7fc699200700 1 --2- 172.21.15.19:0/920244111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564339360c30 0x564339363050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:30.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.053+0000 7fc68be00700 1 -- 172.21.15.19:0/920244111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564339360c30 msgr2=0x564339363050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:30.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.053+0000 7fc68be00700 1 --2- 172.21.15.19:0/920244111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564339360c30 0x564339363050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:30.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.053+0000 7fc68be00700 1 -- 172.21.15.19:0/920244111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564339356c40 msgr2=0x564339357050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:30.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.053+0000 7fc68be00700 1 --2- 172.21.15.19:0/920244111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564339356c40 0x564339357050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:30.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.053+0000 7fc68be00700 1 -- 172.21.15.19:0/920244111 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643391c58d0 con 0x564339357590 2024-05-07T02:40:30.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.054+0000 7fc68be00700 1 --2- 172.21.15.19:0/19546021 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc6780b2a70 0x7fc6780b4f80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc67c009fd0 tx=0x7fc67c009b40 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:30.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.054+0000 7fc68be00700 1 --2- 172.21.15.19:0/920244111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564339357590 0x5643393606f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc67c000b60 tx=0x7fc67c01d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ac2ca9829fb5928 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:30.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.054+0000 7fc689600700 1 -- 172.21.15.19:0/920244111 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c025070 con 0x564339357590 2024-05-07T02:40:30.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.054+0000 7fc689600700 1 -- 172.21.15.19:0/920244111 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc67c011e40 con 0x564339357590 2024-05-07T02:40:30.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.054+0000 7fc689600700 1 -- 172.21.15.19:0/920244111 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c021410 con 0x564339357590 2024-05-07T02:40:30.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.055+0000 7fc6a9d4db80 1 -- 172.21.15.19:0/920244111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564339357590 msgr2=0x5643393606f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:30.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.055+0000 7fc6a9d4db80 1 --2- 172.21.15.19:0/920244111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564339357590 0x5643393606f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc67c000b60 tx=0x7fc67c01d000 comp rx=0 tx=0).stop 2024-05-07T02:40:30.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.055+0000 7fc6a9d4db80 1 -- 172.21.15.19:0/920244111 shutdown_connections 2024-05-07T02:40:30.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.055+0000 7fc6a9d4db80 1 --2- 172.21.15.19:0/920244111 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564339360c30 0x564339363050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:30.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.055+0000 7fc6a9d4db80 1 --2- 172.21.15.19:0/920244111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564339356c40 0x564339357050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:30.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.055+0000 7fc6a9d4db80 1 --2- 172.21.15.19:0/920244111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564339357590 0x5643393606f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:30.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.055+0000 7fc6a9d4db80 1 -- 172.21.15.19:0/920244111 >> 172.21.15.19:0/920244111 conn(0x5643391d8e40 msgr2=0x564339351980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:30.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.055+0000 7fc6a9d4db80 1 -- 172.21.15.19:0/920244111 shutdown_connections 2024-05-07T02:40:30.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.055+0000 7fc6a9d4db80 1 -- 172.21.15.19:0/920244111 wait complete. 2024-05-07T02:40:30.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.056+0000 7fc6a9d4db80 1 -- start start 2024-05-07T02:40:30.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.056+0000 7fc6a9d4db80 1 Processor -- start 2024-05-07T02:40:30.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.057+0000 7fc6a9d4db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564339356c40 0x56433935db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:30.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.057+0000 7fc6a9d4db80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564339357590 0x56433935e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:30.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.057+0000 7fc6a9d4db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564339360c30 0x56433935e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:30.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.057+0000 7fc6a9d4db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5643393639a0 con 0x564339356c40 2024-05-07T02:40:30.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.057+0000 7fc6a9d4db80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5643391deb20 con 0x564339357590 2024-05-07T02:40:30.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.057+0000 7fc6a9d4db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564339363b20 con 0x564339360c30 2024-05-07T02:40:30.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.057+0000 7fc6a9d4db80 10 client.0 _ll_register_callbacks cb 0x56433920c380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:40:30.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.057+0000 7fc693e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564339356c40 0x56433935db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:30.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.057+0000 7fc68be00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564339357590 0x56433935e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:30.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.057+0000 7fc68be00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564339357590 0x56433935e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:38046/0 (socket says 172.21.15.19:38046) 2024-05-07T02:40:30.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.057+0000 7fc693e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564339356c40 0x56433935db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58230/0 (socket says 172.21.15.19:58230) 2024-05-07T02:40:30.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.057+0000 7fc68be00700 1 -- 172.21.15.19:0/3079886044 learned_addr learned my addr 172.21.15.19:0/3079886044 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:40:30.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.057+0000 7fc699200700 1 --2- 172.21.15.19:0/3079886044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564339360c30 0x56433935e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:30.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.058+0000 7fc699200700 1 -- 172.21.15.19:0/3079886044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564339357590 msgr2=0x56433935e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:30.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.058+0000 7fc699200700 1 --2- 172.21.15.19:0/3079886044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564339357590 0x56433935e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:30.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.058+0000 7fc699200700 1 -- 172.21.15.19:0/3079886044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564339356c40 msgr2=0x56433935db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:30.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.058+0000 7fc699200700 1 --2- 172.21.15.19:0/3079886044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564339356c40 0x56433935db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:30.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.058+0000 7fc699200700 1 -- 172.21.15.19:0/3079886044 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56433923e420 con 0x564339360c30 2024-05-07T02:40:30.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.058+0000 7fc693e00700 1 --2- 172.21.15.19:0/3079886044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564339356c40 0x56433935db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:40:30.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.058+0000 7fc68be00700 1 --2- 172.21.15.19:0/3079886044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564339357590 0x56433935e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:40:30.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.058+0000 7fc699200700 1 --2- 172.21.15.19:0/3079886044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564339360c30 0x56433935e600 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc68c0e0480 tx=0x7fc68c0e1e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=891432077572a4d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:30.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.058+0000 7fc682000700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:40:30.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.058+0000 7fc682000700 1 -- 172.21.15.19:0/3079886044 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c0a9810 con 0x564339360c30 2024-05-07T02:40:30.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.058+0000 7fc682000700 1 -- 172.21.15.19:0/3079886044 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9525+0+0 (secure 0 0 0) 0x7fc68c0c46a0 con 0x564339360c30 2024-05-07T02:40:30.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.059+0000 7fc6a9d4db80 1 -- 172.21.15.19:0/3079886044 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5643393230e0 con 0x564339360c30 2024-05-07T02:40:30.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.059+0000 7fc682000700 1 -- 172.21.15.19:0/3079886044 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fc68c0075c0 con 0x564339360c30 2024-05-07T02:40:30.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.059+0000 7fc6a9d4db80 10 client.26455 fetch_fsmap learned FSMap version 49 2024-05-07T02:40:30.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.059+0000 7fc6a9d4db80 1 -- 172.21.15.19:0/3079886044 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x564339363a00 con 0x564339360c30 2024-05-07T02:40:30.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.059+0000 7fc682000700 1 -- 172.21.15.19:0/3079886044 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5643393230e0 con 0x564339360c30 2024-05-07T02:40:30.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.059+0000 7fc682000700 1 -- 172.21.15.19:0/3079886044 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 49) ==== 8366+0+0 (secure 0 0 0) 0x7fc68c011070 con 0x564339360c30 2024-05-07T02:40:30.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.059+0000 7fc6a9d4db80 10 client.26455 fetch_fsmap finished waiting for FSMap version 49 2024-05-07T02:40:30.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.059+0000 7fc6a9d4db80 10 client.26455 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-07T02:40:30.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.060+0000 7fc6a9d4db80 20 client.26455 populate_metadata read hostname 'smithi019' 2024-05-07T02:40:30.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.060+0000 7fc6a9d4db80 1 --2- 172.21.15.19:0/3079886044 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x564339370b90 0x564339372fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:30.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.060+0000 7fc6a9d4db80 4 client.26455 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:40:30.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.060+0000 7fc6a9d4db80 1 -- 172.21.15.19:0/3079886044 --> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564339286400 con 0x564339370b90 2024-05-07T02:40:30.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.060+0000 7fc693e00700 1 --2- 172.21.15.19:0/3079886044 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x564339370b90 0x564339372fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:30.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.061+0000 7fc693e00700 1 --2- 172.21.15.19:0/3079886044 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x564339370b90 0x564339372fe0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fe7885173af55674 server_cookie=c0d34983dbb4e0ca in_seq=0 out_seq=0 2024-05-07T02:40:30.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.061+0000 7fc682000700 10 client.26455 ms_handle_connect on v2:172.21.15.64:6836/2585907777 2024-05-07T02:40:30.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.062+0000 7fc682000700 1 -- 172.21.15.19:0/3079886044 <== mds.4 v2:172.21.15.64:6836/2585907777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564339286400 con 0x564339370b90 2024-05-07T02:40:30.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.062+0000 7fc682000700 10 client.26455 handle_command_reply: tid=0 2024-05-07T02:40:30.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.062+0000 7fc6a9d4db80 1 client.26455 shutdown 2024-05-07T02:40:30.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.062+0000 7fc6a9d4db80 1 -- 172.21.15.19:0/3079886044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564339360c30 msgr2=0x56433935e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:30.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.062+0000 7fc6a9d4db80 1 --2- 172.21.15.19:0/3079886044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564339360c30 0x56433935e600 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc68c0e0480 tx=0x7fc68c0e1e60 comp rx=0 tx=0).stop 2024-05-07T02:40:30.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.062+0000 7fc6a9d4db80 1 -- 172.21.15.19:0/3079886044 shutdown_connections 2024-05-07T02:40:30.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.062+0000 7fc6a9d4db80 1 --2- 172.21.15.19:0/3079886044 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x564339370b90 0x564339372fe0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:30.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.062+0000 7fc6a9d4db80 1 --2- 172.21.15.19:0/3079886044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564339360c30 0x56433935e600 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:30.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.062+0000 7fc6a9d4db80 1 --2- 172.21.15.19:0/3079886044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564339356c40 0x56433935db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:30.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.062+0000 7fc6a9d4db80 1 --2- 172.21.15.19:0/3079886044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564339357590 0x56433935e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:30.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.062+0000 7fc682000700 0 client.26455 ms_handle_reset on v2:172.21.15.64:6836/2585907777 2024-05-07T02:40:30.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.062+0000 7fc6a9d4db80 1 -- 172.21.15.19:0/3079886044 >> 172.21.15.19:0/3079886044 conn(0x5643391d8e40 msgr2=0x564339353600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:30.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.062+0000 7fc6a9d4db80 1 -- 172.21.15.19:0/3079886044 shutdown_connections 2024-05-07T02:40:30.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.062+0000 7fc6a9d4db80 1 -- 172.21.15.19:0/3079886044 wait complete. 2024-05-07T02:40:30.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.062+0000 7fc6a9d4db80 20 client.26455 trim_cache size 0 max 16384 2024-05-07T02:40:30.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.079+0000 7fc6a9d4db80 1 Processor -- start 2024-05-07T02:40:30.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.079+0000 7fc6a9d4db80 1 -- start start 2024-05-07T02:40:30.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.080+0000 7fc6a9d4db80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564339356c40 0x56433935cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:30.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.080+0000 7fc6a9d4db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564339357590 0x56433935e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:30.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.080+0000 7fc6a9d4db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564339360c30 0x56433936cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:30.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.080+0000 7fc6a9d4db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc67000edf0 con 0x564339360c30 2024-05-07T02:40:30.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.080+0000 7fc6a9d4db80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc670010960 con 0x564339356c40 2024-05-07T02:40:30.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.080+0000 7fc6a9d4db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc67000f4c0 con 0x564339357590 2024-05-07T02:40:30.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.080+0000 7fc699200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564339360c30 0x56433936cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:30.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.080+0000 7fc699200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564339360c30 0x56433936cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58236/0 (socket says 172.21.15.19:58236) 2024-05-07T02:40:30.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.080+0000 7fc699200700 1 -- 172.21.15.19:0/3389126482 learned_addr learned my addr 172.21.15.19:0/3389126482 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:40:30.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.080+0000 7fc68be00700 1 --2- 172.21.15.19:0/3389126482 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564339357590 0x56433935e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:30.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.080+0000 7fc693e00700 1 --2- 172.21.15.19:0/3389126482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564339356c40 0x56433935cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:30.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.081+0000 7fc693e00700 1 -- 172.21.15.19:0/3389126482 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564339357590 msgr2=0x56433935e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:30.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.081+0000 7fc693e00700 1 --2- 172.21.15.19:0/3389126482 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564339357590 0x56433935e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:30.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.081+0000 7fc693e00700 1 -- 172.21.15.19:0/3389126482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564339360c30 msgr2=0x56433936cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:30.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.081+0000 7fc693e00700 1 --2- 172.21.15.19:0/3389126482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564339360c30 0x56433936cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:30.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.081+0000 7fc693e00700 1 -- 172.21.15.19:0/3389126482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56433934d730 con 0x564339356c40 2024-05-07T02:40:30.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.081+0000 7fc699200700 1 --2- 172.21.15.19:0/3389126482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564339360c30 0x56433936cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:40:30.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.081+0000 7fc693e00700 1 --2- 172.21.15.19:0/3389126482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564339356c40 0x56433935cae0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x564339370fa0 tx=0x7fc684009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71e8ae5c7e7d7953 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:30.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.081+0000 7fc688c00700 1 -- 172.21.15.19:0/3389126482 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc684004220 con 0x564339356c40 2024-05-07T02:40:30.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.081+0000 7fc688c00700 1 -- 172.21.15.19:0/3389126482 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6840043c0 con 0x564339356c40 2024-05-07T02:40:30.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.081+0000 7fc688c00700 1 -- 172.21.15.19:0/3389126482 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc684020610 con 0x564339356c40 2024-05-07T02:40:30.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.082+0000 7fc6a9d4db80 1 -- 172.21.15.19:0/3389126482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564339356c40 msgr2=0x56433935cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:30.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.082+0000 7fc6a9d4db80 1 --2- 172.21.15.19:0/3389126482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564339356c40 0x56433935cae0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x564339370fa0 tx=0x7fc684009260 comp rx=0 tx=0).stop 2024-05-07T02:40:30.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.082+0000 7fc6a9d4db80 1 -- 172.21.15.19:0/3389126482 shutdown_connections 2024-05-07T02:40:30.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.082+0000 7fc6a9d4db80 1 --2- 172.21.15.19:0/3389126482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564339360c30 0x56433936cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:30.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.082+0000 7fc6a9d4db80 1 --2- 172.21.15.19:0/3389126482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564339356c40 0x56433935cae0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:30.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.082+0000 7fc6a9d4db80 1 --2- 172.21.15.19:0/3389126482 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564339357590 0x56433935e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:30.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.082+0000 7fc6a9d4db80 1 -- 172.21.15.19:0/3389126482 >> 172.21.15.19:0/3389126482 conn(0x5643391d8e40 msgr2=0x56433935db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:30.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.083+0000 7fc6a9d4db80 1 -- 172.21.15.19:0/3389126482 shutdown_connections 2024-05-07T02:40:30.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.083+0000 7fc6a9d4db80 1 -- 172.21.15.19:0/3389126482 wait complete. 2024-05-07T02:40:30.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.083+0000 7fc6a9d4db80 1 -- start start 2024-05-07T02:40:30.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.083+0000 7fc6a9d4db80 1 Processor -- start 2024-05-07T02:40:30.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.084+0000 7fc6a9d4db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564339356c40 0x564339368c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:30.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.084+0000 7fc6a9d4db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564339357590 0x564339369200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:30.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.084+0000 7fc6a9d4db80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564339360c30 0x564339364200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:30.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.084+0000 7fc6a9d4db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc670010290 con 0x564339357590 2024-05-07T02:40:30.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.084+0000 7fc6a9d4db80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc67000fb90 con 0x564339360c30 2024-05-07T02:40:30.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.084+0000 7fc6a9d4db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc67000cdb0 con 0x564339356c40 2024-05-07T02:40:30.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.084+0000 7fc6a9d4db80 10 client.0 _ll_register_callbacks cb 0x56433920c380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:40:30.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.084+0000 7fc68be00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564339357590 0x564339369200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:30.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.084+0000 7fc693e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564339356c40 0x564339368c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:30.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.084+0000 7fc68be00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564339357590 0x564339369200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58242/0 (socket says 172.21.15.19:58242) 2024-05-07T02:40:30.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.084+0000 7fc68be00700 1 -- 172.21.15.19:0/647776254 learned_addr learned my addr 172.21.15.19:0/647776254 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:40:30.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.084+0000 7fc699200700 1 --2- 172.21.15.19:0/647776254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564339360c30 0x564339364200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:30.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.085+0000 7fc693e00700 1 -- 172.21.15.19:0/647776254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564339360c30 msgr2=0x564339364200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:30.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.085+0000 7fc693e00700 1 --2- 172.21.15.19:0/647776254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564339360c30 0x564339364200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:30.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.085+0000 7fc693e00700 1 -- 172.21.15.19:0/647776254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564339357590 msgr2=0x564339369200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:30.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.085+0000 7fc693e00700 1 --2- 172.21.15.19:0/647776254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564339357590 0x564339369200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:30.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.085+0000 7fc693e00700 1 -- 172.21.15.19:0/647776254 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564339364900 con 0x564339356c40 2024-05-07T02:40:30.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.085+0000 7fc68be00700 1 --2- 172.21.15.19:0/647776254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564339357590 0x564339369200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:40:30.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.085+0000 7fc693e00700 1 --2- 172.21.15.19:0/647776254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564339356c40 0x564339368c60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc68400a010 tx=0x7fc6840057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=654d819371b79f4b server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:30.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.085+0000 7fc682000700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:40:30.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.085+0000 7fc682000700 1 -- 172.21.15.19:0/647776254 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc684003680 con 0x564339356c40 2024-05-07T02:40:30.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.086+0000 7fc682000700 1 -- 172.21.15.19:0/647776254 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9525+0+0 (secure 0 0 0) 0x7fc684003820 con 0x564339356c40 2024-05-07T02:40:30.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.086+0000 7fc6a9d4db80 1 -- 172.21.15.19:0/647776254 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6840043c0 con 0x564339356c40 2024-05-07T02:40:30.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.086+0000 7fc682000700 1 -- 172.21.15.19:0/647776254 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fc684003a20 con 0x564339356c40 2024-05-07T02:40:30.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.086+0000 7fc6a9d4db80 10 client.26461 fetch_fsmap learned FSMap version 49 2024-05-07T02:40:30.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.086+0000 7fc6a9d4db80 1 -- 172.21.15.19:0/647776254 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5643393dacf0 con 0x564339356c40 2024-05-07T02:40:30.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.086+0000 7fc682000700 1 -- 172.21.15.19:0/647776254 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6840043c0 con 0x564339356c40 2024-05-07T02:40:30.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.086+0000 7fc682000700 1 -- 172.21.15.19:0/647776254 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 49) ==== 8366+0+0 (secure 0 0 0) 0x7fc684010430 con 0x564339356c40 2024-05-07T02:40:30.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.086+0000 7fc6a9d4db80 10 client.26461 fetch_fsmap finished waiting for FSMap version 49 2024-05-07T02:40:30.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.086+0000 7fc6a9d4db80 10 client.26461 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-07T02:40:30.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.086+0000 7fc6a9d4db80 20 client.26461 populate_metadata read hostname 'smithi019' 2024-05-07T02:40:30.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.087+0000 7fc6a9d4db80 1 --2- 172.21.15.19:0/647776254 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x5643393db390 0x5643393dd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:30.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.087+0000 7fc6a9d4db80 4 client.26461 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:40:30.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.087+0000 7fc6a9d4db80 1 -- 172.21.15.19:0/647776254 --> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5643392804a0 con 0x5643393db390 2024-05-07T02:40:30.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.087+0000 7fc68be00700 1 --2- 172.21.15.19:0/647776254 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x5643393db390 0x5643393dd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:30.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.088+0000 7fc68be00700 1 --2- 172.21.15.19:0/647776254 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x5643393db390 0x5643393dd7b0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fe08b7cde4976242 server_cookie=70b78f330358399b in_seq=0 out_seq=0 2024-05-07T02:40:30.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.088+0000 7fc682000700 10 client.26461 ms_handle_connect on v2:172.21.15.64:6836/2585907777 2024-05-07T02:40:30.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.088+0000 7fc682000700 1 -- 172.21.15.19:0/647776254 <== mds.4 v2:172.21.15.64:6836/2585907777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5643392804a0 con 0x5643393db390 2024-05-07T02:40:30.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.088+0000 7fc682000700 10 client.26461 handle_command_reply: tid=0 2024-05-07T02:40:30.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.088+0000 7fc6a9d4db80 1 client.26461 shutdown 2024-05-07T02:40:30.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.088+0000 7fc6a9d4db80 1 -- 172.21.15.19:0/647776254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564339356c40 msgr2=0x564339368c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:30.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.088+0000 7fc6a9d4db80 1 --2- 172.21.15.19:0/647776254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564339356c40 0x564339368c60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc68400a010 tx=0x7fc6840057f0 comp rx=0 tx=0).stop 2024-05-07T02:40:30.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.088+0000 7fc6a9d4db80 1 -- 172.21.15.19:0/647776254 shutdown_connections 2024-05-07T02:40:30.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.088+0000 7fc6a9d4db80 1 --2- 172.21.15.19:0/647776254 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x5643393db390 0x5643393dd7b0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:30.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.088+0000 7fc6a9d4db80 1 --2- 172.21.15.19:0/647776254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564339360c30 0x564339364200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:30.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.088+0000 7fc6a9d4db80 1 --2- 172.21.15.19:0/647776254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564339356c40 0x564339368c60 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:30.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.088+0000 7fc682000700 0 client.26461 ms_handle_reset on v2:172.21.15.64:6836/2585907777 2024-05-07T02:40:30.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.088+0000 7fc6a9d4db80 1 --2- 172.21.15.19:0/647776254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564339357590 0x564339369200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:30.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.088+0000 7fc6a9d4db80 1 -- 172.21.15.19:0/647776254 >> 172.21.15.19:0/647776254 conn(0x5643391d8e40 msgr2=0x56433935f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:30.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.088+0000 7fc6a9d4db80 1 -- 172.21.15.19:0/647776254 shutdown_connections 2024-05-07T02:40:30.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.088+0000 7fc6a9d4db80 1 -- 172.21.15.19:0/647776254 wait complete. 2024-05-07T02:40:30.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.089+0000 7fc6a9d4db80 20 client.26461 trim_cache size 0 max 16384 2024-05-07T02:40:30.096 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:40:30.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.089+0000 7fc699c00700 1 -- 172.21.15.19:0/19546021 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc6780b2a70 msgr2=0x7fc6780b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:30.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.089+0000 7fc699c00700 1 --2- 172.21.15.19:0/19546021 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc6780b2a70 0x7fc6780b4f80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc67c009fd0 tx=0x7fc67c009b40 comp rx=0 tx=0).stop 2024-05-07T02:40:30.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.089+0000 7fc699c00700 1 -- 172.21.15.19:0/19546021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc69410b8a0 msgr2=0x7fc6941a1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:30.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.089+0000 7fc699c00700 1 --2- 172.21.15.19:0/19546021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc69410b8a0 0x7fc6941a1610 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc68c007a00 tx=0x7fc68c007a30 comp rx=0 tx=0).stop 2024-05-07T02:40:30.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.089+0000 7fc699c00700 1 -- 172.21.15.19:0/19546021 shutdown_connections 2024-05-07T02:40:30.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.089+0000 7fc699c00700 1 --2- 172.21.15.19:0/19546021 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc69410b8a0 0x7fc6941a1610 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:30.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.089+0000 7fc699c00700 1 --2- 172.21.15.19:0/19546021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc694106620 0x7fc69419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:30.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.089+0000 7fc699c00700 1 --2- 172.21.15.19:0/19546021 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc6780b2a70 0x7fc6780b4f80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:30.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.089+0000 7fc699c00700 1 --2- 172.21.15.19:0/19546021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc69410af50 0x7fc69419d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:30.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.089+0000 7fc699c00700 1 -- 172.21.15.19:0/19546021 >> 172.21.15.19:0/19546021 conn(0x7fc694102030 msgr2=0x7fc69410a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:30.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.090+0000 7fc699c00700 1 -- 172.21.15.19:0/19546021 shutdown_connections 2024-05-07T02:40:30.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:30.090+0000 7fc699c00700 1 -- 172.21.15.19:0/19546021 wait complete. 2024-05-07T02:40:30.977 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-07T02:40:31.451 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:40:31.828 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:31 smithi019 ceph-mon[29781]: pgmap v267: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:40:31.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:31 smithi064 ceph-mon[34699]: pgmap v267: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:40:31.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:31 smithi027 ceph-mon[34398]: pgmap v267: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:40:33.605 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:33 smithi019 ceph-mon[29781]: pgmap v268: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:40:33.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:33 smithi064 ceph-mon[34699]: pgmap v268: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:40:33.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.863+0000 7ff947200700 1 -- 172.21.15.19:0/1307364472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9481052b0 msgr2=0x7ff9481056c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:33.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.863+0000 7ff947200700 1 --2- 172.21.15.19:0/1307364472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9481052b0 0x7ff9481056c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff938009990 tx=0x7ff938009ca0 comp rx=0 tx=0).stop 2024-05-07T02:40:33.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.863+0000 7ff947200700 1 -- 172.21.15.19:0/1307364472 shutdown_connections 2024-05-07T02:40:33.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.863+0000 7ff947200700 1 --2- 172.21.15.19:0/1307364472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff94810e720 0x7ff948110c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:33.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.863+0000 7ff947200700 1 --2- 172.21.15.19:0/1307364472 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff9481052b0 0x7ff9481056c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:33.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.863+0000 7ff947200700 1 --2- 172.21.15.19:0/1307364472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff948105c00 0x7ff94810e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:33.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.863+0000 7ff947200700 1 -- 172.21.15.19:0/1307364472 >> 172.21.15.19:0/1307364472 conn(0x7ff948078c40 msgr2=0x7ff948079080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:33.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.863+0000 7ff947200700 1 -- 172.21.15.19:0/1307364472 shutdown_connections 2024-05-07T02:40:33.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.863+0000 7ff947200700 1 -- 172.21.15.19:0/1307364472 wait complete. 2024-05-07T02:40:33.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.864+0000 7ff947200700 1 Processor -- start 2024-05-07T02:40:33.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.864+0000 7ff947200700 1 -- start start 2024-05-07T02:40:33.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.865+0000 7ff947200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9481052b0 0x7ff94819ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:33.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.865+0000 7ff947200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff948105c00 0x7ff94819d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:33.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.865+0000 7ff947200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff94810e720 0x7ff9481a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:33.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.865+0000 7ff947200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff9481142c0 con 0x7ff94810e720 2024-05-07T02:40:33.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.865+0000 7ff947200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff948114140 con 0x7ff9481052b0 2024-05-07T02:40:33.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.865+0000 7ff947200700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff948114440 con 0x7ff948105c00 2024-05-07T02:40:33.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.865+0000 7ff945e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9481052b0 0x7ff94819ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:33.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.865+0000 7ff946800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff94810e720 0x7ff9481a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:33.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.865+0000 7ff945e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9481052b0 0x7ff94819ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:38092/0 (socket says 172.21.15.19:38092) 2024-05-07T02:40:33.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.865+0000 7ff945e00700 1 -- 172.21.15.19:0/2247880018 learned_addr learned my addr 172.21.15.19:0/2247880018 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:40:33.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.866+0000 7ff945400700 1 --2- 172.21.15.19:0/2247880018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff948105c00 0x7ff94819d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:33.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.866+0000 7ff945e00700 1 -- 172.21.15.19:0/2247880018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff948105c00 msgr2=0x7ff94819d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:33.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.866+0000 7ff945e00700 1 --2- 172.21.15.19:0/2247880018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff948105c00 0x7ff94819d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:33.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.866+0000 7ff945e00700 1 -- 172.21.15.19:0/2247880018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff94810e720 msgr2=0x7ff9481a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:33.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.866+0000 7ff945e00700 1 --2- 172.21.15.19:0/2247880018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff94810e720 0x7ff9481a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:33.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.866+0000 7ff945e00700 1 -- 172.21.15.19:0/2247880018 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9481a1d60 con 0x7ff9481052b0 2024-05-07T02:40:33.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.866+0000 7ff945e00700 1 --2- 172.21.15.19:0/2247880018 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9481052b0 0x7ff94819ce90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff938000c00 tx=0x7ff9380057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:33.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.867+0000 7ff93ea00700 1 -- 172.21.15.19:0/2247880018 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff938027070 con 0x7ff9481052b0 2024-05-07T02:40:33.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.867+0000 7ff947200700 1 -- 172.21.15.19:0/2247880018 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9481a1ff0 con 0x7ff9481052b0 2024-05-07T02:40:33.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.867+0000 7ff947200700 1 -- 172.21.15.19:0/2247880018 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9481a9970 con 0x7ff9481052b0 2024-05-07T02:40:33.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.867+0000 7ff93ea00700 1 -- 172.21.15.19:0/2247880018 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff938022470 con 0x7ff9481052b0 2024-05-07T02:40:33.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.867+0000 7ff93ea00700 1 -- 172.21.15.19:0/2247880018 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff938017650 con 0x7ff9481052b0 2024-05-07T02:40:33.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.871+0000 7ff93ea00700 1 -- 172.21.15.19:0/2247880018 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff93802a070 con 0x7ff9481052b0 2024-05-07T02:40:33.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.872+0000 7ff95b488b80 1 Processor -- start 2024-05-07T02:40:33.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.872+0000 7ff95b488b80 1 -- start start 2024-05-07T02:40:33.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.873+0000 7ff93ea00700 1 --2- 172.21.15.19:0/2247880018 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff92c0b2a70 0x7ff92c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:33.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.873+0000 7ff95b488b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fa3c03a90 0x555fa3c03ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:33.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.873+0000 7ff93ea00700 1 -- 172.21.15.19:0/2247880018 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9525+0+0 (secure 0 0 0) 0x7ff9380e36a0 con 0x7ff9481052b0 2024-05-07T02:40:33.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.873+0000 7ff95b488b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fa3c043e0 0x555fa3c0d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:33.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.873+0000 7ff95b488b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa3c0db60 0x555fa3c0ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:33.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.873+0000 7ff95b488b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555fa3b25a20 con 0x555fa3c043e0 2024-05-07T02:40:33.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.873+0000 7ff95b488b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x555fa3ac3760 con 0x555fa3c03a90 2024-05-07T02:40:33.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.873+0000 7ff95b488b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555fa3b219e0 con 0x555fa3c0db60 2024-05-07T02:40:33.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.873+0000 7ff946800700 1 --2- 172.21.15.19:0/2247880018 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff92c0b2a70 0x7ff92c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:33.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.873+0000 7ff945400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fa3c043e0 0x555fa3c0d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:33.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.873+0000 7ff945e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fa3c03a90 0x555fa3c03ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:33.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.873+0000 7ff945400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fa3c043e0 0x555fa3c0d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58276/0 (socket says 172.21.15.19:58276) 2024-05-07T02:40:33.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.873+0000 7ff945400700 1 -- 172.21.15.19:0/2151092573 learned_addr learned my addr 172.21.15.19:0/2151092573 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:40:33.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.874+0000 7ff945e00700 1 -- 172.21.15.19:0/2151092573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa3c0db60 msgr2=0x555fa3c0ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:33.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.874+0000 7ff945e00700 1 --2- 172.21.15.19:0/2151092573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa3c0db60 0x555fa3c0ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:33.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.874+0000 7ff945e00700 1 -- 172.21.15.19:0/2151092573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fa3c043e0 msgr2=0x555fa3c0d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:33.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.874+0000 7ff945e00700 1 --2- 172.21.15.19:0/2151092573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fa3c043e0 0x555fa3c0d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:33.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.874+0000 7ff945e00700 1 -- 172.21.15.19:0/2151092573 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fa3a728d0 con 0x555fa3c03a90 2024-05-07T02:40:33.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.874+0000 7ff945e00700 1 --2- 172.21.15.19:0/2151092573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fa3c03a90 0x555fa3c03ea0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff9380e09a0 tx=0x7ff938020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebbbbcca35e3b454 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:33.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.874+0000 7ff946800700 1 --2- 172.21.15.19:0/2247880018 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff92c0b2a70 0x7ff92c0b4f80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff94000f540 tx=0x7ff940009920 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:33.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.874+0000 7ff933400700 1 -- 172.21.15.19:0/2151092573 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9380aa5c0 con 0x555fa3c03a90 2024-05-07T02:40:33.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.875+0000 7ff933400700 1 -- 172.21.15.19:0/2151092573 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9380ae970 con 0x555fa3c03a90 2024-05-07T02:40:33.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.875+0000 7ff933400700 1 -- 172.21.15.19:0/2151092573 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9380aa820 con 0x555fa3c03a90 2024-05-07T02:40:33.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.875+0000 7ff95b488b80 1 -- 172.21.15.19:0/2151092573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fa3c03a90 msgr2=0x555fa3c03ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:33.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.875+0000 7ff95b488b80 1 --2- 172.21.15.19:0/2151092573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fa3c03a90 0x555fa3c03ea0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff9380e09a0 tx=0x7ff938020000 comp rx=0 tx=0).stop 2024-05-07T02:40:33.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.876+0000 7ff95b488b80 1 -- 172.21.15.19:0/2151092573 shutdown_connections 2024-05-07T02:40:33.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.876+0000 7ff95b488b80 1 --2- 172.21.15.19:0/2151092573 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa3c0db60 0x555fa3c0ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:33.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.876+0000 7ff95b488b80 1 --2- 172.21.15.19:0/2151092573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fa3c03a90 0x555fa3c03ea0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:33.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.876+0000 7ff95b488b80 1 --2- 172.21.15.19:0/2151092573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fa3c043e0 0x555fa3c0d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:33.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.876+0000 7ff95b488b80 1 -- 172.21.15.19:0/2151092573 >> 172.21.15.19:0/2151092573 conn(0x555fa3a85e40 msgr2=0x555fa3bfe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:33.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.876+0000 7ff95b488b80 1 -- 172.21.15.19:0/2151092573 shutdown_connections 2024-05-07T02:40:33.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.876+0000 7ff95b488b80 1 -- 172.21.15.19:0/2151092573 wait complete. 2024-05-07T02:40:33.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.876+0000 7ff95b488b80 1 -- start start 2024-05-07T02:40:33.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.877+0000 7ff95b488b80 1 Processor -- start 2024-05-07T02:40:33.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.877+0000 7ff95b488b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa3c03a90 0x555fa3c0ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:33.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.877+0000 7ff95b488b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fa3c043e0 0x555fa3c0b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:33.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.877+0000 7ff95b488b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fa3c0db60 0x555fa3c0b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:33.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.877+0000 7ff95b488b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555fa3c10720 con 0x555fa3c043e0 2024-05-07T02:40:33.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.877+0000 7ff95b488b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x555fa3a8bb20 con 0x555fa3c0db60 2024-05-07T02:40:33.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.877+0000 7ff95b488b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555fa3c108a0 con 0x555fa3c03a90 2024-05-07T02:40:33.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.878+0000 7ff95b488b80 10 client.0 _ll_register_callbacks cb 0x555fa3accc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:40:33.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.878+0000 7ff945e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa3c03a90 0x555fa3c0ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:33.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.878+0000 7ff945400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fa3c043e0 0x555fa3c0b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:33.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.878+0000 7ff946800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fa3c0db60 0x555fa3c0b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:33.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.878+0000 7ff945e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa3c03a90 0x555fa3c0ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:57468/0 (socket says 172.21.15.19:57468) 2024-05-07T02:40:33.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.878+0000 7ff945e00700 1 -- 172.21.15.19:0/3062724153 learned_addr learned my addr 172.21.15.19:0/3062724153 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:40:33.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.878+0000 7ff945e00700 1 -- 172.21.15.19:0/3062724153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fa3c0db60 msgr2=0x555fa3c0b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:33.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.878+0000 7ff945e00700 1 --2- 172.21.15.19:0/3062724153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fa3c0db60 0x555fa3c0b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:33.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.878+0000 7ff945e00700 1 -- 172.21.15.19:0/3062724153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fa3c043e0 msgr2=0x555fa3c0b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:33.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.878+0000 7ff945e00700 1 --2- 172.21.15.19:0/3062724153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fa3c043e0 0x555fa3c0b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:33.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.878+0000 7ff945e00700 1 -- 172.21.15.19:0/3062724153 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fa3aeb420 con 0x555fa3c03a90 2024-05-07T02:40:33.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.879+0000 7ff945e00700 1 --2- 172.21.15.19:0/3062724153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa3c03a90 0x555fa3c0ab20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff9380a7830 tx=0x7ff9380096e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbeee61bd2d24a2e server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:33.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.879+0000 7ff927e00700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:40:33.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.879+0000 7ff927e00700 1 -- 172.21.15.19:0/3062724153 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9380c4060 con 0x555fa3c03a90 2024-05-07T02:40:33.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.879+0000 7ff95b488b80 1 -- 172.21.15.19:0/3062724153 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9380ae970 con 0x555fa3c03a90 2024-05-07T02:40:33.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.879+0000 7ff927e00700 1 -- 172.21.15.19:0/3062724153 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9525+0+0 (secure 0 0 0) 0x7ff9380aa820 con 0x555fa3c03a90 2024-05-07T02:40:33.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.879+0000 7ff927e00700 1 -- 172.21.15.19:0/3062724153 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7ff9380aaa20 con 0x555fa3c03a90 2024-05-07T02:40:33.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.879+0000 7ff95b488b80 10 client.26470 fetch_fsmap learned FSMap version 49 2024-05-07T02:40:33.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.879+0000 7ff95b488b80 1 -- 172.21.15.19:0/3062724153 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x555fa3c10720 con 0x555fa3c03a90 2024-05-07T02:40:33.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.880+0000 7ff927e00700 1 -- 172.21.15.19:0/3062724153 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9380ae970 con 0x555fa3c03a90 2024-05-07T02:40:33.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.880+0000 7ff927e00700 1 -- 172.21.15.19:0/3062724153 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 49) ==== 8366+0+0 (secure 0 0 0) 0x7ff9380a4ce0 con 0x555fa3c03a90 2024-05-07T02:40:33.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.880+0000 7ff95b488b80 10 client.26470 fetch_fsmap finished waiting for FSMap version 49 2024-05-07T02:40:33.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.880+0000 7ff95b488b80 10 client.26470 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-05-07T02:40:33.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.880+0000 7ff95b488b80 20 client.26470 populate_metadata read hostname 'smithi019' 2024-05-07T02:40:33.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.880+0000 7ff95b488b80 1 --2- 172.21.15.19:0/3062724153 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x555fa3c1d6b0 0x555fa3c1fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:33.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.880+0000 7ff95b488b80 4 client.26470 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:40:33.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.880+0000 7ff95b488b80 1 -- 172.21.15.19:0/3062724153 --> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555fa3bd02e0 con 0x555fa3c1d6b0 2024-05-07T02:40:33.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.880+0000 7ff945400700 1 --2- 172.21.15.19:0/3062724153 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x555fa3c1d6b0 0x555fa3c1fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:33.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.881+0000 7ff945400700 1 --2- 172.21.15.19:0/3062724153 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x555fa3c1d6b0 0x555fa3c1fb30 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4878b2c5b6600e86 server_cookie=f27d096b93fa8671 in_seq=0 out_seq=0 2024-05-07T02:40:33.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.881+0000 7ff927e00700 10 client.26470 ms_handle_connect on v2:172.21.15.27:6836/2583138143 2024-05-07T02:40:33.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.882+0000 7ff927e00700 1 -- 172.21.15.19:0/3062724153 <== mds.3 v2:172.21.15.27:6836/2583138143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555fa3bd02e0 con 0x555fa3c1d6b0 2024-05-07T02:40:33.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.882+0000 7ff927e00700 10 client.26470 handle_command_reply: tid=0 2024-05-07T02:40:33.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.882+0000 7ff95b488b80 1 client.26470 shutdown 2024-05-07T02:40:33.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.882+0000 7ff95b488b80 1 -- 172.21.15.19:0/3062724153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa3c03a90 msgr2=0x555fa3c0ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:33.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.882+0000 7ff95b488b80 1 --2- 172.21.15.19:0/3062724153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa3c03a90 0x555fa3c0ab20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff9380a7830 tx=0x7ff9380096e0 comp rx=0 tx=0).stop 2024-05-07T02:40:33.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.882+0000 7ff95b488b80 1 -- 172.21.15.19:0/3062724153 shutdown_connections 2024-05-07T02:40:33.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.882+0000 7ff95b488b80 1 --2- 172.21.15.19:0/3062724153 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x555fa3c1d6b0 0x555fa3c1fb30 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:33.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.882+0000 7ff95b488b80 1 --2- 172.21.15.19:0/3062724153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fa3c0db60 0x555fa3c0b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:33.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.882+0000 7ff95b488b80 1 --2- 172.21.15.19:0/3062724153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa3c03a90 0x555fa3c0ab20 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:33.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.882+0000 7ff95b488b80 1 --2- 172.21.15.19:0/3062724153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fa3c043e0 0x555fa3c0b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:33.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.882+0000 7ff927e00700 0 client.26470 ms_handle_reset on v2:172.21.15.27:6836/2583138143 2024-05-07T02:40:33.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.882+0000 7ff95b488b80 1 -- 172.21.15.19:0/3062724153 >> 172.21.15.19:0/3062724153 conn(0x555fa3a85e40 msgr2=0x555fa3c00450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:33.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.883+0000 7ff95b488b80 1 -- 172.21.15.19:0/3062724153 shutdown_connections 2024-05-07T02:40:33.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.883+0000 7ff95b488b80 1 -- 172.21.15.19:0/3062724153 wait complete. 2024-05-07T02:40:33.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.883+0000 7ff95b488b80 20 client.26470 trim_cache size 0 max 16384 2024-05-07T02:40:33.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.900+0000 7ff95b488b80 1 Processor -- start 2024-05-07T02:40:33.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.900+0000 7ff95b488b80 1 -- start start 2024-05-07T02:40:33.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.901+0000 7ff95b488b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fa3c03a90 0x555fa3c19770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:33.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.901+0000 7ff95b488b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fa3c043e0 0x555fa3c25330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:33.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.901+0000 7ff95b488b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa3c0db60 0x555fa3c25870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:33.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.901+0000 7ff95b488b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff9481b25c0 con 0x555fa3c043e0 2024-05-07T02:40:33.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.901+0000 7ff95b488b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff9481b4130 con 0x555fa3c03a90 2024-05-07T02:40:33.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.901+0000 7ff95b488b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff9481b2c90 con 0x555fa3c0db60 2024-05-07T02:40:33.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.901+0000 7ff945e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fa3c03a90 0x555fa3c19770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:33.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.901+0000 7ff945400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fa3c043e0 0x555fa3c25330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:33.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.901+0000 7ff945e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fa3c03a90 0x555fa3c19770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:38118/0 (socket says 172.21.15.19:38118) 2024-05-07T02:40:33.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.901+0000 7ff945e00700 1 -- 172.21.15.19:0/1053543813 learned_addr learned my addr 172.21.15.19:0/1053543813 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:40:33.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.901+0000 7ff946800700 1 --2- 172.21.15.19:0/1053543813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa3c0db60 0x555fa3c25870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:33.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.901+0000 7ff945e00700 1 -- 172.21.15.19:0/1053543813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa3c0db60 msgr2=0x555fa3c25870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:33.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.901+0000 7ff945e00700 1 --2- 172.21.15.19:0/1053543813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa3c0db60 0x555fa3c25870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:33.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.901+0000 7ff945e00700 1 -- 172.21.15.19:0/1053543813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fa3c043e0 msgr2=0x555fa3c25330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:33.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.901+0000 7ff945e00700 1 --2- 172.21.15.19:0/1053543813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fa3c043e0 0x555fa3c25330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:33.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.901+0000 7ff945e00700 1 -- 172.21.15.19:0/1053543813 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fa3c04210 con 0x555fa3c03a90 2024-05-07T02:40:33.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.902+0000 7ff945e00700 1 --2- 172.21.15.19:0/1053543813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fa3c03a90 0x555fa3c19770 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff9380aa550 tx=0x7ff938020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3f28aeb01351e4a server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:33.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.902+0000 7ff932a00700 1 -- 172.21.15.19:0/1053543813 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9380c4070 con 0x555fa3c03a90 2024-05-07T02:40:33.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.902+0000 7ff932a00700 1 -- 172.21.15.19:0/1053543813 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9380ae970 con 0x555fa3c03a90 2024-05-07T02:40:33.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.902+0000 7ff932a00700 1 -- 172.21.15.19:0/1053543813 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9380b2f00 con 0x555fa3c03a90 2024-05-07T02:40:33.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.903+0000 7ff95b488b80 1 -- 172.21.15.19:0/1053543813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fa3c03a90 msgr2=0x555fa3c19770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:33.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.903+0000 7ff95b488b80 1 --2- 172.21.15.19:0/1053543813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fa3c03a90 0x555fa3c19770 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff9380aa550 tx=0x7ff938020000 comp rx=0 tx=0).stop 2024-05-07T02:40:33.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.903+0000 7ff95b488b80 1 -- 172.21.15.19:0/1053543813 shutdown_connections 2024-05-07T02:40:33.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.903+0000 7ff95b488b80 1 --2- 172.21.15.19:0/1053543813 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa3c0db60 0x555fa3c25870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:33.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.903+0000 7ff95b488b80 1 --2- 172.21.15.19:0/1053543813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fa3c03a90 0x555fa3c19770 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:33.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.903+0000 7ff95b488b80 1 --2- 172.21.15.19:0/1053543813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fa3c043e0 0x555fa3c25330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:33.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.903+0000 7ff95b488b80 1 -- 172.21.15.19:0/1053543813 >> 172.21.15.19:0/1053543813 conn(0x555fa3a85e40 msgr2=0x555fa3c09410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:33.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.903+0000 7ff95b488b80 1 -- 172.21.15.19:0/1053543813 shutdown_connections 2024-05-07T02:40:33.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.903+0000 7ff95b488b80 1 -- 172.21.15.19:0/1053543813 wait complete. 2024-05-07T02:40:33.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.903+0000 7ff95b488b80 1 -- start start 2024-05-07T02:40:33.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.904+0000 7ff95b488b80 1 Processor -- start 2024-05-07T02:40:33.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.904+0000 7ff95b488b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fa3c03a90 0x555fa3c10ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:33.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.904+0000 7ff95b488b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fa3c043e0 0x555fa3c10ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:33.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.904+0000 7ff95b488b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa3c0db60 0x555fa3c11530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:33.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.904+0000 7ff95b488b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff9481b3a60 con 0x555fa3c043e0 2024-05-07T02:40:33.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.904+0000 7ff95b488b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff9481b3360 con 0x555fa3c03a90 2024-05-07T02:40:33.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.904+0000 7ff95b488b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff9481b0580 con 0x555fa3c0db60 2024-05-07T02:40:33.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.904+0000 7ff95b488b80 10 client.0 _ll_register_callbacks cb 0x555fa3accc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:40:33.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.905+0000 7ff945400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fa3c043e0 0x555fa3c10ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:33.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.905+0000 7ff945e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fa3c03a90 0x555fa3c10ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:33.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.905+0000 7ff945e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fa3c03a90 0x555fa3c10ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:38128/0 (socket says 172.21.15.19:38128) 2024-05-07T02:40:33.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.905+0000 7ff945e00700 1 -- 172.21.15.19:0/3152508010 learned_addr learned my addr 172.21.15.19:0/3152508010 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:40:33.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.905+0000 7ff946800700 1 --2- 172.21.15.19:0/3152508010 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa3c0db60 0x555fa3c11530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:33.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.905+0000 7ff945e00700 1 -- 172.21.15.19:0/3152508010 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa3c0db60 msgr2=0x555fa3c11530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:33.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.905+0000 7ff945e00700 1 --2- 172.21.15.19:0/3152508010 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa3c0db60 0x555fa3c11530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:33.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.905+0000 7ff945e00700 1 -- 172.21.15.19:0/3152508010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fa3c043e0 msgr2=0x555fa3c10ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:33.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.905+0000 7ff945e00700 1 --2- 172.21.15.19:0/3152508010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fa3c043e0 0x555fa3c10ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:33.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.905+0000 7ff945e00700 1 -- 172.21.15.19:0/3152508010 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fa3c86040 con 0x555fa3c03a90 2024-05-07T02:40:33.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.905+0000 7ff945400700 1 --2- 172.21.15.19:0/3152508010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fa3c043e0 0x555fa3c10ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:40:33.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.905+0000 7ff945e00700 1 --2- 172.21.15.19:0/3152508010 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fa3c03a90 0x555fa3c10ab0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff938092300 tx=0x7ff9380053a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d791751dbf0032d server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:33.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.905+0000 7ff927e00700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:40:33.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.906+0000 7ff927e00700 1 -- 172.21.15.19:0/3152508010 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9380c4070 con 0x555fa3c03a90 2024-05-07T02:40:33.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.906+0000 7ff927e00700 1 -- 172.21.15.19:0/3152508010 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9525+0+0 (secure 0 0 0) 0x7ff9380aa820 con 0x555fa3c03a90 2024-05-07T02:40:33.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.906+0000 7ff95b488b80 1 -- 172.21.15.19:0/3152508010 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9380ae970 con 0x555fa3c03a90 2024-05-07T02:40:33.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.906+0000 7ff927e00700 1 -- 172.21.15.19:0/3152508010 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7ff9380a9d30 con 0x555fa3c03a90 2024-05-07T02:40:33.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.906+0000 7ff95b488b80 10 client.26516 fetch_fsmap learned FSMap version 49 2024-05-07T02:40:33.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.906+0000 7ff95b488b80 1 -- 172.21.15.19:0/3152508010 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x555fa3c86040 con 0x555fa3c03a90 2024-05-07T02:40:33.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.906+0000 7ff927e00700 1 -- 172.21.15.19:0/3152508010 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9380ae970 con 0x555fa3c03a90 2024-05-07T02:40:33.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.907+0000 7ff927e00700 1 -- 172.21.15.19:0/3152508010 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 49) ==== 8366+0+0 (secure 0 0 0) 0x7ff9380a4ce0 con 0x555fa3c03a90 2024-05-07T02:40:33.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.907+0000 7ff95b488b80 10 client.26516 fetch_fsmap finished waiting for FSMap version 49 2024-05-07T02:40:33.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.907+0000 7ff95b488b80 10 client.26516 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-05-07T02:40:33.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.907+0000 7ff95b488b80 20 client.26516 populate_metadata read hostname 'smithi019' 2024-05-07T02:40:33.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.907+0000 7ff95b488b80 1 --2- 172.21.15.19:0/3152508010 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x555fa3c87480 0x555fa3c898a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:33.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.907+0000 7ff95b488b80 4 client.26516 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:40:33.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.907+0000 7ff95b488b80 1 -- 172.21.15.19:0/3152508010 --> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555fa3b33400 con 0x555fa3c87480 2024-05-07T02:40:33.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.907+0000 7ff945400700 1 --2- 172.21.15.19:0/3152508010 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x555fa3c87480 0x555fa3c898a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:33.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.908+0000 7ff945400700 1 --2- 172.21.15.19:0/3152508010 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x555fa3c87480 0x555fa3c898a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a02b2ba463d23af server_cookie=e555aa0ea0cc5111 in_seq=0 out_seq=0 2024-05-07T02:40:33.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.908+0000 7ff927e00700 10 client.26516 ms_handle_connect on v2:172.21.15.27:6836/2583138143 2024-05-07T02:40:33.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.908+0000 7ff927e00700 1 -- 172.21.15.19:0/3152508010 <== mds.3 v2:172.21.15.27:6836/2583138143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555fa3b33400 con 0x555fa3c87480 2024-05-07T02:40:33.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.908+0000 7ff927e00700 10 client.26516 handle_command_reply: tid=0 2024-05-07T02:40:33.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.908+0000 7ff95b488b80 1 client.26516 shutdown 2024-05-07T02:40:33.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.908+0000 7ff95b488b80 1 -- 172.21.15.19:0/3152508010 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fa3c03a90 msgr2=0x555fa3c10ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:33.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.908+0000 7ff95b488b80 1 --2- 172.21.15.19:0/3152508010 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fa3c03a90 0x555fa3c10ab0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff938092300 tx=0x7ff9380053a0 comp rx=0 tx=0).stop 2024-05-07T02:40:33.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.909+0000 7ff95b488b80 1 -- 172.21.15.19:0/3152508010 shutdown_connections 2024-05-07T02:40:33.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.909+0000 7ff95b488b80 1 --2- 172.21.15.19:0/3152508010 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x555fa3c87480 0x555fa3c898a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:33.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.909+0000 7ff95b488b80 1 --2- 172.21.15.19:0/3152508010 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555fa3c0db60 0x555fa3c11530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:33.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.909+0000 7ff95b488b80 1 --2- 172.21.15.19:0/3152508010 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555fa3c03a90 0x555fa3c10ab0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:33.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.909+0000 7ff95b488b80 1 --2- 172.21.15.19:0/3152508010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555fa3c043e0 0x555fa3c10ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:33.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.909+0000 7ff95b488b80 1 -- 172.21.15.19:0/3152508010 >> 172.21.15.19:0/3152508010 conn(0x555fa3a85e40 msgr2=0x555fa3c09000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:33.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.909+0000 7ff927e00700 0 client.26516 ms_handle_reset on v2:172.21.15.27:6836/2583138143 2024-05-07T02:40:33.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.909+0000 7ff95b488b80 1 -- 172.21.15.19:0/3152508010 shutdown_connections 2024-05-07T02:40:33.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.909+0000 7ff95b488b80 1 -- 172.21.15.19:0/3152508010 wait complete. 2024-05-07T02:40:33.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.909+0000 7ff95b488b80 20 client.26516 trim_cache size 0 max 16384 2024-05-07T02:40:33.917 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:40:33.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.910+0000 7ff947200700 1 -- 172.21.15.19:0/2247880018 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff92c0b2a70 msgr2=0x7ff92c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:33.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.910+0000 7ff947200700 1 --2- 172.21.15.19:0/2247880018 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff92c0b2a70 0x7ff92c0b4f80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff94000f540 tx=0x7ff940009920 comp rx=0 tx=0).stop 2024-05-07T02:40:33.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.910+0000 7ff947200700 1 -- 172.21.15.19:0/2247880018 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9481052b0 msgr2=0x7ff94819ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:33.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.910+0000 7ff947200700 1 --2- 172.21.15.19:0/2247880018 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9481052b0 0x7ff94819ce90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff938000c00 tx=0x7ff9380057c0 comp rx=0 tx=0).stop 2024-05-07T02:40:33.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.910+0000 7ff947200700 1 -- 172.21.15.19:0/2247880018 shutdown_connections 2024-05-07T02:40:33.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.910+0000 7ff947200700 1 --2- 172.21.15.19:0/2247880018 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff92c0b2a70 0x7ff92c0b4f80 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:33.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.910+0000 7ff947200700 1 --2- 172.21.15.19:0/2247880018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff94810e720 0x7ff9481a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:33.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.910+0000 7ff947200700 1 --2- 172.21.15.19:0/2247880018 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9481052b0 0x7ff94819ce90 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:33.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.910+0000 7ff947200700 1 --2- 172.21.15.19:0/2247880018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff948105c00 0x7ff94819d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:33.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.910+0000 7ff947200700 1 -- 172.21.15.19:0/2247880018 >> 172.21.15.19:0/2247880018 conn(0x7ff948078c40 msgr2=0x7ff9480ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:33.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.910+0000 7ff947200700 1 -- 172.21.15.19:0/2247880018 shutdown_connections 2024-05-07T02:40:33.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:33.910+0000 7ff947200700 1 -- 172.21.15.19:0/2247880018 wait complete. 2024-05-07T02:40:33.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:33 smithi027 ceph-mon[34398]: pgmap v268: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:40:34.780 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-07T02:40:35.255 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:40:35.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:35 smithi064 ceph-mon[34699]: pgmap v269: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:40:35.881 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:35 smithi019 ceph-mon[29781]: pgmap v269: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:40:35.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:35 smithi027 ceph-mon[34398]: pgmap v269: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:40:37.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.823+0000 7f05cda00700 1 -- 172.21.15.19:0/1904485450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05c8109870 msgr2=0x7f05c81111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:37.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.823+0000 7f05cda00700 1 --2- 172.21.15.19:0/1904485450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05c8109870 0x7f05c81111d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f05c000b0d0 tx=0x7f05c000b3e0 comp rx=0 tx=0).stop 2024-05-07T02:40:37.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.823+0000 7f05cda00700 1 -- 172.21.15.19:0/1904485450 shutdown_connections 2024-05-07T02:40:37.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.823+0000 7f05cda00700 1 --2- 172.21.15.19:0/1904485450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05c8109870 0x7f05c81111d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:37.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.823+0000 7f05cda00700 1 --2- 172.21.15.19:0/1904485450 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05c81045f0 0x7f05c8104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:37.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.823+0000 7f05cda00700 1 --2- 172.21.15.19:0/1904485450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05c8108f20 0x7f05c8109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:37.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.823+0000 7f05cda00700 1 -- 172.21.15.19:0/1904485450 >> 172.21.15.19:0/1904485450 conn(0x7f05c8100040 msgr2=0x7f05c8102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:37.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.823+0000 7f05cda00700 1 -- 172.21.15.19:0/1904485450 shutdown_connections 2024-05-07T02:40:37.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.823+0000 7f05cda00700 1 -- 172.21.15.19:0/1904485450 wait complete. 2024-05-07T02:40:37.828 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:37 smithi019 ceph-mon[29781]: pgmap v270: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:40:37.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.825+0000 7f05cda00700 1 Processor -- start 2024-05-07T02:40:37.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.825+0000 7f05cda00700 1 -- start start 2024-05-07T02:40:37.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.826+0000 7f05cda00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05c81045f0 0x7f05c819adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:37.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.826+0000 7f05cda00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05c8108f20 0x7f05c819b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:37.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.826+0000 7f05cda00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05c8109870 0x7f05c819f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:37.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.826+0000 7f05cda00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f05c81144a0 con 0x7f05c81045f0 2024-05-07T02:40:37.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.826+0000 7f05cda00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f05c8114320 con 0x7f05c8108f20 2024-05-07T02:40:37.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.826+0000 7f05cda00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f05c8114620 con 0x7f05c8109870 2024-05-07T02:40:37.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.826+0000 7f05c6a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05c81045f0 0x7f05c819adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:37.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.826+0000 7f05c6000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05c8108f20 0x7f05c819b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:37.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.826+0000 7f05c6000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05c8108f20 0x7f05c819b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:38146/0 (socket says 172.21.15.19:38146) 2024-05-07T02:40:37.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.826+0000 7f05c6000700 1 -- 172.21.15.19:0/1088666027 learned_addr learned my addr 172.21.15.19:0/1088666027 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:40:37.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.826+0000 7f05c7400700 1 --2- 172.21.15.19:0/1088666027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05c8109870 0x7f05c819f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:37.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.827+0000 7f05c6000700 1 -- 172.21.15.19:0/1088666027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05c8109870 msgr2=0x7f05c819f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:37.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.827+0000 7f05c6000700 1 --2- 172.21.15.19:0/1088666027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05c8109870 0x7f05c819f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:37.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.827+0000 7f05c6000700 1 -- 172.21.15.19:0/1088666027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05c81045f0 msgr2=0x7f05c819adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:37.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.827+0000 7f05c6000700 1 --2- 172.21.15.19:0/1088666027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05c81045f0 0x7f05c819adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:37.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.827+0000 7f05c6000700 1 -- 172.21.15.19:0/1088666027 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c819fc80 con 0x7f05c8108f20 2024-05-07T02:40:37.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.827+0000 7f05c6000700 1 --2- 172.21.15.19:0/1088666027 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05c8108f20 0x7f05c819b350 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f05b400e970 tx=0x7f05b400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:37.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.827+0000 7f05bf400700 1 -- 172.21.15.19:0/1088666027 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b4004a30 con 0x7f05c8108f20 2024-05-07T02:40:37.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.827+0000 7f05bf400700 1 -- 172.21.15.19:0/1088666027 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05b400cc90 con 0x7f05c8108f20 2024-05-07T02:40:37.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.827+0000 7f05cda00700 1 -- 172.21.15.19:0/1088666027 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05c819ff70 con 0x7f05c8108f20 2024-05-07T02:40:37.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.828+0000 7f05bf400700 1 -- 172.21.15.19:0/1088666027 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b4005610 con 0x7f05c8108f20 2024-05-07T02:40:37.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.828+0000 7f05cda00700 1 -- 172.21.15.19:0/1088666027 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05c81a7890 con 0x7f05c8108f20 2024-05-07T02:40:37.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.834+0000 7f05bf400700 1 -- 172.21.15.19:0/1088666027 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f05b400c7e0 con 0x7f05c8108f20 2024-05-07T02:40:37.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.834+0000 7f05dc57fb80 1 Processor -- start 2024-05-07T02:40:37.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.834+0000 7f05dc57fb80 1 -- start start 2024-05-07T02:40:37.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.835+0000 7f05dc57fb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b930bc9cd0 0x55b930bca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:37.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.835+0000 7f05dc57fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b930bca620 0x55b930bce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:37.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.835+0000 7f05dc57fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b930bce9e0 0x55b930bd6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:37.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.835+0000 7f05dc57fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b930ae7a20 con 0x55b930bca620 2024-05-07T02:40:37.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.835+0000 7f05dc57fb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b930a85760 con 0x55b930bc9cd0 2024-05-07T02:40:37.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.835+0000 7f05dc57fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b930ae39e0 con 0x55b930bce9e0 2024-05-07T02:40:37.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.835+0000 7f05c6a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b930bca620 0x55b930bce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:37.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.835+0000 7f05c7400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b930bc9cd0 0x55b930bca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:37.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.835+0000 7f05c6a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b930bca620 0x55b930bce3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58334/0 (socket says 172.21.15.19:58334) 2024-05-07T02:40:37.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.835+0000 7f05c6a00700 1 -- 172.21.15.19:0/449898404 learned_addr learned my addr 172.21.15.19:0/449898404 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:40:37.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.835+0000 7f05c6000700 1 --2- 172.21.15.19:0/449898404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b930bce9e0 0x55b930bd6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:37.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.836+0000 7f05c7400700 1 -- 172.21.15.19:0/449898404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b930bce9e0 msgr2=0x55b930bd6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:37.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.836+0000 7f05bf400700 1 --2- 172.21.15.19:0/1088666027 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f05ac0b2a70 0x7f05ac0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:37.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.836+0000 7f05c7400700 1 --2- 172.21.15.19:0/449898404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b930bce9e0 0x55b930bd6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:37.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.836+0000 7f05c7400700 1 -- 172.21.15.19:0/449898404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b930bca620 msgr2=0x55b930bce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:37.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.836+0000 7f05c7400700 1 --2- 172.21.15.19:0/449898404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b930bca620 0x55b930bce3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:37.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.836+0000 7f05c7400700 1 -- 172.21.15.19:0/449898404 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b930a348d0 con 0x55b930bc9cd0 2024-05-07T02:40:37.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.836+0000 7f05c6a00700 1 --2- 172.21.15.19:0/449898404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b930bca620 0x55b930bce3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:40:37.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.836+0000 7f05c6000700 1 --2- 172.21.15.19:0/449898404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b930bce9e0 0x55b930bd6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:40:37.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.836+0000 7f05bf400700 1 -- 172.21.15.19:0/1088666027 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9525+0+0 (secure 0 0 0) 0x7f05b40e2ab0 con 0x7f05c8108f20 2024-05-07T02:40:37.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.837+0000 7f05c7400700 1 --2- 172.21.15.19:0/1088666027 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f05ac0b2a70 0x7f05ac0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:37.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.837+0000 7f05c7400700 1 --2- 172.21.15.19:0/449898404 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b930bc9cd0 0x55b930bca0e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f05c000b770 tx=0x7f05c00098c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a543a67a63c143f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:37.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.837+0000 7f05b3e00700 1 -- 172.21.15.19:0/449898404 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c000bca0 con 0x55b930bc9cd0 2024-05-07T02:40:37.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.837+0000 7f05b3e00700 1 -- 172.21.15.19:0/449898404 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05c0020070 con 0x55b930bc9cd0 2024-05-07T02:40:37.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.837+0000 7f05b3e00700 1 -- 172.21.15.19:0/449898404 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c00075c0 con 0x55b930bc9cd0 2024-05-07T02:40:37.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.838+0000 7f05dc57fb80 1 -- 172.21.15.19:0/449898404 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b930bc9cd0 msgr2=0x55b930bca0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:37.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.838+0000 7f05c7400700 1 --2- 172.21.15.19:0/1088666027 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f05ac0b2a70 0x7f05ac0b4f80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f05c0027000 tx=0x7f05c0007800 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:37.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.838+0000 7f05dc57fb80 1 --2- 172.21.15.19:0/449898404 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b930bc9cd0 0x55b930bca0e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f05c000b770 tx=0x7f05c00098c0 comp rx=0 tx=0).stop 2024-05-07T02:40:37.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.838+0000 7f05dc57fb80 1 -- 172.21.15.19:0/449898404 shutdown_connections 2024-05-07T02:40:37.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.838+0000 7f05dc57fb80 1 --2- 172.21.15.19:0/449898404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b930bce9e0 0x55b930bd6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:37.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.838+0000 7f05dc57fb80 1 --2- 172.21.15.19:0/449898404 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b930bc9cd0 0x55b930bca0e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:37.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.838+0000 7f05dc57fb80 1 --2- 172.21.15.19:0/449898404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b930bca620 0x55b930bce3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:37.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.838+0000 7f05dc57fb80 1 -- 172.21.15.19:0/449898404 >> 172.21.15.19:0/449898404 conn(0x55b930a47e40 msgr2=0x55b930bc0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:37.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.838+0000 7f05dc57fb80 1 -- 172.21.15.19:0/449898404 shutdown_connections 2024-05-07T02:40:37.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.838+0000 7f05dc57fb80 1 -- 172.21.15.19:0/449898404 wait complete. 2024-05-07T02:40:37.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.839+0000 7f05dc57fb80 1 -- start start 2024-05-07T02:40:37.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.839+0000 7f05dc57fb80 1 Processor -- start 2024-05-07T02:40:37.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.840+0000 7f05dc57fb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b930bc9cd0 0x55b930bd6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:37.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.840+0000 7f05dc57fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b930bce9e0 0x55b930bd3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:37.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.840+0000 7f05dc57fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b930bd3d50 0x55b930bd4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:37.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.840+0000 7f05dc57fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b930bd6bd0 con 0x55b930bce9e0 2024-05-07T02:40:37.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.840+0000 7f05dc57fb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55b930a4db20 con 0x55b930bc9cd0 2024-05-07T02:40:37.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.840+0000 7f05dc57fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b930bd6d50 con 0x55b930bd3d50 2024-05-07T02:40:37.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.840+0000 7f05dc57fb80 10 client.0 _ll_register_callbacks cb 0x55b930a7b380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:40:37.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.840+0000 7f05c6a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b930bc9cd0 0x55b930bd6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:37.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.840+0000 7f05c6a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b930bc9cd0 0x55b930bd6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:38168/0 (socket says 172.21.15.19:38168) 2024-05-07T02:40:37.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.840+0000 7f05c6a00700 1 -- 172.21.15.19:0/2095555975 learned_addr learned my addr 172.21.15.19:0/2095555975 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:40:37.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.840+0000 7f05c6000700 1 --2- 172.21.15.19:0/2095555975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b930bce9e0 0x55b930bd3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:37.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.840+0000 7f05c7400700 1 --2- 172.21.15.19:0/2095555975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b930bd3d50 0x55b930bd4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:37.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.841+0000 7f05c6a00700 1 -- 172.21.15.19:0/2095555975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b930bd3d50 msgr2=0x55b930bd4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:37.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.841+0000 7f05c6a00700 1 --2- 172.21.15.19:0/2095555975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b930bd3d50 0x55b930bd4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:37.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.841+0000 7f05c6a00700 1 -- 172.21.15.19:0/2095555975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b930bce9e0 msgr2=0x55b930bd3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:37.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.841+0000 7f05c6a00700 1 --2- 172.21.15.19:0/2095555975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b930bce9e0 0x55b930bd3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:37.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.841+0000 7f05c6a00700 1 -- 172.21.15.19:0/2095555975 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b930aad420 con 0x55b930bc9cd0 2024-05-07T02:40:37.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.841+0000 7f05c6a00700 1 --2- 172.21.15.19:0/2095555975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b930bc9cd0 0x55b930bd6490 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f05b8012840 tx=0x7f05b8012b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b940dd7dbe33f08 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:37.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.841+0000 7f05b0c00700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:40:37.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.841+0000 7f05b0c00700 1 -- 172.21.15.19:0/2095555975 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b8004020 con 0x55b930bc9cd0 2024-05-07T02:40:37.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.841+0000 7f05b0c00700 1 -- 172.21.15.19:0/2095555975 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9525+0+0 (secure 0 0 0) 0x7f05b80041c0 con 0x55b930bc9cd0 2024-05-07T02:40:37.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.841+0000 7f05dc57fb80 1 -- 172.21.15.19:0/2095555975 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05c0020070 con 0x55b930bc9cd0 2024-05-07T02:40:37.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.842+0000 7f05b0c00700 1 -- 172.21.15.19:0/2095555975 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f05b80043c0 con 0x55b930bc9cd0 2024-05-07T02:40:37.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.842+0000 7f05dc57fb80 10 client.26528 fetch_fsmap learned FSMap version 49 2024-05-07T02:40:37.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.842+0000 7f05dc57fb80 1 -- 172.21.15.19:0/2095555975 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55b930be1910 con 0x55b930bc9cd0 2024-05-07T02:40:37.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.842+0000 7f05b0c00700 1 -- 172.21.15.19:0/2095555975 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05c0020070 con 0x55b930bc9cd0 2024-05-07T02:40:37.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.842+0000 7f05b0c00700 1 -- 172.21.15.19:0/2095555975 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 49) ==== 8366+0+0 (secure 0 0 0) 0x7f05b801c740 con 0x55b930bc9cd0 2024-05-07T02:40:37.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.842+0000 7f05dc57fb80 10 client.26528 fetch_fsmap finished waiting for FSMap version 49 2024-05-07T02:40:37.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.842+0000 7f05dc57fb80 10 client.26528 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-07T02:40:37.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.843+0000 7f05dc57fb80 20 client.26528 populate_metadata read hostname 'smithi019' 2024-05-07T02:40:37.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.843+0000 7f05dc57fb80 1 --2- 172.21.15.19:0/2095555975 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55b930be3f50 0x55b930be6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:37.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.843+0000 7f05dc57fb80 4 client.26528 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:40:37.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.843+0000 7f05dc57fb80 1 -- 172.21.15.19:0/2095555975 --> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b930b920e0 con 0x55b930be3f50 2024-05-07T02:40:37.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.843+0000 7f05c6000700 1 --2- 172.21.15.19:0/2095555975 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55b930be3f50 0x55b930be6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:37.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.843+0000 7f05c6000700 1 --2- 172.21.15.19:0/2095555975 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55b930be3f50 0x55b930be6370 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=44e1dd737d9d6b24 server_cookie=3d875d138dc2e632 in_seq=0 out_seq=0 2024-05-07T02:40:37.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.843+0000 7f05b0c00700 10 client.26528 ms_handle_connect on v2:172.21.15.64:6838/1583120004 2024-05-07T02:40:37.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.844+0000 7f05b0c00700 1 -- 172.21.15.19:0/2095555975 <== mds.2 v2:172.21.15.64:6838/1583120004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b930b920e0 con 0x55b930be3f50 2024-05-07T02:40:37.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.844+0000 7f05b0c00700 10 client.26528 handle_command_reply: tid=0 2024-05-07T02:40:37.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.845+0000 7f05dc57fb80 1 client.26528 shutdown 2024-05-07T02:40:37.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.845+0000 7f05dc57fb80 1 -- 172.21.15.19:0/2095555975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b930bc9cd0 msgr2=0x55b930bd6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:37.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.845+0000 7f05dc57fb80 1 --2- 172.21.15.19:0/2095555975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b930bc9cd0 0x55b930bd6490 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f05b8012840 tx=0x7f05b8012b50 comp rx=0 tx=0).stop 2024-05-07T02:40:37.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.845+0000 7f05dc57fb80 1 -- 172.21.15.19:0/2095555975 shutdown_connections 2024-05-07T02:40:37.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.845+0000 7f05dc57fb80 1 --2- 172.21.15.19:0/2095555975 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b930bd3d50 0x55b930bd4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:37.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.845+0000 7f05dc57fb80 1 --2- 172.21.15.19:0/2095555975 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55b930be3f50 0x55b930be6370 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:37.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.845+0000 7f05dc57fb80 1 --2- 172.21.15.19:0/2095555975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b930bc9cd0 0x55b930bd6490 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:37.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.845+0000 7f05dc57fb80 1 --2- 172.21.15.19:0/2095555975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b930bce9e0 0x55b930bd3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:37.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.845+0000 7f05b0c00700 0 client.26528 ms_handle_reset on v2:172.21.15.64:6838/1583120004 2024-05-07T02:40:37.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.845+0000 7f05dc57fb80 1 -- 172.21.15.19:0/2095555975 >> 172.21.15.19:0/2095555975 conn(0x55b930a47e40 msgr2=0x55b930bca0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:37.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.845+0000 7f05dc57fb80 1 -- 172.21.15.19:0/2095555975 shutdown_connections 2024-05-07T02:40:37.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.845+0000 7f05dc57fb80 1 -- 172.21.15.19:0/2095555975 wait complete. 2024-05-07T02:40:37.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.845+0000 7f05dc57fb80 20 client.26528 trim_cache size 0 max 16384 2024-05-07T02:40:37.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:37 smithi064 ceph-mon[34699]: pgmap v270: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:40:37.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.862+0000 7f05dc57fb80 1 Processor -- start 2024-05-07T02:40:37.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.862+0000 7f05dc57fb80 1 -- start start 2024-05-07T02:40:37.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.863+0000 7f05dc57fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b930bc9cd0 0x55b930bebb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:37.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.863+0000 7f05dc57fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b930bce9e0 0x55b930bec040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:37.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.863+0000 7f05dc57fb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b930bd3d50 0x55b930bd30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:37.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.863+0000 7f05dc57fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f05c81ae510 con 0x55b930bce9e0 2024-05-07T02:40:37.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.863+0000 7f05dc57fb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f05c81b0080 con 0x55b930bd3d50 2024-05-07T02:40:37.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.863+0000 7f05dc57fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f05c81aebe0 con 0x55b930bc9cd0 2024-05-07T02:40:37.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.863+0000 7f05c6a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b930bc9cd0 0x55b930bebb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:37.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.863+0000 7f05c6000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b930bce9e0 0x55b930bec040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:37.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.863+0000 7f05c6a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b930bc9cd0 0x55b930bebb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:57520/0 (socket says 172.21.15.19:57520) 2024-05-07T02:40:37.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.863+0000 7f05c6a00700 1 -- 172.21.15.19:0/3388257338 learned_addr learned my addr 172.21.15.19:0/3388257338 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:40:37.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.863+0000 7f05c7400700 1 --2- 172.21.15.19:0/3388257338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b930bd3d50 0x55b930bd30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:37.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.863+0000 7f05c6a00700 1 -- 172.21.15.19:0/3388257338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b930bd3d50 msgr2=0x55b930bd30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:37.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.863+0000 7f05c6a00700 1 --2- 172.21.15.19:0/3388257338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b930bd3d50 0x55b930bd30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:37.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.863+0000 7f05c6a00700 1 -- 172.21.15.19:0/3388257338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b930bce9e0 msgr2=0x55b930bec040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:37.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.863+0000 7f05c6a00700 1 --2- 172.21.15.19:0/3388257338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b930bce9e0 0x55b930bec040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:37.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.864+0000 7f05c6a00700 1 -- 172.21.15.19:0/3388257338 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b930bd0300 con 0x55b930bc9cd0 2024-05-07T02:40:37.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.864+0000 7f05c6000700 1 --2- 172.21.15.19:0/3388257338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b930bce9e0 0x55b930bec040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:40:37.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.864+0000 7f05c6a00700 1 --2- 172.21.15.19:0/3388257338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b930bc9cd0 0x55b930bebb00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f05b8001eb0 tx=0x7f05b801cee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b02409875b4dd989 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:37.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.864+0000 7f05b3400700 1 -- 172.21.15.19:0/3388257338 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b802f070 con 0x55b930bc9cd0 2024-05-07T02:40:37.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.864+0000 7f05b3400700 1 -- 172.21.15.19:0/3388257338 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05b801c920 con 0x55b930bc9cd0 2024-05-07T02:40:37.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.864+0000 7f05b3400700 1 -- 172.21.15.19:0/3388257338 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b8005290 con 0x55b930bc9cd0 2024-05-07T02:40:37.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.865+0000 7f05dc57fb80 1 -- 172.21.15.19:0/3388257338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b930bc9cd0 msgr2=0x55b930bebb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:37.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.865+0000 7f05dc57fb80 1 --2- 172.21.15.19:0/3388257338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b930bc9cd0 0x55b930bebb00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f05b8001eb0 tx=0x7f05b801cee0 comp rx=0 tx=0).stop 2024-05-07T02:40:37.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.865+0000 7f05dc57fb80 1 -- 172.21.15.19:0/3388257338 shutdown_connections 2024-05-07T02:40:37.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.865+0000 7f05dc57fb80 1 --2- 172.21.15.19:0/3388257338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b930bd3d50 0x55b930bd30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:37.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.865+0000 7f05dc57fb80 1 --2- 172.21.15.19:0/3388257338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b930bc9cd0 0x55b930bebb00 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:37.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.865+0000 7f05dc57fb80 1 --2- 172.21.15.19:0/3388257338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b930bce9e0 0x55b930bec040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:37.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.865+0000 7f05dc57fb80 1 -- 172.21.15.19:0/3388257338 >> 172.21.15.19:0/3388257338 conn(0x55b930a47e40 msgr2=0x55b930bd2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:37.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.865+0000 7f05dc57fb80 1 -- 172.21.15.19:0/3388257338 shutdown_connections 2024-05-07T02:40:37.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.865+0000 7f05dc57fb80 1 -- 172.21.15.19:0/3388257338 wait complete. 2024-05-07T02:40:37.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.866+0000 7f05dc57fb80 1 -- start start 2024-05-07T02:40:37.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.866+0000 7f05dc57fb80 1 Processor -- start 2024-05-07T02:40:37.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.867+0000 7f05dc57fb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b930bc9cd0 0x55b930bdbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:37.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.867+0000 7f05dc57fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b930bce9e0 0x55b930bdc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:37.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.867+0000 7f05dc57fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b930bd3d50 0x55b930bd8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:37.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.867+0000 7f05dc57fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f05c81af9b0 con 0x55b930bd3d50 2024-05-07T02:40:37.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.867+0000 7f05dc57fb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f05c81af2b0 con 0x55b930bc9cd0 2024-05-07T02:40:37.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.867+0000 7f05dc57fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f05c81ac4d0 con 0x55b930bce9e0 2024-05-07T02:40:37.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.867+0000 7f05dc57fb80 10 client.0 _ll_register_callbacks cb 0x55b930a7b380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:40:37.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.867+0000 7f05c7400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b930bd3d50 0x55b930bd8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:37.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.867+0000 7f05c6000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b930bce9e0 0x55b930bdc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:37.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.867+0000 7f05c7400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b930bd3d50 0x55b930bd8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58360/0 (socket says 172.21.15.19:58360) 2024-05-07T02:40:37.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.867+0000 7f05c6000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b930bce9e0 0x55b930bdc4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:57526/0 (socket says 172.21.15.19:57526) 2024-05-07T02:40:37.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.867+0000 7f05c6000700 1 -- 172.21.15.19:0/2775321188 learned_addr learned my addr 172.21.15.19:0/2775321188 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:40:37.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.867+0000 7f05c6a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b930bc9cd0 0x55b930bdbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:37.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.867+0000 7f05c6000700 1 -- 172.21.15.19:0/2775321188 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b930bc9cd0 msgr2=0x55b930bdbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:37.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.867+0000 7f05c6000700 1 --2- 172.21.15.19:0/2775321188 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b930bc9cd0 0x55b930bdbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:37.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.867+0000 7f05c6000700 1 -- 172.21.15.19:0/2775321188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b930bd3d50 msgr2=0x55b930bd8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:37.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.867+0000 7f05c6000700 1 --2- 172.21.15.19:0/2775321188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b930bd3d50 0x55b930bd8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:37.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.867+0000 7f05c6000700 1 -- 172.21.15.19:0/2775321188 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b930bd94b0 con 0x55b930bce9e0 2024-05-07T02:40:37.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.868+0000 7f05c6000700 1 --2- 172.21.15.19:0/2775321188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b930bce9e0 0x55b930bdc4e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f05b40a6c10 tx=0x7f05b40050b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d18a74486b8c6cc server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:37.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.868+0000 7f05b0c00700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:40:37.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.868+0000 7f05b0c00700 1 -- 172.21.15.19:0/2775321188 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b40e8070 con 0x55b930bce9e0 2024-05-07T02:40:37.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.868+0000 7f05dc57fb80 1 -- 172.21.15.19:0/2775321188 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05b801c920 con 0x55b930bce9e0 2024-05-07T02:40:37.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.868+0000 7f05b0c00700 1 -- 172.21.15.19:0/2775321188 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9525+0+0 (secure 0 0 0) 0x7f05b4016330 con 0x55b930bce9e0 2024-05-07T02:40:37.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.868+0000 7f05b0c00700 1 -- 172.21.15.19:0/2775321188 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f05b40a9770 con 0x55b930bce9e0 2024-05-07T02:40:37.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.868+0000 7f05dc57fb80 10 client.26494 fetch_fsmap learned FSMap version 49 2024-05-07T02:40:37.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.868+0000 7f05dc57fb80 1 -- 172.21.15.19:0/2775321188 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55b930c4ed40 con 0x55b930bce9e0 2024-05-07T02:40:37.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.869+0000 7f05b0c00700 1 -- 172.21.15.19:0/2775321188 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05b801c920 con 0x55b930bce9e0 2024-05-07T02:40:37.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.869+0000 7f05b0c00700 1 -- 172.21.15.19:0/2775321188 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 49) ==== 8366+0+0 (secure 0 0 0) 0x7f05b40cc0b0 con 0x55b930bce9e0 2024-05-07T02:40:37.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.869+0000 7f05dc57fb80 10 client.26494 fetch_fsmap finished waiting for FSMap version 49 2024-05-07T02:40:37.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.869+0000 7f05dc57fb80 10 client.26494 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-07T02:40:37.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.869+0000 7f05dc57fb80 20 client.26494 populate_metadata read hostname 'smithi019' 2024-05-07T02:40:37.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.869+0000 7f05dc57fb80 1 --2- 172.21.15.19:0/2775321188 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55b930c4f420 0x55b930c51840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:37.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.869+0000 7f05dc57fb80 4 client.26494 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:40:37.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.869+0000 7f05dc57fb80 1 -- 172.21.15.19:0/2775321188 --> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b930af5400 con 0x55b930c4f420 2024-05-07T02:40:37.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.869+0000 7f05c6a00700 1 --2- 172.21.15.19:0/2775321188 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55b930c4f420 0x55b930c51840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:37.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.870+0000 7f05c6a00700 1 --2- 172.21.15.19:0/2775321188 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55b930c4f420 0x55b930c51840 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=667444899cafdd95 server_cookie=881417209da6a805 in_seq=0 out_seq=0 2024-05-07T02:40:37.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.870+0000 7f05b0c00700 10 client.26494 ms_handle_connect on v2:172.21.15.64:6838/1583120004 2024-05-07T02:40:37.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.870+0000 7f05b0c00700 1 -- 172.21.15.19:0/2775321188 <== mds.2 v2:172.21.15.64:6838/1583120004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b930af5400 con 0x55b930c4f420 2024-05-07T02:40:37.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.870+0000 7f05b0c00700 10 client.26494 handle_command_reply: tid=0 2024-05-07T02:40:37.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.870+0000 7f05dc57fb80 1 client.26494 shutdown 2024-05-07T02:40:37.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.870+0000 7f05dc57fb80 1 -- 172.21.15.19:0/2775321188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b930bce9e0 msgr2=0x55b930bdc4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:37.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.870+0000 7f05dc57fb80 1 --2- 172.21.15.19:0/2775321188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b930bce9e0 0x55b930bdc4e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f05b40a6c10 tx=0x7f05b40050b0 comp rx=0 tx=0).stop 2024-05-07T02:40:37.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.871+0000 7f05dc57fb80 1 -- 172.21.15.19:0/2775321188 shutdown_connections 2024-05-07T02:40:37.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.871+0000 7f05dc57fb80 1 --2- 172.21.15.19:0/2775321188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b930bd3d50 0x55b930bd8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:37.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.871+0000 7f05dc57fb80 1 --2- 172.21.15.19:0/2775321188 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55b930c4f420 0x55b930c51840 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:37.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.871+0000 7f05dc57fb80 1 --2- 172.21.15.19:0/2775321188 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55b930bc9cd0 0x55b930bdbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:37.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.871+0000 7f05dc57fb80 1 --2- 172.21.15.19:0/2775321188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b930bce9e0 0x55b930bdc4e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:37.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.871+0000 7f05b0c00700 0 client.26494 ms_handle_reset on v2:172.21.15.64:6838/1583120004 2024-05-07T02:40:37.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.871+0000 7f05dc57fb80 1 -- 172.21.15.19:0/2775321188 >> 172.21.15.19:0/2775321188 conn(0x55b930a47e40 msgr2=0x55b930bba1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:37.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.871+0000 7f05dc57fb80 1 -- 172.21.15.19:0/2775321188 shutdown_connections 2024-05-07T02:40:37.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.871+0000 7f05dc57fb80 1 -- 172.21.15.19:0/2775321188 wait complete. 2024-05-07T02:40:37.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.871+0000 7f05dc57fb80 20 client.26494 trim_cache size 0 max 16384 2024-05-07T02:40:37.880 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:40:37.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.872+0000 7f05cda00700 1 -- 172.21.15.19:0/1088666027 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f05ac0b2a70 msgr2=0x7f05ac0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:37.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.872+0000 7f05cda00700 1 --2- 172.21.15.19:0/1088666027 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f05ac0b2a70 0x7f05ac0b4f80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f05c0027000 tx=0x7f05c0007800 comp rx=0 tx=0).stop 2024-05-07T02:40:37.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.872+0000 7f05cda00700 1 -- 172.21.15.19:0/1088666027 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05c8108f20 msgr2=0x7f05c819b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:37.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.872+0000 7f05cda00700 1 --2- 172.21.15.19:0/1088666027 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05c8108f20 0x7f05c819b350 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f05b400e970 tx=0x7f05b400ec80 comp rx=0 tx=0).stop 2024-05-07T02:40:37.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.872+0000 7f05cda00700 1 -- 172.21.15.19:0/1088666027 shutdown_connections 2024-05-07T02:40:37.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.872+0000 7f05cda00700 1 --2- 172.21.15.19:0/1088666027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f05c8109870 0x7f05c819f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:37.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.872+0000 7f05cda00700 1 --2- 172.21.15.19:0/1088666027 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f05ac0b2a70 0x7f05ac0b4f80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:37.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.872+0000 7f05cda00700 1 --2- 172.21.15.19:0/1088666027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f05c81045f0 0x7f05c819adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:37.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.872+0000 7f05cda00700 1 --2- 172.21.15.19:0/1088666027 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05c8108f20 0x7f05c819b350 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:37.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.872+0000 7f05cda00700 1 -- 172.21.15.19:0/1088666027 >> 172.21.15.19:0/1088666027 conn(0x7f05c8100040 msgr2=0x7f05c8108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:37.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.872+0000 7f05cda00700 1 -- 172.21.15.19:0/1088666027 shutdown_connections 2024-05-07T02:40:37.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:37.872+0000 7f05cda00700 1 -- 172.21.15.19:0/1088666027 wait complete. 2024-05-07T02:40:37.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:37 smithi027 ceph-mon[34398]: pgmap v270: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:40:38.863 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-07T02:40:39.339 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:40:39.859 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:39 smithi019 ceph-mon[29781]: pgmap v271: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:40:39.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:39 smithi064 ceph-mon[34699]: pgmap v271: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:40:39.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:39 smithi027 ceph-mon[34398]: pgmap v271: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:40:41.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:41 smithi064 ceph-mon[34699]: pgmap v272: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:40:41.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.875+0000 7f6094a00700 1 -- 172.21.15.19:0/445885993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60900764b0 msgr2=0x7f6090114c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:41.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.875+0000 7f6094a00700 1 --2- 172.21.15.19:0/445885993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60900764b0 0x7f6090114c40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6078009990 tx=0x7f6078009ca0 comp rx=0 tx=0).stop 2024-05-07T02:40:41.877 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:41 smithi019 ceph-mon[29781]: pgmap v272: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:40:41.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.876+0000 7f6094a00700 1 -- 172.21.15.19:0/445885993 shutdown_connections 2024-05-07T02:40:41.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.876+0000 7f6094a00700 1 --2- 172.21.15.19:0/445885993 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6090115180 0x7f60901175a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:41.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.876+0000 7f6094a00700 1 --2- 172.21.15.19:0/445885993 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6090075b60 0x7f6090075f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:41.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.876+0000 7f6094a00700 1 --2- 172.21.15.19:0/445885993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60900764b0 0x7f6090114c40 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:41.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.876+0000 7f6094a00700 1 -- 172.21.15.19:0/445885993 >> 172.21.15.19:0/445885993 conn(0x7f6090079d10 msgr2=0x7f609007a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:41.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.876+0000 7f6094a00700 1 -- 172.21.15.19:0/445885993 shutdown_connections 2024-05-07T02:40:41.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.876+0000 7f6094a00700 1 -- 172.21.15.19:0/445885993 wait complete. 2024-05-07T02:40:41.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.878+0000 7f6094a00700 1 Processor -- start 2024-05-07T02:40:41.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.878+0000 7f6094a00700 1 -- start start 2024-05-07T02:40:41.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.879+0000 7f6094a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6090075b60 0x7f60901a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:41.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.879+0000 7f6094a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60900764b0 0x7f60901a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:41.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.879+0000 7f6094a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6090115180 0x7f60901a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:41.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.879+0000 7f6094a00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f609011a8e0 con 0x7f60900764b0 2024-05-07T02:40:41.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.879+0000 7f6094a00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f609011a760 con 0x7f6090115180 2024-05-07T02:40:41.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.879+0000 7f6094a00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f609011aa60 con 0x7f6090075b60 2024-05-07T02:40:41.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.879+0000 7f608fe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6090115180 0x7f60901a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:41.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.879+0000 7f608fe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6090115180 0x7f60901a5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:36798/0 (socket says 172.21.15.19:36798) 2024-05-07T02:40:41.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.879+0000 7f608fe00700 1 -- 172.21.15.19:0/3537790357 learned_addr learned my addr 172.21.15.19:0/3537790357 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:40:41.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.879+0000 7f6086a00700 1 --2- 172.21.15.19:0/3537790357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60900764b0 0x7f60901a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:41.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.879+0000 7f608f400700 1 --2- 172.21.15.19:0/3537790357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6090075b60 0x7f60901a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:41.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.880+0000 7f608fe00700 1 -- 172.21.15.19:0/3537790357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6090075b60 msgr2=0x7f60901a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:41.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.880+0000 7f608fe00700 1 --2- 172.21.15.19:0/3537790357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6090075b60 0x7f60901a1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:41.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.880+0000 7f608fe00700 1 -- 172.21.15.19:0/3537790357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60900764b0 msgr2=0x7f60901a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:41.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.880+0000 7f608fe00700 1 --2- 172.21.15.19:0/3537790357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60900764b0 0x7f60901a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:41.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.880+0000 7f608fe00700 1 -- 172.21.15.19:0/3537790357 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60901a6140 con 0x7f6090115180 2024-05-07T02:40:41.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.880+0000 7f608fe00700 1 --2- 172.21.15.19:0/3537790357 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6090115180 0x7f60901a5aa0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f608800ef30 tx=0x7f608800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:41.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.880+0000 7f608cc00700 1 -- 172.21.15.19:0/3537790357 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60880092e0 con 0x7f6090115180 2024-05-07T02:40:41.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.880+0000 7f6094a00700 1 -- 172.21.15.19:0/3537790357 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60901a63d0 con 0x7f6090115180 2024-05-07T02:40:41.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.880+0000 7f608cc00700 1 -- 172.21.15.19:0/3537790357 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6088004bb0 con 0x7f6090115180 2024-05-07T02:40:41.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.881+0000 7f608cc00700 1 -- 172.21.15.19:0/3537790357 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6088004020 con 0x7f6090115180 2024-05-07T02:40:41.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.881+0000 7f6094a00700 1 -- 172.21.15.19:0/3537790357 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60901a68c0 con 0x7f6090115180 2024-05-07T02:40:41.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.885+0000 7f608cc00700 1 -- 172.21.15.19:0/3537790357 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f608801b030 con 0x7f6090115180 2024-05-07T02:40:41.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.886+0000 7f60a4be4b80 1 Processor -- start 2024-05-07T02:40:41.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.886+0000 7f60a4be4b80 1 -- start start 2024-05-07T02:40:41.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.886+0000 7f60a4be4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555609975c40 0x555609976050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:41.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.886+0000 7f60a4be4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555609976590 0x55560997f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:41.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.886+0000 7f60a4be4b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55560997fc30 0x555609982050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:41.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.886+0000 7f60a4be4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555609897a20 con 0x555609975c40 2024-05-07T02:40:41.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.886+0000 7f60a4be4b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x555609835760 con 0x55560997fc30 2024-05-07T02:40:41.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.886+0000 7f60a4be4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5556098939e0 con 0x555609976590 2024-05-07T02:40:41.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.886+0000 7f608cc00700 1 --2- 172.21.15.19:0/3537790357 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f60740b29f0 0x7f60740b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:41.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.887+0000 7f6086a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555609975c40 0x555609976050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:41.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.887+0000 7f608cc00700 1 -- 172.21.15.19:0/3537790357 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9525+0+0 (secure 0 0 0) 0x7f60880e24c0 con 0x7f6090115180 2024-05-07T02:40:41.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.887+0000 7f608f400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555609976590 0x55560997f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:41.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.887+0000 7f608fe00700 1 --2- 172.21.15.19:0/3537790357 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f60740b29f0 0x7f60740b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:41.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.887+0000 7f608f400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555609976590 0x55560997f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:47326/0 (socket says 172.21.15.19:47326) 2024-05-07T02:40:41.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.887+0000 7f608f400700 1 -- 172.21.15.19:0/4195281189 learned_addr learned my addr 172.21.15.19:0/4195281189 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:40:41.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.887+0000 7f6086a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555609975c40 0x555609976050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37850/0 (socket says 172.21.15.19:37850) 2024-05-07T02:40:41.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.887+0000 7f6086a00700 1 --2- 172.21.15.19:0/4195281189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55560997fc30 0x555609982050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:41.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.887+0000 7f608f400700 1 -- 172.21.15.19:0/4195281189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55560997fc30 msgr2=0x555609982050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:41.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.887+0000 7f608f400700 1 --2- 172.21.15.19:0/4195281189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55560997fc30 0x555609982050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:41.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.887+0000 7f608f400700 1 -- 172.21.15.19:0/4195281189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555609975c40 msgr2=0x555609976050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:41.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.887+0000 7f608f400700 1 --2- 172.21.15.19:0/4195281189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555609975c40 0x555609976050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:41.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.887+0000 7f608f400700 1 -- 172.21.15.19:0/4195281189 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556097e48d0 con 0x555609976590 2024-05-07T02:40:41.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.888+0000 7f608fe00700 1 --2- 172.21.15.19:0/3537790357 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f60740b29f0 0x7f60740b4f00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6088009510 tx=0x7f6088009480 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:41.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.888+0000 7f608f400700 1 --2- 172.21.15.19:0/4195281189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555609976590 0x55560997f6f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6080009d80 tx=0x7f6080011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bb76f43a0f7bcce server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:41.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.888+0000 7f6084c00700 1 -- 172.21.15.19:0/4195281189 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6080004230 con 0x555609976590 2024-05-07T02:40:41.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.888+0000 7f6084c00700 1 -- 172.21.15.19:0/4195281189 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60800043d0 con 0x555609976590 2024-05-07T02:40:41.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.888+0000 7f6084c00700 1 -- 172.21.15.19:0/4195281189 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6080019850 con 0x555609976590 2024-05-07T02:40:41.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.889+0000 7f60a4be4b80 1 -- 172.21.15.19:0/4195281189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555609976590 msgr2=0x55560997f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:41.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.889+0000 7f60a4be4b80 1 --2- 172.21.15.19:0/4195281189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555609976590 0x55560997f6f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6080009d80 tx=0x7f6080011720 comp rx=0 tx=0).stop 2024-05-07T02:40:41.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.889+0000 7f60a4be4b80 1 -- 172.21.15.19:0/4195281189 shutdown_connections 2024-05-07T02:40:41.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.889+0000 7f60a4be4b80 1 --2- 172.21.15.19:0/4195281189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55560997fc30 0x555609982050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:41.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.889+0000 7f60a4be4b80 1 --2- 172.21.15.19:0/4195281189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555609975c40 0x555609976050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:41.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.889+0000 7f60a4be4b80 1 --2- 172.21.15.19:0/4195281189 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555609976590 0x55560997f6f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:41.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.889+0000 7f60a4be4b80 1 -- 172.21.15.19:0/4195281189 >> 172.21.15.19:0/4195281189 conn(0x5556097f7e40 msgr2=0x555609970980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:41.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.890+0000 7f60a4be4b80 1 -- 172.21.15.19:0/4195281189 shutdown_connections 2024-05-07T02:40:41.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.890+0000 7f60a4be4b80 1 -- 172.21.15.19:0/4195281189 wait complete. 2024-05-07T02:40:41.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.890+0000 7f60a4be4b80 1 -- start start 2024-05-07T02:40:41.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.891+0000 7f60a4be4b80 1 Processor -- start 2024-05-07T02:40:41.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.891+0000 7f60a4be4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555609975c40 0x55560997cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:41.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.891+0000 7f60a4be4b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555609976590 0x55560997d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:41.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.891+0000 7f60a4be4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55560997fc30 0x55560997d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:41.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.891+0000 7f60a4be4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555609982a00 con 0x555609975c40 2024-05-07T02:40:41.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.891+0000 7f60a4be4b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5556097fdb20 con 0x555609976590 2024-05-07T02:40:41.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.891+0000 7f60a4be4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555609982b80 con 0x55560997fc30 2024-05-07T02:40:41.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.891+0000 7f60a4be4b80 10 client.0 _ll_register_callbacks cb 0x55560982b380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:40:41.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.891+0000 7f6086a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555609975c40 0x55560997cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:41.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.892+0000 7f608f400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555609976590 0x55560997d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:41.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.892+0000 7f6086a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555609975c40 0x55560997cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37862/0 (socket says 172.21.15.19:37862) 2024-05-07T02:40:41.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.892+0000 7f6086a00700 1 -- 172.21.15.19:0/3056775332 learned_addr learned my addr 172.21.15.19:0/3056775332 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:40:41.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.892+0000 7f6086a00700 1 --2- 172.21.15.19:0/3056775332 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55560997fc30 0x55560997d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:41.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.892+0000 7f608f400700 1 -- 172.21.15.19:0/3056775332 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55560997fc30 msgr2=0x55560997d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:41.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.892+0000 7f608f400700 1 --2- 172.21.15.19:0/3056775332 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55560997fc30 0x55560997d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:41.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.892+0000 7f608f400700 1 -- 172.21.15.19:0/3056775332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555609975c40 msgr2=0x55560997cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:41.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.892+0000 7f608f400700 1 --2- 172.21.15.19:0/3056775332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555609975c40 0x55560997cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:41.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.892+0000 7f608f400700 1 -- 172.21.15.19:0/3056775332 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55560985d420 con 0x555609976590 2024-05-07T02:40:41.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.892+0000 7f6086a00700 1 --2- 172.21.15.19:0/3056775332 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55560997fc30 0x55560997d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:40:41.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.893+0000 7f608f400700 1 --2- 172.21.15.19:0/3056775332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555609976590 0x55560997d110 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6080000c00 tx=0x7f6080003930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d636a2d0e11f6fa2 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:41.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.893+0000 7f607d600700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:40:41.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.893+0000 7f607d600700 1 -- 172.21.15.19:0/3056775332 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6080027070 con 0x555609976590 2024-05-07T02:40:41.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.893+0000 7f607d600700 1 -- 172.21.15.19:0/3056775332 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9525+0+0 (secure 0 0 0) 0x7f6080005450 con 0x555609976590 2024-05-07T02:40:41.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.893+0000 7f60a4be4b80 1 -- 172.21.15.19:0/3056775332 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60800043d0 con 0x555609976590 2024-05-07T02:40:41.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.893+0000 7f607d600700 1 -- 172.21.15.19:0/3056775332 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f6080033050 con 0x555609976590 2024-05-07T02:40:41.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.893+0000 7f60a4be4b80 10 client.26546 fetch_fsmap learned FSMap version 50 2024-05-07T02:40:41.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.893+0000 7f60a4be4b80 1 -- 172.21.15.19:0/3056775332 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55560998d600 con 0x555609976590 2024-05-07T02:40:41.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.894+0000 7f607d600700 1 -- 172.21.15.19:0/3056775332 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60800043d0 con 0x555609976590 2024-05-07T02:40:41.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.894+0000 7f607d600700 1 -- 172.21.15.19:0/3056775332 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 50) ==== 8362+0+0 (secure 0 0 0) 0x7f6080004c70 con 0x555609976590 2024-05-07T02:40:41.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.894+0000 7f60a4be4b80 10 client.26546 fetch_fsmap finished waiting for FSMap version 50 2024-05-07T02:40:41.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.894+0000 7f60a4be4b80 10 client.26546 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-07T02:40:41.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.894+0000 7f60a4be4b80 20 client.26546 populate_metadata read hostname 'smithi019' 2024-05-07T02:40:41.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.894+0000 7f60a4be4b80 1 --2- 172.21.15.19:0/3056775332 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x55560998fb30 0x555609991f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:41.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.894+0000 7f60a4be4b80 4 client.26546 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:40:41.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.894+0000 7f60a4be4b80 1 -- 172.21.15.19:0/3056775332 --> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556099420e0 con 0x55560998fb30 2024-05-07T02:40:41.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.894+0000 7f6086a00700 1 --2- 172.21.15.19:0/3056775332 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x55560998fb30 0x555609991f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:41.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.895+0000 7f6086a00700 1 --2- 172.21.15.19:0/3056775332 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x55560998fb30 0x555609991f50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=10761ba432493bf1 server_cookie=af4d5851c08ffd10 in_seq=0 out_seq=0 2024-05-07T02:40:41.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.895+0000 7f607d600700 10 client.26546 ms_handle_connect on v2:172.21.15.19:6840/3905645246 2024-05-07T02:40:41.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.896+0000 7f607d600700 1 -- 172.21.15.19:0/3056775332 <== mds.1 v2:172.21.15.19:6840/3905645246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5556099420e0 con 0x55560998fb30 2024-05-07T02:40:41.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.896+0000 7f607d600700 10 client.26546 handle_command_reply: tid=0 2024-05-07T02:40:41.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.896+0000 7f60a4be4b80 1 client.26546 shutdown 2024-05-07T02:40:41.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.896+0000 7f60a4be4b80 1 -- 172.21.15.19:0/3056775332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555609976590 msgr2=0x55560997d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:41.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.896+0000 7f60a4be4b80 1 --2- 172.21.15.19:0/3056775332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555609976590 0x55560997d110 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6080000c00 tx=0x7f6080003930 comp rx=0 tx=0).stop 2024-05-07T02:40:41.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.896+0000 7f60a4be4b80 1 -- 172.21.15.19:0/3056775332 shutdown_connections 2024-05-07T02:40:41.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.896+0000 7f60a4be4b80 1 --2- 172.21.15.19:0/3056775332 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x55560998fb30 0x555609991f50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:41.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.896+0000 7f60a4be4b80 1 --2- 172.21.15.19:0/3056775332 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55560997fc30 0x55560997d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:41.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.896+0000 7f60a4be4b80 1 --2- 172.21.15.19:0/3056775332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555609975c40 0x55560997cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:41.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.896+0000 7f60a4be4b80 1 --2- 172.21.15.19:0/3056775332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555609976590 0x55560997d110 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:41.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.896+0000 7f607d600700 0 client.26546 ms_handle_reset on v2:172.21.15.19:6840/3905645246 2024-05-07T02:40:41.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.896+0000 7f60a4be4b80 1 -- 172.21.15.19:0/3056775332 >> 172.21.15.19:0/3056775332 conn(0x5556097f7e40 msgr2=0x555609972600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:41.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.897+0000 7f60a4be4b80 1 -- 172.21.15.19:0/3056775332 shutdown_connections 2024-05-07T02:40:41.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.897+0000 7f60a4be4b80 1 -- 172.21.15.19:0/3056775332 wait complete. 2024-05-07T02:40:41.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.897+0000 7f60a4be4b80 20 client.26546 trim_cache size 0 max 16384 2024-05-07T02:40:41.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.915+0000 7f60a4be4b80 1 Processor -- start 2024-05-07T02:40:41.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.915+0000 7f60a4be4b80 1 -- start start 2024-05-07T02:40:41.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.915+0000 7f60a4be4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555609975c40 0x55560997bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:41.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.915+0000 7f60a4be4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555609976590 0x555609974d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:41.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.915+0000 7f60a4be4b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55560997fc30 0x55560998ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:41.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.915+0000 7f60a4be4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f60901b6bd0 con 0x555609976590 2024-05-07T02:40:41.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.915+0000 7f60a4be4b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f60901b8710 con 0x55560997fc30 2024-05-07T02:40:41.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.915+0000 7f60a4be4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f60901b72a0 con 0x555609975c40 2024-05-07T02:40:41.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.916+0000 7f6086a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555609975c40 0x55560997bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:41.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.916+0000 7f608f400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555609976590 0x555609974d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:41.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.916+0000 7f6086a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555609975c40 0x55560997bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:47348/0 (socket says 172.21.15.19:47348) 2024-05-07T02:40:41.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.916+0000 7f6086a00700 1 -- 172.21.15.19:0/2271256973 learned_addr learned my addr 172.21.15.19:0/2271256973 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:40:41.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.916+0000 7f6086a00700 1 --2- 172.21.15.19:0/2271256973 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55560997fc30 0x55560998ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:41.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.917+0000 7f6086a00700 1 -- 172.21.15.19:0/2271256973 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55560997fc30 msgr2=0x55560998ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:41.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.917+0000 7f6086a00700 1 --2- 172.21.15.19:0/2271256973 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55560997fc30 0x55560998ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:41.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.917+0000 7f6086a00700 1 -- 172.21.15.19:0/2271256973 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555609976590 msgr2=0x555609974d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:41.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.917+0000 7f6086a00700 1 --2- 172.21.15.19:0/2271256973 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555609976590 0x555609974d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:41.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.917+0000 7f6086a00700 1 -- 172.21.15.19:0/2271256973 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55560996c730 con 0x555609975c40 2024-05-07T02:40:41.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.917+0000 7f6086a00700 1 --2- 172.21.15.19:0/2271256973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555609975c40 0x55560997bef0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6078009350 tx=0x7f6078005760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=950906beb314e6d server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:41.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.917+0000 7f607fe00700 1 -- 172.21.15.19:0/2271256973 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6078034030 con 0x555609975c40 2024-05-07T02:40:41.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.917+0000 7f607fe00700 1 -- 172.21.15.19:0/2271256973 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6078035040 con 0x555609975c40 2024-05-07T02:40:41.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.917+0000 7f607fe00700 1 -- 172.21.15.19:0/2271256973 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f607802e390 con 0x555609975c40 2024-05-07T02:40:41.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.918+0000 7f60a4be4b80 1 -- 172.21.15.19:0/2271256973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555609975c40 msgr2=0x55560997bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:41.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.918+0000 7f60a4be4b80 1 --2- 172.21.15.19:0/2271256973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555609975c40 0x55560997bef0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6078009350 tx=0x7f6078005760 comp rx=0 tx=0).stop 2024-05-07T02:40:41.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.918+0000 7f60a4be4b80 1 -- 172.21.15.19:0/2271256973 shutdown_connections 2024-05-07T02:40:41.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.919+0000 7f60a4be4b80 1 --2- 172.21.15.19:0/2271256973 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55560997fc30 0x55560998ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:41.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.919+0000 7f60a4be4b80 1 --2- 172.21.15.19:0/2271256973 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555609975c40 0x55560997bef0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:41.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.919+0000 7f60a4be4b80 1 --2- 172.21.15.19:0/2271256973 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555609976590 0x555609974d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:41.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.919+0000 7f60a4be4b80 1 -- 172.21.15.19:0/2271256973 >> 172.21.15.19:0/2271256973 conn(0x5556097f7e40 msgr2=0x555609974610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:41.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.919+0000 7f60a4be4b80 1 -- 172.21.15.19:0/2271256973 shutdown_connections 2024-05-07T02:40:41.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.919+0000 7f60a4be4b80 1 -- 172.21.15.19:0/2271256973 wait complete. 2024-05-07T02:40:41.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.919+0000 7f60a4be4b80 1 -- start start 2024-05-07T02:40:41.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.920+0000 7f60a4be4b80 1 Processor -- start 2024-05-07T02:40:41.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.920+0000 7f60a4be4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555609975c40 0x555609988030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:41.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.920+0000 7f60a4be4b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555609976590 0x555609983390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:41.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.920+0000 7f60a4be4b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55560997fc30 0x5556099838d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:41.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.920+0000 7f60a4be4b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f60901b8040 con 0x555609976590 2024-05-07T02:40:41.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.920+0000 7f60a4be4b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f60901b7970 con 0x55560997fc30 2024-05-07T02:40:41.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.920+0000 7f60a4be4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f60901b4b90 con 0x555609975c40 2024-05-07T02:40:41.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.920+0000 7f60a4be4b80 10 client.0 _ll_register_callbacks cb 0x55560982b380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:40:41.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.921+0000 7f6086a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55560997fc30 0x5556099838d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:41.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.921+0000 7f6086a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55560997fc30 0x5556099838d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:36828/0 (socket says 172.21.15.19:36828) 2024-05-07T02:40:41.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.921+0000 7f6086a00700 1 -- 172.21.15.19:0/1072871018 learned_addr learned my addr 172.21.15.19:0/1072871018 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:40:41.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.921+0000 7f608f400700 1 --2- 172.21.15.19:0/1072871018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555609976590 0x555609983390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:41.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.921+0000 7f6086a00700 1 --2- 172.21.15.19:0/1072871018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555609975c40 0x555609988030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:41.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.921+0000 7f6086a00700 1 -- 172.21.15.19:0/1072871018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555609975c40 msgr2=0x555609988030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:41.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.921+0000 7f6086a00700 1 --2- 172.21.15.19:0/1072871018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555609975c40 0x555609988030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:41.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.921+0000 7f6086a00700 1 -- 172.21.15.19:0/1072871018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555609976590 msgr2=0x555609983390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:41.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.921+0000 7f6086a00700 1 --2- 172.21.15.19:0/1072871018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555609976590 0x555609983390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:41.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.921+0000 7f6086a00700 1 -- 172.21.15.19:0/1072871018 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555609983e10 con 0x55560997fc30 2024-05-07T02:40:41.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.921+0000 7f6086a00700 1 --2- 172.21.15.19:0/1072871018 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55560997fc30 0x5556099838d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6078005850 tx=0x7f60780098c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4efba8a69c0d4b3e server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:41.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.922+0000 7f607d600700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:40:41.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.922+0000 7f607d600700 1 -- 172.21.15.19:0/1072871018 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6078034040 con 0x55560997fc30 2024-05-07T02:40:41.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.922+0000 7f60a4be4b80 1 -- 172.21.15.19:0/1072871018 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6078035040 con 0x55560997fc30 2024-05-07T02:40:41.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.922+0000 7f607d600700 1 -- 172.21.15.19:0/1072871018 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9525+0+0 (secure 0 0 0) 0x7f607802e130 con 0x55560997fc30 2024-05-07T02:40:41.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.922+0000 7f607d600700 1 -- 172.21.15.19:0/1072871018 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f6078003880 con 0x55560997fc30 2024-05-07T02:40:41.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.922+0000 7f60a4be4b80 10 client.26552 fetch_fsmap learned FSMap version 50 2024-05-07T02:40:41.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.922+0000 7f60a4be4b80 1 -- 172.21.15.19:0/1072871018 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5556099f9d50 con 0x55560997fc30 2024-05-07T02:40:41.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.922+0000 7f607d600700 1 -- 172.21.15.19:0/1072871018 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6078035040 con 0x55560997fc30 2024-05-07T02:40:41.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.923+0000 7f607d600700 1 -- 172.21.15.19:0/1072871018 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 50) ==== 8362+0+0 (secure 0 0 0) 0x7f607802e940 con 0x55560997fc30 2024-05-07T02:40:41.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.923+0000 7f60a4be4b80 10 client.26552 fetch_fsmap finished waiting for FSMap version 50 2024-05-07T02:40:41.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.923+0000 7f60a4be4b80 10 client.26552 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-07T02:40:41.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.923+0000 7f60a4be4b80 20 client.26552 populate_metadata read hostname 'smithi019' 2024-05-07T02:40:41.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.923+0000 7f60a4be4b80 1 --2- 172.21.15.19:0/1072871018 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5556099fa2d0 0x5556099fc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:41.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.923+0000 7f60a4be4b80 4 client.26552 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:40:41.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.923+0000 7f60a4be4b80 1 -- 172.21.15.19:0/1072871018 --> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5556098a5400 con 0x5556099fa2d0 2024-05-07T02:40:41.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.923+0000 7f608f400700 1 --2- 172.21.15.19:0/1072871018 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5556099fa2d0 0x5556099fc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:41.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.924+0000 7f608f400700 1 --2- 172.21.15.19:0/1072871018 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5556099fa2d0 0x5556099fc6f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5ed798393d38d501 server_cookie=497d620567687812 in_seq=0 out_seq=0 2024-05-07T02:40:41.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.924+0000 7f607d600700 10 client.26552 ms_handle_connect on v2:172.21.15.19:6840/3905645246 2024-05-07T02:40:41.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.924+0000 7f607d600700 1 -- 172.21.15.19:0/1072871018 <== mds.1 v2:172.21.15.19:6840/3905645246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5556098a5400 con 0x5556099fa2d0 2024-05-07T02:40:41.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.924+0000 7f607d600700 10 client.26552 handle_command_reply: tid=0 2024-05-07T02:40:41.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.924+0000 7f60a4be4b80 1 client.26552 shutdown 2024-05-07T02:40:41.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.924+0000 7f60a4be4b80 1 -- 172.21.15.19:0/1072871018 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55560997fc30 msgr2=0x5556099838d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:41.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.924+0000 7f60a4be4b80 1 --2- 172.21.15.19:0/1072871018 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55560997fc30 0x5556099838d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6078005850 tx=0x7f60780098c0 comp rx=0 tx=0).stop 2024-05-07T02:40:41.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.924+0000 7f60a4be4b80 1 -- 172.21.15.19:0/1072871018 shutdown_connections 2024-05-07T02:40:41.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.924+0000 7f60a4be4b80 1 --2- 172.21.15.19:0/1072871018 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5556099fa2d0 0x5556099fc6f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:41.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.924+0000 7f60a4be4b80 1 --2- 172.21.15.19:0/1072871018 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55560997fc30 0x5556099838d0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:41.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.924+0000 7f60a4be4b80 1 --2- 172.21.15.19:0/1072871018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555609975c40 0x555609988030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:41.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.924+0000 7f60a4be4b80 1 --2- 172.21.15.19:0/1072871018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555609976590 0x555609983390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:41.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.924+0000 7f607d600700 0 client.26552 ms_handle_reset on v2:172.21.15.19:6840/3905645246 2024-05-07T02:40:41.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.924+0000 7f60a4be4b80 1 -- 172.21.15.19:0/1072871018 >> 172.21.15.19:0/1072871018 conn(0x5556097f7e40 msgr2=0x55560997bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:41.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.925+0000 7f60a4be4b80 1 -- 172.21.15.19:0/1072871018 shutdown_connections 2024-05-07T02:40:41.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.925+0000 7f60a4be4b80 1 -- 172.21.15.19:0/1072871018 wait complete. 2024-05-07T02:40:41.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.925+0000 7f60a4be4b80 20 client.26552 trim_cache size 0 max 16384 2024-05-07T02:40:41.934 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:40:41.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.926+0000 7f6094a00700 1 -- 172.21.15.19:0/3537790357 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f60740b29f0 msgr2=0x7f60740b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:41.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.926+0000 7f6094a00700 1 --2- 172.21.15.19:0/3537790357 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f60740b29f0 0x7f60740b4f00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6088009510 tx=0x7f6088009480 comp rx=0 tx=0).stop 2024-05-07T02:40:41.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.926+0000 7f6094a00700 1 -- 172.21.15.19:0/3537790357 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6090115180 msgr2=0x7f60901a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:41.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.926+0000 7f6094a00700 1 --2- 172.21.15.19:0/3537790357 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6090115180 0x7f60901a5aa0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f608800ef30 tx=0x7f608800c3a0 comp rx=0 tx=0).stop 2024-05-07T02:40:41.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.926+0000 7f6094a00700 1 -- 172.21.15.19:0/3537790357 shutdown_connections 2024-05-07T02:40:41.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.926+0000 7f6094a00700 1 --2- 172.21.15.19:0/3537790357 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6090115180 0x7f60901a5aa0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:41.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.926+0000 7f6094a00700 1 --2- 172.21.15.19:0/3537790357 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6090075b60 0x7f60901a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:41.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.926+0000 7f6094a00700 1 --2- 172.21.15.19:0/3537790357 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f60740b29f0 0x7f60740b4f00 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:41.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.926+0000 7f6094a00700 1 --2- 172.21.15.19:0/3537790357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60900764b0 0x7f60901a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:41.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.926+0000 7f6094a00700 1 -- 172.21.15.19:0/3537790357 >> 172.21.15.19:0/3537790357 conn(0x7f6090079d10 msgr2=0x7f6090100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:41.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.926+0000 7f6094a00700 1 -- 172.21.15.19:0/3537790357 shutdown_connections 2024-05-07T02:40:41.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:41.926+0000 7f6094a00700 1 -- 172.21.15.19:0/3537790357 wait complete. 2024-05-07T02:40:41.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:41 smithi027 ceph-mon[34398]: pgmap v272: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:40:42.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:42 smithi019 ceph-mon[29781]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:40:42.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:42 smithi064 ceph-mon[34699]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:40:42.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:42 smithi027 ceph-mon[34398]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-07T02:40:42.958 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-07T02:40:43.445 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:40:43.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:43 smithi064 ceph-mon[34699]: pgmap v273: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:40:43.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:43 smithi019 ceph-mon[29781]: pgmap v273: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:40:43.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:43 smithi027 ceph-mon[34398]: pgmap v273: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:40:45.685 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:45 smithi019 ceph-mon[29781]: pgmap v274: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:40:45.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:45 smithi064 ceph-mon[34699]: pgmap v274: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:40:45.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:45 smithi027 ceph-mon[34398]: pgmap v274: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:40:46.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.073+0000 7fdceae00700 1 -- 172.21.15.19:0/398767183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdce4104400 msgr2=0x7fdce4104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:46.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.073+0000 7fdceae00700 1 --2- 172.21.15.19:0/398767183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdce4104400 0x7fdce4104810 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdcd800b0d0 tx=0x7fdcd800b3e0 comp rx=0 tx=0).stop 2024-05-07T02:40:46.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.074+0000 7fdceae00700 1 -- 172.21.15.19:0/398767183 shutdown_connections 2024-05-07T02:40:46.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.074+0000 7fdceae00700 1 --2- 172.21.15.19:0/398767183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdce4109680 0x7fdce4110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:46.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.074+0000 7fdceae00700 1 --2- 172.21.15.19:0/398767183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdce4104400 0x7fdce4104810 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:46.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.074+0000 7fdceae00700 1 --2- 172.21.15.19:0/398767183 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdce4108d30 0x7fdce4109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:46.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.074+0000 7fdceae00700 1 -- 172.21.15.19:0/398767183 >> 172.21.15.19:0/398767183 conn(0x7fdce4079d10 msgr2=0x7fdce407a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:46.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.074+0000 7fdceae00700 1 -- 172.21.15.19:0/398767183 shutdown_connections 2024-05-07T02:40:46.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.074+0000 7fdceae00700 1 -- 172.21.15.19:0/398767183 wait complete. 2024-05-07T02:40:46.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.075+0000 7fdceae00700 1 Processor -- start 2024-05-07T02:40:46.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.075+0000 7fdceae00700 1 -- start start 2024-05-07T02:40:46.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.076+0000 7fdceae00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdce4104400 0x7fdce419ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:46.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.076+0000 7fdceae00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdce4108d30 0x7fdce419d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:46.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.076+0000 7fdceae00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdce4109680 0x7fdce41a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:46.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.076+0000 7fdceae00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdce41142b0 con 0x7fdce4104400 2024-05-07T02:40:46.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.076+0000 7fdceae00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdce4114130 con 0x7fdce4109680 2024-05-07T02:40:46.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.076+0000 7fdceae00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdce4114430 con 0x7fdce4108d30 2024-05-07T02:40:46.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.076+0000 7fdce9a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdce4104400 0x7fdce419ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:46.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.076+0000 7fdce9000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdce4108d30 0x7fdce419d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:46.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.076+0000 7fdce9a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdce4104400 0x7fdce419ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37906/0 (socket says 172.21.15.19:37906) 2024-05-07T02:40:46.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.076+0000 7fdce9000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdce4108d30 0x7fdce419d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:47380/0 (socket says 172.21.15.19:47380) 2024-05-07T02:40:46.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.076+0000 7fdcea400700 1 --2- 172.21.15.19:0/2224611505 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdce4109680 0x7fdce41a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:46.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.076+0000 7fdce9a00700 1 -- 172.21.15.19:0/2224611505 learned_addr learned my addr 172.21.15.19:0/2224611505 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:40:46.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.077+0000 7fdcea400700 1 -- 172.21.15.19:0/2224611505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdce4108d30 msgr2=0x7fdce419d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:46.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.077+0000 7fdcea400700 1 --2- 172.21.15.19:0/2224611505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdce4108d30 0x7fdce419d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:46.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.077+0000 7fdcea400700 1 -- 172.21.15.19:0/2224611505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdce4104400 msgr2=0x7fdce419ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:46.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.077+0000 7fdcea400700 1 --2- 172.21.15.19:0/2224611505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdce4104400 0x7fdce419ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:46.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.077+0000 7fdcea400700 1 -- 172.21.15.19:0/2224611505 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdce41a1d30 con 0x7fdce4109680 2024-05-07T02:40:46.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.077+0000 7fdcea400700 1 --2- 172.21.15.19:0/2224611505 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdce4109680 0x7fdce41a1690 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc00e900 tx=0x7fdcdc00ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:46.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.077+0000 7fdce2000700 1 -- 172.21.15.19:0/2224611505 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc004220 con 0x7fdce4109680 2024-05-07T02:40:46.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.078+0000 7fdceae00700 1 -- 172.21.15.19:0/2224611505 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdce41a2020 con 0x7fdce4109680 2024-05-07T02:40:46.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.078+0000 7fdce2000700 1 -- 172.21.15.19:0/2224611505 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcdc0043c0 con 0x7fdce4109680 2024-05-07T02:40:46.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.078+0000 7fdce2000700 1 -- 172.21.15.19:0/2224611505 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc003d50 con 0x7fdce4109680 2024-05-07T02:40:46.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.078+0000 7fdceae00700 1 -- 172.21.15.19:0/2224611505 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdce41a9940 con 0x7fdce4109680 2024-05-07T02:40:46.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.082+0000 7fdce2000700 1 -- 172.21.15.19:0/2224611505 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fdcdc00c780 con 0x7fdce4109680 2024-05-07T02:40:46.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.083+0000 7fdcfaef0b80 1 Processor -- start 2024-05-07T02:40:46.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.083+0000 7fdcfaef0b80 1 -- start start 2024-05-07T02:40:46.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.083+0000 7fdcfaef0b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ebedbeac40 0x55ebedbeb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:46.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.083+0000 7fdce2000700 1 --2- 172.21.15.19:0/2224611505 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fdccc0b29f0 0x7fdccc0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:46.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.083+0000 7fdcfaef0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ebedbeb590 0x55ebedbf46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:46.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.084+0000 7fdcfaef0b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebedbf4c30 0x55ebedbf7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:46.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.084+0000 7fdcfaef0b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ebedb0ca20 con 0x55ebedbeb590 2024-05-07T02:40:46.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.084+0000 7fdcfaef0b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ebedaaa760 con 0x55ebedbf4c30 2024-05-07T02:40:46.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.084+0000 7fdcfaef0b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ebedb089e0 con 0x55ebedbeac40 2024-05-07T02:40:46.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.084+0000 7fdce2000700 1 -- 172.21.15.19:0/2224611505 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9525+0+0 (secure 0 0 0) 0x7fdcdc0e2bc0 con 0x7fdce4109680 2024-05-07T02:40:46.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.084+0000 7fdcea400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebedbf4c30 0x55ebedbf7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:46.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.084+0000 7fdcea400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebedbf4c30 0x55ebedbf7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:36850/0 (socket says 172.21.15.19:36850) 2024-05-07T02:40:46.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.084+0000 7fdcea400700 1 -- 172.21.15.19:0/1728782762 learned_addr learned my addr 172.21.15.19:0/1728782762 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:40:46.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.084+0000 7fdce9000700 1 --2- 172.21.15.19:0/1728782762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ebedbeac40 0x55ebedbeb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:46.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.084+0000 7fdce9a00700 1 --2- 172.21.15.19:0/2224611505 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fdccc0b29f0 0x7fdccc0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:46.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.084+0000 7fdce9000700 1 --2- 172.21.15.19:0/1728782762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ebedbeb590 0x55ebedbf46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:46.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.084+0000 7fdcea400700 1 -- 172.21.15.19:0/1728782762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ebedbeac40 msgr2=0x55ebedbeb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:46.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.084+0000 7fdcea400700 1 --2- 172.21.15.19:0/1728782762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ebedbeac40 0x55ebedbeb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:46.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.084+0000 7fdcea400700 1 -- 172.21.15.19:0/1728782762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ebedbeb590 msgr2=0x55ebedbf46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:46.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.084+0000 7fdcea400700 1 --2- 172.21.15.19:0/1728782762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ebedbeb590 0x55ebedbf46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:46.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.084+0000 7fdcea400700 1 -- 172.21.15.19:0/1728782762 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebeda598d0 con 0x55ebedbf4c30 2024-05-07T02:40:46.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.085+0000 7fdce9000700 1 --2- 172.21.15.19:0/1728782762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ebedbeb590 0x55ebedbf46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-07T02:40:46.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.085+0000 7fdcea400700 1 --2- 172.21.15.19:0/1728782762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebedbf4c30 0x55ebedbf7050 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc009230 tx=0x7fdcdc0a93a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=470fbb1e974e01ce server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:46.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.085+0000 7fdce9a00700 1 --2- 172.21.15.19:0/2224611505 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fdccc0b29f0 0x7fdccc0b4f00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8009900 tx=0x7fdcd800b040 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:46.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.085+0000 7fdcd6a00700 1 -- 172.21.15.19:0/1728782762 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc0a5070 con 0x55ebedbf4c30 2024-05-07T02:40:46.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.085+0000 7fdcd6a00700 1 -- 172.21.15.19:0/1728782762 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcdc0b2030 con 0x55ebedbf4c30 2024-05-07T02:40:46.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.085+0000 7fdcd6a00700 1 -- 172.21.15.19:0/1728782762 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc0b28b0 con 0x55ebedbf4c30 2024-05-07T02:40:46.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.086+0000 7fdcfaef0b80 1 -- 172.21.15.19:0/1728782762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebedbf4c30 msgr2=0x55ebedbf7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:46.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.086+0000 7fdcfaef0b80 1 --2- 172.21.15.19:0/1728782762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebedbf4c30 0x55ebedbf7050 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc009230 tx=0x7fdcdc0a93a0 comp rx=0 tx=0).stop 2024-05-07T02:40:46.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.086+0000 7fdcfaef0b80 1 -- 172.21.15.19:0/1728782762 shutdown_connections 2024-05-07T02:40:46.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.086+0000 7fdcfaef0b80 1 --2- 172.21.15.19:0/1728782762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebedbf4c30 0x55ebedbf7050 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:46.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.086+0000 7fdcfaef0b80 1 --2- 172.21.15.19:0/1728782762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ebedbeac40 0x55ebedbeb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:46.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.086+0000 7fdcfaef0b80 1 --2- 172.21.15.19:0/1728782762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ebedbeb590 0x55ebedbf46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:46.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.086+0000 7fdcfaef0b80 1 -- 172.21.15.19:0/1728782762 >> 172.21.15.19:0/1728782762 conn(0x55ebeda6ce40 msgr2=0x55ebedbe5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:46.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.086+0000 7fdcfaef0b80 1 -- 172.21.15.19:0/1728782762 shutdown_connections 2024-05-07T02:40:46.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.086+0000 7fdcfaef0b80 1 -- 172.21.15.19:0/1728782762 wait complete. 2024-05-07T02:40:46.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.087+0000 7fdcfaef0b80 1 -- start start 2024-05-07T02:40:46.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.087+0000 7fdcfaef0b80 1 Processor -- start 2024-05-07T02:40:46.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.088+0000 7fdcfaef0b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebedbeac40 0x55ebedbf1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:46.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.088+0000 7fdcfaef0b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ebedbeb590 0x55ebedbf2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:46.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.088+0000 7fdcfaef0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ebedbf4c30 0x55ebedbf2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:46.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.088+0000 7fdcfaef0b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ebedbf7a00 con 0x55ebedbf4c30 2024-05-07T02:40:46.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.088+0000 7fdcfaef0b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55ebeda72b20 con 0x55ebedbeac40 2024-05-07T02:40:46.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.088+0000 7fdcfaef0b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ebedbf7b80 con 0x55ebedbeb590 2024-05-07T02:40:46.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.088+0000 7fdcfaef0b80 10 client.0 _ll_register_callbacks cb 0x55ebedaa0380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:40:46.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.088+0000 7fdce9000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ebedbeb590 0x55ebedbf2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:46.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.088+0000 7fdcea400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ebedbf4c30 0x55ebedbf2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:46.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.088+0000 7fdce9000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ebedbeb590 0x55ebedbf2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:47394/0 (socket says 172.21.15.19:47394) 2024-05-07T02:40:46.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.088+0000 7fdce9000700 1 -- 172.21.15.19:0/2952830393 learned_addr learned my addr 172.21.15.19:0/2952830393 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:40:46.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.089+0000 7fdce9a00700 1 --2- 172.21.15.19:0/2952830393 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebedbeac40 0x55ebedbf1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:46.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.089+0000 7fdce9000700 1 -- 172.21.15.19:0/2952830393 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebedbeac40 msgr2=0x55ebedbf1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:46.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.089+0000 7fdce9000700 1 --2- 172.21.15.19:0/2952830393 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebedbeac40 0x55ebedbf1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:46.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.089+0000 7fdce9000700 1 -- 172.21.15.19:0/2952830393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ebedbf4c30 msgr2=0x55ebedbf2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:46.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.089+0000 7fdce9000700 1 --2- 172.21.15.19:0/2952830393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ebedbf4c30 0x55ebedbf2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:46.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.089+0000 7fdce9000700 1 -- 172.21.15.19:0/2952830393 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebedad2420 con 0x55ebedbeb590 2024-05-07T02:40:46.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.089+0000 7fdcea400700 1 --2- 172.21.15.19:0/2952830393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ebedbf4c30 0x55ebedbf2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:40:46.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.089+0000 7fdce9000700 1 --2- 172.21.15.19:0/2952830393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ebedbeb590 0x55ebedbf2130 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fdce406c3c0 tx=0x7fdcd0010b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ff07d87cbc24a52 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:46.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.089+0000 7fdccb400700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:40:46.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.090+0000 7fdccb400700 1 -- 172.21.15.19:0/2952830393 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0021060 con 0x55ebedbeb590 2024-05-07T02:40:46.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.090+0000 7fdcfaef0b80 1 -- 172.21.15.19:0/2952830393 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcdc0b2030 con 0x55ebedbeb590 2024-05-07T02:40:46.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.090+0000 7fdccb400700 1 -- 172.21.15.19:0/2952830393 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9525+0+0 (secure 0 0 0) 0x7fdcd0003680 con 0x55ebedbeb590 2024-05-07T02:40:46.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.090+0000 7fdccb400700 1 -- 172.21.15.19:0/2952830393 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fdcd0028050 con 0x55ebedbeb590 2024-05-07T02:40:46.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.090+0000 7fdcfaef0b80 10 client.26524 fetch_fsmap learned FSMap version 50 2024-05-07T02:40:46.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.090+0000 7fdcfaef0b80 1 -- 172.21.15.19:0/2952830393 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55ebedc02670 con 0x55ebedbeb590 2024-05-07T02:40:46.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.090+0000 7fdccb400700 1 -- 172.21.15.19:0/2952830393 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcdc0b2030 con 0x55ebedbeb590 2024-05-07T02:40:46.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.091+0000 7fdccb400700 1 -- 172.21.15.19:0/2952830393 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 50) ==== 8362+0+0 (secure 0 0 0) 0x7fdcd000f9b0 con 0x55ebedbeb590 2024-05-07T02:40:46.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.091+0000 7fdcfaef0b80 10 client.26524 fetch_fsmap finished waiting for FSMap version 50 2024-05-07T02:40:46.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.091+0000 7fdcfaef0b80 10 client.26524 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:40:46.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.091+0000 7fdcfaef0b80 20 client.26524 populate_metadata read hostname 'smithi019' 2024-05-07T02:40:46.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.091+0000 7fdcfaef0b80 1 --2- 172.21.15.19:0/2952830393 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55ebedc04cb0 0x55ebedc070d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:46.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.091+0000 7fdcfaef0b80 4 client.26524 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:40:46.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.091+0000 7fdcfaef0b80 1 -- 172.21.15.19:0/2952830393 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ebedbb70e0 con 0x55ebedc04cb0 2024-05-07T02:40:46.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.091+0000 7fdce9a00700 1 --2- 172.21.15.19:0/2952830393 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55ebedc04cb0 0x55ebedc070d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:46.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.092+0000 7fdce9a00700 1 --2- 172.21.15.19:0/2952830393 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55ebedc04cb0 0x55ebedc070d0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7472eefaf985453f server_cookie=63655bf3995bdd16 in_seq=0 out_seq=0 2024-05-07T02:40:46.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.092+0000 7fdccb400700 10 client.26524 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:40:46.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.093+0000 7fdccb400700 1 -- 172.21.15.19:0/2952830393 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ebedbb70e0 con 0x55ebedc04cb0 2024-05-07T02:40:46.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.093+0000 7fdccb400700 10 client.26524 handle_command_reply: tid=0 2024-05-07T02:40:46.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.093+0000 7fdcfaef0b80 1 client.26524 shutdown 2024-05-07T02:40:46.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.093+0000 7fdcfaef0b80 1 -- 172.21.15.19:0/2952830393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ebedbeb590 msgr2=0x55ebedbf2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:46.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.093+0000 7fdcfaef0b80 1 --2- 172.21.15.19:0/2952830393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ebedbeb590 0x55ebedbf2130 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fdce406c3c0 tx=0x7fdcd0010b70 comp rx=0 tx=0).stop 2024-05-07T02:40:46.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.093+0000 7fdcfaef0b80 1 -- 172.21.15.19:0/2952830393 shutdown_connections 2024-05-07T02:40:46.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.093+0000 7fdcfaef0b80 1 --2- 172.21.15.19:0/2952830393 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55ebedc04cb0 0x55ebedc070d0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:46.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.093+0000 7fdcfaef0b80 1 --2- 172.21.15.19:0/2952830393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ebedbf4c30 0x55ebedbf2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:46.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.093+0000 7fdcfaef0b80 1 --2- 172.21.15.19:0/2952830393 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebedbeac40 0x55ebedbf1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:46.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.093+0000 7fdcfaef0b80 1 --2- 172.21.15.19:0/2952830393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ebedbeb590 0x55ebedbf2130 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:46.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.093+0000 7fdcfaef0b80 1 -- 172.21.15.19:0/2952830393 >> 172.21.15.19:0/2952830393 conn(0x55ebeda6ce40 msgr2=0x55ebedbe7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:46.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.093+0000 7fdccb400700 0 client.26524 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:40:46.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.093+0000 7fdcfaef0b80 1 -- 172.21.15.19:0/2952830393 shutdown_connections 2024-05-07T02:40:46.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.093+0000 7fdcfaef0b80 1 -- 172.21.15.19:0/2952830393 wait complete. 2024-05-07T02:40:46.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.094+0000 7fdcfaef0b80 20 client.26524 trim_cache size 0 max 16384 2024-05-07T02:40:46.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.110+0000 7fdcfaef0b80 1 Processor -- start 2024-05-07T02:40:46.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.111+0000 7fdcfaef0b80 1 -- start start 2024-05-07T02:40:46.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.111+0000 7fdcfaef0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ebedbeac40 0x55ebedbe5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:46.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.111+0000 7fdcfaef0b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ebedbeb590 0x55ebedbf24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:46.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.111+0000 7fdcfaef0b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebedbf4c30 0x55ebedc00ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:46.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.111+0000 7fdcfaef0b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdce41b2740 con 0x55ebedbeac40 2024-05-07T02:40:46.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.111+0000 7fdcfaef0b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdce41b4280 con 0x55ebedbf4c30 2024-05-07T02:40:46.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.111+0000 7fdcfaef0b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdce41b2e10 con 0x55ebedbeb590 2024-05-07T02:40:46.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.111+0000 7fdce9a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ebedbeac40 0x55ebedbe5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:46.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.111+0000 7fdce9a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ebedbeac40 0x55ebedbe5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37930/0 (socket says 172.21.15.19:37930) 2024-05-07T02:40:46.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.111+0000 7fdce9a00700 1 -- 172.21.15.19:0/4067830097 learned_addr learned my addr 172.21.15.19:0/4067830097 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:40:46.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.111+0000 7fdcea400700 1 --2- 172.21.15.19:0/4067830097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebedbf4c30 0x55ebedc00ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:46.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.112+0000 7fdce9000700 1 --2- 172.21.15.19:0/4067830097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ebedbeb590 0x55ebedbf24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:46.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.112+0000 7fdce9a00700 1 -- 172.21.15.19:0/4067830097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ebedbeb590 msgr2=0x55ebedbf24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:46.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.112+0000 7fdce9a00700 1 --2- 172.21.15.19:0/4067830097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ebedbeb590 0x55ebedbf24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:46.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.112+0000 7fdce9a00700 1 -- 172.21.15.19:0/4067830097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebedbf4c30 msgr2=0x55ebedc00ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:46.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.112+0000 7fdce9a00700 1 --2- 172.21.15.19:0/4067830097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebedbf4c30 0x55ebedc00ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:46.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.112+0000 7fdce9a00700 1 -- 172.21.15.19:0/4067830097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebedbe1b20 con 0x55ebedbeac40 2024-05-07T02:40:46.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.112+0000 7fdce9a00700 1 --2- 172.21.15.19:0/4067830097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ebedbeac40 0x55ebedbe5b20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8009c70 tx=0x7fdcd801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86bef5bc2af4921a server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:46.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.113+0000 7fdcd6000700 1 -- 172.21.15.19:0/4067830097 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd8022070 con 0x55ebedbeac40 2024-05-07T02:40:46.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.113+0000 7fdcd6000700 1 -- 172.21.15.19:0/4067830097 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcd8036070 con 0x55ebedbeac40 2024-05-07T02:40:46.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.113+0000 7fdcd6000700 1 -- 172.21.15.19:0/4067830097 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd80079c0 con 0x55ebedbeac40 2024-05-07T02:40:46.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.114+0000 7fdcfaef0b80 1 -- 172.21.15.19:0/4067830097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ebedbeac40 msgr2=0x55ebedbe5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:46.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.114+0000 7fdcfaef0b80 1 --2- 172.21.15.19:0/4067830097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ebedbeac40 0x55ebedbe5b20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8009c70 tx=0x7fdcd801e000 comp rx=0 tx=0).stop 2024-05-07T02:40:46.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.114+0000 7fdcfaef0b80 1 -- 172.21.15.19:0/4067830097 shutdown_connections 2024-05-07T02:40:46.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.114+0000 7fdcfaef0b80 1 --2- 172.21.15.19:0/4067830097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebedbf4c30 0x55ebedc00ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:46.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.114+0000 7fdcfaef0b80 1 --2- 172.21.15.19:0/4067830097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ebedbeac40 0x55ebedbe5b20 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:46.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.114+0000 7fdcfaef0b80 1 --2- 172.21.15.19:0/4067830097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ebedbeb590 0x55ebedbf24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:46.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.114+0000 7fdcfaef0b80 1 -- 172.21.15.19:0/4067830097 >> 172.21.15.19:0/4067830097 conn(0x55ebeda6ce40 msgr2=0x55ebedbe5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:46.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.114+0000 7fdcfaef0b80 1 -- 172.21.15.19:0/4067830097 shutdown_connections 2024-05-07T02:40:46.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.114+0000 7fdcfaef0b80 1 -- 172.21.15.19:0/4067830097 wait complete. 2024-05-07T02:40:46.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.114+0000 7fdcfaef0b80 1 -- start start 2024-05-07T02:40:46.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.115+0000 7fdcfaef0b80 1 Processor -- start 2024-05-07T02:40:46.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.115+0000 7fdcfaef0b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebedbeac40 0x55ebedbf7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:46.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.115+0000 7fdcfaef0b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ebedbeb590 0x55ebedbf8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:46.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.115+0000 7fdcfaef0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ebedbf4c30 0x55ebedbf8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:46.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.115+0000 7fdcfaef0b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdce41b3bb0 con 0x55ebedbf4c30 2024-05-07T02:40:46.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.115+0000 7fdcfaef0b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fdce41b34e0 con 0x55ebedbeac40 2024-05-07T02:40:46.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.115+0000 7fdcfaef0b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdce41b0700 con 0x55ebedbeb590 2024-05-07T02:40:46.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.115+0000 7fdcfaef0b80 10 client.0 _ll_register_callbacks cb 0x55ebedaa0380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:40:46.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.115+0000 7fdcea400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ebedbf4c30 0x55ebedbf8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:46.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.116+0000 7fdce9a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebedbeac40 0x55ebedbf7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:46.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.116+0000 7fdcea400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ebedbf4c30 0x55ebedbf8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37936/0 (socket says 172.21.15.19:37936) 2024-05-07T02:40:46.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.116+0000 7fdcea400700 1 -- 172.21.15.19:0/3256638675 learned_addr learned my addr 172.21.15.19:0/3256638675 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:40:46.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.116+0000 7fdce9000700 1 --2- 172.21.15.19:0/3256638675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ebedbeb590 0x55ebedbf8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:46.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.116+0000 7fdce9a00700 1 -- 172.21.15.19:0/3256638675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ebedbeb590 msgr2=0x55ebedbf8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:46.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.116+0000 7fdce9a00700 1 --2- 172.21.15.19:0/3256638675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ebedbeb590 0x55ebedbf8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:46.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.116+0000 7fdce9a00700 1 -- 172.21.15.19:0/3256638675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ebedbf4c30 msgr2=0x55ebedbf8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:46.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.116+0000 7fdce9a00700 1 --2- 172.21.15.19:0/3256638675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ebedbf4c30 0x55ebedbf8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:46.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.116+0000 7fdce9a00700 1 -- 172.21.15.19:0/3256638675 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebedc6d3f0 con 0x55ebedbeac40 2024-05-07T02:40:46.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.116+0000 7fdcea400700 1 --2- 172.21.15.19:0/3256638675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ebedbf4c30 0x55ebedbf8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:40:46.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.116+0000 7fdce9a00700 1 --2- 172.21.15.19:0/3256638675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebedbeac40 0x55ebedbf7e00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fdcd800b010 tx=0x7fdcd800baa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4730d9c093b8961 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:46.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.116+0000 7fdccb400700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:40:46.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.117+0000 7fdccb400700 1 -- 172.21.15.19:0/3256638675 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd8040030 con 0x55ebedbeac40 2024-05-07T02:40:46.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.117+0000 7fdccb400700 1 -- 172.21.15.19:0/3256638675 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9525+0+0 (secure 0 0 0) 0x7fdcd8030040 con 0x55ebedbeac40 2024-05-07T02:40:46.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.117+0000 7fdcfaef0b80 1 -- 172.21.15.19:0/3256638675 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcd8036070 con 0x55ebedbeac40 2024-05-07T02:40:46.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.117+0000 7fdccb400700 1 -- 172.21.15.19:0/3256638675 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fdcd8031910 con 0x55ebedbeac40 2024-05-07T02:40:46.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.117+0000 7fdcfaef0b80 10 client.26570 fetch_fsmap learned FSMap version 50 2024-05-07T02:40:46.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.117+0000 7fdcfaef0b80 1 -- 172.21.15.19:0/3256638675 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55ebedc6e5d0 con 0x55ebedbeac40 2024-05-07T02:40:46.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.117+0000 7fdccb400700 1 -- 172.21.15.19:0/3256638675 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcd8036070 con 0x55ebedbeac40 2024-05-07T02:40:46.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.118+0000 7fdccb400700 1 -- 172.21.15.19:0/3256638675 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 50) ==== 8362+0+0 (secure 0 0 0) 0x7fdcd8031ce0 con 0x55ebedbeac40 2024-05-07T02:40:46.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.118+0000 7fdcfaef0b80 10 client.26570 fetch_fsmap finished waiting for FSMap version 50 2024-05-07T02:40:46.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.118+0000 7fdcfaef0b80 10 client.26570 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:40:46.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.118+0000 7fdcfaef0b80 20 client.26570 populate_metadata read hostname 'smithi019' 2024-05-07T02:40:46.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.118+0000 7fdcfaef0b80 1 --2- 172.21.15.19:0/3256638675 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55ebedc6ebc0 0x55ebedc70fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:46.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.118+0000 7fdcfaef0b80 4 client.26570 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:40:46.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.118+0000 7fdcfaef0b80 1 -- 172.21.15.19:0/3256638675 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ebedb1a400 con 0x55ebedc6ebc0 2024-05-07T02:40:46.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.118+0000 7fdce9000700 1 --2- 172.21.15.19:0/3256638675 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55ebedc6ebc0 0x55ebedc70fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:46.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.119+0000 7fdce9000700 1 --2- 172.21.15.19:0/3256638675 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55ebedc6ebc0 0x55ebedc70fe0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a270b6f6a24af64 server_cookie=62a8e19e7046bf1d in_seq=0 out_seq=0 2024-05-07T02:40:46.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.119+0000 7fdccb400700 10 client.26570 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:40:46.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.119+0000 7fdccb400700 1 -- 172.21.15.19:0/3256638675 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ebedb1a400 con 0x55ebedc6ebc0 2024-05-07T02:40:46.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.119+0000 7fdccb400700 10 client.26570 handle_command_reply: tid=0 2024-05-07T02:40:46.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.119+0000 7fdcfaef0b80 1 client.26570 shutdown 2024-05-07T02:40:46.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.119+0000 7fdcfaef0b80 1 -- 172.21.15.19:0/3256638675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebedbeac40 msgr2=0x55ebedbf7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:46.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.119+0000 7fdcfaef0b80 1 --2- 172.21.15.19:0/3256638675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebedbeac40 0x55ebedbf7e00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fdcd800b010 tx=0x7fdcd800baa0 comp rx=0 tx=0).stop 2024-05-07T02:40:46.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.119+0000 7fdcfaef0b80 1 -- 172.21.15.19:0/3256638675 shutdown_connections 2024-05-07T02:40:46.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.119+0000 7fdcfaef0b80 1 --2- 172.21.15.19:0/3256638675 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55ebedc6ebc0 0x55ebedc70fe0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:46.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.119+0000 7fdcfaef0b80 1 --2- 172.21.15.19:0/3256638675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ebedbf4c30 0x55ebedbf8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:46.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.119+0000 7fdcfaef0b80 1 --2- 172.21.15.19:0/3256638675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55ebedbeac40 0x55ebedbf7e00 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:46.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.119+0000 7fdcfaef0b80 1 --2- 172.21.15.19:0/3256638675 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ebedbeb590 0x55ebedbf8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:46.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.119+0000 7fdccb400700 0 client.26570 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:40:46.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.119+0000 7fdcfaef0b80 1 -- 172.21.15.19:0/3256638675 >> 172.21.15.19:0/3256638675 conn(0x55ebeda6ce40 msgr2=0x55ebedbe9610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:46.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.120+0000 7fdcfaef0b80 1 -- 172.21.15.19:0/3256638675 shutdown_connections 2024-05-07T02:40:46.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.120+0000 7fdcfaef0b80 1 -- 172.21.15.19:0/3256638675 wait complete. 2024-05-07T02:40:46.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.120+0000 7fdcfaef0b80 20 client.26570 trim_cache size 0 max 16384 2024-05-07T02:40:46.126 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:40:46.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.120+0000 7fdceae00700 1 -- 172.21.15.19:0/2224611505 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fdccc0b29f0 msgr2=0x7fdccc0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:46.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.120+0000 7fdceae00700 1 --2- 172.21.15.19:0/2224611505 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fdccc0b29f0 0x7fdccc0b4f00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8009900 tx=0x7fdcd800b040 comp rx=0 tx=0).stop 2024-05-07T02:40:46.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.120+0000 7fdceae00700 1 -- 172.21.15.19:0/2224611505 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdce4109680 msgr2=0x7fdce41a1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:46.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.120+0000 7fdceae00700 1 --2- 172.21.15.19:0/2224611505 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdce4109680 0x7fdce41a1690 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc00e900 tx=0x7fdcdc00ec10 comp rx=0 tx=0).stop 2024-05-07T02:40:46.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.121+0000 7fdceae00700 1 -- 172.21.15.19:0/2224611505 shutdown_connections 2024-05-07T02:40:46.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.121+0000 7fdceae00700 1 --2- 172.21.15.19:0/2224611505 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdce4109680 0x7fdce41a1690 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:46.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.121+0000 7fdceae00700 1 --2- 172.21.15.19:0/2224611505 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fdccc0b29f0 0x7fdccc0b4f00 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:46.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.121+0000 7fdceae00700 1 --2- 172.21.15.19:0/2224611505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdce4104400 0x7fdce419ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:46.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.121+0000 7fdceae00700 1 --2- 172.21.15.19:0/2224611505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdce4108d30 0x7fdce419d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:46.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.121+0000 7fdceae00700 1 -- 172.21.15.19:0/2224611505 >> 172.21.15.19:0/2224611505 conn(0x7fdce4079d10 msgr2=0x7fdce4101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:46.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.121+0000 7fdceae00700 1 -- 172.21.15.19:0/2224611505 shutdown_connections 2024-05-07T02:40:46.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:46.121+0000 7fdceae00700 1 -- 172.21.15.19:0/2224611505 wait complete. 2024-05-07T02:40:47.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:47 smithi064 ceph-mon[34699]: pgmap v275: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:40:47.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:47 smithi019 ceph-mon[29781]: pgmap v275: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:40:47.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:47 smithi027 ceph-mon[34398]: pgmap v275: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:40:48.265 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-07T02:40:48.265 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-07T02:40:48.729 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:40:49.823 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:49 smithi019 ceph-mon[29781]: pgmap v276: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:40:49.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:49 smithi064 ceph-mon[34699]: pgmap v276: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:40:49.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:49 smithi027 ceph-mon[34398]: pgmap v276: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:40:51.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.267+0000 7fc143200700 1 -- 172.21.15.19:0/1733819279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc13c075d30 msgr2=0x7fc13c076140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:51.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.267+0000 7fc143200700 1 --2- 172.21.15.19:0/1733819279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc13c075d30 0x7fc13c076140 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc12c009990 tx=0x7fc12c009ca0 comp rx=0 tx=0).stop 2024-05-07T02:40:51.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.267+0000 7fc143200700 1 -- 172.21.15.19:0/1733819279 shutdown_connections 2024-05-07T02:40:51.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.267+0000 7fc143200700 1 --2- 172.21.15.19:0/1733819279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc13c110ba0 0x7fc13c113000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:51.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.267+0000 7fc143200700 1 --2- 172.21.15.19:0/1733819279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc13c075d30 0x7fc13c076140 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:51.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.267+0000 7fc143200700 1 --2- 172.21.15.19:0/1733819279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc13c076680 0x7fc13c110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:51.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.267+0000 7fc143200700 1 -- 172.21.15.19:0/1733819279 >> 172.21.15.19:0/1733819279 conn(0x7fc13c100040 msgr2=0x7fc13c102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:51.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.268+0000 7fc143200700 1 -- 172.21.15.19:0/1733819279 shutdown_connections 2024-05-07T02:40:51.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.268+0000 7fc143200700 1 -- 172.21.15.19:0/1733819279 wait complete. 2024-05-07T02:40:51.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.269+0000 7fc143200700 1 Processor -- start 2024-05-07T02:40:51.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.269+0000 7fc143200700 1 -- start start 2024-05-07T02:40:51.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.270+0000 7fc143200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc13c075d30 0x7fc13c19d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:51.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.270+0000 7fc143200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc13c076680 0x7fc13c19d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:51.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.270+0000 7fc143200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc13c110ba0 0x7fc13c1a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:51.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.270+0000 7fc143200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc13c116690 con 0x7fc13c076680 2024-05-07T02:40:51.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.270+0000 7fc143200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc13c116510 con 0x7fc13c075d30 2024-05-07T02:40:51.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.270+0000 7fc143200700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc13c116810 con 0x7fc13c110ba0 2024-05-07T02:40:51.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.270+0000 7fc133e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc13c076680 0x7fc13c19d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:51.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.270+0000 7fc13be00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc13c075d30 0x7fc13c19d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:51.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.270+0000 7fc133e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc13c076680 0x7fc13c19d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50266/0 (socket says 172.21.15.19:50266) 2024-05-07T02:40:51.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.270+0000 7fc133e00700 1 -- 172.21.15.19:0/3013184296 learned_addr learned my addr 172.21.15.19:0/3013184296 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:40:51.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.270+0000 7fc13be00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc13c075d30 0x7fc13c19d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:33326/0 (socket says 172.21.15.19:33326) 2024-05-07T02:40:51.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.270+0000 7fc141200700 1 --2- 172.21.15.19:0/3013184296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc13c110ba0 0x7fc13c1a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:51.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.271+0000 7fc141200700 1 -- 172.21.15.19:0/3013184296 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc13c075d30 msgr2=0x7fc13c19d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:51.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.271+0000 7fc141200700 1 --2- 172.21.15.19:0/3013184296 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc13c075d30 0x7fc13c19d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:51.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.271+0000 7fc141200700 1 -- 172.21.15.19:0/3013184296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc13c076680 msgr2=0x7fc13c19d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:51.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.271+0000 7fc141200700 1 --2- 172.21.15.19:0/3013184296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc13c076680 0x7fc13c19d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:51.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.271+0000 7fc141200700 1 -- 172.21.15.19:0/3013184296 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc13c1a1ef0 con 0x7fc13c110ba0 2024-05-07T02:40:51.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.271+0000 7fc141200700 1 --2- 172.21.15.19:0/3013184296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc13c110ba0 0x7fc13c1a1850 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc13400ef90 tx=0x7fc13400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:51.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.271+0000 7fc139600700 1 -- 172.21.15.19:0/3013184296 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1340092e0 con 0x7fc13c110ba0 2024-05-07T02:40:51.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.271+0000 7fc139600700 1 -- 172.21.15.19:0/3013184296 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc134004bb0 con 0x7fc13c110ba0 2024-05-07T02:40:51.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.271+0000 7fc143200700 1 -- 172.21.15.19:0/3013184296 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc13c1a2180 con 0x7fc13c110ba0 2024-05-07T02:40:51.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.272+0000 7fc139600700 1 -- 172.21.15.19:0/3013184296 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc134004020 con 0x7fc13c110ba0 2024-05-07T02:40:51.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.272+0000 7fc143200700 1 -- 172.21.15.19:0/3013184296 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc13c1a9b00 con 0x7fc13c110ba0 2024-05-07T02:40:51.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.276+0000 7fc139600700 1 -- 172.21.15.19:0/3013184296 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc13401b030 con 0x7fc13c110ba0 2024-05-07T02:40:51.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.277+0000 7fc151e2db80 1 Processor -- start 2024-05-07T02:40:51.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.277+0000 7fc151e2db80 1 -- start start 2024-05-07T02:40:51.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.277+0000 7fc139600700 1 --2- 172.21.15.19:0/3013184296 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc1200b2a70 0x7fc1200b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:51.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.277+0000 7fc151e2db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55931bd224f0 0x55931bd22900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:51.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.277+0000 7fc151e2db80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55931bd1c6a0 0x55931bd1cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:51.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.278+0000 7fc151e2db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55931bd1cff0 0x55931bd291d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:51.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.278+0000 7fc151e2db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55931bc3eaa0 con 0x55931bd224f0 2024-05-07T02:40:51.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.278+0000 7fc151e2db80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55931bbdc7e0 con 0x55931bd1c6a0 2024-05-07T02:40:51.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.278+0000 7fc151e2db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55931bc3aa60 con 0x55931bd1cff0 2024-05-07T02:40:51.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.278+0000 7fc139600700 1 -- 172.21.15.19:0/3013184296 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9525+0+0 (secure 0 0 0) 0x7fc1340e24c0 con 0x7fc13c110ba0 2024-05-07T02:40:51.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.278+0000 7fc133e00700 1 --2- 172.21.15.19:0/3013184296 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc1200b2a70 0x7fc1200b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:51.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.278+0000 7fc13be00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55931bd224f0 0x55931bd22900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:51.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.278+0000 7fc13be00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55931bd224f0 0x55931bd22900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50274/0 (socket says 172.21.15.19:50274) 2024-05-07T02:40:51.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.278+0000 7fc13be00700 1 -- 172.21.15.19:0/643764267 learned_addr learned my addr 172.21.15.19:0/643764267 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:40:51.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.278+0000 7fc141200700 1 --2- 172.21.15.19:0/643764267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55931bd1cff0 0x55931bd291d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:51.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.278+0000 7fc133e00700 1 --2- 172.21.15.19:0/643764267 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55931bd1c6a0 0x55931bd1cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:51.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.279+0000 7fc13be00700 1 -- 172.21.15.19:0/643764267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55931bd1cff0 msgr2=0x55931bd291d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:51.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.279+0000 7fc13be00700 1 --2- 172.21.15.19:0/643764267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55931bd1cff0 0x55931bd291d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:51.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.279+0000 7fc13be00700 1 -- 172.21.15.19:0/643764267 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55931bd1c6a0 msgr2=0x55931bd1cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:51.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.279+0000 7fc13be00700 1 --2- 172.21.15.19:0/643764267 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55931bd1c6a0 0x55931bd1cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:51.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.279+0000 7fc13be00700 1 -- 172.21.15.19:0/643764267 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55931bb8b950 con 0x55931bd224f0 2024-05-07T02:40:51.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.279+0000 7fc141200700 1 --2- 172.21.15.19:0/643764267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55931bd1cff0 0x55931bd291d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:40:51.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.279+0000 7fc13be00700 1 --2- 172.21.15.19:0/643764267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55931bd224f0 0x55931bd22900 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc12c009990 tx=0x7fc12c0096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0a20b8f9ac51c10 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:51.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.279+0000 7fc133e00700 1 --2- 172.21.15.19:0/3013184296 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc1200b2a70 0x7fc1200b4f80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x55931bd18ed0 tx=0x7fc124009990 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:51.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.279+0000 7fc131600700 1 -- 172.21.15.19:0/643764267 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12c027070 con 0x55931bd224f0 2024-05-07T02:40:51.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.280+0000 7fc131600700 1 -- 172.21.15.19:0/643764267 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc12c0056b0 con 0x55931bd224f0 2024-05-07T02:40:51.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.280+0000 7fc131600700 1 -- 172.21.15.19:0/643764267 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12c003ce0 con 0x55931bd224f0 2024-05-07T02:40:51.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.280+0000 7fc151e2db80 1 -- 172.21.15.19:0/643764267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55931bd224f0 msgr2=0x55931bd22900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:51.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.280+0000 7fc151e2db80 1 --2- 172.21.15.19:0/643764267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55931bd224f0 0x55931bd22900 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc12c009990 tx=0x7fc12c0096a0 comp rx=0 tx=0).stop 2024-05-07T02:40:51.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.281+0000 7fc151e2db80 1 -- 172.21.15.19:0/643764267 shutdown_connections 2024-05-07T02:40:51.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.281+0000 7fc151e2db80 1 --2- 172.21.15.19:0/643764267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55931bd1cff0 0x55931bd291d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:51.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.281+0000 7fc151e2db80 1 --2- 172.21.15.19:0/643764267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55931bd224f0 0x55931bd22900 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:51.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.281+0000 7fc151e2db80 1 --2- 172.21.15.19:0/643764267 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55931bd1c6a0 0x55931bd1cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:51.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.281+0000 7fc151e2db80 1 -- 172.21.15.19:0/643764267 >> 172.21.15.19:0/643764267 conn(0x55931bb9eec0 msgr2=0x55931bd17400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:51.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.281+0000 7fc151e2db80 1 -- 172.21.15.19:0/643764267 shutdown_connections 2024-05-07T02:40:51.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.281+0000 7fc151e2db80 1 -- 172.21.15.19:0/643764267 wait complete. 2024-05-07T02:40:51.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.282+0000 7fc151e2db80 1 -- start start 2024-05-07T02:40:51.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.282+0000 7fc151e2db80 1 Processor -- start 2024-05-07T02:40:51.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.282+0000 7fc151e2db80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55931bd1c6a0 0x55931bd26750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:51.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.282+0000 7fc151e2db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55931bd1cff0 0x55931bd26c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:51.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.282+0000 7fc151e2db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55931bd224f0 0x55931bd271d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:51.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.282+0000 7fc151e2db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55931bd29860 con 0x55931bd224f0 2024-05-07T02:40:51.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.282+0000 7fc151e2db80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55931bba4ba0 con 0x55931bd1c6a0 2024-05-07T02:40:51.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.282+0000 7fc151e2db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55931bd299e0 con 0x55931bd1cff0 2024-05-07T02:40:51.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.283+0000 7fc151e2db80 10 client.0 _ll_register_callbacks cb 0x55931bbd2400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:40:51.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.283+0000 7fc141200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55931bd224f0 0x55931bd271d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:51.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.283+0000 7fc141200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55931bd224f0 0x55931bd271d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50286/0 (socket says 172.21.15.19:50286) 2024-05-07T02:40:51.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.283+0000 7fc141200700 1 -- 172.21.15.19:0/4022180515 learned_addr learned my addr 172.21.15.19:0/4022180515 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:40:51.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.283+0000 7fc13be00700 1 --2- 172.21.15.19:0/4022180515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55931bd1c6a0 0x55931bd26750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:51.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.283+0000 7fc133e00700 1 --2- 172.21.15.19:0/4022180515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55931bd1cff0 0x55931bd26c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:51.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.283+0000 7fc13be00700 1 -- 172.21.15.19:0/4022180515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55931bd1cff0 msgr2=0x55931bd26c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:51.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.283+0000 7fc13be00700 1 --2- 172.21.15.19:0/4022180515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55931bd1cff0 0x55931bd26c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:51.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.283+0000 7fc13be00700 1 -- 172.21.15.19:0/4022180515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55931bd224f0 msgr2=0x55931bd271d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:51.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.283+0000 7fc13be00700 1 --2- 172.21.15.19:0/4022180515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55931bd224f0 0x55931bd271d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:51.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.284+0000 7fc13be00700 1 -- 172.21.15.19:0/4022180515 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55931bc044a0 con 0x55931bd1c6a0 2024-05-07T02:40:51.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.284+0000 7fc13be00700 1 --2- 172.21.15.19:0/4022180515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55931bd1c6a0 0x55931bd26750 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc12c0098e0 tx=0x7fc12c009710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4cbc68b49843197 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:51.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.284+0000 7fc12a000700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:40:51.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.284+0000 7fc12a000700 1 -- 172.21.15.19:0/4022180515 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12c032060 con 0x55931bd1c6a0 2024-05-07T02:40:51.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.284+0000 7fc151e2db80 1 -- 172.21.15.19:0/4022180515 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc12c0056b0 con 0x55931bd1c6a0 2024-05-07T02:40:51.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.284+0000 7fc12a000700 1 -- 172.21.15.19:0/4022180515 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9525+0+0 (secure 0 0 0) 0x7fc12c003c90 con 0x55931bd1c6a0 2024-05-07T02:40:51.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.285+0000 7fc12a000700 1 -- 172.21.15.19:0/4022180515 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fc12c039050 con 0x55931bd1c6a0 2024-05-07T02:40:51.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.285+0000 7fc151e2db80 10 client.26582 fetch_fsmap learned FSMap version 50 2024-05-07T02:40:51.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.285+0000 7fc151e2db80 1 -- 172.21.15.19:0/4022180515 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55931bd34500 con 0x55931bd1c6a0 2024-05-07T02:40:51.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.285+0000 7fc12a000700 1 -- 172.21.15.19:0/4022180515 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc12c0056b0 con 0x55931bd1c6a0 2024-05-07T02:40:51.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.285+0000 7fc12a000700 1 -- 172.21.15.19:0/4022180515 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 50) ==== 8362+0+0 (secure 0 0 0) 0x7fc12c00f800 con 0x55931bd1c6a0 2024-05-07T02:40:51.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.285+0000 7fc151e2db80 10 client.26582 fetch_fsmap finished waiting for FSMap version 50 2024-05-07T02:40:51.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.285+0000 7fc151e2db80 10 client.26582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:40:51.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.285+0000 7fc151e2db80 20 client.26582 populate_metadata read hostname 'smithi019' 2024-05-07T02:40:51.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.286+0000 7fc151e2db80 1 --2- 172.21.15.19:0/4022180515 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55931bd369d0 0x55931bd38df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:51.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.286+0000 7fc151e2db80 4 client.26582 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:40:51.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.286+0000 7fc151e2db80 1 -- 172.21.15.19:0/4022180515 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55931bce9160 con 0x55931bd369d0 2024-05-07T02:40:51.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.286+0000 7fc133e00700 1 --2- 172.21.15.19:0/4022180515 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55931bd369d0 0x55931bd38df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:51.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.287+0000 7fc133e00700 1 --2- 172.21.15.19:0/4022180515 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55931bd369d0 0x55931bd38df0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b6e282b0c676448 server_cookie=5d06d2ef61eaedc0 in_seq=0 out_seq=0 2024-05-07T02:40:51.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.287+0000 7fc12a000700 10 client.26582 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:40:51.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.287+0000 7fc12a000700 1 -- 172.21.15.19:0/4022180515 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55931bce9160 con 0x55931bd369d0 2024-05-07T02:40:51.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.287+0000 7fc12a000700 10 client.26582 handle_command_reply: tid=0 2024-05-07T02:40:51.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.288+0000 7fc151e2db80 1 client.26582 shutdown 2024-05-07T02:40:51.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.288+0000 7fc151e2db80 1 -- 172.21.15.19:0/4022180515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55931bd1c6a0 msgr2=0x55931bd26750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:51.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.288+0000 7fc151e2db80 1 --2- 172.21.15.19:0/4022180515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55931bd1c6a0 0x55931bd26750 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc12c0098e0 tx=0x7fc12c009710 comp rx=0 tx=0).stop 2024-05-07T02:40:51.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.288+0000 7fc151e2db80 1 -- 172.21.15.19:0/4022180515 shutdown_connections 2024-05-07T02:40:51.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.288+0000 7fc151e2db80 1 --2- 172.21.15.19:0/4022180515 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55931bd369d0 0x55931bd38df0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:51.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.288+0000 7fc151e2db80 1 --2- 172.21.15.19:0/4022180515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55931bd224f0 0x55931bd271d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:51.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.288+0000 7fc151e2db80 1 --2- 172.21.15.19:0/4022180515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55931bd1c6a0 0x55931bd26750 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:51.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.288+0000 7fc151e2db80 1 --2- 172.21.15.19:0/4022180515 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55931bd1cff0 0x55931bd26c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:51.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.288+0000 7fc151e2db80 1 -- 172.21.15.19:0/4022180515 >> 172.21.15.19:0/4022180515 conn(0x55931bb9eec0 msgr2=0x55931bd1b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:51.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.288+0000 7fc12a000700 0 client.26582 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:40:51.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.288+0000 7fc151e2db80 1 -- 172.21.15.19:0/4022180515 shutdown_connections 2024-05-07T02:40:51.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.288+0000 7fc151e2db80 1 -- 172.21.15.19:0/4022180515 wait complete. 2024-05-07T02:40:51.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.288+0000 7fc151e2db80 20 client.26582 trim_cache size 0 max 16384 2024-05-07T02:40:51.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.311+0000 7fc151e2db80 1 Processor -- start 2024-05-07T02:40:51.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.311+0000 7fc151e2db80 1 -- start start 2024-05-07T02:40:51.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.312+0000 7fc151e2db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55931bd1c6a0 0x55931bd1c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:51.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.312+0000 7fc151e2db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55931bd1cff0 0x55931bd27010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:51.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.312+0000 7fc151e2db80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55931bd224f0 0x55931bd20e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:51.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.312+0000 7fc151e2db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc13c1b2760 con 0x55931bd1cff0 2024-05-07T02:40:51.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.312+0000 7fc151e2db80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc13c1b42a0 con 0x55931bd224f0 2024-05-07T02:40:51.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.312+0000 7fc151e2db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc13c1b2e30 con 0x55931bd1c6a0 2024-05-07T02:40:51.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.312+0000 7fc133e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55931bd1cff0 0x55931bd27010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:51.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.312+0000 7fc133e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55931bd1cff0 0x55931bd27010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50302/0 (socket says 172.21.15.19:50302) 2024-05-07T02:40:51.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.312+0000 7fc133e00700 1 -- 172.21.15.19:0/1399088041 learned_addr learned my addr 172.21.15.19:0/1399088041 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:40:51.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.312+0000 7fc141200700 1 --2- 172.21.15.19:0/1399088041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55931bd224f0 0x55931bd20e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:51.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.312+0000 7fc13be00700 1 --2- 172.21.15.19:0/1399088041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55931bd1c6a0 0x55931bd1c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:51.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.313+0000 7fc133e00700 1 -- 172.21.15.19:0/1399088041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55931bd1c6a0 msgr2=0x55931bd1c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:51.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.313+0000 7fc133e00700 1 --2- 172.21.15.19:0/1399088041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55931bd1c6a0 0x55931bd1c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:51.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.313+0000 7fc133e00700 1 -- 172.21.15.19:0/1399088041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55931bd224f0 msgr2=0x55931bd20e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:51.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.313+0000 7fc133e00700 1 --2- 172.21.15.19:0/1399088041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55931bd224f0 0x55931bd20e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:51.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.313+0000 7fc133e00700 1 -- 172.21.15.19:0/1399088041 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55931bd18ab0 con 0x55931bd1cff0 2024-05-07T02:40:51.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.313+0000 7fc141200700 1 --2- 172.21.15.19:0/1399088041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55931bd224f0 0x55931bd20e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:40:51.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.313+0000 7fc133e00700 1 --2- 172.21.15.19:0/1399088041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55931bd1cff0 0x55931bd27010 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc124009880 tx=0x7fc1240047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a433d6557dac6ad6 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:51.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.313+0000 7fc130c00700 1 -- 172.21.15.19:0/1399088041 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc124013070 con 0x55931bd1cff0 2024-05-07T02:40:51.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.313+0000 7fc130c00700 1 -- 172.21.15.19:0/1399088041 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc124004ce0 con 0x55931bd1cff0 2024-05-07T02:40:51.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.313+0000 7fc130c00700 1 -- 172.21.15.19:0/1399088041 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12402a350 con 0x55931bd1cff0 2024-05-07T02:40:51.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.314+0000 7fc151e2db80 1 -- 172.21.15.19:0/1399088041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55931bd1cff0 msgr2=0x55931bd27010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:51.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.314+0000 7fc151e2db80 1 --2- 172.21.15.19:0/1399088041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55931bd1cff0 0x55931bd27010 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc124009880 tx=0x7fc1240047c0 comp rx=0 tx=0).stop 2024-05-07T02:40:51.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.315+0000 7fc151e2db80 1 -- 172.21.15.19:0/1399088041 shutdown_connections 2024-05-07T02:40:51.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.315+0000 7fc151e2db80 1 --2- 172.21.15.19:0/1399088041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55931bd224f0 0x55931bd20e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:51.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.315+0000 7fc151e2db80 1 --2- 172.21.15.19:0/1399088041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55931bd1c6a0 0x55931bd1c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:51.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.315+0000 7fc151e2db80 1 --2- 172.21.15.19:0/1399088041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55931bd1cff0 0x55931bd27010 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:51.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.315+0000 7fc151e2db80 1 -- 172.21.15.19:0/1399088041 >> 172.21.15.19:0/1399088041 conn(0x55931bb9eec0 msgr2=0x55931bd256b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:51.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.315+0000 7fc151e2db80 1 -- 172.21.15.19:0/1399088041 shutdown_connections 2024-05-07T02:40:51.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.315+0000 7fc151e2db80 1 -- 172.21.15.19:0/1399088041 wait complete. 2024-05-07T02:40:51.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.315+0000 7fc151e2db80 1 -- start start 2024-05-07T02:40:51.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.315+0000 7fc151e2db80 1 Processor -- start 2024-05-07T02:40:51.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.316+0000 7fc151e2db80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55931bd1c6a0 0x55931bd3a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:51.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.316+0000 7fc151e2db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55931bd1cff0 0x55931bd3a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:51.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.316+0000 7fc151e2db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55931bd224f0 0x55931bd9ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:51.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.316+0000 7fc151e2db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc13c1b3bd0 con 0x55931bd1cff0 2024-05-07T02:40:51.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.316+0000 7fc151e2db80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc13c1b3500 con 0x55931bd1c6a0 2024-05-07T02:40:51.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.316+0000 7fc151e2db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc13c1b0720 con 0x55931bd224f0 2024-05-07T02:40:51.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.316+0000 7fc151e2db80 10 client.0 _ll_register_callbacks cb 0x55931bbd2400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:40:51.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.316+0000 7fc13be00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55931bd1c6a0 0x55931bd3a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:51.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.316+0000 7fc13be00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55931bd1c6a0 0x55931bd3a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:33360/0 (socket says 172.21.15.19:33360) 2024-05-07T02:40:51.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.316+0000 7fc13be00700 1 -- 172.21.15.19:0/3050604351 learned_addr learned my addr 172.21.15.19:0/3050604351 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:40:51.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.316+0000 7fc133e00700 1 --2- 172.21.15.19:0/3050604351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55931bd1cff0 0x55931bd3a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:51.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.317+0000 7fc141200700 1 --2- 172.21.15.19:0/3050604351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55931bd224f0 0x55931bd9ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:51.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.317+0000 7fc13be00700 1 -- 172.21.15.19:0/3050604351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55931bd224f0 msgr2=0x55931bd9ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:51.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.317+0000 7fc13be00700 1 --2- 172.21.15.19:0/3050604351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55931bd224f0 0x55931bd9ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:51.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.317+0000 7fc13be00700 1 -- 172.21.15.19:0/3050604351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55931bd1cff0 msgr2=0x55931bd3a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:51.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.317+0000 7fc13be00700 1 --2- 172.21.15.19:0/3050604351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55931bd1cff0 0x55931bd3a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:51.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.317+0000 7fc13be00700 1 -- 172.21.15.19:0/3050604351 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55931bda0460 con 0x55931bd1c6a0 2024-05-07T02:40:51.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.317+0000 7fc13be00700 1 --2- 172.21.15.19:0/3050604351 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55931bd1c6a0 0x55931bd3a120 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc12c00b2e0 tx=0x7fc12c00b420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a4526959abd5cf7 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:51.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.317+0000 7fc12a000700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:40:51.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.317+0000 7fc12a000700 1 -- 172.21.15.19:0/3050604351 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12c027040 con 0x55931bd1c6a0 2024-05-07T02:40:51.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.317+0000 7fc12a000700 1 -- 172.21.15.19:0/3050604351 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9525+0+0 (secure 0 0 0) 0x7fc12c005330 con 0x55931bd1c6a0 2024-05-07T02:40:51.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.317+0000 7fc151e2db80 1 -- 172.21.15.19:0/3050604351 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc124004ce0 con 0x55931bd1c6a0 2024-05-07T02:40:51.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.318+0000 7fc12a000700 1 -- 172.21.15.19:0/3050604351 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fc12c005530 con 0x55931bd1c6a0 2024-05-07T02:40:51.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.318+0000 7fc151e2db80 10 client.26588 fetch_fsmap learned FSMap version 50 2024-05-07T02:40:51.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.318+0000 7fc151e2db80 1 -- 172.21.15.19:0/3050604351 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55931bda1680 con 0x55931bd1c6a0 2024-05-07T02:40:51.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.318+0000 7fc12a000700 1 -- 172.21.15.19:0/3050604351 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc124004ce0 con 0x55931bd1c6a0 2024-05-07T02:40:51.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.318+0000 7fc12a000700 1 -- 172.21.15.19:0/3050604351 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 50) ==== 8362+0+0 (secure 0 0 0) 0x7fc12c00b9d0 con 0x55931bd1c6a0 2024-05-07T02:40:51.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.318+0000 7fc151e2db80 10 client.26588 fetch_fsmap finished waiting for FSMap version 50 2024-05-07T02:40:51.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.318+0000 7fc151e2db80 10 client.26588 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:40:51.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.318+0000 7fc151e2db80 20 client.26588 populate_metadata read hostname 'smithi019' 2024-05-07T02:40:51.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.318+0000 7fc151e2db80 1 --2- 172.21.15.19:0/3050604351 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55931bda1b40 0x55931bda3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:51.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.318+0000 7fc151e2db80 4 client.26588 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-07T02:40:51.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.318+0000 7fc151e2db80 1 -- 172.21.15.19:0/3050604351 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55931bc4c480 con 0x55931bda1b40 2024-05-07T02:40:51.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.319+0000 7fc133e00700 1 --2- 172.21.15.19:0/3050604351 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55931bda1b40 0x55931bda3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:51.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.319+0000 7fc133e00700 1 --2- 172.21.15.19:0/3050604351 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55931bda1b40 0x55931bda3f60 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6114cee3b5e3e416 server_cookie=51702c48f117823c in_seq=0 out_seq=0 2024-05-07T02:40:51.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.319+0000 7fc12a000700 10 client.26588 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:40:51.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.320+0000 7fc12a000700 1 -- 172.21.15.19:0/3050604351 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55931bc4c480 con 0x55931bda1b40 2024-05-07T02:40:51.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.320+0000 7fc12a000700 10 client.26588 handle_command_reply: tid=0 2024-05-07T02:40:51.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.321+0000 7fc151e2db80 1 client.26588 shutdown 2024-05-07T02:40:51.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.321+0000 7fc151e2db80 1 -- 172.21.15.19:0/3050604351 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55931bd1c6a0 msgr2=0x55931bd3a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:51.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.321+0000 7fc151e2db80 1 --2- 172.21.15.19:0/3050604351 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55931bd1c6a0 0x55931bd3a120 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc12c00b2e0 tx=0x7fc12c00b420 comp rx=0 tx=0).stop 2024-05-07T02:40:51.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.321+0000 7fc151e2db80 1 -- 172.21.15.19:0/3050604351 shutdown_connections 2024-05-07T02:40:51.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.321+0000 7fc151e2db80 1 --2- 172.21.15.19:0/3050604351 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55931bda1b40 0x55931bda3f60 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:51.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.321+0000 7fc151e2db80 1 --2- 172.21.15.19:0/3050604351 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55931bd224f0 0x55931bd9ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:51.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.321+0000 7fc151e2db80 1 --2- 172.21.15.19:0/3050604351 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55931bd1c6a0 0x55931bd3a120 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:51.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.321+0000 7fc151e2db80 1 --2- 172.21.15.19:0/3050604351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55931bd1cff0 0x55931bd3a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:51.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.321+0000 7fc151e2db80 1 -- 172.21.15.19:0/3050604351 >> 172.21.15.19:0/3050604351 conn(0x55931bb9eec0 msgr2=0x55931bd26000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:51.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.321+0000 7fc12a000700 0 client.26588 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:40:51.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.321+0000 7fc151e2db80 1 -- 172.21.15.19:0/3050604351 shutdown_connections 2024-05-07T02:40:51.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.321+0000 7fc151e2db80 1 -- 172.21.15.19:0/3050604351 wait complete. 2024-05-07T02:40:51.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.321+0000 7fc151e2db80 20 client.26588 trim_cache size 0 max 16384 2024-05-07T02:40:51.332 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:40:51.332 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-07T02:40:51.332 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "ede59abf-6c48-4091-a279-0ce4d5606e51", 2024-05-07T02:40:51.332 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-07T02:40:51.333 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:40:51.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.322+0000 7fc143200700 1 -- 172.21.15.19:0/3013184296 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc1200b2a70 msgr2=0x7fc1200b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:51.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.322+0000 7fc143200700 1 --2- 172.21.15.19:0/3013184296 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc1200b2a70 0x7fc1200b4f80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x55931bd18ed0 tx=0x7fc124009990 comp rx=0 tx=0).stop 2024-05-07T02:40:51.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.322+0000 7fc143200700 1 -- 172.21.15.19:0/3013184296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc13c110ba0 msgr2=0x7fc13c1a1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:51.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.322+0000 7fc143200700 1 --2- 172.21.15.19:0/3013184296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc13c110ba0 0x7fc13c1a1850 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc13400ef90 tx=0x7fc13400c3a0 comp rx=0 tx=0).stop 2024-05-07T02:40:51.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.322+0000 7fc143200700 1 -- 172.21.15.19:0/3013184296 shutdown_connections 2024-05-07T02:40:51.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.322+0000 7fc143200700 1 --2- 172.21.15.19:0/3013184296 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc13c110ba0 0x7fc13c1a1850 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:51.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.322+0000 7fc143200700 1 --2- 172.21.15.19:0/3013184296 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc13c075d30 0x7fc13c19d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:51.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.322+0000 7fc143200700 1 --2- 172.21.15.19:0/3013184296 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc1200b2a70 0x7fc1200b4f80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:51.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.322+0000 7fc143200700 1 --2- 172.21.15.19:0/3013184296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc13c076680 0x7fc13c19d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:51.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.322+0000 7fc143200700 1 -- 172.21.15.19:0/3013184296 >> 172.21.15.19:0/3013184296 conn(0x7fc13c100040 msgr2=0x7fc13c1006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:51.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.323+0000 7fc143200700 1 -- 172.21.15.19:0/3013184296 shutdown_connections 2024-05-07T02:40:51.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:51.323+0000 7fc143200700 1 -- 172.21.15.19:0/3013184296 wait complete. 2024-05-07T02:40:51.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:51 smithi064 ceph-mon[34699]: pgmap v277: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:40:51.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:51 smithi019 ceph-mon[29781]: pgmap v277: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:40:51.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:51 smithi027 ceph-mon[34398]: pgmap v277: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:40:52.379 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-07T02:40:52.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:52 smithi064 ceph-mon[34699]: scrub queued for path: / 2024-05-07T02:40:52.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:52 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:40:52.860 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:52 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:40:52.860 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:52 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:40:52.860 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:52 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:40:52.860 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:52 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:40:52.860 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:52 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:40:52.861 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:52 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:40:52.861 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:52 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:40:52.861 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:52 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:40:52.861 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:52 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:40:52.861 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:52 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:40:52.861 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:52 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:40:52.862 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:52 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:40:52.862 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:52 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:40:52.862 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:52 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:40:52.862 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:52 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:40:52.862 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:52 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:40:52.863 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:52 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:40:52.863 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:52 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:40:52.863 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:52 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:40:52.863 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:52 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:40:52.863 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:52 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:40:52.863 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:52 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:40:52.864 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:40:52.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:52 smithi019 ceph-mon[29781]: scrub queued for path: / 2024-05-07T02:40:52.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:52 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:40:52.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:52 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:40:52.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:52 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:40:52.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:52 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:40:52.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:52 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:40:52.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:52 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:40:52.897 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:52 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:40:52.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:52 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:40:52.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:52 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:40:52.898 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:52 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:40:52.899 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:52 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:40:52.899 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:52 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:40:52.899 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:52 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:40:52.899 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:52 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:40:52.899 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:52 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:40:52.900 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:52 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:40:52.900 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:52 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:40:52.900 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:52 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:40:52.900 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:52 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:40:52.901 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:52 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:40:52.901 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:52 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:40:52.901 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:52 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:40:52.901 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:52 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:40:52.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:52 smithi027 ceph-mon[34398]: scrub queued for path: / 2024-05-07T02:40:52.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:52 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:40:52.958 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:52 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:40:52.958 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:52 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:40:52.958 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:52 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:40:52.958 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:52 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:40:52.958 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:52 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:40:52.959 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:52 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:40:52.959 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:52 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:40:52.959 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:52 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:40:52.959 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:52 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:40:52.959 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:52 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:40:52.960 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:52 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:40:52.960 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:52 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:40:52.960 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:52 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:40:52.960 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:52 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:40:52.960 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:52 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:40:52.960 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:52 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:40:52.961 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:52 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:40:52.961 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:52 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:40:52.961 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:52 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:40:52.961 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:52 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:40:52.961 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:52 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:40:52.961 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:52 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:40:53.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:53 smithi064 ceph-mon[34699]: pgmap v278: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:40:53.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:53 smithi019 ceph-mon[29781]: pgmap v278: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:40:53.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:53 smithi027 ceph-mon[34398]: pgmap v278: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:40:55.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.332+0000 7f84d4800700 1 -- 172.21.15.19:0/3194002673 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84cc103ba0 msgr2=0x7f84cc1080d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:55.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.332+0000 7f84d4800700 1 --2- 172.21.15.19:0/3194002673 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84cc103ba0 0x7f84cc1080d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f84bc0098f0 tx=0x7f84bc009c00 comp rx=0 tx=0).stop 2024-05-07T02:40:55.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.332+0000 7f84d4800700 1 -- 172.21.15.19:0/3194002673 shutdown_connections 2024-05-07T02:40:55.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.332+0000 7f84d4800700 1 --2- 172.21.15.19:0/3194002673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84cc1086a0 0x7f84cc108b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:55.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.332+0000 7f84d4800700 1 --2- 172.21.15.19:0/3194002673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84cc103250 0x7f84cc103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:55.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.332+0000 7f84d4800700 1 --2- 172.21.15.19:0/3194002673 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84cc103ba0 0x7f84cc1080d0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:55.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.332+0000 7f84d4800700 1 -- 172.21.15.19:0/3194002673 >> 172.21.15.19:0/3194002673 conn(0x7f84cc0fedc0 msgr2=0x7f84cc1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:55.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.332+0000 7f84d4800700 1 -- 172.21.15.19:0/3194002673 shutdown_connections 2024-05-07T02:40:55.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.332+0000 7f84d4800700 1 -- 172.21.15.19:0/3194002673 wait complete. 2024-05-07T02:40:55.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.333+0000 7f84d4800700 1 Processor -- start 2024-05-07T02:40:55.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.333+0000 7f84d4800700 1 -- start start 2024-05-07T02:40:55.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.334+0000 7f84d4800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84cc103250 0x7f84cc1a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:55.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.334+0000 7f84d4800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84cc103ba0 0x7f84cc1a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:55.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.334+0000 7f84d4800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84cc1086a0 0x7f84cc1a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:55.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.334+0000 7f84d4800700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f84cc11aa20 con 0x7f84cc1086a0 2024-05-07T02:40:55.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.334+0000 7f84d4800700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f84cc11a8a0 con 0x7f84cc103250 2024-05-07T02:40:55.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.334+0000 7f84d4800700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f84cc11aba0 con 0x7f84cc103ba0 2024-05-07T02:40:55.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.334+0000 7f84d1400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84cc103ba0 0x7f84cc1a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:55.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.334+0000 7f84d2800700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84cc1086a0 0x7f84cc1a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:55.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.334+0000 7f84d1400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84cc103ba0 0x7f84cc1a19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:34990/0 (socket says 172.21.15.19:34990) 2024-05-07T02:40:55.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.334+0000 7f84d1e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84cc103250 0x7f84cc1a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:55.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.334+0000 7f84d1400700 1 -- 172.21.15.19:0/2722814144 learned_addr learned my addr 172.21.15.19:0/2722814144 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:40:55.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.335+0000 7f84d1400700 1 -- 172.21.15.19:0/2722814144 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84cc103250 msgr2=0x7f84cc1a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:55.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.335+0000 7f84d1400700 1 --2- 172.21.15.19:0/2722814144 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84cc103250 0x7f84cc1a1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:55.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.335+0000 7f84d1400700 1 -- 172.21.15.19:0/2722814144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84cc1086a0 msgr2=0x7f84cc1a5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:55.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.335+0000 7f84d1400700 1 --2- 172.21.15.19:0/2722814144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84cc1086a0 0x7f84cc1a5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:55.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.335+0000 7f84d1400700 1 -- 172.21.15.19:0/2722814144 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84cc1a62e0 con 0x7f84cc103ba0 2024-05-07T02:40:55.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.335+0000 7f84d1400700 1 --2- 172.21.15.19:0/2722814144 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84cc103ba0 0x7f84cc1a19b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f84bc00f990 tx=0x7f84bc00f9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:55.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.335+0000 7f84c6a00700 1 -- 172.21.15.19:0/2722814144 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84bc027070 con 0x7f84cc103ba0 2024-05-07T02:40:55.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.336+0000 7f84d4800700 1 -- 172.21.15.19:0/2722814144 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84cc1a6570 con 0x7f84cc103ba0 2024-05-07T02:40:55.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.336+0000 7f84c6a00700 1 -- 172.21.15.19:0/2722814144 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84bc0055f0 con 0x7f84cc103ba0 2024-05-07T02:40:55.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.336+0000 7f84c6a00700 1 -- 172.21.15.19:0/2722814144 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84bc004930 con 0x7f84cc103ba0 2024-05-07T02:40:55.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.336+0000 7f84d4800700 1 -- 172.21.15.19:0/2722814144 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84cc1adef0 con 0x7f84cc103ba0 2024-05-07T02:40:55.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.339+0000 7f84c6a00700 1 -- 172.21.15.19:0/2722814144 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f84bc02a070 con 0x7f84cc103ba0 2024-05-07T02:40:55.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.340+0000 7f84e3330b80 1 Processor -- start 2024-05-07T02:40:55.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.341+0000 7f84e3330b80 1 -- start start 2024-05-07T02:40:55.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.341+0000 7f84c6a00700 1 --2- 172.21.15.19:0/2722814144 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f84b40b29f0 0x7f84b40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:55.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.341+0000 7f84e3330b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d02097e3b0 0x55d02097e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:55.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.341+0000 7f84e3330b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d020978470 0x55d020978880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:55.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.341+0000 7f84e3330b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d020978dc0 0x55d020985040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:55.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.341+0000 7f84e3330b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d02089aa20 con 0x55d020978470 2024-05-07T02:40:55.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.341+0000 7f84e3330b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d020838760 con 0x55d02097e3b0 2024-05-07T02:40:55.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.341+0000 7f84e3330b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d0208969e0 con 0x55d020978dc0 2024-05-07T02:40:55.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.341+0000 7f84c6a00700 1 -- 172.21.15.19:0/2722814144 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9525+0+0 (secure 0 0 0) 0x7f84bc0e33b0 con 0x7f84cc103ba0 2024-05-07T02:40:55.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.342+0000 7f84d2800700 1 --2- 172.21.15.19:0/2722814144 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f84b40b29f0 0x7f84b40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:55.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.342+0000 7f84d1400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d020978470 0x55d020978880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:55.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.342+0000 7f84d1e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d02097e3b0 0x55d02097e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:55.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.342+0000 7f84d1400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d020978470 0x55d020978880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50334/0 (socket says 172.21.15.19:50334) 2024-05-07T02:40:55.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.342+0000 7f84d1400700 1 -- 172.21.15.19:0/3836181376 learned_addr learned my addr 172.21.15.19:0/3836181376 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:40:55.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.342+0000 7f84d2800700 1 --2- 172.21.15.19:0/3836181376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d020978dc0 0x55d020985040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:55.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.342+0000 7f84d1400700 1 -- 172.21.15.19:0/3836181376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d020978dc0 msgr2=0x55d020985040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:55.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.342+0000 7f84d1400700 1 --2- 172.21.15.19:0/3836181376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d020978dc0 0x55d020985040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:55.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.342+0000 7f84d1400700 1 -- 172.21.15.19:0/3836181376 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d02097e3b0 msgr2=0x55d02097e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:55.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.342+0000 7f84d1400700 1 --2- 172.21.15.19:0/3836181376 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d02097e3b0 0x55d02097e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:55.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.342+0000 7f84d1400700 1 -- 172.21.15.19:0/3836181376 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0207e78d0 con 0x55d020978470 2024-05-07T02:40:55.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.343+0000 7f84d1400700 1 --2- 172.21.15.19:0/3836181376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d020978470 0x55d020978880 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f84bc0a9fd0 tx=0x7f84bc009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ee7563e9018802 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:55.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.343+0000 7f84d2800700 1 --2- 172.21.15.19:0/2722814144 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f84b40b29f0 0x7f84b40b4f00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f84c8009ec0 tx=0x7f84c8013250 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:55.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.343+0000 7f84bb400700 1 -- 172.21.15.19:0/3836181376 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84bc0a9de0 con 0x55d020978470 2024-05-07T02:40:55.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.343+0000 7f84bb400700 1 -- 172.21.15.19:0/3836181376 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84bc0aa9d0 con 0x55d020978470 2024-05-07T02:40:55.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.343+0000 7f84bb400700 1 -- 172.21.15.19:0/3836181376 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84bc0595f0 con 0x55d020978470 2024-05-07T02:40:55.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.344+0000 7f84e3330b80 1 -- 172.21.15.19:0/3836181376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d020978470 msgr2=0x55d020978880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:55.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.344+0000 7f84e3330b80 1 --2- 172.21.15.19:0/3836181376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d020978470 0x55d020978880 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f84bc0a9fd0 tx=0x7f84bc009640 comp rx=0 tx=0).stop 2024-05-07T02:40:55.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.344+0000 7f84e3330b80 1 -- 172.21.15.19:0/3836181376 shutdown_connections 2024-05-07T02:40:55.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.344+0000 7f84e3330b80 1 --2- 172.21.15.19:0/3836181376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d020978dc0 0x55d020985040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:55.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.344+0000 7f84e3330b80 1 --2- 172.21.15.19:0/3836181376 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d02097e3b0 0x55d02097e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:55.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.344+0000 7f84e3330b80 1 --2- 172.21.15.19:0/3836181376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d020978470 0x55d020978880 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:55.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.344+0000 7f84e3330b80 1 -- 172.21.15.19:0/3836181376 >> 172.21.15.19:0/3836181376 conn(0x55d0207fae40 msgr2=0x55d0209732f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:55.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.344+0000 7f84e3330b80 1 -- 172.21.15.19:0/3836181376 shutdown_connections 2024-05-07T02:40:55.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.344+0000 7f84e3330b80 1 -- 172.21.15.19:0/3836181376 wait complete. 2024-05-07T02:40:55.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.345+0000 7f84e3330b80 1 -- start start 2024-05-07T02:40:55.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.345+0000 7f84e3330b80 1 Processor -- start 2024-05-07T02:40:55.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.346+0000 7f84e3330b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d020978470 0x55d020982590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:55.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.346+0000 7f84e3330b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d020978dc0 0x55d020982ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:55.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.346+0000 7f84e3330b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d02097e3b0 0x55d020983010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:55.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.346+0000 7f84e3330b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d020985580 con 0x55d020978dc0 2024-05-07T02:40:55.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.346+0000 7f84e3330b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55d020800b20 con 0x55d020978470 2024-05-07T02:40:55.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.346+0000 7f84e3330b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d020985700 con 0x55d02097e3b0 2024-05-07T02:40:55.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.346+0000 7f84e3330b80 10 client.0 _ll_register_callbacks cb 0x55d020841c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:40:55.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.346+0000 7f84d2800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d02097e3b0 0x55d020983010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:55.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.346+0000 7f84d1e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d020978470 0x55d020982590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:55.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.346+0000 7f84d2800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d02097e3b0 0x55d020983010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:35000/0 (socket says 172.21.15.19:35000) 2024-05-07T02:40:55.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.346+0000 7f84d2800700 1 -- 172.21.15.19:0/2886754774 learned_addr learned my addr 172.21.15.19:0/2886754774 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:40:55.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.346+0000 7f84d1400700 1 --2- 172.21.15.19:0/2886754774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d020978dc0 0x55d020982ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:55.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.347+0000 7f84d2800700 1 -- 172.21.15.19:0/2886754774 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d020978470 msgr2=0x55d020982590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:55.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.347+0000 7f84d2800700 1 --2- 172.21.15.19:0/2886754774 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d020978470 0x55d020982590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:55.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.347+0000 7f84d2800700 1 -- 172.21.15.19:0/2886754774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d020978dc0 msgr2=0x55d020982ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:55.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.347+0000 7f84d2800700 1 --2- 172.21.15.19:0/2886754774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d020978dc0 0x55d020982ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:55.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.347+0000 7f84d2800700 1 -- 172.21.15.19:0/2886754774 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d020860420 con 0x55d02097e3b0 2024-05-07T02:40:55.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.347+0000 7f84d2800700 1 --2- 172.21.15.19:0/2886754774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d02097e3b0 0x55d020983010 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f84c801b9d0 tx=0x7f84c8004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5782a4ee262b18ce server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:55.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.347+0000 7f84afe00700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:40:55.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.347+0000 7f84afe00700 1 -- 172.21.15.19:0/2886754774 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c8021070 con 0x55d02097e3b0 2024-05-07T02:40:55.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.347+0000 7f84e3330b80 1 -- 172.21.15.19:0/2886754774 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84bc0aa9d0 con 0x55d02097e3b0 2024-05-07T02:40:55.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.348+0000 7f84afe00700 1 -- 172.21.15.19:0/2886754774 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9525+0+0 (secure 0 0 0) 0x7f84c8003a00 con 0x55d02097e3b0 2024-05-07T02:40:55.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.348+0000 7f84afe00700 1 -- 172.21.15.19:0/2886754774 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f84c8003c00 con 0x55d02097e3b0 2024-05-07T02:40:55.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.348+0000 7f84e3330b80 10 client.26560 fetch_fsmap learned FSMap version 50 2024-05-07T02:40:55.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.348+0000 7f84e3330b80 1 -- 172.21.15.19:0/2886754774 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55d020990190 con 0x55d02097e3b0 2024-05-07T02:40:55.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.348+0000 7f84afe00700 1 -- 172.21.15.19:0/2886754774 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84bc0aa9d0 con 0x55d02097e3b0 2024-05-07T02:40:55.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.348+0000 7f84afe00700 1 -- 172.21.15.19:0/2886754774 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 50) ==== 8362+0+0 (secure 0 0 0) 0x7f84c8018bb0 con 0x55d02097e3b0 2024-05-07T02:40:55.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.348+0000 7f84e3330b80 10 client.26560 fetch_fsmap finished waiting for FSMap version 50 2024-05-07T02:40:55.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.348+0000 7f84e3330b80 10 client.26560 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:40:55.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.349+0000 7f84e3330b80 20 client.26560 populate_metadata read hostname 'smithi019' 2024-05-07T02:40:55.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.349+0000 7f84e3330b80 1 --2- 172.21.15.19:0/2886754774 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55d020992770 0x55d020994c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:55.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.349+0000 7f84e3330b80 4 client.26560 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:40:55.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.349+0000 7f84e3330b80 1 -- 172.21.15.19:0/2886754774 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0209452e0 con 0x55d020992770 2024-05-07T02:40:55.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.349+0000 7f84d1e00700 1 --2- 172.21.15.19:0/2886754774 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55d020992770 0x55d020994c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:55.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.350+0000 7f84d1e00700 1 --2- 172.21.15.19:0/2886754774 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55d020992770 0x55d020994c10 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8fe775cf0a686f4 server_cookie=6cfded2071bfe4f0 in_seq=0 out_seq=0 2024-05-07T02:40:55.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.350+0000 7f84afe00700 10 client.26560 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:40:55.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.350+0000 7f84afe00700 1 -- 172.21.15.19:0/2886754774 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d0209452e0 con 0x55d020992770 2024-05-07T02:40:55.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.350+0000 7f84afe00700 10 client.26560 handle_command_reply: tid=0 2024-05-07T02:40:55.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.351+0000 7f84e3330b80 1 client.26560 shutdown 2024-05-07T02:40:55.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.351+0000 7f84e3330b80 1 -- 172.21.15.19:0/2886754774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d02097e3b0 msgr2=0x55d020983010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:55.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.351+0000 7f84e3330b80 1 --2- 172.21.15.19:0/2886754774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d02097e3b0 0x55d020983010 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f84c801b9d0 tx=0x7f84c8004b40 comp rx=0 tx=0).stop 2024-05-07T02:40:55.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.351+0000 7f84e3330b80 1 -- 172.21.15.19:0/2886754774 shutdown_connections 2024-05-07T02:40:55.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.351+0000 7f84e3330b80 1 --2- 172.21.15.19:0/2886754774 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55d020992770 0x55d020994c10 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:55.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.351+0000 7f84e3330b80 1 --2- 172.21.15.19:0/2886754774 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d02097e3b0 0x55d020983010 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:55.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.351+0000 7f84e3330b80 1 --2- 172.21.15.19:0/2886754774 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d020978470 0x55d020982590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:55.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.351+0000 7f84e3330b80 1 --2- 172.21.15.19:0/2886754774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d020978dc0 0x55d020982ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:55.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.351+0000 7f84afe00700 0 client.26560 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:40:55.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.351+0000 7f84e3330b80 1 -- 172.21.15.19:0/2886754774 >> 172.21.15.19:0/2886754774 conn(0x55d0207fae40 msgr2=0x55d020977460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:55.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.351+0000 7f84e3330b80 1 -- 172.21.15.19:0/2886754774 shutdown_connections 2024-05-07T02:40:55.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.351+0000 7f84e3330b80 1 -- 172.21.15.19:0/2886754774 wait complete. 2024-05-07T02:40:55.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.351+0000 7f84e3330b80 20 client.26560 trim_cache size 0 max 16384 2024-05-07T02:40:55.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.374+0000 7f84e3330b80 1 Processor -- start 2024-05-07T02:40:55.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.374+0000 7f84e3330b80 1 -- start start 2024-05-07T02:40:55.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.374+0000 7f84e3330b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d020978470 0x55d020977e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:55.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.374+0000 7f84e3330b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d020978dc0 0x55d020981eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:55.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.374+0000 7f84e3330b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d02097e3b0 0x55d02097fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:55.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.374+0000 7f84e3330b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f84cc1b6b50 con 0x55d02097e3b0 2024-05-07T02:40:55.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.374+0000 7f84e3330b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f84cc1b8690 con 0x55d020978dc0 2024-05-07T02:40:55.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.374+0000 7f84e3330b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f84cc1b7220 con 0x55d020978470 2024-05-07T02:40:55.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.375+0000 7f84d1e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d020978470 0x55d020977e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:55.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.375+0000 7f84d1400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d020978dc0 0x55d020981eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:55.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.375+0000 7f84d1e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d020978470 0x55d020977e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:35014/0 (socket says 172.21.15.19:35014) 2024-05-07T02:40:55.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.375+0000 7f84d1400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d020978dc0 0x55d020981eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:33402/0 (socket says 172.21.15.19:33402) 2024-05-07T02:40:55.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.375+0000 7f84d1e00700 1 -- 172.21.15.19:0/1368299402 learned_addr learned my addr 172.21.15.19:0/1368299402 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:40:55.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.375+0000 7f84d2800700 1 --2- 172.21.15.19:0/1368299402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d02097e3b0 0x55d02097fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:55.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.375+0000 7f84d1e00700 1 -- 172.21.15.19:0/1368299402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d020978dc0 msgr2=0x55d020981eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:55.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.375+0000 7f84d1e00700 1 --2- 172.21.15.19:0/1368299402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d020978dc0 0x55d020981eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:55.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.375+0000 7f84d1e00700 1 -- 172.21.15.19:0/1368299402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d02097e3b0 msgr2=0x55d02097fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:55.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.375+0000 7f84d1e00700 1 --2- 172.21.15.19:0/1368299402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d02097e3b0 0x55d02097fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:55.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.375+0000 7f84d1e00700 1 -- 172.21.15.19:0/1368299402 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d020974a70 con 0x55d020978470 2024-05-07T02:40:55.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.375+0000 7f84d1400700 1 --2- 172.21.15.19:0/1368299402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d020978dc0 0x55d020981eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:40:55.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.376+0000 7f84d1e00700 1 --2- 172.21.15.19:0/1368299402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d020978470 0x55d020977e90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x55d02097de70 tx=0x7f84c0009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8a6259e854a599f server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:55.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.376+0000 7f84baa00700 1 -- 172.21.15.19:0/1368299402 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c0026070 con 0x55d020978470 2024-05-07T02:40:55.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.376+0000 7f84baa00700 1 -- 172.21.15.19:0/1368299402 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84c0004310 con 0x55d020978470 2024-05-07T02:40:55.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.376+0000 7f84baa00700 1 -- 172.21.15.19:0/1368299402 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c0020560 con 0x55d020978470 2024-05-07T02:40:55.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.377+0000 7f84e3330b80 1 -- 172.21.15.19:0/1368299402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d020978470 msgr2=0x55d020977e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:55.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.377+0000 7f84e3330b80 1 --2- 172.21.15.19:0/1368299402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d020978470 0x55d020977e90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x55d02097de70 tx=0x7f84c0009310 comp rx=0 tx=0).stop 2024-05-07T02:40:55.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.377+0000 7f84e3330b80 1 -- 172.21.15.19:0/1368299402 shutdown_connections 2024-05-07T02:40:55.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.377+0000 7f84e3330b80 1 --2- 172.21.15.19:0/1368299402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d02097e3b0 0x55d02097fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:55.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.377+0000 7f84e3330b80 1 --2- 172.21.15.19:0/1368299402 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d020978470 0x55d020977e90 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:55.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.377+0000 7f84e3330b80 1 --2- 172.21.15.19:0/1368299402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d020978dc0 0x55d020981eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:55.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.377+0000 7f84e3330b80 1 -- 172.21.15.19:0/1368299402 >> 172.21.15.19:0/1368299402 conn(0x55d0207fae40 msgr2=0x55d020982bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:55.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.377+0000 7f84e3330b80 1 -- 172.21.15.19:0/1368299402 shutdown_connections 2024-05-07T02:40:55.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.377+0000 7f84e3330b80 1 -- 172.21.15.19:0/1368299402 wait complete. 2024-05-07T02:40:55.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.378+0000 7f84e3330b80 1 -- start start 2024-05-07T02:40:55.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.378+0000 7f84e3330b80 1 Processor -- start 2024-05-07T02:40:55.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.379+0000 7f84e3330b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d020978470 0x55d02098a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:55.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.379+0000 7f84e3330b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d020978dc0 0x55d02098ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:55.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.379+0000 7f84e3330b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d02097e3b0 0x55d020987920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:55.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.379+0000 7f84e3330b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f84cc1b7fc0 con 0x55d020978470 2024-05-07T02:40:55.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.379+0000 7f84e3330b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f84cc1b78f0 con 0x55d020978dc0 2024-05-07T02:40:55.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.379+0000 7f84e3330b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f84cc1b4b10 con 0x55d02097e3b0 2024-05-07T02:40:55.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.379+0000 7f84e3330b80 10 client.0 _ll_register_callbacks cb 0x55d020841c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:40:55.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.379+0000 7f84d1e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d020978470 0x55d02098a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:55.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.379+0000 7f84d2800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d02097e3b0 0x55d020987920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:55.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.379+0000 7f84d2800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d02097e3b0 0x55d020987920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:35022/0 (socket says 172.21.15.19:35022) 2024-05-07T02:40:55.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.379+0000 7f84d2800700 1 -- 172.21.15.19:0/2930788220 learned_addr learned my addr 172.21.15.19:0/2930788220 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:40:55.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.379+0000 7f84d1e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d020978470 0x55d02098a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50372/0 (socket says 172.21.15.19:50372) 2024-05-07T02:40:55.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.379+0000 7f84d1400700 1 --2- 172.21.15.19:0/2930788220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d020978dc0 0x55d02098ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:55.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.380+0000 7f84d2800700 1 -- 172.21.15.19:0/2930788220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d020978dc0 msgr2=0x55d02098ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:55.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.380+0000 7f84d2800700 1 --2- 172.21.15.19:0/2930788220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d020978dc0 0x55d02098ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:55.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.380+0000 7f84d2800700 1 -- 172.21.15.19:0/2930788220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d020978470 msgr2=0x55d02098a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:40:55.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.380+0000 7f84d2800700 1 --2- 172.21.15.19:0/2930788220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d020978470 0x55d02098a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:55.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.380+0000 7f84d2800700 1 -- 172.21.15.19:0/2930788220 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d020987e60 con 0x55d02097e3b0 2024-05-07T02:40:55.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.380+0000 7f84d2800700 1 --2- 172.21.15.19:0/2930788220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d02097e3b0 0x55d020987920 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f84c8009fd0 tx=0x7f84c8004540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e3b30b3070a1a97 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:40:55.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.380+0000 7f84afe00700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:40:55.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.380+0000 7f84afe00700 1 -- 172.21.15.19:0/2930788220 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c803e030 con 0x55d02097e3b0 2024-05-07T02:40:55.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.380+0000 7f84e3330b80 1 -- 172.21.15.19:0/2930788220 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84c0004310 con 0x55d02097e3b0 2024-05-07T02:40:55.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.380+0000 7f84afe00700 1 -- 172.21.15.19:0/2930788220 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9525+0+0 (secure 0 0 0) 0x7f84c802c040 con 0x55d02097e3b0 2024-05-07T02:40:55.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.381+0000 7f84afe00700 1 -- 172.21.15.19:0/2930788220 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f84c8004bb0 con 0x55d02097e3b0 2024-05-07T02:40:55.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.381+0000 7f84e3330b80 10 client.26566 fetch_fsmap learned FSMap version 50 2024-05-07T02:40:55.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.381+0000 7f84e3330b80 1 -- 172.21.15.19:0/2930788220 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55d0209fd5c0 con 0x55d02097e3b0 2024-05-07T02:40:55.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.381+0000 7f84afe00700 1 -- 172.21.15.19:0/2930788220 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84c0004310 con 0x55d02097e3b0 2024-05-07T02:40:55.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.381+0000 7f84afe00700 1 -- 172.21.15.19:0/2930788220 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 50) ==== 8362+0+0 (secure 0 0 0) 0x7f84c8005cf0 con 0x55d02097e3b0 2024-05-07T02:40:55.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.381+0000 7f84e3330b80 10 client.26566 fetch_fsmap finished waiting for FSMap version 50 2024-05-07T02:40:55.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.381+0000 7f84e3330b80 10 client.26566 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:40:55.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.381+0000 7f84e3330b80 20 client.26566 populate_metadata read hostname 'smithi019' 2024-05-07T02:40:55.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.381+0000 7f84e3330b80 1 --2- 172.21.15.19:0/2930788220 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55d0209fdcc0 0x55d020a000e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:40:55.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.381+0000 7f84e3330b80 4 client.26566 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-07T02:40:55.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.381+0000 7f84e3330b80 1 -- 172.21.15.19:0/2930788220 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d0208a8400 con 0x55d0209fdcc0 2024-05-07T02:40:55.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.382+0000 7f84d1e00700 1 --2- 172.21.15.19:0/2930788220 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55d0209fdcc0 0x55d020a000e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:40:55.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.382+0000 7f84d1e00700 1 --2- 172.21.15.19:0/2930788220 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55d0209fdcc0 0x55d020a000e0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ca441eab537627f server_cookie=b4f15eb751400e56 in_seq=0 out_seq=0 2024-05-07T02:40:55.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.382+0000 7f84afe00700 10 client.26566 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:40:55.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.382+0000 7f84afe00700 1 -- 172.21.15.19:0/2930788220 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d0208a8400 con 0x55d0209fdcc0 2024-05-07T02:40:55.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.383+0000 7f84afe00700 10 client.26566 handle_command_reply: tid=0 2024-05-07T02:40:55.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.383+0000 7f84e3330b80 1 client.26566 shutdown 2024-05-07T02:40:55.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.383+0000 7f84e3330b80 1 -- 172.21.15.19:0/2930788220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d02097e3b0 msgr2=0x55d020987920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:55.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.383+0000 7f84e3330b80 1 --2- 172.21.15.19:0/2930788220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d02097e3b0 0x55d020987920 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f84c8009fd0 tx=0x7f84c8004540 comp rx=0 tx=0).stop 2024-05-07T02:40:55.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.383+0000 7f84e3330b80 1 -- 172.21.15.19:0/2930788220 shutdown_connections 2024-05-07T02:40:55.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.383+0000 7f84e3330b80 1 --2- 172.21.15.19:0/2930788220 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55d0209fdcc0 0x55d020a000e0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:55.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.383+0000 7f84e3330b80 1 --2- 172.21.15.19:0/2930788220 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d02097e3b0 0x55d020987920 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:55.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.383+0000 7f84e3330b80 1 --2- 172.21.15.19:0/2930788220 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d020978470 0x55d02098a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:55.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.383+0000 7f84e3330b80 1 --2- 172.21.15.19:0/2930788220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d020978dc0 0x55d02098ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:55.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.383+0000 7f84afe00700 0 client.26566 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:40:55.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.383+0000 7f84e3330b80 1 -- 172.21.15.19:0/2930788220 >> 172.21.15.19:0/2930788220 conn(0x55d0207fae40 msgr2=0x55d0209838e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:55.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.383+0000 7f84e3330b80 1 -- 172.21.15.19:0/2930788220 shutdown_connections 2024-05-07T02:40:55.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.383+0000 7f84e3330b80 1 -- 172.21.15.19:0/2930788220 wait complete. 2024-05-07T02:40:55.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.383+0000 7f84e3330b80 20 client.26566 trim_cache size 0 max 16384 2024-05-07T02:40:55.390 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:40:55.390 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-07T02:40:55.390 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-05-07T02:40:55.390 INFO:teuthology.orchestra.run.smithi019.stdout: "ede59abf-6c48-4091-a279-0ce4d5606e51": { 2024-05-07T02:40:55.390 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-05-07T02:40:55.390 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "ede59abf-6c48-4091-a279-0ce4d5606e51", 2024-05-07T02:40:55.391 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-05-07T02:40:55.391 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:40:55.391 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:40:55.391 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:40:55.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.384+0000 7f84d4800700 1 -- 172.21.15.19:0/2722814144 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f84b40b29f0 msgr2=0x7f84b40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:55.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.384+0000 7f84d4800700 1 --2- 172.21.15.19:0/2722814144 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f84b40b29f0 0x7f84b40b4f00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f84c8009ec0 tx=0x7f84c8013250 comp rx=0 tx=0).stop 2024-05-07T02:40:55.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.384+0000 7f84d4800700 1 -- 172.21.15.19:0/2722814144 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84cc103ba0 msgr2=0x7f84cc1a19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:40:55.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.384+0000 7f84d4800700 1 --2- 172.21.15.19:0/2722814144 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84cc103ba0 0x7f84cc1a19b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f84bc00f990 tx=0x7f84bc00f9c0 comp rx=0 tx=0).stop 2024-05-07T02:40:55.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.384+0000 7f84d4800700 1 -- 172.21.15.19:0/2722814144 shutdown_connections 2024-05-07T02:40:55.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.384+0000 7f84d4800700 1 --2- 172.21.15.19:0/2722814144 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f84b40b29f0 0x7f84b40b4f00 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:55.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.384+0000 7f84d4800700 1 --2- 172.21.15.19:0/2722814144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84cc1086a0 0x7f84cc1a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:55.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.384+0000 7f84d4800700 1 --2- 172.21.15.19:0/2722814144 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84cc103250 0x7f84cc1a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:55.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.384+0000 7f84d4800700 1 --2- 172.21.15.19:0/2722814144 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f84cc103ba0 0x7f84cc1a19b0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:40:55.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.384+0000 7f84d4800700 1 -- 172.21.15.19:0/2722814144 >> 172.21.15.19:0/2722814144 conn(0x7f84cc0fedc0 msgr2=0x7f84cc106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:40:55.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.384+0000 7f84d4800700 1 -- 172.21.15.19:0/2722814144 shutdown_connections 2024-05-07T02:40:55.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:40:55.384+0000 7f84d4800700 1 -- 172.21.15.19:0/2722814144 wait complete. 2024-05-07T02:40:55.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:55 smithi019 ceph-mon[29781]: pgmap v279: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:40:55.858 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:55 smithi064 ceph-mon[34699]: pgmap v279: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:40:55.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:55 smithi027 ceph-mon[34398]: pgmap v279: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:40:56.337 INFO:tasks.cephfs.filesystem:scrub status for tag:ede59abf-6c48-4091-a279-0ce4d5606e51 - {'path': '/', 'tag': 'ede59abf-6c48-4091-a279-0ce4d5606e51', 'options': 'recursive,force'} 2024-05-07T02:40:57.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:57 smithi064 ceph-mon[34699]: pgmap v280: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 511 B/s wr, 12 op/s 2024-05-07T02:40:57.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:57 smithi019 ceph-mon[29781]: pgmap v280: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 511 B/s wr, 12 op/s 2024-05-07T02:40:57.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:57 smithi027 ceph-mon[34398]: pgmap v280: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 511 B/s wr, 12 op/s 2024-05-07T02:40:59.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:59 smithi064 ceph-mon[34699]: pgmap v281: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-05-07T02:40:59.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:59 smithi064 ceph-mon[34699]: scrub summary: idle 2024-05-07T02:40:59.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:59 smithi064 ceph-mon[34699]: scrub summary: idle 2024-05-07T02:40:59.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:59 smithi064 ceph-mon[34699]: scrub summary: idle 2024-05-07T02:40:59.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:59 smithi064 ceph-mon[34699]: scrub summary: idle 2024-05-07T02:40:59.860 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:40:59 smithi064 ceph-mon[34699]: scrub summary: idle 2024-05-07T02:40:59.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:59 smithi019 ceph-mon[29781]: pgmap v281: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-05-07T02:40:59.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:59 smithi019 ceph-mon[29781]: scrub summary: idle 2024-05-07T02:40:59.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:59 smithi019 ceph-mon[29781]: scrub summary: idle 2024-05-07T02:40:59.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:59 smithi019 ceph-mon[29781]: scrub summary: idle 2024-05-07T02:40:59.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:59 smithi019 ceph-mon[29781]: scrub summary: idle 2024-05-07T02:40:59.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:40:59 smithi019 ceph-mon[29781]: scrub summary: idle 2024-05-07T02:40:59.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:59 smithi027 ceph-mon[34398]: pgmap v281: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-05-07T02:40:59.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:59 smithi027 ceph-mon[34398]: scrub summary: idle 2024-05-07T02:40:59.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:59 smithi027 ceph-mon[34398]: scrub summary: idle 2024-05-07T02:40:59.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:59 smithi027 ceph-mon[34398]: scrub summary: idle 2024-05-07T02:40:59.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:59 smithi027 ceph-mon[34398]: scrub summary: idle 2024-05-07T02:40:59.958 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:40:59 smithi027 ceph-mon[34398]: scrub summary: idle 2024-05-07T02:41:01.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:01 smithi064 ceph-mon[34699]: pgmap v282: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-05-07T02:41:01.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:01 smithi019 ceph-mon[29781]: pgmap v282: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-05-07T02:41:01.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:01 smithi027 ceph-mon[34398]: pgmap v282: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-05-07T02:41:03.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:03 smithi019 ceph-mon[29781]: pgmap v283: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-05-07T02:41:03.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:03 smithi027 ceph-mon[34398]: pgmap v283: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-05-07T02:41:04.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:03 smithi064 ceph-mon[34699]: pgmap v283: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-05-07T02:41:05.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:05 smithi019 ceph-mon[29781]: pgmap v284: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-05-07T02:41:05.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:05 smithi027 ceph-mon[34398]: pgmap v284: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-05-07T02:41:06.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:05 smithi064 ceph-mon[34699]: pgmap v284: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-05-07T02:41:07.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:07 smithi019 ceph-mon[29781]: pgmap v285: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 45 KiB/s wr, 30 op/s 2024-05-07T02:41:07.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:07 smithi027 ceph-mon[34398]: pgmap v285: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 45 KiB/s wr, 30 op/s 2024-05-07T02:41:08.108 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:07 smithi064 ceph-mon[34699]: pgmap v285: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 45 KiB/s wr, 30 op/s 2024-05-07T02:41:09.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:09 smithi027 ceph-mon[34398]: pgmap v286: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-05-07T02:41:10.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:09 smithi064 ceph-mon[34699]: pgmap v286: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-05-07T02:41:10.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:09 smithi019 ceph-mon[29781]: pgmap v286: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-05-07T02:41:10.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:10 smithi027 ceph-mon[34398]: osdmap e95: 12 total, 12 up, 12 in 2024-05-07T02:41:11.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:10 smithi064 ceph-mon[34699]: osdmap e95: 12 total, 12 up, 12 in 2024-05-07T02:41:11.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:10 smithi019 ceph-mon[29781]: osdmap e95: 12 total, 12 up, 12 in 2024-05-07T02:41:11.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:11 smithi027 ceph-mon[34398]: pgmap v288: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-05-07T02:41:12.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:11 smithi064 ceph-mon[34699]: pgmap v288: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-05-07T02:41:12.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:11 smithi019 ceph-mon[29781]: pgmap v288: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-05-07T02:41:13.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:13 smithi027 ceph-mon[34398]: pgmap v289: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-05-07T02:41:14.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:13 smithi064 ceph-mon[34699]: pgmap v289: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-05-07T02:41:14.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:13 smithi019 ceph-mon[29781]: pgmap v289: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-05-07T02:41:15.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:15 smithi027 ceph-mon[34398]: pgmap v290: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-05-07T02:41:16.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:15 smithi064 ceph-mon[34699]: pgmap v290: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-05-07T02:41:16.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:15 smithi019 ceph-mon[29781]: pgmap v290: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-05-07T02:41:17.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:17 smithi027 ceph-mon[34398]: pgmap v291: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-05-07T02:41:17.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:17 smithi027 ceph-mon[34398]: osdmap e96: 12 total, 12 up, 12 in 2024-05-07T02:41:18.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:17 smithi064 ceph-mon[34699]: pgmap v291: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-05-07T02:41:18.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:17 smithi064 ceph-mon[34699]: osdmap e96: 12 total, 12 up, 12 in 2024-05-07T02:41:18.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:17 smithi019 ceph-mon[29781]: pgmap v291: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-05-07T02:41:18.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:17 smithi019 ceph-mon[29781]: osdmap e96: 12 total, 12 up, 12 in 2024-05-07T02:41:19.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:19 smithi027 ceph-mon[34398]: pgmap v293: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 359 B/s wr, 12 op/s 2024-05-07T02:41:20.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:19 smithi064 ceph-mon[34699]: pgmap v293: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 359 B/s wr, 12 op/s 2024-05-07T02:41:20.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:19 smithi019 ceph-mon[29781]: pgmap v293: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 359 B/s wr, 12 op/s 2024-05-07T02:41:21.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:21 smithi027 ceph-mon[34398]: pgmap v294: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-05-07T02:41:22.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:21 smithi064 ceph-mon[34699]: pgmap v294: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-05-07T02:41:22.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:21 smithi019 ceph-mon[29781]: pgmap v294: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-05-07T02:41:23.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:23 smithi027 ceph-mon[34398]: pgmap v295: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-07T02:41:24.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:23 smithi064 ceph-mon[34699]: pgmap v295: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-07T02:41:24.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:23 smithi019 ceph-mon[29781]: pgmap v295: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-07T02:41:24.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:24 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:41:25.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:24 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:41:25.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:24 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:41:25.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:25 smithi027 ceph-mon[34398]: pgmap v296: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-07T02:41:25.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:25 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:41:25.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:25 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:41:25.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:25 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:41:26.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:25 smithi064 ceph-mon[34699]: pgmap v296: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-07T02:41:26.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:25 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:41:26.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:25 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:41:26.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:25 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:41:26.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:25 smithi019 ceph-mon[29781]: pgmap v296: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-07T02:41:26.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:25 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:41:26.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:25 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:41:26.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:25 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:41:26.338 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-07T02:41:26.824 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:41:27.965 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:27 smithi019 ceph-mon[29781]: pgmap v297: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-07T02:41:28.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:27 smithi064 ceph-mon[34699]: pgmap v297: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-07T02:41:28.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:27 smithi027 ceph-mon[34398]: pgmap v297: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-07T02:41:29.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.554+0000 7f8a58600700 1 -- 172.21.15.19:0/594296722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a5010a990 msgr2=0x7f8a5010ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:29.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.554+0000 7f8a58600700 1 --2- 172.21.15.19:0/594296722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a5010a990 0x7f8a5010ada0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c009990 tx=0x7f8a3c009ca0 comp rx=0 tx=0).stop 2024-05-07T02:41:29.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.554+0000 7f8a58600700 1 -- 172.21.15.19:0/594296722 shutdown_connections 2024-05-07T02:41:29.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.554+0000 7f8a58600700 1 --2- 172.21.15.19:0/594296722 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a5010b2e0 0x7f8a50117810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:29.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.554+0000 7f8a58600700 1 --2- 172.21.15.19:0/594296722 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a50110970 0x7f8a50110d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:29.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.554+0000 7f8a58600700 1 --2- 172.21.15.19:0/594296722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a5010a990 0x7f8a5010ada0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:29.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.554+0000 7f8a58600700 1 -- 172.21.15.19:0/594296722 >> 172.21.15.19:0/594296722 conn(0x7f8a50077c80 msgr2=0x7f8a500780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:41:29.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.554+0000 7f8a58600700 1 -- 172.21.15.19:0/594296722 shutdown_connections 2024-05-07T02:41:29.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.554+0000 7f8a58600700 1 -- 172.21.15.19:0/594296722 wait complete. 2024-05-07T02:41:29.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.555+0000 7f8a58600700 1 Processor -- start 2024-05-07T02:41:29.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.555+0000 7f8a58600700 1 -- start start 2024-05-07T02:41:29.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.556+0000 7f8a58600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a5010a990 0x7f8a501a1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:29.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.556+0000 7f8a58600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a5010b2e0 0x7f8a501a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:29.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.556+0000 7f8a58600700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a50110970 0x7f8a501a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:29.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.556+0000 7f8a58600700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5011aa90 con 0x7f8a5010b2e0 2024-05-07T02:41:29.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.556+0000 7f8a58600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5011a910 con 0x7f8a5010a990 2024-05-07T02:41:29.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.556+0000 7f8a58600700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5011ac10 con 0x7f8a50110970 2024-05-07T02:41:29.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.556+0000 7f8a55c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a5010a990 0x7f8a501a1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:29.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.556+0000 7f8a55200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a5010b2e0 0x7f8a501a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:29.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.556+0000 7f8a55c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a5010a990 0x7f8a501a1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:45286/0 (socket says 172.21.15.19:45286) 2024-05-07T02:41:29.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.556+0000 7f8a55c00700 1 -- 172.21.15.19:0/1802696840 learned_addr learned my addr 172.21.15.19:0/1802696840 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:41:29.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.556+0000 7f8a56600700 1 --2- 172.21.15.19:0/1802696840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a50110970 0x7f8a501a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:29.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.557+0000 7f8a55c00700 1 -- 172.21.15.19:0/1802696840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a50110970 msgr2=0x7f8a501a5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:29.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.557+0000 7f8a55c00700 1 --2- 172.21.15.19:0/1802696840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a50110970 0x7f8a501a5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:29.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.557+0000 7f8a55c00700 1 -- 172.21.15.19:0/1802696840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a5010b2e0 msgr2=0x7f8a501a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:29.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.557+0000 7f8a55c00700 1 --2- 172.21.15.19:0/1802696840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a5010b2e0 0x7f8a501a19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:29.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.557+0000 7f8a55c00700 1 -- 172.21.15.19:0/1802696840 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a501a6310 con 0x7f8a5010a990 2024-05-07T02:41:29.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.557+0000 7f8a55c00700 1 --2- 172.21.15.19:0/1802696840 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a5010a990 0x7f8a501a1440 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8a4400e9f0 tx=0x7f8a4400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:41:29.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.557+0000 7f8a4e000700 1 -- 172.21.15.19:0/1802696840 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a44004a30 con 0x7f8a5010a990 2024-05-07T02:41:29.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.557+0000 7f8a58600700 1 -- 172.21.15.19:0/1802696840 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a501a6600 con 0x7f8a5010a990 2024-05-07T02:41:29.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.557+0000 7f8a4e000700 1 -- 172.21.15.19:0/1802696840 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a4400cc70 con 0x7f8a5010a990 2024-05-07T02:41:29.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.558+0000 7f8a4e000700 1 -- 172.21.15.19:0/1802696840 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a44005530 con 0x7f8a5010a990 2024-05-07T02:41:29.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.558+0000 7f8a58600700 1 -- 172.21.15.19:0/1802696840 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a501adf20 con 0x7f8a5010a990 2024-05-07T02:41:29.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.562+0000 7f8a4e000700 1 -- 172.21.15.19:0/1802696840 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8a44004bd0 con 0x7f8a5010a990 2024-05-07T02:41:29.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.562+0000 7f8a6721bb80 1 Processor -- start 2024-05-07T02:41:29.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.563+0000 7f8a6721bb80 1 -- start start 2024-05-07T02:41:29.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.563+0000 7f8a6721bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555de72bf3b0 0x555de72bf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:29.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.563+0000 7f8a6721bb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555de72b9470 0x555de72b9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:29.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.563+0000 7f8a6721bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555de72b9dc0 0x555de72c6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:29.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.563+0000 7f8a6721bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555de71dba20 con 0x555de72b9dc0 2024-05-07T02:41:29.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.563+0000 7f8a6721bb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x555de7179760 con 0x555de72b9470 2024-05-07T02:41:29.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.563+0000 7f8a6721bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555de71d79e0 con 0x555de72bf3b0 2024-05-07T02:41:29.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.563+0000 7f8a55200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555de72b9dc0 0x555de72c6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:29.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.563+0000 7f8a56600700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555de72bf3b0 0x555de72bf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:29.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.563+0000 7f8a55c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555de72b9470 0x555de72b9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:29.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.564+0000 7f8a56600700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555de72bf3b0 0x555de72bf7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:50192/0 (socket says 172.21.15.19:50192) 2024-05-07T02:41:29.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.564+0000 7f8a56600700 1 -- 172.21.15.19:0/1989268391 learned_addr learned my addr 172.21.15.19:0/1989268391 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:41:29.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.564+0000 7f8a4e000700 1 --2- 172.21.15.19:0/1802696840 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f8a380b29f0 0x7f8a380b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:29.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.564+0000 7f8a55c00700 1 -- 172.21.15.19:0/1989268391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555de72bf3b0 msgr2=0x555de72bf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:29.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.564+0000 7f8a55c00700 1 --2- 172.21.15.19:0/1989268391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555de72bf3b0 0x555de72bf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:29.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.564+0000 7f8a55c00700 1 -- 172.21.15.19:0/1989268391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555de72b9dc0 msgr2=0x555de72c6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:29.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.564+0000 7f8a55c00700 1 --2- 172.21.15.19:0/1989268391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555de72b9dc0 0x555de72c6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:29.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.564+0000 7f8a4e000700 1 -- 172.21.15.19:0/1802696840 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9525+0+0 (secure 0 0 0) 0x7f8a440e2bc0 con 0x7f8a5010a990 2024-05-07T02:41:29.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.564+0000 7f8a55c00700 1 -- 172.21.15.19:0/1989268391 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555de71288d0 con 0x555de72b9470 2024-05-07T02:41:29.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.564+0000 7f8a56600700 1 --2- 172.21.15.19:0/1989268391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555de72bf3b0 0x555de72bf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:41:29.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.564+0000 7f8a55200700 1 --2- 172.21.15.19:0/1989268391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555de72b9dc0 0x555de72c6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:41:29.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.564+0000 7f8a56600700 1 --2- 172.21.15.19:0/1802696840 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f8a380b29f0 0x7f8a380b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:29.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.565+0000 7f8a55c00700 1 --2- 172.21.15.19:0/1989268391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555de72b9470 0x555de72b9880 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8a440dfec0 tx=0x7f8a44004f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca67426f58dba2d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:41:29.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.565+0000 7f8a42a00700 1 -- 172.21.15.19:0/1989268391 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a440a5040 con 0x555de72b9470 2024-05-07T02:41:29.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.565+0000 7f8a42a00700 1 -- 172.21.15.19:0/1989268391 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a440b2070 con 0x555de72b9470 2024-05-07T02:41:29.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.565+0000 7f8a42a00700 1 -- 172.21.15.19:0/1989268391 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4400ce20 con 0x555de72b9470 2024-05-07T02:41:29.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.566+0000 7f8a6721bb80 1 -- 172.21.15.19:0/1989268391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555de72b9470 msgr2=0x555de72b9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:29.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.566+0000 7f8a6721bb80 1 --2- 172.21.15.19:0/1989268391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555de72b9470 0x555de72b9880 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8a440dfec0 tx=0x7f8a44004f70 comp rx=0 tx=0).stop 2024-05-07T02:41:29.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.566+0000 7f8a56600700 1 --2- 172.21.15.19:0/1802696840 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f8a380b29f0 0x7f8a380b4f00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x555de72bee70 tx=0x7f8a48009040 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:41:29.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.566+0000 7f8a6721bb80 1 -- 172.21.15.19:0/1989268391 shutdown_connections 2024-05-07T02:41:29.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.566+0000 7f8a6721bb80 1 --2- 172.21.15.19:0/1989268391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555de72b9dc0 0x555de72c6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:29.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.566+0000 7f8a6721bb80 1 --2- 172.21.15.19:0/1989268391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555de72bf3b0 0x555de72bf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:29.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.566+0000 7f8a6721bb80 1 --2- 172.21.15.19:0/1989268391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555de72b9470 0x555de72b9880 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:29.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.566+0000 7f8a6721bb80 1 -- 172.21.15.19:0/1989268391 >> 172.21.15.19:0/1989268391 conn(0x555de713be40 msgr2=0x555de72b42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:41:29.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.566+0000 7f8a6721bb80 1 -- 172.21.15.19:0/1989268391 shutdown_connections 2024-05-07T02:41:29.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.566+0000 7f8a6721bb80 1 -- 172.21.15.19:0/1989268391 wait complete. 2024-05-07T02:41:29.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.567+0000 7f8a6721bb80 1 -- start start 2024-05-07T02:41:29.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.567+0000 7f8a6721bb80 1 Processor -- start 2024-05-07T02:41:29.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.568+0000 7f8a6721bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555de72b9470 0x555de72c34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:29.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.568+0000 7f8a6721bb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555de72b9dc0 0x555de72c39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:29.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.568+0000 7f8a6721bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555de72bf3b0 0x555de72c3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:29.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.568+0000 7f8a6721bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555de72c6760 con 0x555de72bf3b0 2024-05-07T02:41:29.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.568+0000 7f8a6721bb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x555de7141b20 con 0x555de72b9dc0 2024-05-07T02:41:29.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.568+0000 7f8a6721bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555de72c68e0 con 0x555de72b9470 2024-05-07T02:41:29.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.568+0000 7f8a6721bb80 10 client.0 _ll_register_callbacks cb 0x555de7182c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:41:29.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.568+0000 7f8a55200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555de72b9dc0 0x555de72c39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:29.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.568+0000 7f8a56600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555de72bf3b0 0x555de72c3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:29.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.568+0000 7f8a55c00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555de72b9470 0x555de72c34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:29.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.568+0000 7f8a55200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555de72b9dc0 0x555de72c39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:45314/0 (socket says 172.21.15.19:45314) 2024-05-07T02:41:29.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.568+0000 7f8a55c00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555de72b9470 0x555de72c34a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:50194/0 (socket says 172.21.15.19:50194) 2024-05-07T02:41:29.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.568+0000 7f8a55200700 1 -- 172.21.15.19:0/2153367097 learned_addr learned my addr 172.21.15.19:0/2153367097 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:41:29.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.569+0000 7f8a55200700 1 -- 172.21.15.19:0/2153367097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555de72b9470 msgr2=0x555de72c34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:29.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.569+0000 7f8a55200700 1 --2- 172.21.15.19:0/2153367097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555de72b9470 0x555de72c34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:29.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.569+0000 7f8a55200700 1 -- 172.21.15.19:0/2153367097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555de72bf3b0 msgr2=0x555de72c3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:29.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.569+0000 7f8a55200700 1 --2- 172.21.15.19:0/2153367097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555de72bf3b0 0x555de72c3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:29.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.569+0000 7f8a55200700 1 -- 172.21.15.19:0/2153367097 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555de71a1420 con 0x555de72b9dc0 2024-05-07T02:41:29.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.569+0000 7f8a55c00700 1 --2- 172.21.15.19:0/2153367097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555de72b9470 0x555de72c34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:41:29.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.569+0000 7f8a56600700 1 --2- 172.21.15.19:0/2153367097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555de72bf3b0 0x555de72c3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:41:29.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.569+0000 7f8a55200700 1 --2- 172.21.15.19:0/2153367097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555de72b9dc0 0x555de72c39e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8a5006d120 tx=0x7f8a3c005800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21bfff37011990d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:41:29.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.569+0000 7f8a37400700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:41:29.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.570+0000 7f8a37400700 1 -- 172.21.15.19:0/2153367097 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c0095c0 con 0x555de72b9dc0 2024-05-07T02:41:29.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.570+0000 7f8a6721bb80 1 -- 172.21.15.19:0/2153367097 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a440b2070 con 0x555de72b9dc0 2024-05-07T02:41:29.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.570+0000 7f8a37400700 1 -- 172.21.15.19:0/2153367097 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9525+0+0 (secure 0 0 0) 0x7f8a3c003680 con 0x555de72b9dc0 2024-05-07T02:41:29.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.570+0000 7f8a37400700 1 -- 172.21.15.19:0/2153367097 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f8a3c031050 con 0x555de72b9dc0 2024-05-07T02:41:29.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.570+0000 7f8a6721bb80 10 client.26618 fetch_fsmap learned FSMap version 50 2024-05-07T02:41:29.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.570+0000 7f8a6721bb80 1 -- 172.21.15.19:0/2153367097 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x555de72d1360 con 0x555de72b9dc0 2024-05-07T02:41:29.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.570+0000 7f8a37400700 1 -- 172.21.15.19:0/2153367097 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a440b2070 con 0x555de72b9dc0 2024-05-07T02:41:29.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.571+0000 7f8a37400700 1 -- 172.21.15.19:0/2153367097 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 50) ==== 8362+0+0 (secure 0 0 0) 0x7f8a3c0238b0 con 0x555de72b9dc0 2024-05-07T02:41:29.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.571+0000 7f8a6721bb80 10 client.26618 fetch_fsmap finished waiting for FSMap version 50 2024-05-07T02:41:29.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.571+0000 7f8a6721bb80 10 client.26618 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:41:29.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.571+0000 7f8a6721bb80 20 client.26618 populate_metadata read hostname 'smithi019' 2024-05-07T02:41:29.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.571+0000 7f8a6721bb80 1 --2- 172.21.15.19:0/2153367097 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x555de72d38f0 0x555de72d5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:29.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.571+0000 7f8a6721bb80 4 client.26618 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:41:29.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.571+0000 7f8a6721bb80 1 -- 172.21.15.19:0/2153367097 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555de72862e0 con 0x555de72d38f0 2024-05-07T02:41:29.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.571+0000 7f8a55c00700 1 --2- 172.21.15.19:0/2153367097 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x555de72d38f0 0x555de72d5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:29.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.572+0000 7f8a55c00700 1 --2- 172.21.15.19:0/2153367097 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x555de72d38f0 0x555de72d5d40 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c84f8d529c536f4e server_cookie=1d0073689d3076c1 in_seq=0 out_seq=0 2024-05-07T02:41:29.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.572+0000 7f8a37400700 10 client.26618 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:41:29.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.573+0000 7f8a37400700 1 -- 172.21.15.19:0/2153367097 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555de72862e0 con 0x555de72d38f0 2024-05-07T02:41:29.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.573+0000 7f8a37400700 10 client.26618 handle_command_reply: tid=0 2024-05-07T02:41:29.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.573+0000 7f8a6721bb80 1 client.26618 shutdown 2024-05-07T02:41:29.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.573+0000 7f8a6721bb80 1 -- 172.21.15.19:0/2153367097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555de72b9dc0 msgr2=0x555de72c39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:29.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.573+0000 7f8a6721bb80 1 --2- 172.21.15.19:0/2153367097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555de72b9dc0 0x555de72c39e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8a5006d120 tx=0x7f8a3c005800 comp rx=0 tx=0).stop 2024-05-07T02:41:29.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.573+0000 7f8a6721bb80 1 -- 172.21.15.19:0/2153367097 shutdown_connections 2024-05-07T02:41:29.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.573+0000 7f8a6721bb80 1 --2- 172.21.15.19:0/2153367097 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x555de72d38f0 0x555de72d5d40 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:29.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.573+0000 7f8a6721bb80 1 --2- 172.21.15.19:0/2153367097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555de72bf3b0 0x555de72c3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:29.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.573+0000 7f8a6721bb80 1 --2- 172.21.15.19:0/2153367097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555de72b9470 0x555de72c34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:29.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.573+0000 7f8a37400700 0 client.26618 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:41:29.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.573+0000 7f8a6721bb80 1 --2- 172.21.15.19:0/2153367097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555de72b9dc0 0x555de72c39e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:29.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.573+0000 7f8a6721bb80 1 -- 172.21.15.19:0/2153367097 >> 172.21.15.19:0/2153367097 conn(0x555de713be40 msgr2=0x555de72b8460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:41:29.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.573+0000 7f8a6721bb80 1 -- 172.21.15.19:0/2153367097 shutdown_connections 2024-05-07T02:41:29.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.573+0000 7f8a6721bb80 1 -- 172.21.15.19:0/2153367097 wait complete. 2024-05-07T02:41:29.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.574+0000 7f8a6721bb80 20 client.26618 trim_cache size 0 max 16384 2024-05-07T02:41:29.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.596+0000 7f8a6721bb80 1 Processor -- start 2024-05-07T02:41:29.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.597+0000 7f8a6721bb80 1 -- start start 2024-05-07T02:41:29.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.597+0000 7f8a6721bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555de72b9470 0x555de72b8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:29.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.597+0000 7f8a6721bb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555de72b9dc0 0x555de72c3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:29.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.597+0000 7f8a6721bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555de72bf3b0 0x555de72cf820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:29.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.597+0000 7f8a6721bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8a501b6c90 con 0x555de72b9470 2024-05-07T02:41:29.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.597+0000 7f8a6721bb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8a501b87d0 con 0x555de72b9dc0 2024-05-07T02:41:29.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.597+0000 7f8a6721bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8a501b7360 con 0x555de72bf3b0 2024-05-07T02:41:29.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.597+0000 7f8a56600700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555de72bf3b0 0x555de72cf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:29.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.597+0000 7f8a56600700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555de72bf3b0 0x555de72cf820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:50196/0 (socket says 172.21.15.19:50196) 2024-05-07T02:41:29.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.597+0000 7f8a56600700 1 -- 172.21.15.19:0/1446506268 learned_addr learned my addr 172.21.15.19:0/1446506268 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:41:29.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.597+0000 7f8a55200700 1 --2- 172.21.15.19:0/1446506268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555de72b9dc0 0x555de72c3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:29.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.598+0000 7f8a55c00700 1 --2- 172.21.15.19:0/1446506268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555de72b9470 0x555de72b8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:29.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.598+0000 7f8a56600700 1 -- 172.21.15.19:0/1446506268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555de72b9dc0 msgr2=0x555de72c3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:29.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.598+0000 7f8a56600700 1 --2- 172.21.15.19:0/1446506268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555de72b9dc0 0x555de72c3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:29.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.598+0000 7f8a56600700 1 -- 172.21.15.19:0/1446506268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555de72b9470 msgr2=0x555de72b8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:29.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.598+0000 7f8a56600700 1 --2- 172.21.15.19:0/1446506268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555de72b9470 0x555de72b8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:29.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.598+0000 7f8a56600700 1 -- 172.21.15.19:0/1446506268 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555de72b4830 con 0x555de72bf3b0 2024-05-07T02:41:29.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.598+0000 7f8a55c00700 1 --2- 172.21.15.19:0/1446506268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555de72b9470 0x555de72b8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:41:29.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.598+0000 7f8a55200700 1 --2- 172.21.15.19:0/1446506268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555de72b9dc0 0x555de72c3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:41:29.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.598+0000 7f8a56600700 1 --2- 172.21.15.19:0/1446506268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555de72bf3b0 0x555de72cf820 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8a48007b90 tx=0x7f8a48004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26b710dcde8c603f server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:41:29.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.599+0000 7f8a42000700 1 -- 172.21.15.19:0/1446506268 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a48003e10 con 0x555de72bf3b0 2024-05-07T02:41:29.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.599+0000 7f8a42000700 1 -- 172.21.15.19:0/1446506268 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a48029070 con 0x555de72bf3b0 2024-05-07T02:41:29.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.599+0000 7f8a42000700 1 -- 172.21.15.19:0/1446506268 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a480179f0 con 0x555de72bf3b0 2024-05-07T02:41:29.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.600+0000 7f8a6721bb80 1 -- 172.21.15.19:0/1446506268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555de72bf3b0 msgr2=0x555de72cf820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:29.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.600+0000 7f8a6721bb80 1 --2- 172.21.15.19:0/1446506268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555de72bf3b0 0x555de72cf820 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8a48007b90 tx=0x7f8a48004b40 comp rx=0 tx=0).stop 2024-05-07T02:41:29.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.600+0000 7f8a6721bb80 1 -- 172.21.15.19:0/1446506268 shutdown_connections 2024-05-07T02:41:29.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.600+0000 7f8a6721bb80 1 --2- 172.21.15.19:0/1446506268 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555de72bf3b0 0x555de72cf820 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:29.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.600+0000 7f8a6721bb80 1 --2- 172.21.15.19:0/1446506268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555de72b9470 0x555de72b8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:29.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.600+0000 7f8a6721bb80 1 --2- 172.21.15.19:0/1446506268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555de72b9dc0 0x555de72c3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:29.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.600+0000 7f8a6721bb80 1 -- 172.21.15.19:0/1446506268 >> 172.21.15.19:0/1446506268 conn(0x555de713be40 msgr2=0x555de72ae790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:41:29.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.600+0000 7f8a6721bb80 1 -- 172.21.15.19:0/1446506268 shutdown_connections 2024-05-07T02:41:29.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.600+0000 7f8a6721bb80 1 -- 172.21.15.19:0/1446506268 wait complete. 2024-05-07T02:41:29.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.601+0000 7f8a6721bb80 1 -- start start 2024-05-07T02:41:29.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.601+0000 7f8a6721bb80 1 Processor -- start 2024-05-07T02:41:29.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.601+0000 7f8a6721bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555de72b9470 0x555de72cbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:29.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.601+0000 7f8a6721bb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555de72b9dc0 0x555de72cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:29.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.601+0000 7f8a6721bb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555de72bf3b0 0x555de72c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:29.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.601+0000 7f8a6721bb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8a501b8100 con 0x555de72b9dc0 2024-05-07T02:41:29.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.601+0000 7f8a6721bb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8a501b7a30 con 0x555de72bf3b0 2024-05-07T02:41:29.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.601+0000 7f8a6721bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8a501b4c50 con 0x555de72b9470 2024-05-07T02:41:29.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.601+0000 7f8a6721bb80 10 client.0 _ll_register_callbacks cb 0x555de7182c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:41:29.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.602+0000 7f8a55c00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555de72b9470 0x555de72cbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:29.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.602+0000 7f8a55200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555de72b9dc0 0x555de72cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:29.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.602+0000 7f8a55c00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555de72b9470 0x555de72cbba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:50208/0 (socket says 172.21.15.19:50208) 2024-05-07T02:41:29.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.602+0000 7f8a55c00700 1 -- 172.21.15.19:0/1860592502 learned_addr learned my addr 172.21.15.19:0/1860592502 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:41:29.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.602+0000 7f8a56600700 1 --2- 172.21.15.19:0/1860592502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555de72bf3b0 0x555de72c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:29.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.602+0000 7f8a55c00700 1 -- 172.21.15.19:0/1860592502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555de72bf3b0 msgr2=0x555de72c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:29.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.602+0000 7f8a55c00700 1 --2- 172.21.15.19:0/1860592502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555de72bf3b0 0x555de72c8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:29.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.602+0000 7f8a55c00700 1 -- 172.21.15.19:0/1860592502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555de72b9dc0 msgr2=0x555de72cc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:29.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.602+0000 7f8a55c00700 1 --2- 172.21.15.19:0/1860592502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555de72b9dc0 0x555de72cc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:29.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.602+0000 7f8a55c00700 1 -- 172.21.15.19:0/1860592502 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555de72c90b0 con 0x555de72b9470 2024-05-07T02:41:29.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.603+0000 7f8a55c00700 1 --2- 172.21.15.19:0/1860592502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555de72b9470 0x555de72cbba0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8a440ac010 tx=0x7f8a440a9420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a6c10604f9f2eaf server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:41:29.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.603+0000 7f8a37400700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:41:29.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.603+0000 7f8a37400700 1 -- 172.21.15.19:0/1860592502 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a440a5040 con 0x555de72b9470 2024-05-07T02:41:29.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.603+0000 7f8a37400700 1 -- 172.21.15.19:0/1860592502 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9525+0+0 (secure 0 0 0) 0x7f8a440a9eb0 con 0x555de72b9470 2024-05-07T02:41:29.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.603+0000 7f8a6721bb80 1 -- 172.21.15.19:0/1860592502 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a48029070 con 0x555de72b9470 2024-05-07T02:41:29.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.604+0000 7f8a37400700 1 -- 172.21.15.19:0/1860592502 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f8a440e0350 con 0x555de72b9470 2024-05-07T02:41:29.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.604+0000 7f8a6721bb80 10 client.26584 fetch_fsmap learned FSMap version 50 2024-05-07T02:41:29.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.604+0000 7f8a6721bb80 1 -- 172.21.15.19:0/1860592502 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x555de733e6f0 con 0x555de72b9470 2024-05-07T02:41:29.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.604+0000 7f8a37400700 1 -- 172.21.15.19:0/1860592502 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a48029070 con 0x555de72b9470 2024-05-07T02:41:29.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.604+0000 7f8a37400700 1 -- 172.21.15.19:0/1860592502 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 50) ==== 8362+0+0 (secure 0 0 0) 0x7f8a440b2420 con 0x555de72b9470 2024-05-07T02:41:29.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.604+0000 7f8a6721bb80 10 client.26584 fetch_fsmap finished waiting for FSMap version 50 2024-05-07T02:41:29.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.604+0000 7f8a6721bb80 10 client.26584 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:41:29.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.604+0000 7f8a6721bb80 20 client.26584 populate_metadata read hostname 'smithi019' 2024-05-07T02:41:29.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.604+0000 7f8a6721bb80 1 --2- 172.21.15.19:0/1860592502 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x555de733e980 0x555de7340da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:29.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.604+0000 7f8a6721bb80 4 client.26584 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-07T02:41:29.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.604+0000 7f8a6721bb80 1 -- 172.21.15.19:0/1860592502 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555de71e9400 con 0x555de733e980 2024-05-07T02:41:29.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.605+0000 7f8a55200700 1 --2- 172.21.15.19:0/1860592502 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x555de733e980 0x555de7340da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:29.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.605+0000 7f8a55200700 1 --2- 172.21.15.19:0/1860592502 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x555de733e980 0x555de7340da0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe30320ea38ffc35 server_cookie=7960071fb96933ff in_seq=0 out_seq=0 2024-05-07T02:41:29.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.605+0000 7f8a37400700 10 client.26584 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:41:29.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.605+0000 7f8a37400700 1 -- 172.21.15.19:0/1860592502 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555de71e9400 con 0x555de733e980 2024-05-07T02:41:29.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.605+0000 7f8a37400700 10 client.26584 handle_command_reply: tid=0 2024-05-07T02:41:29.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.605+0000 7f8a6721bb80 1 client.26584 shutdown 2024-05-07T02:41:29.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.606+0000 7f8a6721bb80 1 -- 172.21.15.19:0/1860592502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555de72b9470 msgr2=0x555de72cbba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:29.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.606+0000 7f8a6721bb80 1 --2- 172.21.15.19:0/1860592502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555de72b9470 0x555de72cbba0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8a440ac010 tx=0x7f8a440a9420 comp rx=0 tx=0).stop 2024-05-07T02:41:29.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.606+0000 7f8a6721bb80 1 -- 172.21.15.19:0/1860592502 shutdown_connections 2024-05-07T02:41:29.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.606+0000 7f8a6721bb80 1 --2- 172.21.15.19:0/1860592502 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x555de733e980 0x555de7340da0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:29.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.606+0000 7f8a6721bb80 1 --2- 172.21.15.19:0/1860592502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555de72bf3b0 0x555de72c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:29.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.606+0000 7f8a6721bb80 1 --2- 172.21.15.19:0/1860592502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555de72b9470 0x555de72cbba0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:29.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.606+0000 7f8a6721bb80 1 --2- 172.21.15.19:0/1860592502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555de72b9dc0 0x555de72cc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:29.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.606+0000 7f8a37400700 0 client.26584 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:41:29.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.606+0000 7f8a6721bb80 1 -- 172.21.15.19:0/1860592502 >> 172.21.15.19:0/1860592502 conn(0x555de713be40 msgr2=0x555de72c4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:41:29.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.606+0000 7f8a6721bb80 1 -- 172.21.15.19:0/1860592502 shutdown_connections 2024-05-07T02:41:29.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.606+0000 7f8a6721bb80 1 -- 172.21.15.19:0/1860592502 wait complete. 2024-05-07T02:41:29.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.606+0000 7f8a6721bb80 20 client.26584 trim_cache size 0 max 16384 2024-05-07T02:41:29.614 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:41:29.614 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-07T02:41:29.615 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-07T02:41:29.615 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:41:29.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.607+0000 7f8a58600700 1 -- 172.21.15.19:0/1802696840 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f8a380b29f0 msgr2=0x7f8a380b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:29.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.607+0000 7f8a58600700 1 --2- 172.21.15.19:0/1802696840 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f8a380b29f0 0x7f8a380b4f00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x555de72bee70 tx=0x7f8a48009040 comp rx=0 tx=0).stop 2024-05-07T02:41:29.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.607+0000 7f8a58600700 1 -- 172.21.15.19:0/1802696840 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a5010a990 msgr2=0x7f8a501a1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:29.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.607+0000 7f8a58600700 1 --2- 172.21.15.19:0/1802696840 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a5010a990 0x7f8a501a1440 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8a4400e9f0 tx=0x7f8a4400ed00 comp rx=0 tx=0).stop 2024-05-07T02:41:29.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.607+0000 7f8a58600700 1 -- 172.21.15.19:0/1802696840 shutdown_connections 2024-05-07T02:41:29.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.607+0000 7f8a58600700 1 --2- 172.21.15.19:0/1802696840 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8a50110970 0x7f8a501a5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:29.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.607+0000 7f8a58600700 1 --2- 172.21.15.19:0/1802696840 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a5010a990 0x7f8a501a1440 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:29.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.607+0000 7f8a58600700 1 --2- 172.21.15.19:0/1802696840 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f8a380b29f0 0x7f8a380b4f00 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:29.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.607+0000 7f8a58600700 1 --2- 172.21.15.19:0/1802696840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a5010b2e0 0x7f8a501a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:29.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.607+0000 7f8a58600700 1 -- 172.21.15.19:0/1802696840 >> 172.21.15.19:0/1802696840 conn(0x7f8a50077c80 msgr2=0x7f8a5007b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:41:29.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.607+0000 7f8a58600700 1 -- 172.21.15.19:0/1802696840 shutdown_connections 2024-05-07T02:41:29.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:29.607+0000 7f8a58600700 1 -- 172.21.15.19:0/1802696840 wait complete. 2024-05-07T02:41:30.108 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:29 smithi019 ceph-mon[29781]: pgmap v298: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:41:30.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:29 smithi064 ceph-mon[34699]: pgmap v298: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:41:30.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:29 smithi027 ceph-mon[34398]: pgmap v298: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:41:30.589 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-07T02:41:30.589 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-07T02:41:31.065 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:41:31.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:31 smithi019 ceph-mon[29781]: pgmap v299: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:41:32.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:31 smithi064 ceph-mon[34699]: pgmap v299: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:41:32.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:31 smithi027 ceph-mon[34398]: pgmap v299: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:41:33.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.636+0000 7fc7e4e00700 1 -- 172.21.15.19:0/2666834158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7e01034e0 msgr2=0x7fc7e01038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:33.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.636+0000 7fc7e4e00700 1 --2- 172.21.15.19:0/2666834158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7e01034e0 0x7fc7e01038f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc7d00099c0 tx=0x7fc7d0009cd0 comp rx=0 tx=0).stop 2024-05-07T02:41:33.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.636+0000 7fc7e4e00700 1 -- 172.21.15.19:0/2666834158 shutdown_connections 2024-05-07T02:41:33.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.636+0000 7fc7e4e00700 1 --2- 172.21.15.19:0/2666834158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7e0108760 0x7fc7e01100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:33.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.636+0000 7fc7e4e00700 1 --2- 172.21.15.19:0/2666834158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7e01034e0 0x7fc7e01038f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:33.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.636+0000 7fc7e4e00700 1 --2- 172.21.15.19:0/2666834158 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7e0107e10 0x7fc7e0108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:33.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.636+0000 7fc7e4e00700 1 -- 172.21.15.19:0/2666834158 >> 172.21.15.19:0/2666834158 conn(0x7fc7e00fef30 msgr2=0x7fc7e0101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:41:33.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.636+0000 7fc7e4e00700 1 -- 172.21.15.19:0/2666834158 shutdown_connections 2024-05-07T02:41:33.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.636+0000 7fc7e4e00700 1 -- 172.21.15.19:0/2666834158 wait complete. 2024-05-07T02:41:33.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.637+0000 7fc7e4e00700 1 Processor -- start 2024-05-07T02:41:33.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.638+0000 7fc7e4e00700 1 -- start start 2024-05-07T02:41:33.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.638+0000 7fc7e4e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7e01034e0 0x7fc7e019d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:33.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.638+0000 7fc7e4e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7e0107e10 0x7fc7e019d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:33.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.638+0000 7fc7e4e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7e0108760 0x7fc7e01a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:33.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.638+0000 7fc7e4e00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e0113390 con 0x7fc7e0107e10 2024-05-07T02:41:33.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.638+0000 7fc7e4e00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e0113210 con 0x7fc7e01034e0 2024-05-07T02:41:33.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.638+0000 7fc7e4e00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e0113510 con 0x7fc7e0108760 2024-05-07T02:41:33.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.638+0000 7fc7dde00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7e01034e0 0x7fc7e019d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:33.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.639+0000 7fc7de800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7e0108760 0x7fc7e01a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:33.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.638+0000 7fc7dd400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7e0107e10 0x7fc7e019d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:33.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.639+0000 7fc7dde00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7e01034e0 0x7fc7e019d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:45364/0 (socket says 172.21.15.19:45364) 2024-05-07T02:41:33.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.639+0000 7fc7dde00700 1 -- 172.21.15.19:0/4095407855 learned_addr learned my addr 172.21.15.19:0/4095407855 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:41:33.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.639+0000 7fc7dde00700 1 -- 172.21.15.19:0/4095407855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7e0108760 msgr2=0x7fc7e01a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:33.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.639+0000 7fc7dde00700 1 --2- 172.21.15.19:0/4095407855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7e0108760 0x7fc7e01a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:33.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.639+0000 7fc7dde00700 1 -- 172.21.15.19:0/4095407855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7e0107e10 msgr2=0x7fc7e019d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:33.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.639+0000 7fc7dde00700 1 --2- 172.21.15.19:0/4095407855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7e0107e10 0x7fc7e019d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:33.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.639+0000 7fc7dde00700 1 -- 172.21.15.19:0/4095407855 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e01a1f20 con 0x7fc7e01034e0 2024-05-07T02:41:33.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.640+0000 7fc7dde00700 1 --2- 172.21.15.19:0/4095407855 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7e01034e0 0x7fc7e019d050 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0006010 tx=0x7fc7d0004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:41:33.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.640+0000 7fc7d6a00700 1 -- 172.21.15.19:0/4095407855 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d0026070 con 0x7fc7e01034e0 2024-05-07T02:41:33.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.640+0000 7fc7e4e00700 1 -- 172.21.15.19:0/4095407855 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7e01a21b0 con 0x7fc7e01034e0 2024-05-07T02:41:33.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.640+0000 7fc7d6a00700 1 -- 172.21.15.19:0/4095407855 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7d0003c70 con 0x7fc7e01034e0 2024-05-07T02:41:33.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.640+0000 7fc7d6a00700 1 -- 172.21.15.19:0/4095407855 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d000fb70 con 0x7fc7e01034e0 2024-05-07T02:41:33.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.640+0000 7fc7e4e00700 1 -- 172.21.15.19:0/4095407855 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7e01a9b30 con 0x7fc7e01034e0 2024-05-07T02:41:33.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.641+0000 7fc7e4e00700 1 -- 172.21.15.19:0/4095407855 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7e0104990 con 0x7fc7e01034e0 2024-05-07T02:41:33.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.644+0000 7fc7d6a00700 1 -- 172.21.15.19:0/4095407855 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc7d000fd10 con 0x7fc7e01034e0 2024-05-07T02:41:33.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.645+0000 7fc7d6a00700 1 --2- 172.21.15.19:0/4095407855 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc7c40b29f0 0x7fc7c40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:33.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.646+0000 7fc7d6a00700 1 -- 172.21.15.19:0/4095407855 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9525+0+0 (secure 0 0 0) 0x7fc7d00e3750 con 0x7fc7e01034e0 2024-05-07T02:41:33.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.646+0000 7fc7dd400700 1 --2- 172.21.15.19:0/4095407855 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc7c40b29f0 0x7fc7c40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:33.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.646+0000 7fc7dd400700 1 --2- 172.21.15.19:0/4095407855 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc7c40b29f0 0x7fc7c40b4f00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0107c50 tx=0x7fc7cc009260 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:41:33.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.648+0000 7fc7d6a00700 1 -- 172.21.15.19:0/4095407855 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc7d0013070 con 0x7fc7e01034e0 2024-05-07T02:41:33.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:33 smithi019 ceph-mon[29781]: pgmap v300: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:41:33.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.943+0000 7fc7e4e00700 1 -- 172.21.15.19:0/4095407855 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7e00512c0 con 0x7fc7e01034e0 2024-05-07T02:41:33.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.944+0000 7fc7d6a00700 1 -- 172.21.15.19:0/4095407855 <== mon.2 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+11924 (secure 0 0 0) 0x7fc7d00aad70 con 0x7fc7e01034e0 2024-05-07T02:41:33.945 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:41:33.945 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1763670446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1763670446},{"type":"v1","addr":"172.21.15.19:6839","nonce":1763670446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/477529985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":477529985},{"type":"v1","addr":"172.21.15.64:6833","nonce":477529985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":50,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-07T02:26:14.960465+0000","modified":"2024-05-07T02:40:40.531575+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24539,"mds_2":24520,"mds_3":24515,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6833/2171873162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2171873162},{"type":"v1","addr":"172.21.15.27:6833","nonce":2171873162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/2379917839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2379917839},{"type":"v1","addr":"172.21.15.19:6835","nonce":2379917839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6837/2067515538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2067515538},{"type":"v1","addr":"172.21.15.19:6837","nonce":2067515538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6835/1000540600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1000540600},{"type":"v1","addr":"172.21.15.27:6835","nonce":1000540600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/1997318872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1997318872},{"type":"v1","addr":"172.21.15.64:6835","nonce":1997318872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":34,"addr":"172.21.15.64:6837/2585907777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2585907777},{"type":"v1","addr":"172.21.15.64:6837","nonce":2585907777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":33,"addr":"172.21.15.27:6837/2583138143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2583138143},{"type":"v1","addr":"172.21.15.27:6837","nonce":2583138143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":32,"addr":"172.21.15.64:6839/1583120004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":1583120004},{"type":"v1","addr":"172.21.15.64:6839","nonce":1583120004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":29,"addr":"172.21.15.19:6841/3905645246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3905645246},{"type":"v1","addr":"172.21.15.19:6841","nonce":3905645246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6839/3963268634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3963268634},{"type":"v1","addr":"172.21.15.27:6839","nonce":3963268634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24551,24539,24520,24505,24515]},"id":1}]} 2024-05-07T02:41:33.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.947+0000 7fc7e4e00700 1 -- 172.21.15.19:0/4095407855 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc7c40b29f0 msgr2=0x7fc7c40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:33.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.947+0000 7fc7e4e00700 1 --2- 172.21.15.19:0/4095407855 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc7c40b29f0 0x7fc7c40b4f00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0107c50 tx=0x7fc7cc009260 comp rx=0 tx=0).stop 2024-05-07T02:41:33.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.947+0000 7fc7e4e00700 1 -- 172.21.15.19:0/4095407855 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7e01034e0 msgr2=0x7fc7e019d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:33.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.947+0000 7fc7e4e00700 1 --2- 172.21.15.19:0/4095407855 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7e01034e0 0x7fc7e019d050 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0006010 tx=0x7fc7d0004270 comp rx=0 tx=0).stop 2024-05-07T02:41:33.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.947+0000 7fc7e4e00700 1 -- 172.21.15.19:0/4095407855 shutdown_connections 2024-05-07T02:41:33.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.947+0000 7fc7e4e00700 1 --2- 172.21.15.19:0/4095407855 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc7e0108760 0x7fc7e01a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:33.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.947+0000 7fc7e4e00700 1 --2- 172.21.15.19:0/4095407855 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7e01034e0 0x7fc7e019d050 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:33.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.947+0000 7fc7e4e00700 1 --2- 172.21.15.19:0/4095407855 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc7c40b29f0 0x7fc7c40b4f00 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:33.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.947+0000 7fc7e4e00700 1 --2- 172.21.15.19:0/4095407855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7e0107e10 0x7fc7e019d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:33.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.947+0000 7fc7e4e00700 1 -- 172.21.15.19:0/4095407855 >> 172.21.15.19:0/4095407855 conn(0x7fc7e00fef30 msgr2=0x7fc7e006ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:41:33.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.948+0000 7fc7e4e00700 1 -- 172.21.15.19:0/4095407855 shutdown_connections 2024-05-07T02:41:33.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:33.948+0000 7fc7e4e00700 1 -- 172.21.15.19:0/4095407855 wait complete. 2024-05-07T02:41:33.950 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 50 2024-05-07T02:41:34.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:33 smithi064 ceph-mon[34699]: pgmap v300: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:41:34.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:33 smithi027 ceph-mon[34398]: pgmap v300: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:41:34.899 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-07T02:41:35.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:34 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/4095407855' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:41:35.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:34 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/4095407855' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:41:35.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:34 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/4095407855' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:41:35.360 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:41:35.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:35 smithi019 ceph-mon[29781]: pgmap v301: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:41:36.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:35 smithi064 ceph-mon[34699]: pgmap v301: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:41:36.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:35 smithi027 ceph-mon[34398]: pgmap v301: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:41:37.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.807+0000 7ff3b2400700 1 -- 172.21.15.19:0/5292083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3ac109870 msgr2=0x7ff3ac1111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:37.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.807+0000 7ff3b2400700 1 --2- 172.21.15.19:0/5292083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3ac109870 0x7ff3ac1111d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff3a40099c0 tx=0x7ff3a4009cd0 comp rx=0 tx=0).stop 2024-05-07T02:41:37.809 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:37 smithi019 ceph-mon[29781]: pgmap v302: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:41:37.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.807+0000 7ff3b2400700 1 -- 172.21.15.19:0/5292083 shutdown_connections 2024-05-07T02:41:37.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.807+0000 7ff3b2400700 1 --2- 172.21.15.19:0/5292083 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3ac109870 0x7ff3ac1111d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:37.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.807+0000 7ff3b2400700 1 --2- 172.21.15.19:0/5292083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3ac1045f0 0x7ff3ac104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:37.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.807+0000 7ff3b2400700 1 --2- 172.21.15.19:0/5292083 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3ac108f20 0x7ff3ac109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:37.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.807+0000 7ff3b2400700 1 -- 172.21.15.19:0/5292083 >> 172.21.15.19:0/5292083 conn(0x7ff3ac100040 msgr2=0x7ff3ac102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:41:37.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.807+0000 7ff3b2400700 1 -- 172.21.15.19:0/5292083 shutdown_connections 2024-05-07T02:41:37.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.807+0000 7ff3b2400700 1 -- 172.21.15.19:0/5292083 wait complete. 2024-05-07T02:41:37.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.809+0000 7ff3b2400700 1 Processor -- start 2024-05-07T02:41:37.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.809+0000 7ff3b2400700 1 -- start start 2024-05-07T02:41:37.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.809+0000 7ff3b2400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3ac1045f0 0x7ff3ac19d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:37.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.809+0000 7ff3b2400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3ac108f20 0x7ff3ac19d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:37.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.809+0000 7ff3b2400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3ac109870 0x7ff3ac1a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:37.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.809+0000 7ff3b2400700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac1144a0 con 0x7ff3ac1045f0 2024-05-07T02:41:37.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.809+0000 7ff3b2400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac114320 con 0x7ff3ac108f20 2024-05-07T02:41:37.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.809+0000 7ff3b2400700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac114620 con 0x7ff3ac109870 2024-05-07T02:41:37.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.810+0000 7ff3ab400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3ac1045f0 0x7ff3ac19d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:37.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.810+0000 7ff3a2a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3ac108f20 0x7ff3ac19d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:37.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.810+0000 7ff3abe00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3ac109870 0x7ff3ac1a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:37.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.810+0000 7ff3ab400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3ac1045f0 0x7ff3ac19d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46326/0 (socket says 172.21.15.19:46326) 2024-05-07T02:41:37.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.810+0000 7ff3ab400700 1 -- 172.21.15.19:0/1222105488 learned_addr learned my addr 172.21.15.19:0/1222105488 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:41:37.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.810+0000 7ff3a2a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3ac108f20 0x7ff3ac19d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:45396/0 (socket says 172.21.15.19:45396) 2024-05-07T02:41:37.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.810+0000 7ff3ab400700 1 -- 172.21.15.19:0/1222105488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3ac109870 msgr2=0x7ff3ac1a18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:37.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.810+0000 7ff3ab400700 1 --2- 172.21.15.19:0/1222105488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3ac109870 0x7ff3ac1a18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:37.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.811+0000 7ff3ab400700 1 -- 172.21.15.19:0/1222105488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3ac108f20 msgr2=0x7ff3ac19d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:37.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.811+0000 7ff3ab400700 1 --2- 172.21.15.19:0/1222105488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3ac108f20 0x7ff3ac19d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:37.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.811+0000 7ff3ab400700 1 -- 172.21.15.19:0/1222105488 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3ac1a1f60 con 0x7ff3ac1045f0 2024-05-07T02:41:37.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.811+0000 7ff3a2a00700 1 --2- 172.21.15.19:0/1222105488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3ac108f20 0x7ff3ac19d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:41:37.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.811+0000 7ff3ab400700 1 --2- 172.21.15.19:0/1222105488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3ac1045f0 0x7ff3ac19d090 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff39c00ef10 tx=0x7ff39c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:41:37.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.811+0000 7ff3a8c00700 1 -- 172.21.15.19:0/1222105488 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39c00c9c0 con 0x7ff3ac1045f0 2024-05-07T02:41:37.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.811+0000 7ff3a8c00700 1 -- 172.21.15.19:0/1222105488 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff39c019070 con 0x7ff3ac1045f0 2024-05-07T02:41:37.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.811+0000 7ff3b2400700 1 -- 172.21.15.19:0/1222105488 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3ac1a21f0 con 0x7ff3ac1045f0 2024-05-07T02:41:37.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.811+0000 7ff3a8c00700 1 -- 172.21.15.19:0/1222105488 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39c014480 con 0x7ff3ac1045f0 2024-05-07T02:41:37.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.812+0000 7ff3b2400700 1 -- 172.21.15.19:0/1222105488 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3ac1a9b70 con 0x7ff3ac1045f0 2024-05-07T02:41:37.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.816+0000 7ff3a8c00700 1 -- 172.21.15.19:0/1222105488 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff39c012050 con 0x7ff3ac1045f0 2024-05-07T02:41:37.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.817+0000 7ff3c0f1eb80 1 Processor -- start 2024-05-07T02:41:37.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.817+0000 7ff3c0f1eb80 1 -- start start 2024-05-07T02:41:37.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.817+0000 7ff3c0f1eb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56235bd1c3b0 0x56235bd1c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:37.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.817+0000 7ff3c0f1eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56235bd16470 0x56235bd16880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:37.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.817+0000 7ff3c0f1eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56235bd16dc0 0x56235bd23040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:37.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.817+0000 7ff3c0f1eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56235bc38a20 con 0x56235bd16dc0 2024-05-07T02:41:37.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.817+0000 7ff3c0f1eb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56235bbd6760 con 0x56235bd1c3b0 2024-05-07T02:41:37.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.817+0000 7ff3c0f1eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56235bc349e0 con 0x56235bd16470 2024-05-07T02:41:37.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.818+0000 7ff3a8c00700 1 --2- 172.21.15.19:0/1222105488 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff3900b29f0 0x7ff3900b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:37.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.818+0000 7ff3ab400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56235bd16470 0x56235bd16880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:37.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.818+0000 7ff3ab400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56235bd16470 0x56235bd16880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:50252/0 (socket says 172.21.15.19:50252) 2024-05-07T02:41:37.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.818+0000 7ff3ab400700 1 -- 172.21.15.19:0/358249793 learned_addr learned my addr 172.21.15.19:0/358249793 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:41:37.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.818+0000 7ff3abe00700 1 --2- 172.21.15.19:0/358249793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56235bd1c3b0 0x56235bd1c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:37.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.818+0000 7ff3a8c00700 1 -- 172.21.15.19:0/1222105488 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9525+0+0 (secure 0 0 0) 0x7ff39c0e2530 con 0x7ff3ac1045f0 2024-05-07T02:41:37.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.818+0000 7ff3a2a00700 1 --2- 172.21.15.19:0/358249793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56235bd16dc0 0x56235bd23040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:37.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.818+0000 7ff3abe00700 1 --2- 172.21.15.19:0/1222105488 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff3900b29f0 0x7ff3900b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:37.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.818+0000 7ff3ab400700 1 -- 172.21.15.19:0/358249793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56235bd1c3b0 msgr2=0x56235bd1c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:37.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.818+0000 7ff3ab400700 1 --2- 172.21.15.19:0/358249793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56235bd1c3b0 0x56235bd1c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:37.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.818+0000 7ff3ab400700 1 -- 172.21.15.19:0/358249793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56235bd16dc0 msgr2=0x56235bd23040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:37.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.818+0000 7ff3ab400700 1 --2- 172.21.15.19:0/358249793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56235bd16dc0 0x56235bd23040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:37.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.818+0000 7ff3ab400700 1 -- 172.21.15.19:0/358249793 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56235bb858d0 con 0x56235bd16470 2024-05-07T02:41:37.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.819+0000 7ff3ab400700 1 --2- 172.21.15.19:0/358249793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56235bd16470 0x56235bd16880 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff39c080780 tx=0x7ff39c009eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4358f4757cd01625 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:41:37.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.819+0000 7ff3a0c00700 1 -- 172.21.15.19:0/358249793 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39c0a9ac0 con 0x56235bd16470 2024-05-07T02:41:37.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.819+0000 7ff3abe00700 1 --2- 172.21.15.19:0/1222105488 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff3900b29f0 0x7ff3900b4f00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009990 tx=0x7ff3a4005960 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:41:37.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.819+0000 7ff3a0c00700 1 -- 172.21.15.19:0/358249793 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff39c00cd10 con 0x56235bd16470 2024-05-07T02:41:37.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.819+0000 7ff3a0c00700 1 -- 172.21.15.19:0/358249793 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39c0b1f90 con 0x56235bd16470 2024-05-07T02:41:37.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.820+0000 7ff3c0f1eb80 1 -- 172.21.15.19:0/358249793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56235bd16470 msgr2=0x56235bd16880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:37.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.820+0000 7ff3c0f1eb80 1 --2- 172.21.15.19:0/358249793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56235bd16470 0x56235bd16880 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff39c080780 tx=0x7ff39c009eb0 comp rx=0 tx=0).stop 2024-05-07T02:41:37.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.820+0000 7ff3c0f1eb80 1 -- 172.21.15.19:0/358249793 shutdown_connections 2024-05-07T02:41:37.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.820+0000 7ff3c0f1eb80 1 --2- 172.21.15.19:0/358249793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56235bd16dc0 0x56235bd23040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:37.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.820+0000 7ff3c0f1eb80 1 --2- 172.21.15.19:0/358249793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56235bd1c3b0 0x56235bd1c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:37.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.820+0000 7ff3c0f1eb80 1 --2- 172.21.15.19:0/358249793 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56235bd16470 0x56235bd16880 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:37.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.820+0000 7ff3c0f1eb80 1 -- 172.21.15.19:0/358249793 >> 172.21.15.19:0/358249793 conn(0x56235bb98e40 msgr2=0x56235bd112f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:41:37.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.820+0000 7ff3c0f1eb80 1 -- 172.21.15.19:0/358249793 shutdown_connections 2024-05-07T02:41:37.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.820+0000 7ff3c0f1eb80 1 -- 172.21.15.19:0/358249793 wait complete. 2024-05-07T02:41:37.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.821+0000 7ff3c0f1eb80 1 -- start start 2024-05-07T02:41:37.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.821+0000 7ff3c0f1eb80 1 Processor -- start 2024-05-07T02:41:37.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.822+0000 7ff3c0f1eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56235bd16470 0x56235bd205e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:37.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.822+0000 7ff3c0f1eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56235bd16dc0 0x56235bd20b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:37.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.822+0000 7ff3c0f1eb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56235bd1c3b0 0x56235bd21060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:37.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.822+0000 7ff3c0f1eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56235bd23700 con 0x56235bd16dc0 2024-05-07T02:41:37.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.822+0000 7ff3c0f1eb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56235bb9eb20 con 0x56235bd1c3b0 2024-05-07T02:41:37.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.822+0000 7ff3c0f1eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56235bd23880 con 0x56235bd16470 2024-05-07T02:41:37.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.822+0000 7ff3c0f1eb80 10 client.0 _ll_register_callbacks cb 0x56235bbdfc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:41:37.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.822+0000 7ff3ab400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56235bd16470 0x56235bd205e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:37.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.822+0000 7ff3ab400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56235bd16470 0x56235bd205e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:50264/0 (socket says 172.21.15.19:50264) 2024-05-07T02:41:37.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.822+0000 7ff3ab400700 1 -- 172.21.15.19:0/3147397273 learned_addr learned my addr 172.21.15.19:0/3147397273 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:41:37.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.822+0000 7ff3abe00700 1 --2- 172.21.15.19:0/3147397273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56235bd1c3b0 0x56235bd21060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:37.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.822+0000 7ff3a2a00700 1 --2- 172.21.15.19:0/3147397273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56235bd16dc0 0x56235bd20b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:37.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.823+0000 7ff3ab400700 1 -- 172.21.15.19:0/3147397273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56235bd1c3b0 msgr2=0x56235bd21060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:37.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.823+0000 7ff3ab400700 1 --2- 172.21.15.19:0/3147397273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56235bd1c3b0 0x56235bd21060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:37.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.823+0000 7ff3ab400700 1 -- 172.21.15.19:0/3147397273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56235bd16dc0 msgr2=0x56235bd20b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:37.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.823+0000 7ff3ab400700 1 --2- 172.21.15.19:0/3147397273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56235bd16dc0 0x56235bd20b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:37.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.823+0000 7ff3ab400700 1 -- 172.21.15.19:0/3147397273 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56235bbfe420 con 0x56235bd16470 2024-05-07T02:41:37.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.823+0000 7ff3ab400700 1 --2- 172.21.15.19:0/3147397273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56235bd16470 0x56235bd205e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff39c00eee0 tx=0x7ff39c091080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=347888b758ce24aa server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:41:37.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.823+0000 7ff399600700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:41:37.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.823+0000 7ff399600700 1 -- 172.21.15.19:0/3147397273 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39c0ad770 con 0x56235bd16470 2024-05-07T02:41:37.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.823+0000 7ff3c0f1eb80 1 -- 172.21.15.19:0/3147397273 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff39c00cd10 con 0x56235bd16470 2024-05-07T02:41:37.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.823+0000 7ff399600700 1 -- 172.21.15.19:0/3147397273 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9525+0+0 (secure 0 0 0) 0x7ff39c0a8830 con 0x56235bd16470 2024-05-07T02:41:37.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.824+0000 7ff399600700 1 -- 172.21.15.19:0/3147397273 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7ff39c0a9ac0 con 0x56235bd16470 2024-05-07T02:41:37.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.824+0000 7ff3c0f1eb80 10 client.26602 fetch_fsmap learned FSMap version 50 2024-05-07T02:41:37.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.824+0000 7ff3c0f1eb80 1 -- 172.21.15.19:0/3147397273 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x56235bd23700 con 0x56235bd16470 2024-05-07T02:41:37.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.824+0000 7ff399600700 1 -- 172.21.15.19:0/3147397273 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff39c00cd10 con 0x56235bd16470 2024-05-07T02:41:37.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.824+0000 7ff399600700 1 -- 172.21.15.19:0/3147397273 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 50) ==== 8362+0+0 (secure 0 0 0) 0x7ff39c0a3ae0 con 0x56235bd16470 2024-05-07T02:41:37.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.824+0000 7ff3c0f1eb80 10 client.26602 fetch_fsmap finished waiting for FSMap version 50 2024-05-07T02:41:37.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.824+0000 7ff3c0f1eb80 10 client.26602 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-07T02:41:37.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.824+0000 7ff3c0f1eb80 20 client.26602 populate_metadata read hostname 'smithi019' 2024-05-07T02:41:37.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.824+0000 7ff3c0f1eb80 1 --2- 172.21.15.19:0/3147397273 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x56235bd306d0 0x56235bd32af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:37.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.824+0000 7ff3c0f1eb80 4 client.26602 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:41:37.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.824+0000 7ff3c0f1eb80 1 -- 172.21.15.19:0/3147397273 --> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56235bce32e0 con 0x56235bd306d0 2024-05-07T02:41:37.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.825+0000 7ff3a2a00700 1 --2- 172.21.15.19:0/3147397273 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x56235bd306d0 0x56235bd32af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:37.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.825+0000 7ff3a2a00700 1 --2- 172.21.15.19:0/3147397273 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x56235bd306d0 0x56235bd32af0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b29d0133f902b96f server_cookie=643a4063c7cb3650 in_seq=0 out_seq=0 2024-05-07T02:41:37.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.825+0000 7ff399600700 10 client.26602 ms_handle_connect on v2:172.21.15.64:6836/2585907777 2024-05-07T02:41:37.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.826+0000 7ff399600700 1 -- 172.21.15.19:0/3147397273 <== mds.4 v2:172.21.15.64:6836/2585907777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56235bce32e0 con 0x56235bd306d0 2024-05-07T02:41:37.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.826+0000 7ff399600700 10 client.26602 handle_command_reply: tid=0 2024-05-07T02:41:37.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.826+0000 7ff3c0f1eb80 1 client.26602 shutdown 2024-05-07T02:41:37.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.826+0000 7ff3c0f1eb80 1 -- 172.21.15.19:0/3147397273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56235bd16470 msgr2=0x56235bd205e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:37.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.826+0000 7ff3c0f1eb80 1 --2- 172.21.15.19:0/3147397273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56235bd16470 0x56235bd205e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff39c00eee0 tx=0x7ff39c091080 comp rx=0 tx=0).stop 2024-05-07T02:41:37.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.826+0000 7ff3c0f1eb80 1 -- 172.21.15.19:0/3147397273 shutdown_connections 2024-05-07T02:41:37.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.827+0000 7ff3c0f1eb80 1 --2- 172.21.15.19:0/3147397273 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x56235bd306d0 0x56235bd32af0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:37.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.827+0000 7ff3c0f1eb80 1 --2- 172.21.15.19:0/3147397273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56235bd1c3b0 0x56235bd21060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:37.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.827+0000 7ff399600700 0 client.26602 ms_handle_reset on v2:172.21.15.64:6836/2585907777 2024-05-07T02:41:37.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.827+0000 7ff3c0f1eb80 1 --2- 172.21.15.19:0/3147397273 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56235bd16470 0x56235bd205e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:37.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.827+0000 7ff3c0f1eb80 1 --2- 172.21.15.19:0/3147397273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56235bd16dc0 0x56235bd20b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:37.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.827+0000 7ff3c0f1eb80 1 -- 172.21.15.19:0/3147397273 >> 172.21.15.19:0/3147397273 conn(0x56235bb98e40 msgr2=0x56235bd15460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:41:37.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.827+0000 7ff3c0f1eb80 1 -- 172.21.15.19:0/3147397273 shutdown_connections 2024-05-07T02:41:37.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.827+0000 7ff3c0f1eb80 1 -- 172.21.15.19:0/3147397273 wait complete. 2024-05-07T02:41:37.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.827+0000 7ff3c0f1eb80 20 client.26602 trim_cache size 0 max 16384 2024-05-07T02:41:37.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.844+0000 7ff3c0f1eb80 1 Processor -- start 2024-05-07T02:41:37.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.844+0000 7ff3c0f1eb80 1 -- start start 2024-05-07T02:41:37.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.844+0000 7ff3c0f1eb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56235bd16470 0x56235bd1acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:37.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.845+0000 7ff3c0f1eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56235bd16dc0 0x56235bd1f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:37.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.845+0000 7ff3c0f1eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56235bd1c3b0 0x56235bd1de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:37.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.845+0000 7ff3c0f1eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac1b29e0 con 0x56235bd1c3b0 2024-05-07T02:41:37.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.845+0000 7ff3c0f1eb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac1b4520 con 0x56235bd16470 2024-05-07T02:41:37.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.845+0000 7ff3c0f1eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac1b30b0 con 0x56235bd16dc0 2024-05-07T02:41:37.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.845+0000 7ff3ab400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56235bd16470 0x56235bd1acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:37.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.845+0000 7ff3abe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56235bd1c3b0 0x56235bd1de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:37.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.845+0000 7ff3ab400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56235bd16470 0x56235bd1acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:45416/0 (socket says 172.21.15.19:45416) 2024-05-07T02:41:37.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.845+0000 7ff3ab400700 1 -- 172.21.15.19:0/444215847 learned_addr learned my addr 172.21.15.19:0/444215847 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:41:37.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.845+0000 7ff3a2a00700 1 --2- 172.21.15.19:0/444215847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56235bd16dc0 0x56235bd1f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:37.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.845+0000 7ff3ab400700 1 -- 172.21.15.19:0/444215847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56235bd16dc0 msgr2=0x56235bd1f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:37.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.845+0000 7ff3ab400700 1 --2- 172.21.15.19:0/444215847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56235bd16dc0 0x56235bd1f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:37.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.845+0000 7ff3ab400700 1 -- 172.21.15.19:0/444215847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56235bd1c3b0 msgr2=0x56235bd1de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:37.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.845+0000 7ff3ab400700 1 --2- 172.21.15.19:0/444215847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56235bd1c3b0 0x56235bd1de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:37.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.845+0000 7ff3ab400700 1 -- 172.21.15.19:0/444215847 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56235bd11830 con 0x56235bd16470 2024-05-07T02:41:37.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.846+0000 7ff3a2a00700 1 --2- 172.21.15.19:0/444215847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56235bd16dc0 0x56235bd1f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:41:37.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.846+0000 7ff3ab400700 1 --2- 172.21.15.19:0/444215847 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56235bd16470 0x56235bd1acd0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff39c0a90c0 tx=0x7ff39c009eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50681a5f8397d72e server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:41:37.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.846+0000 7ff39be00700 1 -- 172.21.15.19:0/444215847 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39c0a50b0 con 0x56235bd16470 2024-05-07T02:41:37.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.846+0000 7ff39be00700 1 -- 172.21.15.19:0/444215847 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff39c00cd10 con 0x56235bd16470 2024-05-07T02:41:37.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.846+0000 7ff39be00700 1 -- 172.21.15.19:0/444215847 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39c0a8b30 con 0x56235bd16470 2024-05-07T02:41:37.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.847+0000 7ff3c0f1eb80 1 -- 172.21.15.19:0/444215847 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56235bd16470 msgr2=0x56235bd1acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:37.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.847+0000 7ff3c0f1eb80 1 --2- 172.21.15.19:0/444215847 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56235bd16470 0x56235bd1acd0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff39c0a90c0 tx=0x7ff39c009eb0 comp rx=0 tx=0).stop 2024-05-07T02:41:37.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.847+0000 7ff3c0f1eb80 1 -- 172.21.15.19:0/444215847 shutdown_connections 2024-05-07T02:41:37.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.847+0000 7ff3c0f1eb80 1 --2- 172.21.15.19:0/444215847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56235bd1c3b0 0x56235bd1de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:37.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.847+0000 7ff3c0f1eb80 1 --2- 172.21.15.19:0/444215847 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56235bd16470 0x56235bd1acd0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:37.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.847+0000 7ff3c0f1eb80 1 --2- 172.21.15.19:0/444215847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56235bd16dc0 0x56235bd1f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:37.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.847+0000 7ff3c0f1eb80 1 -- 172.21.15.19:0/444215847 >> 172.21.15.19:0/444215847 conn(0x56235bb98e40 msgr2=0x56235bd1eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:41:37.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.847+0000 7ff3c0f1eb80 1 -- 172.21.15.19:0/444215847 shutdown_connections 2024-05-07T02:41:37.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.847+0000 7ff3c0f1eb80 1 -- 172.21.15.19:0/444215847 wait complete. 2024-05-07T02:41:37.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.848+0000 7ff3c0f1eb80 1 -- start start 2024-05-07T02:41:37.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.848+0000 7ff3c0f1eb80 1 Processor -- start 2024-05-07T02:41:37.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.849+0000 7ff3c0f1eb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56235bd16470 0x56235bd28d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:37.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.849+0000 7ff3c0f1eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56235bd16dc0 0x56235bd37200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:37.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.849+0000 7ff3c0f1eb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56235bd1c3b0 0x56235bd37740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:37.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.849+0000 7ff3c0f1eb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac1b3e50 con 0x56235bd1c3b0 2024-05-07T02:41:37.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.849+0000 7ff3c0f1eb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac1b3780 con 0x56235bd16470 2024-05-07T02:41:37.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.849+0000 7ff3c0f1eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac1b09a0 con 0x56235bd16dc0 2024-05-07T02:41:37.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.849+0000 7ff3c0f1eb80 10 client.0 _ll_register_callbacks cb 0x56235bbdfc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:41:37.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.849+0000 7ff3a2a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56235bd16dc0 0x56235bd37200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:37.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.849+0000 7ff3a2a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56235bd16dc0 0x56235bd37200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:50272/0 (socket says 172.21.15.19:50272) 2024-05-07T02:41:37.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.849+0000 7ff3a2a00700 1 -- 172.21.15.19:0/1423658504 learned_addr learned my addr 172.21.15.19:0/1423658504 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:41:37.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.849+0000 7ff3ab400700 1 --2- 172.21.15.19:0/1423658504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56235bd16470 0x56235bd28d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:37.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.849+0000 7ff3abe00700 1 --2- 172.21.15.19:0/1423658504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56235bd1c3b0 0x56235bd37740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:37.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.849+0000 7ff3a2a00700 1 -- 172.21.15.19:0/1423658504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56235bd16470 msgr2=0x56235bd28d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:37.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.849+0000 7ff3a2a00700 1 --2- 172.21.15.19:0/1423658504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56235bd16470 0x56235bd28d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:37.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.849+0000 7ff3a2a00700 1 -- 172.21.15.19:0/1423658504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56235bd1c3b0 msgr2=0x56235bd37740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:37.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.850+0000 7ff3a2a00700 1 --2- 172.21.15.19:0/1423658504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56235bd1c3b0 0x56235bd37740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:37.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.850+0000 7ff3a2a00700 1 -- 172.21.15.19:0/1423658504 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56235bd37c80 con 0x56235bd16dc0 2024-05-07T02:41:37.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.850+0000 7ff3ab400700 1 --2- 172.21.15.19:0/1423658504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56235bd16470 0x56235bd28d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:41:37.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.850+0000 7ff3a2a00700 1 --2- 172.21.15.19:0/1423658504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56235bd16dc0 0x56235bd37200 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac10a880 tx=0x7ff394004320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83cd6e79b8ec6e5c server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:41:37.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.850+0000 7ff399600700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:41:37.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.850+0000 7ff399600700 1 -- 172.21.15.19:0/1423658504 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff394012e20 con 0x56235bd16dc0 2024-05-07T02:41:37.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.850+0000 7ff3c0f1eb80 1 -- 172.21.15.19:0/1423658504 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff39c00cd10 con 0x56235bd16dc0 2024-05-07T02:41:37.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.850+0000 7ff399600700 1 -- 172.21.15.19:0/1423658504 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9525+0+0 (secure 0 0 0) 0x7ff394004830 con 0x56235bd16dc0 2024-05-07T02:41:37.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.851+0000 7ff399600700 1 -- 172.21.15.19:0/1423658504 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7ff394004a30 con 0x56235bd16dc0 2024-05-07T02:41:37.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.851+0000 7ff3c0f1eb80 10 client.26608 fetch_fsmap learned FSMap version 50 2024-05-07T02:41:37.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.851+0000 7ff3c0f1eb80 1 -- 172.21.15.19:0/1423658504 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x56235bd9a800 con 0x56235bd16dc0 2024-05-07T02:41:37.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.851+0000 7ff399600700 1 -- 172.21.15.19:0/1423658504 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff39c00cd10 con 0x56235bd16dc0 2024-05-07T02:41:37.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.851+0000 7ff399600700 1 -- 172.21.15.19:0/1423658504 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 50) ==== 8362+0+0 (secure 0 0 0) 0x7ff394003ba0 con 0x56235bd16dc0 2024-05-07T02:41:37.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.851+0000 7ff3c0f1eb80 10 client.26608 fetch_fsmap finished waiting for FSMap version 50 2024-05-07T02:41:37.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.851+0000 7ff3c0f1eb80 10 client.26608 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-07T02:41:37.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.851+0000 7ff3c0f1eb80 20 client.26608 populate_metadata read hostname 'smithi019' 2024-05-07T02:41:37.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.851+0000 7ff3c0f1eb80 1 --2- 172.21.15.19:0/1423658504 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x56235bd306d0 0x56235bd9cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:37.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.851+0000 7ff3c0f1eb80 4 client.26608 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:41:37.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.851+0000 7ff3c0f1eb80 1 -- 172.21.15.19:0/1423658504 --> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56235bc46400 con 0x56235bd306d0 2024-05-07T02:41:37.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.852+0000 7ff3ab400700 1 --2- 172.21.15.19:0/1423658504 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x56235bd306d0 0x56235bd9cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:37.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.853+0000 7ff399600700 10 client.26608 ms_handle_connect on v2:172.21.15.64:6836/2585907777 2024-05-07T02:41:37.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.852+0000 7ff3ab400700 1 --2- 172.21.15.19:0/1423658504 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x56235bd306d0 0x56235bd9cf30 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d3d745e965041bc8 server_cookie=6d4aa16048287e19 in_seq=0 out_seq=0 2024-05-07T02:41:37.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.853+0000 7ff399600700 1 -- 172.21.15.19:0/1423658504 <== mds.4 v2:172.21.15.64:6836/2585907777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56235bc46400 con 0x56235bd306d0 2024-05-07T02:41:37.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.853+0000 7ff399600700 10 client.26608 handle_command_reply: tid=0 2024-05-07T02:41:37.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.853+0000 7ff3c0f1eb80 1 client.26608 shutdown 2024-05-07T02:41:37.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.853+0000 7ff3c0f1eb80 1 -- 172.21.15.19:0/1423658504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56235bd16dc0 msgr2=0x56235bd37200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:37.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.853+0000 7ff3c0f1eb80 1 --2- 172.21.15.19:0/1423658504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56235bd16dc0 0x56235bd37200 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac10a880 tx=0x7ff394004320 comp rx=0 tx=0).stop 2024-05-07T02:41:37.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.853+0000 7ff3c0f1eb80 1 -- 172.21.15.19:0/1423658504 shutdown_connections 2024-05-07T02:41:37.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.853+0000 7ff3c0f1eb80 1 --2- 172.21.15.19:0/1423658504 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x56235bd306d0 0x56235bd9cf30 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:37.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.853+0000 7ff3c0f1eb80 1 --2- 172.21.15.19:0/1423658504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56235bd1c3b0 0x56235bd37740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:37.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.853+0000 7ff3c0f1eb80 1 --2- 172.21.15.19:0/1423658504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56235bd16470 0x56235bd28d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:37.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.853+0000 7ff3c0f1eb80 1 --2- 172.21.15.19:0/1423658504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56235bd16dc0 0x56235bd37200 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:37.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.853+0000 7ff399600700 0 client.26608 ms_handle_reset on v2:172.21.15.64:6836/2585907777 2024-05-07T02:41:37.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.853+0000 7ff3c0f1eb80 1 -- 172.21.15.19:0/1423658504 >> 172.21.15.19:0/1423658504 conn(0x56235bb98e40 msgr2=0x56235bd0b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:41:37.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.853+0000 7ff3c0f1eb80 1 -- 172.21.15.19:0/1423658504 shutdown_connections 2024-05-07T02:41:37.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.854+0000 7ff3c0f1eb80 1 -- 172.21.15.19:0/1423658504 wait complete. 2024-05-07T02:41:37.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.854+0000 7ff3c0f1eb80 20 client.26608 trim_cache size 0 max 16384 2024-05-07T02:41:37.864 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:41:37.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.854+0000 7ff3b2400700 1 -- 172.21.15.19:0/1222105488 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff3900b29f0 msgr2=0x7ff3900b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:37.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.854+0000 7ff3b2400700 1 --2- 172.21.15.19:0/1222105488 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff3900b29f0 0x7ff3900b4f00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009990 tx=0x7ff3a4005960 comp rx=0 tx=0).stop 2024-05-07T02:41:37.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.854+0000 7ff3b2400700 1 -- 172.21.15.19:0/1222105488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3ac1045f0 msgr2=0x7ff3ac19d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:37.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.854+0000 7ff3b2400700 1 --2- 172.21.15.19:0/1222105488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3ac1045f0 0x7ff3ac19d090 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff39c00ef10 tx=0x7ff39c00c3a0 comp rx=0 tx=0).stop 2024-05-07T02:41:37.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.855+0000 7ff3b2400700 1 -- 172.21.15.19:0/1222105488 shutdown_connections 2024-05-07T02:41:37.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.855+0000 7ff3b2400700 1 --2- 172.21.15.19:0/1222105488 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff3ac109870 0x7ff3ac1a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:37.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.855+0000 7ff3b2400700 1 --2- 172.21.15.19:0/1222105488 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff3900b29f0 0x7ff3900b4f00 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:37.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.855+0000 7ff3b2400700 1 --2- 172.21.15.19:0/1222105488 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3ac1045f0 0x7ff3ac19d090 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:37.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.855+0000 7ff3b2400700 1 --2- 172.21.15.19:0/1222105488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3ac108f20 0x7ff3ac19d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:37.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.855+0000 7ff3b2400700 1 -- 172.21.15.19:0/1222105488 >> 172.21.15.19:0/1222105488 conn(0x7ff3ac100040 msgr2=0x7ff3ac110d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:41:37.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.855+0000 7ff3b2400700 1 -- 172.21.15.19:0/1222105488 shutdown_connections 2024-05-07T02:41:37.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:37.855+0000 7ff3b2400700 1 -- 172.21.15.19:0/1222105488 wait complete. 2024-05-07T02:41:38.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:37 smithi064 ceph-mon[34699]: pgmap v302: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:41:38.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:37 smithi027 ceph-mon[34398]: pgmap v302: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:41:38.751 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-07T02:41:39.232 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:41:40.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:39 smithi064 ceph-mon[34699]: pgmap v303: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:41:40.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:39 smithi019 ceph-mon[29781]: pgmap v303: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:41:40.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:39 smithi027 ceph-mon[34398]: pgmap v303: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:41:41.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.723+0000 7fea8e600700 1 -- 172.21.15.19:0/2049934531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea88109850 msgr2=0x7fea881111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:41.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.723+0000 7fea8e600700 1 --2- 172.21.15.19:0/2049934531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea88109850 0x7fea881111b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fea8000b0d0 tx=0x7fea8000b3e0 comp rx=0 tx=0).stop 2024-05-07T02:41:41.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.723+0000 7fea8e600700 1 -- 172.21.15.19:0/2049934531 shutdown_connections 2024-05-07T02:41:41.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.723+0000 7fea8e600700 1 --2- 172.21.15.19:0/2049934531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea88109850 0x7fea881111b0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:41.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.723+0000 7fea8e600700 1 --2- 172.21.15.19:0/2049934531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea881045d0 0x7fea881049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:41.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.723+0000 7fea8e600700 1 --2- 172.21.15.19:0/2049934531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea88108f00 0x7fea88109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:41.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.723+0000 7fea8e600700 1 -- 172.21.15.19:0/2049934531 >> 172.21.15.19:0/2049934531 conn(0x7fea880ffea0 msgr2=0x7fea881022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:41:41.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.723+0000 7fea8e600700 1 -- 172.21.15.19:0/2049934531 shutdown_connections 2024-05-07T02:41:41.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.723+0000 7fea8e600700 1 -- 172.21.15.19:0/2049934531 wait complete. 2024-05-07T02:41:41.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.725+0000 7fea8e600700 1 Processor -- start 2024-05-07T02:41:41.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.725+0000 7fea8e600700 1 -- start start 2024-05-07T02:41:41.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.725+0000 7fea8e600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea881045d0 0x7fea8819add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:41.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.725+0000 7fea8e600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea88108f00 0x7fea8819b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:41.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.725+0000 7fea8e600700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea88109850 0x7fea8819f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:41.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.725+0000 7fea8e600700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fea88114480 con 0x7fea88108f00 2024-05-07T02:41:41.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.725+0000 7fea8e600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fea88114300 con 0x7fea881045d0 2024-05-07T02:41:41.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.725+0000 7fea8e600700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fea88114600 con 0x7fea88109850 2024-05-07T02:41:41.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.726+0000 7fea7fe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea88108f00 0x7fea8819b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:41.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.726+0000 7fea87400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea881045d0 0x7fea8819add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:41.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.726+0000 7fea87400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea881045d0 0x7fea8819add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:57472/0 (socket says 172.21.15.19:57472) 2024-05-07T02:41:41.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.726+0000 7fea87400700 1 -- 172.21.15.19:0/2140939669 learned_addr learned my addr 172.21.15.19:0/2140939669 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:41:41.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.726+0000 7fea7fe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea88108f00 0x7fea8819b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37050/0 (socket says 172.21.15.19:37050) 2024-05-07T02:41:41.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.726+0000 7fea87e00700 1 --2- 172.21.15.19:0/2140939669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea88109850 0x7fea8819f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:41.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.726+0000 7fea7fe00700 1 -- 172.21.15.19:0/2140939669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea88109850 msgr2=0x7fea8819f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:41.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.726+0000 7fea7fe00700 1 --2- 172.21.15.19:0/2140939669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea88109850 0x7fea8819f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:41.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.726+0000 7fea7fe00700 1 -- 172.21.15.19:0/2140939669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea881045d0 msgr2=0x7fea8819add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:41.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.726+0000 7fea7fe00700 1 --2- 172.21.15.19:0/2140939669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea881045d0 0x7fea8819add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:41.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.726+0000 7fea7fe00700 1 -- 172.21.15.19:0/2140939669 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea8819fca0 con 0x7fea88108f00 2024-05-07T02:41:41.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.727+0000 7fea7fe00700 1 --2- 172.21.15.19:0/2140939669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea88108f00 0x7fea8819b370 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fea7000e9c0 tx=0x7fea7000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:41:41.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.727+0000 7fea84c00700 1 -- 172.21.15.19:0/2140939669 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7000c9c0 con 0x7fea88108f00 2024-05-07T02:41:41.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.727+0000 7fea84c00700 1 -- 172.21.15.19:0/2140939669 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea7000cb60 con 0x7fea88108f00 2024-05-07T02:41:41.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.727+0000 7fea8e600700 1 -- 172.21.15.19:0/2140939669 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea8819ff90 con 0x7fea88108f00 2024-05-07T02:41:41.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.727+0000 7fea84c00700 1 -- 172.21.15.19:0/2140939669 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea70018680 con 0x7fea88108f00 2024-05-07T02:41:41.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.727+0000 7fea8e600700 1 -- 172.21.15.19:0/2140939669 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea881a78b0 con 0x7fea88108f00 2024-05-07T02:41:41.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.733+0000 7fea84c00700 1 -- 172.21.15.19:0/2140939669 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fea70010430 con 0x7fea88108f00 2024-05-07T02:41:41.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.733+0000 7fea9d1c0b80 1 Processor -- start 2024-05-07T02:41:41.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.734+0000 7fea9d1c0b80 1 -- start start 2024-05-07T02:41:41.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.734+0000 7fea9d1c0b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564cd2ebb3b0 0x564cd2ebb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:41.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.734+0000 7fea9d1c0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564cd2eb5470 0x564cd2eb5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:41.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.734+0000 7fea9d1c0b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564cd2eb5dc0 0x564cd2ec2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:41.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.734+0000 7fea9d1c0b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564cd2dd7a20 con 0x564cd2eb5470 2024-05-07T02:41:41.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.734+0000 7fea9d1c0b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x564cd2d75760 con 0x564cd2eb5dc0 2024-05-07T02:41:41.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.734+0000 7fea9d1c0b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564cd2dd39e0 con 0x564cd2ebb3b0 2024-05-07T02:41:41.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.734+0000 7fea87400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564cd2eb5470 0x564cd2eb5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:41.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.735+0000 7fea7fe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564cd2eb5dc0 0x564cd2ec2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:41.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.735+0000 7fea87400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564cd2eb5470 0x564cd2eb5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37056/0 (socket says 172.21.15.19:37056) 2024-05-07T02:41:41.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.735+0000 7fea87400700 1 -- 172.21.15.19:0/3013439609 learned_addr learned my addr 172.21.15.19:0/3013439609 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:41:41.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.735+0000 7fea87e00700 1 --2- 172.21.15.19:0/3013439609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564cd2ebb3b0 0x564cd2ebb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:41.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.735+0000 7fea84c00700 1 --2- 172.21.15.19:0/2140939669 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fea6c0b29f0 0x7fea6c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:41.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.735+0000 7fea87400700 1 -- 172.21.15.19:0/3013439609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564cd2ebb3b0 msgr2=0x564cd2ebb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:41.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.735+0000 7fea87400700 1 --2- 172.21.15.19:0/3013439609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564cd2ebb3b0 0x564cd2ebb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:41.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.735+0000 7fea87400700 1 -- 172.21.15.19:0/3013439609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564cd2eb5dc0 msgr2=0x564cd2ec2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:41.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.735+0000 7fea87400700 1 --2- 172.21.15.19:0/3013439609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564cd2eb5dc0 0x564cd2ec2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:41.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.735+0000 7fea87400700 1 -- 172.21.15.19:0/3013439609 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cd2d248d0 con 0x564cd2eb5470 2024-05-07T02:41:41.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.735+0000 7fea84c00700 1 -- 172.21.15.19:0/2140939669 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9525+0+0 (secure 0 0 0) 0x7fea700e2b10 con 0x7fea88108f00 2024-05-07T02:41:41.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.735+0000 7fea87e00700 1 --2- 172.21.15.19:0/2140939669 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fea6c0b29f0 0x7fea6c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:41.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.736+0000 7fea87400700 1 --2- 172.21.15.19:0/3013439609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564cd2eb5470 0x564cd2eb5880 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fea78009e50 tx=0x7fea78011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9534a605122ef109 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:41:41.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.736+0000 7fea7cc00700 1 -- 172.21.15.19:0/3013439609 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea78004a30 con 0x564cd2eb5470 2024-05-07T02:41:41.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.736+0000 7fea7cc00700 1 -- 172.21.15.19:0/3013439609 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea780054d0 con 0x564cd2eb5470 2024-05-07T02:41:41.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.736+0000 7fea7cc00700 1 -- 172.21.15.19:0/3013439609 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea78019990 con 0x564cd2eb5470 2024-05-07T02:41:41.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.737+0000 7fea9d1c0b80 1 -- 172.21.15.19:0/3013439609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564cd2eb5470 msgr2=0x564cd2eb5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:41.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.737+0000 7fea9d1c0b80 1 --2- 172.21.15.19:0/3013439609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564cd2eb5470 0x564cd2eb5880 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fea78009e50 tx=0x7fea78011720 comp rx=0 tx=0).stop 2024-05-07T02:41:41.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.737+0000 7fea87e00700 1 --2- 172.21.15.19:0/2140939669 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fea6c0b29f0 0x7fea6c0b4f00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x564cd2ebae70 tx=0x7fea80005040 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:41:41.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.737+0000 7fea9d1c0b80 1 -- 172.21.15.19:0/3013439609 shutdown_connections 2024-05-07T02:41:41.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.737+0000 7fea9d1c0b80 1 --2- 172.21.15.19:0/3013439609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564cd2eb5dc0 0x564cd2ec2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:41.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.737+0000 7fea9d1c0b80 1 --2- 172.21.15.19:0/3013439609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564cd2ebb3b0 0x564cd2ebb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:41.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.737+0000 7fea9d1c0b80 1 --2- 172.21.15.19:0/3013439609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564cd2eb5470 0x564cd2eb5880 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:41.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.737+0000 7fea9d1c0b80 1 -- 172.21.15.19:0/3013439609 >> 172.21.15.19:0/3013439609 conn(0x564cd2d37e40 msgr2=0x564cd2eb02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:41:41.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.737+0000 7fea9d1c0b80 1 -- 172.21.15.19:0/3013439609 shutdown_connections 2024-05-07T02:41:41.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.737+0000 7fea9d1c0b80 1 -- 172.21.15.19:0/3013439609 wait complete. 2024-05-07T02:41:41.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.738+0000 7fea9d1c0b80 1 -- start start 2024-05-07T02:41:41.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.739+0000 7fea9d1c0b80 1 Processor -- start 2024-05-07T02:41:41.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.739+0000 7fea9d1c0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564cd2eb5470 0x564cd2ebf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:41.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.739+0000 7fea9d1c0b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564cd2eb5dc0 0x564cd2ebf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:41.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.739+0000 7fea9d1c0b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564cd2ebb3b0 0x564cd2ebff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:41.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.739+0000 7fea9d1c0b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564cd2ec27c0 con 0x564cd2eb5470 2024-05-07T02:41:41.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.739+0000 7fea9d1c0b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x564cd2d3db20 con 0x564cd2ebb3b0 2024-05-07T02:41:41.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.739+0000 7fea9d1c0b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564cd2ec2940 con 0x564cd2eb5dc0 2024-05-07T02:41:41.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.739+0000 7fea9d1c0b80 10 client.0 _ll_register_callbacks cb 0x564cd2d7ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:41:41.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.740+0000 7fea87400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564cd2eb5470 0x564cd2ebf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:41.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.740+0000 7fea87e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564cd2ebb3b0 0x564cd2ebff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:41.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.740+0000 7fea87400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564cd2eb5470 0x564cd2ebf4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37070/0 (socket says 172.21.15.19:37070) 2024-05-07T02:41:41.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.740+0000 7fea87400700 1 -- 172.21.15.19:0/3546136874 learned_addr learned my addr 172.21.15.19:0/3546136874 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:41:41.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.740+0000 7fea7fe00700 1 --2- 172.21.15.19:0/3546136874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564cd2eb5dc0 0x564cd2ebf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:41.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.740+0000 7fea7fe00700 1 -- 172.21.15.19:0/3546136874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564cd2ebb3b0 msgr2=0x564cd2ebff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:41.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.740+0000 7fea7fe00700 1 --2- 172.21.15.19:0/3546136874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564cd2ebb3b0 0x564cd2ebff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:41.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.740+0000 7fea7fe00700 1 -- 172.21.15.19:0/3546136874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564cd2eb5470 msgr2=0x564cd2ebf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:41.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.740+0000 7fea7fe00700 1 --2- 172.21.15.19:0/3546136874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564cd2eb5470 0x564cd2ebf4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:41.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.740+0000 7fea7fe00700 1 -- 172.21.15.19:0/3546136874 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cd2d9d420 con 0x564cd2eb5dc0 2024-05-07T02:41:41.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.740+0000 7fea87400700 1 --2- 172.21.15.19:0/3546136874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564cd2eb5470 0x564cd2ebf4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:41:41.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.740+0000 7fea7fe00700 1 --2- 172.21.15.19:0/3546136874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564cd2eb5dc0 0x564cd2ebf9e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fea700dfe70 tx=0x7fea700a9f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cfa3c708d43bb74 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:41:41.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.740+0000 7fea75600700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:41:41.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.741+0000 7fea75600700 1 -- 172.21.15.19:0/3546136874 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea700a58c0 con 0x564cd2eb5dc0 2024-05-07T02:41:41.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.741+0000 7fea9d1c0b80 1 -- 172.21.15.19:0/3546136874 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea780054d0 con 0x564cd2eb5dc0 2024-05-07T02:41:41.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.741+0000 7fea75600700 1 -- 172.21.15.19:0/3546136874 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9525+0+0 (secure 0 0 0) 0x7fea70010720 con 0x564cd2eb5dc0 2024-05-07T02:41:41.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.741+0000 7fea75600700 1 -- 172.21.15.19:0/3546136874 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fea700a9170 con 0x564cd2eb5dc0 2024-05-07T02:41:41.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.741+0000 7fea9d1c0b80 10 client.26620 fetch_fsmap learned FSMap version 50 2024-05-07T02:41:41.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.741+0000 7fea9d1c0b80 1 -- 172.21.15.19:0/3546136874 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x564cd2ecd340 con 0x564cd2eb5dc0 2024-05-07T02:41:41.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.741+0000 7fea75600700 1 -- 172.21.15.19:0/3546136874 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea780054d0 con 0x564cd2eb5dc0 2024-05-07T02:41:41.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.741+0000 7fea75600700 1 -- 172.21.15.19:0/3546136874 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 50) ==== 8362+0+0 (secure 0 0 0) 0x7fea700a5370 con 0x564cd2eb5dc0 2024-05-07T02:41:41.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.742+0000 7fea9d1c0b80 10 client.26620 fetch_fsmap finished waiting for FSMap version 50 2024-05-07T02:41:41.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.742+0000 7fea9d1c0b80 10 client.26620 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-05-07T02:41:41.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.742+0000 7fea9d1c0b80 20 client.26620 populate_metadata read hostname 'smithi019' 2024-05-07T02:41:41.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.742+0000 7fea9d1c0b80 1 --2- 172.21.15.19:0/3546136874 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x564cd2ecf8d0 0x564cd2ed1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:41.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.742+0000 7fea9d1c0b80 4 client.26620 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:41:41.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.742+0000 7fea9d1c0b80 1 -- 172.21.15.19:0/3546136874 --> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564cd2e822e0 con 0x564cd2ecf8d0 2024-05-07T02:41:41.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.742+0000 7fea87400700 1 --2- 172.21.15.19:0/3546136874 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x564cd2ecf8d0 0x564cd2ed1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:41.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.743+0000 7fea87400700 1 --2- 172.21.15.19:0/3546136874 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x564cd2ecf8d0 0x564cd2ed1cf0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=19637573bbdaab3a server_cookie=7577f0a9d4d44fe1 in_seq=0 out_seq=0 2024-05-07T02:41:41.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.743+0000 7fea75600700 10 client.26620 ms_handle_connect on v2:172.21.15.27:6836/2583138143 2024-05-07T02:41:41.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.744+0000 7fea75600700 1 -- 172.21.15.19:0/3546136874 <== mds.3 v2:172.21.15.27:6836/2583138143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564cd2e822e0 con 0x564cd2ecf8d0 2024-05-07T02:41:41.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.744+0000 7fea75600700 10 client.26620 handle_command_reply: tid=0 2024-05-07T02:41:41.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.744+0000 7fea9d1c0b80 1 client.26620 shutdown 2024-05-07T02:41:41.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.744+0000 7fea9d1c0b80 1 -- 172.21.15.19:0/3546136874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564cd2eb5dc0 msgr2=0x564cd2ebf9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:41.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.744+0000 7fea9d1c0b80 1 --2- 172.21.15.19:0/3546136874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564cd2eb5dc0 0x564cd2ebf9e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fea700dfe70 tx=0x7fea700a9f40 comp rx=0 tx=0).stop 2024-05-07T02:41:41.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.744+0000 7fea9d1c0b80 1 -- 172.21.15.19:0/3546136874 shutdown_connections 2024-05-07T02:41:41.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.744+0000 7fea9d1c0b80 1 --2- 172.21.15.19:0/3546136874 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x564cd2ecf8d0 0x564cd2ed1cf0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:41.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.744+0000 7fea9d1c0b80 1 --2- 172.21.15.19:0/3546136874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564cd2ebb3b0 0x564cd2ebff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:41.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.744+0000 7fea9d1c0b80 1 --2- 172.21.15.19:0/3546136874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564cd2eb5470 0x564cd2ebf4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:41.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.744+0000 7fea9d1c0b80 1 --2- 172.21.15.19:0/3546136874 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564cd2eb5dc0 0x564cd2ebf9e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:41.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.744+0000 7fea75600700 0 client.26620 ms_handle_reset on v2:172.21.15.27:6836/2583138143 2024-05-07T02:41:41.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.744+0000 7fea9d1c0b80 1 -- 172.21.15.19:0/3546136874 >> 172.21.15.19:0/3546136874 conn(0x564cd2d37e40 msgr2=0x564cd2eb4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:41:41.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.745+0000 7fea9d1c0b80 1 -- 172.21.15.19:0/3546136874 shutdown_connections 2024-05-07T02:41:41.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.745+0000 7fea9d1c0b80 1 -- 172.21.15.19:0/3546136874 wait complete. 2024-05-07T02:41:41.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.745+0000 7fea9d1c0b80 20 client.26620 trim_cache size 0 max 16384 2024-05-07T02:41:41.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.761+0000 7fea9d1c0b80 1 Processor -- start 2024-05-07T02:41:41.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.762+0000 7fea9d1c0b80 1 -- start start 2024-05-07T02:41:41.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.762+0000 7fea9d1c0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564cd2eb5470 0x564cd2ebcf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:41.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.762+0000 7fea9d1c0b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564cd2eb5dc0 0x564cd2ecb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:41.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.762+0000 7fea9d1c0b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564cd2ebb3b0 0x564cd2ed74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:41.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.762+0000 7fea9d1c0b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fea881b0790 con 0x564cd2eb5470 2024-05-07T02:41:41.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.762+0000 7fea9d1c0b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fea881b22d0 con 0x564cd2ebb3b0 2024-05-07T02:41:41.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.762+0000 7fea9d1c0b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fea881b0e60 con 0x564cd2eb5dc0 2024-05-07T02:41:41.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.762+0000 7fea87400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564cd2eb5470 0x564cd2ebcf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:41.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.762+0000 7fea87400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564cd2eb5470 0x564cd2ebcf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37074/0 (socket says 172.21.15.19:37074) 2024-05-07T02:41:41.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.762+0000 7fea87400700 1 -- 172.21.15.19:0/4208299122 learned_addr learned my addr 172.21.15.19:0/4208299122 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:41:41.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.762+0000 7fea7fe00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564cd2eb5dc0 0x564cd2ecb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:41.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.762+0000 7fea87e00700 1 --2- 172.21.15.19:0/4208299122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564cd2ebb3b0 0x564cd2ed74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:41.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.763+0000 7fea87400700 1 -- 172.21.15.19:0/4208299122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564cd2eb5dc0 msgr2=0x564cd2ecb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:41.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.763+0000 7fea87400700 1 --2- 172.21.15.19:0/4208299122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564cd2eb5dc0 0x564cd2ecb850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:41.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.763+0000 7fea87400700 1 -- 172.21.15.19:0/4208299122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564cd2ebb3b0 msgr2=0x564cd2ed74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:41.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.763+0000 7fea87400700 1 --2- 172.21.15.19:0/4208299122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564cd2ebb3b0 0x564cd2ed74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:41.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.763+0000 7fea87400700 1 -- 172.21.15.19:0/4208299122 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cd2eb1180 con 0x564cd2eb5470 2024-05-07T02:41:41.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.763+0000 7fea87e00700 1 --2- 172.21.15.19:0/4208299122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564cd2ebb3b0 0x564cd2ed74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:41:41.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.764+0000 7fea87400700 1 --2- 172.21.15.19:0/4208299122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564cd2eb5470 0x564cd2ebcf10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fea78014040 tx=0x7fea78005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b54262e75dcfd489 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:41:41.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.766+0000 7fea77e00700 1 -- 172.21.15.19:0/4208299122 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea78031070 con 0x564cd2eb5470 2024-05-07T02:41:41.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.766+0000 7fea77e00700 1 -- 172.21.15.19:0/4208299122 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea78003b30 con 0x564cd2eb5470 2024-05-07T02:41:41.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.766+0000 7fea77e00700 1 -- 172.21.15.19:0/4208299122 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7802b530 con 0x564cd2eb5470 2024-05-07T02:41:41.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.767+0000 7fea9d1c0b80 1 -- 172.21.15.19:0/4208299122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564cd2eb5470 msgr2=0x564cd2ebcf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:41.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.767+0000 7fea9d1c0b80 1 --2- 172.21.15.19:0/4208299122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564cd2eb5470 0x564cd2ebcf10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fea78014040 tx=0x7fea78005fa0 comp rx=0 tx=0).stop 2024-05-07T02:41:41.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.767+0000 7fea9d1c0b80 1 -- 172.21.15.19:0/4208299122 shutdown_connections 2024-05-07T02:41:41.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.767+0000 7fea9d1c0b80 1 --2- 172.21.15.19:0/4208299122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564cd2ebb3b0 0x564cd2ed74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:41.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.767+0000 7fea9d1c0b80 1 --2- 172.21.15.19:0/4208299122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564cd2eb5470 0x564cd2ebcf10 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:41.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.767+0000 7fea9d1c0b80 1 --2- 172.21.15.19:0/4208299122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564cd2eb5dc0 0x564cd2ecb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:41.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.767+0000 7fea9d1c0b80 1 -- 172.21.15.19:0/4208299122 >> 172.21.15.19:0/4208299122 conn(0x564cd2d37e40 msgr2=0x564cd2ebddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:41:41.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.767+0000 7fea9d1c0b80 1 -- 172.21.15.19:0/4208299122 shutdown_connections 2024-05-07T02:41:41.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.767+0000 7fea9d1c0b80 1 -- 172.21.15.19:0/4208299122 wait complete. 2024-05-07T02:41:41.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.768+0000 7fea9d1c0b80 1 -- start start 2024-05-07T02:41:41.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.768+0000 7fea9d1c0b80 1 Processor -- start 2024-05-07T02:41:41.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.768+0000 7fea9d1c0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564cd2eb5470 0x564cd2ec7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:41.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.768+0000 7fea9d1c0b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564cd2eb5dc0 0x564cd2ed6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:41.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.769+0000 7fea9d1c0b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564cd2ebb3b0 0x564cd2ed6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:41.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.769+0000 7fea9d1c0b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fea881b1c00 con 0x564cd2eb5470 2024-05-07T02:41:41.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.769+0000 7fea9d1c0b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fea881b1530 con 0x564cd2ebb3b0 2024-05-07T02:41:41.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.769+0000 7fea9d1c0b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fea881ae750 con 0x564cd2eb5dc0 2024-05-07T02:41:41.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.769+0000 7fea9d1c0b80 10 client.0 _ll_register_callbacks cb 0x564cd2d7ec70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:41:41.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.769+0000 7fea87400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564cd2eb5470 0x564cd2ec7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:41.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.769+0000 7fea87e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564cd2ebb3b0 0x564cd2ed6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:41.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.769+0000 7fea7fe00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564cd2eb5dc0 0x564cd2ed6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:41.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.769+0000 7fea87e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564cd2ebb3b0 0x564cd2ed6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:57500/0 (socket says 172.21.15.19:57500) 2024-05-07T02:41:41.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.769+0000 7fea7fe00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564cd2eb5dc0 0x564cd2ed6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:48610/0 (socket says 172.21.15.19:48610) 2024-05-07T02:41:41.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.769+0000 7fea87e00700 1 -- 172.21.15.19:0/1894510696 learned_addr learned my addr 172.21.15.19:0/1894510696 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:41:41.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.769+0000 7fea87400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564cd2eb5470 0x564cd2ec7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37082/0 (socket says 172.21.15.19:37082) 2024-05-07T02:41:41.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.769+0000 7fea87400700 1 -- 172.21.15.19:0/1894510696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564cd2eb5dc0 msgr2=0x564cd2ed6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:41.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.770+0000 7fea87400700 1 --2- 172.21.15.19:0/1894510696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564cd2eb5dc0 0x564cd2ed6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:41.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.770+0000 7fea87400700 1 -- 172.21.15.19:0/1894510696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564cd2ebb3b0 msgr2=0x564cd2ed6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:41.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.770+0000 7fea87400700 1 --2- 172.21.15.19:0/1894510696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564cd2ebb3b0 0x564cd2ed6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:41.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.770+0000 7fea87400700 1 -- 172.21.15.19:0/1894510696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cd2ed70c0 con 0x564cd2eb5470 2024-05-07T02:41:41.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.770+0000 7fea87e00700 1 --2- 172.21.15.19:0/1894510696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564cd2ebb3b0 0x564cd2ed6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:41:41.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.770+0000 7fea7fe00700 1 --2- 172.21.15.19:0/1894510696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564cd2eb5dc0 0x564cd2ed6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:41:41.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.770+0000 7fea75600700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-07T02:41:41.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.770+0000 7fea87400700 1 --2- 172.21.15.19:0/1894510696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564cd2eb5470 0x564cd2ec7dd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x564cd2eaa030 tx=0x7fea78005240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f4f0d8800c8f044 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:41:41.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.770+0000 7fea75600700 1 -- 172.21.15.19:0/1894510696 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea78031070 con 0x564cd2eb5470 2024-05-07T02:41:41.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.770+0000 7fea9d1c0b80 1 -- 172.21.15.19:0/1894510696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea78003b30 con 0x564cd2eb5470 2024-05-07T02:41:41.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.770+0000 7fea75600700 1 -- 172.21.15.19:0/1894510696 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9525+0+0 (secure 0 0 0) 0x7fea78003890 con 0x564cd2eb5470 2024-05-07T02:41:41.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.771+0000 7fea75600700 1 -- 172.21.15.19:0/1894510696 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fea78011a70 con 0x564cd2eb5470 2024-05-07T02:41:41.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.771+0000 7fea9d1c0b80 10 client.16467 fetch_fsmap learned FSMap version 50 2024-05-07T02:41:41.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.771+0000 7fea9d1c0b80 1 -- 172.21.15.19:0/1894510696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x564cd2f39be0 con 0x564cd2eb5470 2024-05-07T02:41:41.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.771+0000 7fea75600700 1 -- 172.21.15.19:0/1894510696 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea78003b30 con 0x564cd2eb5470 2024-05-07T02:41:41.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.771+0000 7fea75600700 1 -- 172.21.15.19:0/1894510696 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 50) ==== 8362+0+0 (secure 0 0 0) 0x7fea78003ce0 con 0x564cd2eb5470 2024-05-07T02:41:41.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.771+0000 7fea9d1c0b80 10 client.16467 fetch_fsmap finished waiting for FSMap version 50 2024-05-07T02:41:41.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.771+0000 7fea9d1c0b80 10 client.16467 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-05-07T02:41:41.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.771+0000 7fea9d1c0b80 20 client.16467 populate_metadata read hostname 'smithi019' 2024-05-07T02:41:41.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.771+0000 7fea9d1c0b80 1 --2- 172.21.15.19:0/1894510696 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x564cd2f39fe0 0x564cd2f3c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:41.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.771+0000 7fea9d1c0b80 4 client.16467 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:41:41.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.771+0000 7fea9d1c0b80 1 -- 172.21.15.19:0/1894510696 --> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564cd2de5400 con 0x564cd2f39fe0 2024-05-07T02:41:41.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.772+0000 7fea7fe00700 1 --2- 172.21.15.19:0/1894510696 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x564cd2f39fe0 0x564cd2f3c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:41.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.772+0000 7fea7fe00700 1 --2- 172.21.15.19:0/1894510696 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x564cd2f39fe0 0x564cd2f3c400 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=44f80326594ea853 server_cookie=3c70ecd341d570ef in_seq=0 out_seq=0 2024-05-07T02:41:41.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.772+0000 7fea75600700 10 client.16467 ms_handle_connect on v2:172.21.15.27:6836/2583138143 2024-05-07T02:41:41.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.772+0000 7fea75600700 1 -- 172.21.15.19:0/1894510696 <== mds.3 v2:172.21.15.27:6836/2583138143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564cd2de5400 con 0x564cd2f39fe0 2024-05-07T02:41:41.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.772+0000 7fea75600700 10 client.16467 handle_command_reply: tid=0 2024-05-07T02:41:41.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.772+0000 7fea9d1c0b80 1 client.16467 shutdown 2024-05-07T02:41:41.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.773+0000 7fea9d1c0b80 1 -- 172.21.15.19:0/1894510696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564cd2eb5470 msgr2=0x564cd2ec7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:41.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.773+0000 7fea9d1c0b80 1 --2- 172.21.15.19:0/1894510696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564cd2eb5470 0x564cd2ec7dd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x564cd2eaa030 tx=0x7fea78005240 comp rx=0 tx=0).stop 2024-05-07T02:41:41.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.773+0000 7fea9d1c0b80 1 -- 172.21.15.19:0/1894510696 shutdown_connections 2024-05-07T02:41:41.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.773+0000 7fea9d1c0b80 1 --2- 172.21.15.19:0/1894510696 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x564cd2f39fe0 0x564cd2f3c400 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:41.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.773+0000 7fea9d1c0b80 1 --2- 172.21.15.19:0/1894510696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x564cd2ebb3b0 0x564cd2ed6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:41.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.773+0000 7fea9d1c0b80 1 --2- 172.21.15.19:0/1894510696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564cd2eb5470 0x564cd2ec7dd0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:41.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.773+0000 7fea9d1c0b80 1 --2- 172.21.15.19:0/1894510696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564cd2eb5dc0 0x564cd2ed6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:41.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.773+0000 7fea9d1c0b80 1 -- 172.21.15.19:0/1894510696 >> 172.21.15.19:0/1894510696 conn(0x564cd2d37e40 msgr2=0x564cd2eb4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:41:41.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.773+0000 7fea75600700 0 client.16467 ms_handle_reset on v2:172.21.15.27:6836/2583138143 2024-05-07T02:41:41.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.773+0000 7fea9d1c0b80 1 -- 172.21.15.19:0/1894510696 shutdown_connections 2024-05-07T02:41:41.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.773+0000 7fea9d1c0b80 1 -- 172.21.15.19:0/1894510696 wait complete. 2024-05-07T02:41:41.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.773+0000 7fea9d1c0b80 20 client.16467 trim_cache size 0 max 16384 2024-05-07T02:41:41.778 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:41:41.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.774+0000 7fea8e600700 1 -- 172.21.15.19:0/2140939669 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fea6c0b29f0 msgr2=0x7fea6c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:41.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.774+0000 7fea8e600700 1 --2- 172.21.15.19:0/2140939669 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fea6c0b29f0 0x7fea6c0b4f00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x564cd2ebae70 tx=0x7fea80005040 comp rx=0 tx=0).stop 2024-05-07T02:41:41.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.774+0000 7fea8e600700 1 -- 172.21.15.19:0/2140939669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea88108f00 msgr2=0x7fea8819b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:41.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.774+0000 7fea8e600700 1 --2- 172.21.15.19:0/2140939669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea88108f00 0x7fea8819b370 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fea7000e9c0 tx=0x7fea7000ecd0 comp rx=0 tx=0).stop 2024-05-07T02:41:41.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.774+0000 7fea8e600700 1 -- 172.21.15.19:0/2140939669 shutdown_connections 2024-05-07T02:41:41.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.774+0000 7fea8e600700 1 --2- 172.21.15.19:0/2140939669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea88109850 0x7fea8819f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:41.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.774+0000 7fea8e600700 1 --2- 172.21.15.19:0/2140939669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea881045d0 0x7fea8819add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:41.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.774+0000 7fea8e600700 1 --2- 172.21.15.19:0/2140939669 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fea6c0b29f0 0x7fea6c0b4f00 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:41.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.774+0000 7fea8e600700 1 --2- 172.21.15.19:0/2140939669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea88108f00 0x7fea8819b370 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:41.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.774+0000 7fea8e600700 1 -- 172.21.15.19:0/2140939669 >> 172.21.15.19:0/2140939669 conn(0x7fea880ffea0 msgr2=0x7fea88108670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:41:41.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.774+0000 7fea8e600700 1 -- 172.21.15.19:0/2140939669 shutdown_connections 2024-05-07T02:41:41.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:41.775+0000 7fea8e600700 1 -- 172.21.15.19:0/2140939669 wait complete. 2024-05-07T02:41:42.008 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:41 smithi027 ceph-mon[34398]: pgmap v304: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:41:42.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:41 smithi064 ceph-mon[34699]: pgmap v304: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:41:42.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:41 smithi019 ceph-mon[29781]: pgmap v304: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:41:42.826 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-07T02:41:43.306 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:41:44.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:43 smithi064 ceph-mon[34699]: pgmap v305: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:41:44.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:43 smithi019 ceph-mon[29781]: pgmap v305: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:41:44.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:43 smithi027 ceph-mon[34398]: pgmap v305: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:41:45.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.701+0000 7fa41a000700 1 -- 172.21.15.19:0/1301406215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa414104400 msgr2=0x7fa414104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:45.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.701+0000 7fa41a000700 1 --2- 172.21.15.19:0/1301406215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa414104400 0x7fa414104810 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa408009990 tx=0x7fa408009ca0 comp rx=0 tx=0).stop 2024-05-07T02:41:45.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.701+0000 7fa41a000700 1 -- 172.21.15.19:0/1301406215 shutdown_connections 2024-05-07T02:41:45.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.701+0000 7fa41a000700 1 --2- 172.21.15.19:0/1301406215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa414109680 0x7fa414110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:45.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.701+0000 7fa41a000700 1 --2- 172.21.15.19:0/1301406215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa414104400 0x7fa414104810 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:45.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.701+0000 7fa41a000700 1 --2- 172.21.15.19:0/1301406215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa414108d30 0x7fa414109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:45.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.701+0000 7fa41a000700 1 -- 172.21.15.19:0/1301406215 >> 172.21.15.19:0/1301406215 conn(0x7fa414079d10 msgr2=0x7fa41407a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:41:45.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.701+0000 7fa41a000700 1 -- 172.21.15.19:0/1301406215 shutdown_connections 2024-05-07T02:41:45.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.701+0000 7fa41a000700 1 -- 172.21.15.19:0/1301406215 wait complete. 2024-05-07T02:41:45.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.702+0000 7fa41a000700 1 Processor -- start 2024-05-07T02:41:45.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.703+0000 7fa41a000700 1 -- start start 2024-05-07T02:41:45.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.703+0000 7fa41a000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa414104400 0x7fa41419cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:45.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.703+0000 7fa41a000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa414108d30 0x7fa41419d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:45.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.703+0000 7fa41a000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa414109680 0x7fa4141a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:45.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.703+0000 7fa41a000700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa4141142b0 con 0x7fa414109680 2024-05-07T02:41:45.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.703+0000 7fa41a000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa414114130 con 0x7fa414104400 2024-05-07T02:41:45.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.703+0000 7fa41a000700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa414114430 con 0x7fa414108d30 2024-05-07T02:41:45.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.703+0000 7fa418c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa414104400 0x7fa41419cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:45.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.703+0000 7fa419600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa414109680 0x7fa4141a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:45.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.703+0000 7fa418c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa414104400 0x7fa41419cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:57520/0 (socket says 172.21.15.19:57520) 2024-05-07T02:41:45.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.703+0000 7fa418c00700 1 -- 172.21.15.19:0/3369024952 learned_addr learned my addr 172.21.15.19:0/3369024952 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:41:45.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.703+0000 7fa413e00700 1 --2- 172.21.15.19:0/3369024952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa414108d30 0x7fa41419d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:45.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.704+0000 7fa418c00700 1 -- 172.21.15.19:0/3369024952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa414108d30 msgr2=0x7fa41419d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:45.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.704+0000 7fa418c00700 1 --2- 172.21.15.19:0/3369024952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa414108d30 0x7fa41419d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:45.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.704+0000 7fa418c00700 1 -- 172.21.15.19:0/3369024952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa414109680 msgr2=0x7fa4141a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:45.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.704+0000 7fa418c00700 1 --2- 172.21.15.19:0/3369024952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa414109680 0x7fa4141a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:45.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.704+0000 7fa418c00700 1 -- 172.21.15.19:0/3369024952 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4141a1d90 con 0x7fa414104400 2024-05-07T02:41:45.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.704+0000 7fa418c00700 1 --2- 172.21.15.19:0/3369024952 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa414104400 0x7fa41419cec0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa4080057c0 tx=0x7fa408005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:41:45.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.704+0000 7fa411600700 1 -- 172.21.15.19:0/3369024952 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa408026070 con 0x7fa414104400 2024-05-07T02:41:45.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.705+0000 7fa411600700 1 -- 172.21.15.19:0/3369024952 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa40800b610 con 0x7fa414104400 2024-05-07T02:41:45.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.705+0000 7fa41a000700 1 -- 172.21.15.19:0/3369024952 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4141a2020 con 0x7fa414104400 2024-05-07T02:41:45.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.705+0000 7fa411600700 1 -- 172.21.15.19:0/3369024952 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa408021e00 con 0x7fa414104400 2024-05-07T02:41:45.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.705+0000 7fa41a000700 1 -- 172.21.15.19:0/3369024952 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4141a2500 con 0x7fa414104400 2024-05-07T02:41:45.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.709+0000 7fa411600700 1 -- 172.21.15.19:0/3369024952 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa408029070 con 0x7fa414104400 2024-05-07T02:41:45.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.709+0000 7fa42a073b80 1 Processor -- start 2024-05-07T02:41:45.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.710+0000 7fa42a073b80 1 -- start start 2024-05-07T02:41:45.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.710+0000 7fa42a073b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e5bf8c40 0x55a5e5bf9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:45.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.710+0000 7fa42a073b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5e5bf9590 0x55a5e5c026f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:45.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.710+0000 7fa42a073b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5e5c02c30 0x55a5e5c05050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:45.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.710+0000 7fa42a073b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a5e5b1aa20 con 0x55a5e5c02c30 2024-05-07T02:41:45.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.710+0000 7fa42a073b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a5e5ab8760 con 0x55a5e5bf8c40 2024-05-07T02:41:45.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.710+0000 7fa42a073b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a5e5b169e0 con 0x55a5e5bf9590 2024-05-07T02:41:45.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.711+0000 7fa418c00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5e5bf9590 0x55a5e5c026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:45.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.711+0000 7fa419600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e5bf8c40 0x55a5e5bf9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:45.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.711+0000 7fa418c00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5e5bf9590 0x55a5e5c026f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:48642/0 (socket says 172.21.15.19:48642) 2024-05-07T02:41:45.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.711+0000 7fa418c00700 1 -- 172.21.15.19:0/600083087 learned_addr learned my addr 172.21.15.19:0/600083087 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:41:45.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.711+0000 7fa413e00700 1 --2- 172.21.15.19:0/600083087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5e5c02c30 0x55a5e5c05050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:45.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.711+0000 7fa418c00700 1 -- 172.21.15.19:0/600083087 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e5bf8c40 msgr2=0x55a5e5bf9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:45.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.711+0000 7fa418c00700 1 --2- 172.21.15.19:0/600083087 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e5bf8c40 0x55a5e5bf9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:45.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.711+0000 7fa418c00700 1 -- 172.21.15.19:0/600083087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5e5c02c30 msgr2=0x55a5e5c05050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:45.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.711+0000 7fa418c00700 1 --2- 172.21.15.19:0/600083087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5e5c02c30 0x55a5e5c05050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:45.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.711+0000 7fa418c00700 1 -- 172.21.15.19:0/600083087 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5e5a678d0 con 0x55a5e5bf9590 2024-05-07T02:41:45.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.711+0000 7fa413e00700 1 --2- 172.21.15.19:0/600083087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5e5c02c30 0x55a5e5c05050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:41:45.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.712+0000 7fa418c00700 1 --2- 172.21.15.19:0/600083087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5e5bf9590 0x55a5e5c026f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa4080e07f0 tx=0x7fa4080e99d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80ea27e1961a3b76 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:41:45.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.712+0000 7fa411600700 1 --2- 172.21.15.19:0/3369024952 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fa3fc0b29f0 0x7fa3fc0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:45.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.712+0000 7fa411600700 1 -- 172.21.15.19:0/3369024952 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9525+0+0 (secure 0 0 0) 0x7fa4080e3520 con 0x7fa414104400 2024-05-07T02:41:45.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.712+0000 7fa419600700 1 --2- 172.21.15.19:0/3369024952 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fa3fc0b29f0 0x7fa3fc0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:45.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.712+0000 7fa406000700 1 -- 172.21.15.19:0/600083087 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4080a6360 con 0x55a5e5bf9590 2024-05-07T02:41:45.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.712+0000 7fa406000700 1 -- 172.21.15.19:0/600083087 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4080f0030 con 0x55a5e5bf9590 2024-05-07T02:41:45.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.712+0000 7fa406000700 1 -- 172.21.15.19:0/600083087 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4080a9c10 con 0x55a5e5bf9590 2024-05-07T02:41:45.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.713+0000 7fa42a073b80 1 -- 172.21.15.19:0/600083087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5e5bf9590 msgr2=0x55a5e5c026f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:45.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.713+0000 7fa419600700 1 --2- 172.21.15.19:0/3369024952 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fa3fc0b29f0 0x7fa3fc0b4f00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x55a5e5bee190 tx=0x7fa40c00b040 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:41:45.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.713+0000 7fa42a073b80 1 --2- 172.21.15.19:0/600083087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5e5bf9590 0x55a5e5c026f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa4080e07f0 tx=0x7fa4080e99d0 comp rx=0 tx=0).stop 2024-05-07T02:41:45.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.713+0000 7fa42a073b80 1 -- 172.21.15.19:0/600083087 shutdown_connections 2024-05-07T02:41:45.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.713+0000 7fa42a073b80 1 --2- 172.21.15.19:0/600083087 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5e5c02c30 0x55a5e5c05050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:45.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.713+0000 7fa42a073b80 1 --2- 172.21.15.19:0/600083087 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e5bf8c40 0x55a5e5bf9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:45.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.713+0000 7fa42a073b80 1 --2- 172.21.15.19:0/600083087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5e5bf9590 0x55a5e5c026f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:45.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.713+0000 7fa42a073b80 1 -- 172.21.15.19:0/600083087 >> 172.21.15.19:0/600083087 conn(0x55a5e5a7ae40 msgr2=0x55a5e5bf3980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:41:45.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.714+0000 7fa42a073b80 1 -- 172.21.15.19:0/600083087 shutdown_connections 2024-05-07T02:41:45.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.714+0000 7fa42a073b80 1 -- 172.21.15.19:0/600083087 wait complete. 2024-05-07T02:41:45.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.714+0000 7fa42a073b80 1 -- start start 2024-05-07T02:41:45.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.715+0000 7fa42a073b80 1 Processor -- start 2024-05-07T02:41:45.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.715+0000 7fa42a073b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5e5bf8c40 0x55a5e5bffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:45.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.715+0000 7fa42a073b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e5bf9590 0x55a5e5c00130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:45.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.715+0000 7fa42a073b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5e5c02c30 0x55a5e5c00670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:45.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.715+0000 7fa42a073b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a5e5c057c0 con 0x55a5e5bf8c40 2024-05-07T02:41:45.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.715+0000 7fa42a073b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55a5e5a80b20 con 0x55a5e5bf9590 2024-05-07T02:41:45.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.715+0000 7fa42a073b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a5e5c05940 con 0x55a5e5c02c30 2024-05-07T02:41:45.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.715+0000 7fa42a073b80 10 client.0 _ll_register_callbacks cb 0x55a5e5aae380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:41:45.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.715+0000 7fa418c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5e5bf8c40 0x55a5e5bffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:45.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.716+0000 7fa418c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5e5bf8c40 0x55a5e5bffbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37130/0 (socket says 172.21.15.19:37130) 2024-05-07T02:41:45.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.716+0000 7fa418c00700 1 -- 172.21.15.19:0/1113252403 learned_addr learned my addr 172.21.15.19:0/1113252403 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:41:45.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.716+0000 7fa413e00700 1 --2- 172.21.15.19:0/1113252403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e5bf9590 0x55a5e5c00130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:45.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.716+0000 7fa419600700 1 --2- 172.21.15.19:0/1113252403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5e5c02c30 0x55a5e5c00670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:45.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.716+0000 7fa418c00700 1 -- 172.21.15.19:0/1113252403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5e5c02c30 msgr2=0x55a5e5c00670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:45.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.716+0000 7fa418c00700 1 --2- 172.21.15.19:0/1113252403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5e5c02c30 0x55a5e5c00670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:45.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.716+0000 7fa418c00700 1 -- 172.21.15.19:0/1113252403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e5bf9590 msgr2=0x55a5e5c00130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:45.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.716+0000 7fa418c00700 1 --2- 172.21.15.19:0/1113252403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e5bf9590 0x55a5e5c00130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:45.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.716+0000 7fa418c00700 1 -- 172.21.15.19:0/1113252403 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5e5ae0420 con 0x55a5e5bf8c40 2024-05-07T02:41:45.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.716+0000 7fa418c00700 1 --2- 172.21.15.19:0/1113252403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5e5bf8c40 0x55a5e5bffbf0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa408009350 tx=0x7fa4080e0820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74804cf0d4775069 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:41:45.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.716+0000 7fa3faa00700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-07T02:41:45.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.717+0000 7fa3faa00700 1 -- 172.21.15.19:0/1113252403 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4080a9df0 con 0x55a5e5bf8c40 2024-05-07T02:41:45.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.717+0000 7fa3faa00700 1 -- 172.21.15.19:0/1113252403 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9525+0+0 (secure 0 0 0) 0x7fa4080a6730 con 0x55a5e5bf8c40 2024-05-07T02:41:45.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.717+0000 7fa42a073b80 1 -- 172.21.15.19:0/1113252403 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4080f0030 con 0x55a5e5bf8c40 2024-05-07T02:41:45.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.717+0000 7fa3faa00700 1 -- 172.21.15.19:0/1113252403 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fa4080aa270 con 0x55a5e5bf8c40 2024-05-07T02:41:45.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.717+0000 7fa42a073b80 10 client.16479 fetch_fsmap learned FSMap version 50 2024-05-07T02:41:45.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.717+0000 7fa42a073b80 1 -- 172.21.15.19:0/1113252403 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55a5e5c057c0 con 0x55a5e5bf8c40 2024-05-07T02:41:45.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.717+0000 7fa3faa00700 1 -- 172.21.15.19:0/1113252403 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4080f0030 con 0x55a5e5bf8c40 2024-05-07T02:41:45.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.718+0000 7fa3faa00700 1 -- 172.21.15.19:0/1113252403 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 50) ==== 8362+0+0 (secure 0 0 0) 0x7fa408021530 con 0x55a5e5bf8c40 2024-05-07T02:41:45.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.718+0000 7fa42a073b80 10 client.16479 fetch_fsmap finished waiting for FSMap version 50 2024-05-07T02:41:45.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.718+0000 7fa42a073b80 10 client.16479 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-07T02:41:45.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.718+0000 7fa42a073b80 20 client.16479 populate_metadata read hostname 'smithi019' 2024-05-07T02:41:45.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.718+0000 7fa42a073b80 1 --2- 172.21.15.19:0/1113252403 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55a5e5c126b0 0x55a5e5c14af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:45.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.718+0000 7fa42a073b80 4 client.16479 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:41:45.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.718+0000 7fa42a073b80 1 -- 172.21.15.19:0/1113252403 --> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5e5bc50e0 con 0x55a5e5c126b0 2024-05-07T02:41:45.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.718+0000 7fa413e00700 1 --2- 172.21.15.19:0/1113252403 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55a5e5c126b0 0x55a5e5c14af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:45.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.719+0000 7fa413e00700 1 --2- 172.21.15.19:0/1113252403 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55a5e5c126b0 0x55a5e5c14af0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bae7ff493eb534a8 server_cookie=4a222357e5f59214 in_seq=0 out_seq=0 2024-05-07T02:41:45.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.719+0000 7fa3faa00700 10 client.16479 ms_handle_connect on v2:172.21.15.64:6838/1583120004 2024-05-07T02:41:45.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.720+0000 7fa3faa00700 1 -- 172.21.15.19:0/1113252403 <== mds.2 v2:172.21.15.64:6838/1583120004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a5e5bc50e0 con 0x55a5e5c126b0 2024-05-07T02:41:45.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.720+0000 7fa3faa00700 10 client.16479 handle_command_reply: tid=0 2024-05-07T02:41:45.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.720+0000 7fa42a073b80 1 client.16479 shutdown 2024-05-07T02:41:45.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.720+0000 7fa42a073b80 1 -- 172.21.15.19:0/1113252403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5e5bf8c40 msgr2=0x55a5e5bffbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:45.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.720+0000 7fa42a073b80 1 --2- 172.21.15.19:0/1113252403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5e5bf8c40 0x55a5e5bffbf0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa408009350 tx=0x7fa4080e0820 comp rx=0 tx=0).stop 2024-05-07T02:41:45.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.720+0000 7fa42a073b80 1 -- 172.21.15.19:0/1113252403 shutdown_connections 2024-05-07T02:41:45.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.720+0000 7fa42a073b80 1 --2- 172.21.15.19:0/1113252403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5e5c02c30 0x55a5e5c00670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:45.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.720+0000 7fa42a073b80 1 --2- 172.21.15.19:0/1113252403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5e5bf8c40 0x55a5e5bffbf0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:45.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.720+0000 7fa42a073b80 1 --2- 172.21.15.19:0/1113252403 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55a5e5c126b0 0x55a5e5c14af0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:45.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.720+0000 7fa42a073b80 1 --2- 172.21.15.19:0/1113252403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e5bf9590 0x55a5e5c00130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:45.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.720+0000 7fa42a073b80 1 -- 172.21.15.19:0/1113252403 >> 172.21.15.19:0/1113252403 conn(0x55a5e5a7ae40 msgr2=0x55a5e5bf5600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:41:45.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.720+0000 7fa3faa00700 0 client.16479 ms_handle_reset on v2:172.21.15.64:6838/1583120004 2024-05-07T02:41:45.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.720+0000 7fa42a073b80 1 -- 172.21.15.19:0/1113252403 shutdown_connections 2024-05-07T02:41:45.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.720+0000 7fa42a073b80 1 -- 172.21.15.19:0/1113252403 wait complete. 2024-05-07T02:41:45.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.720+0000 7fa42a073b80 20 client.16479 trim_cache size 0 max 16384 2024-05-07T02:41:45.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.737+0000 7fa42a073b80 1 Processor -- start 2024-05-07T02:41:45.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.737+0000 7fa42a073b80 1 -- start start 2024-05-07T02:41:45.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.738+0000 7fa42a073b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5e5bf8c40 0x55a5e5bf9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:45.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.738+0000 7fa42a073b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e5bf9590 0x55a5e5c0e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:45.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.738+0000 7fa42a073b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5e5c02c30 0x55a5e5c1a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:45.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.738+0000 7fa42a073b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa4141b2730 con 0x55a5e5c02c30 2024-05-07T02:41:45.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.738+0000 7fa42a073b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa4141b4270 con 0x55a5e5bf9590 2024-05-07T02:41:45.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.738+0000 7fa42a073b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa4141b2e00 con 0x55a5e5bf8c40 2024-05-07T02:41:45.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.738+0000 7fa418c00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5e5bf8c40 0x55a5e5bf9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:45.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.738+0000 7fa419600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5e5c02c30 0x55a5e5c1a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:45.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.738+0000 7fa418c00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5e5bf8c40 0x55a5e5bf9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:48656/0 (socket says 172.21.15.19:48656) 2024-05-07T02:41:45.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.738+0000 7fa418c00700 1 -- 172.21.15.19:0/781625476 learned_addr learned my addr 172.21.15.19:0/781625476 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:41:45.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.738+0000 7fa413e00700 1 --2- 172.21.15.19:0/781625476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e5bf9590 0x55a5e5c0e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:45.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.739+0000 7fa418c00700 1 -- 172.21.15.19:0/781625476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e5bf9590 msgr2=0x55a5e5c0e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:45.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.739+0000 7fa418c00700 1 --2- 172.21.15.19:0/781625476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e5bf9590 0x55a5e5c0e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:45.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.739+0000 7fa418c00700 1 -- 172.21.15.19:0/781625476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5e5c02c30 msgr2=0x55a5e5c1a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:45.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.739+0000 7fa418c00700 1 --2- 172.21.15.19:0/781625476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5e5c02c30 0x55a5e5c1a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:45.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.739+0000 7fa418c00700 1 -- 172.21.15.19:0/781625476 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5e5bfe160 con 0x55a5e5bf8c40 2024-05-07T02:41:45.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.739+0000 7fa418c00700 1 --2- 172.21.15.19:0/781625476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5e5bf8c40 0x55a5e5bf9050 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa4080aa430 tx=0x7fa4080e99d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efac735286e0556 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:41:45.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.739+0000 7fa405600700 1 -- 172.21.15.19:0/781625476 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4080a9df0 con 0x55a5e5bf8c40 2024-05-07T02:41:45.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.739+0000 7fa405600700 1 -- 172.21.15.19:0/781625476 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4080f0020 con 0x55a5e5bf8c40 2024-05-07T02:41:45.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.739+0000 7fa405600700 1 -- 172.21.15.19:0/781625476 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4080a9fb0 con 0x55a5e5bf8c40 2024-05-07T02:41:45.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.740+0000 7fa42a073b80 1 -- 172.21.15.19:0/781625476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5e5bf8c40 msgr2=0x55a5e5bf9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:45.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.740+0000 7fa42a073b80 1 --2- 172.21.15.19:0/781625476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5e5bf8c40 0x55a5e5bf9050 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa4080aa430 tx=0x7fa4080e99d0 comp rx=0 tx=0).stop 2024-05-07T02:41:45.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.740+0000 7fa42a073b80 1 -- 172.21.15.19:0/781625476 shutdown_connections 2024-05-07T02:41:45.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.740+0000 7fa42a073b80 1 --2- 172.21.15.19:0/781625476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5e5c02c30 0x55a5e5c1a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:45.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.740+0000 7fa42a073b80 1 --2- 172.21.15.19:0/781625476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5e5bf8c40 0x55a5e5bf9050 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:45.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.740+0000 7fa42a073b80 1 --2- 172.21.15.19:0/781625476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e5bf9590 0x55a5e5c0e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:45.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.740+0000 7fa42a073b80 1 -- 172.21.15.19:0/781625476 >> 172.21.15.19:0/781625476 conn(0x55a5e5a7ae40 msgr2=0x55a5e5bfe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:41:45.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.740+0000 7fa42a073b80 1 -- 172.21.15.19:0/781625476 shutdown_connections 2024-05-07T02:41:45.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.740+0000 7fa42a073b80 1 -- 172.21.15.19:0/781625476 wait complete. 2024-05-07T02:41:45.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.741+0000 7fa42a073b80 1 -- start start 2024-05-07T02:41:45.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.741+0000 7fa42a073b80 1 Processor -- start 2024-05-07T02:41:45.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.742+0000 7fa42a073b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e5bf8c40 0x55a5e5c05e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:45.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.742+0000 7fa42a073b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5e5bf9590 0x55a5e5c06370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:45.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.742+0000 7fa42a073b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5e5c02c30 0x55a5e5c7ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:45.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.742+0000 7fa42a073b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa4141b3ba0 con 0x55a5e5bf9590 2024-05-07T02:41:45.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.742+0000 7fa42a073b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa4141b34d0 con 0x55a5e5bf8c40 2024-05-07T02:41:45.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.742+0000 7fa42a073b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa4141b06f0 con 0x55a5e5c02c30 2024-05-07T02:41:45.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.742+0000 7fa42a073b80 10 client.0 _ll_register_callbacks cb 0x55a5e5aae380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:41:45.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.742+0000 7fa418c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e5bf8c40 0x55a5e5c05e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:45.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.742+0000 7fa418c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e5bf8c40 0x55a5e5c05e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:57566/0 (socket says 172.21.15.19:57566) 2024-05-07T02:41:45.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.742+0000 7fa418c00700 1 -- 172.21.15.19:0/1730779533 learned_addr learned my addr 172.21.15.19:0/1730779533 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:41:45.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.742+0000 7fa413e00700 1 --2- 172.21.15.19:0/1730779533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5e5bf9590 0x55a5e5c06370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:45.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.742+0000 7fa419600700 1 --2- 172.21.15.19:0/1730779533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5e5c02c30 0x55a5e5c7ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:45.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.742+0000 7fa418c00700 1 -- 172.21.15.19:0/1730779533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5e5c02c30 msgr2=0x55a5e5c7ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:45.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.742+0000 7fa418c00700 1 --2- 172.21.15.19:0/1730779533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5e5c02c30 0x55a5e5c7ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:45.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.742+0000 7fa418c00700 1 -- 172.21.15.19:0/1730779533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5e5bf9590 msgr2=0x55a5e5c06370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:45.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.742+0000 7fa418c00700 1 --2- 172.21.15.19:0/1730779533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5e5bf9590 0x55a5e5c06370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:45.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.742+0000 7fa418c00700 1 -- 172.21.15.19:0/1730779533 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5e5c7b390 con 0x55a5e5bf8c40 2024-05-07T02:41:45.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.743+0000 7fa418c00700 1 --2- 172.21.15.19:0/1730779533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e5bf8c40 0x55a5e5c05e30 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa4080b3290 tx=0x7fa408005a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3d9543b92ef80f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:41:45.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.743+0000 7fa3faa00700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:41:45.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.743+0000 7fa3faa00700 1 -- 172.21.15.19:0/1730779533 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4080a9df0 con 0x55a5e5bf8c40 2024-05-07T02:41:45.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.743+0000 7fa3faa00700 1 -- 172.21.15.19:0/1730779533 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9525+0+0 (secure 0 0 0) 0x7fa4080e3110 con 0x55a5e5bf8c40 2024-05-07T02:41:45.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.743+0000 7fa42a073b80 1 -- 172.21.15.19:0/1730779533 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4080f0020 con 0x55a5e5bf8c40 2024-05-07T02:41:45.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.743+0000 7fa3faa00700 1 -- 172.21.15.19:0/1730779533 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fa4080e0d10 con 0x55a5e5bf8c40 2024-05-07T02:41:45.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.743+0000 7fa42a073b80 10 client.26684 fetch_fsmap learned FSMap version 50 2024-05-07T02:41:45.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.743+0000 7fa42a073b80 1 -- 172.21.15.19:0/1730779533 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55a5e5c7b390 con 0x55a5e5bf8c40 2024-05-07T02:41:45.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.744+0000 7fa3faa00700 1 -- 172.21.15.19:0/1730779533 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4080f0020 con 0x55a5e5bf8c40 2024-05-07T02:41:45.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.744+0000 7fa3faa00700 1 -- 172.21.15.19:0/1730779533 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 50) ==== 8362+0+0 (secure 0 0 0) 0x7fa4080b2de0 con 0x55a5e5bf8c40 2024-05-07T02:41:45.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.744+0000 7fa42a073b80 10 client.26684 fetch_fsmap finished waiting for FSMap version 50 2024-05-07T02:41:45.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.744+0000 7fa42a073b80 10 client.26684 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-07T02:41:45.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.744+0000 7fa42a073b80 20 client.26684 populate_metadata read hostname 'smithi019' 2024-05-07T02:41:45.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.744+0000 7fa42a073b80 1 --2- 172.21.15.19:0/1730779533 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55a5e5c7c7b0 0x55a5e5c7ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:45.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.744+0000 7fa42a073b80 4 client.26684 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:41:45.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.744+0000 7fa42a073b80 1 -- 172.21.15.19:0/1730779533 --> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a5e5b28400 con 0x55a5e5c7c7b0 2024-05-07T02:41:45.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.744+0000 7fa413e00700 1 --2- 172.21.15.19:0/1730779533 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55a5e5c7c7b0 0x55a5e5c7ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:45.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.745+0000 7fa413e00700 1 --2- 172.21.15.19:0/1730779533 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55a5e5c7c7b0 0x55a5e5c7ebd0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=211e84c230c24b59 server_cookie=205437d4f0ea8562 in_seq=0 out_seq=0 2024-05-07T02:41:45.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.745+0000 7fa3faa00700 10 client.26684 ms_handle_connect on v2:172.21.15.64:6838/1583120004 2024-05-07T02:41:45.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.745+0000 7fa3faa00700 1 -- 172.21.15.19:0/1730779533 <== mds.2 v2:172.21.15.64:6838/1583120004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a5e5b28400 con 0x55a5e5c7c7b0 2024-05-07T02:41:45.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.745+0000 7fa3faa00700 10 client.26684 handle_command_reply: tid=0 2024-05-07T02:41:45.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.745+0000 7fa42a073b80 1 client.26684 shutdown 2024-05-07T02:41:45.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.745+0000 7fa42a073b80 1 -- 172.21.15.19:0/1730779533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e5bf8c40 msgr2=0x55a5e5c05e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:45.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.745+0000 7fa42a073b80 1 --2- 172.21.15.19:0/1730779533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e5bf8c40 0x55a5e5c05e30 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa4080b3290 tx=0x7fa408005a00 comp rx=0 tx=0).stop 2024-05-07T02:41:45.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.745+0000 7fa42a073b80 1 -- 172.21.15.19:0/1730779533 shutdown_connections 2024-05-07T02:41:45.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.745+0000 7fa42a073b80 1 --2- 172.21.15.19:0/1730779533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a5e5c02c30 0x55a5e5c7ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:45.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.745+0000 7fa42a073b80 1 --2- 172.21.15.19:0/1730779533 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x55a5e5c7c7b0 0x55a5e5c7ebd0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:45.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.745+0000 7fa42a073b80 1 --2- 172.21.15.19:0/1730779533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55a5e5bf8c40 0x55a5e5c05e30 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:45.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.745+0000 7fa42a073b80 1 --2- 172.21.15.19:0/1730779533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5e5bf9590 0x55a5e5c06370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:45.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.745+0000 7fa3faa00700 0 client.26684 ms_handle_reset on v2:172.21.15.64:6838/1583120004 2024-05-07T02:41:45.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.745+0000 7fa42a073b80 1 -- 172.21.15.19:0/1730779533 >> 172.21.15.19:0/1730779533 conn(0x55a5e5a7ae40 msgr2=0x55a5e5bffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:41:45.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.746+0000 7fa42a073b80 1 -- 172.21.15.19:0/1730779533 shutdown_connections 2024-05-07T02:41:45.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.746+0000 7fa42a073b80 1 -- 172.21.15.19:0/1730779533 wait complete. 2024-05-07T02:41:45.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.746+0000 7fa42a073b80 20 client.26684 trim_cache size 0 max 16384 2024-05-07T02:41:45.755 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:41:45.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.747+0000 7fa41a000700 1 -- 172.21.15.19:0/3369024952 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fa3fc0b29f0 msgr2=0x7fa3fc0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:45.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.747+0000 7fa41a000700 1 --2- 172.21.15.19:0/3369024952 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fa3fc0b29f0 0x7fa3fc0b4f00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x55a5e5bee190 tx=0x7fa40c00b040 comp rx=0 tx=0).stop 2024-05-07T02:41:45.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.747+0000 7fa41a000700 1 -- 172.21.15.19:0/3369024952 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa414104400 msgr2=0x7fa41419cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:45.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.747+0000 7fa41a000700 1 --2- 172.21.15.19:0/3369024952 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa414104400 0x7fa41419cec0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa4080057c0 tx=0x7fa408005ef0 comp rx=0 tx=0).stop 2024-05-07T02:41:45.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.747+0000 7fa41a000700 1 -- 172.21.15.19:0/3369024952 shutdown_connections 2024-05-07T02:41:45.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.747+0000 7fa41a000700 1 --2- 172.21.15.19:0/3369024952 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fa3fc0b29f0 0x7fa3fc0b4f00 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:45.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.747+0000 7fa41a000700 1 --2- 172.21.15.19:0/3369024952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa414109680 0x7fa4141a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:45.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.747+0000 7fa41a000700 1 --2- 172.21.15.19:0/3369024952 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa414104400 0x7fa41419cec0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:45.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.747+0000 7fa41a000700 1 --2- 172.21.15.19:0/3369024952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa414108d30 0x7fa41419d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:45.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.747+0000 7fa41a000700 1 -- 172.21.15.19:0/3369024952 >> 172.21.15.19:0/3369024952 conn(0x7fa414079d10 msgr2=0x7fa414102690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:41:45.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.747+0000 7fa41a000700 1 -- 172.21.15.19:0/3369024952 shutdown_connections 2024-05-07T02:41:45.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:45.747+0000 7fa41a000700 1 -- 172.21.15.19:0/3369024952 wait complete. 2024-05-07T02:41:46.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:45 smithi064 ceph-mon[34699]: pgmap v306: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:41:46.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:45 smithi019 ceph-mon[29781]: pgmap v306: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:41:46.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:45 smithi027 ceph-mon[34398]: pgmap v306: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:41:46.612 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-07T02:41:47.095 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:41:47.895 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:47 smithi019 ceph-mon[29781]: pgmap v307: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:41:48.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:47 smithi064 ceph-mon[34699]: pgmap v307: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:41:48.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:47 smithi027 ceph-mon[34398]: pgmap v307: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:41:49.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.446+0000 7ff98c600700 1 -- 172.21.15.19:0/3142375227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff984106ed0 msgr2=0x7ff984113360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:49.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.446+0000 7ff98c600700 1 --2- 172.21.15.19:0/3142375227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff984106ed0 0x7ff984113360 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff97c009f90 tx=0x7ff97c00b330 comp rx=0 tx=0).stop 2024-05-07T02:41:49.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.446+0000 7ff98c600700 1 -- 172.21.15.19:0/3142375227 shutdown_connections 2024-05-07T02:41:49.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.446+0000 7ff98c600700 1 --2- 172.21.15.19:0/3142375227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff984106ed0 0x7ff984113360 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:49.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.446+0000 7ff98c600700 1 --2- 172.21.15.19:0/3142375227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff98410c5c0 0x7ff98410c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:49.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.446+0000 7ff98c600700 1 --2- 172.21.15.19:0/3142375227 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff984106580 0x7ff984106990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:49.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.446+0000 7ff98c600700 1 -- 172.21.15.19:0/3142375227 >> 172.21.15.19:0/3142375227 conn(0x7ff984102080 msgr2=0x7ff9841044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:41:49.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.446+0000 7ff98c600700 1 -- 172.21.15.19:0/3142375227 shutdown_connections 2024-05-07T02:41:49.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.446+0000 7ff98c600700 1 -- 172.21.15.19:0/3142375227 wait complete. 2024-05-07T02:41:49.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.447+0000 7ff98c600700 1 Processor -- start 2024-05-07T02:41:49.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.447+0000 7ff98c600700 1 -- start start 2024-05-07T02:41:49.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.448+0000 7ff98c600700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff984106580 0x7ff98419d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:49.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.448+0000 7ff98c600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff984106ed0 0x7ff98419d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:49.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.448+0000 7ff98c600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff98410c5c0 0x7ff9841a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:49.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.448+0000 7ff98c600700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff984116740 con 0x7ff984106ed0 2024-05-07T02:41:49.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.448+0000 7ff98c600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff9841165c0 con 0x7ff98410c5c0 2024-05-07T02:41:49.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.448+0000 7ff98c600700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff9841168c0 con 0x7ff984106580 2024-05-07T02:41:49.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.448+0000 7ff989200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff984106ed0 0x7ff98419d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:49.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.448+0000 7ff989c00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff984106580 0x7ff98419d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:49.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.448+0000 7ff989200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff984106ed0 0x7ff98419d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44368/0 (socket says 172.21.15.19:44368) 2024-05-07T02:41:49.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.448+0000 7ff989200700 1 -- 172.21.15.19:0/3935487968 learned_addr learned my addr 172.21.15.19:0/3935487968 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:41:49.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.448+0000 7ff98a600700 1 --2- 172.21.15.19:0/3935487968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff98410c5c0 0x7ff9841a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:49.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.449+0000 7ff98a600700 1 -- 172.21.15.19:0/3935487968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff984106580 msgr2=0x7ff98419d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:49.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.449+0000 7ff98a600700 1 --2- 172.21.15.19:0/3935487968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff984106580 0x7ff98419d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:49.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.449+0000 7ff98a600700 1 -- 172.21.15.19:0/3935487968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff984106ed0 msgr2=0x7ff98419d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:49.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.449+0000 7ff98a600700 1 --2- 172.21.15.19:0/3935487968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff984106ed0 0x7ff98419d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:49.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.449+0000 7ff98a600700 1 -- 172.21.15.19:0/3935487968 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9841a1ee0 con 0x7ff98410c5c0 2024-05-07T02:41:49.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.449+0000 7ff989200700 1 --2- 172.21.15.19:0/3935487968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff984106ed0 0x7ff98419d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:41:49.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.449+0000 7ff989c00700 1 --2- 172.21.15.19:0/3935487968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff984106580 0x7ff98419d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:41:49.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.449+0000 7ff98a600700 1 --2- 172.21.15.19:0/3935487968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff98410c5c0 0x7ff9841a1840 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff97c00b6c0 tx=0x7ff97c00cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:41:49.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.449+0000 7ff982000700 1 -- 172.21.15.19:0/3935487968 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff97c021070 con 0x7ff98410c5c0 2024-05-07T02:41:49.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.450+0000 7ff982000700 1 -- 172.21.15.19:0/3935487968 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff97c007ac0 con 0x7ff98410c5c0 2024-05-07T02:41:49.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.450+0000 7ff98c600700 1 -- 172.21.15.19:0/3935487968 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9841a2170 con 0x7ff98410c5c0 2024-05-07T02:41:49.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.450+0000 7ff982000700 1 -- 172.21.15.19:0/3935487968 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff97c005040 con 0x7ff98410c5c0 2024-05-07T02:41:49.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.450+0000 7ff98c600700 1 -- 172.21.15.19:0/3935487968 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9841a9af0 con 0x7ff98410c5c0 2024-05-07T02:41:49.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.454+0000 7ff982000700 1 -- 172.21.15.19:0/3935487968 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff97c024070 con 0x7ff98410c5c0 2024-05-07T02:41:49.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.454+0000 7ff99b0a2b80 1 Processor -- start 2024-05-07T02:41:49.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.455+0000 7ff99b0a2b80 1 -- start start 2024-05-07T02:41:49.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.455+0000 7ff99b0a2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560562155470 0x560562155880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:49.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.455+0000 7ff99b0a2b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56056214f620 0x56056214fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:49.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.455+0000 7ff99b0a2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56056214ff70 0x56056215c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:49.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.455+0000 7ff99b0a2b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560562071a20 con 0x56056214ff70 2024-05-07T02:41:49.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.455+0000 7ff99b0a2b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x56056200f760 con 0x56056214f620 2024-05-07T02:41:49.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.455+0000 7ff99b0a2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56056206d9e0 con 0x560562155470 2024-05-07T02:41:49.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.456+0000 7ff989c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56056214f620 0x56056214fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:49.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.456+0000 7ff989c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56056214f620 0x56056214fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:45834/0 (socket says 172.21.15.19:45834) 2024-05-07T02:41:49.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.456+0000 7ff989c00700 1 -- 172.21.15.19:0/844462862 learned_addr learned my addr 172.21.15.19:0/844462862 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:41:49.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.456+0000 7ff989200700 1 --2- 172.21.15.19:0/844462862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56056214ff70 0x56056215c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:49.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.456+0000 7ff989200700 1 --2- 172.21.15.19:0/844462862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560562155470 0x560562155880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:49.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.456+0000 7ff982000700 1 --2- 172.21.15.19:0/3935487968 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff96c0b29f0 0x7ff96c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:49.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.456+0000 7ff989c00700 1 -- 172.21.15.19:0/844462862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560562155470 msgr2=0x560562155880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:49.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.456+0000 7ff989c00700 1 --2- 172.21.15.19:0/844462862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560562155470 0x560562155880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:49.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.456+0000 7ff989c00700 1 -- 172.21.15.19:0/844462862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56056214ff70 msgr2=0x56056215c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:49.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.456+0000 7ff989c00700 1 --2- 172.21.15.19:0/844462862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56056214ff70 0x56056215c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:49.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.456+0000 7ff989c00700 1 -- 172.21.15.19:0/844462862 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560561fbe8d0 con 0x56056214f620 2024-05-07T02:41:49.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.456+0000 7ff98a600700 1 --2- 172.21.15.19:0/3935487968 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff96c0b29f0 0x7ff96c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:49.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.456+0000 7ff982000700 1 -- 172.21.15.19:0/3935487968 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9525+0+0 (secure 0 0 0) 0x7ff97c00f020 con 0x7ff98410c5c0 2024-05-07T02:41:49.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.457+0000 7ff989c00700 1 --2- 172.21.15.19:0/844462862 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56056214f620 0x56056214fa30 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff9780115f0 tx=0x7ff978011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ed8843a9642867a server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:41:49.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.457+0000 7ff976a00700 1 -- 172.21.15.19:0/844462862 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff978004a30 con 0x56056214f620 2024-05-07T02:41:49.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.457+0000 7ff976a00700 1 -- 172.21.15.19:0/844462862 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9780054d0 con 0x56056214f620 2024-05-07T02:41:49.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.457+0000 7ff976a00700 1 -- 172.21.15.19:0/844462862 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff97800e870 con 0x56056214f620 2024-05-07T02:41:49.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.458+0000 7ff99b0a2b80 1 -- 172.21.15.19:0/844462862 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56056214f620 msgr2=0x56056214fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:49.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.458+0000 7ff98a600700 1 --2- 172.21.15.19:0/3935487968 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff96c0b29f0 0x7ff96c0b4f00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff97c0e10c0 tx=0x7ff97c009040 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:41:49.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.458+0000 7ff99b0a2b80 1 --2- 172.21.15.19:0/844462862 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56056214f620 0x56056214fa30 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff9780115f0 tx=0x7ff978011900 comp rx=0 tx=0).stop 2024-05-07T02:41:49.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.458+0000 7ff99b0a2b80 1 -- 172.21.15.19:0/844462862 shutdown_connections 2024-05-07T02:41:49.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.458+0000 7ff99b0a2b80 1 --2- 172.21.15.19:0/844462862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56056214ff70 0x56056215c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:49.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.458+0000 7ff99b0a2b80 1 --2- 172.21.15.19:0/844462862 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560562155470 0x560562155880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:49.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.458+0000 7ff99b0a2b80 1 --2- 172.21.15.19:0/844462862 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56056214f620 0x56056214fa30 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:49.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.458+0000 7ff99b0a2b80 1 -- 172.21.15.19:0/844462862 >> 172.21.15.19:0/844462862 conn(0x560561fd1e40 msgr2=0x56056214a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:41:49.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.458+0000 7ff99b0a2b80 1 -- 172.21.15.19:0/844462862 shutdown_connections 2024-05-07T02:41:49.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.458+0000 7ff99b0a2b80 1 -- 172.21.15.19:0/844462862 wait complete. 2024-05-07T02:41:49.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.459+0000 7ff99b0a2b80 1 -- start start 2024-05-07T02:41:49.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.459+0000 7ff99b0a2b80 1 Processor -- start 2024-05-07T02:41:49.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.460+0000 7ff99b0a2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56056214f620 0x560562159640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:49.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.460+0000 7ff99b0a2b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56056214ff70 0x560562159b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:49.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.460+0000 7ff99b0a2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560562155470 0x56056215a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:49.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.460+0000 7ff99b0a2b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56056215c840 con 0x56056214f620 2024-05-07T02:41:49.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.460+0000 7ff99b0a2b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x560561fd7b20 con 0x56056214ff70 2024-05-07T02:41:49.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.460+0000 7ff99b0a2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56056215c9c0 con 0x560562155470 2024-05-07T02:41:49.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.460+0000 7ff99b0a2b80 10 client.0 _ll_register_callbacks cb 0x560562005380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:41:49.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.460+0000 7ff989200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56056214f620 0x560562159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:49.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.460+0000 7ff989c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56056214ff70 0x560562159b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:49.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.460+0000 7ff989c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56056214ff70 0x560562159b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:45844/0 (socket says 172.21.15.19:45844) 2024-05-07T02:41:49.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.460+0000 7ff989c00700 1 -- 172.21.15.19:0/788548824 learned_addr learned my addr 172.21.15.19:0/788548824 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:41:49.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.460+0000 7ff989200700 1 --2- 172.21.15.19:0/788548824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560562155470 0x56056215a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:49.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.461+0000 7ff989c00700 1 -- 172.21.15.19:0/788548824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560562155470 msgr2=0x56056215a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:49.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.461+0000 7ff989c00700 1 --2- 172.21.15.19:0/788548824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560562155470 0x56056215a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:49.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.461+0000 7ff989c00700 1 -- 172.21.15.19:0/788548824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56056214f620 msgr2=0x560562159640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:49.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.461+0000 7ff989c00700 1 --2- 172.21.15.19:0/788548824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56056214f620 0x560562159640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:49.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.461+0000 7ff989c00700 1 -- 172.21.15.19:0/788548824 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560562037420 con 0x56056214ff70 2024-05-07T02:41:49.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.461+0000 7ff989200700 1 --2- 172.21.15.19:0/788548824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56056214f620 0x560562159640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:41:49.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.461+0000 7ff989c00700 1 --2- 172.21.15.19:0/788548824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56056214ff70 0x560562159b80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff978005920 tx=0x7ff978005450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba7f1fa675f4261e server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:41:49.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.461+0000 7ff96b400700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:41:49.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.461+0000 7ff96b400700 1 -- 172.21.15.19:0/788548824 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff978028070 con 0x56056214ff70 2024-05-07T02:41:49.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.462+0000 7ff99b0a2b80 1 -- 172.21.15.19:0/788548824 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9780054d0 con 0x56056214ff70 2024-05-07T02:41:49.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.462+0000 7ff96b400700 1 -- 172.21.15.19:0/788548824 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9525+0+0 (secure 0 0 0) 0x7ff9780048e0 con 0x56056214ff70 2024-05-07T02:41:49.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.462+0000 7ff96b400700 1 -- 172.21.15.19:0/788548824 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7ff978033050 con 0x56056214ff70 2024-05-07T02:41:49.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.462+0000 7ff99b0a2b80 10 client.26696 fetch_fsmap learned FSMap version 50 2024-05-07T02:41:49.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.462+0000 7ff99b0a2b80 1 -- 172.21.15.19:0/788548824 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5605621674f0 con 0x56056214ff70 2024-05-07T02:41:49.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.462+0000 7ff96b400700 1 -- 172.21.15.19:0/788548824 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9780054d0 con 0x56056214ff70 2024-05-07T02:41:49.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.462+0000 7ff96b400700 1 -- 172.21.15.19:0/788548824 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 50) ==== 8362+0+0 (secure 0 0 0) 0x7ff97800e470 con 0x56056214ff70 2024-05-07T02:41:49.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.462+0000 7ff99b0a2b80 10 client.26696 fetch_fsmap finished waiting for FSMap version 50 2024-05-07T02:41:49.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.463+0000 7ff99b0a2b80 10 client.26696 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-07T02:41:49.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.463+0000 7ff99b0a2b80 20 client.26696 populate_metadata read hostname 'smithi019' 2024-05-07T02:41:49.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.463+0000 7ff99b0a2b80 1 --2- 172.21.15.19:0/788548824 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x560562169a80 0x56056216bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:49.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.463+0000 7ff99b0a2b80 4 client.26696 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:41:49.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.463+0000 7ff99b0a2b80 1 -- 172.21.15.19:0/788548824 --> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56056211c0e0 con 0x560562169a80 2024-05-07T02:41:49.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.463+0000 7ff989200700 1 --2- 172.21.15.19:0/788548824 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x560562169a80 0x56056216bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:49.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.464+0000 7ff989200700 1 --2- 172.21.15.19:0/788548824 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x560562169a80 0x56056216bea0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=332751b4e82df517 server_cookie=2409205f9634e25e in_seq=0 out_seq=0 2024-05-07T02:41:49.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.464+0000 7ff96b400700 10 client.26696 ms_handle_connect on v2:172.21.15.19:6840/3905645246 2024-05-07T02:41:49.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.464+0000 7ff96b400700 1 -- 172.21.15.19:0/788548824 <== mds.1 v2:172.21.15.19:6840/3905645246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56056211c0e0 con 0x560562169a80 2024-05-07T02:41:49.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.464+0000 7ff96b400700 10 client.26696 handle_command_reply: tid=0 2024-05-07T02:41:49.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.465+0000 7ff99b0a2b80 1 client.26696 shutdown 2024-05-07T02:41:49.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.465+0000 7ff99b0a2b80 1 -- 172.21.15.19:0/788548824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56056214ff70 msgr2=0x560562159b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:49.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.465+0000 7ff99b0a2b80 1 --2- 172.21.15.19:0/788548824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56056214ff70 0x560562159b80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff978005920 tx=0x7ff978005450 comp rx=0 tx=0).stop 2024-05-07T02:41:49.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.465+0000 7ff99b0a2b80 1 -- 172.21.15.19:0/788548824 shutdown_connections 2024-05-07T02:41:49.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.465+0000 7ff99b0a2b80 1 --2- 172.21.15.19:0/788548824 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x560562169a80 0x56056216bea0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:49.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.465+0000 7ff99b0a2b80 1 --2- 172.21.15.19:0/788548824 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560562155470 0x56056215a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:49.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.465+0000 7ff99b0a2b80 1 --2- 172.21.15.19:0/788548824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56056214f620 0x560562159640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:49.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.465+0000 7ff99b0a2b80 1 --2- 172.21.15.19:0/788548824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56056214ff70 0x560562159b80 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:49.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.465+0000 7ff96b400700 0 client.26696 ms_handle_reset on v2:172.21.15.19:6840/3905645246 2024-05-07T02:41:49.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.465+0000 7ff99b0a2b80 1 -- 172.21.15.19:0/788548824 >> 172.21.15.19:0/788548824 conn(0x560561fd1e40 msgr2=0x56056214e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:41:49.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.465+0000 7ff99b0a2b80 1 -- 172.21.15.19:0/788548824 shutdown_connections 2024-05-07T02:41:49.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.465+0000 7ff99b0a2b80 1 -- 172.21.15.19:0/788548824 wait complete. 2024-05-07T02:41:49.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.465+0000 7ff99b0a2b80 20 client.26696 trim_cache size 0 max 16384 2024-05-07T02:41:49.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.482+0000 7ff99b0a2b80 1 Processor -- start 2024-05-07T02:41:49.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.482+0000 7ff99b0a2b80 1 -- start start 2024-05-07T02:41:49.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.483+0000 7ff99b0a2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56056214f620 0x560562153d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:49.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.483+0000 7ff99b0a2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56056214ff70 0x560562158e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:49.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.483+0000 7ff99b0a2b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560562155470 0x560562156f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:49.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.483+0000 7ff99b0a2b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff9841b2770 con 0x56056214ff70 2024-05-07T02:41:49.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.483+0000 7ff99b0a2b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff9841b42b0 con 0x560562155470 2024-05-07T02:41:49.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.483+0000 7ff99b0a2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff9841b2e40 con 0x56056214f620 2024-05-07T02:41:49.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.483+0000 7ff989c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56056214ff70 0x560562158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:49.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.483+0000 7ff989c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56056214ff70 0x560562158e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44392/0 (socket says 172.21.15.19:44392) 2024-05-07T02:41:49.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.483+0000 7ff989c00700 1 -- 172.21.15.19:0/2057758752 learned_addr learned my addr 172.21.15.19:0/2057758752 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:41:49.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.483+0000 7ff989200700 1 --2- 172.21.15.19:0/2057758752 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56056214f620 0x560562153d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:49.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.483+0000 7ff989200700 1 --2- 172.21.15.19:0/2057758752 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560562155470 0x560562156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:49.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.483+0000 7ff989c00700 1 -- 172.21.15.19:0/2057758752 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56056214f620 msgr2=0x560562153d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:49.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.483+0000 7ff989c00700 1 --2- 172.21.15.19:0/2057758752 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56056214f620 0x560562153d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:49.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.483+0000 7ff989c00700 1 -- 172.21.15.19:0/2057758752 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560562155470 msgr2=0x560562156f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:49.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.483+0000 7ff989c00700 1 --2- 172.21.15.19:0/2057758752 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560562155470 0x560562156f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:49.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.483+0000 7ff989c00700 1 -- 172.21.15.19:0/2057758752 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56056215bee0 con 0x56056214ff70 2024-05-07T02:41:49.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.484+0000 7ff989c00700 1 --2- 172.21.15.19:0/2057758752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56056214ff70 0x560562158e10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff978011510 tx=0x7ff978009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=411cb9d88568cb78 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:41:49.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.484+0000 7ff976000700 1 -- 172.21.15.19:0/2057758752 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff978000e60 con 0x56056214ff70 2024-05-07T02:41:49.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.484+0000 7ff976000700 1 -- 172.21.15.19:0/2057758752 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9780044d0 con 0x56056214ff70 2024-05-07T02:41:49.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.484+0000 7ff976000700 1 -- 172.21.15.19:0/2057758752 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff978030070 con 0x56056214ff70 2024-05-07T02:41:49.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.485+0000 7ff99b0a2b80 1 -- 172.21.15.19:0/2057758752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56056214ff70 msgr2=0x560562158e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:49.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.485+0000 7ff99b0a2b80 1 --2- 172.21.15.19:0/2057758752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56056214ff70 0x560562158e10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff978011510 tx=0x7ff978009310 comp rx=0 tx=0).stop 2024-05-07T02:41:49.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.485+0000 7ff99b0a2b80 1 -- 172.21.15.19:0/2057758752 shutdown_connections 2024-05-07T02:41:49.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.485+0000 7ff99b0a2b80 1 --2- 172.21.15.19:0/2057758752 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560562155470 0x560562156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:49.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.485+0000 7ff99b0a2b80 1 --2- 172.21.15.19:0/2057758752 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56056214f620 0x560562153d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:49.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.485+0000 7ff99b0a2b80 1 --2- 172.21.15.19:0/2057758752 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56056214ff70 0x560562158e10 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:49.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.485+0000 7ff99b0a2b80 1 -- 172.21.15.19:0/2057758752 >> 172.21.15.19:0/2057758752 conn(0x560561fd1e40 msgr2=0x56056214e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:41:49.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.485+0000 7ff99b0a2b80 1 -- 172.21.15.19:0/2057758752 shutdown_connections 2024-05-07T02:41:49.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.485+0000 7ff99b0a2b80 1 -- 172.21.15.19:0/2057758752 wait complete. 2024-05-07T02:41:49.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.486+0000 7ff99b0a2b80 1 -- start start 2024-05-07T02:41:49.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.486+0000 7ff99b0a2b80 1 Processor -- start 2024-05-07T02:41:49.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.487+0000 7ff99b0a2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56056214f620 0x5605621572d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:49.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.487+0000 7ff99b0a2b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56056214ff70 0x560562161ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:49.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.487+0000 7ff99b0a2b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560562155470 0x560562162280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:49.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.487+0000 7ff99b0a2b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff9841b3be0 con 0x56056214ff70 2024-05-07T02:41:49.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.487+0000 7ff99b0a2b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff9841b3510 con 0x560562155470 2024-05-07T02:41:49.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.487+0000 7ff99b0a2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff9841b0730 con 0x56056214f620 2024-05-07T02:41:49.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.487+0000 7ff99b0a2b80 10 client.0 _ll_register_callbacks cb 0x56056216c3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:41:49.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.487+0000 7ff989c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56056214ff70 0x560562161ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:49.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.487+0000 7ff989200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56056214f620 0x5605621572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:49.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.487+0000 7ff989c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56056214ff70 0x560562161ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44394/0 (socket says 172.21.15.19:44394) 2024-05-07T02:41:49.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.487+0000 7ff989c00700 1 -- 172.21.15.19:0/2633104797 learned_addr learned my addr 172.21.15.19:0/2633104797 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:41:49.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.487+0000 7ff989200700 1 --2- 172.21.15.19:0/2633104797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560562155470 0x560562162280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:49.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.487+0000 7ff989c00700 1 -- 172.21.15.19:0/2633104797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56056214f620 msgr2=0x5605621572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:49.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.487+0000 7ff989c00700 1 --2- 172.21.15.19:0/2633104797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56056214f620 0x5605621572d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:49.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.487+0000 7ff989c00700 1 -- 172.21.15.19:0/2633104797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560562155470 msgr2=0x560562162280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:49.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.487+0000 7ff989c00700 1 --2- 172.21.15.19:0/2633104797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560562155470 0x560562162280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:49.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.487+0000 7ff989c00700 1 -- 172.21.15.19:0/2633104797 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560562155cc0 con 0x56056214ff70 2024-05-07T02:41:49.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.487+0000 7ff989200700 1 --2- 172.21.15.19:0/2633104797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56056214f620 0x5605621572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:41:49.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.488+0000 7ff989c00700 1 --2- 172.21.15.19:0/2633104797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56056214ff70 0x560562161ce0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff978000c00 tx=0x7ff97800e6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=641608fc34f93bc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:41:49.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.488+0000 7ff96b400700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-07T02:41:49.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.488+0000 7ff96b400700 1 -- 172.21.15.19:0/2633104797 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff978030040 con 0x56056214ff70 2024-05-07T02:41:49.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.488+0000 7ff99b0a2b80 1 -- 172.21.15.19:0/2633104797 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9780044d0 con 0x56056214ff70 2024-05-07T02:41:49.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.488+0000 7ff96b400700 1 -- 172.21.15.19:0/2633104797 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9525+0+0 (secure 0 0 0) 0x7ff978004ae0 con 0x56056214ff70 2024-05-07T02:41:49.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.488+0000 7ff96b400700 1 -- 172.21.15.19:0/2633104797 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7ff978028040 con 0x56056214ff70 2024-05-07T02:41:49.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.488+0000 7ff99b0a2b80 10 client.16503 fetch_fsmap learned FSMap version 50 2024-05-07T02:41:49.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.489+0000 7ff99b0a2b80 1 -- 172.21.15.19:0/2633104797 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5605621d3440 con 0x56056214ff70 2024-05-07T02:41:49.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.489+0000 7ff96b400700 1 -- 172.21.15.19:0/2633104797 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9780044d0 con 0x56056214ff70 2024-05-07T02:41:49.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.489+0000 7ff96b400700 1 -- 172.21.15.19:0/2633104797 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 50) ==== 8362+0+0 (secure 0 0 0) 0x7ff978019910 con 0x56056214ff70 2024-05-07T02:41:49.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.489+0000 7ff99b0a2b80 10 client.16503 fetch_fsmap finished waiting for FSMap version 50 2024-05-07T02:41:49.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.489+0000 7ff99b0a2b80 10 client.16503 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-07T02:41:49.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.489+0000 7ff99b0a2b80 20 client.16503 populate_metadata read hostname 'smithi019' 2024-05-07T02:41:49.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.489+0000 7ff99b0a2b80 1 --2- 172.21.15.19:0/2633104797 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5605621d3a30 0x5605621d5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:49.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.489+0000 7ff99b0a2b80 4 client.16503 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:41:49.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.489+0000 7ff99b0a2b80 1 -- 172.21.15.19:0/2633104797 --> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56056207f400 con 0x5605621d3a30 2024-05-07T02:41:49.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.489+0000 7ff989200700 1 --2- 172.21.15.19:0/2633104797 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5605621d3a30 0x5605621d5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:49.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.490+0000 7ff989200700 1 --2- 172.21.15.19:0/2633104797 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5605621d3a30 0x5605621d5e50 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cdeeb5bd2e03a35b server_cookie=9db355c5dabd8a8a in_seq=0 out_seq=0 2024-05-07T02:41:49.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.490+0000 7ff96b400700 10 client.16503 ms_handle_connect on v2:172.21.15.19:6840/3905645246 2024-05-07T02:41:49.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.490+0000 7ff96b400700 1 -- 172.21.15.19:0/2633104797 <== mds.1 v2:172.21.15.19:6840/3905645246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56056207f400 con 0x5605621d3a30 2024-05-07T02:41:49.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.490+0000 7ff96b400700 10 client.16503 handle_command_reply: tid=0 2024-05-07T02:41:49.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.490+0000 7ff99b0a2b80 1 client.16503 shutdown 2024-05-07T02:41:49.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.490+0000 7ff99b0a2b80 1 -- 172.21.15.19:0/2633104797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56056214ff70 msgr2=0x560562161ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:49.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.490+0000 7ff99b0a2b80 1 --2- 172.21.15.19:0/2633104797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56056214ff70 0x560562161ce0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff978000c00 tx=0x7ff97800e6a0 comp rx=0 tx=0).stop 2024-05-07T02:41:49.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.491+0000 7ff99b0a2b80 1 -- 172.21.15.19:0/2633104797 shutdown_connections 2024-05-07T02:41:49.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.491+0000 7ff99b0a2b80 1 --2- 172.21.15.19:0/2633104797 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5605621d3a30 0x5605621d5e50 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:49.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.491+0000 7ff99b0a2b80 1 --2- 172.21.15.19:0/2633104797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x560562155470 0x560562162280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:49.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.491+0000 7ff99b0a2b80 1 --2- 172.21.15.19:0/2633104797 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56056214f620 0x5605621572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:49.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.491+0000 7ff99b0a2b80 1 --2- 172.21.15.19:0/2633104797 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56056214ff70 0x560562161ce0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:49.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.491+0000 7ff99b0a2b80 1 -- 172.21.15.19:0/2633104797 >> 172.21.15.19:0/2633104797 conn(0x560561fd1e40 msgr2=0x560562159600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:41:49.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.491+0000 7ff96b400700 0 client.16503 ms_handle_reset on v2:172.21.15.19:6840/3905645246 2024-05-07T02:41:49.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.491+0000 7ff99b0a2b80 1 -- 172.21.15.19:0/2633104797 shutdown_connections 2024-05-07T02:41:49.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.491+0000 7ff99b0a2b80 1 -- 172.21.15.19:0/2633104797 wait complete. 2024-05-07T02:41:49.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.491+0000 7ff99b0a2b80 20 client.16503 trim_cache size 0 max 16384 2024-05-07T02:41:49.499 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:41:49.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.492+0000 7ff98c600700 1 -- 172.21.15.19:0/3935487968 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff96c0b29f0 msgr2=0x7ff96c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:49.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.492+0000 7ff98c600700 1 --2- 172.21.15.19:0/3935487968 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff96c0b29f0 0x7ff96c0b4f00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff97c0e10c0 tx=0x7ff97c009040 comp rx=0 tx=0).stop 2024-05-07T02:41:49.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.492+0000 7ff98c600700 1 -- 172.21.15.19:0/3935487968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff98410c5c0 msgr2=0x7ff9841a1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:49.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.492+0000 7ff98c600700 1 --2- 172.21.15.19:0/3935487968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff98410c5c0 0x7ff9841a1840 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff97c00b6c0 tx=0x7ff97c00cc30 comp rx=0 tx=0).stop 2024-05-07T02:41:49.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.492+0000 7ff98c600700 1 -- 172.21.15.19:0/3935487968 shutdown_connections 2024-05-07T02:41:49.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.492+0000 7ff98c600700 1 --2- 172.21.15.19:0/3935487968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff98410c5c0 0x7ff9841a1840 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:49.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.492+0000 7ff98c600700 1 --2- 172.21.15.19:0/3935487968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff984106580 0x7ff98419d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:49.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.492+0000 7ff98c600700 1 --2- 172.21.15.19:0/3935487968 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff96c0b29f0 0x7ff96c0b4f00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:49.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.492+0000 7ff98c600700 1 --2- 172.21.15.19:0/3935487968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff984106ed0 0x7ff98419d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:49.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.492+0000 7ff98c600700 1 -- 172.21.15.19:0/3935487968 >> 172.21.15.19:0/3935487968 conn(0x7ff984102080 msgr2=0x7ff9841044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:41:49.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.492+0000 7ff98c600700 1 -- 172.21.15.19:0/3935487968 shutdown_connections 2024-05-07T02:41:49.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:49.492+0000 7ff98c600700 1 -- 172.21.15.19:0/3935487968 wait complete. 2024-05-07T02:41:50.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:49 smithi064 ceph-mon[34699]: pgmap v308: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:41:50.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:49 smithi019 ceph-mon[29781]: pgmap v308: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:41:50.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:49 smithi027 ceph-mon[34398]: pgmap v308: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:41:50.521 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-07T02:41:50.986 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:41:52.043 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:51 smithi019 ceph-mon[29781]: pgmap v309: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:41:52.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:51 smithi064 ceph-mon[34699]: pgmap v309: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:41:52.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:51 smithi027 ceph-mon[34398]: pgmap v309: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:41:53.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.700+0000 7f2c28c00700 1 -- 172.21.15.19:0/1110625968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c24106ed0 msgr2=0x7f2c24113360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:53.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.700+0000 7f2c28c00700 1 --2- 172.21.15.19:0/1110625968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c24106ed0 0x7f2c24113360 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c00b0a0 tx=0x7f2c1c00b3b0 comp rx=0 tx=0).stop 2024-05-07T02:41:53.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.700+0000 7f2c28c00700 1 -- 172.21.15.19:0/1110625968 shutdown_connections 2024-05-07T02:41:53.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.700+0000 7f2c28c00700 1 --2- 172.21.15.19:0/1110625968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c24106ed0 0x7f2c24113360 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:53.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.700+0000 7f2c28c00700 1 --2- 172.21.15.19:0/1110625968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c2410c5c0 0x7f2c2410c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:53.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.700+0000 7f2c28c00700 1 --2- 172.21.15.19:0/1110625968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c24106580 0x7f2c24106990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:53.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.700+0000 7f2c28c00700 1 -- 172.21.15.19:0/1110625968 >> 172.21.15.19:0/1110625968 conn(0x7f2c24102080 msgr2=0x7f2c241044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:41:53.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.700+0000 7f2c28c00700 1 -- 172.21.15.19:0/1110625968 shutdown_connections 2024-05-07T02:41:53.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.700+0000 7f2c28c00700 1 -- 172.21.15.19:0/1110625968 wait complete. 2024-05-07T02:41:53.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.702+0000 7f2c28c00700 1 Processor -- start 2024-05-07T02:41:53.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.702+0000 7f2c28c00700 1 -- start start 2024-05-07T02:41:53.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.702+0000 7f2c28c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c24106580 0x7f2c2419cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:53.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.702+0000 7f2c28c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c24106ed0 0x7f2c2419d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:53.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.702+0000 7f2c28c00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c2410c5c0 0x7f2c241a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:53.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.702+0000 7f2c28c00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2c24116740 con 0x7f2c24106580 2024-05-07T02:41:53.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.702+0000 7f2c28c00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2c241165c0 con 0x7f2c24106ed0 2024-05-07T02:41:53.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.702+0000 7f2c28c00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2c241168c0 con 0x7f2c2410c5c0 2024-05-07T02:41:53.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.703+0000 7f2c21e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c24106580 0x7f2c2419cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:53.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.703+0000 7f2c21400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c24106ed0 0x7f2c2419d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:53.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.703+0000 7f2c21e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c24106580 0x7f2c2419cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44404/0 (socket says 172.21.15.19:44404) 2024-05-07T02:41:53.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.703+0000 7f2c21e00700 1 -- 172.21.15.19:0/1740183645 learned_addr learned my addr 172.21.15.19:0/1740183645 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:41:53.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.703+0000 7f2c22800700 1 --2- 172.21.15.19:0/1740183645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c2410c5c0 0x7f2c241a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:53.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.703+0000 7f2c21400700 1 -- 172.21.15.19:0/1740183645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c2410c5c0 msgr2=0x7f2c241a1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:53.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.703+0000 7f2c21400700 1 --2- 172.21.15.19:0/1740183645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c2410c5c0 0x7f2c241a1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:53.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.703+0000 7f2c21400700 1 -- 172.21.15.19:0/1740183645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c24106580 msgr2=0x7f2c2419cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:53.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.703+0000 7f2c21400700 1 --2- 172.21.15.19:0/1740183645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c24106580 0x7f2c2419cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:53.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.703+0000 7f2c21400700 1 -- 172.21.15.19:0/1740183645 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c241a1ec0 con 0x7f2c24106ed0 2024-05-07T02:41:53.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.703+0000 7f2c21e00700 1 --2- 172.21.15.19:0/1740183645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c24106580 0x7f2c2419cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:41:53.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.704+0000 7f2c21400700 1 --2- 172.21.15.19:0/1740183645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c24106ed0 0x7f2c2419d590 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2c24075b80 tx=0x7f2c1000eca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:41:53.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.704+0000 7f2c1aa00700 1 -- 172.21.15.19:0/1740183645 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c10004220 con 0x7f2c24106ed0 2024-05-07T02:41:53.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.704+0000 7f2c28c00700 1 -- 172.21.15.19:0/1740183645 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c241a21b0 con 0x7f2c24106ed0 2024-05-07T02:41:53.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.704+0000 7f2c1aa00700 1 -- 172.21.15.19:0/1740183645 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c100043c0 con 0x7f2c24106ed0 2024-05-07T02:41:53.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.704+0000 7f2c1aa00700 1 -- 172.21.15.19:0/1740183645 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c10003cf0 con 0x7f2c24106ed0 2024-05-07T02:41:53.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.704+0000 7f2c28c00700 1 -- 172.21.15.19:0/1740183645 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c241a9ad0 con 0x7f2c24106ed0 2024-05-07T02:41:53.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.709+0000 7f2c1aa00700 1 -- 172.21.15.19:0/1740183645 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2c1000c7e0 con 0x7f2c24106ed0 2024-05-07T02:41:53.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.709+0000 7f2c377a1b80 1 Processor -- start 2024-05-07T02:41:53.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.709+0000 7f2c377a1b80 1 -- start start 2024-05-07T02:41:53.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.710+0000 7f2c377a1b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555651a5b470 0x555651a5b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:53.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.710+0000 7f2c377a1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555651a55620 0x555651a55a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:53.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.710+0000 7f2c377a1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555651a55f70 0x555651a62150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:53.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.710+0000 7f2c377a1b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555651977a20 con 0x555651a55f70 2024-05-07T02:41:53.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.710+0000 7f2c377a1b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x555651915760 con 0x555651a5b470 2024-05-07T02:41:53.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.710+0000 7f2c377a1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5556519739e0 con 0x555651a55620 2024-05-07T02:41:53.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.710+0000 7f2c21e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555651a55620 0x555651a55a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:53.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.710+0000 7f2c21e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555651a55620 0x555651a55a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:57368/0 (socket says 172.21.15.19:57368) 2024-05-07T02:41:53.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.710+0000 7f2c21e00700 1 -- 172.21.15.19:0/2868559318 learned_addr learned my addr 172.21.15.19:0/2868559318 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:41:53.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.710+0000 7f2c21400700 1 --2- 172.21.15.19:0/2868559318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555651a55f70 0x555651a62150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:53.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.710+0000 7f2c22800700 1 --2- 172.21.15.19:0/2868559318 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555651a5b470 0x555651a5b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:53.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.711+0000 7f2c21e00700 1 -- 172.21.15.19:0/2868559318 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555651a5b470 msgr2=0x555651a5b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:53.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.711+0000 7f2c21e00700 1 --2- 172.21.15.19:0/2868559318 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555651a5b470 0x555651a5b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:53.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.711+0000 7f2c21e00700 1 -- 172.21.15.19:0/2868559318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555651a55f70 msgr2=0x555651a62150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:53.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.711+0000 7f2c21e00700 1 --2- 172.21.15.19:0/2868559318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555651a55f70 0x555651a62150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:53.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.711+0000 7f2c21e00700 1 -- 172.21.15.19:0/2868559318 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556518c48d0 con 0x555651a55620 2024-05-07T02:41:53.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.711+0000 7f2c21400700 1 --2- 172.21.15.19:0/2868559318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555651a55f70 0x555651a62150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:41:53.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.711+0000 7f2c1aa00700 1 --2- 172.21.15.19:0/1740183645 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f2c080b29f0 0x7f2c080b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:53.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.711+0000 7f2c21e00700 1 --2- 172.21.15.19:0/2868559318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555651a55620 0x555651a55a30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2c14011680 tx=0x7f2c14011990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f7d6c1ea89a4ee server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:41:53.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.711+0000 7f2c1aa00700 1 -- 172.21.15.19:0/1740183645 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9525+0+0 (secure 0 0 0) 0x7f2c100e2bb0 con 0x7f2c24106ed0 2024-05-07T02:41:53.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.711+0000 7f2c22800700 1 --2- 172.21.15.19:0/1740183645 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f2c080b29f0 0x7f2c080b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:53.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.711+0000 7f2c0f400700 1 -- 172.21.15.19:0/2868559318 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c14004030 con 0x555651a55620 2024-05-07T02:41:53.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.711+0000 7f2c0f400700 1 -- 172.21.15.19:0/2868559318 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c14003680 con 0x555651a55620 2024-05-07T02:41:53.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.711+0000 7f2c0f400700 1 -- 172.21.15.19:0/2868559318 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c14019740 con 0x555651a55620 2024-05-07T02:41:53.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.712+0000 7f2c377a1b80 1 -- 172.21.15.19:0/2868559318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555651a55620 msgr2=0x555651a55a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:53.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.712+0000 7f2c22800700 1 --2- 172.21.15.19:0/1740183645 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f2c080b29f0 0x7f2c080b4f00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c00b780 tx=0x7f2c1c006210 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:41:53.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.712+0000 7f2c377a1b80 1 --2- 172.21.15.19:0/2868559318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555651a55620 0x555651a55a30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2c14011680 tx=0x7f2c14011990 comp rx=0 tx=0).stop 2024-05-07T02:41:53.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.713+0000 7f2c377a1b80 1 -- 172.21.15.19:0/2868559318 shutdown_connections 2024-05-07T02:41:53.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.713+0000 7f2c377a1b80 1 --2- 172.21.15.19:0/2868559318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555651a55f70 0x555651a62150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:53.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.713+0000 7f2c377a1b80 1 --2- 172.21.15.19:0/2868559318 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555651a5b470 0x555651a5b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:53.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.713+0000 7f2c377a1b80 1 --2- 172.21.15.19:0/2868559318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555651a55620 0x555651a55a30 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:53.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.713+0000 7f2c377a1b80 1 -- 172.21.15.19:0/2868559318 >> 172.21.15.19:0/2868559318 conn(0x5556518d7e40 msgr2=0x555651a50380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:41:53.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.713+0000 7f2c377a1b80 1 -- 172.21.15.19:0/2868559318 shutdown_connections 2024-05-07T02:41:53.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.713+0000 7f2c377a1b80 1 -- 172.21.15.19:0/2868559318 wait complete. 2024-05-07T02:41:53.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.713+0000 7f2c377a1b80 1 -- start start 2024-05-07T02:41:53.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.714+0000 7f2c377a1b80 1 Processor -- start 2024-05-07T02:41:53.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.714+0000 7f2c377a1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555651a55620 0x555651a5f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:53.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.714+0000 7f2c377a1b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555651a55f70 0x555651a5fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:53.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.714+0000 7f2c377a1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555651a5b470 0x555651a600f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:53.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.714+0000 7f2c377a1b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555651a62840 con 0x555651a55620 2024-05-07T02:41:53.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.714+0000 7f2c377a1b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5556518ddb20 con 0x555651a55f70 2024-05-07T02:41:53.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.714+0000 7f2c377a1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555651a629c0 con 0x555651a5b470 2024-05-07T02:41:53.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.714+0000 7f2c377a1b80 10 client.0 _ll_register_callbacks cb 0x55565190b380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:41:53.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.715+0000 7f2c21e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555651a55620 0x555651a5f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:53.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.715+0000 7f2c22800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555651a5b470 0x555651a600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:53.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.715+0000 7f2c21e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555651a55620 0x555651a5f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44424/0 (socket says 172.21.15.19:44424) 2024-05-07T02:41:53.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.715+0000 7f2c21e00700 1 -- 172.21.15.19:0/2603367939 learned_addr learned my addr 172.21.15.19:0/2603367939 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:41:53.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.715+0000 7f2c21400700 1 --2- 172.21.15.19:0/2603367939 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555651a55f70 0x555651a5fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:53.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.715+0000 7f2c22800700 1 -- 172.21.15.19:0/2603367939 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555651a55f70 msgr2=0x555651a5fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:53.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.715+0000 7f2c22800700 1 --2- 172.21.15.19:0/2603367939 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555651a55f70 0x555651a5fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:53.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.715+0000 7f2c22800700 1 -- 172.21.15.19:0/2603367939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555651a55620 msgr2=0x555651a5f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:53.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.715+0000 7f2c22800700 1 --2- 172.21.15.19:0/2603367939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555651a55620 0x555651a5f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:53.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.715+0000 7f2c22800700 1 -- 172.21.15.19:0/2603367939 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55565193d420 con 0x555651a5b470 2024-05-07T02:41:53.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.716+0000 7f2c22800700 1 --2- 172.21.15.19:0/2603367939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555651a5b470 0x555651a600f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c0062a0 tx=0x7f2c1c0099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ed945adf97785c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:41:53.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.716+0000 7f2c03e00700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:41:53.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.716+0000 7f2c03e00700 1 -- 172.21.15.19:0/2603367939 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c02d070 con 0x555651a5b470 2024-05-07T02:41:53.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.716+0000 7f2c377a1b80 1 -- 172.21.15.19:0/2603367939 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c14003680 con 0x555651a5b470 2024-05-07T02:41:53.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.716+0000 7f2c03e00700 1 -- 172.21.15.19:0/2603367939 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9525+0+0 (secure 0 0 0) 0x7f2c1c007c30 con 0x555651a5b470 2024-05-07T02:41:53.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.716+0000 7f2c03e00700 1 -- 172.21.15.19:0/2603367939 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f2c1c023070 con 0x555651a5b470 2024-05-07T02:41:53.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.716+0000 7f2c377a1b80 10 client.26674 fetch_fsmap learned FSMap version 50 2024-05-07T02:41:53.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.716+0000 7f2c377a1b80 1 -- 172.21.15.19:0/2603367939 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x555651a6d4e0 con 0x555651a5b470 2024-05-07T02:41:53.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.717+0000 7f2c03e00700 1 -- 172.21.15.19:0/2603367939 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c14003680 con 0x555651a5b470 2024-05-07T02:41:53.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.717+0000 7f2c03e00700 1 -- 172.21.15.19:0/2603367939 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 50) ==== 8362+0+0 (secure 0 0 0) 0x7f2c1c011070 con 0x555651a5b470 2024-05-07T02:41:53.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.717+0000 7f2c377a1b80 10 client.26674 fetch_fsmap finished waiting for FSMap version 50 2024-05-07T02:41:53.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.717+0000 7f2c377a1b80 10 client.26674 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:41:53.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.717+0000 7f2c377a1b80 20 client.26674 populate_metadata read hostname 'smithi019' 2024-05-07T02:41:53.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.717+0000 7f2c377a1b80 1 --2- 172.21.15.19:0/2603367939 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x555651a6fa70 0x555651a71e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:53.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.717+0000 7f2c377a1b80 4 client.26674 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:41:53.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.717+0000 7f2c377a1b80 1 -- 172.21.15.19:0/2603367939 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555651a220e0 con 0x555651a6fa70 2024-05-07T02:41:53.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.718+0000 7f2c21e00700 1 --2- 172.21.15.19:0/2603367939 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x555651a6fa70 0x555651a71e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:53.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.718+0000 7f2c21e00700 1 --2- 172.21.15.19:0/2603367939 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x555651a6fa70 0x555651a71e90 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f071adf9acec63b server_cookie=a55d6da01c1d4710 in_seq=0 out_seq=0 2024-05-07T02:41:53.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.718+0000 7f2c03e00700 10 client.26674 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:41:53.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.719+0000 7f2c03e00700 1 -- 172.21.15.19:0/2603367939 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555651a220e0 con 0x555651a6fa70 2024-05-07T02:41:53.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.719+0000 7f2c03e00700 10 client.26674 handle_command_reply: tid=0 2024-05-07T02:41:53.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.719+0000 7f2c377a1b80 1 client.26674 shutdown 2024-05-07T02:41:53.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.720+0000 7f2c377a1b80 1 -- 172.21.15.19:0/2603367939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555651a5b470 msgr2=0x555651a600f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:53.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.720+0000 7f2c377a1b80 1 --2- 172.21.15.19:0/2603367939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555651a5b470 0x555651a600f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c0062a0 tx=0x7f2c1c0099e0 comp rx=0 tx=0).stop 2024-05-07T02:41:53.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.720+0000 7f2c377a1b80 1 -- 172.21.15.19:0/2603367939 shutdown_connections 2024-05-07T02:41:53.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.720+0000 7f2c377a1b80 1 --2- 172.21.15.19:0/2603367939 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x555651a6fa70 0x555651a71e90 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:53.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.720+0000 7f2c377a1b80 1 --2- 172.21.15.19:0/2603367939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555651a5b470 0x555651a600f0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:53.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.720+0000 7f2c377a1b80 1 --2- 172.21.15.19:0/2603367939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555651a55620 0x555651a5f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:53.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.720+0000 7f2c377a1b80 1 --2- 172.21.15.19:0/2603367939 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555651a55f70 0x555651a5fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:53.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.720+0000 7f2c377a1b80 1 -- 172.21.15.19:0/2603367939 >> 172.21.15.19:0/2603367939 conn(0x5556518d7e40 msgr2=0x555651a54610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:41:53.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.720+0000 7f2c03e00700 0 client.26674 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:41:53.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.720+0000 7f2c377a1b80 1 -- 172.21.15.19:0/2603367939 shutdown_connections 2024-05-07T02:41:53.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.720+0000 7f2c377a1b80 1 -- 172.21.15.19:0/2603367939 wait complete. 2024-05-07T02:41:53.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.720+0000 7f2c377a1b80 20 client.26674 trim_cache size 0 max 16384 2024-05-07T02:41:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.737+0000 7f2c377a1b80 1 Processor -- start 2024-05-07T02:41:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.737+0000 7f2c377a1b80 1 -- start start 2024-05-07T02:41:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.737+0000 7f2c377a1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555651a55620 0x555651a60060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.737+0000 7f2c377a1b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555651a55f70 0x555651a59d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.737+0000 7f2c377a1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555651a5b470 0x555651a6b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:53.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.737+0000 7f2c377a1b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2c241b2900 con 0x555651a55620 2024-05-07T02:41:53.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.737+0000 7f2c377a1b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2c241b4440 con 0x555651a55f70 2024-05-07T02:41:53.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.737+0000 7f2c377a1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2c241b2fd0 con 0x555651a5b470 2024-05-07T02:41:53.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.738+0000 7f2c21400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555651a55f70 0x555651a59d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:53.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.738+0000 7f2c22800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555651a5b470 0x555651a6b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:53.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.738+0000 7f2c21400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555651a55f70 0x555651a59d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:45904/0 (socket says 172.21.15.19:45904) 2024-05-07T02:41:53.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.738+0000 7f2c21400700 1 -- 172.21.15.19:0/2257171434 learned_addr learned my addr 172.21.15.19:0/2257171434 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:41:53.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.738+0000 7f2c21e00700 1 --2- 172.21.15.19:0/2257171434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555651a55620 0x555651a60060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:53.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.738+0000 7f2c21400700 1 -- 172.21.15.19:0/2257171434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555651a5b470 msgr2=0x555651a6b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:53.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.738+0000 7f2c21400700 1 --2- 172.21.15.19:0/2257171434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555651a5b470 0x555651a6b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:53.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.738+0000 7f2c21400700 1 -- 172.21.15.19:0/2257171434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555651a55620 msgr2=0x555651a60060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:53.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.738+0000 7f2c21400700 1 --2- 172.21.15.19:0/2257171434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555651a55620 0x555651a60060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:53.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.738+0000 7f2c21400700 1 -- 172.21.15.19:0/2257171434 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555651a5bc70 con 0x555651a55f70 2024-05-07T02:41:53.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.739+0000 7f2c21400700 1 --2- 172.21.15.19:0/2257171434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555651a55f70 0x555651a59d90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2c100a6cb0 tx=0x7f2c10003710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bf4bb1cf07e9dc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:41:53.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.739+0000 7f2c0ea00700 1 -- 172.21.15.19:0/2257171434 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c100b2050 con 0x555651a55f70 2024-05-07T02:41:53.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.739+0000 7f2c0ea00700 1 -- 172.21.15.19:0/2257171434 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c100b21f0 con 0x555651a55f70 2024-05-07T02:41:53.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.739+0000 7f2c0ea00700 1 -- 172.21.15.19:0/2257171434 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c100a9a40 con 0x555651a55f70 2024-05-07T02:41:53.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.740+0000 7f2c377a1b80 1 -- 172.21.15.19:0/2257171434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555651a55f70 msgr2=0x555651a59d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:53.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.740+0000 7f2c377a1b80 1 --2- 172.21.15.19:0/2257171434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555651a55f70 0x555651a59d90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2c100a6cb0 tx=0x7f2c10003710 comp rx=0 tx=0).stop 2024-05-07T02:41:53.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.740+0000 7f2c377a1b80 1 -- 172.21.15.19:0/2257171434 shutdown_connections 2024-05-07T02:41:53.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.740+0000 7f2c377a1b80 1 --2- 172.21.15.19:0/2257171434 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555651a5b470 0x555651a6b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:53.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.740+0000 7f2c377a1b80 1 --2- 172.21.15.19:0/2257171434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555651a55620 0x555651a60060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:53.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.740+0000 7f2c377a1b80 1 --2- 172.21.15.19:0/2257171434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555651a55f70 0x555651a59d90 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:53.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.740+0000 7f2c377a1b80 1 -- 172.21.15.19:0/2257171434 >> 172.21.15.19:0/2257171434 conn(0x5556518d7e40 msgr2=0x555651a4fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:41:53.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.740+0000 7f2c377a1b80 1 -- 172.21.15.19:0/2257171434 shutdown_connections 2024-05-07T02:41:53.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.740+0000 7f2c377a1b80 1 -- 172.21.15.19:0/2257171434 wait complete. 2024-05-07T02:41:53.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.741+0000 7f2c377a1b80 1 -- start start 2024-05-07T02:41:53.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.741+0000 7f2c377a1b80 1 Processor -- start 2024-05-07T02:41:53.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.742+0000 7f2c377a1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555651a55620 0x555651a67c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:53.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.742+0000 7f2c377a1b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555651a55f70 0x555651a68190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:53.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.742+0000 7f2c377a1b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555651a5b470 0x555651a76640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:53.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.742+0000 7f2c377a1b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2c241b3d70 con 0x555651a55f70 2024-05-07T02:41:53.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.742+0000 7f2c377a1b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2c241b36a0 con 0x555651a5b470 2024-05-07T02:41:53.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.742+0000 7f2c377a1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2c241b08c0 con 0x555651a55620 2024-05-07T02:41:53.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.742+0000 7f2c377a1b80 10 client.0 _ll_register_callbacks cb 0x55565190b380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:41:53.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.742+0000 7f2c21400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555651a55f70 0x555651a68190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:53.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.742+0000 7f2c22800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555651a5b470 0x555651a76640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:53.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.742+0000 7f2c21400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555651a55f70 0x555651a68190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44448/0 (socket says 172.21.15.19:44448) 2024-05-07T02:41:53.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.742+0000 7f2c21400700 1 -- 172.21.15.19:0/2369454533 learned_addr learned my addr 172.21.15.19:0/2369454533 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:41:53.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.742+0000 7f2c21e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555651a55620 0x555651a67c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:53.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.742+0000 7f2c22800700 1 -- 172.21.15.19:0/2369454533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555651a55620 msgr2=0x555651a67c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:53.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.742+0000 7f2c22800700 1 --2- 172.21.15.19:0/2369454533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555651a55620 0x555651a67c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:53.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.742+0000 7f2c22800700 1 -- 172.21.15.19:0/2369454533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555651a55f70 msgr2=0x555651a68190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:53.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.742+0000 7f2c22800700 1 --2- 172.21.15.19:0/2369454533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555651a55f70 0x555651a68190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:53.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.742+0000 7f2c22800700 1 -- 172.21.15.19:0/2369454533 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555651a76b80 con 0x555651a5b470 2024-05-07T02:41:53.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.743+0000 7f2c22800700 1 --2- 172.21.15.19:0/2369454533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555651a5b470 0x555651a76640 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c009e60 tx=0x7f2c1c005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9dbef880526003ff server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:41:53.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.743+0000 7f2c03e00700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:41:53.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.743+0000 7f2c03e00700 1 -- 172.21.15.19:0/2369454533 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c02d040 con 0x555651a5b470 2024-05-07T02:41:53.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.743+0000 7f2c377a1b80 1 -- 172.21.15.19:0/2369454533 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c100b21f0 con 0x555651a5b470 2024-05-07T02:41:53.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.743+0000 7f2c03e00700 1 -- 172.21.15.19:0/2369454533 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9525+0+0 (secure 0 0 0) 0x7f2c1c005250 con 0x555651a5b470 2024-05-07T02:41:53.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.743+0000 7f2c03e00700 1 -- 172.21.15.19:0/2369454533 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f2c1c005450 con 0x555651a5b470 2024-05-07T02:41:53.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.744+0000 7f2c377a1b80 10 client.26720 fetch_fsmap learned FSMap version 50 2024-05-07T02:41:53.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.744+0000 7f2c377a1b80 1 -- 172.21.15.19:0/2369454533 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x555651ad9940 con 0x555651a5b470 2024-05-07T02:41:53.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.744+0000 7f2c03e00700 1 -- 172.21.15.19:0/2369454533 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c100b21f0 con 0x555651a5b470 2024-05-07T02:41:53.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.744+0000 7f2c03e00700 1 -- 172.21.15.19:0/2369454533 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 50) ==== 8362+0+0 (secure 0 0 0) 0x7f2c1c011040 con 0x555651a5b470 2024-05-07T02:41:53.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.744+0000 7f2c377a1b80 10 client.26720 fetch_fsmap finished waiting for FSMap version 50 2024-05-07T02:41:53.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.744+0000 7f2c377a1b80 10 client.26720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:41:53.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.744+0000 7f2c377a1b80 20 client.26720 populate_metadata read hostname 'smithi019' 2024-05-07T02:41:53.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.744+0000 7f2c377a1b80 1 --2- 172.21.15.19:0/2369454533 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x555651ad9fe0 0x555651adc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:53.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.744+0000 7f2c377a1b80 4 client.26720 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:41:53.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.744+0000 7f2c377a1b80 1 -- 172.21.15.19:0/2369454533 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555651985400 con 0x555651ad9fe0 2024-05-07T02:41:53.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.745+0000 7f2c21e00700 1 --2- 172.21.15.19:0/2369454533 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x555651ad9fe0 0x555651adc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:53.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.745+0000 7f2c21e00700 1 --2- 172.21.15.19:0/2369454533 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x555651ad9fe0 0x555651adc400 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ff8793f03e0044f server_cookie=4f0b7d1e3a59dc59 in_seq=0 out_seq=0 2024-05-07T02:41:53.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.745+0000 7f2c03e00700 10 client.26720 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:41:53.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.745+0000 7f2c03e00700 1 -- 172.21.15.19:0/2369454533 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555651985400 con 0x555651ad9fe0 2024-05-07T02:41:53.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.745+0000 7f2c03e00700 10 client.26720 handle_command_reply: tid=0 2024-05-07T02:41:53.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.746+0000 7f2c377a1b80 1 client.26720 shutdown 2024-05-07T02:41:53.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.746+0000 7f2c377a1b80 1 -- 172.21.15.19:0/2369454533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555651a5b470 msgr2=0x555651a76640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:53.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.746+0000 7f2c377a1b80 1 --2- 172.21.15.19:0/2369454533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555651a5b470 0x555651a76640 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c009e60 tx=0x7f2c1c005f10 comp rx=0 tx=0).stop 2024-05-07T02:41:53.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.746+0000 7f2c377a1b80 1 -- 172.21.15.19:0/2369454533 shutdown_connections 2024-05-07T02:41:53.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.746+0000 7f2c377a1b80 1 --2- 172.21.15.19:0/2369454533 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x555651ad9fe0 0x555651adc400 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:53.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.746+0000 7f2c377a1b80 1 --2- 172.21.15.19:0/2369454533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555651a5b470 0x555651a76640 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:53.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.746+0000 7f2c377a1b80 1 --2- 172.21.15.19:0/2369454533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555651a55620 0x555651a67c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:53.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.746+0000 7f2c377a1b80 1 --2- 172.21.15.19:0/2369454533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555651a55f70 0x555651a68190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:53.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.746+0000 7f2c377a1b80 1 -- 172.21.15.19:0/2369454533 >> 172.21.15.19:0/2369454533 conn(0x5556518d7e40 msgr2=0x555651a4fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:41:53.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.746+0000 7f2c03e00700 0 client.26720 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:41:53.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.746+0000 7f2c377a1b80 1 -- 172.21.15.19:0/2369454533 shutdown_connections 2024-05-07T02:41:53.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.746+0000 7f2c377a1b80 1 -- 172.21.15.19:0/2369454533 wait complete. 2024-05-07T02:41:53.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.746+0000 7f2c377a1b80 20 client.26720 trim_cache size 0 max 16384 2024-05-07T02:41:53.753 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:41:53.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.747+0000 7f2c28c00700 1 -- 172.21.15.19:0/1740183645 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f2c080b29f0 msgr2=0x7f2c080b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:53.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.747+0000 7f2c28c00700 1 --2- 172.21.15.19:0/1740183645 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f2c080b29f0 0x7f2c080b4f00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c00b780 tx=0x7f2c1c006210 comp rx=0 tx=0).stop 2024-05-07T02:41:53.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.747+0000 7f2c28c00700 1 -- 172.21.15.19:0/1740183645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c24106ed0 msgr2=0x7f2c2419d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:53.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.747+0000 7f2c28c00700 1 --2- 172.21.15.19:0/1740183645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c24106ed0 0x7f2c2419d590 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2c24075b80 tx=0x7f2c1000eca0 comp rx=0 tx=0).stop 2024-05-07T02:41:53.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.747+0000 7f2c28c00700 1 -- 172.21.15.19:0/1740183645 shutdown_connections 2024-05-07T02:41:53.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.747+0000 7f2c28c00700 1 --2- 172.21.15.19:0/1740183645 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c2410c5c0 0x7f2c241a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:53.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.747+0000 7f2c28c00700 1 --2- 172.21.15.19:0/1740183645 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f2c080b29f0 0x7f2c080b4f00 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:53.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.747+0000 7f2c28c00700 1 --2- 172.21.15.19:0/1740183645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c24106580 0x7f2c2419cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:53.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.747+0000 7f2c28c00700 1 --2- 172.21.15.19:0/1740183645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c24106ed0 0x7f2c2419d590 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:53.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.747+0000 7f2c28c00700 1 -- 172.21.15.19:0/1740183645 >> 172.21.15.19:0/1740183645 conn(0x7f2c24102080 msgr2=0x7f2c24104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:41:53.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.747+0000 7f2c28c00700 1 -- 172.21.15.19:0/1740183645 shutdown_connections 2024-05-07T02:41:53.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:53.747+0000 7f2c28c00700 1 -- 172.21.15.19:0/1740183645 wait complete. 2024-05-07T02:41:54.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:53 smithi064 ceph-mon[34699]: pgmap v310: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:41:54.135 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:53 smithi019 ceph-mon[29781]: pgmap v310: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:41:54.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:53 smithi027 ceph-mon[34398]: pgmap v310: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:41:55.753 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-07T02:41:55.754 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-07T02:41:56.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:55 smithi064 ceph-mon[34699]: pgmap v311: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:41:56.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:55 smithi019 ceph-mon[29781]: pgmap v311: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:41:56.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:55 smithi027 ceph-mon[34398]: pgmap v311: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:41:56.227 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:41:58.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:57 smithi064 ceph-mon[34699]: pgmap v312: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:41:58.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:57 smithi019 ceph-mon[29781]: pgmap v312: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:41:58.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:57 smithi027 ceph-mon[34398]: pgmap v312: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:41:58.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.769+0000 7fc5d7e00700 1 -- 172.21.15.19:0/3481954720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5d810e700 msgr2=0x7fc5d8110be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:58.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.769+0000 7fc5d7e00700 1 --2- 172.21.15.19:0/3481954720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5d810e700 0x7fc5d8110be0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc5d000b0a0 tx=0x7fc5d000b3b0 comp rx=0 tx=0).stop 2024-05-07T02:41:58.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.770+0000 7fc5d7e00700 1 -- 172.21.15.19:0/3481954720 shutdown_connections 2024-05-07T02:41:58.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.770+0000 7fc5d7e00700 1 --2- 172.21.15.19:0/3481954720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5d810e700 0x7fc5d8110be0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:58.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.770+0000 7fc5d7e00700 1 --2- 172.21.15.19:0/3481954720 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5d8105290 0x7fc5d81056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:58.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.770+0000 7fc5d7e00700 1 --2- 172.21.15.19:0/3481954720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5d8105be0 0x7fc5d810e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:58.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.770+0000 7fc5d7e00700 1 -- 172.21.15.19:0/3481954720 >> 172.21.15.19:0/3481954720 conn(0x7fc5d8078ac0 msgr2=0x7fc5d8078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:41:58.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.770+0000 7fc5d7e00700 1 -- 172.21.15.19:0/3481954720 shutdown_connections 2024-05-07T02:41:58.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.770+0000 7fc5d7e00700 1 -- 172.21.15.19:0/3481954720 wait complete. 2024-05-07T02:41:58.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.772+0000 7fc5d7e00700 1 Processor -- start 2024-05-07T02:41:58.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.773+0000 7fc5d7e00700 1 -- start start 2024-05-07T02:41:58.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.774+0000 7fc5d7e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5d8105290 0x7fc5d81a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:58.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.774+0000 7fc5d7e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5d8105be0 0x7fc5d81a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:58.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.774+0000 7fc5d7e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5d810e700 0x7fc5d819aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:58.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.774+0000 7fc5d7e00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d81142b0 con 0x7fc5d810e700 2024-05-07T02:41:58.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.774+0000 7fc5d7e00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d8114130 con 0x7fc5d8105be0 2024-05-07T02:41:58.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.774+0000 7fc5d7e00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d8114430 con 0x7fc5d8105290 2024-05-07T02:41:58.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.774+0000 7fc5d6a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5d8105290 0x7fc5d81a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:58.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.774+0000 7fc5cfe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5d8105be0 0x7fc5d81a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:58.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.774+0000 7fc5d6a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5d8105290 0x7fc5d81a0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:57418/0 (socket says 172.21.15.19:57418) 2024-05-07T02:41:58.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.774+0000 7fc5cfe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5d8105be0 0x7fc5d81a0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:45926/0 (socket says 172.21.15.19:45926) 2024-05-07T02:41:58.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.774+0000 7fc5d6a00700 1 -- 172.21.15.19:0/3664735195 learned_addr learned my addr 172.21.15.19:0/3664735195 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:41:58.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.775+0000 7fc5cfe00700 1 -- 172.21.15.19:0/3664735195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5d8105290 msgr2=0x7fc5d81a0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:58.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.775+0000 7fc5cfe00700 1 --2- 172.21.15.19:0/3664735195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5d8105290 0x7fc5d81a0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:58.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.775+0000 7fc5cfe00700 1 -- 172.21.15.19:0/3664735195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5d810e700 msgr2=0x7fc5d819aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:58.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.775+0000 7fc5cfe00700 1 --2- 172.21.15.19:0/3664735195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5d810e700 0x7fc5d819aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:58.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.775+0000 7fc5cfe00700 1 -- 172.21.15.19:0/3664735195 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5d819b100 con 0x7fc5d8105be0 2024-05-07T02:41:58.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.775+0000 7fc5cfe00700 1 --2- 172.21.15.19:0/3664735195 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5d8105be0 0x7fc5d81a0d70 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc5c000e9f0 tx=0x7fc5c000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:41:58.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.775+0000 7fc5cf400700 1 -- 172.21.15.19:0/3664735195 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c000cbd0 con 0x7fc5d8105be0 2024-05-07T02:41:58.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.775+0000 7fc5d7e00700 1 -- 172.21.15.19:0/3664735195 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5d819b3f0 con 0x7fc5d8105be0 2024-05-07T02:41:58.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.775+0000 7fc5cf400700 1 -- 172.21.15.19:0/3664735195 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5c000cd70 con 0x7fc5d8105be0 2024-05-07T02:41:58.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.775+0000 7fc5cf400700 1 -- 172.21.15.19:0/3664735195 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c0018680 con 0x7fc5d8105be0 2024-05-07T02:41:58.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.776+0000 7fc5d7e00700 1 -- 172.21.15.19:0/3664735195 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5d81a76c0 con 0x7fc5d8105be0 2024-05-07T02:41:58.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.780+0000 7fc5cf400700 1 -- 172.21.15.19:0/3664735195 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc5c0018820 con 0x7fc5d8105be0 2024-05-07T02:41:58.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.780+0000 7fc5ec754b80 1 Processor -- start 2024-05-07T02:41:58.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.781+0000 7fc5ec754b80 1 -- start start 2024-05-07T02:41:58.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.781+0000 7fc5ec754b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bbba5dcbe0 0x55bbba5dcff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:58.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.781+0000 7fc5ec754b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbba5dd530 0x55bbba5eab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:58.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.781+0000 7fc5ec754b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbba5eb0c0 0x55bbba5ed4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:58.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.781+0000 7fc5ec754b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bbba4feaa0 con 0x55bbba5dd530 2024-05-07T02:41:58.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.781+0000 7fc5ec754b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55bbba49c7e0 con 0x55bbba5dcbe0 2024-05-07T02:41:58.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.781+0000 7fc5ec754b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55bbba4faa60 con 0x55bbba5eb0c0 2024-05-07T02:41:58.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.781+0000 7fc5d6a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbba5dd530 0x55bbba5eab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:58.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.782+0000 7fc5cf400700 1 --2- 172.21.15.19:0/3664735195 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc5bc0b29f0 0x7fc5bc0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:58.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.782+0000 7fc5d7400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bbba5dcbe0 0x55bbba5dcff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:58.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.782+0000 7fc5d6a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbba5dd530 0x55bbba5eab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44490/0 (socket says 172.21.15.19:44490) 2024-05-07T02:41:58.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.782+0000 7fc5d6a00700 1 -- 172.21.15.19:0/2411038688 learned_addr learned my addr 172.21.15.19:0/2411038688 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:41:58.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.782+0000 7fc5cfe00700 1 --2- 172.21.15.19:0/2411038688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbba5eb0c0 0x55bbba5ed4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:58.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.782+0000 7fc5cf400700 1 -- 172.21.15.19:0/3664735195 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9525+0+0 (secure 0 0 0) 0x7fc5c00e2c30 con 0x7fc5d8105be0 2024-05-07T02:41:58.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.782+0000 7fc5d7400700 1 --2- 172.21.15.19:0/3664735195 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc5bc0b29f0 0x7fc5bc0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:58.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.782+0000 7fc5d6a00700 1 -- 172.21.15.19:0/2411038688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbba5eb0c0 msgr2=0x55bbba5ed4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:58.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.782+0000 7fc5d6a00700 1 --2- 172.21.15.19:0/2411038688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbba5eb0c0 0x55bbba5ed4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:58.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.782+0000 7fc5d6a00700 1 -- 172.21.15.19:0/2411038688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bbba5dcbe0 msgr2=0x55bbba5dcff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:58.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.782+0000 7fc5d6a00700 1 --2- 172.21.15.19:0/2411038688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bbba5dcbe0 0x55bbba5dcff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:58.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.782+0000 7fc5d6a00700 1 -- 172.21.15.19:0/2411038688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbba44b950 con 0x55bbba5dd530 2024-05-07T02:41:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.782+0000 7fc5d7400700 1 --2- 172.21.15.19:0/2411038688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bbba5dcbe0 0x55bbba5dcff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:41:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.783+0000 7fc5d6a00700 1 --2- 172.21.15.19:0/2411038688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbba5dd530 0x55bbba5eab80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8009e30 tx=0x7fc5c8011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85e819a97a3b7c65 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:41:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.783+0000 7fc5c7e00700 1 -- 172.21.15.19:0/2411038688 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c801e070 con 0x55bbba5dd530 2024-05-07T02:41:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.783+0000 7fc5c7e00700 1 -- 172.21.15.19:0/2411038688 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5c8019ae0 con 0x55bbba5dd530 2024-05-07T02:41:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.783+0000 7fc5d7400700 1 --2- 172.21.15.19:0/3664735195 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc5bc0b29f0 0x7fc5bc0b4f00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0009fd0 tx=0x7fc5d0006000 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:41:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.783+0000 7fc5c7e00700 1 -- 172.21.15.19:0/2411038688 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c800fac0 con 0x55bbba5dd530 2024-05-07T02:41:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.784+0000 7fc5ec754b80 1 -- 172.21.15.19:0/2411038688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbba5dd530 msgr2=0x55bbba5eab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.784+0000 7fc5ec754b80 1 --2- 172.21.15.19:0/2411038688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbba5dd530 0x55bbba5eab80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8009e30 tx=0x7fc5c8011720 comp rx=0 tx=0).stop 2024-05-07T02:41:58.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.784+0000 7fc5ec754b80 1 -- 172.21.15.19:0/2411038688 shutdown_connections 2024-05-07T02:41:58.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.784+0000 7fc5ec754b80 1 --2- 172.21.15.19:0/2411038688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbba5eb0c0 0x55bbba5ed4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:58.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.784+0000 7fc5ec754b80 1 --2- 172.21.15.19:0/2411038688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bbba5dcbe0 0x55bbba5dcff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:58.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.784+0000 7fc5ec754b80 1 --2- 172.21.15.19:0/2411038688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbba5dd530 0x55bbba5eab80 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:58.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.784+0000 7fc5ec754b80 1 -- 172.21.15.19:0/2411038688 >> 172.21.15.19:0/2411038688 conn(0x55bbba45eec0 msgr2=0x55bbba5d7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:41:58.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.785+0000 7fc5ec754b80 1 -- 172.21.15.19:0/2411038688 shutdown_connections 2024-05-07T02:41:58.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.785+0000 7fc5ec754b80 1 -- 172.21.15.19:0/2411038688 wait complete. 2024-05-07T02:41:58.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.785+0000 7fc5ec754b80 1 -- start start 2024-05-07T02:41:58.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.786+0000 7fc5ec754b80 1 Processor -- start 2024-05-07T02:41:58.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.786+0000 7fc5ec754b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbba5dcbe0 0x55bbba5e80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:58.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.786+0000 7fc5ec754b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbba5dd530 0x55bbba5e8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:58.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.786+0000 7fc5ec754b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bbba5eb0c0 0x55bbba5e8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:58.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.786+0000 7fc5ec754b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bbba5edec0 con 0x55bbba5dcbe0 2024-05-07T02:41:58.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.786+0000 7fc5ec754b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55bbba464ba0 con 0x55bbba5eb0c0 2024-05-07T02:41:58.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.786+0000 7fc5ec754b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55bbba5ee040 con 0x55bbba5dd530 2024-05-07T02:41:58.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.786+0000 7fc5ec754b80 10 client.0 _ll_register_callbacks cb 0x55bbba492400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:41:58.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.786+0000 7fc5d6a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbba5dcbe0 0x55bbba5e80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:58.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.787+0000 7fc5d6a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbba5dcbe0 0x55bbba5e80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44500/0 (socket says 172.21.15.19:44500) 2024-05-07T02:41:58.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.787+0000 7fc5cfe00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbba5dd530 0x55bbba5e8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:58.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.787+0000 7fc5d6a00700 1 -- 172.21.15.19:0/2967162989 learned_addr learned my addr 172.21.15.19:0/2967162989 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:41:58.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.787+0000 7fc5d7400700 1 --2- 172.21.15.19:0/2967162989 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bbba5eb0c0 0x55bbba5e8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:58.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.787+0000 7fc5d6a00700 1 -- 172.21.15.19:0/2967162989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbba5dd530 msgr2=0x55bbba5e8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:58.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.787+0000 7fc5d6a00700 1 --2- 172.21.15.19:0/2967162989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbba5dd530 0x55bbba5e8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:58.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.787+0000 7fc5d6a00700 1 -- 172.21.15.19:0/2967162989 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bbba5eb0c0 msgr2=0x55bbba5e8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:58.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.787+0000 7fc5d6a00700 1 --2- 172.21.15.19:0/2967162989 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bbba5eb0c0 0x55bbba5e8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:58.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.787+0000 7fc5d6a00700 1 -- 172.21.15.19:0/2967162989 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbba4c44a0 con 0x55bbba5dcbe0 2024-05-07T02:41:58.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.787+0000 7fc5cfe00700 1 --2- 172.21.15.19:0/2967162989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbba5dd530 0x55bbba5e8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:41:58.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.787+0000 7fc5d7400700 1 --2- 172.21.15.19:0/2967162989 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bbba5eb0c0 0x55bbba5e8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:41:58.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.788+0000 7fc5d6a00700 1 --2- 172.21.15.19:0/2967162989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbba5dcbe0 0x55bbba5e80d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8000c00 tx=0x7fc5c8019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d91b9529e13ad8 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:41:58.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.788+0000 7fc5c4c00700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-07T02:41:58.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.788+0000 7fc5c4c00700 1 -- 172.21.15.19:0/2967162989 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8029020 con 0x55bbba5dcbe0 2024-05-07T02:41:58.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.788+0000 7fc5c4c00700 1 -- 172.21.15.19:0/2967162989 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9525+0+0 (secure 0 0 0) 0x7fc5c8004240 con 0x55bbba5dcbe0 2024-05-07T02:41:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.788+0000 7fc5ec754b80 1 -- 172.21.15.19:0/2967162989 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5c8019ae0 con 0x55bbba5dcbe0 2024-05-07T02:41:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.788+0000 7fc5c4c00700 1 -- 172.21.15.19:0/2967162989 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fc5c801e050 con 0x55bbba5dcbe0 2024-05-07T02:41:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.788+0000 7fc5ec754b80 10 client.16530 fetch_fsmap learned FSMap version 50 2024-05-07T02:41:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.788+0000 7fc5ec754b80 1 -- 172.21.15.19:0/2967162989 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55bbba5f8b40 con 0x55bbba5dcbe0 2024-05-07T02:41:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.789+0000 7fc5c4c00700 1 -- 172.21.15.19:0/2967162989 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5c8019ae0 con 0x55bbba5dcbe0 2024-05-07T02:41:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.789+0000 7fc5c4c00700 1 -- 172.21.15.19:0/2967162989 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 50) ==== 8362+0+0 (secure 0 0 0) 0x7fc5c8004980 con 0x55bbba5dcbe0 2024-05-07T02:41:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.789+0000 7fc5ec754b80 10 client.16530 fetch_fsmap finished waiting for FSMap version 50 2024-05-07T02:41:58.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.789+0000 7fc5ec754b80 10 client.16530 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:41:58.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.789+0000 7fc5ec754b80 20 client.16530 populate_metadata read hostname 'smithi019' 2024-05-07T02:41:58.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.789+0000 7fc5ec754b80 1 --2- 172.21.15.19:0/2967162989 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55bbba5fb060 0x55bbba5fd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:58.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.789+0000 7fc5ec754b80 4 client.16530 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:41:58.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.789+0000 7fc5ec754b80 1 -- 172.21.15.19:0/2967162989 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbba5a9160 con 0x55bbba5fb060 2024-05-07T02:41:58.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.790+0000 7fc5cfe00700 1 --2- 172.21.15.19:0/2967162989 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55bbba5fb060 0x55bbba5fd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:58.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.790+0000 7fc5cfe00700 1 --2- 172.21.15.19:0/2967162989 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55bbba5fb060 0x55bbba5fd480 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b892e75e06dec5b0 server_cookie=e6689b1ba19fa6ed in_seq=0 out_seq=0 2024-05-07T02:41:58.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.790+0000 7fc5c4c00700 10 client.16530 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:41:58.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.791+0000 7fc5c4c00700 1 -- 172.21.15.19:0/2967162989 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fc5c008ac70 con 0x55bbba5fb060 2024-05-07T02:41:58.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.791+0000 7fc5c4c00700 10 client.16530 handle_command_reply: tid=0 2024-05-07T02:41:58.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.791+0000 7fc5ec754b80 1 client.16530 shutdown 2024-05-07T02:41:58.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.791+0000 7fc5ec754b80 1 -- 172.21.15.19:0/2967162989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbba5dcbe0 msgr2=0x55bbba5e80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:58.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.791+0000 7fc5ec754b80 1 --2- 172.21.15.19:0/2967162989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbba5dcbe0 0x55bbba5e80d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8000c00 tx=0x7fc5c8019660 comp rx=0 tx=0).stop 2024-05-07T02:41:58.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.791+0000 7fc5ec754b80 1 -- 172.21.15.19:0/2967162989 shutdown_connections 2024-05-07T02:41:58.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.791+0000 7fc5ec754b80 1 --2- 172.21.15.19:0/2967162989 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55bbba5fb060 0x55bbba5fd480 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:58.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.791+0000 7fc5ec754b80 1 --2- 172.21.15.19:0/2967162989 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bbba5eb0c0 0x55bbba5e8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:58.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.791+0000 7fc5ec754b80 1 --2- 172.21.15.19:0/2967162989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbba5dcbe0 0x55bbba5e80d0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:58.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.791+0000 7fc5ec754b80 1 --2- 172.21.15.19:0/2967162989 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbba5dd530 0x55bbba5e8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:58.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.791+0000 7fc5ec754b80 1 -- 172.21.15.19:0/2967162989 >> 172.21.15.19:0/2967162989 conn(0x55bbba45eec0 msgr2=0x55bbba5d9680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:41:58.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.792+0000 7fc5c4c00700 0 client.16530 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:41:58.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.792+0000 7fc5ec754b80 1 -- 172.21.15.19:0/2967162989 shutdown_connections 2024-05-07T02:41:58.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.792+0000 7fc5ec754b80 1 -- 172.21.15.19:0/2967162989 wait complete. 2024-05-07T02:41:58.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.792+0000 7fc5ec754b80 20 client.16530 trim_cache size 0 max 16384 2024-05-07T02:41:58.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.816+0000 7fc5ec754b80 1 Processor -- start 2024-05-07T02:41:58.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.816+0000 7fc5ec754b80 1 -- start start 2024-05-07T02:41:58.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.816+0000 7fc5ec754b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbba5dcbe0 0x55bbba5dbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:58.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.816+0000 7fc5ec754b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bbba5dd530 0x55bbba5ea570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:58.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.816+0000 7fc5ec754b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbba5eb0c0 0x55bbba5e72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:58.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.816+0000 7fc5ec754b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d81ae430 con 0x55bbba5eb0c0 2024-05-07T02:41:58.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.816+0000 7fc5ec754b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d81aff70 con 0x55bbba5dd530 2024-05-07T02:41:58.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.816+0000 7fc5ec754b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d81aeb00 con 0x55bbba5dcbe0 2024-05-07T02:41:58.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.817+0000 7fc5d6a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbba5dcbe0 0x55bbba5dbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:58.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.817+0000 7fc5cfe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bbba5dd530 0x55bbba5ea570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:58.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.817+0000 7fc5d6a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbba5dcbe0 0x55bbba5dbd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:57448/0 (socket says 172.21.15.19:57448) 2024-05-07T02:41:58.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.817+0000 7fc5d6a00700 1 -- 172.21.15.19:0/3859698533 learned_addr learned my addr 172.21.15.19:0/3859698533 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:41:58.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.817+0000 7fc5d7400700 1 --2- 172.21.15.19:0/3859698533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbba5eb0c0 0x55bbba5e72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:58.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.817+0000 7fc5d6a00700 1 -- 172.21.15.19:0/3859698533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bbba5dd530 msgr2=0x55bbba5ea570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:58.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.817+0000 7fc5d6a00700 1 --2- 172.21.15.19:0/3859698533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bbba5dd530 0x55bbba5ea570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:58.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.817+0000 7fc5d6a00700 1 -- 172.21.15.19:0/3859698533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbba5eb0c0 msgr2=0x55bbba5e72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:58.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.817+0000 7fc5d6a00700 1 --2- 172.21.15.19:0/3859698533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbba5eb0c0 0x55bbba5e72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:58.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.817+0000 7fc5d6a00700 1 -- 172.21.15.19:0/3859698533 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbba5dd940 con 0x55bbba5dcbe0 2024-05-07T02:41:58.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.817+0000 7fc5cfe00700 1 --2- 172.21.15.19:0/3859698533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bbba5dd530 0x55bbba5ea570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:41:58.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.818+0000 7fc5d6a00700 1 --2- 172.21.15.19:0/3859698533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbba5dcbe0 0x55bbba5dbd00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8004e70 tx=0x7fc5c8011c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d27acc5e2e16958f server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:41:58.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.818+0000 7fc5c7400700 1 -- 172.21.15.19:0/3859698533 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8029040 con 0x55bbba5dcbe0 2024-05-07T02:41:58.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.818+0000 7fc5c7400700 1 -- 172.21.15.19:0/3859698533 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5c801f040 con 0x55bbba5dcbe0 2024-05-07T02:41:58.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.818+0000 7fc5c7400700 1 -- 172.21.15.19:0/3859698533 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8005240 con 0x55bbba5dcbe0 2024-05-07T02:41:58.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.819+0000 7fc5ec754b80 1 -- 172.21.15.19:0/3859698533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbba5dcbe0 msgr2=0x55bbba5dbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:58.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.819+0000 7fc5ec754b80 1 --2- 172.21.15.19:0/3859698533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbba5dcbe0 0x55bbba5dbd00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8004e70 tx=0x7fc5c8011c00 comp rx=0 tx=0).stop 2024-05-07T02:41:58.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.819+0000 7fc5ec754b80 1 -- 172.21.15.19:0/3859698533 shutdown_connections 2024-05-07T02:41:58.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.819+0000 7fc5ec754b80 1 --2- 172.21.15.19:0/3859698533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbba5eb0c0 0x55bbba5e72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:58.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.819+0000 7fc5ec754b80 1 --2- 172.21.15.19:0/3859698533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbba5dcbe0 0x55bbba5dbd00 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:58.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.819+0000 7fc5ec754b80 1 --2- 172.21.15.19:0/3859698533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bbba5dd530 0x55bbba5ea570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:58.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.819+0000 7fc5ec754b80 1 -- 172.21.15.19:0/3859698533 >> 172.21.15.19:0/3859698533 conn(0x55bbba45eec0 msgr2=0x55bbba5e2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:41:58.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.820+0000 7fc5ec754b80 1 -- 172.21.15.19:0/3859698533 shutdown_connections 2024-05-07T02:41:58.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.820+0000 7fc5ec754b80 1 -- 172.21.15.19:0/3859698533 wait complete. 2024-05-07T02:41:58.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.820+0000 7fc5ec754b80 1 -- start start 2024-05-07T02:41:58.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.820+0000 7fc5ec754b80 1 Processor -- start 2024-05-07T02:41:58.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.821+0000 7fc5ec754b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbba5dcbe0 0x55bbba5ef130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:58.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.821+0000 7fc5ec754b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbba5dd530 0x55bbba5ef670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:58.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.821+0000 7fc5ec754b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bbba5eb0c0 0x55bbba5efbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:58.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.821+0000 7fc5ec754b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d81af8a0 con 0x55bbba5dd530 2024-05-07T02:41:58.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.821+0000 7fc5ec754b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d81af1d0 con 0x55bbba5eb0c0 2024-05-07T02:41:58.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.821+0000 7fc5ec754b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d81ac3f0 con 0x55bbba5dcbe0 2024-05-07T02:41:58.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.821+0000 7fc5ec754b80 10 client.0 _ll_register_callbacks cb 0x55bbba492400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:41:58.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.821+0000 7fc5d6a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbba5dcbe0 0x55bbba5ef130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:58.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.821+0000 7fc5d6a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbba5dcbe0 0x55bbba5ef130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:57454/0 (socket says 172.21.15.19:57454) 2024-05-07T02:41:58.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.821+0000 7fc5d6a00700 1 -- 172.21.15.19:0/2869781446 learned_addr learned my addr 172.21.15.19:0/2869781446 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:41:58.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.821+0000 7fc5cfe00700 1 --2- 172.21.15.19:0/2869781446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbba5dd530 0x55bbba5ef670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:58.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.822+0000 7fc5d7400700 1 --2- 172.21.15.19:0/2869781446 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bbba5eb0c0 0x55bbba5efbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:58.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.822+0000 7fc5cfe00700 1 -- 172.21.15.19:0/2869781446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbba5dcbe0 msgr2=0x55bbba5ef130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:58.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.822+0000 7fc5cfe00700 1 --2- 172.21.15.19:0/2869781446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbba5dcbe0 0x55bbba5ef130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:58.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.822+0000 7fc5cfe00700 1 -- 172.21.15.19:0/2869781446 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bbba5eb0c0 msgr2=0x55bbba5efbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:41:58.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.822+0000 7fc5cfe00700 1 --2- 172.21.15.19:0/2869781446 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bbba5eb0c0 0x55bbba5efbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:58.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.822+0000 7fc5cfe00700 1 -- 172.21.15.19:0/2869781446 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbba6648a0 con 0x55bbba5dd530 2024-05-07T02:41:58.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.822+0000 7fc5d6a00700 1 --2- 172.21.15.19:0/2869781446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbba5dcbe0 0x55bbba5ef130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:41:58.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.822+0000 7fc5cfe00700 1 --2- 172.21.15.19:0/2869781446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbba5dd530 0x55bbba5ef670 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc5c00dff30 tx=0x7fc5c00e5080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d254eb1fc0e0bf7d server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:41:58.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.822+0000 7fc5c4c00700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-07T02:41:58.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.823+0000 7fc5c4c00700 1 -- 172.21.15.19:0/2869781446 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c00a9f20 con 0x55bbba5dd530 2024-05-07T02:41:58.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.823+0000 7fc5c4c00700 1 -- 172.21.15.19:0/2869781446 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9525+0+0 (secure 0 0 0) 0x7fc5c00b2490 con 0x55bbba5dd530 2024-05-07T02:41:58.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.823+0000 7fc5ec754b80 1 -- 172.21.15.19:0/2869781446 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5c801f040 con 0x55bbba5dd530 2024-05-07T02:41:58.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.823+0000 7fc5c4c00700 1 -- 172.21.15.19:0/2869781446 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fc5c00aa1c0 con 0x55bbba5dd530 2024-05-07T02:41:58.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.823+0000 7fc5ec754b80 10 client.16536 fetch_fsmap learned FSMap version 50 2024-05-07T02:41:58.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.823+0000 7fc5ec754b80 1 -- 172.21.15.19:0/2869781446 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55bbba665b40 con 0x55bbba5dd530 2024-05-07T02:41:58.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.823+0000 7fc5c4c00700 1 -- 172.21.15.19:0/2869781446 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5c801f040 con 0x55bbba5dd530 2024-05-07T02:41:58.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.823+0000 7fc5c4c00700 1 -- 172.21.15.19:0/2869781446 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 50) ==== 8362+0+0 (secure 0 0 0) 0x7fc5c00cc0b0 con 0x55bbba5dd530 2024-05-07T02:41:58.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.824+0000 7fc5ec754b80 10 client.16536 fetch_fsmap finished waiting for FSMap version 50 2024-05-07T02:41:58.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.824+0000 7fc5ec754b80 10 client.16536 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:41:58.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.824+0000 7fc5ec754b80 20 client.16536 populate_metadata read hostname 'smithi019' 2024-05-07T02:41:58.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.824+0000 7fc5ec754b80 1 --2- 172.21.15.19:0/2869781446 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55bbba6661b0 0x55bbba6685d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:41:58.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.824+0000 7fc5ec754b80 4 client.16536 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-07T02:41:58.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.824+0000 7fc5ec754b80 1 -- 172.21.15.19:0/2869781446 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55bbba50c480 con 0x55bbba6661b0 2024-05-07T02:41:58.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.824+0000 7fc5d6a00700 1 --2- 172.21.15.19:0/2869781446 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55bbba6661b0 0x55bbba6685d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:41:58.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.825+0000 7fc5d6a00700 1 --2- 172.21.15.19:0/2869781446 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55bbba6661b0 0x55bbba6685d0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e641d90e4120f54 server_cookie=38156e89adc34eb8 in_seq=0 out_seq=0 2024-05-07T02:41:58.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.825+0000 7fc5c4c00700 10 client.16536 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:41:58.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.826+0000 7fc5c4c00700 1 -- 172.21.15.19:0/2869781446 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bbba50c480 con 0x55bbba6661b0 2024-05-07T02:41:58.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.826+0000 7fc5c4c00700 10 client.16536 handle_command_reply: tid=0 2024-05-07T02:41:58.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.826+0000 7fc5ec754b80 1 client.16536 shutdown 2024-05-07T02:41:58.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.827+0000 7fc5ec754b80 1 -- 172.21.15.19:0/2869781446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbba5dd530 msgr2=0x55bbba5ef670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:58.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.827+0000 7fc5ec754b80 1 --2- 172.21.15.19:0/2869781446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbba5dd530 0x55bbba5ef670 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc5c00dff30 tx=0x7fc5c00e5080 comp rx=0 tx=0).stop 2024-05-07T02:41:58.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.827+0000 7fc5ec754b80 1 -- 172.21.15.19:0/2869781446 shutdown_connections 2024-05-07T02:41:58.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.827+0000 7fc5ec754b80 1 --2- 172.21.15.19:0/2869781446 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55bbba6661b0 0x55bbba6685d0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:58.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.827+0000 7fc5ec754b80 1 --2- 172.21.15.19:0/2869781446 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55bbba5eb0c0 0x55bbba5efbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:58.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.827+0000 7fc5ec754b80 1 --2- 172.21.15.19:0/2869781446 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bbba5dcbe0 0x55bbba5ef130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:58.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.827+0000 7fc5ec754b80 1 --2- 172.21.15.19:0/2869781446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbba5dd530 0x55bbba5ef670 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:58.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.827+0000 7fc5c4c00700 0 client.16536 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:41:58.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.827+0000 7fc5ec754b80 1 -- 172.21.15.19:0/2869781446 >> 172.21.15.19:0/2869781446 conn(0x55bbba45eec0 msgr2=0x55bbba5e9530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:41:58.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.827+0000 7fc5ec754b80 1 -- 172.21.15.19:0/2869781446 shutdown_connections 2024-05-07T02:41:58.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.827+0000 7fc5ec754b80 1 -- 172.21.15.19:0/2869781446 wait complete. 2024-05-07T02:41:58.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.827+0000 7fc5ec754b80 20 client.16536 trim_cache size 0 max 16384 2024-05-07T02:41:58.836 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:41:58.836 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-05-07T02:41:58.836 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "95d5246f-b66d-4cc4-a0f7-76d373531436", 2024-05-07T02:41:58.836 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-05-07T02:41:58.836 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:41:58.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.828+0000 7fc5d7e00700 1 -- 172.21.15.19:0/3664735195 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc5bc0b29f0 msgr2=0x7fc5bc0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:58.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.828+0000 7fc5d7e00700 1 --2- 172.21.15.19:0/3664735195 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc5bc0b29f0 0x7fc5bc0b4f00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0009fd0 tx=0x7fc5d0006000 comp rx=0 tx=0).stop 2024-05-07T02:41:58.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.828+0000 7fc5d7e00700 1 -- 172.21.15.19:0/3664735195 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5d8105be0 msgr2=0x7fc5d81a0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:41:58.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.828+0000 7fc5d7e00700 1 --2- 172.21.15.19:0/3664735195 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5d8105be0 0x7fc5d81a0d70 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc5c000e9f0 tx=0x7fc5c000ed00 comp rx=0 tx=0).stop 2024-05-07T02:41:58.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.828+0000 7fc5d7e00700 1 -- 172.21.15.19:0/3664735195 shutdown_connections 2024-05-07T02:41:58.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.828+0000 7fc5d7e00700 1 --2- 172.21.15.19:0/3664735195 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fc5bc0b29f0 0x7fc5bc0b4f00 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:58.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.828+0000 7fc5d7e00700 1 --2- 172.21.15.19:0/3664735195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5d810e700 0x7fc5d819aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:58.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.828+0000 7fc5d7e00700 1 --2- 172.21.15.19:0/3664735195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5d8105290 0x7fc5d81a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:58.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.828+0000 7fc5d7e00700 1 --2- 172.21.15.19:0/3664735195 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5d8105be0 0x7fc5d81a0d70 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:41:58.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.828+0000 7fc5d7e00700 1 -- 172.21.15.19:0/3664735195 >> 172.21.15.19:0/3664735195 conn(0x7fc5d8078ac0 msgr2=0x7fc5d80fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:41:58.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.828+0000 7fc5d7e00700 1 -- 172.21.15.19:0/3664735195 shutdown_connections 2024-05-07T02:41:58.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:41:58.828+0000 7fc5d7e00700 1 -- 172.21.15.19:0/3664735195 wait complete. 2024-05-07T02:41:59.776 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-07T02:42:00.012 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:59 smithi019 ceph-mon[29781]: pgmap v313: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:42:00.012 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:59 smithi019 ceph-mon[29781]: scrub queued for path: / 2024-05-07T02:42:00.012 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:59 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:42:00.012 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:59 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:42:00.012 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:59 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:42:00.013 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:59 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:42:00.013 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:59 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:42:00.013 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:59 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:42:00.013 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:59 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:42:00.013 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:59 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:42:00.013 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:59 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:42:00.013 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:59 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:42:00.013 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:59 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:42:00.014 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:59 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:42:00.014 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:59 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:42:00.014 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:59 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:42:00.014 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:59 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:42:00.015 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:59 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:42:00.015 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:59 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:42:00.015 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:59 smithi019 ceph-mon[29781]: scrub summary: active paths [/] 2024-05-07T02:42:00.015 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:59 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:42:00.015 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:59 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:42:00.016 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:41:59 smithi019 ceph-mon[29781]: scrub summary: idle+waiting paths [/] 2024-05-07T02:42:00.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:59 smithi064 ceph-mon[34699]: pgmap v313: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:42:00.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:59 smithi064 ceph-mon[34699]: scrub queued for path: / 2024-05-07T02:42:00.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:59 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:42:00.110 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:59 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:42:00.110 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:59 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:42:00.110 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:59 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:42:00.110 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:59 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:42:00.110 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:59 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:42:00.110 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:59 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:42:00.110 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:59 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:42:00.110 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:59 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:42:00.110 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:59 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:42:00.111 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:59 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:42:00.111 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:59 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:42:00.111 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:59 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:42:00.111 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:59 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:42:00.111 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:59 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:42:00.111 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:59 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:42:00.111 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:59 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:42:00.111 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:59 smithi064 ceph-mon[34699]: scrub summary: active paths [/] 2024-05-07T02:42:00.111 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:59 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:42:00.112 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:59 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:42:00.112 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:41:59 smithi064 ceph-mon[34699]: scrub summary: idle+waiting paths [/] 2024-05-07T02:42:00.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:59 smithi027 ceph-mon[34398]: pgmap v313: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:42:00.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:59 smithi027 ceph-mon[34398]: scrub queued for path: / 2024-05-07T02:42:00.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:59 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:42:00.208 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:59 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:42:00.208 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:59 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:42:00.208 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:59 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:42:00.208 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:59 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:42:00.208 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:59 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:42:00.208 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:59 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:42:00.208 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:59 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:42:00.208 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:59 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:42:00.208 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:59 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:42:00.209 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:59 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:42:00.209 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:59 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:42:00.209 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:59 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:42:00.209 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:59 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:42:00.209 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:59 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:42:00.209 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:59 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:42:00.209 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:59 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:42:00.209 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:59 smithi027 ceph-mon[34398]: scrub summary: active paths [/] 2024-05-07T02:42:00.209 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:59 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:42:00.210 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:59 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:42:00.210 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:41:59 smithi027 ceph-mon[34398]: scrub summary: idle+waiting paths [/] 2024-05-07T02:42:00.230 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:42:02.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:01 smithi064 ceph-mon[34699]: pgmap v314: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:42:02.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:01 smithi019 ceph-mon[29781]: pgmap v314: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:42:02.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:01 smithi027 ceph-mon[34398]: pgmap v314: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:42:02.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.801+0000 7f3259e00700 1 -- 172.21.15.19:0/1751037253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3254109870 msgr2=0x7f32541111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:02.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.801+0000 7f3259e00700 1 --2- 172.21.15.19:0/1751037253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3254109870 0x7f32541111d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f324c00b0d0 tx=0x7f324c00b3e0 comp rx=0 tx=0).stop 2024-05-07T02:42:02.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.802+0000 7f3259e00700 1 -- 172.21.15.19:0/1751037253 shutdown_connections 2024-05-07T02:42:02.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.802+0000 7f3259e00700 1 --2- 172.21.15.19:0/1751037253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3254109870 0x7f32541111d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:02.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.802+0000 7f3259e00700 1 --2- 172.21.15.19:0/1751037253 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32541045f0 0x7f3254104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:02.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.802+0000 7f3259e00700 1 --2- 172.21.15.19:0/1751037253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3254108f20 0x7f3254109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:02.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.802+0000 7f3259e00700 1 -- 172.21.15.19:0/1751037253 >> 172.21.15.19:0/1751037253 conn(0x7f3254100040 msgr2=0x7f3254102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:42:02.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.802+0000 7f3259e00700 1 -- 172.21.15.19:0/1751037253 shutdown_connections 2024-05-07T02:42:02.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.802+0000 7f3259e00700 1 -- 172.21.15.19:0/1751037253 wait complete. 2024-05-07T02:42:02.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.803+0000 7f3259e00700 1 Processor -- start 2024-05-07T02:42:02.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.804+0000 7f3259e00700 1 -- start start 2024-05-07T02:42:02.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.804+0000 7f3259e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32541045f0 0x7f3254075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:02.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.804+0000 7f3259e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3254108f20 0x7f3254076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:02.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.804+0000 7f3259e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3254109870 0x7f3254076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:02.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.804+0000 7f3259e00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f32541144a0 con 0x7f3254108f20 2024-05-07T02:42:02.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.804+0000 7f3259e00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3254114320 con 0x7f3254109870 2024-05-07T02:42:02.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.804+0000 7f3259e00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3254114620 con 0x7f32541045f0 2024-05-07T02:42:02.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.805+0000 7f3252a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32541045f0 0x7f3254075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:02.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.805+0000 7f3252a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32541045f0 0x7f3254075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:41000/0 (socket says 172.21.15.19:41000) 2024-05-07T02:42:02.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.805+0000 7f3252a00700 1 -- 172.21.15.19:0/466777631 learned_addr learned my addr 172.21.15.19:0/466777631 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:42:02.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.805+0000 7f3253400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3254109870 0x7f3254076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:02.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.805+0000 7f3252000700 1 --2- 172.21.15.19:0/466777631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3254108f20 0x7f3254076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:02.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.805+0000 7f3253400700 1 -- 172.21.15.19:0/466777631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32541045f0 msgr2=0x7f3254075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:02.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.805+0000 7f3253400700 1 --2- 172.21.15.19:0/466777631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32541045f0 0x7f3254075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:02.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.805+0000 7f3253400700 1 -- 172.21.15.19:0/466777631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3254108f20 msgr2=0x7f3254076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:02.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.805+0000 7f3253400700 1 --2- 172.21.15.19:0/466777631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3254108f20 0x7f3254076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:02.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.805+0000 7f3253400700 1 -- 172.21.15.19:0/466777631 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32541a91b0 con 0x7f3254109870 2024-05-07T02:42:02.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.806+0000 7f3253400700 1 --2- 172.21.15.19:0/466777631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3254109870 0x7f3254076540 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f324c00b0d0 tx=0x7f324c007a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:42:02.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.806+0000 7f3258a00700 1 -- 172.21.15.19:0/466777631 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324c01f070 con 0x7f3254109870 2024-05-07T02:42:02.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.806+0000 7f3258a00700 1 -- 172.21.15.19:0/466777631 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f324c004c50 con 0x7f3254109870 2024-05-07T02:42:02.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.806+0000 7f3259e00700 1 -- 172.21.15.19:0/466777631 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32541a9440 con 0x7f3254109870 2024-05-07T02:42:02.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.806+0000 7f3258a00700 1 -- 172.21.15.19:0/466777631 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324c01b430 con 0x7f3254109870 2024-05-07T02:42:02.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.806+0000 7f3259e00700 1 -- 172.21.15.19:0/466777631 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32541a9890 con 0x7f3254109870 2024-05-07T02:42:02.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.811+0000 7f3258a00700 1 -- 172.21.15.19:0/466777631 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f324c022070 con 0x7f3254109870 2024-05-07T02:42:02.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.811+0000 7f326885fb80 1 Processor -- start 2024-05-07T02:42:02.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.812+0000 7f326885fb80 1 -- start start 2024-05-07T02:42:02.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.812+0000 7f326885fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56258717f3b0 0x56258717f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:02.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.812+0000 7f326885fb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562587179470 0x562587179880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:02.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.812+0000 7f326885fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562587179dc0 0x562587186040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:02.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.812+0000 7f326885fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56258709ba20 con 0x56258717f3b0 2024-05-07T02:42:02.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.812+0000 7f326885fb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562587039760 con 0x562587179470 2024-05-07T02:42:02.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.812+0000 7f326885fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5625870979e0 con 0x562587179dc0 2024-05-07T02:42:02.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.813+0000 7f3252a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562587179470 0x562587179880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:02.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.813+0000 7f3252a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562587179470 0x562587179880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:47780/0 (socket says 172.21.15.19:47780) 2024-05-07T02:42:02.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.813+0000 7f3252a00700 1 -- 172.21.15.19:0/2719257323 learned_addr learned my addr 172.21.15.19:0/2719257323 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:42:02.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.813+0000 7f3252000700 1 --2- 172.21.15.19:0/2719257323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562587179dc0 0x562587186040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:02.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.813+0000 7f3252000700 1 --2- 172.21.15.19:0/2719257323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56258717f3b0 0x56258717f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:02.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.813+0000 7f3252a00700 1 -- 172.21.15.19:0/2719257323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562587179dc0 msgr2=0x562587186040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:02.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.813+0000 7f3252a00700 1 --2- 172.21.15.19:0/2719257323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562587179dc0 0x562587186040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:02.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.813+0000 7f3252a00700 1 -- 172.21.15.19:0/2719257323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56258717f3b0 msgr2=0x56258717f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:02.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.813+0000 7f3252a00700 1 --2- 172.21.15.19:0/2719257323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56258717f3b0 0x56258717f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:02.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.813+0000 7f3252a00700 1 -- 172.21.15.19:0/2719257323 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562586fe88d0 con 0x562587179470 2024-05-07T02:42:02.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.813+0000 7f3252000700 1 --2- 172.21.15.19:0/2719257323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562587179dc0 0x562587186040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:42:02.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.814+0000 7f3258a00700 1 --2- 172.21.15.19:0/466777631 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f32380b29f0 0x7f32380b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:02.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.814+0000 7f3252a00700 1 --2- 172.21.15.19:0/2719257323 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562587179470 0x562587179880 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3244009e50 tx=0x7f3244011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=428992cca1e45f76 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:42:02.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.814+0000 7f3258a00700 1 -- 172.21.15.19:0/466777631 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9525+0+0 (secure 0 0 0) 0x7f324c00f020 con 0x7f3254109870 2024-05-07T02:42:02.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.814+0000 7f3252000700 1 --2- 172.21.15.19:0/466777631 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f32380b29f0 0x7f32380b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:02.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.814+0000 7f3248c00700 1 -- 172.21.15.19:0/2719257323 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3244004a30 con 0x562587179470 2024-05-07T02:42:02.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.814+0000 7f3248c00700 1 -- 172.21.15.19:0/2719257323 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32440054d0 con 0x562587179470 2024-05-07T02:42:02.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.814+0000 7f3248c00700 1 -- 172.21.15.19:0/2719257323 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3244019940 con 0x562587179470 2024-05-07T02:42:02.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.815+0000 7f326885fb80 1 -- 172.21.15.19:0/2719257323 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562587179470 msgr2=0x562587179880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:02.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.815+0000 7f3252000700 1 --2- 172.21.15.19:0/466777631 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f32380b29f0 0x7f32380b4f00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x56258717ee70 tx=0x7f3240004750 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:42:02.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.815+0000 7f326885fb80 1 --2- 172.21.15.19:0/2719257323 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562587179470 0x562587179880 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3244009e50 tx=0x7f3244011720 comp rx=0 tx=0).stop 2024-05-07T02:42:02.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.815+0000 7f326885fb80 1 -- 172.21.15.19:0/2719257323 shutdown_connections 2024-05-07T02:42:02.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.815+0000 7f326885fb80 1 --2- 172.21.15.19:0/2719257323 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562587179dc0 0x562587186040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:02.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.815+0000 7f326885fb80 1 --2- 172.21.15.19:0/2719257323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56258717f3b0 0x56258717f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:02.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.815+0000 7f326885fb80 1 --2- 172.21.15.19:0/2719257323 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562587179470 0x562587179880 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:02.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.815+0000 7f326885fb80 1 -- 172.21.15.19:0/2719257323 >> 172.21.15.19:0/2719257323 conn(0x562586ffbe40 msgr2=0x5625871742f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:42:02.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.816+0000 7f326885fb80 1 -- 172.21.15.19:0/2719257323 shutdown_connections 2024-05-07T02:42:02.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.816+0000 7f326885fb80 1 -- 172.21.15.19:0/2719257323 wait complete. 2024-05-07T02:42:02.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.816+0000 7f326885fb80 1 -- start start 2024-05-07T02:42:02.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.816+0000 7f326885fb80 1 Processor -- start 2024-05-07T02:42:02.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.817+0000 7f326885fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562587179470 0x562587183550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:02.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.817+0000 7f326885fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562587179dc0 0x562587183a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:02.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.817+0000 7f326885fb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56258717f3b0 0x562587183fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:02.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.817+0000 7f326885fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5625871867c0 con 0x562587179dc0 2024-05-07T02:42:02.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.817+0000 7f326885fb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x562587001b20 con 0x56258717f3b0 2024-05-07T02:42:02.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.817+0000 7f326885fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x562587186940 con 0x562587179470 2024-05-07T02:42:02.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.817+0000 7f326885fb80 10 client.0 _ll_register_callbacks cb 0x562587042c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:42:02.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.817+0000 7f3252000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562587179dc0 0x562587183a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:02.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.817+0000 7f3252a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562587179470 0x562587183550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:02.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.817+0000 7f3252a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562587179470 0x562587183550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:41024/0 (socket says 172.21.15.19:41024) 2024-05-07T02:42:02.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.817+0000 7f3252a00700 1 -- 172.21.15.19:0/831894979 learned_addr learned my addr 172.21.15.19:0/831894979 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:42:02.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.817+0000 7f3253400700 1 --2- 172.21.15.19:0/831894979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56258717f3b0 0x562587183fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:02.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.818+0000 7f3252a00700 1 -- 172.21.15.19:0/831894979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56258717f3b0 msgr2=0x562587183fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:02.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.818+0000 7f3252a00700 1 --2- 172.21.15.19:0/831894979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56258717f3b0 0x562587183fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:02.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.818+0000 7f3252a00700 1 -- 172.21.15.19:0/831894979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562587179dc0 msgr2=0x562587183a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:02.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.818+0000 7f3252a00700 1 --2- 172.21.15.19:0/831894979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562587179dc0 0x562587183a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:02.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.818+0000 7f3252a00700 1 -- 172.21.15.19:0/831894979 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562587061420 con 0x562587179470 2024-05-07T02:42:02.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.818+0000 7f3252a00700 1 --2- 172.21.15.19:0/831894979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562587179470 0x562587183550 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3244006010 tx=0x7f3244004250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f98ce950be23b687 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:42:02.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.818+0000 7f323d600700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:42:02.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.819+0000 7f323d600700 1 -- 172.21.15.19:0/831894979 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3244027070 con 0x562587179470 2024-05-07T02:42:02.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.819+0000 7f323d600700 1 -- 172.21.15.19:0/831894979 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9525+0+0 (secure 0 0 0) 0x7f3244004750 con 0x562587179470 2024-05-07T02:42:02.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.819+0000 7f326885fb80 1 -- 172.21.15.19:0/831894979 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32440054d0 con 0x562587179470 2024-05-07T02:42:02.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.819+0000 7f323d600700 1 -- 172.21.15.19:0/831894979 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f3244033050 con 0x562587179470 2024-05-07T02:42:02.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.819+0000 7f326885fb80 10 client.26710 fetch_fsmap learned FSMap version 50 2024-05-07T02:42:02.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.819+0000 7f326885fb80 1 -- 172.21.15.19:0/831894979 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x562587191420 con 0x562587179470 2024-05-07T02:42:02.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.819+0000 7f323d600700 1 -- 172.21.15.19:0/831894979 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32440054d0 con 0x562587179470 2024-05-07T02:42:02.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.820+0000 7f323d600700 1 -- 172.21.15.19:0/831894979 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 50) ==== 8362+0+0 (secure 0 0 0) 0x7f3244019540 con 0x562587179470 2024-05-07T02:42:02.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.820+0000 7f326885fb80 10 client.26710 fetch_fsmap finished waiting for FSMap version 50 2024-05-07T02:42:02.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.820+0000 7f326885fb80 10 client.26710 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:42:02.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.820+0000 7f326885fb80 20 client.26710 populate_metadata read hostname 'smithi019' 2024-05-07T02:42:02.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.820+0000 7f326885fb80 1 --2- 172.21.15.19:0/831894979 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5625871939b0 0x562587195dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:02.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.820+0000 7f326885fb80 4 client.26710 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:42:02.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.820+0000 7f326885fb80 1 -- 172.21.15.19:0/831894979 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625871462e0 con 0x5625871939b0 2024-05-07T02:42:02.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.820+0000 7f3252000700 1 --2- 172.21.15.19:0/831894979 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5625871939b0 0x562587195dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:02.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.821+0000 7f3252000700 1 --2- 172.21.15.19:0/831894979 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5625871939b0 0x562587195dd0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8aa9a899e653e44b server_cookie=2da51b68aecca3 in_seq=0 out_seq=0 2024-05-07T02:42:02.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.821+0000 7f323d600700 10 client.26710 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:42:02.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.822+0000 7f323d600700 1 -- 172.21.15.19:0/831894979 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5625871462e0 con 0x5625871939b0 2024-05-07T02:42:02.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.822+0000 7f323d600700 10 client.26710 handle_command_reply: tid=0 2024-05-07T02:42:02.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.822+0000 7f326885fb80 1 client.26710 shutdown 2024-05-07T02:42:02.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.822+0000 7f326885fb80 1 -- 172.21.15.19:0/831894979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562587179470 msgr2=0x562587183550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:02.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.822+0000 7f326885fb80 1 --2- 172.21.15.19:0/831894979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562587179470 0x562587183550 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3244006010 tx=0x7f3244004250 comp rx=0 tx=0).stop 2024-05-07T02:42:02.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.822+0000 7f326885fb80 1 -- 172.21.15.19:0/831894979 shutdown_connections 2024-05-07T02:42:02.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.822+0000 7f326885fb80 1 --2- 172.21.15.19:0/831894979 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5625871939b0 0x562587195dd0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:02.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.822+0000 7f326885fb80 1 --2- 172.21.15.19:0/831894979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x56258717f3b0 0x562587183fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:02.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.822+0000 7f326885fb80 1 --2- 172.21.15.19:0/831894979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562587179470 0x562587183550 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:02.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.822+0000 7f326885fb80 1 --2- 172.21.15.19:0/831894979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562587179dc0 0x562587183a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:02.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.822+0000 7f323d600700 0 client.26710 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:42:02.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.822+0000 7f326885fb80 1 -- 172.21.15.19:0/831894979 >> 172.21.15.19:0/831894979 conn(0x562586ffbe40 msgr2=0x562587178460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:42:02.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.823+0000 7f326885fb80 1 -- 172.21.15.19:0/831894979 shutdown_connections 2024-05-07T02:42:02.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.823+0000 7f326885fb80 1 -- 172.21.15.19:0/831894979 wait complete. 2024-05-07T02:42:02.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.823+0000 7f326885fb80 20 client.26710 trim_cache size 0 max 16384 2024-05-07T02:42:02.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.845+0000 7f326885fb80 1 Processor -- start 2024-05-07T02:42:02.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.846+0000 7f326885fb80 1 -- start start 2024-05-07T02:42:02.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.846+0000 7f326885fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562587179470 0x562587178e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:02.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.846+0000 7f326885fb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562587179dc0 0x562587183e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:02.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.846+0000 7f326885fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56258717f3b0 0x562587182420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:02.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.846+0000 7f326885fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f32541b27d0 con 0x562587179470 2024-05-07T02:42:02.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.846+0000 7f326885fb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f32541b4310 con 0x562587179dc0 2024-05-07T02:42:02.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.846+0000 7f326885fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f32541b2ea0 con 0x56258717f3b0 2024-05-07T02:42:02.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.846+0000 7f3252a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562587179470 0x562587178e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:02.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.846+0000 7f3252a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562587179470 0x562587178e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59776/0 (socket says 172.21.15.19:59776) 2024-05-07T02:42:02.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.846+0000 7f3252a00700 1 -- 172.21.15.19:0/3856802259 learned_addr learned my addr 172.21.15.19:0/3856802259 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:42:02.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.846+0000 7f3253400700 1 --2- 172.21.15.19:0/3856802259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56258717f3b0 0x562587182420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:02.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.846+0000 7f3252000700 1 --2- 172.21.15.19:0/3856802259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562587179dc0 0x562587183e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:02.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.847+0000 7f3252a00700 1 -- 172.21.15.19:0/3856802259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56258717f3b0 msgr2=0x562587182420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:02.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.847+0000 7f3252a00700 1 --2- 172.21.15.19:0/3856802259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56258717f3b0 0x562587182420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:02.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.847+0000 7f3252a00700 1 -- 172.21.15.19:0/3856802259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562587179dc0 msgr2=0x562587183e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:02.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.847+0000 7f3252a00700 1 --2- 172.21.15.19:0/3856802259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562587179dc0 0x562587183e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:02.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.847+0000 7f3252a00700 1 -- 172.21.15.19:0/3856802259 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562587181ac0 con 0x562587179470 2024-05-07T02:42:02.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.847+0000 7f3252a00700 1 --2- 172.21.15.19:0/3856802259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562587179470 0x562587178e90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3244014040 tx=0x7f3244011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=997524bf4dad06cf server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:42:02.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.847+0000 7f323fe00700 1 -- 172.21.15.19:0/3856802259 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3244027070 con 0x562587179470 2024-05-07T02:42:02.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.848+0000 7f323fe00700 1 -- 172.21.15.19:0/3856802259 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32440056d0 con 0x562587179470 2024-05-07T02:42:02.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.848+0000 7f323fe00700 1 -- 172.21.15.19:0/3856802259 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3244003680 con 0x562587179470 2024-05-07T02:42:02.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.848+0000 7f326885fb80 1 -- 172.21.15.19:0/3856802259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562587179470 msgr2=0x562587178e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:02.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.848+0000 7f326885fb80 1 --2- 172.21.15.19:0/3856802259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562587179470 0x562587178e90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3244014040 tx=0x7f3244011720 comp rx=0 tx=0).stop 2024-05-07T02:42:02.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.849+0000 7f326885fb80 1 -- 172.21.15.19:0/3856802259 shutdown_connections 2024-05-07T02:42:02.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.849+0000 7f326885fb80 1 --2- 172.21.15.19:0/3856802259 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56258717f3b0 0x562587182420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:02.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.849+0000 7f326885fb80 1 --2- 172.21.15.19:0/3856802259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562587179470 0x562587178e90 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:02.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.849+0000 7f326885fb80 1 --2- 172.21.15.19:0/3856802259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562587179dc0 0x562587183e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:02.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.849+0000 7f326885fb80 1 -- 172.21.15.19:0/3856802259 >> 172.21.15.19:0/3856802259 conn(0x562586ffbe40 msgr2=0x562587181d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:42:02.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.849+0000 7f326885fb80 1 -- 172.21.15.19:0/3856802259 shutdown_connections 2024-05-07T02:42:02.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.849+0000 7f326885fb80 1 -- 172.21.15.19:0/3856802259 wait complete. 2024-05-07T02:42:02.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.849+0000 7f326885fb80 1 -- start start 2024-05-07T02:42:02.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.849+0000 7f326885fb80 1 Processor -- start 2024-05-07T02:42:02.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.850+0000 7f326885fb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562587179470 0x56258718be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:02.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.850+0000 7f326885fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562587179dc0 0x56258719a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:02.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.850+0000 7f326885fb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56258717f3b0 0x56258719ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:02.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.850+0000 7f326885fb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f32541b3c40 con 0x56258717f3b0 2024-05-07T02:42:02.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.850+0000 7f326885fb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f32541b3570 con 0x562587179470 2024-05-07T02:42:02.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.850+0000 7f326885fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f32541b0790 con 0x562587179dc0 2024-05-07T02:42:02.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.850+0000 7f326885fb80 10 client.0 _ll_register_callbacks cb 0x562587042c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:42:02.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.850+0000 7f3252a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562587179470 0x56258718be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:02.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.850+0000 7f3253400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56258717f3b0 0x56258719ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:02.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.850+0000 7f3252a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562587179470 0x56258718be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:47802/0 (socket says 172.21.15.19:47802) 2024-05-07T02:42:02.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.850+0000 7f3252a00700 1 -- 172.21.15.19:0/472762420 learned_addr learned my addr 172.21.15.19:0/472762420 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:42:02.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.851+0000 7f3252000700 1 --2- 172.21.15.19:0/472762420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562587179dc0 0x56258719a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:02.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.851+0000 7f3252a00700 1 -- 172.21.15.19:0/472762420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562587179dc0 msgr2=0x56258719a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:02.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.851+0000 7f3252a00700 1 --2- 172.21.15.19:0/472762420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562587179dc0 0x56258719a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:02.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.851+0000 7f3252a00700 1 -- 172.21.15.19:0/472762420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56258717f3b0 msgr2=0x56258719ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:02.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.851+0000 7f3252a00700 1 --2- 172.21.15.19:0/472762420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56258717f3b0 0x56258719ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:02.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.851+0000 7f3252a00700 1 -- 172.21.15.19:0/472762420 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56258719b140 con 0x562587179470 2024-05-07T02:42:02.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.851+0000 7f3252000700 1 --2- 172.21.15.19:0/472762420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562587179dc0 0x56258719a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:42:02.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.851+0000 7f3253400700 1 --2- 172.21.15.19:0/472762420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56258717f3b0 0x56258719ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:42:02.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.851+0000 7f3252a00700 1 --2- 172.21.15.19:0/472762420 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562587179470 0x56258718be00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f32440092a0 tx=0x7f3244019660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76fe9e28308a3383 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:42:02.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.851+0000 7f323d600700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:42:02.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.852+0000 7f323d600700 1 -- 172.21.15.19:0/472762420 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3244027040 con 0x562587179470 2024-05-07T02:42:02.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.852+0000 7f323d600700 1 -- 172.21.15.19:0/472762420 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9525+0+0 (secure 0 0 0) 0x7f32440054d0 con 0x562587179470 2024-05-07T02:42:02.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.852+0000 7f326885fb80 1 -- 172.21.15.19:0/472762420 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32440056d0 con 0x562587179470 2024-05-07T02:42:02.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.852+0000 7f323d600700 1 -- 172.21.15.19:0/472762420 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f3244048050 con 0x562587179470 2024-05-07T02:42:02.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.852+0000 7f326885fb80 10 client.26756 fetch_fsmap learned FSMap version 50 2024-05-07T02:42:02.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.852+0000 7f326885fb80 1 -- 172.21.15.19:0/472762420 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x562587206460 con 0x562587179470 2024-05-07T02:42:02.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.852+0000 7f323d600700 1 -- 172.21.15.19:0/472762420 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32440056d0 con 0x562587179470 2024-05-07T02:42:02.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.853+0000 7f323d600700 1 -- 172.21.15.19:0/472762420 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 50) ==== 8362+0+0 (secure 0 0 0) 0x7f3244003780 con 0x562587179470 2024-05-07T02:42:02.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.853+0000 7f326885fb80 10 client.26756 fetch_fsmap finished waiting for FSMap version 50 2024-05-07T02:42:02.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.853+0000 7f326885fb80 10 client.26756 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:42:02.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.853+0000 7f326885fb80 20 client.26756 populate_metadata read hostname 'smithi019' 2024-05-07T02:42:02.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.853+0000 7f326885fb80 1 --2- 172.21.15.19:0/472762420 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x562587206a50 0x562587208e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:02.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.853+0000 7f326885fb80 4 client.26756 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-07T02:42:02.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.853+0000 7f326885fb80 1 -- 172.21.15.19:0/472762420 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5625870a9400 con 0x562587206a50 2024-05-07T02:42:02.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.853+0000 7f3252000700 1 --2- 172.21.15.19:0/472762420 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x562587206a50 0x562587208e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:02.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.853+0000 7f3252000700 1 --2- 172.21.15.19:0/472762420 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x562587206a50 0x562587208e70 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c50dfa459d1f4335 server_cookie=a09051e343a18681 in_seq=0 out_seq=0 2024-05-07T02:42:02.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.853+0000 7f323d600700 10 client.26756 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:42:02.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.854+0000 7f323d600700 1 -- 172.21.15.19:0/472762420 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5625870a9400 con 0x562587206a50 2024-05-07T02:42:02.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.854+0000 7f323d600700 10 client.26756 handle_command_reply: tid=0 2024-05-07T02:42:02.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.854+0000 7f326885fb80 1 client.26756 shutdown 2024-05-07T02:42:02.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.854+0000 7f326885fb80 1 -- 172.21.15.19:0/472762420 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562587179470 msgr2=0x56258718be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:02.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.854+0000 7f326885fb80 1 --2- 172.21.15.19:0/472762420 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562587179470 0x56258718be00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f32440092a0 tx=0x7f3244019660 comp rx=0 tx=0).stop 2024-05-07T02:42:02.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.854+0000 7f326885fb80 1 -- 172.21.15.19:0/472762420 shutdown_connections 2024-05-07T02:42:02.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.854+0000 7f326885fb80 1 --2- 172.21.15.19:0/472762420 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x562587206a50 0x562587208e70 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:02.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.854+0000 7f326885fb80 1 --2- 172.21.15.19:0/472762420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56258717f3b0 0x56258719ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:02.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.854+0000 7f326885fb80 1 --2- 172.21.15.19:0/472762420 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x562587179470 0x56258718be00 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:02.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.854+0000 7f326885fb80 1 --2- 172.21.15.19:0/472762420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x562587179dc0 0x56258719a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:02.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.854+0000 7f326885fb80 1 -- 172.21.15.19:0/472762420 >> 172.21.15.19:0/472762420 conn(0x562586ffbe40 msgr2=0x562587178bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:42:02.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.854+0000 7f323d600700 0 client.26756 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:42:02.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.854+0000 7f326885fb80 1 -- 172.21.15.19:0/472762420 shutdown_connections 2024-05-07T02:42:02.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.854+0000 7f326885fb80 1 -- 172.21.15.19:0/472762420 wait complete. 2024-05-07T02:42:02.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.855+0000 7f326885fb80 20 client.26756 trim_cache size 0 max 16384 2024-05-07T02:42:02.867 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:42:02.867 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-07T02:42:02.867 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-05-07T02:42:02.868 INFO:teuthology.orchestra.run.smithi019.stdout: "95d5246f-b66d-4cc4-a0f7-76d373531436": { 2024-05-07T02:42:02.868 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-05-07T02:42:02.868 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "95d5246f-b66d-4cc4-a0f7-76d373531436", 2024-05-07T02:42:02.868 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-05-07T02:42:02.868 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:42:02.868 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:42:02.868 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:42:02.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.855+0000 7f3259e00700 1 -- 172.21.15.19:0/466777631 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f32380b29f0 msgr2=0x7f32380b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:02.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.855+0000 7f3259e00700 1 --2- 172.21.15.19:0/466777631 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f32380b29f0 0x7f32380b4f00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x56258717ee70 tx=0x7f3240004750 comp rx=0 tx=0).stop 2024-05-07T02:42:02.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.855+0000 7f3259e00700 1 -- 172.21.15.19:0/466777631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3254109870 msgr2=0x7f3254076540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:02.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.855+0000 7f3259e00700 1 --2- 172.21.15.19:0/466777631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3254109870 0x7f3254076540 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f324c00b0d0 tx=0x7f324c007a00 comp rx=0 tx=0).stop 2024-05-07T02:42:02.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.855+0000 7f3259e00700 1 -- 172.21.15.19:0/466777631 shutdown_connections 2024-05-07T02:42:02.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.855+0000 7f3259e00700 1 --2- 172.21.15.19:0/466777631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3254109870 0x7f3254076540 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:02.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.855+0000 7f3259e00700 1 --2- 172.21.15.19:0/466777631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f32541045f0 0x7f3254075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:02.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.855+0000 7f3259e00700 1 --2- 172.21.15.19:0/466777631 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f32380b29f0 0x7f32380b4f00 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:02.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.855+0000 7f3259e00700 1 --2- 172.21.15.19:0/466777631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3254108f20 0x7f3254076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:02.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.855+0000 7f3259e00700 1 -- 172.21.15.19:0/466777631 >> 172.21.15.19:0/466777631 conn(0x7f3254100040 msgr2=0x7f3254108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:42:02.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.856+0000 7f3259e00700 1 -- 172.21.15.19:0/466777631 shutdown_connections 2024-05-07T02:42:02.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:02.856+0000 7f3259e00700 1 -- 172.21.15.19:0/466777631 wait complete. 2024-05-07T02:42:03.926 INFO:tasks.cephfs.filesystem:scrub status for tag:95d5246f-b66d-4cc4-a0f7-76d373531436 - {'path': '/', 'tag': '95d5246f-b66d-4cc4-a0f7-76d373531436', 'options': 'recursive,force'} 2024-05-07T02:42:04.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:03 smithi064 ceph-mon[34699]: pgmap v315: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-05-07T02:42:04.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:03 smithi019 ceph-mon[29781]: pgmap v315: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-05-07T02:42:04.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:03 smithi027 ceph-mon[34398]: pgmap v315: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-05-07T02:42:06.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:05 smithi064 ceph-mon[34699]: pgmap v316: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-05-07T02:42:06.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:05 smithi019 ceph-mon[29781]: pgmap v316: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-05-07T02:42:06.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:05 smithi027 ceph-mon[34398]: pgmap v316: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-05-07T02:42:08.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:07 smithi064 ceph-mon[34699]: pgmap v317: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-05-07T02:42:08.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:07 smithi019 ceph-mon[29781]: pgmap v317: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-05-07T02:42:08.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:07 smithi027 ceph-mon[34398]: pgmap v317: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-05-07T02:42:09.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:08 smithi064 ceph-mon[34699]: scrub summary: idle 2024-05-07T02:42:09.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:08 smithi064 ceph-mon[34699]: scrub summary: idle 2024-05-07T02:42:09.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:08 smithi064 ceph-mon[34699]: scrub summary: idle 2024-05-07T02:42:09.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:08 smithi064 ceph-mon[34699]: scrub summary: idle 2024-05-07T02:42:09.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:08 smithi019 ceph-mon[29781]: scrub summary: idle 2024-05-07T02:42:09.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:08 smithi019 ceph-mon[29781]: scrub summary: idle 2024-05-07T02:42:09.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:08 smithi019 ceph-mon[29781]: scrub summary: idle 2024-05-07T02:42:09.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:08 smithi019 ceph-mon[29781]: scrub summary: idle 2024-05-07T02:42:09.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:08 smithi027 ceph-mon[34398]: scrub summary: idle 2024-05-07T02:42:09.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:08 smithi027 ceph-mon[34398]: scrub summary: idle 2024-05-07T02:42:09.208 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:08 smithi027 ceph-mon[34398]: scrub summary: idle 2024-05-07T02:42:09.208 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:08 smithi027 ceph-mon[34398]: scrub summary: idle 2024-05-07T02:42:10.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:09 smithi064 ceph-mon[34699]: pgmap v318: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-05-07T02:42:10.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:09 smithi064 ceph-mon[34699]: scrub summary: idle 2024-05-07T02:42:10.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:09 smithi064 ceph-mon[34699]: osdmap e97: 12 total, 12 up, 12 in 2024-05-07T02:42:10.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:09 smithi019 ceph-mon[29781]: pgmap v318: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-05-07T02:42:10.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:09 smithi019 ceph-mon[29781]: scrub summary: idle 2024-05-07T02:42:10.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:09 smithi019 ceph-mon[29781]: osdmap e97: 12 total, 12 up, 12 in 2024-05-07T02:42:10.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:09 smithi027 ceph-mon[34398]: pgmap v318: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-05-07T02:42:10.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:09 smithi027 ceph-mon[34398]: scrub summary: idle 2024-05-07T02:42:10.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:09 smithi027 ceph-mon[34398]: osdmap e97: 12 total, 12 up, 12 in 2024-05-07T02:42:10.256 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-07T02:42:10.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.544+0000 7f8107e00640 1 -- 172.21.15.19:0/667418299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8108109ae0 msgr2=0x7f8108103680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:10.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.544+0000 7f8107e00640 1 --2- 172.21.15.19:0/667418299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8108109ae0 0x7f8108103680 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f80f8009900 tx=0x7f80f802a3f0 comp rx=0 tx=0).stop 2024-05-07T02:42:10.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.545+0000 7f8107e00640 1 -- 172.21.15.19:0/667418299 shutdown_connections 2024-05-07T02:42:10.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.545+0000 7f8107e00640 1 --2- 172.21.15.19:0/667418299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8108104230 0x7f8108110600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:10.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.545+0000 7f8107e00640 1 --2- 172.21.15.19:0/667418299 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8108109f80 0x7f8108103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:10.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.545+0000 7f8107e00640 1 --2- 172.21.15.19:0/667418299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8108109ae0 0x7f8108103680 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:10.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.545+0000 7f8107e00640 1 -- 172.21.15.19:0/667418299 >> 172.21.15.19:0/667418299 conn(0x7f81080ff1d0 msgr2=0x7f81081015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:42:10.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.545+0000 7f8107e00640 1 -- 172.21.15.19:0/667418299 shutdown_connections 2024-05-07T02:42:10.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.545+0000 7f8107e00640 1 -- 172.21.15.19:0/667418299 wait complete. 2024-05-07T02:42:10.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.546+0000 7f8107e00640 1 Processor -- start 2024-05-07T02:42:10.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.546+0000 7f8107e00640 1 -- start start 2024-05-07T02:42:10.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.546+0000 7f8107e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8108104230 0x7f8108074fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:10.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.546+0000 7f8107e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8108109ae0 0x7f8108075510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:10.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.546+0000 7f8107e00640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8108109f80 0x7f8108075a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:10.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.546+0000 7f8107e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8108113dc0 con 0x7f8108109ae0 2024-05-07T02:42:10.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.546+0000 7f8107e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f8108113c40 con 0x7f8108104230 2024-05-07T02:42:10.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.546+0000 7f8107e00640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8108113f40 con 0x7f8108109f80 2024-05-07T02:42:10.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.547+0000 7f8106000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8108109ae0 0x7f8108075510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:10.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.547+0000 7f8106000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8108109ae0 0x7f8108075510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59180/0 (socket says 172.21.15.19:59180) 2024-05-07T02:42:10.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.547+0000 7f8106000640 1 -- 172.21.15.19:0/2312258816 learned_addr learned my addr 172.21.15.19:0/2312258816 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:42:10.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.547+0000 7f8107400640 1 --2- 172.21.15.19:0/2312258816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8108109f80 0x7f8108075a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:10.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.547+0000 7f8106a00640 1 --2- 172.21.15.19:0/2312258816 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8108104230 0x7f8108074fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:10.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.547+0000 7f8106000640 1 -- 172.21.15.19:0/2312258816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8108109f80 msgr2=0x7f8108075a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:10.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.547+0000 7f8106000640 1 --2- 172.21.15.19:0/2312258816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8108109f80 0x7f8108075a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:10.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.547+0000 7f8106000640 1 -- 172.21.15.19:0/2312258816 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8108104230 msgr2=0x7f8108074fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:10.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.547+0000 7f8106000640 1 --2- 172.21.15.19:0/2312258816 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8108104230 0x7f8108074fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:10.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.547+0000 7f8106000640 1 -- 172.21.15.19:0/2312258816 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81081a8a80 con 0x7f8108109ae0 2024-05-07T02:42:10.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.548+0000 7f8106000640 1 --2- 172.21.15.19:0/2312258816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8108109ae0 0x7f8108075510 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f80f400e980 tx=0x7f80f400ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:42:10.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.548+0000 7f80ff400640 1 -- 172.21.15.19:0/2312258816 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f400ccc0 con 0x7f8108109ae0 2024-05-07T02:42:10.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.548+0000 7f80ff400640 1 -- 172.21.15.19:0/2312258816 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80f40108e0 con 0x7f8108109ae0 2024-05-07T02:42:10.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.548+0000 7f8107e00640 1 -- 172.21.15.19:0/2312258816 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81081a8d70 con 0x7f8108109ae0 2024-05-07T02:42:10.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.548+0000 7f80ff400640 1 -- 172.21.15.19:0/2312258816 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f4018880 con 0x7f8108109ae0 2024-05-07T02:42:10.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.548+0000 7f8107e00640 1 -- 172.21.15.19:0/2312258816 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81081a9200 con 0x7f8108109ae0 2024-05-07T02:42:10.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.549+0000 7f8107e00640 1 -- 172.21.15.19:0/2312258816 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81080696e0 con 0x7f8108109ae0 2024-05-07T02:42:10.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.552+0000 7f80ff400640 1 -- 172.21.15.19:0/2312258816 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f80f401f020 con 0x7f8108109ae0 2024-05-07T02:42:10.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.554+0000 7f80ff400640 1 --2- 172.21.15.19:0/2312258816 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f80ec0b2a10 0x7f80ec0b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:10.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.554+0000 7f80ff400640 1 -- 172.21.15.19:0/2312258816 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9613+0+0 (secure 0 0 0) 0x7f80f40e3090 con 0x7f8108109ae0 2024-05-07T02:42:10.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.554+0000 7f8106a00640 1 --2- 172.21.15.19:0/2312258816 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f80ec0b2a10 0x7f80ec0b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:10.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.555+0000 7f8106a00640 1 --2- 172.21.15.19:0/2312258816 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f80ec0b2a10 0x7f80ec0b4f20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f80f8031440 tx=0x7f80f8005a20 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:42:10.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.557+0000 7f80ff400640 1 -- 172.21.15.19:0/2312258816 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f80f40aa5d0 con 0x7f8108109ae0 2024-05-07T02:42:10.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.794+0000 7f8107e00640 1 -- 172.21.15.19:0/2312258816 --> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f81080ffcf0 con 0x7f80ec0b2a10 2024-05-07T02:42:10.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.811+0000 7f80ff400640 1 -- 172.21.15.19:0/2312258816 <== mgr.25234 v2:172.21.15.19:6800/3265825478 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f81080ffcf0 con 0x7f80ec0b2a10 2024-05-07T02:42:10.813 INFO:teuthology.orchestra.run.smithi019.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-07T00:00:00", "created": "2024-05-07T02:32:08", "first": "2024-05-07T02:33:00", "last": "2024-05-07T02:42:00", "last_pruned": "2024-05-07T02:42:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-05-07T02:42:10.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.814+0000 7f8107e00640 1 -- 172.21.15.19:0/2312258816 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f80ec0b2a10 msgr2=0x7f80ec0b4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:10.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.814+0000 7f8107e00640 1 --2- 172.21.15.19:0/2312258816 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f80ec0b2a10 0x7f80ec0b4f20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f80f8031440 tx=0x7f80f8005a20 comp rx=0 tx=0).stop 2024-05-07T02:42:10.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.814+0000 7f8107e00640 1 -- 172.21.15.19:0/2312258816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8108109ae0 msgr2=0x7f8108075510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:10.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.814+0000 7f8107e00640 1 --2- 172.21.15.19:0/2312258816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8108109ae0 0x7f8108075510 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f80f400e980 tx=0x7f80f400ed80 comp rx=0 tx=0).stop 2024-05-07T02:42:10.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.814+0000 7f8107e00640 1 -- 172.21.15.19:0/2312258816 shutdown_connections 2024-05-07T02:42:10.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.814+0000 7f8107e00640 1 --2- 172.21.15.19:0/2312258816 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f80ec0b2a10 0x7f80ec0b4f20 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:10.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.814+0000 7f8107e00640 1 --2- 172.21.15.19:0/2312258816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8108109f80 0x7f8108075a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:10.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.814+0000 7f8107e00640 1 --2- 172.21.15.19:0/2312258816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8108109ae0 0x7f8108075510 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:10.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.814+0000 7f8107e00640 1 --2- 172.21.15.19:0/2312258816 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8108104230 0x7f8108074fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:10.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.814+0000 7f8107e00640 1 -- 172.21.15.19:0/2312258816 >> 172.21.15.19:0/2312258816 conn(0x7f81080ff1d0 msgr2=0x7f81080ff5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:42:10.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.814+0000 7f8107e00640 1 -- 172.21.15.19:0/2312258816 shutdown_connections 2024-05-07T02:42:10.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:10.814+0000 7f8107e00640 1 -- 172.21.15.19:0/2312258816 wait complete. 2024-05-07T02:42:10.843 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-07T02:42:11.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.131+0000 7f2e84200640 1 -- 172.21.15.19:0/1284191377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e7c076520 msgr2=0x7f2e7c076930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:11.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.131+0000 7f2e84200640 1 --2- 172.21.15.19:0/1284191377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e7c076520 0x7f2e7c076930 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2e70009900 tx=0x7f2e7002a480 comp rx=0 tx=0).stop 2024-05-07T02:42:11.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.132+0000 7f2e84200640 1 -- 172.21.15.19:0/1284191377 shutdown_connections 2024-05-07T02:42:11.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.132+0000 7f2e84200640 1 --2- 172.21.15.19:0/1284191377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e7c1104f0 0x7f2e7c112910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:11.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.132+0000 7f2e84200640 1 --2- 172.21.15.19:0/1284191377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e7c10db90 0x7f2e7c10ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:11.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.132+0000 7f2e84200640 1 --2- 172.21.15.19:0/1284191377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e7c076520 0x7f2e7c076930 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:11.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.132+0000 7f2e84200640 1 -- 172.21.15.19:0/1284191377 >> 172.21.15.19:0/1284191377 conn(0x7f2e7c0ff360 msgr2=0x7f2e7c1017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:42:11.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.132+0000 7f2e84200640 1 -- 172.21.15.19:0/1284191377 shutdown_connections 2024-05-07T02:42:11.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.132+0000 7f2e84200640 1 -- 172.21.15.19:0/1284191377 wait complete. 2024-05-07T02:42:11.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.133+0000 7f2e84200640 1 Processor -- start 2024-05-07T02:42:11.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.134+0000 7f2e84200640 1 -- start start 2024-05-07T02:42:11.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.134+0000 7f2e84200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e7c076520 0x7f2e7c1a25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:11.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.134+0000 7f2e84200640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e7c10db90 0x7f2e7c1a2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:11.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.134+0000 7f2e84200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e7c1104f0 0x7f2e7c19c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:11.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.134+0000 7f2e84200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2e7c116050 con 0x7f2e7c1104f0 2024-05-07T02:42:11.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.134+0000 7f2e84200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2e7c115ed0 con 0x7f2e7c076520 2024-05-07T02:42:11.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.134+0000 7f2e84200640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2e7c1161d0 con 0x7f2e7c10db90 2024-05-07T02:42:11.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.135+0000 7f2e81800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e7c076520 0x7f2e7c1a25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:11.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.135+0000 7f2e80e00640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e7c10db90 0x7f2e7c1a2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:11.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.135+0000 7f2e81800640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e7c076520 0x7f2e7c1a25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:41462/0 (socket says 172.21.15.19:41462) 2024-05-07T02:42:11.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.135+0000 7f2e81800640 1 -- 172.21.15.19:0/4008299363 learned_addr learned my addr 172.21.15.19:0/4008299363 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:42:11.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.135+0000 7f2e82200640 1 --2- 172.21.15.19:0/4008299363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e7c1104f0 0x7f2e7c19c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:11.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.135+0000 7f2e80e00640 1 -- 172.21.15.19:0/4008299363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e7c076520 msgr2=0x7f2e7c1a25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:11.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.135+0000 7f2e80e00640 1 --2- 172.21.15.19:0/4008299363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e7c076520 0x7f2e7c1a25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:11.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.135+0000 7f2e80e00640 1 -- 172.21.15.19:0/4008299363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e7c1104f0 msgr2=0x7f2e7c19c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:11.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.135+0000 7f2e80e00640 1 --2- 172.21.15.19:0/4008299363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e7c1104f0 0x7f2e7c19c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:11.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.135+0000 7f2e80e00640 1 -- 172.21.15.19:0/4008299363 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e7c19ceb0 con 0x7f2e7c10db90 2024-05-07T02:42:11.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.135+0000 7f2e81800640 1 --2- 172.21.15.19:0/4008299363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e7c076520 0x7f2e7c1a25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:42:11.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.135+0000 7f2e80e00640 1 --2- 172.21.15.19:0/4008299363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e7c10db90 0x7f2e7c1a2ae0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2e68005fd0 tx=0x7f2e6800dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:42:11.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.136+0000 7f2e7a000640 1 -- 172.21.15.19:0/4008299363 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e68009c00 con 0x7f2e7c10db90 2024-05-07T02:42:11.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.136+0000 7f2e7a000640 1 -- 172.21.15.19:0/4008299363 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e68009da0 con 0x7f2e7c10db90 2024-05-07T02:42:11.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.136+0000 7f2e84200640 1 -- 172.21.15.19:0/4008299363 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e7c19d1a0 con 0x7f2e7c10db90 2024-05-07T02:42:11.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.136+0000 7f2e7a000640 1 -- 172.21.15.19:0/4008299363 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e68003de0 con 0x7f2e7c10db90 2024-05-07T02:42:11.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.136+0000 7f2e84200640 1 -- 172.21.15.19:0/4008299363 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e7c1a93f0 con 0x7f2e7c10db90 2024-05-07T02:42:11.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.137+0000 7f2e84200640 1 -- 172.21.15.19:0/4008299363 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e7c077840 con 0x7f2e7c10db90 2024-05-07T02:42:11.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.140+0000 7f2e7a000640 1 -- 172.21.15.19:0/4008299363 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2e68003f80 con 0x7f2e7c10db90 2024-05-07T02:42:11.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.141+0000 7f2e7a000640 1 --2- 172.21.15.19:0/4008299363 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f2e640b2a10 0x7f2e640b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:11.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.142+0000 7f2e7a000640 1 -- 172.21.15.19:0/4008299363 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9613+0+0 (secure 0 0 0) 0x7f2e680e2520 con 0x7f2e7c10db90 2024-05-07T02:42:11.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.142+0000 7f2e81800640 1 --2- 172.21.15.19:0/4008299363 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f2e640b2a10 0x7f2e640b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:11.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.142+0000 7f2e81800640 1 --2- 172.21.15.19:0/4008299363 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f2e640b2a10 0x7f2e640b4f20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2e70009230 tx=0x7f2e700058d0 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:42:11.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.146+0000 7f2e7a000640 1 -- 172.21.15.19:0/4008299363 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2e68017020 con 0x7f2e7c10db90 2024-05-07T02:42:11.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.385+0000 7f2e84200640 1 -- 172.21.15.19:0/4008299363 --> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f2e7c111010 con 0x7f2e640b2a10 2024-05-07T02:42:11.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.398+0000 7f2e7a000640 1 -- 172.21.15.19:0/4008299363 <== mgr.25234 v2:172.21.15.19:6800/3265825478 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f2e7c111010 con 0x7f2e640b2a10 2024-05-07T02:42:11.400 INFO:teuthology.orchestra.run.smithi019.stdout:/ 1m 6m3h 2024-05-07T02:42:11.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.401+0000 7f2e84200640 1 -- 172.21.15.19:0/4008299363 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f2e640b2a10 msgr2=0x7f2e640b4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:11.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.401+0000 7f2e84200640 1 --2- 172.21.15.19:0/4008299363 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f2e640b2a10 0x7f2e640b4f20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2e70009230 tx=0x7f2e700058d0 comp rx=0 tx=0).stop 2024-05-07T02:42:11.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.401+0000 7f2e84200640 1 -- 172.21.15.19:0/4008299363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e7c10db90 msgr2=0x7f2e7c1a2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:11.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.401+0000 7f2e84200640 1 --2- 172.21.15.19:0/4008299363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e7c10db90 0x7f2e7c1a2ae0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2e68005fd0 tx=0x7f2e6800dde0 comp rx=0 tx=0).stop 2024-05-07T02:42:11.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.401+0000 7f2e84200640 1 -- 172.21.15.19:0/4008299363 shutdown_connections 2024-05-07T02:42:11.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.401+0000 7f2e84200640 1 --2- 172.21.15.19:0/4008299363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e7c1104f0 0x7f2e7c19c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:11.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.401+0000 7f2e84200640 1 --2- 172.21.15.19:0/4008299363 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f2e640b2a10 0x7f2e640b4f20 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:11.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.401+0000 7f2e84200640 1 --2- 172.21.15.19:0/4008299363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e7c10db90 0x7f2e7c1a2ae0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:11.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.401+0000 7f2e84200640 1 --2- 172.21.15.19:0/4008299363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e7c076520 0x7f2e7c1a25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:11.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.401+0000 7f2e84200640 1 -- 172.21.15.19:0/4008299363 >> 172.21.15.19:0/4008299363 conn(0x7f2e7c0ff360 msgr2=0x7f2e7c110900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:42:11.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.401+0000 7f2e84200640 1 -- 172.21.15.19:0/4008299363 shutdown_connections 2024-05-07T02:42:11.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:11.401+0000 7f2e84200640 1 -- 172.21.15.19:0/4008299363 wait complete. 2024-05-07T02:42:11.431 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-05-07T02:42:11.442 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-05-07T02:42:12.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:11 smithi064 ceph-mon[34699]: pgmap v320: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-05-07T02:42:12.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:11 smithi064 ceph-mon[34699]: from='client.16560 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:42:12.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:11 smithi019 ceph-mon[29781]: pgmap v320: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-05-07T02:42:12.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:11 smithi019 ceph-mon[29781]: from='client.16560 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:42:12.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:11 smithi027 ceph-mon[34398]: pgmap v320: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-05-07T02:42:12.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:11 smithi027 ceph-mon[34398]: from='client.16560 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:42:13.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:12 smithi064 ceph-mon[34699]: from='client.26728 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:42:13.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:12 smithi019 ceph-mon[29781]: from='client.26728 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:42:13.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:12 smithi027 ceph-mon[34398]: from='client.26728 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-07T02:42:14.109 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:13 smithi064 ceph-mon[34699]: pgmap v321: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 204 B/s wr, 11 op/s 2024-05-07T02:42:14.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:13 smithi019 ceph-mon[29781]: pgmap v321: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 204 B/s wr, 11 op/s 2024-05-07T02:42:14.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:13 smithi027 ceph-mon[34398]: pgmap v321: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 204 B/s wr, 11 op/s 2024-05-07T02:42:16.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:15 smithi019 ceph-mon[29781]: pgmap v322: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 204 B/s wr, 11 op/s 2024-05-07T02:42:16.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:15 smithi027 ceph-mon[34398]: pgmap v322: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 204 B/s wr, 11 op/s 2024-05-07T02:42:16.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:15 smithi064 ceph-mon[34699]: pgmap v322: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 204 B/s wr, 11 op/s 2024-05-07T02:42:18.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:17 smithi019 ceph-mon[29781]: pgmap v323: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 204 B/s wr, 13 op/s 2024-05-07T02:42:18.146 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:17 smithi019 ceph-mon[29781]: osdmap e98: 12 total, 12 up, 12 in 2024-05-07T02:42:18.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:17 smithi027 ceph-mon[34398]: pgmap v323: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 204 B/s wr, 13 op/s 2024-05-07T02:42:18.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:17 smithi027 ceph-mon[34398]: osdmap e98: 12 total, 12 up, 12 in 2024-05-07T02:42:18.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:17 smithi064 ceph-mon[34699]: pgmap v323: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 204 B/s wr, 13 op/s 2024-05-07T02:42:18.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:17 smithi064 ceph-mon[34699]: osdmap e98: 12 total, 12 up, 12 in 2024-05-07T02:42:20.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:19 smithi019 ceph-mon[29781]: pgmap v325: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 218 B/s wr, 14 op/s 2024-05-07T02:42:20.161 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:19 smithi064 ceph-mon[34699]: pgmap v325: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 218 B/s wr, 14 op/s 2024-05-07T02:42:20.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:19 smithi027 ceph-mon[34398]: pgmap v325: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 218 B/s wr, 14 op/s 2024-05-07T02:42:22.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:21 smithi019 ceph-mon[29781]: pgmap v326: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 204 B/s wr, 13 op/s 2024-05-07T02:42:22.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:21 smithi027 ceph-mon[34398]: pgmap v326: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 204 B/s wr, 13 op/s 2024-05-07T02:42:22.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:21 smithi064 ceph-mon[34699]: pgmap v326: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 204 B/s wr, 13 op/s 2024-05-07T02:42:24.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:23 smithi019 ceph-mon[29781]: pgmap v327: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 102 B/s wr, 11 op/s 2024-05-07T02:42:24.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:23 smithi027 ceph-mon[34398]: pgmap v327: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 102 B/s wr, 11 op/s 2024-05-07T02:42:24.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:23 smithi064 ceph-mon[34699]: pgmap v327: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 102 B/s wr, 11 op/s 2024-05-07T02:42:26.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:25 smithi019 ceph-mon[29781]: pgmap v328: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 102 B/s wr, 11 op/s 2024-05-07T02:42:26.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:25 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:42:26.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:25 smithi027 ceph-mon[34398]: pgmap v328: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 102 B/s wr, 11 op/s 2024-05-07T02:42:26.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:25 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:42:26.307 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:25 smithi064 ceph-mon[34699]: pgmap v328: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 102 B/s wr, 11 op/s 2024-05-07T02:42:26.307 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:25 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:42:28.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:27 smithi027 ceph-mon[34398]: pgmap v329: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-07T02:42:28.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:27 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:42:28.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:27 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:42:28.208 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:27 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:42:28.208 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:27 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:42:28.208 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:27 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:28.208 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:27 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:28.208 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:27 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:28.208 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:27 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:28.208 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:27 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:28.208 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:27 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:28.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:27 smithi064 ceph-mon[34699]: pgmap v329: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-07T02:42:28.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:27 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:42:28.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:27 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:42:28.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:27 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:42:28.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:27 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:42:28.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:27 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:28.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:27 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:28.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:27 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:28.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:27 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:28.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:27 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:28.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:27 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:28.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:27 smithi019 ceph-mon[29781]: pgmap v329: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-05-07T02:42:28.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:27 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:42:28.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:27 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:42:28.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:27 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:42:28.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:27 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:42:28.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:27 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:28.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:27 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:28.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:27 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:28.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:27 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:28.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:27 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:28.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:27 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:29.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:28 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:29.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:28 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:29.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:28 smithi027 ceph-mon[34398]: Adjusting osd_memory_target on smithi027 to 176.2M 2024-05-07T02:42:29.208 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:28 smithi027 ceph-mon[34398]: Unable to set osd_memory_target on smithi027 to 184763084: error parsing value: Value '184763084' is below minimum 939524096 2024-05-07T02:42:29.208 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:28 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:29.208 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:28 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:29.208 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:28 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:29.208 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:28 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:29.208 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:28 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:29.208 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:28 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:29.208 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:28 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:29.209 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:28 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:29.209 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:28 smithi027 ceph-mon[34398]: Adjusting osd_memory_target on smithi064 to 176.2M 2024-05-07T02:42:29.209 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:28 smithi027 ceph-mon[34398]: Unable to set osd_memory_target on smithi064 to 184793907: error parsing value: Value '184793907' is below minimum 939524096 2024-05-07T02:42:29.209 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:28 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:42:29.209 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:28 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:42:29.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:28 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:29.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:28 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:29.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:28 smithi064 ceph-mon[34699]: Adjusting osd_memory_target on smithi027 to 176.2M 2024-05-07T02:42:29.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:28 smithi064 ceph-mon[34699]: Unable to set osd_memory_target on smithi027 to 184763084: error parsing value: Value '184763084' is below minimum 939524096 2024-05-07T02:42:29.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:28 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:29.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:28 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:29.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:28 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:29.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:28 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:29.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:28 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:29.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:28 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:29.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:28 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:29.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:28 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:29.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:28 smithi064 ceph-mon[34699]: Adjusting osd_memory_target on smithi064 to 176.2M 2024-05-07T02:42:29.361 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:28 smithi064 ceph-mon[34699]: Unable to set osd_memory_target on smithi064 to 184793907: error parsing value: Value '184793907' is below minimum 939524096 2024-05-07T02:42:29.361 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:28 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:42:29.361 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:28 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:42:29.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:28 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:29.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:28 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:29.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:28 smithi019 ceph-mon[29781]: Adjusting osd_memory_target on smithi027 to 176.2M 2024-05-07T02:42:29.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:28 smithi019 ceph-mon[29781]: Unable to set osd_memory_target on smithi027 to 184763084: error parsing value: Value '184763084' is below minimum 939524096 2024-05-07T02:42:29.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:28 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:29.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:28 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:29.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:28 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:29.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:28 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:29.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:28 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:29.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:28 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:29.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:28 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:29.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:28 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:29.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:28 smithi019 ceph-mon[29781]: Adjusting osd_memory_target on smithi064 to 176.2M 2024-05-07T02:42:29.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:28 smithi019 ceph-mon[29781]: Unable to set osd_memory_target on smithi064 to 184793907: error parsing value: Value '184793907' is below minimum 939524096 2024-05-07T02:42:29.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:28 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:42:29.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:28 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:42:30.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:29 smithi027 ceph-mon[34398]: pgmap v330: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-07T02:42:30.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:29 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:30.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:29 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:30.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:29 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:30.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:29 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:30.208 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:29 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:30.208 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:29 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:30.208 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:29 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:30.208 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:29 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:30.208 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:29 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:42:30.208 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:29 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:42:30.208 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:29 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:42:30.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:29 smithi064 ceph-mon[34699]: pgmap v330: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-07T02:42:30.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:29 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:30.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:29 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:30.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:29 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:30.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:29 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:30.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:29 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:30.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:29 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:30.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:29 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:30.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:29 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:30.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:29 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:42:30.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:29 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:42:30.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:29 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:42:30.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:29 smithi019 ceph-mon[29781]: pgmap v330: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-07T02:42:30.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:29 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:30.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:29 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:30.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:29 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:30.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:29 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:30.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:29 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:30.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:29 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:30.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:29 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:30.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:29 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-07T02:42:30.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:29 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:42:30.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:29 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:42:30.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:29 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:42:31.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:30 smithi027 ceph-mon[34398]: Adjusting osd_memory_target on smithi019 to 177.5M 2024-05-07T02:42:31.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:30 smithi027 ceph-mon[34398]: Unable to set osd_memory_target on smithi019 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-05-07T02:42:31.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:30 smithi064 ceph-mon[34699]: Adjusting osd_memory_target on smithi019 to 177.5M 2024-05-07T02:42:31.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:30 smithi064 ceph-mon[34699]: Unable to set osd_memory_target on smithi019 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-05-07T02:42:31.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:30 smithi019 ceph-mon[29781]: Adjusting osd_memory_target on smithi019 to 177.5M 2024-05-07T02:42:31.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:30 smithi019 ceph-mon[29781]: Unable to set osd_memory_target on smithi019 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-05-07T02:42:32.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:31 smithi027 ceph-mon[34398]: pgmap v331: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-07T02:42:32.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:31 smithi064 ceph-mon[34699]: pgmap v331: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-07T02:42:32.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:31 smithi019 ceph-mon[29781]: pgmap v331: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-07T02:42:33.927 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-07T02:42:34.201 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:33 smithi019 ceph-mon[29781]: pgmap v332: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:42:34.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:33 smithi027 ceph-mon[34398]: pgmap v332: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:42:34.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:33 smithi064 ceph-mon[34699]: pgmap v332: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:42:34.410 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:42:36.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:35 smithi027 ceph-mon[34398]: pgmap v333: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:42:36.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:35 smithi064 ceph-mon[34699]: pgmap v333: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:42:36.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:35 smithi019 ceph-mon[29781]: pgmap v333: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:42:37.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.073+0000 7f4355c00700 1 -- 172.21.15.19:0/1449013508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4350104400 msgr2=0x7f4350104810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:37.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.073+0000 7f4355c00700 1 --2- 172.21.15.19:0/1449013508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4350104400 0x7f4350104810 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f43400099c0 tx=0x7f4340009cd0 comp rx=0 tx=0).stop 2024-05-07T02:42:37.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.073+0000 7f4355c00700 1 -- 172.21.15.19:0/1449013508 shutdown_connections 2024-05-07T02:42:37.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.073+0000 7f4355c00700 1 --2- 172.21.15.19:0/1449013508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4350109680 0x7f4350110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:37.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.073+0000 7f4355c00700 1 --2- 172.21.15.19:0/1449013508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4350104400 0x7f4350104810 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:37.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.073+0000 7f4355c00700 1 --2- 172.21.15.19:0/1449013508 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4350108d30 0x7f4350109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:37.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.073+0000 7f4355c00700 1 -- 172.21.15.19:0/1449013508 >> 172.21.15.19:0/1449013508 conn(0x7f4350079d10 msgr2=0x7f435007a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:42:37.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.073+0000 7f4355c00700 1 -- 172.21.15.19:0/1449013508 shutdown_connections 2024-05-07T02:42:37.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.073+0000 7f4355c00700 1 -- 172.21.15.19:0/1449013508 wait complete. 2024-05-07T02:42:37.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.075+0000 7f4355c00700 1 Processor -- start 2024-05-07T02:42:37.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.075+0000 7f4355c00700 1 -- start start 2024-05-07T02:42:37.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.075+0000 7f4355c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4350104400 0x7f435019ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:37.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.075+0000 7f4355c00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4350108d30 0x7f435019d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:37.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.075+0000 7f4355c00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4350109680 0x7f43501a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:37.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.075+0000 7f4355c00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4350114300 con 0x7f4350104400 2024-05-07T02:42:37.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.075+0000 7f4355c00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f4350114180 con 0x7f4350109680 2024-05-07T02:42:37.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.075+0000 7f4355c00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4350114480 con 0x7f4350108d30 2024-05-07T02:42:37.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.076+0000 7f434fe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4350104400 0x7f435019ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:37.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.076+0000 7f434f400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4350108d30 0x7f435019d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:37.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.076+0000 7f434fe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4350104400 0x7f435019ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38872/0 (socket says 172.21.15.19:38872) 2024-05-07T02:42:37.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.076+0000 7f434fe00700 1 -- 172.21.15.19:0/618600068 learned_addr learned my addr 172.21.15.19:0/618600068 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:42:37.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.076+0000 7f4355200700 1 --2- 172.21.15.19:0/618600068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4350109680 0x7f43501a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:37.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.076+0000 7f434fe00700 1 -- 172.21.15.19:0/618600068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4350108d30 msgr2=0x7f435019d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:37.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.076+0000 7f434fe00700 1 --2- 172.21.15.19:0/618600068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4350108d30 0x7f435019d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:37.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.076+0000 7f434fe00700 1 -- 172.21.15.19:0/618600068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4350109680 msgr2=0x7f43501a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:37.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.076+0000 7f434fe00700 1 --2- 172.21.15.19:0/618600068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4350109680 0x7f43501a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:37.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.076+0000 7f434fe00700 1 -- 172.21.15.19:0/618600068 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43501a1d20 con 0x7f4350104400 2024-05-07T02:42:37.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.076+0000 7f4355200700 1 --2- 172.21.15.19:0/618600068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4350109680 0x7f43501a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:42:37.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.076+0000 7f434f400700 1 --2- 172.21.15.19:0/618600068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4350108d30 0x7f435019d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:42:37.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.077+0000 7f434fe00700 1 --2- 172.21.15.19:0/618600068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4350104400 0x7f435019ce50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4340006010 tx=0x7f4340005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:42:37.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.077+0000 7f434cc00700 1 -- 172.21.15.19:0/618600068 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4340026070 con 0x7f4350104400 2024-05-07T02:42:37.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.077+0000 7f4355c00700 1 -- 172.21.15.19:0/618600068 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43501a1fb0 con 0x7f4350104400 2024-05-07T02:42:37.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.077+0000 7f434cc00700 1 -- 172.21.15.19:0/618600068 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4340003c70 con 0x7f4350104400 2024-05-07T02:42:37.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.077+0000 7f4355c00700 1 -- 172.21.15.19:0/618600068 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43501a9930 con 0x7f4350104400 2024-05-07T02:42:37.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.077+0000 7f434cc00700 1 -- 172.21.15.19:0/618600068 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434000fae0 con 0x7f4350104400 2024-05-07T02:42:37.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.081+0000 7f434cc00700 1 -- 172.21.15.19:0/618600068 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4340029070 con 0x7f4350104400 2024-05-07T02:42:37.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.082+0000 7f4365d5db80 1 Processor -- start 2024-05-07T02:42:37.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.082+0000 7f4365d5db80 1 -- start start 2024-05-07T02:42:37.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.082+0000 7f4365d5db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558fca33ea90 0x558fca33eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:37.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.083+0000 7f4365d5db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fca33f3e0 0x558fca348620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:37.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.083+0000 7f434cc00700 1 --2- 172.21.15.19:0/618600068 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f43340b29f0 0x7f43340b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:37.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.083+0000 7f4365d5db80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fca348b60 0x558fca34af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:37.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.083+0000 7f4365d5db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558fca260a20 con 0x558fca33f3e0 2024-05-07T02:42:37.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.083+0000 7f4365d5db80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x558fca1fe760 con 0x558fca348b60 2024-05-07T02:42:37.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.083+0000 7f4365d5db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558fca25c9e0 con 0x558fca33ea90 2024-05-07T02:42:37.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.083+0000 7f434cc00700 1 -- 172.21.15.19:0/618600068 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9525+0+0 (secure 0 0 0) 0x7f43400e31d0 con 0x7f4350104400 2024-05-07T02:42:37.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.083+0000 7f434f400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fca33f3e0 0x558fca348620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:37.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.083+0000 7f4355200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558fca33ea90 0x558fca33eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:37.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.083+0000 7f4355200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558fca33ea90 0x558fca33eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:40368/0 (socket says 172.21.15.19:40368) 2024-05-07T02:42:37.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.083+0000 7f4355200700 1 -- 172.21.15.19:0/367668084 learned_addr learned my addr 172.21.15.19:0/367668084 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:42:37.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.083+0000 7f4355200700 1 --2- 172.21.15.19:0/367668084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fca348b60 0x558fca34af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:37.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.084+0000 7f434fe00700 1 --2- 172.21.15.19:0/618600068 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f43340b29f0 0x7f43340b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:37.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.084+0000 7f4355200700 1 -- 172.21.15.19:0/367668084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fca348b60 msgr2=0x558fca34af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:37.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.084+0000 7f4355200700 1 --2- 172.21.15.19:0/367668084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fca348b60 0x558fca34af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:37.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.084+0000 7f4355200700 1 -- 172.21.15.19:0/367668084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fca33f3e0 msgr2=0x558fca348620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:37.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.084+0000 7f4355200700 1 --2- 172.21.15.19:0/367668084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fca33f3e0 0x558fca348620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:37.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.084+0000 7f4355200700 1 -- 172.21.15.19:0/367668084 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fca1ad8d0 con 0x558fca33ea90 2024-05-07T02:42:37.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.084+0000 7f4355200700 1 --2- 172.21.15.19:0/367668084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558fca33ea90 0x558fca33eea0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4348012ed0 tx=0x7f4348009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3b5d8e5e090d7f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:42:37.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.084+0000 7f4345600700 1 -- 172.21.15.19:0/367668084 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434801f070 con 0x558fca33ea90 2024-05-07T02:42:37.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.084+0000 7f434fe00700 1 --2- 172.21.15.19:0/618600068 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f43340b29f0 0x7f43340b4f00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f43400a6170 tx=0x7f4340009cd0 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:42:37.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.084+0000 7f4345600700 1 -- 172.21.15.19:0/367668084 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43480092d0 con 0x558fca33ea90 2024-05-07T02:42:37.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.084+0000 7f4345600700 1 -- 172.21.15.19:0/367668084 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4348007d20 con 0x558fca33ea90 2024-05-07T02:42:37.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.085+0000 7f4365d5db80 1 -- 172.21.15.19:0/367668084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558fca33ea90 msgr2=0x558fca33eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:37.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.085+0000 7f4365d5db80 1 --2- 172.21.15.19:0/367668084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558fca33ea90 0x558fca33eea0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4348012ed0 tx=0x7f4348009920 comp rx=0 tx=0).stop 2024-05-07T02:42:37.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.086+0000 7f4365d5db80 1 -- 172.21.15.19:0/367668084 shutdown_connections 2024-05-07T02:42:37.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.086+0000 7f4365d5db80 1 --2- 172.21.15.19:0/367668084 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fca348b60 0x558fca34af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:37.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.086+0000 7f4365d5db80 1 --2- 172.21.15.19:0/367668084 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558fca33ea90 0x558fca33eea0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:37.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.086+0000 7f4365d5db80 1 --2- 172.21.15.19:0/367668084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fca33f3e0 0x558fca348620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:37.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.086+0000 7f4365d5db80 1 -- 172.21.15.19:0/367668084 >> 172.21.15.19:0/367668084 conn(0x558fca1c0e40 msgr2=0x558fca339880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:42:37.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.086+0000 7f4365d5db80 1 -- 172.21.15.19:0/367668084 shutdown_connections 2024-05-07T02:42:37.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.086+0000 7f4365d5db80 1 -- 172.21.15.19:0/367668084 wait complete. 2024-05-07T02:42:37.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.086+0000 7f4365d5db80 1 -- start start 2024-05-07T02:42:37.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.087+0000 7f4365d5db80 1 Processor -- start 2024-05-07T02:42:37.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.087+0000 7f4365d5db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fca33ea90 0x558fca345b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:37.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.087+0000 7f4365d5db80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fca33f3e0 0x558fca3460b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:37.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.087+0000 7f4365d5db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558fca348b60 0x558fca3465f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:37.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.087+0000 7f4365d5db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558fca34b960 con 0x558fca33ea90 2024-05-07T02:42:37.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.087+0000 7f4365d5db80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x558fca1c6b20 con 0x558fca33f3e0 2024-05-07T02:42:37.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.087+0000 7f4365d5db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558fca34bae0 con 0x558fca348b60 2024-05-07T02:42:37.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.087+0000 7f4365d5db80 10 client.0 _ll_register_callbacks cb 0x558fca207c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:42:37.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.088+0000 7f434f400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fca33f3e0 0x558fca3460b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:37.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.088+0000 7f4355200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fca33ea90 0x558fca345b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:37.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.088+0000 7f434f400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fca33f3e0 0x558fca3460b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:45316/0 (socket says 172.21.15.19:45316) 2024-05-07T02:42:37.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.088+0000 7f434f400700 1 -- 172.21.15.19:0/2681737539 learned_addr learned my addr 172.21.15.19:0/2681737539 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:42:37.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.088+0000 7f4355200700 1 --2- 172.21.15.19:0/2681737539 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558fca348b60 0x558fca3465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:37.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.088+0000 7f434f400700 1 -- 172.21.15.19:0/2681737539 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558fca348b60 msgr2=0x558fca3465f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:37.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.088+0000 7f434f400700 1 --2- 172.21.15.19:0/2681737539 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558fca348b60 0x558fca3465f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:37.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.088+0000 7f434f400700 1 -- 172.21.15.19:0/2681737539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fca33ea90 msgr2=0x558fca345b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:37.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.088+0000 7f434f400700 1 --2- 172.21.15.19:0/2681737539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fca33ea90 0x558fca345b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:37.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.088+0000 7f434f400700 1 -- 172.21.15.19:0/2681737539 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fca226420 con 0x558fca33f3e0 2024-05-07T02:42:37.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.088+0000 7f4355200700 1 --2- 172.21.15.19:0/2681737539 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558fca348b60 0x558fca3465f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:42:37.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.089+0000 7f434f400700 1 --2- 172.21.15.19:0/2681737539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fca33f3e0 0x558fca3460b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f433c011d10 tx=0x7f433c00f6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dad26b546dc2b4df server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:42:37.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.089+0000 7f433a000700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:42:37.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.089+0000 7f433a000700 1 -- 172.21.15.19:0/2681737539 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433c00fe40 con 0x558fca33f3e0 2024-05-07T02:42:37.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.089+0000 7f433a000700 1 -- 172.21.15.19:0/2681737539 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9525+0+0 (secure 0 0 0) 0x7f433c004050 con 0x558fca33f3e0 2024-05-07T02:42:37.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.089+0000 7f4365d5db80 1 -- 172.21.15.19:0/2681737539 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43480092d0 con 0x558fca33f3e0 2024-05-07T02:42:37.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.089+0000 7f433a000700 1 -- 172.21.15.19:0/2681737539 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f433c025050 con 0x558fca33f3e0 2024-05-07T02:42:37.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.089+0000 7f4365d5db80 10 client.26780 fetch_fsmap learned FSMap version 50 2024-05-07T02:42:37.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.089+0000 7f4365d5db80 1 -- 172.21.15.19:0/2681737539 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x558fca3565f0 con 0x558fca33f3e0 2024-05-07T02:42:37.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.090+0000 7f433a000700 1 -- 172.21.15.19:0/2681737539 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43480092d0 con 0x558fca33f3e0 2024-05-07T02:42:37.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.090+0000 7f433a000700 1 -- 172.21.15.19:0/2681737539 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 50) ==== 8362+0+0 (secure 0 0 0) 0x7f433c01a430 con 0x558fca33f3e0 2024-05-07T02:42:37.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.090+0000 7f4365d5db80 10 client.26780 fetch_fsmap finished waiting for FSMap version 50 2024-05-07T02:42:37.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.090+0000 7f4365d5db80 10 client.26780 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:42:37.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.090+0000 7f4365d5db80 20 client.26780 populate_metadata read hostname 'smithi019' 2024-05-07T02:42:37.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.090+0000 7f4365d5db80 1 --2- 172.21.15.19:0/2681737539 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x558fca358c30 0x558fca35b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:37.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.090+0000 7f4365d5db80 4 client.26780 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:42:37.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.090+0000 7f4365d5db80 1 -- 172.21.15.19:0/2681737539 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558fca30b2e0 con 0x558fca358c30 2024-05-07T02:42:37.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.091+0000 7f4355200700 1 --2- 172.21.15.19:0/2681737539 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x558fca358c30 0x558fca35b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:37.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.091+0000 7f4355200700 1 --2- 172.21.15.19:0/2681737539 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x558fca358c30 0x558fca35b090 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=419d936e99ff31c server_cookie=76d04560ebb67328 in_seq=0 out_seq=0 2024-05-07T02:42:37.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.091+0000 7f433a000700 10 client.26780 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:42:37.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.092+0000 7f433a000700 1 -- 172.21.15.19:0/2681737539 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558fca30b2e0 con 0x558fca358c30 2024-05-07T02:42:37.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.092+0000 7f433a000700 10 client.26780 handle_command_reply: tid=0 2024-05-07T02:42:37.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.092+0000 7f4365d5db80 1 client.26780 shutdown 2024-05-07T02:42:37.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.092+0000 7f4365d5db80 1 -- 172.21.15.19:0/2681737539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fca33f3e0 msgr2=0x558fca3460b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:37.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.092+0000 7f4365d5db80 1 --2- 172.21.15.19:0/2681737539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fca33f3e0 0x558fca3460b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f433c011d10 tx=0x7f433c00f6a0 comp rx=0 tx=0).stop 2024-05-07T02:42:37.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.092+0000 7f4365d5db80 1 -- 172.21.15.19:0/2681737539 shutdown_connections 2024-05-07T02:42:37.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.092+0000 7f4365d5db80 1 --2- 172.21.15.19:0/2681737539 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x558fca358c30 0x558fca35b090 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:37.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.092+0000 7f4365d5db80 1 --2- 172.21.15.19:0/2681737539 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558fca348b60 0x558fca3465f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:37.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.092+0000 7f4365d5db80 1 --2- 172.21.15.19:0/2681737539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fca33ea90 0x558fca345b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:37.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.092+0000 7f4365d5db80 1 --2- 172.21.15.19:0/2681737539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fca33f3e0 0x558fca3460b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:37.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.092+0000 7f433a000700 0 client.26780 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:42:37.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.092+0000 7f4365d5db80 1 -- 172.21.15.19:0/2681737539 >> 172.21.15.19:0/2681737539 conn(0x558fca1c0e40 msgr2=0x558fca33b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:42:37.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.093+0000 7f4365d5db80 1 -- 172.21.15.19:0/2681737539 shutdown_connections 2024-05-07T02:42:37.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.093+0000 7f4365d5db80 1 -- 172.21.15.19:0/2681737539 wait complete. 2024-05-07T02:42:37.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.093+0000 7f4365d5db80 20 client.26780 trim_cache size 0 max 16384 2024-05-07T02:42:37.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.115+0000 7f4365d5db80 1 Processor -- start 2024-05-07T02:42:37.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.115+0000 7f4365d5db80 1 -- start start 2024-05-07T02:42:37.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.115+0000 7f4365d5db80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fca33ea90 0x558fca3460e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:37.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.115+0000 7f4365d5db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fca33f3e0 0x558fca360850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:37.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.115+0000 7f4365d5db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558fca348b60 0x558fca360d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:37.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.115+0000 7f4365d5db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f43501b2650 con 0x558fca33f3e0 2024-05-07T02:42:37.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.115+0000 7f4365d5db80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f43501b4190 con 0x558fca33ea90 2024-05-07T02:42:37.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.115+0000 7f4365d5db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f43501b2d20 con 0x558fca348b60 2024-05-07T02:42:37.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.116+0000 7f4355200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fca33ea90 0x558fca3460e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:37.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.116+0000 7f4355200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fca33ea90 0x558fca3460e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:45332/0 (socket says 172.21.15.19:45332) 2024-05-07T02:42:37.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.116+0000 7f4355200700 1 -- 172.21.15.19:0/60060450 learned_addr learned my addr 172.21.15.19:0/60060450 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:42:37.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.116+0000 7f4355200700 1 --2- 172.21.15.19:0/60060450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558fca348b60 0x558fca360d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:37.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.116+0000 7f434f400700 1 --2- 172.21.15.19:0/60060450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fca33f3e0 0x558fca360850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:37.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.116+0000 7f4355200700 1 -- 172.21.15.19:0/60060450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558fca348b60 msgr2=0x558fca360d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:37.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.116+0000 7f4355200700 1 --2- 172.21.15.19:0/60060450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558fca348b60 0x558fca360d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:37.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.116+0000 7f4355200700 1 -- 172.21.15.19:0/60060450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fca33f3e0 msgr2=0x558fca360850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:37.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.116+0000 7f4355200700 1 --2- 172.21.15.19:0/60060450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fca33f3e0 0x558fca360850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:37.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.116+0000 7f4355200700 1 -- 172.21.15.19:0/60060450 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fca3440e0 con 0x558fca33ea90 2024-05-07T02:42:37.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.116+0000 7f4355200700 1 --2- 172.21.15.19:0/60060450 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fca33ea90 0x558fca3460e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f434800b010 tx=0x7f4348007900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb9e3f1e360da7f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:42:37.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.116+0000 7f4344c00700 1 -- 172.21.15.19:0/60060450 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434801f040 con 0x558fca33ea90 2024-05-07T02:42:37.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.116+0000 7f4344c00700 1 -- 172.21.15.19:0/60060450 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4348003a00 con 0x558fca33ea90 2024-05-07T02:42:37.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.116+0000 7f4344c00700 1 -- 172.21.15.19:0/60060450 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4348007c90 con 0x558fca33ea90 2024-05-07T02:42:37.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.117+0000 7f4365d5db80 1 -- 172.21.15.19:0/60060450 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fca33ea90 msgr2=0x558fca3460e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:37.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.117+0000 7f4365d5db80 1 --2- 172.21.15.19:0/60060450 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fca33ea90 0x558fca3460e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f434800b010 tx=0x7f4348007900 comp rx=0 tx=0).stop 2024-05-07T02:42:37.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.117+0000 7f4365d5db80 1 -- 172.21.15.19:0/60060450 shutdown_connections 2024-05-07T02:42:37.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.117+0000 7f4365d5db80 1 --2- 172.21.15.19:0/60060450 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558fca348b60 0x558fca360d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:37.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.117+0000 7f4365d5db80 1 --2- 172.21.15.19:0/60060450 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fca33ea90 0x558fca3460e0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:37.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.117+0000 7f4365d5db80 1 --2- 172.21.15.19:0/60060450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fca33f3e0 0x558fca360850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:37.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.117+0000 7f4365d5db80 1 -- 172.21.15.19:0/60060450 >> 172.21.15.19:0/60060450 conn(0x558fca1c0e40 msgr2=0x558fca346fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:42:37.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.117+0000 7f4365d5db80 1 -- 172.21.15.19:0/60060450 shutdown_connections 2024-05-07T02:42:37.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.118+0000 7f4365d5db80 1 -- 172.21.15.19:0/60060450 wait complete. 2024-05-07T02:42:37.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.118+0000 7f4365d5db80 1 -- start start 2024-05-07T02:42:37.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.118+0000 7f4365d5db80 1 Processor -- start 2024-05-07T02:42:37.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.119+0000 7f4365d5db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558fca33ea90 0x558fca34ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:37.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.119+0000 7f4365d5db80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fca33f3e0 0x558fca34cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:37.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.119+0000 7f4365d5db80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fca348b60 0x558fca34d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:37.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.119+0000 7f4365d5db80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f43501b3ac0 con 0x558fca33f3e0 2024-05-07T02:42:37.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.119+0000 7f4365d5db80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f43501b33f0 con 0x558fca348b60 2024-05-07T02:42:37.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.119+0000 7f4365d5db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f43501b0610 con 0x558fca33ea90 2024-05-07T02:42:37.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.119+0000 7f4365d5db80 10 client.0 _ll_register_callbacks cb 0x558fca207c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:42:37.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.119+0000 7f434f400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fca33f3e0 0x558fca34cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:37.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.119+0000 7f4355200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558fca33ea90 0x558fca34ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:37.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.119+0000 7f434f400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fca33f3e0 0x558fca34cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38914/0 (socket says 172.21.15.19:38914) 2024-05-07T02:42:37.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.119+0000 7f434f400700 1 -- 172.21.15.19:0/4022195618 learned_addr learned my addr 172.21.15.19:0/4022195618 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:42:37.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.119+0000 7f4355200700 1 --2- 172.21.15.19:0/4022195618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fca348b60 0x558fca34d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:37.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.119+0000 7f434f400700 1 -- 172.21.15.19:0/4022195618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558fca33ea90 msgr2=0x558fca34ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:37.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.119+0000 7f434f400700 1 --2- 172.21.15.19:0/4022195618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558fca33ea90 0x558fca34ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:37.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.119+0000 7f434f400700 1 -- 172.21.15.19:0/4022195618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fca348b60 msgr2=0x558fca34d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:37.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.119+0000 7f434f400700 1 --2- 172.21.15.19:0/4022195618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fca348b60 0x558fca34d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:37.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.120+0000 7f434f400700 1 -- 172.21.15.19:0/4022195618 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fca3c2400 con 0x558fca33f3e0 2024-05-07T02:42:37.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.120+0000 7f434f400700 1 --2- 172.21.15.19:0/4022195618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fca33f3e0 0x558fca34cf50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x558fca339e50 tx=0x7f433c01a670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2612349c526dc26 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:42:37.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.120+0000 7f433a000700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-07T02:42:37.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.120+0000 7f433a000700 1 -- 172.21.15.19:0/4022195618 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433c02d020 con 0x558fca33f3e0 2024-05-07T02:42:37.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.120+0000 7f433a000700 1 -- 172.21.15.19:0/4022195618 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9525+0+0 (secure 0 0 0) 0x7f433c01acd0 con 0x558fca33f3e0 2024-05-07T02:42:37.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.120+0000 7f4365d5db80 1 -- 172.21.15.19:0/4022195618 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4348003a00 con 0x558fca33f3e0 2024-05-07T02:42:37.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.120+0000 7f433a000700 1 -- 172.21.15.19:0/4022195618 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f433c038050 con 0x558fca33f3e0 2024-05-07T02:42:37.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.120+0000 7f4365d5db80 10 client.16584 fetch_fsmap learned FSMap version 50 2024-05-07T02:42:37.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.120+0000 7f4365d5db80 1 -- 172.21.15.19:0/4022195618 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x558fca3c36a0 con 0x558fca33f3e0 2024-05-07T02:42:37.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.121+0000 7f433a000700 1 -- 172.21.15.19:0/4022195618 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4348003a00 con 0x558fca33f3e0 2024-05-07T02:42:37.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.121+0000 7f433a000700 1 -- 172.21.15.19:0/4022195618 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 50) ==== 8362+0+0 (secure 0 0 0) 0x7f433c0038f0 con 0x558fca33f3e0 2024-05-07T02:42:37.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.121+0000 7f4365d5db80 10 client.16584 fetch_fsmap finished waiting for FSMap version 50 2024-05-07T02:42:37.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.121+0000 7f4365d5db80 10 client.16584 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:42:37.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.121+0000 7f4365d5db80 20 client.16584 populate_metadata read hostname 'smithi019' 2024-05-07T02:42:37.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.121+0000 7f4365d5db80 1 --2- 172.21.15.19:0/4022195618 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x558fca3c3e30 0x558fca3c6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:37.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.121+0000 7f4365d5db80 4 client.16584 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-07T02:42:37.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.121+0000 7f4365d5db80 1 -- 172.21.15.19:0/4022195618 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558fca26e400 con 0x558fca3c3e30 2024-05-07T02:42:37.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.122+0000 7f4355200700 1 --2- 172.21.15.19:0/4022195618 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x558fca3c3e30 0x558fca3c6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:37.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.122+0000 7f4355200700 1 --2- 172.21.15.19:0/4022195618 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x558fca3c3e30 0x558fca3c6250 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c3421a4744997b1 server_cookie=6910dd9400447ccc in_seq=0 out_seq=0 2024-05-07T02:42:37.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.122+0000 7f433a000700 10 client.16584 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:42:37.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.122+0000 7f433a000700 1 -- 172.21.15.19:0/4022195618 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558fca26e400 con 0x558fca3c3e30 2024-05-07T02:42:37.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.122+0000 7f433a000700 10 client.16584 handle_command_reply: tid=0 2024-05-07T02:42:37.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.122+0000 7f4365d5db80 1 client.16584 shutdown 2024-05-07T02:42:37.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.123+0000 7f4365d5db80 1 -- 172.21.15.19:0/4022195618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fca33f3e0 msgr2=0x558fca34cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:37.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.123+0000 7f4365d5db80 1 --2- 172.21.15.19:0/4022195618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fca33f3e0 0x558fca34cf50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x558fca339e50 tx=0x7f433c01a670 comp rx=0 tx=0).stop 2024-05-07T02:42:37.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.123+0000 7f4365d5db80 1 -- 172.21.15.19:0/4022195618 shutdown_connections 2024-05-07T02:42:37.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.123+0000 7f4365d5db80 1 --2- 172.21.15.19:0/4022195618 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x558fca3c3e30 0x558fca3c6250 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:37.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.123+0000 7f4365d5db80 1 --2- 172.21.15.19:0/4022195618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558fca348b60 0x558fca34d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:37.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.123+0000 7f4365d5db80 1 --2- 172.21.15.19:0/4022195618 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558fca33ea90 0x558fca34ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:37.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.123+0000 7f4365d5db80 1 --2- 172.21.15.19:0/4022195618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558fca33f3e0 0x558fca34cf50 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:37.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.123+0000 7f433a000700 0 client.16584 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:42:37.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.123+0000 7f4365d5db80 1 -- 172.21.15.19:0/4022195618 >> 172.21.15.19:0/4022195618 conn(0x558fca1c0e40 msgr2=0x558fca3478f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:42:37.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.123+0000 7f4365d5db80 1 -- 172.21.15.19:0/4022195618 shutdown_connections 2024-05-07T02:42:37.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.123+0000 7f4365d5db80 1 -- 172.21.15.19:0/4022195618 wait complete. 2024-05-07T02:42:37.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.123+0000 7f4365d5db80 20 client.16584 trim_cache size 0 max 16384 2024-05-07T02:42:37.130 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:42:37.130 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-05-07T02:42:37.130 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-05-07T02:42:37.131 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:42:37.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.124+0000 7f4355c00700 1 -- 172.21.15.19:0/618600068 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f43340b29f0 msgr2=0x7f43340b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:37.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.124+0000 7f4355c00700 1 --2- 172.21.15.19:0/618600068 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f43340b29f0 0x7f43340b4f00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f43400a6170 tx=0x7f4340009cd0 comp rx=0 tx=0).stop 2024-05-07T02:42:37.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.124+0000 7f4355c00700 1 -- 172.21.15.19:0/618600068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4350104400 msgr2=0x7f435019ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:37.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.124+0000 7f4355c00700 1 --2- 172.21.15.19:0/618600068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4350104400 0x7f435019ce50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4340006010 tx=0x7f4340005280 comp rx=0 tx=0).stop 2024-05-07T02:42:37.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.124+0000 7f4355c00700 1 -- 172.21.15.19:0/618600068 shutdown_connections 2024-05-07T02:42:37.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.124+0000 7f4355c00700 1 --2- 172.21.15.19:0/618600068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4350109680 0x7f43501a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:37.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.124+0000 7f4355c00700 1 --2- 172.21.15.19:0/618600068 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f43340b29f0 0x7f43340b4f00 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:37.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.124+0000 7f4355c00700 1 --2- 172.21.15.19:0/618600068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4350104400 0x7f435019ce50 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:37.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.124+0000 7f4355c00700 1 --2- 172.21.15.19:0/618600068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4350108d30 0x7f435019d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:37.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.124+0000 7f4355c00700 1 -- 172.21.15.19:0/618600068 >> 172.21.15.19:0/618600068 conn(0x7f4350079d10 msgr2=0x7f4350110ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:42:37.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.124+0000 7f4355c00700 1 -- 172.21.15.19:0/618600068 shutdown_connections 2024-05-07T02:42:37.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:37.124+0000 7f4355c00700 1 -- 172.21.15.19:0/618600068 wait complete. 2024-05-07T02:42:37.936 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-07T02:42:37.937 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-07T02:42:38.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:37 smithi019 ceph-mon[29781]: pgmap v334: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:42:38.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:37 smithi027 ceph-mon[34398]: pgmap v334: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:42:38.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:37 smithi064 ceph-mon[34699]: pgmap v334: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:42:38.424 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:42:40.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:39 smithi019 ceph-mon[29781]: pgmap v335: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:42:40.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:39 smithi027 ceph-mon[34398]: pgmap v335: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:42:40.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:39 smithi064 ceph-mon[34699]: pgmap v335: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:42:40.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:40.862+0000 7f1f50e00700 1 -- 172.21.15.19:0/2528312634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f4c10af20 msgr2=0x7f1f4c10b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:40.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:40.862+0000 7f1f50e00700 1 --2- 172.21.15.19:0/2528312634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f4c10af20 0x7f1f4c10b330 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1f340099c0 tx=0x7f1f34009cd0 comp rx=0 tx=0).stop 2024-05-07T02:42:40.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:40.862+0000 7f1f50e00700 1 -- 172.21.15.19:0/2528312634 shutdown_connections 2024-05-07T02:42:40.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:40.862+0000 7f1f50e00700 1 --2- 172.21.15.19:0/2528312634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f4c10b870 0x7f1f4c1131d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:40.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:40.862+0000 7f1f50e00700 1 --2- 172.21.15.19:0/2528312634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f4c106640 0x7f1f4c106a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:40.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:40.862+0000 7f1f50e00700 1 --2- 172.21.15.19:0/2528312634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f4c10af20 0x7f1f4c10b330 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:40.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:40.862+0000 7f1f50e00700 1 -- 172.21.15.19:0/2528312634 >> 172.21.15.19:0/2528312634 conn(0x7f1f4c102030 msgr2=0x7f1f4c104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:42:40.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:40.862+0000 7f1f50e00700 1 -- 172.21.15.19:0/2528312634 shutdown_connections 2024-05-07T02:42:40.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:40.862+0000 7f1f50e00700 1 -- 172.21.15.19:0/2528312634 wait complete. 2024-05-07T02:42:40.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:40.864+0000 7f1f50e00700 1 Processor -- start 2024-05-07T02:42:40.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:40.865+0000 7f1f50e00700 1 -- start start 2024-05-07T02:42:40.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:40.865+0000 7f1f50e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f4c106640 0x7f1f4c19ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:40.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:40.865+0000 7f1f50e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f4c10af20 0x7f1f4c19d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:40.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:40.865+0000 7f1f50e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f4c10b870 0x7f1f4c1a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:40.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:40.865+0000 7f1f50e00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4c1164d0 con 0x7f1f4c106640 2024-05-07T02:42:40.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:40.865+0000 7f1f50e00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4c116350 con 0x7f1f4c10af20 2024-05-07T02:42:40.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:40.865+0000 7f1f50e00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4c116650 con 0x7f1f4c10b870 2024-05-07T02:42:40.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:40.865+0000 7f1f4b400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f4c106640 0x7f1f4c19ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:40.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:40.866+0000 7f1f4b400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f4c106640 0x7f1f4c19ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53208/0 (socket says 172.21.15.19:53208) 2024-05-07T02:42:40.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:40.866+0000 7f1f4b400700 1 -- 172.21.15.19:0/2960307384 learned_addr learned my addr 172.21.15.19:0/2960307384 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:42:40.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:40.866+0000 7f1f43e00700 1 --2- 172.21.15.19:0/2960307384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f4c10af20 0x7f1f4c19d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:40.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:40.866+0000 7f1f4be00700 1 --2- 172.21.15.19:0/2960307384 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f4c10b870 0x7f1f4c1a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:40.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:40.866+0000 7f1f4b400700 1 -- 172.21.15.19:0/2960307384 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f4c10b870 msgr2=0x7f1f4c1a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:40.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:40.866+0000 7f1f4b400700 1 --2- 172.21.15.19:0/2960307384 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f4c10b870 0x7f1f4c1a1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:40.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:40.866+0000 7f1f4b400700 1 -- 172.21.15.19:0/2960307384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f4c10af20 msgr2=0x7f1f4c19d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:40.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:40.866+0000 7f1f4b400700 1 --2- 172.21.15.19:0/2960307384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f4c10af20 0x7f1f4c19d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:40.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:40.866+0000 7f1f4b400700 1 -- 172.21.15.19:0/2960307384 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f4c1a1d10 con 0x7f1f4c106640 2024-05-07T02:42:40.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:40.866+0000 7f1f43e00700 1 --2- 172.21.15.19:0/2960307384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f4c10af20 0x7f1f4c19d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:42:40.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:40.867+0000 7f1f4b400700 1 --2- 172.21.15.19:0/2960307384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f4c106640 0x7f1f4c19ce40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c00ef10 tx=0x7f1f3c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:42:40.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:40.867+0000 7f1f48c00700 1 -- 172.21.15.19:0/2960307384 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c00c9c0 con 0x7f1f4c106640 2024-05-07T02:42:40.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:40.867+0000 7f1f48c00700 1 -- 172.21.15.19:0/2960307384 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f3c019070 con 0x7f1f4c106640 2024-05-07T02:42:40.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:40.867+0000 7f1f50e00700 1 -- 172.21.15.19:0/2960307384 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f4c1a1fa0 con 0x7f1f4c106640 2024-05-07T02:42:40.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:40.867+0000 7f1f48c00700 1 -- 172.21.15.19:0/2960307384 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c014480 con 0x7f1f4c106640 2024-05-07T02:42:40.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:40.867+0000 7f1f50e00700 1 -- 172.21.15.19:0/2960307384 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f4c1a9920 con 0x7f1f4c106640 2024-05-07T02:42:40.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:40.868+0000 7f1f50e00700 1 -- 172.21.15.19:0/2960307384 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f4c0512c0 con 0x7f1f4c106640 2024-05-07T02:42:40.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:40.871+0000 7f1f48c00700 1 -- 172.21.15.19:0/2960307384 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1f3c012050 con 0x7f1f4c106640 2024-05-07T02:42:40.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:40.873+0000 7f1f48c00700 1 --2- 172.21.15.19:0/2960307384 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f1f300b2a70 0x7f1f300b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:40.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:40.873+0000 7f1f48c00700 1 -- 172.21.15.19:0/2960307384 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9525+0+0 (secure 0 0 0) 0x7f1f3c0e2f70 con 0x7f1f4c106640 2024-05-07T02:42:40.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:40.873+0000 7f1f43e00700 1 --2- 172.21.15.19:0/2960307384 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f1f300b2a70 0x7f1f300b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:40.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:40.874+0000 7f1f43e00700 1 --2- 172.21.15.19:0/2960307384 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f1f300b2a70 0x7f1f300b4f80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1f34000c00 tx=0x7f1f3401a040 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:42:40.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:40.876+0000 7f1f48c00700 1 -- 172.21.15.19:0/2960307384 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1f3c010040 con 0x7f1f4c106640 2024-05-07T02:42:41.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:41.167+0000 7f1f50e00700 1 -- 172.21.15.19:0/2960307384 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f4c02e5f0 con 0x7f1f4c106640 2024-05-07T02:42:41.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:41.168+0000 7f1f48c00700 1 -- 172.21.15.19:0/2960307384 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+11924 (secure 0 0 0) 0x7f1f3c0aa500 con 0x7f1f4c106640 2024-05-07T02:42:41.169 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:42:41.170 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1763670446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1763670446},{"type":"v1","addr":"172.21.15.19:6839","nonce":1763670446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/477529985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":477529985},{"type":"v1","addr":"172.21.15.64:6833","nonce":477529985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":50,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-07T02:26:14.960465+0000","modified":"2024-05-07T02:40:40.531575+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24539,"mds_2":24520,"mds_3":24515,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6833/2171873162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2171873162},{"type":"v1","addr":"172.21.15.27:6833","nonce":2171873162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/2379917839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2379917839},{"type":"v1","addr":"172.21.15.19:6835","nonce":2379917839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6837/2067515538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2067515538},{"type":"v1","addr":"172.21.15.19:6837","nonce":2067515538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6835/1000540600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1000540600},{"type":"v1","addr":"172.21.15.27:6835","nonce":1000540600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/1997318872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1997318872},{"type":"v1","addr":"172.21.15.64:6835","nonce":1997318872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":34,"addr":"172.21.15.64:6837/2585907777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2585907777},{"type":"v1","addr":"172.21.15.64:6837","nonce":2585907777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":33,"addr":"172.21.15.27:6837/2583138143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2583138143},{"type":"v1","addr":"172.21.15.27:6837","nonce":2583138143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":32,"addr":"172.21.15.64:6839/1583120004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":1583120004},{"type":"v1","addr":"172.21.15.64:6839","nonce":1583120004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":29,"addr":"172.21.15.19:6841/3905645246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3905645246},{"type":"v1","addr":"172.21.15.19:6841","nonce":3905645246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6839/3963268634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3963268634},{"type":"v1","addr":"172.21.15.27:6839","nonce":3963268634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24551,24539,24520,24505,24515]},"id":1}]} 2024-05-07T02:42:41.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:41.171+0000 7f1f50e00700 1 -- 172.21.15.19:0/2960307384 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f1f300b2a70 msgr2=0x7f1f300b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:41.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:41.171+0000 7f1f50e00700 1 --2- 172.21.15.19:0/2960307384 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f1f300b2a70 0x7f1f300b4f80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1f34000c00 tx=0x7f1f3401a040 comp rx=0 tx=0).stop 2024-05-07T02:42:41.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:41.171+0000 7f1f50e00700 1 -- 172.21.15.19:0/2960307384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f4c106640 msgr2=0x7f1f4c19ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:41.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:41.171+0000 7f1f50e00700 1 --2- 172.21.15.19:0/2960307384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f4c106640 0x7f1f4c19ce40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c00ef10 tx=0x7f1f3c00c3a0 comp rx=0 tx=0).stop 2024-05-07T02:42:41.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:41.172+0000 7f1f50e00700 1 -- 172.21.15.19:0/2960307384 shutdown_connections 2024-05-07T02:42:41.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:41.172+0000 7f1f50e00700 1 --2- 172.21.15.19:0/2960307384 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1f4c10b870 0x7f1f4c1a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:41.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:41.172+0000 7f1f50e00700 1 --2- 172.21.15.19:0/2960307384 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f1f300b2a70 0x7f1f300b4f80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:41.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:41.172+0000 7f1f50e00700 1 --2- 172.21.15.19:0/2960307384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f4c106640 0x7f1f4c19ce40 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:41.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:41.172+0000 7f1f50e00700 1 --2- 172.21.15.19:0/2960307384 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f4c10af20 0x7f1f4c19d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:41.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:41.172+0000 7f1f50e00700 1 -- 172.21.15.19:0/2960307384 >> 172.21.15.19:0/2960307384 conn(0x7f1f4c102030 msgr2=0x7f1f4c10a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:42:41.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:41.172+0000 7f1f50e00700 1 -- 172.21.15.19:0/2960307384 shutdown_connections 2024-05-07T02:42:41.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:41.172+0000 7f1f50e00700 1 -- 172.21.15.19:0/2960307384 wait complete. 2024-05-07T02:42:41.174 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 50 2024-05-07T02:42:42.089 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-05-07T02:42:42.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:41 smithi027 ceph-mon[34398]: pgmap v336: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:42:42.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:41 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/2960307384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:42:42.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:41 smithi064 ceph-mon[34699]: pgmap v336: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:42:42.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:41 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/2960307384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:42:42.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:41 smithi019 ceph-mon[29781]: pgmap v336: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:42:42.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:41 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/2960307384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:42:42.572 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:42:44.335 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:43 smithi019 ceph-mon[29781]: pgmap v337: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:42:44.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:43 smithi064 ceph-mon[34699]: pgmap v337: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:42:44.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:43 smithi027 ceph-mon[34398]: pgmap v337: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:42:44.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.962+0000 7f0d7f400700 1 -- 172.21.15.19:0/3899386425 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d78075d30 msgr2=0x7f0d78076140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:44.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.962+0000 7f0d7f400700 1 --2- 172.21.15.19:0/3899386425 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d78075d30 0x7f0d78076140 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0d68009f90 tx=0x7f0d6800b330 comp rx=0 tx=0).stop 2024-05-07T02:42:44.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.962+0000 7f0d7f400700 1 -- 172.21.15.19:0/3899386425 shutdown_connections 2024-05-07T02:42:44.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.962+0000 7f0d7f400700 1 --2- 172.21.15.19:0/3899386425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d78110b90 0x7f0d78112ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:44.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.962+0000 7f0d7f400700 1 --2- 172.21.15.19:0/3899386425 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d78075d30 0x7f0d78076140 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:44.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.962+0000 7f0d7f400700 1 --2- 172.21.15.19:0/3899386425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d78076680 0x7f0d78110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:44.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.962+0000 7f0d7f400700 1 -- 172.21.15.19:0/3899386425 >> 172.21.15.19:0/3899386425 conn(0x7f0d780ffe70 msgr2=0x7f0d781022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:42:44.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.962+0000 7f0d7f400700 1 -- 172.21.15.19:0/3899386425 shutdown_connections 2024-05-07T02:42:44.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.962+0000 7f0d7f400700 1 -- 172.21.15.19:0/3899386425 wait complete. 2024-05-07T02:42:44.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.963+0000 7f0d7f400700 1 Processor -- start 2024-05-07T02:42:44.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.964+0000 7f0d7f400700 1 -- start start 2024-05-07T02:42:44.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.964+0000 7f0d7f400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d78075d30 0x7f0d7819cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:44.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.964+0000 7f0d7f400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d78076680 0x7f0d7819d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:44.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.964+0000 7f0d7f400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d78110b90 0x7f0d781a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:44.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.964+0000 7f0d7f400700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0d78116680 con 0x7f0d78076680 2024-05-07T02:42:44.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.964+0000 7f0d7f400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0d78116500 con 0x7f0d78075d30 2024-05-07T02:42:44.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.964+0000 7f0d7f400700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0d78116800 con 0x7f0d78110b90 2024-05-07T02:42:44.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.965+0000 7f0d7ca00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d78075d30 0x7f0d7819cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:44.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.965+0000 7f0d6fe00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d78076680 0x7f0d7819d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:44.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.965+0000 7f0d7ca00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d78075d30 0x7f0d7819cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:42018/0 (socket says 172.21.15.19:42018) 2024-05-07T02:42:44.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.965+0000 7f0d7ca00700 1 -- 172.21.15.19:0/1483876712 learned_addr learned my addr 172.21.15.19:0/1483876712 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:42:44.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.965+0000 7f0d7d400700 1 --2- 172.21.15.19:0/1483876712 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d78110b90 0x7f0d781a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:44.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.965+0000 7f0d7ca00700 1 -- 172.21.15.19:0/1483876712 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d78110b90 msgr2=0x7f0d781a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:44.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.965+0000 7f0d7ca00700 1 --2- 172.21.15.19:0/1483876712 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d78110b90 0x7f0d781a1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:44.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.965+0000 7f0d7ca00700 1 -- 172.21.15.19:0/1483876712 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d78076680 msgr2=0x7f0d7819d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:44.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.965+0000 7f0d7ca00700 1 --2- 172.21.15.19:0/1483876712 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d78076680 0x7f0d7819d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:44.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.965+0000 7f0d7ca00700 1 -- 172.21.15.19:0/1483876712 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d781a1ea0 con 0x7f0d78075d30 2024-05-07T02:42:44.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.965+0000 7f0d6fe00700 1 --2- 172.21.15.19:0/1483876712 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d78076680 0x7f0d7819d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:42:44.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.966+0000 7f0d7ca00700 1 --2- 172.21.15.19:0/1483876712 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d78075d30 0x7f0d7819cfd0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0d68009f60 tx=0x7f0d6800ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:42:44.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.966+0000 7f0d76000700 1 -- 172.21.15.19:0/1483876712 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d68021070 con 0x7f0d78075d30 2024-05-07T02:42:44.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.966+0000 7f0d76000700 1 -- 172.21.15.19:0/1483876712 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d68007ac0 con 0x7f0d78075d30 2024-05-07T02:42:44.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.966+0000 7f0d7f400700 1 -- 172.21.15.19:0/1483876712 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d781a2130 con 0x7f0d78075d30 2024-05-07T02:42:44.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.966+0000 7f0d76000700 1 -- 172.21.15.19:0/1483876712 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d68005040 con 0x7f0d78075d30 2024-05-07T02:42:44.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.966+0000 7f0d7f400700 1 -- 172.21.15.19:0/1483876712 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d781a9ab0 con 0x7f0d78075d30 2024-05-07T02:42:44.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.970+0000 7f0d76000700 1 -- 172.21.15.19:0/1483876712 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0d68024070 con 0x7f0d78075d30 2024-05-07T02:42:44.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.971+0000 7f0d8def0b80 1 Processor -- start 2024-05-07T02:42:44.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.971+0000 7f0d8def0b80 1 -- start start 2024-05-07T02:42:44.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.972+0000 7f0d76000700 1 --2- 172.21.15.19:0/1483876712 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f0d5c0b29f0 0x7f0d5c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:44.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.972+0000 7f0d8def0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55793b64a470 0x55793b64a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:44.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.972+0000 7f0d8def0b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55793b644620 0x55793b644a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:44.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.972+0000 7f0d76000700 1 -- 172.21.15.19:0/1483876712 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9525+0+0 (secure 0 0 0) 0x7f0d6800f020 con 0x7f0d78075d30 2024-05-07T02:42:44.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.972+0000 7f0d8def0b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55793b644f70 0x55793b651150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:44.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.972+0000 7f0d8def0b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55793b566a20 con 0x55793b64a470 2024-05-07T02:42:44.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.972+0000 7f0d8def0b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55793b504760 con 0x55793b644620 2024-05-07T02:42:44.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.972+0000 7f0d8def0b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55793b5629e0 con 0x55793b644f70 2024-05-07T02:42:44.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.972+0000 7f0d7d400700 1 --2- 172.21.15.19:0/1483876712 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f0d5c0b29f0 0x7f0d5c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:44.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.972+0000 7f0d6fe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55793b644620 0x55793b644a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:44.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.972+0000 7f0d7ca00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55793b64a470 0x55793b64a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:44.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.972+0000 7f0d6fe00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55793b644620 0x55793b644a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:42028/0 (socket says 172.21.15.19:42028) 2024-05-07T02:42:44.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.972+0000 7f0d6fe00700 1 -- 172.21.15.19:0/1525786688 learned_addr learned my addr 172.21.15.19:0/1525786688 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:42:44.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.972+0000 7f0d7ca00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55793b64a470 0x55793b64a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53238/0 (socket says 172.21.15.19:53238) 2024-05-07T02:42:44.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.973+0000 7f0d7d400700 1 --2- 172.21.15.19:0/1525786688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55793b644f70 0x55793b651150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:44.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.973+0000 7f0d6fe00700 1 -- 172.21.15.19:0/1525786688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55793b644f70 msgr2=0x55793b651150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:44.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.973+0000 7f0d6fe00700 1 --2- 172.21.15.19:0/1525786688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55793b644f70 0x55793b651150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:44.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.973+0000 7f0d6fe00700 1 -- 172.21.15.19:0/1525786688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55793b64a470 msgr2=0x55793b64a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:44.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.973+0000 7f0d6fe00700 1 --2- 172.21.15.19:0/1525786688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55793b64a470 0x55793b64a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:44.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.973+0000 7f0d6fe00700 1 -- 172.21.15.19:0/1525786688 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55793b4b38d0 con 0x55793b644620 2024-05-07T02:42:44.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.973+0000 7f0d7d400700 1 --2- 172.21.15.19:0/1525786688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55793b644f70 0x55793b651150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:42:44.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.973+0000 7f0d6fe00700 1 --2- 172.21.15.19:0/1525786688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55793b644620 0x55793b644a30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0d600115f0 tx=0x7f0d60011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce2ab08e70b358a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:42:44.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.973+0000 7f0d7d400700 1 --2- 172.21.15.19:0/1483876712 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f0d5c0b29f0 0x7f0d5c0b4f00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0d78077bd0 tx=0x7f0d70009630 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:42:44.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.974+0000 7f0d6e000700 1 -- 172.21.15.19:0/1525786688 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6001e070 con 0x55793b644620 2024-05-07T02:42:44.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.974+0000 7f0d6e000700 1 -- 172.21.15.19:0/1525786688 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d60019ae0 con 0x55793b644620 2024-05-07T02:42:44.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.974+0000 7f0d6e000700 1 -- 172.21.15.19:0/1525786688 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6000eb90 con 0x55793b644620 2024-05-07T02:42:44.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.975+0000 7f0d8def0b80 1 -- 172.21.15.19:0/1525786688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55793b644620 msgr2=0x55793b644a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:44.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.975+0000 7f0d8def0b80 1 --2- 172.21.15.19:0/1525786688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55793b644620 0x55793b644a30 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0d600115f0 tx=0x7f0d60011900 comp rx=0 tx=0).stop 2024-05-07T02:42:44.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.975+0000 7f0d8def0b80 1 -- 172.21.15.19:0/1525786688 shutdown_connections 2024-05-07T02:42:44.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.975+0000 7f0d8def0b80 1 --2- 172.21.15.19:0/1525786688 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55793b644f70 0x55793b651150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:44.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.975+0000 7f0d8def0b80 1 --2- 172.21.15.19:0/1525786688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55793b64a470 0x55793b64a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:44.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.975+0000 7f0d8def0b80 1 --2- 172.21.15.19:0/1525786688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55793b644620 0x55793b644a30 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:44.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.975+0000 7f0d8def0b80 1 -- 172.21.15.19:0/1525786688 >> 172.21.15.19:0/1525786688 conn(0x55793b4c6e40 msgr2=0x55793b63f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:42:44.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.975+0000 7f0d8def0b80 1 -- 172.21.15.19:0/1525786688 shutdown_connections 2024-05-07T02:42:44.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.975+0000 7f0d8def0b80 1 -- 172.21.15.19:0/1525786688 wait complete. 2024-05-07T02:42:44.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.975+0000 7f0d8def0b80 1 -- start start 2024-05-07T02:42:44.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.976+0000 7f0d8def0b80 1 Processor -- start 2024-05-07T02:42:44.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.976+0000 7f0d8def0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55793b644620 0x55793b64e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:44.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.976+0000 7f0d8def0b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55793b644f70 0x55793b64eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:44.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.976+0000 7f0d8def0b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55793b64a470 0x55793b64f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:44.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.976+0000 7f0d8def0b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55793b651840 con 0x55793b644620 2024-05-07T02:42:44.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.976+0000 7f0d8def0b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x55793b4ccb20 con 0x55793b64a470 2024-05-07T02:42:44.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.976+0000 7f0d8def0b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55793b6519c0 con 0x55793b644f70 2024-05-07T02:42:44.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.976+0000 7f0d8def0b80 10 client.0 _ll_register_callbacks cb 0x55793b4fa380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:42:44.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.977+0000 7f0d6fe00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55793b644f70 0x55793b64eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:44.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.977+0000 7f0d7ca00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55793b644620 0x55793b64e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:44.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.977+0000 7f0d6fe00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55793b644f70 0x55793b64eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:55712/0 (socket says 172.21.15.19:55712) 2024-05-07T02:42:44.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.977+0000 7f0d6fe00700 1 -- 172.21.15.19:0/973885361 learned_addr learned my addr 172.21.15.19:0/973885361 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:42:44.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.977+0000 7f0d7d400700 1 --2- 172.21.15.19:0/973885361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55793b64a470 0x55793b64f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:44.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.977+0000 7f0d6fe00700 1 -- 172.21.15.19:0/973885361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55793b64a470 msgr2=0x55793b64f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:44.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.977+0000 7f0d6fe00700 1 --2- 172.21.15.19:0/973885361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55793b64a470 0x55793b64f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:44.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.977+0000 7f0d6fe00700 1 -- 172.21.15.19:0/973885361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55793b644620 msgr2=0x55793b64e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:44.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.977+0000 7f0d6fe00700 1 --2- 172.21.15.19:0/973885361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55793b644620 0x55793b64e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:44.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.977+0000 7f0d6fe00700 1 -- 172.21.15.19:0/973885361 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55793b52c420 con 0x55793b644f70 2024-05-07T02:42:44.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.977+0000 7f0d6fe00700 1 --2- 172.21.15.19:0/973885361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55793b644f70 0x55793b64eb60 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0d6001c040 tx=0x7f0d6000e700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e50ef3aadd2c5fe server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:42:44.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.978+0000 7f0d66a00700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:42:44.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.978+0000 7f0d66a00700 1 -- 172.21.15.19:0/973885361 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d60027020 con 0x55793b644f70 2024-05-07T02:42:44.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.978+0000 7f0d8def0b80 1 -- 172.21.15.19:0/973885361 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d60019ae0 con 0x55793b644f70 2024-05-07T02:42:44.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.978+0000 7f0d66a00700 1 -- 172.21.15.19:0/973885361 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9525+0+0 (secure 0 0 0) 0x7f0d60004040 con 0x55793b644f70 2024-05-07T02:42:44.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.978+0000 7f0d66a00700 1 -- 172.21.15.19:0/973885361 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f0d60032050 con 0x55793b644f70 2024-05-07T02:42:44.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.978+0000 7f0d8def0b80 10 client.26764 fetch_fsmap learned FSMap version 50 2024-05-07T02:42:44.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.979+0000 7f0d8def0b80 1 -- 172.21.15.19:0/973885361 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55793b65c440 con 0x55793b644f70 2024-05-07T02:42:44.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.979+0000 7f0d66a00700 1 -- 172.21.15.19:0/973885361 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d60019ae0 con 0x55793b644f70 2024-05-07T02:42:44.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.979+0000 7f0d66a00700 1 -- 172.21.15.19:0/973885361 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 50) ==== 8362+0+0 (secure 0 0 0) 0x7f0d6000e910 con 0x55793b644f70 2024-05-07T02:42:44.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.979+0000 7f0d8def0b80 10 client.26764 fetch_fsmap finished waiting for FSMap version 50 2024-05-07T02:42:44.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.979+0000 7f0d8def0b80 10 client.26764 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-07T02:42:44.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.979+0000 7f0d8def0b80 20 client.26764 populate_metadata read hostname 'smithi019' 2024-05-07T02:42:44.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.979+0000 7f0d8def0b80 1 --2- 172.21.15.19:0/973885361 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x55793b65e910 0x55793b660d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:44.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.979+0000 7f0d8def0b80 4 client.26764 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:42:44.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.979+0000 7f0d8def0b80 1 -- 172.21.15.19:0/973885361 --> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55793b6110e0 con 0x55793b65e910 2024-05-07T02:42:44.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.980+0000 7f0d7ca00700 1 --2- 172.21.15.19:0/973885361 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x55793b65e910 0x55793b660d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:44.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.980+0000 7f0d7ca00700 1 --2- 172.21.15.19:0/973885361 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x55793b65e910 0x55793b660d30 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f0fb330e33c3044 server_cookie=e8af47c9f46adf34 in_seq=0 out_seq=0 2024-05-07T02:42:44.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.980+0000 7f0d66a00700 10 client.26764 ms_handle_connect on v2:172.21.15.64:6836/2585907777 2024-05-07T02:42:44.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.981+0000 7f0d66a00700 1 -- 172.21.15.19:0/973885361 <== mds.4 v2:172.21.15.64:6836/2585907777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55793b6110e0 con 0x55793b65e910 2024-05-07T02:42:44.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.981+0000 7f0d66a00700 10 client.26764 handle_command_reply: tid=0 2024-05-07T02:42:44.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.981+0000 7f0d8def0b80 1 client.26764 shutdown 2024-05-07T02:42:44.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.982+0000 7f0d8def0b80 1 -- 172.21.15.19:0/973885361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55793b644f70 msgr2=0x55793b64eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:44.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.982+0000 7f0d8def0b80 1 --2- 172.21.15.19:0/973885361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55793b644f70 0x55793b64eb60 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0d6001c040 tx=0x7f0d6000e700 comp rx=0 tx=0).stop 2024-05-07T02:42:44.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.982+0000 7f0d8def0b80 1 -- 172.21.15.19:0/973885361 shutdown_connections 2024-05-07T02:42:44.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.982+0000 7f0d8def0b80 1 --2- 172.21.15.19:0/973885361 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x55793b65e910 0x55793b660d30 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:44.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.982+0000 7f0d8def0b80 1 --2- 172.21.15.19:0/973885361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55793b64a470 0x55793b64f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:44.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.982+0000 7f0d8def0b80 1 --2- 172.21.15.19:0/973885361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55793b644620 0x55793b64e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:44.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.982+0000 7f0d8def0b80 1 --2- 172.21.15.19:0/973885361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55793b644f70 0x55793b64eb60 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:44.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.982+0000 7f0d66a00700 0 client.26764 ms_handle_reset on v2:172.21.15.64:6836/2585907777 2024-05-07T02:42:44.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.982+0000 7f0d8def0b80 1 -- 172.21.15.19:0/973885361 >> 172.21.15.19:0/973885361 conn(0x55793b4c6e40 msgr2=0x55793b643610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:42:44.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.982+0000 7f0d8def0b80 1 -- 172.21.15.19:0/973885361 shutdown_connections 2024-05-07T02:42:44.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.982+0000 7f0d8def0b80 1 -- 172.21.15.19:0/973885361 wait complete. 2024-05-07T02:42:44.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.982+0000 7f0d8def0b80 20 client.26764 trim_cache size 0 max 16384 2024-05-07T02:42:44.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.999+0000 7f0d8def0b80 1 Processor -- start 2024-05-07T02:42:45.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:44.999+0000 7f0d8def0b80 1 -- start start 2024-05-07T02:42:45.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.000+0000 7f0d8def0b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55793b644620 0x55793b639bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:45.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.000+0000 7f0d8def0b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55793b644f70 0x55793b64ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:45.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.000+0000 7f0d8def0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55793b64a470 0x55793b64bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:45.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.000+0000 7f0d8def0b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0d781b2890 con 0x55793b64a470 2024-05-07T02:42:45.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.000+0000 7f0d8def0b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0d781b43d0 con 0x55793b644620 2024-05-07T02:42:45.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.000+0000 7f0d8def0b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0d781b2f60 con 0x55793b644f70 2024-05-07T02:42:45.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.000+0000 7f0d6fe00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55793b644f70 0x55793b64ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:45.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.000+0000 7f0d6fe00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55793b644f70 0x55793b64ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:55726/0 (socket says 172.21.15.19:55726) 2024-05-07T02:42:45.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.000+0000 7f0d6fe00700 1 -- 172.21.15.19:0/861048044 learned_addr learned my addr 172.21.15.19:0/861048044 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:42:45.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.000+0000 7f0d7ca00700 1 --2- 172.21.15.19:0/861048044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55793b644620 0x55793b639bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:45.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.000+0000 7f0d7d400700 1 --2- 172.21.15.19:0/861048044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55793b64a470 0x55793b64bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:45.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.001+0000 7f0d6fe00700 1 -- 172.21.15.19:0/861048044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55793b644620 msgr2=0x55793b639bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:45.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.001+0000 7f0d6fe00700 1 --2- 172.21.15.19:0/861048044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55793b644620 0x55793b639bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:45.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.001+0000 7f0d6fe00700 1 -- 172.21.15.19:0/861048044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55793b64a470 msgr2=0x55793b64bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:45.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.001+0000 7f0d6fe00700 1 --2- 172.21.15.19:0/861048044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55793b64a470 0x55793b64bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:45.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.001+0000 7f0d6fe00700 1 -- 172.21.15.19:0/861048044 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55793b63bb80 con 0x55793b644f70 2024-05-07T02:42:45.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.001+0000 7f0d7d400700 1 --2- 172.21.15.19:0/861048044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55793b64a470 0x55793b64bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-07T02:42:45.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.001+0000 7f0d6fe00700 1 --2- 172.21.15.19:0/861048044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55793b644f70 0x55793b64ddf0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0d60014040 tx=0x7f0d60011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a18bc64b3908dce server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:42:45.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.001+0000 7f0d6d600700 1 -- 172.21.15.19:0/861048044 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6003b070 con 0x55793b644f70 2024-05-07T02:42:45.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.001+0000 7f0d6d600700 1 -- 172.21.15.19:0/861048044 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d6001f040 con 0x55793b644f70 2024-05-07T02:42:45.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.001+0000 7f0d6d600700 1 -- 172.21.15.19:0/861048044 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d60027070 con 0x55793b644f70 2024-05-07T02:42:45.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.002+0000 7f0d8def0b80 1 -- 172.21.15.19:0/861048044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55793b644f70 msgr2=0x55793b64ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:45.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.002+0000 7f0d8def0b80 1 --2- 172.21.15.19:0/861048044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55793b644f70 0x55793b64ddf0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0d60014040 tx=0x7f0d60011900 comp rx=0 tx=0).stop 2024-05-07T02:42:45.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.002+0000 7f0d8def0b80 1 -- 172.21.15.19:0/861048044 shutdown_connections 2024-05-07T02:42:45.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.002+0000 7f0d8def0b80 1 --2- 172.21.15.19:0/861048044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55793b64a470 0x55793b64bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:45.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.002+0000 7f0d8def0b80 1 --2- 172.21.15.19:0/861048044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55793b644620 0x55793b639bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:45.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.002+0000 7f0d8def0b80 1 --2- 172.21.15.19:0/861048044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55793b644f70 0x55793b64ddf0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:45.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.002+0000 7f0d8def0b80 1 -- 172.21.15.19:0/861048044 >> 172.21.15.19:0/861048044 conn(0x55793b4c6e40 msgr2=0x55793b643610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:42:45.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.002+0000 7f0d8def0b80 1 -- 172.21.15.19:0/861048044 shutdown_connections 2024-05-07T02:42:45.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.003+0000 7f0d8def0b80 1 -- 172.21.15.19:0/861048044 wait complete. 2024-05-07T02:42:45.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.003+0000 7f0d8def0b80 1 -- start start 2024-05-07T02:42:45.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.003+0000 7f0d8def0b80 1 Processor -- start 2024-05-07T02:42:45.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.004+0000 7f0d8def0b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55793b644620 0x55793b657260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:45.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.004+0000 7f0d8def0b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55793b644f70 0x55793b665910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:45.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.004+0000 7f0d8def0b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55793b64a470 0x55793b665e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:45.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.004+0000 7f0d8def0b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0d781b3d00 con 0x55793b64a470 2024-05-07T02:42:45.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.004+0000 7f0d8def0b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0d781b3630 con 0x55793b644f70 2024-05-07T02:42:45.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.004+0000 7f0d8def0b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0d781b0850 con 0x55793b644620 2024-05-07T02:42:45.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.004+0000 7f0d8def0b80 10 client.0 _ll_register_callbacks cb 0x55793b4fa380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:42:45.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.004+0000 7f0d7ca00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55793b644620 0x55793b657260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:45.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.004+0000 7f0d7d400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55793b64a470 0x55793b665e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:45.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.004+0000 7f0d7ca00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55793b644620 0x55793b657260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:55732/0 (socket says 172.21.15.19:55732) 2024-05-07T02:42:45.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.004+0000 7f0d7ca00700 1 -- 172.21.15.19:0/758833881 learned_addr learned my addr 172.21.15.19:0/758833881 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:42:45.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.005+0000 7f0d6fe00700 1 --2- 172.21.15.19:0/758833881 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55793b644f70 0x55793b665910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:45.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.005+0000 7f0d7ca00700 1 -- 172.21.15.19:0/758833881 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55793b644f70 msgr2=0x55793b665910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:45.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.005+0000 7f0d7ca00700 1 --2- 172.21.15.19:0/758833881 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55793b644f70 0x55793b665910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:45.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.005+0000 7f0d7ca00700 1 -- 172.21.15.19:0/758833881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55793b64a470 msgr2=0x55793b665e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:45.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.005+0000 7f0d7ca00700 1 --2- 172.21.15.19:0/758833881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55793b64a470 0x55793b665e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:45.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.005+0000 7f0d7ca00700 1 -- 172.21.15.19:0/758833881 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55793b666620 con 0x55793b644620 2024-05-07T02:42:45.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.005+0000 7f0d7ca00700 1 --2- 172.21.15.19:0/758833881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55793b644620 0x55793b657260 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0d680e1210 tx=0x7f0d6800ce50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a93f749ee507cdc2 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:42:45.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.005+0000 7f0d66a00700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:42:45.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.005+0000 7f0d66a00700 1 -- 172.21.15.19:0/758833881 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d68070060 con 0x55793b644620 2024-05-07T02:42:45.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.006+0000 7f0d66a00700 1 -- 172.21.15.19:0/758833881 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9525+0+0 (secure 0 0 0) 0x7f0d680a7060 con 0x55793b644620 2024-05-07T02:42:45.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.006+0000 7f0d8def0b80 1 -- 172.21.15.19:0/758833881 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d6001f040 con 0x55793b644620 2024-05-07T02:42:45.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.006+0000 7f0d66a00700 1 -- 172.21.15.19:0/758833881 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f0d680ae040 con 0x55793b644620 2024-05-07T02:42:45.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.006+0000 7f0d8def0b80 10 client.26770 fetch_fsmap learned FSMap version 50 2024-05-07T02:42:45.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.006+0000 7f0d8def0b80 1 -- 172.21.15.19:0/758833881 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55793b6d1650 con 0x55793b644620 2024-05-07T02:42:45.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.006+0000 7f0d66a00700 1 -- 172.21.15.19:0/758833881 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d6001f040 con 0x55793b644620 2024-05-07T02:42:45.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.006+0000 7f0d66a00700 1 -- 172.21.15.19:0/758833881 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 50) ==== 8362+0+0 (secure 0 0 0) 0x7f0d68092040 con 0x55793b644620 2024-05-07T02:42:45.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.006+0000 7f0d8def0b80 10 client.26770 fetch_fsmap finished waiting for FSMap version 50 2024-05-07T02:42:45.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.006+0000 7f0d8def0b80 10 client.26770 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-05-07T02:42:45.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.006+0000 7f0d8def0b80 20 client.26770 populate_metadata read hostname 'smithi019' 2024-05-07T02:42:45.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.007+0000 7f0d8def0b80 1 --2- 172.21.15.19:0/758833881 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x55793b6d1c90 0x55793b6d40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:45.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.007+0000 7f0d8def0b80 4 client.26770 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:42:45.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.007+0000 7f0d8def0b80 1 -- 172.21.15.19:0/758833881 --> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55793b574400 con 0x55793b6d1c90 2024-05-07T02:42:45.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.007+0000 7f0d6fe00700 1 --2- 172.21.15.19:0/758833881 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x55793b6d1c90 0x55793b6d40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:45.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.007+0000 7f0d6fe00700 1 --2- 172.21.15.19:0/758833881 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x55793b6d1c90 0x55793b6d40b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6df4d8639383304b server_cookie=95f9238a57368aea in_seq=0 out_seq=0 2024-05-07T02:42:45.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.007+0000 7f0d66a00700 10 client.26770 ms_handle_connect on v2:172.21.15.64:6836/2585907777 2024-05-07T02:42:45.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.008+0000 7f0d66a00700 1 -- 172.21.15.19:0/758833881 <== mds.4 v2:172.21.15.64:6836/2585907777 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55793b574400 con 0x55793b6d1c90 2024-05-07T02:42:45.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.008+0000 7f0d66a00700 10 client.26770 handle_command_reply: tid=0 2024-05-07T02:42:45.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.008+0000 7f0d8def0b80 1 client.26770 shutdown 2024-05-07T02:42:45.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.008+0000 7f0d8def0b80 1 -- 172.21.15.19:0/758833881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55793b644620 msgr2=0x55793b657260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:45.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.008+0000 7f0d8def0b80 1 --2- 172.21.15.19:0/758833881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55793b644620 0x55793b657260 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0d680e1210 tx=0x7f0d6800ce50 comp rx=0 tx=0).stop 2024-05-07T02:42:45.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.008+0000 7f0d8def0b80 1 -- 172.21.15.19:0/758833881 shutdown_connections 2024-05-07T02:42:45.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.008+0000 7f0d8def0b80 1 --2- 172.21.15.19:0/758833881 >> [v2:172.21.15.64:6836/2585907777,v1:172.21.15.64:6837/2585907777] conn(0x55793b6d1c90 0x55793b6d40b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:45.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.008+0000 7f0d8def0b80 1 --2- 172.21.15.19:0/758833881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55793b64a470 0x55793b665e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:45.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.008+0000 7f0d8def0b80 1 --2- 172.21.15.19:0/758833881 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55793b644620 0x55793b657260 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:45.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.008+0000 7f0d8def0b80 1 --2- 172.21.15.19:0/758833881 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55793b644f70 0x55793b665910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:45.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.008+0000 7f0d8def0b80 1 -- 172.21.15.19:0/758833881 >> 172.21.15.19:0/758833881 conn(0x55793b4c6e40 msgr2=0x55793b639950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:42:45.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.008+0000 7f0d66a00700 0 client.26770 ms_handle_reset on v2:172.21.15.64:6836/2585907777 2024-05-07T02:42:45.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.008+0000 7f0d8def0b80 1 -- 172.21.15.19:0/758833881 shutdown_connections 2024-05-07T02:42:45.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.008+0000 7f0d8def0b80 1 -- 172.21.15.19:0/758833881 wait complete. 2024-05-07T02:42:45.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.008+0000 7f0d8def0b80 20 client.26770 trim_cache size 0 max 16384 2024-05-07T02:42:45.015 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:42:45.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.009+0000 7f0d7f400700 1 -- 172.21.15.19:0/1483876712 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f0d5c0b29f0 msgr2=0x7f0d5c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:45.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.009+0000 7f0d7f400700 1 --2- 172.21.15.19:0/1483876712 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f0d5c0b29f0 0x7f0d5c0b4f00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0d78077bd0 tx=0x7f0d70009630 comp rx=0 tx=0).stop 2024-05-07T02:42:45.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.009+0000 7f0d7f400700 1 -- 172.21.15.19:0/1483876712 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d78075d30 msgr2=0x7f0d7819cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:45.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.009+0000 7f0d7f400700 1 --2- 172.21.15.19:0/1483876712 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d78075d30 0x7f0d7819cfd0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0d68009f60 tx=0x7f0d6800ccb0 comp rx=0 tx=0).stop 2024-05-07T02:42:45.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.009+0000 7f0d7f400700 1 -- 172.21.15.19:0/1483876712 shutdown_connections 2024-05-07T02:42:45.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.009+0000 7f0d7f400700 1 --2- 172.21.15.19:0/1483876712 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d78110b90 0x7f0d781a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:45.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.009+0000 7f0d7f400700 1 --2- 172.21.15.19:0/1483876712 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d78075d30 0x7f0d7819cfd0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:45.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.009+0000 7f0d7f400700 1 --2- 172.21.15.19:0/1483876712 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f0d5c0b29f0 0x7f0d5c0b4f00 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:45.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.009+0000 7f0d7f400700 1 --2- 172.21.15.19:0/1483876712 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d78076680 0x7f0d7819d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:45.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.009+0000 7f0d7f400700 1 -- 172.21.15.19:0/1483876712 >> 172.21.15.19:0/1483876712 conn(0x7f0d780ffe70 msgr2=0x7f0d78100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:42:45.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.009+0000 7f0d7f400700 1 -- 172.21.15.19:0/1483876712 shutdown_connections 2024-05-07T02:42:45.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:45.010+0000 7f0d7f400700 1 -- 172.21.15.19:0/1483876712 wait complete. 2024-05-07T02:42:45.910 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-05-07T02:42:46.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:45 smithi019 ceph-mon[29781]: pgmap v338: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-07T02:42:46.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:45 smithi064 ceph-mon[34699]: pgmap v338: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-07T02:42:46.382 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:42:46.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:45 smithi027 ceph-mon[34398]: pgmap v338: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-07T02:42:48.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:47 smithi019 ceph-mon[29781]: pgmap v339: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:42:48.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:47 smithi064 ceph-mon[34699]: pgmap v339: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:42:48.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:47 smithi027 ceph-mon[34398]: pgmap v339: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:42:48.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.821+0000 7fa817200700 1 -- 172.21.15.19:0/1773548473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa818109680 msgr2=0x7fa818110fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:48.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.821+0000 7fa817200700 1 --2- 172.21.15.19:0/1773548473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa818109680 0x7fa818110fe0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa81000b0d0 tx=0x7fa81000b3e0 comp rx=0 tx=0).stop 2024-05-07T02:42:48.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.821+0000 7fa817200700 1 -- 172.21.15.19:0/1773548473 shutdown_connections 2024-05-07T02:42:48.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.821+0000 7fa817200700 1 --2- 172.21.15.19:0/1773548473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa818109680 0x7fa818110fe0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:48.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.822+0000 7fa817200700 1 --2- 172.21.15.19:0/1773548473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa818104400 0x7fa818104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:48.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.822+0000 7fa817200700 1 --2- 172.21.15.19:0/1773548473 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa818108d30 0x7fa818109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:48.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.822+0000 7fa817200700 1 -- 172.21.15.19:0/1773548473 >> 172.21.15.19:0/1773548473 conn(0x7fa818079d10 msgr2=0x7fa81807a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:42:48.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.822+0000 7fa817200700 1 -- 172.21.15.19:0/1773548473 shutdown_connections 2024-05-07T02:42:48.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.822+0000 7fa817200700 1 -- 172.21.15.19:0/1773548473 wait complete. 2024-05-07T02:42:48.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.823+0000 7fa817200700 1 Processor -- start 2024-05-07T02:42:48.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.823+0000 7fa817200700 1 -- start start 2024-05-07T02:42:48.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.824+0000 7fa817200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa818104400 0x7fa81819ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:48.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.824+0000 7fa817200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa818108d30 0x7fa81819d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:48.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.824+0000 7fa817200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa818109680 0x7fa8181a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:48.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.824+0000 7fa817200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa818114300 con 0x7fa818104400 2024-05-07T02:42:48.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.824+0000 7fa817200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa818114180 con 0x7fa818108d30 2024-05-07T02:42:48.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.824+0000 7fa817200700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa818114480 con 0x7fa818109680 2024-05-07T02:42:48.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.824+0000 7fa816800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa818109680 0x7fa8181a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:48.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.824+0000 7fa80d400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa818108d30 0x7fa81819d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:48.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.824+0000 7fa815e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa818104400 0x7fa81819ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:48.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.824+0000 7fa816800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa818109680 0x7fa8181a1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:55740/0 (socket says 172.21.15.19:55740) 2024-05-07T02:42:48.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.824+0000 7fa80d400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa818108d30 0x7fa81819d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:42056/0 (socket says 172.21.15.19:42056) 2024-05-07T02:42:48.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.824+0000 7fa816800700 1 -- 172.21.15.19:0/3942870734 learned_addr learned my addr 172.21.15.19:0/3942870734 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:42:48.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.825+0000 7fa815e00700 1 -- 172.21.15.19:0/3942870734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa818109680 msgr2=0x7fa8181a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:48.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.825+0000 7fa815e00700 1 --2- 172.21.15.19:0/3942870734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa818109680 0x7fa8181a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:48.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.825+0000 7fa815e00700 1 -- 172.21.15.19:0/3942870734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa818108d30 msgr2=0x7fa81819d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:48.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.825+0000 7fa815e00700 1 --2- 172.21.15.19:0/3942870734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa818108d30 0x7fa81819d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:48.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.825+0000 7fa815e00700 1 -- 172.21.15.19:0/3942870734 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8181a1d20 con 0x7fa818104400 2024-05-07T02:42:48.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.825+0000 7fa80d400700 1 --2- 172.21.15.19:0/3942870734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa818108d30 0x7fa81819d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:42:48.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.825+0000 7fa815e00700 1 --2- 172.21.15.19:0/3942870734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa818104400 0x7fa81819ce50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa80800e9f0 tx=0x7fa80800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:42:48.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.826+0000 7fa80f400700 1 -- 172.21.15.19:0/3942870734 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80800c9c0 con 0x7fa818104400 2024-05-07T02:42:48.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.826+0000 7fa80f400700 1 -- 172.21.15.19:0/3942870734 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa80800cb60 con 0x7fa818104400 2024-05-07T02:42:48.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.826+0000 7fa80f400700 1 -- 172.21.15.19:0/3942870734 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa808018680 con 0x7fa818104400 2024-05-07T02:42:48.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.826+0000 7fa817200700 1 -- 172.21.15.19:0/3942870734 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8181a2010 con 0x7fa818104400 2024-05-07T02:42:48.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.826+0000 7fa817200700 1 -- 172.21.15.19:0/3942870734 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8181a9930 con 0x7fa818104400 2024-05-07T02:42:48.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.831+0000 7fa80f400700 1 -- 172.21.15.19:0/3942870734 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa808010430 con 0x7fa818104400 2024-05-07T02:42:48.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.832+0000 7fa82b528b80 1 Processor -- start 2024-05-07T02:42:48.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.832+0000 7fa82b528b80 1 -- start start 2024-05-07T02:42:48.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.833+0000 7fa82b528b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568769a0a90 0x5568769a0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:48.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.833+0000 7fa82b528b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568769a13e0 0x5568769aa620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:48.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.833+0000 7fa80f400700 1 --2- 172.21.15.19:0/3942870734 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fa7f80b2a70 0x7fa7f80b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:48.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.833+0000 7fa82b528b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5568769aab60 0x5568769acf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:48.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.833+0000 7fa82b528b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5568768c2a20 con 0x5568769a13e0 2024-05-07T02:42:48.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.833+0000 7fa82b528b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556876860760 con 0x5568769aab60 2024-05-07T02:42:48.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.833+0000 7fa82b528b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5568768be9e0 con 0x5568769a0a90 2024-05-07T02:42:48.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.833+0000 7fa80f400700 1 -- 172.21.15.19:0/3942870734 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9525+0+0 (secure 0 0 0) 0x7fa8080e2b50 con 0x7fa818104400 2024-05-07T02:42:48.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.833+0000 7fa815e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568769a13e0 0x5568769aa620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:48.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.833+0000 7fa816800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568769a0a90 0x5568769a0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:48.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.833+0000 7fa815e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568769a13e0 0x5568769aa620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49256/0 (socket says 172.21.15.19:49256) 2024-05-07T02:42:48.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.833+0000 7fa815e00700 1 -- 172.21.15.19:0/3181814878 learned_addr learned my addr 172.21.15.19:0/3181814878 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:42:48.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.833+0000 7fa816800700 1 --2- 172.21.15.19:0/3181814878 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5568769aab60 0x5568769acf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:48.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.833+0000 7fa80d400700 1 --2- 172.21.15.19:0/3942870734 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fa7f80b2a70 0x7fa7f80b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:48.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.834+0000 7fa815e00700 1 -- 172.21.15.19:0/3181814878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568769a0a90 msgr2=0x5568769a0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:48.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.834+0000 7fa815e00700 1 --2- 172.21.15.19:0/3181814878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568769a0a90 0x5568769a0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:48.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.834+0000 7fa815e00700 1 -- 172.21.15.19:0/3181814878 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5568769aab60 msgr2=0x5568769acf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:48.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.834+0000 7fa815e00700 1 --2- 172.21.15.19:0/3181814878 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5568769aab60 0x5568769acf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:48.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.834+0000 7fa815e00700 1 -- 172.21.15.19:0/3181814878 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55687680f8d0 con 0x5568769a13e0 2024-05-07T02:42:48.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.834+0000 7fa816800700 1 --2- 172.21.15.19:0/3181814878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568769a0a90 0x5568769a0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:42:48.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.834+0000 7fa815e00700 1 --2- 172.21.15.19:0/3181814878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568769a13e0 0x5568769aa620 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa8080dfeb0 tx=0x7fa8080b22d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=859a84e1b0ac2245 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:42:48.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.835+0000 7fa80d400700 1 --2- 172.21.15.19:0/3942870734 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fa7f80b2a70 0x7fa7f80b4f80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa818110030 tx=0x7fa800009260 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:42:48.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.835+0000 7fa807400700 1 -- 172.21.15.19:0/3181814878 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8080a99e0 con 0x5568769a13e0 2024-05-07T02:42:48.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.835+0000 7fa807400700 1 -- 172.21.15.19:0/3181814878 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8080add90 con 0x5568769a13e0 2024-05-07T02:42:48.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.835+0000 7fa807400700 1 -- 172.21.15.19:0/3181814878 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8080a9ea0 con 0x5568769a13e0 2024-05-07T02:42:48.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.836+0000 7fa82b528b80 1 -- 172.21.15.19:0/3181814878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568769a13e0 msgr2=0x5568769aa620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:48.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.836+0000 7fa82b528b80 1 --2- 172.21.15.19:0/3181814878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568769a13e0 0x5568769aa620 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa8080dfeb0 tx=0x7fa8080b22d0 comp rx=0 tx=0).stop 2024-05-07T02:42:48.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.836+0000 7fa82b528b80 1 -- 172.21.15.19:0/3181814878 shutdown_connections 2024-05-07T02:42:48.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.836+0000 7fa82b528b80 1 --2- 172.21.15.19:0/3181814878 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5568769aab60 0x5568769acf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:48.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.836+0000 7fa82b528b80 1 --2- 172.21.15.19:0/3181814878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568769a0a90 0x5568769a0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:48.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.836+0000 7fa82b528b80 1 --2- 172.21.15.19:0/3181814878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568769a13e0 0x5568769aa620 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:48.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.836+0000 7fa82b528b80 1 -- 172.21.15.19:0/3181814878 >> 172.21.15.19:0/3181814878 conn(0x556876822e40 msgr2=0x55687699b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:42:48.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.836+0000 7fa82b528b80 1 -- 172.21.15.19:0/3181814878 shutdown_connections 2024-05-07T02:42:48.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.836+0000 7fa82b528b80 1 -- 172.21.15.19:0/3181814878 wait complete. 2024-05-07T02:42:48.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.837+0000 7fa82b528b80 1 -- start start 2024-05-07T02:42:48.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.837+0000 7fa82b528b80 1 Processor -- start 2024-05-07T02:42:48.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.838+0000 7fa82b528b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5568769a0a90 0x5568769a7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:48.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.838+0000 7fa82b528b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568769a13e0 0x5568769a8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:48.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.838+0000 7fa82b528b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568769aab60 0x5568769a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:48.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.838+0000 7fa82b528b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5568769ad900 con 0x5568769a13e0 2024-05-07T02:42:48.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.838+0000 7fa82b528b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556876828b20 con 0x5568769a0a90 2024-05-07T02:42:48.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.838+0000 7fa82b528b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5568769ada80 con 0x5568769aab60 2024-05-07T02:42:48.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.838+0000 7fa82b528b80 10 client.0 _ll_register_callbacks cb 0x556876869c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:42:48.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.838+0000 7fa815e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5568769a0a90 0x5568769a7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:48.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.838+0000 7fa816800700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568769aab60 0x5568769a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:48.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.838+0000 7fa80d400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568769a13e0 0x5568769a8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:48.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.838+0000 7fa815e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5568769a0a90 0x5568769a7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:50628/0 (socket says 172.21.15.19:50628) 2024-05-07T02:42:48.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.838+0000 7fa815e00700 1 -- 172.21.15.19:0/3059509361 learned_addr learned my addr 172.21.15.19:0/3059509361 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:42:48.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.839+0000 7fa815e00700 1 -- 172.21.15.19:0/3059509361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568769aab60 msgr2=0x5568769a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:48.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.839+0000 7fa815e00700 1 --2- 172.21.15.19:0/3059509361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568769aab60 0x5568769a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:48.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.839+0000 7fa815e00700 1 -- 172.21.15.19:0/3059509361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568769a13e0 msgr2=0x5568769a8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:48.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.839+0000 7fa815e00700 1 --2- 172.21.15.19:0/3059509361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568769a13e0 0x5568769a8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:48.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.839+0000 7fa815e00700 1 -- 172.21.15.19:0/3059509361 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556876888420 con 0x5568769a0a90 2024-05-07T02:42:48.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.839+0000 7fa815e00700 1 --2- 172.21.15.19:0/3059509361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5568769a0a90 0x5568769a7ac0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa8080a9970 tx=0x7fa8080a9650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c81c990e7ab3edf server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:42:48.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.839+0000 7fa7ffe00700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:42:48.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.839+0000 7fa7ffe00700 1 -- 172.21.15.19:0/3059509361 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8080c2060 con 0x5568769a0a90 2024-05-07T02:42:48.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.839+0000 7fa82b528b80 1 -- 172.21.15.19:0/3059509361 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8080add90 con 0x5568769a0a90 2024-05-07T02:42:48.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.839+0000 7fa7ffe00700 1 -- 172.21.15.19:0/3059509361 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9525+0+0 (secure 0 0 0) 0x7fa8080b23b0 con 0x5568769a0a90 2024-05-07T02:42:48.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.839+0000 7fa7ffe00700 1 -- 172.21.15.19:0/3059509361 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fa8080b2840 con 0x5568769a0a90 2024-05-07T02:42:48.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.839+0000 7fa82b528b80 10 client.26822 fetch_fsmap learned FSMap version 50 2024-05-07T02:42:48.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.839+0000 7fa82b528b80 1 -- 172.21.15.19:0/3059509361 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5568769b84d0 con 0x5568769a0a90 2024-05-07T02:42:48.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.840+0000 7fa7ffe00700 1 -- 172.21.15.19:0/3059509361 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8080add90 con 0x5568769a0a90 2024-05-07T02:42:48.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.840+0000 7fa7ffe00700 1 -- 172.21.15.19:0/3059509361 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 50) ==== 8362+0+0 (secure 0 0 0) 0x7fa8080a5470 con 0x5568769a0a90 2024-05-07T02:42:48.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.840+0000 7fa82b528b80 10 client.26822 fetch_fsmap finished waiting for FSMap version 50 2024-05-07T02:42:48.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.840+0000 7fa82b528b80 10 client.26822 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-05-07T02:42:48.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.840+0000 7fa82b528b80 20 client.26822 populate_metadata read hostname 'smithi019' 2024-05-07T02:42:48.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.840+0000 7fa82b528b80 1 --2- 172.21.15.19:0/3059509361 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x5568769ba9a0 0x5568769bcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:48.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.840+0000 7fa82b528b80 4 client.26822 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:42:48.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.840+0000 7fa82b528b80 1 -- 172.21.15.19:0/3059509361 --> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55687696d2e0 con 0x5568769ba9a0 2024-05-07T02:42:48.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.840+0000 7fa80d400700 1 --2- 172.21.15.19:0/3059509361 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x5568769ba9a0 0x5568769bcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:48.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.841+0000 7fa80d400700 1 --2- 172.21.15.19:0/3059509361 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x5568769ba9a0 0x5568769bcdc0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8c3c152ec8374654 server_cookie=46a72d0bf8627f3 in_seq=0 out_seq=0 2024-05-07T02:42:48.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.841+0000 7fa7ffe00700 10 client.26822 ms_handle_connect on v2:172.21.15.27:6836/2583138143 2024-05-07T02:42:48.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.842+0000 7fa7ffe00700 1 -- 172.21.15.19:0/3059509361 <== mds.3 v2:172.21.15.27:6836/2583138143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55687696d2e0 con 0x5568769ba9a0 2024-05-07T02:42:48.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.842+0000 7fa7ffe00700 10 client.26822 handle_command_reply: tid=0 2024-05-07T02:42:48.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.842+0000 7fa82b528b80 1 client.26822 shutdown 2024-05-07T02:42:48.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.842+0000 7fa82b528b80 1 -- 172.21.15.19:0/3059509361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5568769a0a90 msgr2=0x5568769a7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:48.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.842+0000 7fa82b528b80 1 --2- 172.21.15.19:0/3059509361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5568769a0a90 0x5568769a7ac0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa8080a9970 tx=0x7fa8080a9650 comp rx=0 tx=0).stop 2024-05-07T02:42:48.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.842+0000 7fa82b528b80 1 -- 172.21.15.19:0/3059509361 shutdown_connections 2024-05-07T02:42:48.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.842+0000 7fa82b528b80 1 --2- 172.21.15.19:0/3059509361 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x5568769ba9a0 0x5568769bcdc0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:48.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.842+0000 7fa82b528b80 1 --2- 172.21.15.19:0/3059509361 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568769aab60 0x5568769a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:48.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.842+0000 7fa82b528b80 1 --2- 172.21.15.19:0/3059509361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5568769a0a90 0x5568769a7ac0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:48.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.842+0000 7fa7ffe00700 0 client.26822 ms_handle_reset on v2:172.21.15.27:6836/2583138143 2024-05-07T02:42:48.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.842+0000 7fa82b528b80 1 --2- 172.21.15.19:0/3059509361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568769a13e0 0x5568769a8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:48.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.842+0000 7fa82b528b80 1 -- 172.21.15.19:0/3059509361 >> 172.21.15.19:0/3059509361 conn(0x556876822e40 msgr2=0x55687699d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:42:48.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.842+0000 7fa82b528b80 1 -- 172.21.15.19:0/3059509361 shutdown_connections 2024-05-07T02:42:48.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.843+0000 7fa82b528b80 1 -- 172.21.15.19:0/3059509361 wait complete. 2024-05-07T02:42:48.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.843+0000 7fa82b528b80 20 client.26822 trim_cache size 0 max 16384 2024-05-07T02:42:48.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.860+0000 7fa82b528b80 1 Processor -- start 2024-05-07T02:42:48.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.860+0000 7fa82b528b80 1 -- start start 2024-05-07T02:42:48.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.860+0000 7fa82b528b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568769a0a90 0x5568769a84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:48.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.860+0000 7fa82b528b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5568769a13e0 0x5568769b6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:48.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.860+0000 7fa82b528b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568769aab60 0x5568769bfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:48.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.860+0000 7fa82b528b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa8181b2870 con 0x5568769a0a90 2024-05-07T02:42:48.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.860+0000 7fa82b528b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa8181b43b0 con 0x5568769a13e0 2024-05-07T02:42:48.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.860+0000 7fa82b528b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa8181b2f40 con 0x5568769aab60 2024-05-07T02:42:48.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.860+0000 7fa815e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568769a0a90 0x5568769a84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:48.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.860+0000 7fa80d400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5568769a13e0 0x5568769b6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:48.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.861+0000 7fa80d400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5568769a13e0 0x5568769b6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:50634/0 (socket says 172.21.15.19:50634) 2024-05-07T02:42:48.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.861+0000 7fa80d400700 1 -- 172.21.15.19:0/2550969995 learned_addr learned my addr 172.21.15.19:0/2550969995 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:42:48.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.861+0000 7fa816800700 1 --2- 172.21.15.19:0/2550969995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568769aab60 0x5568769bfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:48.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.861+0000 7fa815e00700 1 -- 172.21.15.19:0/2550969995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568769aab60 msgr2=0x5568769bfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:48.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.861+0000 7fa815e00700 1 --2- 172.21.15.19:0/2550969995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568769aab60 0x5568769bfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:48.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.861+0000 7fa815e00700 1 -- 172.21.15.19:0/2550969995 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5568769a13e0 msgr2=0x5568769b6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:48.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.861+0000 7fa815e00700 1 --2- 172.21.15.19:0/2550969995 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5568769a13e0 0x5568769b6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:48.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.861+0000 7fa815e00700 1 -- 172.21.15.19:0/2550969995 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568769a1830 con 0x5568769a0a90 2024-05-07T02:42:48.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.861+0000 7fa80d400700 1 --2- 172.21.15.19:0/2550969995 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5568769a13e0 0x5568769b6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:42:48.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.861+0000 7fa815e00700 1 --2- 172.21.15.19:0/2550969995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568769a0a90 0x5568769a84b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa8080a96e0 tx=0x7fa8080a98e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c52539650930f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:42:48.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.862+0000 7fa806a00700 1 -- 172.21.15.19:0/2550969995 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80808a030 con 0x5568769a0a90 2024-05-07T02:42:48.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.862+0000 7fa806a00700 1 -- 172.21.15.19:0/2550969995 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8080add90 con 0x5568769a0a90 2024-05-07T02:42:48.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.862+0000 7fa806a00700 1 -- 172.21.15.19:0/2550969995 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8080c2070 con 0x5568769a0a90 2024-05-07T02:42:48.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.862+0000 7fa82b528b80 1 -- 172.21.15.19:0/2550969995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568769a0a90 msgr2=0x5568769a84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:48.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.862+0000 7fa82b528b80 1 --2- 172.21.15.19:0/2550969995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568769a0a90 0x5568769a84b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa8080a96e0 tx=0x7fa8080a98e0 comp rx=0 tx=0).stop 2024-05-07T02:42:48.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.863+0000 7fa82b528b80 1 -- 172.21.15.19:0/2550969995 shutdown_connections 2024-05-07T02:42:48.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.863+0000 7fa82b528b80 1 --2- 172.21.15.19:0/2550969995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568769aab60 0x5568769bfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:48.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.863+0000 7fa82b528b80 1 --2- 172.21.15.19:0/2550969995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568769a0a90 0x5568769a84b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:48.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.863+0000 7fa82b528b80 1 --2- 172.21.15.19:0/2550969995 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5568769a13e0 0x5568769b6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:48.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.863+0000 7fa82b528b80 1 -- 172.21.15.19:0/2550969995 >> 172.21.15.19:0/2550969995 conn(0x556876822e40 msgr2=0x5568769a6a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:42:48.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.863+0000 7fa82b528b80 1 -- 172.21.15.19:0/2550969995 shutdown_connections 2024-05-07T02:42:48.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.863+0000 7fa82b528b80 1 -- 172.21.15.19:0/2550969995 wait complete. 2024-05-07T02:42:48.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.863+0000 7fa82b528b80 1 -- start start 2024-05-07T02:42:48.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.863+0000 7fa82b528b80 1 Processor -- start 2024-05-07T02:42:48.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.864+0000 7fa82b528b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568769a0a90 0x5568769c04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:48.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.864+0000 7fa82b528b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568769a13e0 0x5568769bd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:48.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.864+0000 7fa82b528b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5568769aab60 0x5568769bdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:48.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.864+0000 7fa82b528b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa8181b3ce0 con 0x5568769a13e0 2024-05-07T02:42:48.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.864+0000 7fa82b528b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fa8181b3610 con 0x5568769aab60 2024-05-07T02:42:48.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.864+0000 7fa82b528b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa8181b0830 con 0x5568769a0a90 2024-05-07T02:42:48.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.864+0000 7fa82b528b80 10 client.0 _ll_register_callbacks cb 0x556876869c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:42:48.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.864+0000 7fa80d400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568769a13e0 0x5568769bd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:48.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.864+0000 7fa80d400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568769a13e0 0x5568769bd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49282/0 (socket says 172.21.15.19:49282) 2024-05-07T02:42:48.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.864+0000 7fa80d400700 1 -- 172.21.15.19:0/3682980280 learned_addr learned my addr 172.21.15.19:0/3682980280 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:42:48.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.865+0000 7fa815e00700 1 --2- 172.21.15.19:0/3682980280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568769a0a90 0x5568769c04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:48.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.865+0000 7fa816800700 1 --2- 172.21.15.19:0/3682980280 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5568769aab60 0x5568769bdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:48.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.865+0000 7fa80d400700 1 -- 172.21.15.19:0/3682980280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568769a0a90 msgr2=0x5568769c04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:48.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.865+0000 7fa80d400700 1 --2- 172.21.15.19:0/3682980280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568769a0a90 0x5568769c04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:48.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.865+0000 7fa80d400700 1 -- 172.21.15.19:0/3682980280 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5568769aab60 msgr2=0x5568769bdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:48.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.865+0000 7fa80d400700 1 --2- 172.21.15.19:0/3682980280 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5568769aab60 0x5568769bdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:48.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.865+0000 7fa80d400700 1 -- 172.21.15.19:0/3682980280 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568769be660 con 0x5568769a13e0 2024-05-07T02:42:48.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.865+0000 7fa815e00700 1 --2- 172.21.15.19:0/3682980280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568769a0a90 0x5568769c04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:42:48.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.865+0000 7fa80d400700 1 --2- 172.21.15.19:0/3682980280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568769a13e0 0x5568769bd840 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa81806c3c0 tx=0x7fa8000047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85568627d1ab523b server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:42:48.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.865+0000 7fa7ffe00700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-07T02:42:48.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.866+0000 7fa7ffe00700 1 -- 172.21.15.19:0/3682980280 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8000044c0 con 0x5568769a13e0 2024-05-07T02:42:48.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.866+0000 7fa7ffe00700 1 -- 172.21.15.19:0/3682980280 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9525+0+0 (secure 0 0 0) 0x7fa80000ec60 con 0x5568769a13e0 2024-05-07T02:42:48.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.866+0000 7fa82b528b80 1 -- 172.21.15.19:0/3682980280 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8080add90 con 0x5568769a13e0 2024-05-07T02:42:48.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.866+0000 7fa7ffe00700 1 -- 172.21.15.19:0/3682980280 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fa800037050 con 0x5568769a13e0 2024-05-07T02:42:48.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.866+0000 7fa82b528b80 10 client.16623 fetch_fsmap learned FSMap version 50 2024-05-07T02:42:48.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.866+0000 7fa82b528b80 1 -- 172.21.15.19:0/3682980280 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x556876a24fc0 con 0x5568769a13e0 2024-05-07T02:42:48.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.866+0000 7fa7ffe00700 1 -- 172.21.15.19:0/3682980280 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8080add90 con 0x5568769a13e0 2024-05-07T02:42:48.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.867+0000 7fa7ffe00700 1 -- 172.21.15.19:0/3682980280 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 50) ==== 8362+0+0 (secure 0 0 0) 0x7fa800027ce0 con 0x5568769a13e0 2024-05-07T02:42:48.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.867+0000 7fa82b528b80 10 client.16623 fetch_fsmap finished waiting for FSMap version 50 2024-05-07T02:42:48.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.867+0000 7fa82b528b80 10 client.16623 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-05-07T02:42:48.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.867+0000 7fa82b528b80 20 client.16623 populate_metadata read hostname 'smithi019' 2024-05-07T02:42:48.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.867+0000 7fa82b528b80 1 --2- 172.21.15.19:0/3682980280 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x556876a256e0 0x556876a27b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:48.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.867+0000 7fa82b528b80 4 client.16623 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:42:48.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.867+0000 7fa82b528b80 1 -- 172.21.15.19:0/3682980280 --> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5568768d0400 con 0x556876a256e0 2024-05-07T02:42:48.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.867+0000 7fa815e00700 1 --2- 172.21.15.19:0/3682980280 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x556876a256e0 0x556876a27b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:48.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.868+0000 7fa815e00700 1 --2- 172.21.15.19:0/3682980280 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x556876a256e0 0x556876a27b00 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ce8d1460e638eb66 server_cookie=7a1ceb6898d48cdb in_seq=0 out_seq=0 2024-05-07T02:42:48.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.868+0000 7fa7ffe00700 10 client.16623 ms_handle_connect on v2:172.21.15.27:6836/2583138143 2024-05-07T02:42:48.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.868+0000 7fa7ffe00700 1 -- 172.21.15.19:0/3682980280 <== mds.3 v2:172.21.15.27:6836/2583138143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5568768d0400 con 0x556876a256e0 2024-05-07T02:42:48.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.868+0000 7fa7ffe00700 10 client.16623 handle_command_reply: tid=0 2024-05-07T02:42:48.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.868+0000 7fa82b528b80 1 client.16623 shutdown 2024-05-07T02:42:48.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.868+0000 7fa82b528b80 1 -- 172.21.15.19:0/3682980280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568769a13e0 msgr2=0x5568769bd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:48.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.868+0000 7fa82b528b80 1 --2- 172.21.15.19:0/3682980280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568769a13e0 0x5568769bd840 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa81806c3c0 tx=0x7fa8000047c0 comp rx=0 tx=0).stop 2024-05-07T02:42:48.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.868+0000 7fa82b528b80 1 -- 172.21.15.19:0/3682980280 shutdown_connections 2024-05-07T02:42:48.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.868+0000 7fa82b528b80 1 --2- 172.21.15.19:0/3682980280 >> [v2:172.21.15.27:6836/2583138143,v1:172.21.15.27:6837/2583138143] conn(0x556876a256e0 0x556876a27b00 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:48.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.868+0000 7fa82b528b80 1 --2- 172.21.15.19:0/3682980280 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5568769aab60 0x5568769bdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:48.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.868+0000 7fa82b528b80 1 --2- 172.21.15.19:0/3682980280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5568769a0a90 0x5568769c04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:48.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.868+0000 7fa82b528b80 1 --2- 172.21.15.19:0/3682980280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5568769a13e0 0x5568769bd840 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:48.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.868+0000 7fa82b528b80 1 -- 172.21.15.19:0/3682980280 >> 172.21.15.19:0/3682980280 conn(0x556876822e40 msgr2=0x5568769a9100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:42:48.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.868+0000 7fa7ffe00700 0 client.16623 ms_handle_reset on v2:172.21.15.27:6836/2583138143 2024-05-07T02:42:48.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.868+0000 7fa82b528b80 1 -- 172.21.15.19:0/3682980280 shutdown_connections 2024-05-07T02:42:48.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.868+0000 7fa82b528b80 1 -- 172.21.15.19:0/3682980280 wait complete. 2024-05-07T02:42:48.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.869+0000 7fa82b528b80 20 client.16623 trim_cache size 0 max 16384 2024-05-07T02:42:48.887 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:42:48.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.869+0000 7fa817200700 1 -- 172.21.15.19:0/3942870734 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fa7f80b2a70 msgr2=0x7fa7f80b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:48.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.869+0000 7fa817200700 1 --2- 172.21.15.19:0/3942870734 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fa7f80b2a70 0x7fa7f80b4f80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa818110030 tx=0x7fa800009260 comp rx=0 tx=0).stop 2024-05-07T02:42:48.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.869+0000 7fa817200700 1 -- 172.21.15.19:0/3942870734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa818104400 msgr2=0x7fa81819ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:48.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.869+0000 7fa817200700 1 --2- 172.21.15.19:0/3942870734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa818104400 0x7fa81819ce50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa80800e9f0 tx=0x7fa80800ed00 comp rx=0 tx=0).stop 2024-05-07T02:42:48.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.869+0000 7fa817200700 1 -- 172.21.15.19:0/3942870734 shutdown_connections 2024-05-07T02:42:48.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.869+0000 7fa817200700 1 --2- 172.21.15.19:0/3942870734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa818109680 0x7fa8181a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:48.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.869+0000 7fa817200700 1 --2- 172.21.15.19:0/3942870734 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fa7f80b2a70 0x7fa7f80b4f80 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:48.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.869+0000 7fa817200700 1 --2- 172.21.15.19:0/3942870734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa818104400 0x7fa81819ce50 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:48.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.869+0000 7fa817200700 1 --2- 172.21.15.19:0/3942870734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa818108d30 0x7fa81819d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:48.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.869+0000 7fa817200700 1 -- 172.21.15.19:0/3942870734 >> 172.21.15.19:0/3942870734 conn(0x7fa818079d10 msgr2=0x7fa818110500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:42:48.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.870+0000 7fa817200700 1 -- 172.21.15.19:0/3942870734 shutdown_connections 2024-05-07T02:42:48.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:48.870+0000 7fa817200700 1 -- 172.21.15.19:0/3942870734 wait complete. 2024-05-07T02:42:49.770 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-07T02:42:50.246 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:42:50.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:49 smithi064 ceph-mon[34699]: pgmap v340: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-07T02:42:50.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:49 smithi019 ceph-mon[29781]: pgmap v340: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-07T02:42:50.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:49 smithi027 ceph-mon[34398]: pgmap v340: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-07T02:42:52.316 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:51 smithi019 ceph-mon[29781]: pgmap v341: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-07T02:42:52.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:51 smithi064 ceph-mon[34699]: pgmap v341: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-07T02:42:52.531 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:51 smithi027 ceph-mon[34398]: pgmap v341: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-05-07T02:42:52.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.729+0000 7f3f91200700 1 -- 172.21.15.19:0/3653140835 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f8c10b110 msgr2=0x7f3f8c117620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:52.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.729+0000 7f3f91200700 1 --2- 172.21.15.19:0/3653140835 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f8c10b110 0x7f3f8c117620 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3f8400b0a0 tx=0x7f3f8400b3b0 comp rx=0 tx=0).stop 2024-05-07T02:42:52.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.729+0000 7f3f91200700 1 -- 172.21.15.19:0/3653140835 shutdown_connections 2024-05-07T02:42:52.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.729+0000 7f3f91200700 1 --2- 172.21.15.19:0/3653140835 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f8c10b110 0x7f3f8c117620 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:52.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.729+0000 7f3f91200700 1 --2- 172.21.15.19:0/3653140835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f8c10f6c0 0x7f3f8c10fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:52.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.729+0000 7f3f91200700 1 --2- 172.21.15.19:0/3653140835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f8c10a7c0 0x7f3f8c10abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:52.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.729+0000 7f3f91200700 1 -- 172.21.15.19:0/3653140835 >> 172.21.15.19:0/3653140835 conn(0x7f3f8c076b00 msgr2=0x7f3f8c076f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:42:52.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.729+0000 7f3f91200700 1 -- 172.21.15.19:0/3653140835 shutdown_connections 2024-05-07T02:42:52.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.730+0000 7f3f91200700 1 -- 172.21.15.19:0/3653140835 wait complete. 2024-05-07T02:42:52.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.731+0000 7f3f91200700 1 Processor -- start 2024-05-07T02:42:52.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.731+0000 7f3f91200700 1 -- start start 2024-05-07T02:42:52.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.732+0000 7f3f91200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f8c10a7c0 0x7f3f8c1a1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:52.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.732+0000 7f3f91200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f8c10b110 0x7f3f8c1a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:52.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.732+0000 7f3f91200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f8c10f6c0 0x7f3f8c1a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:52.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.732+0000 7f3f91200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8c11a8b0 con 0x7f3f8c10f6c0 2024-05-07T02:42:52.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.732+0000 7f3f91200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8c11a730 con 0x7f3f8c10b110 2024-05-07T02:42:52.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.732+0000 7f3f91200700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8c11aa30 con 0x7f3f8c10a7c0 2024-05-07T02:42:52.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.732+0000 7f3f8b400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f8c10a7c0 0x7f3f8c1a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:52.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.732+0000 7f3f8b400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f8c10a7c0 0x7f3f8c1a1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:36254/0 (socket says 172.21.15.19:36254) 2024-05-07T02:42:52.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.732+0000 7f3f8aa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f8c10b110 0x7f3f8c1a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:52.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.732+0000 7f3f8b400700 1 -- 172.21.15.19:0/831487962 learned_addr learned my addr 172.21.15.19:0/831487962 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:42:52.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.732+0000 7f3f8be00700 1 --2- 172.21.15.19:0/831487962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f8c10f6c0 0x7f3f8c1a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:52.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.733+0000 7f3f8b400700 1 -- 172.21.15.19:0/831487962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f8c10b110 msgr2=0x7f3f8c1a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:52.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.733+0000 7f3f8b400700 1 --2- 172.21.15.19:0/831487962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f8c10b110 0x7f3f8c1a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:52.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.733+0000 7f3f8b400700 1 -- 172.21.15.19:0/831487962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f8c10f6c0 msgr2=0x7f3f8c1a5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:52.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.733+0000 7f3f8b400700 1 --2- 172.21.15.19:0/831487962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f8c10f6c0 0x7f3f8c1a5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:52.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.733+0000 7f3f8b400700 1 -- 172.21.15.19:0/831487962 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f8c1a6030 con 0x7f3f8c10a7c0 2024-05-07T02:42:52.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.733+0000 7f3f8b400700 1 --2- 172.21.15.19:0/831487962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f8c10a7c0 0x7f3f8c1a1160 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c00e9a0 tx=0x7f3f7c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:42:52.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.733+0000 7f3f83e00700 1 -- 172.21.15.19:0/831487962 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c00ca10 con 0x7f3f8c10a7c0 2024-05-07T02:42:52.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.733+0000 7f3f83e00700 1 -- 172.21.15.19:0/831487962 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f7c00cbb0 con 0x7f3f8c10a7c0 2024-05-07T02:42:52.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.733+0000 7f3f91200700 1 -- 172.21.15.19:0/831487962 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f8c1a6320 con 0x7f3f8c10a7c0 2024-05-07T02:42:52.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.733+0000 7f3f83e00700 1 -- 172.21.15.19:0/831487962 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c018680 con 0x7f3f8c10a7c0 2024-05-07T02:42:52.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.734+0000 7f3f91200700 1 -- 172.21.15.19:0/831487962 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f8c1adc40 con 0x7f3f8c10a7c0 2024-05-07T02:42:52.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.738+0000 7f3f83e00700 1 -- 172.21.15.19:0/831487962 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3f7c010430 con 0x7f3f8c10a7c0 2024-05-07T02:42:52.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.738+0000 7f3fa12bcb80 1 Processor -- start 2024-05-07T02:42:52.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.739+0000 7f3fa12bcb80 1 -- start start 2024-05-07T02:42:52.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.739+0000 7f3fa12bcb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564b1076a90 0x5564b1076ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:52.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.739+0000 7f3fa12bcb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564b10773e0 0x5564b1080620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:52.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.739+0000 7f3fa12bcb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564b1080b60 0x5564b1082f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:52.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.739+0000 7f3fa12bcb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5564b0f98a20 con 0x5564b1076a90 2024-05-07T02:42:52.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.739+0000 7f3fa12bcb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5564b0f36760 con 0x5564b10773e0 2024-05-07T02:42:52.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.739+0000 7f3fa12bcb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5564b0f949e0 con 0x5564b1080b60 2024-05-07T02:42:52.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.740+0000 7f3f8be00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564b1076a90 0x5564b1076ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:52.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.740+0000 7f3f8b400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564b10773e0 0x5564b1080620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:52.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.740+0000 7f3f8b400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564b10773e0 0x5564b1080620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:50682/0 (socket says 172.21.15.19:50682) 2024-05-07T02:42:52.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.740+0000 7f3f8b400700 1 -- 172.21.15.19:0/3539804878 learned_addr learned my addr 172.21.15.19:0/3539804878 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:42:52.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.740+0000 7f3f8be00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564b1076a90 0x5564b1076ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49300/0 (socket says 172.21.15.19:49300) 2024-05-07T02:42:52.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.740+0000 7f3f8aa00700 1 --2- 172.21.15.19:0/3539804878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564b1080b60 0x5564b1082f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:52.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.740+0000 7f3f8b400700 1 -- 172.21.15.19:0/3539804878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564b1080b60 msgr2=0x5564b1082f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:52.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.740+0000 7f3f8b400700 1 --2- 172.21.15.19:0/3539804878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564b1080b60 0x5564b1082f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:52.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.740+0000 7f3f8b400700 1 -- 172.21.15.19:0/3539804878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564b1076a90 msgr2=0x5564b1076ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:52.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.740+0000 7f3f8b400700 1 --2- 172.21.15.19:0/3539804878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564b1076a90 0x5564b1076ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:52.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.740+0000 7f3f8b400700 1 -- 172.21.15.19:0/3539804878 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564b0ee58d0 con 0x5564b10773e0 2024-05-07T02:42:52.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.741+0000 7f3f83e00700 1 --2- 172.21.15.19:0/831487962 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f3f700b29f0 0x7f3f700b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:52.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.741+0000 7f3f8b400700 1 --2- 172.21.15.19:0/3539804878 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564b10773e0 0x5564b1080620 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c0dfdc0 tx=0x7f3f7c0e6e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce0ee80d3114327 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:42:52.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.741+0000 7f3f83e00700 1 -- 172.21.15.19:0/831487962 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9525+0+0 (secure 0 0 0) 0x7f3f7c0e2b80 con 0x7f3f8c10a7c0 2024-05-07T02:42:52.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.741+0000 7f3f8be00700 1 --2- 172.21.15.19:0/831487962 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f3f700b29f0 0x7f3f700b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:52.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.741+0000 7f3f80c00700 1 -- 172.21.15.19:0/3539804878 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c0bee60 con 0x5564b10773e0 2024-05-07T02:42:52.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.741+0000 7f3f80c00700 1 -- 172.21.15.19:0/3539804878 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f7c0a9520 con 0x5564b10773e0 2024-05-07T02:42:52.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.741+0000 7f3f80c00700 1 -- 172.21.15.19:0/3539804878 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c0a9180 con 0x5564b10773e0 2024-05-07T02:42:52.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.742+0000 7f3fa12bcb80 1 -- 172.21.15.19:0/3539804878 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564b10773e0 msgr2=0x5564b1080620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:52.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.742+0000 7f3f8be00700 1 --2- 172.21.15.19:0/831487962 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f3f700b29f0 0x7f3f700b4f00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x5564b1072e40 tx=0x7f3f8400b720 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:42:52.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.742+0000 7f3fa12bcb80 1 --2- 172.21.15.19:0/3539804878 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564b10773e0 0x5564b1080620 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c0dfdc0 tx=0x7f3f7c0e6e40 comp rx=0 tx=0).stop 2024-05-07T02:42:52.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.742+0000 7f3fa12bcb80 1 -- 172.21.15.19:0/3539804878 shutdown_connections 2024-05-07T02:42:52.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.742+0000 7f3fa12bcb80 1 --2- 172.21.15.19:0/3539804878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564b1080b60 0x5564b1082f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:52.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.742+0000 7f3fa12bcb80 1 --2- 172.21.15.19:0/3539804878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564b1076a90 0x5564b1076ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:52.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.742+0000 7f3fa12bcb80 1 --2- 172.21.15.19:0/3539804878 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564b10773e0 0x5564b1080620 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:52.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.742+0000 7f3fa12bcb80 1 -- 172.21.15.19:0/3539804878 >> 172.21.15.19:0/3539804878 conn(0x5564b0ef8e40 msgr2=0x5564b1071880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:42:52.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.742+0000 7f3fa12bcb80 1 -- 172.21.15.19:0/3539804878 shutdown_connections 2024-05-07T02:42:52.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.742+0000 7f3fa12bcb80 1 -- 172.21.15.19:0/3539804878 wait complete. 2024-05-07T02:42:52.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.743+0000 7f3fa12bcb80 1 -- start start 2024-05-07T02:42:52.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.743+0000 7f3fa12bcb80 1 Processor -- start 2024-05-07T02:42:52.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.744+0000 7f3fa12bcb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564b1076a90 0x5564b107db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:52.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.744+0000 7f3fa12bcb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564b10773e0 0x5564b107e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:52.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.744+0000 7f3fa12bcb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564b1080b60 0x5564b107e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:52.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.744+0000 7f3fa12bcb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5564b1083960 con 0x5564b1080b60 2024-05-07T02:42:52.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.744+0000 7f3fa12bcb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5564b0efeb20 con 0x5564b10773e0 2024-05-07T02:42:52.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.744+0000 7f3fa12bcb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5564b1083ae0 con 0x5564b1076a90 2024-05-07T02:42:52.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.744+0000 7f3fa12bcb80 10 client.0 _ll_register_callbacks cb 0x5564b0f3fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:42:52.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.744+0000 7f3f8aa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564b10773e0 0x5564b107e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:52.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.744+0000 7f3f8aa00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564b10773e0 0x5564b107e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:50698/0 (socket says 172.21.15.19:50698) 2024-05-07T02:42:52.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.744+0000 7f3f8aa00700 1 -- 172.21.15.19:0/4229986787 learned_addr learned my addr 172.21.15.19:0/4229986787 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:42:52.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.744+0000 7f3f8b400700 1 --2- 172.21.15.19:0/4229986787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564b1076a90 0x5564b107db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:52.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.744+0000 7f3f8be00700 1 --2- 172.21.15.19:0/4229986787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564b1080b60 0x5564b107e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:52.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.745+0000 7f3f8aa00700 1 -- 172.21.15.19:0/4229986787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564b1076a90 msgr2=0x5564b107db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:52.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.745+0000 7f3f8aa00700 1 --2- 172.21.15.19:0/4229986787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564b1076a90 0x5564b107db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:52.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.745+0000 7f3f8aa00700 1 -- 172.21.15.19:0/4229986787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564b1080b60 msgr2=0x5564b107e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:52.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.745+0000 7f3f8aa00700 1 --2- 172.21.15.19:0/4229986787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564b1080b60 0x5564b107e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:52.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.745+0000 7f3f8aa00700 1 -- 172.21.15.19:0/4229986787 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564b0f5e420 con 0x5564b10773e0 2024-05-07T02:42:52.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.745+0000 7f3f8b400700 1 --2- 172.21.15.19:0/4229986787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564b1076a90 0x5564b107db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:42:52.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.745+0000 7f3f8aa00700 1 --2- 172.21.15.19:0/4229986787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564b10773e0 0x5564b107e050 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3f780127f0 tx=0x7f3f78012b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f42b87dcdb28233 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:42:52.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.745+0000 7f3f75600700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:42:52.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.745+0000 7f3f75600700 1 -- 172.21.15.19:0/4229986787 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7800f4e0 con 0x5564b10773e0 2024-05-07T02:42:52.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.745+0000 7f3f75600700 1 -- 172.21.15.19:0/4229986787 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9525+0+0 (secure 0 0 0) 0x7f3f7800f680 con 0x5564b10773e0 2024-05-07T02:42:52.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.745+0000 7f3fa12bcb80 1 -- 172.21.15.19:0/4229986787 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f7c0a9520 con 0x5564b10773e0 2024-05-07T02:42:52.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.746+0000 7f3f75600700 1 -- 172.21.15.19:0/4229986787 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f3f7800f880 con 0x5564b10773e0 2024-05-07T02:42:52.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.746+0000 7f3fa12bcb80 10 client.26840 fetch_fsmap learned FSMap version 50 2024-05-07T02:42:52.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.746+0000 7f3fa12bcb80 1 -- 172.21.15.19:0/4229986787 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5564b108e630 con 0x5564b10773e0 2024-05-07T02:42:52.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.746+0000 7f3f75600700 1 -- 172.21.15.19:0/4229986787 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f7c0a9520 con 0x5564b10773e0 2024-05-07T02:42:52.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.746+0000 7f3f75600700 1 -- 172.21.15.19:0/4229986787 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 50) ==== 8362+0+0 (secure 0 0 0) 0x7f3f78010a70 con 0x5564b10773e0 2024-05-07T02:42:52.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.746+0000 7f3fa12bcb80 10 client.26840 fetch_fsmap finished waiting for FSMap version 50 2024-05-07T02:42:52.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.746+0000 7f3fa12bcb80 10 client.26840 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-07T02:42:52.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.746+0000 7f3fa12bcb80 20 client.26840 populate_metadata read hostname 'smithi019' 2024-05-07T02:42:52.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.746+0000 7f3fa12bcb80 1 --2- 172.21.15.19:0/4229986787 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x5564b1090c70 0x5564b1093090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:52.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.746+0000 7f3fa12bcb80 4 client.26840 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:42:52.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.746+0000 7f3fa12bcb80 1 -- 172.21.15.19:0/4229986787 --> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564b10432e0 con 0x5564b1090c70 2024-05-07T02:42:52.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.747+0000 7f3f8b400700 1 --2- 172.21.15.19:0/4229986787 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x5564b1090c70 0x5564b1093090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:52.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.747+0000 7f3f8b400700 1 --2- 172.21.15.19:0/4229986787 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x5564b1090c70 0x5564b1093090 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=deaf80dfadb155a9 server_cookie=ef2d3e914a1cd56 in_seq=0 out_seq=0 2024-05-07T02:42:52.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.747+0000 7f3f75600700 10 client.26840 ms_handle_connect on v2:172.21.15.64:6838/1583120004 2024-05-07T02:42:52.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.748+0000 7f3f75600700 1 -- 172.21.15.19:0/4229986787 <== mds.2 v2:172.21.15.64:6838/1583120004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5564b10432e0 con 0x5564b1090c70 2024-05-07T02:42:52.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.748+0000 7f3f75600700 10 client.26840 handle_command_reply: tid=0 2024-05-07T02:42:52.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.748+0000 7f3fa12bcb80 1 client.26840 shutdown 2024-05-07T02:42:52.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.749+0000 7f3fa12bcb80 1 -- 172.21.15.19:0/4229986787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564b10773e0 msgr2=0x5564b107e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:52.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.749+0000 7f3fa12bcb80 1 --2- 172.21.15.19:0/4229986787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564b10773e0 0x5564b107e050 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3f780127f0 tx=0x7f3f78012b00 comp rx=0 tx=0).stop 2024-05-07T02:42:52.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.749+0000 7f3fa12bcb80 1 -- 172.21.15.19:0/4229986787 shutdown_connections 2024-05-07T02:42:52.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.749+0000 7f3fa12bcb80 1 --2- 172.21.15.19:0/4229986787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564b1080b60 0x5564b107e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:52.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.749+0000 7f3fa12bcb80 1 --2- 172.21.15.19:0/4229986787 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564b1076a90 0x5564b107db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:52.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.749+0000 7f3fa12bcb80 1 --2- 172.21.15.19:0/4229986787 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x5564b1090c70 0x5564b1093090 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:52.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.749+0000 7f3fa12bcb80 1 --2- 172.21.15.19:0/4229986787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564b10773e0 0x5564b107e050 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:52.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.749+0000 7f3fa12bcb80 1 -- 172.21.15.19:0/4229986787 >> 172.21.15.19:0/4229986787 conn(0x5564b0ef8e40 msgr2=0x5564b1073450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:42:52.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.749+0000 7f3f75600700 0 client.26840 ms_handle_reset on v2:172.21.15.64:6838/1583120004 2024-05-07T02:42:52.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.749+0000 7f3fa12bcb80 1 -- 172.21.15.19:0/4229986787 shutdown_connections 2024-05-07T02:42:52.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.749+0000 7f3fa12bcb80 1 -- 172.21.15.19:0/4229986787 wait complete. 2024-05-07T02:42:52.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.749+0000 7f3fa12bcb80 20 client.26840 trim_cache size 0 max 16384 2024-05-07T02:42:52.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.766+0000 7f3fa12bcb80 1 Processor -- start 2024-05-07T02:42:52.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.766+0000 7f3fa12bcb80 1 -- start start 2024-05-07T02:42:52.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.766+0000 7f3fa12bcb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564b1076a90 0x5564b107ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:52.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.766+0000 7f3fa12bcb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564b10773e0 0x5564b107e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:52.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.766+0000 7f3fa12bcb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564b1080b60 0x5564b108c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:52.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.766+0000 7f3fa12bcb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8c1b6ab0 con 0x5564b10773e0 2024-05-07T02:42:52.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.766+0000 7f3fa12bcb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8c1b85f0 con 0x5564b1080b60 2024-05-07T02:42:52.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.766+0000 7f3fa12bcb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8c1b7180 con 0x5564b1076a90 2024-05-07T02:42:52.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.767+0000 7f3f8b400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564b1076a90 0x5564b107ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:52.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.767+0000 7f3f8aa00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564b10773e0 0x5564b107e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:52.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.767+0000 7f3f8b400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564b1076a90 0x5564b107ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:36274/0 (socket says 172.21.15.19:36274) 2024-05-07T02:42:52.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.767+0000 7f3f8b400700 1 -- 172.21.15.19:0/1728836920 learned_addr learned my addr 172.21.15.19:0/1728836920 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:42:52.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.767+0000 7f3f8be00700 1 --2- 172.21.15.19:0/1728836920 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564b1080b60 0x5564b108c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:52.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.767+0000 7f3f8b400700 1 -- 172.21.15.19:0/1728836920 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564b1080b60 msgr2=0x5564b108c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:52.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.767+0000 7f3f8b400700 1 --2- 172.21.15.19:0/1728836920 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564b1080b60 0x5564b108c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:52.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.767+0000 7f3f8b400700 1 -- 172.21.15.19:0/1728836920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564b10773e0 msgr2=0x5564b107e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:52.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.767+0000 7f3f8b400700 1 --2- 172.21.15.19:0/1728836920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564b10773e0 0x5564b107e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:52.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.767+0000 7f3f8b400700 1 -- 172.21.15.19:0/1728836920 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564b107c040 con 0x5564b1076a90 2024-05-07T02:42:52.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.768+0000 7f3f8b400700 1 --2- 172.21.15.19:0/1728836920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564b1076a90 0x5564b107ce30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c0a5a90 tx=0x7f3f7c0e6040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f0005f2190d4176 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:42:52.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.768+0000 7f3f77e00700 1 -- 172.21.15.19:0/1728836920 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c0bee60 con 0x5564b1076a90 2024-05-07T02:42:52.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.768+0000 7f3f77e00700 1 -- 172.21.15.19:0/1728836920 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f7c0b21a0 con 0x5564b1076a90 2024-05-07T02:42:52.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.768+0000 7f3f77e00700 1 -- 172.21.15.19:0/1728836920 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c0aa110 con 0x5564b1076a90 2024-05-07T02:42:52.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.769+0000 7f3fa12bcb80 1 -- 172.21.15.19:0/1728836920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564b1076a90 msgr2=0x5564b107ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:52.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.769+0000 7f3fa12bcb80 1 --2- 172.21.15.19:0/1728836920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564b1076a90 0x5564b107ce30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c0a5a90 tx=0x7f3f7c0e6040 comp rx=0 tx=0).stop 2024-05-07T02:42:52.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.769+0000 7f3fa12bcb80 1 -- 172.21.15.19:0/1728836920 shutdown_connections 2024-05-07T02:42:52.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.769+0000 7f3fa12bcb80 1 --2- 172.21.15.19:0/1728836920 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564b1080b60 0x5564b108c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:52.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.769+0000 7f3fa12bcb80 1 --2- 172.21.15.19:0/1728836920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564b1076a90 0x5564b107ce30 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:52.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.769+0000 7f3fa12bcb80 1 --2- 172.21.15.19:0/1728836920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564b10773e0 0x5564b107e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:52.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.769+0000 7f3fa12bcb80 1 -- 172.21.15.19:0/1728836920 >> 172.21.15.19:0/1728836920 conn(0x5564b0ef8e40 msgr2=0x5564b107dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:42:52.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.769+0000 7f3fa12bcb80 1 -- 172.21.15.19:0/1728836920 shutdown_connections 2024-05-07T02:42:52.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.769+0000 7f3fa12bcb80 1 -- 172.21.15.19:0/1728836920 wait complete. 2024-05-07T02:42:52.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.769+0000 7f3fa12bcb80 1 -- start start 2024-05-07T02:42:52.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.770+0000 7f3fa12bcb80 1 Processor -- start 2024-05-07T02:42:52.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.770+0000 7f3fa12bcb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564b1076a90 0x5564b1088f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:52.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.770+0000 7f3fa12bcb80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564b10773e0 0x5564b1089440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:52.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.770+0000 7f3fa12bcb80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564b1080b60 0x5564b1084440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:52.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.770+0000 7f3fa12bcb80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8c1b7f20 con 0x5564b1080b60 2024-05-07T02:42:52.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.770+0000 7f3fa12bcb80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8c1b7850 con 0x5564b10773e0 2024-05-07T02:42:52.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.770+0000 7f3fa12bcb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8c1b4a70 con 0x5564b1076a90 2024-05-07T02:42:52.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.770+0000 7f3fa12bcb80 10 client.0 _ll_register_callbacks cb 0x5564b0f3fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:42:52.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.771+0000 7f3f8b400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564b1076a90 0x5564b1088f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:52.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.771+0000 7f3f8b400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564b1076a90 0x5564b1088f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:36282/0 (socket says 172.21.15.19:36282) 2024-05-07T02:42:52.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.771+0000 7f3f8b400700 1 -- 172.21.15.19:0/3013382697 learned_addr learned my addr 172.21.15.19:0/3013382697 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:42:52.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.771+0000 7f3f8aa00700 1 --2- 172.21.15.19:0/3013382697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564b10773e0 0x5564b1089440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:52.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.771+0000 7f3f8be00700 1 --2- 172.21.15.19:0/3013382697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564b1080b60 0x5564b1084440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:52.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.771+0000 7f3f8b400700 1 -- 172.21.15.19:0/3013382697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564b10773e0 msgr2=0x5564b1089440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:52.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.771+0000 7f3f8b400700 1 --2- 172.21.15.19:0/3013382697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564b10773e0 0x5564b1089440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:52.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.771+0000 7f3f8b400700 1 -- 172.21.15.19:0/3013382697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564b1080b60 msgr2=0x5564b1084440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:52.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.771+0000 7f3f8b400700 1 --2- 172.21.15.19:0/3013382697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564b1080b60 0x5564b1084440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:52.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.771+0000 7f3f8b400700 1 -- 172.21.15.19:0/3013382697 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564b1084c10 con 0x5564b1076a90 2024-05-07T02:42:52.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.772+0000 7f3f8b400700 1 --2- 172.21.15.19:0/3013382697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564b1076a90 0x5564b1088f00 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c0e6ab0 tx=0x7f3f7c009790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff1219737e454da9 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:42:52.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.772+0000 7f3f75600700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-07T02:42:52.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.772+0000 7f3f75600700 1 -- 172.21.15.19:0/3013382697 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c08f060 con 0x5564b1076a90 2024-05-07T02:42:52.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.772+0000 7f3fa12bcb80 1 -- 172.21.15.19:0/3013382697 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f7c0b21a0 con 0x5564b1076a90 2024-05-07T02:42:52.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.772+0000 7f3f75600700 1 -- 172.21.15.19:0/3013382697 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9525+0+0 (secure 0 0 0) 0x7f3f7c0a6000 con 0x5564b1076a90 2024-05-07T02:42:52.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.772+0000 7f3f75600700 1 -- 172.21.15.19:0/3013382697 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f3f7c0a9360 con 0x5564b1076a90 2024-05-07T02:42:52.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.772+0000 7f3fa12bcb80 10 client.26806 fetch_fsmap learned FSMap version 50 2024-05-07T02:42:52.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.772+0000 7f3fa12bcb80 1 -- 172.21.15.19:0/3013382697 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5564b10faf40 con 0x5564b1076a90 2024-05-07T02:42:52.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.773+0000 7f3f75600700 1 -- 172.21.15.19:0/3013382697 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f7c0b21a0 con 0x5564b1076a90 2024-05-07T02:42:52.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.773+0000 7f3f75600700 1 -- 172.21.15.19:0/3013382697 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 50) ==== 8362+0+0 (secure 0 0 0) 0x7f3f7c0e0100 con 0x5564b1076a90 2024-05-07T02:42:52.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.773+0000 7f3fa12bcb80 10 client.26806 fetch_fsmap finished waiting for FSMap version 50 2024-05-07T02:42:52.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.773+0000 7f3fa12bcb80 10 client.26806 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-07T02:42:52.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.773+0000 7f3fa12bcb80 20 client.26806 populate_metadata read hostname 'smithi019' 2024-05-07T02:42:52.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.773+0000 7f3fa12bcb80 1 --2- 172.21.15.19:0/3013382697 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x5564b10fb110 0x5564b10fd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:52.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.773+0000 7f3fa12bcb80 4 client.26806 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:42:52.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.773+0000 7f3fa12bcb80 1 -- 172.21.15.19:0/3013382697 --> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5564b0fa6400 con 0x5564b10fb110 2024-05-07T02:42:52.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.773+0000 7f3f8aa00700 1 --2- 172.21.15.19:0/3013382697 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x5564b10fb110 0x5564b10fd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:52.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.774+0000 7f3f8aa00700 1 --2- 172.21.15.19:0/3013382697 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x5564b10fb110 0x5564b10fd530 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=48fd384dad03405e server_cookie=696c953409be716a in_seq=0 out_seq=0 2024-05-07T02:42:52.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.774+0000 7f3f75600700 10 client.26806 ms_handle_connect on v2:172.21.15.64:6838/1583120004 2024-05-07T02:42:52.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.774+0000 7f3f75600700 1 -- 172.21.15.19:0/3013382697 <== mds.2 v2:172.21.15.64:6838/1583120004 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5564b0fa6400 con 0x5564b10fb110 2024-05-07T02:42:52.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.774+0000 7f3f75600700 10 client.26806 handle_command_reply: tid=0 2024-05-07T02:42:52.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.774+0000 7f3fa12bcb80 1 client.26806 shutdown 2024-05-07T02:42:52.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.774+0000 7f3fa12bcb80 1 -- 172.21.15.19:0/3013382697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564b1076a90 msgr2=0x5564b1088f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:52.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.774+0000 7f3fa12bcb80 1 --2- 172.21.15.19:0/3013382697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564b1076a90 0x5564b1088f00 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c0e6ab0 tx=0x7f3f7c009790 comp rx=0 tx=0).stop 2024-05-07T02:42:52.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.774+0000 7f3fa12bcb80 1 -- 172.21.15.19:0/3013382697 shutdown_connections 2024-05-07T02:42:52.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.774+0000 7f3fa12bcb80 1 --2- 172.21.15.19:0/3013382697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5564b1080b60 0x5564b1084440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:52.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.774+0000 7f3fa12bcb80 1 --2- 172.21.15.19:0/3013382697 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5564b1076a90 0x5564b1088f00 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:52.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.774+0000 7f3fa12bcb80 1 --2- 172.21.15.19:0/3013382697 >> [v2:172.21.15.64:6838/1583120004,v1:172.21.15.64:6839/1583120004] conn(0x5564b10fb110 0x5564b10fd530 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:52.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.774+0000 7f3fa12bcb80 1 --2- 172.21.15.19:0/3013382697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5564b10773e0 0x5564b1089440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:52.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.774+0000 7f3fa12bcb80 1 -- 172.21.15.19:0/3013382697 >> 172.21.15.19:0/3013382697 conn(0x5564b0ef8e40 msgr2=0x5564b107cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:42:52.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.774+0000 7f3f75600700 0 client.26806 ms_handle_reset on v2:172.21.15.64:6838/1583120004 2024-05-07T02:42:52.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.775+0000 7f3fa12bcb80 1 -- 172.21.15.19:0/3013382697 shutdown_connections 2024-05-07T02:42:52.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.775+0000 7f3fa12bcb80 1 -- 172.21.15.19:0/3013382697 wait complete. 2024-05-07T02:42:52.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.775+0000 7f3fa12bcb80 20 client.26806 trim_cache size 0 max 16384 2024-05-07T02:42:52.781 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:42:52.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.775+0000 7f3f91200700 1 -- 172.21.15.19:0/831487962 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f3f700b29f0 msgr2=0x7f3f700b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:52.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.775+0000 7f3f91200700 1 --2- 172.21.15.19:0/831487962 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f3f700b29f0 0x7f3f700b4f00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x5564b1072e40 tx=0x7f3f8400b720 comp rx=0 tx=0).stop 2024-05-07T02:42:52.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.775+0000 7f3f91200700 1 -- 172.21.15.19:0/831487962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f8c10a7c0 msgr2=0x7f3f8c1a1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:52.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.775+0000 7f3f91200700 1 --2- 172.21.15.19:0/831487962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f8c10a7c0 0x7f3f8c1a1160 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c00e9a0 tx=0x7f3f7c00ecb0 comp rx=0 tx=0).stop 2024-05-07T02:42:52.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.776+0000 7f3f91200700 1 -- 172.21.15.19:0/831487962 shutdown_connections 2024-05-07T02:42:52.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.776+0000 7f3f91200700 1 --2- 172.21.15.19:0/831487962 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f3f700b29f0 0x7f3f700b4f00 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:52.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.776+0000 7f3f91200700 1 --2- 172.21.15.19:0/831487962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f8c10f6c0 0x7f3f8c1a5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:52.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.776+0000 7f3f91200700 1 --2- 172.21.15.19:0/831487962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f8c10a7c0 0x7f3f8c1a1160 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:52.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.776+0000 7f3f91200700 1 --2- 172.21.15.19:0/831487962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f8c10b110 0x7f3f8c1a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:52.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.776+0000 7f3f91200700 1 -- 172.21.15.19:0/831487962 >> 172.21.15.19:0/831487962 conn(0x7f3f8c076b00 msgr2=0x7f3f8c1087c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:42:52.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.776+0000 7f3f91200700 1 -- 172.21.15.19:0/831487962 shutdown_connections 2024-05-07T02:42:52.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:52.776+0000 7f3f91200700 1 -- 172.21.15.19:0/831487962 wait complete. 2024-05-07T02:42:53.721 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-07T02:42:54.182 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:42:54.329 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:53 smithi019 ceph-mon[29781]: pgmap v342: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:42:54.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:53 smithi064 ceph-mon[34699]: pgmap v342: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:42:54.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:53 smithi027 ceph-mon[34398]: pgmap v342: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:42:56.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:55 smithi019 ceph-mon[29781]: pgmap v343: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:42:56.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:55 smithi064 ceph-mon[34699]: pgmap v343: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:42:56.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:55 smithi027 ceph-mon[34398]: pgmap v343: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:42:56.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.548+0000 7f15d9600700 1 -- 172.21.15.19:0/2323940162 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15d4106ed0 msgr2=0x7f15d4113360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:56.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.548+0000 7f15d9600700 1 --2- 172.21.15.19:0/2323940162 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15d4106ed0 0x7f15d4113360 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f15cc00b0a0 tx=0x7f15cc00b3b0 comp rx=0 tx=0).stop 2024-05-07T02:42:56.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.548+0000 7f15d9600700 1 -- 172.21.15.19:0/2323940162 shutdown_connections 2024-05-07T02:42:56.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.548+0000 7f15d9600700 1 --2- 172.21.15.19:0/2323940162 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15d4106ed0 0x7f15d4113360 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:56.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.548+0000 7f15d9600700 1 --2- 172.21.15.19:0/2323940162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15d410c5c0 0x7f15d410c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:56.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.548+0000 7f15d9600700 1 --2- 172.21.15.19:0/2323940162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d4106580 0x7f15d4106990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:56.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.548+0000 7f15d9600700 1 -- 172.21.15.19:0/2323940162 >> 172.21.15.19:0/2323940162 conn(0x7f15d4102080 msgr2=0x7f15d41044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:42:56.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.548+0000 7f15d9600700 1 -- 172.21.15.19:0/2323940162 shutdown_connections 2024-05-07T02:42:56.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.548+0000 7f15d9600700 1 -- 172.21.15.19:0/2323940162 wait complete. 2024-05-07T02:42:56.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.550+0000 7f15d9600700 1 Processor -- start 2024-05-07T02:42:56.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.550+0000 7f15d9600700 1 -- start start 2024-05-07T02:42:56.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.550+0000 7f15d9600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d4106580 0x7f15d4110730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:56.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.550+0000 7f15d9600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15d4106ed0 0x7f15d4110c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:56.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.550+0000 7f15d9600700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15d410c5c0 0x7f15d41111b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:56.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.550+0000 7f15d9600700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f15d4116740 con 0x7f15d4106580 2024-05-07T02:42:56.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.550+0000 7f15d9600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f15d41165c0 con 0x7f15d4106ed0 2024-05-07T02:42:56.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.550+0000 7f15d9600700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f15d41168c0 con 0x7f15d410c5c0 2024-05-07T02:42:56.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.551+0000 7f15d2a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d4106580 0x7f15d4110730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:56.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.551+0000 7f15d2a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d4106580 0x7f15d4110730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49344/0 (socket says 172.21.15.19:49344) 2024-05-07T02:42:56.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.551+0000 7f15d3400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15d410c5c0 0x7f15d41111b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:56.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.551+0000 7f15d2a00700 1 -- 172.21.15.19:0/3298337073 learned_addr learned my addr 172.21.15.19:0/3298337073 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:42:56.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.551+0000 7f15d2000700 1 --2- 172.21.15.19:0/3298337073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15d4106ed0 0x7f15d4110c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:56.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.551+0000 7f15d3400700 1 -- 172.21.15.19:0/3298337073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15d4106ed0 msgr2=0x7f15d4110c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:56.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.551+0000 7f15d3400700 1 --2- 172.21.15.19:0/3298337073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15d4106ed0 0x7f15d4110c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:56.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.551+0000 7f15d3400700 1 -- 172.21.15.19:0/3298337073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d4106580 msgr2=0x7f15d4110730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:56.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.551+0000 7f15d3400700 1 --2- 172.21.15.19:0/3298337073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d4106580 0x7f15d4110730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:56.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.551+0000 7f15d3400700 1 -- 172.21.15.19:0/3298337073 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d41ad580 con 0x7f15d410c5c0 2024-05-07T02:42:56.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.552+0000 7f15d3400700 1 --2- 172.21.15.19:0/3298337073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15d410c5c0 0x7f15d41111b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f15cc00d010 tx=0x7f15cc004c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:42:56.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.552+0000 7f15cb400700 1 -- 172.21.15.19:0/3298337073 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc021070 con 0x7f15d410c5c0 2024-05-07T02:42:56.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.552+0000 7f15d9600700 1 -- 172.21.15.19:0/3298337073 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15d41ad810 con 0x7f15d410c5c0 2024-05-07T02:42:56.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.552+0000 7f15cb400700 1 -- 172.21.15.19:0/3298337073 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15cc007e30 con 0x7f15d410c5c0 2024-05-07T02:42:56.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.552+0000 7f15cb400700 1 -- 172.21.15.19:0/3298337073 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc0045a0 con 0x7f15d410c5c0 2024-05-07T02:42:56.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.552+0000 7f15d9600700 1 -- 172.21.15.19:0/3298337073 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15d41add10 con 0x7f15d410c5c0 2024-05-07T02:42:56.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.556+0000 7f15cb400700 1 -- 172.21.15.19:0/3298337073 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f15cc004740 con 0x7f15d410c5c0 2024-05-07T02:42:56.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.557+0000 7f15e8150b80 1 Processor -- start 2024-05-07T02:42:56.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.558+0000 7f15e8150b80 1 -- start start 2024-05-07T02:42:56.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.558+0000 7f15e8150b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573cf84c3b0 0x5573cf84c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:56.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.558+0000 7f15e8150b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5573cf846470 0x5573cf846880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:56.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.558+0000 7f15e8150b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5573cf846dc0 0x5573cf853040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:56.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.558+0000 7f15e8150b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5573cf768a20 con 0x5573cf84c3b0 2024-05-07T02:42:56.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.558+0000 7f15e8150b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5573cf706760 con 0x5573cf846dc0 2024-05-07T02:42:56.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.558+0000 7f15e8150b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5573cf7649e0 con 0x5573cf846470 2024-05-07T02:42:56.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.559+0000 7f15cb400700 1 --2- 172.21.15.19:0/3298337073 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f15b80b29f0 0x7f15b80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:56.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.559+0000 7f15d2000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573cf84c3b0 0x5573cf84c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:56.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.559+0000 7f15d2000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573cf84c3b0 0x5573cf84c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49354/0 (socket says 172.21.15.19:49354) 2024-05-07T02:42:56.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.559+0000 7f15d2000700 1 -- 172.21.15.19:0/351583743 learned_addr learned my addr 172.21.15.19:0/351583743 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:42:56.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.559+0000 7f15d2a00700 1 --2- 172.21.15.19:0/351583743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5573cf846470 0x5573cf846880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:56.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.559+0000 7f15d2000700 1 --2- 172.21.15.19:0/351583743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5573cf846dc0 0x5573cf853040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:56.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.559+0000 7f15d3400700 1 --2- 172.21.15.19:0/3298337073 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f15b80b29f0 0x7f15b80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:56.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.559+0000 7f15cb400700 1 -- 172.21.15.19:0/3298337073 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9525+0+0 (secure 0 0 0) 0x7f15cc00f020 con 0x7f15d410c5c0 2024-05-07T02:42:56.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.559+0000 7f15d2a00700 1 -- 172.21.15.19:0/351583743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5573cf846dc0 msgr2=0x5573cf853040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:56.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.559+0000 7f15d2a00700 1 --2- 172.21.15.19:0/351583743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5573cf846dc0 0x5573cf853040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:56.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.559+0000 7f15d2a00700 1 -- 172.21.15.19:0/351583743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573cf84c3b0 msgr2=0x5573cf84c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:56.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.559+0000 7f15d2a00700 1 --2- 172.21.15.19:0/351583743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573cf84c3b0 0x5573cf84c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:56.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.559+0000 7f15d2a00700 1 -- 172.21.15.19:0/351583743 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573cf6b58d0 con 0x5573cf846470 2024-05-07T02:42:56.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.560+0000 7f15d2000700 1 --2- 172.21.15.19:0/351583743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573cf84c3b0 0x5573cf84c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:42:56.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.560+0000 7f15d2a00700 1 --2- 172.21.15.19:0/351583743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5573cf846470 0x5573cf846880 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f15c4005950 tx=0x7f15c4011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf8130b9974a6775 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:42:56.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.560+0000 7f15d3400700 1 --2- 172.21.15.19:0/3298337073 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f15b80b29f0 0x7f15b80b4f00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f15cc0a7f80 tx=0x7f15cc009040 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:42:56.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.560+0000 7f15bfe00700 1 -- 172.21.15.19:0/351583743 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c4004250 con 0x5573cf846470 2024-05-07T02:42:56.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.560+0000 7f15bfe00700 1 -- 172.21.15.19:0/351583743 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15c40043f0 con 0x5573cf846470 2024-05-07T02:42:56.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.560+0000 7f15bfe00700 1 -- 172.21.15.19:0/351583743 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c40055d0 con 0x5573cf846470 2024-05-07T02:42:56.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.561+0000 7f15e8150b80 1 -- 172.21.15.19:0/351583743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5573cf846470 msgr2=0x5573cf846880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:56.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.561+0000 7f15e8150b80 1 --2- 172.21.15.19:0/351583743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5573cf846470 0x5573cf846880 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f15c4005950 tx=0x7f15c4011720 comp rx=0 tx=0).stop 2024-05-07T02:42:56.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.561+0000 7f15e8150b80 1 -- 172.21.15.19:0/351583743 shutdown_connections 2024-05-07T02:42:56.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.561+0000 7f15e8150b80 1 --2- 172.21.15.19:0/351583743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5573cf846dc0 0x5573cf853040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:56.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.561+0000 7f15e8150b80 1 --2- 172.21.15.19:0/351583743 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573cf84c3b0 0x5573cf84c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:56.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.561+0000 7f15e8150b80 1 --2- 172.21.15.19:0/351583743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5573cf846470 0x5573cf846880 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:56.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.561+0000 7f15e8150b80 1 -- 172.21.15.19:0/351583743 >> 172.21.15.19:0/351583743 conn(0x5573cf6c8e40 msgr2=0x5573cf8412f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:42:56.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.561+0000 7f15e8150b80 1 -- 172.21.15.19:0/351583743 shutdown_connections 2024-05-07T02:42:56.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.562+0000 7f15e8150b80 1 -- 172.21.15.19:0/351583743 wait complete. 2024-05-07T02:42:56.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.562+0000 7f15e8150b80 1 -- start start 2024-05-07T02:42:56.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.562+0000 7f15e8150b80 1 Processor -- start 2024-05-07T02:42:56.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.563+0000 7f15e8150b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573cf846470 0x5573cf850520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:56.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.563+0000 7f15e8150b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5573cf846dc0 0x5573cf850a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:56.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.563+0000 7f15e8150b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5573cf84c3b0 0x5573cf850fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:56.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.563+0000 7f15e8150b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5573cf8537c0 con 0x5573cf846470 2024-05-07T02:42:56.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.563+0000 7f15e8150b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x5573cf6ceb20 con 0x5573cf846dc0 2024-05-07T02:42:56.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.563+0000 7f15e8150b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5573cf853940 con 0x5573cf84c3b0 2024-05-07T02:42:56.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.563+0000 7f15e8150b80 10 client.0 _ll_register_callbacks cb 0x5573cf70fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:42:56.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.563+0000 7f15d2000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573cf846470 0x5573cf850520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:56.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.563+0000 7f15d2a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5573cf846dc0 0x5573cf850a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:56.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.563+0000 7f15d2000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573cf846470 0x5573cf850520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49370/0 (socket says 172.21.15.19:49370) 2024-05-07T02:42:56.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.563+0000 7f15d2000700 1 -- 172.21.15.19:0/3404151506 learned_addr learned my addr 172.21.15.19:0/3404151506 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:42:56.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.564+0000 7f15d2000700 1 --2- 172.21.15.19:0/3404151506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5573cf84c3b0 0x5573cf850fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:56.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.564+0000 7f15d2000700 1 -- 172.21.15.19:0/3404151506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5573cf84c3b0 msgr2=0x5573cf850fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:56.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.564+0000 7f15d2000700 1 --2- 172.21.15.19:0/3404151506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5573cf84c3b0 0x5573cf850fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:56.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.564+0000 7f15d2000700 1 -- 172.21.15.19:0/3404151506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5573cf846dc0 msgr2=0x5573cf850a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:56.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.564+0000 7f15d2000700 1 --2- 172.21.15.19:0/3404151506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5573cf846dc0 0x5573cf850a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:56.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.564+0000 7f15d2000700 1 -- 172.21.15.19:0/3404151506 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573cf72e420 con 0x5573cf846470 2024-05-07T02:42:56.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.564+0000 7f15d2a00700 1 --2- 172.21.15.19:0/3404151506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5573cf846dc0 0x5573cf850a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:42:56.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.564+0000 7f15d2000700 1 --2- 172.21.15.19:0/3404151506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573cf846470 0x5573cf850520 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f15c0013430 tx=0x7f15c00058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b6c258ea47c1aa7 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:42:56.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.564+0000 7f15bcc00700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-07T02:42:56.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.564+0000 7f15bcc00700 1 -- 172.21.15.19:0/3404151506 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c0011d20 con 0x5573cf846470 2024-05-07T02:42:56.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.565+0000 7f15bcc00700 1 -- 172.21.15.19:0/3404151506 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9525+0+0 (secure 0 0 0) 0x7f15c0004030 con 0x5573cf846470 2024-05-07T02:42:56.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.565+0000 7f15e8150b80 1 -- 172.21.15.19:0/3404151506 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15c40043f0 con 0x5573cf846470 2024-05-07T02:42:56.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.565+0000 7f15bcc00700 1 -- 172.21.15.19:0/3404151506 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f15c002a050 con 0x5573cf846470 2024-05-07T02:42:56.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.565+0000 7f15e8150b80 10 client.16653 fetch_fsmap learned FSMap version 50 2024-05-07T02:42:56.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.565+0000 7f15e8150b80 1 -- 172.21.15.19:0/3404151506 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5573cf85e410 con 0x5573cf846470 2024-05-07T02:42:56.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.565+0000 7f15bcc00700 1 -- 172.21.15.19:0/3404151506 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15c40043f0 con 0x5573cf846470 2024-05-07T02:42:56.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.565+0000 7f15bcc00700 1 -- 172.21.15.19:0/3404151506 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 50) ==== 8362+0+0 (secure 0 0 0) 0x7f15c00253d0 con 0x5573cf846470 2024-05-07T02:42:56.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.566+0000 7f15e8150b80 10 client.16653 fetch_fsmap finished waiting for FSMap version 50 2024-05-07T02:42:56.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.566+0000 7f15e8150b80 10 client.16653 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-07T02:42:56.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.566+0000 7f15e8150b80 20 client.16653 populate_metadata read hostname 'smithi019' 2024-05-07T02:42:56.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.566+0000 7f15e8150b80 1 --2- 172.21.15.19:0/3404151506 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5573cf860a50 0x5573cf862ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:56.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.566+0000 7f15e8150b80 4 client.16653 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:42:56.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.566+0000 7f15e8150b80 1 -- 172.21.15.19:0/3404151506 --> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573cf8132e0 con 0x5573cf860a50 2024-05-07T02:42:56.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.566+0000 7f15d2a00700 1 --2- 172.21.15.19:0/3404151506 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5573cf860a50 0x5573cf862ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:56.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.567+0000 7f15bcc00700 10 client.16653 ms_handle_connect on v2:172.21.15.19:6840/3905645246 2024-05-07T02:42:56.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.567+0000 7f15d2a00700 1 --2- 172.21.15.19:0/3404151506 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5573cf860a50 0x5573cf862ef0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4856ad3643b20468 server_cookie=8cfaa11f80e80dc2 in_seq=0 out_seq=0 2024-05-07T02:42:56.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.568+0000 7f15bcc00700 1 -- 172.21.15.19:0/3404151506 <== mds.1 v2:172.21.15.19:6840/3905645246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5573cf8132e0 con 0x5573cf860a50 2024-05-07T02:42:56.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.568+0000 7f15bcc00700 10 client.16653 handle_command_reply: tid=0 2024-05-07T02:42:56.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.568+0000 7f15e8150b80 1 client.16653 shutdown 2024-05-07T02:42:56.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.568+0000 7f15e8150b80 1 -- 172.21.15.19:0/3404151506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573cf846470 msgr2=0x5573cf850520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:56.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.568+0000 7f15e8150b80 1 --2- 172.21.15.19:0/3404151506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573cf846470 0x5573cf850520 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f15c0013430 tx=0x7f15c00058e0 comp rx=0 tx=0).stop 2024-05-07T02:42:56.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.568+0000 7f15e8150b80 1 -- 172.21.15.19:0/3404151506 shutdown_connections 2024-05-07T02:42:56.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.568+0000 7f15e8150b80 1 --2- 172.21.15.19:0/3404151506 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5573cf860a50 0x5573cf862ef0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:56.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.568+0000 7f15e8150b80 1 --2- 172.21.15.19:0/3404151506 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5573cf84c3b0 0x5573cf850fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:56.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.568+0000 7f15e8150b80 1 --2- 172.21.15.19:0/3404151506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573cf846470 0x5573cf850520 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:56.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.568+0000 7f15e8150b80 1 --2- 172.21.15.19:0/3404151506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5573cf846dc0 0x5573cf850a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:56.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.568+0000 7f15e8150b80 1 -- 172.21.15.19:0/3404151506 >> 172.21.15.19:0/3404151506 conn(0x5573cf6c8e40 msgr2=0x5573cf845460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:42:56.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.568+0000 7f15bcc00700 0 client.16653 ms_handle_reset on v2:172.21.15.19:6840/3905645246 2024-05-07T02:42:56.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.568+0000 7f15e8150b80 1 -- 172.21.15.19:0/3404151506 shutdown_connections 2024-05-07T02:42:56.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.568+0000 7f15e8150b80 1 -- 172.21.15.19:0/3404151506 wait complete. 2024-05-07T02:42:56.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.569+0000 7f15e8150b80 20 client.16653 trim_cache size 0 max 16384 2024-05-07T02:42:56.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.586+0000 7f15e8150b80 1 Processor -- start 2024-05-07T02:42:56.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.586+0000 7f15e8150b80 1 -- start start 2024-05-07T02:42:56.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.586+0000 7f15e8150b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5573cf846470 0x5573cf85c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:56.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.586+0000 7f15e8150b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5573cf846dc0 0x5573cf8686b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:56.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.586+0000 7f15e8150b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573cf84c3b0 0x5573cf868bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:56.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.586+0000 7f15e8150b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f15d41b6b70 con 0x5573cf84c3b0 2024-05-07T02:42:56.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.586+0000 7f15e8150b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f15d41b86b0 con 0x5573cf846470 2024-05-07T02:42:56.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.586+0000 7f15e8150b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f15d41b7240 con 0x5573cf846dc0 2024-05-07T02:42:56.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.587+0000 7f15d2a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5573cf846dc0 0x5573cf8686b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:56.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.587+0000 7f15d2000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5573cf846470 0x5573cf85c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:56.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.587+0000 7f15d2a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5573cf846dc0 0x5573cf8686b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:36336/0 (socket says 172.21.15.19:36336) 2024-05-07T02:42:56.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.587+0000 7f15d2a00700 1 -- 172.21.15.19:0/691398470 learned_addr learned my addr 172.21.15.19:0/691398470 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:42:56.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.587+0000 7f15d2000700 1 --2- 172.21.15.19:0/691398470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573cf84c3b0 0x5573cf868bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:56.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.587+0000 7f15d2a00700 1 -- 172.21.15.19:0/691398470 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5573cf846470 msgr2=0x5573cf85c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:56.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.587+0000 7f15d2a00700 1 --2- 172.21.15.19:0/691398470 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5573cf846470 0x5573cf85c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:56.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.587+0000 7f15d2a00700 1 -- 172.21.15.19:0/691398470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573cf84c3b0 msgr2=0x5573cf868bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:56.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.587+0000 7f15d2a00700 1 --2- 172.21.15.19:0/691398470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573cf84c3b0 0x5573cf868bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:56.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.587+0000 7f15d2a00700 1 -- 172.21.15.19:0/691398470 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573cf84ea90 con 0x5573cf846dc0 2024-05-07T02:42:56.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.587+0000 7f15d2000700 1 --2- 172.21.15.19:0/691398470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573cf84c3b0 0x5573cf868bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:42:56.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.587+0000 7f15d2a00700 1 --2- 172.21.15.19:0/691398470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5573cf846dc0 0x5573cf8686b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f15c4009660 tx=0x7f15c4005e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=925fd9545bed719d server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:42:56.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.588+0000 7f15bf400700 1 -- 172.21.15.19:0/691398470 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c40043a0 con 0x5573cf846dc0 2024-05-07T02:42:56.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.588+0000 7f15bf400700 1 -- 172.21.15.19:0/691398470 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15c4030070 con 0x5573cf846dc0 2024-05-07T02:42:56.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.588+0000 7f15bf400700 1 -- 172.21.15.19:0/691398470 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c4003cc0 con 0x5573cf846dc0 2024-05-07T02:42:56.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.588+0000 7f15e8150b80 1 -- 172.21.15.19:0/691398470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5573cf846dc0 msgr2=0x5573cf8686b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:56.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.588+0000 7f15e8150b80 1 --2- 172.21.15.19:0/691398470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5573cf846dc0 0x5573cf8686b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f15c4009660 tx=0x7f15c4005e00 comp rx=0 tx=0).stop 2024-05-07T02:42:56.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.589+0000 7f15e8150b80 1 -- 172.21.15.19:0/691398470 shutdown_connections 2024-05-07T02:42:56.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.589+0000 7f15e8150b80 1 --2- 172.21.15.19:0/691398470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573cf84c3b0 0x5573cf868bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:56.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.589+0000 7f15e8150b80 1 --2- 172.21.15.19:0/691398470 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5573cf846470 0x5573cf85c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:56.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.589+0000 7f15e8150b80 1 --2- 172.21.15.19:0/691398470 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5573cf846dc0 0x5573cf8686b0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:56.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.589+0000 7f15e8150b80 1 -- 172.21.15.19:0/691398470 >> 172.21.15.19:0/691398470 conn(0x5573cf6c8e40 msgr2=0x5573cf84b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:42:56.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.589+0000 7f15e8150b80 1 -- 172.21.15.19:0/691398470 shutdown_connections 2024-05-07T02:42:56.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.589+0000 7f15e8150b80 1 -- 172.21.15.19:0/691398470 wait complete. 2024-05-07T02:42:56.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.589+0000 7f15e8150b80 1 -- start start 2024-05-07T02:42:56.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.589+0000 7f15e8150b80 1 Processor -- start 2024-05-07T02:42:56.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.590+0000 7f15e8150b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5573cf846470 0x5573cf858bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:56.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.590+0000 7f15e8150b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573cf846dc0 0x5573cf859110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:56.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.590+0000 7f15e8150b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5573cf84c3b0 0x5573cf84c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:56.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.590+0000 7f15e8150b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f15d41b7fe0 con 0x5573cf846dc0 2024-05-07T02:42:56.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.590+0000 7f15e8150b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f15d41b7910 con 0x5573cf846470 2024-05-07T02:42:56.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.590+0000 7f15e8150b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f15d41b4b30 con 0x5573cf84c3b0 2024-05-07T02:42:56.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.590+0000 7f15e8150b80 10 client.0 _ll_register_callbacks cb 0x5573cf842120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:42:56.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.590+0000 7f15d2a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573cf846dc0 0x5573cf859110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:56.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.590+0000 7f15d2000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5573cf846470 0x5573cf858bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:56.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.591+0000 7f15d2a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573cf846dc0 0x5573cf859110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49400/0 (socket says 172.21.15.19:49400) 2024-05-07T02:42:56.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.591+0000 7f15d2a00700 1 -- 172.21.15.19:0/4008565631 learned_addr learned my addr 172.21.15.19:0/4008565631 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:42:56.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.591+0000 7f15d2000700 1 --2- 172.21.15.19:0/4008565631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5573cf84c3b0 0x5573cf84c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:56.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.591+0000 7f15d2a00700 1 -- 172.21.15.19:0/4008565631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5573cf84c3b0 msgr2=0x5573cf84c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:56.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.591+0000 7f15d2a00700 1 --2- 172.21.15.19:0/4008565631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5573cf84c3b0 0x5573cf84c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:56.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.591+0000 7f15d2a00700 1 -- 172.21.15.19:0/4008565631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5573cf846470 msgr2=0x5573cf858bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:42:56.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.591+0000 7f15d2a00700 1 --2- 172.21.15.19:0/4008565631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5573cf846470 0x5573cf858bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:56.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.591+0000 7f15d2a00700 1 -- 172.21.15.19:0/4008565631 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573cf84cd40 con 0x5573cf846dc0 2024-05-07T02:42:56.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.591+0000 7f15d2000700 1 --2- 172.21.15.19:0/4008565631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5573cf846470 0x5573cf858bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:42:56.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.591+0000 7f15d2a00700 1 --2- 172.21.15.19:0/4008565631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573cf846dc0 0x5573cf859110 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f15c4003ae0 tx=0x7f15c4009350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92138af4b9c3eb34 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:42:56.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.591+0000 7f15bcc00700 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-05-07T02:42:56.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.592+0000 7f15bcc00700 1 -- 172.21.15.19:0/4008565631 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c4003740 con 0x5573cf846dc0 2024-05-07T02:42:56.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.592+0000 7f15e8150b80 1 -- 172.21.15.19:0/4008565631 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15c4030070 con 0x5573cf846dc0 2024-05-07T02:42:56.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.592+0000 7f15bcc00700 1 -- 172.21.15.19:0/4008565631 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9525+0+0 (secure 0 0 0) 0x7f15c4003b40 con 0x5573cf846dc0 2024-05-07T02:42:56.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.592+0000 7f15bcc00700 1 -- 172.21.15.19:0/4008565631 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f15c4003d60 con 0x5573cf846dc0 2024-05-07T02:42:56.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.592+0000 7f15e8150b80 10 client.16659 fetch_fsmap learned FSMap version 50 2024-05-07T02:42:56.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.592+0000 7f15e8150b80 1 -- 172.21.15.19:0/4008565631 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5573cf8ca8e0 con 0x5573cf846dc0 2024-05-07T02:42:56.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.592+0000 7f15bcc00700 1 -- 172.21.15.19:0/4008565631 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15c4030070 con 0x5573cf846dc0 2024-05-07T02:42:56.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.593+0000 7f15bcc00700 1 -- 172.21.15.19:0/4008565631 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 50) ==== 8362+0+0 (secure 0 0 0) 0x7f15c4018070 con 0x5573cf846dc0 2024-05-07T02:42:56.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.593+0000 7f15e8150b80 10 client.16659 fetch_fsmap finished waiting for FSMap version 50 2024-05-07T02:42:56.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.593+0000 7f15e8150b80 10 client.16659 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-05-07T02:42:56.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.593+0000 7f15e8150b80 20 client.16659 populate_metadata read hostname 'smithi019' 2024-05-07T02:42:56.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.593+0000 7f15e8150b80 1 --2- 172.21.15.19:0/4008565631 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5573cf8cacf0 0x5573cf8cd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:42:56.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.593+0000 7f15e8150b80 4 client.16659 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:42:56.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.593+0000 7f15e8150b80 1 -- 172.21.15.19:0/4008565631 --> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5573cf776400 con 0x5573cf8cacf0 2024-05-07T02:42:56.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.593+0000 7f15d2000700 1 --2- 172.21.15.19:0/4008565631 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5573cf8cacf0 0x5573cf8cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:42:56.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.594+0000 7f15d2000700 1 --2- 172.21.15.19:0/4008565631 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5573cf8cacf0 0x5573cf8cd110 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=772d4d0a21862561 server_cookie=372a3e32900a8dfb in_seq=0 out_seq=0 2024-05-07T02:42:56.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.594+0000 7f15bcc00700 10 client.16659 ms_handle_connect on v2:172.21.15.19:6840/3905645246 2024-05-07T02:42:56.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.594+0000 7f15bcc00700 1 -- 172.21.15.19:0/4008565631 <== mds.1 v2:172.21.15.19:6840/3905645246 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5573cf776400 con 0x5573cf8cacf0 2024-05-07T02:42:56.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.594+0000 7f15bcc00700 10 client.16659 handle_command_reply: tid=0 2024-05-07T02:42:56.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.594+0000 7f15e8150b80 1 client.16659 shutdown 2024-05-07T02:42:56.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.594+0000 7f15e8150b80 1 -- 172.21.15.19:0/4008565631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573cf846dc0 msgr2=0x5573cf859110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:56.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.594+0000 7f15e8150b80 1 --2- 172.21.15.19:0/4008565631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573cf846dc0 0x5573cf859110 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f15c4003ae0 tx=0x7f15c4009350 comp rx=0 tx=0).stop 2024-05-07T02:42:56.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.594+0000 7f15e8150b80 1 -- 172.21.15.19:0/4008565631 shutdown_connections 2024-05-07T02:42:56.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.594+0000 7f15e8150b80 1 --2- 172.21.15.19:0/4008565631 >> [v2:172.21.15.19:6840/3905645246,v1:172.21.15.19:6841/3905645246] conn(0x5573cf8cacf0 0x5573cf8cd110 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:56.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.594+0000 7f15e8150b80 1 --2- 172.21.15.19:0/4008565631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5573cf84c3b0 0x5573cf84c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:56.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.594+0000 7f15e8150b80 1 --2- 172.21.15.19:0/4008565631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5573cf846470 0x5573cf858bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:56.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.594+0000 7f15e8150b80 1 --2- 172.21.15.19:0/4008565631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5573cf846dc0 0x5573cf859110 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:56.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.594+0000 7f15e8150b80 1 -- 172.21.15.19:0/4008565631 >> 172.21.15.19:0/4008565631 conn(0x5573cf6c8e40 msgr2=0x5573cf83b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:42:56.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.595+0000 7f15bcc00700 0 client.16659 ms_handle_reset on v2:172.21.15.19:6840/3905645246 2024-05-07T02:42:56.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.595+0000 7f15e8150b80 1 -- 172.21.15.19:0/4008565631 shutdown_connections 2024-05-07T02:42:56.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.595+0000 7f15e8150b80 1 -- 172.21.15.19:0/4008565631 wait complete. 2024-05-07T02:42:56.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.595+0000 7f15e8150b80 20 client.16659 trim_cache size 0 max 16384 2024-05-07T02:42:56.601 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:42:56.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.595+0000 7f15d9600700 1 -- 172.21.15.19:0/3298337073 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f15b80b29f0 msgr2=0x7f15b80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:56.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.595+0000 7f15d9600700 1 --2- 172.21.15.19:0/3298337073 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f15b80b29f0 0x7f15b80b4f00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f15cc0a7f80 tx=0x7f15cc009040 comp rx=0 tx=0).stop 2024-05-07T02:42:56.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.595+0000 7f15d9600700 1 -- 172.21.15.19:0/3298337073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15d410c5c0 msgr2=0x7f15d41111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:42:56.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.595+0000 7f15d9600700 1 --2- 172.21.15.19:0/3298337073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15d410c5c0 0x7f15d41111b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f15cc00d010 tx=0x7f15cc004c80 comp rx=0 tx=0).stop 2024-05-07T02:42:56.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.596+0000 7f15d9600700 1 -- 172.21.15.19:0/3298337073 shutdown_connections 2024-05-07T02:42:56.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.596+0000 7f15d9600700 1 --2- 172.21.15.19:0/3298337073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f15d410c5c0 0x7f15d41111b0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:56.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.596+0000 7f15d9600700 1 --2- 172.21.15.19:0/3298337073 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f15b80b29f0 0x7f15b80b4f00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:56.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.596+0000 7f15d9600700 1 --2- 172.21.15.19:0/3298337073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15d4106580 0x7f15d4110730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:56.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.596+0000 7f15d9600700 1 --2- 172.21.15.19:0/3298337073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15d4106ed0 0x7f15d4110c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:42:56.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.596+0000 7f15d9600700 1 -- 172.21.15.19:0/3298337073 >> 172.21.15.19:0/3298337073 conn(0x7f15d4102080 msgr2=0x7f15d4104650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:42:56.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.596+0000 7f15d9600700 1 -- 172.21.15.19:0/3298337073 shutdown_connections 2024-05-07T02:42:56.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:42:56.596+0000 7f15d9600700 1 -- 172.21.15.19:0/3298337073 wait complete. 2024-05-07T02:42:57.456 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-07T02:42:57.962 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:42:58.271 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:42:57 smithi019 ceph-mon[29781]: pgmap v344: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:42:58.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:42:57 smithi064 ceph-mon[34699]: pgmap v344: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:42:58.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:42:57 smithi027 ceph-mon[34398]: pgmap v344: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:43:00.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.350+0000 7fcd3e200700 1 -- 172.21.15.19:0/2312570394 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd381031b0 msgr2=0x7fcd38111110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:00.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.350+0000 7fcd3e200700 1 --2- 172.21.15.19:0/2312570394 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd381031b0 0x7fcd38111110 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fcd3000b0a0 tx=0x7fcd3000b3b0 comp rx=0 tx=0).stop 2024-05-07T02:43:00.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.350+0000 7fcd3e200700 1 -- 172.21.15.19:0/2312570394 shutdown_connections 2024-05-07T02:43:00.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.350+0000 7fcd3e200700 1 --2- 172.21.15.19:0/2312570394 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd381031b0 0x7fcd38111110 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:00.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.350+0000 7fcd3e200700 1 --2- 172.21.15.19:0/2312570394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd3810a3d0 0x7fcd3810a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:00.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.350+0000 7fcd3e200700 1 --2- 172.21.15.19:0/2312570394 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd38102860 0x7fcd38102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:00.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.350+0000 7fcd3e200700 1 -- 172.21.15.19:0/2312570394 >> 172.21.15.19:0/2312570394 conn(0x7fcd380fe6f0 msgr2=0x7fcd38100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:43:00.352 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:43:00 smithi019 ceph-mon[29781]: pgmap v345: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:43:00.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.351+0000 7fcd3e200700 1 -- 172.21.15.19:0/2312570394 shutdown_connections 2024-05-07T02:43:00.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.351+0000 7fcd3e200700 1 -- 172.21.15.19:0/2312570394 wait complete. 2024-05-07T02:43:00.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.352+0000 7fcd3e200700 1 Processor -- start 2024-05-07T02:43:00.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.352+0000 7fcd3e200700 1 -- start start 2024-05-07T02:43:00.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.353+0000 7fcd3e200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd38102860 0x7fcd38103a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:00.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.353+0000 7fcd3e200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd381031b0 0x7fcd38103f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:00.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.353+0000 7fcd3e200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd3810a3d0 0x7fcd381044c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:00.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.353+0000 7fcd3e200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcd381144c0 con 0x7fcd3810a3d0 2024-05-07T02:43:00.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.353+0000 7fcd3e200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcd38114340 con 0x7fcd38102860 2024-05-07T02:43:00.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.353+0000 7fcd3e200700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcd38114640 con 0x7fcd381031b0 2024-05-07T02:43:00.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.353+0000 7fcd37400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd38102860 0x7fcd38103a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:00.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.353+0000 7fcd37400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd38102860 0x7fcd38103a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:33372/0 (socket says 172.21.15.19:33372) 2024-05-07T02:43:00.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.353+0000 7fcd37400700 1 -- 172.21.15.19:0/741754585 learned_addr learned my addr 172.21.15.19:0/741754585 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:43:00.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.354+0000 7fcd37e00700 1 --2- 172.21.15.19:0/741754585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd3810a3d0 0x7fcd381044c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:00.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.354+0000 7fcd36a00700 1 --2- 172.21.15.19:0/741754585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd381031b0 0x7fcd38103f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:00.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.354+0000 7fcd37400700 1 -- 172.21.15.19:0/741754585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd381031b0 msgr2=0x7fcd38103f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:00.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.354+0000 7fcd37400700 1 --2- 172.21.15.19:0/741754585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd381031b0 0x7fcd38103f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:00.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.354+0000 7fcd37400700 1 -- 172.21.15.19:0/741754585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd3810a3d0 msgr2=0x7fcd381044c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:00.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.354+0000 7fcd37400700 1 --2- 172.21.15.19:0/741754585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd3810a3d0 0x7fcd381044c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:00.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.354+0000 7fcd37400700 1 -- 172.21.15.19:0/741754585 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd381a9230 con 0x7fcd38102860 2024-05-07T02:43:00.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.354+0000 7fcd37400700 1 --2- 172.21.15.19:0/741754585 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd38102860 0x7fcd38103a40 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fcd2800d980 tx=0x7fcd2800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:43:00.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.355+0000 7fcd2fe00700 1 -- 172.21.15.19:0/741754585 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd28016070 con 0x7fcd38102860 2024-05-07T02:43:00.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.355+0000 7fcd2fe00700 1 -- 172.21.15.19:0/741754585 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd28004830 con 0x7fcd38102860 2024-05-07T02:43:00.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.355+0000 7fcd3e200700 1 -- 172.21.15.19:0/741754585 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd381a9520 con 0x7fcd38102860 2024-05-07T02:43:00.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.355+0000 7fcd2fe00700 1 -- 172.21.15.19:0/741754585 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd28005540 con 0x7fcd38102860 2024-05-07T02:43:00.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.355+0000 7fcd3e200700 1 -- 172.21.15.19:0/741754585 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd381a9a60 con 0x7fcd38102860 2024-05-07T02:43:00.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:43:00 smithi064 ceph-mon[34699]: pgmap v345: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:43:00.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.360+0000 7fcd2fe00700 1 -- 172.21.15.19:0/741754585 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fcd28004220 con 0x7fcd38102860 2024-05-07T02:43:00.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.361+0000 7fcd4ce15b80 1 Processor -- start 2024-05-07T02:43:00.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.361+0000 7fcd4ce15b80 1 -- start start 2024-05-07T02:43:00.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.362+0000 7fcd4ce15b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566251a73b0 0x5566251a77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:00.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.362+0000 7fcd4ce15b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566251a1470 0x5566251a1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:00.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.362+0000 7fcd4ce15b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566251a1dc0 0x5566251ae040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:00.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.362+0000 7fcd4ce15b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5566250c3a20 con 0x5566251a73b0 2024-05-07T02:43:00.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.362+0000 7fcd4ce15b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556625061760 con 0x5566251a1dc0 2024-05-07T02:43:00.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.362+0000 7fcd4ce15b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5566250bf9e0 con 0x5566251a1470 2024-05-07T02:43:00.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.362+0000 7fcd2fe00700 1 --2- 172.21.15.19:0/741754585 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fcd1c0b29f0 0x7fcd1c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:00.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.362+0000 7fcd36a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566251a1dc0 0x5566251ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:00.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.362+0000 7fcd37e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566251a73b0 0x5566251a77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:00.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.362+0000 7fcd36a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566251a1dc0 0x5566251ae040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:33376/0 (socket says 172.21.15.19:33376) 2024-05-07T02:43:00.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.362+0000 7fcd36a00700 1 -- 172.21.15.19:0/3935961941 learned_addr learned my addr 172.21.15.19:0/3935961941 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:43:00.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.363+0000 7fcd2fe00700 1 -- 172.21.15.19:0/741754585 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9525+0+0 (secure 0 0 0) 0x7fcd280e27a0 con 0x7fcd38102860 2024-05-07T02:43:00.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.363+0000 7fcd37e00700 1 --2- 172.21.15.19:0/741754585 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fcd1c0b29f0 0x7fcd1c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:00.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.363+0000 7fcd37400700 1 --2- 172.21.15.19:0/3935961941 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566251a1470 0x5566251a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:00.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.363+0000 7fcd36a00700 1 -- 172.21.15.19:0/3935961941 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566251a1470 msgr2=0x5566251a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:43:00.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.363+0000 7fcd36a00700 1 --2- 172.21.15.19:0/3935961941 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566251a1470 0x5566251a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:00.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.363+0000 7fcd36a00700 1 -- 172.21.15.19:0/3935961941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566251a73b0 msgr2=0x5566251a77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:43:00.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.363+0000 7fcd36a00700 1 --2- 172.21.15.19:0/3935961941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566251a73b0 0x5566251a77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:00.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.363+0000 7fcd36a00700 1 -- 172.21.15.19:0/3935961941 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566250108d0 con 0x5566251a1dc0 2024-05-07T02:43:00.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.363+0000 7fcd37400700 1 --2- 172.21.15.19:0/3935961941 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566251a1470 0x5566251a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-07T02:43:00.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.364+0000 7fcd36a00700 1 --2- 172.21.15.19:0/3935961941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566251a1dc0 0x5566251ae040 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fcd24009cf0 tx=0x7fcd24011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9ef0e14f352932a server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:43:00.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.364+0000 7fcd37e00700 1 --2- 172.21.15.19:0/741754585 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fcd1c0b29f0 0x7fcd1c0b4f00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcd3000b780 tx=0x7fcd30009c40 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:43:00.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.364+0000 7fcd2cc00700 1 -- 172.21.15.19:0/3935961941 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd24011cc0 con 0x5566251a1dc0 2024-05-07T02:43:00.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.364+0000 7fcd2cc00700 1 -- 172.21.15.19:0/3935961941 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd24004500 con 0x5566251a1dc0 2024-05-07T02:43:00.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.364+0000 7fcd2cc00700 1 -- 172.21.15.19:0/3935961941 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd24019640 con 0x5566251a1dc0 2024-05-07T02:43:00.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.365+0000 7fcd4ce15b80 1 -- 172.21.15.19:0/3935961941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566251a1dc0 msgr2=0x5566251ae040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:00.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.365+0000 7fcd4ce15b80 1 --2- 172.21.15.19:0/3935961941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566251a1dc0 0x5566251ae040 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fcd24009cf0 tx=0x7fcd24011440 comp rx=0 tx=0).stop 2024-05-07T02:43:00.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.365+0000 7fcd4ce15b80 1 -- 172.21.15.19:0/3935961941 shutdown_connections 2024-05-07T02:43:00.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.365+0000 7fcd4ce15b80 1 --2- 172.21.15.19:0/3935961941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566251a1dc0 0x5566251ae040 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:00.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.365+0000 7fcd4ce15b80 1 --2- 172.21.15.19:0/3935961941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566251a73b0 0x5566251a77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:00.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.365+0000 7fcd4ce15b80 1 --2- 172.21.15.19:0/3935961941 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566251a1470 0x5566251a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:00.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.365+0000 7fcd4ce15b80 1 -- 172.21.15.19:0/3935961941 >> 172.21.15.19:0/3935961941 conn(0x556625023e40 msgr2=0x55662519c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:43:00.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.365+0000 7fcd4ce15b80 1 -- 172.21.15.19:0/3935961941 shutdown_connections 2024-05-07T02:43:00.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.366+0000 7fcd4ce15b80 1 -- 172.21.15.19:0/3935961941 wait complete. 2024-05-07T02:43:00.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.366+0000 7fcd4ce15b80 1 -- start start 2024-05-07T02:43:00.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.367+0000 7fcd4ce15b80 1 Processor -- start 2024-05-07T02:43:00.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.368+0000 7fcd4ce15b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566251a1470 0x5566251ab520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:00.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.368+0000 7fcd4ce15b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566251a1dc0 0x5566251aba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:00.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.368+0000 7fcd4ce15b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566251a73b0 0x5566251abfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:00.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.368+0000 7fcd4ce15b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5566251ae7c0 con 0x5566251a73b0 2024-05-07T02:43:00.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.368+0000 7fcd4ce15b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x556625029b20 con 0x5566251a1dc0 2024-05-07T02:43:00.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.368+0000 7fcd4ce15b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5566251ae940 con 0x5566251a1470 2024-05-07T02:43:00.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.368+0000 7fcd4ce15b80 10 client.0 _ll_register_callbacks cb 0x55662506ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:43:00.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.368+0000 7fcd36a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566251a1dc0 0x5566251aba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:00.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.368+0000 7fcd37400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566251a1470 0x5566251ab520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:00.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.368+0000 7fcd36a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566251a1dc0 0x5566251aba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:33388/0 (socket says 172.21.15.19:33388) 2024-05-07T02:43:00.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.368+0000 7fcd36a00700 1 -- 172.21.15.19:0/521008598 learned_addr learned my addr 172.21.15.19:0/521008598 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:43:00.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.368+0000 7fcd37e00700 1 --2- 172.21.15.19:0/521008598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566251a73b0 0x5566251abfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:00.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.369+0000 7fcd36a00700 1 -- 172.21.15.19:0/521008598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566251a1470 msgr2=0x5566251ab520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:43:00.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.369+0000 7fcd36a00700 1 --2- 172.21.15.19:0/521008598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566251a1470 0x5566251ab520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:00.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.369+0000 7fcd36a00700 1 -- 172.21.15.19:0/521008598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566251a73b0 msgr2=0x5566251abfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:43:00.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.369+0000 7fcd36a00700 1 --2- 172.21.15.19:0/521008598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566251a73b0 0x5566251abfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:00.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.369+0000 7fcd36a00700 1 -- 172.21.15.19:0/521008598 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556625089420 con 0x5566251a1dc0 2024-05-07T02:43:00.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.369+0000 7fcd37400700 1 --2- 172.21.15.19:0/521008598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566251a1470 0x5566251ab520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:43:00.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.369+0000 7fcd36a00700 1 --2- 172.21.15.19:0/521008598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566251a1dc0 0x5566251aba60 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcd24000c00 tx=0x7fcd240057e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e41a93cb8953143 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:43:00.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.369+0000 7fcd21600700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:43:00.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.370+0000 7fcd21600700 1 -- 172.21.15.19:0/521008598 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd24004260 con 0x5566251a1dc0 2024-05-07T02:43:00.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.370+0000 7fcd21600700 1 -- 172.21.15.19:0/521008598 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9525+0+0 (secure 0 0 0) 0x7fcd24011ac0 con 0x5566251a1dc0 2024-05-07T02:43:00.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.370+0000 7fcd4ce15b80 1 -- 172.21.15.19:0/521008598 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd24004500 con 0x5566251a1dc0 2024-05-07T02:43:00.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.370+0000 7fcd21600700 1 -- 172.21.15.19:0/521008598 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fcd24031050 con 0x5566251a1dc0 2024-05-07T02:43:00.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.370+0000 7fcd4ce15b80 10 client.26876 fetch_fsmap learned FSMap version 50 2024-05-07T02:43:00.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.370+0000 7fcd4ce15b80 1 -- 172.21.15.19:0/521008598 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5566251c1bd0 con 0x5566251a1dc0 2024-05-07T02:43:00.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.370+0000 7fcd21600700 1 -- 172.21.15.19:0/521008598 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd24004500 con 0x5566251a1dc0 2024-05-07T02:43:00.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.371+0000 7fcd21600700 1 -- 172.21.15.19:0/521008598 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 50) ==== 8362+0+0 (secure 0 0 0) 0x7fcd24018070 con 0x5566251a1dc0 2024-05-07T02:43:00.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.371+0000 7fcd4ce15b80 10 client.26876 fetch_fsmap finished waiting for FSMap version 50 2024-05-07T02:43:00.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.371+0000 7fcd4ce15b80 10 client.26876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:43:00.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.371+0000 7fcd4ce15b80 20 client.26876 populate_metadata read hostname 'smithi019' 2024-05-07T02:43:00.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.371+0000 7fcd4ce15b80 1 --2- 172.21.15.19:0/521008598 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5566251c40f0 0x5566251c6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:00.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.371+0000 7fcd4ce15b80 4 client.26876 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-07T02:43:00.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.371+0000 7fcd4ce15b80 1 -- 172.21.15.19:0/521008598 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55662516e2e0 con 0x5566251c40f0 2024-05-07T02:43:00.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.371+0000 7fcd37400700 1 --2- 172.21.15.19:0/521008598 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5566251c40f0 0x5566251c6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:00.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.372+0000 7fcd37400700 1 --2- 172.21.15.19:0/521008598 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5566251c40f0 0x5566251c6510 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=390f8ad372891977 server_cookie=ca25dcbb4ee8dee5 in_seq=0 out_seq=0 2024-05-07T02:43:00.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.372+0000 7fcd21600700 10 client.26876 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:43:00.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.373+0000 7fcd21600700 1 -- 172.21.15.19:0/521008598 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55662516e2e0 con 0x5566251c40f0 2024-05-07T02:43:00.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.373+0000 7fcd21600700 10 client.26876 handle_command_reply: tid=0 2024-05-07T02:43:00.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.373+0000 7fcd4ce15b80 1 client.26876 shutdown 2024-05-07T02:43:00.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.373+0000 7fcd4ce15b80 1 -- 172.21.15.19:0/521008598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566251a1dc0 msgr2=0x5566251aba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:00.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.373+0000 7fcd4ce15b80 1 --2- 172.21.15.19:0/521008598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566251a1dc0 0x5566251aba60 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcd24000c00 tx=0x7fcd240057e0 comp rx=0 tx=0).stop 2024-05-07T02:43:00.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.374+0000 7fcd4ce15b80 1 -- 172.21.15.19:0/521008598 shutdown_connections 2024-05-07T02:43:00.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.374+0000 7fcd4ce15b80 1 --2- 172.21.15.19:0/521008598 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x5566251c40f0 0x5566251c6510 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:00.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.374+0000 7fcd4ce15b80 1 --2- 172.21.15.19:0/521008598 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566251a73b0 0x5566251abfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:00.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.374+0000 7fcd4ce15b80 1 --2- 172.21.15.19:0/521008598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566251a1470 0x5566251ab520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:00.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.374+0000 7fcd4ce15b80 1 --2- 172.21.15.19:0/521008598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566251a1dc0 0x5566251aba60 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:00.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.374+0000 7fcd4ce15b80 1 -- 172.21.15.19:0/521008598 >> 172.21.15.19:0/521008598 conn(0x556625023e40 msgr2=0x5566251a0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:43:00.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.374+0000 7fcd21600700 0 client.26876 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:43:00.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.374+0000 7fcd4ce15b80 1 -- 172.21.15.19:0/521008598 shutdown_connections 2024-05-07T02:43:00.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.374+0000 7fcd4ce15b80 1 -- 172.21.15.19:0/521008598 wait complete. 2024-05-07T02:43:00.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.374+0000 7fcd4ce15b80 20 client.26876 trim_cache size 0 max 16384 2024-05-07T02:43:00.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.395+0000 7fcd4ce15b80 1 Processor -- start 2024-05-07T02:43:00.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.395+0000 7fcd4ce15b80 1 -- start start 2024-05-07T02:43:00.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.395+0000 7fcd4ce15b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566251a1470 0x5566251abde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:00.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.395+0000 7fcd4ce15b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566251a1dc0 0x5566251a8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:00.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.395+0000 7fcd4ce15b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566251a73b0 0x5566251a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:00.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.395+0000 7fcd4ce15b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcd381ae5b0 con 0x5566251a1470 2024-05-07T02:43:00.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.395+0000 7fcd4ce15b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcd381b00f0 con 0x5566251a73b0 2024-05-07T02:43:00.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.395+0000 7fcd4ce15b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcd381aec80 con 0x5566251a1dc0 2024-05-07T02:43:00.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.396+0000 7fcd36a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566251a1dc0 0x5566251a8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:00.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.396+0000 7fcd36a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566251a1dc0 0x5566251a8f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:38238/0 (socket says 172.21.15.19:38238) 2024-05-07T02:43:00.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.396+0000 7fcd36a00700 1 -- 172.21.15.19:0/3726535581 learned_addr learned my addr 172.21.15.19:0/3726535581 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:43:00.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.396+0000 7fcd37e00700 1 --2- 172.21.15.19:0/3726535581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566251a73b0 0x5566251a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:00.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.396+0000 7fcd36a00700 1 -- 172.21.15.19:0/3726535581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566251a73b0 msgr2=0x5566251a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:43:00.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.396+0000 7fcd36a00700 1 --2- 172.21.15.19:0/3726535581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566251a73b0 0x5566251a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:00.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.396+0000 7fcd37400700 1 --2- 172.21.15.19:0/3726535581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566251a1470 0x5566251abde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:00.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.396+0000 7fcd36a00700 1 -- 172.21.15.19:0/3726535581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566251a1470 msgr2=0x5566251abde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:43:00.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.396+0000 7fcd36a00700 1 --2- 172.21.15.19:0/3726535581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566251a1470 0x5566251abde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:00.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.396+0000 7fcd36a00700 1 -- 172.21.15.19:0/3726535581 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55662519c830 con 0x5566251a1dc0 2024-05-07T02:43:00.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.396+0000 7fcd37e00700 1 --2- 172.21.15.19:0/3726535581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566251a73b0 0x5566251a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-07T02:43:00.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.397+0000 7fcd36a00700 1 --2- 172.21.15.19:0/3726535581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566251a1dc0 0x5566251a8f10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcd24011480 tx=0x7fcd24009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73f74a1c10697f4a server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:43:00.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.397+0000 7fcd23e00700 1 -- 172.21.15.19:0/3726535581 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd24003e00 con 0x5566251a1dc0 2024-05-07T02:43:00.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.397+0000 7fcd23e00700 1 -- 172.21.15.19:0/3726535581 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd24028070 con 0x5566251a1dc0 2024-05-07T02:43:00.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.397+0000 7fcd23e00700 1 -- 172.21.15.19:0/3726535581 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd24004fe0 con 0x5566251a1dc0 2024-05-07T02:43:00.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.398+0000 7fcd4ce15b80 1 -- 172.21.15.19:0/3726535581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566251a1dc0 msgr2=0x5566251a8f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:00.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.398+0000 7fcd4ce15b80 1 --2- 172.21.15.19:0/3726535581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566251a1dc0 0x5566251a8f10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcd24011480 tx=0x7fcd24009bf0 comp rx=0 tx=0).stop 2024-05-07T02:43:00.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.398+0000 7fcd4ce15b80 1 -- 172.21.15.19:0/3726535581 shutdown_connections 2024-05-07T02:43:00.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.398+0000 7fcd4ce15b80 1 --2- 172.21.15.19:0/3726535581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566251a73b0 0x5566251a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:00.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.398+0000 7fcd4ce15b80 1 --2- 172.21.15.19:0/3726535581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566251a1470 0x5566251abde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:00.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.398+0000 7fcd4ce15b80 1 --2- 172.21.15.19:0/3726535581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566251a1dc0 0x5566251a8f10 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:00.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.398+0000 7fcd4ce15b80 1 -- 172.21.15.19:0/3726535581 >> 172.21.15.19:0/3726535581 conn(0x556625023e40 msgr2=0x5566251a0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:43:00.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.398+0000 7fcd4ce15b80 1 -- 172.21.15.19:0/3726535581 shutdown_connections 2024-05-07T02:43:00.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.398+0000 7fcd4ce15b80 1 -- 172.21.15.19:0/3726535581 wait complete. 2024-05-07T02:43:00.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.399+0000 7fcd4ce15b80 1 -- start start 2024-05-07T02:43:00.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.399+0000 7fcd4ce15b80 1 Processor -- start 2024-05-07T02:43:00.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.400+0000 7fcd4ce15b80 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566251a1470 0x5566251b7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:00.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.400+0000 7fcd4ce15b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566251a1dc0 0x5566251b7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:00.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.400+0000 7fcd4ce15b80 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566251a73b0 0x5566251b7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:00.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.400+0000 7fcd4ce15b80 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcd381afa20 con 0x5566251a1470 2024-05-07T02:43:00.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.400+0000 7fcd4ce15b80 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fcd381af350 con 0x5566251a73b0 2024-05-07T02:43:00.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.400+0000 7fcd4ce15b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcd381ac570 con 0x5566251a1dc0 2024-05-07T02:43:00.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.400+0000 7fcd4ce15b80 10 client.0 _ll_register_callbacks cb 0x55662506ac70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-07T02:43:00.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.400+0000 7fcd37400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566251a1470 0x5566251b7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:00.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.400+0000 7fcd36a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566251a1dc0 0x5566251b7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:00.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.400+0000 7fcd37400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566251a1470 0x5566251b7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57642/0 (socket says 172.21.15.19:57642) 2024-05-07T02:43:00.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.400+0000 7fcd37400700 1 -- 172.21.15.19:0/3954697866 learned_addr learned my addr 172.21.15.19:0/3954697866 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:43:00.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.400+0000 7fcd37e00700 1 --2- 172.21.15.19:0/3954697866 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566251a73b0 0x5566251b7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:00.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.401+0000 7fcd37e00700 1 -- 172.21.15.19:0/3954697866 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566251a1dc0 msgr2=0x5566251b7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:43:00.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.401+0000 7fcd37e00700 1 --2- 172.21.15.19:0/3954697866 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566251a1dc0 0x5566251b7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:00.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.401+0000 7fcd37e00700 1 -- 172.21.15.19:0/3954697866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566251a1470 msgr2=0x5566251b7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-07T02:43:00.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.401+0000 7fcd37e00700 1 --2- 172.21.15.19:0/3954697866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566251a1470 0x5566251b7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:00.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.401+0000 7fcd37e00700 1 -- 172.21.15.19:0/3954697866 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55662522c6b0 con 0x5566251a73b0 2024-05-07T02:43:00.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.401+0000 7fcd37e00700 1 --2- 172.21.15.19:0/3954697866 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566251a73b0 0x5566251b7ea0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fcd300062a0 tx=0x7fcd30026000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=716b6b77410cf3a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:43:00.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.401+0000 7fcd21600700 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2024-05-07T02:43:00.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.402+0000 7fcd21600700 1 -- 172.21.15.19:0/3954697866 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd30020660 con 0x5566251a73b0 2024-05-07T02:43:00.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.402+0000 7fcd4ce15b80 1 -- 172.21.15.19:0/3954697866 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd24028070 con 0x5566251a73b0 2024-05-07T02:43:00.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.402+0000 7fcd21600700 1 -- 172.21.15.19:0/3954697866 <== mon.2 v2:172.21.15.27:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9525+0+0 (secure 0 0 0) 0x7fcd30020800 con 0x5566251a73b0 2024-05-07T02:43:00.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.402+0000 7fcd21600700 1 -- 172.21.15.19:0/3954697866 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fcd3003b050 con 0x5566251a73b0 2024-05-07T02:43:00.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.402+0000 7fcd4ce15b80 10 client.26879 fetch_fsmap learned FSMap version 50 2024-05-07T02:43:00.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.402+0000 7fcd4ce15b80 1 -- 172.21.15.19:0/3954697866 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55662522d710 con 0x5566251a73b0 2024-05-07T02:43:00.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.402+0000 7fcd21600700 1 -- 172.21.15.19:0/3954697866 <== mon.2 v2:172.21.15.27:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd24028070 con 0x5566251a73b0 2024-05-07T02:43:00.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.403+0000 7fcd21600700 1 -- 172.21.15.19:0/3954697866 <== mon.2 v2:172.21.15.27:3300/0 5 ==== fsmap(e 50) ==== 8362+0+0 (secure 0 0 0) 0x7fcd30011070 con 0x5566251a73b0 2024-05-07T02:43:00.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.403+0000 7fcd4ce15b80 10 client.26879 fetch_fsmap finished waiting for FSMap version 50 2024-05-07T02:43:00.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.403+0000 7fcd4ce15b80 10 client.26879 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-07T02:43:00.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.403+0000 7fcd4ce15b80 20 client.26879 populate_metadata read hostname 'smithi019' 2024-05-07T02:43:00.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.403+0000 7fcd4ce15b80 1 --2- 172.21.15.19:0/3954697866 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55662522ddf0 0x556625230210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:00.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.403+0000 7fcd4ce15b80 4 client.26879 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-07T02:43:00.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.403+0000 7fcd4ce15b80 1 -- 172.21.15.19:0/3954697866 --> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5566250d1400 con 0x55662522ddf0 2024-05-07T02:43:00.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.403+0000 7fcd37400700 1 --2- 172.21.15.19:0/3954697866 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55662522ddf0 0x556625230210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:00.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.404+0000 7fcd37400700 1 --2- 172.21.15.19:0/3954697866 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55662522ddf0 0x556625230210 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94cfa4cc455caccd server_cookie=3067976b006e3ae1 in_seq=0 out_seq=0 2024-05-07T02:43:00.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.404+0000 7fcd21600700 10 client.26879 ms_handle_connect on v2:172.21.15.27:6838/3963268634 2024-05-07T02:43:00.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.404+0000 7fcd21600700 1 -- 172.21.15.19:0/3954697866 <== mds.0 v2:172.21.15.27:6838/3963268634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5566250d1400 con 0x55662522ddf0 2024-05-07T02:43:00.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.404+0000 7fcd21600700 10 client.26879 handle_command_reply: tid=0 2024-05-07T02:43:00.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.404+0000 7fcd4ce15b80 1 client.26879 shutdown 2024-05-07T02:43:00.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.404+0000 7fcd4ce15b80 1 -- 172.21.15.19:0/3954697866 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566251a73b0 msgr2=0x5566251b7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:00.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.404+0000 7fcd4ce15b80 1 --2- 172.21.15.19:0/3954697866 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566251a73b0 0x5566251b7ea0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fcd300062a0 tx=0x7fcd30026000 comp rx=0 tx=0).stop 2024-05-07T02:43:00.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.404+0000 7fcd4ce15b80 1 -- 172.21.15.19:0/3954697866 shutdown_connections 2024-05-07T02:43:00.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.404+0000 7fcd4ce15b80 1 --2- 172.21.15.19:0/3954697866 >> [v2:172.21.15.27:6838/3963268634,v1:172.21.15.27:6839/3963268634] conn(0x55662522ddf0 0x556625230210 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:00.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.404+0000 7fcd4ce15b80 1 --2- 172.21.15.19:0/3954697866 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5566251a73b0 0x5566251b7ea0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:00.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.404+0000 7fcd4ce15b80 1 --2- 172.21.15.19:0/3954697866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5566251a1470 0x5566251b7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:00.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.404+0000 7fcd4ce15b80 1 --2- 172.21.15.19:0/3954697866 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5566251a1dc0 0x5566251b7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:00.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.404+0000 7fcd21600700 0 client.26879 ms_handle_reset on v2:172.21.15.27:6838/3963268634 2024-05-07T02:43:00.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.404+0000 7fcd4ce15b80 1 -- 172.21.15.19:0/3954697866 >> 172.21.15.19:0/3954697866 conn(0x556625023e40 msgr2=0x5566251abcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:43:00.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.405+0000 7fcd4ce15b80 1 -- 172.21.15.19:0/3954697866 shutdown_connections 2024-05-07T02:43:00.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.405+0000 7fcd4ce15b80 1 -- 172.21.15.19:0/3954697866 wait complete. 2024-05-07T02:43:00.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.405+0000 7fcd4ce15b80 20 client.26879 trim_cache size 0 max 16384 2024-05-07T02:43:00.409 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-05-07T02:43:00.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.406+0000 7fcd3e200700 1 -- 172.21.15.19:0/741754585 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fcd1c0b29f0 msgr2=0x7fcd1c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:00.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.406+0000 7fcd3e200700 1 --2- 172.21.15.19:0/741754585 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fcd1c0b29f0 0x7fcd1c0b4f00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcd3000b780 tx=0x7fcd30009c40 comp rx=0 tx=0).stop 2024-05-07T02:43:00.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.406+0000 7fcd3e200700 1 -- 172.21.15.19:0/741754585 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd38102860 msgr2=0x7fcd38103a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:00.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.406+0000 7fcd3e200700 1 --2- 172.21.15.19:0/741754585 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd38102860 0x7fcd38103a40 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fcd2800d980 tx=0x7fcd2800dc90 comp rx=0 tx=0).stop 2024-05-07T02:43:00.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.406+0000 7fcd3e200700 1 -- 172.21.15.19:0/741754585 shutdown_connections 2024-05-07T02:43:00.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.406+0000 7fcd3e200700 1 --2- 172.21.15.19:0/741754585 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fcd1c0b29f0 0x7fcd1c0b4f00 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:00.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.406+0000 7fcd3e200700 1 --2- 172.21.15.19:0/741754585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd3810a3d0 0x7fcd381044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:00.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.406+0000 7fcd3e200700 1 --2- 172.21.15.19:0/741754585 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd38102860 0x7fcd38103a40 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:00.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.406+0000 7fcd3e200700 1 --2- 172.21.15.19:0/741754585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcd381031b0 0x7fcd38103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:00.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.406+0000 7fcd3e200700 1 -- 172.21.15.19:0/741754585 >> 172.21.15.19:0/741754585 conn(0x7fcd380fe6f0 msgr2=0x7fcd38100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:43:00.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.406+0000 7fcd3e200700 1 -- 172.21.15.19:0/741754585 shutdown_connections 2024-05-07T02:43:00.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:00.406+0000 7fcd3e200700 1 -- 172.21.15.19:0/741754585 wait complete. 2024-05-07T02:43:00.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:43:00 smithi027 ceph-mon[34398]: pgmap v345: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:43:01.444 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/github.com_batrick_ceph_366c9130c497dee9e73178650cf710ed21269b18/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/github.com_batrick_ceph_366c9130c497dee9e73178650cf710ed21269b18/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/github.com_batrick_ceph_366c9130c497dee9e73178650cf710ed21269b18/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-05-07T02:43:01.450 INFO:tasks.fwd_scrub:done joining 2024-05-07T02:43:01.450 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-07T02:43:01.461 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-07T02:43:01.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.755+0000 7ff3a1a00640 1 -- 172.21.15.19:0/1622823175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff39c1041d0 msgr2=0x7ff39c1105a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:01.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.755+0000 7ff3a1a00640 1 --2- 172.21.15.19:0/1622823175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff39c1041d0 0x7ff39c1105a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff394009ca0 tx=0x7ff39402a8c0 comp rx=0 tx=0).stop 2024-05-07T02:43:01.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.755+0000 7ff3a1a00640 1 -- 172.21.15.19:0/1622823175 shutdown_connections 2024-05-07T02:43:01.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.755+0000 7ff3a1a00640 1 --2- 172.21.15.19:0/1622823175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff39c1041d0 0x7ff39c1105a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:01.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.755+0000 7ff3a1a00640 1 --2- 172.21.15.19:0/1622823175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff39c109f20 0x7ff39c103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:01.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.755+0000 7ff3a1a00640 1 --2- 172.21.15.19:0/1622823175 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff39c109a80 0x7ff39c103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:01.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.755+0000 7ff3a1a00640 1 -- 172.21.15.19:0/1622823175 >> 172.21.15.19:0/1622823175 conn(0x7ff39c0ff160 msgr2=0x7ff39c1015b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:43:01.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.756+0000 7ff3a1a00640 1 -- 172.21.15.19:0/1622823175 shutdown_connections 2024-05-07T02:43:01.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.756+0000 7ff3a1a00640 1 -- 172.21.15.19:0/1622823175 wait complete. 2024-05-07T02:43:01.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.757+0000 7ff3a1a00640 1 Processor -- start 2024-05-07T02:43:01.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.757+0000 7ff3a1a00640 1 -- start start 2024-05-07T02:43:01.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.757+0000 7ff3a1a00640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff39c1041d0 0x7ff39c0760e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:01.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.757+0000 7ff3a1a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff39c109a80 0x7ff39c076620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:01.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.757+0000 7ff3a1a00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff39c109f20 0x7ff39c076b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:01.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.757+0000 7ff3a1a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff39c113cc0 con 0x7ff39c109a80 2024-05-07T02:43:01.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.757+0000 7ff3a1a00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7ff39c113b40 con 0x7ff39c109f20 2024-05-07T02:43:01.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.757+0000 7ff3a1a00640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff39c113e40 con 0x7ff39c1041d0 2024-05-07T02:43:01.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.758+0000 7ff39b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff39c109a80 0x7ff39c076620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:01.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.758+0000 7ff39be00640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff39c1041d0 0x7ff39c0760e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:01.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.758+0000 7ff39b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff39c109a80 0x7ff39c076620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57658/0 (socket says 172.21.15.19:57658) 2024-05-07T02:43:01.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.758+0000 7ff39b400640 1 -- 172.21.15.19:0/1026088863 learned_addr learned my addr 172.21.15.19:0/1026088863 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:43:01.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.758+0000 7ff3a1000640 1 --2- 172.21.15.19:0/1026088863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff39c109f20 0x7ff39c076b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:01.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.758+0000 7ff39be00640 1 -- 172.21.15.19:0/1026088863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff39c109f20 msgr2=0x7ff39c076b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:01.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.758+0000 7ff39be00640 1 --2- 172.21.15.19:0/1026088863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff39c109f20 0x7ff39c076b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:01.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.758+0000 7ff39be00640 1 -- 172.21.15.19:0/1026088863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff39c109a80 msgr2=0x7ff39c076620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:01.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.758+0000 7ff39be00640 1 --2- 172.21.15.19:0/1026088863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff39c109a80 0x7ff39c076620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:01.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.758+0000 7ff39be00640 1 -- 172.21.15.19:0/1026088863 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff39c1a88c0 con 0x7ff39c1041d0 2024-05-07T02:43:01.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.759+0000 7ff39be00640 1 --2- 172.21.15.19:0/1026088863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff39c1041d0 0x7ff39c0760e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff38c00e9c0 tx=0x7ff38c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:43:01.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.759+0000 7ff398c00640 1 -- 172.21.15.19:0/1026088863 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c00cd80 con 0x7ff39c1041d0 2024-05-07T02:43:01.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.759+0000 7ff3a1a00640 1 -- 172.21.15.19:0/1026088863 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff39c1a8bb0 con 0x7ff39c1041d0 2024-05-07T02:43:01.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.759+0000 7ff398c00640 1 -- 172.21.15.19:0/1026088863 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff38c0035f0 con 0x7ff39c1041d0 2024-05-07T02:43:01.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.759+0000 7ff398c00640 1 -- 172.21.15.19:0/1026088863 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c010690 con 0x7ff39c1041d0 2024-05-07T02:43:01.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.759+0000 7ff3a1a00640 1 -- 172.21.15.19:0/1026088863 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff39c1a9150 con 0x7ff39c1041d0 2024-05-07T02:43:01.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.760+0000 7ff3a1a00640 1 -- 172.21.15.19:0/1026088863 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff39c06a2d0 con 0x7ff39c1041d0 2024-05-07T02:43:01.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.763+0000 7ff398c00640 1 -- 172.21.15.19:0/1026088863 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff38c0037a0 con 0x7ff39c1041d0 2024-05-07T02:43:01.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.764+0000 7ff398c00640 1 --2- 172.21.15.19:0/1026088863 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff3800b2a10 0x7ff3800b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:01.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.765+0000 7ff398c00640 1 -- 172.21.15.19:0/1026088863 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9525+0+0 (secure 0 0 0) 0x7ff38c0e2e00 con 0x7ff39c1041d0 2024-05-07T02:43:01.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.765+0000 7ff39b400640 1 --2- 172.21.15.19:0/1026088863 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff3800b2a10 0x7ff3800b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:01.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.766+0000 7ff39b400640 1 --2- 172.21.15.19:0/1026088863 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff3800b2a10 0x7ff3800b4f20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff39c107b90 tx=0x7ff388005f90 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:43:01.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.767+0000 7ff398c00640 1 -- 172.21.15.19:0/1026088863 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff38c0aa420 con 0x7ff39c1041d0 2024-05-07T02:43:01.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.984+0000 7ff3a1a00640 1 -- 172.21.15.19:0/1026088863 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff39c077610 con 0x7ff39c1041d0 2024-05-07T02:43:01.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.987+0000 7ff398c00640 1 -- 172.21.15.19:0/1026088863 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+25281 (secure 0 0 0) 0x7ff38c0a9d20 con 0x7ff39c1041d0 2024-05-07T02:43:01.988 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:43:01.988 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":98,"fsid":"9ff84c16-0c17-11ef-bc97-c7b262605968","created":"2024-05-07T02:16:19.507657+0000","modified":"2024-05-07T02:42:17.341804+0000","last_up_change":"2024-05-07T02:24:55.025124+0000","last_in_change":"2024-05-07T02:24:33.842422+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-07T02:20:21.439697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-07T02:26:06.178814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-07T02:26:10.705693+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":97,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-07T02:26:23.534133+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":97,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b437cf22-ec8e-48f0-86a2-2413a0ab40dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6803","nonce":2390539683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6805","nonce":2390539683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6809","nonce":2390539683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6807","nonce":2390539683}]},"public_addr":"172.21.15.19:6803/2390539683","cluster_addr":"172.21.15.19:6805/2390539683","heartbeat_back_addr":"172.21.15.19:6809/2390539683","heartbeat_front_addr":"172.21.15.19:6807/2390539683","state":["exists","up"]},{"osd":1,"uuid":"dc753e16-634d-4c14-abd2-935281e1499e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6801","nonce":4292470615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6803","nonce":4292470615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6807","nonce":4292470615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6805","nonce":4292470615}]},"public_addr":"172.21.15.27:6801/4292470615","cluster_addr":"172.21.15.27:6803/4292470615","heartbeat_back_addr":"172.21.15.27:6807/4292470615","heartbeat_front_addr":"172.21.15.27:6805/4292470615","state":["exists","up"]},{"osd":2,"uuid":"40570071-646e-4e66-9a16-c55ab743ac86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6801","nonce":1180552939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6803","nonce":1180552939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6807","nonce":1180552939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6805","nonce":1180552939}]},"public_addr":"172.21.15.64:6801/1180552939","cluster_addr":"172.21.15.64:6803/1180552939","heartbeat_back_addr":"172.21.15.64:6807/1180552939","heartbeat_front_addr":"172.21.15.64:6805/1180552939","state":["exists","up"]},{"osd":3,"uuid":"ceaa4252-4dfa-427a-9b1a-76adf3b8e609","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6811","nonce":387118141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6813","nonce":387118141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6817","nonce":387118141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6815","nonce":387118141}]},"public_addr":"172.21.15.19:6811/387118141","cluster_addr":"172.21.15.19:6813/387118141","heartbeat_back_addr":"172.21.15.19:6817/387118141","heartbeat_front_addr":"172.21.15.19:6815/387118141","state":["exists","up"]},{"osd":4,"uuid":"5a6d1ebc-62ca-444e-8f29-1980418d0a8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6809","nonce":3662186068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6811","nonce":3662186068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6815","nonce":3662186068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6813","nonce":3662186068}]},"public_addr":"172.21.15.27:6809/3662186068","cluster_addr":"172.21.15.27:6811/3662186068","heartbeat_back_addr":"172.21.15.27:6815/3662186068","heartbeat_front_addr":"172.21.15.27:6813/3662186068","state":["exists","up"]},{"osd":5,"uuid":"60c207e9-55e8-4bed-87c3-f6a06a2a1fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6809","nonce":1264591849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6811","nonce":1264591849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6815","nonce":1264591849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6813","nonce":1264591849}]},"public_addr":"172.21.15.64:6809/1264591849","cluster_addr":"172.21.15.64:6811/1264591849","heartbeat_back_addr":"172.21.15.64:6815/1264591849","heartbeat_front_addr":"172.21.15.64:6813/1264591849","state":["exists","up"]},{"osd":6,"uuid":"f3e038e0-4026-45f1-a923-c07cbb307f64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6819","nonce":2158875543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6821","nonce":2158875543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6825","nonce":2158875543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6823","nonce":2158875543}]},"public_addr":"172.21.15.19:6819/2158875543","cluster_addr":"172.21.15.19:6821/2158875543","heartbeat_back_addr":"172.21.15.19:6825/2158875543","heartbeat_front_addr":"172.21.15.19:6823/2158875543","state":["exists","up"]},{"osd":7,"uuid":"d69e34d4-2028-43b8-8ada-ff4a9b579620","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6817","nonce":2756612091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6819","nonce":2756612091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6823","nonce":2756612091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6821","nonce":2756612091}]},"public_addr":"172.21.15.27:6817/2756612091","cluster_addr":"172.21.15.27:6819/2756612091","heartbeat_back_addr":"172.21.15.27:6823/2756612091","heartbeat_front_addr":"172.21.15.27:6821/2756612091","state":["exists","up"]},{"osd":8,"uuid":"6c1a4179-75b6-42ff-ac29-378801e6f8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6817","nonce":1987024836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6819","nonce":1987024836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6823","nonce":1987024836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6821","nonce":1987024836}]},"public_addr":"172.21.15.64:6817/1987024836","cluster_addr":"172.21.15.64:6819/1987024836","heartbeat_back_addr":"172.21.15.64:6823/1987024836","heartbeat_front_addr":"172.21.15.64:6821/1987024836","state":["exists","up"]},{"osd":9,"uuid":"9df7ff66-d39f-4709-8324-74cf9e0e052d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6827","nonce":4124921646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6829","nonce":4124921646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6833","nonce":4124921646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6831","nonce":4124921646}]},"public_addr":"172.21.15.19:6827/4124921646","cluster_addr":"172.21.15.19:6829/4124921646","heartbeat_back_addr":"172.21.15.19:6833/4124921646","heartbeat_front_addr":"172.21.15.19:6831/4124921646","state":["exists","up"]},{"osd":10,"uuid":"71eaa9a7-667d-4cb7-935c-42177d20a279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6825","nonce":1428931817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6827","nonce":1428931817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6831","nonce":1428931817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6829","nonce":1428931817}]},"public_addr":"172.21.15.27:6825/1428931817","cluster_addr":"172.21.15.27:6827/1428931817","heartbeat_back_addr":"172.21.15.27:6831/1428931817","heartbeat_front_addr":"172.21.15.27:6829/1428931817","state":["exists","up"]},{"osd":11,"uuid":"3330627f-ab5d-4e6e-83cf-fc8b74c54ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6825","nonce":4045070679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6827","nonce":4045070679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6831","nonce":4045070679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6829","nonce":4045070679}]},"public_addr":"172.21.15.64:6825/4045070679","cluster_addr":"172.21.15.64:6827/4045070679","heartbeat_back_addr":"172.21.15.64:6831/4045070679","heartbeat_front_addr":"172.21.15.64:6829/4045070679","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:19:27.902633+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:19:54.305685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:20:18.727053+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:20:53.579199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:21:21.058018+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:21:47.091494+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:22:23.269018+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:22:51.863628+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:23:18.537774+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:23:55.763993+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:24:24.956178+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:24:52.681201+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/1821831886":"2024-05-08T02:31:49.815663+0000","172.21.15.19:6801/2059564551":"2024-05-08T02:31:49.815663+0000","172.21.15.19:6800/2059564551":"2024-05-08T02:31:49.815663+0000","172.21.15.19:0/1956516258":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/1816772181":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/686554066":"2024-05-08T02:31:49.815663+0000","172.21.15.19:0/2602127700":"2024-05-08T02:31:49.815663+0000","172.21.15.19:6801/174304059":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/3466480009":"2024-05-08T02:31:49.815663+0000","172.21.15.19:0/1514051750":"2024-05-08T02:16:54.732592+0000","172.21.15.19:6800/174304059":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/3930045871":"2024-05-08T02:16:54.732592+0000","172.21.15.19:6800/3614957309":"2024-05-08T02:16:54.732592+0000","172.21.15.19:0/1328365399":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/139593344":"2024-05-08T02:16:54.732592+0000","172.21.15.19:6801/3614957309":"2024-05-08T02:16:54.732592+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-07T02:43:01.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.990+0000 7ff3a1a00640 1 -- 172.21.15.19:0/1026088863 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff3800b2a10 msgr2=0x7ff3800b4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:01.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.990+0000 7ff3a1a00640 1 --2- 172.21.15.19:0/1026088863 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff3800b2a10 0x7ff3800b4f20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff39c107b90 tx=0x7ff388005f90 comp rx=0 tx=0).stop 2024-05-07T02:43:01.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.990+0000 7ff3a1a00640 1 -- 172.21.15.19:0/1026088863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff39c1041d0 msgr2=0x7ff39c0760e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:01.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.990+0000 7ff3a1a00640 1 --2- 172.21.15.19:0/1026088863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff39c1041d0 0x7ff39c0760e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff38c00e9c0 tx=0x7ff38c00ee80 comp rx=0 tx=0).stop 2024-05-07T02:43:01.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.990+0000 7ff3a1a00640 1 -- 172.21.15.19:0/1026088863 shutdown_connections 2024-05-07T02:43:01.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.990+0000 7ff3a1a00640 1 --2- 172.21.15.19:0/1026088863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff39c109f20 0x7ff39c076b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:01.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.990+0000 7ff3a1a00640 1 --2- 172.21.15.19:0/1026088863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff39c109a80 0x7ff39c076620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:01.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.990+0000 7ff3a1a00640 1 --2- 172.21.15.19:0/1026088863 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7ff3800b2a10 0x7ff3800b4f20 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:01.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.990+0000 7ff3a1a00640 1 --2- 172.21.15.19:0/1026088863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff39c1041d0 0x7ff39c0760e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:01.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.990+0000 7ff3a1a00640 1 -- 172.21.15.19:0/1026088863 >> 172.21.15.19:0/1026088863 conn(0x7ff39c0ff160 msgr2=0x7ff39c1061e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:43:01.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.990+0000 7ff3a1a00640 1 -- 172.21.15.19:0/1026088863 shutdown_connections 2024-05-07T02:43:01.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:01.990+0000 7ff3a1a00640 1 -- 172.21.15.19:0/1026088863 wait complete. 2024-05-07T02:43:02.019 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-07T02:20:21.439697+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-07T02:26:06.178814+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-07T02:26:10.705693+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 97, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-07T02:26:23.534133+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 97, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-07T02:43:02.020 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-07T02:43:02.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.310+0000 7f0426000640 1 -- 172.21.15.19:0/963722027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f042010c140 msgr2=0x7f0420105d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:02.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.310+0000 7f0426000640 1 --2- 172.21.15.19:0/963722027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f042010c140 0x7f0420105d80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0410009900 tx=0x7f041002a410 comp rx=0 tx=0).stop 2024-05-07T02:43:02.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.311+0000 7f0426000640 1 -- 172.21.15.19:0/963722027 shutdown_connections 2024-05-07T02:43:02.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.311+0000 7f0426000640 1 --2- 172.21.15.19:0/963722027 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04201063f0 0x7f04201127c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:02.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.311+0000 7f0426000640 1 --2- 172.21.15.19:0/963722027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f042010c140 0x7f0420105d80 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:02.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.311+0000 7f0426000640 1 --2- 172.21.15.19:0/963722027 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f042010bca0 0x7f0420105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:02.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.311+0000 7f0426000640 1 -- 172.21.15.19:0/963722027 >> 172.21.15.19:0/963722027 conn(0x7f0420101300 msgr2=0x7f0420103770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:43:02.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.311+0000 7f0426000640 1 -- 172.21.15.19:0/963722027 shutdown_connections 2024-05-07T02:43:02.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.311+0000 7f0426000640 1 -- 172.21.15.19:0/963722027 wait complete. 2024-05-07T02:43:02.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.312+0000 7f0426000640 1 Processor -- start 2024-05-07T02:43:02.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.312+0000 7f0426000640 1 -- start start 2024-05-07T02:43:02.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.312+0000 7f0426000640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04201063f0 0x7f04201a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:02.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.312+0000 7f0426000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f042010bca0 0x7f04201a2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:02.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.312+0000 7f0426000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f042010c140 0x7f042019c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:02.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.312+0000 7f0426000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0420115ee0 con 0x7f042010c140 2024-05-07T02:43:02.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.312+0000 7f0426000640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0420115d60 con 0x7f042010bca0 2024-05-07T02:43:02.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.312+0000 7f0426000640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0420116060 con 0x7f04201063f0 2024-05-07T02:43:02.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.313+0000 7f0424c00640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04201063f0 0x7f04201a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:02.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.313+0000 7f0425600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f042010c140 0x7f042019c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:02.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.313+0000 7f0424c00640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04201063f0 0x7f04201a2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:38270/0 (socket says 172.21.15.19:38270) 2024-05-07T02:43:02.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.313+0000 7f0424c00640 1 -- 172.21.15.19:0/3064921905 learned_addr learned my addr 172.21.15.19:0/3064921905 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:43:02.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.313+0000 7f041fe00640 1 --2- 172.21.15.19:0/3064921905 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f042010bca0 0x7f04201a2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:02.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.313+0000 7f0424c00640 1 -- 172.21.15.19:0/3064921905 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f042010bca0 msgr2=0x7f04201a2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:02.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.313+0000 7f0424c00640 1 --2- 172.21.15.19:0/3064921905 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f042010bca0 0x7f04201a2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:02.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.313+0000 7f0424c00640 1 -- 172.21.15.19:0/3064921905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f042010c140 msgr2=0x7f042019c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:02.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.313+0000 7f0424c00640 1 --2- 172.21.15.19:0/3064921905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f042010c140 0x7f042019c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:02.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.313+0000 7f0424c00640 1 -- 172.21.15.19:0/3064921905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f042019ce10 con 0x7f04201063f0 2024-05-07T02:43:02.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.314+0000 7f0424c00640 1 --2- 172.21.15.19:0/3064921905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04201063f0 0x7f04201a2560 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f041400e9f0 tx=0x7f041400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:43:02.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.314+0000 7f041d600640 1 -- 172.21.15.19:0/3064921905 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041400ccc0 con 0x7f04201063f0 2024-05-07T02:43:02.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.314+0000 7f041d600640 1 -- 172.21.15.19:0/3064921905 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04140054d0 con 0x7f04201063f0 2024-05-07T02:43:02.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.314+0000 7f0426000640 1 -- 172.21.15.19:0/3064921905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f042019d0a0 con 0x7f04201063f0 2024-05-07T02:43:02.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.314+0000 7f041d600640 1 -- 172.21.15.19:0/3064921905 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0414010430 con 0x7f04201063f0 2024-05-07T02:43:02.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.314+0000 7f0426000640 1 -- 172.21.15.19:0/3064921905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04201a9300 con 0x7f04201063f0 2024-05-07T02:43:02.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.315+0000 7f0426000640 1 -- 172.21.15.19:0/3064921905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0420106f80 con 0x7f04201063f0 2024-05-07T02:43:02.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.318+0000 7f041d600640 1 -- 172.21.15.19:0/3064921905 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f04140046c0 con 0x7f04201063f0 2024-05-07T02:43:02.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.319+0000 7f041d600640 1 --2- 172.21.15.19:0/3064921905 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f04080b2960 0x7f04080b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:02.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.320+0000 7f041d600640 1 -- 172.21.15.19:0/3064921905 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9525+0+0 (secure 0 0 0) 0x7f04140e2d40 con 0x7f04201063f0 2024-05-07T02:43:02.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.320+0000 7f041fe00640 1 --2- 172.21.15.19:0/3064921905 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f04080b2960 0x7f04080b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:02.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.321+0000 7f041fe00640 1 --2- 172.21.15.19:0/3064921905 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f04080b2960 0x7f04080b4e70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f041002aa00 tx=0x7f0410005a20 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:43:02.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.324+0000 7f041d600640 1 -- 172.21.15.19:0/3064921905 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f04140aa2d0 con 0x7f04201063f0 2024-05-07T02:43:02.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:43:02 smithi064 ceph-mon[34699]: pgmap v346: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:43:02.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:43:02 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/1026088863' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-07T02:43:02.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:43:02 smithi019 ceph-mon[29781]: pgmap v346: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:43:02.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:43:02 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/1026088863' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-07T02:43:02.459 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:43:02 smithi027 ceph-mon[34398]: pgmap v346: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:43:02.460 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:43:02 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/1026088863' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-07T02:43:02.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.536+0000 7f0426000640 1 -- 172.21.15.19:0/3064921905 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f04201a94d0 con 0x7f04201063f0 2024-05-07T02:43:02.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.537+0000 7f041d600640 1 -- 172.21.15.19:0/3064921905 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) ==== 93+0+10 (secure 0 0 0) 0x7f04140a9bd0 con 0x7f04201063f0 2024-05-07T02:43:02.538 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-05-07T02:43:02.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.539+0000 7f0426000640 1 -- 172.21.15.19:0/3064921905 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f04080b2960 msgr2=0x7f04080b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:02.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.539+0000 7f0426000640 1 --2- 172.21.15.19:0/3064921905 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f04080b2960 0x7f04080b4e70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f041002aa00 tx=0x7f0410005a20 comp rx=0 tx=0).stop 2024-05-07T02:43:02.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.540+0000 7f0426000640 1 -- 172.21.15.19:0/3064921905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04201063f0 msgr2=0x7f04201a2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:02.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.540+0000 7f0426000640 1 --2- 172.21.15.19:0/3064921905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04201063f0 0x7f04201a2560 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f041400e9f0 tx=0x7f041400edf0 comp rx=0 tx=0).stop 2024-05-07T02:43:02.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.540+0000 7f0426000640 1 -- 172.21.15.19:0/3064921905 shutdown_connections 2024-05-07T02:43:02.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.540+0000 7f0426000640 1 --2- 172.21.15.19:0/3064921905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f042010c140 0x7f042019c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:02.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.540+0000 7f0426000640 1 --2- 172.21.15.19:0/3064921905 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f042010bca0 0x7f04201a2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:02.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.540+0000 7f0426000640 1 --2- 172.21.15.19:0/3064921905 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f04080b2960 0x7f04080b4e70 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:02.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.540+0000 7f0426000640 1 --2- 172.21.15.19:0/3064921905 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04201063f0 0x7f04201a2560 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:02.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.540+0000 7f0426000640 1 -- 172.21.15.19:0/3064921905 >> 172.21.15.19:0/3064921905 conn(0x7f0420101300 msgr2=0x7f04201107b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:43:02.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.540+0000 7f0426000640 1 -- 172.21.15.19:0/3064921905 shutdown_connections 2024-05-07T02:43:02.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.540+0000 7f0426000640 1 -- 172.21.15.19:0/3064921905 wait complete. 2024-05-07T02:43:02.567 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-07T02:43:02.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.852+0000 7f1fdb600640 1 -- 172.21.15.19:0/900885275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fd410c310 msgr2=0x7f1fd4105f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:02.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.852+0000 7f1fdb600640 1 --2- 172.21.15.19:0/900885275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fd410c310 0x7f1fd4105f50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0009860 tx=0x7f1fc002a340 comp rx=0 tx=0).stop 2024-05-07T02:43:02.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.852+0000 7f1fdb600640 1 -- 172.21.15.19:0/900885275 shutdown_connections 2024-05-07T02:43:02.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.852+0000 7f1fdb600640 1 --2- 172.21.15.19:0/900885275 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fd41065c0 0x7f1fd4112990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:02.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.852+0000 7f1fdb600640 1 --2- 172.21.15.19:0/900885275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fd410c310 0x7f1fd4105f50 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:02.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.852+0000 7f1fdb600640 1 --2- 172.21.15.19:0/900885275 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fd410be70 0x7f1fd4105a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:02.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.852+0000 7f1fdb600640 1 -- 172.21.15.19:0/900885275 >> 172.21.15.19:0/900885275 conn(0x7f1fd4077100 msgr2=0x7f1fd4077540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:43:02.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.852+0000 7f1fdb600640 1 -- 172.21.15.19:0/900885275 shutdown_connections 2024-05-07T02:43:02.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.852+0000 7f1fdb600640 1 -- 172.21.15.19:0/900885275 wait complete. 2024-05-07T02:43:02.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.853+0000 7f1fdb600640 1 Processor -- start 2024-05-07T02:43:02.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.853+0000 7f1fdb600640 1 -- start start 2024-05-07T02:43:02.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.854+0000 7f1fdb600640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fd41065c0 0x7f1fd41a26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:02.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.854+0000 7f1fdb600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fd410be70 0x7f1fd41a2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:02.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.854+0000 7f1fdb600640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fd410c310 0x7f1fd419c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:02.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.854+0000 7f1fdb600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd4116060 con 0x7f1fd410be70 2024-05-07T02:43:02.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.854+0000 7f1fdb600640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd4115ee0 con 0x7f1fd410c310 2024-05-07T02:43:02.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.854+0000 7f1fdb600640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd41161e0 con 0x7f1fd41065c0 2024-05-07T02:43:02.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.854+0000 7f1fd3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fd410be70 0x7f1fd41a2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:02.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.854+0000 7f1fd8c00640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fd41065c0 0x7f1fd41a26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:02.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.854+0000 7f1fd3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fd410be70 0x7f1fd41a2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57686/0 (socket says 172.21.15.19:57686) 2024-05-07T02:43:02.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.854+0000 7f1fd3e00640 1 -- 172.21.15.19:0/1453176885 learned_addr learned my addr 172.21.15.19:0/1453176885 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:43:02.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.854+0000 7f1fd9600640 1 --2- 172.21.15.19:0/1453176885 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fd410c310 0x7f1fd419c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:02.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.855+0000 7f1fd8c00640 1 -- 172.21.15.19:0/1453176885 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fd410c310 msgr2=0x7f1fd419c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:02.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.855+0000 7f1fd8c00640 1 --2- 172.21.15.19:0/1453176885 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fd410c310 0x7f1fd419c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:02.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.855+0000 7f1fd8c00640 1 -- 172.21.15.19:0/1453176885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fd410be70 msgr2=0x7f1fd41a2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:02.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.855+0000 7f1fd8c00640 1 --2- 172.21.15.19:0/1453176885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fd410be70 0x7f1fd41a2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:02.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.855+0000 7f1fd8c00640 1 -- 172.21.15.19:0/1453176885 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fd419cfa0 con 0x7f1fd41065c0 2024-05-07T02:43:02.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.855+0000 7f1fd8c00640 1 --2- 172.21.15.19:0/1453176885 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fd41065c0 0x7f1fd41a26f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1fc800e9c0 tx=0x7f1fc800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:43:02.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.855+0000 7f1fd1600640 1 -- 172.21.15.19:0/1453176885 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc800ccc0 con 0x7f1fd41065c0 2024-05-07T02:43:02.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.855+0000 7f1fd1600640 1 -- 172.21.15.19:0/1453176885 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fc8005020 con 0x7f1fd41065c0 2024-05-07T02:43:02.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.855+0000 7f1fd1600640 1 -- 172.21.15.19:0/1453176885 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc8004a00 con 0x7f1fd41065c0 2024-05-07T02:43:02.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.855+0000 7f1fdb600640 1 -- 172.21.15.19:0/1453176885 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fd419d290 con 0x7f1fd41065c0 2024-05-07T02:43:02.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.856+0000 7f1fdb600640 1 -- 172.21.15.19:0/1453176885 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fd41a9570 con 0x7f1fd41065c0 2024-05-07T02:43:02.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.857+0000 7f1fdb600640 1 -- 172.21.15.19:0/1453176885 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fd406a340 con 0x7f1fd41065c0 2024-05-07T02:43:02.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.859+0000 7f1fd1600640 1 -- 172.21.15.19:0/1453176885 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1fc80051d0 con 0x7f1fd41065c0 2024-05-07T02:43:02.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.861+0000 7f1fd1600640 1 --2- 172.21.15.19:0/1453176885 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f1fbc0b2960 0x7f1fbc0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:02.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.861+0000 7f1fd1600640 1 -- 172.21.15.19:0/1453176885 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9525+0+0 (secure 0 0 0) 0x7f1fc80e2a50 con 0x7f1fd41065c0 2024-05-07T02:43:02.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.861+0000 7f1fd3e00640 1 --2- 172.21.15.19:0/1453176885 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f1fbc0b2960 0x7f1fbc0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:02.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.862+0000 7f1fd3e00640 1 --2- 172.21.15.19:0/1453176885 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f1fbc0b2960 0x7f1fbc0b4e70 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1fc00057c0 tx=0x7f1fc00058d0 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:43:02.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:02.863+0000 7f1fd1600640 1 -- 172.21.15.19:0/1453176885 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1fc80aa070 con 0x7f1fd41065c0 2024-05-07T02:43:03.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.070+0000 7f1fdb600640 1 -- 172.21.15.19:0/1453176885 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1fd41a9740 con 0x7f1fd41065c0 2024-05-07T02:43:03.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.071+0000 7f1fd1600640 1 -- 172.21.15.19:0/1453176885 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v98) ==== 104+0+11 (secure 0 0 0) 0x7f1fc80a9970 con 0x7f1fd41065c0 2024-05-07T02:43:03.072 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-05-07T02:43:03.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.074+0000 7f1fdb600640 1 -- 172.21.15.19:0/1453176885 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f1fbc0b2960 msgr2=0x7f1fbc0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:03.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.074+0000 7f1fdb600640 1 --2- 172.21.15.19:0/1453176885 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f1fbc0b2960 0x7f1fbc0b4e70 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1fc00057c0 tx=0x7f1fc00058d0 comp rx=0 tx=0).stop 2024-05-07T02:43:03.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.074+0000 7f1fdb600640 1 -- 172.21.15.19:0/1453176885 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fd41065c0 msgr2=0x7f1fd41a26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:03.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.074+0000 7f1fdb600640 1 --2- 172.21.15.19:0/1453176885 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fd41065c0 0x7f1fd41a26f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1fc800e9c0 tx=0x7f1fc800ee80 comp rx=0 tx=0).stop 2024-05-07T02:43:03.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.074+0000 7f1fdb600640 1 -- 172.21.15.19:0/1453176885 shutdown_connections 2024-05-07T02:43:03.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.074+0000 7f1fdb600640 1 --2- 172.21.15.19:0/1453176885 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fd410c310 0x7f1fd419c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:03.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.074+0000 7f1fdb600640 1 --2- 172.21.15.19:0/1453176885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1fd410be70 0x7f1fd41a2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:03.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.074+0000 7f1fdb600640 1 --2- 172.21.15.19:0/1453176885 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f1fbc0b2960 0x7f1fbc0b4e70 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:03.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.074+0000 7f1fdb600640 1 --2- 172.21.15.19:0/1453176885 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1fd41065c0 0x7f1fd41a26f0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:03.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.074+0000 7f1fdb600640 1 -- 172.21.15.19:0/1453176885 >> 172.21.15.19:0/1453176885 conn(0x7f1fd4077100 msgr2=0x7f1fd4104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:43:03.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.074+0000 7f1fdb600640 1 -- 172.21.15.19:0/1453176885 shutdown_connections 2024-05-07T02:43:03.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.074+0000 7f1fdb600640 1 -- 172.21.15.19:0/1453176885 wait complete. 2024-05-07T02:43:03.103 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-07T02:43:03.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:43:03 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/3064921905' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-07T02:43:03.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.376+0000 7f0015200640 1 -- 172.21.15.19:0/3769380473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00101041e0 msgr2=0x7f00101105b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:03.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.376+0000 7f0015200640 1 --2- 172.21.15.19:0/3769380473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00101041e0 0x7f00101105b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0008009ca0 tx=0x7f000802a8c0 comp rx=0 tx=0).stop 2024-05-07T02:43:03.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.376+0000 7f0015200640 1 -- 172.21.15.19:0/3769380473 shutdown_connections 2024-05-07T02:43:03.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.376+0000 7f0015200640 1 --2- 172.21.15.19:0/3769380473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00101041e0 0x7f00101105b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:03.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.376+0000 7f0015200640 1 --2- 172.21.15.19:0/3769380473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0010109f30 0x7f0010103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:03.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.376+0000 7f0015200640 1 --2- 172.21.15.19:0/3769380473 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0010109a90 0x7f0010103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:03.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.376+0000 7f0015200640 1 -- 172.21.15.19:0/3769380473 >> 172.21.15.19:0/3769380473 conn(0x7f00100791a0 msgr2=0x7f00100795e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:43:03.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.376+0000 7f0015200640 1 -- 172.21.15.19:0/3769380473 shutdown_connections 2024-05-07T02:43:03.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.376+0000 7f0015200640 1 -- 172.21.15.19:0/3769380473 wait complete. 2024-05-07T02:43:03.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.377+0000 7f0015200640 1 Processor -- start 2024-05-07T02:43:03.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.378+0000 7f0015200640 1 -- start start 2024-05-07T02:43:03.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.378+0000 7f0015200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00101041e0 0x7f00101a2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:03.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.378+0000 7f0015200640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0010109a90 0x7f00101a2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:03.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.378+0000 7f0015200640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0010109f30 0x7f001019c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:03.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.378+0000 7f0015200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0010113c80 con 0x7f00101041e0 2024-05-07T02:43:03.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.378+0000 7f0015200640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0010113b00 con 0x7f0010109f30 2024-05-07T02:43:03.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.378+0000 7f0015200640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0010113e00 con 0x7f0010109a90 2024-05-07T02:43:03.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.378+0000 7f000f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00101041e0 0x7f00101a2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:03.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.378+0000 7f000ea00640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0010109a90 0x7f00101a2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:03.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.378+0000 7f000fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0010109f30 0x7f001019c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:03.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.378+0000 7f000f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00101041e0 0x7f00101a2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57702/0 (socket says 172.21.15.19:57702) 2024-05-07T02:43:03.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.378+0000 7f000ea00640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0010109a90 0x7f00101a2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:38302/0 (socket says 172.21.15.19:38302) 2024-05-07T02:43:03.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.378+0000 7f000f400640 1 -- 172.21.15.19:0/3767567777 learned_addr learned my addr 172.21.15.19:0/3767567777 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:43:03.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.379+0000 7f000fe00640 1 -- 172.21.15.19:0/3767567777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0010109a90 msgr2=0x7f00101a2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:03.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.379+0000 7f000fe00640 1 --2- 172.21.15.19:0/3767567777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0010109a90 0x7f00101a2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:03.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.379+0000 7f000fe00640 1 -- 172.21.15.19:0/3767567777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00101041e0 msgr2=0x7f00101a2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:03.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.379+0000 7f000fe00640 1 --2- 172.21.15.19:0/3767567777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00101041e0 0x7f00101a2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:03.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.379+0000 7f000fe00640 1 -- 172.21.15.19:0/3767567777 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f001019cdf0 con 0x7f0010109f30 2024-05-07T02:43:03.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.379+0000 7f000fe00640 1 --2- 172.21.15.19:0/3767567777 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0010109f30 0x7f001019c6c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f000802ade0 tx=0x7f0008031660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:43:03.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.379+0000 7f0007e00640 1 -- 172.21.15.19:0/3767567777 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0008003b70 con 0x7f0010109f30 2024-05-07T02:43:03.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.380+0000 7f0007e00640 1 -- 172.21.15.19:0/3767567777 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0008031910 con 0x7f0010109f30 2024-05-07T02:43:03.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.380+0000 7f0015200640 1 -- 172.21.15.19:0/3767567777 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f001019d020 con 0x7f0010109f30 2024-05-07T02:43:03.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.380+0000 7f0007e00640 1 -- 172.21.15.19:0/3767567777 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000802f950 con 0x7f0010109f30 2024-05-07T02:43:03.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.380+0000 7f0015200640 1 -- 172.21.15.19:0/3767567777 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00101a92e0 con 0x7f0010109f30 2024-05-07T02:43:03.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.381+0000 7f0015200640 1 -- 172.21.15.19:0/3767567777 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0010104d70 con 0x7f0010109f30 2024-05-07T02:43:03.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.383+0000 7f0007e00640 1 -- 172.21.15.19:0/3767567777 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f000802faf0 con 0x7f0010109f30 2024-05-07T02:43:03.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.385+0000 7f0007e00640 1 --2- 172.21.15.19:0/3767567777 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7efff40b29e0 0x7efff40b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:03.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.385+0000 7f0007e00640 1 -- 172.21.15.19:0/3767567777 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9525+0+0 (secure 0 0 0) 0x7f0008103d30 con 0x7f0010109f30 2024-05-07T02:43:03.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.385+0000 7f000f400640 1 --2- 172.21.15.19:0/3767567777 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7efff40b29e0 0x7efff40b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:03.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.386+0000 7f000f400640 1 --2- 172.21.15.19:0/3767567777 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7efff40b29e0 0x7efff40b4ef0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f00000095f0 tx=0x7f0000005f90 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:43:03.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.387+0000 7f0007e00640 1 -- 172.21.15.19:0/3767567777 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f00080cb3d0 con 0x7f0010109f30 2024-05-07T02:43:03.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:43:03 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/3064921905' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-07T02:43:03.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:43:03 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/3064921905' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-07T02:43:03.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.603+0000 7f0015200640 1 -- 172.21.15.19:0/3767567777 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f00101a94b0 con 0x7f0010109f30 2024-05-07T02:43:03.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.604+0000 7f0007e00640 1 -- 172.21.15.19:0/3767567777 <== mon.2 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v98) ==== 100+0+11 (secure 0 0 0) 0x7f0008003d10 con 0x7f0010109f30 2024-05-07T02:43:03.604 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-05-07T02:43:03.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.606+0000 7f0015200640 1 -- 172.21.15.19:0/3767567777 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7efff40b29e0 msgr2=0x7efff40b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:03.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.606+0000 7f0015200640 1 --2- 172.21.15.19:0/3767567777 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7efff40b29e0 0x7efff40b4ef0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f00000095f0 tx=0x7f0000005f90 comp rx=0 tx=0).stop 2024-05-07T02:43:03.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.606+0000 7f0015200640 1 -- 172.21.15.19:0/3767567777 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0010109f30 msgr2=0x7f001019c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:03.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.606+0000 7f0015200640 1 --2- 172.21.15.19:0/3767567777 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0010109f30 0x7f001019c6c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f000802ade0 tx=0x7f0008031660 comp rx=0 tx=0).stop 2024-05-07T02:43:03.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.607+0000 7f0015200640 1 -- 172.21.15.19:0/3767567777 shutdown_connections 2024-05-07T02:43:03.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.607+0000 7f0015200640 1 --2- 172.21.15.19:0/3767567777 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0010109f30 0x7f001019c6c0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:03.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.607+0000 7f0015200640 1 --2- 172.21.15.19:0/3767567777 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7efff40b29e0 0x7efff40b4ef0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:03.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.607+0000 7f0015200640 1 --2- 172.21.15.19:0/3767567777 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0010109a90 0x7f00101a2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:03.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.607+0000 7f0015200640 1 --2- 172.21.15.19:0/3767567777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00101041e0 0x7f00101a2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:03.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.607+0000 7f0015200640 1 -- 172.21.15.19:0/3767567777 >> 172.21.15.19:0/3767567777 conn(0x7f00100791a0 msgr2=0x7f00100779d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:43:03.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.607+0000 7f0015200640 1 -- 172.21.15.19:0/3767567777 shutdown_connections 2024-05-07T02:43:03.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.607+0000 7f0015200640 1 -- 172.21.15.19:0/3767567777 wait complete. 2024-05-07T02:43:03.635 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-07T02:43:03.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.921+0000 7f180fe00640 1 -- 172.21.15.19:0/3726816341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1810104b60 msgr2=0x7f1810104f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:03.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.921+0000 7f180fe00640 1 --2- 172.21.15.19:0/3726816341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1810104b60 0x7f1810104f70 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1800009900 tx=0x7f180002a3f0 comp rx=0 tx=0).stop 2024-05-07T02:43:03.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.922+0000 7f180fe00640 1 -- 172.21.15.19:0/3726816341 shutdown_connections 2024-05-07T02:43:03.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.922+0000 7f180fe00640 1 --2- 172.21.15.19:0/3726816341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f181010dfb0 0x7f18101104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:03.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.922+0000 7f180fe00640 1 --2- 172.21.15.19:0/3726816341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f181010b5c0 0x7f181010d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:03.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.922+0000 7f180fe00640 1 --2- 172.21.15.19:0/3726816341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1810104b60 0x7f1810104f70 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:03.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.922+0000 7f180fe00640 1 -- 172.21.15.19:0/3726816341 >> 172.21.15.19:0/3726816341 conn(0x7f18100fe220 msgr2=0x7f1810100640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:43:03.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.922+0000 7f180fe00640 1 -- 172.21.15.19:0/3726816341 shutdown_connections 2024-05-07T02:43:03.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.922+0000 7f180fe00640 1 -- 172.21.15.19:0/3726816341 wait complete. 2024-05-07T02:43:03.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.923+0000 7f180fe00640 1 Processor -- start 2024-05-07T02:43:03.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.924+0000 7f180fe00640 1 -- start start 2024-05-07T02:43:03.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.924+0000 7f180fe00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1810104b60 0x7f18101a24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:03.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.924+0000 7f180fe00640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f181010b5c0 0x7f18101a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:03.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.924+0000 7f180fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f181010dfb0 0x7f181019c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:03.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.924+0000 7f180fe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1810113c10 con 0x7f181010dfb0 2024-05-07T02:43:03.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.924+0000 7f180fe00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f1810113a90 con 0x7f1810104b60 2024-05-07T02:43:03.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.924+0000 7f180fe00640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1810113d90 con 0x7f181010b5c0 2024-05-07T02:43:03.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.924+0000 7f180f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f181010dfb0 0x7f181019c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:03.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.925+0000 7f180f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f181010dfb0 0x7f181019c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57718/0 (socket says 172.21.15.19:57718) 2024-05-07T02:43:03.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.925+0000 7f180f400640 1 -- 172.21.15.19:0/4174476187 learned_addr learned my addr 172.21.15.19:0/4174476187 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:43:03.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.925+0000 7f180ea00640 1 --2- 172.21.15.19:0/4174476187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1810104b60 0x7f18101a24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:03.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.925+0000 7f1807e00640 1 --2- 172.21.15.19:0/4174476187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f181010b5c0 0x7f18101a2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:03.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.925+0000 7f180f400640 1 -- 172.21.15.19:0/4174476187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f181010b5c0 msgr2=0x7f18101a2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:03.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.925+0000 7f180f400640 1 --2- 172.21.15.19:0/4174476187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f181010b5c0 0x7f18101a2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:03.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.925+0000 7f180f400640 1 -- 172.21.15.19:0/4174476187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1810104b60 msgr2=0x7f18101a24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:03.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.925+0000 7f180f400640 1 --2- 172.21.15.19:0/4174476187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1810104b60 0x7f18101a24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:03.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.925+0000 7f180f400640 1 -- 172.21.15.19:0/4174476187 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f181019cda0 con 0x7f181010dfb0 2024-05-07T02:43:03.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.925+0000 7f180f400640 1 --2- 172.21.15.19:0/4174476187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f181010dfb0 0x7f181019c670 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f180800eb10 tx=0x7f180800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:43:03.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.926+0000 7f1807400640 1 -- 172.21.15.19:0/4174476187 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1808004b20 con 0x7f181010dfb0 2024-05-07T02:43:03.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.926+0000 7f1807400640 1 -- 172.21.15.19:0/4174476187 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1808004cc0 con 0x7f181010dfb0 2024-05-07T02:43:03.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.926+0000 7f180fe00640 1 -- 172.21.15.19:0/4174476187 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f181019d030 con 0x7f181010dfb0 2024-05-07T02:43:03.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.926+0000 7f1807400640 1 -- 172.21.15.19:0/4174476187 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1808010670 con 0x7f181010dfb0 2024-05-07T02:43:03.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.926+0000 7f180fe00640 1 -- 172.21.15.19:0/4174476187 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18101a9290 con 0x7f181010dfb0 2024-05-07T02:43:03.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.927+0000 7f180fe00640 1 -- 172.21.15.19:0/4174476187 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f181006aa10 con 0x7f181010dfb0 2024-05-07T02:43:03.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.930+0000 7f1807400640 1 -- 172.21.15.19:0/4174476187 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1808005020 con 0x7f181010dfb0 2024-05-07T02:43:03.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.931+0000 7f1807400640 1 --2- 172.21.15.19:0/4174476187 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f17f40b2960 0x7f17f40b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:03.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.932+0000 7f1807400640 1 -- 172.21.15.19:0/4174476187 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9525+0+0 (secure 0 0 0) 0x7f18080e2f70 con 0x7f181010dfb0 2024-05-07T02:43:03.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.932+0000 7f180ea00640 1 --2- 172.21.15.19:0/4174476187 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f17f40b2960 0x7f17f40b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:03.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.933+0000 7f180ea00640 1 --2- 172.21.15.19:0/4174476187 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f17f40b2960 0x7f17f40b4e70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1800031440 tx=0x7f1800009560 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:43:03.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:03.936+0000 7f1807400640 1 -- 172.21.15.19:0/4174476187 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f18080aa500 con 0x7f181010dfb0 2024-05-07T02:43:04.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.152+0000 7f180fe00640 1 -- 172.21.15.19:0/4174476187 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f18101a9460 con 0x7f181010dfb0 2024-05-07T02:43:04.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.153+0000 7f1807400640 1 -- 172.21.15.19:0/4174476187 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v98) ==== 103+0+11 (secure 0 0 0) 0x7f18080a9e00 con 0x7f181010dfb0 2024-05-07T02:43:04.154 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-05-07T02:43:04.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.157+0000 7f180fe00640 1 -- 172.21.15.19:0/4174476187 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f17f40b2960 msgr2=0x7f17f40b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:04.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.157+0000 7f180fe00640 1 --2- 172.21.15.19:0/4174476187 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f17f40b2960 0x7f17f40b4e70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1800031440 tx=0x7f1800009560 comp rx=0 tx=0).stop 2024-05-07T02:43:04.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.157+0000 7f180fe00640 1 -- 172.21.15.19:0/4174476187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f181010dfb0 msgr2=0x7f181019c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:04.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.157+0000 7f180fe00640 1 --2- 172.21.15.19:0/4174476187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f181010dfb0 0x7f181019c670 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f180800eb10 tx=0x7f180800c2d0 comp rx=0 tx=0).stop 2024-05-07T02:43:04.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.157+0000 7f180fe00640 1 -- 172.21.15.19:0/4174476187 shutdown_connections 2024-05-07T02:43:04.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.157+0000 7f180fe00640 1 --2- 172.21.15.19:0/4174476187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f181010dfb0 0x7f181019c670 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:04.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.157+0000 7f180fe00640 1 --2- 172.21.15.19:0/4174476187 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f17f40b2960 0x7f17f40b4e70 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:04.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.157+0000 7f180fe00640 1 --2- 172.21.15.19:0/4174476187 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f181010b5c0 0x7f18101a2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:04.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.157+0000 7f180fe00640 1 --2- 172.21.15.19:0/4174476187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1810104b60 0x7f18101a24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:04.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.157+0000 7f180fe00640 1 -- 172.21.15.19:0/4174476187 >> 172.21.15.19:0/4174476187 conn(0x7f18100fe220 msgr2=0x7f181010e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:43:04.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.157+0000 7f180fe00640 1 -- 172.21.15.19:0/4174476187 shutdown_connections 2024-05-07T02:43:04.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.157+0000 7f180fe00640 1 -- 172.21.15.19:0/4174476187 wait complete. 2024-05-07T02:43:04.187 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-07T02:43:04.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:43:04 smithi064 ceph-mon[34699]: pgmap v347: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-05-07T02:43:04.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:43:04 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/1453176885' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-07T02:43:04.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:43:04 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/3767567777' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-07T02:43:04.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:43:04 smithi019 ceph-mon[29781]: pgmap v347: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-05-07T02:43:04.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:43:04 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/1453176885' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-07T02:43:04.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:43:04 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/3767567777' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-07T02:43:04.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:43:04 smithi027 ceph-mon[34398]: pgmap v347: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-05-07T02:43:04.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:43:04 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/1453176885' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-07T02:43:04.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:43:04 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/3767567777' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-07T02:43:04.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.478+0000 7fd123e00640 1 -- 172.21.15.19:0/2250125852 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd124075670 msgr2=0x7fd124075a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:04.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.478+0000 7fd123e00640 1 --2- 172.21.15.19:0/2250125852 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd124075670 0x7fd124075a80 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fd114009900 tx=0x7fd11402a480 comp rx=0 tx=0).stop 2024-05-07T02:43:04.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.478+0000 7fd123e00640 1 -- 172.21.15.19:0/2250125852 shutdown_connections 2024-05-07T02:43:04.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.478+0000 7fd123e00640 1 --2- 172.21.15.19:0/2250125852 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd124110270 0x7fd1241126d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:04.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.478+0000 7fd123e00640 1 --2- 172.21.15.19:0/2250125852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd12410d910 0x7fd12410fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:04.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.478+0000 7fd123e00640 1 --2- 172.21.15.19:0/2250125852 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd124075670 0x7fd124075a80 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:04.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.478+0000 7fd123e00640 1 -- 172.21.15.19:0/2250125852 >> 172.21.15.19:0/2250125852 conn(0x7fd12407a190 msgr2=0x7fd12407a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:43:04.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.478+0000 7fd123e00640 1 -- 172.21.15.19:0/2250125852 shutdown_connections 2024-05-07T02:43:04.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.478+0000 7fd123e00640 1 -- 172.21.15.19:0/2250125852 wait complete. 2024-05-07T02:43:04.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.479+0000 7fd123e00640 1 Processor -- start 2024-05-07T02:43:04.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.479+0000 7fd123e00640 1 -- start start 2024-05-07T02:43:04.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.480+0000 7fd123e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd124075670 0x7fd12419c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:04.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.480+0000 7fd123e00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd12410d910 0x7fd12419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:04.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.480+0000 7fd123e00640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd124110270 0x7fd1241a0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:04.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.480+0000 7fd123e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd124115e90 con 0x7fd124075670 2024-05-07T02:43:04.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.480+0000 7fd123e00640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fd124115d10 con 0x7fd12410d910 2024-05-07T02:43:04.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.480+0000 7fd123e00640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd124116010 con 0x7fd124110270 2024-05-07T02:43:04.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.480+0000 7fd122a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd124075670 0x7fd12419c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:04.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.480+0000 7fd122a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd124075670 0x7fd12419c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57746/0 (socket says 172.21.15.19:57746) 2024-05-07T02:43:04.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.480+0000 7fd122a00640 1 -- 172.21.15.19:0/3964688001 learned_addr learned my addr 172.21.15.19:0/3964688001 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:43:04.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.480+0000 7fd11a000640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd12410d910 0x7fd12419cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:04.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.480+0000 7fd123400640 1 --2- 172.21.15.19:0/3964688001 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd124110270 0x7fd1241a0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:04.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.480+0000 7fd123400640 1 -- 172.21.15.19:0/3964688001 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd12410d910 msgr2=0x7fd12419cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:04.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.480+0000 7fd123400640 1 --2- 172.21.15.19:0/3964688001 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd12410d910 0x7fd12419cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:04.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.481+0000 7fd123400640 1 -- 172.21.15.19:0/3964688001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd124075670 msgr2=0x7fd12419c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:04.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.481+0000 7fd123400640 1 --2- 172.21.15.19:0/3964688001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd124075670 0x7fd12419c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:04.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.481+0000 7fd123400640 1 -- 172.21.15.19:0/3964688001 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1241a1630 con 0x7fd124110270 2024-05-07T02:43:04.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.481+0000 7fd122a00640 1 --2- 172.21.15.19:0/3964688001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd124075670 0x7fd12419c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:43:04.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.481+0000 7fd123400640 1 --2- 172.21.15.19:0/3964688001 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd124110270 0x7fd1241a0ff0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd11c009fd0 tx=0x7fd11c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:43:04.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.481+0000 7fd11be00640 1 -- 172.21.15.19:0/3964688001 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11c00ce20 con 0x7fd124110270 2024-05-07T02:43:04.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.481+0000 7fd11be00640 1 -- 172.21.15.19:0/3964688001 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd11c005020 con 0x7fd124110270 2024-05-07T02:43:04.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.481+0000 7fd123e00640 1 -- 172.21.15.19:0/3964688001 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1241a1920 con 0x7fd124110270 2024-05-07T02:43:04.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.481+0000 7fd11be00640 1 -- 172.21.15.19:0/3964688001 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11c010690 con 0x7fd124110270 2024-05-07T02:43:04.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.481+0000 7fd123e00640 1 -- 172.21.15.19:0/3964688001 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1241a1e20 con 0x7fd124110270 2024-05-07T02:43:04.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.483+0000 7fd123e00640 1 -- 172.21.15.19:0/3964688001 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd124069e80 con 0x7fd124110270 2024-05-07T02:43:04.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.485+0000 7fd11be00640 1 -- 172.21.15.19:0/3964688001 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd11c0054d0 con 0x7fd124110270 2024-05-07T02:43:04.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.489+0000 7fd11be00640 1 --2- 172.21.15.19:0/3964688001 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fd1080b2a10 0x7fd1080b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:04.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.489+0000 7fd11be00640 1 -- 172.21.15.19:0/3964688001 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9525+0+0 (secure 0 0 0) 0x7fd11c0e30a0 con 0x7fd124110270 2024-05-07T02:43:04.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.489+0000 7fd122a00640 1 --2- 172.21.15.19:0/3964688001 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fd1080b2a10 0x7fd1080b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:04.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.490+0000 7fd11be00640 1 -- 172.21.15.19:0/3964688001 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd1241a1e20 con 0x7fd124110270 2024-05-07T02:43:04.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.491+0000 7fd122a00640 1 --2- 172.21.15.19:0/3964688001 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fd1080b2a10 0x7fd1080b4f20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd1140096c0 tx=0x7fd114005a20 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:43:04.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.758+0000 7fd123e00640 1 -- 172.21.15.19:0/3964688001 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fd1241a9420 con 0x7fd124110270 2024-05-07T02:43:04.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.771+0000 7fd11be00640 1 -- 172.21.15.19:0/3964688001 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+816174 (secure 0 0 0) 0x7fd11c0aa630 con 0x7fd124110270 2024-05-07T02:43:04.784 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:43:04.784 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:43:04.785 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 26, 2024-05-07T02:43:04.785 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.785 INFO:teuthology.orchestra.run.smithi019.stdout: "active_gid": 25234, 2024-05-07T02:43:04.785 INFO:teuthology.orchestra.run.smithi019.stdout: "active_name": "x", 2024-05-07T02:43:04.785 INFO:teuthology.orchestra.run.smithi019.stdout: "active_addrs": { 2024-05-07T02:43:04.785 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-05-07T02:43:04.785 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:04.785 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-05-07T02:43:04.785 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6800", 2024-05-07T02:43:04.786 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3265825478 2024-05-07T02:43:04.786 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.786 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:04.786 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-05-07T02:43:04.786 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6801", 2024-05-07T02:43:04.786 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3265825478 2024-05-07T02:43:04.786 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:04.786 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-05-07T02:43:04.786 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.787 INFO:teuthology.orchestra.run.smithi019.stdout: "active_addr": "172.21.15.19:6801/3265825478", 2024-05-07T02:43:04.787 INFO:teuthology.orchestra.run.smithi019.stdout: "active_change": "2024-05-07T02:31:49.816143+0000", 2024-05-07T02:43:04.787 INFO:teuthology.orchestra.run.smithi019.stdout: "active_mgr_features": 4540701547738038271, 2024-05-07T02:43:04.787 INFO:teuthology.orchestra.run.smithi019.stdout: "available": true, 2024-05-07T02:43:04.787 INFO:teuthology.orchestra.run.smithi019.stdout: "standbys": [ 2024-05-07T02:43:04.787 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:04.787 INFO:teuthology.orchestra.run.smithi019.stdout: "gid": 25228, 2024-05-07T02:43:04.787 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "z", 2024-05-07T02:43:04.787 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_features": 4540701547738038271, 2024-05-07T02:43:04.788 INFO:teuthology.orchestra.run.smithi019.stdout: "available_modules": [ 2024-05-07T02:43:04.788 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:04.788 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "alerts", 2024-05-07T02:43:04.788 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:04.788 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:04.788 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:04.788 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-07T02:43:04.788 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-07T02:43:04.788 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:04.789 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.789 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:04.789 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-07T02:43:04.789 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.789 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.789 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.789 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How frequently to reexamine health status", 2024-05-07T02:43:04.789 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.789 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.789 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.790 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.790 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:04.790 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:04.790 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.790 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.790 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:04.790 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:04.790 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.790 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.791 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:04.791 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:04.791 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:04.791 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:04.791 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:04.791 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:04.791 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:04.791 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:04.791 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:04.792 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.792 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.792 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.792 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.792 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:04.792 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:04.792 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:04.792 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.792 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:04.793 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:04.793 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.793 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.793 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.793 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:04.793 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.793 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.793 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.793 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.793 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:04.794 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:04.794 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.794 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.794 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:04.794 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:04.794 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.794 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.794 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:04.794 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:04.795 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:04.795 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:04.795 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:04.795 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:04.795 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:04.795 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:04.795 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:04.795 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.795 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.796 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.796 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.796 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:04.796 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:04.796 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:04.796 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.796 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:04.796 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:04.797 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.797 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.797 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.797 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:04.797 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.797 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.797 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.797 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.797 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_destination": { 2024-05-07T02:43:04.797 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_destination", 2024-05-07T02:43:04.798 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.798 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.798 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:04.798 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:04.798 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.798 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.798 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.798 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email address to send alerts to", 2024-05-07T02:43:04.799 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.799 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.799 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.799 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.799 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_from_name": { 2024-05-07T02:43:04.799 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_from_name", 2024-05-07T02:43:04.799 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.799 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.799 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:04.800 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "Ceph", 2024-05-07T02:43:04.800 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.800 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.800 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.800 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email From: name", 2024-05-07T02:43:04.800 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.800 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.800 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.800 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.800 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_host": { 2024-05-07T02:43:04.801 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_host", 2024-05-07T02:43:04.801 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.801 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.801 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:04.801 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:04.801 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.801 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.801 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.801 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP server", 2024-05-07T02:43:04.802 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.802 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.802 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.802 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.802 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_password": { 2024-05-07T02:43:04.802 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_password", 2024-05-07T02:43:04.802 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.802 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.802 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:04.803 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:04.803 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.803 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.803 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.803 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Password to authenticate with", 2024-05-07T02:43:04.803 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.803 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.803 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.803 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.804 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_port": { 2024-05-07T02:43:04.804 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_port", 2024-05-07T02:43:04.804 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:04.804 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.804 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:04.804 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "465", 2024-05-07T02:43:04.804 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.804 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.804 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.805 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP port", 2024-05-07T02:43:04.805 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.805 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.805 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.805 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.805 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_sender": { 2024-05-07T02:43:04.805 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_sender", 2024-05-07T02:43:04.805 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.805 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.805 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:04.806 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:04.806 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.806 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.806 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.806 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP envelope sender", 2024-05-07T02:43:04.806 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.806 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.807 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.807 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.807 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_ssl": { 2024-05-07T02:43:04.807 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_ssl", 2024-05-07T02:43:04.807 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:04.807 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.807 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:04.807 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:04.807 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.808 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.808 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.808 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-07T02:43:04.808 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.808 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.808 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.808 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.808 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_user": { 2024-05-07T02:43:04.808 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_user", 2024-05-07T02:43:04.809 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.809 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.809 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:04.809 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:04.809 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.809 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.809 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.809 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "User to authenticate as", 2024-05-07T02:43:04.809 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.809 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.810 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.810 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.810 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:04.810 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:04.810 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:04.810 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:04.810 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:04.810 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:04.811 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.811 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.811 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.811 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:04.811 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.811 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.811 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.811 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:04.811 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:04.811 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.812 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:04.812 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "balancer", 2024-05-07T02:43:04.812 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:04.812 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:04.812 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:04.812 INFO:teuthology.orchestra.run.smithi019.stdout: "active": { 2024-05-07T02:43:04.812 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "active", 2024-05-07T02:43:04.812 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:04.812 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.813 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:04.813 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:04.813 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.813 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.813 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.813 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically balance PGs across cluster", 2024-05-07T02:43:04.813 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.813 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.813 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.814 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.814 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_time": { 2024-05-07T02:43:04.814 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_time", 2024-05-07T02:43:04.814 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.814 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.814 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:04.814 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0000", 2024-05-07T02:43:04.814 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.814 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.815 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.815 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "beginning time of day to automatically balance", 2024-05-07T02:43:04.815 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-07T02:43:04.815 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.815 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.815 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.815 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_weekday": { 2024-05-07T02:43:04.815 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_weekday", 2024-05-07T02:43:04.815 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-07T02:43:04.816 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.816 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:04.816 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:04.816 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-05-07T02:43:04.816 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-05-07T02:43:04.816 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.816 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-07T02:43:04.816 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-07T02:43:04.816 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.817 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.817 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.817 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_max_iterations": { 2024-05-07T02:43:04.817 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_max_iterations", 2024-05-07T02:43:04.817 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-07T02:43:04.817 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.817 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:04.817 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "25", 2024-05-07T02:43:04.817 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-07T02:43:04.817 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "250", 2024-05-07T02:43:04.818 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.818 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-07T02:43:04.818 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.818 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.818 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.818 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.818 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_metrics": { 2024-05-07T02:43:04.818 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_metrics", 2024-05-07T02:43:04.819 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.819 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.819 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:04.819 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "pgs,objects,bytes", 2024-05-07T02:43:04.819 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.819 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.819 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.819 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-07T02:43:04.819 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-07T02:43:04.820 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.820 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.820 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.820 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_step": { 2024-05-07T02:43:04.820 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_step", 2024-05-07T02:43:04.820 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-07T02:43:04.820 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.820 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:04.820 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.5", 2024-05-07T02:43:04.821 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0.001", 2024-05-07T02:43:04.821 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "0.999", 2024-05-07T02:43:04.821 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.821 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "aggressiveness of optimization", 2024-05-07T02:43:04.821 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-07T02:43:04.821 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.821 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.821 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.821 INFO:teuthology.orchestra.run.smithi019.stdout: "end_time": { 2024-05-07T02:43:04.821 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_time", 2024-05-07T02:43:04.822 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.822 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.822 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:04.822 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2359", 2024-05-07T02:43:04.822 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.822 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.822 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.822 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ending time of day to automatically balance", 2024-05-07T02:43:04.823 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-07T02:43:04.823 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.823 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.823 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.823 INFO:teuthology.orchestra.run.smithi019.stdout: "end_weekday": { 2024-05-07T02:43:04.823 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_weekday", 2024-05-07T02:43:04.823 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-07T02:43:04.823 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.823 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:04.823 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:04.824 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-05-07T02:43:04.824 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-05-07T02:43:04.824 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.824 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-07T02:43:04.824 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-07T02:43:04.824 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.824 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.824 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.824 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:04.825 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:04.825 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.825 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.825 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:04.825 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:04.825 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.825 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.825 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:04.825 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:04.826 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:04.826 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:04.826 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:04.826 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:04.826 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:04.826 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:04.826 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:04.827 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.827 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.827 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.827 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.827 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:04.827 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:04.827 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:04.827 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.827 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:04.827 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:04.828 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.828 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.828 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.828 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:04.828 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.828 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.828 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.828 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.828 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:04.829 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:04.829 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.829 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.829 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:04.829 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:04.829 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.829 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.829 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:04.829 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:04.830 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:04.830 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:04.830 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:04.830 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:04.830 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:04.830 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:04.830 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:04.830 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.831 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.831 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.831 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.831 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:04.831 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:04.831 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:04.831 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.831 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:04.831 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:04.832 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.832 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.832 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.832 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:04.832 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.832 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.832 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.832 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.832 INFO:teuthology.orchestra.run.smithi019.stdout: "min_score": { 2024-05-07T02:43:04.833 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_score", 2024-05-07T02:43:04.833 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-07T02:43:04.833 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.833 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:04.833 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:04.833 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.833 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.833 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.833 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-07T02:43:04.833 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.834 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.834 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.834 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.834 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-05-07T02:43:04.834 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-05-07T02:43:04.834 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.834 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.834 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:04.835 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "upmap", 2024-05-07T02:43:04.835 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.835 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.835 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:04.835 INFO:teuthology.orchestra.run.smithi019.stdout: "crush-compat", 2024-05-07T02:43:04.835 INFO:teuthology.orchestra.run.smithi019.stdout: "none", 2024-05-07T02:43:04.835 INFO:teuthology.orchestra.run.smithi019.stdout: "read", 2024-05-07T02:43:04.835 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap", 2024-05-07T02:43:04.835 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap-read" 2024-05-07T02:43:04.835 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:04.836 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Balancer mode", 2024-05-07T02:43:04.836 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.836 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.836 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.836 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.836 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_ids": { 2024-05-07T02:43:04.836 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_ids", 2024-05-07T02:43:04.836 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.836 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.837 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:04.837 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:04.837 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.837 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.837 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.837 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-07T02:43:04.837 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.837 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.837 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.838 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.838 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-07T02:43:04.838 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-07T02:43:04.838 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:04.838 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.838 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:04.838 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-07T02:43:04.838 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.838 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.839 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.839 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-07T02:43:04.839 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.839 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.839 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.839 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.839 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:04.839 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:04.839 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:04.840 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:04.840 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:04.840 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:04.840 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.840 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.840 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.840 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:04.840 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.841 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.841 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.841 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.841 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_deviation": { 2024-05-07T02:43:04.841 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_deviation", 2024-05-07T02:43:04.841 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:04.841 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.841 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:04.841 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-07T02:43:04.842 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-07T02:43:04.842 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.842 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.842 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-07T02:43:04.842 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-07T02:43:04.842 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.842 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.842 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.842 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_optimizations": { 2024-05-07T02:43:04.843 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_optimizations", 2024-05-07T02:43:04.843 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-07T02:43:04.843 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.843 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:04.843 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-07T02:43:04.843 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.843 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.843 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.843 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-07T02:43:04.844 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.844 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.844 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.844 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:04.844 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:04.844 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.844 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:04.844 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm", 2024-05-07T02:43:04.844 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:04.845 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:04.845 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:04.845 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_down_multiplier": { 2024-05-07T02:43:04.845 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_down_multiplier", 2024-05-07T02:43:04.845 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-07T02:43:04.845 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.845 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.845 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-05-07T02:43:04.845 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.845 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.846 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.846 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-07T02:43:04.846 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.846 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.846 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.846 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.846 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_refresh_rate": { 2024-05-07T02:43:04.847 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_refresh_rate", 2024-05-07T02:43:04.847 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:04.847 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.847 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.847 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "20", 2024-05-07T02:43:04.847 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.847 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.847 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.847 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-07T02:43:04.848 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.848 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.848 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.848 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.848 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_starting_port": { 2024-05-07T02:43:04.848 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_starting_port", 2024-05-07T02:43:04.848 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:04.848 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.848 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.848 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4721", 2024-05-07T02:43:04.849 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.849 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.849 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.849 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-07T02:43:04.849 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.849 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.849 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.849 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.849 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_ptrace": { 2024-05-07T02:43:04.850 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_ptrace", 2024-05-07T02:43:04.850 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:04.850 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.850 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.850 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:04.850 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.850 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.850 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.850 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-07T02:43:04.851 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-07T02:43:04.851 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.851 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.851 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.851 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_interval": { 2024-05-07T02:43:04.851 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_interval", 2024-05-07T02:43:04.851 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:04.851 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.851 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.852 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-07T02:43:04.852 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.852 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.852 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.852 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-07T02:43:04.852 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.852 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.852 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.852 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.852 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_memory_target_ratio": { 2024-05-07T02:43:04.853 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_memory_target_ratio", 2024-05-07T02:43:04.853 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-07T02:43:04.853 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.853 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.853 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.7", 2024-05-07T02:43:04.853 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.853 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.853 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.853 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-07T02:43:04.854 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.854 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.854 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.854 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.854 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm_log_destination": { 2024-05-07T02:43:04.854 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm_log_destination", 2024-05-07T02:43:04.854 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.854 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.855 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.855 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:04.855 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.855 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.855 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:04.855 INFO:teuthology.orchestra.run.smithi019.stdout: "file", 2024-05-07T02:43:04.855 INFO:teuthology.orchestra.run.smithi019.stdout: "file,syslog", 2024-05-07T02:43:04.855 INFO:teuthology.orchestra.run.smithi019.stdout: "syslog" 2024-05-07T02:43:04.855 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:04.855 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-07T02:43:04.856 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.856 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.856 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.856 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.856 INFO:teuthology.orchestra.run.smithi019.stdout: "cgroups_split": { 2024-05-07T02:43:04.856 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cgroups_split", 2024-05-07T02:43:04.856 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:04.856 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.856 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.857 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:04.857 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.857 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.857 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.857 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-07T02:43:04.857 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.857 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.857 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.857 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.858 INFO:teuthology.orchestra.run.smithi019.stdout: "config_checks_enabled": { 2024-05-07T02:43:04.858 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_checks_enabled", 2024-05-07T02:43:04.858 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:04.858 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.858 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.858 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:04.858 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.858 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.858 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.859 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-07T02:43:04.859 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.859 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.859 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.859 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.859 INFO:teuthology.orchestra.run.smithi019.stdout: "config_dashboard": { 2024-05-07T02:43:04.859 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_dashboard", 2024-05-07T02:43:04.859 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:04.859 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.859 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.860 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:04.860 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.860 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.860 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.860 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-07T02:43:04.860 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.860 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.860 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.860 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.861 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_alertmanager": { 2024-05-07T02:43:04.861 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_alertmanager", 2024-05-07T02:43:04.861 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.861 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.861 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.861 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-07T02:43:04.861 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.861 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.861 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.862 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-07T02:43:04.862 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.862 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.862 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.862 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.862 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_base": { 2024-05-07T02:43:04.862 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_base", 2024-05-07T02:43:04.862 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.863 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.863 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:04.863 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-07T02:43:04.863 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.863 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.863 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.863 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Container image name, without the tag", 2024-05-07T02:43:04.863 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.863 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.863 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.864 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.864 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_elasticsearch": { 2024-05-07T02:43:04.864 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_elasticsearch", 2024-05-07T02:43:04.864 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.864 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.864 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.864 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-07T02:43:04.864 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.864 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.865 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.865 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "elasticsearch container image", 2024-05-07T02:43:04.865 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.865 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.865 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.865 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.865 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_grafana": { 2024-05-07T02:43:04.865 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_grafana", 2024-05-07T02:43:04.866 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.866 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.866 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.866 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-07T02:43:04.866 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.866 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.866 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.866 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-07T02:43:04.866 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.867 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.867 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.867 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.867 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_haproxy": { 2024-05-07T02:43:04.867 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_haproxy", 2024-05-07T02:43:04.867 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.867 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.867 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.867 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-07T02:43:04.868 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.868 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.868 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.868 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "HAproxy container image", 2024-05-07T02:43:04.868 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.868 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.868 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.868 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.868 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_agent": { 2024-05-07T02:43:04.869 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_agent", 2024-05-07T02:43:04.869 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.869 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.869 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.869 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-07T02:43:04.869 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.869 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.869 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.869 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger agent container image", 2024-05-07T02:43:04.870 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.870 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.870 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.870 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.870 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_collector": { 2024-05-07T02:43:04.870 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_collector", 2024-05-07T02:43:04.870 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.870 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.870 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.871 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-07T02:43:04.871 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.871 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.871 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.871 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger collector container image", 2024-05-07T02:43:04.871 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.871 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.871 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.871 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.872 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_query": { 2024-05-07T02:43:04.872 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_query", 2024-05-07T02:43:04.872 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.872 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.872 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.872 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-07T02:43:04.872 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.872 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.872 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.872 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger query container image", 2024-05-07T02:43:04.873 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.873 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.873 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.873 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.873 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_keepalived": { 2024-05-07T02:43:04.873 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_keepalived", 2024-05-07T02:43:04.873 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.873 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.873 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.874 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-07T02:43:04.874 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.874 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.874 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.874 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Keepalived container image", 2024-05-07T02:43:04.874 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.874 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.874 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.874 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.875 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_loki": { 2024-05-07T02:43:04.875 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_loki", 2024-05-07T02:43:04.875 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.875 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.875 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.875 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-07T02:43:04.875 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.875 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.875 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.876 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Loki container image", 2024-05-07T02:43:04.876 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.876 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.876 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.876 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.876 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_node_exporter": { 2024-05-07T02:43:04.876 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_node_exporter", 2024-05-07T02:43:04.876 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.876 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.876 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.877 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-07T02:43:04.877 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.877 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.877 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.877 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-07T02:43:04.877 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.877 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.877 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.877 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.878 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_nvmeof": { 2024-05-07T02:43:04.878 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_nvmeof", 2024-05-07T02:43:04.878 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.878 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.878 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.878 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-07T02:43:04.878 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.878 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.878 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.879 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Nvme-of container image", 2024-05-07T02:43:04.879 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.879 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.879 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.879 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.879 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_prometheus": { 2024-05-07T02:43:04.879 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_prometheus", 2024-05-07T02:43:04.879 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.879 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.879 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.880 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-07T02:43:04.880 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.880 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.880 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.880 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-07T02:43:04.880 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.880 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.880 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.880 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.881 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_promtail": { 2024-05-07T02:43:04.881 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_promtail", 2024-05-07T02:43:04.881 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.881 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.881 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.881 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-07T02:43:04.881 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.881 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.881 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.882 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Promtail container image", 2024-05-07T02:43:04.882 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.882 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.882 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.882 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.882 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_samba": { 2024-05-07T02:43:04.882 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_samba", 2024-05-07T02:43:04.882 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.882 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.882 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.883 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-07T02:43:04.883 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.883 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.883 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.883 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Samba/SMB container image", 2024-05-07T02:43:04.883 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.883 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.883 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.884 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.884 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_snmp_gateway": { 2024-05-07T02:43:04.884 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_snmp_gateway", 2024-05-07T02:43:04.884 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.884 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.884 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.884 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-07T02:43:04.884 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.884 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.885 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.885 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SNMP Gateway container image", 2024-05-07T02:43:04.885 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.885 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.885 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.885 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.885 INFO:teuthology.orchestra.run.smithi019.stdout: "container_init": { 2024-05-07T02:43:04.885 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_init", 2024-05-07T02:43:04.885 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:04.886 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.886 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.886 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:04.886 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.886 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.886 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.886 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Run podman/docker with `--init`", 2024-05-07T02:43:04.886 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.886 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.887 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.887 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.887 INFO:teuthology.orchestra.run.smithi019.stdout: "daemon_cache_timeout": { 2024-05-07T02:43:04.887 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "daemon_cache_timeout", 2024-05-07T02:43:04.887 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:04.887 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.887 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.887 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-07T02:43:04.887 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.888 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.888 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.888 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-07T02:43:04.888 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.888 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.888 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.888 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.888 INFO:teuthology.orchestra.run.smithi019.stdout: "default_cephadm_command_timeout": { 2024-05-07T02:43:04.888 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_cephadm_command_timeout", 2024-05-07T02:43:04.888 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:04.889 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.889 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.889 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "900", 2024-05-07T02:43:04.889 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.889 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.889 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.889 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-07T02:43:04.889 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.889 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.890 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.890 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.890 INFO:teuthology.orchestra.run.smithi019.stdout: "default_registry": { 2024-05-07T02:43:04.890 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_registry", 2024-05-07T02:43:04.890 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.890 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.890 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.890 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io", 2024-05-07T02:43:04.890 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.891 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.891 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.891 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-07T02:43:04.891 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.891 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.891 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.891 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.891 INFO:teuthology.orchestra.run.smithi019.stdout: "device_cache_timeout": { 2024-05-07T02:43:04.891 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_cache_timeout", 2024-05-07T02:43:04.891 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:04.892 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.892 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.892 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1800", 2024-05-07T02:43:04.892 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.892 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.892 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.892 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache device inventory", 2024-05-07T02:43:04.892 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.893 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.893 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.893 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.893 INFO:teuthology.orchestra.run.smithi019.stdout: "device_enhanced_scan": { 2024-05-07T02:43:04.893 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_enhanced_scan", 2024-05-07T02:43:04.893 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:04.893 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.893 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.893 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:04.893 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.894 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.894 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.894 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-07T02:43:04.894 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.894 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.894 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.894 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.894 INFO:teuthology.orchestra.run.smithi019.stdout: "facts_cache_timeout": { 2024-05-07T02:43:04.894 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "facts_cache_timeout", 2024-05-07T02:43:04.895 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:04.895 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.895 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.895 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-07T02:43:04.895 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.895 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.895 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.895 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache host facts data", 2024-05-07T02:43:04.895 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.896 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.896 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.896 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.896 INFO:teuthology.orchestra.run.smithi019.stdout: "grafana_dashboards_path": { 2024-05-07T02:43:04.896 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "grafana_dashboards_path", 2024-05-07T02:43:04.896 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.896 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.896 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.896 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-07T02:43:04.897 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.897 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.897 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.897 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-07T02:43:04.897 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.897 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.897 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.897 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.897 INFO:teuthology.orchestra.run.smithi019.stdout: "host_check_interval": { 2024-05-07T02:43:04.898 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "host_check_interval", 2024-05-07T02:43:04.898 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:04.898 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.898 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.898 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-07T02:43:04.898 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.898 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.898 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.898 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to perform a host check", 2024-05-07T02:43:04.899 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.899 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.899 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.899 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.899 INFO:teuthology.orchestra.run.smithi019.stdout: "hw_monitoring": { 2024-05-07T02:43:04.899 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hw_monitoring", 2024-05-07T02:43:04.899 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:04.899 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.899 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.900 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:04.900 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.900 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.900 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.900 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-07T02:43:04.900 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.900 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.900 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.900 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.901 INFO:teuthology.orchestra.run.smithi019.stdout: "inventory_list_all": { 2024-05-07T02:43:04.901 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "inventory_list_all", 2024-05-07T02:43:04.901 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:04.901 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.901 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.901 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:04.901 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.901 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.901 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.901 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-07T02:43:04.902 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.902 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.902 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.902 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.902 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:04.902 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:04.902 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.903 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.903 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:04.903 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:04.903 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.903 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.903 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:04.903 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:04.903 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:04.903 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:04.904 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:04.904 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:04.904 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:04.904 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:04.904 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:04.904 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.904 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.904 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.904 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.904 INFO:teuthology.orchestra.run.smithi019.stdout: "log_refresh_metadata": { 2024-05-07T02:43:04.905 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_refresh_metadata", 2024-05-07T02:43:04.905 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:04.905 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.905 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.905 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:04.905 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.905 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.905 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.905 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-07T02:43:04.906 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.906 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.906 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.906 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.906 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:04.906 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:04.906 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:04.910 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.910 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.910 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:04.910 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.911 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.911 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.911 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-07T02:43:04.911 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.911 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.911 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.911 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.911 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:04.911 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:04.911 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.912 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.912 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:04.912 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:04.912 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.912 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.912 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:04.912 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:04.912 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:04.912 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:04.913 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:04.913 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:04.913 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:04.913 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:04.913 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:04.913 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.913 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.913 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.913 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.914 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:04.914 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:04.914 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:04.914 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.914 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:04.914 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:04.914 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.914 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.914 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.915 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:04.915 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.915 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.915 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.915 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.915 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-07T02:43:04.915 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-07T02:43:04.915 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:04.915 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.916 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.916 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:04.916 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.916 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.916 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.916 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-07T02:43:04.916 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.916 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.916 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.917 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.917 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-07T02:43:04.917 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-07T02:43:04.917 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.917 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.917 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.917 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "*", 2024-05-07T02:43:04.917 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.917 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.918 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.918 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-07T02:43:04.918 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.918 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.918 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.918 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.918 INFO:teuthology.orchestra.run.smithi019.stdout: "max_count_per_host": { 2024-05-07T02:43:04.918 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_count_per_host", 2024-05-07T02:43:04.918 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:04.919 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.919 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.919 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-07T02:43:04.919 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.919 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.919 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.919 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of daemons per service per host", 2024-05-07T02:43:04.919 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.919 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.920 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.920 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.920 INFO:teuthology.orchestra.run.smithi019.stdout: "max_osd_draining_count": { 2024-05-07T02:43:04.920 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_osd_draining_count", 2024-05-07T02:43:04.920 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:04.920 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.920 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.920 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-07T02:43:04.920 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.920 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.921 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.921 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-07T02:43:04.921 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.921 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.921 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.921 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.921 INFO:teuthology.orchestra.run.smithi019.stdout: "migration_current": { 2024-05-07T02:43:04.921 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "migration_current", 2024-05-07T02:43:04.921 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:04.922 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.922 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.922 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:04.922 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.922 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.922 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.922 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "internal - do not modify", 2024-05-07T02:43:04.922 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.923 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.923 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.923 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.923 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-05-07T02:43:04.923 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-05-07T02:43:04.923 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.923 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.923 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.923 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "root", 2024-05-07T02:43:04.923 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.924 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.924 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:04.924 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm-package", 2024-05-07T02:43:04.924 INFO:teuthology.orchestra.run.smithi019.stdout: "root" 2024-05-07T02:43:04.924 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:04.924 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "mode for remote execution of cephadm", 2024-05-07T02:43:04.924 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.924 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.924 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.925 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.925 INFO:teuthology.orchestra.run.smithi019.stdout: "oob_default_addr": { 2024-05-07T02:43:04.925 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "oob_default_addr", 2024-05-07T02:43:04.925 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.925 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.925 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.925 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "169.254.1.1", 2024-05-07T02:43:04.925 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.925 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.926 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.926 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-07T02:43:04.926 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.926 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.926 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.926 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.926 INFO:teuthology.orchestra.run.smithi019.stdout: "prometheus_alerts_path": { 2024-05-07T02:43:04.926 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus_alerts_path", 2024-05-07T02:43:04.927 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.927 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.927 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.927 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-07T02:43:04.927 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.927 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.927 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.927 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-07T02:43:04.927 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.927 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.928 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.928 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.928 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_insecure": { 2024-05-07T02:43:04.928 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_insecure", 2024-05-07T02:43:04.928 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:04.928 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.928 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.928 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:04.928 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.929 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.929 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.929 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-07T02:43:04.929 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.929 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.929 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.929 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.929 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_password": { 2024-05-07T02:43:04.929 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_password", 2024-05-07T02:43:04.930 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.930 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.930 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.930 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:04.930 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.930 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.930 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.930 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-07T02:43:04.931 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.931 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.931 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.931 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.931 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_url": { 2024-05-07T02:43:04.931 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_url", 2024-05-07T02:43:04.931 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.931 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.931 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.932 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:04.932 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.932 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.932 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.932 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-07T02:43:04.932 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.932 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.932 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.932 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.932 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_username": { 2024-05-07T02:43:04.933 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_username", 2024-05-07T02:43:04.933 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.933 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.933 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.933 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:04.933 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.933 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.933 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.933 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-07T02:43:04.934 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.934 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.934 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.934 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.934 INFO:teuthology.orchestra.run.smithi019.stdout: "secure_monitoring_stack": { 2024-05-07T02:43:04.934 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "secure_monitoring_stack", 2024-05-07T02:43:04.934 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:04.934 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.934 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.935 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:04.935 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.935 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.935 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.935 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-07T02:43:04.935 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.935 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.935 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.935 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.936 INFO:teuthology.orchestra.run.smithi019.stdout: "service_discovery_port": { 2024-05-07T02:43:04.936 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "service_discovery_port", 2024-05-07T02:43:04.936 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:04.936 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.936 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.936 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8765", 2024-05-07T02:43:04.936 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.936 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.936 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.937 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "cephadm service discovery port", 2024-05-07T02:43:04.937 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.937 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.937 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.937 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.937 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:04.937 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:04.937 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:04.937 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:04.938 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:04.938 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:04.938 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.938 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.938 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.938 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:04.938 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.938 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.938 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.939 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.939 INFO:teuthology.orchestra.run.smithi019.stdout: "ssh_config_file": { 2024-05-07T02:43:04.939 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssh_config_file", 2024-05-07T02:43:04.939 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.939 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.939 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.939 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:04.939 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.939 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.940 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.940 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-07T02:43:04.940 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.940 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.940 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.940 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.940 INFO:teuthology.orchestra.run.smithi019.stdout: "use_agent": { 2024-05-07T02:43:04.940 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_agent", 2024-05-07T02:43:04.940 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:04.941 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.941 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.941 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:04.941 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.941 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.941 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.941 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-07T02:43:04.941 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.941 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.942 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.942 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.942 INFO:teuthology.orchestra.run.smithi019.stdout: "use_repo_digest": { 2024-05-07T02:43:04.942 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_repo_digest", 2024-05-07T02:43:04.942 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:04.942 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.942 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.942 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:04.942 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.943 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.943 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.943 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-07T02:43:04.943 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.943 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.943 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.943 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.943 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_failed_host_check": { 2024-05-07T02:43:04.943 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_failed_host_check", 2024-05-07T02:43:04.943 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:04.944 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.944 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.944 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:04.944 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.944 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.944 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.944 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if the host check fails", 2024-05-07T02:43:04.944 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.944 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.945 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.945 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.945 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_daemons": { 2024-05-07T02:43:04.945 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_daemons", 2024-05-07T02:43:04.945 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:04.945 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.945 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.945 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:04.945 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.946 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.946 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.946 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-07T02:43:04.946 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.946 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.946 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.946 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.946 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_hosts": { 2024-05-07T02:43:04.946 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_hosts", 2024-05-07T02:43:04.947 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:04.947 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.947 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.947 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:04.947 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.947 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.947 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.947 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-07T02:43:04.947 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.948 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.948 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.948 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:04.948 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:04.948 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.948 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:04.948 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crash", 2024-05-07T02:43:04.948 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:04.948 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:04.949 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:04.949 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:04.949 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:04.949 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.949 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.949 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:04.949 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:04.949 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.949 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.949 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:04.950 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:04.950 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:04.950 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:04.950 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:04.950 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:04.950 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:04.950 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:04.950 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:04.951 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.951 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.951 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.951 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.951 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:04.951 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:04.951 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:04.951 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.951 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:04.952 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:04.952 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.952 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.952 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.952 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:04.952 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.952 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.952 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.952 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.953 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:04.953 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:04.953 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.953 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.953 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:04.953 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:04.953 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.953 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.953 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:04.954 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:04.954 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:04.954 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:04.954 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:04.954 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:04.954 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:04.954 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:04.954 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:04.954 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.955 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.955 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.955 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.955 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:04.955 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:04.955 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:04.955 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.955 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:04.955 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:04.956 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.956 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.956 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.956 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:04.956 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.956 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.956 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.956 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.956 INFO:teuthology.orchestra.run.smithi019.stdout: "retain_interval": { 2024-05-07T02:43:04.956 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retain_interval", 2024-05-07T02:43:04.957 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:04.957 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.957 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:04.957 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "31536000", 2024-05-07T02:43:04.957 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.957 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.957 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.957 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-07T02:43:04.957 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.958 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.958 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.958 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.958 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:04.958 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:04.958 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:04.958 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:04.958 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:04.958 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:04.959 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.959 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.959 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.959 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:04.959 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.959 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.959 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.959 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.959 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_recent_interval": { 2024-05-07T02:43:04.960 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_recent_interval", 2024-05-07T02:43:04.960 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:04.960 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.960 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:04.960 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1209600", 2024-05-07T02:43:04.960 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.960 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.960 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.960 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-07T02:43:04.961 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.961 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.961 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.961 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:04.961 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:04.961 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.961 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:04.961 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dashboard", 2024-05-07T02:43:04.961 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:04.962 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:04.962 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:04.962 INFO:teuthology.orchestra.run.smithi019.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-07T02:43:04.962 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-07T02:43:04.962 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:04.962 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.962 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.962 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-07T02:43:04.962 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.963 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.963 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.963 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:04.963 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.963 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.963 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.963 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.963 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_HOST": { 2024-05-07T02:43:04.963 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-07T02:43:04.964 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.964 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.964 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.964 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:04.964 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.964 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.964 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.964 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:04.964 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.965 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.965 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.965 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.965 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-07T02:43:04.965 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-07T02:43:04.965 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:04.965 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.965 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.965 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:04.966 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.966 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.966 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.966 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:04.966 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.966 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.966 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.966 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.966 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_ENABLED": { 2024-05-07T02:43:04.967 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_ENABLED", 2024-05-07T02:43:04.967 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:04.967 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.967 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.967 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:04.967 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.967 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.967 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.967 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:04.968 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.968 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.968 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.968 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.968 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-07T02:43:04.968 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-07T02:43:04.968 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:04.968 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.968 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.969 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:04.969 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.969 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.969 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.969 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:04.969 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.969 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.969 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.969 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.970 INFO:teuthology.orchestra.run.smithi019.stdout: "ENABLE_BROWSABLE_API": { 2024-05-07T02:43:04.970 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-07T02:43:04.970 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:04.970 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.970 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.970 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:04.970 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.970 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.970 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.971 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:04.971 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.971 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.971 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.971 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.971 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-07T02:43:04.971 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-07T02:43:04.971 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:04.971 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.972 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.972 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:04.972 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.972 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.972 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.972 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:04.972 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.972 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.972 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.972 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.973 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-07T02:43:04.973 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-07T02:43:04.973 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:04.973 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.973 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.973 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:04.973 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.973 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.973 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.974 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:04.974 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.974 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.974 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.974 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.974 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-07T02:43:04.974 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-07T02:43:04.974 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:04.974 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.975 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.975 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:04.975 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.975 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.975 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.975 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:04.975 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.975 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.975 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.976 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.976 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-07T02:43:04.976 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-07T02:43:04.976 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:04.976 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.976 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.976 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:04.976 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.976 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.976 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.977 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:04.977 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.977 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.977 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.977 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.977 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-07T02:43:04.977 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-07T02:43:04.977 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:04.977 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.978 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.978 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:04.978 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.978 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.978 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.978 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:04.978 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.978 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.978 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.979 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.979 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-07T02:43:04.979 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-07T02:43:04.979 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:04.979 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.979 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.979 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:04.979 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.979 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.980 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.980 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:04.980 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.980 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.980 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.980 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.980 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-07T02:43:04.980 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-07T02:43:04.980 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:04.981 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.981 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.981 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:04.981 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.981 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.981 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.981 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:04.981 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.981 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.982 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.982 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.982 INFO:teuthology.orchestra.run.smithi019.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-07T02:43:04.982 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-07T02:43:04.982 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.982 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.982 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.982 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:04.982 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.983 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.983 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.983 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:04.983 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.983 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.983 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.983 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.983 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_PASSWORD": { 2024-05-07T02:43:04.983 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-07T02:43:04.984 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.984 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.984 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.984 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-05-07T02:43:04.984 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.984 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.984 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.984 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:04.984 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.985 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.985 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.985 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.985 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-07T02:43:04.985 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-07T02:43:04.985 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:04.985 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.985 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.986 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:04.986 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.986 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.986 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.986 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:04.986 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.986 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.986 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.986 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.987 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_URL": { 2024-05-07T02:43:04.987 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_URL", 2024-05-07T02:43:04.987 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.987 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.987 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.987 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:04.987 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.987 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.987 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.988 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:04.988 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.988 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.988 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.988 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.988 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_USERNAME": { 2024-05-07T02:43:04.988 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-07T02:43:04.988 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.988 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.989 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.989 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-05-07T02:43:04.989 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.989 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.989 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.989 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:04.989 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.989 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.989 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.990 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.990 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-07T02:43:04.990 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-07T02:43:04.990 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.990 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.990 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.990 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:04.990 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.990 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.991 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.991 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:04.991 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.991 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.991 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.991 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.991 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-07T02:43:04.991 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-07T02:43:04.991 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:04.992 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.992 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.992 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:04.992 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.992 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.992 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.992 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:04.992 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.992 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.992 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.993 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.993 INFO:teuthology.orchestra.run.smithi019.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-07T02:43:04.993 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-07T02:43:04.993 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:04.993 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.993 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.993 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:04.993 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.993 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.994 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.994 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:04.994 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.994 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.994 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.994 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.994 INFO:teuthology.orchestra.run.smithi019.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-07T02:43:04.994 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-07T02:43:04.994 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.995 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.995 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.995 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:04.995 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.995 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.995 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.995 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:04.995 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.995 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.996 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.996 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.996 INFO:teuthology.orchestra.run.smithi019.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-07T02:43:04.996 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-07T02:43:04.996 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.996 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.996 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.996 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "[]", 2024-05-07T02:43:04.996 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.996 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.997 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.997 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:04.997 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.997 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.997 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.997 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.997 INFO:teuthology.orchestra.run.smithi019.stdout: "MULTICLUSTER_CONFIG": { 2024-05-07T02:43:04.997 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-07T02:43:04.997 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.998 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.998 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.998 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "{}", 2024-05-07T02:43:04.998 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:04.998 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:04.998 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:04.998 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:04.998 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:04.998 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:04.999 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:04.999 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:04.999 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_HOST": { 2024-05-07T02:43:04.999 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-07T02:43:04.999 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:04.999 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:04.999 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:04.999 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:04.999 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.000 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.000 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.000 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.000 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.000 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.000 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.000 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.000 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-07T02:43:05.000 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-07T02:43:05.001 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.001 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.001 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.001 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.001 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.001 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.001 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.001 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.001 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.001 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.002 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.002 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.002 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-07T02:43:05.002 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-07T02:43:05.002 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.002 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.002 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.002 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.002 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.003 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.003 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.003 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.003 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.003 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.003 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.003 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.003 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-07T02:43:05.003 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-07T02:43:05.004 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.004 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.004 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.004 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.004 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.004 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.004 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.004 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.004 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.005 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.005 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.005 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.005 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-07T02:43:05.005 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-07T02:43:05.005 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.005 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.005 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.005 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.006 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.006 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.006 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.006 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.006 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.006 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.006 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.006 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.007 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-07T02:43:05.007 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-07T02:43:05.007 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.007 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.007 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.007 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.007 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.007 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.007 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.007 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.008 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.008 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.008 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.008 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.008 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-07T02:43:05.008 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-07T02:43:05.008 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.008 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.008 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.009 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.009 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.009 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.009 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.009 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.009 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.009 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.009 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.009 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.009 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-07T02:43:05.010 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-07T02:43:05.010 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.010 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.010 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.010 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.010 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.010 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.010 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.010 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.011 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.011 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.011 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.011 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.011 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-07T02:43:05.011 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-07T02:43:05.011 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.011 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.012 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.012 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.012 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.012 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.012 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.012 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.012 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.012 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.012 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.012 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.013 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_ENABLED": { 2024-05-07T02:43:05.013 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-07T02:43:05.013 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.013 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.013 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.013 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.013 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.013 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.013 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.014 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.014 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.014 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.014 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.014 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.014 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-07T02:43:05.014 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-07T02:43:05.014 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.014 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.015 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.015 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-07T02:43:05.015 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.015 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.015 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.015 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.015 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.015 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.016 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.016 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.016 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-07T02:43:05.016 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-07T02:43:05.016 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.016 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.016 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.016 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-07T02:43:05.016 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.017 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.017 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.017 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.017 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.017 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.017 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.017 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.017 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-07T02:43:05.017 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-07T02:43:05.018 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.018 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.018 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.018 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8", 2024-05-07T02:43:05.018 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.018 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.018 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.018 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.018 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.019 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.019 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.019 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.019 INFO:teuthology.orchestra.run.smithi019.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-07T02:43:05.019 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-07T02:43:05.019 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.019 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.019 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.019 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "45", 2024-05-07T02:43:05.020 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.020 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.020 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.020 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.020 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.020 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.020 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.020 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.020 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ACCESS_KEY": { 2024-05-07T02:43:05.021 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-07T02:43:05.021 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.021 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.021 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.021 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.021 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.021 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.021 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.021 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.022 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.022 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.022 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.022 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.022 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-07T02:43:05.022 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-07T02:43:05.022 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.022 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.022 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.023 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-05-07T02:43:05.023 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.023 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.023 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.023 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.023 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.023 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.023 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.023 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.024 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SECRET_KEY": { 2024-05-07T02:43:05.024 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-07T02:43:05.024 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.024 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.024 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.024 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.024 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.024 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.024 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.025 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.025 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.025 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.025 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.025 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.025 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SSL_VERIFY": { 2024-05-07T02:43:05.025 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-07T02:43:05.025 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.025 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.026 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.026 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.026 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.026 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.026 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.026 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.026 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.026 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.026 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.027 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.027 INFO:teuthology.orchestra.run.smithi019.stdout: "UNSAFE_TLS_v1_2": { 2024-05-07T02:43:05.027 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-07T02:43:05.027 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.027 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.027 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.027 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.027 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.027 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.028 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.028 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.028 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.028 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.028 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.028 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.028 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-07T02:43:05.028 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-07T02:43:05.028 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.028 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.029 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.029 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.029 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.029 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.029 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.029 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.029 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.029 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.029 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.030 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.030 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-07T02:43:05.030 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-07T02:43:05.030 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.030 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.030 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.030 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-07T02:43:05.030 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.030 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.031 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.031 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.031 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.031 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.031 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.031 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.031 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-07T02:43:05.031 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-07T02:43:05.031 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.032 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.032 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.032 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-07T02:43:05.032 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.032 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.032 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.032 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.032 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.032 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.033 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.033 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.033 INFO:teuthology.orchestra.run.smithi019.stdout: "cross_origin_url": { 2024-05-07T02:43:05.033 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cross_origin_url", 2024-05-07T02:43:05.033 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.033 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.033 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.033 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.033 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.033 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.034 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.034 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.034 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.034 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.034 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.034 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.034 INFO:teuthology.orchestra.run.smithi019.stdout: "crt_file": { 2024-05-07T02:43:05.034 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crt_file", 2024-05-07T02:43:05.034 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.035 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.035 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.035 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.035 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.035 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.035 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.035 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.035 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.035 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.036 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.036 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.036 INFO:teuthology.orchestra.run.smithi019.stdout: "debug": { 2024-05-07T02:43:05.036 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "debug", 2024-05-07T02:43:05.036 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.036 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.036 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.036 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.036 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.037 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.037 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.037 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable/disable debug options", 2024-05-07T02:43:05.037 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.037 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.037 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.037 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.037 INFO:teuthology.orchestra.run.smithi019.stdout: "jwt_token_ttl": { 2024-05-07T02:43:05.037 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "jwt_token_ttl", 2024-05-07T02:43:05.037 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.038 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.038 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.038 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "28800", 2024-05-07T02:43:05.038 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.038 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.038 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.038 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.038 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.039 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.039 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.039 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.039 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-05-07T02:43:05.039 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-05-07T02:43:05.039 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.039 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.039 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.039 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.040 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.040 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.040 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.040 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.040 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.040 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.040 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.040 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.040 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.040 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.041 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.041 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.041 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.041 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.041 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.041 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.041 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.041 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.041 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.042 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.042 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.042 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.042 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.042 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.042 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.042 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.042 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.042 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.043 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.043 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.043 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.043 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.043 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.043 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.043 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.043 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.043 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.044 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.044 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.044 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.044 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.044 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.044 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.044 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.044 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.044 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.044 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.045 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.045 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.045 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.045 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.045 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.045 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.045 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.045 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.045 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.046 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.046 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.046 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.046 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.046 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.046 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.046 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.046 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.046 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.047 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.047 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.047 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.047 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.047 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.047 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.047 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.047 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.047 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.048 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.048 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.048 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.048 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.048 INFO:teuthology.orchestra.run.smithi019.stdout: "motd": { 2024-05-07T02:43:05.048 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "motd", 2024-05-07T02:43:05.048 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.048 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.048 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.049 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.049 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.049 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.049 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.049 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "The message of the day", 2024-05-07T02:43:05.049 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.049 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.049 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.049 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.049 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect_resolve_ip_addr": { 2024-05-07T02:43:05.050 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "redirect_resolve_ip_addr", 2024-05-07T02:43:05.050 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.050 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.050 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.050 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.050 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.050 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.050 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.051 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.051 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.051 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.051 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.051 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.051 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-05-07T02:43:05.051 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-05-07T02:43:05.051 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.051 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.051 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.052 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-05-07T02:43:05.052 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.052 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.052 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.052 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.052 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.052 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.052 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.052 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.053 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-05-07T02:43:05.053 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-05-07T02:43:05.053 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.053 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.053 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.053 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8080", 2024-05-07T02:43:05.053 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.053 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.053 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.054 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.054 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.054 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.054 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.054 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.054 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.054 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.054 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.059 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.059 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.059 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.059 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.059 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.059 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.059 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.059 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.060 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.060 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.060 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.060 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-05-07T02:43:05.060 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-05-07T02:43:05.060 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.060 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.060 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.060 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.061 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.061 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.061 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.061 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.061 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.061 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.061 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.061 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.062 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl_server_port": { 2024-05-07T02:43:05.062 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl_server_port", 2024-05-07T02:43:05.062 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.062 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.062 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.062 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8443", 2024-05-07T02:43:05.062 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.062 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.062 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.063 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.063 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.063 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.063 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.063 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.063 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-05-07T02:43:05.063 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-05-07T02:43:05.063 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.063 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.063 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.064 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "redirect", 2024-05-07T02:43:05.064 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.064 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.064 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.064 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.064 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect" 2024-05-07T02:43:05.064 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.064 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.064 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.065 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.065 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.065 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.065 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-05-07T02:43:05.065 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-05-07T02:43:05.065 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.065 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.065 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.065 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-05-07T02:43:05.066 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-05-07T02:43:05.066 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-05-07T02:43:05.066 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.066 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.066 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.066 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.066 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.066 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.067 INFO:teuthology.orchestra.run.smithi019.stdout: "url_prefix": { 2024-05-07T02:43:05.067 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url_prefix", 2024-05-07T02:43:05.067 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.067 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.067 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.067 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.067 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.067 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.067 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.068 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.068 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.068 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.068 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.068 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.068 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.068 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.068 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.068 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "devicehealth", 2024-05-07T02:43:05.069 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.069 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.069 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.069 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_monitoring": { 2024-05-07T02:43:05.069 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_monitoring", 2024-05-07T02:43:05.069 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.069 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.069 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.069 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.070 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.070 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.070 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.070 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "monitor device health metrics", 2024-05-07T02:43:05.070 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.070 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.070 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.070 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.070 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.071 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.071 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.071 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.071 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.071 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.071 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.071 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.071 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.071 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.072 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.072 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.072 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.072 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.072 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.072 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.072 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.072 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.073 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.073 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.073 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.073 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.073 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.073 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.073 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.073 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.073 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.073 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.074 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.074 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.074 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.074 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.074 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.074 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.074 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.074 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.075 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.075 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.075 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.075 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.075 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.075 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.075 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.075 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.075 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.076 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.076 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.076 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.076 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.076 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.076 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.076 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.076 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.076 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.076 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.077 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.077 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.077 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.077 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.077 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.077 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.077 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.077 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.077 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.078 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.078 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.078 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.078 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.078 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.078 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.078 INFO:teuthology.orchestra.run.smithi019.stdout: "mark_out_threshold": { 2024-05-07T02:43:05.078 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mark_out_threshold", 2024-05-07T02:43:05.078 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:05.079 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.079 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.079 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2419200", 2024-05-07T02:43:05.079 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.079 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.079 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.079 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-07T02:43:05.079 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.079 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.080 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.080 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.080 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_name": { 2024-05-07T02:43:05.080 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_name", 2024-05-07T02:43:05.080 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.080 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.080 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.080 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "device_health_metrics", 2024-05-07T02:43:05.080 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.081 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.081 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.081 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-07T02:43:05.081 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.081 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.081 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.081 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.081 INFO:teuthology.orchestra.run.smithi019.stdout: "retention_period": { 2024-05-07T02:43:05.081 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retention_period", 2024-05-07T02:43:05.082 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:05.082 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.082 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.082 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15552000", 2024-05-07T02:43:05.082 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.082 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.082 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.082 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain device health metrics", 2024-05-07T02:43:05.083 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.083 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.083 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.083 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.083 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_frequency": { 2024-05-07T02:43:05.083 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_frequency", 2024-05-07T02:43:05.083 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:05.083 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.083 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.084 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-05-07T02:43:05.084 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.084 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.084 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.084 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-07T02:43:05.084 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.084 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.084 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.084 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.085 INFO:teuthology.orchestra.run.smithi019.stdout: "self_heal": { 2024-05-07T02:43:05.085 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "self_heal", 2024-05-07T02:43:05.085 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.085 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.085 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.085 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.085 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.085 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.085 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.086 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-07T02:43:05.086 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.086 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.086 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.086 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.086 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-07T02:43:05.086 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-07T02:43:05.086 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:05.087 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.087 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.087 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-07T02:43:05.087 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.087 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.087 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.087 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and check device health", 2024-05-07T02:43:05.087 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.087 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.087 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.088 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.088 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.088 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.088 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.088 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.088 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.088 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.088 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.088 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.089 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.089 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.089 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.089 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.089 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.089 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.089 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_threshold": { 2024-05-07T02:43:05.089 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_threshold", 2024-05-07T02:43:05.089 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:05.090 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.090 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.090 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7257600", 2024-05-07T02:43:05.090 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.090 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.090 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.090 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-07T02:43:05.090 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.090 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.091 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.091 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.091 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.091 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.091 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.091 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "diskprediction_local", 2024-05-07T02:43:05.091 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.091 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.091 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.092 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.092 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.092 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.092 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.092 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.092 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.092 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.092 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.093 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.093 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.093 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.093 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.093 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.093 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.093 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.093 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.093 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.094 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.094 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.094 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.094 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.094 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.094 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.094 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.094 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.095 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.095 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.095 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.095 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.095 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.095 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.095 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.095 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.095 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.096 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.096 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.096 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.096 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.096 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.096 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.096 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.096 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.096 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.096 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.097 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.097 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.097 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.097 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.097 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.097 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.097 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.097 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.097 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.098 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.098 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.098 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.098 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.098 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.098 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.098 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.098 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.098 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.099 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.099 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.099 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.099 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.099 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.099 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.099 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.099 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.099 INFO:teuthology.orchestra.run.smithi019.stdout: "predict_interval": { 2024-05-07T02:43:05.100 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predict_interval", 2024-05-07T02:43:05.100 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.100 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.100 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.100 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-05-07T02:43:05.100 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.100 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.100 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.100 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.101 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.101 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.101 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.101 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.101 INFO:teuthology.orchestra.run.smithi019.stdout: "predictor_model": { 2024-05-07T02:43:05.101 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predictor_model", 2024-05-07T02:43:05.101 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.101 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.101 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.102 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "prophetstor", 2024-05-07T02:43:05.102 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.102 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.102 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.102 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.102 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.102 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.102 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.102 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.103 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-07T02:43:05.103 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-07T02:43:05.103 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.103 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.103 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.103 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-07T02:43:05.103 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.103 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.103 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.104 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.104 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.104 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.104 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.104 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.104 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.104 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.104 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.104 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.104 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.105 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.105 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.105 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.105 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.105 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.105 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.105 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.105 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.105 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.106 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.106 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.106 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.106 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "influx", 2024-05-07T02:43:05.106 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": false, 2024-05-07T02:43:05.106 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "influxdb python module not found", 2024-05-07T02:43:05.106 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.106 INFO:teuthology.orchestra.run.smithi019.stdout: "batch_size": { 2024-05-07T02:43:05.107 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "batch_size", 2024-05-07T02:43:05.107 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.107 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.107 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.107 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5000", 2024-05-07T02:43:05.107 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.107 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.107 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.107 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-07T02:43:05.107 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.108 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.108 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.108 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.108 INFO:teuthology.orchestra.run.smithi019.stdout: "database": { 2024-05-07T02:43:05.108 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "database", 2024-05-07T02:43:05.108 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.108 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.108 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.108 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "ceph", 2024-05-07T02:43:05.109 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.109 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.109 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.109 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-07T02:43:05.109 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.109 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.109 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.109 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.109 INFO:teuthology.orchestra.run.smithi019.stdout: "hostname": { 2024-05-07T02:43:05.109 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hostname", 2024-05-07T02:43:05.110 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.110 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.110 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.110 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.110 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.110 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.110 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.110 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server hostname", 2024-05-07T02:43:05.111 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.111 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.111 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.111 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.111 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-07T02:43:05.111 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-07T02:43:05.111 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:05.111 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.111 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.111 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "30", 2024-05-07T02:43:05.112 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "5", 2024-05-07T02:43:05.112 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.112 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.112 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-07T02:43:05.112 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.112 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.112 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.112 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.112 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.113 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.113 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.113 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.113 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.113 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.113 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.113 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.113 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.113 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.114 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.114 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.114 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.114 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.114 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.114 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.114 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.114 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.115 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.115 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.115 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.115 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.115 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.115 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.115 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.115 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.115 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.116 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.116 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.116 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.116 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.116 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.116 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.116 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.116 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.116 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.117 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.117 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.117 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.117 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.117 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.117 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.117 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.117 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.117 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.118 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.118 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.118 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.118 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.118 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.118 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.118 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.118 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.119 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.119 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.119 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.119 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.119 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.119 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.119 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.119 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.119 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.119 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.120 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.120 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.120 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.120 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.120 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.120 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.120 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.120 INFO:teuthology.orchestra.run.smithi019.stdout: "password": { 2024-05-07T02:43:05.120 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "password", 2024-05-07T02:43:05.121 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.121 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.121 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.121 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.121 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.121 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.121 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.121 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "password of InfluxDB server user", 2024-05-07T02:43:05.121 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.122 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.122 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.122 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.122 INFO:teuthology.orchestra.run.smithi019.stdout: "port": { 2024-05-07T02:43:05.122 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "port", 2024-05-07T02:43:05.122 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.122 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.122 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.123 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8086", 2024-05-07T02:43:05.123 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.123 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.123 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.123 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server port", 2024-05-07T02:43:05.123 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.123 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.123 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.123 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.124 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.124 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.124 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.124 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.124 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.124 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.124 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.124 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.124 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.125 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.125 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.125 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.125 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.125 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.125 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-05-07T02:43:05.125 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-05-07T02:43:05.125 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.125 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.125 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.126 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "false", 2024-05-07T02:43:05.126 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.126 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.126 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.126 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-07T02:43:05.126 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.126 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.126 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.127 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.127 INFO:teuthology.orchestra.run.smithi019.stdout: "threads": { 2024-05-07T02:43:05.127 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threads", 2024-05-07T02:43:05.127 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.127 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.127 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.127 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-07T02:43:05.127 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-07T02:43:05.127 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "32", 2024-05-07T02:43:05.128 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.128 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-07T02:43:05.128 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.128 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.128 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.128 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.128 INFO:teuthology.orchestra.run.smithi019.stdout: "username": { 2024-05-07T02:43:05.128 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "username", 2024-05-07T02:43:05.128 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.129 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.129 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.129 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.129 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.129 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.129 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.129 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "username of InfluxDB server user", 2024-05-07T02:43:05.129 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.129 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.130 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.130 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.130 INFO:teuthology.orchestra.run.smithi019.stdout: "verify_ssl": { 2024-05-07T02:43:05.130 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "verify_ssl", 2024-05-07T02:43:05.130 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.130 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.130 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.130 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "true", 2024-05-07T02:43:05.130 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.131 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.131 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.131 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-07T02:43:05.131 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.131 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.131 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.131 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.131 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.131 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.132 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.132 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "insights", 2024-05-07T02:43:05.132 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.132 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.132 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.132 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.132 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.132 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.132 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.133 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.133 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.133 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.133 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.133 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.133 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.133 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.133 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.133 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.134 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.134 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.134 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.134 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.134 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.134 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.134 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.135 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.135 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.135 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.135 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.135 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.135 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.135 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.135 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.135 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.135 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.136 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.136 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.136 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.136 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.136 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.136 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.136 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.136 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.136 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.137 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.137 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.137 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.137 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.137 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.137 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.137 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.137 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.137 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.138 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.138 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.138 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.138 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.138 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.138 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.138 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.138 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.139 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.139 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.139 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.139 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.139 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.139 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.139 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.139 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.139 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.140 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.140 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.140 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.140 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.140 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.140 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.140 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.140 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.140 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.141 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.141 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.141 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.141 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.141 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.141 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.141 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.141 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.141 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.142 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.142 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.142 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.142 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.142 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "iostat", 2024-05-07T02:43:05.142 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.142 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.142 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.143 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.143 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.143 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.143 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.143 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.143 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.143 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.143 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.143 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.144 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.144 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.144 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.144 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.144 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.144 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.144 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.144 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.144 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.145 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.145 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.145 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.145 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.145 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.145 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.145 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.145 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.145 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.145 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.146 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.146 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.146 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.146 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.146 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.146 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.146 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.146 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.147 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.147 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.147 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.147 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.147 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.147 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.147 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.147 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.147 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.147 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.148 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.148 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.148 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.148 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.148 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.148 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.148 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.148 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.148 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.149 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.149 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.149 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.149 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.149 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.149 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.149 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.149 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.149 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.149 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.150 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.150 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.150 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.150 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.150 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.150 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.150 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.150 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.151 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.151 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.151 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.151 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.151 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.151 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.151 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.151 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.151 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.152 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.152 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.152 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.152 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.152 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.152 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "k8sevents", 2024-05-07T02:43:05.152 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.152 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.152 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.152 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_event_retention_days": { 2024-05-07T02:43:05.153 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ceph_event_retention_days", 2024-05-07T02:43:05.153 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.153 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.153 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.153 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7", 2024-05-07T02:43:05.153 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.153 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.153 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.153 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-07T02:43:05.154 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.154 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.154 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.154 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.154 INFO:teuthology.orchestra.run.smithi019.stdout: "config_check_secs": { 2024-05-07T02:43:05.154 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_check_secs", 2024-05-07T02:43:05.154 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.154 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.154 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.155 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-07T02:43:05.155 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "10", 2024-05-07T02:43:05.155 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.155 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.155 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-07T02:43:05.155 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.155 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.155 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.155 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.156 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.156 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.156 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.156 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.156 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.156 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.156 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.156 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.156 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.157 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.157 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.157 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.157 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.157 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.157 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.157 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.157 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.157 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.157 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.158 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.158 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.158 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.158 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.158 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.158 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.158 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.158 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.159 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.159 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.159 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.159 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.159 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.159 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.159 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.159 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.159 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.159 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.160 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.160 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.160 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.160 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.160 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.160 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.160 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.160 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.160 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.161 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.161 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.161 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.161 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.161 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.161 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.161 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.161 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.161 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.162 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.162 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.162 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.162 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.162 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.162 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.162 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.162 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.162 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.163 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.163 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.163 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.163 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.163 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.163 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.163 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.163 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.163 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.164 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.164 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.164 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.164 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.164 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.164 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.164 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.164 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.164 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.165 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.165 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.165 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.165 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.165 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.165 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "localpool", 2024-05-07T02:43:05.165 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.165 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.165 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.166 INFO:teuthology.orchestra.run.smithi019.stdout: "failure_domain": { 2024-05-07T02:43:05.166 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "failure_domain", 2024-05-07T02:43:05.166 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.166 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.166 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.166 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "host", 2024-05-07T02:43:05.166 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.166 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.166 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.167 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "failure domain for any created local pool", 2024-05-07T02:43:05.167 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-07T02:43:05.167 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.167 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.167 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.167 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.167 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.167 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.167 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.168 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.168 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.168 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.168 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.168 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.168 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.168 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.168 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.168 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.169 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.169 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.169 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.169 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.169 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.169 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.169 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.169 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.169 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.170 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.170 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.170 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.170 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.170 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.170 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.170 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.170 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.170 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.171 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.171 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.171 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.171 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.171 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.171 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.171 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.171 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.171 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.172 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.172 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.172 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.172 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.172 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.172 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.172 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.172 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.172 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.172 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.173 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.173 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.173 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.173 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.173 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.173 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.173 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.173 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.173 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.174 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.174 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.174 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.174 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.174 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.174 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.174 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.174 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.174 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.175 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.175 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.175 INFO:teuthology.orchestra.run.smithi019.stdout: "min_size": { 2024-05-07T02:43:05.175 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_size", 2024-05-07T02:43:05.175 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.175 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.175 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.175 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.175 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.176 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.176 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.176 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default min_size for any created local pool", 2024-05-07T02:43:05.176 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-07T02:43:05.176 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.176 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.176 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.176 INFO:teuthology.orchestra.run.smithi019.stdout: "num_rep": { 2024-05-07T02:43:05.176 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "num_rep", 2024-05-07T02:43:05.177 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.177 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.177 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.177 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3", 2024-05-07T02:43:05.177 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.177 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.177 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.177 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default replica count for any created local pool", 2024-05-07T02:43:05.177 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.178 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.178 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.178 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.178 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num": { 2024-05-07T02:43:05.178 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_num", 2024-05-07T02:43:05.178 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.178 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.178 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.178 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "128", 2024-05-07T02:43:05.179 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.179 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.179 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.179 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default pg_num for any created local pool", 2024-05-07T02:43:05.179 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.179 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.179 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.179 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.179 INFO:teuthology.orchestra.run.smithi019.stdout: "prefix": { 2024-05-07T02:43:05.180 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prefix", 2024-05-07T02:43:05.180 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.180 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.180 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.180 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.180 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.180 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.180 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.180 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name prefix for any created local pool", 2024-05-07T02:43:05.181 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.181 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.181 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.181 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.181 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.181 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.181 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.181 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.181 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.182 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.182 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.182 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.182 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.182 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.182 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.182 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.182 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.182 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.183 INFO:teuthology.orchestra.run.smithi019.stdout: "subtree": { 2024-05-07T02:43:05.183 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "subtree", 2024-05-07T02:43:05.183 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.183 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.183 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.183 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "rack", 2024-05-07T02:43:05.183 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.183 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.183 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.184 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-07T02:43:05.184 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-07T02:43:05.184 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.184 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.184 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.184 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.184 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.184 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.184 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mds_autoscaler", 2024-05-07T02:43:05.185 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.185 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.185 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.185 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.185 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.185 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.185 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.185 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.185 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.186 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.186 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.186 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.186 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.186 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.186 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.186 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.186 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.186 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.187 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.187 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.187 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.187 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.187 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.187 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.187 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.187 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.187 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.188 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.188 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.188 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.188 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.188 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.188 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.188 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.188 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.188 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.189 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.189 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.189 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.189 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.189 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.189 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.189 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.189 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.189 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.190 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.190 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.190 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.190 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.190 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.190 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.190 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.190 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.190 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.191 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.191 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.191 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.191 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.191 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.191 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.191 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.191 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.191 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.192 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.192 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.192 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.192 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.192 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.192 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.192 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.192 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.192 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.193 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.193 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.193 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.193 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.193 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.193 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.193 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.193 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.193 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.193 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.194 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.194 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.194 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.194 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.194 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.194 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.194 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.194 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.194 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirroring", 2024-05-07T02:43:05.195 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.195 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.195 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.195 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.195 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.195 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.195 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.195 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.195 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.195 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.196 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.196 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.196 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.196 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.196 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.196 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.196 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.196 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.196 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.197 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.197 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.197 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.197 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.197 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.197 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.197 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.197 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.197 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.198 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.198 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.198 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.198 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.198 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.198 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.198 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.198 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.199 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.199 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.199 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.199 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.199 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.199 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.199 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.199 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.199 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.200 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.200 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.200 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.200 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.200 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.200 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.200 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.200 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.200 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.201 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.201 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.201 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.201 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.201 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.201 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.201 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.201 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.201 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.201 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.202 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.202 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.202 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.202 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.202 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.202 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.202 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.202 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.203 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.203 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.203 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.203 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.203 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.203 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.203 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.203 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.204 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.204 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.204 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.204 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.204 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.204 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.204 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.204 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.204 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.205 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.205 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "nfs", 2024-05-07T02:43:05.205 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.205 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.205 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.205 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.205 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.205 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.205 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.206 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.206 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.206 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.206 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.206 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.206 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.206 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.206 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.206 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.207 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.207 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.207 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.207 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.207 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.207 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.207 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.207 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.207 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.208 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.208 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.208 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.208 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.208 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.208 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.208 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.208 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.208 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.209 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.209 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.209 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.209 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.209 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.209 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.209 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.209 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.209 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.210 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.210 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.210 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.210 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.210 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.210 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.210 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.210 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.210 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.211 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.211 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.211 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.211 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.211 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.211 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.211 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.211 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.211 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.212 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.212 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.212 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.212 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.212 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.212 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.212 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.212 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.212 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.213 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.213 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.213 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.213 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.213 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.213 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.213 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.213 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.213 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.214 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.214 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.214 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.214 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.214 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.214 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.214 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.214 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.214 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.215 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.215 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.215 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-05-07T02:43:05.215 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.215 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.215 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.215 INFO:teuthology.orchestra.run.smithi019.stdout: "fail_fs": { 2024-05-07T02:43:05.215 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "fail_fs", 2024-05-07T02:43:05.215 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.216 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.216 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.216 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.216 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.216 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.216 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.216 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-07T02:43:05.216 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.216 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.217 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.217 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.217 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.217 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.217 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.217 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.217 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.217 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.217 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.218 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.218 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.218 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.218 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.218 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.218 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.218 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.218 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.218 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.219 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.219 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.219 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.219 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.219 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.219 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.219 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.219 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.219 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.220 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.220 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.220 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.220 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.220 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.220 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.220 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.220 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.221 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.221 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.221 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.221 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.221 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.221 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.221 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.221 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.221 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.222 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.222 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.222 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.222 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.222 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.222 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.222 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.222 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.222 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.223 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.223 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.223 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.223 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.223 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.223 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.223 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.223 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.223 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.224 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.224 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.224 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.224 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.224 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.224 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.224 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.224 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.224 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.225 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.225 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator": { 2024-05-07T02:43:05.225 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-05-07T02:43:05.225 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.225 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.225 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.225 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.225 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.225 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.226 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.226 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm", 2024-05-07T02:43:05.226 INFO:teuthology.orchestra.run.smithi019.stdout: "rook", 2024-05-07T02:43:05.226 INFO:teuthology.orchestra.run.smithi019.stdout: "test_orchestrator" 2024-05-07T02:43:05.226 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.226 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Orchestrator backend", 2024-05-07T02:43:05.226 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.226 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.226 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.227 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.227 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.227 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.227 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.227 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.227 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.227 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.227 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.228 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.228 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.228 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.228 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.228 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.228 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.228 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.228 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.228 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.229 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.229 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_perf_query", 2024-05-07T02:43:05.229 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.229 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.229 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.229 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.229 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.229 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.229 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.230 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.230 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.230 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.230 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.230 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.230 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.230 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.230 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.230 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.231 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.231 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.231 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.231 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.231 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.231 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.231 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.231 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.231 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.232 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.232 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.232 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.232 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.232 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.232 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.232 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.232 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.232 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.233 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.233 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.233 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.233 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.233 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.233 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.233 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.233 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.233 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.234 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.234 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.234 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.234 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.234 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.234 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.234 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.234 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.235 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.235 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.235 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.235 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.235 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.235 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.235 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.235 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.235 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.236 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.236 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.236 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.236 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.236 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.236 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.236 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.236 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.236 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.237 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.237 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.237 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.237 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.237 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.237 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.237 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.237 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.237 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.238 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.238 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.238 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.238 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.238 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.238 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.238 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.238 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.238 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.239 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.239 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.239 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.239 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_support", 2024-05-07T02:43:05.239 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.239 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.239 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.239 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.239 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.240 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.240 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.240 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.240 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.240 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.240 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.240 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.240 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.241 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.241 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.241 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.241 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.241 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.241 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.241 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.241 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.241 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.242 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.242 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.242 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.242 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.242 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.242 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.242 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.242 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.243 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.243 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.243 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.243 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.243 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.243 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.243 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.243 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.243 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.244 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.244 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.244 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.244 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.244 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.244 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.244 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.244 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.244 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.245 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.245 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.245 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.245 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.245 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.245 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.245 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.245 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.245 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.246 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.246 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.246 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.246 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.246 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.246 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.246 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.246 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.246 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.247 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.247 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.247 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.247 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.247 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.247 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.247 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.247 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.247 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.248 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.248 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.248 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.248 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.248 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.248 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.248 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.248 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.248 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.249 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.249 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.249 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.249 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.249 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.249 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.249 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_autoscaler", 2024-05-07T02:43:05.249 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.249 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.250 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.250 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.250 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.250 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.250 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.250 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.250 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.250 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.250 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.251 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.251 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.251 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.251 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.251 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.251 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.251 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.251 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.251 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.252 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.252 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.252 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.252 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.252 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.252 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.252 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.252 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.252 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.253 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.253 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.253 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.253 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.253 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.253 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.253 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.253 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.253 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.254 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.254 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.254 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.254 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.254 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.254 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.254 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.254 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.254 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.255 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.255 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.255 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.255 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.255 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.255 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.255 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.255 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.255 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.256 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.256 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.256 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.256 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.256 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.256 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.256 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.256 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.256 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.256 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.257 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.257 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.257 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.257 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.257 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.257 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.257 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.257 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-07T02:43:05.257 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-07T02:43:05.258 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:05.258 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.258 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.258 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-07T02:43:05.258 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.258 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.258 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.258 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.258 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.259 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.259 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.259 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.259 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.259 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.259 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.259 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.259 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.259 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.260 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.260 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.260 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.260 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.260 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.260 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.260 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.260 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.260 INFO:teuthology.orchestra.run.smithi019.stdout: "threshold": { 2024-05-07T02:43:05.261 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threshold", 2024-05-07T02:43:05.261 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-07T02:43:05.261 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.261 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.261 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-05-07T02:43:05.261 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1.0", 2024-05-07T02:43:05.261 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.261 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.261 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "scaling threshold", 2024-05-07T02:43:05.262 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-07T02:43:05.262 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.262 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.262 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.262 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.262 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.262 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.262 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "progress", 2024-05-07T02:43:05.262 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.263 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.263 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.263 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_pg_recovery_event": { 2024-05-07T02:43:05.263 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_pg_recovery_event", 2024-05-07T02:43:05.263 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.263 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.263 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.263 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.263 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.264 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.264 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.264 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-07T02:43:05.264 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.264 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.264 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.264 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.264 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-05-07T02:43:05.264 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-05-07T02:43:05.265 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.265 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.265 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.265 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.265 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.265 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.265 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.265 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.265 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.266 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.266 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.266 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.266 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.266 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.266 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.266 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.266 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.266 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.267 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.267 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.267 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.267 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.267 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.267 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.267 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.267 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.267 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.268 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.268 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.268 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.268 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.268 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.268 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.268 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.268 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.268 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.269 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.269 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.269 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.269 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.269 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.269 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.269 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.269 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.270 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.270 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.270 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.270 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.270 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.270 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.270 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.270 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.270 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.271 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.271 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.271 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.271 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.271 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.271 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.271 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.271 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.271 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.272 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.272 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.272 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.272 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.272 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.272 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.272 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.272 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.272 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.273 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.273 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.273 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.273 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.273 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.273 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.273 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.273 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.273 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.274 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.274 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.274 INFO:teuthology.orchestra.run.smithi019.stdout: "max_completed_events": { 2024-05-07T02:43:05.274 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_completed_events", 2024-05-07T02:43:05.274 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.274 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.274 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.274 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "50", 2024-05-07T02:43:05.274 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.275 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.275 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.275 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "number of past completed events to remember", 2024-05-07T02:43:05.275 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.275 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.275 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.275 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.275 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-07T02:43:05.275 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-07T02:43:05.276 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:05.276 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.276 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.276 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-07T02:43:05.276 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.276 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.276 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.276 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long the module is going to sleep", 2024-05-07T02:43:05.276 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.277 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.277 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.277 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.277 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.277 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.277 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.277 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.277 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.277 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.278 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.278 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.278 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.278 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.278 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.278 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.278 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.278 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.279 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.279 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.279 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.279 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus", 2024-05-07T02:43:05.279 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.279 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.279 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.279 INFO:teuthology.orchestra.run.smithi019.stdout: "cache": { 2024-05-07T02:43:05.279 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cache", 2024-05-07T02:43:05.280 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.280 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.280 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.280 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.280 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.280 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.280 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.280 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.280 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.281 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.281 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.281 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.281 INFO:teuthology.orchestra.run.smithi019.stdout: "exclude_perf_counters": { 2024-05-07T02:43:05.281 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "exclude_perf_counters", 2024-05-07T02:43:05.281 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.281 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.281 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.281 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.281 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.282 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.282 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.282 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-07T02:43:05.282 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-07T02:43:05.282 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.282 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.282 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.282 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.283 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.283 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.283 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.283 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.283 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.283 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.283 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.283 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.283 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.284 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.284 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.284 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.284 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.284 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.284 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.284 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.285 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.285 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.285 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.285 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.285 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.285 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.285 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.285 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.285 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.286 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.286 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.286 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.286 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.286 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.286 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.286 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.286 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.287 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.287 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.287 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.287 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.287 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.287 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.287 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.287 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.288 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.288 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.288 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.288 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.288 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.288 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.288 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.288 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.288 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.288 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.289 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.289 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.289 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.289 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.289 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.289 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.289 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.289 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.290 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.290 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.290 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.290 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.290 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.290 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.290 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.290 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.290 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.291 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.291 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools": { 2024-05-07T02:43:05.291 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools", 2024-05-07T02:43:05.291 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.291 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.291 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.291 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.291 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.291 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.292 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.292 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.292 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.292 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.292 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.292 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.292 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-07T02:43:05.292 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-07T02:43:05.292 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.293 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.293 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.293 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "300", 2024-05-07T02:43:05.293 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.293 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.293 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.293 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.293 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.294 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.294 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.294 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.294 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_interval": { 2024-05-07T02:43:05.294 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_interval", 2024-05-07T02:43:05.294 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-07T02:43:05.294 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.294 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.294 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15.0", 2024-05-07T02:43:05.295 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.295 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.295 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.295 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.295 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.295 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.295 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.295 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.295 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-05-07T02:43:05.296 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-05-07T02:43:05.296 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.296 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.296 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.296 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-05-07T02:43:05.296 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.296 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.296 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.296 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-07T02:43:05.297 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.297 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.297 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.297 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.297 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-05-07T02:43:05.297 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-05-07T02:43:05.297 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.297 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.297 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.298 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "9283", 2024-05-07T02:43:05.298 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.298 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.298 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.298 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-07T02:43:05.298 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.298 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.298 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.298 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.299 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.299 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.299 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.299 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.299 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.299 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.299 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.299 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.299 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.300 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.300 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.300 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.300 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.300 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.300 INFO:teuthology.orchestra.run.smithi019.stdout: "stale_cache_strategy": { 2024-05-07T02:43:05.300 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stale_cache_strategy", 2024-05-07T02:43:05.300 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.300 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.301 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.301 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "log", 2024-05-07T02:43:05.301 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.301 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.301 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.301 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.301 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.301 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.301 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.301 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.302 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-05-07T02:43:05.302 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-05-07T02:43:05.302 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.302 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.302 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.302 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "default", 2024-05-07T02:43:05.302 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.302 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.302 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.303 INFO:teuthology.orchestra.run.smithi019.stdout: "default", 2024-05-07T02:43:05.303 INFO:teuthology.orchestra.run.smithi019.stdout: "error" 2024-05-07T02:43:05.303 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.303 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.303 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.303 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.303 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.303 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.303 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-05-07T02:43:05.304 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-05-07T02:43:05.304 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.304 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.304 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.304 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-05-07T02:43:05.304 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-05-07T02:43:05.304 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-05-07T02:43:05.304 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.304 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.305 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.305 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.305 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.305 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.305 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.305 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.305 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.305 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_support", 2024-05-07T02:43:05.305 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.306 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.306 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.306 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.306 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.306 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.306 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.306 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.306 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.307 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.307 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.307 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.307 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.307 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.307 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.307 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.307 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.307 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.307 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.308 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.308 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.308 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.308 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.308 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.308 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.308 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.308 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.308 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.309 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.309 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.309 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.309 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.309 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.309 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.309 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.309 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.309 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.310 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.310 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.310 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.310 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.310 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.310 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.310 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.310 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.310 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.311 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.311 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.311 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.311 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.311 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.311 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.311 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.311 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.311 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.312 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.312 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.312 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.312 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.312 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.312 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.312 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.312 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.312 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.313 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.313 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.313 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.313 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.313 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.313 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.313 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.313 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.313 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.314 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_snap_create": { 2024-05-07T02:43:05.314 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_snap_create", 2024-05-07T02:43:05.314 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.314 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.314 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.318 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-07T02:43:05.318 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.318 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.318 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.319 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.319 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.319 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.319 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.319 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.319 INFO:teuthology.orchestra.run.smithi019.stdout: "mirror_snapshot_schedule": { 2024-05-07T02:43:05.319 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirror_snapshot_schedule", 2024-05-07T02:43:05.319 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.319 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.320 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.320 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.320 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.320 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.320 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.320 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.320 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.320 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.320 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.321 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.321 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.321 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.321 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.321 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.321 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.321 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.321 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.321 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.322 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.322 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.322 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.322 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.322 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.322 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.322 INFO:teuthology.orchestra.run.smithi019.stdout: "trash_purge_schedule": { 2024-05-07T02:43:05.322 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "trash_purge_schedule", 2024-05-07T02:43:05.322 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.323 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.323 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.323 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.323 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.323 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.323 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.323 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.323 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.323 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.324 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.324 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.324 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.324 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.324 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.324 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "restful", 2024-05-07T02:43:05.324 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.324 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.324 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.325 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_auth": { 2024-05-07T02:43:05.325 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_auth", 2024-05-07T02:43:05.325 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.325 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.325 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.325 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.325 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.325 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.325 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.326 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.326 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.326 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.326 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.326 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.326 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-05-07T02:43:05.326 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-05-07T02:43:05.326 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.326 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.327 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.327 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.327 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.327 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.327 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.327 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.327 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.327 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.327 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.327 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.328 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.328 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.328 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.328 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.328 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.328 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.328 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.328 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.328 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.329 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.329 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.329 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.329 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.329 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.329 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.329 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.329 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.329 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.330 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.330 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.330 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.330 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.330 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.330 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.330 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.330 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.330 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.331 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.331 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.331 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.331 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.331 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.331 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.331 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.331 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.332 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.332 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.332 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.332 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.332 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.332 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.332 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.332 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.332 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.333 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.333 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.333 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.333 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.333 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.333 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.333 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.333 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.333 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.334 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.334 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.334 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.334 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.334 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.334 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.334 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.334 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.334 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.335 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.335 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.335 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.335 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.335 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.335 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.335 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.335 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.335 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-05-07T02:43:05.336 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-05-07T02:43:05.336 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.336 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.336 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.336 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.336 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.336 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.336 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.336 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.337 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.337 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.337 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.337 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.337 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-05-07T02:43:05.337 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-05-07T02:43:05.337 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.337 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.337 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.338 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.338 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.338 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.338 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.338 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.338 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.338 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.338 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.338 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.339 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.339 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.339 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.339 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.339 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.339 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.339 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.339 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.339 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.340 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.340 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.340 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.340 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.340 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.340 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.340 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.340 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.340 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rgw", 2024-05-07T02:43:05.341 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.341 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.341 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.341 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.341 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.341 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.341 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.341 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.341 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.342 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.342 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.342 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.342 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.342 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.342 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.342 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.342 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.343 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.343 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.343 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.343 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.343 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.343 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.343 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.343 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.343 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.344 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.344 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.344 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.344 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.344 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.344 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.344 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.344 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.344 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.345 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.345 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.345 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.345 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.345 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.345 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.345 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.345 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.345 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.345 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.346 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.346 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.346 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.346 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.346 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.346 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.346 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.346 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.346 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.347 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.347 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.347 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.347 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.347 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.347 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.347 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.347 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.347 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.348 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.348 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.348 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.348 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.348 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.348 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.348 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.348 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.348 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.349 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.349 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.349 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.349 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.349 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.349 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.349 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.349 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.349 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.350 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.350 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.350 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.350 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.350 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.350 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.350 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.350 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.350 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.351 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rook", 2024-05-07T02:43:05.351 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.351 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.351 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.351 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.351 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.351 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.351 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.351 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.352 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.352 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.352 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.352 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.352 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.352 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.352 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.352 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.352 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.353 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.353 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.353 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.353 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.353 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.353 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.353 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.353 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.354 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.354 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.354 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.354 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.354 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.354 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.354 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.354 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.354 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.355 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.355 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.355 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.355 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.355 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.355 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.355 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.355 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.355 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.356 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.356 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.356 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.356 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.356 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.356 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.356 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.356 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.356 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.357 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.357 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.357 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.357 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.357 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.357 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.357 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.357 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.357 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.358 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.358 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.358 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.358 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.358 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.358 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.358 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.358 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.359 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.359 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.359 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.359 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.359 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.359 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.359 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.359 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.359 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.359 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.360 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.360 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.360 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.360 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.360 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.360 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.360 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.360 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.361 INFO:teuthology.orchestra.run.smithi019.stdout: "storage_class": { 2024-05-07T02:43:05.361 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "storage_class", 2024-05-07T02:43:05.361 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.361 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.361 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.361 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "local", 2024-05-07T02:43:05.361 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.361 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.361 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.361 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-07T02:43:05.362 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.362 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.362 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.362 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.362 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.362 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.362 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.362 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "selftest", 2024-05-07T02:43:05.363 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.363 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.363 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.363 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.363 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.363 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.363 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.363 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.363 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.363 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.364 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.364 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.364 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.364 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.364 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.364 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.364 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.364 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.364 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.365 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.365 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.365 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.365 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.365 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.365 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.365 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.365 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.365 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.366 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.366 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.366 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.366 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.366 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.366 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.366 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.366 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.366 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.367 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.367 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.367 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.367 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.367 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.367 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.367 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.367 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.367 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.368 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.368 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.368 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.368 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.368 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.368 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.368 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.368 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.368 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.369 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.369 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.369 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.369 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.369 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.369 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.369 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.369 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.369 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.369 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.370 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.370 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.370 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.370 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.370 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.370 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.370 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.370 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.371 INFO:teuthology.orchestra.run.smithi019.stdout: "roption1": { 2024-05-07T02:43:05.371 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption1", 2024-05-07T02:43:05.371 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.371 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.371 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.371 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.371 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.371 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.371 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.372 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.372 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.372 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.372 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.372 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.372 INFO:teuthology.orchestra.run.smithi019.stdout: "roption2": { 2024-05-07T02:43:05.372 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption2", 2024-05-07T02:43:05.372 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.372 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.373 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.373 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "xyz", 2024-05-07T02:43:05.373 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.373 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.373 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.373 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.373 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.373 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.373 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.373 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.374 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption1": { 2024-05-07T02:43:05.374 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption1", 2024-05-07T02:43:05.374 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.374 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.374 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.374 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.374 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.374 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.375 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.375 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.375 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.375 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.375 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.375 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.375 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption2": { 2024-05-07T02:43:05.375 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption2", 2024-05-07T02:43:05.375 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.376 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.376 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.376 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.376 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.376 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.376 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.376 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.376 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.376 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.377 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.377 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.377 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption3": { 2024-05-07T02:43:05.377 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption3", 2024-05-07T02:43:05.377 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-07T02:43:05.377 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.377 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.377 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.377 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.378 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.378 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.378 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.378 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.378 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.378 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.378 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.378 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption4": { 2024-05-07T02:43:05.379 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption4", 2024-05-07T02:43:05.379 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.379 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.379 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.379 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.379 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.379 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.379 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.379 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.380 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.380 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.380 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.380 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.380 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption5": { 2024-05-07T02:43:05.380 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption5", 2024-05-07T02:43:05.380 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.380 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.380 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.380 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.381 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.381 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.381 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.381 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.381 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.381 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.381 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.381 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.381 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption6": { 2024-05-07T02:43:05.382 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption6", 2024-05-07T02:43:05.382 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.382 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.382 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.382 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.382 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.382 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.382 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.383 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.383 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.383 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.383 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.383 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.383 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption7": { 2024-05-07T02:43:05.383 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption7", 2024-05-07T02:43:05.383 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.383 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.384 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.384 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.384 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-07T02:43:05.384 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "42", 2024-05-07T02:43:05.384 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.384 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.384 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.384 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.384 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.384 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.385 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.385 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.385 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.385 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.385 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.385 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.385 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.385 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.385 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.386 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.386 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.386 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.386 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.386 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.386 INFO:teuthology.orchestra.run.smithi019.stdout: "testkey": { 2024-05-07T02:43:05.386 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testkey", 2024-05-07T02:43:05.386 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.387 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.387 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.387 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.387 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.387 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.387 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.387 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.387 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.387 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.387 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.388 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.388 INFO:teuthology.orchestra.run.smithi019.stdout: "testlkey": { 2024-05-07T02:43:05.388 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testlkey", 2024-05-07T02:43:05.388 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.388 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.388 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.388 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.388 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.389 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.389 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.389 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.389 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.389 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.389 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.389 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.389 INFO:teuthology.orchestra.run.smithi019.stdout: "testnewline": { 2024-05-07T02:43:05.389 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testnewline", 2024-05-07T02:43:05.389 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.390 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.390 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.390 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.390 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.390 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.390 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.390 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.390 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.390 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.391 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.391 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.391 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.391 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.391 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.391 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smb", 2024-05-07T02:43:05.391 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.391 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.391 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.392 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.392 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.392 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.392 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.392 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.392 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.392 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.392 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.392 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.393 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.393 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.393 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.393 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.393 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.393 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.393 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.393 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.393 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.394 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.394 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.394 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.394 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.394 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.394 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.394 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.394 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.394 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.395 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.395 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.395 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.395 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.395 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.395 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.395 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.395 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.395 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.396 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.396 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.396 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.396 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.396 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.396 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.396 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.396 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.397 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.397 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.397 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.397 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.397 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.397 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.397 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.397 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.397 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.398 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.398 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.398 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.398 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.398 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.398 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.398 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.398 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.399 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.399 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.399 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.399 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.399 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.399 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.399 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.399 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.399 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.400 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.400 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.400 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.400 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.400 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.400 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.400 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.400 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.400 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.401 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.401 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.401 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.401 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.401 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.401 INFO:teuthology.orchestra.run.smithi019.stdout: "update_orchestration": { 2024-05-07T02:43:05.401 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "update_orchestration", 2024-05-07T02:43:05.401 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.401 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.402 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.402 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.402 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.402 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.402 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.402 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-07T02:43:05.402 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.402 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.403 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.403 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.403 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.403 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.403 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.403 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snap_schedule", 2024-05-07T02:43:05.403 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.403 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.403 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.403 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_m_granularity": { 2024-05-07T02:43:05.404 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_m_granularity", 2024-05-07T02:43:05.404 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.404 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.404 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.404 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.404 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.404 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.404 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.404 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow minute scheduled snapshots", 2024-05-07T02:43:05.405 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.405 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.405 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.405 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.405 INFO:teuthology.orchestra.run.smithi019.stdout: "dump_on_update": { 2024-05-07T02:43:05.405 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dump_on_update", 2024-05-07T02:43:05.405 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.405 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.405 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.406 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.406 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.406 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.406 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.406 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "dump database to debug log on update", 2024-05-07T02:43:05.406 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.406 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.406 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.406 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.407 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.407 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.407 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.407 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.407 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.407 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.407 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.407 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.408 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.408 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.408 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.408 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.408 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.408 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.408 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.408 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.408 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.409 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.409 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.409 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.409 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.409 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.409 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.409 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.409 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.409 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.409 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.410 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.410 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.410 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.410 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.410 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.410 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.410 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.410 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.411 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.411 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.411 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.411 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.411 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.411 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.411 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.411 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.411 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.411 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.412 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.412 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.412 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.412 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.412 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.412 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.412 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.412 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.412 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.413 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.413 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.413 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.413 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.413 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.413 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.413 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.413 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.413 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.414 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.414 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.414 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.414 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.414 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.414 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.414 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.414 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.414 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.415 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.415 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.415 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.415 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.415 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.415 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.415 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.415 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.415 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.416 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.416 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.416 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.416 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.416 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.416 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.416 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stats", 2024-05-07T02:43:05.416 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.416 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.417 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.417 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.417 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.417 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.417 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.417 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.417 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.417 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.417 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.417 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.418 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.418 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.418 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.418 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.418 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.418 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.418 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.418 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.419 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.419 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.419 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.419 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.419 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.419 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.419 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.419 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.419 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.420 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.420 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.420 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.420 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.420 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.420 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.420 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.420 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.421 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.421 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.421 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.421 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.421 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.421 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.421 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.421 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.421 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.422 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.422 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.422 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.422 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.422 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.422 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.422 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.422 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.422 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.423 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.423 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.423 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.423 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.423 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.423 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.423 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.423 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.423 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.424 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.424 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.424 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.424 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.424 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.424 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.424 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.424 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.424 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.424 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.425 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.425 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.425 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.425 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.425 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.425 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.425 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.425 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.425 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.426 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.426 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.426 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.426 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.426 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.426 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.426 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.426 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "status", 2024-05-07T02:43:05.426 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.427 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.427 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.427 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.427 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.427 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.427 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.427 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.427 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.427 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.428 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.428 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.428 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.428 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.428 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.428 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.428 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.428 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.428 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.429 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.429 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.429 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.429 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.429 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.429 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.429 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.429 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.429 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.430 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.430 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.430 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.430 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.430 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.430 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.430 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.430 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.430 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.431 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.431 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.431 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.431 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.431 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.431 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.431 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.431 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.431 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.432 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.432 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.432 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.432 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.432 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.432 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.432 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.432 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.432 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.432 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.433 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.433 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.433 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.433 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.433 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.433 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.433 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.433 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.434 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.434 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.434 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.434 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.434 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.434 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.434 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.434 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.434 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.435 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.435 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.435 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.435 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.435 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.435 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.435 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.435 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.435 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.436 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.436 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.436 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.436 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.436 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.436 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.436 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.436 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.436 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telegraf", 2024-05-07T02:43:05.437 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.437 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.437 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.437 INFO:teuthology.orchestra.run.smithi019.stdout: "address": { 2024-05-07T02:43:05.437 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "address", 2024-05-07T02:43:05.437 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.437 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.437 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.437 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-07T02:43:05.438 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.438 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.438 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.438 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.438 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.438 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.438 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.438 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.438 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-07T02:43:05.439 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-07T02:43:05.439 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:05.439 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.439 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.439 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15", 2024-05-07T02:43:05.439 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.439 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.439 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.439 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.440 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.440 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.440 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.440 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.440 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.440 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.440 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.440 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.441 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.441 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.441 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.441 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.441 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.441 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.441 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.441 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.441 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.442 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.442 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.442 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.442 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.442 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.442 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.442 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.442 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.442 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.443 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.443 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.443 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.443 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.443 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.443 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.443 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.443 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.443 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.444 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.444 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.444 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.444 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.444 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.444 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.444 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.444 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.444 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.445 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.445 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.445 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.445 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.445 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.445 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.445 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.445 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.445 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.445 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.446 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.446 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.446 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.446 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.446 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.446 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.446 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.446 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.447 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.447 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.447 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.447 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.447 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.447 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.447 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.447 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.447 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.448 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.448 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.448 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.448 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.448 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.448 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.448 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.448 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.448 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.449 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.449 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.449 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.449 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.449 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.449 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.449 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.449 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.449 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.450 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.450 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.450 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telemetry", 2024-05-07T02:43:05.450 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.450 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.450 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.450 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_basic": { 2024-05-07T02:43:05.450 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_basic", 2024-05-07T02:43:05.450 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.451 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.451 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.451 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.451 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.451 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.451 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.451 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-07T02:43:05.451 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.451 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.452 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.452 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.452 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_crash": { 2024-05-07T02:43:05.452 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_crash", 2024-05-07T02:43:05.452 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.452 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.452 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.452 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.452 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.453 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.453 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.453 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-07T02:43:05.453 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.453 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.453 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.453 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.453 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_device": { 2024-05-07T02:43:05.453 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_device", 2024-05-07T02:43:05.454 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.454 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.454 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.454 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.454 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.454 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.454 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.454 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-07T02:43:05.454 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.455 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.455 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.455 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.455 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_ident": { 2024-05-07T02:43:05.455 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_ident", 2024-05-07T02:43:05.455 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.455 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.455 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.455 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.456 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.456 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.456 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.456 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-07T02:43:05.456 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.456 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.456 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.456 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.456 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_perf": { 2024-05-07T02:43:05.456 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_perf", 2024-05-07T02:43:05.457 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.457 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.457 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.457 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.457 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.457 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.457 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.457 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-07T02:43:05.458 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.458 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.458 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.458 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.458 INFO:teuthology.orchestra.run.smithi019.stdout: "contact": { 2024-05-07T02:43:05.458 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "contact", 2024-05-07T02:43:05.458 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.458 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.458 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.459 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.459 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.459 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.459 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.459 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.459 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.459 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.459 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.459 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.459 INFO:teuthology.orchestra.run.smithi019.stdout: "description": { 2024-05-07T02:43:05.460 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "description", 2024-05-07T02:43:05.460 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.460 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.460 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.460 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.460 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.460 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.460 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.461 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.461 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.461 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.461 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.461 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.461 INFO:teuthology.orchestra.run.smithi019.stdout: "device_url": { 2024-05-07T02:43:05.461 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_url", 2024-05-07T02:43:05.461 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.461 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.462 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.462 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-07T02:43:05.462 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.462 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.462 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.462 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.462 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.462 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.462 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.463 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.463 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-05-07T02:43:05.463 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-05-07T02:43:05.463 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.463 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.463 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.463 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.463 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.463 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.464 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.464 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.464 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.464 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.464 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.464 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.464 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-07T02:43:05.464 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-07T02:43:05.464 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.465 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.465 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.465 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "24", 2024-05-07T02:43:05.465 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "8", 2024-05-07T02:43:05.465 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.465 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.465 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.465 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.465 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.466 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.466 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.466 INFO:teuthology.orchestra.run.smithi019.stdout: "last_opt_revision": { 2024-05-07T02:43:05.466 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "last_opt_revision", 2024-05-07T02:43:05.466 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.466 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.466 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.466 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1", 2024-05-07T02:43:05.466 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.467 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.467 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.467 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.467 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.467 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.467 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.467 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.467 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard": { 2024-05-07T02:43:05.467 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard", 2024-05-07T02:43:05.467 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.468 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.468 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.468 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.468 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.468 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.468 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.468 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.468 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.468 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.469 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.469 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.469 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard_description": { 2024-05-07T02:43:05.469 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard_description", 2024-05-07T02:43:05.469 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.469 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.469 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.469 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.470 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.470 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.470 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.470 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.470 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.470 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.470 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.470 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.470 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.471 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.471 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.471 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.471 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.471 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.471 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.471 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.471 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.471 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.472 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.472 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.472 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.472 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.472 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.472 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.472 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.472 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.472 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.473 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.473 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.473 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.473 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.473 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.473 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.473 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.473 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.473 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.474 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.474 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.474 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.474 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.474 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.474 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.474 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.474 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.474 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.475 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.475 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.475 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.475 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.475 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.475 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.475 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.475 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.475 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.476 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.476 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.476 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.476 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.476 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.476 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.476 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.476 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.476 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.477 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.477 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.477 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.477 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.477 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.477 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.477 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.477 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.477 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.478 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.478 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.478 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.478 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.478 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.478 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.478 INFO:teuthology.orchestra.run.smithi019.stdout: "organization": { 2024-05-07T02:43:05.478 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "organization", 2024-05-07T02:43:05.478 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.479 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.479 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.479 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.479 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.479 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.479 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.479 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.479 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.479 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.480 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.480 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.480 INFO:teuthology.orchestra.run.smithi019.stdout: "proxy": { 2024-05-07T02:43:05.480 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "proxy", 2024-05-07T02:43:05.480 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.480 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.480 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.480 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.480 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.480 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.481 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.481 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.481 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.481 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.481 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.481 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.481 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.481 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.481 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.482 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.482 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.482 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.482 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.482 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.482 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.482 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.482 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.483 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.483 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.483 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.483 INFO:teuthology.orchestra.run.smithi019.stdout: "url": { 2024-05-07T02:43:05.483 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url", 2024-05-07T02:43:05.483 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.483 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.483 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.483 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-07T02:43:05.484 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.484 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.484 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.484 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.484 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.484 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.484 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.484 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.484 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.485 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.485 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.485 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "test_orchestrator", 2024-05-07T02:43:05.485 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.485 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.485 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.485 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.485 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.485 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.486 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.486 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.486 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.486 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.486 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.486 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.486 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.486 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.486 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.487 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.487 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.487 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.487 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.487 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.487 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.487 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.487 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.487 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.488 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.488 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.488 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.488 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.488 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.488 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.488 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.488 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.488 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.489 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.489 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.489 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.489 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.489 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.489 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.489 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.489 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.489 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.490 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.490 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.490 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.490 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.490 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.490 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.490 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.490 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.491 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.491 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.491 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.491 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.491 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.491 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.491 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.491 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.491 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.492 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.492 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.492 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.492 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.492 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.492 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.492 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.492 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.492 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.493 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.493 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.493 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.493 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.493 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.493 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.493 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.493 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.493 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.494 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.494 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.494 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.494 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.494 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.494 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.494 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.494 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.494 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.495 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.495 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.495 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.495 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.495 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "volumes", 2024-05-07T02:43:05.495 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.495 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.495 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.495 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.496 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.496 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.496 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.496 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.496 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.496 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.496 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.496 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.496 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.497 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.497 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.497 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.497 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.497 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.497 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.497 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.497 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.497 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.498 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.498 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.498 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.498 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.498 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.498 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.498 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.498 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.498 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.499 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.499 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.499 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.499 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.499 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.499 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.499 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.499 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.499 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.500 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.500 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.500 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.500 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.500 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.500 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.500 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.500 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.500 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.501 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.501 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.501 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.501 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.501 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.501 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.501 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.501 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.501 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.501 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.502 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.502 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.502 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.502 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.502 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.502 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.502 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.502 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.502 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.503 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.503 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.503 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.503 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.503 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.503 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_clones": { 2024-05-07T02:43:05.503 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_clones", 2024-05-07T02:43:05.503 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.503 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.504 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.504 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4", 2024-05-07T02:43:05.504 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.504 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.504 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.504 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-07T02:43:05.504 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.504 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.504 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.505 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.505 INFO:teuthology.orchestra.run.smithi019.stdout: "periodic_async_work": { 2024-05-07T02:43:05.505 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "periodic_async_work", 2024-05-07T02:43:05.505 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.505 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.505 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.505 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.505 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.505 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.506 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.506 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Periodically check for async work", 2024-05-07T02:43:05.506 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.506 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.506 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.506 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.506 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_delay": { 2024-05-07T02:43:05.506 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_delay", 2024-05-07T02:43:05.506 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.507 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.507 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.507 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.507 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.507 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.507 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.507 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-07T02:43:05.507 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.507 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.508 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.508 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.508 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_no_wait": { 2024-05-07T02:43:05.508 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_no_wait", 2024-05-07T02:43:05.508 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.508 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.508 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.508 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.508 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.509 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.509 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.509 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-07T02:43:05.509 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.509 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.509 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.509 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.509 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.509 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.510 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.510 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.510 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.510 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.510 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.510 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.510 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.510 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.510 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.511 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.511 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.511 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.511 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.511 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.511 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.511 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix", 2024-05-07T02:43:05.511 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.512 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.512 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.512 INFO:teuthology.orchestra.run.smithi019.stdout: "discovery_interval": { 2024-05-07T02:43:05.512 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "discovery_interval", 2024-05-07T02:43:05.512 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-07T02:43:05.512 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.512 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.512 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "100", 2024-05-07T02:43:05.512 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.512 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.513 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.513 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.513 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.513 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.513 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.513 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.513 INFO:teuthology.orchestra.run.smithi019.stdout: "identifier": { 2024-05-07T02:43:05.513 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "identifier", 2024-05-07T02:43:05.513 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.514 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.514 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.514 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.514 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.514 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.514 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.514 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.514 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.515 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.515 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.515 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.515 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-07T02:43:05.515 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-07T02:43:05.515 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:05.515 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.515 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.515 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-07T02:43:05.515 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.516 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.516 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.516 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.516 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.516 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.516 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.516 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.516 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.516 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.517 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.517 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.517 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.517 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.517 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.517 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.517 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.517 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.517 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.517 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.518 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.518 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.518 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.518 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.518 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.518 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.518 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.518 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.519 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.519 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.519 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.519 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.519 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.519 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.519 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.519 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.519 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.519 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.520 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.520 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.520 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.520 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.520 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.520 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.520 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.520 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.521 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.521 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.521 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.521 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.521 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.521 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.521 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.521 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.521 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.521 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.522 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.522 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.522 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.522 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.522 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.522 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.522 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.522 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.523 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.523 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.523 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.523 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.523 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.523 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.523 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.523 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.523 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.524 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.524 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.524 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.524 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.524 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.524 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.524 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.524 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.524 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.525 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.525 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.525 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.525 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.525 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.525 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.525 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.525 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.525 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.526 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.526 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_host": { 2024-05-07T02:43:05.526 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_host", 2024-05-07T02:43:05.526 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.526 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.526 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.526 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.526 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.526 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.527 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.527 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.527 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.527 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.527 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.527 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.527 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_port": { 2024-05-07T02:43:05.527 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_port", 2024-05-07T02:43:05.527 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.528 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.528 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.528 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10051", 2024-05-07T02:43:05.528 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.528 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.528 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.528 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.528 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.528 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.529 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.529 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.529 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_sender": { 2024-05-07T02:43:05.529 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_sender", 2024-05-07T02:43:05.529 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.529 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.529 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.529 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-07T02:43:05.529 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.529 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.530 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.530 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.530 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.530 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.530 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.530 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.530 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.530 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.531 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-05-07T02:43:05.531 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.531 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.531 INFO:teuthology.orchestra.run.smithi019.stdout: "gid": 25238, 2024-05-07T02:43:05.531 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "y", 2024-05-07T02:43:05.531 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_features": 4540701547738038271, 2024-05-07T02:43:05.531 INFO:teuthology.orchestra.run.smithi019.stdout: "available_modules": [ 2024-05-07T02:43:05.531 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.531 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "alerts", 2024-05-07T02:43:05.532 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.532 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.532 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.532 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-07T02:43:05.532 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-07T02:43:05.532 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:05.532 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.532 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.532 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-07T02:43:05.533 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.533 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.533 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.533 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How frequently to reexamine health status", 2024-05-07T02:43:05.533 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.533 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.533 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.533 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.533 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.534 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.534 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.534 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.534 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.534 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.534 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.534 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.534 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.534 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.535 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.535 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.535 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.535 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.535 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.535 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.535 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.535 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.535 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.536 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.536 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.536 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.536 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.536 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.536 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.536 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.536 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.536 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.537 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.537 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.537 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.537 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.537 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.537 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.537 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.537 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.537 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.538 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.538 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.538 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.538 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.538 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.538 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.538 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.538 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.538 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.539 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.539 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.539 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.539 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.539 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.539 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.539 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.539 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.539 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.540 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.540 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.540 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.540 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.540 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.540 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.540 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.540 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.540 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.541 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.541 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.541 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.541 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.541 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.541 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.541 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_destination": { 2024-05-07T02:43:05.541 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_destination", 2024-05-07T02:43:05.541 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.542 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.542 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.542 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.542 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.542 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.542 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.542 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email address to send alerts to", 2024-05-07T02:43:05.542 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.543 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.543 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.543 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.543 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_from_name": { 2024-05-07T02:43:05.543 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_from_name", 2024-05-07T02:43:05.543 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.543 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.543 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.543 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "Ceph", 2024-05-07T02:43:05.544 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.544 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.544 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.544 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email From: name", 2024-05-07T02:43:05.544 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.544 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.544 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.544 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.544 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_host": { 2024-05-07T02:43:05.545 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_host", 2024-05-07T02:43:05.545 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.545 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.545 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.545 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.545 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.545 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.545 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.545 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP server", 2024-05-07T02:43:05.546 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.546 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.546 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.546 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.546 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_password": { 2024-05-07T02:43:05.546 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_password", 2024-05-07T02:43:05.546 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.546 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.547 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.547 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.547 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.547 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.547 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.547 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Password to authenticate with", 2024-05-07T02:43:05.547 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.547 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.547 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.548 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.548 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_port": { 2024-05-07T02:43:05.548 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_port", 2024-05-07T02:43:05.548 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.548 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.548 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.548 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "465", 2024-05-07T02:43:05.548 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.548 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.549 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.549 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP port", 2024-05-07T02:43:05.549 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.549 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.549 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.549 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.549 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_sender": { 2024-05-07T02:43:05.549 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_sender", 2024-05-07T02:43:05.549 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.550 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.550 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.550 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.550 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.550 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.550 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.550 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP envelope sender", 2024-05-07T02:43:05.550 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.550 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.551 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.551 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.551 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_ssl": { 2024-05-07T02:43:05.551 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_ssl", 2024-05-07T02:43:05.551 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.551 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.551 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.551 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.551 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.552 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.552 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.552 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-07T02:43:05.552 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.552 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.552 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.552 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.552 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_user": { 2024-05-07T02:43:05.552 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_user", 2024-05-07T02:43:05.553 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.553 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.553 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.553 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.553 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.553 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.553 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.553 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "User to authenticate as", 2024-05-07T02:43:05.553 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.554 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.554 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.554 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.554 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.554 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.554 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.554 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.554 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.554 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.555 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.555 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.555 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.555 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.555 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.555 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.555 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.555 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.555 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.556 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.556 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.556 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "balancer", 2024-05-07T02:43:05.556 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.556 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.556 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.556 INFO:teuthology.orchestra.run.smithi019.stdout: "active": { 2024-05-07T02:43:05.556 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "active", 2024-05-07T02:43:05.556 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.557 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.557 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.557 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.557 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.557 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.557 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.557 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically balance PGs across cluster", 2024-05-07T02:43:05.557 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.557 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.558 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.558 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.558 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_time": { 2024-05-07T02:43:05.558 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_time", 2024-05-07T02:43:05.558 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.558 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.558 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.558 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0000", 2024-05-07T02:43:05.558 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.559 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.559 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.559 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "beginning time of day to automatically balance", 2024-05-07T02:43:05.559 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-07T02:43:05.559 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.559 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.559 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.559 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_weekday": { 2024-05-07T02:43:05.559 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_weekday", 2024-05-07T02:43:05.560 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-07T02:43:05.560 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.560 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.560 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.560 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-05-07T02:43:05.560 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-05-07T02:43:05.560 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.560 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-07T02:43:05.560 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-07T02:43:05.561 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.561 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.561 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.561 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_max_iterations": { 2024-05-07T02:43:05.561 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_max_iterations", 2024-05-07T02:43:05.561 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-07T02:43:05.561 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.561 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.561 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "25", 2024-05-07T02:43:05.562 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-07T02:43:05.562 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "250", 2024-05-07T02:43:05.562 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.562 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-07T02:43:05.562 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.562 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.562 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.562 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.562 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_metrics": { 2024-05-07T02:43:05.563 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_metrics", 2024-05-07T02:43:05.563 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.563 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.563 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.563 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "pgs,objects,bytes", 2024-05-07T02:43:05.563 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.563 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.563 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.563 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-07T02:43:05.564 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-07T02:43:05.564 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.564 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.564 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.564 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_step": { 2024-05-07T02:43:05.564 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_step", 2024-05-07T02:43:05.564 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-07T02:43:05.564 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.564 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.565 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.5", 2024-05-07T02:43:05.565 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0.001", 2024-05-07T02:43:05.565 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "0.999", 2024-05-07T02:43:05.565 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.565 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "aggressiveness of optimization", 2024-05-07T02:43:05.565 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-07T02:43:05.565 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.565 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.565 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.566 INFO:teuthology.orchestra.run.smithi019.stdout: "end_time": { 2024-05-07T02:43:05.566 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_time", 2024-05-07T02:43:05.566 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.566 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.566 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.566 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2359", 2024-05-07T02:43:05.566 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.566 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.566 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.567 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ending time of day to automatically balance", 2024-05-07T02:43:05.567 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-07T02:43:05.567 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.567 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.567 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.567 INFO:teuthology.orchestra.run.smithi019.stdout: "end_weekday": { 2024-05-07T02:43:05.567 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_weekday", 2024-05-07T02:43:05.567 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-07T02:43:05.568 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.568 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.568 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.568 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-05-07T02:43:05.568 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-05-07T02:43:05.568 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.568 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-07T02:43:05.568 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-07T02:43:05.568 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.569 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.569 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.569 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.569 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.569 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.569 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.569 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.569 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.569 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.570 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.570 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.570 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.570 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.570 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.570 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.570 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.570 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.570 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.571 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.571 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.571 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.571 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.571 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.571 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.571 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.571 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.571 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.572 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.572 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.572 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.572 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.572 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.572 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.572 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.572 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.572 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.573 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.573 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.573 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.573 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.573 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.573 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.573 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.573 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.573 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.574 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.574 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.574 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.574 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.574 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.574 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.574 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.574 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.574 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.575 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.575 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.575 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.575 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.575 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.575 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.575 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.575 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.575 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.576 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.576 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.576 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.576 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.576 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.576 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.576 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.576 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.576 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.577 INFO:teuthology.orchestra.run.smithi019.stdout: "min_score": { 2024-05-07T02:43:05.577 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_score", 2024-05-07T02:43:05.577 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-07T02:43:05.577 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.577 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.577 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.577 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.577 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.577 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.578 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-07T02:43:05.578 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.578 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.578 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.578 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.578 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-05-07T02:43:05.578 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-05-07T02:43:05.578 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.578 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.579 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.579 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "upmap", 2024-05-07T02:43:05.579 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.579 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.579 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.579 INFO:teuthology.orchestra.run.smithi019.stdout: "crush-compat", 2024-05-07T02:43:05.579 INFO:teuthology.orchestra.run.smithi019.stdout: "none", 2024-05-07T02:43:05.579 INFO:teuthology.orchestra.run.smithi019.stdout: "read", 2024-05-07T02:43:05.579 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap", 2024-05-07T02:43:05.580 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap-read" 2024-05-07T02:43:05.580 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.580 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Balancer mode", 2024-05-07T02:43:05.580 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.580 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.580 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.580 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.580 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_ids": { 2024-05-07T02:43:05.580 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_ids", 2024-05-07T02:43:05.581 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.581 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.581 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.581 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.581 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.581 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.581 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.581 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-07T02:43:05.581 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.582 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.582 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.582 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.582 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-07T02:43:05.582 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-07T02:43:05.582 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:05.582 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.582 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.582 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-07T02:43:05.583 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.583 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.583 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.583 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-07T02:43:05.583 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.583 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.583 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.583 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.583 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.583 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.584 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.584 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.584 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.584 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.584 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.584 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.584 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.584 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.584 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.585 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.585 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.585 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.585 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_deviation": { 2024-05-07T02:43:05.585 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_deviation", 2024-05-07T02:43:05.585 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.585 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.585 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.585 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-07T02:43:05.586 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-07T02:43:05.586 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.586 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.586 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-07T02:43:05.586 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-07T02:43:05.586 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.586 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.586 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.587 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_optimizations": { 2024-05-07T02:43:05.587 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_optimizations", 2024-05-07T02:43:05.587 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-07T02:43:05.587 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.587 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.587 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-07T02:43:05.587 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.587 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.587 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.587 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-07T02:43:05.588 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.588 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.588 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.588 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.588 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.588 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.588 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.588 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm", 2024-05-07T02:43:05.588 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.589 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.589 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.589 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_down_multiplier": { 2024-05-07T02:43:05.589 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_down_multiplier", 2024-05-07T02:43:05.589 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-07T02:43:05.589 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.589 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.589 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-05-07T02:43:05.589 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.590 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.590 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.590 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-07T02:43:05.590 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.590 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.590 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.590 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.590 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_refresh_rate": { 2024-05-07T02:43:05.590 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_refresh_rate", 2024-05-07T02:43:05.591 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:05.591 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.591 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.591 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "20", 2024-05-07T02:43:05.591 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.591 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.591 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.591 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-07T02:43:05.591 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.592 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.592 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.592 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.592 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_starting_port": { 2024-05-07T02:43:05.592 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_starting_port", 2024-05-07T02:43:05.592 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.592 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.592 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.592 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4721", 2024-05-07T02:43:05.593 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.593 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.593 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.593 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-07T02:43:05.593 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.593 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.593 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.593 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.593 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_ptrace": { 2024-05-07T02:43:05.594 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_ptrace", 2024-05-07T02:43:05.594 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.594 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.594 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.594 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.594 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.594 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.594 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.594 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-07T02:43:05.595 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-07T02:43:05.595 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.595 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.595 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.595 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_interval": { 2024-05-07T02:43:05.595 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_interval", 2024-05-07T02:43:05.595 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:05.595 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.595 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.596 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-07T02:43:05.596 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.596 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.596 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.596 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-07T02:43:05.596 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.596 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.596 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.596 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.597 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_memory_target_ratio": { 2024-05-07T02:43:05.597 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_memory_target_ratio", 2024-05-07T02:43:05.597 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-07T02:43:05.597 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.597 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.597 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.7", 2024-05-07T02:43:05.597 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.597 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.597 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.597 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-07T02:43:05.598 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.598 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.598 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.598 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.598 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm_log_destination": { 2024-05-07T02:43:05.598 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm_log_destination", 2024-05-07T02:43:05.598 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.598 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.598 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.599 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.599 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.599 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.599 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.599 INFO:teuthology.orchestra.run.smithi019.stdout: "file", 2024-05-07T02:43:05.599 INFO:teuthology.orchestra.run.smithi019.stdout: "file,syslog", 2024-05-07T02:43:05.599 INFO:teuthology.orchestra.run.smithi019.stdout: "syslog" 2024-05-07T02:43:05.599 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.599 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-07T02:43:05.600 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.600 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.600 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.600 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.600 INFO:teuthology.orchestra.run.smithi019.stdout: "cgroups_split": { 2024-05-07T02:43:05.600 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cgroups_split", 2024-05-07T02:43:05.600 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.600 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.600 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.601 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.601 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.601 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.601 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.601 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-07T02:43:05.601 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.601 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.601 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.601 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.602 INFO:teuthology.orchestra.run.smithi019.stdout: "config_checks_enabled": { 2024-05-07T02:43:05.602 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_checks_enabled", 2024-05-07T02:43:05.602 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.602 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.602 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.602 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.602 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.602 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.602 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.603 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-07T02:43:05.603 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.603 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.603 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.603 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.603 INFO:teuthology.orchestra.run.smithi019.stdout: "config_dashboard": { 2024-05-07T02:43:05.603 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_dashboard", 2024-05-07T02:43:05.603 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.603 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.604 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.604 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.604 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.604 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.604 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.604 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-07T02:43:05.604 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.604 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.604 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.605 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.605 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_alertmanager": { 2024-05-07T02:43:05.605 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_alertmanager", 2024-05-07T02:43:05.605 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.605 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.605 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.605 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-07T02:43:05.605 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.606 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.606 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.606 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-07T02:43:05.606 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.606 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.606 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.606 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.606 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_base": { 2024-05-07T02:43:05.606 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_base", 2024-05-07T02:43:05.607 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.607 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.607 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.607 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-07T02:43:05.607 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.607 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.607 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.607 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Container image name, without the tag", 2024-05-07T02:43:05.607 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.608 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.608 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.608 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.608 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_elasticsearch": { 2024-05-07T02:43:05.608 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_elasticsearch", 2024-05-07T02:43:05.608 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.608 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.608 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.608 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-07T02:43:05.609 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.609 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.609 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.609 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "elasticsearch container image", 2024-05-07T02:43:05.609 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.609 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.609 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.609 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.609 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_grafana": { 2024-05-07T02:43:05.610 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_grafana", 2024-05-07T02:43:05.610 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.610 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.610 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.610 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-07T02:43:05.610 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.610 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.610 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.610 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-07T02:43:05.611 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.611 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.611 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.611 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.611 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_haproxy": { 2024-05-07T02:43:05.611 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_haproxy", 2024-05-07T02:43:05.611 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.611 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.612 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.612 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-07T02:43:05.612 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.612 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.612 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.612 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "HAproxy container image", 2024-05-07T02:43:05.612 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.612 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.612 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.613 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.613 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_agent": { 2024-05-07T02:43:05.613 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_agent", 2024-05-07T02:43:05.613 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.613 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.613 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.613 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-07T02:43:05.613 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.613 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.614 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.614 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger agent container image", 2024-05-07T02:43:05.614 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.614 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.614 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.614 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.614 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_collector": { 2024-05-07T02:43:05.614 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_collector", 2024-05-07T02:43:05.614 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.615 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.615 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.615 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-07T02:43:05.615 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.615 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.615 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.615 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger collector container image", 2024-05-07T02:43:05.615 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.615 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.616 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.616 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.616 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_query": { 2024-05-07T02:43:05.616 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_query", 2024-05-07T02:43:05.616 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.616 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.616 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.616 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-07T02:43:05.616 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.616 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.617 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.617 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger query container image", 2024-05-07T02:43:05.617 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.617 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.617 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.617 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.617 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_keepalived": { 2024-05-07T02:43:05.617 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_keepalived", 2024-05-07T02:43:05.617 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.618 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.618 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.618 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-07T02:43:05.618 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.618 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.618 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.618 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Keepalived container image", 2024-05-07T02:43:05.618 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.619 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.619 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.619 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.619 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_loki": { 2024-05-07T02:43:05.619 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_loki", 2024-05-07T02:43:05.619 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.619 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.619 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.619 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-07T02:43:05.620 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.620 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.620 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.620 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Loki container image", 2024-05-07T02:43:05.620 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.620 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.620 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.620 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.620 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_node_exporter": { 2024-05-07T02:43:05.621 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_node_exporter", 2024-05-07T02:43:05.621 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.621 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.621 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.621 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-07T02:43:05.621 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.621 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.621 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.621 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-07T02:43:05.622 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.622 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.622 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.622 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.622 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_nvmeof": { 2024-05-07T02:43:05.622 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_nvmeof", 2024-05-07T02:43:05.622 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.622 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.623 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.623 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-07T02:43:05.623 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.623 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.623 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.623 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Nvme-of container image", 2024-05-07T02:43:05.623 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.623 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.623 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.623 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.624 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_prometheus": { 2024-05-07T02:43:05.624 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_prometheus", 2024-05-07T02:43:05.624 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.624 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.624 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.624 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-07T02:43:05.624 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.624 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.625 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.625 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-07T02:43:05.625 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.625 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.625 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.625 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.625 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_promtail": { 2024-05-07T02:43:05.625 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_promtail", 2024-05-07T02:43:05.625 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.625 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.626 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.626 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-07T02:43:05.626 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.626 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.626 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.626 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Promtail container image", 2024-05-07T02:43:05.626 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.626 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.627 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.627 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.627 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_samba": { 2024-05-07T02:43:05.627 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_samba", 2024-05-07T02:43:05.627 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.627 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.627 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.627 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-07T02:43:05.627 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.627 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.628 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.628 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Samba/SMB container image", 2024-05-07T02:43:05.628 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.628 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.628 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.628 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.628 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_snmp_gateway": { 2024-05-07T02:43:05.628 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_snmp_gateway", 2024-05-07T02:43:05.628 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.629 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.629 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.629 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-07T02:43:05.629 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.629 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.629 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.629 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SNMP Gateway container image", 2024-05-07T02:43:05.629 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.629 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.630 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.630 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.630 INFO:teuthology.orchestra.run.smithi019.stdout: "container_init": { 2024-05-07T02:43:05.630 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_init", 2024-05-07T02:43:05.630 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.630 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.630 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.630 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.630 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.631 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.631 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.631 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Run podman/docker with `--init`", 2024-05-07T02:43:05.631 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.631 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.631 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.631 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.631 INFO:teuthology.orchestra.run.smithi019.stdout: "daemon_cache_timeout": { 2024-05-07T02:43:05.631 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "daemon_cache_timeout", 2024-05-07T02:43:05.632 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:05.632 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.632 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.632 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-07T02:43:05.632 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.632 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.632 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.632 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-07T02:43:05.632 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.633 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.633 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.633 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.633 INFO:teuthology.orchestra.run.smithi019.stdout: "default_cephadm_command_timeout": { 2024-05-07T02:43:05.633 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_cephadm_command_timeout", 2024-05-07T02:43:05.633 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.633 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.633 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.633 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "900", 2024-05-07T02:43:05.633 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.634 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.634 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.634 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-07T02:43:05.634 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.634 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.634 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.634 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.634 INFO:teuthology.orchestra.run.smithi019.stdout: "default_registry": { 2024-05-07T02:43:05.635 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_registry", 2024-05-07T02:43:05.635 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.635 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.635 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.635 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io", 2024-05-07T02:43:05.635 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.635 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.635 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.635 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-07T02:43:05.636 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.636 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.636 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.636 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.636 INFO:teuthology.orchestra.run.smithi019.stdout: "device_cache_timeout": { 2024-05-07T02:43:05.636 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_cache_timeout", 2024-05-07T02:43:05.636 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:05.636 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.636 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.636 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1800", 2024-05-07T02:43:05.637 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.637 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.637 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.637 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache device inventory", 2024-05-07T02:43:05.637 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.637 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.637 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.637 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.637 INFO:teuthology.orchestra.run.smithi019.stdout: "device_enhanced_scan": { 2024-05-07T02:43:05.638 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_enhanced_scan", 2024-05-07T02:43:05.638 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.638 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.638 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.638 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.638 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.638 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.638 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.638 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-07T02:43:05.639 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.639 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.639 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.639 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.639 INFO:teuthology.orchestra.run.smithi019.stdout: "facts_cache_timeout": { 2024-05-07T02:43:05.639 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "facts_cache_timeout", 2024-05-07T02:43:05.639 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:05.639 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.639 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.640 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-07T02:43:05.640 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.640 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.640 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.640 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache host facts data", 2024-05-07T02:43:05.640 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.640 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.640 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.640 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.641 INFO:teuthology.orchestra.run.smithi019.stdout: "grafana_dashboards_path": { 2024-05-07T02:43:05.641 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "grafana_dashboards_path", 2024-05-07T02:43:05.641 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.641 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.641 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.641 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-07T02:43:05.641 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.641 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.642 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.642 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-07T02:43:05.642 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.642 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.642 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.642 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.642 INFO:teuthology.orchestra.run.smithi019.stdout: "host_check_interval": { 2024-05-07T02:43:05.642 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "host_check_interval", 2024-05-07T02:43:05.642 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:05.643 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.643 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.643 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-07T02:43:05.643 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.643 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.643 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.643 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to perform a host check", 2024-05-07T02:43:05.643 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.643 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.644 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.644 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.644 INFO:teuthology.orchestra.run.smithi019.stdout: "hw_monitoring": { 2024-05-07T02:43:05.644 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hw_monitoring", 2024-05-07T02:43:05.644 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.644 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.644 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.644 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.644 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.645 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.645 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.645 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-07T02:43:05.645 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.645 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.645 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.645 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.645 INFO:teuthology.orchestra.run.smithi019.stdout: "inventory_list_all": { 2024-05-07T02:43:05.645 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "inventory_list_all", 2024-05-07T02:43:05.646 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.646 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.646 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.646 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.646 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.646 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.646 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.646 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-07T02:43:05.646 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.647 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.647 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.647 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.647 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.647 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.647 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.647 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.647 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.647 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.648 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.648 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.648 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.648 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.648 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.648 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.648 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.648 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.648 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.649 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.649 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.649 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.649 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.649 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.649 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.649 INFO:teuthology.orchestra.run.smithi019.stdout: "log_refresh_metadata": { 2024-05-07T02:43:05.649 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_refresh_metadata", 2024-05-07T02:43:05.649 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.650 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.650 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.650 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.650 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.650 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.650 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.650 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-07T02:43:05.650 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.650 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.651 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.651 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.651 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.651 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.651 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.651 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.651 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.651 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.651 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.652 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.652 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.652 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-07T02:43:05.652 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.652 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.652 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.652 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.652 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.652 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.653 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.653 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.653 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.653 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.653 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.653 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.653 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.653 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.653 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.653 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.654 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.654 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.654 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.654 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.654 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.654 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.654 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.654 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.655 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.655 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.655 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.655 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.655 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.655 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.655 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.655 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.655 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.656 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.656 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.656 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.656 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.656 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.656 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.656 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-07T02:43:05.656 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-07T02:43:05.656 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.657 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.657 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.657 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.657 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.657 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.657 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.657 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-07T02:43:05.657 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.657 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.658 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.658 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.658 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-07T02:43:05.658 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-07T02:43:05.658 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.658 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.658 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.658 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "*", 2024-05-07T02:43:05.658 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.659 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.659 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.659 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-07T02:43:05.659 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.659 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.659 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.659 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.659 INFO:teuthology.orchestra.run.smithi019.stdout: "max_count_per_host": { 2024-05-07T02:43:05.659 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_count_per_host", 2024-05-07T02:43:05.660 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.660 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.660 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.660 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-07T02:43:05.660 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.660 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.660 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.660 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of daemons per service per host", 2024-05-07T02:43:05.660 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.661 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.661 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.661 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.661 INFO:teuthology.orchestra.run.smithi019.stdout: "max_osd_draining_count": { 2024-05-07T02:43:05.661 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_osd_draining_count", 2024-05-07T02:43:05.661 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.661 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.661 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.661 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-07T02:43:05.662 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.662 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.662 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.662 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-07T02:43:05.662 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.662 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.662 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.662 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.663 INFO:teuthology.orchestra.run.smithi019.stdout: "migration_current": { 2024-05-07T02:43:05.663 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "migration_current", 2024-05-07T02:43:05.663 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.663 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.663 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.663 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.663 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.663 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.663 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.664 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "internal - do not modify", 2024-05-07T02:43:05.664 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.664 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.664 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.664 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.664 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-05-07T02:43:05.664 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-05-07T02:43:05.664 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.664 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.665 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.665 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "root", 2024-05-07T02:43:05.665 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.665 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.665 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.665 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm-package", 2024-05-07T02:43:05.665 INFO:teuthology.orchestra.run.smithi019.stdout: "root" 2024-05-07T02:43:05.665 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.665 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "mode for remote execution of cephadm", 2024-05-07T02:43:05.666 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.666 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.666 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.666 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.666 INFO:teuthology.orchestra.run.smithi019.stdout: "oob_default_addr": { 2024-05-07T02:43:05.666 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "oob_default_addr", 2024-05-07T02:43:05.666 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.666 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.667 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.667 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "169.254.1.1", 2024-05-07T02:43:05.667 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.667 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.667 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.667 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-07T02:43:05.667 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.667 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.667 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.667 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.668 INFO:teuthology.orchestra.run.smithi019.stdout: "prometheus_alerts_path": { 2024-05-07T02:43:05.668 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus_alerts_path", 2024-05-07T02:43:05.668 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.668 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.668 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.668 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-07T02:43:05.668 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.668 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.668 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.669 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-07T02:43:05.669 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.669 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.669 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.669 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.669 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_insecure": { 2024-05-07T02:43:05.669 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_insecure", 2024-05-07T02:43:05.669 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.669 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.670 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.670 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.670 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.670 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.670 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.670 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-07T02:43:05.670 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.670 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.670 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.671 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.671 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_password": { 2024-05-07T02:43:05.671 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_password", 2024-05-07T02:43:05.671 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.671 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.671 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.671 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.671 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.671 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.672 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.672 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-07T02:43:05.672 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.672 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.672 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.672 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.672 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_url": { 2024-05-07T02:43:05.672 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_url", 2024-05-07T02:43:05.672 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.672 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.673 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.673 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.673 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.673 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.673 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.673 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-07T02:43:05.673 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.673 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.673 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.674 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.674 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_username": { 2024-05-07T02:43:05.674 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_username", 2024-05-07T02:43:05.674 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.674 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.674 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.674 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.674 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.674 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.675 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.675 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-07T02:43:05.675 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.675 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.675 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.675 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.675 INFO:teuthology.orchestra.run.smithi019.stdout: "secure_monitoring_stack": { 2024-05-07T02:43:05.675 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "secure_monitoring_stack", 2024-05-07T02:43:05.675 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.675 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.676 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.676 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.676 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.676 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.676 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.676 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-07T02:43:05.676 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.676 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.676 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.677 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.677 INFO:teuthology.orchestra.run.smithi019.stdout: "service_discovery_port": { 2024-05-07T02:43:05.677 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "service_discovery_port", 2024-05-07T02:43:05.677 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.677 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.677 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.677 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8765", 2024-05-07T02:43:05.677 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.677 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.678 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.678 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "cephadm service discovery port", 2024-05-07T02:43:05.678 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.678 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.678 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.678 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.678 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.678 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.678 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.679 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.679 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.679 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.679 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.679 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.679 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.679 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.679 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.679 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.679 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.680 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.680 INFO:teuthology.orchestra.run.smithi019.stdout: "ssh_config_file": { 2024-05-07T02:43:05.680 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssh_config_file", 2024-05-07T02:43:05.680 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.680 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.680 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.680 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.680 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.680 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.681 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.681 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-07T02:43:05.681 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.681 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.681 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.681 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.681 INFO:teuthology.orchestra.run.smithi019.stdout: "use_agent": { 2024-05-07T02:43:05.681 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_agent", 2024-05-07T02:43:05.681 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.682 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.682 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.682 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.682 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.682 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.682 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.682 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-07T02:43:05.682 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.683 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.683 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.683 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.683 INFO:teuthology.orchestra.run.smithi019.stdout: "use_repo_digest": { 2024-05-07T02:43:05.683 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_repo_digest", 2024-05-07T02:43:05.683 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.683 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.683 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.683 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.683 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.684 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.684 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.684 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-07T02:43:05.684 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.684 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.684 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.684 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.684 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_failed_host_check": { 2024-05-07T02:43:05.684 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_failed_host_check", 2024-05-07T02:43:05.685 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.685 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.685 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.685 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.685 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.685 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.685 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.685 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if the host check fails", 2024-05-07T02:43:05.685 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.685 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.686 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.686 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.686 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_daemons": { 2024-05-07T02:43:05.686 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_daemons", 2024-05-07T02:43:05.686 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.686 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.686 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.686 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.687 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.687 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.687 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.687 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-07T02:43:05.687 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.687 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.687 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.687 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.687 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_hosts": { 2024-05-07T02:43:05.687 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_hosts", 2024-05-07T02:43:05.688 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.688 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.688 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.688 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.688 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.688 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.688 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.688 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-07T02:43:05.688 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.689 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.689 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.689 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.689 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.689 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.689 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.689 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crash", 2024-05-07T02:43:05.689 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.689 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.690 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.690 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.690 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.690 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.690 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.690 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.690 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.690 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.690 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.691 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.691 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.691 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.691 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.691 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.691 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.691 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.691 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.691 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.692 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.692 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.692 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.692 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.692 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.692 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.692 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.692 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.692 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.693 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.693 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.693 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.693 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.693 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.693 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.693 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.693 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.693 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.693 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.694 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.694 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.694 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.694 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.694 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.694 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.694 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.694 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.695 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.695 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.695 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.695 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.695 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.695 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.695 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.695 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.695 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.696 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.696 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.696 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.696 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.696 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.696 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.696 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.696 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.696 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.696 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.697 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.697 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.697 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.697 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.697 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.697 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.697 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.697 INFO:teuthology.orchestra.run.smithi019.stdout: "retain_interval": { 2024-05-07T02:43:05.697 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retain_interval", 2024-05-07T02:43:05.698 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:05.698 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.698 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.698 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "31536000", 2024-05-07T02:43:05.698 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.698 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.698 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.698 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-07T02:43:05.699 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.699 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.699 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.699 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.699 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.699 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.699 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.699 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.699 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.699 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.700 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.700 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.700 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.700 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.700 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.700 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.700 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.700 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.700 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_recent_interval": { 2024-05-07T02:43:05.701 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_recent_interval", 2024-05-07T02:43:05.701 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:05.701 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.701 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.701 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1209600", 2024-05-07T02:43:05.701 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.701 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.701 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.701 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-07T02:43:05.702 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.702 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.702 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.702 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.702 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.702 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.702 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.702 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dashboard", 2024-05-07T02:43:05.703 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.703 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.703 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.703 INFO:teuthology.orchestra.run.smithi019.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-07T02:43:05.703 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-07T02:43:05.703 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.703 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.703 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.703 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-07T02:43:05.704 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.704 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.704 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.704 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.704 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.704 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.704 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.704 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.704 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_HOST": { 2024-05-07T02:43:05.704 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-07T02:43:05.705 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.705 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.705 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.705 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.705 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.705 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.705 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.705 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.705 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.706 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.706 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.706 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.706 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-07T02:43:05.706 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-07T02:43:05.706 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.706 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.706 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.706 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.707 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.707 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.707 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.707 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.707 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.707 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.707 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.707 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.708 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_ENABLED": { 2024-05-07T02:43:05.708 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_ENABLED", 2024-05-07T02:43:05.708 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.708 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.708 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.708 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.708 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.708 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.708 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.709 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.709 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.709 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.709 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.709 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.709 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-07T02:43:05.709 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-07T02:43:05.709 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.709 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.709 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.710 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.710 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.710 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.710 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.710 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.710 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.710 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.710 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.711 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.711 INFO:teuthology.orchestra.run.smithi019.stdout: "ENABLE_BROWSABLE_API": { 2024-05-07T02:43:05.711 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-07T02:43:05.711 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.711 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.711 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.711 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.711 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.711 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.712 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.712 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.712 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.712 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.712 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.712 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.712 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-07T02:43:05.712 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-07T02:43:05.712 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.713 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.713 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.713 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.713 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.713 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.713 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.713 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.713 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.713 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.713 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.714 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.714 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-07T02:43:05.714 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-07T02:43:05.714 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.714 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.714 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.714 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.714 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.715 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.715 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.715 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.715 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.715 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.715 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.715 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.715 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-07T02:43:05.715 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-07T02:43:05.716 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.716 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.716 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.716 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.716 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.716 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.716 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.716 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.716 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.716 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.717 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.717 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.717 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-07T02:43:05.717 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-07T02:43:05.717 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.717 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.717 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.717 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.717 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.718 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.718 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.718 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.718 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.718 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.718 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.718 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.718 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-07T02:43:05.718 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-07T02:43:05.719 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.719 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.719 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.719 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.719 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.719 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.719 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.719 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.719 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.719 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.720 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.720 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.720 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-07T02:43:05.720 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-07T02:43:05.720 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.720 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.720 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.720 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.720 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.720 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.721 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.721 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.721 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.721 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.721 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.721 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.721 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-07T02:43:05.721 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-07T02:43:05.721 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.722 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.722 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.722 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.722 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.722 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.722 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.722 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.722 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.723 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.723 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.723 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.723 INFO:teuthology.orchestra.run.smithi019.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-07T02:43:05.723 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-07T02:43:05.723 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.723 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.723 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.723 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.724 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.724 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.724 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.724 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.724 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.724 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.724 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.724 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.724 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_PASSWORD": { 2024-05-07T02:43:05.725 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-07T02:43:05.725 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.725 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.725 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.725 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-05-07T02:43:05.725 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.725 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.725 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.725 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.725 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.726 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.726 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.726 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.726 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-07T02:43:05.726 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-07T02:43:05.726 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.726 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.726 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.726 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.727 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.727 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.727 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.730 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.731 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.731 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.731 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.731 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.731 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_URL": { 2024-05-07T02:43:05.731 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_URL", 2024-05-07T02:43:05.731 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.731 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.731 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.731 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.732 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.732 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.732 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.732 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.732 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.732 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.732 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.732 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.732 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_USERNAME": { 2024-05-07T02:43:05.733 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-07T02:43:05.733 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.733 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.733 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.733 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-05-07T02:43:05.733 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.733 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.733 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.733 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.734 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.734 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.734 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.734 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.734 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-07T02:43:05.734 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-07T02:43:05.734 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.734 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.734 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.735 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.735 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.735 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.735 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.735 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.735 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.735 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.735 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.735 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.735 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-07T02:43:05.736 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-07T02:43:05.736 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.736 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.736 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.736 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.736 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.736 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.736 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.736 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.737 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.737 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.737 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.737 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.737 INFO:teuthology.orchestra.run.smithi019.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-07T02:43:05.737 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-07T02:43:05.737 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.737 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.737 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.738 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.738 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.738 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.738 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.738 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.738 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.738 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.738 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.738 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.739 INFO:teuthology.orchestra.run.smithi019.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-07T02:43:05.739 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-07T02:43:05.739 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.739 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.739 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.739 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.739 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.739 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.739 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.739 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.740 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.740 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.740 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.740 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.740 INFO:teuthology.orchestra.run.smithi019.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-07T02:43:05.740 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-07T02:43:05.740 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.740 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.740 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.741 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "[]", 2024-05-07T02:43:05.741 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.741 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.741 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.741 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.741 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.741 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.741 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.741 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.742 INFO:teuthology.orchestra.run.smithi019.stdout: "MULTICLUSTER_CONFIG": { 2024-05-07T02:43:05.742 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-07T02:43:05.742 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.742 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.742 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.742 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "{}", 2024-05-07T02:43:05.742 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.742 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.742 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.743 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.743 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.743 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.743 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.743 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.743 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_HOST": { 2024-05-07T02:43:05.743 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-07T02:43:05.743 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.743 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.744 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.744 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.744 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.744 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.744 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.744 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.744 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.744 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.744 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.745 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.745 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-07T02:43:05.745 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-07T02:43:05.745 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.745 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.745 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.745 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.745 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.745 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.746 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.746 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.746 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.746 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.746 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.746 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.746 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-07T02:43:05.746 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-07T02:43:05.746 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.747 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.747 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.747 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.747 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.747 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.747 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.747 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.747 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.747 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.748 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.748 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.748 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-07T02:43:05.748 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-07T02:43:05.748 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.748 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.748 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.748 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.748 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.749 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.749 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.749 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.749 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.749 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.749 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.749 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.749 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-07T02:43:05.749 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-07T02:43:05.750 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.750 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.750 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.750 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.750 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.750 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.750 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.750 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.750 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.751 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.751 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.751 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.751 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-07T02:43:05.751 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-07T02:43:05.751 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.751 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.751 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.751 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.752 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.752 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.752 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.752 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.752 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.752 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.752 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.752 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.753 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-07T02:43:05.753 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-07T02:43:05.753 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.753 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.753 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.753 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.753 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.753 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.753 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.754 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.754 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.754 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.754 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.754 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.754 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-07T02:43:05.754 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-07T02:43:05.754 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.754 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.755 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.755 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.755 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.755 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.755 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.755 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.755 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.755 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.755 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.755 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.756 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-07T02:43:05.756 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-07T02:43:05.756 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.756 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.756 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.756 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.756 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.756 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.756 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.757 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.757 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.757 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.757 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.757 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.757 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_ENABLED": { 2024-05-07T02:43:05.757 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-07T02:43:05.757 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.757 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.758 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.758 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.758 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.758 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.758 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.758 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.758 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.758 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.758 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.759 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.759 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-07T02:43:05.759 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-07T02:43:05.759 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.759 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.759 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.759 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-07T02:43:05.759 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.759 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.760 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.760 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.760 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.760 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.760 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.760 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.760 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-07T02:43:05.760 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-07T02:43:05.760 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.761 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.761 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.761 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-07T02:43:05.761 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.761 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.761 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.761 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.761 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.761 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.761 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.762 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.762 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-07T02:43:05.762 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-07T02:43:05.762 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.762 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.762 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.762 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8", 2024-05-07T02:43:05.762 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.763 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.763 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.763 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.763 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.763 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.763 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.763 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.763 INFO:teuthology.orchestra.run.smithi019.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-07T02:43:05.763 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-07T02:43:05.764 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.764 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.764 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.764 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "45", 2024-05-07T02:43:05.764 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.764 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.764 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.764 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.764 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.764 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.765 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.765 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.765 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ACCESS_KEY": { 2024-05-07T02:43:05.765 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-07T02:43:05.765 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.765 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.765 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.765 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.765 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.766 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.766 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.766 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.766 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.766 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.766 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.766 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.766 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-07T02:43:05.767 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-07T02:43:05.767 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.767 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.767 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.767 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-05-07T02:43:05.767 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.767 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.767 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.767 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.768 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.768 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.768 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.768 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.768 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SECRET_KEY": { 2024-05-07T02:43:05.768 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-07T02:43:05.768 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.768 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.768 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.769 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.769 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.769 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.769 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.769 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.769 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.769 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.769 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.769 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.770 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SSL_VERIFY": { 2024-05-07T02:43:05.770 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-07T02:43:05.770 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.770 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.770 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.770 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.770 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.770 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.771 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.771 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.771 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.771 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.771 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.771 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.771 INFO:teuthology.orchestra.run.smithi019.stdout: "UNSAFE_TLS_v1_2": { 2024-05-07T02:43:05.771 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-07T02:43:05.771 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.772 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.772 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.772 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.772 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.772 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.772 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.772 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.772 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.772 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.773 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.773 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.773 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-07T02:43:05.773 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-07T02:43:05.773 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.773 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.773 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.773 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.773 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.773 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.774 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.774 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.774 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.774 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.774 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.774 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.774 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-07T02:43:05.774 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-07T02:43:05.774 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.775 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.775 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.775 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-07T02:43:05.775 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.775 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.775 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.775 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.775 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.775 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.776 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.776 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.776 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-07T02:43:05.776 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-07T02:43:05.776 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.776 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.776 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.776 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-07T02:43:05.776 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.777 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.777 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.777 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.777 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.777 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.777 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.777 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.777 INFO:teuthology.orchestra.run.smithi019.stdout: "cross_origin_url": { 2024-05-07T02:43:05.777 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cross_origin_url", 2024-05-07T02:43:05.777 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.778 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.778 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.778 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.778 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.778 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.778 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.778 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.778 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.778 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.779 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.779 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.779 INFO:teuthology.orchestra.run.smithi019.stdout: "crt_file": { 2024-05-07T02:43:05.779 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crt_file", 2024-05-07T02:43:05.779 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.779 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.779 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.779 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.779 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.780 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.780 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.780 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.780 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.780 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.780 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.780 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.780 INFO:teuthology.orchestra.run.smithi019.stdout: "debug": { 2024-05-07T02:43:05.780 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "debug", 2024-05-07T02:43:05.781 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.781 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.781 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.781 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.781 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.781 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.781 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.781 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable/disable debug options", 2024-05-07T02:43:05.781 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.782 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.782 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.782 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.782 INFO:teuthology.orchestra.run.smithi019.stdout: "jwt_token_ttl": { 2024-05-07T02:43:05.782 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "jwt_token_ttl", 2024-05-07T02:43:05.782 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.782 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.782 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.782 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "28800", 2024-05-07T02:43:05.783 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.783 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.783 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.783 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.783 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.783 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.783 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.783 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.783 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-05-07T02:43:05.784 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-05-07T02:43:05.784 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.784 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.784 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.784 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.784 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.784 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.784 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.784 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.785 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.785 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.785 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.785 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.785 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.785 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.785 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.785 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.785 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.786 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.786 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.786 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.786 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.786 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.786 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.786 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.786 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.786 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.787 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.787 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.787 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.787 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.787 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.787 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.787 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.787 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.787 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.788 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.788 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.788 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.788 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.788 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.788 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.788 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.788 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.788 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.789 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.789 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.789 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.789 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.789 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.789 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.789 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.789 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.789 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.789 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.790 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.790 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.790 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.790 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.790 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.790 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.790 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.790 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.790 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.791 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.791 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.791 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.791 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.791 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.791 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.791 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.791 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.791 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.792 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.792 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.792 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.792 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.792 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.792 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.792 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.792 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.792 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.793 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.793 INFO:teuthology.orchestra.run.smithi019.stdout: "motd": { 2024-05-07T02:43:05.793 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "motd", 2024-05-07T02:43:05.793 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.793 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.793 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.793 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.793 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.793 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.794 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.794 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "The message of the day", 2024-05-07T02:43:05.794 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.794 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.794 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.794 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.794 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect_resolve_ip_addr": { 2024-05-07T02:43:05.794 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "redirect_resolve_ip_addr", 2024-05-07T02:43:05.794 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.795 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.795 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.795 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.795 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.795 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.795 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.795 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.795 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.795 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.796 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.796 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.796 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-05-07T02:43:05.796 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-05-07T02:43:05.796 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.796 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.796 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.796 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-05-07T02:43:05.796 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.797 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.797 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.797 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.797 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.797 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.797 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.797 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.797 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-05-07T02:43:05.797 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-05-07T02:43:05.798 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.798 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.798 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.798 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8080", 2024-05-07T02:43:05.798 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.798 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.798 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.798 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.798 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.799 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.799 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.799 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.799 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.799 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.799 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.799 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.799 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.799 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.800 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.800 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.800 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.800 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.800 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.800 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.800 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.800 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.800 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-05-07T02:43:05.800 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-05-07T02:43:05.801 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.801 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.801 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.801 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.801 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.801 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.801 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.801 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.801 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.802 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.802 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.802 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.802 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl_server_port": { 2024-05-07T02:43:05.802 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl_server_port", 2024-05-07T02:43:05.802 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.802 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.802 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.802 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8443", 2024-05-07T02:43:05.803 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.803 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.803 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.803 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.803 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.803 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.803 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.803 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.803 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-05-07T02:43:05.804 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-05-07T02:43:05.804 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.804 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.804 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.804 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "redirect", 2024-05-07T02:43:05.804 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.804 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.804 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.804 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.804 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect" 2024-05-07T02:43:05.805 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.805 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.805 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.805 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.805 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.805 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.805 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-05-07T02:43:05.805 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-05-07T02:43:05.805 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.806 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.806 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.806 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-05-07T02:43:05.806 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-05-07T02:43:05.806 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-05-07T02:43:05.806 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.806 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.806 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.806 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.807 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.807 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.807 INFO:teuthology.orchestra.run.smithi019.stdout: "url_prefix": { 2024-05-07T02:43:05.807 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url_prefix", 2024-05-07T02:43:05.807 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.807 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.807 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.807 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.808 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.808 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.808 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.808 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.808 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.808 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.808 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.808 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.808 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.809 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.809 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.809 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "devicehealth", 2024-05-07T02:43:05.809 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.809 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.809 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.809 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_monitoring": { 2024-05-07T02:43:05.809 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_monitoring", 2024-05-07T02:43:05.809 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.810 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.810 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.810 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.810 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.810 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.810 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.810 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "monitor device health metrics", 2024-05-07T02:43:05.810 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.810 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.811 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.811 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.811 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.811 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.811 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.811 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.811 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.811 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.811 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.812 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.812 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.812 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.812 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.812 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.812 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.812 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.812 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.812 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.812 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.813 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.813 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.813 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.813 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.813 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.813 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.813 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.813 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.813 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.814 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.814 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.814 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.814 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.814 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.814 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.814 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.814 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.814 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.815 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.815 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.815 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.815 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.815 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.815 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.815 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.815 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.815 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.816 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.816 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.816 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.816 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.816 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.816 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.816 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.816 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.816 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.817 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.817 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.817 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.817 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.817 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.817 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.817 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.817 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.817 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.817 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.818 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.818 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.818 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.818 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.818 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.818 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.818 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.818 INFO:teuthology.orchestra.run.smithi019.stdout: "mark_out_threshold": { 2024-05-07T02:43:05.818 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mark_out_threshold", 2024-05-07T02:43:05.819 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:05.819 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.819 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.819 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2419200", 2024-05-07T02:43:05.819 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.819 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.819 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.819 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-07T02:43:05.819 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.820 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.820 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.820 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.820 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_name": { 2024-05-07T02:43:05.820 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_name", 2024-05-07T02:43:05.820 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.820 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.820 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.820 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "device_health_metrics", 2024-05-07T02:43:05.821 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.821 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.821 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.821 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-07T02:43:05.821 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.821 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.821 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.821 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.821 INFO:teuthology.orchestra.run.smithi019.stdout: "retention_period": { 2024-05-07T02:43:05.821 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retention_period", 2024-05-07T02:43:05.822 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:05.822 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.822 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.822 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15552000", 2024-05-07T02:43:05.822 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.822 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.822 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.822 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain device health metrics", 2024-05-07T02:43:05.822 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.823 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.823 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.823 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.823 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_frequency": { 2024-05-07T02:43:05.823 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_frequency", 2024-05-07T02:43:05.823 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:05.823 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.823 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.823 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-05-07T02:43:05.824 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.824 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.824 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.824 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-07T02:43:05.824 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.824 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.824 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.824 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.824 INFO:teuthology.orchestra.run.smithi019.stdout: "self_heal": { 2024-05-07T02:43:05.824 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "self_heal", 2024-05-07T02:43:05.825 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.825 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.825 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.825 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:05.825 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.825 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.825 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.825 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-07T02:43:05.825 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.826 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.826 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.826 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.826 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-07T02:43:05.826 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-07T02:43:05.826 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:05.826 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.826 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.826 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-07T02:43:05.827 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.827 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.827 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.827 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and check device health", 2024-05-07T02:43:05.827 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.827 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.827 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.827 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.827 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.828 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.828 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.828 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.828 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.828 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.828 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.828 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.828 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.829 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.829 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.829 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.829 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.829 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.829 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_threshold": { 2024-05-07T02:43:05.829 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_threshold", 2024-05-07T02:43:05.829 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:05.829 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.829 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.830 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7257600", 2024-05-07T02:43:05.830 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.830 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.830 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.830 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-07T02:43:05.830 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.830 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.830 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.830 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.831 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.831 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.831 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.831 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "diskprediction_local", 2024-05-07T02:43:05.831 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.831 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.831 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.831 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.831 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.832 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.832 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.832 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.832 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.832 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.832 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.832 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.832 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.832 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.833 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.833 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.833 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.833 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.833 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.833 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.833 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.833 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.833 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.833 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.834 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.834 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.834 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.834 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.834 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.834 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.834 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.834 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.834 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.835 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.835 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.835 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.835 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.835 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.835 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.835 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.835 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.835 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.836 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.836 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.836 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.836 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.836 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.836 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.836 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.836 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.836 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.837 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.837 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.837 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.837 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.837 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.837 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.837 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.837 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.837 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.837 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.838 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.838 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.838 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.838 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.838 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.838 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.838 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.838 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.838 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.839 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.839 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.839 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.839 INFO:teuthology.orchestra.run.smithi019.stdout: "predict_interval": { 2024-05-07T02:43:05.839 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predict_interval", 2024-05-07T02:43:05.839 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.839 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.839 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.839 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-05-07T02:43:05.840 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.840 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.840 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.840 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.840 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.840 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.840 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.840 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.840 INFO:teuthology.orchestra.run.smithi019.stdout: "predictor_model": { 2024-05-07T02:43:05.841 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predictor_model", 2024-05-07T02:43:05.841 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.841 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.841 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.841 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "prophetstor", 2024-05-07T02:43:05.841 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.841 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.841 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.841 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.842 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.842 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.842 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.842 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.842 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-07T02:43:05.842 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-07T02:43:05.842 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.842 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.842 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.843 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-07T02:43:05.843 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.843 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.843 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.843 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.843 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.843 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.843 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.843 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.844 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.844 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.844 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.844 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.844 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.844 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.844 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.844 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.844 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.845 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.845 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.845 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.845 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.845 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.845 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.845 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.845 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.845 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "influx", 2024-05-07T02:43:05.846 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": false, 2024-05-07T02:43:05.846 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "influxdb python module not found", 2024-05-07T02:43:05.846 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.846 INFO:teuthology.orchestra.run.smithi019.stdout: "batch_size": { 2024-05-07T02:43:05.846 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "batch_size", 2024-05-07T02:43:05.846 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.846 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.846 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.846 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5000", 2024-05-07T02:43:05.847 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.847 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.847 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.847 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-07T02:43:05.847 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.847 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.847 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.847 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.847 INFO:teuthology.orchestra.run.smithi019.stdout: "database": { 2024-05-07T02:43:05.848 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "database", 2024-05-07T02:43:05.848 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.848 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.848 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.848 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "ceph", 2024-05-07T02:43:05.848 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.848 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.848 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.849 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-07T02:43:05.849 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.849 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.849 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.849 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.849 INFO:teuthology.orchestra.run.smithi019.stdout: "hostname": { 2024-05-07T02:43:05.849 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hostname", 2024-05-07T02:43:05.849 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.849 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.849 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.850 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.850 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.850 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.850 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.850 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server hostname", 2024-05-07T02:43:05.850 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.850 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.850 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.851 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.851 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-07T02:43:05.851 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-07T02:43:05.851 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:05.851 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.851 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.851 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "30", 2024-05-07T02:43:05.851 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "5", 2024-05-07T02:43:05.851 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.852 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.852 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-07T02:43:05.852 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.852 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.852 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.852 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.852 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.852 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.852 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.853 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.853 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.853 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.853 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.853 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.853 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.853 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.853 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.853 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.854 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.854 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.854 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.854 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.854 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.854 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.854 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.854 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.855 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.855 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.855 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.855 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.855 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.855 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.855 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.855 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.855 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.855 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.856 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.856 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.856 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.856 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.856 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.856 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.856 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.856 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.856 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.857 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.857 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.857 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.857 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.857 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.857 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.857 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.857 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.857 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.858 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.858 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.858 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.858 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.858 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.858 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.858 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.858 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.858 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.859 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.859 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.859 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.859 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.859 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.859 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.859 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.859 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.859 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.859 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.860 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.860 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.860 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.860 INFO:teuthology.orchestra.run.smithi019.stdout: "password": { 2024-05-07T02:43:05.860 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "password", 2024-05-07T02:43:05.860 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.860 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.860 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.860 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.860 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.861 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.861 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.861 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "password of InfluxDB server user", 2024-05-07T02:43:05.861 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.861 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.861 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.861 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.861 INFO:teuthology.orchestra.run.smithi019.stdout: "port": { 2024-05-07T02:43:05.861 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "port", 2024-05-07T02:43:05.862 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.862 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.862 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.862 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8086", 2024-05-07T02:43:05.862 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.862 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.862 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.862 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server port", 2024-05-07T02:43:05.862 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.863 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.863 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.863 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.863 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.863 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.863 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.863 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.863 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.863 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.863 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.864 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.864 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.864 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.864 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.864 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.864 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.864 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.864 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-05-07T02:43:05.864 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-05-07T02:43:05.865 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.865 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.865 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.865 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "false", 2024-05-07T02:43:05.865 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.865 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.865 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.865 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-07T02:43:05.865 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.865 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.865 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.866 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.866 INFO:teuthology.orchestra.run.smithi019.stdout: "threads": { 2024-05-07T02:43:05.866 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threads", 2024-05-07T02:43:05.866 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.866 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.866 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.866 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-07T02:43:05.866 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-07T02:43:05.867 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "32", 2024-05-07T02:43:05.867 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.867 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-07T02:43:05.867 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.867 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.867 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.867 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.867 INFO:teuthology.orchestra.run.smithi019.stdout: "username": { 2024-05-07T02:43:05.867 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "username", 2024-05-07T02:43:05.867 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.868 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.868 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.868 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.868 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.868 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.868 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.868 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "username of InfluxDB server user", 2024-05-07T02:43:05.868 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.868 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.869 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.869 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.869 INFO:teuthology.orchestra.run.smithi019.stdout: "verify_ssl": { 2024-05-07T02:43:05.869 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "verify_ssl", 2024-05-07T02:43:05.869 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.869 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.869 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.869 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "true", 2024-05-07T02:43:05.869 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.869 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.870 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.870 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-07T02:43:05.870 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.870 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.870 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.870 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.870 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.870 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.870 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.871 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "insights", 2024-05-07T02:43:05.871 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.871 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.871 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.871 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.871 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.871 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.871 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.871 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.872 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.872 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.872 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.872 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.872 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.872 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.872 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.872 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.872 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.872 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.873 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.873 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.873 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.873 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.873 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.873 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.873 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.873 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.873 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.873 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.874 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.874 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.874 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.874 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.874 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.874 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.874 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.874 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.874 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.875 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.875 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.875 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.875 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.875 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.875 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.875 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.875 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.875 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.875 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.876 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.876 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.876 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.876 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.876 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.876 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.876 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.876 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.876 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.877 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.877 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.877 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.877 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.877 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.877 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.877 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.877 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.877 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.878 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.878 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.878 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.878 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.878 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.878 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.878 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.878 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.878 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.879 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.879 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.879 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.879 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.879 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.879 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.879 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.879 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.879 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.880 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.880 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.880 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.880 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.880 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.880 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.880 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.880 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "iostat", 2024-05-07T02:43:05.880 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.880 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.881 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.881 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.881 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.881 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.881 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.881 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.881 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.881 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.881 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.881 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.882 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.882 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.882 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.882 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.882 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.882 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.882 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.882 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.883 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.883 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.883 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.883 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.883 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.883 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.883 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.883 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.883 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.883 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.884 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.884 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.884 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.884 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.884 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.884 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.884 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.884 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.884 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.885 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.885 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.885 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.885 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.885 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.885 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.885 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.885 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.885 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.885 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.886 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.886 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.886 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.886 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.886 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.886 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.886 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.886 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.887 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.887 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.887 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.887 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.887 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.887 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.887 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.887 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.887 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.888 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.888 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.888 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.888 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.888 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.888 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.888 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.888 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.888 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.889 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.889 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.889 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.889 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.889 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.889 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.889 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.889 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.889 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.890 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.890 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.890 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.890 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.890 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.890 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.890 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "k8sevents", 2024-05-07T02:43:05.890 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.891 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.891 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.891 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_event_retention_days": { 2024-05-07T02:43:05.891 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ceph_event_retention_days", 2024-05-07T02:43:05.891 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.891 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.891 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.891 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7", 2024-05-07T02:43:05.891 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.891 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.892 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.892 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-07T02:43:05.892 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.892 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.892 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.892 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.892 INFO:teuthology.orchestra.run.smithi019.stdout: "config_check_secs": { 2024-05-07T02:43:05.892 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_check_secs", 2024-05-07T02:43:05.892 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.893 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.893 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.893 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-07T02:43:05.893 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "10", 2024-05-07T02:43:05.893 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.893 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.893 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-07T02:43:05.893 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.893 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.894 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.894 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.894 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.894 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.894 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.894 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.894 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.894 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.894 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.895 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.895 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.895 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.895 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.895 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.895 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.895 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.895 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.895 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.896 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.896 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.896 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.896 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.896 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.896 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.896 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.896 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.896 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.897 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.897 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.897 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.897 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.897 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.897 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.897 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.897 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.897 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.898 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.898 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.898 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.898 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.898 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.898 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.898 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.898 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.899 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.899 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.899 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.899 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.899 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.899 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.899 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.899 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.899 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.899 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.900 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.900 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.900 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.900 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.900 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.900 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.900 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.900 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.900 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.901 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.901 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.901 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.901 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.901 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.901 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.901 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.901 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.901 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.902 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.902 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.902 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.902 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.902 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.902 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.902 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.902 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.902 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.903 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.903 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.903 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.903 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.903 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.903 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.903 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.903 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.903 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "localpool", 2024-05-07T02:43:05.904 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.904 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.904 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.904 INFO:teuthology.orchestra.run.smithi019.stdout: "failure_domain": { 2024-05-07T02:43:05.904 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "failure_domain", 2024-05-07T02:43:05.904 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.904 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.904 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.904 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "host", 2024-05-07T02:43:05.905 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.905 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.905 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.905 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "failure domain for any created local pool", 2024-05-07T02:43:05.905 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-07T02:43:05.905 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.905 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.905 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.905 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.905 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.906 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.906 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.906 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.906 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.906 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.906 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.906 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.906 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.906 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.907 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.907 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.907 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.907 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.907 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.907 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.907 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.907 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.907 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.908 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.908 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.908 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.908 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.908 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.908 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.908 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.908 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.909 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.909 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.909 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.909 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.909 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.909 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.909 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.909 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.909 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.909 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.910 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.910 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.910 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.910 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.910 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.910 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.910 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.910 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.910 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.911 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.911 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.911 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.911 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.911 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.911 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.911 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.911 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.911 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.912 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.912 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.912 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.912 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.912 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.912 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.912 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.912 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.912 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.913 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.913 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.913 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.913 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.913 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.913 INFO:teuthology.orchestra.run.smithi019.stdout: "min_size": { 2024-05-07T02:43:05.913 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_size", 2024-05-07T02:43:05.913 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.913 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.913 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.914 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.914 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.914 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.914 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.914 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default min_size for any created local pool", 2024-05-07T02:43:05.914 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-07T02:43:05.914 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.914 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.914 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.915 INFO:teuthology.orchestra.run.smithi019.stdout: "num_rep": { 2024-05-07T02:43:05.915 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "num_rep", 2024-05-07T02:43:05.915 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.915 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.915 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.915 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3", 2024-05-07T02:43:05.915 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.915 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.915 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.916 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default replica count for any created local pool", 2024-05-07T02:43:05.916 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.916 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.916 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.916 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.916 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num": { 2024-05-07T02:43:05.916 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_num", 2024-05-07T02:43:05.916 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.916 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.916 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.917 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "128", 2024-05-07T02:43:05.917 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.917 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.917 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.917 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default pg_num for any created local pool", 2024-05-07T02:43:05.917 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.917 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.917 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.917 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.918 INFO:teuthology.orchestra.run.smithi019.stdout: "prefix": { 2024-05-07T02:43:05.918 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prefix", 2024-05-07T02:43:05.918 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.918 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.918 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.918 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.918 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.918 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.919 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.919 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name prefix for any created local pool", 2024-05-07T02:43:05.919 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.919 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.919 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.919 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.919 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.919 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.919 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.919 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.920 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.920 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.920 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.920 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.920 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.920 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.920 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.920 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.920 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.921 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.921 INFO:teuthology.orchestra.run.smithi019.stdout: "subtree": { 2024-05-07T02:43:05.921 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "subtree", 2024-05-07T02:43:05.921 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.921 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.921 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.921 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "rack", 2024-05-07T02:43:05.921 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.921 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.921 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.922 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-07T02:43:05.922 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-07T02:43:05.922 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.922 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.922 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.922 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.922 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.922 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.923 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mds_autoscaler", 2024-05-07T02:43:05.923 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.923 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.923 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.923 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.923 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.923 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.923 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.923 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.924 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.924 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.924 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.924 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.924 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.924 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.924 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.924 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.924 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.924 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.925 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.925 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.925 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.925 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.925 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.925 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.925 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.925 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.925 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.925 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.926 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.926 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.926 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.926 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.926 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.926 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.926 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.926 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.926 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.927 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.927 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.927 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.927 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.927 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.927 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.927 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.927 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.927 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.928 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.928 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.928 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.928 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.928 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.928 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.928 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.928 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.928 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.928 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.929 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.929 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.929 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.929 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.929 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.929 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.929 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.929 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.929 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.930 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.930 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.930 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.930 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.930 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.930 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.930 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.930 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.930 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.931 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.931 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.931 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.931 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.931 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.931 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.931 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.931 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.931 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.932 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.932 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.932 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.932 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.932 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.932 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.932 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.932 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirroring", 2024-05-07T02:43:05.932 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.933 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.933 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.933 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.933 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.933 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.933 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.933 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.933 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.933 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.933 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.934 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.934 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.934 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.934 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.934 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.934 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.934 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.934 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.935 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.935 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.935 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.935 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.935 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.935 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.935 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.935 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.935 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.935 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.936 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.936 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.936 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.936 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.936 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.936 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.936 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.936 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.936 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.937 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.937 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.937 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.937 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.937 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.937 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.937 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.937 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.937 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.938 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.938 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.938 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.938 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.938 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.938 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.938 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.938 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.938 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.939 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.939 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.939 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.939 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.939 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.939 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.939 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.939 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.939 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.940 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.940 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.940 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.940 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.940 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.940 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.940 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.940 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.940 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.940 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.941 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.941 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.941 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.941 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.941 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.941 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.941 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.941 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.941 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.942 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.942 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.942 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.942 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.942 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.942 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.942 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "nfs", 2024-05-07T02:43:05.942 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.943 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.943 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.943 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.943 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.943 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.943 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.943 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.943 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.943 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.943 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.944 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.944 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.944 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.944 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.944 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.944 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.944 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.944 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.944 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.945 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.945 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.945 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.945 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.945 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.945 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.945 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.945 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.945 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.946 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.946 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.946 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.946 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.946 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.946 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.946 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.946 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.946 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.947 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.947 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.947 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.947 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.947 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.947 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.947 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.947 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.947 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.948 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.948 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.948 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.948 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.948 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.948 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.948 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.948 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.949 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.949 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.949 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.949 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.949 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.949 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.949 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.949 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.949 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.950 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.950 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.950 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.950 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.950 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.950 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.950 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.950 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.950 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.951 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.951 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.951 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.951 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.951 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.951 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.951 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.951 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.951 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.951 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.952 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.952 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.952 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.952 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.952 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.952 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.952 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.952 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-05-07T02:43:05.953 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.953 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.953 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.953 INFO:teuthology.orchestra.run.smithi019.stdout: "fail_fs": { 2024-05-07T02:43:05.953 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "fail_fs", 2024-05-07T02:43:05.953 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.953 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.953 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.953 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.954 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.954 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.954 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.954 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-07T02:43:05.954 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.954 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.954 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.954 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.955 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.955 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.955 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.955 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.955 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.955 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.955 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.955 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.955 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.955 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.956 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.956 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.956 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.956 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.956 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.956 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.956 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.956 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.956 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.957 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.957 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.957 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.957 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.957 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.957 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.957 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.957 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.957 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.957 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.958 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.958 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.958 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.958 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.958 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.958 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.958 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.958 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.958 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.959 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.959 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.959 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.959 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.959 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.959 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.959 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.959 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.959 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.960 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.960 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.960 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.960 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.960 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.960 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.960 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.960 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.960 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.961 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.961 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.961 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.961 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.961 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.961 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.961 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.961 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.961 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.961 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.962 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.962 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.962 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.962 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.962 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator": { 2024-05-07T02:43:05.962 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-05-07T02:43:05.962 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.962 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.962 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.963 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.963 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.963 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.963 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.963 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm", 2024-05-07T02:43:05.963 INFO:teuthology.orchestra.run.smithi019.stdout: "rook", 2024-05-07T02:43:05.963 INFO:teuthology.orchestra.run.smithi019.stdout: "test_orchestrator" 2024-05-07T02:43:05.963 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.963 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Orchestrator backend", 2024-05-07T02:43:05.964 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.964 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.964 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.964 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.964 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.964 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.964 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.964 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.964 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.965 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.965 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.965 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.965 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.965 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.965 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.965 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.965 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.965 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.966 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.966 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.966 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.966 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_perf_query", 2024-05-07T02:43:05.966 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.966 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.966 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.966 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.966 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.967 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.967 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.967 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.967 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.967 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.967 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.967 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.967 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.967 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.967 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.968 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.968 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.968 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.968 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.968 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.968 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.968 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.968 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.968 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.969 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.969 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.969 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.969 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.969 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.969 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.969 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.969 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.969 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.970 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.970 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.970 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.970 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.970 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.970 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.970 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.970 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.970 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.971 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.971 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.971 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.971 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.971 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.971 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.971 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.971 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.971 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.971 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.972 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.972 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.972 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.972 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.972 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.972 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.972 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.972 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.972 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.973 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.973 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.973 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.973 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.973 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.973 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.973 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.973 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.973 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.974 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.974 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.974 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.974 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.974 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.974 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.974 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.974 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.974 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.975 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.975 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.975 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.975 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.975 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.975 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.975 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.975 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.975 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.976 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.976 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.976 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_support", 2024-05-07T02:43:05.976 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.976 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.976 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.976 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.976 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.976 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.976 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.977 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.977 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.977 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.977 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.977 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.977 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.977 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.977 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.977 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.977 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.978 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.978 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.978 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.978 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.978 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.978 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.978 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.978 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.978 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.979 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.979 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.979 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.979 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.979 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.979 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.979 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.979 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.979 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.980 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.980 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.980 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.980 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.980 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.980 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.980 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.980 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.980 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.981 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.981 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.981 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.981 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.981 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.981 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.981 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.981 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.981 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.982 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.982 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.982 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.982 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.982 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.982 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.982 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.982 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.983 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.983 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.983 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.983 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.983 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.983 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.983 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.983 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.983 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.983 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.984 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.984 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.984 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.984 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.984 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.984 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.984 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.984 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.984 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.985 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.985 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.985 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.985 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.985 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.985 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.985 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.985 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.985 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.986 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.986 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_autoscaler", 2024-05-07T02:43:05.986 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.986 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.986 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.986 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:05.986 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:05.986 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.986 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.987 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.987 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:05.987 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.987 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.987 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.987 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.987 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.987 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.987 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.987 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.988 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.988 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.988 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.988 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.988 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.988 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.988 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.988 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:05.988 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:05.989 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.989 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.989 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.989 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.989 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.989 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.989 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.989 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.989 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.990 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.990 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.990 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.990 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:05.990 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:05.990 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:05.990 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.990 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.990 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:05.991 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.991 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.991 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:05.991 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:05.991 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:05.991 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:05.991 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:05.991 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:05.991 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:05.992 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:05.992 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.992 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.992 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.992 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.992 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.992 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:05.992 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:05.992 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:05.993 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.993 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.993 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:05.993 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.993 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.993 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.993 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.993 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.993 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.994 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.994 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.994 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-07T02:43:05.994 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-07T02:43:05.994 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:05.994 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.994 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.994 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-07T02:43:05.994 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.995 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.995 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.995 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.995 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.995 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.995 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.995 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.995 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:05.995 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:05.996 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:05.996 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:05.996 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:05.996 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:05.996 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:05.996 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.996 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.996 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:05.997 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:05.997 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.997 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.997 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.997 INFO:teuthology.orchestra.run.smithi019.stdout: "threshold": { 2024-05-07T02:43:05.997 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threshold", 2024-05-07T02:43:05.997 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-07T02:43:05.997 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:05.997 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:05.998 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-05-07T02:43:05.998 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1.0", 2024-05-07T02:43:05.998 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:05.998 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:05.998 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "scaling threshold", 2024-05-07T02:43:05.998 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-07T02:43:05.998 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:05.998 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:05.998 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.998 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:05.999 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:05.999 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:05.999 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "progress", 2024-05-07T02:43:05.999 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:05.999 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:05.999 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:05.999 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_pg_recovery_event": { 2024-05-07T02:43:05.999 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_pg_recovery_event", 2024-05-07T02:43:05.999 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.000 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.000 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.000 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.000 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.000 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.000 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.000 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-07T02:43:06.000 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.000 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.000 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.001 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.001 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-05-07T02:43:06.001 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-05-07T02:43:06.001 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.001 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.001 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.001 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.001 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.001 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.002 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.002 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.002 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.002 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.002 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.002 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.002 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.002 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.002 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.003 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.003 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.003 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.003 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.003 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.003 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.003 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.003 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.003 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.004 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.004 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.004 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.004 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.004 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.004 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.004 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.004 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.004 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.005 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.005 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.005 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.005 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.005 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.005 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.005 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.005 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.006 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.006 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.006 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.006 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.006 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.006 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.006 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.006 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.006 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.007 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.007 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.007 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.007 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.007 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.007 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.007 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.007 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.007 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.007 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.008 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.008 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.008 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.008 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.008 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.008 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.008 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.008 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.008 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.009 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.009 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.009 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.009 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.009 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.009 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.009 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.009 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.009 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.010 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.010 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.010 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.010 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.010 INFO:teuthology.orchestra.run.smithi019.stdout: "max_completed_events": { 2024-05-07T02:43:06.010 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_completed_events", 2024-05-07T02:43:06.010 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.010 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.010 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.011 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "50", 2024-05-07T02:43:06.011 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.011 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.011 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.011 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "number of past completed events to remember", 2024-05-07T02:43:06.011 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.011 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.011 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.011 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.012 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-07T02:43:06.012 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-07T02:43:06.012 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:06.012 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.012 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.012 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-07T02:43:06.012 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.012 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.012 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.012 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long the module is going to sleep", 2024-05-07T02:43:06.013 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.013 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.013 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.013 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.013 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.013 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.013 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.013 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.013 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.014 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.014 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.014 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.014 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.014 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.014 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.014 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.014 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.014 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.014 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.015 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.015 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.015 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus", 2024-05-07T02:43:06.015 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.015 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.015 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.015 INFO:teuthology.orchestra.run.smithi019.stdout: "cache": { 2024-05-07T02:43:06.015 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cache", 2024-05-07T02:43:06.015 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.016 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.016 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.016 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.016 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.016 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.016 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.016 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.016 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.016 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.017 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.017 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.017 INFO:teuthology.orchestra.run.smithi019.stdout: "exclude_perf_counters": { 2024-05-07T02:43:06.017 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "exclude_perf_counters", 2024-05-07T02:43:06.017 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.017 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.017 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.017 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.017 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.018 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.018 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.018 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-07T02:43:06.018 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-07T02:43:06.018 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.018 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.018 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.018 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.018 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.019 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.019 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.019 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.019 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.019 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.019 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.019 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.019 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.019 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.020 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.020 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.020 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.020 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.020 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.020 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.020 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.020 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.020 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.020 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.021 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.021 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.021 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.021 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.021 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.021 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.021 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.021 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.021 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.022 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.022 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.022 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.022 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.022 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.022 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.022 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.022 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.022 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.023 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.023 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.023 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.023 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.023 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.023 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.023 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.023 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.023 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.024 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.024 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.024 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.024 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.024 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.024 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.024 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.024 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.024 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.025 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.025 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.025 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.025 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.025 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.025 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.025 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.025 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.025 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.026 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.026 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.026 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.026 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.026 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools": { 2024-05-07T02:43:06.026 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools", 2024-05-07T02:43:06.026 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.026 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.026 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.027 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.027 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.027 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.027 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.027 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.027 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.027 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.027 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.027 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.028 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-07T02:43:06.028 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-07T02:43:06.028 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.028 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.028 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.028 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "300", 2024-05-07T02:43:06.028 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.028 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.028 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.028 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.029 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.029 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.029 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.029 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.029 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_interval": { 2024-05-07T02:43:06.029 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_interval", 2024-05-07T02:43:06.029 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-07T02:43:06.029 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.029 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.030 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15.0", 2024-05-07T02:43:06.030 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.030 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.030 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.030 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.030 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.030 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.030 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.030 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.031 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-05-07T02:43:06.031 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-05-07T02:43:06.031 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.031 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.031 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.031 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-05-07T02:43:06.031 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.031 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.031 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.032 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-07T02:43:06.032 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.032 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.032 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.032 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.032 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-05-07T02:43:06.032 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-05-07T02:43:06.032 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.032 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.033 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.033 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "9283", 2024-05-07T02:43:06.033 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.033 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.033 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.033 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-07T02:43:06.033 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.033 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.033 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.034 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.034 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.034 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.034 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.034 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.034 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.034 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.034 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.034 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.035 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.035 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.035 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.035 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.035 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.035 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.035 INFO:teuthology.orchestra.run.smithi019.stdout: "stale_cache_strategy": { 2024-05-07T02:43:06.035 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stale_cache_strategy", 2024-05-07T02:43:06.035 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.035 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.036 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.036 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "log", 2024-05-07T02:43:06.036 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.036 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.036 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.036 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.036 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.037 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.037 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.037 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.037 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-05-07T02:43:06.037 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-05-07T02:43:06.037 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.037 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.037 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.037 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "default", 2024-05-07T02:43:06.038 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.038 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.038 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.038 INFO:teuthology.orchestra.run.smithi019.stdout: "default", 2024-05-07T02:43:06.038 INFO:teuthology.orchestra.run.smithi019.stdout: "error" 2024-05-07T02:43:06.038 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.038 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.038 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.038 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.039 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.039 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.039 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-05-07T02:43:06.039 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-05-07T02:43:06.039 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.039 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.039 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.039 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-05-07T02:43:06.039 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-05-07T02:43:06.040 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-05-07T02:43:06.040 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.040 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.040 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.040 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.040 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.040 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.040 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.040 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.041 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.041 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_support", 2024-05-07T02:43:06.041 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.041 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.041 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.041 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.041 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.041 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.041 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.042 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.042 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.042 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.042 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.042 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.042 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.042 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.042 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.042 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.043 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.043 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.043 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.043 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.043 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.043 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.043 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.043 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.043 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.044 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.044 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.044 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.044 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.044 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.044 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.044 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.044 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.044 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.045 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.045 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.045 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.045 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.045 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.045 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.045 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.045 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.046 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.046 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.046 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.046 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.046 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.046 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.046 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.046 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.046 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.047 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.047 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.047 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.047 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.047 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.047 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.047 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.047 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.047 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.048 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.048 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.048 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.048 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.048 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.048 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.048 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.048 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.048 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.049 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.049 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.049 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.049 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.049 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_snap_create": { 2024-05-07T02:43:06.049 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_snap_create", 2024-05-07T02:43:06.049 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.049 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.049 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.049 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-07T02:43:06.050 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.050 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.050 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.050 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.050 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.050 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.050 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.050 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.051 INFO:teuthology.orchestra.run.smithi019.stdout: "mirror_snapshot_schedule": { 2024-05-07T02:43:06.051 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirror_snapshot_schedule", 2024-05-07T02:43:06.051 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.051 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.051 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.051 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.051 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.051 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.051 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.051 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.052 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.052 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.052 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.052 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.052 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.052 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.052 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.052 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.052 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.052 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.053 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.053 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.053 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.053 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.053 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.053 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.053 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.053 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.054 INFO:teuthology.orchestra.run.smithi019.stdout: "trash_purge_schedule": { 2024-05-07T02:43:06.054 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "trash_purge_schedule", 2024-05-07T02:43:06.054 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.054 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.054 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.054 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.054 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.054 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.054 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.055 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.055 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.055 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.055 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.055 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.055 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.055 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.055 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.055 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "restful", 2024-05-07T02:43:06.056 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.056 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.056 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.056 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_auth": { 2024-05-07T02:43:06.056 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_auth", 2024-05-07T02:43:06.056 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.056 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.056 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.056 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.056 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.057 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.057 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.057 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.057 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.057 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.057 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.057 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.057 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-05-07T02:43:06.057 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-05-07T02:43:06.058 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.058 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.058 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.058 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.058 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.058 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.058 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.058 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.059 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.059 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.059 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.059 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.059 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.059 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.059 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.059 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.059 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.059 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.060 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.060 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.060 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.060 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.060 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.060 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.060 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.060 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.060 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.060 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.061 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.061 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.061 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.061 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.061 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.061 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.061 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.061 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.061 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.062 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.062 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.062 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.062 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.062 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.062 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.062 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.062 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.062 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.062 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.063 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.063 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.063 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.063 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.063 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.063 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.063 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.063 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.063 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.064 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.064 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.064 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.064 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.064 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.064 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.064 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.064 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.064 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.065 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.065 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.065 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.065 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.065 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.065 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.065 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.065 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.065 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.066 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.066 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.066 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.066 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.066 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.066 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.066 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.066 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.067 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-05-07T02:43:06.067 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-05-07T02:43:06.067 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.067 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.067 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.067 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.067 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.067 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.067 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.067 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.068 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.068 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.068 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.068 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.068 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-05-07T02:43:06.068 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-05-07T02:43:06.068 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.068 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.068 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.069 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.069 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.069 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.069 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.069 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.069 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.069 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.069 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.069 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.069 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.070 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.070 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.070 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.070 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.070 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.070 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.070 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.070 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.071 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.071 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.071 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.071 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.071 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.071 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.071 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.071 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.071 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rgw", 2024-05-07T02:43:06.072 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.072 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.072 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.072 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.072 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.072 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.072 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.072 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.072 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.072 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.073 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.073 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.073 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.073 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.073 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.073 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.073 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.073 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.073 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.074 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.074 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.074 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.074 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.074 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.074 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.074 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.074 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.074 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.075 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.075 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.075 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.075 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.075 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.075 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.075 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.075 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.075 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.076 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.076 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.076 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.076 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.076 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.076 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.076 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.076 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.076 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.077 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.077 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.077 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.077 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.077 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.077 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.077 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.077 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.077 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.078 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.078 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.078 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.078 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.078 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.078 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.078 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.078 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.078 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.079 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.079 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.079 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.079 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.079 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.079 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.079 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.079 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.080 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.080 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.080 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.080 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.080 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.080 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.080 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.080 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.080 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.081 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.081 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.081 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.081 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.081 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.081 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.081 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.081 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.081 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.082 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rook", 2024-05-07T02:43:06.082 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.082 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.082 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.082 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.082 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.082 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.082 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.083 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.083 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.083 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.083 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.083 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.083 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.083 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.083 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.083 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.084 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.084 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.084 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.084 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.084 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.084 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.084 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.084 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.084 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.085 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.085 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.085 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.085 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.085 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.085 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.085 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.085 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.085 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.086 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.086 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.086 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.086 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.086 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.086 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.086 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.086 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.086 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.087 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.087 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.087 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.087 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.087 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.087 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.087 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.087 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.087 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.088 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.088 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.088 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.088 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.088 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.088 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.088 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.088 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.089 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.089 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.089 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.089 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.089 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.089 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.089 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.089 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.089 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.090 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.090 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.090 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.090 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.090 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.090 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.090 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.090 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.090 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.091 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.091 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.091 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.091 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.091 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.091 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.091 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.091 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.091 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.092 INFO:teuthology.orchestra.run.smithi019.stdout: "storage_class": { 2024-05-07T02:43:06.092 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "storage_class", 2024-05-07T02:43:06.092 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.092 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.092 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.092 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "local", 2024-05-07T02:43:06.092 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.092 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.092 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.093 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-07T02:43:06.093 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.093 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.093 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.093 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.093 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.093 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.093 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.093 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "selftest", 2024-05-07T02:43:06.094 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.094 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.094 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.094 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.094 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.094 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.094 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.094 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.094 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.095 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.095 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.095 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.095 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.095 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.095 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.095 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.095 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.095 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.096 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.096 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.096 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.096 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.096 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.096 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.096 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.096 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.096 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.096 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.097 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.097 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.097 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.097 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.097 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.097 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.097 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.097 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.097 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.097 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.098 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.098 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.098 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.098 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.098 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.098 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.098 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.098 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.099 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.099 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.099 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.099 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.099 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.099 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.099 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.099 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.099 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.099 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.100 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.100 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.100 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.100 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.100 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.100 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.100 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.100 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.100 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.101 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.101 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.101 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.101 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.101 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.101 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.101 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.101 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.101 INFO:teuthology.orchestra.run.smithi019.stdout: "roption1": { 2024-05-07T02:43:06.102 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption1", 2024-05-07T02:43:06.102 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.102 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.102 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.102 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.102 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.102 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.102 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.102 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.103 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.103 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.103 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.103 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.103 INFO:teuthology.orchestra.run.smithi019.stdout: "roption2": { 2024-05-07T02:43:06.103 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption2", 2024-05-07T02:43:06.103 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.103 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.103 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.103 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "xyz", 2024-05-07T02:43:06.104 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.104 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.104 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.104 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.104 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.104 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.104 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.104 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.104 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption1": { 2024-05-07T02:43:06.105 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption1", 2024-05-07T02:43:06.105 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.105 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.105 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.105 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.105 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.105 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.105 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.105 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.105 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.106 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.106 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.106 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.106 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption2": { 2024-05-07T02:43:06.106 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption2", 2024-05-07T02:43:06.106 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.106 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.106 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.107 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.107 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.107 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.107 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.107 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.107 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.107 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.107 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.107 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.107 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption3": { 2024-05-07T02:43:06.108 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption3", 2024-05-07T02:43:06.108 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-07T02:43:06.108 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.108 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.108 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.108 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.108 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.108 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.108 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.108 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.109 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.109 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.109 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.109 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption4": { 2024-05-07T02:43:06.109 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption4", 2024-05-07T02:43:06.109 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.109 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.109 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.109 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.109 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.110 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.110 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.110 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.110 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.110 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.110 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.110 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.110 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption5": { 2024-05-07T02:43:06.111 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption5", 2024-05-07T02:43:06.111 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.111 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.111 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.111 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.111 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.111 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.111 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.111 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.111 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.112 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.112 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.112 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.112 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption6": { 2024-05-07T02:43:06.112 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption6", 2024-05-07T02:43:06.112 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.112 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.112 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.112 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.113 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.113 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.113 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.113 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.113 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.113 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.113 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.113 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.113 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption7": { 2024-05-07T02:43:06.113 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption7", 2024-05-07T02:43:06.114 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.114 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.114 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.114 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.114 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-07T02:43:06.114 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "42", 2024-05-07T02:43:06.114 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.114 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.115 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.115 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.115 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.115 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.115 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.115 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.115 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.115 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.115 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.116 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.116 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.116 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.116 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.116 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.116 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.116 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.116 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.116 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.117 INFO:teuthology.orchestra.run.smithi019.stdout: "testkey": { 2024-05-07T02:43:06.117 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testkey", 2024-05-07T02:43:06.117 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.117 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.117 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.117 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.117 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.117 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.117 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.118 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.118 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.118 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.118 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.118 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.118 INFO:teuthology.orchestra.run.smithi019.stdout: "testlkey": { 2024-05-07T02:43:06.118 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testlkey", 2024-05-07T02:43:06.118 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.119 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.119 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.119 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.119 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.119 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.119 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.119 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.119 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.119 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.120 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.120 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.120 INFO:teuthology.orchestra.run.smithi019.stdout: "testnewline": { 2024-05-07T02:43:06.120 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testnewline", 2024-05-07T02:43:06.120 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.120 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.120 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.120 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.120 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.121 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.121 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.121 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.121 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.121 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.121 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.121 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.121 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.121 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.121 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.122 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smb", 2024-05-07T02:43:06.122 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.122 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.122 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.122 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.122 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.122 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.122 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.123 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.123 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.123 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.123 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.123 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.123 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.123 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.123 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.123 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.124 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.124 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.124 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.124 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.124 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.124 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.124 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.124 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.124 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.125 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.125 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.125 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.125 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.125 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.125 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.125 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.125 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.125 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.126 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.126 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.126 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.126 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.126 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.126 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.126 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.126 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.127 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.127 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.127 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.127 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.127 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.127 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.127 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.127 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.127 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.127 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.128 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.128 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.128 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.128 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.128 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.128 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.128 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.128 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.129 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.129 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.129 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.129 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.129 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.129 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.129 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.129 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.129 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.130 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.130 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.130 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.130 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.130 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.130 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.130 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.130 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.130 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.131 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.131 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.131 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.131 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.131 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.131 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.131 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.131 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.131 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.132 INFO:teuthology.orchestra.run.smithi019.stdout: "update_orchestration": { 2024-05-07T02:43:06.132 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "update_orchestration", 2024-05-07T02:43:06.132 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.132 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.132 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.132 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.132 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.132 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.132 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.133 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-07T02:43:06.133 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.133 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.133 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.133 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.133 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.133 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.133 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.133 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snap_schedule", 2024-05-07T02:43:06.134 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.134 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.134 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.134 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_m_granularity": { 2024-05-07T02:43:06.134 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_m_granularity", 2024-05-07T02:43:06.134 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.134 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.134 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.134 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.135 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.135 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.135 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.135 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow minute scheduled snapshots", 2024-05-07T02:43:06.135 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.135 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.135 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.135 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.135 INFO:teuthology.orchestra.run.smithi019.stdout: "dump_on_update": { 2024-05-07T02:43:06.136 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dump_on_update", 2024-05-07T02:43:06.136 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.136 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.136 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.136 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.136 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.136 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.136 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.136 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "dump database to debug log on update", 2024-05-07T02:43:06.137 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.137 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.137 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.137 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.137 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.137 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.137 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.137 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.137 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.138 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.138 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.138 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.138 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.138 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.138 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.138 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.138 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.138 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.139 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.139 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.139 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.139 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.139 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.139 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.139 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.139 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.139 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.140 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.140 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.140 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.140 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.140 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.140 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.140 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.140 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.140 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.141 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.141 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.141 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.141 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.141 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.141 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.141 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.141 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.141 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.142 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.142 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.142 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.142 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.142 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.142 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.142 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.142 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.142 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.143 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.143 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.143 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.143 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.143 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.143 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.143 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.143 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.143 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.143 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.144 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.144 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.144 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.144 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.144 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.144 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.144 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.144 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.144 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.145 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.145 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.145 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.145 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.145 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.145 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.145 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.145 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.145 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.146 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.146 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.146 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.146 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.146 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.146 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.146 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.146 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.146 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.147 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stats", 2024-05-07T02:43:06.147 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.147 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.147 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.147 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.147 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.147 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.147 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.147 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.148 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.148 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.148 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.148 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.148 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.148 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.148 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.148 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.148 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.148 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.149 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.149 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.149 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.149 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.149 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.149 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.149 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.149 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.149 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.150 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.150 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.150 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.150 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.150 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.150 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.150 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.150 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.150 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.151 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.151 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.151 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.151 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.151 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.151 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.151 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.151 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.151 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.152 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.152 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.152 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.152 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.152 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.152 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.152 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.152 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.152 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.152 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.153 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.153 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.153 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.153 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.153 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.153 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.153 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.153 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.153 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.154 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.154 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.154 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.154 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.154 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.154 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.154 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.154 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.154 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.155 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.155 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.155 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.155 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.155 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.155 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.155 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.155 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.155 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.156 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.156 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.156 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.156 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.156 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.156 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.156 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.156 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.156 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "status", 2024-05-07T02:43:06.156 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.157 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.157 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.157 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.157 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.157 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.157 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.157 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.157 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.157 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.158 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.158 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.158 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.158 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.158 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.158 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.158 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.158 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.159 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.159 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.159 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.159 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.159 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.159 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.159 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.159 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.159 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.159 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.160 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.160 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.160 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.160 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.160 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.160 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.160 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.160 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.160 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.161 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.161 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.161 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.161 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.161 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.161 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.161 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.161 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.161 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.162 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.162 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.162 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.162 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.162 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.162 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.162 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.162 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.162 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.163 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.163 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.163 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.163 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.163 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.163 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.163 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.163 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.163 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.164 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.164 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.164 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.164 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.164 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.164 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.164 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.164 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.164 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.165 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.165 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.165 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.165 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.165 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.165 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.165 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.165 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.165 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.166 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.166 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.166 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.166 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.166 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.166 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.166 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.166 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.166 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telegraf", 2024-05-07T02:43:06.167 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.167 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.167 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.167 INFO:teuthology.orchestra.run.smithi019.stdout: "address": { 2024-05-07T02:43:06.167 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "address", 2024-05-07T02:43:06.167 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.167 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.167 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.167 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-07T02:43:06.168 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.168 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.168 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.168 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.168 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.168 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.168 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.168 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.169 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-07T02:43:06.169 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-07T02:43:06.169 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:06.169 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.169 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.169 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15", 2024-05-07T02:43:06.169 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.169 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.169 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.170 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.170 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.170 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.170 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.170 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.170 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.170 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.170 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.171 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.171 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.171 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.171 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.171 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.171 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.171 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.171 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.171 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.172 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.172 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.172 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.172 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.172 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.172 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.172 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.172 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.172 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.173 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.173 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.173 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.173 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.173 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.173 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.173 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.173 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.173 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.174 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.174 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.174 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.174 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.174 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.174 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.174 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.174 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.175 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.175 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.175 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.175 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.175 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.175 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.175 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.175 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.175 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.175 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.176 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.176 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.176 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.176 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.176 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.176 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.176 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.176 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.176 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.177 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.177 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.177 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.177 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.177 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.177 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.177 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.177 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.177 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.178 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.178 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.178 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.178 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.178 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.178 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.178 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.178 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.178 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.179 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.179 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.179 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.179 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.179 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.179 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.179 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.179 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.179 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.180 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.180 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.180 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.180 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telemetry", 2024-05-07T02:43:06.180 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.180 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.180 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.180 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_basic": { 2024-05-07T02:43:06.180 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_basic", 2024-05-07T02:43:06.180 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.181 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.181 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.181 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.181 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.181 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.181 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.181 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-07T02:43:06.181 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.182 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.182 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.182 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.182 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_crash": { 2024-05-07T02:43:06.182 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_crash", 2024-05-07T02:43:06.182 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.182 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.182 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.182 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.183 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.183 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.183 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.183 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-07T02:43:06.183 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.183 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.183 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.183 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.183 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_device": { 2024-05-07T02:43:06.184 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_device", 2024-05-07T02:43:06.184 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.184 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.184 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.184 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.184 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.184 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.184 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.184 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-07T02:43:06.185 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.185 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.185 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.185 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.185 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_ident": { 2024-05-07T02:43:06.185 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_ident", 2024-05-07T02:43:06.185 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.185 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.185 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.186 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.186 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.186 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.186 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.186 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-07T02:43:06.186 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.186 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.186 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.187 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.187 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_perf": { 2024-05-07T02:43:06.187 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_perf", 2024-05-07T02:43:06.187 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.187 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.187 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.187 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.187 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.187 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.188 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.188 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-07T02:43:06.188 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.188 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.188 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.188 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.188 INFO:teuthology.orchestra.run.smithi019.stdout: "contact": { 2024-05-07T02:43:06.188 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "contact", 2024-05-07T02:43:06.188 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.188 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.189 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.189 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.189 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.189 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.189 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.189 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.189 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.189 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.189 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.190 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.190 INFO:teuthology.orchestra.run.smithi019.stdout: "description": { 2024-05-07T02:43:06.190 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "description", 2024-05-07T02:43:06.190 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.190 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.190 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.190 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.190 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.190 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.191 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.191 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.191 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.191 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.191 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.191 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.191 INFO:teuthology.orchestra.run.smithi019.stdout: "device_url": { 2024-05-07T02:43:06.191 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_url", 2024-05-07T02:43:06.191 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.192 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.192 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.192 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-07T02:43:06.192 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.192 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.192 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.192 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.192 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.192 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.193 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.193 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.193 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-05-07T02:43:06.193 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-05-07T02:43:06.193 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.193 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.193 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.193 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.193 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.194 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.194 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.194 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.194 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.194 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.194 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.194 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.194 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-07T02:43:06.194 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-07T02:43:06.195 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.195 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.195 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.195 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "24", 2024-05-07T02:43:06.195 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "8", 2024-05-07T02:43:06.195 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.195 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.195 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.195 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.196 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.196 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.196 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.196 INFO:teuthology.orchestra.run.smithi019.stdout: "last_opt_revision": { 2024-05-07T02:43:06.196 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "last_opt_revision", 2024-05-07T02:43:06.196 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.196 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.196 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.196 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1", 2024-05-07T02:43:06.197 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.197 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.197 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.197 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.197 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.197 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.197 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.197 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.197 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard": { 2024-05-07T02:43:06.197 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard", 2024-05-07T02:43:06.198 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.198 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.198 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.198 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.198 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.198 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.198 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.198 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.198 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.199 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.199 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.199 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.199 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard_description": { 2024-05-07T02:43:06.199 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard_description", 2024-05-07T02:43:06.199 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.199 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.199 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.199 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.200 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.200 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.200 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.200 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.200 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.200 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.200 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.200 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.200 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.200 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.201 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.201 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.201 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.201 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.201 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.201 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.201 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.201 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.201 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.202 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.202 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.202 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.202 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.202 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.202 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.202 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.202 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.202 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.203 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.203 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.203 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.203 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.203 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.203 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.203 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.203 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.203 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.204 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.204 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.204 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.204 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.204 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.204 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.204 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.204 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.204 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.205 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.205 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.205 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.205 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.205 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.205 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.205 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.205 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.205 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.205 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.206 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.206 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.206 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.206 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.206 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.206 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.206 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.206 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.207 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.207 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.207 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.207 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.207 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.207 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.207 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.207 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.207 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.208 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.208 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.208 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.208 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.208 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.208 INFO:teuthology.orchestra.run.smithi019.stdout: "organization": { 2024-05-07T02:43:06.208 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "organization", 2024-05-07T02:43:06.208 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.208 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.209 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.209 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.209 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.209 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.209 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.209 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.209 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.209 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.209 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.210 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.210 INFO:teuthology.orchestra.run.smithi019.stdout: "proxy": { 2024-05-07T02:43:06.210 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "proxy", 2024-05-07T02:43:06.210 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.210 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.210 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.210 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.210 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.210 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.211 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.211 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.211 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.211 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.211 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.211 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.211 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.211 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.211 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.212 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.212 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.212 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.212 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.212 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.212 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.212 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.212 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.213 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.213 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.213 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.213 INFO:teuthology.orchestra.run.smithi019.stdout: "url": { 2024-05-07T02:43:06.213 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url", 2024-05-07T02:43:06.213 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.213 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.213 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.213 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-07T02:43:06.214 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.214 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.214 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.214 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.214 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.214 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.214 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.214 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.214 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.215 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.215 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.215 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "test_orchestrator", 2024-05-07T02:43:06.215 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.215 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.215 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.215 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.215 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.215 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.216 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.216 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.216 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.216 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.216 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.216 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.216 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.216 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.216 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.217 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.217 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.217 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.217 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.217 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.217 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.217 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.217 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.217 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.217 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.218 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.218 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.218 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.218 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.218 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.218 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.218 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.218 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.219 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.219 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.219 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.219 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.219 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.219 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.219 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.219 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.219 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.220 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.220 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.220 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.220 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.220 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.220 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.220 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.220 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.220 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.221 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.221 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.221 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.221 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.221 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.221 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.221 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.221 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.221 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.222 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.222 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.222 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.222 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.222 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.222 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.222 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.222 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.222 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.223 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.223 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.223 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.223 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.223 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.223 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.223 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.223 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.223 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.224 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.224 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.224 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.224 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.224 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.224 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.224 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.224 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.224 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.225 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.225 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.225 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.225 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "volumes", 2024-05-07T02:43:06.225 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.225 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.225 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.225 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.226 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.226 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.226 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.226 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.226 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.226 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.226 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.226 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.226 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.227 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.227 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.227 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.227 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.227 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.227 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.227 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.227 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.227 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.228 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.228 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.228 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.228 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.228 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.228 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.228 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.228 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.228 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.228 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.229 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.229 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.229 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.229 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.229 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.229 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.229 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.229 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.229 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.230 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.230 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.230 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.230 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.230 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.230 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.230 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.230 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.231 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.231 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.231 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.231 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.231 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.231 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.231 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.231 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.231 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.231 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.232 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.232 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.232 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.232 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.232 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.232 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.232 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.232 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.232 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.233 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.233 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.233 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.233 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.233 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.233 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_clones": { 2024-05-07T02:43:06.233 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_clones", 2024-05-07T02:43:06.233 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.233 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.234 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.234 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4", 2024-05-07T02:43:06.234 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.234 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.234 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.234 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-07T02:43:06.234 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.234 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.234 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.235 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.235 INFO:teuthology.orchestra.run.smithi019.stdout: "periodic_async_work": { 2024-05-07T02:43:06.235 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "periodic_async_work", 2024-05-07T02:43:06.235 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.235 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.235 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.235 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.235 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.235 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.236 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.236 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Periodically check for async work", 2024-05-07T02:43:06.236 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.236 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.236 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.236 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.236 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_delay": { 2024-05-07T02:43:06.236 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_delay", 2024-05-07T02:43:06.236 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.236 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.237 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.237 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.237 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.237 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.237 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.237 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-07T02:43:06.237 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.237 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.237 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.237 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.238 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_no_wait": { 2024-05-07T02:43:06.238 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_no_wait", 2024-05-07T02:43:06.238 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.238 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.238 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.238 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.238 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.238 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.239 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.239 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-07T02:43:06.239 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.239 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.239 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.239 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.239 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.239 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.239 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.239 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.240 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.240 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.240 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.240 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.240 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.240 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.240 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.240 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.240 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.241 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.241 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.241 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.241 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.241 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix", 2024-05-07T02:43:06.241 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.241 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.241 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.241 INFO:teuthology.orchestra.run.smithi019.stdout: "discovery_interval": { 2024-05-07T02:43:06.242 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "discovery_interval", 2024-05-07T02:43:06.242 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-07T02:43:06.242 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.242 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.242 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "100", 2024-05-07T02:43:06.242 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.242 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.242 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.242 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.243 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.243 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.243 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.243 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.243 INFO:teuthology.orchestra.run.smithi019.stdout: "identifier": { 2024-05-07T02:43:06.243 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "identifier", 2024-05-07T02:43:06.243 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.243 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.243 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.244 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.244 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.244 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.244 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.244 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.244 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.244 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.244 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.244 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.244 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-07T02:43:06.245 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-07T02:43:06.245 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:06.245 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.245 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.245 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-07T02:43:06.245 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.245 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.245 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.245 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.246 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.246 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.246 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.246 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.246 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.246 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.246 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.246 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.246 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.247 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.247 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.247 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.247 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.247 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.247 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.247 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.247 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.247 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.247 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.248 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.248 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.248 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.248 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.248 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.248 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.248 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.248 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.248 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.249 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.249 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.249 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.249 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.249 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.249 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.249 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.249 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.249 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.250 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.250 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.250 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.250 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.250 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.250 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.250 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.250 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.251 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.251 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.251 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.251 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.251 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.251 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.251 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.251 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.251 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.252 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.252 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.252 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.252 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.252 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.252 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.252 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.252 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.252 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.253 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.253 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.253 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.253 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.253 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.253 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.253 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.253 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.253 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.253 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.254 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.254 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.254 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.254 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.254 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.254 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.254 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.254 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.255 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.255 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.255 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.255 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.255 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.255 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.255 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.255 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_host": { 2024-05-07T02:43:06.255 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_host", 2024-05-07T02:43:06.256 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.256 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.256 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.256 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.256 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.256 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.256 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.256 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.256 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.257 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.257 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.257 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.257 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_port": { 2024-05-07T02:43:06.257 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_port", 2024-05-07T02:43:06.257 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.257 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.257 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.257 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10051", 2024-05-07T02:43:06.258 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.258 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.258 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.258 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.258 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.258 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.258 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.258 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.258 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_sender": { 2024-05-07T02:43:06.259 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_sender", 2024-05-07T02:43:06.259 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.259 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.259 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.259 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-07T02:43:06.259 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.259 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.259 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.260 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.260 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.260 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.260 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.260 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.260 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.260 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.260 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-05-07T02:43:06.260 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.261 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.261 INFO:teuthology.orchestra.run.smithi019.stdout: "modules": [ 2024-05-07T02:43:06.261 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm", 2024-05-07T02:43:06.261 INFO:teuthology.orchestra.run.smithi019.stdout: "dashboard", 2024-05-07T02:43:06.261 INFO:teuthology.orchestra.run.smithi019.stdout: "iostat", 2024-05-07T02:43:06.261 INFO:teuthology.orchestra.run.smithi019.stdout: "nfs", 2024-05-07T02:43:06.261 INFO:teuthology.orchestra.run.smithi019.stdout: "restful", 2024-05-07T02:43:06.261 INFO:teuthology.orchestra.run.smithi019.stdout: "snap_schedule" 2024-05-07T02:43:06.261 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.261 INFO:teuthology.orchestra.run.smithi019.stdout: "available_modules": [ 2024-05-07T02:43:06.262 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.262 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "alerts", 2024-05-07T02:43:06.262 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.262 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.262 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.262 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-07T02:43:06.262 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-07T02:43:06.262 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:06.263 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.263 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.263 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-07T02:43:06.263 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.263 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.263 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.263 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How frequently to reexamine health status", 2024-05-07T02:43:06.263 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.263 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.263 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.264 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.264 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.264 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.264 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.264 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.264 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.264 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.264 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.264 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.265 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.265 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.265 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.265 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.265 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.265 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.265 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.265 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.265 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.266 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.266 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.266 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.266 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.266 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.266 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.266 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.266 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.266 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.267 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.267 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.267 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.267 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.267 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.267 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.267 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.267 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.267 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.268 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.268 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.268 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.268 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.268 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.268 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.268 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.268 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.268 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.269 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.269 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.269 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.269 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.269 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.269 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.269 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.269 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.269 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.270 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.270 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.270 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.270 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.270 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.270 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.270 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.270 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.270 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.271 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.271 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.271 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.271 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.271 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.271 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.271 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.271 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.271 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_destination": { 2024-05-07T02:43:06.272 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_destination", 2024-05-07T02:43:06.272 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.272 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.272 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.272 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.272 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.272 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.272 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.272 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email address to send alerts to", 2024-05-07T02:43:06.273 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.273 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.273 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.273 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.273 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_from_name": { 2024-05-07T02:43:06.273 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_from_name", 2024-05-07T02:43:06.273 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.273 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.273 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.274 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "Ceph", 2024-05-07T02:43:06.274 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.274 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.274 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.274 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email From: name", 2024-05-07T02:43:06.274 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.274 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.274 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.275 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.275 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_host": { 2024-05-07T02:43:06.275 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_host", 2024-05-07T02:43:06.275 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.275 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.275 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.275 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.275 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.275 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.276 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.276 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP server", 2024-05-07T02:43:06.276 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.276 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.276 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.276 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.276 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_password": { 2024-05-07T02:43:06.276 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_password", 2024-05-07T02:43:06.276 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.277 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.277 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.277 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.277 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.277 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.277 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.277 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Password to authenticate with", 2024-05-07T02:43:06.277 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.277 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.277 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.278 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.278 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_port": { 2024-05-07T02:43:06.278 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_port", 2024-05-07T02:43:06.278 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.278 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.278 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.278 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "465", 2024-05-07T02:43:06.278 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.278 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.279 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.279 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP port", 2024-05-07T02:43:06.279 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.279 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.279 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.279 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.279 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_sender": { 2024-05-07T02:43:06.279 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_sender", 2024-05-07T02:43:06.279 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.280 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.280 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.280 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.280 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.280 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.280 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.280 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP envelope sender", 2024-05-07T02:43:06.280 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.280 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.280 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.281 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.281 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_ssl": { 2024-05-07T02:43:06.281 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_ssl", 2024-05-07T02:43:06.281 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.281 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.281 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.281 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.281 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.281 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.282 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.282 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-07T02:43:06.282 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.282 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.282 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.282 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.282 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_user": { 2024-05-07T02:43:06.282 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_user", 2024-05-07T02:43:06.283 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.283 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.283 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.283 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.283 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.283 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.283 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.283 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "User to authenticate as", 2024-05-07T02:43:06.283 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.283 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.284 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.284 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.284 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.284 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.284 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.284 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.284 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.284 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.284 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.285 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.285 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.285 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.285 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.285 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.285 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.285 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.285 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.285 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.286 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.286 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "balancer", 2024-05-07T02:43:06.286 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.286 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.286 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.286 INFO:teuthology.orchestra.run.smithi019.stdout: "active": { 2024-05-07T02:43:06.286 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "active", 2024-05-07T02:43:06.286 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.286 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.287 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.287 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.287 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.287 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.287 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.287 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically balance PGs across cluster", 2024-05-07T02:43:06.287 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.287 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.287 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.287 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.288 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_time": { 2024-05-07T02:43:06.288 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_time", 2024-05-07T02:43:06.288 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.288 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.288 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.288 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0000", 2024-05-07T02:43:06.288 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.288 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.288 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.289 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "beginning time of day to automatically balance", 2024-05-07T02:43:06.289 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-07T02:43:06.289 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.289 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.289 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.289 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_weekday": { 2024-05-07T02:43:06.289 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_weekday", 2024-05-07T02:43:06.289 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-07T02:43:06.289 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.289 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.290 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.290 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-05-07T02:43:06.290 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-05-07T02:43:06.290 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.290 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-07T02:43:06.290 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-07T02:43:06.290 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.290 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.291 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.291 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_max_iterations": { 2024-05-07T02:43:06.291 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_max_iterations", 2024-05-07T02:43:06.291 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-07T02:43:06.291 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.291 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.291 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "25", 2024-05-07T02:43:06.291 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-07T02:43:06.291 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "250", 2024-05-07T02:43:06.291 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.292 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-07T02:43:06.292 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.292 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.292 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.292 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.292 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_metrics": { 2024-05-07T02:43:06.292 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_metrics", 2024-05-07T02:43:06.292 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.292 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.293 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.293 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "pgs,objects,bytes", 2024-05-07T02:43:06.293 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.293 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.293 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.293 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-07T02:43:06.293 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-07T02:43:06.293 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.293 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.294 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.294 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_step": { 2024-05-07T02:43:06.294 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_step", 2024-05-07T02:43:06.294 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-07T02:43:06.294 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.294 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.294 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.5", 2024-05-07T02:43:06.294 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0.001", 2024-05-07T02:43:06.295 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "0.999", 2024-05-07T02:43:06.295 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.295 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "aggressiveness of optimization", 2024-05-07T02:43:06.295 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-07T02:43:06.295 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.295 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.295 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.295 INFO:teuthology.orchestra.run.smithi019.stdout: "end_time": { 2024-05-07T02:43:06.295 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_time", 2024-05-07T02:43:06.296 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.296 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.296 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.296 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2359", 2024-05-07T02:43:06.296 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.296 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.296 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.296 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ending time of day to automatically balance", 2024-05-07T02:43:06.296 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-07T02:43:06.296 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.297 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.297 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.297 INFO:teuthology.orchestra.run.smithi019.stdout: "end_weekday": { 2024-05-07T02:43:06.297 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_weekday", 2024-05-07T02:43:06.297 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-07T02:43:06.297 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.297 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.297 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.297 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-05-07T02:43:06.298 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-05-07T02:43:06.298 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.298 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-07T02:43:06.298 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-07T02:43:06.298 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.298 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.298 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.298 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.298 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.299 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.299 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.299 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.299 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.299 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.299 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.299 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.299 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.299 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.300 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.300 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.300 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.300 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.300 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.300 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.300 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.300 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.300 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.301 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.301 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.301 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.301 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.301 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.301 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.301 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.301 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.301 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.301 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.302 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.302 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.302 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.302 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.302 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.302 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.302 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.302 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.303 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.303 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.303 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.303 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.303 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.303 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.303 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.303 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.303 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.304 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.304 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.304 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.304 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.304 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.304 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.304 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.304 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.304 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.305 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.305 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.305 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.305 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.305 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.305 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.305 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.305 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.305 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.306 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.306 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.306 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.306 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.306 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.306 INFO:teuthology.orchestra.run.smithi019.stdout: "min_score": { 2024-05-07T02:43:06.306 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_score", 2024-05-07T02:43:06.306 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-07T02:43:06.306 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.307 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.307 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.307 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.307 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.307 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.307 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-07T02:43:06.307 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.307 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.307 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.308 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.308 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-05-07T02:43:06.308 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-05-07T02:43:06.308 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.308 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.308 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.308 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "upmap", 2024-05-07T02:43:06.308 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.308 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.308 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.309 INFO:teuthology.orchestra.run.smithi019.stdout: "crush-compat", 2024-05-07T02:43:06.309 INFO:teuthology.orchestra.run.smithi019.stdout: "none", 2024-05-07T02:43:06.309 INFO:teuthology.orchestra.run.smithi019.stdout: "read", 2024-05-07T02:43:06.309 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap", 2024-05-07T02:43:06.309 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap-read" 2024-05-07T02:43:06.309 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.309 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Balancer mode", 2024-05-07T02:43:06.309 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.309 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.310 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.310 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.310 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_ids": { 2024-05-07T02:43:06.310 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_ids", 2024-05-07T02:43:06.310 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.310 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.310 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.310 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.310 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.311 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.311 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.311 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-07T02:43:06.311 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.311 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.311 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.311 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.311 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-07T02:43:06.311 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-07T02:43:06.312 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:06.312 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.312 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.312 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-07T02:43:06.312 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.312 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.312 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.312 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-07T02:43:06.312 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.313 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.313 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.313 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.313 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.313 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.313 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.313 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.313 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.313 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.314 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.314 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.314 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.314 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.314 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.314 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.314 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.314 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.315 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_deviation": { 2024-05-07T02:43:06.315 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_deviation", 2024-05-07T02:43:06.315 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.315 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.315 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.315 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-07T02:43:06.315 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-07T02:43:06.315 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.315 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.316 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-07T02:43:06.316 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-07T02:43:06.316 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.316 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.316 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.316 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_optimizations": { 2024-05-07T02:43:06.316 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_optimizations", 2024-05-07T02:43:06.316 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-07T02:43:06.316 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.316 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.317 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-07T02:43:06.317 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.317 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.317 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.317 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-07T02:43:06.317 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.317 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.317 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.317 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.318 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.318 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.318 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.318 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm", 2024-05-07T02:43:06.318 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.318 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.318 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.318 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_down_multiplier": { 2024-05-07T02:43:06.319 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_down_multiplier", 2024-05-07T02:43:06.319 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-07T02:43:06.319 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.319 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.319 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-05-07T02:43:06.319 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.319 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.319 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.319 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-07T02:43:06.320 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.320 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.320 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.320 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.320 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_refresh_rate": { 2024-05-07T02:43:06.320 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_refresh_rate", 2024-05-07T02:43:06.320 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:06.320 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.320 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.321 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "20", 2024-05-07T02:43:06.321 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.321 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.321 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.321 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-07T02:43:06.321 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.321 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.321 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.321 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.321 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_starting_port": { 2024-05-07T02:43:06.322 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_starting_port", 2024-05-07T02:43:06.322 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.322 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.322 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.322 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4721", 2024-05-07T02:43:06.322 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.322 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.322 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.323 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-07T02:43:06.323 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.323 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.323 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.323 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.323 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_ptrace": { 2024-05-07T02:43:06.323 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_ptrace", 2024-05-07T02:43:06.323 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.323 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.324 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.324 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.324 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.324 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.324 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.324 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-07T02:43:06.324 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-07T02:43:06.324 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.324 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.325 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.325 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_interval": { 2024-05-07T02:43:06.325 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_interval", 2024-05-07T02:43:06.325 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:06.325 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.325 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.325 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-07T02:43:06.325 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.325 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.326 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.326 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-07T02:43:06.326 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.326 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.326 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.326 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.326 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_memory_target_ratio": { 2024-05-07T02:43:06.326 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_memory_target_ratio", 2024-05-07T02:43:06.327 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-07T02:43:06.327 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.327 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.327 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.7", 2024-05-07T02:43:06.327 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.327 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.327 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.327 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-07T02:43:06.328 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.328 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.328 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.328 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.328 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm_log_destination": { 2024-05-07T02:43:06.328 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm_log_destination", 2024-05-07T02:43:06.328 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.328 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.328 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.329 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.329 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.329 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.329 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.329 INFO:teuthology.orchestra.run.smithi019.stdout: "file", 2024-05-07T02:43:06.329 INFO:teuthology.orchestra.run.smithi019.stdout: "file,syslog", 2024-05-07T02:43:06.329 INFO:teuthology.orchestra.run.smithi019.stdout: "syslog" 2024-05-07T02:43:06.329 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.329 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-07T02:43:06.330 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.330 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.330 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.330 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.330 INFO:teuthology.orchestra.run.smithi019.stdout: "cgroups_split": { 2024-05-07T02:43:06.330 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cgroups_split", 2024-05-07T02:43:06.330 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.330 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.330 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.331 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.331 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.331 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.331 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.331 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-07T02:43:06.331 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.331 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.331 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.331 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.332 INFO:teuthology.orchestra.run.smithi019.stdout: "config_checks_enabled": { 2024-05-07T02:43:06.332 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_checks_enabled", 2024-05-07T02:43:06.332 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.332 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.332 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.332 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.332 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.332 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.332 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.333 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-07T02:43:06.333 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.333 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.333 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.333 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.333 INFO:teuthology.orchestra.run.smithi019.stdout: "config_dashboard": { 2024-05-07T02:43:06.333 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_dashboard", 2024-05-07T02:43:06.333 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.333 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.333 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.334 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.334 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.334 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.334 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.334 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-07T02:43:06.334 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.334 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.334 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.334 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.335 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_alertmanager": { 2024-05-07T02:43:06.335 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_alertmanager", 2024-05-07T02:43:06.335 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.335 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.335 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.335 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-07T02:43:06.335 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.335 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.335 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.336 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-07T02:43:06.336 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.336 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.336 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.336 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.336 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_base": { 2024-05-07T02:43:06.336 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_base", 2024-05-07T02:43:06.336 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.336 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.337 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.337 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-07T02:43:06.337 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.337 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.337 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.337 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Container image name, without the tag", 2024-05-07T02:43:06.337 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.337 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.337 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.337 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.338 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_elasticsearch": { 2024-05-07T02:43:06.338 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_elasticsearch", 2024-05-07T02:43:06.338 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.338 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.338 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.338 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-07T02:43:06.338 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.338 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.342 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.342 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "elasticsearch container image", 2024-05-07T02:43:06.342 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.342 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.343 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.343 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.343 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_grafana": { 2024-05-07T02:43:06.343 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_grafana", 2024-05-07T02:43:06.343 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.343 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.343 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.343 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-07T02:43:06.343 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.344 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.344 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.344 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-07T02:43:06.344 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.344 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.344 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.344 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.344 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_haproxy": { 2024-05-07T02:43:06.344 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_haproxy", 2024-05-07T02:43:06.344 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.345 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.345 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.345 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-07T02:43:06.345 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.345 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.345 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.345 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "HAproxy container image", 2024-05-07T02:43:06.345 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.345 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.346 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.346 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.346 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_agent": { 2024-05-07T02:43:06.346 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_agent", 2024-05-07T02:43:06.346 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.346 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.346 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.346 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-07T02:43:06.346 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.347 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.347 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.347 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger agent container image", 2024-05-07T02:43:06.347 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.347 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.347 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.347 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.347 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_collector": { 2024-05-07T02:43:06.347 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_collector", 2024-05-07T02:43:06.348 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.348 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.348 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.348 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-07T02:43:06.348 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.348 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.348 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.348 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger collector container image", 2024-05-07T02:43:06.348 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.348 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.349 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.349 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.349 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_query": { 2024-05-07T02:43:06.349 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_query", 2024-05-07T02:43:06.349 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.349 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.349 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.349 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-07T02:43:06.349 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.350 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.350 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.350 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger query container image", 2024-05-07T02:43:06.350 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.350 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.350 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.350 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.350 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_keepalived": { 2024-05-07T02:43:06.350 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_keepalived", 2024-05-07T02:43:06.351 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.351 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.351 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.351 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-07T02:43:06.351 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.351 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.351 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.351 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Keepalived container image", 2024-05-07T02:43:06.351 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.351 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.352 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.352 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.352 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_loki": { 2024-05-07T02:43:06.352 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_loki", 2024-05-07T02:43:06.352 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.352 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.352 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.352 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-07T02:43:06.352 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.353 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.353 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.353 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Loki container image", 2024-05-07T02:43:06.353 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.353 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.353 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.353 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.353 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_node_exporter": { 2024-05-07T02:43:06.353 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_node_exporter", 2024-05-07T02:43:06.353 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.354 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.354 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.354 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-07T02:43:06.354 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.354 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.354 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.354 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-07T02:43:06.354 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.354 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.355 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.355 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.355 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_nvmeof": { 2024-05-07T02:43:06.355 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_nvmeof", 2024-05-07T02:43:06.355 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.355 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.355 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.355 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-07T02:43:06.356 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.356 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.356 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.356 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Nvme-of container image", 2024-05-07T02:43:06.356 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.356 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.356 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.356 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.356 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_prometheus": { 2024-05-07T02:43:06.356 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_prometheus", 2024-05-07T02:43:06.357 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.357 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.357 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.357 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-07T02:43:06.357 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.357 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.357 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.357 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-05-07T02:43:06.358 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.358 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.358 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.358 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.358 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_promtail": { 2024-05-07T02:43:06.358 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_promtail", 2024-05-07T02:43:06.358 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.358 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.358 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.358 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-07T02:43:06.359 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.359 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.359 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.359 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Promtail container image", 2024-05-07T02:43:06.359 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.359 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.359 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.359 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.359 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_samba": { 2024-05-07T02:43:06.360 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_samba", 2024-05-07T02:43:06.360 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.360 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.360 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.360 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-07T02:43:06.360 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.360 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.360 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.360 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Samba/SMB container image", 2024-05-07T02:43:06.361 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.361 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.361 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.361 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.361 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_snmp_gateway": { 2024-05-07T02:43:06.361 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_snmp_gateway", 2024-05-07T02:43:06.361 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.361 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.361 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.362 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-07T02:43:06.362 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.362 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.362 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.362 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SNMP Gateway container image", 2024-05-07T02:43:06.362 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.362 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.362 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.362 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.363 INFO:teuthology.orchestra.run.smithi019.stdout: "container_init": { 2024-05-07T02:43:06.363 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_init", 2024-05-07T02:43:06.363 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.363 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.363 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.363 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.363 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.363 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.363 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.364 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Run podman/docker with `--init`", 2024-05-07T02:43:06.364 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.364 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.364 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.364 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.364 INFO:teuthology.orchestra.run.smithi019.stdout: "daemon_cache_timeout": { 2024-05-07T02:43:06.364 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "daemon_cache_timeout", 2024-05-07T02:43:06.364 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:06.364 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.365 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.365 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-07T02:43:06.365 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.365 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.365 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.365 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-07T02:43:06.365 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.365 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.365 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.366 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.366 INFO:teuthology.orchestra.run.smithi019.stdout: "default_cephadm_command_timeout": { 2024-05-07T02:43:06.366 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_cephadm_command_timeout", 2024-05-07T02:43:06.366 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.366 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.366 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.366 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "900", 2024-05-07T02:43:06.366 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.367 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.367 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.367 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-07T02:43:06.367 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.367 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.367 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.367 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.367 INFO:teuthology.orchestra.run.smithi019.stdout: "default_registry": { 2024-05-07T02:43:06.367 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_registry", 2024-05-07T02:43:06.368 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.368 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.368 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.368 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io", 2024-05-07T02:43:06.368 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.368 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.368 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.368 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-07T02:43:06.368 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.369 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.369 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.369 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.369 INFO:teuthology.orchestra.run.smithi019.stdout: "device_cache_timeout": { 2024-05-07T02:43:06.369 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_cache_timeout", 2024-05-07T02:43:06.369 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:06.369 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.369 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.369 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1800", 2024-05-07T02:43:06.370 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.370 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.370 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.370 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache device inventory", 2024-05-07T02:43:06.370 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.370 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.370 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.370 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.371 INFO:teuthology.orchestra.run.smithi019.stdout: "device_enhanced_scan": { 2024-05-07T02:43:06.371 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_enhanced_scan", 2024-05-07T02:43:06.371 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.371 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.371 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.371 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.371 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.371 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.371 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.372 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-07T02:43:06.372 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.372 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.372 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.372 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.372 INFO:teuthology.orchestra.run.smithi019.stdout: "facts_cache_timeout": { 2024-05-07T02:43:06.372 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "facts_cache_timeout", 2024-05-07T02:43:06.372 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:06.372 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.373 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.373 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-07T02:43:06.373 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.373 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.373 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.373 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache host facts data", 2024-05-07T02:43:06.373 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.373 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.373 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.374 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.374 INFO:teuthology.orchestra.run.smithi019.stdout: "grafana_dashboards_path": { 2024-05-07T02:43:06.374 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "grafana_dashboards_path", 2024-05-07T02:43:06.374 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.374 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.374 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.374 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-07T02:43:06.374 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.374 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.375 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.375 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-07T02:43:06.375 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.375 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.375 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.375 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.375 INFO:teuthology.orchestra.run.smithi019.stdout: "host_check_interval": { 2024-05-07T02:43:06.375 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "host_check_interval", 2024-05-07T02:43:06.375 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:06.376 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.376 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.376 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-07T02:43:06.376 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.376 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.376 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.376 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to perform a host check", 2024-05-07T02:43:06.376 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.376 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.377 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.377 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.377 INFO:teuthology.orchestra.run.smithi019.stdout: "hw_monitoring": { 2024-05-07T02:43:06.377 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hw_monitoring", 2024-05-07T02:43:06.377 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.377 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.377 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.377 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.377 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.378 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.378 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.378 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-07T02:43:06.378 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.378 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.378 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.378 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.378 INFO:teuthology.orchestra.run.smithi019.stdout: "inventory_list_all": { 2024-05-07T02:43:06.378 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "inventory_list_all", 2024-05-07T02:43:06.379 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.379 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.379 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.379 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.379 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.379 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.379 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.379 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-07T02:43:06.379 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.380 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.380 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.380 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.380 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.380 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.380 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.380 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.380 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.380 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.381 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.381 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.381 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.381 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.381 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.381 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.381 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.381 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.381 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.382 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.382 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.382 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.382 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.382 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.382 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.382 INFO:teuthology.orchestra.run.smithi019.stdout: "log_refresh_metadata": { 2024-05-07T02:43:06.382 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_refresh_metadata", 2024-05-07T02:43:06.382 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.383 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.383 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.383 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.383 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.383 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.383 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.383 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-07T02:43:06.383 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.383 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.384 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.384 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.384 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.384 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.384 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.384 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.384 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.384 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.384 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.385 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.385 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.385 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-07T02:43:06.385 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.385 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.385 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.385 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.385 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.385 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.386 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.386 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.386 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.386 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.386 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.386 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.386 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.386 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.386 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.387 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.387 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.387 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.387 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.387 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.387 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.387 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.387 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.388 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.388 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.388 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.388 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.388 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.388 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.388 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.388 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.388 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.389 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.389 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.389 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.389 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.389 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.389 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.389 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.389 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-07T02:43:06.389 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-07T02:43:06.389 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.390 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.390 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.390 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.390 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.390 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.390 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.390 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-07T02:43:06.390 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.391 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.391 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.391 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.391 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-07T02:43:06.391 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-07T02:43:06.391 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.391 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.391 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.391 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "*", 2024-05-07T02:43:06.391 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.392 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.392 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.392 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-07T02:43:06.392 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.392 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.392 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.392 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.392 INFO:teuthology.orchestra.run.smithi019.stdout: "max_count_per_host": { 2024-05-07T02:43:06.392 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_count_per_host", 2024-05-07T02:43:06.393 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.393 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.393 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.393 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-07T02:43:06.393 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.393 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.393 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.393 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of daemons per service per host", 2024-05-07T02:43:06.393 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.394 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.394 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.394 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.394 INFO:teuthology.orchestra.run.smithi019.stdout: "max_osd_draining_count": { 2024-05-07T02:43:06.394 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_osd_draining_count", 2024-05-07T02:43:06.394 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.395 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.395 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.395 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-07T02:43:06.395 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.395 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.395 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.395 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-07T02:43:06.395 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.395 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.396 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.396 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.396 INFO:teuthology.orchestra.run.smithi019.stdout: "migration_current": { 2024-05-07T02:43:06.396 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "migration_current", 2024-05-07T02:43:06.396 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.396 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.396 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.396 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.396 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.397 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.397 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.397 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "internal - do not modify", 2024-05-07T02:43:06.397 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.397 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.397 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.397 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.397 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-05-07T02:43:06.397 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-05-07T02:43:06.398 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.398 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.398 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.398 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "root", 2024-05-07T02:43:06.398 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.398 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.398 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.398 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm-package", 2024-05-07T02:43:06.398 INFO:teuthology.orchestra.run.smithi019.stdout: "root" 2024-05-07T02:43:06.399 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.399 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "mode for remote execution of cephadm", 2024-05-07T02:43:06.399 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.399 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.399 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.399 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.399 INFO:teuthology.orchestra.run.smithi019.stdout: "oob_default_addr": { 2024-05-07T02:43:06.399 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "oob_default_addr", 2024-05-07T02:43:06.399 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.400 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.400 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.400 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "169.254.1.1", 2024-05-07T02:43:06.400 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.400 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.400 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.400 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-07T02:43:06.400 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.401 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.401 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.401 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.401 INFO:teuthology.orchestra.run.smithi019.stdout: "prometheus_alerts_path": { 2024-05-07T02:43:06.401 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus_alerts_path", 2024-05-07T02:43:06.401 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.401 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.401 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.401 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-07T02:43:06.401 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.402 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.402 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.402 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-07T02:43:06.402 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.402 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.402 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.402 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.402 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_insecure": { 2024-05-07T02:43:06.403 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_insecure", 2024-05-07T02:43:06.403 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.403 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.403 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.403 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.403 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.403 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.403 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.403 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-07T02:43:06.404 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.404 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.404 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.404 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.404 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_password": { 2024-05-07T02:43:06.404 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_password", 2024-05-07T02:43:06.404 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.404 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.404 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.405 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.405 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.405 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.405 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.405 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-07T02:43:06.405 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.405 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.405 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.405 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.406 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_url": { 2024-05-07T02:43:06.406 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_url", 2024-05-07T02:43:06.406 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.406 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.406 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.406 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.406 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.406 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.406 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.407 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-07T02:43:06.407 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.407 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.407 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.407 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.407 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_username": { 2024-05-07T02:43:06.407 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_username", 2024-05-07T02:43:06.407 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.407 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.408 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.408 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.408 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.408 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.408 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.408 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-07T02:43:06.408 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.408 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.408 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.409 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.409 INFO:teuthology.orchestra.run.smithi019.stdout: "secure_monitoring_stack": { 2024-05-07T02:43:06.409 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "secure_monitoring_stack", 2024-05-07T02:43:06.409 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.409 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.409 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.409 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.409 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.409 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.410 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.410 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-07T02:43:06.410 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.410 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.410 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.410 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.410 INFO:teuthology.orchestra.run.smithi019.stdout: "service_discovery_port": { 2024-05-07T02:43:06.410 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "service_discovery_port", 2024-05-07T02:43:06.411 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.411 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.411 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.411 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8765", 2024-05-07T02:43:06.411 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.411 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.411 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.411 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "cephadm service discovery port", 2024-05-07T02:43:06.411 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.412 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.412 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.412 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.412 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.412 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.412 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.412 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.412 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.412 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.413 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.413 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.413 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.413 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.413 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.413 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.413 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.413 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.413 INFO:teuthology.orchestra.run.smithi019.stdout: "ssh_config_file": { 2024-05-07T02:43:06.413 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssh_config_file", 2024-05-07T02:43:06.414 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.414 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.414 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.414 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.414 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.414 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.414 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.414 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-07T02:43:06.415 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.415 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.415 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.415 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.415 INFO:teuthology.orchestra.run.smithi019.stdout: "use_agent": { 2024-05-07T02:43:06.415 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_agent", 2024-05-07T02:43:06.415 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.415 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.415 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.415 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.416 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.416 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.416 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.416 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-07T02:43:06.416 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.416 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.416 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.416 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.416 INFO:teuthology.orchestra.run.smithi019.stdout: "use_repo_digest": { 2024-05-07T02:43:06.417 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_repo_digest", 2024-05-07T02:43:06.417 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.417 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.417 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.417 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.417 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.417 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.417 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.417 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-07T02:43:06.418 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.418 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.418 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.418 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.418 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_failed_host_check": { 2024-05-07T02:43:06.418 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_failed_host_check", 2024-05-07T02:43:06.418 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.418 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.418 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.419 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.419 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.419 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.419 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.419 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if the host check fails", 2024-05-07T02:43:06.419 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.419 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.419 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.419 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.420 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_daemons": { 2024-05-07T02:43:06.420 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_daemons", 2024-05-07T02:43:06.420 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.420 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.420 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.420 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.420 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.420 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.420 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.420 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-07T02:43:06.421 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.421 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.421 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.421 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.421 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_hosts": { 2024-05-07T02:43:06.421 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_hosts", 2024-05-07T02:43:06.421 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.421 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.421 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.422 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.422 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.422 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.422 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.422 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-07T02:43:06.422 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.422 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.422 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.423 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.423 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.423 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.423 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.423 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crash", 2024-05-07T02:43:06.423 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.423 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.423 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.423 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.423 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.424 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.424 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.424 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.424 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.424 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.424 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.424 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.424 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.424 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.424 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.425 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.425 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.425 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.425 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.425 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.425 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.425 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.425 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.425 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.426 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.426 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.426 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.426 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.426 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.426 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.426 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.426 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.427 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.427 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.427 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.427 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.427 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.427 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.427 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.427 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.427 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.427 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.428 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.428 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.428 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.428 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.428 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.428 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.428 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.428 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.428 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.428 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.429 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.429 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.429 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.429 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.429 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.429 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.429 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.429 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.429 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.430 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.430 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.430 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.430 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.430 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.430 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.430 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.430 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.431 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.431 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.431 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.431 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.431 INFO:teuthology.orchestra.run.smithi019.stdout: "retain_interval": { 2024-05-07T02:43:06.431 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retain_interval", 2024-05-07T02:43:06.431 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:06.431 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.431 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.431 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "31536000", 2024-05-07T02:43:06.432 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.432 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.432 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.432 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-07T02:43:06.432 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.432 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.432 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.432 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.432 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.433 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.433 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.433 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.433 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.433 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.433 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.433 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.433 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.433 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.433 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.434 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.434 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.434 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.434 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_recent_interval": { 2024-05-07T02:43:06.434 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_recent_interval", 2024-05-07T02:43:06.434 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:06.434 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.434 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.434 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1209600", 2024-05-07T02:43:06.435 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.435 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.435 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.435 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-07T02:43:06.435 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.435 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.435 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.435 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.435 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.436 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.436 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.436 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dashboard", 2024-05-07T02:43:06.436 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.436 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.436 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.436 INFO:teuthology.orchestra.run.smithi019.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-07T02:43:06.436 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-07T02:43:06.436 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.437 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.437 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.437 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-07T02:43:06.437 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.437 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.437 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.437 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.437 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.437 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.437 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.438 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.438 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_HOST": { 2024-05-07T02:43:06.438 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-07T02:43:06.438 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.438 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.438 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.438 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.438 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.438 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.439 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.439 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.439 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.439 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.439 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.439 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.439 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-07T02:43:06.439 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-07T02:43:06.439 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.440 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.440 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.440 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.440 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.440 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.440 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.440 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.440 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.440 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.440 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.441 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.441 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_ENABLED": { 2024-05-07T02:43:06.441 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_ENABLED", 2024-05-07T02:43:06.441 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.441 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.441 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.441 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.441 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.441 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.442 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.442 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.442 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.442 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.442 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.442 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.442 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-07T02:43:06.442 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-07T02:43:06.442 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.443 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.443 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.443 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.443 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.443 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.443 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.443 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.443 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.443 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.444 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.444 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.444 INFO:teuthology.orchestra.run.smithi019.stdout: "ENABLE_BROWSABLE_API": { 2024-05-07T02:43:06.444 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-07T02:43:06.444 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.444 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.444 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.444 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.444 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.445 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.445 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.445 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.445 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.445 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.445 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.445 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.445 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-07T02:43:06.445 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-07T02:43:06.446 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.446 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.446 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.446 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.446 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.446 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.446 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.446 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.446 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.447 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.447 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.447 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.447 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-07T02:43:06.447 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-07T02:43:06.447 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.447 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.447 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.447 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.448 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.448 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.448 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.448 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.448 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.448 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.448 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.448 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.448 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-07T02:43:06.449 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-07T02:43:06.449 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.449 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.449 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.449 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.449 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.449 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.449 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.449 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.449 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.450 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.450 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.450 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.450 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-07T02:43:06.450 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-07T02:43:06.450 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.450 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.450 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.450 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.451 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.451 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.451 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.451 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.451 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.451 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.451 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.451 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.451 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-07T02:43:06.452 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-07T02:43:06.452 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.452 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.452 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.452 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.452 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.452 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.452 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.452 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.453 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.453 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.453 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.453 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.453 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-07T02:43:06.453 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-07T02:43:06.453 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.453 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.453 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.454 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.454 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.454 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.454 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.454 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.454 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.454 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.454 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.454 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.455 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-07T02:43:06.455 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-07T02:43:06.455 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.455 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.455 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.455 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.455 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.455 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.455 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.455 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.456 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.456 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.456 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.456 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.456 INFO:teuthology.orchestra.run.smithi019.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-07T02:43:06.456 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-07T02:43:06.456 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.456 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.456 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.457 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.457 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.457 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.457 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.457 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.457 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.457 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.457 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.457 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.458 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_PASSWORD": { 2024-05-07T02:43:06.458 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-07T02:43:06.458 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.458 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.458 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.458 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-05-07T02:43:06.458 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.458 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.458 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.459 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.459 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.459 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.459 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.459 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.459 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-07T02:43:06.459 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-07T02:43:06.459 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.459 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.460 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.460 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.460 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.460 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.460 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.460 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.460 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.460 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.460 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.461 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.461 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_URL": { 2024-05-07T02:43:06.461 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_URL", 2024-05-07T02:43:06.461 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.461 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.461 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.461 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.461 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.461 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.462 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.462 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.462 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.462 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.462 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.462 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.462 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_USERNAME": { 2024-05-07T02:43:06.462 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-07T02:43:06.462 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.463 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.463 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.463 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-05-07T02:43:06.463 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.463 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.463 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.463 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.463 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.463 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.464 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.464 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.464 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-07T02:43:06.464 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-07T02:43:06.464 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.464 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.464 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.464 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.464 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.465 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.465 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.465 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.465 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.465 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.465 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.465 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.465 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-07T02:43:06.465 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-07T02:43:06.466 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.466 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.466 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.466 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.466 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.466 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.466 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.466 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.467 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.467 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.467 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.467 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.467 INFO:teuthology.orchestra.run.smithi019.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-07T02:43:06.467 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-07T02:43:06.467 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.467 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.467 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.468 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.468 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.468 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.468 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.468 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.468 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.468 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.468 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.468 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.469 INFO:teuthology.orchestra.run.smithi019.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-07T02:43:06.469 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-07T02:43:06.469 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.469 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.469 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.469 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.469 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.469 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.469 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.470 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.470 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.470 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.470 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.470 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.470 INFO:teuthology.orchestra.run.smithi019.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-07T02:43:06.470 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-07T02:43:06.470 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.470 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.471 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.471 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "[]", 2024-05-07T02:43:06.471 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.471 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.471 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.471 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.471 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.471 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.471 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.472 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.472 INFO:teuthology.orchestra.run.smithi019.stdout: "MULTICLUSTER_CONFIG": { 2024-05-07T02:43:06.472 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-07T02:43:06.472 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.472 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.472 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.472 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "{}", 2024-05-07T02:43:06.472 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.472 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.473 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.473 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.473 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.473 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.473 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.473 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.473 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_HOST": { 2024-05-07T02:43:06.473 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-07T02:43:06.473 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.474 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.474 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.474 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.474 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.474 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.474 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.474 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.474 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.475 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.475 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.475 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.475 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-07T02:43:06.475 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-07T02:43:06.475 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.475 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.475 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.475 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.475 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.476 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.476 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.476 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.476 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.476 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.476 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.476 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.476 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-07T02:43:06.476 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-07T02:43:06.477 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.477 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.477 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.477 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.477 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.477 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.477 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.477 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.477 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.478 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.478 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.478 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.478 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-07T02:43:06.478 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-07T02:43:06.478 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.478 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.478 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.478 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.479 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.479 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.479 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.479 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.479 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.479 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.479 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.479 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.479 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-07T02:43:06.480 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-07T02:43:06.480 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.480 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.480 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.480 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.480 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.480 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.480 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.480 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.481 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.481 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.481 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.481 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.481 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-07T02:43:06.481 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-07T02:43:06.481 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.481 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.481 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.482 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.482 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.482 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.482 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.482 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.482 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.482 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.482 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.482 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.483 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-07T02:43:06.483 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-07T02:43:06.483 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.483 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.483 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.483 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.483 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.483 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.483 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.484 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.484 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.484 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.484 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.484 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.484 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-07T02:43:06.484 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-07T02:43:06.484 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.484 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.485 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.485 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.485 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.485 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.485 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.485 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.485 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.485 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.485 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.485 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.486 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-07T02:43:06.486 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-07T02:43:06.486 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.486 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.486 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.486 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.486 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.486 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.487 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.487 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.487 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.487 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.487 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.487 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.487 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_ENABLED": { 2024-05-07T02:43:06.487 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-07T02:43:06.487 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.488 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.488 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.488 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.488 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.488 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.488 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.488 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.488 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.488 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.489 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.489 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.489 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-07T02:43:06.489 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-07T02:43:06.489 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.489 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.489 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.489 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-07T02:43:06.489 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.489 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.490 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.490 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.490 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.490 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.490 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.490 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.490 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-07T02:43:06.490 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-07T02:43:06.491 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.491 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.491 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.491 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-07T02:43:06.491 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.491 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.491 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.491 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.491 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.491 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.492 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.492 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.492 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-07T02:43:06.492 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-07T02:43:06.492 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.492 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.492 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.492 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8", 2024-05-07T02:43:06.493 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.493 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.493 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.493 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.493 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.493 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.493 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.493 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.493 INFO:teuthology.orchestra.run.smithi019.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-07T02:43:06.493 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-07T02:43:06.494 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.494 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.494 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.494 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "45", 2024-05-07T02:43:06.494 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.494 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.494 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.494 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.495 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.495 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.495 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.495 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.495 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ACCESS_KEY": { 2024-05-07T02:43:06.495 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-07T02:43:06.495 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.495 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.495 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.496 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.496 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.496 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.496 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.496 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.496 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.496 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.496 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.496 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.497 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-07T02:43:06.497 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-07T02:43:06.497 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.497 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.497 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.497 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-05-07T02:43:06.497 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.497 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.497 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.498 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.498 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.498 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.498 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.498 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.498 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SECRET_KEY": { 2024-05-07T02:43:06.498 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-07T02:43:06.498 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.498 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.499 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.499 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.499 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.499 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.499 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.499 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.499 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.499 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.499 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.500 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.500 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SSL_VERIFY": { 2024-05-07T02:43:06.500 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-07T02:43:06.500 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.500 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.500 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.500 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.500 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.500 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.500 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.501 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.501 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.501 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.501 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.501 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.501 INFO:teuthology.orchestra.run.smithi019.stdout: "UNSAFE_TLS_v1_2": { 2024-05-07T02:43:06.501 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-07T02:43:06.501 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.501 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.502 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.502 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.502 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.502 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.502 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.502 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.502 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.502 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.502 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.503 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.503 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-07T02:43:06.503 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-07T02:43:06.503 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.503 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.503 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.503 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.503 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.503 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.504 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.504 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.504 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.504 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.504 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.504 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.504 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-07T02:43:06.504 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-07T02:43:06.504 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.504 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.505 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.505 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-07T02:43:06.505 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.505 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.505 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.505 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.505 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.506 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.506 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.506 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.506 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-07T02:43:06.507 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-07T02:43:06.507 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.507 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.507 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.507 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-07T02:43:06.507 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.508 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.508 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.508 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.508 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.508 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.509 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.509 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.509 INFO:teuthology.orchestra.run.smithi019.stdout: "cross_origin_url": { 2024-05-07T02:43:06.509 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cross_origin_url", 2024-05-07T02:43:06.509 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.509 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.510 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.510 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.510 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.510 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.510 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.511 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.511 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.511 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.511 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.511 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.512 INFO:teuthology.orchestra.run.smithi019.stdout: "crt_file": { 2024-05-07T02:43:06.512 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crt_file", 2024-05-07T02:43:06.512 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.512 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.512 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.513 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.513 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.513 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.513 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.513 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.514 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.514 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.514 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.514 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.514 INFO:teuthology.orchestra.run.smithi019.stdout: "debug": { 2024-05-07T02:43:06.514 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "debug", 2024-05-07T02:43:06.514 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.514 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.515 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.515 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.515 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.515 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.515 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.515 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable/disable debug options", 2024-05-07T02:43:06.515 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.515 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.515 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.515 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.516 INFO:teuthology.orchestra.run.smithi019.stdout: "jwt_token_ttl": { 2024-05-07T02:43:06.516 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "jwt_token_ttl", 2024-05-07T02:43:06.516 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.516 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.516 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.516 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "28800", 2024-05-07T02:43:06.516 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.516 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.516 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.516 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.517 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.517 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.517 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.517 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.517 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-05-07T02:43:06.517 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-05-07T02:43:06.517 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.517 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.517 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.517 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.518 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.518 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.518 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.518 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.518 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.518 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.518 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.518 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.518 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.519 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.519 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.519 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.519 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.519 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.519 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.519 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.519 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.519 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.520 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.520 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.520 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.520 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.520 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.520 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.520 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.520 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.520 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.521 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.521 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.521 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.521 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.521 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.521 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.521 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.521 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.521 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.522 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.522 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.522 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.522 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.522 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.522 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.522 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.522 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.522 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.523 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.523 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.523 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.523 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.523 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.523 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.523 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.523 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.523 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.523 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.524 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.524 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.524 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.524 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.524 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.524 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.524 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.524 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.524 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.525 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.525 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.525 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.525 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.525 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.525 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.525 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.525 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.525 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.526 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.526 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.526 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.526 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.526 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.526 INFO:teuthology.orchestra.run.smithi019.stdout: "motd": { 2024-05-07T02:43:06.526 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "motd", 2024-05-07T02:43:06.526 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.526 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.527 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.527 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.527 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.527 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.527 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.527 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "The message of the day", 2024-05-07T02:43:06.527 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.527 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.527 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.528 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.528 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect_resolve_ip_addr": { 2024-05-07T02:43:06.528 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "redirect_resolve_ip_addr", 2024-05-07T02:43:06.528 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.528 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.528 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.528 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.528 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.528 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.529 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.529 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.529 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.529 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.529 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.529 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.529 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-05-07T02:43:06.529 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-05-07T02:43:06.529 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.529 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.530 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.530 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-05-07T02:43:06.530 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.530 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.530 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.530 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.530 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.530 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.530 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.531 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.531 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-05-07T02:43:06.531 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-05-07T02:43:06.531 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.531 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.531 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.531 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8080", 2024-05-07T02:43:06.531 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.531 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.532 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.532 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.532 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.532 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.532 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.532 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.532 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.532 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.532 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.533 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.533 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.533 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.533 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.533 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.533 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.533 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.533 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.533 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.533 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.534 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.534 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-05-07T02:43:06.534 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-05-07T02:43:06.534 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.534 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.534 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.534 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.534 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.535 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.535 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.535 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.535 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.535 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.535 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.535 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.535 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl_server_port": { 2024-05-07T02:43:06.535 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl_server_port", 2024-05-07T02:43:06.535 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.536 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.536 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.536 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8443", 2024-05-07T02:43:06.536 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.536 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.536 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.536 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.536 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.536 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.537 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.537 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.537 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-05-07T02:43:06.537 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-05-07T02:43:06.537 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.537 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.537 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.537 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "redirect", 2024-05-07T02:43:06.537 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.537 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.538 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.538 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.538 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect" 2024-05-07T02:43:06.538 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.538 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.538 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.538 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.538 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.539 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.539 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-05-07T02:43:06.539 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-05-07T02:43:06.539 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.539 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.539 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.539 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-05-07T02:43:06.539 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-05-07T02:43:06.539 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-05-07T02:43:06.540 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.540 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.540 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.540 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.540 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.540 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.540 INFO:teuthology.orchestra.run.smithi019.stdout: "url_prefix": { 2024-05-07T02:43:06.540 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url_prefix", 2024-05-07T02:43:06.540 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.540 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.541 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.541 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.541 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.541 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.541 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.541 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.541 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.541 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.541 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.542 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.542 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.542 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.542 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.542 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "devicehealth", 2024-05-07T02:43:06.542 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.542 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.542 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.542 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_monitoring": { 2024-05-07T02:43:06.543 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_monitoring", 2024-05-07T02:43:06.543 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.543 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.543 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.543 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.543 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.543 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.543 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.543 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "monitor device health metrics", 2024-05-07T02:43:06.544 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.544 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.544 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.544 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.544 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.544 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.544 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.544 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.544 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.545 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.545 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.545 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.545 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.545 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.545 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.545 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.545 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.545 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.546 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.546 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.546 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.546 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.546 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.546 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.546 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.546 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.546 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.547 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.547 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.547 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.547 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.547 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.547 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.547 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.547 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.547 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.548 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.548 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.548 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.548 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.548 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.548 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.548 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.548 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.548 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.549 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.549 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.549 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.549 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.549 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.549 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.549 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.549 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.549 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.549 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.550 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.550 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.550 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.550 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.550 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.550 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.550 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.550 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.550 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.551 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.551 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.551 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.551 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.551 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.551 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.551 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.551 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.551 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.552 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.552 INFO:teuthology.orchestra.run.smithi019.stdout: "mark_out_threshold": { 2024-05-07T02:43:06.552 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mark_out_threshold", 2024-05-07T02:43:06.552 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:06.552 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.552 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.552 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2419200", 2024-05-07T02:43:06.552 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.552 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.553 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.553 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-07T02:43:06.553 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.553 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.553 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.553 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.553 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_name": { 2024-05-07T02:43:06.553 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_name", 2024-05-07T02:43:06.553 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.553 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.554 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.554 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "device_health_metrics", 2024-05-07T02:43:06.554 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.554 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.554 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.554 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-07T02:43:06.554 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.554 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.555 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.555 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.555 INFO:teuthology.orchestra.run.smithi019.stdout: "retention_period": { 2024-05-07T02:43:06.555 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retention_period", 2024-05-07T02:43:06.555 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:06.555 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.555 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.555 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15552000", 2024-05-07T02:43:06.555 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.555 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.556 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.556 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain device health metrics", 2024-05-07T02:43:06.556 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.556 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.556 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.556 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.556 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_frequency": { 2024-05-07T02:43:06.556 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_frequency", 2024-05-07T02:43:06.556 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:06.557 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.557 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.557 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-05-07T02:43:06.557 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.557 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.557 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.557 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-07T02:43:06.557 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.557 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.557 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.558 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.558 INFO:teuthology.orchestra.run.smithi019.stdout: "self_heal": { 2024-05-07T02:43:06.558 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "self_heal", 2024-05-07T02:43:06.558 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.558 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.558 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.558 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.558 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.559 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.559 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.559 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-07T02:43:06.559 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.559 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.559 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.559 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.559 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-07T02:43:06.559 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-07T02:43:06.559 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:06.560 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.560 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.560 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-07T02:43:06.560 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.560 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.560 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.560 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and check device health", 2024-05-07T02:43:06.560 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.560 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.561 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.561 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.561 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.561 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.561 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.561 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.561 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.561 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.561 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.561 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.562 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.562 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.562 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.562 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.562 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.562 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.562 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_threshold": { 2024-05-07T02:43:06.562 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_threshold", 2024-05-07T02:43:06.562 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:06.563 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.563 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.563 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7257600", 2024-05-07T02:43:06.563 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.563 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.563 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.563 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-07T02:43:06.563 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.563 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.564 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.564 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.564 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.564 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.564 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.564 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "diskprediction_local", 2024-05-07T02:43:06.564 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.564 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.564 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.565 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.565 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.565 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.565 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.565 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.565 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.565 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.565 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.565 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.566 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.566 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.566 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.566 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.566 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.566 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.566 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.566 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.566 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.567 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.567 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.567 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.567 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.567 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.567 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.567 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.567 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.567 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.567 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.568 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.568 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.568 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.568 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.568 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.568 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.568 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.568 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.568 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.569 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.569 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.569 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.569 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.569 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.569 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.569 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.569 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.569 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.570 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.570 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.570 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.570 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.570 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.570 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.570 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.570 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.570 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.571 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.571 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.571 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.571 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.571 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.571 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.571 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.571 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.572 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.572 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.572 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.572 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.572 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.572 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.572 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.572 INFO:teuthology.orchestra.run.smithi019.stdout: "predict_interval": { 2024-05-07T02:43:06.572 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predict_interval", 2024-05-07T02:43:06.573 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.573 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.573 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.573 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-05-07T02:43:06.573 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.573 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.573 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.573 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.573 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.573 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.574 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.574 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.574 INFO:teuthology.orchestra.run.smithi019.stdout: "predictor_model": { 2024-05-07T02:43:06.574 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predictor_model", 2024-05-07T02:43:06.574 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.574 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.574 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.574 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "prophetstor", 2024-05-07T02:43:06.574 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.575 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.575 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.575 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.575 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.575 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.575 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.575 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.575 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-07T02:43:06.575 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-07T02:43:06.576 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.576 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.576 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.576 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-05-07T02:43:06.576 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.576 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.576 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.576 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.576 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.577 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.577 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.577 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.577 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.577 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.577 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.577 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.577 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.577 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.578 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.578 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.578 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.578 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.578 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.578 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.578 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.578 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.579 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.579 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.579 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.579 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "influx", 2024-05-07T02:43:06.579 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": false, 2024-05-07T02:43:06.579 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "influxdb python module not found", 2024-05-07T02:43:06.579 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.579 INFO:teuthology.orchestra.run.smithi019.stdout: "batch_size": { 2024-05-07T02:43:06.579 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "batch_size", 2024-05-07T02:43:06.579 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.580 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.580 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.580 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5000", 2024-05-07T02:43:06.580 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.580 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.580 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.580 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-07T02:43:06.580 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.580 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.581 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.581 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.581 INFO:teuthology.orchestra.run.smithi019.stdout: "database": { 2024-05-07T02:43:06.581 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "database", 2024-05-07T02:43:06.581 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.581 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.581 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.581 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "ceph", 2024-05-07T02:43:06.581 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.582 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.582 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.582 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-07T02:43:06.582 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.582 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.582 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.582 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.582 INFO:teuthology.orchestra.run.smithi019.stdout: "hostname": { 2024-05-07T02:43:06.582 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hostname", 2024-05-07T02:43:06.583 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.583 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.583 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.583 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.583 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.583 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.583 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.583 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server hostname", 2024-05-07T02:43:06.584 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.584 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.584 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.584 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.584 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-07T02:43:06.584 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-07T02:43:06.584 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:06.584 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.584 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.585 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "30", 2024-05-07T02:43:06.585 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "5", 2024-05-07T02:43:06.585 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.585 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.585 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-07T02:43:06.585 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.585 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.585 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.585 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.586 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.586 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.586 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.586 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.586 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.586 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.586 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.586 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.586 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.587 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.587 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.587 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.587 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.587 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.587 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.587 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.587 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.587 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.588 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.588 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.588 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.588 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.588 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.588 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.588 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.588 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.588 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.589 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.589 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.589 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.589 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.589 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.589 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.589 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.589 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.589 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.590 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.590 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.590 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.590 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.590 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.590 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.590 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.590 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.590 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.591 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.591 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.591 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.591 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.591 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.591 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.591 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.591 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.591 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.592 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.592 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.592 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.592 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.592 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.592 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.592 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.592 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.592 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.593 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.593 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.593 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.593 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.593 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.593 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.593 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.593 INFO:teuthology.orchestra.run.smithi019.stdout: "password": { 2024-05-07T02:43:06.593 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "password", 2024-05-07T02:43:06.594 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.594 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.594 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.594 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.594 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.594 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.594 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.594 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "password of InfluxDB server user", 2024-05-07T02:43:06.594 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.595 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.595 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.595 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.595 INFO:teuthology.orchestra.run.smithi019.stdout: "port": { 2024-05-07T02:43:06.595 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "port", 2024-05-07T02:43:06.595 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.595 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.595 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.596 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8086", 2024-05-07T02:43:06.596 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.596 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.596 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.596 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server port", 2024-05-07T02:43:06.596 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.596 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.596 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.596 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.597 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.597 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.597 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.597 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.597 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.597 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.597 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.597 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.597 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.598 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.598 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.598 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.598 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.598 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.598 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-05-07T02:43:06.598 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-05-07T02:43:06.598 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.598 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.599 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.599 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "false", 2024-05-07T02:43:06.599 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.599 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.599 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.599 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-07T02:43:06.599 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.599 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.600 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.600 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.600 INFO:teuthology.orchestra.run.smithi019.stdout: "threads": { 2024-05-07T02:43:06.600 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threads", 2024-05-07T02:43:06.600 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.600 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.600 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.600 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-07T02:43:06.600 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-07T02:43:06.600 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "32", 2024-05-07T02:43:06.601 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.601 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-07T02:43:06.601 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.601 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.601 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.601 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.601 INFO:teuthology.orchestra.run.smithi019.stdout: "username": { 2024-05-07T02:43:06.601 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "username", 2024-05-07T02:43:06.602 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.602 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.602 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.602 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.602 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.602 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.602 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.602 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "username of InfluxDB server user", 2024-05-07T02:43:06.602 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.603 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.603 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.603 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.603 INFO:teuthology.orchestra.run.smithi019.stdout: "verify_ssl": { 2024-05-07T02:43:06.603 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "verify_ssl", 2024-05-07T02:43:06.603 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.603 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.603 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.603 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "true", 2024-05-07T02:43:06.604 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.604 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.604 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.604 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-07T02:43:06.604 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.604 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.604 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.604 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.604 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.605 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.605 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.605 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "insights", 2024-05-07T02:43:06.605 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.605 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.605 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.605 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.605 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.605 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.606 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.606 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.606 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.606 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.606 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.606 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.606 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.606 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.607 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.607 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.607 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.607 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.607 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.607 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.607 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.607 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.607 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.607 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.608 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.608 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.608 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.608 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.608 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.608 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.608 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.608 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.608 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.609 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.609 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.609 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.609 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.609 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.609 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.609 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.609 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.609 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.610 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.610 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.610 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.610 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.610 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.610 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.610 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.610 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.611 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.611 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.611 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.611 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.611 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.611 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.611 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.611 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.611 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.611 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.612 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.612 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.612 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.612 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.612 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.612 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.612 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.612 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.612 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.613 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.613 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.613 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.613 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.613 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.613 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.613 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.613 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.613 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.614 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.614 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.614 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.614 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.614 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.614 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.614 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.614 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.615 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.615 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.615 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.615 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.615 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "iostat", 2024-05-07T02:43:06.615 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.615 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.615 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.615 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.616 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.616 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.616 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.616 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.616 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.616 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.616 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.616 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.616 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.617 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.617 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.617 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.617 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.617 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.617 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.617 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.617 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.617 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.618 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.618 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.618 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.618 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.618 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.618 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.618 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.618 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.618 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.619 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.619 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.619 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.619 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.619 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.619 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.619 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.619 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.619 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.620 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.620 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.620 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.620 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.620 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.620 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.620 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.620 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.620 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.621 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.621 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.621 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.621 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.621 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.621 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.621 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.621 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.622 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.622 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.622 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.622 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.622 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.622 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.622 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.622 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.622 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.623 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.623 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.623 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.623 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.623 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.623 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.623 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.623 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.623 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.624 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.624 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.624 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.624 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.624 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.624 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.624 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.624 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.624 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.625 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.625 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.625 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.625 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.625 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.625 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.625 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "k8sevents", 2024-05-07T02:43:06.625 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.625 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.626 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.626 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_event_retention_days": { 2024-05-07T02:43:06.626 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ceph_event_retention_days", 2024-05-07T02:43:06.626 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.626 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.626 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.626 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7", 2024-05-07T02:43:06.626 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.626 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.627 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.627 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-07T02:43:06.627 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.627 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.627 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.627 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.627 INFO:teuthology.orchestra.run.smithi019.stdout: "config_check_secs": { 2024-05-07T02:43:06.627 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_check_secs", 2024-05-07T02:43:06.627 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.628 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.628 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.628 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-07T02:43:06.628 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "10", 2024-05-07T02:43:06.628 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.628 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.628 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-07T02:43:06.628 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.628 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.629 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.629 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.629 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.629 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.629 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.629 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.629 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.629 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.629 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.630 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.630 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.630 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.630 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.630 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.630 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.630 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.630 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.631 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.631 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.631 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.631 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.631 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.631 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.631 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.631 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.631 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.632 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.632 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.632 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.632 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.632 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.632 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.632 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.632 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.632 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.633 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.633 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.633 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.633 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.633 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.633 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.633 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.633 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.633 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.634 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.634 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.634 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.634 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.634 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.634 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.634 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.634 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.634 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.635 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.635 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.635 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.635 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.635 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.635 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.635 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.635 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.635 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.636 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.636 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.636 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.636 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.636 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.636 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.636 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.636 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.636 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.637 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.637 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.637 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.637 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.637 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.637 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.637 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.637 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.638 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.638 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.638 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.638 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.638 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.638 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.638 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.638 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.638 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.639 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.639 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "localpool", 2024-05-07T02:43:06.639 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.639 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.639 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.639 INFO:teuthology.orchestra.run.smithi019.stdout: "failure_domain": { 2024-05-07T02:43:06.639 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "failure_domain", 2024-05-07T02:43:06.639 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.639 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.640 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.640 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "host", 2024-05-07T02:43:06.640 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.640 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.640 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.640 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "failure domain for any created local pool", 2024-05-07T02:43:06.640 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-07T02:43:06.640 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.640 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.641 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.641 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.641 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.641 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.641 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.641 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.641 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.641 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.641 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.642 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.642 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.642 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.642 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.642 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.642 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.642 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.642 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.642 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.643 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.643 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.643 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.643 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.643 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.643 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.643 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.643 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.643 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.644 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.644 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.644 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.644 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.644 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.644 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.644 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.644 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.644 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.645 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.645 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.645 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.645 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.645 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.645 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.645 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.645 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.645 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.646 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.646 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.646 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.646 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.646 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.646 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.646 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.646 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.646 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.647 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.647 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.647 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.647 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.647 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.647 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.647 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.647 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.647 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.648 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.648 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.648 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.648 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.648 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.648 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.648 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.648 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.648 INFO:teuthology.orchestra.run.smithi019.stdout: "min_size": { 2024-05-07T02:43:06.648 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_size", 2024-05-07T02:43:06.649 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.649 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.649 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.649 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.649 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.649 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.649 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.649 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default min_size for any created local pool", 2024-05-07T02:43:06.650 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-07T02:43:06.650 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.650 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.650 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.650 INFO:teuthology.orchestra.run.smithi019.stdout: "num_rep": { 2024-05-07T02:43:06.650 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "num_rep", 2024-05-07T02:43:06.650 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.650 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.650 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.651 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3", 2024-05-07T02:43:06.651 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.651 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.651 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.651 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default replica count for any created local pool", 2024-05-07T02:43:06.651 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.651 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.651 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.651 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.652 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num": { 2024-05-07T02:43:06.652 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_num", 2024-05-07T02:43:06.652 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.652 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.652 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.652 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "128", 2024-05-07T02:43:06.652 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.652 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.652 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.653 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default pg_num for any created local pool", 2024-05-07T02:43:06.653 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.653 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.653 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.653 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.653 INFO:teuthology.orchestra.run.smithi019.stdout: "prefix": { 2024-05-07T02:43:06.653 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prefix", 2024-05-07T02:43:06.653 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.653 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.654 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.654 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.654 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.654 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.654 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.654 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name prefix for any created local pool", 2024-05-07T02:43:06.654 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.654 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.655 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.655 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.655 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.655 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.655 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.655 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.655 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.655 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.655 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.656 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.656 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.656 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.656 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.656 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.656 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.656 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.656 INFO:teuthology.orchestra.run.smithi019.stdout: "subtree": { 2024-05-07T02:43:06.656 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "subtree", 2024-05-07T02:43:06.657 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.657 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.657 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.657 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "rack", 2024-05-07T02:43:06.657 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.657 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.657 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.657 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-07T02:43:06.657 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-07T02:43:06.657 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.658 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.658 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.658 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.658 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.658 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.658 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mds_autoscaler", 2024-05-07T02:43:06.658 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.658 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.659 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.659 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.659 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.659 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.659 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.659 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.659 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.659 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.659 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.660 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.660 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.660 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.660 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.660 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.660 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.660 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.660 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.660 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.660 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.661 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.661 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.661 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.661 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.661 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.661 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.661 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.661 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.662 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.662 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.662 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.662 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.662 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.662 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.662 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.662 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.662 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.663 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.663 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.663 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.663 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.663 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.663 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.663 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.663 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.663 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.664 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.664 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.664 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.664 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.664 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.664 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.664 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.664 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.664 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.665 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.665 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.665 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.665 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.665 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.665 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.665 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.665 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.665 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.666 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.666 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.666 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.666 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.666 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.666 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.666 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.666 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.666 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.667 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.667 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.667 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.667 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.667 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.667 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.667 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.667 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.667 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.668 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.668 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.668 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.668 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.668 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.668 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.668 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.668 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirroring", 2024-05-07T02:43:06.668 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.669 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.669 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.669 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.669 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.669 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.669 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.669 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.669 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.669 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.670 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.670 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.670 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.670 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.670 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.670 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.670 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.670 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.670 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.671 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.671 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.671 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.671 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.671 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.671 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.671 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.671 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.671 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.672 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.672 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.672 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.672 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.672 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.672 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.672 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.672 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.672 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.673 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.673 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.673 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.673 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.673 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.673 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.673 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.673 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.674 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.674 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.674 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.674 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.674 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.674 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.674 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.674 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.674 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.675 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.675 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.675 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.675 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.675 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.675 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.675 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.675 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.675 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.676 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.676 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.676 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.676 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.676 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.676 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.676 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.676 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.676 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.677 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.677 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.677 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.677 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.677 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.677 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.677 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.677 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.677 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.678 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.678 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.678 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.678 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.678 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.678 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.678 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.678 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.679 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.679 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "nfs", 2024-05-07T02:43:06.679 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.679 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.679 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.679 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.679 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.679 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.679 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.679 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.680 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.680 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.680 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.680 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.680 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.680 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.680 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.680 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.680 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.681 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.681 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.681 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.681 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.681 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.681 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.681 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.681 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.681 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.682 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.682 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.682 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.682 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.682 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.682 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.682 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.682 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.683 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.683 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.683 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.683 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.683 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.683 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.683 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.683 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.683 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.684 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.684 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.684 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.684 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.684 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.684 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.684 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.684 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.684 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.685 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.685 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.685 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.685 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.685 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.685 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.685 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.685 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.685 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.686 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.686 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.686 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.686 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.686 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.686 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.686 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.686 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.686 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.687 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.687 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.687 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.687 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.687 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.687 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.687 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.687 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.687 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.688 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.688 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.688 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.688 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.688 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.688 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.688 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.688 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.688 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.689 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.689 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.689 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-05-07T02:43:06.689 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.689 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.689 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.689 INFO:teuthology.orchestra.run.smithi019.stdout: "fail_fs": { 2024-05-07T02:43:06.689 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "fail_fs", 2024-05-07T02:43:06.689 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.690 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.690 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.690 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.690 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.690 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.690 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.690 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-07T02:43:06.690 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.690 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.691 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.691 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.691 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.691 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.691 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.691 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.691 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.691 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.691 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.692 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.692 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.692 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.692 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.692 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.692 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.692 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.692 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.692 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.693 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.693 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.693 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.693 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.693 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.693 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.693 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.693 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.693 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.693 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.694 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.694 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.694 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.694 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.694 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.694 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.694 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.694 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.695 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.695 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.695 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.695 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.695 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.695 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.695 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.695 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.695 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.696 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.696 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.696 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.696 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.696 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.696 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.696 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.696 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.696 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.697 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.697 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.697 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.697 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.697 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.697 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.697 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.697 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.697 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.698 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.698 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.698 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.698 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.698 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.698 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.698 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.698 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.698 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.699 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator": { 2024-05-07T02:43:06.699 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-05-07T02:43:06.699 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.699 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.699 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.699 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.699 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.699 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.699 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.700 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm", 2024-05-07T02:43:06.700 INFO:teuthology.orchestra.run.smithi019.stdout: "rook", 2024-05-07T02:43:06.700 INFO:teuthology.orchestra.run.smithi019.stdout: "test_orchestrator" 2024-05-07T02:43:06.700 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.700 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Orchestrator backend", 2024-05-07T02:43:06.700 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.700 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.700 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.700 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.701 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.701 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.701 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.701 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.701 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.701 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.701 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.701 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.701 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.702 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.702 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.702 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.702 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.702 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.702 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.702 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.702 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.702 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_perf_query", 2024-05-07T02:43:06.703 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.703 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.703 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.703 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.703 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.703 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.703 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.703 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.703 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.704 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.704 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.704 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.704 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.704 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.704 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.704 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.704 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.704 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.705 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.705 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.705 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.705 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.705 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.705 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.705 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.705 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.705 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.705 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.706 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.706 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.706 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.706 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.706 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.706 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.706 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.706 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.706 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.707 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.707 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.707 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.707 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.707 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.707 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.707 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.707 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.707 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.707 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.708 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.708 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.708 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.708 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.708 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.708 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.708 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.708 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.709 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.709 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.709 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.709 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.709 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.709 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.709 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.709 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.709 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.710 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.710 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.710 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.710 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.710 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.710 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.710 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.710 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.711 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.711 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.711 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.711 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.711 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.711 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.711 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.711 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.711 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.711 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.712 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.712 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.712 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.712 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.712 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.712 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.712 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.712 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.712 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_support", 2024-05-07T02:43:06.712 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.713 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.713 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.713 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.713 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.713 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.713 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.713 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.713 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.713 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.714 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.714 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.714 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.714 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.714 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.714 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.714 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.714 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.715 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.715 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.715 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.715 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.715 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.715 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.715 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.715 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.715 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.715 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.716 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.716 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.716 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.716 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.716 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.716 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.716 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.716 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.716 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.717 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.717 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.717 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.717 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.717 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.717 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.717 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.717 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.717 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.718 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.718 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.718 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.718 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.718 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.718 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.718 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.718 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.719 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.719 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.719 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.719 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.719 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.719 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.719 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.719 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.719 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.720 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.720 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.720 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.720 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.720 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.720 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.720 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.720 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.720 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.721 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.721 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.721 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.721 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.721 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.721 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.721 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.721 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.721 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.722 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.722 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.722 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.722 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.722 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.722 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.722 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.722 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.722 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.723 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_autoscaler", 2024-05-07T02:43:06.723 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.723 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.723 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.723 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.723 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.723 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.723 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.723 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.724 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.724 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.724 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.724 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.724 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.724 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.724 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.724 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.724 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.725 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.725 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.725 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.725 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.725 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.725 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.725 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.725 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.725 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.726 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.726 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.726 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.726 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.726 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.726 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.726 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.726 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.726 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.727 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.727 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.727 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.727 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.727 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.727 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.727 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.727 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.727 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.728 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.728 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.728 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.728 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.728 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.728 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.728 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.728 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.728 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.729 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.729 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.729 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.729 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.729 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.729 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.729 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.729 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.729 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.730 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.730 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.730 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.730 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.730 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.730 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.730 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.730 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.731 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.731 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.731 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.731 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-07T02:43:06.731 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-07T02:43:06.731 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:06.731 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.731 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.731 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-07T02:43:06.732 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.732 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.732 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.732 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.732 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.732 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.732 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.732 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.732 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.733 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.733 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.733 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.733 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.733 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.733 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.733 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.733 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.733 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.734 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.734 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.734 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.734 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.734 INFO:teuthology.orchestra.run.smithi019.stdout: "threshold": { 2024-05-07T02:43:06.734 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threshold", 2024-05-07T02:43:06.734 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-07T02:43:06.734 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.735 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.735 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-05-07T02:43:06.735 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1.0", 2024-05-07T02:43:06.735 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.735 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.735 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "scaling threshold", 2024-05-07T02:43:06.735 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-07T02:43:06.735 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.735 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.736 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.736 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.736 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.736 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.736 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "progress", 2024-05-07T02:43:06.736 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.736 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.736 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.736 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_pg_recovery_event": { 2024-05-07T02:43:06.737 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_pg_recovery_event", 2024-05-07T02:43:06.737 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.737 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.737 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.737 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.737 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.737 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.737 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.737 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-07T02:43:06.738 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.738 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.738 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.738 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.738 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-05-07T02:43:06.738 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-05-07T02:43:06.738 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.738 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.739 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.739 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.739 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.739 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.739 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.739 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.739 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.739 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.739 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.740 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.740 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.740 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.740 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.740 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.740 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.740 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.740 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.740 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.741 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.741 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.741 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.741 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.741 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.741 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.741 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.741 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.741 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.742 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.742 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.742 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.742 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.742 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.742 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.742 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.742 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.742 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.743 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.743 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.743 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.743 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.743 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.743 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.743 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.743 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.743 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.744 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.744 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.744 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.744 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.744 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.744 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.744 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.744 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.744 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.745 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.745 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.745 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.745 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.745 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.745 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.745 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.745 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.745 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.746 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.746 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.746 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.746 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.746 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.746 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.746 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.746 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.747 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.747 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.747 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.747 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.747 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.747 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.747 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.750 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.750 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.750 INFO:teuthology.orchestra.run.smithi019.stdout: "max_completed_events": { 2024-05-07T02:43:06.751 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_completed_events", 2024-05-07T02:43:06.751 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.751 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.751 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.751 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "50", 2024-05-07T02:43:06.751 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.751 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.751 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.751 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "number of past completed events to remember", 2024-05-07T02:43:06.752 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.752 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.752 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.752 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.752 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-05-07T02:43:06.752 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-05-07T02:43:06.752 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:06.752 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.752 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.752 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-05-07T02:43:06.753 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.753 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.753 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.753 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long the module is going to sleep", 2024-05-07T02:43:06.753 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.753 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.753 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.753 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.753 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.754 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.754 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.754 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.754 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.754 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.754 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.754 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.754 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.755 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.755 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.755 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.755 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.755 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.755 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.755 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.755 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.755 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus", 2024-05-07T02:43:06.756 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.756 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.756 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.756 INFO:teuthology.orchestra.run.smithi019.stdout: "cache": { 2024-05-07T02:43:06.756 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cache", 2024-05-07T02:43:06.756 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.756 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.756 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.756 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.756 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.757 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.757 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.757 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.757 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.757 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.757 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.757 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.757 INFO:teuthology.orchestra.run.smithi019.stdout: "exclude_perf_counters": { 2024-05-07T02:43:06.758 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "exclude_perf_counters", 2024-05-07T02:43:06.758 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.758 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.758 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.758 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.758 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.758 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.758 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.758 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-07T02:43:06.759 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-07T02:43:06.759 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.759 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.759 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.759 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.759 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.759 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.759 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.759 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.760 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.760 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.760 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.760 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.760 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.760 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.760 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.760 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.761 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.761 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.761 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.761 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.761 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.761 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.761 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.761 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.761 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.762 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.762 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.762 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.762 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.762 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.762 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.762 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.762 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.762 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.763 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.763 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.763 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.763 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.763 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.763 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.763 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.763 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.763 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.764 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.764 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.764 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.764 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.764 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.764 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.764 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.764 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.764 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.765 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.765 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.765 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.765 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.765 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.765 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.765 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.765 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.765 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.766 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.766 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.766 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.766 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.766 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.766 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.766 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.766 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.766 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.767 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.767 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.767 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.767 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools": { 2024-05-07T02:43:06.767 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools", 2024-05-07T02:43:06.767 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.767 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.767 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.767 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.768 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.768 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.768 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.768 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.768 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.768 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.768 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.768 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.768 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-07T02:43:06.769 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-07T02:43:06.769 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.769 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.769 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.769 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "300", 2024-05-07T02:43:06.769 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.769 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.769 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.769 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.770 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.770 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.770 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.770 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.770 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_interval": { 2024-05-07T02:43:06.770 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_interval", 2024-05-07T02:43:06.770 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-07T02:43:06.770 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.770 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.771 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15.0", 2024-05-07T02:43:06.771 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.771 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.771 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.771 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.771 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.771 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.771 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.771 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.772 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-05-07T02:43:06.772 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-05-07T02:43:06.772 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.772 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.772 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.772 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-05-07T02:43:06.772 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.772 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.772 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.773 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-07T02:43:06.773 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.773 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.773 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.773 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.773 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-05-07T02:43:06.773 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-05-07T02:43:06.773 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.773 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.774 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.774 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "9283", 2024-05-07T02:43:06.774 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.774 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.774 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.774 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-07T02:43:06.774 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.774 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.774 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.775 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.775 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.775 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.775 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.775 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.775 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.775 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.775 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.775 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.776 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.776 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.776 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.776 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.776 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.776 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.776 INFO:teuthology.orchestra.run.smithi019.stdout: "stale_cache_strategy": { 2024-05-07T02:43:06.776 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stale_cache_strategy", 2024-05-07T02:43:06.776 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.777 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.777 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.777 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "log", 2024-05-07T02:43:06.777 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.777 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.777 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.777 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.777 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.777 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.778 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.778 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.778 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-05-07T02:43:06.778 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-05-07T02:43:06.778 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.778 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.778 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.778 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "default", 2024-05-07T02:43:06.778 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.779 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.779 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.779 INFO:teuthology.orchestra.run.smithi019.stdout: "default", 2024-05-07T02:43:06.779 INFO:teuthology.orchestra.run.smithi019.stdout: "error" 2024-05-07T02:43:06.779 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.779 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.779 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.779 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.779 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.780 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.780 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-05-07T02:43:06.780 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-05-07T02:43:06.780 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.780 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.780 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.780 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-05-07T02:43:06.780 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-05-07T02:43:06.780 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-05-07T02:43:06.781 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.781 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.781 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.781 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.781 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.781 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.781 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.781 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.781 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.782 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_support", 2024-05-07T02:43:06.782 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.782 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.782 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.782 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.782 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.782 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.782 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.782 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.783 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.783 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.783 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.783 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.783 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.783 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.783 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.783 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.783 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.784 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.784 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.784 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.784 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.784 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.784 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.784 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.784 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.785 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.785 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.785 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.785 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.785 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.785 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.785 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.785 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.785 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.785 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.786 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.786 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.786 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.786 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.786 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.786 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.786 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.786 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.787 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.787 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.787 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.787 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.787 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.787 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.787 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.787 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.787 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.788 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.788 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.788 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.788 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.788 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.788 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.788 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.788 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.788 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.789 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.789 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.789 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.789 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.789 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.789 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.789 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.789 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.789 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.790 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.790 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.790 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.790 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_snap_create": { 2024-05-07T02:43:06.790 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_snap_create", 2024-05-07T02:43:06.790 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.790 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.790 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.790 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-05-07T02:43:06.791 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.791 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.791 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.791 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.791 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.791 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.791 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.791 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.791 INFO:teuthology.orchestra.run.smithi019.stdout: "mirror_snapshot_schedule": { 2024-05-07T02:43:06.792 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirror_snapshot_schedule", 2024-05-07T02:43:06.792 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.792 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.792 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.792 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.792 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.792 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.792 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.792 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.793 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.793 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.793 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.793 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.793 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.793 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.793 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.793 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.793 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.794 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.794 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.794 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.794 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.794 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.794 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.794 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.794 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.794 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.795 INFO:teuthology.orchestra.run.smithi019.stdout: "trash_purge_schedule": { 2024-05-07T02:43:06.795 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "trash_purge_schedule", 2024-05-07T02:43:06.795 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.795 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.795 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.795 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.795 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.795 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.795 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.795 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.796 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.796 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.796 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.796 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.796 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.796 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.796 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.796 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "restful", 2024-05-07T02:43:06.796 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.797 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.797 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.797 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_auth": { 2024-05-07T02:43:06.797 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_auth", 2024-05-07T02:43:06.797 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.797 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.797 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.797 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.797 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.797 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.798 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.798 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.798 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.798 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.798 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.798 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.798 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-05-07T02:43:06.798 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-05-07T02:43:06.799 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.799 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.799 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.799 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.799 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.799 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.799 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.799 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.799 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.799 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.800 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.800 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.800 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.800 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.800 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.800 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.800 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.800 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.800 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.801 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.801 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.801 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.801 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.801 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.801 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.801 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.801 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.801 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.801 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.802 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.802 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.802 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.802 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.802 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.802 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.802 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.802 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.802 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.803 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.803 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.803 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.803 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.803 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.803 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.803 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.803 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.803 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.804 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.804 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.804 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.804 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.804 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.804 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.804 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.804 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.804 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.805 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.805 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.805 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.805 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.805 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.805 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.805 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.805 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.805 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.806 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.806 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.806 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.806 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.806 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.806 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.806 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.806 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.806 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.807 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.807 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.807 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.807 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.807 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.807 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.807 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.807 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.807 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-05-07T02:43:06.808 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-05-07T02:43:06.808 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.808 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.808 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.808 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.808 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.808 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.808 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.808 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.808 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.809 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.809 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.809 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.809 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-05-07T02:43:06.809 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-05-07T02:43:06.809 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.809 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.809 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.809 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.810 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.810 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.810 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.810 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.810 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.810 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.810 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.810 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.810 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.811 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.811 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.811 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.811 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.811 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.811 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.811 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.811 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.811 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.811 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.812 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.812 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.812 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.812 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.812 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.812 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.812 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rgw", 2024-05-07T02:43:06.812 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.812 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.813 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.813 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.813 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.813 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.813 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.813 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.813 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.813 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.813 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.813 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.814 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.814 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.814 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.814 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.814 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.814 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.814 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.814 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.814 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.815 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.815 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.815 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.815 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.815 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.815 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.815 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.815 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.815 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.816 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.816 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.816 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.816 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.816 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.816 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.816 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.816 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.817 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.817 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.817 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.817 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.817 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.817 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.817 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.817 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.817 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.817 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.818 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.818 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.818 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.818 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.818 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.818 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.818 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.818 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.818 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.819 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.819 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.819 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.819 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.819 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.819 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.819 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.819 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.819 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.820 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.820 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.820 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.820 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.820 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.820 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.820 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.820 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.820 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.821 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.821 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.821 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.821 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.821 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.821 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.821 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.821 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.822 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.822 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.822 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.822 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.822 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.822 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.822 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.822 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rook", 2024-05-07T02:43:06.822 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.823 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.823 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.823 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.823 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.823 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.823 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.823 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.823 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.823 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.824 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.824 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.824 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.824 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.824 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.824 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.824 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.824 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.824 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.825 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.825 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.825 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.825 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.825 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.825 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.825 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.825 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.825 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.826 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.826 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.826 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.826 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.826 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.826 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.826 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.826 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.826 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.827 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.827 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.827 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.827 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.827 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.827 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.827 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.827 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.827 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.827 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.828 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.828 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.828 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.828 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.828 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.828 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.828 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.828 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.828 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.829 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.829 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.829 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.829 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.829 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.829 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.829 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.829 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.829 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.830 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.830 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.830 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.830 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.830 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.830 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.830 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.830 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.830 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.831 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.831 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.831 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.831 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.831 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.831 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.831 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.831 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.831 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.832 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.832 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.832 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.832 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.832 INFO:teuthology.orchestra.run.smithi019.stdout: "storage_class": { 2024-05-07T02:43:06.832 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "storage_class", 2024-05-07T02:43:06.832 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.832 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.832 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.833 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "local", 2024-05-07T02:43:06.833 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.833 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.833 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.833 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-07T02:43:06.833 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.833 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.833 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.833 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.834 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.834 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.834 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.834 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "selftest", 2024-05-07T02:43:06.834 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.834 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.834 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.834 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.835 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.835 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.835 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.835 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.835 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.835 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.835 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.835 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.835 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.835 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.836 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.836 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.836 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.836 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.836 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.836 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.836 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.836 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.836 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.837 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.837 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.837 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.837 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.837 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.837 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.837 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.837 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.837 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.838 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.838 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.838 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.838 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.838 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.838 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.838 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.838 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.838 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.839 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.839 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.839 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.839 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.839 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.839 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.839 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.839 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.839 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.840 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.840 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.840 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.840 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.840 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.840 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.840 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.840 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.840 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.840 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.841 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.841 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.841 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.841 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.841 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.841 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.841 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.841 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.841 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.842 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.842 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.842 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.842 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.842 INFO:teuthology.orchestra.run.smithi019.stdout: "roption1": { 2024-05-07T02:43:06.842 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption1", 2024-05-07T02:43:06.842 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.842 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.842 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.843 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.843 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.843 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.843 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.843 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.843 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.843 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.843 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.844 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.844 INFO:teuthology.orchestra.run.smithi019.stdout: "roption2": { 2024-05-07T02:43:06.844 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption2", 2024-05-07T02:43:06.844 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.844 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.844 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.844 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "xyz", 2024-05-07T02:43:06.844 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.844 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.845 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.845 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.845 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.845 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.845 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.845 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.845 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption1": { 2024-05-07T02:43:06.845 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption1", 2024-05-07T02:43:06.845 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.846 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.846 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.846 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.846 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.846 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.846 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.846 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.846 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.847 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.847 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.847 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.847 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption2": { 2024-05-07T02:43:06.847 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption2", 2024-05-07T02:43:06.847 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.847 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.847 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.847 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.848 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.848 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.848 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.848 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.848 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.848 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.848 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.848 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.848 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption3": { 2024-05-07T02:43:06.848 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption3", 2024-05-07T02:43:06.849 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-05-07T02:43:06.849 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.849 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.849 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.849 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.849 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.849 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.849 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.849 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.850 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.850 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.850 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.850 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption4": { 2024-05-07T02:43:06.850 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption4", 2024-05-07T02:43:06.850 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.850 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.850 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.850 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.851 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.851 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.851 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.851 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.851 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.851 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.851 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.851 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.851 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption5": { 2024-05-07T02:43:06.852 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption5", 2024-05-07T02:43:06.852 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.852 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.852 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.852 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.852 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.852 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.852 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.852 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.853 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.853 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.853 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.853 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.853 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption6": { 2024-05-07T02:43:06.853 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption6", 2024-05-07T02:43:06.853 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.853 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.853 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.854 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.854 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.854 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.854 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.854 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.854 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.854 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.854 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.854 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.855 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption7": { 2024-05-07T02:43:06.855 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption7", 2024-05-07T02:43:06.855 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.855 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.855 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.855 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.855 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-05-07T02:43:06.855 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "42", 2024-05-07T02:43:06.855 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.856 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.856 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.856 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.856 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.856 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.856 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.856 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.856 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.856 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.856 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.857 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.857 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.857 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.857 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.857 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.857 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.857 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.857 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.857 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.858 INFO:teuthology.orchestra.run.smithi019.stdout: "testkey": { 2024-05-07T02:43:06.858 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testkey", 2024-05-07T02:43:06.858 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.858 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.858 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.858 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.858 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.858 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.858 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.859 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.859 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.859 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.859 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.859 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.859 INFO:teuthology.orchestra.run.smithi019.stdout: "testlkey": { 2024-05-07T02:43:06.859 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testlkey", 2024-05-07T02:43:06.859 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.859 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.860 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.860 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.860 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.860 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.860 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.860 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.860 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.860 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.860 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.861 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.861 INFO:teuthology.orchestra.run.smithi019.stdout: "testnewline": { 2024-05-07T02:43:06.861 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testnewline", 2024-05-07T02:43:06.861 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.861 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.861 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.861 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.861 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.861 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.862 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.862 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.862 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.862 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.862 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.862 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.862 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.862 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.862 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.863 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smb", 2024-05-07T02:43:06.863 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.863 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.863 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.863 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.863 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.863 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.863 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.863 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.864 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.864 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.864 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.864 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.864 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.864 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.864 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.864 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.864 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.865 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.865 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.865 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.865 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.865 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.865 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.865 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.865 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.865 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.866 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.866 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.866 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.866 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.866 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.866 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.866 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.866 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.866 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.867 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.867 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.867 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.867 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.867 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.867 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.867 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.867 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.867 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.868 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.868 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.868 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.868 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.868 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.868 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.868 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.868 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.868 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.868 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.869 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.869 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.869 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.869 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.869 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.869 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.869 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.869 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.869 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.870 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.870 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.870 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.870 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.870 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.870 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.870 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.870 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.870 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.871 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.871 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.871 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.871 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.871 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.871 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.871 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.871 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.872 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.872 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.872 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.872 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.872 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.872 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.872 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.872 INFO:teuthology.orchestra.run.smithi019.stdout: "update_orchestration": { 2024-05-07T02:43:06.872 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "update_orchestration", 2024-05-07T02:43:06.873 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.873 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.873 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.873 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.873 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.873 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.873 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.873 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-07T02:43:06.873 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.874 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.874 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.874 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.874 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.874 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.874 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.874 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snap_schedule", 2024-05-07T02:43:06.874 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.875 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.875 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.875 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_m_granularity": { 2024-05-07T02:43:06.875 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_m_granularity", 2024-05-07T02:43:06.875 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.875 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.875 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.875 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.875 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.876 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.876 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.876 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow minute scheduled snapshots", 2024-05-07T02:43:06.876 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.876 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.876 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.876 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.876 INFO:teuthology.orchestra.run.smithi019.stdout: "dump_on_update": { 2024-05-07T02:43:06.876 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dump_on_update", 2024-05-07T02:43:06.877 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.877 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.877 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.877 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.877 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.877 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.877 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.877 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "dump database to debug log on update", 2024-05-07T02:43:06.877 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.878 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.878 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.878 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.878 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.878 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.878 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.878 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.878 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.878 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.879 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.879 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.879 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.879 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.879 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.879 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.879 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.879 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.879 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.880 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.880 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.880 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.880 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.880 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.880 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.880 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.880 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.880 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.881 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.881 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.881 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.881 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.881 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.881 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.881 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.881 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.881 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.881 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.882 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.882 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.882 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.882 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.882 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.882 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.882 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.882 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.883 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.883 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.883 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.883 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.883 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.883 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.883 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.883 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.883 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.884 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.884 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.884 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.884 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.884 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.884 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.884 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.884 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.884 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.885 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.885 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.885 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.885 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.885 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.885 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.885 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.885 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.885 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.886 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.886 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.886 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.886 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.886 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.886 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.886 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.886 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.886 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.887 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.887 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.887 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.887 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.887 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.887 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.887 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.887 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.887 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.888 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stats", 2024-05-07T02:43:06.888 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.888 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.888 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.888 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.888 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.888 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.888 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.888 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.889 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.889 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.889 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.889 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.889 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.889 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.889 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.889 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.889 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.890 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.890 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.890 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.890 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.890 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.890 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.890 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.890 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.890 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.891 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.891 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.891 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.891 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.891 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.891 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.891 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.891 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.891 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.892 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.892 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.892 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.892 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.892 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.892 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.892 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.892 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.892 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.893 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.893 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.893 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.893 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.893 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.893 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.893 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.893 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.893 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.893 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.894 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.894 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.894 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.894 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.894 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.894 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.894 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.894 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.894 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.895 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.895 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.895 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.895 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.895 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.895 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.895 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.895 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.895 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.896 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.896 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.896 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.896 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.896 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.896 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.896 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.896 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.896 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.897 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.897 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.897 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.897 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.897 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.897 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.897 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.897 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.897 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.898 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "status", 2024-05-07T02:43:06.898 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.898 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.898 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.898 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.898 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.898 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.898 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.899 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.899 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.899 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.899 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.899 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.899 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.899 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.899 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.899 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.900 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.900 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.900 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.900 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.900 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.900 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.900 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.900 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.900 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.901 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.901 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.901 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.901 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.901 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.901 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.901 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.901 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.901 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.901 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.902 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.902 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.902 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.902 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.902 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.902 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.902 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.902 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.903 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.903 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.903 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.903 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.903 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.903 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.903 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.903 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.903 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.904 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.904 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.904 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.904 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.904 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.904 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.904 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.904 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.904 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.904 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.905 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.905 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.905 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.905 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.905 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.905 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.905 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.905 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.905 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.906 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.906 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.906 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.906 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.906 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.906 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.906 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.906 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.907 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.907 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.907 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.907 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.907 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.907 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.907 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.907 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.907 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.908 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.908 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.908 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telegraf", 2024-05-07T02:43:06.908 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.908 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.908 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.908 INFO:teuthology.orchestra.run.smithi019.stdout: "address": { 2024-05-07T02:43:06.908 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "address", 2024-05-07T02:43:06.908 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.908 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.909 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.909 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-07T02:43:06.909 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.909 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.909 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.909 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.909 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.909 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.909 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.910 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.910 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-07T02:43:06.910 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-07T02:43:06.910 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:06.910 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.910 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.910 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15", 2024-05-07T02:43:06.910 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.910 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.911 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.911 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.911 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.911 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.911 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.911 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.911 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.911 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.911 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.912 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.912 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.912 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.912 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.912 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.912 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.912 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.912 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.912 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.913 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.913 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.913 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.913 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.913 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.913 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.913 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.913 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.913 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.913 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.914 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.914 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.914 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.914 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.914 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.914 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.914 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.914 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.914 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.915 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.915 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.915 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.915 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.915 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.915 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.915 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.915 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.915 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.916 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.916 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.916 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.916 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.916 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.916 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.916 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.916 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.916 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.916 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.917 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.917 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.917 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.917 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.917 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.917 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.917 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.917 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.918 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.918 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.918 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.918 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.918 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.918 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.918 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.918 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.918 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.919 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.919 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.919 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.919 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.919 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.919 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.919 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.919 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.919 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.919 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.920 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.920 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.920 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.920 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.920 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.920 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.920 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.920 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.920 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.920 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.921 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telemetry", 2024-05-07T02:43:06.921 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.921 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.921 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.921 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_basic": { 2024-05-07T02:43:06.921 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_basic", 2024-05-07T02:43:06.921 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.921 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.921 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.922 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.922 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.922 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.922 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.922 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-07T02:43:06.922 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.922 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.922 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.922 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.923 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_crash": { 2024-05-07T02:43:06.923 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_crash", 2024-05-07T02:43:06.923 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.923 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.923 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.923 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.923 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.923 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.923 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.923 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-07T02:43:06.924 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.924 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.924 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.924 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.924 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_device": { 2024-05-07T02:43:06.924 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_device", 2024-05-07T02:43:06.924 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.924 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.924 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.925 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.925 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.925 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.925 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.925 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-07T02:43:06.925 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.925 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.925 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.925 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.926 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_ident": { 2024-05-07T02:43:06.926 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_ident", 2024-05-07T02:43:06.926 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.926 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.926 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.926 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.926 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.926 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.926 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.927 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-07T02:43:06.927 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.927 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.927 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.927 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.927 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_perf": { 2024-05-07T02:43:06.927 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_perf", 2024-05-07T02:43:06.927 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.927 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.927 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.928 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.928 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.928 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.928 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.928 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-07T02:43:06.928 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.928 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.928 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.928 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.929 INFO:teuthology.orchestra.run.smithi019.stdout: "contact": { 2024-05-07T02:43:06.929 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "contact", 2024-05-07T02:43:06.929 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.929 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.929 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.929 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.929 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.929 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.929 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.929 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.930 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.930 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.930 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.930 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.930 INFO:teuthology.orchestra.run.smithi019.stdout: "description": { 2024-05-07T02:43:06.930 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "description", 2024-05-07T02:43:06.930 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.930 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.930 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.931 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.931 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.931 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.931 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.931 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.931 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.931 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.931 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.931 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.932 INFO:teuthology.orchestra.run.smithi019.stdout: "device_url": { 2024-05-07T02:43:06.932 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_url", 2024-05-07T02:43:06.932 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.932 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.932 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.932 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-07T02:43:06.932 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.932 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.932 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.933 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.933 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.933 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.933 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.933 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.933 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-05-07T02:43:06.933 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-05-07T02:43:06.933 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.933 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.934 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.934 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.934 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.934 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.934 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.934 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.934 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.934 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.934 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.935 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.935 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-07T02:43:06.935 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-07T02:43:06.935 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.935 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.935 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.935 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "24", 2024-05-07T02:43:06.935 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "8", 2024-05-07T02:43:06.935 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.936 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.936 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.936 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.936 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.936 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.936 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.936 INFO:teuthology.orchestra.run.smithi019.stdout: "last_opt_revision": { 2024-05-07T02:43:06.936 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "last_opt_revision", 2024-05-07T02:43:06.936 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.936 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.937 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.937 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1", 2024-05-07T02:43:06.937 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.937 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.937 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.937 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.937 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.937 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.937 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.938 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.938 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard": { 2024-05-07T02:43:06.938 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard", 2024-05-07T02:43:06.938 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.938 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.938 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.938 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.938 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.938 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.939 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.939 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.939 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.939 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.939 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.939 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.939 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard_description": { 2024-05-07T02:43:06.939 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard_description", 2024-05-07T02:43:06.939 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.940 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.940 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.940 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.940 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.940 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.940 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.940 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.940 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.940 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.940 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.941 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.941 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.941 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.941 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.941 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.941 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.941 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.941 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.941 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.942 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.942 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.942 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.942 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.942 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.942 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.942 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.942 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.942 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.942 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.943 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.943 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.943 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.943 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.943 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.943 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.943 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.943 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.943 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.944 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.944 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.944 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.944 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.944 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.944 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.944 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.944 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.945 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.945 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.945 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.945 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.945 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.945 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.945 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.945 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.945 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.946 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.946 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.946 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.946 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.946 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.946 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.946 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.946 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.946 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.947 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.947 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.947 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.947 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.947 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.947 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.947 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.947 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.947 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.948 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.948 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.948 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.948 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.948 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.948 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.948 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.948 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.948 INFO:teuthology.orchestra.run.smithi019.stdout: "organization": { 2024-05-07T02:43:06.949 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "organization", 2024-05-07T02:43:06.949 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.949 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.949 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.949 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.949 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.949 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.949 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.949 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.949 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.950 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.950 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.950 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.950 INFO:teuthology.orchestra.run.smithi019.stdout: "proxy": { 2024-05-07T02:43:06.950 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "proxy", 2024-05-07T02:43:06.950 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.950 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.950 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.955 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.955 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.955 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.955 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.955 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.955 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.955 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.955 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.956 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.956 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.956 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.956 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.956 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.956 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.956 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.956 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.956 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.956 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.957 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.957 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.957 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.957 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.957 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.957 INFO:teuthology.orchestra.run.smithi019.stdout: "url": { 2024-05-07T02:43:06.957 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url", 2024-05-07T02:43:06.957 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.958 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.958 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.958 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-07T02:43:06.958 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.958 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.958 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.958 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.958 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.958 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.959 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.959 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.959 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.959 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.959 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.959 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "test_orchestrator", 2024-05-07T02:43:06.959 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.959 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.960 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.960 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.960 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.960 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.960 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.960 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.960 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.960 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.960 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.961 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.961 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.961 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.961 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.961 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.961 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.961 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.961 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.961 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.961 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.962 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.962 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.962 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.962 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.962 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.962 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.962 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.962 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.963 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.963 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.963 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.963 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.963 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.963 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.963 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.963 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.963 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.963 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.964 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.964 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.964 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.964 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.964 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.964 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.964 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.964 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.964 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.965 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.965 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.965 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.965 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.965 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.965 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.965 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.965 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.965 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.966 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.966 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.966 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.966 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.966 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.966 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.966 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.966 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.967 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.967 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.967 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.967 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.967 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.967 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.967 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.967 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.967 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.968 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.968 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.968 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.968 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.968 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.968 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.968 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.968 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.968 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.969 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.969 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.969 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.969 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.969 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.969 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.969 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.969 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "volumes", 2024-05-07T02:43:06.969 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.970 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.970 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.970 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.970 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.970 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.970 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.970 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.970 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.970 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.971 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.971 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.971 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.971 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.971 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.971 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.971 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.971 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.971 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.971 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.972 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.972 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.972 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.972 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.972 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.972 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.972 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.972 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.972 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.973 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.973 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.973 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.973 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.973 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.973 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.973 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.973 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.973 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.974 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.974 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.974 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.974 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.974 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.974 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.974 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.974 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.974 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.975 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.975 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.975 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.975 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.975 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.975 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.975 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.975 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.975 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.975 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.976 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.976 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.976 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.976 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.976 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.976 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.976 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.976 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.976 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.976 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.977 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.977 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.977 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.977 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.977 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.977 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.977 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_clones": { 2024-05-07T02:43:06.977 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_clones", 2024-05-07T02:43:06.977 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.978 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.978 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.978 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4", 2024-05-07T02:43:06.978 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.978 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.978 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.978 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-07T02:43:06.978 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.978 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.979 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.979 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.979 INFO:teuthology.orchestra.run.smithi019.stdout: "periodic_async_work": { 2024-05-07T02:43:06.979 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "periodic_async_work", 2024-05-07T02:43:06.979 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.979 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.979 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.979 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.979 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.980 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.980 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.980 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Periodically check for async work", 2024-05-07T02:43:06.980 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.980 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.980 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.980 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.980 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_delay": { 2024-05-07T02:43:06.980 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_delay", 2024-05-07T02:43:06.981 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.981 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.981 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.981 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.981 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.981 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.981 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.981 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-07T02:43:06.981 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.982 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.982 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.982 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.982 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_no_wait": { 2024-05-07T02:43:06.982 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_no_wait", 2024-05-07T02:43:06.982 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.982 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.982 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.982 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-05-07T02:43:06.982 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.983 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.983 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.983 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-07T02:43:06.983 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.983 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.983 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.983 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.983 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.983 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.983 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.984 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.984 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.984 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.984 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.984 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.984 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.984 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.984 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.984 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.985 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.985 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.985 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:06.985 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.985 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:06.985 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix", 2024-05-07T02:43:06.985 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-05-07T02:43:06.985 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-05-07T02:43:06.985 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-05-07T02:43:06.985 INFO:teuthology.orchestra.run.smithi019.stdout: "discovery_interval": { 2024-05-07T02:43:06.986 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "discovery_interval", 2024-05-07T02:43:06.986 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-05-07T02:43:06.986 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.986 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.986 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "100", 2024-05-07T02:43:06.986 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.986 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.986 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.986 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.987 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.987 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.987 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.987 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.987 INFO:teuthology.orchestra.run.smithi019.stdout: "identifier": { 2024-05-07T02:43:06.987 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "identifier", 2024-05-07T02:43:06.987 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.987 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.987 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.988 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.988 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.988 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.988 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.988 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.988 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.988 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.988 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.988 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.989 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-05-07T02:43:06.989 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-05-07T02:43:06.989 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-05-07T02:43:06.989 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.989 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:06.989 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-05-07T02:43:06.989 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.989 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.989 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.990 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.990 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.990 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.990 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.990 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.990 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-05-07T02:43:06.990 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-05-07T02:43:06.990 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.990 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.991 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.991 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:06.991 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.991 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.991 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.991 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.991 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.991 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.991 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.991 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.992 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.992 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.992 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.992 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.992 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.992 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.992 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.992 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-05-07T02:43:06.992 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-05-07T02:43:06.993 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.993 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.993 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.993 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.993 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.993 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.993 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.993 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.993 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.994 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.994 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.994 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.994 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-05-07T02:43:06.994 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-05-07T02:43:06.994 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:06.994 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.994 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.994 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-05-07T02:43:06.995 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.995 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.995 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-05-07T02:43:06.995 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-05-07T02:43:06.995 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-05-07T02:43:06.995 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-05-07T02:43:06.995 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-05-07T02:43:06.995 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-05-07T02:43:06.995 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-05-07T02:43:06.995 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:06.996 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.996 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.996 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.996 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.996 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.996 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-05-07T02:43:06.996 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-05-07T02:43:06.996 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-05-07T02:43:06.996 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:06.997 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.997 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-05-07T02:43:06.997 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.997 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.997 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.997 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.997 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.997 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.997 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.998 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.998 INFO:teuthology.orchestra.run.smithi019.stdout: "sqlite3_killpoint": { 2024-05-07T02:43:06.998 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sqlite3_killpoint", 2024-05-07T02:43:06.998 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:06.998 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "dev", 2024-05-07T02:43:06.998 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-05-07T02:43:06.998 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-05-07T02:43:06.998 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:06.998 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:06.999 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:06.999 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:06.999 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:06.999 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:06.999 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:06.999 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:06.999 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_host": { 2024-05-07T02:43:06.999 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_host", 2024-05-07T02:43:06.999 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:07.000 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:07.000 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:07.000 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-05-07T02:43:07.000 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:07.000 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:07.000 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:07.000 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:07.000 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:07.000 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:07.001 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:07.001 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:07.001 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_port": { 2024-05-07T02:43:07.001 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_port", 2024-05-07T02:43:07.001 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-05-07T02:43:07.001 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:07.001 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:07.001 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10051", 2024-05-07T02:43:07.001 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:07.002 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:07.002 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:07.002 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:07.002 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:07.002 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:07.002 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:07.002 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:07.002 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_sender": { 2024-05-07T02:43:07.002 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_sender", 2024-05-07T02:43:07.003 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-05-07T02:43:07.003 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-05-07T02:43:07.003 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-05-07T02:43:07.003 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-07T02:43:07.003 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-05-07T02:43:07.003 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-05-07T02:43:07.003 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-05-07T02:43:07.003 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-05-07T02:43:07.003 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-05-07T02:43:07.003 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-05-07T02:43:07.004 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-05-07T02:43:07.004 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:07.004 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:07.004 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:07.004 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:07.004 INFO:teuthology.orchestra.run.smithi019.stdout: "services": { 2024-05-07T02:43:07.004 INFO:teuthology.orchestra.run.smithi019.stdout: "dashboard": "https://172.21.15.19:8443/" 2024-05-07T02:43:07.004 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:07.004 INFO:teuthology.orchestra.run.smithi019.stdout: "always_on_modules": { 2024-05-07T02:43:07.005 INFO:teuthology.orchestra.run.smithi019.stdout: "octopus": [ 2024-05-07T02:43:07.005 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-05-07T02:43:07.005 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-05-07T02:43:07.005 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-05-07T02:43:07.005 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-05-07T02:43:07.005 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-05-07T02:43:07.005 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-05-07T02:43:07.005 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-05-07T02:43:07.005 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-05-07T02:43:07.005 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-05-07T02:43:07.006 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-05-07T02:43:07.006 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:07.006 INFO:teuthology.orchestra.run.smithi019.stdout: "pacific": [ 2024-05-07T02:43:07.006 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-05-07T02:43:07.006 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-05-07T02:43:07.006 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-05-07T02:43:07.006 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-05-07T02:43:07.006 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-05-07T02:43:07.006 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-05-07T02:43:07.007 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-05-07T02:43:07.007 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-05-07T02:43:07.007 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-05-07T02:43:07.007 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-05-07T02:43:07.007 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:07.007 INFO:teuthology.orchestra.run.smithi019.stdout: "quincy": [ 2024-05-07T02:43:07.007 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-05-07T02:43:07.007 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-05-07T02:43:07.007 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-05-07T02:43:07.007 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-05-07T02:43:07.008 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-05-07T02:43:07.008 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-05-07T02:43:07.008 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-05-07T02:43:07.008 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-05-07T02:43:07.008 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-05-07T02:43:07.008 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-05-07T02:43:07.008 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:07.008 INFO:teuthology.orchestra.run.smithi019.stdout: "reef": [ 2024-05-07T02:43:07.008 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-05-07T02:43:07.009 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-05-07T02:43:07.009 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-05-07T02:43:07.009 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-05-07T02:43:07.009 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-05-07T02:43:07.009 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-05-07T02:43:07.009 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-05-07T02:43:07.009 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-05-07T02:43:07.009 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-05-07T02:43:07.009 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-05-07T02:43:07.010 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-05-07T02:43:07.010 INFO:teuthology.orchestra.run.smithi019.stdout: "squid": [ 2024-05-07T02:43:07.010 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-05-07T02:43:07.010 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-05-07T02:43:07.010 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-05-07T02:43:07.010 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-05-07T02:43:07.010 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-05-07T02:43:07.010 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-05-07T02:43:07.010 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-05-07T02:43:07.011 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-05-07T02:43:07.011 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-05-07T02:43:07.011 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-05-07T02:43:07.011 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-05-07T02:43:07.011 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:07.011 INFO:teuthology.orchestra.run.smithi019.stdout: "last_failure_osd_epoch": 90, 2024-05-07T02:43:07.011 INFO:teuthology.orchestra.run.smithi019.stdout: "active_clients": [ 2024-05-07T02:43:07.011 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:07.011 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "devicehealth", 2024-05-07T02:43:07.012 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-05-07T02:43:07.012 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:07.012 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-05-07T02:43:07.012 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-05-07T02:43:07.012 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3078448930 2024-05-07T02:43:07.012 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:07.012 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-05-07T02:43:07.012 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:07.012 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:07.012 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "libcephsqlite", 2024-05-07T02:43:07.013 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-05-07T02:43:07.013 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:07.013 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-05-07T02:43:07.013 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-05-07T02:43:07.013 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 4174441237 2024-05-07T02:43:07.013 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:07.013 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-05-07T02:43:07.013 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:07.013 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:07.014 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_support", 2024-05-07T02:43:07.014 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-05-07T02:43:07.014 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:07.014 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-05-07T02:43:07.014 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-05-07T02:43:07.014 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1881782964 2024-05-07T02:43:07.014 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:07.014 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-05-07T02:43:07.014 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:07.015 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:07.015 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snap_schedule", 2024-05-07T02:43:07.015 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-05-07T02:43:07.015 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:07.015 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-05-07T02:43:07.015 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-05-07T02:43:07.015 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3388887101 2024-05-07T02:43:07.015 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:07.015 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-05-07T02:43:07.016 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:07.016 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:07.016 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snap_schedule", 2024-05-07T02:43:07.016 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-05-07T02:43:07.016 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:07.016 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-05-07T02:43:07.016 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-05-07T02:43:07.016 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 2333792422 2024-05-07T02:43:07.016 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:07.017 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-05-07T02:43:07.017 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:07.017 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:07.017 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "volumes", 2024-05-07T02:43:07.017 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-05-07T02:43:07.017 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-05-07T02:43:07.017 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-05-07T02:43:07.017 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-05-07T02:43:07.017 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 923015436 2024-05-07T02:43:07.017 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:07.018 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-05-07T02:43:07.018 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:07.018 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-05-07T02:43:07.018 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:43:07.019 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:43:05 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/4174476187' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-07T02:43:07.019 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:43:05 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/3964688001' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-07T02:43:07.020 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:43:06 smithi027 ceph-mon[34398]: pgmap v348: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-05-07T02:43:07.020 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:43:05 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/4174476187' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-07T02:43:07.020 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:43:05 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/3964688001' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-07T02:43:07.020 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:43:06 smithi064 ceph-mon[34699]: pgmap v348: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-05-07T02:43:07.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.783+0000 7fd123e00640 1 -- 172.21.15.19:0/3964688001 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fd1080b2a10 msgr2=0x7fd1080b4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:07.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.783+0000 7fd123e00640 1 --2- 172.21.15.19:0/3964688001 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fd1080b2a10 0x7fd1080b4f20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd1140096c0 tx=0x7fd114005a20 comp rx=0 tx=0).stop 2024-05-07T02:43:07.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.783+0000 7fd123e00640 1 -- 172.21.15.19:0/3964688001 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd124110270 msgr2=0x7fd1241a0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:07.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.783+0000 7fd123e00640 1 --2- 172.21.15.19:0/3964688001 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd124110270 0x7fd1241a0ff0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd11c009fd0 tx=0x7fd11c00ef10 comp rx=0 tx=0).stop 2024-05-07T02:43:07.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.783+0000 7fd123e00640 1 -- 172.21.15.19:0/3964688001 shutdown_connections 2024-05-07T02:43:07.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.783+0000 7fd123e00640 1 --2- 172.21.15.19:0/3964688001 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fd1080b2a10 0x7fd1080b4f20 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:07.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.783+0000 7fd123e00640 1 --2- 172.21.15.19:0/3964688001 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd124110270 0x7fd1241a0ff0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:07.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.783+0000 7fd123e00640 1 --2- 172.21.15.19:0/3964688001 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd12410d910 0x7fd12419cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:07.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.783+0000 7fd123e00640 1 --2- 172.21.15.19:0/3964688001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd124075670 0x7fd12419c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:07.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.783+0000 7fd123e00640 1 -- 172.21.15.19:0/3964688001 >> 172.21.15.19:0/3964688001 conn(0x7fd12407a190 msgr2=0x7fd1241021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:43:07.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.784+0000 7fd123e00640 1 -- 172.21.15.19:0/3964688001 shutdown_connections 2024-05-07T02:43:07.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:04.784+0000 7fd123e00640 1 -- 172.21.15.19:0/3964688001 wait complete. 2024-05-07T02:43:07.030 DEBUG:tasks.check_counter:Getting stats from a 2024-05-07T02:43:07.030 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-07T02:43:07.031 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:43:05 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/4174476187' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-07T02:43:07.031 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:43:05 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/3964688001' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-07T02:43:07.031 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:43:06 smithi019 ceph-mon[29781]: pgmap v348: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-05-07T02:43:07.567 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:43:08.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:43:08 smithi064 ceph-mon[34699]: pgmap v349: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-05-07T02:43:08.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:43:08 smithi019 ceph-mon[29781]: pgmap v349: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-05-07T02:43:08.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:43:08 smithi027 ceph-mon[34398]: pgmap v349: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-05-07T02:43:10.175 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:43:10 smithi019 ceph-mon[29781]: pgmap v350: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-05-07T02:43:10.175 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:43:10 smithi019 ceph-mon[29781]: osdmap e99: 12 total, 12 up, 12 in 2024-05-07T02:43:10.180 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:43:10.180 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-05-07T02:43:10.181 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 19913, 2024-05-07T02:43:10.181 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 31677, 2024-05-07T02:43:10.181 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 6757394, 2024-05-07T02:43:10.181 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 24167292, 2024-05-07T02:43:10.181 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 13, 2024-05-07T02:43:10.181 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 7, 2024-05-07T02:43:10.181 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 7.793747557, 2024-05-07T02:43:10.181 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 4.471141187, 2024-05-07T02:43:10.182 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 3.728050964, 2024-05-07T02:43:10.182 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.179041620, 2024-05-07T02:43:10.182 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-07T02:43:10.182 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 31671, 2024-05-07T02:43:10.182 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 2.890501145, 2024-05-07T02:43:10.182 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000091266 2024-05-07T02:43:10.182 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.182 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-07T02:43:10.183 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 41045, 2024-05-07T02:43:10.183 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.034317811, 2024-05-07T02:43:10.183 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000836 2024-05-07T02:43:10.183 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.183 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 928, 2024-05-07T02:43:10.183 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 976 2024-05-07T02:43:10.183 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.183 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-05-07T02:43:10.183 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 4115, 2024-05-07T02:43:10.184 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 4327, 2024-05-07T02:43:10.184 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 1204135, 2024-05-07T02:43:10.184 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 14671589, 2024-05-07T02:43:10.184 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 19, 2024-05-07T02:43:10.184 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 7, 2024-05-07T02:43:10.184 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.916548640, 2024-05-07T02:43:10.184 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.383482136, 2024-05-07T02:43:10.184 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.224082758, 2024-05-07T02:43:10.185 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.237965412, 2024-05-07T02:43:10.185 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-07T02:43:10.185 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 4323, 2024-05-07T02:43:10.185 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.552519129, 2024-05-07T02:43:10.185 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000127809 2024-05-07T02:43:10.185 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.185 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-07T02:43:10.185 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 265, 2024-05-07T02:43:10.186 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000146916, 2024-05-07T02:43:10.186 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000554 2024-05-07T02:43:10.186 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.186 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-05-07T02:43:10.186 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 511264 2024-05-07T02:43:10.186 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.186 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-05-07T02:43:10.186 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 30281, 2024-05-07T02:43:10.186 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 26669, 2024-05-07T02:43:10.187 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 10783107, 2024-05-07T02:43:10.187 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 6916525, 2024-05-07T02:43:10.187 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 25, 2024-05-07T02:43:10.187 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 6, 2024-05-07T02:43:10.187 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 6.126267441, 2024-05-07T02:43:10.187 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 2.704798199, 2024-05-07T02:43:10.187 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 4.218025637, 2024-05-07T02:43:10.187 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.089001280, 2024-05-07T02:43:10.188 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-07T02:43:10.188 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 26674, 2024-05-07T02:43:10.188 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 2.123580866, 2024-05-07T02:43:10.188 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000079612 2024-05-07T02:43:10.188 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.188 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-07T02:43:10.188 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 53587, 2024-05-07T02:43:10.188 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.027417570, 2024-05-07T02:43:10.188 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000511 2024-05-07T02:43:10.189 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.189 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 1932736, 2024-05-07T02:43:10.189 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 186672 2024-05-07T02:43:10.189 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.189 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-05-07T02:43:10.189 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 1, 2024-05-07T02:43:10.189 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-05-07T02:43:10.189 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.189 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-MDSRank": { 2024-05-07T02:43:10.189 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-05-07T02:43:10.190 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-05-07T02:43:10.190 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 5472, 2024-05-07T02:43:10.190 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 8.530481406, 2024-05-07T02:43:10.190 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.001558933 2024-05-07T02:43:10.190 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:10.190 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.190 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-PurgeQueue": { 2024-05-07T02:43:10.190 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-05-07T02:43:10.191 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-05-07T02:43:10.191 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 4557, 2024-05-07T02:43:10.191 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.884134532, 2024-05-07T02:43:10.191 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000194016 2024-05-07T02:43:10.191 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:10.191 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.191 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-05-07T02:43:10.191 INFO:teuthology.orchestra.run.smithi019.stdout: "request": 14157, 2024-05-07T02:43:10.191 INFO:teuthology.orchestra.run.smithi019.stdout: "reply": 13607, 2024-05-07T02:43:10.192 INFO:teuthology.orchestra.run.smithi019.stdout: "reply_latency": { 2024-05-07T02:43:10.192 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 13607, 2024-05-07T02:43:10.192 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 28.855466597, 2024-05-07T02:43:10.192 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.002120633 2024-05-07T02:43:10.192 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.192 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_reply": 0, 2024-05-07T02:43:10.192 INFO:teuthology.orchestra.run.smithi019.stdout: "forward": 550, 2024-05-07T02:43:10.192 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_complete": 0, 2024-05-07T02:43:10.193 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_keys": 0, 2024-05-07T02:43:10.193 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_commit": 11, 2024-05-07T02:43:10.193 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_split": 0, 2024-05-07T02:43:10.193 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_merge": 0, 2024-05-07T02:43:10.193 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes": 83, 2024-05-07T02:43:10.193 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_top": 15, 2024-05-07T02:43:10.193 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_bottom": 14, 2024-05-07T02:43:10.193 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pin_tail": 54, 2024-05-07T02:43:10.193 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pinned": 61, 2024-05-07T02:43:10.194 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_expired": 27, 2024-05-07T02:43:10.194 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_with_caps": 2, 2024-05-07T02:43:10.194 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": 2, 2024-05-07T02:43:10.194 INFO:teuthology.orchestra.run.smithi019.stdout: "subtrees": 4, 2024-05-07T02:43:10.194 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse": 28057, 2024-05-07T02:43:10.194 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_hit": 10813, 2024-05-07T02:43:10.194 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_forward": 549, 2024-05-07T02:43:10.194 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_discover": 4, 2024-05-07T02:43:10.195 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_dir_fetch": 0, 2024-05-07T02:43:10.195 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_remote_ino": 0, 2024-05-07T02:43:10.195 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_lock": 0, 2024-05-07T02:43:10.195 INFO:teuthology.orchestra.run.smithi019.stdout: "load_cent": 0, 2024-05-07T02:43:10.195 INFO:teuthology.orchestra.run.smithi019.stdout: "q": 0, 2024-05-07T02:43:10.195 INFO:teuthology.orchestra.run.smithi019.stdout: "exported": 1, 2024-05-07T02:43:10.195 INFO:teuthology.orchestra.run.smithi019.stdout: "exported_inodes": 25, 2024-05-07T02:43:10.195 INFO:teuthology.orchestra.run.smithi019.stdout: "imported": 2, 2024-05-07T02:43:10.195 INFO:teuthology.orchestra.run.smithi019.stdout: "imported_inodes": 75, 2024-05-07T02:43:10.196 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_dir_fetch": 0, 2024-05-07T02:43:10.196 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_backtrace_fetch": 28, 2024-05-07T02:43:10.196 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_peer_discover": 0, 2024-05-07T02:43:10.196 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rfiles": 67, 2024-05-07T02:43:10.196 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rbytes": 21176629, 2024-05-07T02:43:10.196 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rsnaps": 6, 2024-05-07T02:43:10.196 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_fetch": 28, 2024-05-07T02:43:10.196 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_set_tag": 0, 2024-05-07T02:43:10.196 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_repaired": 0, 2024-05-07T02:43:10.197 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_inotable_repaired": 0, 2024-05-07T02:43:10.197 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_inodes": 0, 2024-05-07T02:43:10.197 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_base_inodes": 0, 2024-05-07T02:43:10.197 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dirfrag_rstats": 0, 2024-05-07T02:43:10.197 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_file_inodes": 28, 2024-05-07T02:43:10.197 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_inode_file_caps": 0, 2024-05-07T02:43:10.197 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_revoke": 397, 2024-05-07T02:43:10.197 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_grant": 7620, 2024-05-07T02:43:10.197 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_trunc": 0, 2024-05-07T02:43:10.198 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-07T02:43:10.198 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-07T02:43:10.198 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps": 872, 2024-05-07T02:43:10.198 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps_dirty": 88, 2024-05-07T02:43:10.198 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_cap_release": 138, 2024-05-07T02:43:10.198 INFO:teuthology.orchestra.run.smithi019.stdout: "process_request_cap_release": 12974 2024-05-07T02:43:10.198 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.198 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_cache": { 2024-05-07T02:43:10.199 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update": 3, 2024-05-07T02:43:10.199 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update_receipt": 6, 2024-05-07T02:43:10.199 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_try_discover": 4, 2024-05-07T02:43:10.199 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_send_discover": 7, 2024-05-07T02:43:10.199 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_handle_discover": 3, 2024-05-07T02:43:10.199 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays": 0, 2024-05-07T02:43:10.199 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_delayed": 0, 2024-05-07T02:43:10.199 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_enqueuing": 0, 2024-05-07T02:43:10.199 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_created": 2312, 2024-05-07T02:43:10.200 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_enqueued": 2312, 2024-05-07T02:43:10.200 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_reintegrated": 0, 2024-05-07T02:43:10.200 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_migrated": 0, 2024-05-07T02:43:10.200 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_processing": 0, 2024-05-07T02:43:10.200 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_enqueued": 0, 2024-05-07T02:43:10.200 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_prioritized": 0, 2024-05-07T02:43:10.200 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_started": 0, 2024-05-07T02:43:10.200 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_completed": 0, 2024-05-07T02:43:10.201 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_quiesce_path": 0, 2024-05-07T02:43:10.201 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_quiesce_inode": 0, 2024-05-07T02:43:10.201 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_enqueue_scrub": 0, 2024-05-07T02:43:10.201 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_exportdir": 1, 2024-05-07T02:43:10.201 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_flush": 0, 2024-05-07T02:43:10.201 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragmentdir": 0, 2024-05-07T02:43:10.201 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragstats": 0, 2024-05-07T02:43:10.201 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_inodestats": 0 2024-05-07T02:43:10.201 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.202 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_log": { 2024-05-07T02:43:10.202 INFO:teuthology.orchestra.run.smithi019.stdout: "evlrg": 0, 2024-05-07T02:43:10.202 INFO:teuthology.orchestra.run.smithi019.stdout: "evadd": 9827, 2024-05-07T02:43:10.202 INFO:teuthology.orchestra.run.smithi019.stdout: "evex": 0, 2024-05-07T02:43:10.202 INFO:teuthology.orchestra.run.smithi019.stdout: "evtrm": 0, 2024-05-07T02:43:10.202 INFO:teuthology.orchestra.run.smithi019.stdout: "ev": 9827, 2024-05-07T02:43:10.202 INFO:teuthology.orchestra.run.smithi019.stdout: "evexg": 0, 2024-05-07T02:43:10.202 INFO:teuthology.orchestra.run.smithi019.stdout: "evexd": 0, 2024-05-07T02:43:10.203 INFO:teuthology.orchestra.run.smithi019.stdout: "segadd": 11, 2024-05-07T02:43:10.203 INFO:teuthology.orchestra.run.smithi019.stdout: "segex": 0, 2024-05-07T02:43:10.203 INFO:teuthology.orchestra.run.smithi019.stdout: "segtrm": 0, 2024-05-07T02:43:10.203 INFO:teuthology.orchestra.run.smithi019.stdout: "seg": 11, 2024-05-07T02:43:10.203 INFO:teuthology.orchestra.run.smithi019.stdout: "segmjr": 2, 2024-05-07T02:43:10.203 INFO:teuthology.orchestra.run.smithi019.stdout: "segexg": 0, 2024-05-07T02:43:10.203 INFO:teuthology.orchestra.run.smithi019.stdout: "segexd": 0, 2024-05-07T02:43:10.203 INFO:teuthology.orchestra.run.smithi019.stdout: "expos": 4194304, 2024-05-07T02:43:10.203 INFO:teuthology.orchestra.run.smithi019.stdout: "wrpos": 18082553, 2024-05-07T02:43:10.204 INFO:teuthology.orchestra.run.smithi019.stdout: "rdpos": 0, 2024-05-07T02:43:10.204 INFO:teuthology.orchestra.run.smithi019.stdout: "jlat": { 2024-05-07T02:43:10.204 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2597, 2024-05-07T02:43:10.204 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 7.535949505, 2024-05-07T02:43:10.204 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.002901790 2024-05-07T02:43:10.204 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.204 INFO:teuthology.orchestra.run.smithi019.stdout: "replayed": 0 2024-05-07T02:43:10.204 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.204 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_mem": { 2024-05-07T02:43:10.205 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 52, 2024-05-07T02:43:10.205 INFO:teuthology.orchestra.run.smithi019.stdout: "ino+": 2386, 2024-05-07T02:43:10.205 INFO:teuthology.orchestra.run.smithi019.stdout: "ino-": 2334, 2024-05-07T02:43:10.205 INFO:teuthology.orchestra.run.smithi019.stdout: "dir": 28, 2024-05-07T02:43:10.205 INFO:teuthology.orchestra.run.smithi019.stdout: "dir+": 35, 2024-05-07T02:43:10.205 INFO:teuthology.orchestra.run.smithi019.stdout: "dir-": 7, 2024-05-07T02:43:10.205 INFO:teuthology.orchestra.run.smithi019.stdout: "dn": 83, 2024-05-07T02:43:10.205 INFO:teuthology.orchestra.run.smithi019.stdout: "dn+": 3702, 2024-05-07T02:43:10.206 INFO:teuthology.orchestra.run.smithi019.stdout: "dn-": 3619, 2024-05-07T02:43:10.206 INFO:teuthology.orchestra.run.smithi019.stdout: "cap": 2, 2024-05-07T02:43:10.206 INFO:teuthology.orchestra.run.smithi019.stdout: "cap+": 2396, 2024-05-07T02:43:10.206 INFO:teuthology.orchestra.run.smithi019.stdout: "cap-": 2394, 2024-05-07T02:43:10.206 INFO:teuthology.orchestra.run.smithi019.stdout: "rss": 125604, 2024-05-07T02:43:10.206 INFO:teuthology.orchestra.run.smithi019.stdout: "heap": 223632 2024-05-07T02:43:10.206 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.206 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_server": { 2024-05-07T02:43:10.207 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_client_request": 25606, 2024-05-07T02:43:10.207 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_server_request": 0, 2024-05-07T02:43:10.207 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_request": 14157, 2024-05-07T02:43:10.207 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_session": 13, 2024-05-07T02:43:10.207 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_peer_request": 4630, 2024-05-07T02:43:10.207 INFO:teuthology.orchestra.run.smithi019.stdout: "req_create_latency": { 2024-05-07T02:43:10.207 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2312, 2024-05-07T02:43:10.207 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 12.130793828, 2024-05-07T02:43:10.207 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.005246883 2024-05-07T02:43:10.207 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.208 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getattr_latency": { 2024-05-07T02:43:10.208 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 70, 2024-05-07T02:43:10.208 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.125243271, 2024-05-07T02:43:10.208 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.001789189 2024-05-07T02:43:10.208 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.208 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getfilelock_latency": { 2024-05-07T02:43:10.208 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:10.208 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:10.209 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:10.209 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.209 INFO:teuthology.orchestra.run.smithi019.stdout: "req_link_latency": { 2024-05-07T02:43:10.209 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:10.209 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:10.209 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:10.209 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.209 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookup_latency": { 2024-05-07T02:43:10.209 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2873, 2024-05-07T02:43:10.209 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 1.651752003, 2024-05-07T02:43:10.210 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000574922 2024-05-07T02:43:10.210 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.210 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookuphash_latency": { 2024-05-07T02:43:10.210 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:10.210 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:10.210 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:10.210 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.210 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupino_latency": { 2024-05-07T02:43:10.211 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:10.211 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:10.211 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:10.211 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.211 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupname_latency": { 2024-05-07T02:43:10.211 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:10.211 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:10.211 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:10.212 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.212 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupparent_latency": { 2024-05-07T02:43:10.212 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:10.212 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:10.212 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:10.212 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.212 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupsnap_latency": { 2024-05-07T02:43:10.212 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:10.213 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:10.213 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:10.213 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.213 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lssnap_latency": { 2024-05-07T02:43:10.213 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:10.214 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:10.214 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:10.214 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.214 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mkdir_latency": { 2024-05-07T02:43:10.214 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:10.214 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:10.214 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:10.215 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.215 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mknod_latency": { 2024-05-07T02:43:10.215 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:10.215 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:10.215 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:10.215 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.215 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mksnap_latency": { 2024-05-07T02:43:10.215 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:10.215 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:10.216 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:10.216 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.216 INFO:teuthology.orchestra.run.smithi019.stdout: "req_open_latency": { 2024-05-07T02:43:10.216 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:10.216 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:10.216 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:10.216 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.216 INFO:teuthology.orchestra.run.smithi019.stdout: "req_readdir_latency": { 2024-05-07T02:43:10.217 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 6040, 2024-05-07T02:43:10.217 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 5.973678723, 2024-05-07T02:43:10.217 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000989019 2024-05-07T02:43:10.217 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.217 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rename_latency": { 2024-05-07T02:43:10.217 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:10.217 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:10.217 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:10.218 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.218 INFO:teuthology.orchestra.run.smithi019.stdout: "req_renamesnap_latency": { 2024-05-07T02:43:10.218 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:10.218 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:10.218 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:10.218 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.218 INFO:teuthology.orchestra.run.smithi019.stdout: "req_snapdiff_latency": { 2024-05-07T02:43:10.218 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:10.218 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:10.219 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:10.219 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.219 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmdir_latency": { 2024-05-07T02:43:10.219 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:10.219 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:10.219 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:10.219 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.219 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmsnap_latency": { 2024-05-07T02:43:10.219 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:10.220 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:10.220 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:10.220 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.220 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmxattr_latency": { 2024-05-07T02:43:10.220 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:10.220 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:10.220 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:10.220 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.221 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setattr_latency": { 2024-05-07T02:43:10.221 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:10.221 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:10.221 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:10.221 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.221 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setdirlayout_latency": { 2024-05-07T02:43:10.221 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:10.221 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:10.221 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:10.222 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.222 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setfilelock_latency": { 2024-05-07T02:43:10.222 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:10.222 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:10.222 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:10.222 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.222 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setlayout_latency": { 2024-05-07T02:43:10.222 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:10.222 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:10.223 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:10.223 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.223 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setxattr_latency": { 2024-05-07T02:43:10.223 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:10.223 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:10.223 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:10.223 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.223 INFO:teuthology.orchestra.run.smithi019.stdout: "req_symlink_latency": { 2024-05-07T02:43:10.224 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:10.224 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:10.224 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:10.224 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.224 INFO:teuthology.orchestra.run.smithi019.stdout: "req_unlink_latency": { 2024-05-07T02:43:10.224 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2312, 2024-05-07T02:43:10.224 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 8.973998772, 2024-05-07T02:43:10.224 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.003881487 2024-05-07T02:43:10.224 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.225 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_revoke_eviction": 0, 2024-05-07T02:43:10.225 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_acquisition_throttle": 0, 2024-05-07T02:43:10.225 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getvxattr_latency": { 2024-05-07T02:43:10.225 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:10.225 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:10.225 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:10.225 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:10.225 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.226 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_sessions": { 2024-05-07T02:43:10.226 INFO:teuthology.orchestra.run.smithi019.stdout: "session_count": 1, 2024-05-07T02:43:10.226 INFO:teuthology.orchestra.run.smithi019.stdout: "session_add": 2, 2024-05-07T02:43:10.226 INFO:teuthology.orchestra.run.smithi019.stdout: "session_remove": 1, 2024-05-07T02:43:10.226 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_open": 1, 2024-05-07T02:43:10.226 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_stale": 0, 2024-05-07T02:43:10.226 INFO:teuthology.orchestra.run.smithi019.stdout: "total_load": 3853, 2024-05-07T02:43:10.226 INFO:teuthology.orchestra.run.smithi019.stdout: "average_load": 3853, 2024-05-07T02:43:10.226 INFO:teuthology.orchestra.run.smithi019.stdout: "avg_session_uptime": 596, 2024-05-07T02:43:10.227 INFO:teuthology.orchestra.run.smithi019.stdout: "mdthresh_evicted": 0 2024-05-07T02:43:10.227 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.227 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-05-07T02:43:10.227 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-05-07T02:43:10.227 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-05-07T02:43:10.227 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-05-07T02:43:10.227 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-05-07T02:43:10.227 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-05-07T02:43:10.227 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-05-07T02:43:10.228 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-07T02:43:10.228 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-05-07T02:43:10.228 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-07T02:43:10.228 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-05-07T02:43:10.228 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-05-07T02:43:10.228 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-05-07T02:43:10.228 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-07T02:43:10.228 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-05-07T02:43:10.228 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-05-07T02:43:10.229 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-05-07T02:43:10.229 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-05-07T02:43:10.229 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-05-07T02:43:10.229 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-07T02:43:10.229 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-05-07T02:43:10.229 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-07T02:43:10.229 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-05-07T02:43:10.229 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-05-07T02:43:10.229 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-05-07T02:43:10.230 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-05-07T02:43:10.230 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-05-07T02:43:10.230 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-07T02:43:10.230 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-05-07T02:43:10.230 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-05-07T02:43:10.230 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-05-07T02:43:10.230 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-05-07T02:43:10.230 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-05-07T02:43:10.231 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-05-07T02:43:10.231 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-05-07T02:43:10.231 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-05-07T02:43:10.231 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-05-07T02:43:10.231 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 293781, 2024-05-07T02:43:10.231 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 137, 2024-05-07T02:43:10.231 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 176, 2024-05-07T02:43:10.231 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 2, 2024-05-07T02:43:10.233 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-05-07T02:43:10.233 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-05-07T02:43:10.233 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-05-07T02:43:10.233 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-05-07T02:43:10.233 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-05-07T02:43:10.233 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-05-07T02:43:10.233 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 9608, 2024-05-07T02:43:10.234 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 184, 2024-05-07T02:43:10.234 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-05-07T02:43:10.234 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-05-07T02:43:10.234 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-05-07T02:43:10.234 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-05-07T02:43:10.234 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 250397, 2024-05-07T02:43:10.234 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 1546, 2024-05-07T02:43:10.234 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-05-07T02:43:10.235 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-05-07T02:43:10.235 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-05-07T02:43:10.235 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-05-07T02:43:10.235 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.235 INFO:teuthology.orchestra.run.smithi019.stdout: "objecter": { 2024-05-07T02:43:10.235 INFO:teuthology.orchestra.run.smithi019.stdout: "op_active": 0, 2024-05-07T02:43:10.235 INFO:teuthology.orchestra.run.smithi019.stdout: "op_laggy": 0, 2024-05-07T02:43:10.235 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send": 8081, 2024-05-07T02:43:10.235 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send_bytes": 14357098, 2024-05-07T02:43:10.236 INFO:teuthology.orchestra.run.smithi019.stdout: "op_resend": 33, 2024-05-07T02:43:10.236 INFO:teuthology.orchestra.run.smithi019.stdout: "op_reply": 8081, 2024-05-07T02:43:10.236 INFO:teuthology.orchestra.run.smithi019.stdout: "op_latency": { 2024-05-07T02:43:10.236 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 8081, 2024-05-07T02:43:10.236 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 32.288670672, 2024-05-07T02:43:10.236 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.003995628 2024-05-07T02:43:10.236 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.236 INFO:teuthology.orchestra.run.smithi019.stdout: "op_inflight": 0, 2024-05-07T02:43:10.236 INFO:teuthology.orchestra.run.smithi019.stdout: "oplen_avg": { 2024-05-07T02:43:10.237 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 8081, 2024-05-07T02:43:10.237 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 8411 2024-05-07T02:43:10.237 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.237 INFO:teuthology.orchestra.run.smithi019.stdout: "op": 8081, 2024-05-07T02:43:10.237 INFO:teuthology.orchestra.run.smithi019.stdout: "op_r": 245, 2024-05-07T02:43:10.237 INFO:teuthology.orchestra.run.smithi019.stdout: "op_w": 7836, 2024-05-07T02:43:10.237 INFO:teuthology.orchestra.run.smithi019.stdout: "op_rmw": 0, 2024-05-07T02:43:10.237 INFO:teuthology.orchestra.run.smithi019.stdout: "op_pg": 0, 2024-05-07T02:43:10.237 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_stat": 0, 2024-05-07T02:43:10.238 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_create": 10, 2024-05-07T02:43:10.238 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_read": 217, 2024-05-07T02:43:10.238 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_write": 2814, 2024-05-07T02:43:10.238 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writefull": 144, 2024-05-07T02:43:10.238 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writesame": 0, 2024-05-07T02:43:10.238 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_append": 0, 2024-05-07T02:43:10.238 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_zero": 0, 2024-05-07T02:43:10.238 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_truncate": 0, 2024-05-07T02:43:10.239 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_delete": 4639, 2024-05-07T02:43:10.239 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_mapext": 0, 2024-05-07T02:43:10.239 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_sparse_read": 0, 2024-05-07T02:43:10.239 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_clonerange": 0, 2024-05-07T02:43:10.239 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_getxattr": 28, 2024-05-07T02:43:10.239 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_setxattr": 20, 2024-05-07T02:43:10.239 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_cmpxattr": 0, 2024-05-07T02:43:10.239 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_rmxattr": 0, 2024-05-07T02:43:10.239 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_resetxattrs": 0, 2024-05-07T02:43:10.240 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_call": 0, 2024-05-07T02:43:10.240 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_watch": 0, 2024-05-07T02:43:10.240 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_notify": 0, 2024-05-07T02:43:10.240 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_src_cmpxattr": 0, 2024-05-07T02:43:10.240 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls": 0, 2024-05-07T02:43:10.240 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls_filter": 0, 2024-05-07T02:43:10.240 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_other": 0, 2024-05-07T02:43:10.240 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_active": 0, 2024-05-07T02:43:10.241 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_send": 0, 2024-05-07T02:43:10.241 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_resend": 0, 2024-05-07T02:43:10.241 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_ping": 0, 2024-05-07T02:43:10.241 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_active": 0, 2024-05-07T02:43:10.241 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_send": 0, 2024-05-07T02:43:10.241 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_resend": 0, 2024-05-07T02:43:10.241 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_active": 0, 2024-05-07T02:43:10.241 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_send": 0, 2024-05-07T02:43:10.241 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_resend": 0, 2024-05-07T02:43:10.242 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_active": 0, 2024-05-07T02:43:10.242 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_send": 0, 2024-05-07T02:43:10.242 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_resend": 0, 2024-05-07T02:43:10.242 INFO:teuthology.orchestra.run.smithi019.stdout: "command_active": 0, 2024-05-07T02:43:10.242 INFO:teuthology.orchestra.run.smithi019.stdout: "command_send": 0, 2024-05-07T02:43:10.242 INFO:teuthology.orchestra.run.smithi019.stdout: "command_resend": 0, 2024-05-07T02:43:10.242 INFO:teuthology.orchestra.run.smithi019.stdout: "map_epoch": 99, 2024-05-07T02:43:10.242 INFO:teuthology.orchestra.run.smithi019.stdout: "map_full": 0, 2024-05-07T02:43:10.242 INFO:teuthology.orchestra.run.smithi019.stdout: "map_inc": 12, 2024-05-07T02:43:10.243 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_sessions": 12, 2024-05-07T02:43:10.243 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_open": 12, 2024-05-07T02:43:10.243 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_close": 0, 2024-05-07T02:43:10.243 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_laggy": 0, 2024-05-07T02:43:10.243 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_wr": 402, 2024-05-07T02:43:10.243 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_rd": 0, 2024-05-07T02:43:10.243 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_del": 137 2024-05-07T02:43:10.243 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.243 INFO:teuthology.orchestra.run.smithi019.stdout: "oft": { 2024-05-07T02:43:10.244 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_objs": 1, 2024-05-07T02:43:10.244 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_kv_pairs": 0, 2024-05-07T02:43:10.244 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_updates": 0, 2024-05-07T02:43:10.244 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_removes": 0 2024-05-07T02:43:10.244 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.244 INFO:teuthology.orchestra.run.smithi019.stdout: "osd-slow-ops": { 2024-05-07T02:43:10.244 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_ops_count": 0 2024-05-07T02:43:10.244 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.244 INFO:teuthology.orchestra.run.smithi019.stdout: "purge_queue": { 2024-05-07T02:43:10.245 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops": 0, 2024-05-07T02:43:10.245 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops_high_water": 10, 2024-05-07T02:43:10.245 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing": 0, 2024-05-07T02:43:10.245 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_high_water": 5, 2024-05-07T02:43:10.245 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed_ops": 4624, 2024-05-07T02:43:10.245 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed": 2312, 2024-05-07T02:43:10.245 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_item_in_journal": 0 2024-05-07T02:43:10.245 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.246 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-07T02:43:10.246 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-05-07T02:43:10.246 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-05-07T02:43:10.246 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-05-07T02:43:10.246 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 54305, 2024-05-07T02:43:10.246 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 13834885, 2024-05-07T02:43:10.246 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:10.246 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 54305, 2024-05-07T02:43:10.246 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-05-07T02:43:10.247 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-05-07T02:43:10.247 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 54305, 2024-05-07T02:43:10.247 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 13834885, 2024-05-07T02:43:10.247 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-05-07T02:43:10.247 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:10.247 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:10.247 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:10.247 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:10.248 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.248 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_bytes": { 2024-05-07T02:43:10.248 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-05-07T02:43:10.248 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-05-07T02:43:10.248 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-05-07T02:43:10.248 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-05-07T02:43:10.248 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-05-07T02:43:10.248 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:10.249 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-05-07T02:43:10.249 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 8081, 2024-05-07T02:43:10.249 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 14609130, 2024-05-07T02:43:10.249 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 8081, 2024-05-07T02:43:10.249 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 14609130, 2024-05-07T02:43:10.249 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-05-07T02:43:10.249 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:10.249 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:10.249 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:10.250 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:10.250 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.250 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_ops": { 2024-05-07T02:43:10.250 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-05-07T02:43:10.250 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 1024, 2024-05-07T02:43:10.250 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-05-07T02:43:10.250 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-05-07T02:43:10.250 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-05-07T02:43:10.250 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:10.251 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-05-07T02:43:10.251 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 8081, 2024-05-07T02:43:10.251 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 8081, 2024-05-07T02:43:10.251 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 8081, 2024-05-07T02:43:10.251 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 8081, 2024-05-07T02:43:10.251 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-05-07T02:43:10.251 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:10.251 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:10.252 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:10.252 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:10.252 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.252 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle": { 2024-05-07T02:43:10.252 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-05-07T02:43:10.252 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-05-07T02:43:10.252 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-05-07T02:43:10.252 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 2312, 2024-05-07T02:43:10.253 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 252032, 2024-05-07T02:43:10.253 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:10.253 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 2312, 2024-05-07T02:43:10.253 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-05-07T02:43:10.253 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-05-07T02:43:10.253 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 217, 2024-05-07T02:43:10.253 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 252032, 2024-05-07T02:43:10.254 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-05-07T02:43:10.254 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:10.254 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:10.254 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:10.254 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:10.254 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:10.254 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle-0x55778d4445b0": { 2024-05-07T02:43:10.254 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-05-07T02:43:10.254 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-05-07T02:43:10.255 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-05-07T02:43:10.255 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 9827, 2024-05-07T02:43:10.255 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 13888249, 2024-05-07T02:43:10.255 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:10.255 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 9827, 2024-05-07T02:43:10.255 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-05-07T02:43:10.255 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-05-07T02:43:10.255 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 2597, 2024-05-07T02:43:10.255 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 13888249, 2024-05-07T02:43:10.256 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-05-07T02:43:10.256 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:10.256 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:10.256 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:10.256 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:10.256 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:10.256 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:43:10.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:43:10 smithi064 ceph-mon[34699]: pgmap v350: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-05-07T02:43:10.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:43:10 smithi064 ceph-mon[34699]: osdmap e99: 12 total, 12 up, 12 in 2024-05-07T02:43:10.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:43:10 smithi027 ceph-mon[34398]: pgmap v350: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-05-07T02:43:10.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:43:10 smithi027 ceph-mon[34398]: osdmap e99: 12 total, 12 up, 12 in 2024-05-07T02:43:11.337 INFO:tasks.check_counter:Daemon mds.a mds.exported=1 2024-05-07T02:43:11.337 INFO:tasks.check_counter:Daemon mds.a mds.imported=2 2024-05-07T02:43:11.337 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=6 2024-05-07T02:43:11.337 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-05-07T02:43:11.337 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-05-07T02:43:11.337 DEBUG:tasks.check_counter:Getting stats from d 2024-05-07T02:43:11.338 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-07T02:43:11.810 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:43:12.229 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:43:12 smithi019 ceph-mon[29781]: pgmap v352: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-05-07T02:43:12.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:43:12 smithi064 ceph-mon[34699]: pgmap v352: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-05-07T02:43:12.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:43:12 smithi027 ceph-mon[34398]: pgmap v352: 129 pgs: 129 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-05-07T02:43:14.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:43:14 smithi064 ceph-mon[34699]: pgmap v353: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-07T02:43:14.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:43:14 smithi019 ceph-mon[29781]: pgmap v353: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-07T02:43:14.425 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:43:14.425 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-05-07T02:43:14.425 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 841, 2024-05-07T02:43:14.425 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 843, 2024-05-07T02:43:14.426 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 4409381, 2024-05-07T02:43:14.426 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 267502, 2024-05-07T02:43:14.426 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 8, 2024-05-07T02:43:14.426 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 3, 2024-05-07T02:43:14.426 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.207602678, 2024-05-07T02:43:14.426 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.069344556, 2024-05-07T02:43:14.426 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.067592631, 2024-05-07T02:43:14.426 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.049577003, 2024-05-07T02:43:14.427 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-07T02:43:14.427 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 841, 2024-05-07T02:43:14.427 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.079193062, 2024-05-07T02:43:14.427 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000094165 2024-05-07T02:43:14.427 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.427 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-07T02:43:14.427 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:14.427 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:14.427 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:14.428 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.428 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-07T02:43:14.428 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 0 2024-05-07T02:43:14.428 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.428 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-05-07T02:43:14.428 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 821, 2024-05-07T02:43:14.428 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 1034, 2024-05-07T02:43:14.428 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 8478964, 2024-05-07T02:43:14.428 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 770951, 2024-05-07T02:43:14.429 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 7, 2024-05-07T02:43:14.429 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 5, 2024-05-07T02:43:14.429 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.257565111, 2024-05-07T02:43:14.429 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.105721312, 2024-05-07T02:43:14.429 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.077831139, 2024-05-07T02:43:14.429 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.049756988, 2024-05-07T02:43:14.429 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-07T02:43:14.429 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1032, 2024-05-07T02:43:14.429 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.105586476, 2024-05-07T02:43:14.430 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000102312 2024-05-07T02:43:14.430 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.430 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-07T02:43:14.430 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:14.430 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:14.430 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:14.430 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.430 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-05-07T02:43:14.430 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 511776 2024-05-07T02:43:14.431 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.431 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-05-07T02:43:14.431 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 1475, 2024-05-07T02:43:14.431 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 1443, 2024-05-07T02:43:14.431 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 6459850, 2024-05-07T02:43:14.431 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 541846, 2024-05-07T02:43:14.431 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 4, 2024-05-07T02:43:14.431 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 3, 2024-05-07T02:43:14.431 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.440999261, 2024-05-07T02:43:14.432 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.157909901, 2024-05-07T02:43:14.432 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.209856822, 2024-05-07T02:43:14.432 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.083652855, 2024-05-07T02:43:14.432 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-07T02:43:14.432 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1442, 2024-05-07T02:43:14.432 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.147890206, 2024-05-07T02:43:14.432 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000102559 2024-05-07T02:43:14.432 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.432 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-07T02:43:14.433 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:14.433 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:14.433 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:14.433 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.433 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 1900144, 2024-05-07T02:43:14.433 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 179008 2024-05-07T02:43:14.433 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.433 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-05-07T02:43:14.433 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 1, 2024-05-07T02:43:14.434 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-05-07T02:43:14.434 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.434 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-MDSRank": { 2024-05-07T02:43:14.434 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-05-07T02:43:14.434 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-05-07T02:43:14.434 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 4855, 2024-05-07T02:43:14.434 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.277266128, 2024-05-07T02:43:14.434 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000057109 2024-05-07T02:43:14.434 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:14.435 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.435 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-PurgeQueue": { 2024-05-07T02:43:14.435 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-05-07T02:43:14.435 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-05-07T02:43:14.435 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:14.435 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:14.435 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:14.435 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:14.435 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.436 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-05-07T02:43:14.436 INFO:teuthology.orchestra.run.smithi019.stdout: "request": 0, 2024-05-07T02:43:14.436 INFO:teuthology.orchestra.run.smithi019.stdout: "reply": 0, 2024-05-07T02:43:14.436 INFO:teuthology.orchestra.run.smithi019.stdout: "reply_latency": { 2024-05-07T02:43:14.436 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:14.436 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:14.436 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:14.436 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.436 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_reply": 0, 2024-05-07T02:43:14.437 INFO:teuthology.orchestra.run.smithi019.stdout: "forward": 0, 2024-05-07T02:43:14.437 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_complete": 0, 2024-05-07T02:43:14.437 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_keys": 0, 2024-05-07T02:43:14.437 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_commit": 0, 2024-05-07T02:43:14.437 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_split": 0, 2024-05-07T02:43:14.437 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_merge": 0, 2024-05-07T02:43:14.437 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes": 1836, 2024-05-07T02:43:14.437 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_top": 11, 2024-05-07T02:43:14.437 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_bottom": 1825, 2024-05-07T02:43:14.438 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pin_tail": 0, 2024-05-07T02:43:14.438 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pinned": 1821, 2024-05-07T02:43:14.438 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_expired": 0, 2024-05-07T02:43:14.438 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_with_caps": 0, 2024-05-07T02:43:14.438 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": 0, 2024-05-07T02:43:14.438 INFO:teuthology.orchestra.run.smithi019.stdout: "subtrees": 5, 2024-05-07T02:43:14.438 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse": 0, 2024-05-07T02:43:14.438 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_hit": 0, 2024-05-07T02:43:14.439 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_forward": 0, 2024-05-07T02:43:14.439 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_discover": 0, 2024-05-07T02:43:14.439 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_dir_fetch": 0, 2024-05-07T02:43:14.439 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_remote_ino": 0, 2024-05-07T02:43:14.439 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_lock": 0, 2024-05-07T02:43:14.439 INFO:teuthology.orchestra.run.smithi019.stdout: "load_cent": 0, 2024-05-07T02:43:14.439 INFO:teuthology.orchestra.run.smithi019.stdout: "q": 0, 2024-05-07T02:43:14.439 INFO:teuthology.orchestra.run.smithi019.stdout: "exported": 0, 2024-05-07T02:43:14.439 INFO:teuthology.orchestra.run.smithi019.stdout: "exported_inodes": 0, 2024-05-07T02:43:14.440 INFO:teuthology.orchestra.run.smithi019.stdout: "imported": 0, 2024-05-07T02:43:14.440 INFO:teuthology.orchestra.run.smithi019.stdout: "imported_inodes": 0, 2024-05-07T02:43:14.440 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_dir_fetch": 0, 2024-05-07T02:43:14.440 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_backtrace_fetch": 0, 2024-05-07T02:43:14.440 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_peer_discover": 0, 2024-05-07T02:43:14.440 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rfiles": 0, 2024-05-07T02:43:14.440 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rbytes": 0, 2024-05-07T02:43:14.440 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rsnaps": 0, 2024-05-07T02:43:14.440 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_fetch": 0, 2024-05-07T02:43:14.441 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_set_tag": 0, 2024-05-07T02:43:14.441 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_repaired": 0, 2024-05-07T02:43:14.441 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_inotable_repaired": 0, 2024-05-07T02:43:14.441 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_inodes": 0, 2024-05-07T02:43:14.441 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_base_inodes": 0, 2024-05-07T02:43:14.441 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dirfrag_rstats": 0, 2024-05-07T02:43:14.441 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_file_inodes": 0, 2024-05-07T02:43:14.441 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_inode_file_caps": 0, 2024-05-07T02:43:14.441 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_revoke": 0, 2024-05-07T02:43:14.442 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_grant": 0, 2024-05-07T02:43:14.442 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_trunc": 0, 2024-05-07T02:43:14.442 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-07T02:43:14.442 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-07T02:43:14.442 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps": 0, 2024-05-07T02:43:14.442 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps_dirty": 0, 2024-05-07T02:43:14.443 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_cap_release": 0, 2024-05-07T02:43:14.443 INFO:teuthology.orchestra.run.smithi019.stdout: "process_request_cap_release": 0 2024-05-07T02:43:14.443 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.443 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_cache": { 2024-05-07T02:43:14.443 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update": 0, 2024-05-07T02:43:14.443 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update_receipt": 0, 2024-05-07T02:43:14.443 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_try_discover": 0, 2024-05-07T02:43:14.443 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_send_discover": 0, 2024-05-07T02:43:14.444 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_handle_discover": 0, 2024-05-07T02:43:14.444 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays": 0, 2024-05-07T02:43:14.444 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_delayed": 0, 2024-05-07T02:43:14.444 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_enqueuing": 0, 2024-05-07T02:43:14.444 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_created": 0, 2024-05-07T02:43:14.444 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_enqueued": 0, 2024-05-07T02:43:14.444 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_reintegrated": 0, 2024-05-07T02:43:14.444 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_migrated": 0, 2024-05-07T02:43:14.445 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_processing": 0, 2024-05-07T02:43:14.445 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_enqueued": 0, 2024-05-07T02:43:14.445 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_prioritized": 0, 2024-05-07T02:43:14.445 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_started": 0, 2024-05-07T02:43:14.445 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_completed": 0, 2024-05-07T02:43:14.445 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_quiesce_path": 0, 2024-05-07T02:43:14.445 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_quiesce_inode": 0, 2024-05-07T02:43:14.445 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_enqueue_scrub": 0, 2024-05-07T02:43:14.445 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_exportdir": 0, 2024-05-07T02:43:14.446 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_flush": 0, 2024-05-07T02:43:14.446 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragmentdir": 0, 2024-05-07T02:43:14.446 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragstats": 0, 2024-05-07T02:43:14.446 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_inodestats": 0 2024-05-07T02:43:14.446 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.446 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_log": { 2024-05-07T02:43:14.446 INFO:teuthology.orchestra.run.smithi019.stdout: "evlrg": 0, 2024-05-07T02:43:14.446 INFO:teuthology.orchestra.run.smithi019.stdout: "evadd": 0, 2024-05-07T02:43:14.446 INFO:teuthology.orchestra.run.smithi019.stdout: "evex": 0, 2024-05-07T02:43:14.447 INFO:teuthology.orchestra.run.smithi019.stdout: "evtrm": 0, 2024-05-07T02:43:14.447 INFO:teuthology.orchestra.run.smithi019.stdout: "ev": 11686, 2024-05-07T02:43:14.447 INFO:teuthology.orchestra.run.smithi019.stdout: "evexg": 0, 2024-05-07T02:43:14.447 INFO:teuthology.orchestra.run.smithi019.stdout: "evexd": 0, 2024-05-07T02:43:14.447 INFO:teuthology.orchestra.run.smithi019.stdout: "segadd": 0, 2024-05-07T02:43:14.447 INFO:teuthology.orchestra.run.smithi019.stdout: "segex": 0, 2024-05-07T02:43:14.447 INFO:teuthology.orchestra.run.smithi019.stdout: "segtrm": 0, 2024-05-07T02:43:14.447 INFO:teuthology.orchestra.run.smithi019.stdout: "seg": 13, 2024-05-07T02:43:14.447 INFO:teuthology.orchestra.run.smithi019.stdout: "segmjr": 2, 2024-05-07T02:43:14.448 INFO:teuthology.orchestra.run.smithi019.stdout: "segexg": 0, 2024-05-07T02:43:14.448 INFO:teuthology.orchestra.run.smithi019.stdout: "segexd": 0, 2024-05-07T02:43:14.448 INFO:teuthology.orchestra.run.smithi019.stdout: "expos": 4194304, 2024-05-07T02:43:14.448 INFO:teuthology.orchestra.run.smithi019.stdout: "wrpos": 20851604, 2024-05-07T02:43:14.448 INFO:teuthology.orchestra.run.smithi019.stdout: "rdpos": 20851030, 2024-05-07T02:43:14.448 INFO:teuthology.orchestra.run.smithi019.stdout: "jlat": { 2024-05-07T02:43:14.448 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:14.448 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:14.448 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:14.449 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.449 INFO:teuthology.orchestra.run.smithi019.stdout: "replayed": 11686 2024-05-07T02:43:14.449 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.449 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_mem": { 2024-05-07T02:43:14.449 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 115, 2024-05-07T02:43:14.449 INFO:teuthology.orchestra.run.smithi019.stdout: "ino+": 1843, 2024-05-07T02:43:14.449 INFO:teuthology.orchestra.run.smithi019.stdout: "ino-": 1728, 2024-05-07T02:43:14.449 INFO:teuthology.orchestra.run.smithi019.stdout: "dir": 24, 2024-05-07T02:43:14.449 INFO:teuthology.orchestra.run.smithi019.stdout: "dir+": 25, 2024-05-07T02:43:14.450 INFO:teuthology.orchestra.run.smithi019.stdout: "dir-": 1, 2024-05-07T02:43:14.450 INFO:teuthology.orchestra.run.smithi019.stdout: "dn": 1836, 2024-05-07T02:43:14.450 INFO:teuthology.orchestra.run.smithi019.stdout: "dn+": 1860, 2024-05-07T02:43:14.450 INFO:teuthology.orchestra.run.smithi019.stdout: "dn-": 24, 2024-05-07T02:43:14.450 INFO:teuthology.orchestra.run.smithi019.stdout: "cap": 0, 2024-05-07T02:43:14.450 INFO:teuthology.orchestra.run.smithi019.stdout: "cap+": 0, 2024-05-07T02:43:14.450 INFO:teuthology.orchestra.run.smithi019.stdout: "cap-": 0, 2024-05-07T02:43:14.450 INFO:teuthology.orchestra.run.smithi019.stdout: "rss": 121068, 2024-05-07T02:43:14.450 INFO:teuthology.orchestra.run.smithi019.stdout: "heap": 215440 2024-05-07T02:43:14.451 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.451 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_server": { 2024-05-07T02:43:14.451 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_client_request": 0, 2024-05-07T02:43:14.451 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_server_request": 0, 2024-05-07T02:43:14.451 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_request": 0, 2024-05-07T02:43:14.451 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_session": 0, 2024-05-07T02:43:14.451 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_peer_request": 0, 2024-05-07T02:43:14.451 INFO:teuthology.orchestra.run.smithi019.stdout: "req_create_latency": { 2024-05-07T02:43:14.451 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:14.452 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:14.452 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:14.452 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.452 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getattr_latency": { 2024-05-07T02:43:14.452 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:14.452 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:14.452 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:14.452 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.452 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getfilelock_latency": { 2024-05-07T02:43:14.453 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:14.453 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:14.453 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:14.453 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.453 INFO:teuthology.orchestra.run.smithi019.stdout: "req_link_latency": { 2024-05-07T02:43:14.453 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:14.453 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:14.453 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:14.453 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.454 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookup_latency": { 2024-05-07T02:43:14.454 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:14.454 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:14.454 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:14.454 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.454 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookuphash_latency": { 2024-05-07T02:43:14.454 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:14.454 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:14.454 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:14.455 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.455 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupino_latency": { 2024-05-07T02:43:14.455 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:14.455 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:14.455 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:14.455 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.455 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupname_latency": { 2024-05-07T02:43:14.455 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:14.455 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:14.456 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:14.456 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.456 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupparent_latency": { 2024-05-07T02:43:14.456 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:14.456 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:14.456 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:14.456 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.456 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupsnap_latency": { 2024-05-07T02:43:14.456 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:14.457 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:14.457 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:14.457 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.457 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lssnap_latency": { 2024-05-07T02:43:14.457 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:14.457 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:14.457 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:14.457 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.457 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mkdir_latency": { 2024-05-07T02:43:14.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:43:14 smithi027 ceph-mon[34398]: pgmap v353: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-07T02:43:14.459 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:14.459 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:14.459 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:14.459 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.459 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mknod_latency": { 2024-05-07T02:43:14.459 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:14.459 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:14.459 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:14.460 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.460 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mksnap_latency": { 2024-05-07T02:43:14.460 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:14.460 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:14.460 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:14.460 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.460 INFO:teuthology.orchestra.run.smithi019.stdout: "req_open_latency": { 2024-05-07T02:43:14.461 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:14.461 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:14.461 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:14.461 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.461 INFO:teuthology.orchestra.run.smithi019.stdout: "req_readdir_latency": { 2024-05-07T02:43:14.461 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:14.461 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:14.461 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:14.461 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.462 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rename_latency": { 2024-05-07T02:43:14.462 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:14.462 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:14.462 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:14.462 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.462 INFO:teuthology.orchestra.run.smithi019.stdout: "req_renamesnap_latency": { 2024-05-07T02:43:14.462 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:14.463 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:14.463 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:14.463 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.463 INFO:teuthology.orchestra.run.smithi019.stdout: "req_snapdiff_latency": { 2024-05-07T02:43:14.463 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:14.463 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:14.463 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:14.464 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.464 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmdir_latency": { 2024-05-07T02:43:14.464 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:14.464 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:14.464 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:14.464 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.464 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmsnap_latency": { 2024-05-07T02:43:14.465 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:14.465 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:14.465 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:14.465 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.465 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmxattr_latency": { 2024-05-07T02:43:14.465 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:14.465 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:14.465 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:14.465 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.466 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setattr_latency": { 2024-05-07T02:43:14.466 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:14.466 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:14.466 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:14.466 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.466 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setdirlayout_latency": { 2024-05-07T02:43:14.466 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:14.466 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:14.466 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:14.467 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.467 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setfilelock_latency": { 2024-05-07T02:43:14.467 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:14.467 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:14.467 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:14.467 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.467 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setlayout_latency": { 2024-05-07T02:43:14.467 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:14.468 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:14.468 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:14.468 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.468 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setxattr_latency": { 2024-05-07T02:43:14.468 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:14.468 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:14.468 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:14.468 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.468 INFO:teuthology.orchestra.run.smithi019.stdout: "req_symlink_latency": { 2024-05-07T02:43:14.468 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:14.469 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:14.469 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:14.469 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.469 INFO:teuthology.orchestra.run.smithi019.stdout: "req_unlink_latency": { 2024-05-07T02:43:14.469 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:14.469 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:14.469 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:14.469 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.470 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_revoke_eviction": 0, 2024-05-07T02:43:14.470 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_acquisition_throttle": 0, 2024-05-07T02:43:14.470 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getvxattr_latency": { 2024-05-07T02:43:14.470 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:14.470 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:14.470 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:14.470 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:14.470 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.470 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_sessions": { 2024-05-07T02:43:14.471 INFO:teuthology.orchestra.run.smithi019.stdout: "session_count": 2, 2024-05-07T02:43:14.471 INFO:teuthology.orchestra.run.smithi019.stdout: "session_add": 4, 2024-05-07T02:43:14.471 INFO:teuthology.orchestra.run.smithi019.stdout: "session_remove": 2, 2024-05-07T02:43:14.471 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_open": 4, 2024-05-07T02:43:14.471 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_stale": 0, 2024-05-07T02:43:14.471 INFO:teuthology.orchestra.run.smithi019.stdout: "total_load": 0, 2024-05-07T02:43:14.472 INFO:teuthology.orchestra.run.smithi019.stdout: "average_load": 0, 2024-05-07T02:43:14.472 INFO:teuthology.orchestra.run.smithi019.stdout: "avg_session_uptime": 3709, 2024-05-07T02:43:14.472 INFO:teuthology.orchestra.run.smithi019.stdout: "mdthresh_evicted": 0 2024-05-07T02:43:14.472 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.472 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-05-07T02:43:14.472 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-05-07T02:43:14.472 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-05-07T02:43:14.472 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-05-07T02:43:14.472 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-05-07T02:43:14.473 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-05-07T02:43:14.473 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-05-07T02:43:14.473 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-07T02:43:14.473 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-05-07T02:43:14.473 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-07T02:43:14.473 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-05-07T02:43:14.473 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-05-07T02:43:14.473 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-05-07T02:43:14.473 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-07T02:43:14.474 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-05-07T02:43:14.474 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-05-07T02:43:14.474 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-05-07T02:43:14.474 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-05-07T02:43:14.474 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-05-07T02:43:14.474 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-07T02:43:14.474 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-05-07T02:43:14.474 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-07T02:43:14.474 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-05-07T02:43:14.475 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-05-07T02:43:14.475 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-05-07T02:43:14.475 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-05-07T02:43:14.475 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-05-07T02:43:14.475 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-07T02:43:14.475 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-05-07T02:43:14.476 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-05-07T02:43:14.476 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-05-07T02:43:14.476 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-05-07T02:43:14.476 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-05-07T02:43:14.476 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-05-07T02:43:14.476 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-05-07T02:43:14.476 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-05-07T02:43:14.476 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-05-07T02:43:14.476 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 213704, 2024-05-07T02:43:14.477 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 168, 2024-05-07T02:43:14.477 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 0, 2024-05-07T02:43:14.477 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 0, 2024-05-07T02:43:14.477 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-05-07T02:43:14.477 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-05-07T02:43:14.477 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-05-07T02:43:14.477 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-05-07T02:43:14.477 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-05-07T02:43:14.477 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-05-07T02:43:14.478 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 9848, 2024-05-07T02:43:14.478 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 190, 2024-05-07T02:43:14.478 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-05-07T02:43:14.478 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-05-07T02:43:14.478 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-05-07T02:43:14.478 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-05-07T02:43:14.478 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 1617234, 2024-05-07T02:43:14.478 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 9931, 2024-05-07T02:43:14.478 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-05-07T02:43:14.479 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-05-07T02:43:14.479 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-05-07T02:43:14.479 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-05-07T02:43:14.479 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.479 INFO:teuthology.orchestra.run.smithi019.stdout: "objecter": { 2024-05-07T02:43:14.479 INFO:teuthology.orchestra.run.smithi019.stdout: "op_active": 0, 2024-05-07T02:43:14.479 INFO:teuthology.orchestra.run.smithi019.stdout: "op_laggy": 0, 2024-05-07T02:43:14.479 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send": 2797, 2024-05-07T02:43:14.480 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send_bytes": 21, 2024-05-07T02:43:14.480 INFO:teuthology.orchestra.run.smithi019.stdout: "op_resend": 3, 2024-05-07T02:43:14.480 INFO:teuthology.orchestra.run.smithi019.stdout: "op_reply": 2797, 2024-05-07T02:43:14.480 INFO:teuthology.orchestra.run.smithi019.stdout: "op_latency": { 2024-05-07T02:43:14.480 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2797, 2024-05-07T02:43:14.480 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 2.880970573, 2024-05-07T02:43:14.480 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.001030021 2024-05-07T02:43:14.480 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.480 INFO:teuthology.orchestra.run.smithi019.stdout: "op_inflight": 0, 2024-05-07T02:43:14.481 INFO:teuthology.orchestra.run.smithi019.stdout: "oplen_avg": { 2024-05-07T02:43:14.481 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2797, 2024-05-07T02:43:14.481 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 2798 2024-05-07T02:43:14.481 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.481 INFO:teuthology.orchestra.run.smithi019.stdout: "op": 2797, 2024-05-07T02:43:14.481 INFO:teuthology.orchestra.run.smithi019.stdout: "op_r": 2797, 2024-05-07T02:43:14.481 INFO:teuthology.orchestra.run.smithi019.stdout: "op_w": 0, 2024-05-07T02:43:14.481 INFO:teuthology.orchestra.run.smithi019.stdout: "op_rmw": 0, 2024-05-07T02:43:14.481 INFO:teuthology.orchestra.run.smithi019.stdout: "op_pg": 0, 2024-05-07T02:43:14.482 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_stat": 1610, 2024-05-07T02:43:14.482 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_create": 0, 2024-05-07T02:43:14.482 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_read": 1185, 2024-05-07T02:43:14.482 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_write": 0, 2024-05-07T02:43:14.482 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writefull": 0, 2024-05-07T02:43:14.482 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writesame": 0, 2024-05-07T02:43:14.482 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_append": 0, 2024-05-07T02:43:14.482 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_zero": 0, 2024-05-07T02:43:14.483 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_truncate": 0, 2024-05-07T02:43:14.483 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_delete": 0, 2024-05-07T02:43:14.483 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_mapext": 0, 2024-05-07T02:43:14.483 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_sparse_read": 0, 2024-05-07T02:43:14.483 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_clonerange": 0, 2024-05-07T02:43:14.483 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_getxattr": 1, 2024-05-07T02:43:14.483 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_setxattr": 0, 2024-05-07T02:43:14.483 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_cmpxattr": 0, 2024-05-07T02:43:14.483 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_rmxattr": 0, 2024-05-07T02:43:14.484 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_resetxattrs": 0, 2024-05-07T02:43:14.484 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_call": 0, 2024-05-07T02:43:14.484 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_watch": 0, 2024-05-07T02:43:14.484 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_notify": 0, 2024-05-07T02:43:14.484 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_src_cmpxattr": 0, 2024-05-07T02:43:14.484 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls": 0, 2024-05-07T02:43:14.484 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls_filter": 0, 2024-05-07T02:43:14.484 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_other": 0, 2024-05-07T02:43:14.484 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_active": 0, 2024-05-07T02:43:14.485 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_send": 0, 2024-05-07T02:43:14.485 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_resend": 0, 2024-05-07T02:43:14.485 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_ping": 0, 2024-05-07T02:43:14.485 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_active": 0, 2024-05-07T02:43:14.485 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_send": 0, 2024-05-07T02:43:14.485 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_resend": 0, 2024-05-07T02:43:14.485 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_active": 0, 2024-05-07T02:43:14.485 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_send": 0, 2024-05-07T02:43:14.485 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_resend": 0, 2024-05-07T02:43:14.486 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_active": 0, 2024-05-07T02:43:14.486 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_send": 0, 2024-05-07T02:43:14.486 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_resend": 0, 2024-05-07T02:43:14.486 INFO:teuthology.orchestra.run.smithi019.stdout: "command_active": 0, 2024-05-07T02:43:14.486 INFO:teuthology.orchestra.run.smithi019.stdout: "command_send": 0, 2024-05-07T02:43:14.486 INFO:teuthology.orchestra.run.smithi019.stdout: "command_resend": 0, 2024-05-07T02:43:14.486 INFO:teuthology.orchestra.run.smithi019.stdout: "map_epoch": 99, 2024-05-07T02:43:14.486 INFO:teuthology.orchestra.run.smithi019.stdout: "map_full": 0, 2024-05-07T02:43:14.486 INFO:teuthology.orchestra.run.smithi019.stdout: "map_inc": 12, 2024-05-07T02:43:14.487 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_sessions": 7, 2024-05-07T02:43:14.487 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_open": 8, 2024-05-07T02:43:14.487 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_close": 1, 2024-05-07T02:43:14.487 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_laggy": 0, 2024-05-07T02:43:14.487 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_wr": 0, 2024-05-07T02:43:14.487 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_rd": 2, 2024-05-07T02:43:14.487 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_del": 0 2024-05-07T02:43:14.487 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.487 INFO:teuthology.orchestra.run.smithi019.stdout: "oft": { 2024-05-07T02:43:14.488 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_objs": 0, 2024-05-07T02:43:14.488 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_kv_pairs": 0, 2024-05-07T02:43:14.488 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_updates": 0, 2024-05-07T02:43:14.488 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_removes": 0 2024-05-07T02:43:14.488 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.488 INFO:teuthology.orchestra.run.smithi019.stdout: "osd-slow-ops": { 2024-05-07T02:43:14.488 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_ops_count": 0 2024-05-07T02:43:14.488 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.488 INFO:teuthology.orchestra.run.smithi019.stdout: "purge_queue": { 2024-05-07T02:43:14.489 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops": 0, 2024-05-07T02:43:14.489 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops_high_water": 0, 2024-05-07T02:43:14.489 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing": 0, 2024-05-07T02:43:14.489 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_high_water": 0, 2024-05-07T02:43:14.489 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed_ops": 0, 2024-05-07T02:43:14.489 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed": 0, 2024-05-07T02:43:14.489 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_item_in_journal": 0 2024-05-07T02:43:14.489 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.490 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-07T02:43:14.490 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-05-07T02:43:14.490 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-05-07T02:43:14.490 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-05-07T02:43:14.490 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 3133, 2024-05-07T02:43:14.490 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 19046198, 2024-05-07T02:43:14.490 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:14.491 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 3133, 2024-05-07T02:43:14.491 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-05-07T02:43:14.491 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-05-07T02:43:14.491 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 3133, 2024-05-07T02:43:14.491 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 19046198, 2024-05-07T02:43:14.491 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-05-07T02:43:14.491 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:14.491 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:14.492 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:14.492 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:14.492 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.492 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_bytes": { 2024-05-07T02:43:14.492 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-05-07T02:43:14.492 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-05-07T02:43:14.492 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-05-07T02:43:14.492 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-05-07T02:43:14.492 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-05-07T02:43:14.493 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:14.493 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-05-07T02:43:14.493 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 2797, 2024-05-07T02:43:14.493 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 16657305, 2024-05-07T02:43:14.493 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 2797, 2024-05-07T02:43:14.493 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 16657305, 2024-05-07T02:43:14.493 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-05-07T02:43:14.493 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:14.494 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:14.494 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:14.494 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:14.494 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.494 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_ops": { 2024-05-07T02:43:14.494 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-05-07T02:43:14.494 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 1024, 2024-05-07T02:43:14.495 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-05-07T02:43:14.495 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-05-07T02:43:14.495 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-05-07T02:43:14.495 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:14.495 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-05-07T02:43:14.495 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 2797, 2024-05-07T02:43:14.495 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 2797, 2024-05-07T02:43:14.495 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 2797, 2024-05-07T02:43:14.495 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 2797, 2024-05-07T02:43:14.496 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-05-07T02:43:14.496 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:14.496 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:14.496 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:14.496 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:14.496 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.496 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle": { 2024-05-07T02:43:14.496 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-05-07T02:43:14.497 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-05-07T02:43:14.497 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-05-07T02:43:14.497 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-05-07T02:43:14.497 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-05-07T02:43:14.497 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:14.497 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-05-07T02:43:14.497 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-05-07T02:43:14.497 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-05-07T02:43:14.497 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 0, 2024-05-07T02:43:14.498 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 0, 2024-05-07T02:43:14.498 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-05-07T02:43:14.498 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:14.498 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:14.498 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:14.498 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:14.498 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:14.498 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle-0x55e9bb7a08f0": { 2024-05-07T02:43:14.499 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-05-07T02:43:14.499 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-05-07T02:43:14.499 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-05-07T02:43:14.499 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-05-07T02:43:14.499 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-05-07T02:43:14.499 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:14.499 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-05-07T02:43:14.499 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-05-07T02:43:14.499 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-05-07T02:43:14.500 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 0, 2024-05-07T02:43:14.500 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 0, 2024-05-07T02:43:14.500 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-05-07T02:43:14.500 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:14.500 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:14.500 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:14.500 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:14.500 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:14.500 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:43:15.547 INFO:tasks.check_counter:Daemon mds.d mds.exported=0 2024-05-07T02:43:15.548 INFO:tasks.check_counter:Daemon mds.d mds.imported=0 2024-05-07T02:43:15.548 INFO:tasks.check_counter:Daemon mds.d mds.root_rsnaps=0 2024-05-07T02:43:15.548 INFO:tasks.check_counter:Daemon mds.d mds_server.req_mksnap_latency.avgcount=0 2024-05-07T02:43:15.548 INFO:tasks.check_counter:Daemon mds.d mds_server.req_rmsnap_latency.avgcount=0 2024-05-07T02:43:15.548 DEBUG:tasks.check_counter:Getting stats from g 2024-05-07T02:43:15.548 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-07T02:43:16.021 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:43:16.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:43:16 smithi064 ceph-mon[34699]: pgmap v354: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-07T02:43:16.375 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:43:16 smithi019 ceph-mon[29781]: pgmap v354: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-07T02:43:16.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:43:16 smithi027 ceph-mon[34398]: pgmap v354: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-07T02:43:18.183 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:43:18 smithi019 ceph-mon[29781]: pgmap v355: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-07T02:43:18.183 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:43:18 smithi019 ceph-mon[29781]: osdmap e100: 12 total, 12 up, 12 in 2024-05-07T02:43:18.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:43:18 smithi064 ceph-mon[34699]: pgmap v355: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-07T02:43:18.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:43:18 smithi064 ceph-mon[34699]: osdmap e100: 12 total, 12 up, 12 in 2024-05-07T02:43:18.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:43:18 smithi027 ceph-mon[34398]: pgmap v355: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-07T02:43:18.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:43:18 smithi027 ceph-mon[34398]: osdmap e100: 12 total, 12 up, 12 in 2024-05-07T02:43:18.736 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:43:18.736 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-05-07T02:43:18.736 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 0, 2024-05-07T02:43:18.736 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 2, 2024-05-07T02:43:18.737 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 0, 2024-05-07T02:43:18.737 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 0, 2024-05-07T02:43:18.737 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 4, 2024-05-07T02:43:18.737 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 0, 2024-05-07T02:43:18.737 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.004297472, 2024-05-07T02:43:18.737 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.000000000, 2024-05-07T02:43:18.737 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.000844653, 2024-05-07T02:43:18.737 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-07T02:43:18.738 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-07T02:43:18.738 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:18.738 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:18.738 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:18.738 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:18.738 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-07T02:43:18.738 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:18.738 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:18.738 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:18.739 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:18.739 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-07T02:43:18.739 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 0 2024-05-07T02:43:18.739 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:18.739 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-05-07T02:43:18.739 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 6, 2024-05-07T02:43:18.739 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 216, 2024-05-07T02:43:18.739 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 3264, 2024-05-07T02:43:18.739 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 195344, 2024-05-07T02:43:18.740 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 5, 2024-05-07T02:43:18.740 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 3, 2024-05-07T02:43:18.740 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.044492179, 2024-05-07T02:43:18.740 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.033380787, 2024-05-07T02:43:18.740 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.003902275, 2024-05-07T02:43:18.740 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-07T02:43:18.740 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-07T02:43:18.740 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 214, 2024-05-07T02:43:18.741 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.043670208, 2024-05-07T02:43:18.741 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000204066 2024-05-07T02:43:18.741 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:18.741 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-07T02:43:18.741 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:18.741 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:18.741 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:18.741 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:18.741 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-05-07T02:43:18.742 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 195344 2024-05-07T02:43:18.742 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:18.742 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-05-07T02:43:18.742 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 292, 2024-05-07T02:43:18.742 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 281, 2024-05-07T02:43:18.742 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 1730560, 2024-05-07T02:43:18.742 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 173504, 2024-05-07T02:43:18.742 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 2, 2024-05-07T02:43:18.743 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 1, 2024-05-07T02:43:18.743 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.131512423, 2024-05-07T02:43:18.743 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.047009260, 2024-05-07T02:43:18.743 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.094891414, 2024-05-07T02:43:18.743 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.013902280, 2024-05-07T02:43:18.743 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-07T02:43:18.743 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 280, 2024-05-07T02:43:18.743 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.027046184, 2024-05-07T02:43:18.743 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000096593 2024-05-07T02:43:18.744 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:18.744 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-07T02:43:18.744 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:18.744 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:18.744 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:18.744 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:18.744 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 1730560, 2024-05-07T02:43:18.744 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 173504 2024-05-07T02:43:18.745 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:18.745 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-05-07T02:43:18.745 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 0, 2024-05-07T02:43:18.745 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-05-07T02:43:18.745 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:18.745 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-05-07T02:43:18.745 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-05-07T02:43:18.745 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-05-07T02:43:18.745 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-05-07T02:43:18.746 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-05-07T02:43:18.746 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-05-07T02:43:18.746 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-05-07T02:43:18.746 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-07T02:43:18.746 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-05-07T02:43:18.746 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-07T02:43:18.746 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-05-07T02:43:18.746 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-05-07T02:43:18.747 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-05-07T02:43:18.747 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-07T02:43:18.747 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-05-07T02:43:18.747 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-05-07T02:43:18.747 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-05-07T02:43:18.747 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-05-07T02:43:18.747 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-05-07T02:43:18.747 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-07T02:43:18.748 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-05-07T02:43:18.748 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-07T02:43:18.748 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-05-07T02:43:18.748 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-05-07T02:43:18.748 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-05-07T02:43:18.748 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-05-07T02:43:18.748 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-05-07T02:43:18.748 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-07T02:43:18.748 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-05-07T02:43:18.749 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-05-07T02:43:18.749 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-05-07T02:43:18.749 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-05-07T02:43:18.749 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-05-07T02:43:18.749 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-05-07T02:43:18.749 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-05-07T02:43:18.749 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-05-07T02:43:18.749 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-05-07T02:43:18.749 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 148714, 2024-05-07T02:43:18.750 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 30, 2024-05-07T02:43:18.750 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 0, 2024-05-07T02:43:18.750 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 0, 2024-05-07T02:43:18.750 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-05-07T02:43:18.750 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-05-07T02:43:18.750 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-05-07T02:43:18.750 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-05-07T02:43:18.750 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-05-07T02:43:18.751 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-05-07T02:43:18.751 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 0, 2024-05-07T02:43:18.751 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 0, 2024-05-07T02:43:18.751 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-05-07T02:43:18.751 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-05-07T02:43:18.751 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-05-07T02:43:18.751 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-05-07T02:43:18.751 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 928, 2024-05-07T02:43:18.751 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 2, 2024-05-07T02:43:18.752 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-05-07T02:43:18.752 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-05-07T02:43:18.752 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-05-07T02:43:18.752 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-05-07T02:43:18.752 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:18.752 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-07T02:43:18.752 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-05-07T02:43:18.752 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-05-07T02:43:18.753 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-05-07T02:43:18.753 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 294, 2024-05-07T02:43:18.753 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 1689330, 2024-05-07T02:43:18.753 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:18.753 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 294, 2024-05-07T02:43:18.753 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-05-07T02:43:18.753 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-05-07T02:43:18.753 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 294, 2024-05-07T02:43:18.754 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 1689330, 2024-05-07T02:43:18.754 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-05-07T02:43:18.754 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:18.754 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:18.754 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:18.754 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:18.754 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:18.754 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:43:19.722 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-05-07T02:43:19.722 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-05-07T02:43:19.722 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-05-07T02:43:19.722 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-05-07T02:43:19.722 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-05-07T02:43:19.723 DEBUG:tasks.check_counter:Getting stats from j 2024-05-07T02:43:19.723 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-07T02:43:20.193 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:43:20.345 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:43:20 smithi019 ceph-mon[29781]: pgmap v357: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 223 B/s wr, 11 op/s 2024-05-07T02:43:20.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:43:20 smithi064 ceph-mon[34699]: pgmap v357: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 223 B/s wr, 11 op/s 2024-05-07T02:43:20.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:43:20 smithi027 ceph-mon[34398]: pgmap v357: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 223 B/s wr, 11 op/s 2024-05-07T02:43:22.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:43:22 smithi064 ceph-mon[34699]: pgmap v358: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-07T02:43:22.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:43:22 smithi019 ceph-mon[29781]: pgmap v358: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-07T02:43:22.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:43:22 smithi027 ceph-mon[34398]: pgmap v358: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-05-07T02:43:22.855 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-05-07T02:43:22.855 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-05-07T02:43:22.856 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 2135, 2024-05-07T02:43:22.856 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 2135, 2024-05-07T02:43:22.856 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 7752115, 2024-05-07T02:43:22.856 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 678107, 2024-05-07T02:43:22.856 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 7, 2024-05-07T02:43:22.856 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 3, 2024-05-07T02:43:22.856 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.492217830, 2024-05-07T02:43:22.856 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.171865225, 2024-05-07T02:43:22.856 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.144701014, 2024-05-07T02:43:22.857 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.128804819, 2024-05-07T02:43:22.857 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-07T02:43:22.857 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2134, 2024-05-07T02:43:22.857 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.213685361, 2024-05-07T02:43:22.857 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000100133 2024-05-07T02:43:22.857 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.857 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-07T02:43:22.857 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:22.857 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:22.858 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:22.858 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.858 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-07T02:43:22.858 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 448 2024-05-07T02:43:22.858 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.858 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-05-07T02:43:22.858 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 519, 2024-05-07T02:43:22.858 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 737, 2024-05-07T02:43:22.859 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 8519162, 2024-05-07T02:43:22.859 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 679562, 2024-05-07T02:43:22.859 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 7, 2024-05-07T02:43:22.859 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 5, 2024-05-07T02:43:22.859 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.197976299, 2024-05-07T02:43:22.859 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.073881466, 2024-05-07T02:43:22.859 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.068627970, 2024-05-07T02:43:22.859 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.036231845, 2024-05-07T02:43:22.859 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-07T02:43:22.860 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 734, 2024-05-07T02:43:22.860 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.084645410, 2024-05-07T02:43:22.860 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000115320 2024-05-07T02:43:22.860 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.860 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-07T02:43:22.860 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:22.860 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:22.861 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:22.861 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.861 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-07T02:43:22.861 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 515088 2024-05-07T02:43:22.861 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.861 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-05-07T02:43:22.861 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 508, 2024-05-07T02:43:22.862 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 476, 2024-05-07T02:43:22.862 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 6139180, 2024-05-07T02:43:22.862 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 235182, 2024-05-07T02:43:22.862 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 5, 2024-05-07T02:43:22.862 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 3, 2024-05-07T02:43:22.862 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.191947023, 2024-05-07T02:43:22.862 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.051349556, 2024-05-07T02:43:22.862 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.154772054, 2024-05-07T02:43:22.863 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.024190493, 2024-05-07T02:43:22.863 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-05-07T02:43:22.863 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 475, 2024-05-07T02:43:22.863 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.052879479, 2024-05-07T02:43:22.863 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000111325 2024-05-07T02:43:22.863 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.863 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-05-07T02:43:22.863 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:22.863 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:22.864 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:22.864 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.864 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 1902368, 2024-05-07T02:43:22.864 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 181440 2024-05-07T02:43:22.864 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.864 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-05-07T02:43:22.864 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 1, 2024-05-07T02:43:22.864 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-05-07T02:43:22.865 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.865 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-MDSRank": { 2024-05-07T02:43:22.865 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-05-07T02:43:22.865 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-05-07T02:43:22.865 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 4841, 2024-05-07T02:43:22.865 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.285795056, 2024-05-07T02:43:22.865 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000059036 2024-05-07T02:43:22.865 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:22.865 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.866 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-PurgeQueue": { 2024-05-07T02:43:22.866 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-05-07T02:43:22.866 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-05-07T02:43:22.866 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:22.866 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:22.866 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:22.866 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:22.866 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.866 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-05-07T02:43:22.867 INFO:teuthology.orchestra.run.smithi019.stdout: "request": 0, 2024-05-07T02:43:22.867 INFO:teuthology.orchestra.run.smithi019.stdout: "reply": 0, 2024-05-07T02:43:22.867 INFO:teuthology.orchestra.run.smithi019.stdout: "reply_latency": { 2024-05-07T02:43:22.867 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:22.867 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:22.867 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:22.867 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.868 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_reply": 0, 2024-05-07T02:43:22.868 INFO:teuthology.orchestra.run.smithi019.stdout: "forward": 0, 2024-05-07T02:43:22.868 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_complete": 0, 2024-05-07T02:43:22.868 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_keys": 0, 2024-05-07T02:43:22.868 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_commit": 0, 2024-05-07T02:43:22.868 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_split": 0, 2024-05-07T02:43:22.868 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_merge": 0, 2024-05-07T02:43:22.868 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes": 2121, 2024-05-07T02:43:22.868 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_top": 4, 2024-05-07T02:43:22.869 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_bottom": 2117, 2024-05-07T02:43:22.869 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pin_tail": 0, 2024-05-07T02:43:22.869 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pinned": 2116, 2024-05-07T02:43:22.869 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_expired": 0, 2024-05-07T02:43:22.869 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_with_caps": 0, 2024-05-07T02:43:22.869 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": 0, 2024-05-07T02:43:22.870 INFO:teuthology.orchestra.run.smithi019.stdout: "subtrees": 6, 2024-05-07T02:43:22.870 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse": 0, 2024-05-07T02:43:22.870 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_hit": 0, 2024-05-07T02:43:22.870 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_forward": 0, 2024-05-07T02:43:22.870 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_discover": 0, 2024-05-07T02:43:22.870 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_dir_fetch": 0, 2024-05-07T02:43:22.870 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_remote_ino": 0, 2024-05-07T02:43:22.870 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_lock": 0, 2024-05-07T02:43:22.870 INFO:teuthology.orchestra.run.smithi019.stdout: "load_cent": 0, 2024-05-07T02:43:22.871 INFO:teuthology.orchestra.run.smithi019.stdout: "q": 0, 2024-05-07T02:43:22.871 INFO:teuthology.orchestra.run.smithi019.stdout: "exported": 0, 2024-05-07T02:43:22.871 INFO:teuthology.orchestra.run.smithi019.stdout: "exported_inodes": 0, 2024-05-07T02:43:22.871 INFO:teuthology.orchestra.run.smithi019.stdout: "imported": 0, 2024-05-07T02:43:22.871 INFO:teuthology.orchestra.run.smithi019.stdout: "imported_inodes": 0, 2024-05-07T02:43:22.871 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_dir_fetch": 0, 2024-05-07T02:43:22.872 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_backtrace_fetch": 0, 2024-05-07T02:43:22.872 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_peer_discover": 0, 2024-05-07T02:43:22.872 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rfiles": 0, 2024-05-07T02:43:22.872 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rbytes": 0, 2024-05-07T02:43:22.872 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rsnaps": 0, 2024-05-07T02:43:22.872 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_fetch": 0, 2024-05-07T02:43:22.872 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_set_tag": 0, 2024-05-07T02:43:22.872 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_repaired": 0, 2024-05-07T02:43:22.873 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_inotable_repaired": 0, 2024-05-07T02:43:22.873 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_inodes": 0, 2024-05-07T02:43:22.873 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_base_inodes": 0, 2024-05-07T02:43:22.873 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dirfrag_rstats": 0, 2024-05-07T02:43:22.873 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_file_inodes": 0, 2024-05-07T02:43:22.873 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_inode_file_caps": 0, 2024-05-07T02:43:22.873 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_revoke": 0, 2024-05-07T02:43:22.874 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_grant": 0, 2024-05-07T02:43:22.874 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_trunc": 0, 2024-05-07T02:43:22.874 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-07T02:43:22.874 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-07T02:43:22.874 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps": 0, 2024-05-07T02:43:22.874 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps_dirty": 0, 2024-05-07T02:43:22.874 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_cap_release": 0, 2024-05-07T02:43:22.874 INFO:teuthology.orchestra.run.smithi019.stdout: "process_request_cap_release": 0 2024-05-07T02:43:22.875 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.875 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_cache": { 2024-05-07T02:43:22.875 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update": 0, 2024-05-07T02:43:22.875 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update_receipt": 0, 2024-05-07T02:43:22.875 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_try_discover": 0, 2024-05-07T02:43:22.875 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_send_discover": 0, 2024-05-07T02:43:22.875 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_handle_discover": 0, 2024-05-07T02:43:22.875 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays": 0, 2024-05-07T02:43:22.876 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_delayed": 0, 2024-05-07T02:43:22.876 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_enqueuing": 0, 2024-05-07T02:43:22.876 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_created": 0, 2024-05-07T02:43:22.876 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_enqueued": 0, 2024-05-07T02:43:22.876 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_reintegrated": 0, 2024-05-07T02:43:22.876 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_migrated": 0, 2024-05-07T02:43:22.876 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_processing": 0, 2024-05-07T02:43:22.876 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_enqueued": 0, 2024-05-07T02:43:22.877 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_prioritized": 0, 2024-05-07T02:43:22.877 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_started": 0, 2024-05-07T02:43:22.877 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_completed": 0, 2024-05-07T02:43:22.877 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_quiesce_path": 0, 2024-05-07T02:43:22.877 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_quiesce_inode": 0, 2024-05-07T02:43:22.877 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_enqueue_scrub": 0, 2024-05-07T02:43:22.878 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_exportdir": 0, 2024-05-07T02:43:22.878 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_flush": 0, 2024-05-07T02:43:22.878 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragmentdir": 0, 2024-05-07T02:43:22.878 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragstats": 0, 2024-05-07T02:43:22.878 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_inodestats": 0 2024-05-07T02:43:22.878 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.878 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_log": { 2024-05-07T02:43:22.879 INFO:teuthology.orchestra.run.smithi019.stdout: "evlrg": 0, 2024-05-07T02:43:22.879 INFO:teuthology.orchestra.run.smithi019.stdout: "evadd": 0, 2024-05-07T02:43:22.879 INFO:teuthology.orchestra.run.smithi019.stdout: "evex": 0, 2024-05-07T02:43:22.879 INFO:teuthology.orchestra.run.smithi019.stdout: "evtrm": 0, 2024-05-07T02:43:22.879 INFO:teuthology.orchestra.run.smithi019.stdout: "ev": 13128, 2024-05-07T02:43:22.879 INFO:teuthology.orchestra.run.smithi019.stdout: "evexg": 0, 2024-05-07T02:43:22.879 INFO:teuthology.orchestra.run.smithi019.stdout: "evexd": 0, 2024-05-07T02:43:22.880 INFO:teuthology.orchestra.run.smithi019.stdout: "segadd": 0, 2024-05-07T02:43:22.880 INFO:teuthology.orchestra.run.smithi019.stdout: "segex": 0, 2024-05-07T02:43:22.880 INFO:teuthology.orchestra.run.smithi019.stdout: "segtrm": 0, 2024-05-07T02:43:22.880 INFO:teuthology.orchestra.run.smithi019.stdout: "seg": 15, 2024-05-07T02:43:22.880 INFO:teuthology.orchestra.run.smithi019.stdout: "segmjr": 3, 2024-05-07T02:43:22.880 INFO:teuthology.orchestra.run.smithi019.stdout: "segexg": 0, 2024-05-07T02:43:22.881 INFO:teuthology.orchestra.run.smithi019.stdout: "segexd": 0, 2024-05-07T02:43:22.881 INFO:teuthology.orchestra.run.smithi019.stdout: "expos": 4194304, 2024-05-07T02:43:22.881 INFO:teuthology.orchestra.run.smithi019.stdout: "wrpos": 23905906, 2024-05-07T02:43:22.881 INFO:teuthology.orchestra.run.smithi019.stdout: "rdpos": 23905332, 2024-05-07T02:43:22.881 INFO:teuthology.orchestra.run.smithi019.stdout: "jlat": { 2024-05-07T02:43:22.881 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:22.882 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:22.882 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:22.882 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.882 INFO:teuthology.orchestra.run.smithi019.stdout: "replayed": 13128 2024-05-07T02:43:22.882 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.882 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_mem": { 2024-05-07T02:43:22.882 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 102, 2024-05-07T02:43:22.882 INFO:teuthology.orchestra.run.smithi019.stdout: "ino+": 2096, 2024-05-07T02:43:22.882 INFO:teuthology.orchestra.run.smithi019.stdout: "ino-": 1994, 2024-05-07T02:43:22.883 INFO:teuthology.orchestra.run.smithi019.stdout: "dir": 25, 2024-05-07T02:43:22.883 INFO:teuthology.orchestra.run.smithi019.stdout: "dir+": 25, 2024-05-07T02:43:22.883 INFO:teuthology.orchestra.run.smithi019.stdout: "dir-": 0, 2024-05-07T02:43:22.883 INFO:teuthology.orchestra.run.smithi019.stdout: "dn": 2121, 2024-05-07T02:43:22.883 INFO:teuthology.orchestra.run.smithi019.stdout: "dn+": 2121, 2024-05-07T02:43:22.883 INFO:teuthology.orchestra.run.smithi019.stdout: "dn-": 0, 2024-05-07T02:43:22.883 INFO:teuthology.orchestra.run.smithi019.stdout: "cap": 0, 2024-05-07T02:43:22.883 INFO:teuthology.orchestra.run.smithi019.stdout: "cap+": 0, 2024-05-07T02:43:22.883 INFO:teuthology.orchestra.run.smithi019.stdout: "cap-": 0, 2024-05-07T02:43:22.884 INFO:teuthology.orchestra.run.smithi019.stdout: "rss": 121956, 2024-05-07T02:43:22.884 INFO:teuthology.orchestra.run.smithi019.stdout: "heap": 215440 2024-05-07T02:43:22.884 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.884 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_server": { 2024-05-07T02:43:22.884 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_client_request": 0, 2024-05-07T02:43:22.884 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_server_request": 0, 2024-05-07T02:43:22.884 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_request": 0, 2024-05-07T02:43:22.884 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_session": 0, 2024-05-07T02:43:22.885 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_peer_request": 0, 2024-05-07T02:43:22.885 INFO:teuthology.orchestra.run.smithi019.stdout: "req_create_latency": { 2024-05-07T02:43:22.885 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:22.885 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:22.885 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:22.885 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.885 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getattr_latency": { 2024-05-07T02:43:22.885 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:22.885 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:22.886 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:22.886 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.886 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getfilelock_latency": { 2024-05-07T02:43:22.886 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:22.886 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:22.886 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:22.886 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.886 INFO:teuthology.orchestra.run.smithi019.stdout: "req_link_latency": { 2024-05-07T02:43:22.886 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:22.887 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:22.887 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:22.887 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.887 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookup_latency": { 2024-05-07T02:43:22.887 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:22.887 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:22.887 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:22.887 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.887 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookuphash_latency": { 2024-05-07T02:43:22.888 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:22.888 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:22.888 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:22.888 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.888 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupino_latency": { 2024-05-07T02:43:22.888 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:22.888 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:22.888 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:22.888 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.889 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupname_latency": { 2024-05-07T02:43:22.889 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:22.889 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:22.889 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:22.889 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.889 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupparent_latency": { 2024-05-07T02:43:22.889 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:22.889 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:22.890 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:22.890 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.890 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupsnap_latency": { 2024-05-07T02:43:22.890 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:22.890 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:22.890 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:22.890 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.890 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lssnap_latency": { 2024-05-07T02:43:22.891 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:22.891 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:22.891 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:22.891 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.891 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mkdir_latency": { 2024-05-07T02:43:22.915 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:22.915 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:22.915 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:22.916 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.916 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mknod_latency": { 2024-05-07T02:43:22.916 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:22.916 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:22.916 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:22.916 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.916 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mksnap_latency": { 2024-05-07T02:43:22.916 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:22.916 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:22.917 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:22.917 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.917 INFO:teuthology.orchestra.run.smithi019.stdout: "req_open_latency": { 2024-05-07T02:43:22.917 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:22.917 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:22.917 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:22.917 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.917 INFO:teuthology.orchestra.run.smithi019.stdout: "req_readdir_latency": { 2024-05-07T02:43:22.918 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:22.918 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:22.918 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:22.918 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.918 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rename_latency": { 2024-05-07T02:43:22.918 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:22.918 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:22.918 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:22.919 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.919 INFO:teuthology.orchestra.run.smithi019.stdout: "req_renamesnap_latency": { 2024-05-07T02:43:22.919 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:22.919 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:22.919 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:22.919 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.919 INFO:teuthology.orchestra.run.smithi019.stdout: "req_snapdiff_latency": { 2024-05-07T02:43:22.919 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:22.920 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:22.920 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:22.920 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.920 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmdir_latency": { 2024-05-07T02:43:22.920 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:22.920 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:22.920 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:22.920 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.920 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmsnap_latency": { 2024-05-07T02:43:22.921 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:22.921 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:22.921 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:22.921 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.921 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmxattr_latency": { 2024-05-07T02:43:22.921 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:22.921 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:22.921 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:22.922 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.922 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setattr_latency": { 2024-05-07T02:43:22.922 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:22.922 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:22.922 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:22.922 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.922 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setdirlayout_latency": { 2024-05-07T02:43:22.922 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:22.923 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:22.923 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:22.923 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.923 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setfilelock_latency": { 2024-05-07T02:43:22.923 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:22.923 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:22.923 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:22.924 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.924 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setlayout_latency": { 2024-05-07T02:43:22.924 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:22.924 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:22.924 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:22.924 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.924 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setxattr_latency": { 2024-05-07T02:43:22.924 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:22.924 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:22.925 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:22.925 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.925 INFO:teuthology.orchestra.run.smithi019.stdout: "req_symlink_latency": { 2024-05-07T02:43:22.925 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:22.925 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:22.925 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:22.925 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.925 INFO:teuthology.orchestra.run.smithi019.stdout: "req_unlink_latency": { 2024-05-07T02:43:22.925 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:22.926 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:22.926 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:22.926 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.926 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_revoke_eviction": 0, 2024-05-07T02:43:22.926 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_acquisition_throttle": 0, 2024-05-07T02:43:22.926 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getvxattr_latency": { 2024-05-07T02:43:22.926 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:22.926 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:22.927 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:22.927 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:22.927 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.927 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_sessions": { 2024-05-07T02:43:22.927 INFO:teuthology.orchestra.run.smithi019.stdout: "session_count": 2, 2024-05-07T02:43:22.927 INFO:teuthology.orchestra.run.smithi019.stdout: "session_add": 3, 2024-05-07T02:43:22.927 INFO:teuthology.orchestra.run.smithi019.stdout: "session_remove": 1, 2024-05-07T02:43:22.927 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_open": 3, 2024-05-07T02:43:22.927 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_stale": 0, 2024-05-07T02:43:22.929 INFO:teuthology.orchestra.run.smithi019.stdout: "total_load": 0, 2024-05-07T02:43:22.929 INFO:teuthology.orchestra.run.smithi019.stdout: "average_load": 0, 2024-05-07T02:43:22.929 INFO:teuthology.orchestra.run.smithi019.stdout: "avg_session_uptime": 2961, 2024-05-07T02:43:22.929 INFO:teuthology.orchestra.run.smithi019.stdout: "mdthresh_evicted": 0 2024-05-07T02:43:22.929 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.929 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-05-07T02:43:22.930 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-05-07T02:43:22.930 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-05-07T02:43:22.930 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-05-07T02:43:22.930 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-05-07T02:43:22.930 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-05-07T02:43:22.930 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-05-07T02:43:22.930 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-07T02:43:22.930 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-05-07T02:43:22.930 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-07T02:43:22.931 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-05-07T02:43:22.931 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-05-07T02:43:22.931 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-05-07T02:43:22.931 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-07T02:43:22.931 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-05-07T02:43:22.931 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-05-07T02:43:22.931 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-05-07T02:43:22.931 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-05-07T02:43:22.931 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-05-07T02:43:22.932 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-07T02:43:22.932 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-05-07T02:43:22.932 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-07T02:43:22.932 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-05-07T02:43:22.932 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-05-07T02:43:22.932 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-05-07T02:43:22.932 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-05-07T02:43:22.932 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-05-07T02:43:22.932 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-07T02:43:22.932 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-05-07T02:43:22.933 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-05-07T02:43:22.933 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-05-07T02:43:22.933 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-05-07T02:43:22.933 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-05-07T02:43:22.933 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-05-07T02:43:22.933 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-05-07T02:43:22.933 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-05-07T02:43:22.933 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-05-07T02:43:22.934 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 212102, 2024-05-07T02:43:22.934 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 151, 2024-05-07T02:43:22.934 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 0, 2024-05-07T02:43:22.934 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 0, 2024-05-07T02:43:22.934 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-05-07T02:43:22.934 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-05-07T02:43:22.934 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-05-07T02:43:22.934 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-05-07T02:43:22.934 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-05-07T02:43:22.935 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-05-07T02:43:22.935 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 9608, 2024-05-07T02:43:22.935 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 184, 2024-05-07T02:43:22.935 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-05-07T02:43:22.935 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-05-07T02:43:22.935 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-05-07T02:43:22.935 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-05-07T02:43:22.935 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 1791299, 2024-05-07T02:43:22.935 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 10777, 2024-05-07T02:43:22.936 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-05-07T02:43:22.936 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-05-07T02:43:22.936 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-05-07T02:43:22.936 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-05-07T02:43:22.936 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.936 INFO:teuthology.orchestra.run.smithi019.stdout: "objecter": { 2024-05-07T02:43:22.936 INFO:teuthology.orchestra.run.smithi019.stdout: "op_active": 0, 2024-05-07T02:43:22.936 INFO:teuthology.orchestra.run.smithi019.stdout: "op_laggy": 0, 2024-05-07T02:43:22.937 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send": 2817, 2024-05-07T02:43:22.937 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send_bytes": 21, 2024-05-07T02:43:22.937 INFO:teuthology.orchestra.run.smithi019.stdout: "op_resend": 3, 2024-05-07T02:43:22.937 INFO:teuthology.orchestra.run.smithi019.stdout: "op_reply": 2817, 2024-05-07T02:43:22.937 INFO:teuthology.orchestra.run.smithi019.stdout: "op_latency": { 2024-05-07T02:43:22.937 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2817, 2024-05-07T02:43:22.937 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 2.799971413, 2024-05-07T02:43:22.937 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000993955 2024-05-07T02:43:22.938 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.938 INFO:teuthology.orchestra.run.smithi019.stdout: "op_inflight": 0, 2024-05-07T02:43:22.938 INFO:teuthology.orchestra.run.smithi019.stdout: "oplen_avg": { 2024-05-07T02:43:22.938 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 2817, 2024-05-07T02:43:22.938 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 2818 2024-05-07T02:43:22.938 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.938 INFO:teuthology.orchestra.run.smithi019.stdout: "op": 2817, 2024-05-07T02:43:22.938 INFO:teuthology.orchestra.run.smithi019.stdout: "op_r": 2817, 2024-05-07T02:43:22.939 INFO:teuthology.orchestra.run.smithi019.stdout: "op_w": 0, 2024-05-07T02:43:22.939 INFO:teuthology.orchestra.run.smithi019.stdout: "op_rmw": 0, 2024-05-07T02:43:22.939 INFO:teuthology.orchestra.run.smithi019.stdout: "op_pg": 0, 2024-05-07T02:43:22.939 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_stat": 1623, 2024-05-07T02:43:22.939 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_create": 0, 2024-05-07T02:43:22.939 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_read": 1192, 2024-05-07T02:43:22.939 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_write": 0, 2024-05-07T02:43:22.939 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writefull": 0, 2024-05-07T02:43:22.939 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writesame": 0, 2024-05-07T02:43:22.940 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_append": 0, 2024-05-07T02:43:22.940 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_zero": 0, 2024-05-07T02:43:22.940 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_truncate": 0, 2024-05-07T02:43:22.940 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_delete": 0, 2024-05-07T02:43:22.940 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_mapext": 0, 2024-05-07T02:43:22.940 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_sparse_read": 0, 2024-05-07T02:43:22.940 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_clonerange": 0, 2024-05-07T02:43:22.940 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_getxattr": 1, 2024-05-07T02:43:22.940 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_setxattr": 0, 2024-05-07T02:43:22.941 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_cmpxattr": 0, 2024-05-07T02:43:22.941 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_rmxattr": 0, 2024-05-07T02:43:22.941 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_resetxattrs": 0, 2024-05-07T02:43:22.941 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_call": 0, 2024-05-07T02:43:22.941 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_watch": 0, 2024-05-07T02:43:22.941 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_notify": 0, 2024-05-07T02:43:22.941 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_src_cmpxattr": 0, 2024-05-07T02:43:22.941 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls": 0, 2024-05-07T02:43:22.941 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls_filter": 0, 2024-05-07T02:43:22.942 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_other": 0, 2024-05-07T02:43:22.942 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_active": 0, 2024-05-07T02:43:22.942 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_send": 0, 2024-05-07T02:43:22.942 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_resend": 0, 2024-05-07T02:43:22.942 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_ping": 0, 2024-05-07T02:43:22.942 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_active": 0, 2024-05-07T02:43:22.942 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_send": 0, 2024-05-07T02:43:22.942 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_resend": 0, 2024-05-07T02:43:22.942 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_active": 0, 2024-05-07T02:43:22.943 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_send": 0, 2024-05-07T02:43:22.943 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_resend": 0, 2024-05-07T02:43:22.943 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_active": 0, 2024-05-07T02:43:22.943 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_send": 0, 2024-05-07T02:43:22.943 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_resend": 0, 2024-05-07T02:43:22.943 INFO:teuthology.orchestra.run.smithi019.stdout: "command_active": 0, 2024-05-07T02:43:22.943 INFO:teuthology.orchestra.run.smithi019.stdout: "command_send": 0, 2024-05-07T02:43:22.943 INFO:teuthology.orchestra.run.smithi019.stdout: "command_resend": 0, 2024-05-07T02:43:22.943 INFO:teuthology.orchestra.run.smithi019.stdout: "map_epoch": 100, 2024-05-07T02:43:22.944 INFO:teuthology.orchestra.run.smithi019.stdout: "map_full": 0, 2024-05-07T02:43:22.944 INFO:teuthology.orchestra.run.smithi019.stdout: "map_inc": 13, 2024-05-07T02:43:22.944 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_sessions": 7, 2024-05-07T02:43:22.944 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_open": 8, 2024-05-07T02:43:22.944 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_close": 1, 2024-05-07T02:43:22.944 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_laggy": 0, 2024-05-07T02:43:22.944 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_wr": 0, 2024-05-07T02:43:22.944 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_rd": 2, 2024-05-07T02:43:22.945 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_del": 0 2024-05-07T02:43:22.945 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.945 INFO:teuthology.orchestra.run.smithi019.stdout: "oft": { 2024-05-07T02:43:22.945 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_objs": 0, 2024-05-07T02:43:22.945 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_kv_pairs": 0, 2024-05-07T02:43:22.945 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_updates": 0, 2024-05-07T02:43:22.945 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_removes": 0 2024-05-07T02:43:22.945 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.945 INFO:teuthology.orchestra.run.smithi019.stdout: "osd-slow-ops": { 2024-05-07T02:43:22.946 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_ops_count": 0 2024-05-07T02:43:22.946 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.946 INFO:teuthology.orchestra.run.smithi019.stdout: "purge_queue": { 2024-05-07T02:43:22.946 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops": 0, 2024-05-07T02:43:22.946 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops_high_water": 0, 2024-05-07T02:43:22.946 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing": 0, 2024-05-07T02:43:22.946 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_high_water": 0, 2024-05-07T02:43:22.946 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed_ops": 0, 2024-05-07T02:43:22.946 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed": 0, 2024-05-07T02:43:22.947 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_item_in_journal": 0 2024-05-07T02:43:22.947 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.947 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-07T02:43:22.947 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-05-07T02:43:22.947 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-05-07T02:43:22.947 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-05-07T02:43:22.947 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 3158, 2024-05-07T02:43:22.947 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 22105969, 2024-05-07T02:43:22.948 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:22.948 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 3158, 2024-05-07T02:43:22.948 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-05-07T02:43:22.948 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-05-07T02:43:22.948 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 3158, 2024-05-07T02:43:22.948 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 22105969, 2024-05-07T02:43:22.948 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-05-07T02:43:22.948 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:22.948 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:22.949 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:22.949 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:22.949 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.949 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_bytes": { 2024-05-07T02:43:22.949 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-05-07T02:43:22.949 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-05-07T02:43:22.949 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-05-07T02:43:22.949 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-05-07T02:43:22.949 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-05-07T02:43:22.949 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:22.950 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-05-07T02:43:22.950 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 2817, 2024-05-07T02:43:22.950 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 19711607, 2024-05-07T02:43:22.950 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 2817, 2024-05-07T02:43:22.950 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 19711607, 2024-05-07T02:43:22.951 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-05-07T02:43:22.951 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:22.951 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:22.951 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:22.951 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:22.951 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.951 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_ops": { 2024-05-07T02:43:22.951 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-05-07T02:43:22.952 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 1024, 2024-05-07T02:43:22.952 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-05-07T02:43:22.952 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-05-07T02:43:22.952 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-05-07T02:43:22.952 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:22.952 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-05-07T02:43:22.952 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 2817, 2024-05-07T02:43:22.952 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 2817, 2024-05-07T02:43:22.952 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 2817, 2024-05-07T02:43:22.953 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 2817, 2024-05-07T02:43:22.953 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-05-07T02:43:22.953 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:22.953 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:22.953 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:22.953 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:22.953 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.953 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle": { 2024-05-07T02:43:22.954 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-05-07T02:43:22.954 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-05-07T02:43:22.954 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-05-07T02:43:22.954 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-05-07T02:43:22.954 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-05-07T02:43:22.954 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:22.954 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-05-07T02:43:22.954 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-05-07T02:43:22.954 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-05-07T02:43:22.955 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 0, 2024-05-07T02:43:22.955 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 0, 2024-05-07T02:43:22.955 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-05-07T02:43:22.955 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:22.955 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:22.955 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:22.955 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:22.955 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-05-07T02:43:22.955 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle-0x5585d391c8f0": { 2024-05-07T02:43:22.956 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-05-07T02:43:22.956 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-05-07T02:43:22.956 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-05-07T02:43:22.956 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-05-07T02:43:22.956 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-05-07T02:43:22.956 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:22.956 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-05-07T02:43:22.956 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-05-07T02:43:22.956 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-05-07T02:43:22.957 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 0, 2024-05-07T02:43:22.957 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 0, 2024-05-07T02:43:22.957 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-05-07T02:43:22.957 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-05-07T02:43:22.957 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-05-07T02:43:22.957 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-05-07T02:43:22.957 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:22.957 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-05-07T02:43:22.957 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-05-07T02:43:24.039 INFO:tasks.check_counter:Daemon mds.j mds.exported=0 2024-05-07T02:43:24.039 INFO:tasks.check_counter:Daemon mds.j mds.imported=0 2024-05-07T02:43:24.039 INFO:tasks.check_counter:Daemon mds.j mds.root_rsnaps=0 2024-05-07T02:43:24.040 INFO:tasks.check_counter:Daemon mds.j mds_server.req_mksnap_latency.avgcount=0 2024-05-07T02:43:24.040 INFO:tasks.check_counter:Daemon mds.j mds_server.req_rmsnap_latency.avgcount=0 2024-05-07T02:43:24.040 DEBUG:tasks.check_counter:Getting stats from b 2024-05-07T02:43:24.040 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-07T02:43:24.314 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.b/config 2024-05-07T02:43:24.361 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:43:24 smithi064 ceph-mon[34699]: pgmap v359: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-07T02:43:24.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:43:24 smithi019 ceph-mon[29781]: pgmap v359: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-07T02:43:24.437 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:43:24 smithi027 ceph-mon[34398]: pgmap v359: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-07T02:43:26.175 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-05-07T02:43:26.175 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-0": { 2024-05-07T02:43:26.175 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 34958, 2024-05-07T02:43:26.175 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 40270, 2024-05-07T02:43:26.175 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 8520605, 2024-05-07T02:43:26.176 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 46456437, 2024-05-07T02:43:26.176 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 93, 2024-05-07T02:43:26.176 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 6, 2024-05-07T02:43:26.176 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 2.677323036, 2024-05-07T02:43:26.176 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 1.132884076, 2024-05-07T02:43:26.176 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 1.525812075, 2024-05-07T02:43:26.176 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.212323329, 2024-05-07T02:43:26.176 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-05-07T02:43:26.176 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 40267, 2024-05-07T02:43:26.177 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 1.005150144, 2024-05-07T02:43:26.177 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000024962 2024-05-07T02:43:26.177 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.177 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-05-07T02:43:26.177 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 61512, 2024-05-07T02:43:26.177 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.011588160, 2024-05-07T02:43:26.177 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000188 2024-05-07T02:43:26.177 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.177 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 435248, 2024-05-07T02:43:26.178 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 130656 2024-05-07T02:43:26.178 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.178 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-1": { 2024-05-07T02:43:26.178 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 75756, 2024-05-07T02:43:26.178 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 90129, 2024-05-07T02:43:26.178 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 22207236, 2024-05-07T02:43:26.178 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 90082254, 2024-05-07T02:43:26.178 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 14, 2024-05-07T02:43:26.179 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 7, 2024-05-07T02:43:26.179 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 4.886495158, 2024-05-07T02:43:26.179 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 1.930551610, 2024-05-07T02:43:26.179 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 3.392307016, 2024-05-07T02:43:26.179 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.217657310, 2024-05-07T02:43:26.179 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-05-07T02:43:26.179 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 90128, 2024-05-07T02:43:26.179 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 1.679926139, 2024-05-07T02:43:26.179 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000018639 2024-05-07T02:43:26.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.180 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-05-07T02:43:26.180 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 134977, 2024-05-07T02:43:26.180 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.027532366, 2024-05-07T02:43:26.180 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000203 2024-05-07T02:43:26.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.180 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-05-07T02:43:26.180 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 207200 2024-05-07T02:43:26.181 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.181 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-2": { 2024-05-07T02:43:26.181 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 45618, 2024-05-07T02:43:26.181 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 56590, 2024-05-07T02:43:26.181 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 12750829, 2024-05-07T02:43:26.181 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 38648975, 2024-05-07T02:43:26.181 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 12, 2024-05-07T02:43:26.181 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 6, 2024-05-07T02:43:26.181 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 3.142004759, 2024-05-07T02:43:26.182 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 1.206627557, 2024-05-07T02:43:26.182 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 2.204334320, 2024-05-07T02:43:26.182 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.128453560, 2024-05-07T02:43:26.182 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-05-07T02:43:26.182 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 56589, 2024-05-07T02:43:26.182 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 1.078883539, 2024-05-07T02:43:26.182 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000019065 2024-05-07T02:43:26.182 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.182 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-05-07T02:43:26.183 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 91084, 2024-05-07T02:43:26.183 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.017358195, 2024-05-07T02:43:26.183 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000190 2024-05-07T02:43:26.183 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.183 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 1949904, 2024-05-07T02:43:26.183 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 541728 2024-05-07T02:43:26.183 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.183 INFO:teuthology.orchestra.run.smithi027.stdout: "cct": { 2024-05-07T02:43:26.184 INFO:teuthology.orchestra.run.smithi027.stdout: "total_workers": 1, 2024-05-07T02:43:26.184 INFO:teuthology.orchestra.run.smithi027.stdout: "unhealthy_workers": 0 2024-05-07T02:43:26.184 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.184 INFO:teuthology.orchestra.run.smithi027.stdout: "finisher-MDSRank": { 2024-05-07T02:43:26.184 INFO:teuthology.orchestra.run.smithi027.stdout: "queue_len": 0, 2024-05-07T02:43:26.184 INFO:teuthology.orchestra.run.smithi027.stdout: "complete_latency": { 2024-05-07T02:43:26.184 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 26219, 2024-05-07T02:43:26.184 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 14.080341770, 2024-05-07T02:43:26.184 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000537028 2024-05-07T02:43:26.185 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-07T02:43:26.185 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.185 INFO:teuthology.orchestra.run.smithi027.stdout: "finisher-PurgeQueue": { 2024-05-07T02:43:26.185 INFO:teuthology.orchestra.run.smithi027.stdout: "queue_len": 0, 2024-05-07T02:43:26.185 INFO:teuthology.orchestra.run.smithi027.stdout: "complete_latency": { 2024-05-07T02:43:26.185 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 4064, 2024-05-07T02:43:26.186 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.347643242, 2024-05-07T02:43:26.186 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000085542 2024-05-07T02:43:26.186 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-07T02:43:26.186 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.186 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2024-05-07T02:43:26.186 INFO:teuthology.orchestra.run.smithi027.stdout: "request": 26817, 2024-05-07T02:43:26.186 INFO:teuthology.orchestra.run.smithi027.stdout: "reply": 22735, 2024-05-07T02:43:26.186 INFO:teuthology.orchestra.run.smithi027.stdout: "reply_latency": { 2024-05-07T02:43:26.187 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 22735, 2024-05-07T02:43:26.187 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 171.965508954, 2024-05-07T02:43:26.187 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.007563910 2024-05-07T02:43:26.187 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.187 INFO:teuthology.orchestra.run.smithi027.stdout: "slow_reply": 0, 2024-05-07T02:43:26.187 INFO:teuthology.orchestra.run.smithi027.stdout: "forward": 4082, 2024-05-07T02:43:26.187 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_fetch_complete": 3, 2024-05-07T02:43:26.187 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_fetch_keys": 0, 2024-05-07T02:43:26.188 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_commit": 21, 2024-05-07T02:43:26.188 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_split": 1, 2024-05-07T02:43:26.188 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_merge": 0, 2024-05-07T02:43:26.188 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes": 134, 2024-05-07T02:43:26.188 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_top": 37, 2024-05-07T02:43:26.188 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_bottom": 56, 2024-05-07T02:43:26.188 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_pin_tail": 41, 2024-05-07T02:43:26.188 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_pinned": 82, 2024-05-07T02:43:26.188 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_expired": 364, 2024-05-07T02:43:26.189 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_with_caps": 22, 2024-05-07T02:43:26.189 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": 22, 2024-05-07T02:43:26.189 INFO:teuthology.orchestra.run.smithi027.stdout: "subtrees": 18, 2024-05-07T02:43:26.189 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse": 41076, 2024-05-07T02:43:26.189 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_hit": 30316, 2024-05-07T02:43:26.189 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_forward": 4076, 2024-05-07T02:43:26.189 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_discover": 30, 2024-05-07T02:43:26.189 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_dir_fetch": 0, 2024-05-07T02:43:26.189 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_remote_ino": 0, 2024-05-07T02:43:26.190 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_lock": 68, 2024-05-07T02:43:26.190 INFO:teuthology.orchestra.run.smithi027.stdout: "load_cent": 0, 2024-05-07T02:43:26.190 INFO:teuthology.orchestra.run.smithi027.stdout: "q": 0, 2024-05-07T02:43:26.190 INFO:teuthology.orchestra.run.smithi027.stdout: "exported": 13, 2024-05-07T02:43:26.190 INFO:teuthology.orchestra.run.smithi027.stdout: "exported_inodes": 221, 2024-05-07T02:43:26.190 INFO:teuthology.orchestra.run.smithi027.stdout: "imported": 1, 2024-05-07T02:43:26.190 INFO:teuthology.orchestra.run.smithi027.stdout: "imported_inodes": 25, 2024-05-07T02:43:26.190 INFO:teuthology.orchestra.run.smithi027.stdout: "openino_dir_fetch": 0, 2024-05-07T02:43:26.190 INFO:teuthology.orchestra.run.smithi027.stdout: "openino_backtrace_fetch": 495, 2024-05-07T02:43:26.191 INFO:teuthology.orchestra.run.smithi027.stdout: "openino_peer_discover": 0, 2024-05-07T02:43:26.191 INFO:teuthology.orchestra.run.smithi027.stdout: "root_rfiles": 67, 2024-05-07T02:43:26.191 INFO:teuthology.orchestra.run.smithi027.stdout: "root_rbytes": 21176629, 2024-05-07T02:43:26.191 INFO:teuthology.orchestra.run.smithi027.stdout: "root_rsnaps": 6, 2024-05-07T02:43:26.191 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_backtrace_fetch": 495, 2024-05-07T02:43:26.191 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_set_tag": 0, 2024-05-07T02:43:26.191 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_backtrace_repaired": 0, 2024-05-07T02:43:26.191 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_inotable_repaired": 0, 2024-05-07T02:43:26.192 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_dir_inodes": 196, 2024-05-07T02:43:26.192 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_dir_base_inodes": 10, 2024-05-07T02:43:26.192 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_dirfrag_rstats": 186, 2024-05-07T02:43:26.192 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_file_inodes": 299, 2024-05-07T02:43:26.192 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_inode_file_caps": 14, 2024-05-07T02:43:26.192 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_revoke": 9139, 2024-05-07T02:43:26.192 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_grant": 8568, 2024-05-07T02:43:26.192 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_trunc": 0, 2024-05-07T02:43:26.192 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-07T02:43:26.193 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-07T02:43:26.193 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_caps": 13275, 2024-05-07T02:43:26.193 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_caps_dirty": 2211, 2024-05-07T02:43:26.193 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_cap_release": 1281, 2024-05-07T02:43:26.193 INFO:teuthology.orchestra.run.smithi027.stdout: "process_request_cap_release": 9782 2024-05-07T02:43:26.193 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.193 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_cache": { 2024-05-07T02:43:26.193 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_update": 8, 2024-05-07T02:43:26.193 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_update_receipt": 0, 2024-05-07T02:43:26.194 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_try_discover": 0, 2024-05-07T02:43:26.194 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_send_discover": 30, 2024-05-07T02:43:26.194 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_handle_discover": 53, 2024-05-07T02:43:26.194 INFO:teuthology.orchestra.run.smithi027.stdout: "num_strays": 0, 2024-05-07T02:43:26.194 INFO:teuthology.orchestra.run.smithi027.stdout: "num_strays_delayed": 0, 2024-05-07T02:43:26.194 INFO:teuthology.orchestra.run.smithi027.stdout: "num_strays_enqueuing": 0, 2024-05-07T02:43:26.194 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_created": 1980, 2024-05-07T02:43:26.194 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_enqueued": 1980, 2024-05-07T02:43:26.194 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_reintegrated": 0, 2024-05-07T02:43:26.195 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_migrated": 0, 2024-05-07T02:43:26.195 INFO:teuthology.orchestra.run.smithi027.stdout: "num_recovering_processing": 0, 2024-05-07T02:43:26.195 INFO:teuthology.orchestra.run.smithi027.stdout: "num_recovering_enqueued": 0, 2024-05-07T02:43:26.195 INFO:teuthology.orchestra.run.smithi027.stdout: "num_recovering_prioritized": 0, 2024-05-07T02:43:26.195 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_started": 0, 2024-05-07T02:43:26.195 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_completed": 0, 2024-05-07T02:43:26.195 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_quiesce_path": 0, 2024-05-07T02:43:26.195 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_quiesce_inode": 0, 2024-05-07T02:43:26.196 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_enqueue_scrub": 10, 2024-05-07T02:43:26.196 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_exportdir": 17, 2024-05-07T02:43:26.196 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_flush": 0, 2024-05-07T02:43:26.196 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_fragmentdir": 1, 2024-05-07T02:43:26.196 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_fragstats": 0, 2024-05-07T02:43:26.196 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_inodestats": 0 2024-05-07T02:43:26.196 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.196 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_log": { 2024-05-07T02:43:26.196 INFO:teuthology.orchestra.run.smithi027.stdout: "evlrg": 0, 2024-05-07T02:43:26.197 INFO:teuthology.orchestra.run.smithi027.stdout: "evadd": 20028, 2024-05-07T02:43:26.197 INFO:teuthology.orchestra.run.smithi027.stdout: "evex": 0, 2024-05-07T02:43:26.197 INFO:teuthology.orchestra.run.smithi027.stdout: "evtrm": 0, 2024-05-07T02:43:26.197 INFO:teuthology.orchestra.run.smithi027.stdout: "ev": 20028, 2024-05-07T02:43:26.197 INFO:teuthology.orchestra.run.smithi027.stdout: "evexg": 0, 2024-05-07T02:43:26.197 INFO:teuthology.orchestra.run.smithi027.stdout: "evexd": 0, 2024-05-07T02:43:26.197 INFO:teuthology.orchestra.run.smithi027.stdout: "segadd": 30, 2024-05-07T02:43:26.197 INFO:teuthology.orchestra.run.smithi027.stdout: "segex": 0, 2024-05-07T02:43:26.197 INFO:teuthology.orchestra.run.smithi027.stdout: "segtrm": 0, 2024-05-07T02:43:26.198 INFO:teuthology.orchestra.run.smithi027.stdout: "seg": 30, 2024-05-07T02:43:26.198 INFO:teuthology.orchestra.run.smithi027.stdout: "segmjr": 3, 2024-05-07T02:43:26.198 INFO:teuthology.orchestra.run.smithi027.stdout: "segexg": 0, 2024-05-07T02:43:26.198 INFO:teuthology.orchestra.run.smithi027.stdout: "segexd": 0, 2024-05-07T02:43:26.198 INFO:teuthology.orchestra.run.smithi027.stdout: "expos": 4194304, 2024-05-07T02:43:26.198 INFO:teuthology.orchestra.run.smithi027.stdout: "wrpos": 94439849, 2024-05-07T02:43:26.198 INFO:teuthology.orchestra.run.smithi027.stdout: "rdpos": 0, 2024-05-07T02:43:26.198 INFO:teuthology.orchestra.run.smithi027.stdout: "jlat": { 2024-05-07T02:43:26.198 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 12977, 2024-05-07T02:43:26.199 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 35.898430444, 2024-05-07T02:43:26.199 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.002766311 2024-05-07T02:43:26.199 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.199 INFO:teuthology.orchestra.run.smithi027.stdout: "replayed": 0 2024-05-07T02:43:26.199 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.199 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_mem": { 2024-05-07T02:43:26.199 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 126, 2024-05-07T02:43:26.199 INFO:teuthology.orchestra.run.smithi027.stdout: "ino+": 2335, 2024-05-07T02:43:26.199 INFO:teuthology.orchestra.run.smithi027.stdout: "ino-": 2209, 2024-05-07T02:43:26.200 INFO:teuthology.orchestra.run.smithi027.stdout: "dir": 49, 2024-05-07T02:43:26.200 INFO:teuthology.orchestra.run.smithi027.stdout: "dir+": 57, 2024-05-07T02:43:26.200 INFO:teuthology.orchestra.run.smithi027.stdout: "dir-": 8, 2024-05-07T02:43:26.200 INFO:teuthology.orchestra.run.smithi027.stdout: "dn": 134, 2024-05-07T02:43:26.200 INFO:teuthology.orchestra.run.smithi027.stdout: "dn+": 4653, 2024-05-07T02:43:26.200 INFO:teuthology.orchestra.run.smithi027.stdout: "dn-": 4519, 2024-05-07T02:43:26.200 INFO:teuthology.orchestra.run.smithi027.stdout: "cap": 22, 2024-05-07T02:43:26.200 INFO:teuthology.orchestra.run.smithi027.stdout: "cap+": 2356, 2024-05-07T02:43:26.201 INFO:teuthology.orchestra.run.smithi027.stdout: "cap-": 2334, 2024-05-07T02:43:26.201 INFO:teuthology.orchestra.run.smithi027.stdout: "rss": 139744, 2024-05-07T02:43:26.201 INFO:teuthology.orchestra.run.smithi027.stdout: "heap": 231824 2024-05-07T02:43:26.201 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.201 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_server": { 2024-05-07T02:43:26.201 INFO:teuthology.orchestra.run.smithi027.stdout: "dispatch_client_request": 39634, 2024-05-07T02:43:26.201 INFO:teuthology.orchestra.run.smithi027.stdout: "dispatch_server_request": 15019, 2024-05-07T02:43:26.201 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_request": 26817, 2024-05-07T02:43:26.201 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_session": 304, 2024-05-07T02:43:26.202 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_peer_request": 30078, 2024-05-07T02:43:26.202 INFO:teuthology.orchestra.run.smithi027.stdout: "req_create_latency": { 2024-05-07T02:43:26.202 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 2055, 2024-05-07T02:43:26.202 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 2.911613711, 2024-05-07T02:43:26.202 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.001416843 2024-05-07T02:43:26.202 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.202 INFO:teuthology.orchestra.run.smithi027.stdout: "req_getattr_latency": { 2024-05-07T02:43:26.202 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 9570, 2024-05-07T02:43:26.202 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 72.032074465, 2024-05-07T02:43:26.203 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.007526862 2024-05-07T02:43:26.203 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.203 INFO:teuthology.orchestra.run.smithi027.stdout: "req_getfilelock_latency": { 2024-05-07T02:43:26.203 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:26.203 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:26.203 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:26.203 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.203 INFO:teuthology.orchestra.run.smithi027.stdout: "req_link_latency": { 2024-05-07T02:43:26.203 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:26.204 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:26.204 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:26.204 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.204 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookup_latency": { 2024-05-07T02:43:26.204 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 2941, 2024-05-07T02:43:26.204 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 63.693602170, 2024-05-07T02:43:26.204 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.021657124 2024-05-07T02:43:26.204 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.204 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookuphash_latency": { 2024-05-07T02:43:26.205 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:26.205 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:26.205 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:26.205 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.205 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupino_latency": { 2024-05-07T02:43:26.205 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 1, 2024-05-07T02:43:26.205 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000186875, 2024-05-07T02:43:26.205 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000186875 2024-05-07T02:43:26.205 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.206 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupname_latency": { 2024-05-07T02:43:26.206 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:26.206 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:26.206 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:26.206 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.206 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupparent_latency": { 2024-05-07T02:43:26.206 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:26.206 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:26.207 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:26.207 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.207 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupsnap_latency": { 2024-05-07T02:43:26.207 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:26.207 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:26.207 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:26.207 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.207 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lssnap_latency": { 2024-05-07T02:43:26.207 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 11, 2024-05-07T02:43:26.208 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.002687863, 2024-05-07T02:43:26.208 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000244351 2024-05-07T02:43:26.208 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.208 INFO:teuthology.orchestra.run.smithi027.stdout: "req_mkdir_latency": { 2024-05-07T02:43:26.208 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 20, 2024-05-07T02:43:26.208 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.025916994, 2024-05-07T02:43:26.208 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.001295849 2024-05-07T02:43:26.208 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.209 INFO:teuthology.orchestra.run.smithi027.stdout: "req_mknod_latency": { 2024-05-07T02:43:26.209 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:26.209 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:26.209 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:26.209 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.209 INFO:teuthology.orchestra.run.smithi027.stdout: "req_mksnap_latency": { 2024-05-07T02:43:26.209 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 11, 2024-05-07T02:43:26.209 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 2.368697799, 2024-05-07T02:43:26.209 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.215336163 2024-05-07T02:43:26.210 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.210 INFO:teuthology.orchestra.run.smithi027.stdout: "req_open_latency": { 2024-05-07T02:43:26.210 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 1, 2024-05-07T02:43:26.210 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000245127, 2024-05-07T02:43:26.210 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000245127 2024-05-07T02:43:26.210 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.210 INFO:teuthology.orchestra.run.smithi027.stdout: "req_readdir_latency": { 2024-05-07T02:43:26.210 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 5797, 2024-05-07T02:43:26.211 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 8.530737333, 2024-05-07T02:43:26.211 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.001471577 2024-05-07T02:43:26.211 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.211 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rename_latency": { 2024-05-07T02:43:26.211 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 297, 2024-05-07T02:43:26.211 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 19.032522412, 2024-05-07T02:43:26.211 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.064082567 2024-05-07T02:43:26.211 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.211 INFO:teuthology.orchestra.run.smithi027.stdout: "req_renamesnap_latency": { 2024-05-07T02:43:26.212 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:26.212 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:26.212 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:26.212 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.212 INFO:teuthology.orchestra.run.smithi027.stdout: "req_snapdiff_latency": { 2024-05-07T02:43:26.212 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:26.212 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:26.212 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:26.212 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.213 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rmdir_latency": { 2024-05-07T02:43:26.213 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:26.213 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:26.213 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:26.213 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.213 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rmsnap_latency": { 2024-05-07T02:43:26.213 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 5, 2024-05-07T02:43:26.213 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.029754134, 2024-05-07T02:43:26.214 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.005950826 2024-05-07T02:43:26.214 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.214 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rmxattr_latency": { 2024-05-07T02:43:26.214 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:26.214 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:26.214 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:26.214 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.214 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setattr_latency": { 2024-05-07T02:43:26.215 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 2, 2024-05-07T02:43:26.215 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.001408191, 2024-05-07T02:43:26.215 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000704095 2024-05-07T02:43:26.215 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.215 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setdirlayout_latency": { 2024-05-07T02:43:26.215 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:26.215 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:26.215 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:26.216 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.216 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setfilelock_latency": { 2024-05-07T02:43:26.216 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 36, 2024-05-07T02:43:26.216 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.013023379, 2024-05-07T02:43:26.216 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000361760 2024-05-07T02:43:26.216 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.216 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setlayout_latency": { 2024-05-07T02:43:26.216 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:26.216 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:26.217 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:26.217 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.217 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setxattr_latency": { 2024-05-07T02:43:26.217 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 8, 2024-05-07T02:43:26.217 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.024637601, 2024-05-07T02:43:26.217 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.003079700 2024-05-07T02:43:26.217 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.217 INFO:teuthology.orchestra.run.smithi027.stdout: "req_symlink_latency": { 2024-05-07T02:43:26.217 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:26.218 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:26.218 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:26.218 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.218 INFO:teuthology.orchestra.run.smithi027.stdout: "req_unlink_latency": { 2024-05-07T02:43:26.218 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 1980, 2024-05-07T02:43:26.218 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 3.298400900, 2024-05-07T02:43:26.218 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.001665859 2024-05-07T02:43:26.218 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.219 INFO:teuthology.orchestra.run.smithi027.stdout: "cap_revoke_eviction": 0, 2024-05-07T02:43:26.219 INFO:teuthology.orchestra.run.smithi027.stdout: "cap_acquisition_throttle": 0, 2024-05-07T02:43:26.219 INFO:teuthology.orchestra.run.smithi027.stdout: "req_getvxattr_latency": { 2024-05-07T02:43:26.219 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:26.219 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:26.219 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:26.219 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-07T02:43:26.219 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.219 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_sessions": { 2024-05-07T02:43:26.220 INFO:teuthology.orchestra.run.smithi027.stdout: "session_count": 2, 2024-05-07T02:43:26.220 INFO:teuthology.orchestra.run.smithi027.stdout: "session_add": 5, 2024-05-07T02:43:26.220 INFO:teuthology.orchestra.run.smithi027.stdout: "session_remove": 3, 2024-05-07T02:43:26.220 INFO:teuthology.orchestra.run.smithi027.stdout: "sessions_open": 2, 2024-05-07T02:43:26.220 INFO:teuthology.orchestra.run.smithi027.stdout: "sessions_stale": 0, 2024-05-07T02:43:26.220 INFO:teuthology.orchestra.run.smithi027.stdout: "total_load": 385, 2024-05-07T02:43:26.220 INFO:teuthology.orchestra.run.smithi027.stdout: "average_load": 192, 2024-05-07T02:43:26.220 INFO:teuthology.orchestra.run.smithi027.stdout: "avg_session_uptime": 660, 2024-05-07T02:43:26.220 INFO:teuthology.orchestra.run.smithi027.stdout: "mdthresh_evicted": 0 2024-05-07T02:43:26.221 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.221 INFO:teuthology.orchestra.run.smithi027.stdout: "mempool": { 2024-05-07T02:43:26.221 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_bytes": 0, 2024-05-07T02:43:26.221 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_items": 0, 2024-05-07T02:43:26.221 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_bytes": 0, 2024-05-07T02:43:26.221 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_items": 0, 2024-05-07T02:43:26.221 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_bytes": 0, 2024-05-07T02:43:26.221 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_items": 0, 2024-05-07T02:43:26.222 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-07T02:43:26.222 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_items": 0, 2024-05-07T02:43:26.222 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-07T02:43:26.222 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_items": 0, 2024-05-07T02:43:26.222 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_bytes": 0, 2024-05-07T02:43:26.222 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_items": 0, 2024-05-07T02:43:26.222 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-07T02:43:26.222 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_items": 0, 2024-05-07T02:43:26.223 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_bytes": 0, 2024-05-07T02:43:26.223 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_items": 0, 2024-05-07T02:43:26.223 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_bytes": 0, 2024-05-07T02:43:26.223 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_items": 0, 2024-05-07T02:43:26.223 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-07T02:43:26.223 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_items": 0, 2024-05-07T02:43:26.223 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-07T02:43:26.223 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_items": 0, 2024-05-07T02:43:26.224 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_bytes": 0, 2024-05-07T02:43:26.224 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_items": 0, 2024-05-07T02:43:26.224 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_bytes": 0, 2024-05-07T02:43:26.224 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_items": 0, 2024-05-07T02:43:26.224 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-07T02:43:26.224 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_items": 0, 2024-05-07T02:43:26.224 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_bytes": 0, 2024-05-07T02:43:26.224 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_items": 0, 2024-05-07T02:43:26.224 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_bytes": 0, 2024-05-07T02:43:26.225 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_items": 0, 2024-05-07T02:43:26.225 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_bytes": 0, 2024-05-07T02:43:26.225 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_items": 0, 2024-05-07T02:43:26.225 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_bytes": 0, 2024-05-07T02:43:26.225 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_items": 0, 2024-05-07T02:43:26.225 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_bytes": 343044, 2024-05-07T02:43:26.225 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_items": 192, 2024-05-07T02:43:26.225 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_bytes": 0, 2024-05-07T02:43:26.225 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_items": 0, 2024-05-07T02:43:26.226 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_bytes": 0, 2024-05-07T02:43:26.226 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_items": 0, 2024-05-07T02:43:26.226 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_bytes": 0, 2024-05-07T02:43:26.226 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_items": 0, 2024-05-07T02:43:26.226 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_bytes": 0, 2024-05-07T02:43:26.226 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_items": 0, 2024-05-07T02:43:26.226 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_bytes": 9608, 2024-05-07T02:43:26.226 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_items": 184, 2024-05-07T02:43:26.227 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_bytes": 0, 2024-05-07T02:43:26.227 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_items": 0, 2024-05-07T02:43:26.227 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_bytes": 0, 2024-05-07T02:43:26.227 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_items": 0, 2024-05-07T02:43:26.227 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_bytes": 641109, 2024-05-07T02:43:26.227 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_items": 5904, 2024-05-07T02:43:26.227 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_bytes": 0, 2024-05-07T02:43:26.227 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_items": 0, 2024-05-07T02:43:26.227 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_bytes": 0, 2024-05-07T02:43:26.228 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_items": 0 2024-05-07T02:43:26.228 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.228 INFO:teuthology.orchestra.run.smithi027.stdout: "objecter": { 2024-05-07T02:43:26.228 INFO:teuthology.orchestra.run.smithi027.stdout: "op_active": 0, 2024-05-07T02:43:26.228 INFO:teuthology.orchestra.run.smithi027.stdout: "op_laggy": 0, 2024-05-07T02:43:26.228 INFO:teuthology.orchestra.run.smithi027.stdout: "op_send": 18605, 2024-05-07T02:43:26.228 INFO:teuthology.orchestra.run.smithi027.stdout: "op_send_bytes": 90837328, 2024-05-07T02:43:26.228 INFO:teuthology.orchestra.run.smithi027.stdout: "op_resend": 37, 2024-05-07T02:43:26.228 INFO:teuthology.orchestra.run.smithi027.stdout: "op_reply": 18605, 2024-05-07T02:43:26.228 INFO:teuthology.orchestra.run.smithi027.stdout: "op_latency": { 2024-05-07T02:43:26.229 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 18605, 2024-05-07T02:43:26.229 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 78.714985839, 2024-05-07T02:43:26.229 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.004230851 2024-05-07T02:43:26.229 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.229 INFO:teuthology.orchestra.run.smithi027.stdout: "op_inflight": 0, 2024-05-07T02:43:26.229 INFO:teuthology.orchestra.run.smithi027.stdout: "oplen_avg": { 2024-05-07T02:43:26.229 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 18605, 2024-05-07T02:43:26.229 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 18909 2024-05-07T02:43:26.229 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.230 INFO:teuthology.orchestra.run.smithi027.stdout: "op": 18605, 2024-05-07T02:43:26.230 INFO:teuthology.orchestra.run.smithi027.stdout: "op_r": 638, 2024-05-07T02:43:26.230 INFO:teuthology.orchestra.run.smithi027.stdout: "op_w": 17967, 2024-05-07T02:43:26.230 INFO:teuthology.orchestra.run.smithi027.stdout: "op_rmw": 0, 2024-05-07T02:43:26.230 INFO:teuthology.orchestra.run.smithi027.stdout: "op_pg": 0, 2024-05-07T02:43:26.230 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_stat": 0, 2024-05-07T02:43:26.230 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_create": 11, 2024-05-07T02:43:26.230 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_read": 130, 2024-05-07T02:43:26.231 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_write": 13097, 2024-05-07T02:43:26.231 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_writefull": 111, 2024-05-07T02:43:26.231 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_writesame": 0, 2024-05-07T02:43:26.231 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_append": 0, 2024-05-07T02:43:26.231 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_zero": 0, 2024-05-07T02:43:26.231 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_truncate": 1, 2024-05-07T02:43:26.231 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_delete": 4493, 2024-05-07T02:43:26.231 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_mapext": 0, 2024-05-07T02:43:26.231 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_sparse_read": 0, 2024-05-07T02:43:26.232 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_clonerange": 0, 2024-05-07T02:43:26.232 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_getxattr": 505, 2024-05-07T02:43:26.232 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_setxattr": 22, 2024-05-07T02:43:26.232 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_cmpxattr": 0, 2024-05-07T02:43:26.232 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_rmxattr": 0, 2024-05-07T02:43:26.232 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_resetxattrs": 0, 2024-05-07T02:43:26.232 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_call": 0, 2024-05-07T02:43:26.232 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_watch": 0, 2024-05-07T02:43:26.232 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_notify": 0, 2024-05-07T02:43:26.233 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_src_cmpxattr": 0, 2024-05-07T02:43:26.233 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_pgls": 0, 2024-05-07T02:43:26.233 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_pgls_filter": 0, 2024-05-07T02:43:26.233 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_other": 0, 2024-05-07T02:43:26.233 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_active": 0, 2024-05-07T02:43:26.233 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_send": 0, 2024-05-07T02:43:26.233 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_resend": 0, 2024-05-07T02:43:26.233 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_ping": 0, 2024-05-07T02:43:26.233 INFO:teuthology.orchestra.run.smithi027.stdout: "poolop_active": 0, 2024-05-07T02:43:26.234 INFO:teuthology.orchestra.run.smithi027.stdout: "poolop_send": 0, 2024-05-07T02:43:26.234 INFO:teuthology.orchestra.run.smithi027.stdout: "poolop_resend": 0, 2024-05-07T02:43:26.234 INFO:teuthology.orchestra.run.smithi027.stdout: "poolstat_active": 0, 2024-05-07T02:43:26.234 INFO:teuthology.orchestra.run.smithi027.stdout: "poolstat_send": 0, 2024-05-07T02:43:26.234 INFO:teuthology.orchestra.run.smithi027.stdout: "poolstat_resend": 0, 2024-05-07T02:43:26.234 INFO:teuthology.orchestra.run.smithi027.stdout: "statfs_active": 0, 2024-05-07T02:43:26.234 INFO:teuthology.orchestra.run.smithi027.stdout: "statfs_send": 0, 2024-05-07T02:43:26.234 INFO:teuthology.orchestra.run.smithi027.stdout: "statfs_resend": 0, 2024-05-07T02:43:26.235 INFO:teuthology.orchestra.run.smithi027.stdout: "command_active": 0, 2024-05-07T02:43:26.235 INFO:teuthology.orchestra.run.smithi027.stdout: "command_send": 0, 2024-05-07T02:43:26.235 INFO:teuthology.orchestra.run.smithi027.stdout: "command_resend": 0, 2024-05-07T02:43:26.235 INFO:teuthology.orchestra.run.smithi027.stdout: "map_epoch": 100, 2024-05-07T02:43:26.235 INFO:teuthology.orchestra.run.smithi027.stdout: "map_full": 0, 2024-05-07T02:43:26.235 INFO:teuthology.orchestra.run.smithi027.stdout: "map_inc": 22, 2024-05-07T02:43:26.235 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_sessions": 12, 2024-05-07T02:43:26.235 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_session_open": 12, 2024-05-07T02:43:26.235 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_session_close": 0, 2024-05-07T02:43:26.236 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_laggy": 0, 2024-05-07T02:43:26.236 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_wr": 451, 2024-05-07T02:43:26.236 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_rd": 6, 2024-05-07T02:43:26.236 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_del": 82 2024-05-07T02:43:26.236 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.236 INFO:teuthology.orchestra.run.smithi027.stdout: "oft": { 2024-05-07T02:43:26.236 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_objs": 1, 2024-05-07T02:43:26.236 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_kv_pairs": 0, 2024-05-07T02:43:26.237 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_updates": 0, 2024-05-07T02:43:26.237 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_removes": 0 2024-05-07T02:43:26.237 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.237 INFO:teuthology.orchestra.run.smithi027.stdout: "osd-slow-ops": { 2024-05-07T02:43:26.237 INFO:teuthology.orchestra.run.smithi027.stdout: "slow_ops_count": 0 2024-05-07T02:43:26.237 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.237 INFO:teuthology.orchestra.run.smithi027.stdout: "purge_queue": { 2024-05-07T02:43:26.237 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing_ops": 0, 2024-05-07T02:43:26.237 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing_ops_high_water": 10, 2024-05-07T02:43:26.237 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing": 0, 2024-05-07T02:43:26.238 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing_high_water": 5, 2024-05-07T02:43:26.238 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executed_ops": 3960, 2024-05-07T02:43:26.238 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executed": 1980, 2024-05-07T02:43:26.238 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_item_in_journal": 0 2024-05-07T02:43:26.238 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.238 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-07T02:43:26.238 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-05-07T02:43:26.238 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-05-07T02:43:26.239 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-05-07T02:43:26.239 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 156328, 2024-05-07T02:43:26.239 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 29339760, 2024-05-07T02:43:26.239 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:26.239 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 156328, 2024-05-07T02:43:26.239 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-05-07T02:43:26.239 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-05-07T02:43:26.239 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 156328, 2024-05-07T02:43:26.239 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 29339760, 2024-05-07T02:43:26.240 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-05-07T02:43:26.240 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:26.240 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:26.240 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:26.240 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-07T02:43:26.240 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.240 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-objecter_bytes": { 2024-05-07T02:43:26.240 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-05-07T02:43:26.240 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-05-07T02:43:26.241 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-05-07T02:43:26.241 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-05-07T02:43:26.241 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-05-07T02:43:26.241 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:26.241 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-05-07T02:43:26.241 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 18605, 2024-05-07T02:43:26.241 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 91053252, 2024-05-07T02:43:26.241 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 18605, 2024-05-07T02:43:26.241 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 91053252, 2024-05-07T02:43:26.242 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-05-07T02:43:26.242 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:26.242 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:26.242 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:26.242 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-07T02:43:26.242 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.242 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-objecter_ops": { 2024-05-07T02:43:26.242 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-05-07T02:43:26.242 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 1024, 2024-05-07T02:43:26.243 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-05-07T02:43:26.243 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-05-07T02:43:26.243 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-05-07T02:43:26.243 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:26.243 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-05-07T02:43:26.243 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 18605, 2024-05-07T02:43:26.243 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 18605, 2024-05-07T02:43:26.243 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 18605, 2024-05-07T02:43:26.243 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 18605, 2024-05-07T02:43:26.244 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-05-07T02:43:26.244 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:26.244 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:26.244 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:26.244 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-07T02:43:26.244 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.244 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-write_buf_throttle": { 2024-05-07T02:43:26.244 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-05-07T02:43:26.245 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 3758096384, 2024-05-07T02:43:26.245 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-05-07T02:43:26.245 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 1980, 2024-05-07T02:43:26.245 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 215972, 2024-05-07T02:43:26.245 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:26.245 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 1980, 2024-05-07T02:43:26.245 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-05-07T02:43:26.245 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-05-07T02:43:26.245 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 120, 2024-05-07T02:43:26.245 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 215972, 2024-05-07T02:43:26.246 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-05-07T02:43:26.246 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:26.246 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:26.246 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:26.246 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-07T02:43:26.246 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:26.246 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-write_buf_throttle-0x562b5c5ae4e0": { 2024-05-07T02:43:26.246 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-05-07T02:43:26.247 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 3758096384, 2024-05-07T02:43:26.247 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-05-07T02:43:26.247 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 20028, 2024-05-07T02:43:26.247 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 90245545, 2024-05-07T02:43:26.247 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:26.247 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 20028, 2024-05-07T02:43:26.247 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-05-07T02:43:26.247 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-05-07T02:43:26.247 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 12977, 2024-05-07T02:43:26.248 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 90245545, 2024-05-07T02:43:26.248 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-05-07T02:43:26.248 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:26.248 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:26.248 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:26.248 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-07T02:43:26.248 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-07T02:43:26.248 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-05-07T02:43:26.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:43:26 smithi064 ceph-mon[34699]: pgmap v360: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-07T02:43:26.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:43:26 smithi019 ceph-mon[29781]: pgmap v360: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-07T02:43:26.397 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:43:26 smithi027 ceph-mon[34398]: pgmap v360: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-07T02:43:27.032 INFO:tasks.check_counter:Daemon mds.b mds.exported=13 2024-05-07T02:43:27.032 INFO:tasks.check_counter:Daemon mds.b mds.imported=1 2024-05-07T02:43:27.032 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-05-07T02:43:27.032 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=11 2024-05-07T02:43:27.032 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=5 2024-05-07T02:43:27.032 DEBUG:tasks.check_counter:Getting stats from e 2024-05-07T02:43:27.033 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-07T02:43:27.290 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.b/config 2024-05-07T02:43:28.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:43:28 smithi064 ceph-mon[34699]: pgmap v361: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-07T02:43:28.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:43:28 smithi019 ceph-mon[29781]: pgmap v361: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-07T02:43:28.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:43:28 smithi027 ceph-mon[34398]: pgmap v361: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-07T02:43:29.104 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-05-07T02:43:29.104 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-0": { 2024-05-07T02:43:29.105 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 2398, 2024-05-07T02:43:29.105 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 2398, 2024-05-07T02:43:29.105 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 13270807, 2024-05-07T02:43:29.105 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 761747, 2024-05-07T02:43:29.105 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 6, 2024-05-07T02:43:29.105 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 2, 2024-05-07T02:43:29.105 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.209758406, 2024-05-07T02:43:29.105 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.064625579, 2024-05-07T02:43:29.106 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.061668470, 2024-05-07T02:43:29.106 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.065799103, 2024-05-07T02:43:29.106 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-05-07T02:43:29.106 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 2397, 2024-05-07T02:43:29.106 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.076155636, 2024-05-07T02:43:29.106 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000031771 2024-05-07T02:43:29.106 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.106 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-05-07T02:43:29.106 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:29.107 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:29.107 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:29.107 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.107 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-07T02:43:29.107 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 448 2024-05-07T02:43:29.107 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.107 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-1": { 2024-05-07T02:43:29.107 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 730, 2024-05-07T02:43:29.107 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 698, 2024-05-07T02:43:29.108 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 3308611, 2024-05-07T02:43:29.108 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 305840, 2024-05-07T02:43:29.108 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 4, 2024-05-07T02:43:29.108 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 3, 2024-05-07T02:43:29.108 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.098253199, 2024-05-07T02:43:29.108 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.027416472, 2024-05-07T02:43:29.108 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.061920399, 2024-05-07T02:43:29.108 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.018998034, 2024-05-07T02:43:29.108 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-05-07T02:43:29.109 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 697, 2024-05-07T02:43:29.109 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.026837042, 2024-05-07T02:43:29.109 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000038503 2024-05-07T02:43:29.109 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.109 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-05-07T02:43:29.109 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:29.109 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:29.109 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:29.109 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.110 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 1902368, 2024-05-07T02:43:29.110 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 181440 2024-05-07T02:43:29.110 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.110 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-2": { 2024-05-07T02:43:29.110 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 3, 2024-05-07T02:43:29.110 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 221, 2024-05-07T02:43:29.110 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 588, 2024-05-07T02:43:29.110 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 517022, 2024-05-07T02:43:29.110 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 7, 2024-05-07T02:43:29.111 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 4, 2024-05-07T02:43:29.111 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.013519570, 2024-05-07T02:43:29.111 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.008516444, 2024-05-07T02:43:29.111 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.002636843, 2024-05-07T02:43:29.111 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000017572, 2024-05-07T02:43:29.111 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-05-07T02:43:29.111 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 218, 2024-05-07T02:43:29.111 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.017652971, 2024-05-07T02:43:29.111 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000080976 2024-05-07T02:43:29.112 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.112 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-05-07T02:43:29.112 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:29.112 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:29.112 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:29.112 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.112 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-07T02:43:29.112 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 516704 2024-05-07T02:43:29.113 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.113 INFO:teuthology.orchestra.run.smithi027.stdout: "cct": { 2024-05-07T02:43:29.113 INFO:teuthology.orchestra.run.smithi027.stdout: "total_workers": 1, 2024-05-07T02:43:29.113 INFO:teuthology.orchestra.run.smithi027.stdout: "unhealthy_workers": 0 2024-05-07T02:43:29.113 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.113 INFO:teuthology.orchestra.run.smithi027.stdout: "finisher-MDSRank": { 2024-05-07T02:43:29.113 INFO:teuthology.orchestra.run.smithi027.stdout: "queue_len": 0, 2024-05-07T02:43:29.113 INFO:teuthology.orchestra.run.smithi027.stdout: "complete_latency": { 2024-05-07T02:43:29.113 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 4859, 2024-05-07T02:43:29.113 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.124394945, 2024-05-07T02:43:29.114 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000025600 2024-05-07T02:43:29.114 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-07T02:43:29.114 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.114 INFO:teuthology.orchestra.run.smithi027.stdout: "finisher-PurgeQueue": { 2024-05-07T02:43:29.114 INFO:teuthology.orchestra.run.smithi027.stdout: "queue_len": 0, 2024-05-07T02:43:29.114 INFO:teuthology.orchestra.run.smithi027.stdout: "complete_latency": { 2024-05-07T02:43:29.114 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:29.114 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:29.115 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:29.115 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-07T02:43:29.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.115 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2024-05-07T02:43:29.115 INFO:teuthology.orchestra.run.smithi027.stdout: "request": 0, 2024-05-07T02:43:29.115 INFO:teuthology.orchestra.run.smithi027.stdout: "reply": 0, 2024-05-07T02:43:29.115 INFO:teuthology.orchestra.run.smithi027.stdout: "reply_latency": { 2024-05-07T02:43:29.115 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:29.115 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:29.116 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:29.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.116 INFO:teuthology.orchestra.run.smithi027.stdout: "slow_reply": 0, 2024-05-07T02:43:29.116 INFO:teuthology.orchestra.run.smithi027.stdout: "forward": 0, 2024-05-07T02:43:29.116 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_fetch_complete": 0, 2024-05-07T02:43:29.116 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_fetch_keys": 0, 2024-05-07T02:43:29.116 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_commit": 0, 2024-05-07T02:43:29.116 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_split": 0, 2024-05-07T02:43:29.116 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_merge": 0, 2024-05-07T02:43:29.117 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes": 2406, 2024-05-07T02:43:29.117 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_top": 24, 2024-05-07T02:43:29.117 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_bottom": 2382, 2024-05-07T02:43:29.117 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_pin_tail": 0, 2024-05-07T02:43:29.117 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_pinned": 2372, 2024-05-07T02:43:29.117 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_expired": 0, 2024-05-07T02:43:29.117 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_with_caps": 0, 2024-05-07T02:43:29.117 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": 0, 2024-05-07T02:43:29.117 INFO:teuthology.orchestra.run.smithi027.stdout: "subtrees": 3, 2024-05-07T02:43:29.118 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse": 0, 2024-05-07T02:43:29.118 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_hit": 0, 2024-05-07T02:43:29.118 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_forward": 0, 2024-05-07T02:43:29.118 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_discover": 0, 2024-05-07T02:43:29.118 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_dir_fetch": 0, 2024-05-07T02:43:29.118 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_remote_ino": 0, 2024-05-07T02:43:29.118 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_lock": 0, 2024-05-07T02:43:29.118 INFO:teuthology.orchestra.run.smithi027.stdout: "load_cent": 0, 2024-05-07T02:43:29.119 INFO:teuthology.orchestra.run.smithi027.stdout: "q": 0, 2024-05-07T02:43:29.119 INFO:teuthology.orchestra.run.smithi027.stdout: "exported": 0, 2024-05-07T02:43:29.119 INFO:teuthology.orchestra.run.smithi027.stdout: "exported_inodes": 0, 2024-05-07T02:43:29.119 INFO:teuthology.orchestra.run.smithi027.stdout: "imported": 0, 2024-05-07T02:43:29.119 INFO:teuthology.orchestra.run.smithi027.stdout: "imported_inodes": 0, 2024-05-07T02:43:29.119 INFO:teuthology.orchestra.run.smithi027.stdout: "openino_dir_fetch": 0, 2024-05-07T02:43:29.119 INFO:teuthology.orchestra.run.smithi027.stdout: "openino_backtrace_fetch": 0, 2024-05-07T02:43:29.119 INFO:teuthology.orchestra.run.smithi027.stdout: "openino_peer_discover": 0, 2024-05-07T02:43:29.119 INFO:teuthology.orchestra.run.smithi027.stdout: "root_rfiles": 0, 2024-05-07T02:43:29.120 INFO:teuthology.orchestra.run.smithi027.stdout: "root_rbytes": 0, 2024-05-07T02:43:29.120 INFO:teuthology.orchestra.run.smithi027.stdout: "root_rsnaps": 0, 2024-05-07T02:43:29.120 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_backtrace_fetch": 0, 2024-05-07T02:43:29.120 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_set_tag": 0, 2024-05-07T02:43:29.120 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_backtrace_repaired": 0, 2024-05-07T02:43:29.120 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_inotable_repaired": 0, 2024-05-07T02:43:29.120 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_dir_inodes": 0, 2024-05-07T02:43:29.120 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_dir_base_inodes": 0, 2024-05-07T02:43:29.120 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_dirfrag_rstats": 0, 2024-05-07T02:43:29.121 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_file_inodes": 0, 2024-05-07T02:43:29.121 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_inode_file_caps": 0, 2024-05-07T02:43:29.121 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_revoke": 0, 2024-05-07T02:43:29.121 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_grant": 0, 2024-05-07T02:43:29.121 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_trunc": 0, 2024-05-07T02:43:29.121 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-07T02:43:29.121 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-07T02:43:29.121 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_caps": 0, 2024-05-07T02:43:29.121 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_caps_dirty": 0, 2024-05-07T02:43:29.121 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_cap_release": 0, 2024-05-07T02:43:29.122 INFO:teuthology.orchestra.run.smithi027.stdout: "process_request_cap_release": 0 2024-05-07T02:43:29.122 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.122 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_cache": { 2024-05-07T02:43:29.122 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_update": 0, 2024-05-07T02:43:29.122 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_update_receipt": 0, 2024-05-07T02:43:29.122 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_try_discover": 0, 2024-05-07T02:43:29.122 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_send_discover": 0, 2024-05-07T02:43:29.122 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_handle_discover": 0, 2024-05-07T02:43:29.123 INFO:teuthology.orchestra.run.smithi027.stdout: "num_strays": 0, 2024-05-07T02:43:29.123 INFO:teuthology.orchestra.run.smithi027.stdout: "num_strays_delayed": 0, 2024-05-07T02:43:29.123 INFO:teuthology.orchestra.run.smithi027.stdout: "num_strays_enqueuing": 0, 2024-05-07T02:43:29.123 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_created": 0, 2024-05-07T02:43:29.123 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_enqueued": 0, 2024-05-07T02:43:29.123 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_reintegrated": 0, 2024-05-07T02:43:29.123 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_migrated": 0, 2024-05-07T02:43:29.123 INFO:teuthology.orchestra.run.smithi027.stdout: "num_recovering_processing": 0, 2024-05-07T02:43:29.123 INFO:teuthology.orchestra.run.smithi027.stdout: "num_recovering_enqueued": 0, 2024-05-07T02:43:29.124 INFO:teuthology.orchestra.run.smithi027.stdout: "num_recovering_prioritized": 0, 2024-05-07T02:43:29.124 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_started": 0, 2024-05-07T02:43:29.124 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_completed": 0, 2024-05-07T02:43:29.124 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_quiesce_path": 0, 2024-05-07T02:43:29.124 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_quiesce_inode": 0, 2024-05-07T02:43:29.124 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_enqueue_scrub": 0, 2024-05-07T02:43:29.124 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_exportdir": 0, 2024-05-07T02:43:29.124 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_flush": 0, 2024-05-07T02:43:29.125 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_fragmentdir": 0, 2024-05-07T02:43:29.125 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_fragstats": 0, 2024-05-07T02:43:29.125 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_inodestats": 0 2024-05-07T02:43:29.125 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.125 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_log": { 2024-05-07T02:43:29.125 INFO:teuthology.orchestra.run.smithi027.stdout: "evlrg": 0, 2024-05-07T02:43:29.125 INFO:teuthology.orchestra.run.smithi027.stdout: "evadd": 0, 2024-05-07T02:43:29.125 INFO:teuthology.orchestra.run.smithi027.stdout: "evex": 0, 2024-05-07T02:43:29.125 INFO:teuthology.orchestra.run.smithi027.stdout: "evtrm": 0, 2024-05-07T02:43:29.126 INFO:teuthology.orchestra.run.smithi027.stdout: "ev": 9827, 2024-05-07T02:43:29.126 INFO:teuthology.orchestra.run.smithi027.stdout: "evexg": 0, 2024-05-07T02:43:29.126 INFO:teuthology.orchestra.run.smithi027.stdout: "evexd": 0, 2024-05-07T02:43:29.126 INFO:teuthology.orchestra.run.smithi027.stdout: "segadd": 0, 2024-05-07T02:43:29.126 INFO:teuthology.orchestra.run.smithi027.stdout: "segex": 0, 2024-05-07T02:43:29.126 INFO:teuthology.orchestra.run.smithi027.stdout: "segtrm": 0, 2024-05-07T02:43:29.126 INFO:teuthology.orchestra.run.smithi027.stdout: "seg": 11, 2024-05-07T02:43:29.126 INFO:teuthology.orchestra.run.smithi027.stdout: "segmjr": 2, 2024-05-07T02:43:29.127 INFO:teuthology.orchestra.run.smithi027.stdout: "segexg": 0, 2024-05-07T02:43:29.127 INFO:teuthology.orchestra.run.smithi027.stdout: "segexd": 0, 2024-05-07T02:43:29.127 INFO:teuthology.orchestra.run.smithi027.stdout: "expos": 4194304, 2024-05-07T02:43:29.127 INFO:teuthology.orchestra.run.smithi027.stdout: "wrpos": 18082553, 2024-05-07T02:43:29.127 INFO:teuthology.orchestra.run.smithi027.stdout: "rdpos": 18081954, 2024-05-07T02:43:29.127 INFO:teuthology.orchestra.run.smithi027.stdout: "jlat": { 2024-05-07T02:43:29.127 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:29.127 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:29.128 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:29.128 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.128 INFO:teuthology.orchestra.run.smithi027.stdout: "replayed": 9827 2024-05-07T02:43:29.128 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.128 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_mem": { 2024-05-07T02:43:29.128 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 46, 2024-05-07T02:43:29.128 INFO:teuthology.orchestra.run.smithi027.stdout: "ino+": 2380, 2024-05-07T02:43:29.128 INFO:teuthology.orchestra.run.smithi027.stdout: "ino-": 2334, 2024-05-07T02:43:29.128 INFO:teuthology.orchestra.run.smithi027.stdout: "dir": 22, 2024-05-07T02:43:29.129 INFO:teuthology.orchestra.run.smithi027.stdout: "dir+": 23, 2024-05-07T02:43:29.129 INFO:teuthology.orchestra.run.smithi027.stdout: "dir-": 1, 2024-05-07T02:43:29.129 INFO:teuthology.orchestra.run.smithi027.stdout: "dn": 2406, 2024-05-07T02:43:29.129 INFO:teuthology.orchestra.run.smithi027.stdout: "dn+": 2432, 2024-05-07T02:43:29.129 INFO:teuthology.orchestra.run.smithi027.stdout: "dn-": 26, 2024-05-07T02:43:29.129 INFO:teuthology.orchestra.run.smithi027.stdout: "cap": 0, 2024-05-07T02:43:29.129 INFO:teuthology.orchestra.run.smithi027.stdout: "cap+": 0, 2024-05-07T02:43:29.129 INFO:teuthology.orchestra.run.smithi027.stdout: "cap-": 0, 2024-05-07T02:43:29.129 INFO:teuthology.orchestra.run.smithi027.stdout: "rss": 123084, 2024-05-07T02:43:29.130 INFO:teuthology.orchestra.run.smithi027.stdout: "heap": 215440 2024-05-07T02:43:29.130 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.130 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_server": { 2024-05-07T02:43:29.130 INFO:teuthology.orchestra.run.smithi027.stdout: "dispatch_client_request": 0, 2024-05-07T02:43:29.130 INFO:teuthology.orchestra.run.smithi027.stdout: "dispatch_server_request": 0, 2024-05-07T02:43:29.130 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_request": 0, 2024-05-07T02:43:29.130 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_session": 0, 2024-05-07T02:43:29.130 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_peer_request": 0, 2024-05-07T02:43:29.131 INFO:teuthology.orchestra.run.smithi027.stdout: "req_create_latency": { 2024-05-07T02:43:29.131 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:29.131 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:29.131 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:29.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.131 INFO:teuthology.orchestra.run.smithi027.stdout: "req_getattr_latency": { 2024-05-07T02:43:29.131 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:29.131 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:29.132 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:29.132 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.132 INFO:teuthology.orchestra.run.smithi027.stdout: "req_getfilelock_latency": { 2024-05-07T02:43:29.132 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:29.132 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:29.132 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:29.132 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.132 INFO:teuthology.orchestra.run.smithi027.stdout: "req_link_latency": { 2024-05-07T02:43:29.132 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:29.132 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:29.133 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:29.133 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.133 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookup_latency": { 2024-05-07T02:43:29.133 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:29.133 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:29.133 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:29.133 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.133 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookuphash_latency": { 2024-05-07T02:43:29.134 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:29.134 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:29.134 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:29.134 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.134 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupino_latency": { 2024-05-07T02:43:29.134 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:29.134 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:29.134 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:29.134 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.135 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupname_latency": { 2024-05-07T02:43:29.135 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:29.135 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:29.135 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:29.135 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.135 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupparent_latency": { 2024-05-07T02:43:29.135 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:29.135 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:29.135 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:29.136 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.136 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupsnap_latency": { 2024-05-07T02:43:29.136 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:29.136 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:29.136 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:29.136 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.136 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lssnap_latency": { 2024-05-07T02:43:29.136 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:29.136 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:29.137 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:29.137 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.137 INFO:teuthology.orchestra.run.smithi027.stdout: "req_mkdir_latency": { 2024-05-07T02:43:29.138 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:29.138 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:29.138 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:29.138 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.138 INFO:teuthology.orchestra.run.smithi027.stdout: "req_mknod_latency": { 2024-05-07T02:43:29.138 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:29.138 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:29.138 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:29.139 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.139 INFO:teuthology.orchestra.run.smithi027.stdout: "req_mksnap_latency": { 2024-05-07T02:43:29.139 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:29.139 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:29.139 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:29.139 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.139 INFO:teuthology.orchestra.run.smithi027.stdout: "req_open_latency": { 2024-05-07T02:43:29.139 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:29.140 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:29.140 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:29.140 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.140 INFO:teuthology.orchestra.run.smithi027.stdout: "req_readdir_latency": { 2024-05-07T02:43:29.140 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:29.140 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:29.140 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:29.140 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.140 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rename_latency": { 2024-05-07T02:43:29.141 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:29.141 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:29.141 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:29.141 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.141 INFO:teuthology.orchestra.run.smithi027.stdout: "req_renamesnap_latency": { 2024-05-07T02:43:29.141 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:29.141 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:29.141 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:29.142 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.142 INFO:teuthology.orchestra.run.smithi027.stdout: "req_snapdiff_latency": { 2024-05-07T02:43:29.142 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:29.142 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:29.142 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:29.142 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.142 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rmdir_latency": { 2024-05-07T02:43:29.142 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:29.142 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:29.143 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:29.143 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.143 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rmsnap_latency": { 2024-05-07T02:43:29.143 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:29.143 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:29.143 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:29.143 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.143 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rmxattr_latency": { 2024-05-07T02:43:29.143 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:29.144 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:29.144 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:29.144 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.144 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setattr_latency": { 2024-05-07T02:43:29.144 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:29.144 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:29.144 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:29.144 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.144 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setdirlayout_latency": { 2024-05-07T02:43:29.145 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:29.145 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:29.145 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:29.145 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.145 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setfilelock_latency": { 2024-05-07T02:43:29.145 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:29.145 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:29.145 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:29.145 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.146 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setlayout_latency": { 2024-05-07T02:43:29.146 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:29.146 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:29.146 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:29.146 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.146 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setxattr_latency": { 2024-05-07T02:43:29.146 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:29.146 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:29.146 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:29.147 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.147 INFO:teuthology.orchestra.run.smithi027.stdout: "req_symlink_latency": { 2024-05-07T02:43:29.147 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:29.147 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:29.147 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:29.147 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.147 INFO:teuthology.orchestra.run.smithi027.stdout: "req_unlink_latency": { 2024-05-07T02:43:29.147 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:29.147 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:29.148 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:29.148 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.148 INFO:teuthology.orchestra.run.smithi027.stdout: "cap_revoke_eviction": 0, 2024-05-07T02:43:29.148 INFO:teuthology.orchestra.run.smithi027.stdout: "cap_acquisition_throttle": 0, 2024-05-07T02:43:29.148 INFO:teuthology.orchestra.run.smithi027.stdout: "req_getvxattr_latency": { 2024-05-07T02:43:29.148 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:29.148 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:29.148 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:29.148 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-07T02:43:29.149 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.149 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_sessions": { 2024-05-07T02:43:29.149 INFO:teuthology.orchestra.run.smithi027.stdout: "session_count": 1, 2024-05-07T02:43:29.149 INFO:teuthology.orchestra.run.smithi027.stdout: "session_add": 2, 2024-05-07T02:43:29.149 INFO:teuthology.orchestra.run.smithi027.stdout: "session_remove": 1, 2024-05-07T02:43:29.149 INFO:teuthology.orchestra.run.smithi027.stdout: "sessions_open": 2, 2024-05-07T02:43:29.149 INFO:teuthology.orchestra.run.smithi027.stdout: "sessions_stale": 0, 2024-05-07T02:43:29.149 INFO:teuthology.orchestra.run.smithi027.stdout: "total_load": 0, 2024-05-07T02:43:29.149 INFO:teuthology.orchestra.run.smithi027.stdout: "average_load": 0, 2024-05-07T02:43:29.150 INFO:teuthology.orchestra.run.smithi027.stdout: "avg_session_uptime": 4205, 2024-05-07T02:43:29.150 INFO:teuthology.orchestra.run.smithi027.stdout: "mdthresh_evicted": 0 2024-05-07T02:43:29.150 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.150 INFO:teuthology.orchestra.run.smithi027.stdout: "mempool": { 2024-05-07T02:43:29.150 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_bytes": 0, 2024-05-07T02:43:29.150 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_items": 0, 2024-05-07T02:43:29.150 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_bytes": 0, 2024-05-07T02:43:29.150 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_items": 0, 2024-05-07T02:43:29.151 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_bytes": 0, 2024-05-07T02:43:29.151 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_items": 0, 2024-05-07T02:43:29.151 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-07T02:43:29.151 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_items": 0, 2024-05-07T02:43:29.151 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-07T02:43:29.151 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_items": 0, 2024-05-07T02:43:29.151 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_bytes": 0, 2024-05-07T02:43:29.151 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_items": 0, 2024-05-07T02:43:29.151 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-07T02:43:29.152 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_items": 0, 2024-05-07T02:43:29.152 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_bytes": 0, 2024-05-07T02:43:29.152 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_items": 0, 2024-05-07T02:43:29.152 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_bytes": 0, 2024-05-07T02:43:29.152 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_items": 0, 2024-05-07T02:43:29.152 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-07T02:43:29.152 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_items": 0, 2024-05-07T02:43:29.152 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-07T02:43:29.152 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_items": 0, 2024-05-07T02:43:29.153 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_bytes": 0, 2024-05-07T02:43:29.153 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_items": 0, 2024-05-07T02:43:29.153 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_bytes": 0, 2024-05-07T02:43:29.153 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_items": 0, 2024-05-07T02:43:29.153 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-07T02:43:29.153 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_items": 0, 2024-05-07T02:43:29.153 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_bytes": 0, 2024-05-07T02:43:29.153 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_items": 0, 2024-05-07T02:43:29.153 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_bytes": 0, 2024-05-07T02:43:29.154 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_items": 0, 2024-05-07T02:43:29.154 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_bytes": 0, 2024-05-07T02:43:29.154 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_items": 0, 2024-05-07T02:43:29.154 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_bytes": 0, 2024-05-07T02:43:29.154 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_items": 0, 2024-05-07T02:43:29.154 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_bytes": 192033, 2024-05-07T02:43:29.154 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_items": 84, 2024-05-07T02:43:29.154 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_bytes": 0, 2024-05-07T02:43:29.154 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_items": 0, 2024-05-07T02:43:29.155 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_bytes": 0, 2024-05-07T02:43:29.155 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_items": 0, 2024-05-07T02:43:29.155 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_bytes": 0, 2024-05-07T02:43:29.155 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_items": 0, 2024-05-07T02:43:29.155 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_bytes": 0, 2024-05-07T02:43:29.155 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_items": 0, 2024-05-07T02:43:29.155 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_bytes": 9608, 2024-05-07T02:43:29.155 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_items": 184, 2024-05-07T02:43:29.155 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_bytes": 0, 2024-05-07T02:43:29.156 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_items": 0, 2024-05-07T02:43:29.156 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_bytes": 0, 2024-05-07T02:43:29.156 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_items": 0, 2024-05-07T02:43:29.156 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_bytes": 1853899, 2024-05-07T02:43:29.156 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_items": 10518, 2024-05-07T02:43:29.156 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_bytes": 0, 2024-05-07T02:43:29.156 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_items": 0, 2024-05-07T02:43:29.156 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_bytes": 0, 2024-05-07T02:43:29.156 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_items": 0 2024-05-07T02:43:29.157 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.157 INFO:teuthology.orchestra.run.smithi027.stdout: "objecter": { 2024-05-07T02:43:29.157 INFO:teuthology.orchestra.run.smithi027.stdout: "op_active": 0, 2024-05-07T02:43:29.157 INFO:teuthology.orchestra.run.smithi027.stdout: "op_laggy": 0, 2024-05-07T02:43:29.157 INFO:teuthology.orchestra.run.smithi027.stdout: "op_send": 2786, 2024-05-07T02:43:29.157 INFO:teuthology.orchestra.run.smithi027.stdout: "op_send_bytes": 21, 2024-05-07T02:43:29.157 INFO:teuthology.orchestra.run.smithi027.stdout: "op_resend": 3, 2024-05-07T02:43:29.157 INFO:teuthology.orchestra.run.smithi027.stdout: "op_reply": 2786, 2024-05-07T02:43:29.157 INFO:teuthology.orchestra.run.smithi027.stdout: "op_latency": { 2024-05-07T02:43:29.158 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 2786, 2024-05-07T02:43:29.158 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 1.730955767, 2024-05-07T02:43:29.158 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000621305 2024-05-07T02:43:29.158 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.158 INFO:teuthology.orchestra.run.smithi027.stdout: "op_inflight": 0, 2024-05-07T02:43:29.158 INFO:teuthology.orchestra.run.smithi027.stdout: "oplen_avg": { 2024-05-07T02:43:29.158 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 2786, 2024-05-07T02:43:29.158 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 2787 2024-05-07T02:43:29.158 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.159 INFO:teuthology.orchestra.run.smithi027.stdout: "op": 2786, 2024-05-07T02:43:29.159 INFO:teuthology.orchestra.run.smithi027.stdout: "op_r": 2786, 2024-05-07T02:43:29.159 INFO:teuthology.orchestra.run.smithi027.stdout: "op_w": 0, 2024-05-07T02:43:29.159 INFO:teuthology.orchestra.run.smithi027.stdout: "op_rmw": 0, 2024-05-07T02:43:29.159 INFO:teuthology.orchestra.run.smithi027.stdout: "op_pg": 0, 2024-05-07T02:43:29.159 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_stat": 1638, 2024-05-07T02:43:29.159 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_create": 0, 2024-05-07T02:43:29.159 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_read": 1146, 2024-05-07T02:43:29.159 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_write": 0, 2024-05-07T02:43:29.160 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_writefull": 0, 2024-05-07T02:43:29.160 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_writesame": 0, 2024-05-07T02:43:29.160 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_append": 0, 2024-05-07T02:43:29.160 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_zero": 0, 2024-05-07T02:43:29.160 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_truncate": 0, 2024-05-07T02:43:29.160 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_delete": 0, 2024-05-07T02:43:29.160 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_mapext": 0, 2024-05-07T02:43:29.160 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_sparse_read": 0, 2024-05-07T02:43:29.160 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_clonerange": 0, 2024-05-07T02:43:29.161 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_getxattr": 1, 2024-05-07T02:43:29.161 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_setxattr": 0, 2024-05-07T02:43:29.161 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_cmpxattr": 0, 2024-05-07T02:43:29.161 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_rmxattr": 0, 2024-05-07T02:43:29.161 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_resetxattrs": 0, 2024-05-07T02:43:29.161 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_call": 0, 2024-05-07T02:43:29.161 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_watch": 0, 2024-05-07T02:43:29.161 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_notify": 0, 2024-05-07T02:43:29.162 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_src_cmpxattr": 0, 2024-05-07T02:43:29.162 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_pgls": 0, 2024-05-07T02:43:29.162 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_pgls_filter": 0, 2024-05-07T02:43:29.162 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_other": 0, 2024-05-07T02:43:29.162 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_active": 0, 2024-05-07T02:43:29.162 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_send": 0, 2024-05-07T02:43:29.162 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_resend": 0, 2024-05-07T02:43:29.162 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_ping": 0, 2024-05-07T02:43:29.162 INFO:teuthology.orchestra.run.smithi027.stdout: "poolop_active": 0, 2024-05-07T02:43:29.163 INFO:teuthology.orchestra.run.smithi027.stdout: "poolop_send": 0, 2024-05-07T02:43:29.163 INFO:teuthology.orchestra.run.smithi027.stdout: "poolop_resend": 0, 2024-05-07T02:43:29.163 INFO:teuthology.orchestra.run.smithi027.stdout: "poolstat_active": 0, 2024-05-07T02:43:29.163 INFO:teuthology.orchestra.run.smithi027.stdout: "poolstat_send": 0, 2024-05-07T02:43:29.163 INFO:teuthology.orchestra.run.smithi027.stdout: "poolstat_resend": 0, 2024-05-07T02:43:29.163 INFO:teuthology.orchestra.run.smithi027.stdout: "statfs_active": 0, 2024-05-07T02:43:29.163 INFO:teuthology.orchestra.run.smithi027.stdout: "statfs_send": 0, 2024-05-07T02:43:29.163 INFO:teuthology.orchestra.run.smithi027.stdout: "statfs_resend": 0, 2024-05-07T02:43:29.163 INFO:teuthology.orchestra.run.smithi027.stdout: "command_active": 0, 2024-05-07T02:43:29.164 INFO:teuthology.orchestra.run.smithi027.stdout: "command_send": 0, 2024-05-07T02:43:29.164 INFO:teuthology.orchestra.run.smithi027.stdout: "command_resend": 0, 2024-05-07T02:43:29.164 INFO:teuthology.orchestra.run.smithi027.stdout: "map_epoch": 100, 2024-05-07T02:43:29.164 INFO:teuthology.orchestra.run.smithi027.stdout: "map_full": 0, 2024-05-07T02:43:29.164 INFO:teuthology.orchestra.run.smithi027.stdout: "map_inc": 13, 2024-05-07T02:43:29.164 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_sessions": 5, 2024-05-07T02:43:29.164 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_session_open": 6, 2024-05-07T02:43:29.164 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_session_close": 1, 2024-05-07T02:43:29.164 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_laggy": 0, 2024-05-07T02:43:29.165 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_wr": 0, 2024-05-07T02:43:29.165 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_rd": 2, 2024-05-07T02:43:29.165 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_del": 0 2024-05-07T02:43:29.165 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.165 INFO:teuthology.orchestra.run.smithi027.stdout: "oft": { 2024-05-07T02:43:29.165 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_objs": 0, 2024-05-07T02:43:29.165 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_kv_pairs": 0, 2024-05-07T02:43:29.165 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_updates": 0, 2024-05-07T02:43:29.166 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_removes": 0 2024-05-07T02:43:29.166 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.166 INFO:teuthology.orchestra.run.smithi027.stdout: "osd-slow-ops": { 2024-05-07T02:43:29.166 INFO:teuthology.orchestra.run.smithi027.stdout: "slow_ops_count": 0 2024-05-07T02:43:29.166 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.166 INFO:teuthology.orchestra.run.smithi027.stdout: "purge_queue": { 2024-05-07T02:43:29.166 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing_ops": 0, 2024-05-07T02:43:29.166 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing_ops_high_water": 0, 2024-05-07T02:43:29.166 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing": 0, 2024-05-07T02:43:29.167 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing_high_water": 0, 2024-05-07T02:43:29.167 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executed_ops": 0, 2024-05-07T02:43:29.167 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executed": 0, 2024-05-07T02:43:29.167 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_item_in_journal": 0 2024-05-07T02:43:29.167 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.167 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-07T02:43:29.167 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-05-07T02:43:29.167 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-05-07T02:43:29.167 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-05-07T02:43:29.168 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 3127, 2024-05-07T02:43:29.168 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 16278308, 2024-05-07T02:43:29.168 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:29.168 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 3127, 2024-05-07T02:43:29.168 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-05-07T02:43:29.168 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-05-07T02:43:29.168 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 3127, 2024-05-07T02:43:29.168 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 16278308, 2024-05-07T02:43:29.169 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-05-07T02:43:29.169 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:29.169 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:29.169 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:29.169 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-07T02:43:29.169 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.169 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-objecter_bytes": { 2024-05-07T02:43:29.169 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-05-07T02:43:29.169 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-05-07T02:43:29.170 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-05-07T02:43:29.170 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-05-07T02:43:29.170 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-05-07T02:43:29.170 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:29.170 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-05-07T02:43:29.170 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 2786, 2024-05-07T02:43:29.170 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 13888254, 2024-05-07T02:43:29.170 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 2786, 2024-05-07T02:43:29.170 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 13888254, 2024-05-07T02:43:29.171 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-05-07T02:43:29.171 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:29.171 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:29.171 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:29.171 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-07T02:43:29.171 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.171 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-objecter_ops": { 2024-05-07T02:43:29.171 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-05-07T02:43:29.172 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 1024, 2024-05-07T02:43:29.172 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-05-07T02:43:29.172 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-05-07T02:43:29.172 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-05-07T02:43:29.172 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:29.172 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-05-07T02:43:29.172 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 2786, 2024-05-07T02:43:29.172 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 2786, 2024-05-07T02:43:29.172 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 2786, 2024-05-07T02:43:29.173 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 2786, 2024-05-07T02:43:29.173 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-05-07T02:43:29.173 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:29.173 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:29.173 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:29.173 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-07T02:43:29.173 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.173 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-write_buf_throttle": { 2024-05-07T02:43:29.174 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-05-07T02:43:29.174 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 3758096384, 2024-05-07T02:43:29.174 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-05-07T02:43:29.174 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-05-07T02:43:29.174 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-05-07T02:43:29.174 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:29.174 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-05-07T02:43:29.174 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-05-07T02:43:29.175 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-05-07T02:43:29.175 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 0, 2024-05-07T02:43:29.175 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 0, 2024-05-07T02:43:29.175 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-05-07T02:43:29.175 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:29.175 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:29.175 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:29.175 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-07T02:43:29.175 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:29.176 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-write_buf_throttle-0x5587e905a410": { 2024-05-07T02:43:29.176 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-05-07T02:43:29.176 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 3758096384, 2024-05-07T02:43:29.176 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-05-07T02:43:29.176 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-05-07T02:43:29.176 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-05-07T02:43:29.176 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:29.176 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-05-07T02:43:29.176 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-05-07T02:43:29.177 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-05-07T02:43:29.177 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 0, 2024-05-07T02:43:29.177 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 0, 2024-05-07T02:43:29.177 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-05-07T02:43:29.177 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:29.177 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:29.177 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:29.177 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-07T02:43:29.177 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-07T02:43:29.178 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-05-07T02:43:29.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:43:29 smithi027 ceph-mon[34398]: pgmap v362: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:43:29.608 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:43:29 smithi064 ceph-mon[34699]: pgmap v362: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:43:29.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:43:29 smithi019 ceph-mon[29781]: pgmap v362: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-07T02:43:30.055 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-05-07T02:43:30.055 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-05-07T02:43:30.055 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-05-07T02:43:30.056 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-05-07T02:43:30.056 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-05-07T02:43:30.056 DEBUG:tasks.check_counter:Getting stats from h 2024-05-07T02:43:30.056 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-07T02:43:30.316 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.b/config 2024-05-07T02:43:30.439 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:43:30 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:43:30.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:43:30 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:43:30.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:43:30 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-07T02:43:31.427 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:43:31 smithi027 ceph-mon[34398]: pgmap v363: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:43:31.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:43:31 smithi064 ceph-mon[34699]: pgmap v363: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:43:31.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:43:31 smithi019 ceph-mon[29781]: pgmap v363: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:43:32.025 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-05-07T02:43:32.025 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-0": { 2024-05-07T02:43:32.025 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 20950, 2024-05-07T02:43:32.025 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 31492, 2024-05-07T02:43:32.026 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 8559239, 2024-05-07T02:43:32.026 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 28536589, 2024-05-07T02:43:32.026 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 35, 2024-05-07T02:43:32.026 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 8, 2024-05-07T02:43:32.026 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 1.501598033, 2024-05-07T02:43:32.026 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.594986290, 2024-05-07T02:43:32.026 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.987177183, 2024-05-07T02:43:32.027 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.083429025, 2024-05-07T02:43:32.027 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-05-07T02:43:32.027 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 31484, 2024-05-07T02:43:32.027 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.667566077, 2024-05-07T02:43:32.027 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000021203 2024-05-07T02:43:32.027 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.027 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-05-07T02:43:32.027 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 41045, 2024-05-07T02:43:32.028 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.008814969, 2024-05-07T02:43:32.028 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000214 2024-05-07T02:43:32.028 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.028 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 1928496, 2024-05-07T02:43:32.028 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 540256 2024-05-07T02:43:32.028 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.028 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-1": { 2024-05-07T02:43:32.028 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 36290, 2024-05-07T02:43:32.029 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 30305, 2024-05-07T02:43:32.029 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 10794384, 2024-05-07T02:43:32.029 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 12393393, 2024-05-07T02:43:32.029 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 14, 2024-05-07T02:43:32.029 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 5, 2024-05-07T02:43:32.029 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 2.259828485, 2024-05-07T02:43:32.029 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.951762406, 2024-05-07T02:43:32.029 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 1.527769873, 2024-05-07T02:43:32.029 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.106324553, 2024-05-07T02:43:32.030 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-05-07T02:43:32.030 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 30307, 2024-05-07T02:43:32.030 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.593814016, 2024-05-07T02:43:32.030 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000019593 2024-05-07T02:43:32.030 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.030 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-05-07T02:43:32.030 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 60295, 2024-05-07T02:43:32.030 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.010790947, 2024-05-07T02:43:32.031 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000178 2024-05-07T02:43:32.031 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.031 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 2048, 2024-05-07T02:43:32.031 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 263856 2024-05-07T02:43:32.031 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.031 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-2": { 2024-05-07T02:43:32.031 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 2941, 2024-05-07T02:43:32.031 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 2943, 2024-05-07T02:43:32.031 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 911711, 2024-05-07T02:43:32.032 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 8129764, 2024-05-07T02:43:32.032 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 9, 2024-05-07T02:43:32.032 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 6, 2024-05-07T02:43:32.032 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.226837894, 2024-05-07T02:43:32.032 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.068719263, 2024-05-07T02:43:32.032 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.064665550, 2024-05-07T02:43:32.032 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.081139963, 2024-05-07T02:43:32.032 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-05-07T02:43:32.032 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 2944, 2024-05-07T02:43:32.033 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.113410444, 2024-05-07T02:43:32.033 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000038522 2024-05-07T02:43:32.033 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.033 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-05-07T02:43:32.033 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 223, 2024-05-07T02:43:32.033 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000043491, 2024-05-07T02:43:32.033 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000195 2024-05-07T02:43:32.033 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.033 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-07T02:43:32.034 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 448 2024-05-07T02:43:32.034 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.034 INFO:teuthology.orchestra.run.smithi027.stdout: "cct": { 2024-05-07T02:43:32.034 INFO:teuthology.orchestra.run.smithi027.stdout: "total_workers": 1, 2024-05-07T02:43:32.034 INFO:teuthology.orchestra.run.smithi027.stdout: "unhealthy_workers": 0 2024-05-07T02:43:32.034 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.034 INFO:teuthology.orchestra.run.smithi027.stdout: "finisher-MDSRank": { 2024-05-07T02:43:32.035 INFO:teuthology.orchestra.run.smithi027.stdout: "queue_len": 0, 2024-05-07T02:43:32.035 INFO:teuthology.orchestra.run.smithi027.stdout: "complete_latency": { 2024-05-07T02:43:32.035 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 6862, 2024-05-07T02:43:32.035 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 4.650503810, 2024-05-07T02:43:32.035 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000677718 2024-05-07T02:43:32.035 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-07T02:43:32.035 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.035 INFO:teuthology.orchestra.run.smithi027.stdout: "finisher-PurgeQueue": { 2024-05-07T02:43:32.035 INFO:teuthology.orchestra.run.smithi027.stdout: "queue_len": 0, 2024-05-07T02:43:32.036 INFO:teuthology.orchestra.run.smithi027.stdout: "complete_latency": { 2024-05-07T02:43:32.036 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 4290, 2024-05-07T02:43:32.036 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.298915598, 2024-05-07T02:43:32.036 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000069677 2024-05-07T02:43:32.036 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-07T02:43:32.036 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.036 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2024-05-07T02:43:32.036 INFO:teuthology.orchestra.run.smithi027.stdout: "request": 12357, 2024-05-07T02:43:32.037 INFO:teuthology.orchestra.run.smithi027.stdout: "reply": 11818, 2024-05-07T02:43:32.037 INFO:teuthology.orchestra.run.smithi027.stdout: "reply_latency": { 2024-05-07T02:43:32.037 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 11818, 2024-05-07T02:43:32.037 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 15.354385832, 2024-05-07T02:43:32.037 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.001299237 2024-05-07T02:43:32.037 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.037 INFO:teuthology.orchestra.run.smithi027.stdout: "slow_reply": 0, 2024-05-07T02:43:32.037 INFO:teuthology.orchestra.run.smithi027.stdout: "forward": 539, 2024-05-07T02:43:32.037 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_fetch_complete": 0, 2024-05-07T02:43:32.038 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_fetch_keys": 0, 2024-05-07T02:43:32.038 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_commit": 11, 2024-05-07T02:43:32.038 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_split": 0, 2024-05-07T02:43:32.038 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_merge": 0, 2024-05-07T02:43:32.038 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes": 140, 2024-05-07T02:43:32.038 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_top": 12, 2024-05-07T02:43:32.038 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_bottom": 128, 2024-05-07T02:43:32.038 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_pin_tail": 0, 2024-05-07T02:43:32.038 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_pinned": 122, 2024-05-07T02:43:32.039 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_expired": 1, 2024-05-07T02:43:32.039 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_with_caps": 4, 2024-05-07T02:43:32.039 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": 4, 2024-05-07T02:43:32.039 INFO:teuthology.orchestra.run.smithi027.stdout: "subtrees": 6, 2024-05-07T02:43:32.039 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse": 30844, 2024-05-07T02:43:32.039 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_hit": 14781, 2024-05-07T02:43:32.039 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_forward": 539, 2024-05-07T02:43:32.039 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_discover": 9, 2024-05-07T02:43:32.040 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_dir_fetch": 0, 2024-05-07T02:43:32.040 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_remote_ino": 0, 2024-05-07T02:43:32.040 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_lock": 738, 2024-05-07T02:43:32.040 INFO:teuthology.orchestra.run.smithi027.stdout: "load_cent": 0, 2024-05-07T02:43:32.040 INFO:teuthology.orchestra.run.smithi027.stdout: "q": 0, 2024-05-07T02:43:32.040 INFO:teuthology.orchestra.run.smithi027.stdout: "exported": 0, 2024-05-07T02:43:32.040 INFO:teuthology.orchestra.run.smithi027.stdout: "exported_inodes": 0, 2024-05-07T02:43:32.040 INFO:teuthology.orchestra.run.smithi027.stdout: "imported": 4, 2024-05-07T02:43:32.041 INFO:teuthology.orchestra.run.smithi027.stdout: "imported_inodes": 41, 2024-05-07T02:43:32.041 INFO:teuthology.orchestra.run.smithi027.stdout: "openino_dir_fetch": 0, 2024-05-07T02:43:32.041 INFO:teuthology.orchestra.run.smithi027.stdout: "openino_backtrace_fetch": 98, 2024-05-07T02:43:32.041 INFO:teuthology.orchestra.run.smithi027.stdout: "openino_peer_discover": 0, 2024-05-07T02:43:32.041 INFO:teuthology.orchestra.run.smithi027.stdout: "root_rfiles": 67, 2024-05-07T02:43:32.041 INFO:teuthology.orchestra.run.smithi027.stdout: "root_rbytes": 21176629, 2024-05-07T02:43:32.041 INFO:teuthology.orchestra.run.smithi027.stdout: "root_rsnaps": 6, 2024-05-07T02:43:32.041 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_backtrace_fetch": 98, 2024-05-07T02:43:32.041 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_set_tag": 0, 2024-05-07T02:43:32.042 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_backtrace_repaired": 0, 2024-05-07T02:43:32.042 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_inotable_repaired": 0, 2024-05-07T02:43:32.042 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_dir_inodes": 0, 2024-05-07T02:43:32.042 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_dir_base_inodes": 0, 2024-05-07T02:43:32.042 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_dirfrag_rstats": 0, 2024-05-07T02:43:32.042 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_file_inodes": 98, 2024-05-07T02:43:32.042 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_inode_file_caps": 0, 2024-05-07T02:43:32.042 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_revoke": 343, 2024-05-07T02:43:32.042 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_grant": 8479, 2024-05-07T02:43:32.043 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_trunc": 0, 2024-05-07T02:43:32.043 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-07T02:43:32.043 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-07T02:43:32.043 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_caps": 3548, 2024-05-07T02:43:32.043 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_caps_dirty": 1855, 2024-05-07T02:43:32.043 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_cap_release": 1222, 2024-05-07T02:43:32.043 INFO:teuthology.orchestra.run.smithi027.stdout: "process_request_cap_release": 11553 2024-05-07T02:43:32.043 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.043 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_cache": { 2024-05-07T02:43:32.044 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_update": 0, 2024-05-07T02:43:32.044 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_update_receipt": 9, 2024-05-07T02:43:32.044 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_try_discover": 6, 2024-05-07T02:43:32.044 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_send_discover": 15, 2024-05-07T02:43:32.044 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_handle_discover": 0, 2024-05-07T02:43:32.044 INFO:teuthology.orchestra.run.smithi027.stdout: "num_strays": 0, 2024-05-07T02:43:32.044 INFO:teuthology.orchestra.run.smithi027.stdout: "num_strays_delayed": 0, 2024-05-07T02:43:32.044 INFO:teuthology.orchestra.run.smithi027.stdout: "num_strays_enqueuing": 0, 2024-05-07T02:43:32.044 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_created": 1994, 2024-05-07T02:43:32.045 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_enqueued": 1994, 2024-05-07T02:43:32.045 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_reintegrated": 0, 2024-05-07T02:43:32.045 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_migrated": 0, 2024-05-07T02:43:32.045 INFO:teuthology.orchestra.run.smithi027.stdout: "num_recovering_processing": 0, 2024-05-07T02:43:32.045 INFO:teuthology.orchestra.run.smithi027.stdout: "num_recovering_enqueued": 0, 2024-05-07T02:43:32.045 INFO:teuthology.orchestra.run.smithi027.stdout: "num_recovering_prioritized": 0, 2024-05-07T02:43:32.045 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_started": 0, 2024-05-07T02:43:32.045 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_completed": 0, 2024-05-07T02:43:32.045 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_quiesce_path": 0, 2024-05-07T02:43:32.046 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_quiesce_inode": 0, 2024-05-07T02:43:32.046 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_enqueue_scrub": 0, 2024-05-07T02:43:32.046 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_exportdir": 0, 2024-05-07T02:43:32.046 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_flush": 0, 2024-05-07T02:43:32.046 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_fragmentdir": 0, 2024-05-07T02:43:32.046 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_fragstats": 0, 2024-05-07T02:43:32.046 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_inodestats": 0 2024-05-07T02:43:32.047 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.047 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_log": { 2024-05-07T02:43:32.047 INFO:teuthology.orchestra.run.smithi027.stdout: "evlrg": 0, 2024-05-07T02:43:32.047 INFO:teuthology.orchestra.run.smithi027.stdout: "evadd": 13128, 2024-05-07T02:43:32.047 INFO:teuthology.orchestra.run.smithi027.stdout: "evex": 0, 2024-05-07T02:43:32.047 INFO:teuthology.orchestra.run.smithi027.stdout: "evtrm": 0, 2024-05-07T02:43:32.047 INFO:teuthology.orchestra.run.smithi027.stdout: "ev": 13128, 2024-05-07T02:43:32.047 INFO:teuthology.orchestra.run.smithi027.stdout: "evexg": 0, 2024-05-07T02:43:32.047 INFO:teuthology.orchestra.run.smithi027.stdout: "evexd": 0, 2024-05-07T02:43:32.048 INFO:teuthology.orchestra.run.smithi027.stdout: "segadd": 15, 2024-05-07T02:43:32.048 INFO:teuthology.orchestra.run.smithi027.stdout: "segex": 0, 2024-05-07T02:43:32.048 INFO:teuthology.orchestra.run.smithi027.stdout: "segtrm": 0, 2024-05-07T02:43:32.048 INFO:teuthology.orchestra.run.smithi027.stdout: "seg": 15, 2024-05-07T02:43:32.048 INFO:teuthology.orchestra.run.smithi027.stdout: "segmjr": 3, 2024-05-07T02:43:32.048 INFO:teuthology.orchestra.run.smithi027.stdout: "segexg": 0, 2024-05-07T02:43:32.048 INFO:teuthology.orchestra.run.smithi027.stdout: "segexd": 0, 2024-05-07T02:43:32.049 INFO:teuthology.orchestra.run.smithi027.stdout: "expos": 4194304, 2024-05-07T02:43:32.049 INFO:teuthology.orchestra.run.smithi027.stdout: "wrpos": 23905906, 2024-05-07T02:43:32.049 INFO:teuthology.orchestra.run.smithi027.stdout: "rdpos": 0, 2024-05-07T02:43:32.049 INFO:teuthology.orchestra.run.smithi027.stdout: "jlat": { 2024-05-07T02:43:32.049 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 3256, 2024-05-07T02:43:32.049 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 12.176623298, 2024-05-07T02:43:32.049 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.003739749 2024-05-07T02:43:32.049 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.050 INFO:teuthology.orchestra.run.smithi027.stdout: "replayed": 0 2024-05-07T02:43:32.050 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.050 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_mem": { 2024-05-07T02:43:32.050 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 104, 2024-05-07T02:43:32.050 INFO:teuthology.orchestra.run.smithi027.stdout: "ino+": 2098, 2024-05-07T02:43:32.050 INFO:teuthology.orchestra.run.smithi027.stdout: "ino-": 1994, 2024-05-07T02:43:32.050 INFO:teuthology.orchestra.run.smithi027.stdout: "dir": 28, 2024-05-07T02:43:32.051 INFO:teuthology.orchestra.run.smithi027.stdout: "dir+": 37, 2024-05-07T02:43:32.051 INFO:teuthology.orchestra.run.smithi027.stdout: "dir-": 9, 2024-05-07T02:43:32.051 INFO:teuthology.orchestra.run.smithi027.stdout: "dn": 140, 2024-05-07T02:43:32.051 INFO:teuthology.orchestra.run.smithi027.stdout: "dn+": 3169, 2024-05-07T02:43:32.051 INFO:teuthology.orchestra.run.smithi027.stdout: "dn-": 3029, 2024-05-07T02:43:32.051 INFO:teuthology.orchestra.run.smithi027.stdout: "cap": 4, 2024-05-07T02:43:32.051 INFO:teuthology.orchestra.run.smithi027.stdout: "cap+": 2015, 2024-05-07T02:43:32.051 INFO:teuthology.orchestra.run.smithi027.stdout: "cap-": 2011, 2024-05-07T02:43:32.052 INFO:teuthology.orchestra.run.smithi027.stdout: "rss": 129536, 2024-05-07T02:43:32.052 INFO:teuthology.orchestra.run.smithi027.stdout: "heap": 223632 2024-05-07T02:43:32.052 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.052 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_server": { 2024-05-07T02:43:32.052 INFO:teuthology.orchestra.run.smithi027.stdout: "dispatch_client_request": 24232, 2024-05-07T02:43:32.052 INFO:teuthology.orchestra.run.smithi027.stdout: "dispatch_server_request": 0, 2024-05-07T02:43:32.052 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_request": 12357, 2024-05-07T02:43:32.052 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_session": 266, 2024-05-07T02:43:32.052 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_peer_request": 4721, 2024-05-07T02:43:32.053 INFO:teuthology.orchestra.run.smithi027.stdout: "req_create_latency": { 2024-05-07T02:43:32.053 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 1985, 2024-05-07T02:43:32.053 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 4.498648993, 2024-05-07T02:43:32.053 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.002266321 2024-05-07T02:43:32.053 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.053 INFO:teuthology.orchestra.run.smithi027.stdout: "req_getattr_latency": { 2024-05-07T02:43:32.053 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 1, 2024-05-07T02:43:32.053 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.001971613, 2024-05-07T02:43:32.053 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.001971613 2024-05-07T02:43:32.054 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.054 INFO:teuthology.orchestra.run.smithi027.stdout: "req_getfilelock_latency": { 2024-05-07T02:43:32.054 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:32.054 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:32.054 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:32.054 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.054 INFO:teuthology.orchestra.run.smithi027.stdout: "req_link_latency": { 2024-05-07T02:43:32.054 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:32.055 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:32.055 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:32.055 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.055 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookup_latency": { 2024-05-07T02:43:32.055 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 3733, 2024-05-07T02:43:32.055 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 3.928045288, 2024-05-07T02:43:32.055 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.001052248 2024-05-07T02:43:32.056 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.056 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookuphash_latency": { 2024-05-07T02:43:32.056 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:32.056 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:32.056 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:32.056 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.056 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupino_latency": { 2024-05-07T02:43:32.056 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:32.056 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:32.057 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:32.057 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.057 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupname_latency": { 2024-05-07T02:43:32.057 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:32.057 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:32.057 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:32.057 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.057 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupparent_latency": { 2024-05-07T02:43:32.058 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:32.058 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:32.058 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:32.058 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.058 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupsnap_latency": { 2024-05-07T02:43:32.058 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:32.058 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:32.058 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:32.058 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.059 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lssnap_latency": { 2024-05-07T02:43:32.059 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:32.060 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:32.060 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:32.060 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.060 INFO:teuthology.orchestra.run.smithi027.stdout: "req_mkdir_latency": { 2024-05-07T02:43:32.060 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:32.060 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:32.060 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:32.060 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.060 INFO:teuthology.orchestra.run.smithi027.stdout: "req_mknod_latency": { 2024-05-07T02:43:32.061 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:32.061 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:32.061 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:32.061 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.061 INFO:teuthology.orchestra.run.smithi027.stdout: "req_mksnap_latency": { 2024-05-07T02:43:32.061 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:32.061 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:32.062 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:32.062 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.062 INFO:teuthology.orchestra.run.smithi027.stdout: "req_open_latency": { 2024-05-07T02:43:32.062 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:32.062 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:32.062 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:32.062 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.062 INFO:teuthology.orchestra.run.smithi027.stdout: "req_readdir_latency": { 2024-05-07T02:43:32.063 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 3365, 2024-05-07T02:43:32.063 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 1.491292132, 2024-05-07T02:43:32.063 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000443177 2024-05-07T02:43:32.063 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.063 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rename_latency": { 2024-05-07T02:43:32.063 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 738, 2024-05-07T02:43:32.063 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 1.714642389, 2024-05-07T02:43:32.063 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.002323363 2024-05-07T02:43:32.063 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.064 INFO:teuthology.orchestra.run.smithi027.stdout: "req_renamesnap_latency": { 2024-05-07T02:43:32.064 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:32.064 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:32.064 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:32.064 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.065 INFO:teuthology.orchestra.run.smithi027.stdout: "req_snapdiff_latency": { 2024-05-07T02:43:32.065 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:32.065 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:32.065 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:32.065 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.065 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rmdir_latency": { 2024-05-07T02:43:32.065 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:32.065 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:32.065 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:32.066 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.066 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rmsnap_latency": { 2024-05-07T02:43:32.066 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:32.066 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:32.066 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:32.066 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.066 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rmxattr_latency": { 2024-05-07T02:43:32.066 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:32.066 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:32.067 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:32.067 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.067 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setattr_latency": { 2024-05-07T02:43:32.067 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 2, 2024-05-07T02:43:32.067 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.012154286, 2024-05-07T02:43:32.067 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.006077143 2024-05-07T02:43:32.067 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.067 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setdirlayout_latency": { 2024-05-07T02:43:32.068 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:32.068 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:32.068 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:32.068 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.068 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setfilelock_latency": { 2024-05-07T02:43:32.068 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:32.068 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:32.068 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:32.068 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.069 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setlayout_latency": { 2024-05-07T02:43:32.069 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:32.069 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:32.069 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:32.069 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.069 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setxattr_latency": { 2024-05-07T02:43:32.069 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:32.069 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:32.069 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:32.070 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.070 INFO:teuthology.orchestra.run.smithi027.stdout: "req_symlink_latency": { 2024-05-07T02:43:32.070 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:32.070 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:32.070 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:32.070 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.070 INFO:teuthology.orchestra.run.smithi027.stdout: "req_unlink_latency": { 2024-05-07T02:43:32.070 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 1994, 2024-05-07T02:43:32.071 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 3.707631131, 2024-05-07T02:43:32.071 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.001859393 2024-05-07T02:43:32.071 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.071 INFO:teuthology.orchestra.run.smithi027.stdout: "cap_revoke_eviction": 0, 2024-05-07T02:43:32.071 INFO:teuthology.orchestra.run.smithi027.stdout: "cap_acquisition_throttle": 0, 2024-05-07T02:43:32.071 INFO:teuthology.orchestra.run.smithi027.stdout: "req_getvxattr_latency": { 2024-05-07T02:43:32.071 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:32.071 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:32.072 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:32.072 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-07T02:43:32.072 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.072 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_sessions": { 2024-05-07T02:43:32.072 INFO:teuthology.orchestra.run.smithi027.stdout: "session_count": 2, 2024-05-07T02:43:32.072 INFO:teuthology.orchestra.run.smithi027.stdout: "session_add": 3, 2024-05-07T02:43:32.072 INFO:teuthology.orchestra.run.smithi027.stdout: "session_remove": 1, 2024-05-07T02:43:32.072 INFO:teuthology.orchestra.run.smithi027.stdout: "sessions_open": 2, 2024-05-07T02:43:32.072 INFO:teuthology.orchestra.run.smithi027.stdout: "sessions_stale": 0, 2024-05-07T02:43:32.073 INFO:teuthology.orchestra.run.smithi027.stdout: "total_load": 3415, 2024-05-07T02:43:32.073 INFO:teuthology.orchestra.run.smithi027.stdout: "average_load": 1707, 2024-05-07T02:43:32.073 INFO:teuthology.orchestra.run.smithi027.stdout: "avg_session_uptime": 623, 2024-05-07T02:43:32.073 INFO:teuthology.orchestra.run.smithi027.stdout: "mdthresh_evicted": 0 2024-05-07T02:43:32.073 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.073 INFO:teuthology.orchestra.run.smithi027.stdout: "mempool": { 2024-05-07T02:43:32.073 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_bytes": 0, 2024-05-07T02:43:32.073 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_items": 0, 2024-05-07T02:43:32.073 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_bytes": 0, 2024-05-07T02:43:32.074 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_items": 0, 2024-05-07T02:43:32.074 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_bytes": 0, 2024-05-07T02:43:32.074 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_items": 0, 2024-05-07T02:43:32.074 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-07T02:43:32.074 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_items": 0, 2024-05-07T02:43:32.074 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-07T02:43:32.074 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_items": 0, 2024-05-07T02:43:32.074 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_bytes": 0, 2024-05-07T02:43:32.075 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_items": 0, 2024-05-07T02:43:32.075 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-07T02:43:32.075 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_items": 0, 2024-05-07T02:43:32.075 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_bytes": 0, 2024-05-07T02:43:32.075 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_items": 0, 2024-05-07T02:43:32.075 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_bytes": 0, 2024-05-07T02:43:32.075 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_items": 0, 2024-05-07T02:43:32.076 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-07T02:43:32.076 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_items": 0, 2024-05-07T02:43:32.076 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-07T02:43:32.076 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_items": 0, 2024-05-07T02:43:32.076 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_bytes": 0, 2024-05-07T02:43:32.076 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_items": 0, 2024-05-07T02:43:32.076 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_bytes": 0, 2024-05-07T02:43:32.076 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_items": 0, 2024-05-07T02:43:32.076 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-07T02:43:32.077 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_items": 0, 2024-05-07T02:43:32.077 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_bytes": 0, 2024-05-07T02:43:32.077 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_items": 0, 2024-05-07T02:43:32.077 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_bytes": 0, 2024-05-07T02:43:32.077 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_items": 0, 2024-05-07T02:43:32.077 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_bytes": 0, 2024-05-07T02:43:32.077 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_items": 0, 2024-05-07T02:43:32.077 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_bytes": 0, 2024-05-07T02:43:32.077 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_items": 0, 2024-05-07T02:43:32.078 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_bytes": 311975, 2024-05-07T02:43:32.078 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_items": 189, 2024-05-07T02:43:32.078 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_bytes": 0, 2024-05-07T02:43:32.078 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_items": 0, 2024-05-07T02:43:32.078 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_bytes": 0, 2024-05-07T02:43:32.078 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_items": 0, 2024-05-07T02:43:32.078 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_bytes": 0, 2024-05-07T02:43:32.078 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_items": 0, 2024-05-07T02:43:32.078 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_bytes": 0, 2024-05-07T02:43:32.079 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_items": 0, 2024-05-07T02:43:32.079 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_bytes": 9608, 2024-05-07T02:43:32.079 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_items": 184, 2024-05-07T02:43:32.079 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_bytes": 0, 2024-05-07T02:43:32.079 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_items": 0, 2024-05-07T02:43:32.079 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_bytes": 0, 2024-05-07T02:43:32.079 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_items": 0, 2024-05-07T02:43:32.079 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_bytes": 410475, 2024-05-07T02:43:32.079 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_items": 3201, 2024-05-07T02:43:32.080 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_bytes": 0, 2024-05-07T02:43:32.080 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_items": 0, 2024-05-07T02:43:32.080 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_bytes": 0, 2024-05-07T02:43:32.080 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_items": 0 2024-05-07T02:43:32.080 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.080 INFO:teuthology.orchestra.run.smithi027.stdout: "objecter": { 2024-05-07T02:43:32.080 INFO:teuthology.orchestra.run.smithi027.stdout: "op_active": 0, 2024-05-07T02:43:32.080 INFO:teuthology.orchestra.run.smithi027.stdout: "op_laggy": 0, 2024-05-07T02:43:32.080 INFO:teuthology.orchestra.run.smithi027.stdout: "op_send": 8217, 2024-05-07T02:43:32.081 INFO:teuthology.orchestra.run.smithi027.stdout: "op_send_bytes": 20187818, 2024-05-07T02:43:32.081 INFO:teuthology.orchestra.run.smithi027.stdout: "op_resend": 33, 2024-05-07T02:43:32.081 INFO:teuthology.orchestra.run.smithi027.stdout: "op_reply": 8217, 2024-05-07T02:43:32.081 INFO:teuthology.orchestra.run.smithi027.stdout: "op_latency": { 2024-05-07T02:43:32.081 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 8217, 2024-05-07T02:43:32.081 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 36.891055918, 2024-05-07T02:43:32.081 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.004489601 2024-05-07T02:43:32.081 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.081 INFO:teuthology.orchestra.run.smithi027.stdout: "op_inflight": 0, 2024-05-07T02:43:32.082 INFO:teuthology.orchestra.run.smithi027.stdout: "oplen_avg": { 2024-05-07T02:43:32.082 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 8217, 2024-05-07T02:43:32.082 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 8590 2024-05-07T02:43:32.082 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.082 INFO:teuthology.orchestra.run.smithi027.stdout: "op": 8217, 2024-05-07T02:43:32.082 INFO:teuthology.orchestra.run.smithi027.stdout: "op_r": 315, 2024-05-07T02:43:32.082 INFO:teuthology.orchestra.run.smithi027.stdout: "op_w": 7902, 2024-05-07T02:43:32.082 INFO:teuthology.orchestra.run.smithi027.stdout: "op_rmw": 0, 2024-05-07T02:43:32.083 INFO:teuthology.orchestra.run.smithi027.stdout: "op_pg": 0, 2024-05-07T02:43:32.083 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_stat": 0, 2024-05-07T02:43:32.083 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_create": 10, 2024-05-07T02:43:32.083 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_read": 217, 2024-05-07T02:43:32.083 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_write": 3473, 2024-05-07T02:43:32.083 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_writefull": 146, 2024-05-07T02:43:32.083 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_writesame": 0, 2024-05-07T02:43:32.083 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_append": 0, 2024-05-07T02:43:32.083 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_zero": 0, 2024-05-07T02:43:32.083 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_truncate": 0, 2024-05-07T02:43:32.084 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_delete": 4004, 2024-05-07T02:43:32.084 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_mapext": 0, 2024-05-07T02:43:32.084 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_sparse_read": 0, 2024-05-07T02:43:32.084 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_clonerange": 0, 2024-05-07T02:43:32.084 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_getxattr": 98, 2024-05-07T02:43:32.084 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_setxattr": 20, 2024-05-07T02:43:32.084 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_cmpxattr": 0, 2024-05-07T02:43:32.084 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_rmxattr": 0, 2024-05-07T02:43:32.084 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_resetxattrs": 0, 2024-05-07T02:43:32.085 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_call": 0, 2024-05-07T02:43:32.085 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_watch": 0, 2024-05-07T02:43:32.085 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_notify": 0, 2024-05-07T02:43:32.085 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_src_cmpxattr": 0, 2024-05-07T02:43:32.085 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_pgls": 0, 2024-05-07T02:43:32.085 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_pgls_filter": 0, 2024-05-07T02:43:32.086 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_other": 0, 2024-05-07T02:43:32.086 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_active": 0, 2024-05-07T02:43:32.086 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_send": 0, 2024-05-07T02:43:32.086 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_resend": 0, 2024-05-07T02:43:32.086 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_ping": 0, 2024-05-07T02:43:32.086 INFO:teuthology.orchestra.run.smithi027.stdout: "poolop_active": 0, 2024-05-07T02:43:32.086 INFO:teuthology.orchestra.run.smithi027.stdout: "poolop_send": 0, 2024-05-07T02:43:32.086 INFO:teuthology.orchestra.run.smithi027.stdout: "poolop_resend": 0, 2024-05-07T02:43:32.087 INFO:teuthology.orchestra.run.smithi027.stdout: "poolstat_active": 0, 2024-05-07T02:43:32.087 INFO:teuthology.orchestra.run.smithi027.stdout: "poolstat_send": 0, 2024-05-07T02:43:32.087 INFO:teuthology.orchestra.run.smithi027.stdout: "poolstat_resend": 0, 2024-05-07T02:43:32.087 INFO:teuthology.orchestra.run.smithi027.stdout: "statfs_active": 0, 2024-05-07T02:43:32.087 INFO:teuthology.orchestra.run.smithi027.stdout: "statfs_send": 0, 2024-05-07T02:43:32.087 INFO:teuthology.orchestra.run.smithi027.stdout: "statfs_resend": 0, 2024-05-07T02:43:32.087 INFO:teuthology.orchestra.run.smithi027.stdout: "command_active": 0, 2024-05-07T02:43:32.088 INFO:teuthology.orchestra.run.smithi027.stdout: "command_send": 0, 2024-05-07T02:43:32.088 INFO:teuthology.orchestra.run.smithi027.stdout: "command_resend": 0, 2024-05-07T02:43:32.088 INFO:teuthology.orchestra.run.smithi027.stdout: "map_epoch": 100, 2024-05-07T02:43:32.088 INFO:teuthology.orchestra.run.smithi027.stdout: "map_full": 0, 2024-05-07T02:43:32.088 INFO:teuthology.orchestra.run.smithi027.stdout: "map_inc": 13, 2024-05-07T02:43:32.088 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_sessions": 12, 2024-05-07T02:43:32.088 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_session_open": 12, 2024-05-07T02:43:32.088 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_session_close": 0, 2024-05-07T02:43:32.088 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_laggy": 0, 2024-05-07T02:43:32.089 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_wr": 501, 2024-05-07T02:43:32.089 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_rd": 0, 2024-05-07T02:43:32.089 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_del": 121 2024-05-07T02:43:32.089 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.089 INFO:teuthology.orchestra.run.smithi027.stdout: "oft": { 2024-05-07T02:43:32.089 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_objs": 1, 2024-05-07T02:43:32.089 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_kv_pairs": 0, 2024-05-07T02:43:32.089 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_updates": 0, 2024-05-07T02:43:32.089 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_removes": 0 2024-05-07T02:43:32.090 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.090 INFO:teuthology.orchestra.run.smithi027.stdout: "osd-slow-ops": { 2024-05-07T02:43:32.090 INFO:teuthology.orchestra.run.smithi027.stdout: "slow_ops_count": 0 2024-05-07T02:43:32.090 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.090 INFO:teuthology.orchestra.run.smithi027.stdout: "purge_queue": { 2024-05-07T02:43:32.090 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing_ops": 0, 2024-05-07T02:43:32.090 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing_ops_high_water": 10, 2024-05-07T02:43:32.090 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing": 0, 2024-05-07T02:43:32.090 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing_high_water": 5, 2024-05-07T02:43:32.091 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executed_ops": 3988, 2024-05-07T02:43:32.091 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executed": 1994, 2024-05-07T02:43:32.091 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_item_in_journal": 0 2024-05-07T02:43:32.091 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.091 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-07T02:43:32.091 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-05-07T02:43:32.091 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-05-07T02:43:32.091 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-05-07T02:43:32.092 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 60177, 2024-05-07T02:43:32.092 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 14826225, 2024-05-07T02:43:32.092 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:32.092 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 60177, 2024-05-07T02:43:32.092 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-05-07T02:43:32.092 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-05-07T02:43:32.092 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 60177, 2024-05-07T02:43:32.092 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 14826225, 2024-05-07T02:43:32.093 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-05-07T02:43:32.093 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:32.093 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:32.093 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:32.093 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-07T02:43:32.093 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.093 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-objecter_bytes": { 2024-05-07T02:43:32.093 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-05-07T02:43:32.093 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-05-07T02:43:32.093 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-05-07T02:43:32.094 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-05-07T02:43:32.094 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-05-07T02:43:32.094 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:32.094 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-05-07T02:43:32.094 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 8217, 2024-05-07T02:43:32.094 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 20405436, 2024-05-07T02:43:32.094 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 8217, 2024-05-07T02:43:32.095 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 20405436, 2024-05-07T02:43:32.095 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-05-07T02:43:32.095 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:32.095 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:32.095 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:32.095 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-07T02:43:32.095 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.095 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-objecter_ops": { 2024-05-07T02:43:32.096 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-05-07T02:43:32.096 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 1024, 2024-05-07T02:43:32.096 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-05-07T02:43:32.096 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-05-07T02:43:32.096 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-05-07T02:43:32.096 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:32.096 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-05-07T02:43:32.096 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 8217, 2024-05-07T02:43:32.096 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 8217, 2024-05-07T02:43:32.097 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 8217, 2024-05-07T02:43:32.097 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 8217, 2024-05-07T02:43:32.097 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-05-07T02:43:32.097 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:32.097 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:32.097 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:32.097 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-07T02:43:32.097 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.097 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-write_buf_throttle": { 2024-05-07T02:43:32.098 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-05-07T02:43:32.098 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 3758096384, 2024-05-07T02:43:32.098 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-05-07T02:43:32.098 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 1994, 2024-05-07T02:43:32.098 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 217618, 2024-05-07T02:43:32.098 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:32.098 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 1994, 2024-05-07T02:43:32.098 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-05-07T02:43:32.098 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-05-07T02:43:32.099 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 217, 2024-05-07T02:43:32.099 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 217618, 2024-05-07T02:43:32.099 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-05-07T02:43:32.099 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:32.099 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:32.099 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:32.099 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-07T02:43:32.099 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:32.100 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-write_buf_throttle-0x562bb7e925b0": { 2024-05-07T02:43:32.100 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-05-07T02:43:32.100 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 3758096384, 2024-05-07T02:43:32.100 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-05-07T02:43:32.100 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 13128, 2024-05-07T02:43:32.100 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 19711602, 2024-05-07T02:43:32.100 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:32.100 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 13128, 2024-05-07T02:43:32.100 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-05-07T02:43:32.101 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-05-07T02:43:32.101 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 3256, 2024-05-07T02:43:32.101 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 19711602, 2024-05-07T02:43:32.101 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-05-07T02:43:32.101 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:32.101 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:32.101 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:32.101 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-07T02:43:32.101 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-07T02:43:32.102 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-05-07T02:43:32.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:43:32 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:43:32.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:43:32 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:43:32.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:43:32 smithi027 ceph-mon[34398]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:43:32.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:43:32 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:43:32.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:43:32 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:43:32.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:43:32 smithi064 ceph-mon[34699]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:43:32.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:43:32 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-07T02:43:32.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:43:32 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-07T02:43:32.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:43:32 smithi019 ceph-mon[29781]: from='mgr.25234 ' entity='mgr.x' 2024-05-07T02:43:32.959 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-05-07T02:43:32.960 INFO:tasks.check_counter:Daemon mds.h mds.imported=4 2024-05-07T02:43:32.960 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-05-07T02:43:32.960 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-05-07T02:43:32.960 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-05-07T02:43:32.960 DEBUG:tasks.check_counter:Getting stats from k 2024-05-07T02:43:32.960 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-07T02:43:33.211 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.b/config 2024-05-07T02:43:33.453 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:43:33 smithi027 ceph-mon[34398]: pgmap v364: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:43:33.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:43:33 smithi064 ceph-mon[34699]: pgmap v364: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:43:33.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:43:33 smithi019 ceph-mon[29781]: pgmap v364: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:43:35.071 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-05-07T02:43:35.071 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-0": { 2024-05-07T02:43:35.071 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 1292, 2024-05-07T02:43:35.071 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 1260, 2024-05-07T02:43:35.071 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 2222256, 2024-05-07T02:43:35.072 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 486104, 2024-05-07T02:43:35.072 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 5, 2024-05-07T02:43:35.072 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 1, 2024-05-07T02:43:35.072 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.176309447, 2024-05-07T02:43:35.072 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.052288258, 2024-05-07T02:43:35.072 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.080602652, 2024-05-07T02:43:35.072 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.044297120, 2024-05-07T02:43:35.072 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-05-07T02:43:35.073 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 1259, 2024-05-07T02:43:35.073 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.063763739, 2024-05-07T02:43:35.073 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000050646 2024-05-07T02:43:35.073 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.073 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-05-07T02:43:35.073 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:35.073 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:35.073 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:35.073 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.074 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 1903728, 2024-05-07T02:43:35.074 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 184640 2024-05-07T02:43:35.074 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.074 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-1": { 2024-05-07T02:43:35.074 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 1933, 2024-05-07T02:43:35.074 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 2155, 2024-05-07T02:43:35.074 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 7150280, 2024-05-07T02:43:35.074 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 1211701, 2024-05-07T02:43:35.075 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 8, 2024-05-07T02:43:35.075 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 5, 2024-05-07T02:43:35.075 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.163528908, 2024-05-07T02:43:35.075 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.053420012, 2024-05-07T02:43:35.075 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.042558565, 2024-05-07T02:43:35.075 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.053839086, 2024-05-07T02:43:35.075 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-05-07T02:43:35.075 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 2151, 2024-05-07T02:43:35.075 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.096146567, 2024-05-07T02:43:35.076 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000044698 2024-05-07T02:43:35.076 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.076 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-05-07T02:43:35.076 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:35.076 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:35.076 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:35.076 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.076 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-07T02:43:35.077 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 597904 2024-05-07T02:43:35.077 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.077 INFO:teuthology.orchestra.run.smithi027.stdout: "AsyncMessenger::Worker-2": { 2024-05-07T02:43:35.077 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_messages": 8, 2024-05-07T02:43:35.077 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages": 8, 2024-05-07T02:43:35.077 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_bytes": 4573, 2024-05-07T02:43:35.078 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_bytes": 1732, 2024-05-07T02:43:35.078 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_created_connections": 7, 2024-05-07T02:43:35.078 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_active_connections": 4, 2024-05-07T02:43:35.078 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_total_time": 0.002707507, 2024-05-07T02:43:35.078 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_send_time": 0.000111243, 2024-05-07T02:43:35.078 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_recv_time": 0.002281618, 2024-05-07T02:43:35.078 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_running_fast_dispatch_time": 0.000081871, 2024-05-07T02:43:35.078 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_messages_queue_lat": { 2024-05-07T02:43:35.079 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 7, 2024-05-07T02:43:35.079 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.003461458, 2024-05-07T02:43:35.079 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000494494 2024-05-07T02:43:35.079 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.079 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_handle_ack_lat": { 2024-05-07T02:43:35.079 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:35.079 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:35.079 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:35.080 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.080 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-07T02:43:35.080 INFO:teuthology.orchestra.run.smithi027.stdout: "msgr_send_encrypted_bytes": 448 2024-05-07T02:43:35.080 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.080 INFO:teuthology.orchestra.run.smithi027.stdout: "cct": { 2024-05-07T02:43:35.080 INFO:teuthology.orchestra.run.smithi027.stdout: "total_workers": 1, 2024-05-07T02:43:35.080 INFO:teuthology.orchestra.run.smithi027.stdout: "unhealthy_workers": 0 2024-05-07T02:43:35.080 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.080 INFO:teuthology.orchestra.run.smithi027.stdout: "finisher-MDSRank": { 2024-05-07T02:43:35.081 INFO:teuthology.orchestra.run.smithi027.stdout: "queue_len": 0, 2024-05-07T02:43:35.081 INFO:teuthology.orchestra.run.smithi027.stdout: "complete_latency": { 2024-05-07T02:43:35.081 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 4852, 2024-05-07T02:43:35.081 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.145927727, 2024-05-07T02:43:35.081 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000030075 2024-05-07T02:43:35.081 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-07T02:43:35.081 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.081 INFO:teuthology.orchestra.run.smithi027.stdout: "finisher-PurgeQueue": { 2024-05-07T02:43:35.081 INFO:teuthology.orchestra.run.smithi027.stdout: "queue_len": 0, 2024-05-07T02:43:35.082 INFO:teuthology.orchestra.run.smithi027.stdout: "complete_latency": { 2024-05-07T02:43:35.082 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:35.082 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:35.082 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:35.082 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-07T02:43:35.082 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.082 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2024-05-07T02:43:35.082 INFO:teuthology.orchestra.run.smithi027.stdout: "request": 0, 2024-05-07T02:43:35.083 INFO:teuthology.orchestra.run.smithi027.stdout: "reply": 0, 2024-05-07T02:43:35.083 INFO:teuthology.orchestra.run.smithi027.stdout: "reply_latency": { 2024-05-07T02:43:35.083 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:35.083 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:35.083 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:35.083 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.083 INFO:teuthology.orchestra.run.smithi027.stdout: "slow_reply": 0, 2024-05-07T02:43:35.083 INFO:teuthology.orchestra.run.smithi027.stdout: "forward": 0, 2024-05-07T02:43:35.084 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_fetch_complete": 0, 2024-05-07T02:43:35.084 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_fetch_keys": 0, 2024-05-07T02:43:35.084 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_commit": 0, 2024-05-07T02:43:35.084 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_split": 0, 2024-05-07T02:43:35.084 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_merge": 0, 2024-05-07T02:43:35.084 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes": 848, 2024-05-07T02:43:35.084 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_top": 23, 2024-05-07T02:43:35.084 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_bottom": 825, 2024-05-07T02:43:35.084 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_pin_tail": 0, 2024-05-07T02:43:35.085 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_pinned": 816, 2024-05-07T02:43:35.085 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_expired": 0, 2024-05-07T02:43:35.085 INFO:teuthology.orchestra.run.smithi027.stdout: "inodes_with_caps": 0, 2024-05-07T02:43:35.085 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": 0, 2024-05-07T02:43:35.085 INFO:teuthology.orchestra.run.smithi027.stdout: "subtrees": 6, 2024-05-07T02:43:35.085 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse": 0, 2024-05-07T02:43:35.085 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_hit": 0, 2024-05-07T02:43:35.085 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_forward": 0, 2024-05-07T02:43:35.085 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_discover": 0, 2024-05-07T02:43:35.086 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_dir_fetch": 0, 2024-05-07T02:43:35.086 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_remote_ino": 0, 2024-05-07T02:43:35.086 INFO:teuthology.orchestra.run.smithi027.stdout: "traverse_lock": 0, 2024-05-07T02:43:35.086 INFO:teuthology.orchestra.run.smithi027.stdout: "load_cent": 0, 2024-05-07T02:43:35.086 INFO:teuthology.orchestra.run.smithi027.stdout: "q": 0, 2024-05-07T02:43:35.086 INFO:teuthology.orchestra.run.smithi027.stdout: "exported": 0, 2024-05-07T02:43:35.086 INFO:teuthology.orchestra.run.smithi027.stdout: "exported_inodes": 0, 2024-05-07T02:43:35.086 INFO:teuthology.orchestra.run.smithi027.stdout: "imported": 0, 2024-05-07T02:43:35.086 INFO:teuthology.orchestra.run.smithi027.stdout: "imported_inodes": 0, 2024-05-07T02:43:35.087 INFO:teuthology.orchestra.run.smithi027.stdout: "openino_dir_fetch": 0, 2024-05-07T02:43:35.087 INFO:teuthology.orchestra.run.smithi027.stdout: "openino_backtrace_fetch": 0, 2024-05-07T02:43:35.087 INFO:teuthology.orchestra.run.smithi027.stdout: "openino_peer_discover": 0, 2024-05-07T02:43:35.087 INFO:teuthology.orchestra.run.smithi027.stdout: "root_rfiles": 0, 2024-05-07T02:43:35.087 INFO:teuthology.orchestra.run.smithi027.stdout: "root_rbytes": 0, 2024-05-07T02:43:35.087 INFO:teuthology.orchestra.run.smithi027.stdout: "root_rsnaps": 0, 2024-05-07T02:43:35.087 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_backtrace_fetch": 0, 2024-05-07T02:43:35.087 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_set_tag": 0, 2024-05-07T02:43:35.088 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_backtrace_repaired": 0, 2024-05-07T02:43:35.088 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_inotable_repaired": 0, 2024-05-07T02:43:35.088 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_dir_inodes": 0, 2024-05-07T02:43:35.088 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_dir_base_inodes": 0, 2024-05-07T02:43:35.088 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_dirfrag_rstats": 0, 2024-05-07T02:43:35.088 INFO:teuthology.orchestra.run.smithi027.stdout: "scrub_file_inodes": 0, 2024-05-07T02:43:35.089 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_inode_file_caps": 0, 2024-05-07T02:43:35.089 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_revoke": 0, 2024-05-07T02:43:35.089 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_grant": 0, 2024-05-07T02:43:35.089 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_trunc": 0, 2024-05-07T02:43:35.089 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-07T02:43:35.089 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-07T02:43:35.089 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_caps": 0, 2024-05-07T02:43:35.089 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_caps_dirty": 0, 2024-05-07T02:43:35.089 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_cap_release": 0, 2024-05-07T02:43:35.090 INFO:teuthology.orchestra.run.smithi027.stdout: "process_request_cap_release": 0 2024-05-07T02:43:35.090 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.090 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_cache": { 2024-05-07T02:43:35.090 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_update": 0, 2024-05-07T02:43:35.090 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_update_receipt": 0, 2024-05-07T02:43:35.090 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_try_discover": 0, 2024-05-07T02:43:35.090 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_send_discover": 0, 2024-05-07T02:43:35.090 INFO:teuthology.orchestra.run.smithi027.stdout: "dir_handle_discover": 0, 2024-05-07T02:43:35.091 INFO:teuthology.orchestra.run.smithi027.stdout: "num_strays": 0, 2024-05-07T02:43:35.091 INFO:teuthology.orchestra.run.smithi027.stdout: "num_strays_delayed": 0, 2024-05-07T02:43:35.091 INFO:teuthology.orchestra.run.smithi027.stdout: "num_strays_enqueuing": 0, 2024-05-07T02:43:35.091 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_created": 0, 2024-05-07T02:43:35.091 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_enqueued": 0, 2024-05-07T02:43:35.091 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_reintegrated": 0, 2024-05-07T02:43:35.091 INFO:teuthology.orchestra.run.smithi027.stdout: "strays_migrated": 0, 2024-05-07T02:43:35.091 INFO:teuthology.orchestra.run.smithi027.stdout: "num_recovering_processing": 0, 2024-05-07T02:43:35.092 INFO:teuthology.orchestra.run.smithi027.stdout: "num_recovering_enqueued": 0, 2024-05-07T02:43:35.092 INFO:teuthology.orchestra.run.smithi027.stdout: "num_recovering_prioritized": 0, 2024-05-07T02:43:35.092 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_started": 0, 2024-05-07T02:43:35.092 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_completed": 0, 2024-05-07T02:43:35.092 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_quiesce_path": 0, 2024-05-07T02:43:35.092 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_quiesce_inode": 0, 2024-05-07T02:43:35.092 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_enqueue_scrub": 0, 2024-05-07T02:43:35.092 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_exportdir": 0, 2024-05-07T02:43:35.092 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_flush": 0, 2024-05-07T02:43:35.093 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_fragmentdir": 0, 2024-05-07T02:43:35.093 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_fragstats": 0, 2024-05-07T02:43:35.093 INFO:teuthology.orchestra.run.smithi027.stdout: "ireq_inodestats": 0 2024-05-07T02:43:35.093 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.093 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_log": { 2024-05-07T02:43:35.093 INFO:teuthology.orchestra.run.smithi027.stdout: "evlrg": 0, 2024-05-07T02:43:35.093 INFO:teuthology.orchestra.run.smithi027.stdout: "evadd": 0, 2024-05-07T02:43:35.093 INFO:teuthology.orchestra.run.smithi027.stdout: "evex": 0, 2024-05-07T02:43:35.093 INFO:teuthology.orchestra.run.smithi027.stdout: "evtrm": 0, 2024-05-07T02:43:35.094 INFO:teuthology.orchestra.run.smithi027.stdout: "ev": 4776, 2024-05-07T02:43:35.094 INFO:teuthology.orchestra.run.smithi027.stdout: "evexg": 0, 2024-05-07T02:43:35.094 INFO:teuthology.orchestra.run.smithi027.stdout: "evexd": 0, 2024-05-07T02:43:35.094 INFO:teuthology.orchestra.run.smithi027.stdout: "segadd": 0, 2024-05-07T02:43:35.094 INFO:teuthology.orchestra.run.smithi027.stdout: "segex": 0, 2024-05-07T02:43:35.094 INFO:teuthology.orchestra.run.smithi027.stdout: "segtrm": 0, 2024-05-07T02:43:35.094 INFO:teuthology.orchestra.run.smithi027.stdout: "seg": 6, 2024-05-07T02:43:35.094 INFO:teuthology.orchestra.run.smithi027.stdout: "segmjr": 2, 2024-05-07T02:43:35.095 INFO:teuthology.orchestra.run.smithi027.stdout: "segexg": 0, 2024-05-07T02:43:35.095 INFO:teuthology.orchestra.run.smithi027.stdout: "segexd": 0, 2024-05-07T02:43:35.095 INFO:teuthology.orchestra.run.smithi027.stdout: "expos": 4194304, 2024-05-07T02:43:35.095 INFO:teuthology.orchestra.run.smithi027.stdout: "wrpos": 10853360, 2024-05-07T02:43:35.095 INFO:teuthology.orchestra.run.smithi027.stdout: "rdpos": 10852786, 2024-05-07T02:43:35.095 INFO:teuthology.orchestra.run.smithi027.stdout: "jlat": { 2024-05-07T02:43:35.095 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:35.095 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:35.095 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:35.096 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.096 INFO:teuthology.orchestra.run.smithi027.stdout: "replayed": 4776 2024-05-07T02:43:35.096 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.096 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_mem": { 2024-05-07T02:43:35.096 INFO:teuthology.orchestra.run.smithi027.stdout: "ino": 124, 2024-05-07T02:43:35.096 INFO:teuthology.orchestra.run.smithi027.stdout: "ino+": 835, 2024-05-07T02:43:35.096 INFO:teuthology.orchestra.run.smithi027.stdout: "ino-": 711, 2024-05-07T02:43:35.096 INFO:teuthology.orchestra.run.smithi027.stdout: "dir": 20, 2024-05-07T02:43:35.097 INFO:teuthology.orchestra.run.smithi027.stdout: "dir+": 20, 2024-05-07T02:43:35.097 INFO:teuthology.orchestra.run.smithi027.stdout: "dir-": 0, 2024-05-07T02:43:35.097 INFO:teuthology.orchestra.run.smithi027.stdout: "dn": 848, 2024-05-07T02:43:35.097 INFO:teuthology.orchestra.run.smithi027.stdout: "dn+": 848, 2024-05-07T02:43:35.097 INFO:teuthology.orchestra.run.smithi027.stdout: "dn-": 0, 2024-05-07T02:43:35.097 INFO:teuthology.orchestra.run.smithi027.stdout: "cap": 0, 2024-05-07T02:43:35.097 INFO:teuthology.orchestra.run.smithi027.stdout: "cap+": 0, 2024-05-07T02:43:35.098 INFO:teuthology.orchestra.run.smithi027.stdout: "cap-": 0, 2024-05-07T02:43:35.098 INFO:teuthology.orchestra.run.smithi027.stdout: "rss": 120972, 2024-05-07T02:43:35.098 INFO:teuthology.orchestra.run.smithi027.stdout: "heap": 215440 2024-05-07T02:43:35.098 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.099 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_server": { 2024-05-07T02:43:35.099 INFO:teuthology.orchestra.run.smithi027.stdout: "dispatch_client_request": 0, 2024-05-07T02:43:35.099 INFO:teuthology.orchestra.run.smithi027.stdout: "dispatch_server_request": 0, 2024-05-07T02:43:35.099 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_request": 0, 2024-05-07T02:43:35.099 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_client_session": 0, 2024-05-07T02:43:35.099 INFO:teuthology.orchestra.run.smithi027.stdout: "handle_peer_request": 0, 2024-05-07T02:43:35.099 INFO:teuthology.orchestra.run.smithi027.stdout: "req_create_latency": { 2024-05-07T02:43:35.099 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:35.099 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:35.100 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:35.100 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.100 INFO:teuthology.orchestra.run.smithi027.stdout: "req_getattr_latency": { 2024-05-07T02:43:35.100 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:35.100 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:35.100 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:35.100 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.100 INFO:teuthology.orchestra.run.smithi027.stdout: "req_getfilelock_latency": { 2024-05-07T02:43:35.101 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:35.101 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:35.101 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:35.101 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.101 INFO:teuthology.orchestra.run.smithi027.stdout: "req_link_latency": { 2024-05-07T02:43:35.101 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:35.101 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:35.101 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:35.101 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.102 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookup_latency": { 2024-05-07T02:43:35.102 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:35.102 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:35.102 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:35.102 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.102 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookuphash_latency": { 2024-05-07T02:43:35.102 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:35.102 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:35.103 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:35.103 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.103 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupino_latency": { 2024-05-07T02:43:35.103 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:35.103 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:35.103 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:35.103 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.104 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupname_latency": { 2024-05-07T02:43:35.104 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:35.104 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:35.104 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:35.104 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.104 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupparent_latency": { 2024-05-07T02:43:35.104 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:35.104 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:35.104 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:35.105 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.105 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lookupsnap_latency": { 2024-05-07T02:43:35.105 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:35.105 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:35.105 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:35.105 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.105 INFO:teuthology.orchestra.run.smithi027.stdout: "req_lssnap_latency": { 2024-05-07T02:43:35.105 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:35.105 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:35.106 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:35.106 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.106 INFO:teuthology.orchestra.run.smithi027.stdout: "req_mkdir_latency": { 2024-05-07T02:43:35.107 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:35.107 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:35.107 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:35.107 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.107 INFO:teuthology.orchestra.run.smithi027.stdout: "req_mknod_latency": { 2024-05-07T02:43:35.107 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:35.107 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:35.108 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:35.108 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.108 INFO:teuthology.orchestra.run.smithi027.stdout: "req_mksnap_latency": { 2024-05-07T02:43:35.108 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:35.108 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:35.108 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:35.109 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.109 INFO:teuthology.orchestra.run.smithi027.stdout: "req_open_latency": { 2024-05-07T02:43:35.109 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:35.109 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:35.109 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:35.109 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.109 INFO:teuthology.orchestra.run.smithi027.stdout: "req_readdir_latency": { 2024-05-07T02:43:35.110 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:35.110 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:35.110 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:35.110 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.110 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rename_latency": { 2024-05-07T02:43:35.110 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:35.110 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:35.110 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:35.111 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.111 INFO:teuthology.orchestra.run.smithi027.stdout: "req_renamesnap_latency": { 2024-05-07T02:43:35.111 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:35.111 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:35.111 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:35.111 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.111 INFO:teuthology.orchestra.run.smithi027.stdout: "req_snapdiff_latency": { 2024-05-07T02:43:35.111 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:35.112 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:35.112 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:35.112 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.112 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rmdir_latency": { 2024-05-07T02:43:35.112 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:35.112 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:35.112 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:35.112 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.112 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rmsnap_latency": { 2024-05-07T02:43:35.113 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:35.113 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:35.113 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:35.113 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.113 INFO:teuthology.orchestra.run.smithi027.stdout: "req_rmxattr_latency": { 2024-05-07T02:43:35.113 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:35.113 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:35.113 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:35.113 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.114 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setattr_latency": { 2024-05-07T02:43:35.114 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:35.114 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:35.114 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:35.114 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.114 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setdirlayout_latency": { 2024-05-07T02:43:35.114 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:35.114 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:35.115 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:35.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.115 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setfilelock_latency": { 2024-05-07T02:43:35.115 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:35.115 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:35.115 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:35.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.115 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setlayout_latency": { 2024-05-07T02:43:35.116 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:35.116 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:35.116 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:35.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.116 INFO:teuthology.orchestra.run.smithi027.stdout: "req_setxattr_latency": { 2024-05-07T02:43:35.116 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:35.116 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:35.116 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:35.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.117 INFO:teuthology.orchestra.run.smithi027.stdout: "req_symlink_latency": { 2024-05-07T02:43:35.117 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:35.117 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:35.117 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:35.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.117 INFO:teuthology.orchestra.run.smithi027.stdout: "req_unlink_latency": { 2024-05-07T02:43:35.117 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:35.117 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:35.117 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:35.118 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.118 INFO:teuthology.orchestra.run.smithi027.stdout: "cap_revoke_eviction": 0, 2024-05-07T02:43:35.118 INFO:teuthology.orchestra.run.smithi027.stdout: "cap_acquisition_throttle": 0, 2024-05-07T02:43:35.118 INFO:teuthology.orchestra.run.smithi027.stdout: "req_getvxattr_latency": { 2024-05-07T02:43:35.118 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:35.118 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:35.118 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:35.118 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-07T02:43:35.119 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.119 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_sessions": { 2024-05-07T02:43:35.119 INFO:teuthology.orchestra.run.smithi027.stdout: "session_count": 1, 2024-05-07T02:43:35.119 INFO:teuthology.orchestra.run.smithi027.stdout: "session_add": 3, 2024-05-07T02:43:35.119 INFO:teuthology.orchestra.run.smithi027.stdout: "session_remove": 2, 2024-05-07T02:43:35.119 INFO:teuthology.orchestra.run.smithi027.stdout: "sessions_open": 2, 2024-05-07T02:43:35.120 INFO:teuthology.orchestra.run.smithi027.stdout: "sessions_stale": 0, 2024-05-07T02:43:35.120 INFO:teuthology.orchestra.run.smithi027.stdout: "total_load": 0, 2024-05-07T02:43:35.120 INFO:teuthology.orchestra.run.smithi027.stdout: "average_load": 0, 2024-05-07T02:43:35.120 INFO:teuthology.orchestra.run.smithi027.stdout: "avg_session_uptime": 4210, 2024-05-07T02:43:35.120 INFO:teuthology.orchestra.run.smithi027.stdout: "mdthresh_evicted": 0 2024-05-07T02:43:35.120 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.120 INFO:teuthology.orchestra.run.smithi027.stdout: "mempool": { 2024-05-07T02:43:35.120 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_bytes": 0, 2024-05-07T02:43:35.120 INFO:teuthology.orchestra.run.smithi027.stdout: "bloom_filter_items": 0, 2024-05-07T02:43:35.121 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_bytes": 0, 2024-05-07T02:43:35.121 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_alloc_items": 0, 2024-05-07T02:43:35.121 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_bytes": 0, 2024-05-07T02:43:35.121 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_data_items": 0, 2024-05-07T02:43:35.121 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-07T02:43:35.121 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_onode_items": 0, 2024-05-07T02:43:35.121 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-07T02:43:35.121 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_meta_items": 0, 2024-05-07T02:43:35.121 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_bytes": 0, 2024-05-07T02:43:35.122 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_other_items": 0, 2024-05-07T02:43:35.122 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-07T02:43:35.122 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_cache_buffer_items": 0, 2024-05-07T02:43:35.122 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_bytes": 0, 2024-05-07T02:43:35.122 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_extent_items": 0, 2024-05-07T02:43:35.122 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_bytes": 0, 2024-05-07T02:43:35.122 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_blob_items": 0, 2024-05-07T02:43:35.122 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-07T02:43:35.123 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_shared_blob_items": 0, 2024-05-07T02:43:35.123 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-07T02:43:35.123 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_inline_bl_items": 0, 2024-05-07T02:43:35.123 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_bytes": 0, 2024-05-07T02:43:35.123 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_fsck_items": 0, 2024-05-07T02:43:35.123 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_bytes": 0, 2024-05-07T02:43:35.123 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_txc_items": 0, 2024-05-07T02:43:35.123 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-07T02:43:35.123 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_deferred_items": 0, 2024-05-07T02:43:35.124 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_bytes": 0, 2024-05-07T02:43:35.124 INFO:teuthology.orchestra.run.smithi027.stdout: "bluestore_writing_items": 0, 2024-05-07T02:43:35.124 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_bytes": 0, 2024-05-07T02:43:35.124 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_items": 0, 2024-05-07T02:43:35.124 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_bytes": 0, 2024-05-07T02:43:35.124 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_reader_items": 0, 2024-05-07T02:43:35.125 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_bytes": 0, 2024-05-07T02:43:35.125 INFO:teuthology.orchestra.run.smithi027.stdout: "bluefs_file_writer_items": 0, 2024-05-07T02:43:35.125 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_bytes": 203261, 2024-05-07T02:43:35.125 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_anon_items": 173, 2024-05-07T02:43:35.125 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_bytes": 0, 2024-05-07T02:43:35.126 INFO:teuthology.orchestra.run.smithi027.stdout: "buffer_meta_items": 0, 2024-05-07T02:43:35.126 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_bytes": 0, 2024-05-07T02:43:35.126 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_items": 0, 2024-05-07T02:43:35.126 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_bytes": 0, 2024-05-07T02:43:35.126 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_mapbl_items": 0, 2024-05-07T02:43:35.126 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_bytes": 0, 2024-05-07T02:43:35.126 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_pglog_items": 0, 2024-05-07T02:43:35.126 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_bytes": 9608, 2024-05-07T02:43:35.127 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_items": 184, 2024-05-07T02:43:35.127 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_bytes": 0, 2024-05-07T02:43:35.127 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_mapping_items": 0, 2024-05-07T02:43:35.127 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_bytes": 0, 2024-05-07T02:43:35.127 INFO:teuthology.orchestra.run.smithi027.stdout: "pgmap_items": 0, 2024-05-07T02:43:35.127 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_bytes": 931383, 2024-05-07T02:43:35.127 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_co_items": 6188, 2024-05-07T02:43:35.127 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_bytes": 0, 2024-05-07T02:43:35.128 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_1_items": 0, 2024-05-07T02:43:35.128 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_bytes": 0, 2024-05-07T02:43:35.128 INFO:teuthology.orchestra.run.smithi027.stdout: "unittest_2_items": 0 2024-05-07T02:43:35.128 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.129 INFO:teuthology.orchestra.run.smithi027.stdout: "objecter": { 2024-05-07T02:43:35.129 INFO:teuthology.orchestra.run.smithi027.stdout: "op_active": 0, 2024-05-07T02:43:35.129 INFO:teuthology.orchestra.run.smithi027.stdout: "op_laggy": 0, 2024-05-07T02:43:35.129 INFO:teuthology.orchestra.run.smithi027.stdout: "op_send": 2882, 2024-05-07T02:43:35.130 INFO:teuthology.orchestra.run.smithi027.stdout: "op_send_bytes": 21, 2024-05-07T02:43:35.130 INFO:teuthology.orchestra.run.smithi027.stdout: "op_resend": 3, 2024-05-07T02:43:35.130 INFO:teuthology.orchestra.run.smithi027.stdout: "op_reply": 2882, 2024-05-07T02:43:35.130 INFO:teuthology.orchestra.run.smithi027.stdout: "op_latency": { 2024-05-07T02:43:35.131 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 2882, 2024-05-07T02:43:35.131 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 2.045947783, 2024-05-07T02:43:35.131 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000709905 2024-05-07T02:43:35.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.132 INFO:teuthology.orchestra.run.smithi027.stdout: "op_inflight": 0, 2024-05-07T02:43:35.132 INFO:teuthology.orchestra.run.smithi027.stdout: "oplen_avg": { 2024-05-07T02:43:35.132 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 2882, 2024-05-07T02:43:35.132 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 2883 2024-05-07T02:43:35.132 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.133 INFO:teuthology.orchestra.run.smithi027.stdout: "op": 2882, 2024-05-07T02:43:35.133 INFO:teuthology.orchestra.run.smithi027.stdout: "op_r": 2882, 2024-05-07T02:43:35.133 INFO:teuthology.orchestra.run.smithi027.stdout: "op_w": 0, 2024-05-07T02:43:35.134 INFO:teuthology.orchestra.run.smithi027.stdout: "op_rmw": 0, 2024-05-07T02:43:35.134 INFO:teuthology.orchestra.run.smithi027.stdout: "op_pg": 0, 2024-05-07T02:43:35.134 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_stat": 1789, 2024-05-07T02:43:35.134 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_create": 0, 2024-05-07T02:43:35.135 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_read": 1091, 2024-05-07T02:43:35.135 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_write": 0, 2024-05-07T02:43:35.135 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_writefull": 0, 2024-05-07T02:43:35.136 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_writesame": 0, 2024-05-07T02:43:35.136 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_append": 0, 2024-05-07T02:43:35.136 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_zero": 0, 2024-05-07T02:43:35.136 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_truncate": 0, 2024-05-07T02:43:35.136 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_delete": 0, 2024-05-07T02:43:35.137 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_mapext": 0, 2024-05-07T02:43:35.137 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_sparse_read": 0, 2024-05-07T02:43:35.137 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_clonerange": 0, 2024-05-07T02:43:35.137 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_getxattr": 1, 2024-05-07T02:43:35.137 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_setxattr": 0, 2024-05-07T02:43:35.137 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_cmpxattr": 0, 2024-05-07T02:43:35.137 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_rmxattr": 0, 2024-05-07T02:43:35.137 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_resetxattrs": 0, 2024-05-07T02:43:35.137 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_call": 0, 2024-05-07T02:43:35.138 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_watch": 0, 2024-05-07T02:43:35.138 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_notify": 0, 2024-05-07T02:43:35.138 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_src_cmpxattr": 0, 2024-05-07T02:43:35.138 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_pgls": 0, 2024-05-07T02:43:35.138 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_pgls_filter": 0, 2024-05-07T02:43:35.138 INFO:teuthology.orchestra.run.smithi027.stdout: "osdop_other": 0, 2024-05-07T02:43:35.138 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_active": 0, 2024-05-07T02:43:35.138 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_send": 0, 2024-05-07T02:43:35.138 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_resend": 0, 2024-05-07T02:43:35.139 INFO:teuthology.orchestra.run.smithi027.stdout: "linger_ping": 0, 2024-05-07T02:43:35.139 INFO:teuthology.orchestra.run.smithi027.stdout: "poolop_active": 0, 2024-05-07T02:43:35.139 INFO:teuthology.orchestra.run.smithi027.stdout: "poolop_send": 0, 2024-05-07T02:43:35.139 INFO:teuthology.orchestra.run.smithi027.stdout: "poolop_resend": 0, 2024-05-07T02:43:35.139 INFO:teuthology.orchestra.run.smithi027.stdout: "poolstat_active": 0, 2024-05-07T02:43:35.139 INFO:teuthology.orchestra.run.smithi027.stdout: "poolstat_send": 0, 2024-05-07T02:43:35.139 INFO:teuthology.orchestra.run.smithi027.stdout: "poolstat_resend": 0, 2024-05-07T02:43:35.139 INFO:teuthology.orchestra.run.smithi027.stdout: "statfs_active": 0, 2024-05-07T02:43:35.139 INFO:teuthology.orchestra.run.smithi027.stdout: "statfs_send": 0, 2024-05-07T02:43:35.140 INFO:teuthology.orchestra.run.smithi027.stdout: "statfs_resend": 0, 2024-05-07T02:43:35.140 INFO:teuthology.orchestra.run.smithi027.stdout: "command_active": 0, 2024-05-07T02:43:35.140 INFO:teuthology.orchestra.run.smithi027.stdout: "command_send": 0, 2024-05-07T02:43:35.140 INFO:teuthology.orchestra.run.smithi027.stdout: "command_resend": 0, 2024-05-07T02:43:35.140 INFO:teuthology.orchestra.run.smithi027.stdout: "map_epoch": 100, 2024-05-07T02:43:35.140 INFO:teuthology.orchestra.run.smithi027.stdout: "map_full": 0, 2024-05-07T02:43:35.140 INFO:teuthology.orchestra.run.smithi027.stdout: "map_inc": 13, 2024-05-07T02:43:35.140 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_sessions": 6, 2024-05-07T02:43:35.140 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_session_open": 8, 2024-05-07T02:43:35.141 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_session_close": 2, 2024-05-07T02:43:35.141 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_laggy": 0, 2024-05-07T02:43:35.141 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_wr": 0, 2024-05-07T02:43:35.141 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_rd": 2, 2024-05-07T02:43:35.141 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_del": 0 2024-05-07T02:43:35.141 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.141 INFO:teuthology.orchestra.run.smithi027.stdout: "oft": { 2024-05-07T02:43:35.141 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_objs": 0, 2024-05-07T02:43:35.141 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_kv_pairs": 0, 2024-05-07T02:43:35.142 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_updates": 0, 2024-05-07T02:43:35.142 INFO:teuthology.orchestra.run.smithi027.stdout: "omap_total_removes": 0 2024-05-07T02:43:35.142 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.142 INFO:teuthology.orchestra.run.smithi027.stdout: "osd-slow-ops": { 2024-05-07T02:43:35.142 INFO:teuthology.orchestra.run.smithi027.stdout: "slow_ops_count": 0 2024-05-07T02:43:35.142 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.142 INFO:teuthology.orchestra.run.smithi027.stdout: "purge_queue": { 2024-05-07T02:43:35.142 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing_ops": 0, 2024-05-07T02:43:35.143 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing_ops_high_water": 0, 2024-05-07T02:43:35.143 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing": 0, 2024-05-07T02:43:35.143 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executing_high_water": 0, 2024-05-07T02:43:35.143 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executed_ops": 0, 2024-05-07T02:43:35.143 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_executed": 0, 2024-05-07T02:43:35.144 INFO:teuthology.orchestra.run.smithi027.stdout: "pq_item_in_journal": 0 2024-05-07T02:43:35.144 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.144 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-07T02:43:35.144 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-05-07T02:43:35.144 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-05-07T02:43:35.144 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-05-07T02:43:35.144 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 3229, 2024-05-07T02:43:35.144 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 9065928, 2024-05-07T02:43:35.145 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:35.145 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 3229, 2024-05-07T02:43:35.145 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-05-07T02:43:35.145 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-05-07T02:43:35.145 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 3229, 2024-05-07T02:43:35.145 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 9065928, 2024-05-07T02:43:35.145 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-05-07T02:43:35.145 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:35.145 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:35.146 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:35.146 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-07T02:43:35.146 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.146 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-objecter_bytes": { 2024-05-07T02:43:35.146 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-05-07T02:43:35.146 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 104857600, 2024-05-07T02:43:35.147 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-05-07T02:43:35.147 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-05-07T02:43:35.147 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-05-07T02:43:35.147 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:35.148 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-05-07T02:43:35.148 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 2882, 2024-05-07T02:43:35.148 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 6659649, 2024-05-07T02:43:35.148 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 2882, 2024-05-07T02:43:35.148 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 6659649, 2024-05-07T02:43:35.148 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-05-07T02:43:35.148 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:35.149 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:35.149 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:35.149 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-07T02:43:35.149 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.149 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-objecter_ops": { 2024-05-07T02:43:35.149 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-05-07T02:43:35.149 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 1024, 2024-05-07T02:43:35.149 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-05-07T02:43:35.149 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-05-07T02:43:35.150 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-05-07T02:43:35.150 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:35.150 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-05-07T02:43:35.150 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 2882, 2024-05-07T02:43:35.150 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 2882, 2024-05-07T02:43:35.150 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 2882, 2024-05-07T02:43:35.150 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 2882, 2024-05-07T02:43:35.150 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-05-07T02:43:35.151 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:35.151 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:35.151 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:35.151 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-07T02:43:35.151 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.151 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-write_buf_throttle": { 2024-05-07T02:43:35.151 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-05-07T02:43:35.151 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 3758096384, 2024-05-07T02:43:35.151 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-05-07T02:43:35.152 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-05-07T02:43:35.152 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-05-07T02:43:35.152 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:35.152 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-05-07T02:43:35.152 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-05-07T02:43:35.152 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-05-07T02:43:35.152 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 0, 2024-05-07T02:43:35.152 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 0, 2024-05-07T02:43:35.152 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-05-07T02:43:35.153 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:35.153 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:35.153 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:35.153 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-07T02:43:35.153 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2024-05-07T02:43:35.153 INFO:teuthology.orchestra.run.smithi027.stdout: "throttle-write_buf_throttle-0x55c2e00408f0": { 2024-05-07T02:43:35.153 INFO:teuthology.orchestra.run.smithi027.stdout: "val": 0, 2024-05-07T02:43:35.153 INFO:teuthology.orchestra.run.smithi027.stdout: "max": 3758096384, 2024-05-07T02:43:35.153 INFO:teuthology.orchestra.run.smithi027.stdout: "get_started": 0, 2024-05-07T02:43:35.154 INFO:teuthology.orchestra.run.smithi027.stdout: "get": 0, 2024-05-07T02:43:35.154 INFO:teuthology.orchestra.run.smithi027.stdout: "get_sum": 0, 2024-05-07T02:43:35.154 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:35.154 INFO:teuthology.orchestra.run.smithi027.stdout: "get_or_fail_success": 0, 2024-05-07T02:43:35.154 INFO:teuthology.orchestra.run.smithi027.stdout: "take": 0, 2024-05-07T02:43:35.154 INFO:teuthology.orchestra.run.smithi027.stdout: "take_sum": 0, 2024-05-07T02:43:35.154 INFO:teuthology.orchestra.run.smithi027.stdout: "put": 0, 2024-05-07T02:43:35.154 INFO:teuthology.orchestra.run.smithi027.stdout: "put_sum": 0, 2024-05-07T02:43:35.155 INFO:teuthology.orchestra.run.smithi027.stdout: "wait": { 2024-05-07T02:43:35.155 INFO:teuthology.orchestra.run.smithi027.stdout: "avgcount": 0, 2024-05-07T02:43:35.155 INFO:teuthology.orchestra.run.smithi027.stdout: "sum": 0.000000000, 2024-05-07T02:43:35.155 INFO:teuthology.orchestra.run.smithi027.stdout: "avgtime": 0.000000000 2024-05-07T02:43:35.155 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-07T02:43:35.155 INFO:teuthology.orchestra.run.smithi027.stdout: } 2024-05-07T02:43:35.155 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-05-07T02:43:35.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:43:35 smithi064 ceph-mon[34699]: pgmap v365: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:43:35.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:43:35 smithi019 ceph-mon[29781]: pgmap v365: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:43:35.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:43:35 smithi027 ceph-mon[34398]: pgmap v365: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:43:35.734 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-05-07T02:43:35.734 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-05-07T02:43:35.735 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-05-07T02:43:35.735 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-05-07T02:43:35.735 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-05-07T02:43:35.735 DEBUG:tasks.check_counter:Getting stats from c 2024-05-07T02:43:35.735 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-07T02:43:35.997 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.c/config 2024-05-07T02:43:37.428 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:43:37 smithi064 ceph-mon[34699]: pgmap v366: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:43:37.647 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:43:37 smithi019 ceph-mon[29781]: pgmap v366: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:43:38.369 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:43:37 smithi027 ceph-mon[34398]: pgmap v366: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:43:38.369 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-07T02:43:38.370 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-0": { 2024-05-07T02:43:38.370 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 10051, 2024-05-07T02:43:38.370 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 14472, 2024-05-07T02:43:38.370 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 5135852, 2024-05-07T02:43:38.370 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 17734930, 2024-05-07T02:43:38.370 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 10, 2024-05-07T02:43:38.370 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 4, 2024-05-07T02:43:38.371 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.815860747, 2024-05-07T02:43:38.371 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.313068715, 2024-05-07T02:43:38.371 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.509746878, 2024-05-07T02:43:38.371 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.074292921, 2024-05-07T02:43:38.371 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-05-07T02:43:38.371 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 14471, 2024-05-07T02:43:38.371 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.304857037, 2024-05-07T02:43:38.371 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000021066 2024-05-07T02:43:38.371 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.372 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-05-07T02:43:38.372 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 16923, 2024-05-07T02:43:38.372 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.003551907, 2024-05-07T02:43:38.372 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000209 2024-05-07T02:43:38.372 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.372 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 1921504, 2024-05-07T02:43:38.372 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 195824 2024-05-07T02:43:38.372 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.373 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-1": { 2024-05-07T02:43:38.373 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 23184, 2024-05-07T02:43:38.373 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 15958, 2024-05-07T02:43:38.373 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 7104664, 2024-05-07T02:43:38.373 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 4155897, 2024-05-07T02:43:38.373 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 19, 2024-05-07T02:43:38.373 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 7, 2024-05-07T02:43:38.373 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 1.359148822, 2024-05-07T02:43:38.373 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.507320297, 2024-05-07T02:43:38.374 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 1.093876436, 2024-05-07T02:43:38.374 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.020908552, 2024-05-07T02:43:38.374 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-05-07T02:43:38.374 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 15955, 2024-05-07T02:43:38.374 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.341790657, 2024-05-07T02:43:38.374 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000021422 2024-05-07T02:43:38.374 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.374 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-05-07T02:43:38.375 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 37729, 2024-05-07T02:43:38.375 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.006846125, 2024-05-07T02:43:38.375 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000181 2024-05-07T02:43:38.375 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.375 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 4192, 2024-05-07T02:43:38.375 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 257616 2024-05-07T02:43:38.375 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.375 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-2": { 2024-05-07T02:43:38.375 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 786, 2024-05-07T02:43:38.376 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 930, 2024-05-07T02:43:38.376 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 282951, 2024-05-07T02:43:38.376 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 900195, 2024-05-07T02:43:38.376 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 29, 2024-05-07T02:43:38.376 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 6, 2024-05-07T02:43:38.376 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.073118223, 2024-05-07T02:43:38.376 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.023768753, 2024-05-07T02:43:38.376 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.026871032, 2024-05-07T02:43:38.377 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.017442627, 2024-05-07T02:43:38.377 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-05-07T02:43:38.377 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 928, 2024-05-07T02:43:38.377 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.076043245, 2024-05-07T02:43:38.377 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000081943 2024-05-07T02:43:38.377 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.377 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-05-07T02:43:38.377 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 145, 2024-05-07T02:43:38.378 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000034420, 2024-05-07T02:43:38.378 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000237 2024-05-07T02:43:38.378 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.378 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-05-07T02:43:38.378 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 344016 2024-05-07T02:43:38.378 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.378 INFO:teuthology.orchestra.run.smithi064.stdout: "cct": { 2024-05-07T02:43:38.378 INFO:teuthology.orchestra.run.smithi064.stdout: "total_workers": 1, 2024-05-07T02:43:38.379 INFO:teuthology.orchestra.run.smithi064.stdout: "unhealthy_workers": 0 2024-05-07T02:43:38.379 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.379 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-MDSRank": { 2024-05-07T02:43:38.379 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2024-05-07T02:43:38.379 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2024-05-07T02:43:38.379 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 2842, 2024-05-07T02:43:38.379 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 1.480963031, 2024-05-07T02:43:38.379 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000521098 2024-05-07T02:43:38.379 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-07T02:43:38.380 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.380 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-PurgeQueue": { 2024-05-07T02:43:38.380 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2024-05-07T02:43:38.380 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2024-05-07T02:43:38.380 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 1813, 2024-05-07T02:43:38.380 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.101374155, 2024-05-07T02:43:38.380 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000055915 2024-05-07T02:43:38.380 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-07T02:43:38.381 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.381 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": { 2024-05-07T02:43:38.381 INFO:teuthology.orchestra.run.smithi064.stdout: "request": 4839, 2024-05-07T02:43:38.381 INFO:teuthology.orchestra.run.smithi064.stdout: "reply": 4838, 2024-05-07T02:43:38.381 INFO:teuthology.orchestra.run.smithi064.stdout: "reply_latency": { 2024-05-07T02:43:38.381 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 4838, 2024-05-07T02:43:38.381 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 8.550055097, 2024-05-07T02:43:38.381 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.001767270 2024-05-07T02:43:38.381 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.382 INFO:teuthology.orchestra.run.smithi064.stdout: "slow_reply": 0, 2024-05-07T02:43:38.382 INFO:teuthology.orchestra.run.smithi064.stdout: "forward": 1, 2024-05-07T02:43:38.382 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_fetch_complete": 0, 2024-05-07T02:43:38.382 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_fetch_keys": 0, 2024-05-07T02:43:38.382 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_commit": 11, 2024-05-07T02:43:38.383 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_split": 0, 2024-05-07T02:43:38.383 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_merge": 0, 2024-05-07T02:43:38.383 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes": 144, 2024-05-07T02:43:38.383 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_top": 24, 2024-05-07T02:43:38.383 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_bottom": 120, 2024-05-07T02:43:38.383 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_pin_tail": 0, 2024-05-07T02:43:38.383 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_pinned": 110, 2024-05-07T02:43:38.383 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_expired": 0, 2024-05-07T02:43:38.384 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_with_caps": 6, 2024-05-07T02:43:38.384 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": 6, 2024-05-07T02:43:38.384 INFO:teuthology.orchestra.run.smithi064.stdout: "subtrees": 6, 2024-05-07T02:43:38.384 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse": 10969, 2024-05-07T02:43:38.384 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_hit": 5946, 2024-05-07T02:43:38.384 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_forward": 0, 2024-05-07T02:43:38.384 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_discover": 5, 2024-05-07T02:43:38.384 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_dir_fetch": 0, 2024-05-07T02:43:38.384 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_remote_ino": 0, 2024-05-07T02:43:38.385 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_lock": 225, 2024-05-07T02:43:38.385 INFO:teuthology.orchestra.run.smithi064.stdout: "load_cent": 1000, 2024-05-07T02:43:38.385 INFO:teuthology.orchestra.run.smithi064.stdout: "q": 1, 2024-05-07T02:43:38.385 INFO:teuthology.orchestra.run.smithi064.stdout: "exported": 0, 2024-05-07T02:43:38.385 INFO:teuthology.orchestra.run.smithi064.stdout: "exported_inodes": 0, 2024-05-07T02:43:38.385 INFO:teuthology.orchestra.run.smithi064.stdout: "imported": 4, 2024-05-07T02:43:38.385 INFO:teuthology.orchestra.run.smithi064.stdout: "imported_inodes": 67, 2024-05-07T02:43:38.385 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_dir_fetch": 0, 2024-05-07T02:43:38.386 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_backtrace_fetch": 88, 2024-05-07T02:43:38.386 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_peer_discover": 1, 2024-05-07T02:43:38.386 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rfiles": 67, 2024-05-07T02:43:38.386 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rbytes": 21176629, 2024-05-07T02:43:38.386 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rsnaps": 6, 2024-05-07T02:43:38.386 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_backtrace_fetch": 88, 2024-05-07T02:43:38.386 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_set_tag": 0, 2024-05-07T02:43:38.386 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_backtrace_repaired": 0, 2024-05-07T02:43:38.386 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_inotable_repaired": 0, 2024-05-07T02:43:38.387 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dir_inodes": 0, 2024-05-07T02:43:38.387 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dir_base_inodes": 0, 2024-05-07T02:43:38.387 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dirfrag_rstats": 0, 2024-05-07T02:43:38.387 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_file_inodes": 88, 2024-05-07T02:43:38.387 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_inode_file_caps": 0, 2024-05-07T02:43:38.387 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_revoke": 404, 2024-05-07T02:43:38.387 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_grant": 3178, 2024-05-07T02:43:38.387 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_trunc": 0, 2024-05-07T02:43:38.388 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-07T02:43:38.388 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-07T02:43:38.388 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_caps": 1381, 2024-05-07T02:43:38.388 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_caps_dirty": 520, 2024-05-07T02:43:38.388 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_cap_release": 249, 2024-05-07T02:43:38.388 INFO:teuthology.orchestra.run.smithi064.stdout: "process_request_cap_release": 4785 2024-05-07T02:43:38.388 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.388 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_cache": { 2024-05-07T02:43:38.388 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_update": 0, 2024-05-07T02:43:38.389 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_update_receipt": 9, 2024-05-07T02:43:38.389 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_try_discover": 6, 2024-05-07T02:43:38.389 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_send_discover": 10, 2024-05-07T02:43:38.389 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_handle_discover": 8, 2024-05-07T02:43:38.389 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays": 0, 2024-05-07T02:43:38.389 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays_delayed": 0, 2024-05-07T02:43:38.389 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays_enqueuing": 0, 2024-05-07T02:43:38.389 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_created": 711, 2024-05-07T02:43:38.389 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_enqueued": 711, 2024-05-07T02:43:38.390 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_reintegrated": 0, 2024-05-07T02:43:38.390 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_migrated": 0, 2024-05-07T02:43:38.390 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_processing": 0, 2024-05-07T02:43:38.390 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_enqueued": 0, 2024-05-07T02:43:38.390 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_prioritized": 0, 2024-05-07T02:43:38.390 INFO:teuthology.orchestra.run.smithi064.stdout: "recovery_started": 0, 2024-05-07T02:43:38.390 INFO:teuthology.orchestra.run.smithi064.stdout: "recovery_completed": 0, 2024-05-07T02:43:38.390 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_quiesce_path": 0, 2024-05-07T02:43:38.391 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_quiesce_inode": 0, 2024-05-07T02:43:38.391 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_enqueue_scrub": 0, 2024-05-07T02:43:38.391 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_exportdir": 0, 2024-05-07T02:43:38.391 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_flush": 0, 2024-05-07T02:43:38.391 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_fragmentdir": 0, 2024-05-07T02:43:38.391 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_fragstats": 0, 2024-05-07T02:43:38.391 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_inodestats": 0 2024-05-07T02:43:38.391 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.391 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_log": { 2024-05-07T02:43:38.392 INFO:teuthology.orchestra.run.smithi064.stdout: "evlrg": 0, 2024-05-07T02:43:38.392 INFO:teuthology.orchestra.run.smithi064.stdout: "evadd": 4776, 2024-05-07T02:43:38.392 INFO:teuthology.orchestra.run.smithi064.stdout: "evex": 0, 2024-05-07T02:43:38.392 INFO:teuthology.orchestra.run.smithi064.stdout: "evtrm": 0, 2024-05-07T02:43:38.392 INFO:teuthology.orchestra.run.smithi064.stdout: "ev": 4776, 2024-05-07T02:43:38.392 INFO:teuthology.orchestra.run.smithi064.stdout: "evexg": 0, 2024-05-07T02:43:38.392 INFO:teuthology.orchestra.run.smithi064.stdout: "evexd": 0, 2024-05-07T02:43:38.392 INFO:teuthology.orchestra.run.smithi064.stdout: "segadd": 6, 2024-05-07T02:43:38.392 INFO:teuthology.orchestra.run.smithi064.stdout: "segex": 0, 2024-05-07T02:43:38.393 INFO:teuthology.orchestra.run.smithi064.stdout: "segtrm": 0, 2024-05-07T02:43:38.393 INFO:teuthology.orchestra.run.smithi064.stdout: "seg": 6, 2024-05-07T02:43:38.393 INFO:teuthology.orchestra.run.smithi064.stdout: "segmjr": 2, 2024-05-07T02:43:38.393 INFO:teuthology.orchestra.run.smithi064.stdout: "segexg": 0, 2024-05-07T02:43:38.393 INFO:teuthology.orchestra.run.smithi064.stdout: "segexd": 0, 2024-05-07T02:43:38.393 INFO:teuthology.orchestra.run.smithi064.stdout: "expos": 4194304, 2024-05-07T02:43:38.393 INFO:teuthology.orchestra.run.smithi064.stdout: "wrpos": 10853360, 2024-05-07T02:43:38.393 INFO:teuthology.orchestra.run.smithi064.stdout: "rdpos": 0, 2024-05-07T02:43:38.393 INFO:teuthology.orchestra.run.smithi064.stdout: "jlat": { 2024-05-07T02:43:38.394 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 1302, 2024-05-07T02:43:38.394 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 4.865029262, 2024-05-07T02:43:38.394 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.003736581 2024-05-07T02:43:38.394 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.394 INFO:teuthology.orchestra.run.smithi064.stdout: "replayed": 0 2024-05-07T02:43:38.394 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.394 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_mem": { 2024-05-07T02:43:38.394 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 131, 2024-05-07T02:43:38.395 INFO:teuthology.orchestra.run.smithi064.stdout: "ino+": 842, 2024-05-07T02:43:38.395 INFO:teuthology.orchestra.run.smithi064.stdout: "ino-": 711, 2024-05-07T02:43:38.395 INFO:teuthology.orchestra.run.smithi064.stdout: "dir": 28, 2024-05-07T02:43:38.395 INFO:teuthology.orchestra.run.smithi064.stdout: "dir+": 35, 2024-05-07T02:43:38.395 INFO:teuthology.orchestra.run.smithi064.stdout: "dir-": 7, 2024-05-07T02:43:38.395 INFO:teuthology.orchestra.run.smithi064.stdout: "dn": 144, 2024-05-07T02:43:38.395 INFO:teuthology.orchestra.run.smithi064.stdout: "dn+": 1036, 2024-05-07T02:43:38.395 INFO:teuthology.orchestra.run.smithi064.stdout: "dn-": 892, 2024-05-07T02:43:38.395 INFO:teuthology.orchestra.run.smithi064.stdout: "cap": 6, 2024-05-07T02:43:38.396 INFO:teuthology.orchestra.run.smithi064.stdout: "cap+": 807, 2024-05-07T02:43:38.396 INFO:teuthology.orchestra.run.smithi064.stdout: "cap-": 801, 2024-05-07T02:43:38.396 INFO:teuthology.orchestra.run.smithi064.stdout: "rss": 129752, 2024-05-07T02:43:38.396 INFO:teuthology.orchestra.run.smithi064.stdout: "heap": 223632 2024-05-07T02:43:38.396 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.396 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_server": { 2024-05-07T02:43:38.396 INFO:teuthology.orchestra.run.smithi064.stdout: "dispatch_client_request": 9298, 2024-05-07T02:43:38.396 INFO:teuthology.orchestra.run.smithi064.stdout: "dispatch_server_request": 0, 2024-05-07T02:43:38.396 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_request": 4839, 2024-05-07T02:43:38.397 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_session": 243, 2024-05-07T02:43:38.397 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_peer_request": 1687, 2024-05-07T02:43:38.397 INFO:teuthology.orchestra.run.smithi064.stdout: "req_create_latency": { 2024-05-07T02:43:38.397 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 711, 2024-05-07T02:43:38.397 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 2.105250538, 2024-05-07T02:43:38.397 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.002960971 2024-05-07T02:43:38.397 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.397 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getattr_latency": { 2024-05-07T02:43:38.397 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 7, 2024-05-07T02:43:38.398 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.021163238, 2024-05-07T02:43:38.398 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.003023319 2024-05-07T02:43:38.398 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.398 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getfilelock_latency": { 2024-05-07T02:43:38.398 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:38.398 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:38.398 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:38.398 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.399 INFO:teuthology.orchestra.run.smithi064.stdout: "req_link_latency": { 2024-05-07T02:43:38.399 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:38.399 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:38.399 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:38.399 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.399 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookup_latency": { 2024-05-07T02:43:38.399 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 1025, 2024-05-07T02:43:38.399 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 1.187278070, 2024-05-07T02:43:38.400 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.001158320 2024-05-07T02:43:38.400 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.400 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookuphash_latency": { 2024-05-07T02:43:38.400 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:38.400 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:38.400 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:38.400 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.400 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupino_latency": { 2024-05-07T02:43:38.400 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:38.401 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:38.401 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:38.401 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.401 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupname_latency": { 2024-05-07T02:43:38.401 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:38.401 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:38.401 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:38.401 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.402 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupparent_latency": { 2024-05-07T02:43:38.402 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:38.402 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:38.402 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:38.402 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.402 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupsnap_latency": { 2024-05-07T02:43:38.402 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:38.402 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:38.403 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:38.403 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.403 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lssnap_latency": { 2024-05-07T02:43:38.403 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:38.403 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:38.403 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:38.403 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.404 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mkdir_latency": { 2024-05-07T02:43:38.404 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:38.404 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:38.404 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:38.404 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.404 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mknod_latency": { 2024-05-07T02:43:38.404 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:38.404 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:38.404 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:38.405 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.405 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mksnap_latency": { 2024-05-07T02:43:38.405 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:38.405 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:38.405 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:38.405 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.405 INFO:teuthology.orchestra.run.smithi064.stdout: "req_open_latency": { 2024-05-07T02:43:38.405 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:38.405 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:38.406 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:38.406 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.406 INFO:teuthology.orchestra.run.smithi064.stdout: "req_readdir_latency": { 2024-05-07T02:43:38.406 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 1979, 2024-05-07T02:43:38.406 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 1.039000591, 2024-05-07T02:43:38.406 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000525012 2024-05-07T02:43:38.406 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.406 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rename_latency": { 2024-05-07T02:43:38.407 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 225, 2024-05-07T02:43:38.407 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.689778298, 2024-05-07T02:43:38.407 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.003065681 2024-05-07T02:43:38.407 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.407 INFO:teuthology.orchestra.run.smithi064.stdout: "req_renamesnap_latency": { 2024-05-07T02:43:38.407 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:38.407 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:38.407 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:38.407 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.408 INFO:teuthology.orchestra.run.smithi064.stdout: "req_snapdiff_latency": { 2024-05-07T02:43:38.408 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:38.408 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:38.408 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:38.408 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.408 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmdir_latency": { 2024-05-07T02:43:38.408 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:38.408 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:38.408 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:38.409 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.409 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmsnap_latency": { 2024-05-07T02:43:38.409 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:38.409 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:38.409 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:38.409 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.409 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmxattr_latency": { 2024-05-07T02:43:38.409 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:38.409 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:38.410 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:38.410 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.410 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setattr_latency": { 2024-05-07T02:43:38.410 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:38.410 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:38.410 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:38.410 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.410 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setdirlayout_latency": { 2024-05-07T02:43:38.410 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:38.411 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:38.411 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:38.411 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.411 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setfilelock_latency": { 2024-05-07T02:43:38.411 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 180, 2024-05-07T02:43:38.411 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.080537621, 2024-05-07T02:43:38.411 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000447431 2024-05-07T02:43:38.411 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.411 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setlayout_latency": { 2024-05-07T02:43:38.412 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:38.412 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:38.412 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:38.412 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.412 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setxattr_latency": { 2024-05-07T02:43:38.412 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:38.412 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:38.412 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:38.412 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.413 INFO:teuthology.orchestra.run.smithi064.stdout: "req_symlink_latency": { 2024-05-07T02:43:38.413 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:38.413 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:38.413 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:38.413 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.413 INFO:teuthology.orchestra.run.smithi064.stdout: "req_unlink_latency": { 2024-05-07T02:43:38.413 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 711, 2024-05-07T02:43:38.413 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 3.427046741, 2024-05-07T02:43:38.413 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.004820037 2024-05-07T02:43:38.414 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.414 INFO:teuthology.orchestra.run.smithi064.stdout: "cap_revoke_eviction": 0, 2024-05-07T02:43:38.414 INFO:teuthology.orchestra.run.smithi064.stdout: "cap_acquisition_throttle": 0, 2024-05-07T02:43:38.414 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getvxattr_latency": { 2024-05-07T02:43:38.414 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:38.414 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:38.414 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:38.414 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-07T02:43:38.415 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.415 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_sessions": { 2024-05-07T02:43:38.415 INFO:teuthology.orchestra.run.smithi064.stdout: "session_count": 1, 2024-05-07T02:43:38.415 INFO:teuthology.orchestra.run.smithi064.stdout: "session_add": 3, 2024-05-07T02:43:38.415 INFO:teuthology.orchestra.run.smithi064.stdout: "session_remove": 2, 2024-05-07T02:43:38.415 INFO:teuthology.orchestra.run.smithi064.stdout: "sessions_open": 1, 2024-05-07T02:43:38.415 INFO:teuthology.orchestra.run.smithi064.stdout: "sessions_stale": 0, 2024-05-07T02:43:38.415 INFO:teuthology.orchestra.run.smithi064.stdout: "total_load": 2028, 2024-05-07T02:43:38.415 INFO:teuthology.orchestra.run.smithi064.stdout: "average_load": 2028, 2024-05-07T02:43:38.416 INFO:teuthology.orchestra.run.smithi064.stdout: "avg_session_uptime": 683, 2024-05-07T02:43:38.416 INFO:teuthology.orchestra.run.smithi064.stdout: "mdthresh_evicted": 0 2024-05-07T02:43:38.416 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.416 INFO:teuthology.orchestra.run.smithi064.stdout: "mempool": { 2024-05-07T02:43:38.416 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_bytes": 0, 2024-05-07T02:43:38.416 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_items": 0, 2024-05-07T02:43:38.416 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_bytes": 0, 2024-05-07T02:43:38.416 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_items": 0, 2024-05-07T02:43:38.416 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_bytes": 0, 2024-05-07T02:43:38.417 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_items": 0, 2024-05-07T02:43:38.417 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-07T02:43:38.417 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_items": 0, 2024-05-07T02:43:38.417 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-07T02:43:38.417 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_items": 0, 2024-05-07T02:43:38.417 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_bytes": 0, 2024-05-07T02:43:38.417 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_items": 0, 2024-05-07T02:43:38.417 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-07T02:43:38.417 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_items": 0, 2024-05-07T02:43:38.418 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_bytes": 0, 2024-05-07T02:43:38.418 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_items": 0, 2024-05-07T02:43:38.418 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_bytes": 0, 2024-05-07T02:43:38.418 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_items": 0, 2024-05-07T02:43:38.418 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-07T02:43:38.418 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_items": 0, 2024-05-07T02:43:38.418 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-07T02:43:38.418 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_items": 0, 2024-05-07T02:43:38.418 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_bytes": 0, 2024-05-07T02:43:38.419 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_items": 0, 2024-05-07T02:43:38.419 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_bytes": 0, 2024-05-07T02:43:38.419 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_items": 0, 2024-05-07T02:43:38.419 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-07T02:43:38.419 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_items": 0, 2024-05-07T02:43:38.419 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_bytes": 0, 2024-05-07T02:43:38.419 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_items": 0, 2024-05-07T02:43:38.419 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_bytes": 0, 2024-05-07T02:43:38.419 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_items": 0, 2024-05-07T02:43:38.420 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_bytes": 0, 2024-05-07T02:43:38.420 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_items": 0, 2024-05-07T02:43:38.420 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_bytes": 0, 2024-05-07T02:43:38.420 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_items": 0, 2024-05-07T02:43:38.420 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_bytes": 321348, 2024-05-07T02:43:38.420 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_items": 207, 2024-05-07T02:43:38.420 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_bytes": 88, 2024-05-07T02:43:38.420 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_items": 1, 2024-05-07T02:43:38.420 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_bytes": 0, 2024-05-07T02:43:38.421 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_items": 0, 2024-05-07T02:43:38.421 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_bytes": 0, 2024-05-07T02:43:38.421 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_items": 0, 2024-05-07T02:43:38.421 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_bytes": 0, 2024-05-07T02:43:38.421 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_items": 0, 2024-05-07T02:43:38.421 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_bytes": 9608, 2024-05-07T02:43:38.421 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_items": 184, 2024-05-07T02:43:38.421 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_bytes": 0, 2024-05-07T02:43:38.421 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_items": 0, 2024-05-07T02:43:38.422 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_bytes": 0, 2024-05-07T02:43:38.422 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_items": 0, 2024-05-07T02:43:38.422 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_bytes": 464775, 2024-05-07T02:43:38.422 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_items": 3480, 2024-05-07T02:43:38.422 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_bytes": 0, 2024-05-07T02:43:38.422 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_items": 0, 2024-05-07T02:43:38.422 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_bytes": 0, 2024-05-07T02:43:38.422 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_items": 0 2024-05-07T02:43:38.422 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.423 INFO:teuthology.orchestra.run.smithi064.stdout: "objecter": { 2024-05-07T02:43:38.423 INFO:teuthology.orchestra.run.smithi064.stdout: "op_active": 0, 2024-05-07T02:43:38.423 INFO:teuthology.orchestra.run.smithi064.stdout: "op_laggy": 0, 2024-05-07T02:43:38.423 INFO:teuthology.orchestra.run.smithi064.stdout: "op_send": 3433, 2024-05-07T02:43:38.423 INFO:teuthology.orchestra.run.smithi064.stdout: "op_send_bytes": 6850430, 2024-05-07T02:43:38.423 INFO:teuthology.orchestra.run.smithi064.stdout: "op_resend": 33, 2024-05-07T02:43:38.423 INFO:teuthology.orchestra.run.smithi064.stdout: "op_reply": 3433, 2024-05-07T02:43:38.423 INFO:teuthology.orchestra.run.smithi064.stdout: "op_latency": { 2024-05-07T02:43:38.424 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 3433, 2024-05-07T02:43:38.424 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 14.885711927, 2024-05-07T02:43:38.424 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.004336065 2024-05-07T02:43:38.424 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.424 INFO:teuthology.orchestra.run.smithi064.stdout: "op_inflight": 0, 2024-05-07T02:43:38.424 INFO:teuthology.orchestra.run.smithi064.stdout: "oplen_avg": { 2024-05-07T02:43:38.424 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 3433, 2024-05-07T02:43:38.424 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 3663 2024-05-07T02:43:38.424 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.425 INFO:teuthology.orchestra.run.smithi064.stdout: "op": 3433, 2024-05-07T02:43:38.425 INFO:teuthology.orchestra.run.smithi064.stdout: "op_r": 248, 2024-05-07T02:43:38.425 INFO:teuthology.orchestra.run.smithi064.stdout: "op_w": 3185, 2024-05-07T02:43:38.425 INFO:teuthology.orchestra.run.smithi064.stdout: "op_rmw": 0, 2024-05-07T02:43:38.425 INFO:teuthology.orchestra.run.smithi064.stdout: "op_pg": 0, 2024-05-07T02:43:38.425 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_stat": 0, 2024-05-07T02:43:38.425 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_create": 10, 2024-05-07T02:43:38.425 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_read": 160, 2024-05-07T02:43:38.425 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_write": 1462, 2024-05-07T02:43:38.426 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_writefull": 116, 2024-05-07T02:43:38.426 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_writesame": 0, 2024-05-07T02:43:38.426 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_append": 0, 2024-05-07T02:43:38.426 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_zero": 0, 2024-05-07T02:43:38.426 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_truncate": 0, 2024-05-07T02:43:38.426 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_delete": 1435, 2024-05-07T02:43:38.426 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_mapext": 0, 2024-05-07T02:43:38.426 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_sparse_read": 0, 2024-05-07T02:43:38.426 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_clonerange": 0, 2024-05-07T02:43:38.427 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_getxattr": 88, 2024-05-07T02:43:38.427 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_setxattr": 20, 2024-05-07T02:43:38.427 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_cmpxattr": 0, 2024-05-07T02:43:38.427 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_rmxattr": 0, 2024-05-07T02:43:38.427 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_resetxattrs": 0, 2024-05-07T02:43:38.427 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_call": 0, 2024-05-07T02:43:38.427 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_watch": 0, 2024-05-07T02:43:38.427 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_notify": 0, 2024-05-07T02:43:38.427 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_src_cmpxattr": 0, 2024-05-07T02:43:38.428 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_pgls": 0, 2024-05-07T02:43:38.428 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_pgls_filter": 0, 2024-05-07T02:43:38.428 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_other": 0, 2024-05-07T02:43:38.428 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_active": 0, 2024-05-07T02:43:38.428 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_send": 0, 2024-05-07T02:43:38.428 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_resend": 0, 2024-05-07T02:43:38.428 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_ping": 0, 2024-05-07T02:43:38.428 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_active": 0, 2024-05-07T02:43:38.429 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_send": 0, 2024-05-07T02:43:38.429 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_resend": 0, 2024-05-07T02:43:38.429 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_active": 0, 2024-05-07T02:43:38.429 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_send": 0, 2024-05-07T02:43:38.429 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_resend": 0, 2024-05-07T02:43:38.429 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_active": 0, 2024-05-07T02:43:38.429 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_send": 0, 2024-05-07T02:43:38.429 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_resend": 0, 2024-05-07T02:43:38.429 INFO:teuthology.orchestra.run.smithi064.stdout: "command_active": 0, 2024-05-07T02:43:38.430 INFO:teuthology.orchestra.run.smithi064.stdout: "command_send": 0, 2024-05-07T02:43:38.430 INFO:teuthology.orchestra.run.smithi064.stdout: "command_resend": 0, 2024-05-07T02:43:38.430 INFO:teuthology.orchestra.run.smithi064.stdout: "map_epoch": 100, 2024-05-07T02:43:38.430 INFO:teuthology.orchestra.run.smithi064.stdout: "map_full": 0, 2024-05-07T02:43:38.430 INFO:teuthology.orchestra.run.smithi064.stdout: "map_inc": 13, 2024-05-07T02:43:38.430 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_sessions": 12, 2024-05-07T02:43:38.430 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_session_open": 12, 2024-05-07T02:43:38.430 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_session_close": 0, 2024-05-07T02:43:38.430 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_laggy": 0, 2024-05-07T02:43:38.431 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_wr": 276, 2024-05-07T02:43:38.431 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_rd": 0, 2024-05-07T02:43:38.431 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_del": 96 2024-05-07T02:43:38.431 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.431 INFO:teuthology.orchestra.run.smithi064.stdout: "oft": { 2024-05-07T02:43:38.431 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_objs": 1, 2024-05-07T02:43:38.431 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_kv_pairs": 0, 2024-05-07T02:43:38.431 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_updates": 0, 2024-05-07T02:43:38.431 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_removes": 0 2024-05-07T02:43:38.432 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.432 INFO:teuthology.orchestra.run.smithi064.stdout: "osd-slow-ops": { 2024-05-07T02:43:38.432 INFO:teuthology.orchestra.run.smithi064.stdout: "slow_ops_count": 0 2024-05-07T02:43:38.432 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.432 INFO:teuthology.orchestra.run.smithi064.stdout: "purge_queue": { 2024-05-07T02:43:38.432 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_ops": 0, 2024-05-07T02:43:38.432 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_ops_high_water": 10, 2024-05-07T02:43:38.432 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing": 0, 2024-05-07T02:43:38.432 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_high_water": 5, 2024-05-07T02:43:38.433 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executed_ops": 1422, 2024-05-07T02:43:38.433 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executed": 711, 2024-05-07T02:43:38.433 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_item_in_journal": 0 2024-05-07T02:43:38.433 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.433 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-07T02:43:38.433 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-05-07T02:43:38.433 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2024-05-07T02:43:38.433 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-05-07T02:43:38.433 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 34017, 2024-05-07T02:43:38.434 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 9439036, 2024-05-07T02:43:38.434 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:38.434 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 34017, 2024-05-07T02:43:38.434 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-05-07T02:43:38.434 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-05-07T02:43:38.434 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 34017, 2024-05-07T02:43:38.434 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 9439036, 2024-05-07T02:43:38.434 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-05-07T02:43:38.434 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:38.435 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:38.435 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:38.435 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-07T02:43:38.435 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.435 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-objecter_bytes": { 2024-05-07T02:43:38.435 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-05-07T02:43:38.435 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2024-05-07T02:43:38.435 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-05-07T02:43:38.435 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2024-05-07T02:43:38.436 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2024-05-07T02:43:38.436 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:38.436 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2024-05-07T02:43:38.436 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 3433, 2024-05-07T02:43:38.436 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 6927993, 2024-05-07T02:43:38.436 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 3433, 2024-05-07T02:43:38.436 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 6927993, 2024-05-07T02:43:38.436 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-05-07T02:43:38.436 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:38.437 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:38.437 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:38.437 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-07T02:43:38.437 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.437 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-objecter_ops": { 2024-05-07T02:43:38.437 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-05-07T02:43:38.437 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 1024, 2024-05-07T02:43:38.437 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-05-07T02:43:38.437 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2024-05-07T02:43:38.438 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2024-05-07T02:43:38.438 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:38.438 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2024-05-07T02:43:38.438 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 3433, 2024-05-07T02:43:38.438 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 3433, 2024-05-07T02:43:38.438 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 3433, 2024-05-07T02:43:38.438 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 3433, 2024-05-07T02:43:38.438 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-05-07T02:43:38.438 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:38.439 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:38.439 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:38.439 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-07T02:43:38.439 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.439 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-write_buf_throttle": { 2024-05-07T02:43:38.439 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-05-07T02:43:38.439 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 3758096384, 2024-05-07T02:43:38.439 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-05-07T02:43:38.439 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 711, 2024-05-07T02:43:38.440 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 77563, 2024-05-07T02:43:38.440 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:38.440 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 711, 2024-05-07T02:43:38.440 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-05-07T02:43:38.440 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-05-07T02:43:38.440 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 160, 2024-05-07T02:43:38.440 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 77563, 2024-05-07T02:43:38.440 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-05-07T02:43:38.440 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:38.441 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:38.441 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:38.441 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-07T02:43:38.441 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:38.441 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-write_buf_throttle-0x55e3e33f4410": { 2024-05-07T02:43:38.441 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-05-07T02:43:38.441 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 3758096384, 2024-05-07T02:43:38.441 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-05-07T02:43:38.441 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 4776, 2024-05-07T02:43:38.442 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 6659056, 2024-05-07T02:43:38.442 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:38.442 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 4776, 2024-05-07T02:43:38.442 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-05-07T02:43:38.442 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-05-07T02:43:38.442 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 1302, 2024-05-07T02:43:38.442 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 6659056, 2024-05-07T02:43:38.442 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-05-07T02:43:38.443 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:38.443 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:38.443 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:38.443 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-07T02:43:38.443 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-07T02:43:38.443 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-07T02:43:38.984 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-05-07T02:43:38.984 INFO:tasks.check_counter:Daemon mds.c mds.imported=4 2024-05-07T02:43:38.985 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-05-07T02:43:38.985 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-05-07T02:43:38.985 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-05-07T02:43:38.985 DEBUG:tasks.check_counter:Getting stats from f 2024-05-07T02:43:38.985 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-07T02:43:39.241 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.c/config 2024-05-07T02:43:39.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:43:39 smithi064 ceph-mon[34699]: pgmap v367: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:43:39.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:43:39 smithi019 ceph-mon[29781]: pgmap v367: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:43:39.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:43:39 smithi027 ceph-mon[34398]: pgmap v367: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:43:41.109 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-07T02:43:41.109 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-0": { 2024-05-07T02:43:41.109 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 1960, 2024-05-07T02:43:41.109 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 1962, 2024-05-07T02:43:41.109 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 34131607, 2024-05-07T02:43:41.109 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 623290, 2024-05-07T02:43:41.110 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 8, 2024-05-07T02:43:41.110 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 4, 2024-05-07T02:43:41.110 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.219140511, 2024-05-07T02:43:41.110 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.062550200, 2024-05-07T02:43:41.110 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.069768144, 2024-05-07T02:43:41.110 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.067004835, 2024-05-07T02:43:41.110 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-05-07T02:43:41.110 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 1960, 2024-05-07T02:43:41.110 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.090667358, 2024-05-07T02:43:41.111 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000046258 2024-05-07T02:43:41.111 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.111 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-05-07T02:43:41.111 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:41.111 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:41.111 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:41.111 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.111 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-07T02:43:41.111 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 0 2024-05-07T02:43:41.112 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.112 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-1": { 2024-05-07T02:43:41.112 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 737, 2024-05-07T02:43:41.112 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 958, 2024-05-07T02:43:41.112 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 31715165, 2024-05-07T02:43:41.112 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 834614, 2024-05-07T02:43:41.112 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 9, 2024-05-07T02:43:41.112 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 6, 2024-05-07T02:43:41.112 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.093903152, 2024-05-07T02:43:41.112 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.030322317, 2024-05-07T02:43:41.113 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.036839449, 2024-05-07T02:43:41.113 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.018362814, 2024-05-07T02:43:41.113 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-05-07T02:43:41.113 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 955, 2024-05-07T02:43:41.113 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.055158477, 2024-05-07T02:43:41.113 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000057757 2024-05-07T02:43:41.113 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.113 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-05-07T02:43:41.113 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:41.114 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:41.114 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:41.114 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.114 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-07T02:43:41.114 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 601136 2024-05-07T02:43:41.114 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.114 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-2": { 2024-05-07T02:43:41.114 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 909, 2024-05-07T02:43:41.114 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 875, 2024-05-07T02:43:41.115 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 27212505, 2024-05-07T02:43:41.115 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 362363, 2024-05-07T02:43:41.115 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 5, 2024-05-07T02:43:41.115 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 4, 2024-05-07T02:43:41.115 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.127252192, 2024-05-07T02:43:41.115 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.037032794, 2024-05-07T02:43:41.115 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.074125126, 2024-05-07T02:43:41.115 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.023834876, 2024-05-07T02:43:41.115 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-05-07T02:43:41.116 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 875, 2024-05-07T02:43:41.116 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.037750264, 2024-05-07T02:43:41.116 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000043143 2024-05-07T02:43:41.116 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.116 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-05-07T02:43:41.116 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:41.116 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:41.116 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:41.116 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.117 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 1906016, 2024-05-07T02:43:41.117 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 183328 2024-05-07T02:43:41.117 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.117 INFO:teuthology.orchestra.run.smithi064.stdout: "cct": { 2024-05-07T02:43:41.117 INFO:teuthology.orchestra.run.smithi064.stdout: "total_workers": 1, 2024-05-07T02:43:41.117 INFO:teuthology.orchestra.run.smithi064.stdout: "unhealthy_workers": 0 2024-05-07T02:43:41.117 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.117 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-MDSRank": { 2024-05-07T02:43:41.118 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2024-05-07T02:43:41.118 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2024-05-07T02:43:41.118 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 5458, 2024-05-07T02:43:41.118 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.161040083, 2024-05-07T02:43:41.118 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000029505 2024-05-07T02:43:41.118 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-07T02:43:41.118 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.118 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-PurgeQueue": { 2024-05-07T02:43:41.118 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2024-05-07T02:43:41.119 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2024-05-07T02:43:41.119 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:41.119 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:41.119 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:41.119 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-07T02:43:41.119 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.119 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": { 2024-05-07T02:43:41.119 INFO:teuthology.orchestra.run.smithi064.stdout: "request": 0, 2024-05-07T02:43:41.119 INFO:teuthology.orchestra.run.smithi064.stdout: "reply": 0, 2024-05-07T02:43:41.120 INFO:teuthology.orchestra.run.smithi064.stdout: "reply_latency": { 2024-05-07T02:43:41.120 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:41.120 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:41.120 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:41.120 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.120 INFO:teuthology.orchestra.run.smithi064.stdout: "slow_reply": 0, 2024-05-07T02:43:41.120 INFO:teuthology.orchestra.run.smithi064.stdout: "forward": 0, 2024-05-07T02:43:41.120 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_fetch_complete": 0, 2024-05-07T02:43:41.120 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_fetch_keys": 0, 2024-05-07T02:43:41.120 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_commit": 0, 2024-05-07T02:43:41.121 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_split": 0, 2024-05-07T02:43:41.121 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_merge": 0, 2024-05-07T02:43:41.121 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes": 2101, 2024-05-07T02:43:41.121 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_top": 11, 2024-05-07T02:43:41.121 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_bottom": 2090, 2024-05-07T02:43:41.121 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_pin_tail": 0, 2024-05-07T02:43:41.121 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_pinned": 2086, 2024-05-07T02:43:41.121 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_expired": 0, 2024-05-07T02:43:41.121 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_with_caps": 0, 2024-05-07T02:43:41.122 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": 0, 2024-05-07T02:43:41.122 INFO:teuthology.orchestra.run.smithi064.stdout: "subtrees": 14, 2024-05-07T02:43:41.122 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse": 0, 2024-05-07T02:43:41.122 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_hit": 0, 2024-05-07T02:43:41.122 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_forward": 0, 2024-05-07T02:43:41.122 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_discover": 0, 2024-05-07T02:43:41.122 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_dir_fetch": 0, 2024-05-07T02:43:41.122 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_remote_ino": 0, 2024-05-07T02:43:41.122 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_lock": 0, 2024-05-07T02:43:41.123 INFO:teuthology.orchestra.run.smithi064.stdout: "load_cent": 0, 2024-05-07T02:43:41.123 INFO:teuthology.orchestra.run.smithi064.stdout: "q": 0, 2024-05-07T02:43:41.123 INFO:teuthology.orchestra.run.smithi064.stdout: "exported": 0, 2024-05-07T02:43:41.123 INFO:teuthology.orchestra.run.smithi064.stdout: "exported_inodes": 0, 2024-05-07T02:43:41.123 INFO:teuthology.orchestra.run.smithi064.stdout: "imported": 0, 2024-05-07T02:43:41.123 INFO:teuthology.orchestra.run.smithi064.stdout: "imported_inodes": 0, 2024-05-07T02:43:41.123 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_dir_fetch": 0, 2024-05-07T02:43:41.123 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_backtrace_fetch": 0, 2024-05-07T02:43:41.124 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_peer_discover": 0, 2024-05-07T02:43:41.124 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rfiles": 67, 2024-05-07T02:43:41.124 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rbytes": 21176629, 2024-05-07T02:43:41.124 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rsnaps": 6, 2024-05-07T02:43:41.124 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_backtrace_fetch": 0, 2024-05-07T02:43:41.124 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_set_tag": 0, 2024-05-07T02:43:41.124 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_backtrace_repaired": 0, 2024-05-07T02:43:41.124 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_inotable_repaired": 0, 2024-05-07T02:43:41.124 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dir_inodes": 0, 2024-05-07T02:43:41.124 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dir_base_inodes": 0, 2024-05-07T02:43:41.125 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dirfrag_rstats": 0, 2024-05-07T02:43:41.125 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_file_inodes": 0, 2024-05-07T02:43:41.125 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_inode_file_caps": 0, 2024-05-07T02:43:41.125 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_revoke": 0, 2024-05-07T02:43:41.125 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_grant": 0, 2024-05-07T02:43:41.125 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_trunc": 0, 2024-05-07T02:43:41.125 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-07T02:43:41.125 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-07T02:43:41.126 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_caps": 0, 2024-05-07T02:43:41.126 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_caps_dirty": 0, 2024-05-07T02:43:41.126 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_cap_release": 0, 2024-05-07T02:43:41.126 INFO:teuthology.orchestra.run.smithi064.stdout: "process_request_cap_release": 0 2024-05-07T02:43:41.126 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.126 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_cache": { 2024-05-07T02:43:41.126 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_update": 0, 2024-05-07T02:43:41.126 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_update_receipt": 0, 2024-05-07T02:43:41.126 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_try_discover": 0, 2024-05-07T02:43:41.127 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_send_discover": 0, 2024-05-07T02:43:41.127 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_handle_discover": 0, 2024-05-07T02:43:41.127 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays": 0, 2024-05-07T02:43:41.127 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays_delayed": 0, 2024-05-07T02:43:41.127 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays_enqueuing": 0, 2024-05-07T02:43:41.127 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_created": 0, 2024-05-07T02:43:41.127 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_enqueued": 0, 2024-05-07T02:43:41.127 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_reintegrated": 0, 2024-05-07T02:43:41.127 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_migrated": 0, 2024-05-07T02:43:41.128 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_processing": 0, 2024-05-07T02:43:41.128 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_enqueued": 0, 2024-05-07T02:43:41.128 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_prioritized": 0, 2024-05-07T02:43:41.128 INFO:teuthology.orchestra.run.smithi064.stdout: "recovery_started": 0, 2024-05-07T02:43:41.128 INFO:teuthology.orchestra.run.smithi064.stdout: "recovery_completed": 0, 2024-05-07T02:43:41.128 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_quiesce_path": 0, 2024-05-07T02:43:41.128 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_quiesce_inode": 0, 2024-05-07T02:43:41.128 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_enqueue_scrub": 0, 2024-05-07T02:43:41.128 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_exportdir": 0, 2024-05-07T02:43:41.129 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_flush": 0, 2024-05-07T02:43:41.129 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_fragmentdir": 0, 2024-05-07T02:43:41.129 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_fragstats": 0, 2024-05-07T02:43:41.129 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_inodestats": 0 2024-05-07T02:43:41.129 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.129 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_log": { 2024-05-07T02:43:41.129 INFO:teuthology.orchestra.run.smithi064.stdout: "evlrg": 0, 2024-05-07T02:43:41.129 INFO:teuthology.orchestra.run.smithi064.stdout: "evadd": 0, 2024-05-07T02:43:41.129 INFO:teuthology.orchestra.run.smithi064.stdout: "evex": 0, 2024-05-07T02:43:41.130 INFO:teuthology.orchestra.run.smithi064.stdout: "evtrm": 0, 2024-05-07T02:43:41.130 INFO:teuthology.orchestra.run.smithi064.stdout: "ev": 20028, 2024-05-07T02:43:41.130 INFO:teuthology.orchestra.run.smithi064.stdout: "evexg": 0, 2024-05-07T02:43:41.130 INFO:teuthology.orchestra.run.smithi064.stdout: "evexd": 0, 2024-05-07T02:43:41.130 INFO:teuthology.orchestra.run.smithi064.stdout: "segadd": 0, 2024-05-07T02:43:41.130 INFO:teuthology.orchestra.run.smithi064.stdout: "segex": 0, 2024-05-07T02:43:41.130 INFO:teuthology.orchestra.run.smithi064.stdout: "segtrm": 0, 2024-05-07T02:43:41.130 INFO:teuthology.orchestra.run.smithi064.stdout: "seg": 30, 2024-05-07T02:43:41.130 INFO:teuthology.orchestra.run.smithi064.stdout: "segmjr": 3, 2024-05-07T02:43:41.131 INFO:teuthology.orchestra.run.smithi064.stdout: "segexg": 0, 2024-05-07T02:43:41.131 INFO:teuthology.orchestra.run.smithi064.stdout: "segexd": 0, 2024-05-07T02:43:41.131 INFO:teuthology.orchestra.run.smithi064.stdout: "expos": 4194304, 2024-05-07T02:43:41.131 INFO:teuthology.orchestra.run.smithi064.stdout: "wrpos": 94439849, 2024-05-07T02:43:41.131 INFO:teuthology.orchestra.run.smithi064.stdout: "rdpos": 94439713, 2024-05-07T02:43:41.131 INFO:teuthology.orchestra.run.smithi064.stdout: "jlat": { 2024-05-07T02:43:41.131 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:41.131 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:41.131 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:41.132 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.132 INFO:teuthology.orchestra.run.smithi064.stdout: "replayed": 20028 2024-05-07T02:43:41.132 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.132 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_mem": { 2024-05-07T02:43:41.132 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 114, 2024-05-07T02:43:41.132 INFO:teuthology.orchestra.run.smithi064.stdout: "ino+": 2215, 2024-05-07T02:43:41.132 INFO:teuthology.orchestra.run.smithi064.stdout: "ino-": 2101, 2024-05-07T02:43:41.132 INFO:teuthology.orchestra.run.smithi064.stdout: "dir": 39, 2024-05-07T02:43:41.132 INFO:teuthology.orchestra.run.smithi064.stdout: "dir+": 40, 2024-05-07T02:43:41.133 INFO:teuthology.orchestra.run.smithi064.stdout: "dir-": 1, 2024-05-07T02:43:41.133 INFO:teuthology.orchestra.run.smithi064.stdout: "dn": 2101, 2024-05-07T02:43:41.133 INFO:teuthology.orchestra.run.smithi064.stdout: "dn+": 2271, 2024-05-07T02:43:41.133 INFO:teuthology.orchestra.run.smithi064.stdout: "dn-": 170, 2024-05-07T02:43:41.133 INFO:teuthology.orchestra.run.smithi064.stdout: "cap": 0, 2024-05-07T02:43:41.133 INFO:teuthology.orchestra.run.smithi064.stdout: "cap+": 0, 2024-05-07T02:43:41.133 INFO:teuthology.orchestra.run.smithi064.stdout: "cap-": 0, 2024-05-07T02:43:41.133 INFO:teuthology.orchestra.run.smithi064.stdout: "rss": 123052, 2024-05-07T02:43:41.133 INFO:teuthology.orchestra.run.smithi064.stdout: "heap": 215440 2024-05-07T02:43:41.134 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.134 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_server": { 2024-05-07T02:43:41.134 INFO:teuthology.orchestra.run.smithi064.stdout: "dispatch_client_request": 0, 2024-05-07T02:43:41.134 INFO:teuthology.orchestra.run.smithi064.stdout: "dispatch_server_request": 0, 2024-05-07T02:43:41.134 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_request": 0, 2024-05-07T02:43:41.134 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_session": 0, 2024-05-07T02:43:41.134 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_peer_request": 0, 2024-05-07T02:43:41.134 INFO:teuthology.orchestra.run.smithi064.stdout: "req_create_latency": { 2024-05-07T02:43:41.135 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:41.135 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:41.135 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:41.135 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.135 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getattr_latency": { 2024-05-07T02:43:41.135 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:41.135 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:41.135 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:41.135 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.136 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getfilelock_latency": { 2024-05-07T02:43:41.136 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:41.136 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:41.136 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:41.136 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.136 INFO:teuthology.orchestra.run.smithi064.stdout: "req_link_latency": { 2024-05-07T02:43:41.136 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:41.136 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:41.136 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:41.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.137 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookup_latency": { 2024-05-07T02:43:41.137 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:41.137 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:41.137 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:41.137 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.137 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookuphash_latency": { 2024-05-07T02:43:41.137 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:41.137 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:41.138 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:41.138 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.138 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupino_latency": { 2024-05-07T02:43:41.138 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:41.138 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:41.138 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:41.138 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.138 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupname_latency": { 2024-05-07T02:43:41.138 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:41.139 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:41.139 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:41.139 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.139 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupparent_latency": { 2024-05-07T02:43:41.139 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:41.139 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:41.139 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:41.139 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.139 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupsnap_latency": { 2024-05-07T02:43:41.140 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:41.140 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:41.140 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:41.140 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.140 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lssnap_latency": { 2024-05-07T02:43:41.140 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:41.140 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:41.140 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:41.140 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.141 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mkdir_latency": { 2024-05-07T02:43:41.141 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:41.141 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:41.141 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:41.142 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.142 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mknod_latency": { 2024-05-07T02:43:41.142 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:41.142 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:41.142 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:41.142 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.142 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mksnap_latency": { 2024-05-07T02:43:41.142 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:41.143 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:41.143 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:41.143 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.145 INFO:teuthology.orchestra.run.smithi064.stdout: "req_open_latency": { 2024-05-07T02:43:41.146 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:41.146 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:41.146 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:41.146 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.146 INFO:teuthology.orchestra.run.smithi064.stdout: "req_readdir_latency": { 2024-05-07T02:43:41.146 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:41.146 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:41.146 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:41.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.147 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rename_latency": { 2024-05-07T02:43:41.147 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:41.147 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:41.147 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:41.147 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.147 INFO:teuthology.orchestra.run.smithi064.stdout: "req_renamesnap_latency": { 2024-05-07T02:43:41.147 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:41.147 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:41.147 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:41.148 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.148 INFO:teuthology.orchestra.run.smithi064.stdout: "req_snapdiff_latency": { 2024-05-07T02:43:41.148 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:41.148 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:41.148 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:41.148 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.148 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmdir_latency": { 2024-05-07T02:43:41.148 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:41.148 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:41.149 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:41.149 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.149 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmsnap_latency": { 2024-05-07T02:43:41.149 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:41.149 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:41.149 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:41.149 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.149 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmxattr_latency": { 2024-05-07T02:43:41.149 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:41.150 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:41.150 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:41.150 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.150 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setattr_latency": { 2024-05-07T02:43:41.150 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:41.150 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:41.150 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:41.150 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.151 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setdirlayout_latency": { 2024-05-07T02:43:41.151 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:41.151 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:41.151 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:41.151 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.151 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setfilelock_latency": { 2024-05-07T02:43:41.151 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:41.151 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:41.151 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:41.152 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.152 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setlayout_latency": { 2024-05-07T02:43:41.152 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:41.152 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:41.152 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:41.152 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.152 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setxattr_latency": { 2024-05-07T02:43:41.152 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:41.152 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:41.153 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:41.153 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.153 INFO:teuthology.orchestra.run.smithi064.stdout: "req_symlink_latency": { 2024-05-07T02:43:41.153 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:41.153 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:41.153 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:41.153 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.153 INFO:teuthology.orchestra.run.smithi064.stdout: "req_unlink_latency": { 2024-05-07T02:43:41.153 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:41.154 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:41.154 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:41.154 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.154 INFO:teuthology.orchestra.run.smithi064.stdout: "cap_revoke_eviction": 0, 2024-05-07T02:43:41.154 INFO:teuthology.orchestra.run.smithi064.stdout: "cap_acquisition_throttle": 0, 2024-05-07T02:43:41.154 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getvxattr_latency": { 2024-05-07T02:43:41.154 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:41.154 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:41.154 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:41.155 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-07T02:43:41.155 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.155 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_sessions": { 2024-05-07T02:43:41.155 INFO:teuthology.orchestra.run.smithi064.stdout: "session_count": 2, 2024-05-07T02:43:41.155 INFO:teuthology.orchestra.run.smithi064.stdout: "session_add": 5, 2024-05-07T02:43:41.155 INFO:teuthology.orchestra.run.smithi064.stdout: "session_remove": 3, 2024-05-07T02:43:41.155 INFO:teuthology.orchestra.run.smithi064.stdout: "sessions_open": 3, 2024-05-07T02:43:41.155 INFO:teuthology.orchestra.run.smithi064.stdout: "sessions_stale": 0, 2024-05-07T02:43:41.155 INFO:teuthology.orchestra.run.smithi064.stdout: "total_load": 0, 2024-05-07T02:43:41.156 INFO:teuthology.orchestra.run.smithi064.stdout: "average_load": 0, 2024-05-07T02:43:41.156 INFO:teuthology.orchestra.run.smithi064.stdout: "avg_session_uptime": 3464, 2024-05-07T02:43:41.156 INFO:teuthology.orchestra.run.smithi064.stdout: "mdthresh_evicted": 0 2024-05-07T02:43:41.156 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.156 INFO:teuthology.orchestra.run.smithi064.stdout: "mempool": { 2024-05-07T02:43:41.156 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_bytes": 0, 2024-05-07T02:43:41.156 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_items": 0, 2024-05-07T02:43:41.156 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_bytes": 0, 2024-05-07T02:43:41.156 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_items": 0, 2024-05-07T02:43:41.157 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_bytes": 0, 2024-05-07T02:43:41.157 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_items": 0, 2024-05-07T02:43:41.157 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-07T02:43:41.157 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_items": 0, 2024-05-07T02:43:41.157 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-07T02:43:41.157 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_items": 0, 2024-05-07T02:43:41.157 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_bytes": 0, 2024-05-07T02:43:41.157 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_items": 0, 2024-05-07T02:43:41.157 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-07T02:43:41.158 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_items": 0, 2024-05-07T02:43:41.158 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_bytes": 0, 2024-05-07T02:43:41.158 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_items": 0, 2024-05-07T02:43:41.158 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_bytes": 0, 2024-05-07T02:43:41.158 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_items": 0, 2024-05-07T02:43:41.158 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-07T02:43:41.158 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_items": 0, 2024-05-07T02:43:41.158 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-07T02:43:41.159 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_items": 0, 2024-05-07T02:43:41.159 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_bytes": 0, 2024-05-07T02:43:41.159 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_items": 0, 2024-05-07T02:43:41.159 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_bytes": 0, 2024-05-07T02:43:41.159 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_items": 0, 2024-05-07T02:43:41.159 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-07T02:43:41.159 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_items": 0, 2024-05-07T02:43:41.159 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_bytes": 0, 2024-05-07T02:43:41.159 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_items": 0, 2024-05-07T02:43:41.160 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_bytes": 0, 2024-05-07T02:43:41.160 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_items": 0, 2024-05-07T02:43:41.160 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_bytes": 0, 2024-05-07T02:43:41.160 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_items": 0, 2024-05-07T02:43:41.160 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_bytes": 0, 2024-05-07T02:43:41.160 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_items": 0, 2024-05-07T02:43:41.160 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_bytes": 238730, 2024-05-07T02:43:41.160 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_items": 255, 2024-05-07T02:43:41.160 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_bytes": 0, 2024-05-07T02:43:41.160 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_items": 0, 2024-05-07T02:43:41.161 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_bytes": 0, 2024-05-07T02:43:41.161 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_items": 0, 2024-05-07T02:43:41.161 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_bytes": 0, 2024-05-07T02:43:41.161 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_items": 0, 2024-05-07T02:43:41.161 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_bytes": 0, 2024-05-07T02:43:41.161 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_items": 0, 2024-05-07T02:43:41.161 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_bytes": 9608, 2024-05-07T02:43:41.161 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_items": 184, 2024-05-07T02:43:41.161 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_bytes": 0, 2024-05-07T02:43:41.162 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_items": 0, 2024-05-07T02:43:41.162 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_bytes": 0, 2024-05-07T02:43:41.162 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_items": 0, 2024-05-07T02:43:41.162 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_bytes": 1939677, 2024-05-07T02:43:41.162 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_items": 12728, 2024-05-07T02:43:41.162 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_bytes": 0, 2024-05-07T02:43:41.162 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_items": 0, 2024-05-07T02:43:41.162 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_bytes": 0, 2024-05-07T02:43:41.163 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_items": 0 2024-05-07T02:43:41.163 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.163 INFO:teuthology.orchestra.run.smithi064.stdout: "objecter": { 2024-05-07T02:43:41.163 INFO:teuthology.orchestra.run.smithi064.stdout: "op_active": 0, 2024-05-07T02:43:41.163 INFO:teuthology.orchestra.run.smithi064.stdout: "op_laggy": 0, 2024-05-07T02:43:41.163 INFO:teuthology.orchestra.run.smithi064.stdout: "op_send": 3257, 2024-05-07T02:43:41.163 INFO:teuthology.orchestra.run.smithi064.stdout: "op_send_bytes": 26, 2024-05-07T02:43:41.163 INFO:teuthology.orchestra.run.smithi064.stdout: "op_resend": 4, 2024-05-07T02:43:41.163 INFO:teuthology.orchestra.run.smithi064.stdout: "op_reply": 3257, 2024-05-07T02:43:41.164 INFO:teuthology.orchestra.run.smithi064.stdout: "op_latency": { 2024-05-07T02:43:41.164 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 3257, 2024-05-07T02:43:41.164 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 2.136958857, 2024-05-07T02:43:41.164 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000656112 2024-05-07T02:43:41.164 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.164 INFO:teuthology.orchestra.run.smithi064.stdout: "op_inflight": 0, 2024-05-07T02:43:41.164 INFO:teuthology.orchestra.run.smithi064.stdout: "oplen_avg": { 2024-05-07T02:43:41.164 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 3257, 2024-05-07T02:43:41.164 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 3258 2024-05-07T02:43:41.165 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.165 INFO:teuthology.orchestra.run.smithi064.stdout: "op": 3257, 2024-05-07T02:43:41.165 INFO:teuthology.orchestra.run.smithi064.stdout: "op_r": 3257, 2024-05-07T02:43:41.165 INFO:teuthology.orchestra.run.smithi064.stdout: "op_w": 0, 2024-05-07T02:43:41.165 INFO:teuthology.orchestra.run.smithi064.stdout: "op_rmw": 0, 2024-05-07T02:43:41.165 INFO:teuthology.orchestra.run.smithi064.stdout: "op_pg": 0, 2024-05-07T02:43:41.165 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_stat": 1913, 2024-05-07T02:43:41.165 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_create": 0, 2024-05-07T02:43:41.165 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_read": 1341, 2024-05-07T02:43:41.166 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_write": 0, 2024-05-07T02:43:41.166 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_writefull": 0, 2024-05-07T02:43:41.166 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_writesame": 0, 2024-05-07T02:43:41.166 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_append": 0, 2024-05-07T02:43:41.166 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_zero": 0, 2024-05-07T02:43:41.166 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_truncate": 0, 2024-05-07T02:43:41.166 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_delete": 0, 2024-05-07T02:43:41.166 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_mapext": 0, 2024-05-07T02:43:41.166 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_sparse_read": 0, 2024-05-07T02:43:41.167 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_clonerange": 0, 2024-05-07T02:43:41.167 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_getxattr": 2, 2024-05-07T02:43:41.167 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_setxattr": 0, 2024-05-07T02:43:41.167 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_cmpxattr": 0, 2024-05-07T02:43:41.167 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_rmxattr": 0, 2024-05-07T02:43:41.167 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_resetxattrs": 0, 2024-05-07T02:43:41.167 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_call": 0, 2024-05-07T02:43:41.167 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_watch": 0, 2024-05-07T02:43:41.167 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_notify": 0, 2024-05-07T02:43:41.168 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_src_cmpxattr": 0, 2024-05-07T02:43:41.168 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_pgls": 0, 2024-05-07T02:43:41.168 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_pgls_filter": 0, 2024-05-07T02:43:41.168 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_other": 0, 2024-05-07T02:43:41.168 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_active": 0, 2024-05-07T02:43:41.168 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_send": 0, 2024-05-07T02:43:41.168 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_resend": 0, 2024-05-07T02:43:41.168 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_ping": 0, 2024-05-07T02:43:41.168 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_active": 0, 2024-05-07T02:43:41.169 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_send": 0, 2024-05-07T02:43:41.169 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_resend": 0, 2024-05-07T02:43:41.169 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_active": 0, 2024-05-07T02:43:41.169 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_send": 0, 2024-05-07T02:43:41.169 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_resend": 0, 2024-05-07T02:43:41.169 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_active": 0, 2024-05-07T02:43:41.169 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_send": 0, 2024-05-07T02:43:41.169 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_resend": 0, 2024-05-07T02:43:41.169 INFO:teuthology.orchestra.run.smithi064.stdout: "command_active": 0, 2024-05-07T02:43:41.170 INFO:teuthology.orchestra.run.smithi064.stdout: "command_send": 0, 2024-05-07T02:43:41.170 INFO:teuthology.orchestra.run.smithi064.stdout: "command_resend": 0, 2024-05-07T02:43:41.170 INFO:teuthology.orchestra.run.smithi064.stdout: "map_epoch": 100, 2024-05-07T02:43:41.170 INFO:teuthology.orchestra.run.smithi064.stdout: "map_full": 0, 2024-05-07T02:43:41.170 INFO:teuthology.orchestra.run.smithi064.stdout: "map_inc": 13, 2024-05-07T02:43:41.170 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_sessions": 10, 2024-05-07T02:43:41.170 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_session_open": 10, 2024-05-07T02:43:41.170 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_session_close": 0, 2024-05-07T02:43:41.170 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_laggy": 0, 2024-05-07T02:43:41.171 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_wr": 0, 2024-05-07T02:43:41.171 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_rd": 2, 2024-05-07T02:43:41.171 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_del": 0 2024-05-07T02:43:41.171 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.171 INFO:teuthology.orchestra.run.smithi064.stdout: "oft": { 2024-05-07T02:43:41.171 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_objs": 0, 2024-05-07T02:43:41.171 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_kv_pairs": 0, 2024-05-07T02:43:41.171 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_updates": 0, 2024-05-07T02:43:41.171 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_removes": 0 2024-05-07T02:43:41.172 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.172 INFO:teuthology.orchestra.run.smithi064.stdout: "osd-slow-ops": { 2024-05-07T02:43:41.172 INFO:teuthology.orchestra.run.smithi064.stdout: "slow_ops_count": 0 2024-05-07T02:43:41.172 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.172 INFO:teuthology.orchestra.run.smithi064.stdout: "purge_queue": { 2024-05-07T02:43:41.172 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_ops": 0, 2024-05-07T02:43:41.172 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_ops_high_water": 0, 2024-05-07T02:43:41.172 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing": 0, 2024-05-07T02:43:41.172 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_high_water": 0, 2024-05-07T02:43:41.173 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executed_ops": 0, 2024-05-07T02:43:41.173 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executed": 0, 2024-05-07T02:43:41.173 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_item_in_journal": 0 2024-05-07T02:43:41.173 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.173 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-07T02:43:41.173 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-05-07T02:43:41.173 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2024-05-07T02:43:41.173 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-05-07T02:43:41.173 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 3602, 2024-05-07T02:43:41.173 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 92714636, 2024-05-07T02:43:41.174 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:41.174 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 3602, 2024-05-07T02:43:41.174 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-05-07T02:43:41.174 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-05-07T02:43:41.174 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 3602, 2024-05-07T02:43:41.174 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 92714636, 2024-05-07T02:43:41.174 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-05-07T02:43:41.174 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:41.175 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:41.175 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:41.175 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-07T02:43:41.175 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.175 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-objecter_bytes": { 2024-05-07T02:43:41.175 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-05-07T02:43:41.175 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2024-05-07T02:43:41.175 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-05-07T02:43:41.175 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2024-05-07T02:43:41.176 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2024-05-07T02:43:41.176 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:41.176 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2024-05-07T02:43:41.176 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 3257, 2024-05-07T02:43:41.176 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 90245555, 2024-05-07T02:43:41.176 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 3257, 2024-05-07T02:43:41.176 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 90245555, 2024-05-07T02:43:41.176 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-05-07T02:43:41.176 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:41.177 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:41.177 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:41.177 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-07T02:43:41.177 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.177 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-objecter_ops": { 2024-05-07T02:43:41.177 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-05-07T02:43:41.177 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 1024, 2024-05-07T02:43:41.177 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-05-07T02:43:41.177 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2024-05-07T02:43:41.178 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2024-05-07T02:43:41.178 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:41.178 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2024-05-07T02:43:41.178 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 3257, 2024-05-07T02:43:41.178 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 3257, 2024-05-07T02:43:41.178 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 3257, 2024-05-07T02:43:41.178 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 3257, 2024-05-07T02:43:41.178 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-05-07T02:43:41.178 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:41.179 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:41.179 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:41.179 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-07T02:43:41.179 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.179 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-write_buf_throttle": { 2024-05-07T02:43:41.179 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-05-07T02:43:41.179 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 3758096384, 2024-05-07T02:43:41.179 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-05-07T02:43:41.179 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2024-05-07T02:43:41.180 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2024-05-07T02:43:41.180 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:41.180 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2024-05-07T02:43:41.180 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-05-07T02:43:41.180 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-05-07T02:43:41.180 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 0, 2024-05-07T02:43:41.180 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 0, 2024-05-07T02:43:41.180 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-05-07T02:43:41.180 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:41.181 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:41.181 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:41.181 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-07T02:43:41.181 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:41.181 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-write_buf_throttle-0x5600f783e410": { 2024-05-07T02:43:41.181 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-05-07T02:43:41.181 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 3758096384, 2024-05-07T02:43:41.181 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-05-07T02:43:41.181 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2024-05-07T02:43:41.181 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2024-05-07T02:43:41.182 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:41.182 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2024-05-07T02:43:41.182 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-05-07T02:43:41.182 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-05-07T02:43:41.182 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 0, 2024-05-07T02:43:41.182 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 0, 2024-05-07T02:43:41.182 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-05-07T02:43:41.182 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:41.182 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:41.183 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:41.183 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-07T02:43:41.183 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-07T02:43:41.183 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-07T02:43:41.585 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:43:41 smithi064 ceph-mon[34699]: pgmap v368: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:43:41.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:43:41 smithi019 ceph-mon[29781]: pgmap v368: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:43:41.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:43:41 smithi027 ceph-mon[34398]: pgmap v368: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:43:41.909 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-05-07T02:43:41.910 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-05-07T02:43:41.910 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-05-07T02:43:41.910 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-05-07T02:43:41.910 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-05-07T02:43:41.910 DEBUG:tasks.check_counter:Getting stats from i 2024-05-07T02:43:41.910 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-07T02:43:42.160 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.c/config 2024-05-07T02:43:43.561 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:43:43 smithi064 ceph-mon[34699]: pgmap v369: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:43:43.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:43:43 smithi019 ceph-mon[29781]: pgmap v369: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:43:43.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:43:43 smithi027 ceph-mon[34398]: pgmap v369: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:43:43.842 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-07T02:43:43.842 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-0": { 2024-05-07T02:43:43.842 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 34467, 2024-05-07T02:43:43.843 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 28625, 2024-05-07T02:43:43.843 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 10287986, 2024-05-07T02:43:43.843 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 11987035, 2024-05-07T02:43:43.843 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 27, 2024-05-07T02:43:43.843 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 7, 2024-05-07T02:43:43.843 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 1.993709169, 2024-05-07T02:43:43.843 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.745704692, 2024-05-07T02:43:43.843 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 1.486576969, 2024-05-07T02:43:43.843 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.074625634, 2024-05-07T02:43:43.844 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-05-07T02:43:43.844 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 28628, 2024-05-07T02:43:43.844 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.589271168, 2024-05-07T02:43:43.844 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000020583 2024-05-07T02:43:43.844 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.844 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-05-07T02:43:43.844 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 57890, 2024-05-07T02:43:43.844 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.010228469, 2024-05-07T02:43:43.844 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000176 2024-05-07T02:43:43.845 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.845 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-05-07T02:43:43.845 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 345520 2024-05-07T02:43:43.845 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.845 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-1": { 2024-05-07T02:43:43.845 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 21427, 2024-05-07T02:43:43.845 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 31846, 2024-05-07T02:43:43.845 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 6935138, 2024-05-07T02:43:43.846 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 30718461, 2024-05-07T02:43:43.846 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 13, 2024-05-07T02:43:43.846 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 7, 2024-05-07T02:43:43.846 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 1.556972446, 2024-05-07T02:43:43.846 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.620322124, 2024-05-07T02:43:43.846 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.988217031, 2024-05-07T02:43:43.846 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.110722368, 2024-05-07T02:43:43.846 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-05-07T02:43:43.846 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 31842, 2024-05-07T02:43:43.847 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.663295394, 2024-05-07T02:43:43.847 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000020830 2024-05-07T02:43:43.847 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.847 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-05-07T02:43:43.847 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 38854, 2024-05-07T02:43:43.847 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.008068608, 2024-05-07T02:43:43.847 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000207 2024-05-07T02:43:43.847 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.847 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 3744, 2024-05-07T02:43:43.848 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 187312 2024-05-07T02:43:43.848 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.848 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-2": { 2024-05-07T02:43:43.848 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 2998, 2024-05-07T02:43:43.848 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 2960, 2024-05-07T02:43:43.848 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 2958665, 2024-05-07T02:43:43.848 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 5675871, 2024-05-07T02:43:43.848 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 19, 2024-05-07T02:43:43.848 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 7, 2024-05-07T02:43:43.848 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.267724556, 2024-05-07T02:43:43.849 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.079104880, 2024-05-07T02:43:43.849 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.116082700, 2024-05-07T02:43:43.849 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.076537600, 2024-05-07T02:43:43.849 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-05-07T02:43:43.849 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 2956, 2024-05-07T02:43:43.849 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.102616003, 2024-05-07T02:43:43.849 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000034714 2024-05-07T02:43:43.849 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.849 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-05-07T02:43:43.850 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 268, 2024-05-07T02:43:43.850 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000073881, 2024-05-07T02:43:43.850 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000275 2024-05-07T02:43:43.850 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.850 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 1933440, 2024-05-07T02:43:43.850 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 198480 2024-05-07T02:43:43.850 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.850 INFO:teuthology.orchestra.run.smithi064.stdout: "cct": { 2024-05-07T02:43:43.851 INFO:teuthology.orchestra.run.smithi064.stdout: "total_workers": 1, 2024-05-07T02:43:43.851 INFO:teuthology.orchestra.run.smithi064.stdout: "unhealthy_workers": 0 2024-05-07T02:43:43.851 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.851 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-MDSRank": { 2024-05-07T02:43:43.851 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2024-05-07T02:43:43.851 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2024-05-07T02:43:43.851 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 8159, 2024-05-07T02:43:43.851 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 3.730307304, 2024-05-07T02:43:43.851 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000457201 2024-05-07T02:43:43.852 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-07T02:43:43.852 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.852 INFO:teuthology.orchestra.run.smithi064.stdout: "finisher-PurgeQueue": { 2024-05-07T02:43:43.852 INFO:teuthology.orchestra.run.smithi064.stdout: "queue_len": 0, 2024-05-07T02:43:43.852 INFO:teuthology.orchestra.run.smithi064.stdout: "complete_latency": { 2024-05-07T02:43:43.852 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 3904, 2024-05-07T02:43:43.852 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.236760772, 2024-05-07T02:43:43.852 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000060645 2024-05-07T02:43:43.852 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-07T02:43:43.852 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.853 INFO:teuthology.orchestra.run.smithi064.stdout: "mds": { 2024-05-07T02:43:43.853 INFO:teuthology.orchestra.run.smithi064.stdout: "request": 10209, 2024-05-07T02:43:43.853 INFO:teuthology.orchestra.run.smithi064.stdout: "reply": 9458, 2024-05-07T02:43:43.853 INFO:teuthology.orchestra.run.smithi064.stdout: "reply_latency": { 2024-05-07T02:43:43.853 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 9458, 2024-05-07T02:43:43.853 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 49.132218788, 2024-05-07T02:43:43.853 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.005194778 2024-05-07T02:43:43.853 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.853 INFO:teuthology.orchestra.run.smithi064.stdout: "slow_reply": 0, 2024-05-07T02:43:43.854 INFO:teuthology.orchestra.run.smithi064.stdout: "forward": 751, 2024-05-07T02:43:43.854 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_fetch_complete": 0, 2024-05-07T02:43:43.854 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_fetch_keys": 0, 2024-05-07T02:43:43.854 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_commit": 11, 2024-05-07T02:43:43.854 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_split": 0, 2024-05-07T02:43:43.854 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_merge": 0, 2024-05-07T02:43:43.854 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes": 144, 2024-05-07T02:43:43.854 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_top": 18, 2024-05-07T02:43:43.854 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_bottom": 95, 2024-05-07T02:43:43.855 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_pin_tail": 31, 2024-05-07T02:43:43.855 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_pinned": 119, 2024-05-07T02:43:43.855 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_expired": 97, 2024-05-07T02:43:43.855 INFO:teuthology.orchestra.run.smithi064.stdout: "inodes_with_caps": 6, 2024-05-07T02:43:43.855 INFO:teuthology.orchestra.run.smithi064.stdout: "caps": 6, 2024-05-07T02:43:43.855 INFO:teuthology.orchestra.run.smithi064.stdout: "subtrees": 11, 2024-05-07T02:43:43.855 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse": 25100, 2024-05-07T02:43:43.855 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_hit": 12287, 2024-05-07T02:43:43.855 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_forward": 749, 2024-05-07T02:43:43.856 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_discover": 28, 2024-05-07T02:43:43.856 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_dir_fetch": 0, 2024-05-07T02:43:43.856 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_remote_ino": 0, 2024-05-07T02:43:43.856 INFO:teuthology.orchestra.run.smithi064.stdout: "traverse_lock": 567, 2024-05-07T02:43:43.856 INFO:teuthology.orchestra.run.smithi064.stdout: "load_cent": 0, 2024-05-07T02:43:43.856 INFO:teuthology.orchestra.run.smithi064.stdout: "q": 0, 2024-05-07T02:43:43.856 INFO:teuthology.orchestra.run.smithi064.stdout: "exported": 1, 2024-05-07T02:43:43.856 INFO:teuthology.orchestra.run.smithi064.stdout: "exported_inodes": 23, 2024-05-07T02:43:43.856 INFO:teuthology.orchestra.run.smithi064.stdout: "imported": 4, 2024-05-07T02:43:43.857 INFO:teuthology.orchestra.run.smithi064.stdout: "imported_inodes": 61, 2024-05-07T02:43:43.857 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_dir_fetch": 0, 2024-05-07T02:43:43.857 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_backtrace_fetch": 163, 2024-05-07T02:43:43.857 INFO:teuthology.orchestra.run.smithi064.stdout: "openino_peer_discover": 0, 2024-05-07T02:43:43.857 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rfiles": 67, 2024-05-07T02:43:43.857 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rbytes": 21176629, 2024-05-07T02:43:43.857 INFO:teuthology.orchestra.run.smithi064.stdout: "root_rsnaps": 6, 2024-05-07T02:43:43.857 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_backtrace_fetch": 163, 2024-05-07T02:43:43.857 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_set_tag": 0, 2024-05-07T02:43:43.858 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_backtrace_repaired": 0, 2024-05-07T02:43:43.858 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_inotable_repaired": 0, 2024-05-07T02:43:43.858 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dir_inodes": 0, 2024-05-07T02:43:43.858 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dir_base_inodes": 0, 2024-05-07T02:43:43.858 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_dirfrag_rstats": 0, 2024-05-07T02:43:43.858 INFO:teuthology.orchestra.run.smithi064.stdout: "scrub_file_inodes": 163, 2024-05-07T02:43:43.858 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_inode_file_caps": 0, 2024-05-07T02:43:43.858 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_revoke": 1874, 2024-05-07T02:43:43.859 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_grant": 8515, 2024-05-07T02:43:43.859 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_trunc": 0, 2024-05-07T02:43:43.859 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-07T02:43:43.859 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-07T02:43:43.859 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_caps": 5473, 2024-05-07T02:43:43.859 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_caps_dirty": 2422, 2024-05-07T02:43:43.859 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_cap_release": 622, 2024-05-07T02:43:43.859 INFO:teuthology.orchestra.run.smithi064.stdout: "process_request_cap_release": 9688 2024-05-07T02:43:43.859 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.860 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_cache": { 2024-05-07T02:43:43.860 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_update": 0, 2024-05-07T02:43:43.860 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_update_receipt": 9, 2024-05-07T02:43:43.860 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_try_discover": 6, 2024-05-07T02:43:43.860 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_send_discover": 33, 2024-05-07T02:43:43.860 INFO:teuthology.orchestra.run.smithi064.stdout: "dir_handle_discover": 31, 2024-05-07T02:43:43.860 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays": 0, 2024-05-07T02:43:43.860 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays_delayed": 0, 2024-05-07T02:43:43.860 INFO:teuthology.orchestra.run.smithi064.stdout: "num_strays_enqueuing": 0, 2024-05-07T02:43:43.861 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_created": 1703, 2024-05-07T02:43:43.861 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_enqueued": 1703, 2024-05-07T02:43:43.861 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_reintegrated": 0, 2024-05-07T02:43:43.861 INFO:teuthology.orchestra.run.smithi064.stdout: "strays_migrated": 0, 2024-05-07T02:43:43.861 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_processing": 0, 2024-05-07T02:43:43.861 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_enqueued": 0, 2024-05-07T02:43:43.861 INFO:teuthology.orchestra.run.smithi064.stdout: "num_recovering_prioritized": 0, 2024-05-07T02:43:43.861 INFO:teuthology.orchestra.run.smithi064.stdout: "recovery_started": 0, 2024-05-07T02:43:43.861 INFO:teuthology.orchestra.run.smithi064.stdout: "recovery_completed": 0, 2024-05-07T02:43:43.862 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_quiesce_path": 0, 2024-05-07T02:43:43.862 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_quiesce_inode": 0, 2024-05-07T02:43:43.862 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_enqueue_scrub": 0, 2024-05-07T02:43:43.862 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_exportdir": 2, 2024-05-07T02:43:43.862 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_flush": 0, 2024-05-07T02:43:43.862 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_fragmentdir": 0, 2024-05-07T02:43:43.862 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_fragstats": 0, 2024-05-07T02:43:43.862 INFO:teuthology.orchestra.run.smithi064.stdout: "ireq_inodestats": 0 2024-05-07T02:43:43.862 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.863 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_log": { 2024-05-07T02:43:43.863 INFO:teuthology.orchestra.run.smithi064.stdout: "evlrg": 0, 2024-05-07T02:43:43.863 INFO:teuthology.orchestra.run.smithi064.stdout: "evadd": 11686, 2024-05-07T02:43:43.863 INFO:teuthology.orchestra.run.smithi064.stdout: "evex": 0, 2024-05-07T02:43:43.863 INFO:teuthology.orchestra.run.smithi064.stdout: "evtrm": 0, 2024-05-07T02:43:43.863 INFO:teuthology.orchestra.run.smithi064.stdout: "ev": 11686, 2024-05-07T02:43:43.863 INFO:teuthology.orchestra.run.smithi064.stdout: "evexg": 0, 2024-05-07T02:43:43.863 INFO:teuthology.orchestra.run.smithi064.stdout: "evexd": 0, 2024-05-07T02:43:43.863 INFO:teuthology.orchestra.run.smithi064.stdout: "segadd": 13, 2024-05-07T02:43:43.864 INFO:teuthology.orchestra.run.smithi064.stdout: "segex": 0, 2024-05-07T02:43:43.864 INFO:teuthology.orchestra.run.smithi064.stdout: "segtrm": 0, 2024-05-07T02:43:43.864 INFO:teuthology.orchestra.run.smithi064.stdout: "seg": 13, 2024-05-07T02:43:43.864 INFO:teuthology.orchestra.run.smithi064.stdout: "segmjr": 2, 2024-05-07T02:43:43.864 INFO:teuthology.orchestra.run.smithi064.stdout: "segexg": 0, 2024-05-07T02:43:43.864 INFO:teuthology.orchestra.run.smithi064.stdout: "segexd": 0, 2024-05-07T02:43:43.864 INFO:teuthology.orchestra.run.smithi064.stdout: "expos": 4194304, 2024-05-07T02:43:43.864 INFO:teuthology.orchestra.run.smithi064.stdout: "wrpos": 20851604, 2024-05-07T02:43:43.864 INFO:teuthology.orchestra.run.smithi064.stdout: "rdpos": 0, 2024-05-07T02:43:43.865 INFO:teuthology.orchestra.run.smithi064.stdout: "jlat": { 2024-05-07T02:43:43.865 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 3894, 2024-05-07T02:43:43.865 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 13.690168988, 2024-05-07T02:43:43.865 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.003515708 2024-05-07T02:43:43.865 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.865 INFO:teuthology.orchestra.run.smithi064.stdout: "replayed": 0 2024-05-07T02:43:43.865 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.865 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_mem": { 2024-05-07T02:43:43.865 INFO:teuthology.orchestra.run.smithi064.stdout: "ino": 127, 2024-05-07T02:43:43.866 INFO:teuthology.orchestra.run.smithi064.stdout: "ino+": 1895, 2024-05-07T02:43:43.866 INFO:teuthology.orchestra.run.smithi064.stdout: "ino-": 1768, 2024-05-07T02:43:43.866 INFO:teuthology.orchestra.run.smithi064.stdout: "dir": 41, 2024-05-07T02:43:43.866 INFO:teuthology.orchestra.run.smithi064.stdout: "dir+": 45, 2024-05-07T02:43:43.866 INFO:teuthology.orchestra.run.smithi064.stdout: "dir-": 4, 2024-05-07T02:43:43.866 INFO:teuthology.orchestra.run.smithi064.stdout: "dn": 144, 2024-05-07T02:43:43.866 INFO:teuthology.orchestra.run.smithi064.stdout: "dn+": 2037, 2024-05-07T02:43:43.866 INFO:teuthology.orchestra.run.smithi064.stdout: "dn-": 1893, 2024-05-07T02:43:43.867 INFO:teuthology.orchestra.run.smithi064.stdout: "cap": 6, 2024-05-07T02:43:43.867 INFO:teuthology.orchestra.run.smithi064.stdout: "cap+": 1817, 2024-05-07T02:43:43.867 INFO:teuthology.orchestra.run.smithi064.stdout: "cap-": 1811, 2024-05-07T02:43:43.867 INFO:teuthology.orchestra.run.smithi064.stdout: "rss": 134024, 2024-05-07T02:43:43.867 INFO:teuthology.orchestra.run.smithi064.stdout: "heap": 223632 2024-05-07T02:43:43.867 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.867 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_server": { 2024-05-07T02:43:43.867 INFO:teuthology.orchestra.run.smithi064.stdout: "dispatch_client_request": 20708, 2024-05-07T02:43:43.867 INFO:teuthology.orchestra.run.smithi064.stdout: "dispatch_server_request": 220, 2024-05-07T02:43:43.868 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_request": 10209, 2024-05-07T02:43:43.868 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_client_session": 492, 2024-05-07T02:43:43.868 INFO:teuthology.orchestra.run.smithi064.stdout: "handle_peer_request": 4159, 2024-05-07T02:43:43.868 INFO:teuthology.orchestra.run.smithi064.stdout: "req_create_latency": { 2024-05-07T02:43:43.868 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 1704, 2024-05-07T02:43:43.868 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 21.848908877, 2024-05-07T02:43:43.868 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.012822129 2024-05-07T02:43:43.868 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.868 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getattr_latency": { 2024-05-07T02:43:43.869 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 59, 2024-05-07T02:43:43.869 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.043311571, 2024-05-07T02:43:43.869 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000734094 2024-05-07T02:43:43.869 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.869 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getfilelock_latency": { 2024-05-07T02:43:43.869 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:43.869 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:43.869 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:43.869 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.870 INFO:teuthology.orchestra.run.smithi064.stdout: "req_link_latency": { 2024-05-07T02:43:43.870 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:43.870 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:43.870 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:43.870 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.870 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookup_latency": { 2024-05-07T02:43:43.870 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 2365, 2024-05-07T02:43:43.870 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 20.449115567, 2024-05-07T02:43:43.870 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.008646560 2024-05-07T02:43:43.871 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.871 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookuphash_latency": { 2024-05-07T02:43:43.871 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:43.871 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:43.871 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:43.871 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.871 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupino_latency": { 2024-05-07T02:43:43.871 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 1, 2024-05-07T02:43:43.871 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000149245, 2024-05-07T02:43:43.872 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000149245 2024-05-07T02:43:43.872 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.872 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupname_latency": { 2024-05-07T02:43:43.872 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:43.872 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:43.872 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:43.872 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.872 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupparent_latency": { 2024-05-07T02:43:43.872 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:43.872 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:43.873 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:43.873 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.873 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lookupsnap_latency": { 2024-05-07T02:43:43.873 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:43.873 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:43.873 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:43.873 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.873 INFO:teuthology.orchestra.run.smithi064.stdout: "req_lssnap_latency": { 2024-05-07T02:43:43.874 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:43.874 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:43.874 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:43.875 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.875 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mkdir_latency": { 2024-05-07T02:43:43.875 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:43.875 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:43.875 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:43.875 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.875 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mknod_latency": { 2024-05-07T02:43:43.875 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:43.875 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:43.876 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:43.876 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.876 INFO:teuthology.orchestra.run.smithi064.stdout: "req_mksnap_latency": { 2024-05-07T02:43:43.876 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:43.876 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:43.876 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:43.876 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.876 INFO:teuthology.orchestra.run.smithi064.stdout: "req_open_latency": { 2024-05-07T02:43:43.876 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:43.877 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:43.877 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:43.877 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.877 INFO:teuthology.orchestra.run.smithi064.stdout: "req_readdir_latency": { 2024-05-07T02:43:43.877 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 2987, 2024-05-07T02:43:43.877 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 1.592702667, 2024-05-07T02:43:43.877 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000533211 2024-05-07T02:43:43.877 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.877 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rename_latency": { 2024-05-07T02:43:43.878 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 572, 2024-05-07T02:43:43.878 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 1.687174760, 2024-05-07T02:43:43.878 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.002949606 2024-05-07T02:43:43.878 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.878 INFO:teuthology.orchestra.run.smithi064.stdout: "req_renamesnap_latency": { 2024-05-07T02:43:43.878 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:43.878 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:43.878 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:43.878 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.879 INFO:teuthology.orchestra.run.smithi064.stdout: "req_snapdiff_latency": { 2024-05-07T02:43:43.879 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:43.879 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:43.879 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:43.879 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.879 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmdir_latency": { 2024-05-07T02:43:43.879 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:43.879 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:43.879 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:43.880 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.880 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmsnap_latency": { 2024-05-07T02:43:43.880 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:43.880 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:43.880 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:43.880 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.880 INFO:teuthology.orchestra.run.smithi064.stdout: "req_rmxattr_latency": { 2024-05-07T02:43:43.880 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:43.880 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:43.881 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:43.881 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.881 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setattr_latency": { 2024-05-07T02:43:43.881 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 3, 2024-05-07T02:43:43.881 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.019068401, 2024-05-07T02:43:43.881 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.006356133 2024-05-07T02:43:43.881 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.881 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setdirlayout_latency": { 2024-05-07T02:43:43.881 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:43.881 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:43.882 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:43.882 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.882 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setfilelock_latency": { 2024-05-07T02:43:43.882 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 64, 2024-05-07T02:43:43.882 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.027840823, 2024-05-07T02:43:43.882 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000435012 2024-05-07T02:43:43.882 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.882 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setlayout_latency": { 2024-05-07T02:43:43.883 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:43.883 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:43.883 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:43.883 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.883 INFO:teuthology.orchestra.run.smithi064.stdout: "req_setxattr_latency": { 2024-05-07T02:43:43.883 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:43.883 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:43.883 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:43.883 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.884 INFO:teuthology.orchestra.run.smithi064.stdout: "req_symlink_latency": { 2024-05-07T02:43:43.884 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:43.884 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:43.884 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:43.884 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.884 INFO:teuthology.orchestra.run.smithi064.stdout: "req_unlink_latency": { 2024-05-07T02:43:43.884 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 1703, 2024-05-07T02:43:43.884 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 3.463946877, 2024-05-07T02:43:43.884 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.002034026 2024-05-07T02:43:43.884 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.885 INFO:teuthology.orchestra.run.smithi064.stdout: "cap_revoke_eviction": 0, 2024-05-07T02:43:43.885 INFO:teuthology.orchestra.run.smithi064.stdout: "cap_acquisition_throttle": 0, 2024-05-07T02:43:43.885 INFO:teuthology.orchestra.run.smithi064.stdout: "req_getvxattr_latency": { 2024-05-07T02:43:43.885 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:43.885 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:43.885 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:43.885 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-07T02:43:43.885 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.885 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_sessions": { 2024-05-07T02:43:43.886 INFO:teuthology.orchestra.run.smithi064.stdout: "session_count": 2, 2024-05-07T02:43:43.886 INFO:teuthology.orchestra.run.smithi064.stdout: "session_add": 4, 2024-05-07T02:43:43.886 INFO:teuthology.orchestra.run.smithi064.stdout: "session_remove": 2, 2024-05-07T02:43:43.886 INFO:teuthology.orchestra.run.smithi064.stdout: "sessions_open": 2, 2024-05-07T02:43:43.886 INFO:teuthology.orchestra.run.smithi064.stdout: "sessions_stale": 0, 2024-05-07T02:43:43.886 INFO:teuthology.orchestra.run.smithi064.stdout: "total_load": 2516, 2024-05-07T02:43:43.886 INFO:teuthology.orchestra.run.smithi064.stdout: "average_load": 1258, 2024-05-07T02:43:43.886 INFO:teuthology.orchestra.run.smithi064.stdout: "avg_session_uptime": 676, 2024-05-07T02:43:43.886 INFO:teuthology.orchestra.run.smithi064.stdout: "mdthresh_evicted": 0 2024-05-07T02:43:43.887 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.887 INFO:teuthology.orchestra.run.smithi064.stdout: "mempool": { 2024-05-07T02:43:43.887 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_bytes": 0, 2024-05-07T02:43:43.887 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_items": 0, 2024-05-07T02:43:43.887 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_bytes": 0, 2024-05-07T02:43:43.887 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_items": 0, 2024-05-07T02:43:43.887 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_bytes": 0, 2024-05-07T02:43:43.887 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_items": 0, 2024-05-07T02:43:43.887 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-07T02:43:43.888 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_items": 0, 2024-05-07T02:43:43.888 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-07T02:43:43.888 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_items": 0, 2024-05-07T02:43:43.888 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_bytes": 0, 2024-05-07T02:43:43.888 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_items": 0, 2024-05-07T02:43:43.888 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-07T02:43:43.888 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_items": 0, 2024-05-07T02:43:43.888 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_bytes": 0, 2024-05-07T02:43:43.888 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_items": 0, 2024-05-07T02:43:43.889 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_bytes": 0, 2024-05-07T02:43:43.889 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_items": 0, 2024-05-07T02:43:43.889 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-07T02:43:43.889 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_items": 0, 2024-05-07T02:43:43.889 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-07T02:43:43.889 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_items": 0, 2024-05-07T02:43:43.889 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_bytes": 0, 2024-05-07T02:43:43.889 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_items": 0, 2024-05-07T02:43:43.889 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_bytes": 0, 2024-05-07T02:43:43.890 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_items": 0, 2024-05-07T02:43:43.890 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-07T02:43:43.890 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_items": 0, 2024-05-07T02:43:43.890 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_bytes": 0, 2024-05-07T02:43:43.890 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_items": 0, 2024-05-07T02:43:43.890 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_bytes": 0, 2024-05-07T02:43:43.890 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_items": 0, 2024-05-07T02:43:43.890 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_bytes": 0, 2024-05-07T02:43:43.890 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_items": 0, 2024-05-07T02:43:43.891 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_bytes": 0, 2024-05-07T02:43:43.891 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_items": 0, 2024-05-07T02:43:43.891 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_bytes": 304379, 2024-05-07T02:43:43.891 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_items": 195, 2024-05-07T02:43:43.891 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_bytes": 264, 2024-05-07T02:43:43.891 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_items": 3, 2024-05-07T02:43:43.891 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_bytes": 0, 2024-05-07T02:43:43.891 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_items": 0, 2024-05-07T02:43:43.892 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_bytes": 0, 2024-05-07T02:43:43.892 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_items": 0, 2024-05-07T02:43:43.892 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_bytes": 0, 2024-05-07T02:43:43.892 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_items": 0, 2024-05-07T02:43:43.892 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_bytes": 9608, 2024-05-07T02:43:43.892 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_items": 184, 2024-05-07T02:43:43.893 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_bytes": 0, 2024-05-07T02:43:43.893 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_items": 0, 2024-05-07T02:43:43.893 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_bytes": 0, 2024-05-07T02:43:43.893 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_items": 0, 2024-05-07T02:43:43.893 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_bytes": 508877, 2024-05-07T02:43:43.893 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_items": 3475, 2024-05-07T02:43:43.893 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_bytes": 0, 2024-05-07T02:43:43.894 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_items": 0, 2024-05-07T02:43:43.894 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_bytes": 0, 2024-05-07T02:43:43.894 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_items": 0 2024-05-07T02:43:43.894 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.894 INFO:teuthology.orchestra.run.smithi064.stdout: "objecter": { 2024-05-07T02:43:43.894 INFO:teuthology.orchestra.run.smithi064.stdout: "op_active": 0, 2024-05-07T02:43:43.894 INFO:teuthology.orchestra.run.smithi064.stdout: "op_laggy": 0, 2024-05-07T02:43:43.894 INFO:teuthology.orchestra.run.smithi064.stdout: "op_send": 8430, 2024-05-07T02:43:43.894 INFO:teuthology.orchestra.run.smithi064.stdout: "op_send_bytes": 17068687, 2024-05-07T02:43:43.895 INFO:teuthology.orchestra.run.smithi064.stdout: "op_resend": 28, 2024-05-07T02:43:43.895 INFO:teuthology.orchestra.run.smithi064.stdout: "op_reply": 8430, 2024-05-07T02:43:43.895 INFO:teuthology.orchestra.run.smithi064.stdout: "op_latency": { 2024-05-07T02:43:43.895 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 8430, 2024-05-07T02:43:43.895 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 36.675291099, 2024-05-07T02:43:43.895 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.004350568 2024-05-07T02:43:43.895 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.895 INFO:teuthology.orchestra.run.smithi064.stdout: "op_inflight": 0, 2024-05-07T02:43:43.895 INFO:teuthology.orchestra.run.smithi064.stdout: "oplen_avg": { 2024-05-07T02:43:43.896 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 8430, 2024-05-07T02:43:43.896 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 8855 2024-05-07T02:43:43.896 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.896 INFO:teuthology.orchestra.run.smithi064.stdout: "op": 8430, 2024-05-07T02:43:43.896 INFO:teuthology.orchestra.run.smithi064.stdout: "op_r": 411, 2024-05-07T02:43:43.896 INFO:teuthology.orchestra.run.smithi064.stdout: "op_w": 8019, 2024-05-07T02:43:43.896 INFO:teuthology.orchestra.run.smithi064.stdout: "op_rmw": 0, 2024-05-07T02:43:43.896 INFO:teuthology.orchestra.run.smithi064.stdout: "op_pg": 0, 2024-05-07T02:43:43.896 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_stat": 0, 2024-05-07T02:43:43.897 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_create": 10, 2024-05-07T02:43:43.897 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_read": 248, 2024-05-07T02:43:43.897 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_write": 4142, 2024-05-07T02:43:43.897 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_writefull": 167, 2024-05-07T02:43:43.897 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_writesame": 0, 2024-05-07T02:43:43.897 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_append": 0, 2024-05-07T02:43:43.897 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_zero": 0, 2024-05-07T02:43:43.897 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_truncate": 0, 2024-05-07T02:43:43.897 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_delete": 3421, 2024-05-07T02:43:43.897 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_mapext": 0, 2024-05-07T02:43:43.898 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_sparse_read": 0, 2024-05-07T02:43:43.898 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_clonerange": 0, 2024-05-07T02:43:43.898 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_getxattr": 163, 2024-05-07T02:43:43.898 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_setxattr": 20, 2024-05-07T02:43:43.898 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_cmpxattr": 0, 2024-05-07T02:43:43.898 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_rmxattr": 0, 2024-05-07T02:43:43.898 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_resetxattrs": 0, 2024-05-07T02:43:43.898 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_call": 0, 2024-05-07T02:43:43.899 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_watch": 0, 2024-05-07T02:43:43.899 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_notify": 0, 2024-05-07T02:43:43.899 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_src_cmpxattr": 0, 2024-05-07T02:43:43.899 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_pgls": 0, 2024-05-07T02:43:43.899 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_pgls_filter": 0, 2024-05-07T02:43:43.899 INFO:teuthology.orchestra.run.smithi064.stdout: "osdop_other": 0, 2024-05-07T02:43:43.899 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_active": 0, 2024-05-07T02:43:43.899 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_send": 0, 2024-05-07T02:43:43.899 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_resend": 0, 2024-05-07T02:43:43.900 INFO:teuthology.orchestra.run.smithi064.stdout: "linger_ping": 0, 2024-05-07T02:43:43.900 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_active": 0, 2024-05-07T02:43:43.900 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_send": 0, 2024-05-07T02:43:43.900 INFO:teuthology.orchestra.run.smithi064.stdout: "poolop_resend": 0, 2024-05-07T02:43:43.900 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_active": 0, 2024-05-07T02:43:43.900 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_send": 0, 2024-05-07T02:43:43.900 INFO:teuthology.orchestra.run.smithi064.stdout: "poolstat_resend": 0, 2024-05-07T02:43:43.900 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_active": 0, 2024-05-07T02:43:43.900 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_send": 0, 2024-05-07T02:43:43.901 INFO:teuthology.orchestra.run.smithi064.stdout: "statfs_resend": 0, 2024-05-07T02:43:43.901 INFO:teuthology.orchestra.run.smithi064.stdout: "command_active": 0, 2024-05-07T02:43:43.901 INFO:teuthology.orchestra.run.smithi064.stdout: "command_send": 0, 2024-05-07T02:43:43.901 INFO:teuthology.orchestra.run.smithi064.stdout: "command_resend": 0, 2024-05-07T02:43:43.901 INFO:teuthology.orchestra.run.smithi064.stdout: "map_epoch": 100, 2024-05-07T02:43:43.901 INFO:teuthology.orchestra.run.smithi064.stdout: "map_full": 0, 2024-05-07T02:43:43.901 INFO:teuthology.orchestra.run.smithi064.stdout: "map_inc": 13, 2024-05-07T02:43:43.901 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_sessions": 12, 2024-05-07T02:43:43.901 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_session_open": 12, 2024-05-07T02:43:43.902 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_session_close": 0, 2024-05-07T02:43:43.902 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_laggy": 0, 2024-05-07T02:43:43.902 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_wr": 513, 2024-05-07T02:43:43.902 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_rd": 0, 2024-05-07T02:43:43.902 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_del": 171 2024-05-07T02:43:43.902 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.902 INFO:teuthology.orchestra.run.smithi064.stdout: "oft": { 2024-05-07T02:43:43.902 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_objs": 1, 2024-05-07T02:43:43.902 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_kv_pairs": 0, 2024-05-07T02:43:43.903 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_updates": 0, 2024-05-07T02:43:43.903 INFO:teuthology.orchestra.run.smithi064.stdout: "omap_total_removes": 0 2024-05-07T02:43:43.903 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.903 INFO:teuthology.orchestra.run.smithi064.stdout: "osd-slow-ops": { 2024-05-07T02:43:43.903 INFO:teuthology.orchestra.run.smithi064.stdout: "slow_ops_count": 1 2024-05-07T02:43:43.903 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.903 INFO:teuthology.orchestra.run.smithi064.stdout: "purge_queue": { 2024-05-07T02:43:43.903 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_ops": 0, 2024-05-07T02:43:43.903 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_ops_high_water": 10, 2024-05-07T02:43:43.904 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing": 0, 2024-05-07T02:43:43.904 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executing_high_water": 5, 2024-05-07T02:43:43.904 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executed_ops": 3406, 2024-05-07T02:43:43.904 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_executed": 1703, 2024-05-07T02:43:43.904 INFO:teuthology.orchestra.run.smithi064.stdout: "pq_item_in_journal": 0 2024-05-07T02:43:43.904 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.904 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-07T02:43:43.904 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-05-07T02:43:43.904 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2024-05-07T02:43:43.905 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-05-07T02:43:43.905 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 58888, 2024-05-07T02:43:43.905 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 14858497, 2024-05-07T02:43:43.905 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:43.905 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 58888, 2024-05-07T02:43:43.905 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-05-07T02:43:43.905 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-05-07T02:43:43.905 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 58888, 2024-05-07T02:43:43.905 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 14858497, 2024-05-07T02:43:43.906 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-05-07T02:43:43.906 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:43.906 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:43.906 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:43.906 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-07T02:43:43.906 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.906 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-objecter_bytes": { 2024-05-07T02:43:43.906 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-05-07T02:43:43.906 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2024-05-07T02:43:43.907 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-05-07T02:43:43.907 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2024-05-07T02:43:43.907 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2024-05-07T02:43:43.907 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:43.907 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2024-05-07T02:43:43.907 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 8430, 2024-05-07T02:43:43.907 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 17254530, 2024-05-07T02:43:43.907 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 8430, 2024-05-07T02:43:43.907 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 17254530, 2024-05-07T02:43:43.908 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-05-07T02:43:43.908 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:43.908 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:43.908 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:43.908 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-07T02:43:43.908 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.908 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-objecter_ops": { 2024-05-07T02:43:43.908 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-05-07T02:43:43.908 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 1024, 2024-05-07T02:43:43.909 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-05-07T02:43:43.909 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 0, 2024-05-07T02:43:43.909 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 0, 2024-05-07T02:43:43.909 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:43.909 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 0, 2024-05-07T02:43:43.909 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 8430, 2024-05-07T02:43:43.909 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 8430, 2024-05-07T02:43:43.909 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 8430, 2024-05-07T02:43:43.909 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 8430, 2024-05-07T02:43:43.909 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-05-07T02:43:43.910 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:43.910 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:43.910 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:43.910 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-07T02:43:43.910 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.910 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-write_buf_throttle": { 2024-05-07T02:43:43.910 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-05-07T02:43:43.910 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 3758096384, 2024-05-07T02:43:43.911 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-05-07T02:43:43.911 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 1703, 2024-05-07T02:43:43.911 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 185843, 2024-05-07T02:43:43.911 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:43.911 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 1703, 2024-05-07T02:43:43.911 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-05-07T02:43:43.911 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-05-07T02:43:43.911 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 248, 2024-05-07T02:43:43.911 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 185843, 2024-05-07T02:43:43.912 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-05-07T02:43:43.912 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:43.912 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:43.912 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:43.912 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-07T02:43:43.912 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:43.912 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-write_buf_throttle-0x555a719a4410": { 2024-05-07T02:43:43.912 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-05-07T02:43:43.912 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 3758096384, 2024-05-07T02:43:43.912 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-05-07T02:43:43.913 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 11686, 2024-05-07T02:43:43.913 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 16657300, 2024-05-07T02:43:43.913 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:43.913 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 11686, 2024-05-07T02:43:43.913 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-05-07T02:43:43.913 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-05-07T02:43:43.913 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 3894, 2024-05-07T02:43:43.913 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 16657300, 2024-05-07T02:43:43.913 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-05-07T02:43:43.914 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:43.914 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:43.914 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:43.914 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-07T02:43:43.914 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-07T02:43:43.914 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-07T02:43:44.740 INFO:tasks.check_counter:Daemon mds.i mds.exported=1 2024-05-07T02:43:44.740 INFO:tasks.check_counter:Daemon mds.i mds.imported=4 2024-05-07T02:43:44.740 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-05-07T02:43:44.740 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-05-07T02:43:44.740 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-05-07T02:43:44.740 DEBUG:tasks.check_counter:Getting stats from l 2024-05-07T02:43:44.741 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-07T02:43:44.993 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.c/config 2024-05-07T02:43:45.386 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:43:45 smithi064 ceph-mon[34699]: pgmap v370: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:43:45.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:43:45 smithi019 ceph-mon[29781]: pgmap v370: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:43:45.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:43:45 smithi027 ceph-mon[34398]: pgmap v370: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:43:46.789 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-07T02:43:46.789 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-0": { 2024-05-07T02:43:46.790 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 4, 2024-05-07T02:43:46.790 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 4, 2024-05-07T02:43:46.790 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 2944, 2024-05-07T02:43:46.790 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 448, 2024-05-07T02:43:46.790 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 4, 2024-05-07T02:43:46.790 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 0, 2024-05-07T02:43:46.790 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.001558014, 2024-05-07T02:43:46.790 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.000037006, 2024-05-07T02:43:46.791 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.000617208, 2024-05-07T02:43:46.791 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-07T02:43:46.791 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-05-07T02:43:46.791 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 3, 2024-05-07T02:43:46.791 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000812812, 2024-05-07T02:43:46.791 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000270937 2024-05-07T02:43:46.791 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:46.791 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-05-07T02:43:46.792 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:46.792 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:46.792 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:46.792 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:46.792 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-07T02:43:46.792 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 448 2024-05-07T02:43:46.792 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:46.792 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-1": { 2024-05-07T02:43:46.792 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 308, 2024-05-07T02:43:46.793 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 297, 2024-05-07T02:43:46.793 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 1735536, 2024-05-07T02:43:46.793 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 183440, 2024-05-07T02:43:46.793 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 2, 2024-05-07T02:43:46.793 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 1, 2024-05-07T02:43:46.793 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.062010651, 2024-05-07T02:43:46.793 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.018435207, 2024-05-07T02:43:46.793 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.046968771, 2024-05-07T02:43:46.793 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.007446049, 2024-05-07T02:43:46.794 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-05-07T02:43:46.794 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 296, 2024-05-07T02:43:46.794 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.015704649, 2024-05-07T02:43:46.794 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000053056 2024-05-07T02:43:46.794 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:46.794 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-05-07T02:43:46.794 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:46.794 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:46.795 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:46.795 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:46.795 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 1735536, 2024-05-07T02:43:46.795 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 183440 2024-05-07T02:43:46.795 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:46.795 INFO:teuthology.orchestra.run.smithi064.stdout: "AsyncMessenger::Worker-2": { 2024-05-07T02:43:46.795 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_messages": 3, 2024-05-07T02:43:46.795 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages": 229, 2024-05-07T02:43:46.795 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_bytes": 480, 2024-05-07T02:43:46.796 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_bytes": 276240, 2024-05-07T02:43:46.796 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_created_connections": 6, 2024-05-07T02:43:46.796 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_active_connections": 3, 2024-05-07T02:43:46.796 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_total_time": 0.015055857, 2024-05-07T02:43:46.796 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_send_time": 0.010326534, 2024-05-07T02:43:46.796 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_recv_time": 0.001830558, 2024-05-07T02:43:46.796 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-07T02:43:46.796 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_messages_queue_lat": { 2024-05-07T02:43:46.796 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 225, 2024-05-07T02:43:46.797 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.026104949, 2024-05-07T02:43:46.797 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000116021 2024-05-07T02:43:46.797 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:46.797 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_handle_ack_lat": { 2024-05-07T02:43:46.797 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:46.797 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:46.797 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:46.797 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:46.797 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_recv_encrypted_bytes": 480, 2024-05-07T02:43:46.798 INFO:teuthology.orchestra.run.smithi064.stdout: "msgr_send_encrypted_bytes": 276240 2024-05-07T02:43:46.798 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:46.798 INFO:teuthology.orchestra.run.smithi064.stdout: "cct": { 2024-05-07T02:43:46.798 INFO:teuthology.orchestra.run.smithi064.stdout: "total_workers": 0, 2024-05-07T02:43:46.798 INFO:teuthology.orchestra.run.smithi064.stdout: "unhealthy_workers": 0 2024-05-07T02:43:46.798 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:46.798 INFO:teuthology.orchestra.run.smithi064.stdout: "mempool": { 2024-05-07T02:43:46.798 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_bytes": 0, 2024-05-07T02:43:46.799 INFO:teuthology.orchestra.run.smithi064.stdout: "bloom_filter_items": 0, 2024-05-07T02:43:46.799 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_bytes": 0, 2024-05-07T02:43:46.799 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_alloc_items": 0, 2024-05-07T02:43:46.799 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_bytes": 0, 2024-05-07T02:43:46.800 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_data_items": 0, 2024-05-07T02:43:46.800 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-07T02:43:46.800 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_onode_items": 0, 2024-05-07T02:43:46.800 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-07T02:43:46.800 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_meta_items": 0, 2024-05-07T02:43:46.800 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_bytes": 0, 2024-05-07T02:43:46.800 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_other_items": 0, 2024-05-07T02:43:46.801 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-07T02:43:46.801 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_cache_buffer_items": 0, 2024-05-07T02:43:46.801 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_bytes": 0, 2024-05-07T02:43:46.801 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_extent_items": 0, 2024-05-07T02:43:46.801 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_bytes": 0, 2024-05-07T02:43:46.801 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_blob_items": 0, 2024-05-07T02:43:46.801 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-07T02:43:46.801 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_shared_blob_items": 0, 2024-05-07T02:43:46.801 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-07T02:43:46.802 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_inline_bl_items": 0, 2024-05-07T02:43:46.802 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_bytes": 0, 2024-05-07T02:43:46.802 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_fsck_items": 0, 2024-05-07T02:43:46.802 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_bytes": 0, 2024-05-07T02:43:46.802 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_txc_items": 0, 2024-05-07T02:43:46.802 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-07T02:43:46.802 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_deferred_items": 0, 2024-05-07T02:43:46.802 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_bytes": 0, 2024-05-07T02:43:46.803 INFO:teuthology.orchestra.run.smithi064.stdout: "bluestore_writing_items": 0, 2024-05-07T02:43:46.803 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_bytes": 0, 2024-05-07T02:43:46.803 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_items": 0, 2024-05-07T02:43:46.803 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_bytes": 0, 2024-05-07T02:43:46.803 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_reader_items": 0, 2024-05-07T02:43:46.803 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_bytes": 0, 2024-05-07T02:43:46.803 INFO:teuthology.orchestra.run.smithi064.stdout: "bluefs_file_writer_items": 0, 2024-05-07T02:43:46.803 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_bytes": 148714, 2024-05-07T02:43:46.803 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_anon_items": 30, 2024-05-07T02:43:46.804 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_bytes": 0, 2024-05-07T02:43:46.804 INFO:teuthology.orchestra.run.smithi064.stdout: "buffer_meta_items": 0, 2024-05-07T02:43:46.804 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_bytes": 0, 2024-05-07T02:43:46.804 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_items": 0, 2024-05-07T02:43:46.804 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_bytes": 0, 2024-05-07T02:43:46.804 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_mapbl_items": 0, 2024-05-07T02:43:46.804 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_bytes": 0, 2024-05-07T02:43:46.804 INFO:teuthology.orchestra.run.smithi064.stdout: "osd_pglog_items": 0, 2024-05-07T02:43:46.804 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_bytes": 0, 2024-05-07T02:43:46.804 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_items": 0, 2024-05-07T02:43:46.805 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_bytes": 0, 2024-05-07T02:43:46.805 INFO:teuthology.orchestra.run.smithi064.stdout: "osdmap_mapping_items": 0, 2024-05-07T02:43:46.805 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_bytes": 0, 2024-05-07T02:43:46.805 INFO:teuthology.orchestra.run.smithi064.stdout: "pgmap_items": 0, 2024-05-07T02:43:46.805 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_bytes": 928, 2024-05-07T02:43:46.805 INFO:teuthology.orchestra.run.smithi064.stdout: "mds_co_items": 2, 2024-05-07T02:43:46.805 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_bytes": 0, 2024-05-07T02:43:46.805 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_1_items": 0, 2024-05-07T02:43:46.805 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_bytes": 0, 2024-05-07T02:43:46.806 INFO:teuthology.orchestra.run.smithi064.stdout: "unittest_2_items": 0 2024-05-07T02:43:46.806 INFO:teuthology.orchestra.run.smithi064.stdout: }, 2024-05-07T02:43:46.806 INFO:teuthology.orchestra.run.smithi064.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-07T02:43:46.806 INFO:teuthology.orchestra.run.smithi064.stdout: "val": 0, 2024-05-07T02:43:46.806 INFO:teuthology.orchestra.run.smithi064.stdout: "max": 104857600, 2024-05-07T02:43:46.806 INFO:teuthology.orchestra.run.smithi064.stdout: "get_started": 0, 2024-05-07T02:43:46.806 INFO:teuthology.orchestra.run.smithi064.stdout: "get": 311, 2024-05-07T02:43:46.806 INFO:teuthology.orchestra.run.smithi064.stdout: "get_sum": 1692074, 2024-05-07T02:43:46.807 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_fail": 0, 2024-05-07T02:43:46.807 INFO:teuthology.orchestra.run.smithi064.stdout: "get_or_fail_success": 311, 2024-05-07T02:43:46.807 INFO:teuthology.orchestra.run.smithi064.stdout: "take": 0, 2024-05-07T02:43:46.807 INFO:teuthology.orchestra.run.smithi064.stdout: "take_sum": 0, 2024-05-07T02:43:46.807 INFO:teuthology.orchestra.run.smithi064.stdout: "put": 311, 2024-05-07T02:43:46.807 INFO:teuthology.orchestra.run.smithi064.stdout: "put_sum": 1692074, 2024-05-07T02:43:46.807 INFO:teuthology.orchestra.run.smithi064.stdout: "wait": { 2024-05-07T02:43:46.807 INFO:teuthology.orchestra.run.smithi064.stdout: "avgcount": 0, 2024-05-07T02:43:46.807 INFO:teuthology.orchestra.run.smithi064.stdout: "sum": 0.000000000, 2024-05-07T02:43:46.808 INFO:teuthology.orchestra.run.smithi064.stdout: "avgtime": 0.000000000 2024-05-07T02:43:46.808 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-07T02:43:46.808 INFO:teuthology.orchestra.run.smithi064.stdout: } 2024-05-07T02:43:46.808 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-07T02:43:47.434 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:43:47 smithi064 ceph-mon[34699]: pgmap v371: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:43:47.463 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-05-07T02:43:47.463 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-05-07T02:43:47.463 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-05-07T02:43:47.463 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-05-07T02:43:47.463 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-05-07T02:43:47.464 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-07T02:43:47.474 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-07T02:43:47.475 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-07T02:43:47.475 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-07T02:43:47.537 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-05-07T02:43:47.537 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-07T02:43:47.595 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph osd blocklist ls 2024-05-07T02:43:47.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:43:47 smithi019 ceph-mon[29781]: pgmap v371: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:43:47.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:43:47 smithi027 ceph-mon[34398]: pgmap v371: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:43:48.080 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:43:49.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:43:49 smithi064 ceph-mon[34699]: pgmap v372: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:43:49.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:43:49 smithi019 ceph-mon[29781]: pgmap v372: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:43:49.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:43:49 smithi027 ceph-mon[34398]: pgmap v372: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:43:50.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.682+0000 7f42d8200700 1 -- 172.21.15.19:0/777821983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42d010e910 msgr2=0x7f42d0110df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:50.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.682+0000 7f42d8200700 1 --2- 172.21.15.19:0/777821983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42d010e910 0x7f42d0110df0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f42c800b0a0 tx=0x7f42c800b3b0 comp rx=0 tx=0).stop 2024-05-07T02:43:50.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.682+0000 7f42d8200700 1 -- 172.21.15.19:0/777821983 shutdown_connections 2024-05-07T02:43:50.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.682+0000 7f42d8200700 1 --2- 172.21.15.19:0/777821983 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42d010e910 0x7f42d0110df0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:50.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.682+0000 7f42d8200700 1 --2- 172.21.15.19:0/777821983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42d01054e0 0x7f42d01058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:50.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.682+0000 7f42d8200700 1 --2- 172.21.15.19:0/777821983 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42d0105e30 0x7f42d010e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:50.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.682+0000 7f42d8200700 1 -- 172.21.15.19:0/777821983 >> 172.21.15.19:0/777821983 conn(0x7f42d00fef30 msgr2=0x7f42d0101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:43:50.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.682+0000 7f42d8200700 1 -- 172.21.15.19:0/777821983 shutdown_connections 2024-05-07T02:43:50.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.682+0000 7f42d8200700 1 -- 172.21.15.19:0/777821983 wait complete. 2024-05-07T02:43:50.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.683+0000 7f42d8200700 1 Processor -- start 2024-05-07T02:43:50.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.683+0000 7f42d8200700 1 -- start start 2024-05-07T02:43:50.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.684+0000 7f42d8200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42d01054e0 0x7f42d019ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:50.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.684+0000 7f42d8200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42d0105e30 0x7f42d019b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:50.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.684+0000 7f42d8200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42d010e910 0x7f42d019f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:50.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.684+0000 7f42d8200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f42d01144c0 con 0x7f42d010e910 2024-05-07T02:43:50.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.684+0000 7f42d8200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f42d0114340 con 0x7f42d0105e30 2024-05-07T02:43:50.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.684+0000 7f42d8200700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f42d0114640 con 0x7f42d01054e0 2024-05-07T02:43:50.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.684+0000 7f42d4e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42d0105e30 0x7f42d019b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:50.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.684+0000 7f42d4e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42d0105e30 0x7f42d019b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:37434/0 (socket says 172.21.15.19:37434) 2024-05-07T02:43:50.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.684+0000 7f42d4e00700 1 -- 172.21.15.19:0/3262116652 learned_addr learned my addr 172.21.15.19:0/3262116652 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:43:50.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.684+0000 7f42d5800700 1 --2- 172.21.15.19:0/3262116652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42d01054e0 0x7f42d019ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:50.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.684+0000 7f42d6200700 1 --2- 172.21.15.19:0/3262116652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42d010e910 0x7f42d019f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:50.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.685+0000 7f42d5800700 1 -- 172.21.15.19:0/3262116652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42d0105e30 msgr2=0x7f42d019b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:50.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.685+0000 7f42d5800700 1 --2- 172.21.15.19:0/3262116652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42d0105e30 0x7f42d019b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:50.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.685+0000 7f42d5800700 1 -- 172.21.15.19:0/3262116652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42d010e910 msgr2=0x7f42d019f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:50.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.685+0000 7f42d5800700 1 --2- 172.21.15.19:0/3262116652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42d010e910 0x7f42d019f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:50.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.685+0000 7f42d5800700 1 -- 172.21.15.19:0/3262116652 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d019fc60 con 0x7f42d01054e0 2024-05-07T02:43:50.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.685+0000 7f42d5800700 1 --2- 172.21.15.19:0/3262116652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42d01054e0 0x7f42d019ad90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f42c400e9f0 tx=0x7f42c400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:43:50.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.685+0000 7f42ce000700 1 -- 172.21.15.19:0/3262116652 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c400c9c0 con 0x7f42d01054e0 2024-05-07T02:43:50.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.686+0000 7f42ce000700 1 -- 172.21.15.19:0/3262116652 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42c400cb60 con 0x7f42d01054e0 2024-05-07T02:43:50.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.686+0000 7f42d8200700 1 -- 172.21.15.19:0/3262116652 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42d019ff50 con 0x7f42d01054e0 2024-05-07T02:43:50.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.686+0000 7f42ce000700 1 -- 172.21.15.19:0/3262116652 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c4018680 con 0x7f42d01054e0 2024-05-07T02:43:50.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.686+0000 7f42d8200700 1 -- 172.21.15.19:0/3262116652 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42d01a7870 con 0x7f42d01054e0 2024-05-07T02:43:50.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.687+0000 7f42d8200700 1 -- 172.21.15.19:0/3262116652 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42d00512c0 con 0x7f42d01054e0 2024-05-07T02:43:50.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.690+0000 7f42ce000700 1 -- 172.21.15.19:0/3262116652 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f42c404ab30 con 0x7f42d01054e0 2024-05-07T02:43:50.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.691+0000 7f42ce000700 1 --2- 172.21.15.19:0/3262116652 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f42b80b2a70 0x7f42b80b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:50.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.691+0000 7f42ce000700 1 -- 172.21.15.19:0/3262116652 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9525+0+0 (secure 0 0 0) 0x7f42c40e2c30 con 0x7f42d01054e0 2024-05-07T02:43:50.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.691+0000 7f42d4e00700 1 --2- 172.21.15.19:0/3262116652 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f42b80b2a70 0x7f42b80b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:50.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.692+0000 7f42d4e00700 1 --2- 172.21.15.19:0/3262116652 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f42b80b2a70 0x7f42b80b4f80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f42bc009680 tx=0x7f42bc009260 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:43:50.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.693+0000 7f42ce000700 1 -- 172.21.15.19:0/3262116652 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f42c404ae20 con 0x7f42d01054e0 2024-05-07T02:43:50.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.951+0000 7f42d8200700 1 -- 172.21.15.19:0/3262116652 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f42d002e5f0 con 0x7f42d01054e0 2024-05-07T02:43:50.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.952+0000 7f42ce000700 1 -- 172.21.15.19:0/3262116652 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v100) ==== 81+0+942 (secure 0 0 0) 0x7f42c40aa220 con 0x7f42d01054e0 2024-05-07T02:43:50.953 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/1821831886 2024-05-08T02:31:49.815663+0000 2024-05-07T02:43:50.954 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/2059564551 2024-05-08T02:31:49.815663+0000 2024-05-07T02:43:50.954 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/2059564551 2024-05-08T02:31:49.815663+0000 2024-05-07T02:43:50.954 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/1956516258 2024-05-08T02:17:31.252868+0000 2024-05-07T02:43:50.954 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/1816772181 2024-05-08T02:17:31.252868+0000 2024-05-07T02:43:50.954 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/686554066 2024-05-08T02:31:49.815663+0000 2024-05-07T02:43:50.954 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/2602127700 2024-05-08T02:31:49.815663+0000 2024-05-07T02:43:50.954 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/174304059 2024-05-08T02:17:31.252868+0000 2024-05-07T02:43:50.954 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/3466480009 2024-05-08T02:31:49.815663+0000 2024-05-07T02:43:50.955 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/1514051750 2024-05-08T02:16:54.732592+0000 2024-05-07T02:43:50.955 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/174304059 2024-05-08T02:17:31.252868+0000 2024-05-07T02:43:50.955 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/3930045871 2024-05-08T02:16:54.732592+0000 2024-05-07T02:43:50.955 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/3614957309 2024-05-08T02:16:54.732592+0000 2024-05-07T02:43:50.955 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/1328365399 2024-05-08T02:17:31.252868+0000 2024-05-07T02:43:50.955 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/139593344 2024-05-08T02:16:54.732592+0000 2024-05-07T02:43:50.955 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/3614957309 2024-05-08T02:16:54.732592+0000 2024-05-07T02:43:50.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.955+0000 7f42d8200700 1 -- 172.21.15.19:0/3262116652 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f42b80b2a70 msgr2=0x7f42b80b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:50.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.955+0000 7f42d8200700 1 --2- 172.21.15.19:0/3262116652 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f42b80b2a70 0x7f42b80b4f80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f42bc009680 tx=0x7f42bc009260 comp rx=0 tx=0).stop 2024-05-07T02:43:50.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.955+0000 7f42d8200700 1 -- 172.21.15.19:0/3262116652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42d01054e0 msgr2=0x7f42d019ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:50.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.955+0000 7f42d8200700 1 --2- 172.21.15.19:0/3262116652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42d01054e0 0x7f42d019ad90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f42c400e9f0 tx=0x7f42c400ed00 comp rx=0 tx=0).stop 2024-05-07T02:43:50.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.955+0000 7f42d8200700 1 -- 172.21.15.19:0/3262116652 shutdown_connections 2024-05-07T02:43:50.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.955+0000 7f42d8200700 1 --2- 172.21.15.19:0/3262116652 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f42b80b2a70 0x7f42b80b4f80 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:50.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.955+0000 7f42d8200700 1 --2- 172.21.15.19:0/3262116652 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42d010e910 0x7f42d019f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:50.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.955+0000 7f42d8200700 1 --2- 172.21.15.19:0/3262116652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f42d01054e0 0x7f42d019ad90 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:50.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.955+0000 7f42d8200700 1 --2- 172.21.15.19:0/3262116652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42d0105e30 0x7f42d019b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:50.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.955+0000 7f42d8200700 1 -- 172.21.15.19:0/3262116652 >> 172.21.15.19:0/3262116652 conn(0x7f42d00fef30 msgr2=0x7f42d00ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:43:50.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.955+0000 7f42d8200700 1 -- 172.21.15.19:0/3262116652 shutdown_connections 2024-05-07T02:43:50.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:50.955+0000 7f42d8200700 1 -- 172.21.15.19:0/3262116652 wait complete. 2024-05-07T02:43:50.958 INFO:teuthology.orchestra.run.smithi019.stderr:listed 16 entries 2024-05-07T02:43:51.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:43:51 smithi064 ceph-mon[34699]: pgmap v373: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:43:51.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:43:51 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/3262116652' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-07T02:43:51.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:43:51 smithi019 ceph-mon[29781]: pgmap v373: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:43:51.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:43:51 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/3262116652' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-07T02:43:51.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:43:51 smithi027 ceph-mon[34398]: pgmap v373: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:43:51.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:43:51 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/3262116652' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-07T02:43:51.799 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-07T02:43:51.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-07T02:43:51.800 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-07T02:43:52.027 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-05-07T02:43:52.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-07T02:43:52.027 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-07T02:43:52.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-07T02:43:52.116 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-05-07T02:43:52.188 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-07T02:43:52.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-07T02:43:52.189 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-07T02:43:52.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-07T02:43:52.271 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link delete ceph-brx 2024-05-07T02:43:52.366 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-07T02:43:52.377 INFO:tasks.cephadm:Teardown begin 2024-05-07T02:43:52.377 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-07T02:43:52.377 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-07T02:43:52.800 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:43:53.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:43:53 smithi064 ceph-mon[34699]: pgmap v374: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:43:53.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:43:53 smithi019 ceph-mon[29781]: pgmap v374: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:43:53.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:43:53 smithi027 ceph-mon[34398]: pgmap v374: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-07T02:43:55.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.547+0000 7f5af7200700 1 -- 172.21.15.19:0/3682971032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5af8103880 msgr2=0x7f5af81105a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:55.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.547+0000 7f5af7200700 1 --2- 172.21.15.19:0/3682971032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5af8103880 0x7f5af81105a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f5ae40098f0 tx=0x7f5ae4009c00 comp rx=0 tx=0).stop 2024-05-07T02:43:55.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.547+0000 7f5af7200700 1 -- 172.21.15.19:0/3682971032 shutdown_connections 2024-05-07T02:43:55.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.547+0000 7f5af7200700 1 --2- 172.21.15.19:0/3682971032 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5af8110ae0 0x7f5af8112f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:55.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.547+0000 7f5af7200700 1 --2- 172.21.15.19:0/3682971032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5af8102f30 0x7f5af8103340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:55.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.547+0000 7f5af7200700 1 --2- 172.21.15.19:0/3682971032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5af8103880 0x7f5af81105a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:55.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.547+0000 7f5af7200700 1 -- 172.21.15.19:0/3682971032 >> 172.21.15.19:0/3682971032 conn(0x7f5af8078ac0 msgr2=0x7f5af8078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:43:55.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.547+0000 7f5af7200700 1 -- 172.21.15.19:0/3682971032 shutdown_connections 2024-05-07T02:43:55.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.547+0000 7f5af7200700 1 -- 172.21.15.19:0/3682971032 wait complete. 2024-05-07T02:43:55.549 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:43:55 smithi019 ceph-mon[29781]: pgmap v375: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:43:55.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.549+0000 7f5af7200700 1 Processor -- start 2024-05-07T02:43:55.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.549+0000 7f5af7200700 1 -- start start 2024-05-07T02:43:55.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.549+0000 7f5af7200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5af8102f30 0x7f5af819cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:55.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.549+0000 7f5af7200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5af8103880 0x7f5af819d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:55.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.549+0000 7f5af7200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5af8110ae0 0x7f5af81a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:55.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.549+0000 7f5af7200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5af81164e0 con 0x7f5af8102f30 2024-05-07T02:43:55.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.549+0000 7f5af7200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f5af8116360 con 0x7f5af8103880 2024-05-07T02:43:55.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.550+0000 7f5af7200700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5af8116660 con 0x7f5af8110ae0 2024-05-07T02:43:55.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.550+0000 7f5af5e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5af8102f30 0x7f5af819cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:55.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.550+0000 7f5af5400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5af8103880 0x7f5af819d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:55.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.550+0000 7f5af5e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5af8102f30 0x7f5af819cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37302/0 (socket says 172.21.15.19:37302) 2024-05-07T02:43:55.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.550+0000 7f5af5e00700 1 -- 172.21.15.19:0/3679242533 learned_addr learned my addr 172.21.15.19:0/3679242533 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:43:55.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.550+0000 7f5af6800700 1 --2- 172.21.15.19:0/3679242533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5af8110ae0 0x7f5af81a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:55.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.550+0000 7f5af5400700 1 -- 172.21.15.19:0/3679242533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5af8110ae0 msgr2=0x7f5af81a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:55.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.550+0000 7f5af5400700 1 --2- 172.21.15.19:0/3679242533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5af8110ae0 0x7f5af81a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:55.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.550+0000 7f5af5400700 1 -- 172.21.15.19:0/3679242533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5af8102f30 msgr2=0x7f5af819cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:55.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.550+0000 7f5af5400700 1 --2- 172.21.15.19:0/3679242533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5af8102f30 0x7f5af819cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:55.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.550+0000 7f5af5400700 1 -- 172.21.15.19:0/3679242533 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af81a1e20 con 0x7f5af8103880 2024-05-07T02:43:55.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.551+0000 7f5af5400700 1 --2- 172.21.15.19:0/3679242533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5af8103880 0x7f5af819d4f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4009230 tx=0x7f5ae400f970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:43:55.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.551+0000 7f5aeea00700 1 -- 172.21.15.19:0/3679242533 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4005040 con 0x7f5af8103880 2024-05-07T02:43:55.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.551+0000 7f5af7200700 1 -- 172.21.15.19:0/3679242533 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5af81a20b0 con 0x7f5af8103880 2024-05-07T02:43:55.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.551+0000 7f5aeea00700 1 -- 172.21.15.19:0/3679242533 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ae4005690 con 0x7f5af8103880 2024-05-07T02:43:55.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.551+0000 7f5aeea00700 1 -- 172.21.15.19:0/3679242533 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4003990 con 0x7f5af8103880 2024-05-07T02:43:55.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.551+0000 7f5af7200700 1 -- 172.21.15.19:0/3679242533 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5af81a25b0 con 0x7f5af8103880 2024-05-07T02:43:55.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.553+0000 7f5af7200700 1 -- 172.21.15.19:0/3679242533 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5af80512c0 con 0x7f5af8103880 2024-05-07T02:43:55.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.555+0000 7f5aeea00700 1 -- 172.21.15.19:0/3679242533 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5ae400b7f0 con 0x7f5af8103880 2024-05-07T02:43:55.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.556+0000 7f5aeea00700 1 --2- 172.21.15.19:0/3679242533 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f5adc0b29f0 0x7f5adc0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:55.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.557+0000 7f5aeea00700 1 -- 172.21.15.19:0/3679242533 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9525+0+0 (secure 0 0 0) 0x7f5ae40e39c0 con 0x7f5af8103880 2024-05-07T02:43:55.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.557+0000 7f5af5e00700 1 --2- 172.21.15.19:0/3679242533 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f5adc0b29f0 0x7f5adc0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:55.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.558+0000 7f5af5e00700 1 --2- 172.21.15.19:0/3679242533 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f5adc0b29f0 0x7f5adc0b4f00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8005d10 tx=0x7f5ae8009310 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:43:55.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.559+0000 7f5aeea00700 1 -- 172.21.15.19:0/3679242533 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5ae405fdf0 con 0x7f5af8103880 2024-05-07T02:43:55.609 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:43:55 smithi064 ceph-mon[34699]: pgmap v375: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:43:55.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:43:55 smithi027 ceph-mon[34398]: pgmap v375: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-07T02:43:55.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.845+0000 7f5af7200700 1 -- 172.21.15.19:0/3679242533 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5af802e5f0 con 0x7f5af8103880 2024-05-07T02:43:55.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.846+0000 7f5aeea00700 1 -- 172.21.15.19:0/3679242533 <== mon.2 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+11924 (secure 0 0 0) 0x7f5af81a25b0 con 0x7f5af8103880 2024-05-07T02:43:55.847 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:43:55.847 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1763670446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1763670446},{"type":"v1","addr":"172.21.15.19:6839","nonce":1763670446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/477529985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":477529985},{"type":"v1","addr":"172.21.15.64:6833","nonce":477529985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":50,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-07T02:26:14.960465+0000","modified":"2024-05-07T02:40:40.531575+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24539,"mds_2":24520,"mds_3":24515,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6833/2171873162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2171873162},{"type":"v1","addr":"172.21.15.27:6833","nonce":2171873162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/2379917839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2379917839},{"type":"v1","addr":"172.21.15.19:6835","nonce":2379917839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6837/2067515538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2067515538},{"type":"v1","addr":"172.21.15.19:6837","nonce":2067515538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6835/1000540600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1000540600},{"type":"v1","addr":"172.21.15.27:6835","nonce":1000540600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/1997318872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1997318872},{"type":"v1","addr":"172.21.15.64:6835","nonce":1997318872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":34,"addr":"172.21.15.64:6837/2585907777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2585907777},{"type":"v1","addr":"172.21.15.64:6837","nonce":2585907777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":33,"addr":"172.21.15.27:6837/2583138143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2583138143},{"type":"v1","addr":"172.21.15.27:6837","nonce":2583138143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":32,"addr":"172.21.15.64:6839/1583120004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":1583120004},{"type":"v1","addr":"172.21.15.64:6839","nonce":1583120004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":29,"addr":"172.21.15.19:6841/3905645246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3905645246},{"type":"v1","addr":"172.21.15.19:6841","nonce":3905645246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6839/3963268634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3963268634},{"type":"v1","addr":"172.21.15.27:6839","nonce":3963268634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24551,24539,24520,24505,24515]},"id":1}]} 2024-05-07T02:43:55.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.849+0000 7f5af7200700 1 -- 172.21.15.19:0/3679242533 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f5adc0b29f0 msgr2=0x7f5adc0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:55.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.849+0000 7f5af7200700 1 --2- 172.21.15.19:0/3679242533 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f5adc0b29f0 0x7f5adc0b4f00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8005d10 tx=0x7f5ae8009310 comp rx=0 tx=0).stop 2024-05-07T02:43:55.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.849+0000 7f5af7200700 1 -- 172.21.15.19:0/3679242533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5af8103880 msgr2=0x7f5af819d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:55.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.849+0000 7f5af7200700 1 --2- 172.21.15.19:0/3679242533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5af8103880 0x7f5af819d4f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4009230 tx=0x7f5ae400f970 comp rx=0 tx=0).stop 2024-05-07T02:43:55.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.849+0000 7f5af7200700 1 -- 172.21.15.19:0/3679242533 shutdown_connections 2024-05-07T02:43:55.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.849+0000 7f5af7200700 1 --2- 172.21.15.19:0/3679242533 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5af8110ae0 0x7f5af81a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:55.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.849+0000 7f5af7200700 1 --2- 172.21.15.19:0/3679242533 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f5adc0b29f0 0x7f5adc0b4f00 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:55.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.849+0000 7f5af7200700 1 --2- 172.21.15.19:0/3679242533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5af8102f30 0x7f5af819cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:55.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.849+0000 7f5af7200700 1 --2- 172.21.15.19:0/3679242533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5af8103880 0x7f5af819d4f0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:55.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.849+0000 7f5af7200700 1 -- 172.21.15.19:0/3679242533 >> 172.21.15.19:0/3679242533 conn(0x7f5af8078ac0 msgr2=0x7f5af81120c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:43:55.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.850+0000 7f5af7200700 1 -- 172.21.15.19:0/3679242533 shutdown_connections 2024-05-07T02:43:55.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:55.850+0000 7f5af7200700 1 -- 172.21.15.19:0/3679242533 wait complete. 2024-05-07T02:43:55.853 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 50 2024-05-07T02:43:56.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:43:56 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/3679242533' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:43:56.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:43:56 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/3679242533' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:43:56.725 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-07T02:43:56.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:43:56 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/3679242533' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:43:57.180 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:43:57.582 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:43:57 smithi019 ceph-mon[29781]: pgmap v376: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.2 KiB/s wr, 15 op/s 2024-05-07T02:43:57.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:43:57 smithi027 ceph-mon[34398]: pgmap v376: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.2 KiB/s wr, 15 op/s 2024-05-07T02:43:57.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:43:57 smithi064 ceph-mon[34699]: pgmap v376: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.2 KiB/s wr, 15 op/s 2024-05-07T02:43:59.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.627+0000 7f2ceb600700 1 -- 172.21.15.19:0/3808122978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ce41045f0 msgr2=0x7f2ce4104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:59.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.627+0000 7f2ceb600700 1 --2- 172.21.15.19:0/3808122978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ce41045f0 0x7f2ce4104a00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2cd80099c0 tx=0x7f2cd8009cd0 comp rx=0 tx=0).stop 2024-05-07T02:43:59.629 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:43:59 smithi019 ceph-mon[29781]: pgmap v377: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-07T02:43:59.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.627+0000 7f2ceb600700 1 -- 172.21.15.19:0/3808122978 shutdown_connections 2024-05-07T02:43:59.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.627+0000 7f2ceb600700 1 --2- 172.21.15.19:0/3808122978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ce4109870 0x7f2ce41111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:59.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.627+0000 7f2ceb600700 1 --2- 172.21.15.19:0/3808122978 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ce41045f0 0x7f2ce4104a00 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:59.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.627+0000 7f2ceb600700 1 --2- 172.21.15.19:0/3808122978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ce4108f20 0x7f2ce4109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:59.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.627+0000 7f2ceb600700 1 -- 172.21.15.19:0/3808122978 >> 172.21.15.19:0/3808122978 conn(0x7f2ce4100040 msgr2=0x7f2ce4102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:43:59.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.627+0000 7f2ceb600700 1 -- 172.21.15.19:0/3808122978 shutdown_connections 2024-05-07T02:43:59.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.627+0000 7f2ceb600700 1 -- 172.21.15.19:0/3808122978 wait complete. 2024-05-07T02:43:59.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.629+0000 7f2ceb600700 1 Processor -- start 2024-05-07T02:43:59.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.629+0000 7f2ceb600700 1 -- start start 2024-05-07T02:43:59.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.629+0000 7f2ceb600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ce41045f0 0x7f2ce41a21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:59.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.629+0000 7f2ceb600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ce4108f20 0x7f2ce41a2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:59.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.629+0000 7f2ceb600700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ce4109870 0x7f2ce41ad250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:59.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.629+0000 7f2ceb600700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce41144f0 con 0x7f2ce41045f0 2024-05-07T02:43:59.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.629+0000 7f2ceb600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce4114370 con 0x7f2ce4108f20 2024-05-07T02:43:59.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.629+0000 7f2ceb600700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce4114670 con 0x7f2ce4109870 2024-05-07T02:43:59.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.630+0000 7f2ce8c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ce41045f0 0x7f2ce41a21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:59.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.630+0000 7f2ce3e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ce4108f20 0x7f2ce41a2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:59.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.630+0000 7f2ce8c00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ce41045f0 0x7f2ce41a21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36106/0 (socket says 172.21.15.19:36106) 2024-05-07T02:43:59.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.630+0000 7f2ce8c00700 1 -- 172.21.15.19:0/941329895 learned_addr learned my addr 172.21.15.19:0/941329895 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:43:59.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.630+0000 7f2ce9600700 1 --2- 172.21.15.19:0/941329895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ce4109870 0x7f2ce41ad250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:59.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.630+0000 7f2ce8c00700 1 -- 172.21.15.19:0/941329895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ce4109870 msgr2=0x7f2ce41ad250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:59.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.630+0000 7f2ce8c00700 1 --2- 172.21.15.19:0/941329895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ce4109870 0x7f2ce41ad250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:59.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.630+0000 7f2ce8c00700 1 -- 172.21.15.19:0/941329895 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ce4108f20 msgr2=0x7f2ce41a2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:59.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.630+0000 7f2ce8c00700 1 --2- 172.21.15.19:0/941329895 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ce4108f20 0x7f2ce41a2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:59.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.630+0000 7f2ce8c00700 1 -- 172.21.15.19:0/941329895 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce41ad790 con 0x7f2ce41045f0 2024-05-07T02:43:59.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.630+0000 7f2ce3e00700 1 --2- 172.21.15.19:0/941329895 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ce4108f20 0x7f2ce41a2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:43:59.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.631+0000 7f2ce8c00700 1 --2- 172.21.15.19:0/941329895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ce41045f0 0x7f2ce41a21d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8005910 tx=0x7f2cd8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:43:59.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.631+0000 7f2ce1600700 1 -- 172.21.15.19:0/941329895 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8026070 con 0x7f2ce41045f0 2024-05-07T02:43:59.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.631+0000 7f2ceb600700 1 -- 172.21.15.19:0/941329895 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ce41ad960 con 0x7f2ce41045f0 2024-05-07T02:43:59.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.631+0000 7f2ce1600700 1 -- 172.21.15.19:0/941329895 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cd8003c70 con 0x7f2ce41045f0 2024-05-07T02:43:59.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.631+0000 7f2ce1600700 1 -- 172.21.15.19:0/941329895 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd800fae0 con 0x7f2ce41045f0 2024-05-07T02:43:59.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.631+0000 7f2ceb600700 1 -- 172.21.15.19:0/941329895 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ce41add50 con 0x7f2ce41045f0 2024-05-07T02:43:59.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.632+0000 7f2ceb600700 1 -- 172.21.15.19:0/941329895 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ce4105aa0 con 0x7f2ce41045f0 2024-05-07T02:43:59.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.635+0000 7f2ce1600700 1 -- 172.21.15.19:0/941329895 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2cd8029070 con 0x7f2ce41045f0 2024-05-07T02:43:59.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.637+0000 7f2ce1600700 1 --2- 172.21.15.19:0/941329895 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f2ccc0b29f0 0x7f2ccc0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:43:59.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.637+0000 7f2ce1600700 1 -- 172.21.15.19:0/941329895 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9525+0+0 (secure 0 0 0) 0x7f2cd80e3a60 con 0x7f2ce41045f0 2024-05-07T02:43:59.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.637+0000 7f2ce3e00700 1 --2- 172.21.15.19:0/941329895 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f2ccc0b29f0 0x7f2ccc0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:43:59.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.638+0000 7f2ce3e00700 1 --2- 172.21.15.19:0/941329895 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f2ccc0b29f0 0x7f2ccc0b4f00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4103550 tx=0x7f2cd0009260 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:43:59.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.639+0000 7f2ce1600700 1 -- 172.21.15.19:0/941329895 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2cd8013070 con 0x7f2ce41045f0 2024-05-07T02:43:59.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:43:59 smithi027 ceph-mon[34398]: pgmap v377: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-07T02:43:59.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:43:59 smithi064 ceph-mon[34699]: pgmap v377: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-07T02:43:59.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.932+0000 7f2ceb600700 1 -- 172.21.15.19:0/941329895 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ce402e5f0 con 0x7f2ce41045f0 2024-05-07T02:43:59.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.933+0000 7f2ce1600700 1 -- 172.21.15.19:0/941329895 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+11924 (secure 0 0 0) 0x7f2cd80ab080 con 0x7f2ce41045f0 2024-05-07T02:43:59.935 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:43:59.935 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14406,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/1763670446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":1763670446},{"type":"v1","addr":"172.21.15.19:6839","nonce":1763670446}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/477529985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":477529985},{"type":"v1","addr":"172.21.15.64:6833","nonce":477529985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":50,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-07T02:26:14.960465+0000","modified":"2024-05-07T02:40:40.531575+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24539,"mds_2":24520,"mds_3":24515,"mds_4":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24458":{"gid":24458,"name":"k","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6833/2171873162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2171873162},{"type":"v1","addr":"172.21.15.27:6833","nonce":2171873162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"j","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6835/2379917839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":2379917839},{"type":"v1","addr":"172.21.15.19:6835","nonce":2379917839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6837/2067515538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2067515538},{"type":"v1","addr":"172.21.15.19:6837","nonce":2067515538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.27:6835/1000540600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1000540600},{"type":"v1","addr":"172.21.15.27:6835","nonce":1000540600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24499":{"gid":24499,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.64:6835/1997318872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":1997318872},{"type":"v1","addr":"172.21.15.64:6835","nonce":1997318872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"c","rank":4,"incarnation":26,"state":"up:active","state_seq":34,"addr":"172.21.15.64:6837/2585907777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2585907777},{"type":"v1","addr":"172.21.15.64:6837","nonce":2585907777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":33,"addr":"172.21.15.27:6837/2583138143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6836","nonce":2583138143},{"type":"v1","addr":"172.21.15.27:6837","nonce":2583138143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":32,"addr":"172.21.15.64:6839/1583120004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":1583120004},{"type":"v1","addr":"172.21.15.64:6839","nonce":1583120004}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"a","rank":1,"incarnation":20,"state":"up:active","state_seq":29,"addr":"172.21.15.19:6841/3905645246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":3905645246},{"type":"v1","addr":"172.21.15.19:6841","nonce":3905645246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.27:6839/3963268634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6838","nonce":3963268634},{"type":"v1","addr":"172.21.15.27:6839","nonce":3963268634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24551,24539,24520,24505,24515]},"id":1}]} 2024-05-07T02:43:59.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.937+0000 7f2ceb600700 1 -- 172.21.15.19:0/941329895 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f2ccc0b29f0 msgr2=0x7f2ccc0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:59.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.937+0000 7f2ceb600700 1 --2- 172.21.15.19:0/941329895 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f2ccc0b29f0 0x7f2ccc0b4f00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4103550 tx=0x7f2cd0009260 comp rx=0 tx=0).stop 2024-05-07T02:43:59.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.937+0000 7f2ceb600700 1 -- 172.21.15.19:0/941329895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ce41045f0 msgr2=0x7f2ce41a21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:43:59.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.937+0000 7f2ceb600700 1 --2- 172.21.15.19:0/941329895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ce41045f0 0x7f2ce41a21d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8005910 tx=0x7f2cd8004270 comp rx=0 tx=0).stop 2024-05-07T02:43:59.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.937+0000 7f2ceb600700 1 -- 172.21.15.19:0/941329895 shutdown_connections 2024-05-07T02:43:59.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.937+0000 7f2ceb600700 1 --2- 172.21.15.19:0/941329895 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2ce4109870 0x7f2ce41ad250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:59.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.937+0000 7f2ceb600700 1 --2- 172.21.15.19:0/941329895 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f2ccc0b29f0 0x7f2ccc0b4f00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:59.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.937+0000 7f2ceb600700 1 --2- 172.21.15.19:0/941329895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ce41045f0 0x7f2ce41a21d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:59.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.937+0000 7f2ceb600700 1 --2- 172.21.15.19:0/941329895 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ce4108f20 0x7f2ce41a2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:43:59.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.937+0000 7f2ceb600700 1 -- 172.21.15.19:0/941329895 >> 172.21.15.19:0/941329895 conn(0x7f2ce4100040 msgr2=0x7f2ce4107720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:43:59.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.937+0000 7f2ceb600700 1 -- 172.21.15.19:0/941329895 shutdown_connections 2024-05-07T02:43:59.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:43:59.938+0000 7f2ceb600700 1 -- 172.21.15.19:0/941329895 wait complete. 2024-05-07T02:43:59.939 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 50 2024-05-07T02:44:00.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:00 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/941329895' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:44:00.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:00 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/941329895' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:44:00.827 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-07T02:44:00.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:00 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/941329895' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-07T02:44:01.305 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:44:01.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:01 smithi019 ceph-mon[29781]: pgmap v378: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-07T02:44:01.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:01 smithi027 ceph-mon[34398]: pgmap v378: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-07T02:44:01.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:01 smithi064 ceph-mon[34699]: pgmap v378: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-07T02:44:03.646 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:03 smithi019 ceph-mon[29781]: pgmap v379: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 46 KiB/s wr, 32 op/s 2024-05-07T02:44:03.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.699+0000 7fca2e400700 1 -- 172.21.15.19:0/366348808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca28106ed0 msgr2=0x7fca28113360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:44:03.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.699+0000 7fca2e400700 1 --2- 172.21.15.19:0/366348808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca28106ed0 0x7fca28113360 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fca2000b0a0 tx=0x7fca2000b3b0 comp rx=0 tx=0).stop 2024-05-07T02:44:03.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.700+0000 7fca2e400700 1 -- 172.21.15.19:0/366348808 shutdown_connections 2024-05-07T02:44:03.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.700+0000 7fca2e400700 1 --2- 172.21.15.19:0/366348808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca28106ed0 0x7fca28113360 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:03.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.700+0000 7fca2e400700 1 --2- 172.21.15.19:0/366348808 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca2810c5c0 0x7fca2810c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:03.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.700+0000 7fca2e400700 1 --2- 172.21.15.19:0/366348808 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca28106580 0x7fca28106990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:03.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.700+0000 7fca2e400700 1 -- 172.21.15.19:0/366348808 >> 172.21.15.19:0/366348808 conn(0x7fca28102080 msgr2=0x7fca281044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:44:03.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.700+0000 7fca2e400700 1 -- 172.21.15.19:0/366348808 shutdown_connections 2024-05-07T02:44:03.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.700+0000 7fca2e400700 1 -- 172.21.15.19:0/366348808 wait complete. 2024-05-07T02:44:03.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.701+0000 7fca2e400700 1 Processor -- start 2024-05-07T02:44:03.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.701+0000 7fca2e400700 1 -- start start 2024-05-07T02:44:03.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.701+0000 7fca2e400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca28106580 0x7fca2819cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:44:03.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.702+0000 7fca2e400700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca28106ed0 0x7fca2819d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:44:03.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.702+0000 7fca2e400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca2810c5c0 0x7fca281a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:44:03.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.702+0000 7fca2e400700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fca281166f0 con 0x7fca2810c5c0 2024-05-07T02:44:03.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.702+0000 7fca2e400700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7fca28116570 con 0x7fca28106580 2024-05-07T02:44:03.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.702+0000 7fca2e400700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fca28116870 con 0x7fca28106ed0 2024-05-07T02:44:03.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.702+0000 7fca1fe00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca28106ed0 0x7fca2819d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:44:03.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.702+0000 7fca27e00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca2810c5c0 0x7fca281a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:44:03.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.702+0000 7fca1fe00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca28106ed0 0x7fca2819d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:53356/0 (socket says 172.21.15.19:53356) 2024-05-07T02:44:03.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.702+0000 7fca1fe00700 1 -- 172.21.15.19:0/653175343 learned_addr learned my addr 172.21.15.19:0/653175343 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:44:03.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.702+0000 7fca27400700 1 --2- 172.21.15.19:0/653175343 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca28106580 0x7fca2819cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:44:03.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.703+0000 7fca27e00700 1 -- 172.21.15.19:0/653175343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca28106ed0 msgr2=0x7fca2819d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:44:03.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.703+0000 7fca27e00700 1 --2- 172.21.15.19:0/653175343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca28106ed0 0x7fca2819d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:03.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.703+0000 7fca27e00700 1 -- 172.21.15.19:0/653175343 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca28106580 msgr2=0x7fca2819cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:44:03.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.703+0000 7fca27e00700 1 --2- 172.21.15.19:0/653175343 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca28106580 0x7fca2819cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:03.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.703+0000 7fca27e00700 1 -- 172.21.15.19:0/653175343 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca281a1e90 con 0x7fca2810c5c0 2024-05-07T02:44:03.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.703+0000 7fca27e00700 1 --2- 172.21.15.19:0/653175343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca2810c5c0 0x7fca281a17f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fca2000b070 tx=0x7fca20004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:44:03.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.703+0000 7fca24c00700 1 -- 172.21.15.19:0/653175343 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca20021070 con 0x7fca2810c5c0 2024-05-07T02:44:03.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.703+0000 7fca24c00700 1 -- 172.21.15.19:0/653175343 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca20007e30 con 0x7fca2810c5c0 2024-05-07T02:44:03.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.703+0000 7fca24c00700 1 -- 172.21.15.19:0/653175343 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca200045d0 con 0x7fca2810c5c0 2024-05-07T02:44:03.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.703+0000 7fca2e400700 1 -- 172.21.15.19:0/653175343 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca281a2120 con 0x7fca2810c5c0 2024-05-07T02:44:03.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.703+0000 7fca2e400700 1 -- 172.21.15.19:0/653175343 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca281a9aa0 con 0x7fca2810c5c0 2024-05-07T02:44:03.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.705+0000 7fca2e400700 1 -- 172.21.15.19:0/653175343 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca2810e980 con 0x7fca2810c5c0 2024-05-07T02:44:03.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:03 smithi027 ceph-mon[34398]: pgmap v379: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 46 KiB/s wr, 32 op/s 2024-05-07T02:44:03.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.709+0000 7fca24c00700 1 -- 172.21.15.19:0/653175343 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fca20003c00 con 0x7fca2810c5c0 2024-05-07T02:44:03.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.710+0000 7fca24c00700 1 --2- 172.21.15.19:0/653175343 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fca0c0b2a70 0x7fca0c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:44:03.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.711+0000 7fca24c00700 1 -- 172.21.15.19:0/653175343 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9525+0+0 (secure 0 0 0) 0x7fca2000f020 con 0x7fca2810c5c0 2024-05-07T02:44:03.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.711+0000 7fca27400700 1 --2- 172.21.15.19:0/653175343 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fca0c0b2a70 0x7fca0c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:44:03.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.712+0000 7fca27400700 1 --2- 172.21.15.19:0/653175343 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fca0c0b2a70 0x7fca0c0b4f80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fca281079f0 tx=0x7fca18009260 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:44:03.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.713+0000 7fca24c00700 1 -- 172.21.15.19:0/653175343 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fca2002a020 con 0x7fca2810c5c0 2024-05-07T02:44:03.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:03 smithi064 ceph-mon[34699]: pgmap v379: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 46 KiB/s wr, 32 op/s 2024-05-07T02:44:03.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.958+0000 7fca2e400700 1 -- 172.21.15.19:0/653175343 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fca2819e320 con 0x7fca2810c5c0 2024-05-07T02:44:03.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.962+0000 7fca24c00700 1 -- 172.21.15.19:0/653175343 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+25282 (secure 0 0 0) 0x7fca200ab6e0 con 0x7fca2810c5c0 2024-05-07T02:44:03.963 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-05-07T02:44:03.964 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":100,"fsid":"9ff84c16-0c17-11ef-bc97-c7b262605968","created":"2024-05-07T02:16:19.507657+0000","modified":"2024-05-07T02:43:17.374636+0000","last_up_change":"2024-05-07T02:24:55.025124+0000","last_in_change":"2024-05-07T02:24:33.842422+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-07T02:20:21.439697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-07T02:26:06.178814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-07T02:26:10.705693+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-07T02:26:23.534133+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b437cf22-ec8e-48f0-86a2-2413a0ab40dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6803","nonce":2390539683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6805","nonce":2390539683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6809","nonce":2390539683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":2390539683},{"type":"v1","addr":"172.21.15.19:6807","nonce":2390539683}]},"public_addr":"172.21.15.19:6803/2390539683","cluster_addr":"172.21.15.19:6805/2390539683","heartbeat_back_addr":"172.21.15.19:6809/2390539683","heartbeat_front_addr":"172.21.15.19:6807/2390539683","state":["exists","up"]},{"osd":1,"uuid":"dc753e16-634d-4c14-abd2-935281e1499e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6801","nonce":4292470615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6803","nonce":4292470615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6807","nonce":4292470615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":4292470615},{"type":"v1","addr":"172.21.15.27:6805","nonce":4292470615}]},"public_addr":"172.21.15.27:6801/4292470615","cluster_addr":"172.21.15.27:6803/4292470615","heartbeat_back_addr":"172.21.15.27:6807/4292470615","heartbeat_front_addr":"172.21.15.27:6805/4292470615","state":["exists","up"]},{"osd":2,"uuid":"40570071-646e-4e66-9a16-c55ab743ac86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6801","nonce":1180552939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6803","nonce":1180552939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6807","nonce":1180552939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":1180552939},{"type":"v1","addr":"172.21.15.64:6805","nonce":1180552939}]},"public_addr":"172.21.15.64:6801/1180552939","cluster_addr":"172.21.15.64:6803/1180552939","heartbeat_back_addr":"172.21.15.64:6807/1180552939","heartbeat_front_addr":"172.21.15.64:6805/1180552939","state":["exists","up"]},{"osd":3,"uuid":"ceaa4252-4dfa-427a-9b1a-76adf3b8e609","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6811","nonce":387118141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6813","nonce":387118141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6817","nonce":387118141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":387118141},{"type":"v1","addr":"172.21.15.19:6815","nonce":387118141}]},"public_addr":"172.21.15.19:6811/387118141","cluster_addr":"172.21.15.19:6813/387118141","heartbeat_back_addr":"172.21.15.19:6817/387118141","heartbeat_front_addr":"172.21.15.19:6815/387118141","state":["exists","up"]},{"osd":4,"uuid":"5a6d1ebc-62ca-444e-8f29-1980418d0a8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6809","nonce":3662186068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6811","nonce":3662186068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6815","nonce":3662186068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":3662186068},{"type":"v1","addr":"172.21.15.27:6813","nonce":3662186068}]},"public_addr":"172.21.15.27:6809/3662186068","cluster_addr":"172.21.15.27:6811/3662186068","heartbeat_back_addr":"172.21.15.27:6815/3662186068","heartbeat_front_addr":"172.21.15.27:6813/3662186068","state":["exists","up"]},{"osd":5,"uuid":"60c207e9-55e8-4bed-87c3-f6a06a2a1fd1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6809","nonce":1264591849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6811","nonce":1264591849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6815","nonce":1264591849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1264591849},{"type":"v1","addr":"172.21.15.64:6813","nonce":1264591849}]},"public_addr":"172.21.15.64:6809/1264591849","cluster_addr":"172.21.15.64:6811/1264591849","heartbeat_back_addr":"172.21.15.64:6815/1264591849","heartbeat_front_addr":"172.21.15.64:6813/1264591849","state":["exists","up"]},{"osd":6,"uuid":"f3e038e0-4026-45f1-a923-c07cbb307f64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6819","nonce":2158875543}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6821","nonce":2158875543}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6825","nonce":2158875543}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2158875543},{"type":"v1","addr":"172.21.15.19:6823","nonce":2158875543}]},"public_addr":"172.21.15.19:6819/2158875543","cluster_addr":"172.21.15.19:6821/2158875543","heartbeat_back_addr":"172.21.15.19:6825/2158875543","heartbeat_front_addr":"172.21.15.19:6823/2158875543","state":["exists","up"]},{"osd":7,"uuid":"d69e34d4-2028-43b8-8ada-ff4a9b579620","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6817","nonce":2756612091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6819","nonce":2756612091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6823","nonce":2756612091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":2756612091},{"type":"v1","addr":"172.21.15.27:6821","nonce":2756612091}]},"public_addr":"172.21.15.27:6817/2756612091","cluster_addr":"172.21.15.27:6819/2756612091","heartbeat_back_addr":"172.21.15.27:6823/2756612091","heartbeat_front_addr":"172.21.15.27:6821/2756612091","state":["exists","up"]},{"osd":8,"uuid":"6c1a4179-75b6-42ff-ac29-378801e6f8f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6817","nonce":1987024836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6819","nonce":1987024836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6823","nonce":1987024836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":1987024836},{"type":"v1","addr":"172.21.15.64:6821","nonce":1987024836}]},"public_addr":"172.21.15.64:6817/1987024836","cluster_addr":"172.21.15.64:6819/1987024836","heartbeat_back_addr":"172.21.15.64:6823/1987024836","heartbeat_front_addr":"172.21.15.64:6821/1987024836","state":["exists","up"]},{"osd":9,"uuid":"9df7ff66-d39f-4709-8324-74cf9e0e052d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6827","nonce":4124921646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6829","nonce":4124921646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6833","nonce":4124921646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":4124921646},{"type":"v1","addr":"172.21.15.19:6831","nonce":4124921646}]},"public_addr":"172.21.15.19:6827/4124921646","cluster_addr":"172.21.15.19:6829/4124921646","heartbeat_back_addr":"172.21.15.19:6833/4124921646","heartbeat_front_addr":"172.21.15.19:6831/4124921646","state":["exists","up"]},{"osd":10,"uuid":"71eaa9a7-667d-4cb7-935c-42177d20a279","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6825","nonce":1428931817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6827","nonce":1428931817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6831","nonce":1428931817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":1428931817},{"type":"v1","addr":"172.21.15.27:6829","nonce":1428931817}]},"public_addr":"172.21.15.27:6825/1428931817","cluster_addr":"172.21.15.27:6827/1428931817","heartbeat_back_addr":"172.21.15.27:6831/1428931817","heartbeat_front_addr":"172.21.15.27:6829/1428931817","state":["exists","up"]},{"osd":11,"uuid":"3330627f-ab5d-4e6e-83cf-fc8b74c54ee3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6825","nonce":4045070679}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6827","nonce":4045070679}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6831","nonce":4045070679}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":4045070679},{"type":"v1","addr":"172.21.15.64:6829","nonce":4045070679}]},"public_addr":"172.21.15.64:6825/4045070679","cluster_addr":"172.21.15.64:6827/4045070679","heartbeat_back_addr":"172.21.15.64:6831/4045070679","heartbeat_front_addr":"172.21.15.64:6829/4045070679","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:19:27.902633+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:19:54.305685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:20:18.727053+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:20:53.579199+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:21:21.058018+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:21:47.091494+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:22:23.269018+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:22:51.863628+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:23:18.537774+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:23:55.763993+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:24:24.956178+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-07T02:24:52.681201+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/1821831886":"2024-05-08T02:31:49.815663+0000","172.21.15.19:6801/2059564551":"2024-05-08T02:31:49.815663+0000","172.21.15.19:6800/2059564551":"2024-05-08T02:31:49.815663+0000","172.21.15.19:0/1956516258":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/1816772181":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/686554066":"2024-05-08T02:31:49.815663+0000","172.21.15.19:0/2602127700":"2024-05-08T02:31:49.815663+0000","172.21.15.19:6801/174304059":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/3466480009":"2024-05-08T02:31:49.815663+0000","172.21.15.19:0/1514051750":"2024-05-08T02:16:54.732592+0000","172.21.15.19:6800/174304059":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/3930045871":"2024-05-08T02:16:54.732592+0000","172.21.15.19:6800/3614957309":"2024-05-08T02:16:54.732592+0000","172.21.15.19:0/1328365399":"2024-05-08T02:17:31.252868+0000","172.21.15.19:0/139593344":"2024-05-08T02:16:54.732592+0000","172.21.15.19:6801/3614957309":"2024-05-08T02:16:54.732592+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-07T02:44:03.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.966+0000 7fca2e400700 1 -- 172.21.15.19:0/653175343 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fca0c0b2a70 msgr2=0x7fca0c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:44:03.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.966+0000 7fca2e400700 1 --2- 172.21.15.19:0/653175343 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fca0c0b2a70 0x7fca0c0b4f80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fca281079f0 tx=0x7fca18009260 comp rx=0 tx=0).stop 2024-05-07T02:44:03.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.966+0000 7fca2e400700 1 -- 172.21.15.19:0/653175343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca2810c5c0 msgr2=0x7fca281a17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:44:03.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.966+0000 7fca2e400700 1 --2- 172.21.15.19:0/653175343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca2810c5c0 0x7fca281a17f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fca2000b070 tx=0x7fca20004d00 comp rx=0 tx=0).stop 2024-05-07T02:44:03.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.966+0000 7fca2e400700 1 -- 172.21.15.19:0/653175343 shutdown_connections 2024-05-07T02:44:03.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.966+0000 7fca2e400700 1 --2- 172.21.15.19:0/653175343 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7fca0c0b2a70 0x7fca0c0b4f80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:03.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.966+0000 7fca2e400700 1 --2- 172.21.15.19:0/653175343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca2810c5c0 0x7fca281a17f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:03.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.966+0000 7fca2e400700 1 --2- 172.21.15.19:0/653175343 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca28106580 0x7fca2819cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:03.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.966+0000 7fca2e400700 1 --2- 172.21.15.19:0/653175343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca28106ed0 0x7fca2819d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:03.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.966+0000 7fca2e400700 1 -- 172.21.15.19:0/653175343 >> 172.21.15.19:0/653175343 conn(0x7fca28102080 msgr2=0x7fca28104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:44:03.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.967+0000 7fca2e400700 1 -- 172.21.15.19:0/653175343 shutdown_connections 2024-05-07T02:44:03.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:03.967+0000 7fca2e400700 1 -- 172.21.15.19:0/653175343 wait complete. 2024-05-07T02:44:04.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:04 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/653175343' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-07T02:44:04.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:04 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/653175343' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-07T02:44:04.829 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph fs fail cephfs 2024-05-07T02:44:04.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:04 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/653175343' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-07T02:44:05.292 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:44:05.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:05 smithi027 ceph-mon[34398]: pgmap v380: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 46 KiB/s wr, 27 op/s 2024-05-07T02:44:05.780 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:05 smithi019 ceph-mon[29781]: pgmap v380: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 46 KiB/s wr, 27 op/s 2024-05-07T02:44:05.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:05 smithi064 ceph-mon[34699]: pgmap v380: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 46 KiB/s wr, 27 op/s 2024-05-07T02:44:07.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.629+0000 7f9e8a600700 1 -- 172.21.15.19:0/4178920534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e841045d0 msgr2=0x7f9e841049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:44:07.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.629+0000 7f9e8a600700 1 --2- 172.21.15.19:0/4178920534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e841045d0 0x7f9e841049e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9e740099c0 tx=0x7f9e74009cd0 comp rx=0 tx=0).stop 2024-05-07T02:44:07.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.629+0000 7f9e8a600700 1 -- 172.21.15.19:0/4178920534 shutdown_connections 2024-05-07T02:44:07.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.629+0000 7f9e8a600700 1 --2- 172.21.15.19:0/4178920534 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e84109850 0x7f9e841111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:07.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.629+0000 7f9e8a600700 1 --2- 172.21.15.19:0/4178920534 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e841045d0 0x7f9e841049e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:07.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.629+0000 7f9e8a600700 1 --2- 172.21.15.19:0/4178920534 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e84108f00 0x7f9e84109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:07.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.629+0000 7f9e8a600700 1 -- 172.21.15.19:0/4178920534 >> 172.21.15.19:0/4178920534 conn(0x7f9e840ffea0 msgr2=0x7f9e841022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:44:07.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.629+0000 7f9e8a600700 1 -- 172.21.15.19:0/4178920534 shutdown_connections 2024-05-07T02:44:07.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.629+0000 7f9e8a600700 1 -- 172.21.15.19:0/4178920534 wait complete. 2024-05-07T02:44:07.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.631+0000 7f9e8a600700 1 Processor -- start 2024-05-07T02:44:07.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.631+0000 7f9e8a600700 1 -- start start 2024-05-07T02:44:07.633 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:07 smithi019 ceph-mon[29781]: pgmap v381: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 46 KiB/s wr, 32 op/s 2024-05-07T02:44:07.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.632+0000 7f9e8a600700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e841045d0 0x7f9e8419d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:44:07.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.632+0000 7f9e8a600700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e84108f00 0x7f9e8419d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:44:07.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.632+0000 7f9e8a600700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e84109850 0x7f9e841a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:44:07.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.632+0000 7f9e8a600700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9e841144d0 con 0x7f9e84109850 2024-05-07T02:44:07.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.632+0000 7f9e8a600700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f9e84114350 con 0x7f9e841045d0 2024-05-07T02:44:07.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.632+0000 7f9e8a600700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9e84114650 con 0x7f9e84108f00 2024-05-07T02:44:07.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.632+0000 7f9e82a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e84108f00 0x7f9e8419d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:44:07.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.632+0000 7f9e82a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e84108f00 0x7f9e8419d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:53368/0 (socket says 172.21.15.19:53368) 2024-05-07T02:44:07.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.632+0000 7f9e82a00700 1 -- 172.21.15.19:0/866997923 learned_addr learned my addr 172.21.15.19:0/866997923 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:44:07.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.632+0000 7f9e83400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e841045d0 0x7f9e8419d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:44:07.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.633+0000 7f9e83e00700 1 --2- 172.21.15.19:0/866997923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e84109850 0x7f9e841a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:44:07.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.633+0000 7f9e82a00700 1 -- 172.21.15.19:0/866997923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e841045d0 msgr2=0x7f9e8419d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:44:07.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.633+0000 7f9e82a00700 1 --2- 172.21.15.19:0/866997923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e841045d0 0x7f9e8419d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:07.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.633+0000 7f9e82a00700 1 -- 172.21.15.19:0/866997923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e84109850 msgr2=0x7f9e841a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:44:07.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.633+0000 7f9e82a00700 1 --2- 172.21.15.19:0/866997923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e84109850 0x7f9e841a18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:07.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.633+0000 7f9e82a00700 1 -- 172.21.15.19:0/866997923 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e841a1f50 con 0x7f9e84108f00 2024-05-07T02:44:07.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.633+0000 7f9e83400700 1 --2- 172.21.15.19:0/866997923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e841045d0 0x7f9e8419d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:44:07.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.634+0000 7f9e82a00700 1 --2- 172.21.15.19:0/866997923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e84108f00 0x7f9e8419d620 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9e7000e970 tx=0x7f9e7000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:44:07.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.634+0000 7f9e7be00700 1 -- 172.21.15.19:0/866997923 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e70004a30 con 0x7f9e84108f00 2024-05-07T02:44:07.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.634+0000 7f9e8a600700 1 -- 172.21.15.19:0/866997923 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e841a2240 con 0x7f9e84108f00 2024-05-07T02:44:07.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.634+0000 7f9e7be00700 1 -- 172.21.15.19:0/866997923 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e7000cc90 con 0x7f9e84108f00 2024-05-07T02:44:07.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.634+0000 7f9e7be00700 1 -- 172.21.15.19:0/866997923 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e70005610 con 0x7f9e84108f00 2024-05-07T02:44:07.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.634+0000 7f9e8a600700 1 -- 172.21.15.19:0/866997923 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e841a9b60 con 0x7f9e84108f00 2024-05-07T02:44:07.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.636+0000 7f9e8a600700 1 -- 172.21.15.19:0/866997923 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e840512c0 con 0x7f9e84108f00 2024-05-07T02:44:07.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.638+0000 7f9e7be00700 1 -- 172.21.15.19:0/866997923 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9e7000c7e0 con 0x7f9e84108f00 2024-05-07T02:44:07.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.639+0000 7f9e7be00700 1 --2- 172.21.15.19:0/866997923 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f9e680b29f0 0x7f9e680b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:44:07.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.639+0000 7f9e7be00700 1 -- 172.21.15.19:0/866997923 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9525+0+0 (secure 0 0 0) 0x7f9e700e2bd0 con 0x7f9e84108f00 2024-05-07T02:44:07.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.640+0000 7f9e83400700 1 --2- 172.21.15.19:0/866997923 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f9e680b29f0 0x7f9e680b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:44:07.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.640+0000 7f9e83400700 1 --2- 172.21.15.19:0/866997923 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f9e680b29f0 0x7f9e680b4f00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9e74000c00 tx=0x7f9e7401a040 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:44:07.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.644+0000 7f9e7be00700 1 -- 172.21.15.19:0/866997923 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9e700aa160 con 0x7f9e84108f00 2024-05-07T02:44:07.707 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:07 smithi027 ceph-mon[34398]: pgmap v381: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 46 KiB/s wr, 32 op/s 2024-05-07T02:44:07.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:07 smithi064 ceph-mon[34699]: pgmap v381: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 46 KiB/s wr, 32 op/s 2024-05-07T02:44:07.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.936+0000 7f9e8a600700 1 -- 172.21.15.19:0/866997923 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f9e841a9d30 con 0x7f9e84108f00 2024-05-07T02:44:07.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.948+0000 7f9e7be00700 1 -- 172.21.15.19:0/866997923 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v51) ==== 161+0+0 (secure 0 0 0) 0x7f9e700a9a60 con 0x7f9e84108f00 2024-05-07T02:44:07.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.955+0000 7f9e8a600700 1 -- 172.21.15.19:0/866997923 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f9e680b29f0 msgr2=0x7f9e680b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:44:07.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.955+0000 7f9e8a600700 1 --2- 172.21.15.19:0/866997923 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f9e680b29f0 0x7f9e680b4f00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9e74000c00 tx=0x7f9e7401a040 comp rx=0 tx=0).stop 2024-05-07T02:44:07.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.955+0000 7f9e8a600700 1 -- 172.21.15.19:0/866997923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e84108f00 msgr2=0x7f9e8419d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:44:07.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.955+0000 7f9e8a600700 1 --2- 172.21.15.19:0/866997923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e84108f00 0x7f9e8419d620 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9e7000e970 tx=0x7f9e7000ec80 comp rx=0 tx=0).stop 2024-05-07T02:44:07.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.955+0000 7f9e8a600700 1 -- 172.21.15.19:0/866997923 shutdown_connections 2024-05-07T02:44:07.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.955+0000 7f9e8a600700 1 --2- 172.21.15.19:0/866997923 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f9e680b29f0 0x7f9e680b4f00 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:07.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.955+0000 7f9e8a600700 1 --2- 172.21.15.19:0/866997923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e84109850 0x7f9e841a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:07.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.955+0000 7f9e8a600700 1 --2- 172.21.15.19:0/866997923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e841045d0 0x7f9e8419d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:07.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.955+0000 7f9e8a600700 1 --2- 172.21.15.19:0/866997923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e84108f00 0x7f9e8419d620 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:07.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.955+0000 7f9e8a600700 1 -- 172.21.15.19:0/866997923 >> 172.21.15.19:0/866997923 conn(0x7f9e840ffea0 msgr2=0x7f9e8410a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:44:07.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.956+0000 7f9e8a600700 1 -- 172.21.15.19:0/866997923 shutdown_connections 2024-05-07T02:44:07.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:07.956+0000 7f9e8a600700 1 -- 172.21.15.19:0/866997923 wait complete. 2024-05-07T02:44:07.958 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-07T02:44:08.359 INFO:journalctl@ceph.mds.c.smithi064.stdout:May 07 02:44:08 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-c[65500]: ignoring --setuser ceph since I am not root 2024-05-07T02:44:08.360 INFO:journalctl@ceph.mds.c.smithi064.stdout:May 07 02:44:08 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-c[65500]: ignoring --setgroup ceph since I am not root 2024-05-07T02:44:08.360 INFO:journalctl@ceph.mds.c.smithi064.stdout:May 07 02:44:08 smithi064 ceph-mds[65504]: -- 172.21.15.64:0/1585266052 <== mon.1 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dc9f051a40 con 0x55dc9f099180 2024-05-07T02:44:08.360 INFO:journalctl@ceph.mds.c.smithi064.stdout:May 07 02:44:08 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-c[65500]: starting mds.c at 2024-05-07T02:44:08.360 INFO:journalctl@ceph.mds.i.smithi064.stdout:May 07 02:44:08 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-i[65809]: ignoring --setuser ceph since I am not root 2024-05-07T02:44:08.360 INFO:journalctl@ceph.mds.i.smithi064.stdout:May 07 02:44:08 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-i[65809]: ignoring --setgroup ceph since I am not root 2024-05-07T02:44:08.361 INFO:journalctl@ceph.mds.i.smithi064.stdout:May 07 02:44:08 smithi064 ceph-mds[65813]: -- 172.21.15.64:0/3814059404 <== mon.2 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564a7e75ba40 con 0x564a7e7a3600 2024-05-07T02:44:08.361 INFO:journalctl@ceph.mds.i.smithi064.stdout:May 07 02:44:08 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-i[65809]: starting mds.i at 2024-05-07T02:44:08.361 INFO:journalctl@ceph.mds.f.smithi064.stdout:May 07 02:44:08 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-f[65192]: ignoring --setuser ceph since I am not root 2024-05-07T02:44:08.361 INFO:journalctl@ceph.mds.f.smithi064.stdout:May 07 02:44:08 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-f[65192]: ignoring --setgroup ceph since I am not root 2024-05-07T02:44:08.361 INFO:journalctl@ceph.mds.f.smithi064.stdout:May 07 02:44:08 smithi064 ceph-mds[65196]: -- 172.21.15.64:0/1088784352 <== mon.1 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c8c7493a40 con 0x55c8c74db600 2024-05-07T02:44:08.361 INFO:journalctl@ceph.mds.f.smithi064.stdout:May 07 02:44:08 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-f[65192]: starting mds.f at 2024-05-07T02:44:08.394 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:07 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mon-a[29772]: 2024-05-07T02:44:07.938+0000 7f9ce9600700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-07T02:44:08.394 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 07 02:44:08 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-d[70814]: ignoring --setuser ceph since I am not root 2024-05-07T02:44:08.395 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 07 02:44:08 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-d[70814]: ignoring --setgroup ceph since I am not root 2024-05-07T02:44:08.395 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 07 02:44:08 smithi019 ceph-mds[70819]: -- 172.21.15.19:0/1324295551 <== mon.2 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5577fb745a40 con 0x5577fb78d180 2024-05-07T02:44:08.395 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 07 02:44:08 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-d[70814]: starting mds.d at 2024-05-07T02:44:08.395 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 07 02:44:08 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-j[70358]: ignoring --setuser ceph since I am not root 2024-05-07T02:44:08.395 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 07 02:44:08 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-j[70358]: ignoring --setgroup ceph since I am not root 2024-05-07T02:44:08.395 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 07 02:44:08 smithi019 ceph-mds[70362]: -- 172.21.15.19:0/410459174 <== mon.2 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f1843aba40 con 0x55f1843f3a80 2024-05-07T02:44:08.396 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 07 02:44:08 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-j[70358]: starting mds.j at 2024-05-07T02:44:08.396 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 07 02:44:08 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-a[72008]: ignoring --setuser ceph since I am not root 2024-05-07T02:44:08.396 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 07 02:44:08 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-a[72008]: ignoring --setgroup ceph since I am not root 2024-05-07T02:44:08.396 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 07 02:44:08 smithi019 ceph-mds[72017]: -- 172.21.15.19:0/2283638155 <== mon.2 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5564409d1a40 con 0x556440a19180 2024-05-07T02:44:08.396 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 07 02:44:08 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-a[72008]: starting mds.a at 2024-05-07T02:44:08.426 INFO:journalctl@ceph.mds.e.smithi027.stdout:May 07 02:44:08 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-e[64547]: ignoring --setuser ceph since I am not root 2024-05-07T02:44:08.426 INFO:journalctl@ceph.mds.e.smithi027.stdout:May 07 02:44:08 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-e[64547]: ignoring --setgroup ceph since I am not root 2024-05-07T02:44:08.427 INFO:journalctl@ceph.mds.e.smithi027.stdout:May 07 02:44:08 smithi027 ceph-mds[64551]: -- 172.21.15.27:0/3437894787 <== mon.2 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5580d0953a40 con 0x5580d099b600 2024-05-07T02:44:08.427 INFO:journalctl@ceph.mds.e.smithi027.stdout:May 07 02:44:08 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-e[64547]: starting mds.e at 2024-05-07T02:44:08.427 INFO:journalctl@ceph.mds.h.smithi027.stdout:May 07 02:44:08 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-h[64849]: ignoring --setuser ceph since I am not root 2024-05-07T02:44:08.427 INFO:journalctl@ceph.mds.h.smithi027.stdout:May 07 02:44:08 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-h[64849]: ignoring --setgroup ceph since I am not root 2024-05-07T02:44:08.427 INFO:journalctl@ceph.mds.h.smithi027.stdout:May 07 02:44:08 smithi027 ceph-mds[64853]: -- 172.21.15.27:0/133183577 <== mon.2 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5559487eda40 con 0x555948835600 2024-05-07T02:44:08.427 INFO:journalctl@ceph.mds.h.smithi027.stdout:May 07 02:44:08 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-h[64849]: starting mds.h at 2024-05-07T02:44:08.428 INFO:journalctl@ceph.mds.b.smithi027.stdout:May 07 02:44:08 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-b[65159]: ignoring --setuser ceph since I am not root 2024-05-07T02:44:08.428 INFO:journalctl@ceph.mds.b.smithi027.stdout:May 07 02:44:08 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-b[65159]: ignoring --setgroup ceph since I am not root 2024-05-07T02:44:08.428 INFO:journalctl@ceph.mds.b.smithi027.stdout:May 07 02:44:08 smithi027 ceph-mds[65163]: -- 172.21.15.27:0/1287228608 <== mon.2 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c4583f3a40 con 0x55c45843ba80 2024-05-07T02:44:08.428 INFO:journalctl@ceph.mds.b.smithi027.stdout:May 07 02:44:08 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-b[65159]: starting mds.b at 2024-05-07T02:44:08.428 INFO:journalctl@ceph.mds.k.smithi027.stdout:May 07 02:44:08 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-k[64223]: ignoring --setuser ceph since I am not root 2024-05-07T02:44:08.429 INFO:journalctl@ceph.mds.k.smithi027.stdout:May 07 02:44:08 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-k[64223]: ignoring --setgroup ceph since I am not root 2024-05-07T02:44:08.429 INFO:journalctl@ceph.mds.k.smithi027.stdout:May 07 02:44:08 smithi027 ceph-mds[64227]: -- 172.21.15.27:0/687696585 <== mon.2 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559273c29a40 con 0x559273c71a80 2024-05-07T02:44:08.429 INFO:journalctl@ceph.mds.k.smithi027.stdout:May 07 02:44:08 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-k[64223]: starting mds.k at 2024-05-07T02:44:08.645 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:08 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/866997923' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-07T02:44:08.784 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:08 smithi019 ceph-mon[29781]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-07T02:44:08.785 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:08 smithi019 ceph-mon[29781]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-07T02:44:08.785 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:08 smithi019 ceph-mon[29781]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-07T02:44:08.785 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:08 smithi019 ceph-mon[29781]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-07T02:44:08.785 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:08 smithi019 ceph-mon[29781]: osdmap e101: 12 total, 12 up, 12 in 2024-05-07T02:44:08.785 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:08 smithi019 ceph-mon[29781]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-07T02:44:08.786 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:08 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/866997923' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-07T02:44:08.786 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:08 smithi027 ceph-mon[34398]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-07T02:44:08.787 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:08 smithi027 ceph-mon[34398]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-07T02:44:08.787 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:08 smithi027 ceph-mon[34398]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-07T02:44:08.787 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:08 smithi027 ceph-mon[34398]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-07T02:44:08.787 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:08 smithi027 ceph-mon[34398]: osdmap e101: 12 total, 12 up, 12 in 2024-05-07T02:44:08.787 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:08 smithi027 ceph-mon[34398]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-07T02:44:08.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:08 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/866997923' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-07T02:44:08.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:08 smithi064 ceph-mon[34699]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-07T02:44:08.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:08 smithi064 ceph-mon[34699]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-07T02:44:08.859 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:08 smithi064 ceph-mon[34699]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-07T02:44:08.860 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:08 smithi064 ceph-mon[34699]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-07T02:44:08.860 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:08 smithi064 ceph-mon[34699]: osdmap e101: 12 total, 12 up, 12 in 2024-05-07T02:44:08.860 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:08 smithi064 ceph-mon[34699]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-07T02:44:08.930 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-05-07T02:44:09.386 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:44:10.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:09 smithi027 ceph-mon[34398]: pgmap v383: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-05-07T02:44:10.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:09 smithi027 ceph-mon[34398]: mds.? [v2:172.21.15.64:6834/3350586754,v1:172.21.15.64:6835/3350586754] up:boot 2024-05-07T02:44:10.208 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:09 smithi027 ceph-mon[34398]: mds.? [v2:172.21.15.27:6832/4159555926,v1:172.21.15.27:6833/4159555926] up:boot 2024-05-07T02:44:10.208 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:09 smithi027 ceph-mon[34398]: mds.? [v2:172.21.15.27:6834/1891720838,v1:172.21.15.27:6835/1891720838] up:boot 2024-05-07T02:44:10.208 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:09 smithi027 ceph-mon[34398]: mds.? [v2:172.21.15.64:6836/1496948674,v1:172.21.15.64:6837/1496948674] up:boot 2024-05-07T02:44:10.208 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:09 smithi027 ceph-mon[34398]: mds.? [v2:172.21.15.64:6838/3882639956,v1:172.21.15.64:6839/3882639956] up:boot 2024-05-07T02:44:10.208 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:09 smithi027 ceph-mon[34398]: mds.? [v2:172.21.15.27:6836/1235433349,v1:172.21.15.27:6838/1235433349] up:boot 2024-05-07T02:44:10.208 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:09 smithi027 ceph-mon[34398]: mds.? [v2:172.21.15.27:6837/212517942,v1:172.21.15.27:6839/212517942] up:boot 2024-05-07T02:44:10.208 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:09 smithi027 ceph-mon[34398]: mds.? [v2:172.21.15.19:6834/3476411519,v1:172.21.15.19:6835/3476411519] up:boot 2024-05-07T02:44:10.208 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:09 smithi027 ceph-mon[34398]: mds.? [v2:172.21.15.19:6836/1539267845,v1:172.21.15.19:6837/1539267845] up:boot 2024-05-07T02:44:10.208 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:09 smithi027 ceph-mon[34398]: mds.? [v2:172.21.15.19:6840/570157709,v1:172.21.15.19:6841/570157709] up:boot 2024-05-07T02:44:10.209 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:09 smithi027 ceph-mon[34398]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-05-07T02:44:10.209 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:09 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-07T02:44:10.209 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:09 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-07T02:44:10.209 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:09 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-07T02:44:10.209 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:09 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-07T02:44:10.209 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:09 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-07T02:44:10.209 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:09 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-07T02:44:10.209 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:09 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-07T02:44:10.210 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:09 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-07T02:44:10.210 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:09 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-07T02:44:10.210 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:09 smithi027 ceph-mon[34398]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-07T02:44:10.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:09 smithi064 ceph-mon[34699]: pgmap v383: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-05-07T02:44:10.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:09 smithi064 ceph-mon[34699]: mds.? [v2:172.21.15.64:6834/3350586754,v1:172.21.15.64:6835/3350586754] up:boot 2024-05-07T02:44:10.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:09 smithi064 ceph-mon[34699]: mds.? [v2:172.21.15.27:6832/4159555926,v1:172.21.15.27:6833/4159555926] up:boot 2024-05-07T02:44:10.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:09 smithi064 ceph-mon[34699]: mds.? [v2:172.21.15.27:6834/1891720838,v1:172.21.15.27:6835/1891720838] up:boot 2024-05-07T02:44:10.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:09 smithi064 ceph-mon[34699]: mds.? [v2:172.21.15.64:6836/1496948674,v1:172.21.15.64:6837/1496948674] up:boot 2024-05-07T02:44:10.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:09 smithi064 ceph-mon[34699]: mds.? [v2:172.21.15.64:6838/3882639956,v1:172.21.15.64:6839/3882639956] up:boot 2024-05-07T02:44:10.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:09 smithi064 ceph-mon[34699]: mds.? [v2:172.21.15.27:6836/1235433349,v1:172.21.15.27:6838/1235433349] up:boot 2024-05-07T02:44:10.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:09 smithi064 ceph-mon[34699]: mds.? [v2:172.21.15.27:6837/212517942,v1:172.21.15.27:6839/212517942] up:boot 2024-05-07T02:44:10.361 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:09 smithi064 ceph-mon[34699]: mds.? [v2:172.21.15.19:6834/3476411519,v1:172.21.15.19:6835/3476411519] up:boot 2024-05-07T02:44:10.361 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:09 smithi064 ceph-mon[34699]: mds.? [v2:172.21.15.19:6836/1539267845,v1:172.21.15.19:6837/1539267845] up:boot 2024-05-07T02:44:10.361 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:09 smithi064 ceph-mon[34699]: mds.? [v2:172.21.15.19:6840/570157709,v1:172.21.15.19:6841/570157709] up:boot 2024-05-07T02:44:10.361 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:09 smithi064 ceph-mon[34699]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-05-07T02:44:10.361 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:09 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-07T02:44:10.361 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:09 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-07T02:44:10.361 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:09 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-07T02:44:10.361 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:09 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-07T02:44:10.361 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:09 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-07T02:44:10.362 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:09 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-07T02:44:10.362 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:09 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-07T02:44:10.362 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:09 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-07T02:44:10.362 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:09 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-07T02:44:10.362 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:09 smithi064 ceph-mon[34699]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-07T02:44:10.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:09 smithi019 ceph-mon[29781]: pgmap v383: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-05-07T02:44:10.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:09 smithi019 ceph-mon[29781]: mds.? [v2:172.21.15.64:6834/3350586754,v1:172.21.15.64:6835/3350586754] up:boot 2024-05-07T02:44:10.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:09 smithi019 ceph-mon[29781]: mds.? [v2:172.21.15.27:6832/4159555926,v1:172.21.15.27:6833/4159555926] up:boot 2024-05-07T02:44:10.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:09 smithi019 ceph-mon[29781]: mds.? [v2:172.21.15.27:6834/1891720838,v1:172.21.15.27:6835/1891720838] up:boot 2024-05-07T02:44:10.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:09 smithi019 ceph-mon[29781]: mds.? [v2:172.21.15.64:6836/1496948674,v1:172.21.15.64:6837/1496948674] up:boot 2024-05-07T02:44:10.397 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:09 smithi019 ceph-mon[29781]: mds.? [v2:172.21.15.64:6838/3882639956,v1:172.21.15.64:6839/3882639956] up:boot 2024-05-07T02:44:10.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:09 smithi019 ceph-mon[29781]: mds.? [v2:172.21.15.27:6836/1235433349,v1:172.21.15.27:6838/1235433349] up:boot 2024-05-07T02:44:10.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:09 smithi019 ceph-mon[29781]: mds.? [v2:172.21.15.27:6837/212517942,v1:172.21.15.27:6839/212517942] up:boot 2024-05-07T02:44:10.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:09 smithi019 ceph-mon[29781]: mds.? [v2:172.21.15.19:6834/3476411519,v1:172.21.15.19:6835/3476411519] up:boot 2024-05-07T02:44:10.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:09 smithi019 ceph-mon[29781]: mds.? [v2:172.21.15.19:6836/1539267845,v1:172.21.15.19:6837/1539267845] up:boot 2024-05-07T02:44:10.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:09 smithi019 ceph-mon[29781]: mds.? [v2:172.21.15.19:6840/570157709,v1:172.21.15.19:6841/570157709] up:boot 2024-05-07T02:44:10.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:09 smithi019 ceph-mon[29781]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-05-07T02:44:10.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:09 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-07T02:44:10.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:09 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-07T02:44:10.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:09 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-07T02:44:10.399 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:09 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-07T02:44:10.399 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:09 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-07T02:44:10.399 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:09 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-07T02:44:10.399 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:09 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-07T02:44:10.399 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:09 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-07T02:44:10.399 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:09 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-07T02:44:10.399 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:09 smithi019 ceph-mon[29781]: from='mgr.25234 172.21.15.19:0/1562175059' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-07T02:44:11.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:11.817+0000 7f2c17200700 1 -- 172.21.15.19:0/477544563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c18106620 msgr2=0x7f2c18106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:44:11.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:11.817+0000 7f2c17200700 1 --2- 172.21.15.19:0/477544563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c18106620 0x7f2c18106a30 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2c08009990 tx=0x7f2c08009ca0 comp rx=0 tx=0).stop 2024-05-07T02:44:11.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:11.817+0000 7f2c17200700 1 -- 172.21.15.19:0/477544563 shutdown_connections 2024-05-07T02:44:11.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:11.817+0000 7f2c17200700 1 --2- 172.21.15.19:0/477544563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c1810b8a0 0x7f2c18113200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:11.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:11.817+0000 7f2c17200700 1 --2- 172.21.15.19:0/477544563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c18106620 0x7f2c18106a30 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:11.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:11.817+0000 7f2c17200700 1 --2- 172.21.15.19:0/477544563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c1810af50 0x7f2c1810b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:11.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:11.817+0000 7f2c17200700 1 -- 172.21.15.19:0/477544563 >> 172.21.15.19:0/477544563 conn(0x7f2c18102030 msgr2=0x7f2c18104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:44:11.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:11.817+0000 7f2c17200700 1 -- 172.21.15.19:0/477544563 shutdown_connections 2024-05-07T02:44:11.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:11.817+0000 7f2c17200700 1 -- 172.21.15.19:0/477544563 wait complete. 2024-05-07T02:44:11.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:11.819+0000 7f2c17200700 1 Processor -- start 2024-05-07T02:44:11.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:11.819+0000 7f2c17200700 1 -- start start 2024-05-07T02:44:11.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:11.820+0000 7f2c17200700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c18106620 0x7f2c18077cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:44:11.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:11.820+0000 7f2c17200700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c1810af50 0x7f2c18078210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:44:11.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:11.820+0000 7f2c17200700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c1810b8a0 0x7f2c18078750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:44:11.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:11.820+0000 7f2c17200700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2c18116520 con 0x7f2c1810b8a0 2024-05-07T02:44:11.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:11.820+0000 7f2c17200700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f2c181163a0 con 0x7f2c18106620 2024-05-07T02:44:11.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:11.820+0000 7f2c17200700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2c181166a0 con 0x7f2c1810af50 2024-05-07T02:44:11.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:11.820+0000 7f2c15e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c18106620 0x7f2c18077cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:44:11.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:11.820+0000 7f2c15e00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c18106620 0x7f2c18077cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:55722/0 (socket says 172.21.15.19:55722) 2024-05-07T02:44:11.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:11.820+0000 7f2c15e00700 1 -- 172.21.15.19:0/2088234348 learned_addr learned my addr 172.21.15.19:0/2088234348 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:44:11.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:11.820+0000 7f2c15400700 1 --2- 172.21.15.19:0/2088234348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c1810af50 0x7f2c18078210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:44:11.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:11.820+0000 7f2c16800700 1 --2- 172.21.15.19:0/2088234348 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c1810b8a0 0x7f2c18078750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:44:11.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:11.821+0000 7f2c15e00700 1 -- 172.21.15.19:0/2088234348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c1810af50 msgr2=0x7f2c18078210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:44:11.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:11.821+0000 7f2c15e00700 1 --2- 172.21.15.19:0/2088234348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c1810af50 0x7f2c18078210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:11.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:11.821+0000 7f2c15e00700 1 -- 172.21.15.19:0/2088234348 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c1810b8a0 msgr2=0x7f2c18078750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:44:11.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:11.821+0000 7f2c15e00700 1 --2- 172.21.15.19:0/2088234348 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c1810b8a0 0x7f2c18078750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:11.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:11.821+0000 7f2c15e00700 1 -- 172.21.15.19:0/2088234348 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c181ad410 con 0x7f2c18106620 2024-05-07T02:44:11.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:11.821+0000 7f2c15e00700 1 --2- 172.21.15.19:0/2088234348 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c18106620 0x7f2c18077cd0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2c08000c00 tx=0x7f2c080057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:44:11.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:11.821+0000 7f2c0ea00700 1 -- 172.21.15.19:0/2088234348 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c08027070 con 0x7f2c18106620 2024-05-07T02:44:11.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:11.822+0000 7f2c0ea00700 1 -- 172.21.15.19:0/2088234348 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c0800b610 con 0x7f2c18106620 2024-05-07T02:44:11.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:11.822+0000 7f2c17200700 1 -- 172.21.15.19:0/2088234348 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c181ad640 con 0x7f2c18106620 2024-05-07T02:44:11.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:11.822+0000 7f2c0ea00700 1 -- 172.21.15.19:0/2088234348 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c080173f0 con 0x7f2c18106620 2024-05-07T02:44:11.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:11.822+0000 7f2c17200700 1 -- 172.21.15.19:0/2088234348 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c181ada90 con 0x7f2c18106620 2024-05-07T02:44:11.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:11.823+0000 7f2c17200700 1 -- 172.21.15.19:0/2088234348 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c18107ad0 con 0x7f2c18106620 2024-05-07T02:44:11.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:11.825+0000 7f2c0ea00700 1 -- 172.21.15.19:0/2088234348 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2c0802a070 con 0x7f2c18106620 2024-05-07T02:44:11.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:11.827+0000 7f2c0ea00700 1 --2- 172.21.15.19:0/2088234348 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f2bfc0b29f0 0x7f2bfc0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:44:11.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:11.827+0000 7f2c0ea00700 1 -- 172.21.15.19:0/2088234348 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9867+0+0 (secure 0 0 0) 0x7f2c080e3b80 con 0x7f2c18106620 2024-05-07T02:44:11.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:11.827+0000 7f2c15400700 1 --2- 172.21.15.19:0/2088234348 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f2bfc0b29f0 0x7f2bfc0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:44:11.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:11.828+0000 7f2c15400700 1 --2- 172.21.15.19:0/2088234348 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f2bfc0b29f0 0x7f2bfc0b4f00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2c181097d0 tx=0x7f2c04009260 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:44:11.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:11.832+0000 7f2c0ea00700 1 -- 172.21.15.19:0/2088234348 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2c08013070 con 0x7f2c18106620 2024-05-07T02:44:12.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:12.126+0000 7f2c17200700 1 -- 172.21.15.19:0/2088234348 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f2c18079200 con 0x7f2c18106620 2024-05-07T02:44:12.127 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:11 smithi019 ceph-mon[29781]: pgmap v384: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-05-07T02:44:12.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:11 smithi064 ceph-mon[34699]: pgmap v384: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-05-07T02:44:12.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:11 smithi027 ceph-mon[34398]: pgmap v384: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-05-07T02:44:12.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:12.974+0000 7f2c0ea00700 1 -- 172.21.15.19:0/2088234348 <== mon.2 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v53) ==== 104+0+0 (secure 0 0 0) 0x7f2c080ab050 con 0x7f2c18106620 2024-05-07T02:44:12.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:12.978+0000 7f2c17200700 1 -- 172.21.15.19:0/2088234348 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f2bfc0b29f0 msgr2=0x7f2bfc0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:44:12.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:12.978+0000 7f2c17200700 1 --2- 172.21.15.19:0/2088234348 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f2bfc0b29f0 0x7f2bfc0b4f00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2c181097d0 tx=0x7f2c04009260 comp rx=0 tx=0).stop 2024-05-07T02:44:12.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:12.978+0000 7f2c17200700 1 -- 172.21.15.19:0/2088234348 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c18106620 msgr2=0x7f2c18077cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:44:12.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:12.978+0000 7f2c17200700 1 --2- 172.21.15.19:0/2088234348 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c18106620 0x7f2c18077cd0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2c08000c00 tx=0x7f2c080057c0 comp rx=0 tx=0).stop 2024-05-07T02:44:12.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:12.979+0000 7f2c17200700 1 -- 172.21.15.19:0/2088234348 shutdown_connections 2024-05-07T02:44:12.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:12.979+0000 7f2c17200700 1 --2- 172.21.15.19:0/2088234348 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f2bfc0b29f0 0x7f2bfc0b4f00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:12.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:12.979+0000 7f2c17200700 1 --2- 172.21.15.19:0/2088234348 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c1810b8a0 0x7f2c18078750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:12.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:12.979+0000 7f2c17200700 1 --2- 172.21.15.19:0/2088234348 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c18106620 0x7f2c18077cd0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:12.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:12.979+0000 7f2c17200700 1 --2- 172.21.15.19:0/2088234348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2c1810af50 0x7f2c18078210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:12.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:12.979+0000 7f2c17200700 1 -- 172.21.15.19:0/2088234348 >> 172.21.15.19:0/2088234348 conn(0x7f2c18102030 msgr2=0x7f2c1810a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:44:12.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:12.979+0000 7f2c17200700 1 -- 172.21.15.19:0/2088234348 shutdown_connections 2024-05-07T02:44:12.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:12.979+0000 7f2c17200700 1 -- 172.21.15.19:0/2088234348 wait complete. 2024-05-07T02:44:13.357 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:12 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/2088234348' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-07T02:44:13.358 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:12 smithi027 ceph-mon[34398]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-07T02:44:13.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:12 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/2088234348' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-07T02:44:13.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:12 smithi064 ceph-mon[34699]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-07T02:44:13.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:12 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/2088234348' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-07T02:44:13.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:12 smithi019 ceph-mon[29781]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-07T02:44:13.973 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-07T02:44:14.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:13 smithi064 ceph-mon[34699]: pgmap v385: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 102 B/s wr, 7 op/s 2024-05-07T02:44:14.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:13 smithi064 ceph-mon[34699]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-07T02:44:14.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:13 smithi064 ceph-mon[34699]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-07T02:44:14.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:13 smithi064 ceph-mon[34699]: Cluster is now healthy 2024-05-07T02:44:14.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:13 smithi064 ceph-mon[34699]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-07T02:44:14.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:13 smithi064 ceph-mon[34699]: fsmap 12 up:standby 2024-05-07T02:44:14.370 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:13 smithi027 ceph-mon[34398]: pgmap v385: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 102 B/s wr, 7 op/s 2024-05-07T02:44:14.371 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:13 smithi027 ceph-mon[34398]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-07T02:44:14.371 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:13 smithi027 ceph-mon[34398]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-07T02:44:14.371 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:13 smithi027 ceph-mon[34398]: Cluster is now healthy 2024-05-07T02:44:14.371 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:13 smithi027 ceph-mon[34398]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-07T02:44:14.371 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:13 smithi027 ceph-mon[34398]: fsmap 12 up:standby 2024-05-07T02:44:14.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:13 smithi019 ceph-mon[29781]: pgmap v385: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 102 B/s wr, 7 op/s 2024-05-07T02:44:14.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:13 smithi019 ceph-mon[29781]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-07T02:44:14.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:13 smithi019 ceph-mon[29781]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-07T02:44:14.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:13 smithi019 ceph-mon[29781]: Cluster is now healthy 2024-05-07T02:44:14.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:13 smithi019 ceph-mon[29781]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-07T02:44:14.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:13 smithi019 ceph-mon[29781]: fsmap 12 up:standby 2024-05-07T02:44:14.446 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:44:16.304 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:15 smithi019 ceph-mon[29781]: pgmap v386: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 102 B/s wr, 7 op/s 2024-05-07T02:44:16.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:15 smithi064 ceph-mon[34699]: pgmap v386: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 102 B/s wr, 7 op/s 2024-05-07T02:44:16.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:15 smithi027 ceph-mon[34398]: pgmap v386: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 102 B/s wr, 7 op/s 2024-05-07T02:44:16.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:16.910+0000 7f8820a00700 1 -- 172.21.15.19:0/1014396341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f881c077560 msgr2=0x7f881c1106a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:44:16.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:16.910+0000 7f8820a00700 1 --2- 172.21.15.19:0/1014396341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f881c077560 0x7f881c1106a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f88080098f0 tx=0x7f8808009c00 comp rx=0 tx=0).stop 2024-05-07T02:44:16.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:16.911+0000 7f8820a00700 1 -- 172.21.15.19:0/1014396341 shutdown_connections 2024-05-07T02:44:16.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:16.911+0000 7f8820a00700 1 --2- 172.21.15.19:0/1014396341 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f881c110be0 0x7f881c113000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:16.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:16.911+0000 7f8820a00700 1 --2- 172.21.15.19:0/1014396341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f881c076c10 0x7f881c077020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:16.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:16.911+0000 7f8820a00700 1 --2- 172.21.15.19:0/1014396341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f881c077560 0x7f881c1106a0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:16.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:16.911+0000 7f8820a00700 1 -- 172.21.15.19:0/1014396341 >> 172.21.15.19:0/1014396341 conn(0x7f881c0ffea0 msgr2=0x7f881c1022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:44:16.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:16.911+0000 7f8820a00700 1 -- 172.21.15.19:0/1014396341 shutdown_connections 2024-05-07T02:44:16.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:16.911+0000 7f8820a00700 1 -- 172.21.15.19:0/1014396341 wait complete. 2024-05-07T02:44:16.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:16.912+0000 7f8820a00700 1 Processor -- start 2024-05-07T02:44:16.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:16.912+0000 7f8820a00700 1 -- start start 2024-05-07T02:44:16.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:16.913+0000 7f8820a00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f881c076c10 0x7f881c19d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:44:16.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:16.913+0000 7f8820a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f881c077560 0x7f881c19d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:44:16.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:16.913+0000 7f8820a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f881c110be0 0x7f881c1a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:44:16.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:16.913+0000 7f8820a00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f881c1166a0 con 0x7f881c077560 2024-05-07T02:44:16.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:16.913+0000 7f8820a00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f881c116520 con 0x7f881c110be0 2024-05-07T02:44:16.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:16.913+0000 7f8820a00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f881c116820 con 0x7f881c076c10 2024-05-07T02:44:16.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:16.913+0000 7f8819400700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f881c077560 0x7f881c19d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:44:16.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:16.913+0000 7f881a800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f881c110be0 0x7f881c1a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:44:16.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:16.913+0000 7f8819e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f881c076c10 0x7f881c19d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:44:16.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:16.913+0000 7f881a800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f881c110be0 0x7f881c1a18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:55746/0 (socket says 172.21.15.19:55746) 2024-05-07T02:44:16.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:16.913+0000 7f8819e00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f881c076c10 0x7f881c19d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.19:59970/0 (socket says 172.21.15.19:59970) 2024-05-07T02:44:16.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:16.913+0000 7f881a800700 1 -- 172.21.15.19:0/83707504 learned_addr learned my addr 172.21.15.19:0/83707504 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:44:16.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:16.914+0000 7f881a800700 1 -- 172.21.15.19:0/83707504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f881c076c10 msgr2=0x7f881c19d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:44:16.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:16.914+0000 7f881a800700 1 --2- 172.21.15.19:0/83707504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f881c076c10 0x7f881c19d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:16.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:16.914+0000 7f881a800700 1 -- 172.21.15.19:0/83707504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f881c077560 msgr2=0x7f881c19d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:44:16.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:16.914+0000 7f881a800700 1 --2- 172.21.15.19:0/83707504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f881c077560 0x7f881c19d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:16.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:16.914+0000 7f881a800700 1 -- 172.21.15.19:0/83707504 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f881c1a1f60 con 0x7f881c110be0 2024-05-07T02:44:16.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:16.914+0000 7f8819e00700 1 --2- 172.21.15.19:0/83707504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f881c076c10 0x7f881c19d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-07T02:44:16.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:16.914+0000 7f881a800700 1 --2- 172.21.15.19:0/83707504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f881c110be0 0x7f881c1a18c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f881400ef90 tx=0x7f881400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:44:16.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:16.914+0000 7f8812a00700 1 -- 172.21.15.19:0/83707504 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88140092e0 con 0x7f881c110be0 2024-05-07T02:44:16.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:16.914+0000 7f8820a00700 1 -- 172.21.15.19:0/83707504 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f881c1a21f0 con 0x7f881c110be0 2024-05-07T02:44:16.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:16.914+0000 7f8812a00700 1 -- 172.21.15.19:0/83707504 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8814004bb0 con 0x7f881c110be0 2024-05-07T02:44:16.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:16.914+0000 7f8812a00700 1 -- 172.21.15.19:0/83707504 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8814004280 con 0x7f881c110be0 2024-05-07T02:44:16.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:16.915+0000 7f8820a00700 1 -- 172.21.15.19:0/83707504 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f881c1a26b0 con 0x7f881c110be0 2024-05-07T02:44:16.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:16.916+0000 7f8820a00700 1 -- 172.21.15.19:0/83707504 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f881c0512c0 con 0x7f881c110be0 2024-05-07T02:44:16.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:16.918+0000 7f8812a00700 1 -- 172.21.15.19:0/83707504 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f881401b030 con 0x7f881c110be0 2024-05-07T02:44:16.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:16.920+0000 7f8812a00700 1 --2- 172.21.15.19:0/83707504 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f88000b29f0 0x7f88000b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:44:16.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:16.920+0000 7f8812a00700 1 -- 172.21.15.19:0/83707504 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9867+0+0 (secure 0 0 0) 0x7f88140e27a0 con 0x7f881c110be0 2024-05-07T02:44:16.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:16.920+0000 7f8819e00700 1 --2- 172.21.15.19:0/83707504 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f88000b29f0 0x7f88000b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:44:16.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:16.921+0000 7f8819e00700 1 --2- 172.21.15.19:0/83707504 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f88000b29f0 0x7f88000b4f00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f880c009750 tx=0x7f880c009310 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:44:16.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:16.922+0000 7f8812a00700 1 -- 172.21.15.19:0/83707504 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8814022020 con 0x7f881c110be0 2024-05-07T02:44:17.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:17.159+0000 7f8820a00700 1 -- 172.21.15.19:0/83707504 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f881c02e5f0 con 0x7f881c110be0 2024-05-07T02:44:17.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:17.982+0000 7f8812a00700 1 -- 172.21.15.19:0/83707504 <== mon.2 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v102) ==== 181+0+0 (secure 0 0 0) 0x7f8814010040 con 0x7f881c110be0 2024-05-07T02:44:17.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:17.987+0000 7f8820a00700 1 -- 172.21.15.19:0/83707504 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f88000b29f0 msgr2=0x7f88000b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:44:17.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:17.987+0000 7f8820a00700 1 --2- 172.21.15.19:0/83707504 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f88000b29f0 0x7f88000b4f00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f880c009750 tx=0x7f880c009310 comp rx=0 tx=0).stop 2024-05-07T02:44:17.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:17.988+0000 7f8820a00700 1 -- 172.21.15.19:0/83707504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f881c110be0 msgr2=0x7f881c1a18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:44:17.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:17.988+0000 7f8820a00700 1 --2- 172.21.15.19:0/83707504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f881c110be0 0x7f881c1a18c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f881400ef90 tx=0x7f881400c3a0 comp rx=0 tx=0).stop 2024-05-07T02:44:17.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:17.988+0000 7f8820a00700 1 -- 172.21.15.19:0/83707504 shutdown_connections 2024-05-07T02:44:17.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:17.988+0000 7f8820a00700 1 --2- 172.21.15.19:0/83707504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f881c110be0 0x7f881c1a18c0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:17.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:17.988+0000 7f8820a00700 1 --2- 172.21.15.19:0/83707504 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f881c076c10 0x7f881c19d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:17.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:17.988+0000 7f8820a00700 1 --2- 172.21.15.19:0/83707504 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f88000b29f0 0x7f88000b4f00 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:17.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:17.988+0000 7f8820a00700 1 --2- 172.21.15.19:0/83707504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f881c077560 0x7f881c19d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:17.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:17.988+0000 7f8820a00700 1 -- 172.21.15.19:0/83707504 >> 172.21.15.19:0/83707504 conn(0x7f881c0ffea0 msgr2=0x7f881c100590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:44:17.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:17.988+0000 7f8820a00700 1 -- 172.21.15.19:0/83707504 shutdown_connections 2024-05-07T02:44:17.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:17.989+0000 7f8820a00700 1 -- 172.21.15.19:0/83707504 wait complete. 2024-05-07T02:44:17.993 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' removed 2024-05-07T02:44:18.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:17 smithi064 ceph-mon[34699]: pgmap v387: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-05-07T02:44:18.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:17 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/83707504' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-07T02:44:18.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:17 smithi064 ceph-mon[34699]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-07T02:44:18.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:17 smithi019 ceph-mon[29781]: pgmap v387: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-05-07T02:44:18.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:17 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/83707504' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-07T02:44:18.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:17 smithi019 ceph-mon[29781]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-07T02:44:18.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:17 smithi027 ceph-mon[34398]: pgmap v387: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-05-07T02:44:18.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:17 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/83707504' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-07T02:44:18.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:17 smithi027 ceph-mon[34398]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-07T02:44:19.020 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-07T02:44:19.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:18 smithi064 ceph-mon[34699]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-07T02:44:19.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:18 smithi064 ceph-mon[34699]: osdmap e102: 12 total, 12 up, 12 in 2024-05-07T02:44:19.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:18 smithi019 ceph-mon[29781]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-07T02:44:19.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:18 smithi019 ceph-mon[29781]: osdmap e102: 12 total, 12 up, 12 in 2024-05-07T02:44:19.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:18 smithi027 ceph-mon[34398]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-07T02:44:19.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:18 smithi027 ceph-mon[34398]: osdmap e102: 12 total, 12 up, 12 in 2024-05-07T02:44:19.484 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:44:20.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:20 smithi064 ceph-mon[34699]: pgmap v389: 97 pgs: 97 active+clean; 23 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:44:20.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:20 smithi019 ceph-mon[29781]: pgmap v389: 97 pgs: 97 active+clean; 23 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:44:20.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:20 smithi027 ceph-mon[34398]: pgmap v389: 97 pgs: 97 active+clean; 23 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:44:22.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:22.012+0000 7f0c9dc00700 1 -- 172.21.15.19:0/2475062088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c98106ed0 msgr2=0x7f0c98113360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:44:22.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:22.012+0000 7f0c9dc00700 1 --2- 172.21.15.19:0/2475062088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c98106ed0 0x7f0c98113360 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0c9000b0a0 tx=0x7f0c9000b3b0 comp rx=0 tx=0).stop 2024-05-07T02:44:22.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:22.013+0000 7f0c9dc00700 1 -- 172.21.15.19:0/2475062088 shutdown_connections 2024-05-07T02:44:22.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:22.013+0000 7f0c9dc00700 1 --2- 172.21.15.19:0/2475062088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c98106ed0 0x7f0c98113360 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:22.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:22.013+0000 7f0c9dc00700 1 --2- 172.21.15.19:0/2475062088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c9810c5c0 0x7f0c9810c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:22.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:22.013+0000 7f0c9dc00700 1 --2- 172.21.15.19:0/2475062088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c98106580 0x7f0c98106990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:22.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:22.013+0000 7f0c9dc00700 1 -- 172.21.15.19:0/2475062088 >> 172.21.15.19:0/2475062088 conn(0x7f0c98102080 msgr2=0x7f0c981044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:44:22.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:22.013+0000 7f0c9dc00700 1 -- 172.21.15.19:0/2475062088 shutdown_connections 2024-05-07T02:44:22.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:22.013+0000 7f0c9dc00700 1 -- 172.21.15.19:0/2475062088 wait complete. 2024-05-07T02:44:22.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:22.014+0000 7f0c9dc00700 1 Processor -- start 2024-05-07T02:44:22.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:22.015+0000 7f0c9dc00700 1 -- start start 2024-05-07T02:44:22.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:22.015+0000 7f0c9dc00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c98106580 0x7f0c9819cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:44:22.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:22.015+0000 7f0c9dc00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c98106ed0 0x7f0c9819d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:44:22.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:22.015+0000 7f0c9dc00700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c9810c5c0 0x7f0c981a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:44:22.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:22.015+0000 7f0c9dc00700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0c98116710 con 0x7f0c98106ed0 2024-05-07T02:44:22.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:22.015+0000 7f0c9dc00700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0c98116590 con 0x7f0c98106580 2024-05-07T02:44:22.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:22.015+0000 7f0c9dc00700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0c98116890 con 0x7f0c9810c5c0 2024-05-07T02:44:22.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:22.016+0000 7f0c8e000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c98106ed0 0x7f0c9819d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:44:22.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:22.016+0000 7f0c96a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c98106580 0x7f0c9819cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:44:22.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:22.016+0000 7f0c8e000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c98106ed0 0x7f0c9819d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52826/0 (socket says 172.21.15.19:52826) 2024-05-07T02:44:22.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:22.016+0000 7f0c96a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c98106580 0x7f0c9819cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:49234/0 (socket says 172.21.15.19:49234) 2024-05-07T02:44:22.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:22.016+0000 7f0c8e000700 1 -- 172.21.15.19:0/4031851531 learned_addr learned my addr 172.21.15.19:0/4031851531 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:44:22.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:22.016+0000 7f0c97400700 1 --2- 172.21.15.19:0/4031851531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c9810c5c0 0x7f0c981a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:44:22.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:22.016+0000 7f0c96a00700 1 -- 172.21.15.19:0/4031851531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c9810c5c0 msgr2=0x7f0c981a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:44:22.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:22.016+0000 7f0c96a00700 1 --2- 172.21.15.19:0/4031851531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c9810c5c0 0x7f0c981a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:22.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:22.016+0000 7f0c96a00700 1 -- 172.21.15.19:0/4031851531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c98106ed0 msgr2=0x7f0c9819d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:44:22.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:22.016+0000 7f0c96a00700 1 --2- 172.21.15.19:0/4031851531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c98106ed0 0x7f0c9819d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:22.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:22.016+0000 7f0c96a00700 1 -- 172.21.15.19:0/4031851531 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c981a1eb0 con 0x7f0c98106580 2024-05-07T02:44:22.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:22.016+0000 7f0c96a00700 1 --2- 172.21.15.19:0/4031851531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c98106580 0x7f0c9819cfe0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0c8800e9c0 tx=0x7f0c8800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:44:22.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:22.017+0000 7f0c8fe00700 1 -- 172.21.15.19:0/4031851531 <== mon.2 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c8800c9c0 con 0x7f0c98106580 2024-05-07T02:44:22.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:22.017+0000 7f0c8fe00700 1 -- 172.21.15.19:0/4031851531 <== mon.2 v2:172.21.15.27:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c8800cb60 con 0x7f0c98106580 2024-05-07T02:44:22.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:22.017+0000 7f0c9dc00700 1 -- 172.21.15.19:0/4031851531 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c981a21a0 con 0x7f0c98106580 2024-05-07T02:44:22.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:22.017+0000 7f0c8fe00700 1 -- 172.21.15.19:0/4031851531 <== mon.2 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c88018680 con 0x7f0c98106580 2024-05-07T02:44:22.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:22.017+0000 7f0c9dc00700 1 -- 172.21.15.19:0/4031851531 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c981a9ac0 con 0x7f0c98106580 2024-05-07T02:44:22.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:22.018+0000 7f0c9dc00700 1 -- 172.21.15.19:0/4031851531 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c980512c0 con 0x7f0c98106580 2024-05-07T02:44:22.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:22.021+0000 7f0c8fe00700 1 -- 172.21.15.19:0/4031851531 <== mon.2 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0c88010430 con 0x7f0c98106580 2024-05-07T02:44:22.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:22.022+0000 7f0c8fe00700 1 --2- 172.21.15.19:0/4031851531 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f0c7c0b2a70 0x7f0c7c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:44:22.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:22.022+0000 7f0c8fe00700 1 -- 172.21.15.19:0/4031851531 <== mon.2 v2:172.21.15.27:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9313+0+0 (secure 0 0 0) 0x7f0c880e2ac0 con 0x7f0c98106580 2024-05-07T02:44:22.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:22.023+0000 7f0c8e000700 1 --2- 172.21.15.19:0/4031851531 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f0c7c0b2a70 0x7f0c7c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:44:22.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:22.023+0000 7f0c8e000700 1 --2- 172.21.15.19:0/4031851531 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f0c7c0b2a70 0x7f0c7c0b4f80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0c98103c00 tx=0x7f0c80009260 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:44:22.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:22.027+0000 7f0c8fe00700 1 -- 172.21.15.19:0/4031851531 <== mon.2 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0c880aa120 con 0x7f0c98106580 2024-05-07T02:44:22.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:22 smithi019 ceph-mon[29781]: pgmap v390: 97 pgs: 97 active+clean; 23 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:44:22.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:22.264+0000 7f0c9dc00700 1 -- 172.21.15.19:0/4031851531 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0c9802e5f0 con 0x7f0c98106580 2024-05-07T02:44:22.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:22 smithi064 ceph-mon[34699]: pgmap v390: 97 pgs: 97 active+clean; 23 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:44:22.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:22 smithi027 ceph-mon[34398]: pgmap v390: 97 pgs: 97 active+clean; 23 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:44:23.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:23.024+0000 7f0c8fe00700 1 -- 172.21.15.19:0/4031851531 <== mon.2 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v103) ==== 169+0+0 (secure 0 0 0) 0x7f0c880b21a0 con 0x7f0c98106580 2024-05-07T02:44:23.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:23.028+0000 7f0c9dc00700 1 -- 172.21.15.19:0/4031851531 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f0c7c0b2a70 msgr2=0x7f0c7c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:44:23.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:23.028+0000 7f0c9dc00700 1 --2- 172.21.15.19:0/4031851531 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f0c7c0b2a70 0x7f0c7c0b4f80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0c98103c00 tx=0x7f0c80009260 comp rx=0 tx=0).stop 2024-05-07T02:44:23.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:23.029+0000 7f0c9dc00700 1 -- 172.21.15.19:0/4031851531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c98106580 msgr2=0x7f0c9819cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:44:23.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:23.029+0000 7f0c9dc00700 1 --2- 172.21.15.19:0/4031851531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c98106580 0x7f0c9819cfe0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0c8800e9c0 tx=0x7f0c8800ecd0 comp rx=0 tx=0).stop 2024-05-07T02:44:23.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:23.029+0000 7f0c9dc00700 1 -- 172.21.15.19:0/4031851531 shutdown_connections 2024-05-07T02:44:23.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:23.029+0000 7f0c9dc00700 1 --2- 172.21.15.19:0/4031851531 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c9810c5c0 0x7f0c981a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:23.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:23.029+0000 7f0c9dc00700 1 --2- 172.21.15.19:0/4031851531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c98106580 0x7f0c9819cfe0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:23.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:23.029+0000 7f0c9dc00700 1 --2- 172.21.15.19:0/4031851531 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f0c7c0b2a70 0x7f0c7c0b4f80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:23.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:23.029+0000 7f0c9dc00700 1 --2- 172.21.15.19:0/4031851531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c98106ed0 0x7f0c9819d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:23.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:23.029+0000 7f0c9dc00700 1 -- 172.21.15.19:0/4031851531 >> 172.21.15.19:0/4031851531 conn(0x7f0c98102080 msgr2=0x7f0c98104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:44:23.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:23.030+0000 7f0c9dc00700 1 -- 172.21.15.19:0/4031851531 shutdown_connections 2024-05-07T02:44:23.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:23.030+0000 7f0c9dc00700 1 -- 172.21.15.19:0/4031851531 wait complete. 2024-05-07T02:44:23.032 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' removed 2024-05-07T02:44:23.342 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:23 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/4031851531' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-07T02:44:23.342 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:23 smithi019 ceph-mon[29781]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-07T02:44:23.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:23 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/4031851531' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-07T02:44:23.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:23 smithi064 ceph-mon[34699]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-07T02:44:23.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:23 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/4031851531' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-07T02:44:23.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:23 smithi027 ceph-mon[34398]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-07T02:44:24.045 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-05-07T02:44:24.292 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:24 smithi019 ceph-mon[29781]: pgmap v391: 97 pgs: 97 active+clean; 23 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:44:24.292 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:24 smithi019 ceph-mon[29781]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-07T02:44:24.293 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:24 smithi019 ceph-mon[29781]: osdmap e103: 12 total, 12 up, 12 in 2024-05-07T02:44:24.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:24 smithi064 ceph-mon[34699]: pgmap v391: 97 pgs: 97 active+clean; 23 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:44:24.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:24 smithi064 ceph-mon[34699]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-07T02:44:24.360 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:24 smithi064 ceph-mon[34699]: osdmap e103: 12 total, 12 up, 12 in 2024-05-07T02:44:24.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:24 smithi027 ceph-mon[34398]: pgmap v391: 97 pgs: 97 active+clean; 23 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:44:24.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:24 smithi027 ceph-mon[34398]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-07T02:44:24.458 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:24 smithi027 ceph-mon[34398]: osdmap e103: 12 total, 12 up, 12 in 2024-05-07T02:44:24.486 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/mon.a/config 2024-05-07T02:44:26.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:26 smithi064 ceph-mon[34699]: pgmap v393: 33 pgs: 33 active+clean; 23 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:44:26.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:26 smithi019 ceph-mon[29781]: pgmap v393: 33 pgs: 33 active+clean; 23 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:44:26.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:26 smithi027 ceph-mon[34398]: pgmap v393: 33 pgs: 33 active+clean; 23 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:44:27.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:27.108+0000 7f0b25000700 1 -- 172.21.15.19:0/2630343582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b20076430 msgr2=0x7f0b201104f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:44:27.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:27.108+0000 7f0b25000700 1 --2- 172.21.15.19:0/2630343582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b20076430 0x7f0b201104f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0b08009990 tx=0x7f0b08009ca0 comp rx=0 tx=0).stop 2024-05-07T02:44:27.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:27.108+0000 7f0b25000700 1 -- 172.21.15.19:0/2630343582 shutdown_connections 2024-05-07T02:44:27.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:27.108+0000 7f0b25000700 1 --2- 172.21.15.19:0/2630343582 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b20110a30 0x7f0b20112e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:27.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:27.108+0000 7f0b25000700 1 --2- 172.21.15.19:0/2630343582 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b20075ae0 0x7f0b20075ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:27.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:27.108+0000 7f0b25000700 1 --2- 172.21.15.19:0/2630343582 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b20076430 0x7f0b201104f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:27.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:27.108+0000 7f0b25000700 1 -- 172.21.15.19:0/2630343582 >> 172.21.15.19:0/2630343582 conn(0x7f0b20079d10 msgr2=0x7f0b2007a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:44:27.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:27.109+0000 7f0b25000700 1 -- 172.21.15.19:0/2630343582 shutdown_connections 2024-05-07T02:44:27.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:27.109+0000 7f0b25000700 1 -- 172.21.15.19:0/2630343582 wait complete. 2024-05-07T02:44:27.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:27.110+0000 7f0b25000700 1 Processor -- start 2024-05-07T02:44:27.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:27.110+0000 7f0b25000700 1 -- start start 2024-05-07T02:44:27.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:27.111+0000 7f0b25000700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b20075ae0 0x7f0b2019cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:44:27.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:27.111+0000 7f0b25000700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b20076430 0x7f0b2019d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:44:27.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:27.111+0000 7f0b25000700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b20110a30 0x7f0b201a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:44:27.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:27.111+0000 7f0b25000700 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0b20116500 con 0x7f0b20076430 2024-05-07T02:44:27.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:27.111+0000 7f0b25000700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 -- 0x7f0b20116380 con 0x7f0b20075ae0 2024-05-07T02:44:27.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:27.111+0000 7f0b25000700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0b20116680 con 0x7f0b20110a30 2024-05-07T02:44:27.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:27.111+0000 7f0b16a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b20076430 0x7f0b2019d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:44:27.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:27.111+0000 7f0b1f400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b20075ae0 0x7f0b2019cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:44:27.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:27.111+0000 7f0b16a00700 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b20076430 0x7f0b2019d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52852/0 (socket says 172.21.15.19:52852) 2024-05-07T02:44:27.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:27.111+0000 7f0b16a00700 1 -- 172.21.15.19:0/391428912 learned_addr learned my addr 172.21.15.19:0/391428912 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-05-07T02:44:27.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:27.111+0000 7f0b1fe00700 1 --2- 172.21.15.19:0/391428912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b20110a30 0x7f0b201a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:44:27.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:27.111+0000 7f0b1f400700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b20075ae0 0x7f0b2019cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.19:49250/0 (socket says 172.21.15.19:49250) 2024-05-07T02:44:27.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:27.112+0000 7f0b1fe00700 1 -- 172.21.15.19:0/391428912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b20075ae0 msgr2=0x7f0b2019cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:44:27.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:27.112+0000 7f0b1fe00700 1 --2- 172.21.15.19:0/391428912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b20075ae0 0x7f0b2019cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:27.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:27.112+0000 7f0b1fe00700 1 -- 172.21.15.19:0/391428912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b20076430 msgr2=0x7f0b2019d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:44:27.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:27.112+0000 7f0b1fe00700 1 --2- 172.21.15.19:0/391428912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b20076430 0x7f0b2019d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:27.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:27.112+0000 7f0b1fe00700 1 -- 172.21.15.19:0/391428912 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b201a1dc0 con 0x7f0b20110a30 2024-05-07T02:44:27.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:27.112+0000 7f0b1fe00700 1 --2- 172.21.15.19:0/391428912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b20110a30 0x7f0b201a1720 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0b1800efc0 tx=0x7f0b1800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:44:27.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:27.112+0000 7f0b1cc00700 1 -- 172.21.15.19:0/391428912 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b180092e0 con 0x7f0b20110a30 2024-05-07T02:44:27.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:27.112+0000 7f0b1cc00700 1 -- 172.21.15.19:0/391428912 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b18004bb0 con 0x7f0b20110a30 2024-05-07T02:44:27.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:27.112+0000 7f0b25000700 1 -- 172.21.15.19:0/391428912 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b201a2050 con 0x7f0b20110a30 2024-05-07T02:44:27.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:27.112+0000 7f0b1cc00700 1 -- 172.21.15.19:0/391428912 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18004020 con 0x7f0b20110a30 2024-05-07T02:44:27.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:27.113+0000 7f0b25000700 1 -- 172.21.15.19:0/391428912 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b201a99d0 con 0x7f0b20110a30 2024-05-07T02:44:27.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:27.114+0000 7f0b25000700 1 -- 172.21.15.19:0/391428912 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b20077220 con 0x7f0b20110a30 2024-05-07T02:44:27.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:27.116+0000 7f0b1cc00700 1 -- 172.21.15.19:0/391428912 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0b1801b030 con 0x7f0b20110a30 2024-05-07T02:44:27.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:27.118+0000 7f0b1cc00700 1 --2- 172.21.15.19:0/391428912 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f0b040b2a70 0x7f0b040b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-07T02:44:27.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:27.118+0000 7f0b1cc00700 1 -- 172.21.15.19:0/391428912 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8754+0+0 (secure 0 0 0) 0x7f0b180e2180 con 0x7f0b20110a30 2024-05-07T02:44:27.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:27.118+0000 7f0b1f400700 1 --2- 172.21.15.19:0/391428912 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f0b040b2a70 0x7f0b040b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-07T02:44:27.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:27.119+0000 7f0b1f400700 1 --2- 172.21.15.19:0/391428912 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f0b040b2a70 0x7f0b040b4f80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0b10007be0 tx=0x7f0b100073d0 comp rx=0 tx=0).ready entity=mgr.25234 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-07T02:44:27.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:27.123+0000 7f0b1cc00700 1 -- 172.21.15.19:0/391428912 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0b18022020 con 0x7f0b20110a30 2024-05-07T02:44:27.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:27.365+0000 7f0b25000700 1 -- 172.21.15.19:0/391428912 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0b2019e250 con 0x7f0b20110a30 2024-05-07T02:44:28.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:28.037+0000 7f0b1cc00700 1 -- 172.21.15.19:0/391428912 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v104) ==== 178+0+0 (secure 0 0 0) 0x7f0b18010040 con 0x7f0b20110a30 2024-05-07T02:44:28.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:28.045+0000 7f0b25000700 1 -- 172.21.15.19:0/391428912 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f0b040b2a70 msgr2=0x7f0b040b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:44:28.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:28.045+0000 7f0b25000700 1 --2- 172.21.15.19:0/391428912 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f0b040b2a70 0x7f0b040b4f80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0b10007be0 tx=0x7f0b100073d0 comp rx=0 tx=0).stop 2024-05-07T02:44:28.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:28.045+0000 7f0b25000700 1 -- 172.21.15.19:0/391428912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b20110a30 msgr2=0x7f0b201a1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-07T02:44:28.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:28.045+0000 7f0b25000700 1 --2- 172.21.15.19:0/391428912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b20110a30 0x7f0b201a1720 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0b1800efc0 tx=0x7f0b1800c3a0 comp rx=0 tx=0).stop 2024-05-07T02:44:28.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:28.046+0000 7f0b25000700 1 -- 172.21.15.19:0/391428912 shutdown_connections 2024-05-07T02:44:28.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:28.046+0000 7f0b25000700 1 --2- 172.21.15.19:0/391428912 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0b20110a30 0x7f0b201a1720 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:28.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:28.046+0000 7f0b25000700 1 --2- 172.21.15.19:0/391428912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b20075ae0 0x7f0b2019cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:28.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:28.046+0000 7f0b25000700 1 --2- 172.21.15.19:0/391428912 >> [v2:172.21.15.19:6800/3265825478,v1:172.21.15.19:6801/3265825478] conn(0x7f0b040b2a70 0x7f0b040b4f80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:28.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:28.046+0000 7f0b25000700 1 --2- 172.21.15.19:0/391428912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b20076430 0x7f0b2019d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-07T02:44:28.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:28.046+0000 7f0b25000700 1 -- 172.21.15.19:0/391428912 >> 172.21.15.19:0/391428912 conn(0x7f0b20079d10 msgr2=0x7f0b20100fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-07T02:44:28.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:28.047+0000 7f0b25000700 1 -- 172.21.15.19:0/391428912 shutdown_connections 2024-05-07T02:44:28.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-05-07T02:44:28.049+0000 7f0b25000700 1 -- 172.21.15.19:0/391428912 wait complete. 2024-05-07T02:44:28.054 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data_ec' removed 2024-05-07T02:44:28.361 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:28 smithi064 ceph-mon[34699]: pgmap v394: 33 pgs: 33 active+clean; 23 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:44:28.362 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:28 smithi064 ceph-mon[34699]: from='client.? 172.21.15.19:0/391428912' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-07T02:44:28.362 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:28 smithi064 ceph-mon[34699]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-07T02:44:28.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:28 smithi019 ceph-mon[29781]: pgmap v394: 33 pgs: 33 active+clean; 23 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:44:28.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:28 smithi019 ceph-mon[29781]: from='client.? 172.21.15.19:0/391428912' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-07T02:44:28.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:28 smithi019 ceph-mon[29781]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-07T02:44:28.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:28 smithi027 ceph-mon[34398]: pgmap v394: 33 pgs: 33 active+clean; 23 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:44:28.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:28 smithi027 ceph-mon[34398]: from='client.? 172.21.15.19:0/391428912' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-07T02:44:28.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:28 smithi027 ceph-mon[34398]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-07T02:44:29.139 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-07T02:44:29.217 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-07T02:44:29.253 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-07T02:44:29.289 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-07T02:44:29.289 DEBUG:teuthology.orchestra.run.smithi019:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-07T02:44:29.345 DEBUG:teuthology.orchestra.run.smithi027:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-07T02:44:29.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:29 smithi064 ceph-mon[34699]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-05-07T02:44:29.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:29 smithi064 ceph-mon[34699]: osdmap e104: 12 total, 12 up, 12 in 2024-05-07T02:44:29.363 DEBUG:teuthology.orchestra.run.smithi064:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-07T02:44:29.381 INFO:tasks.cephadm:Stopping all daemons... 2024-05-07T02:44:29.381 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-07T02:44:29.381 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mon.a 2024-05-07T02:44:29.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:29 smithi019 ceph-mon[29781]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-05-07T02:44:29.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:29 smithi019 ceph-mon[29781]: osdmap e104: 12 total, 12 up, 12 in 2024-05-07T02:44:29.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:29 smithi027 ceph-mon[34398]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-05-07T02:44:29.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:29 smithi027 ceph-mon[34398]: osdmap e104: 12 total, 12 up, 12 in 2024-05-07T02:44:29.896 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:29 smithi019 systemd[1]: Stopping Ceph mon.a for 9ff84c16-0c17-11ef-bc97-c7b262605968... 2024-05-07T02:44:30.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:30 smithi064 ceph-mon[34699]: pgmap v396: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:44:30.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:30 smithi019 ceph-mon[29781]: pgmap v396: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:44:30.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:30 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mon-a[29772]: 2024-05-07T02:44:30.129+0000 7f9cf0400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-07T02:44:30.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:30 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mon-a[29772]: 2024-05-07T02:44:30.129+0000 7f9cf0400700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-07T02:44:30.396 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:30 smithi019 podman[113775]: 2024-05-07 02:44:30.295455302 +0000 UTC m=+0.476528945 container died 6ca858b3ad851c88261b42f04e44b87c84c0d6606971b4d88acda280161d24c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.build-date=20240430, ceph=True, GIT_BRANCH=HEAD) 2024-05-07T02:44:30.457 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:30 smithi027 ceph-mon[34398]: pgmap v396: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-07T02:44:30.830 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:30 smithi019 podman[113775]: 2024-05-07 02:44:30.821055977 +0000 UTC m=+1.002129630 container cleanup 6ca858b3ad851c88261b42f04e44b87c84c0d6606971b4d88acda280161d24c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True) 2024-05-07T02:44:30.830 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:30 smithi019 bash[113775]: ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mon-a 2024-05-07T02:44:31.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:May 07 02:44:31 smithi019 podman[113790]: 2024-05-07 02:44:31.048393121 +0000 UTC m=+0.738795627 container remove 6ca858b3ad851c88261b42f04e44b87c84c0d6606971b4d88acda280161d24c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mon-a, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-07T02:44:31.577 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mon.a.service' 2024-05-07T02:44:31.751 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-07T02:44:31.751 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-07T02:44:31.751 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-07T02:44:31.752 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mon.b 2024-05-07T02:44:32.207 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:31 smithi027 systemd[1]: Stopping Ceph mon.b for 9ff84c16-0c17-11ef-bc97-c7b262605968... 2024-05-07T02:44:32.587 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:32 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mon-b[34394]: 2024-05-07T02:44:32.211+0000 7f46c7143700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-07T02:44:32.588 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:32 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mon-b[34394]: 2024-05-07T02:44:32.211+0000 7f46c7143700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-07T02:44:32.588 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:32 smithi027 podman[72185]: 2024-05-07 02:44:32.283034648 +0000 UTC m=+0.322769093 container died 8527eb176b97e0c657a1d7b8d84c9aa482539a672a167da8ce3dda282aa64082 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mon-b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-07T02:44:32.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:32 smithi027 podman[72185]: 2024-05-07 02:44:32.68685141 +0000 UTC m=+0.726585847 container cleanup 8527eb176b97e0c657a1d7b8d84c9aa482539a672a167da8ce3dda282aa64082 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mon-b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a) 2024-05-07T02:44:32.957 INFO:journalctl@ceph.mon.b.smithi027.stdout:May 07 02:44:32 smithi027 bash[72185]: ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mon-b 2024-05-07T02:44:33.116 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mon.b.service' 2024-05-07T02:44:33.165 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-07T02:44:33.165 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-07T02:44:33.166 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-07T02:44:33.166 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mon.c 2024-05-07T02:44:33.494 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:33 smithi064 systemd[1]: Stopping Ceph mon.c for 9ff84c16-0c17-11ef-bc97-c7b262605968... 2024-05-07T02:44:33.776 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:33 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mon-c[34695]: 2024-05-07T02:44:33.520+0000 7f12256da700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-07T02:44:33.776 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:33 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mon-c[34695]: 2024-05-07T02:44:33.520+0000 7f12256da700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-07T02:44:34.080 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:33 smithi064 podman[72851]: 2024-05-07 02:44:33.776178723 +0000 UTC m=+0.409493317 container died 3f1234cd50e5c679770ad811ecad3e5e64a6232ae6b45064b05f317c81b22bcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430) 2024-05-07T02:44:34.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:34 smithi064 podman[72851]: 2024-05-07 02:44:34.179875987 +0000 UTC m=+0.813190584 container cleanup 3f1234cd50e5c679770ad811ecad3e5e64a6232ae6b45064b05f317c81b22bcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mon-c, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-07T02:44:34.359 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 07 02:44:34 smithi064 bash[72851]: ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mon-c 2024-05-07T02:44:34.612 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mon.c.service' 2024-05-07T02:44:34.657 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-07T02:44:34.657 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-07T02:44:34.657 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-07T02:44:34.657 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mgr.x 2024-05-07T02:44:35.145 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:44:34 smithi019 systemd[1]: Stopping Ceph mgr.x for 9ff84c16-0c17-11ef-bc97-c7b262605968... 2024-05-07T02:44:35.646 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:44:35 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:44:35.240+0000 7fa7dd800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-07T02:44:35.646 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:44:35 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x[30011]: 2024-05-07T02:44:35.240+0000 7fa7dd800700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-05-07T02:44:35.646 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:44:35 smithi019 podman[113993]: 2024-05-07 02:44:35.345685502 +0000 UTC m=+0.226922248 container died 4c3f8d297faab2f2313221454c39d5782efabd0930beca29d0ee3c3c00fc61fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-07T02:44:35.986 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:44:35 smithi019 podman[113993]: 2024-05-07 02:44:35.986189392 +0000 UTC m=+0.867426137 container cleanup 4c3f8d297faab2f2313221454c39d5782efabd0930beca29d0ee3c3c00fc61fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x, org.label-schema.build-date=20240430, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a) 2024-05-07T02:44:36.395 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:44:35 smithi019 bash[113993]: ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x 2024-05-07T02:44:36.396 INFO:journalctl@ceph.mgr.x.smithi019.stdout:May 07 02:44:36 smithi019 podman[114007]: 2024-05-07 02:44:36.213165446 +0000 UTC m=+0.851659250 container remove 4c3f8d297faab2f2313221454c39d5782efabd0930beca29d0ee3c3c00fc61fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-x, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-07T02:44:36.676 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mgr.x.service' 2024-05-07T02:44:36.786 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-07T02:44:36.786 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-07T02:44:36.786 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-07T02:44:36.786 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mgr.y 2024-05-07T02:44:37.147 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:44:36 smithi027 systemd[1]: Stopping Ceph mgr.y for 9ff84c16-0c17-11ef-bc97-c7b262605968... 2024-05-07T02:44:37.451 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:44:37 smithi027 podman[72338]: 2024-05-07 02:44:37.147680497 +0000 UTC m=+0.162518546 container died ae5c7d337e67e0a843e5f745994c0da9483ca858c81053ef236167249e7ad1cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0) 2024-05-07T02:44:37.707 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:44:37 smithi027 podman[72338]: 2024-05-07 02:44:37.567084408 +0000 UTC m=+0.581922449 container cleanup ae5c7d337e67e0a843e5f745994c0da9483ca858c81053ef236167249e7ad1cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0) 2024-05-07T02:44:37.707 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:44:37 smithi027 bash[72338]: ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y 2024-05-07T02:44:38.000 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mgr.y.service' 2024-05-07T02:44:38.021 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:44:37 smithi027 podman[72352]: 2024-05-07 02:44:37.76899814 +0000 UTC m=+0.624403819 container remove ae5c7d337e67e0a843e5f745994c0da9483ca858c81053ef236167249e7ad1cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-07T02:44:38.022 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:44:37 smithi027 systemd[1]: ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-05-07T02:44:38.022 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:44:37 smithi027 systemd[1]: ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mgr.y.service: Failed with result 'exit-code'. 2024-05-07T02:44:38.022 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:44:37 smithi027 systemd[1]: Stopped Ceph mgr.y for 9ff84c16-0c17-11ef-bc97-c7b262605968. 2024-05-07T02:44:38.022 INFO:journalctl@ceph.mgr.y.smithi027.stdout:May 07 02:44:37 smithi027 systemd[1]: ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mgr.y.service: Consumed 23.082s CPU time. 2024-05-07T02:44:38.034 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-07T02:44:38.035 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-07T02:44:38.035 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-07T02:44:38.035 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mgr.z 2024-05-07T02:44:38.359 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:44:38 smithi064 systemd[1]: Stopping Ceph mgr.z for 9ff84c16-0c17-11ef-bc97-c7b262605968... 2024-05-07T02:44:38.757 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:44:38 smithi064 podman[72972]: 2024-05-07 02:44:38.390618346 +0000 UTC m=+0.154247047 container died d79d6701e1613fcaaea532bd15d03b769a00e2d42f2746a53a8e3a98e76a713a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True) 2024-05-07T02:44:39.109 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:44:38 smithi064 podman[72972]: 2024-05-07 02:44:38.860489941 +0000 UTC m=+0.624118631 container cleanup d79d6701e1613fcaaea532bd15d03b769a00e2d42f2746a53a8e3a98e76a713a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-07T02:44:39.109 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 07 02:44:38 smithi064 bash[72972]: ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mgr-z 2024-05-07T02:44:39.295 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mgr.z.service' 2024-05-07T02:44:39.329 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-07T02:44:39.329 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-07T02:44:39.330 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-07T02:44:39.330 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@osd.0 2024-05-07T02:44:39.895 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 07 02:44:39 smithi019 systemd[1]: Stopping Ceph osd.0 for 9ff84c16-0c17-11ef-bc97-c7b262605968... 2024-05-07T02:44:40.396 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 07 02:44:39 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-0[44288]: 2024-05-07T02:44:39.965+0000 7fa54aa00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-07T02:44:40.396 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 07 02:44:39 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-0[44288]: 2024-05-07T02:44:39.965+0000 7fa54aa00700 -1 osd.0 104 *** Got signal Terminated *** 2024-05-07T02:44:40.396 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 07 02:44:39 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-0[44288]: 2024-05-07T02:44:39.965+0000 7fa54aa00700 -1 osd.0 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-07T02:44:45.895 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 07 02:44:45 smithi019 podman[114123]: 2024-05-07 02:44:45.508526604 +0000 UTC m=+5.707954106 container died 990183035c6c016f9b032524f7892727cdf4d6b3537f4ce800811d90e72fb051 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True) 2024-05-07T02:44:46.395 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 07 02:44:46 smithi019 podman[114123]: 2024-05-07 02:44:46.133652823 +0000 UTC m=+6.333080353 container cleanup 990183035c6c016f9b032524f7892727cdf4d6b3537f4ce800811d90e72fb051 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-0, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-07T02:44:46.395 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 07 02:44:46 smithi019 bash[114123]: ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-0 2024-05-07T02:44:46.895 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 07 02:44:46 smithi019 podman[114145]: 2024-05-07 02:44:46.596241938 +0000 UTC m=+1.078647376 container remove 990183035c6c016f9b032524f7892727cdf4d6b3537f4ce800811d90e72fb051 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-0, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-07T02:44:47.896 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 07 02:44:47 smithi019 podman[114231]: 2024-05-07 02:44:47.409566392 +0000 UTC m=+0.041375250 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-07T02:44:47.897 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 07 02:44:47 smithi019 podman[114231]: 2024-05-07 02:44:47.572428211 +0000 UTC m=+0.204237119 container create 9eacdb2e59090e723d4980480f13838bffb2af69a140c889ee74b6c2326eba78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-07T02:44:48.396 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 07 02:44:47 smithi019 podman[114231]: 2024-05-07 02:44:47.986151353 +0000 UTC m=+0.617960247 container init 9eacdb2e59090e723d4980480f13838bffb2af69a140c889ee74b6c2326eba78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-0-deactivate, ceph=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-07T02:44:48.396 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 07 02:44:47 smithi019 podman[114231]: 2024-05-07 02:44:47.999469727 +0000 UTC m=+0.631278625 container start 9eacdb2e59090e723d4980480f13838bffb2af69a140c889ee74b6c2326eba78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-07T02:44:48.396 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 07 02:44:48 smithi019 podman[114231]: 2024-05-07 02:44:48.109342634 +0000 UTC m=+0.741151526 container attach 9eacdb2e59090e723d4980480f13838bffb2af69a140c889ee74b6c2326eba78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0) 2024-05-07T02:44:48.895 INFO:journalctl@ceph.osd.0.smithi019.stdout:May 07 02:44:48 smithi019 podman[114231]: 2024-05-07 02:44:48.478418654 +0000 UTC m=+1.110227548 container died 9eacdb2e59090e723d4980480f13838bffb2af69a140c889ee74b6c2326eba78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-07T02:44:49.432 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@osd.0.service' 2024-05-07T02:44:49.551 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-07T02:44:49.551 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-07T02:44:49.551 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-07T02:44:49.551 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@osd.1 2024-05-07T02:44:49.915 INFO:journalctl@ceph.osd.1.smithi027.stdout:May 07 02:44:49 smithi027 systemd[1]: Stopping Ceph osd.1 for 9ff84c16-0c17-11ef-bc97-c7b262605968... 2024-05-07T02:44:50.207 INFO:journalctl@ceph.osd.1.smithi027.stdout:May 07 02:44:49 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-1[39018]: 2024-05-07T02:44:49.913+0000 7f817b1cb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-07T02:44:50.207 INFO:journalctl@ceph.osd.1.smithi027.stdout:May 07 02:44:49 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-1[39018]: 2024-05-07T02:44:49.913+0000 7f817b1cb700 -1 osd.1 104 *** Got signal Terminated *** 2024-05-07T02:44:50.207 INFO:journalctl@ceph.osd.1.smithi027.stdout:May 07 02:44:49 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-1[39018]: 2024-05-07T02:44:49.913+0000 7f817b1cb700 -1 osd.1 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-07T02:44:55.635 INFO:journalctl@ceph.osd.1.smithi027.stdout:May 07 02:44:55 smithi027 podman[72474]: 2024-05-07 02:44:55.373345795 +0000 UTC m=+5.618777598 container died 3cb7380e3695d3315d1ce0c91b6037548adfdbd012fa17082e10f3151ac00562 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-1, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-07T02:44:55.957 INFO:journalctl@ceph.osd.1.smithi027.stdout:May 07 02:44:55 smithi027 podman[72474]: 2024-05-07 02:44:55.731293073 +0000 UTC m=+5.976724875 container cleanup 3cb7380e3695d3315d1ce0c91b6037548adfdbd012fa17082e10f3151ac00562 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-1, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-07T02:44:55.957 INFO:journalctl@ceph.osd.1.smithi027.stdout:May 07 02:44:55 smithi027 bash[72474]: ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-1 2024-05-07T02:44:56.360 INFO:journalctl@ceph.osd.1.smithi027.stdout:May 07 02:44:56 smithi027 podman[72491]: 2024-05-07 02:44:56.025104888 +0000 UTC m=+0.648465891 container remove 3cb7380e3695d3315d1ce0c91b6037548adfdbd012fa17082e10f3151ac00562 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-07T02:44:56.631 INFO:journalctl@ceph.osd.1.smithi027.stdout:May 07 02:44:56 smithi027 podman[72583]: 2024-05-07 02:44:56.262848495 +0000 UTC m=+0.018649793 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-07T02:44:56.631 INFO:journalctl@ceph.osd.1.smithi027.stdout:May 07 02:44:56 smithi027 podman[72583]: 2024-05-07 02:44:56.380011504 +0000 UTC m=+0.135832571 container create 340bf2f42e4ff12defea4bcadd27ca12d11365a0abf258c6b86e32dc56464ed1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-07T02:44:56.957 INFO:journalctl@ceph.osd.1.smithi027.stdout:May 07 02:44:56 smithi027 podman[72583]: 2024-05-07 02:44:56.755063036 +0000 UTC m=+0.510864313 container init 340bf2f42e4ff12defea4bcadd27ca12d11365a0abf258c6b86e32dc56464ed1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-07T02:44:56.958 INFO:journalctl@ceph.osd.1.smithi027.stdout:May 07 02:44:56 smithi027 podman[72583]: 2024-05-07 02:44:56.758254273 +0000 UTC m=+0.514055549 container start 340bf2f42e4ff12defea4bcadd27ca12d11365a0abf258c6b86e32dc56464ed1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=) 2024-05-07T02:44:56.958 INFO:journalctl@ceph.osd.1.smithi027.stdout:May 07 02:44:56 smithi027 podman[72583]: 2024-05-07 02:44:56.846948298 +0000 UTC m=+0.602749578 container attach 340bf2f42e4ff12defea4bcadd27ca12d11365a0abf258c6b86e32dc56464ed1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-05-07T02:44:57.367 INFO:journalctl@ceph.osd.1.smithi027.stdout:May 07 02:44:57 smithi027 podman[72583]: 2024-05-07 02:44:57.062103708 +0000 UTC m=+0.817904993 container died 340bf2f42e4ff12defea4bcadd27ca12d11365a0abf258c6b86e32dc56464ed1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-1-deactivate, ceph=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-07T02:44:57.687 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@osd.1.service' 2024-05-07T02:44:57.726 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-07T02:44:57.727 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-07T02:44:57.727 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-07T02:44:57.727 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@osd.2 2024-05-07T02:44:58.056 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 07 02:44:57 smithi064 systemd[1]: Stopping Ceph osd.2 for 9ff84c16-0c17-11ef-bc97-c7b262605968... 2024-05-07T02:44:58.338 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 07 02:44:58 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-2[39873]: 2024-05-07T02:44:58.055+0000 7fbd5a9a2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-07T02:44:58.338 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 07 02:44:58 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-2[39873]: 2024-05-07T02:44:58.055+0000 7fbd5a9a2700 -1 osd.2 104 *** Got signal Terminated *** 2024-05-07T02:44:58.339 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 07 02:44:58 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-2[39873]: 2024-05-07T02:44:58.055+0000 7fbd5a9a2700 -1 osd.2 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-07T02:45:03.859 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 07 02:45:03 smithi064 podman[73095]: 2024-05-07 02:45:03.56590113 +0000 UTC m=+5.632804842 container died 29b568914917afbc8bfecf75e3e4ea5bbc9e9081d3471c8ddd7e3da9f69c40b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-07T02:45:04.249 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 07 02:45:03 smithi064 podman[73095]: 2024-05-07 02:45:03.965284404 +0000 UTC m=+6.032188118 container cleanup 29b568914917afbc8bfecf75e3e4ea5bbc9e9081d3471c8ddd7e3da9f69c40b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS) 2024-05-07T02:45:04.250 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 07 02:45:03 smithi064 bash[73095]: ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-2 2024-05-07T02:45:04.548 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 07 02:45:04 smithi064 podman[73124]: 2024-05-07 02:45:04.24990695 +0000 UTC m=+0.674946240 container remove 29b568914917afbc8bfecf75e3e4ea5bbc9e9081d3471c8ddd7e3da9f69c40b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-2, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430) 2024-05-07T02:45:04.804 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 07 02:45:04 smithi064 podman[73217]: 2024-05-07 02:45:04.4519457 +0000 UTC m=+0.018945499 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-07T02:45:04.804 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 07 02:45:04 smithi064 podman[73217]: 2024-05-07 02:45:04.576292676 +0000 UTC m=+0.143292494 container create d6812d8c95c6162598f63428290326f7c955b124d2d13625655abfdd6c5ad3f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-2-deactivate, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240430, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-07T02:45:05.109 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 07 02:45:04 smithi064 podman[73217]: 2024-05-07 02:45:04.803970183 +0000 UTC m=+0.370969976 container init d6812d8c95c6162598f63428290326f7c955b124d2d13625655abfdd6c5ad3f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-07T02:45:05.109 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 07 02:45:04 smithi064 podman[73217]: 2024-05-07 02:45:04.807834152 +0000 UTC m=+0.374833934 container start d6812d8c95c6162598f63428290326f7c955b124d2d13625655abfdd6c5ad3f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-07T02:45:05.109 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 07 02:45:04 smithi064 podman[73217]: 2024-05-07 02:45:04.870534513 +0000 UTC m=+0.437534303 container attach d6812d8c95c6162598f63428290326f7c955b124d2d13625655abfdd6c5ad3f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a) 2024-05-07T02:45:05.465 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 07 02:45:05 smithi064 podman[73217]: 2024-05-07 02:45:05.115224612 +0000 UTC m=+0.682224401 container died d6812d8c95c6162598f63428290326f7c955b124d2d13625655abfdd6c5ad3f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-2-deactivate, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True) 2024-05-07T02:45:05.776 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@osd.2.service' 2024-05-07T02:45:05.822 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-07T02:45:05.823 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-07T02:45:05.823 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-07T02:45:05.823 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@osd.3 2024-05-07T02:45:06.395 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 07 02:45:05 smithi019 systemd[1]: Stopping Ceph osd.3 for 9ff84c16-0c17-11ef-bc97-c7b262605968... 2024-05-07T02:45:06.609 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:06 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:06.255+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:06.895 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 07 02:45:06 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-3[51201]: 2024-05-07T02:45:06.418+0000 7f589b200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-07T02:45:06.896 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 07 02:45:06 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-3[51201]: 2024-05-07T02:45:06.418+0000 7f589b200700 -1 osd.3 104 *** Got signal Terminated *** 2024-05-07T02:45:06.896 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 07 02:45:06 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-3[51201]: 2024-05-07T02:45:06.418+0000 7f589b200700 -1 osd.3 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-07T02:45:07.609 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:07 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:07.243+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:08.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:08 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:08.085+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:08.609 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:08 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:08.287+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:08.895 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 07 02:45:08 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-3[51201]: 2024-05-07T02:45:08.395+0000 7f5891a00700 -1 osd.3 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.564441+0000 front 2024-05-07T02:44:43.564172+0000 (oldest deadline 2024-05-07T02:45:08.264180+0000) 2024-05-07T02:45:09.359 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:09 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:09.327+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:09.359 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:09 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5[46831]: 2024-05-07T02:45:09.095+0000 7f3178bbf700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.944071+0000 front 2024-05-07T02:44:44.944281+0000 (oldest deadline 2024-05-07T02:45:08.443981+0000) 2024-05-07T02:45:09.395 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:09 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:09.065+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:09.707 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:09 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:09.366+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:09.895 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 07 02:45:09 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-3[51201]: 2024-05-07T02:45:09.413+0000 7f5891a00700 -1 osd.3 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.564441+0000 front 2024-05-07T02:44:43.564172+0000 (oldest deadline 2024-05-07T02:45:08.264180+0000) 2024-05-07T02:45:10.359 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:10 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5[46831]: 2024-05-07T02:45:10.097+0000 7f3178bbf700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.944071+0000 front 2024-05-07T02:44:44.944281+0000 (oldest deadline 2024-05-07T02:45:08.443981+0000) 2024-05-07T02:45:10.395 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:10 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:10.050+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:10.707 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:10 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:10.356+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:10.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:10 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:10.361+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:10.895 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 07 02:45:10 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-3[51201]: 2024-05-07T02:45:10.395+0000 7f5891a00700 -1 osd.3 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.564441+0000 front 2024-05-07T02:44:43.564172+0000 (oldest deadline 2024-05-07T02:45:08.264180+0000) 2024-05-07T02:45:11.358 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:11 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5[46831]: 2024-05-07T02:45:11.096+0000 7f3178bbf700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.944071+0000 front 2024-05-07T02:44:44.944281+0000 (oldest deadline 2024-05-07T02:45:08.443981+0000) 2024-05-07T02:45:11.395 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:11 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:11.054+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:11.609 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:11 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:11.358+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:11.707 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:11 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:11.363+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:11.894 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 07 02:45:11 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-3[51201]: 2024-05-07T02:45:11.403+0000 7f5891a00700 -1 osd.3 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.564441+0000 front 2024-05-07T02:44:43.564172+0000 (oldest deadline 2024-05-07T02:45:08.264180+0000) 2024-05-07T02:45:12.145 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:12 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:12.088+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:12.146 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 07 02:45:11 smithi019 podman[114338]: 2024-05-07 02:45:11.893881441 +0000 UTC m=+5.621106035 container died a69af090c734c7014adfd0a7d1d3c2409d17936156c170087406de759798c335 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-3, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True) 2024-05-07T02:45:12.596 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 07 02:45:12 smithi019 podman[114338]: 2024-05-07 02:45:12.361210794 +0000 UTC m=+6.088435426 container cleanup a69af090c734c7014adfd0a7d1d3c2409d17936156c170087406de759798c335 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-3, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-07T02:45:12.596 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 07 02:45:12 smithi019 bash[114338]: ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-3 2024-05-07T02:45:12.609 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:12 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5[46831]: 2024-05-07T02:45:12.112+0000 7f3178bbf700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.944071+0000 front 2024-05-07T02:44:44.944281+0000 (oldest deadline 2024-05-07T02:45:08.443981+0000) 2024-05-07T02:45:12.609 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:12 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:12.347+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:12.707 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:12 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:12.368+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:12.895 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 07 02:45:12 smithi019 podman[114369]: 2024-05-07 02:45:12.595672498 +0000 UTC m=+0.694742527 container remove a69af090c734c7014adfd0a7d1d3c2409d17936156c170087406de759798c335 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5) 2024-05-07T02:45:13.395 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:13 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:13.064+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:13.396 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:13 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5[46831]: 2024-05-07T02:45:13.129+0000 7f3178bbf700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.944071+0000 front 2024-05-07T02:44:44.944281+0000 (oldest deadline 2024-05-07T02:45:08.443981+0000) 2024-05-07T02:45:13.707 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:13 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:13.386+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:13.802 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 07 02:45:13 smithi019 podman[114453]: 2024-05-07 02:45:13.351271629 +0000 UTC m=+0.037843830 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-07T02:45:13.803 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 07 02:45:13 smithi019 podman[114453]: 2024-05-07 02:45:13.463352198 +0000 UTC m=+0.149924419 container create 9c092a4f3eebfc1f372e6d4c5de4ff21aebfd3cdbe23bb481f9592a5372b9e01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-07T02:45:13.858 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:13 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:13.396+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:14.054 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 07 02:45:13 smithi019 podman[114453]: 2024-05-07 02:45:13.80093471 +0000 UTC m=+0.487506974 container init 9c092a4f3eebfc1f372e6d4c5de4ff21aebfd3cdbe23bb481f9592a5372b9e01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-3-deactivate, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-07T02:45:14.054 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 07 02:45:13 smithi019 podman[114453]: 2024-05-07 02:45:13.813568803 +0000 UTC m=+0.500141005 container start 9c092a4f3eebfc1f372e6d4c5de4ff21aebfd3cdbe23bb481f9592a5372b9e01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-07T02:45:14.054 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 07 02:45:13 smithi019 podman[114453]: 2024-05-07 02:45:13.907683135 +0000 UTC m=+0.594255393 container attach 9c092a4f3eebfc1f372e6d4c5de4ff21aebfd3cdbe23bb481f9592a5372b9e01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-07T02:45:14.307 INFO:journalctl@ceph.osd.3.smithi019.stdout:May 07 02:45:14 smithi019 podman[114453]: 2024-05-07 02:45:14.283720258 +0000 UTC m=+0.970292523 container died 9c092a4f3eebfc1f372e6d4c5de4ff21aebfd3cdbe23bb481f9592a5372b9e01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.vendor=CentOS) 2024-05-07T02:45:14.308 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:14 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:14.051+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:14.434 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:14 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5[46831]: 2024-05-07T02:45:14.147+0000 7f3178bbf700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.944071+0000 front 2024-05-07T02:44:44.944281+0000 (oldest deadline 2024-05-07T02:45:08.443981+0000) 2024-05-07T02:45:14.707 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:14 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:14.412+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:14.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:14 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:14.434+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:15.078 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@osd.3.service' 2024-05-07T02:45:15.196 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-07T02:45:15.197 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-07T02:45:15.197 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-07T02:45:15.197 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@osd.4 2024-05-07T02:45:15.359 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:15 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5[46831]: 2024-05-07T02:45:15.107+0000 7f3178bbf700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.944071+0000 front 2024-05-07T02:44:44.944281+0000 (oldest deadline 2024-05-07T02:45:08.443981+0000) 2024-05-07T02:45:15.395 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:15 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:15.058+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:15.505 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:15 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:15.397+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:15.506 INFO:journalctl@ceph.osd.4.smithi027.stdout:May 07 02:45:15 smithi027 systemd[1]: Stopping Ceph osd.4 for 9ff84c16-0c17-11ef-bc97-c7b262605968... 2024-05-07T02:45:15.781 INFO:journalctl@ceph.osd.4.smithi027.stdout:May 07 02:45:15 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-4[45836]: 2024-05-07T02:45:15.504+0000 7f8656056700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-07T02:45:15.781 INFO:journalctl@ceph.osd.4.smithi027.stdout:May 07 02:45:15 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-4[45836]: 2024-05-07T02:45:15.504+0000 7f8656056700 -1 osd.4 104 *** Got signal Terminated *** 2024-05-07T02:45:15.781 INFO:journalctl@ceph.osd.4.smithi027.stdout:May 07 02:45:15 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-4[45836]: 2024-05-07T02:45:15.504+0000 7f8656056700 -1 osd.4 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-07T02:45:15.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:15 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:15.398+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:16.395 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:16 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:16.088+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:16.609 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:16 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5[46831]: 2024-05-07T02:45:16.152+0000 7f3178bbf700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.944071+0000 front 2024-05-07T02:44:44.944281+0000 (oldest deadline 2024-05-07T02:45:08.443981+0000) 2024-05-07T02:45:16.609 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:16 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5[46831]: 2024-05-07T02:45:16.152+0000 7f3178bbf700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:49.545487+0000 front 2024-05-07T02:44:49.545611+0000 (oldest deadline 2024-05-07T02:45:15.445486+0000) 2024-05-07T02:45:16.610 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:16 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:16.398+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:16.610 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:16 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:16.398+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:45:16.707 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:16 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:16.383+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:17.359 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:17 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5[46831]: 2024-05-07T02:45:17.106+0000 7f3178bbf700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.944071+0000 front 2024-05-07T02:44:44.944281+0000 (oldest deadline 2024-05-07T02:45:08.443981+0000) 2024-05-07T02:45:17.359 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:17 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5[46831]: 2024-05-07T02:45:17.106+0000 7f3178bbf700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:49.545487+0000 front 2024-05-07T02:44:49.545611+0000 (oldest deadline 2024-05-07T02:45:15.445486+0000) 2024-05-07T02:45:17.395 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:17 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:17.132+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:17.395 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:17 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:17.132+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.135339+0000 front 2024-05-07T02:44:54.135247+0000 (oldest deadline 2024-05-07T02:45:17.035496+0000) 2024-05-07T02:45:17.396 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:17 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-6[58135]: 2024-05-07T02:45:17.257+0000 7f242ea00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:51.829136+0000 front 2024-05-07T02:44:51.829028+0000 (oldest deadline 2024-05-07T02:45:17.129289+0000) 2024-05-07T02:45:17.707 INFO:journalctl@ceph.osd.4.smithi027.stdout:May 07 02:45:17 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-4[45836]: 2024-05-07T02:45:17.367+0000 7f864e604700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.118205+0000 front 2024-05-07T02:44:53.118238+0000 (oldest deadline 2024-05-07T02:45:16.618332+0000) 2024-05-07T02:45:17.707 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:17 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:17.338+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:17.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:17 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:17.448+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:17.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:17 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:17.448+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:45:18.418 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:18 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5[46831]: 2024-05-07T02:45:18.109+0000 7f3178bbf700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.944071+0000 front 2024-05-07T02:44:44.944281+0000 (oldest deadline 2024-05-07T02:45:08.443981+0000) 2024-05-07T02:45:18.419 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:18 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5[46831]: 2024-05-07T02:45:18.109+0000 7f3178bbf700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:49.545487+0000 front 2024-05-07T02:44:49.545611+0000 (oldest deadline 2024-05-07T02:45:15.445486+0000) 2024-05-07T02:45:18.645 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:18 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-6[58135]: 2024-05-07T02:45:18.273+0000 7f242ea00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:51.829136+0000 front 2024-05-07T02:44:51.829028+0000 (oldest deadline 2024-05-07T02:45:17.129289+0000) 2024-05-07T02:45:18.645 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:18 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:18.155+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:18.646 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:18 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:18.155+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.135339+0000 front 2024-05-07T02:44:54.135247+0000 (oldest deadline 2024-05-07T02:45:17.035496+0000) 2024-05-07T02:45:18.707 INFO:journalctl@ceph.osd.4.smithi027.stdout:May 07 02:45:18 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-4[45836]: 2024-05-07T02:45:18.351+0000 7f864e604700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.118205+0000 front 2024-05-07T02:44:53.118238+0000 (oldest deadline 2024-05-07T02:45:16.618332+0000) 2024-05-07T02:45:18.707 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:18 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:18.307+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:18.707 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:18 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:18.307+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:45:18.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:18 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:18.418+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:18.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:18 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:18.418+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:45:18.860 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:18 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:18.539+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.527515+0000 front 2024-05-07T02:44:52.527624+0000 (oldest deadline 2024-05-07T02:45:17.827549+0000) 2024-05-07T02:45:19.207 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:18 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:18.730+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.600018+0000 front 2024-05-07T02:44:52.600047+0000 (oldest deadline 2024-05-07T02:45:17.900188+0000) 2024-05-07T02:45:19.495 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:19 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:19.388+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:19.495 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:19 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:19.388+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:45:19.496 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:19 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5[46831]: 2024-05-07T02:45:19.143+0000 7f3178bbf700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.944071+0000 front 2024-05-07T02:44:44.944281+0000 (oldest deadline 2024-05-07T02:45:08.443981+0000) 2024-05-07T02:45:19.496 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:19 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5[46831]: 2024-05-07T02:45:19.143+0000 7f3178bbf700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:49.545487+0000 front 2024-05-07T02:44:49.545611+0000 (oldest deadline 2024-05-07T02:45:15.445486+0000) 2024-05-07T02:45:19.496 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:19 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:19.495+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.527515+0000 front 2024-05-07T02:44:52.527624+0000 (oldest deadline 2024-05-07T02:45:17.827549+0000) 2024-05-07T02:45:19.645 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:19 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-6[58135]: 2024-05-07T02:45:19.312+0000 7f242ea00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:51.829136+0000 front 2024-05-07T02:44:51.829028+0000 (oldest deadline 2024-05-07T02:45:17.129289+0000) 2024-05-07T02:45:19.646 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:19 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:19.164+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:19.646 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:19 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:19.164+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.135339+0000 front 2024-05-07T02:44:54.135247+0000 (oldest deadline 2024-05-07T02:45:17.035496+0000) 2024-05-07T02:45:19.707 INFO:journalctl@ceph.osd.4.smithi027.stdout:May 07 02:45:19 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-4[45836]: 2024-05-07T02:45:19.361+0000 7f864e604700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.118205+0000 front 2024-05-07T02:44:53.118238+0000 (oldest deadline 2024-05-07T02:45:16.618332+0000) 2024-05-07T02:45:19.707 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:19 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:19.334+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:19.707 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:19 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:19.334+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:45:20.207 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:19 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:19.748+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.600018+0000 front 2024-05-07T02:44:52.600047+0000 (oldest deadline 2024-05-07T02:45:17.900188+0000) 2024-05-07T02:45:20.509 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:20 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:20.365+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:20.509 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:20 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:20.365+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:45:20.509 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:20 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5[46831]: 2024-05-07T02:45:20.158+0000 7f3178bbf700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.944071+0000 front 2024-05-07T02:44:44.944281+0000 (oldest deadline 2024-05-07T02:45:08.443981+0000) 2024-05-07T02:45:20.509 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:20 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5[46831]: 2024-05-07T02:45:20.158+0000 7f3178bbf700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:49.545487+0000 front 2024-05-07T02:44:49.545611+0000 (oldest deadline 2024-05-07T02:45:15.445486+0000) 2024-05-07T02:45:20.645 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:20 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-6[58135]: 2024-05-07T02:45:20.355+0000 7f242ea00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:51.829136+0000 front 2024-05-07T02:44:51.829028+0000 (oldest deadline 2024-05-07T02:45:17.129289+0000) 2024-05-07T02:45:20.646 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:20 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:20.174+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:20.646 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:20 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:20.174+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.135339+0000 front 2024-05-07T02:44:54.135247+0000 (oldest deadline 2024-05-07T02:45:17.035496+0000) 2024-05-07T02:45:20.707 INFO:journalctl@ceph.osd.4.smithi027.stdout:May 07 02:45:20 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-4[45836]: 2024-05-07T02:45:20.399+0000 7f864e604700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.118205+0000 front 2024-05-07T02:44:53.118238+0000 (oldest deadline 2024-05-07T02:45:16.618332+0000) 2024-05-07T02:45:20.707 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:20 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:20.374+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:20.708 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:20 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:20.374+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:45:20.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:20 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:20.508+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.527515+0000 front 2024-05-07T02:44:52.527624+0000 (oldest deadline 2024-05-07T02:45:17.827549+0000) 2024-05-07T02:45:21.148 INFO:journalctl@ceph.osd.4.smithi027.stdout:May 07 02:45:20 smithi027 podman[72689]: 2024-05-07 02:45:20.877317422 +0000 UTC m=+5.483674211 container died afc34376c518b4a91775ae51f36c54501edd7f7a035cbc608993897464544b9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-4, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-05-07T02:45:21.149 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:20 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:20.756+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.600018+0000 front 2024-05-07T02:44:52.600047+0000 (oldest deadline 2024-05-07T02:45:17.900188+0000) 2024-05-07T02:45:21.412 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:21 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5[46831]: 2024-05-07T02:45:21.115+0000 7f3178bbf700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.944071+0000 front 2024-05-07T02:44:44.944281+0000 (oldest deadline 2024-05-07T02:45:08.443981+0000) 2024-05-07T02:45:21.412 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:21 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5[46831]: 2024-05-07T02:45:21.115+0000 7f3178bbf700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:49.545487+0000 front 2024-05-07T02:44:49.545611+0000 (oldest deadline 2024-05-07T02:45:15.445486+0000) 2024-05-07T02:45:21.457 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:21 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:21.337+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:21.458 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:21 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:21.337+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:45:21.458 INFO:journalctl@ceph.osd.4.smithi027.stdout:May 07 02:45:21 smithi027 podman[72689]: 2024-05-07 02:45:21.247648537 +0000 UTC m=+5.854005334 container cleanup afc34376c518b4a91775ae51f36c54501edd7f7a035cbc608993897464544b9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-4, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-07T02:45:21.458 INFO:journalctl@ceph.osd.4.smithi027.stdout:May 07 02:45:21 smithi027 bash[72689]: ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-4 2024-05-07T02:45:21.645 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:21 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-6[58135]: 2024-05-07T02:45:21.336+0000 7f242ea00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:51.829136+0000 front 2024-05-07T02:44:51.829028+0000 (oldest deadline 2024-05-07T02:45:17.129289+0000) 2024-05-07T02:45:21.646 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:21 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:21.222+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:21.646 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:21 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:21.222+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.135339+0000 front 2024-05-07T02:44:54.135247+0000 (oldest deadline 2024-05-07T02:45:17.035496+0000) 2024-05-07T02:45:21.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:21 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:21.538+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.527515+0000 front 2024-05-07T02:44:52.527624+0000 (oldest deadline 2024-05-07T02:45:17.827549+0000) 2024-05-07T02:45:21.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:21 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:21.411+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:21.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:21 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:21.411+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:45:21.879 INFO:journalctl@ceph.osd.4.smithi027.stdout:May 07 02:45:21 smithi027 podman[72718]: 2024-05-07 02:45:21.548861853 +0000 UTC m=+0.670475942 container remove afc34376c518b4a91775ae51f36c54501edd7f7a035cbc608993897464544b9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-4, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-07T02:45:21.879 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:21 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:21.752+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.600018+0000 front 2024-05-07T02:44:52.600047+0000 (oldest deadline 2024-05-07T02:45:17.900188+0000) 2024-05-07T02:45:22.130 INFO:journalctl@ceph.osd.4.smithi027.stdout:May 07 02:45:21 smithi027 podman[72805]: 2024-05-07 02:45:21.782568212 +0000 UTC m=+0.018633077 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-07T02:45:22.131 INFO:journalctl@ceph.osd.4.smithi027.stdout:May 07 02:45:21 smithi027 podman[72805]: 2024-05-07 02:45:21.883084934 +0000 UTC m=+0.119149834 container create 3c39c47fafa1462cd7a5e62782356aab35aea5bcc2381aa87eaa5cb937466cf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-07T02:45:22.131 INFO:journalctl@ceph.osd.4.smithi027.stdout:May 07 02:45:22 smithi027 podman[72805]: 2024-05-07 02:45:22.127375773 +0000 UTC m=+0.363440619 container init 3c39c47fafa1462cd7a5e62782356aab35aea5bcc2381aa87eaa5cb937466cf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-07T02:45:22.131 INFO:journalctl@ceph.osd.4.smithi027.stdout:May 07 02:45:22 smithi027 podman[72805]: 2024-05-07 02:45:22.130641033 +0000 UTC m=+0.366705887 container start 3c39c47fafa1462cd7a5e62782356aab35aea5bcc2381aa87eaa5cb937466cf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-4-deactivate, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-07T02:45:22.359 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:22 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5[46831]: 2024-05-07T02:45:22.078+0000 7f3178bbf700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.944071+0000 front 2024-05-07T02:44:44.944281+0000 (oldest deadline 2024-05-07T02:45:08.443981+0000) 2024-05-07T02:45:22.359 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:22 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5[46831]: 2024-05-07T02:45:22.078+0000 7f3178bbf700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:49.545487+0000 front 2024-05-07T02:44:49.545611+0000 (oldest deadline 2024-05-07T02:45:15.445486+0000) 2024-05-07T02:45:22.437 INFO:journalctl@ceph.osd.4.smithi027.stdout:May 07 02:45:22 smithi027 podman[72805]: 2024-05-07 02:45:22.218946617 +0000 UTC m=+0.455011474 container attach 3c39c47fafa1462cd7a5e62782356aab35aea5bcc2381aa87eaa5cb937466cf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-4-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.build-date=20240430) 2024-05-07T02:45:22.438 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:22 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:22.341+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:22.438 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:22 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:22.341+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:45:22.645 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:22 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-6[58135]: 2024-05-07T02:45:22.338+0000 7f242ea00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:51.829136+0000 front 2024-05-07T02:44:51.829028+0000 (oldest deadline 2024-05-07T02:45:17.129289+0000) 2024-05-07T02:45:22.645 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:22 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:22.178+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:22.646 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:22 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:22.178+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.135339+0000 front 2024-05-07T02:44:54.135247+0000 (oldest deadline 2024-05-07T02:45:17.035496+0000) 2024-05-07T02:45:22.704 INFO:journalctl@ceph.osd.4.smithi027.stdout:May 07 02:45:22 smithi027 podman[72805]: 2024-05-07 02:45:22.438373013 +0000 UTC m=+0.674437867 container died 3c39c47fafa1462cd7a5e62782356aab35aea5bcc2381aa87eaa5cb937466cf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS) 2024-05-07T02:45:22.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:22 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:22.505+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.527515+0000 front 2024-05-07T02:44:52.527624+0000 (oldest deadline 2024-05-07T02:45:17.827549+0000) 2024-05-07T02:45:22.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:22 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:22.383+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:22.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:22 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:22.383+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:45:22.957 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:22 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:22.753+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.600018+0000 front 2024-05-07T02:44:52.600047+0000 (oldest deadline 2024-05-07T02:45:17.900188+0000) 2024-05-07T02:45:23.039 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@osd.4.service' 2024-05-07T02:45:23.081 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-07T02:45:23.081 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-07T02:45:23.081 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-07T02:45:23.081 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@osd.5 2024-05-07T02:45:23.359 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:23 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5[46831]: 2024-05-07T02:45:23.128+0000 7f3178bbf700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.944071+0000 front 2024-05-07T02:44:44.944281+0000 (oldest deadline 2024-05-07T02:45:08.443981+0000) 2024-05-07T02:45:23.359 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:23 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5[46831]: 2024-05-07T02:45:23.128+0000 7f3178bbf700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:49.545487+0000 front 2024-05-07T02:44:49.545611+0000 (oldest deadline 2024-05-07T02:45:15.445486+0000) 2024-05-07T02:45:23.359 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:23 smithi064 systemd[1]: Stopping Ceph osd.5 for 9ff84c16-0c17-11ef-bc97-c7b262605968... 2024-05-07T02:45:23.645 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:23 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-6[58135]: 2024-05-07T02:45:23.377+0000 7f242ea00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:51.829136+0000 front 2024-05-07T02:44:51.829028+0000 (oldest deadline 2024-05-07T02:45:17.129289+0000) 2024-05-07T02:45:23.646 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:23 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:23.180+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:23.646 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:23 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:23.180+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.135339+0000 front 2024-05-07T02:44:54.135247+0000 (oldest deadline 2024-05-07T02:45:17.035496+0000) 2024-05-07T02:45:23.707 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:23 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:23.359+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:23.707 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:23 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:23.359+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:45:23.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:23 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:23.552+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.527515+0000 front 2024-05-07T02:44:52.527624+0000 (oldest deadline 2024-05-07T02:45:17.827549+0000) 2024-05-07T02:45:23.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:23 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:23.391+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:23.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:23 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:23.391+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:45:23.860 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:23 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5[46831]: 2024-05-07T02:45:23.385+0000 7f317c609700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-07T02:45:23.860 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:23 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5[46831]: 2024-05-07T02:45:23.385+0000 7f317c609700 -1 osd.5 104 *** Got signal Terminated *** 2024-05-07T02:45:23.860 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:23 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5[46831]: 2024-05-07T02:45:23.385+0000 7f317c609700 -1 osd.5 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-07T02:45:24.207 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:23 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:23.739+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.600018+0000 front 2024-05-07T02:44:52.600047+0000 (oldest deadline 2024-05-07T02:45:17.900188+0000) 2024-05-07T02:45:24.452 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:24 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-6[58135]: 2024-05-07T02:45:24.346+0000 7f242ea00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:51.829136+0000 front 2024-05-07T02:44:51.829028+0000 (oldest deadline 2024-05-07T02:45:17.129289+0000) 2024-05-07T02:45:24.453 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:24 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:24.165+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:24.453 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:24 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:24.165+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.135339+0000 front 2024-05-07T02:44:54.135247+0000 (oldest deadline 2024-05-07T02:45:17.035496+0000) 2024-05-07T02:45:24.558 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:24 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5[46831]: 2024-05-07T02:45:24.153+0000 7f3178bbf700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.944071+0000 front 2024-05-07T02:44:44.944281+0000 (oldest deadline 2024-05-07T02:45:08.443981+0000) 2024-05-07T02:45:24.558 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:24 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5[46831]: 2024-05-07T02:45:24.153+0000 7f3178bbf700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:49.545487+0000 front 2024-05-07T02:44:49.545611+0000 (oldest deadline 2024-05-07T02:45:15.445486+0000) 2024-05-07T02:45:24.558 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:24 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:24.344+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:24.559 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:24 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:24.344+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:45:24.707 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:24 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:24.347+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:24.707 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:24 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:24.347+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:45:24.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:24 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:24.557+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.527515+0000 front 2024-05-07T02:44:52.527624+0000 (oldest deadline 2024-05-07T02:45:17.827549+0000) 2024-05-07T02:45:25.207 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:24 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:24.740+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.600018+0000 front 2024-05-07T02:44:52.600047+0000 (oldest deadline 2024-05-07T02:45:17.900188+0000) 2024-05-07T02:45:25.207 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:24 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:24.740+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:44:57.900629+0000 front 2024-05-07T02:44:57.900606+0000 (oldest deadline 2024-05-07T02:45:23.800730+0000) 2024-05-07T02:45:25.538 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:25 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5[46831]: 2024-05-07T02:45:25.110+0000 7f3178bbf700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.944071+0000 front 2024-05-07T02:44:44.944281+0000 (oldest deadline 2024-05-07T02:45:08.443981+0000) 2024-05-07T02:45:25.539 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:25 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5[46831]: 2024-05-07T02:45:25.110+0000 7f3178bbf700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:49.545487+0000 front 2024-05-07T02:44:49.545611+0000 (oldest deadline 2024-05-07T02:45:15.445486+0000) 2024-05-07T02:45:25.539 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:25 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:25.318+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:25.539 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:25 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:25.318+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:45:25.645 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:25 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-6[58135]: 2024-05-07T02:45:25.307+0000 7f242ea00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:51.829136+0000 front 2024-05-07T02:44:51.829028+0000 (oldest deadline 2024-05-07T02:45:17.129289+0000) 2024-05-07T02:45:25.646 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:25 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:25.175+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:25.646 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:25 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:25.175+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.135339+0000 front 2024-05-07T02:44:54.135247+0000 (oldest deadline 2024-05-07T02:45:17.035496+0000) 2024-05-07T02:45:25.646 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:25 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:25.175+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.938208+0000 front 2024-05-07T02:45:02.938286+0000 (oldest deadline 2024-05-07T02:45:24.638420+0000) 2024-05-07T02:45:25.708 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:25 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:25.396+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:25.818 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:25 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:25.396+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:45:25.860 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:25 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:25.537+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.527515+0000 front 2024-05-07T02:44:52.527624+0000 (oldest deadline 2024-05-07T02:45:17.827549+0000) 2024-05-07T02:45:25.860 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:25 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:25.537+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.528686+0000 front 2024-05-07T02:45:02.528717+0000 (oldest deadline 2024-05-07T02:45:25.428840+0000) 2024-05-07T02:45:26.207 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:25 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:25.720+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.600018+0000 front 2024-05-07T02:44:52.600047+0000 (oldest deadline 2024-05-07T02:45:17.900188+0000) 2024-05-07T02:45:26.207 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:25 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:25.720+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:44:57.900629+0000 front 2024-05-07T02:44:57.900606+0000 (oldest deadline 2024-05-07T02:45:23.800730+0000) 2024-05-07T02:45:26.359 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:26 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5[46831]: 2024-05-07T02:45:26.086+0000 7f3178bbf700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.944071+0000 front 2024-05-07T02:44:44.944281+0000 (oldest deadline 2024-05-07T02:45:08.443981+0000) 2024-05-07T02:45:26.359 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:26 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5[46831]: 2024-05-07T02:45:26.086+0000 7f3178bbf700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:49.545487+0000 front 2024-05-07T02:44:49.545611+0000 (oldest deadline 2024-05-07T02:45:15.445486+0000) 2024-05-07T02:45:26.359 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:26 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:26.309+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:26.360 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:26 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:26.309+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:45:26.645 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:26 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-6[58135]: 2024-05-07T02:45:26.350+0000 7f242ea00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:51.829136+0000 front 2024-05-07T02:44:51.829028+0000 (oldest deadline 2024-05-07T02:45:17.129289+0000) 2024-05-07T02:45:26.646 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:26 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:26.158+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:26.646 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:26 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:26.158+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.135339+0000 front 2024-05-07T02:44:54.135247+0000 (oldest deadline 2024-05-07T02:45:17.035496+0000) 2024-05-07T02:45:26.646 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:26 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:26.158+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.938208+0000 front 2024-05-07T02:45:02.938286+0000 (oldest deadline 2024-05-07T02:45:24.638420+0000) 2024-05-07T02:45:26.707 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:26 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:26.384+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:26.707 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:26 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:26.384+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:45:26.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:26 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:26.531+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.527515+0000 front 2024-05-07T02:44:52.527624+0000 (oldest deadline 2024-05-07T02:45:17.827549+0000) 2024-05-07T02:45:26.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:26 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:26.531+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.528686+0000 front 2024-05-07T02:45:02.528717+0000 (oldest deadline 2024-05-07T02:45:25.428840+0000) 2024-05-07T02:45:27.207 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:26 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:26.768+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.600018+0000 front 2024-05-07T02:44:52.600047+0000 (oldest deadline 2024-05-07T02:45:17.900188+0000) 2024-05-07T02:45:27.207 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:26 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:26.768+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:44:57.900629+0000 front 2024-05-07T02:44:57.900606+0000 (oldest deadline 2024-05-07T02:45:23.800730+0000) 2024-05-07T02:45:27.336 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:27 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5[46831]: 2024-05-07T02:45:27.074+0000 7f3178bbf700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.944071+0000 front 2024-05-07T02:44:44.944281+0000 (oldest deadline 2024-05-07T02:45:08.443981+0000) 2024-05-07T02:45:27.337 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:27 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5[46831]: 2024-05-07T02:45:27.074+0000 7f3178bbf700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:49.545487+0000 front 2024-05-07T02:44:49.545611+0000 (oldest deadline 2024-05-07T02:45:15.445486+0000) 2024-05-07T02:45:27.395 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:27 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:27.120+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:27.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:27 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:27.120+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.135339+0000 front 2024-05-07T02:44:54.135247+0000 (oldest deadline 2024-05-07T02:45:17.035496+0000) 2024-05-07T02:45:27.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:27 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:27.120+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.938208+0000 front 2024-05-07T02:45:02.938286+0000 (oldest deadline 2024-05-07T02:45:24.638420+0000) 2024-05-07T02:45:27.609 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:27 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:27.555+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.527515+0000 front 2024-05-07T02:44:52.527624+0000 (oldest deadline 2024-05-07T02:45:17.827549+0000) 2024-05-07T02:45:27.609 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:27 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:27.555+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.528686+0000 front 2024-05-07T02:45:02.528717+0000 (oldest deadline 2024-05-07T02:45:25.428840+0000) 2024-05-07T02:45:27.610 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:27 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:27.336+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:27.610 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:27 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:27.336+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:45:27.610 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:27 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:27.336+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:45:27.707 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:27 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:27.396+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:27.707 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:27 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:27.396+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:45:27.895 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:27 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-6[58135]: 2024-05-07T02:45:27.398+0000 7f242ea00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:51.829136+0000 front 2024-05-07T02:44:51.829028+0000 (oldest deadline 2024-05-07T02:45:17.129289+0000) 2024-05-07T02:45:28.207 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:27 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:27.788+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.600018+0000 front 2024-05-07T02:44:52.600047+0000 (oldest deadline 2024-05-07T02:45:17.900188+0000) 2024-05-07T02:45:28.207 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:27 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:27.788+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:44:57.900629+0000 front 2024-05-07T02:44:57.900606+0000 (oldest deadline 2024-05-07T02:45:23.800730+0000) 2024-05-07T02:45:28.359 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:28 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5[46831]: 2024-05-07T02:45:28.098+0000 7f3178bbf700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.944071+0000 front 2024-05-07T02:44:44.944281+0000 (oldest deadline 2024-05-07T02:45:08.443981+0000) 2024-05-07T02:45:28.359 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:28 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5[46831]: 2024-05-07T02:45:28.098+0000 7f3178bbf700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:49.545487+0000 front 2024-05-07T02:44:49.545611+0000 (oldest deadline 2024-05-07T02:45:15.445486+0000) 2024-05-07T02:45:28.360 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:28 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:28.333+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:28.360 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:28 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:28.333+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:45:28.360 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:28 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:28.333+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:45:28.395 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:28 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:28.116+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:28.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:28 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:28.116+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.135339+0000 front 2024-05-07T02:44:54.135247+0000 (oldest deadline 2024-05-07T02:45:17.035496+0000) 2024-05-07T02:45:28.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:28 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:28.116+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.938208+0000 front 2024-05-07T02:45:02.938286+0000 (oldest deadline 2024-05-07T02:45:24.638420+0000) 2024-05-07T02:45:28.649 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:28 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:28.506+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.527515+0000 front 2024-05-07T02:44:52.527624+0000 (oldest deadline 2024-05-07T02:45:17.827549+0000) 2024-05-07T02:45:28.650 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:28 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:28.506+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.528686+0000 front 2024-05-07T02:45:02.528717+0000 (oldest deadline 2024-05-07T02:45:25.428840+0000) 2024-05-07T02:45:28.707 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:28 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:28.410+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:28.707 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:28 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:28.410+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:45:28.707 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:28 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:28.410+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:01.844911+0000 front 2024-05-07T02:45:01.844958+0000 (oldest deadline 2024-05-07T02:45:27.745107+0000) 2024-05-07T02:45:28.895 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:28 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-6[58135]: 2024-05-07T02:45:28.431+0000 7f242ea00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:51.829136+0000 front 2024-05-07T02:44:51.829028+0000 (oldest deadline 2024-05-07T02:45:17.129289+0000) 2024-05-07T02:45:28.896 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:28 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-6[58135]: 2024-05-07T02:45:28.431+0000 7f242ea00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.333280+0000 front 2024-05-07T02:45:02.333209+0000 (oldest deadline 2024-05-07T02:45:28.134185+0000) 2024-05-07T02:45:29.109 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:28 smithi064 podman[73327]: 2024-05-07 02:45:28.850544271 +0000 UTC m=+5.569323792 container died 8e48787ef25a2edf3282be2a6912814951af289f747b7b645c5ee331e1f2382a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.build-date=20240430, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-07T02:45:29.207 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:28 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:28.768+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.600018+0000 front 2024-05-07T02:44:52.600047+0000 (oldest deadline 2024-05-07T02:45:17.900188+0000) 2024-05-07T02:45:29.207 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:28 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:28.768+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:44:57.900629+0000 front 2024-05-07T02:44:57.900606+0000 (oldest deadline 2024-05-07T02:45:23.800730+0000) 2024-05-07T02:45:29.395 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:29 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-6[58135]: 2024-05-07T02:45:29.387+0000 7f242ea00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:51.829136+0000 front 2024-05-07T02:44:51.829028+0000 (oldest deadline 2024-05-07T02:45:17.129289+0000) 2024-05-07T02:45:29.396 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:29 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-6[58135]: 2024-05-07T02:45:29.387+0000 7f242ea00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.333280+0000 front 2024-05-07T02:45:02.333209+0000 (oldest deadline 2024-05-07T02:45:28.134185+0000) 2024-05-07T02:45:29.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:29 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:29.139+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:29.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:29 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:29.139+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.135339+0000 front 2024-05-07T02:44:54.135247+0000 (oldest deadline 2024-05-07T02:45:17.035496+0000) 2024-05-07T02:45:29.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:29 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:29.139+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.938208+0000 front 2024-05-07T02:45:02.938286+0000 (oldest deadline 2024-05-07T02:45:24.638420+0000) 2024-05-07T02:45:29.529 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:29 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:29.307+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:29.529 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:29 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:29.307+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:45:29.529 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:29 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:29.307+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:45:29.529 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:29 smithi064 podman[73327]: 2024-05-07 02:45:29.244277339 +0000 UTC m=+5.963056859 container cleanup 8e48787ef25a2edf3282be2a6912814951af289f747b7b645c5ee331e1f2382a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True) 2024-05-07T02:45:29.529 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:29 smithi064 bash[73327]: ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5 2024-05-07T02:45:29.707 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:29 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:29.406+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:29.707 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:29 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:29.406+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:45:29.707 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:29 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:29.406+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:01.844911+0000 front 2024-05-07T02:45:01.844958+0000 (oldest deadline 2024-05-07T02:45:27.745107+0000) 2024-05-07T02:45:29.837 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:29 smithi064 podman[73353]: 2024-05-07 02:45:29.529149645 +0000 UTC m=+0.678506203 container remove 8e48787ef25a2edf3282be2a6912814951af289f747b7b645c5ee331e1f2382a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=) 2024-05-07T02:45:29.838 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:29 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:29.534+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.527515+0000 front 2024-05-07T02:44:52.527624+0000 (oldest deadline 2024-05-07T02:45:17.827549+0000) 2024-05-07T02:45:29.838 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:29 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:29.534+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.528686+0000 front 2024-05-07T02:45:02.528717+0000 (oldest deadline 2024-05-07T02:45:25.428840+0000) 2024-05-07T02:45:30.109 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:29 smithi064 podman[73445]: 2024-05-07 02:45:29.740362362 +0000 UTC m=+0.018156705 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-07T02:45:30.109 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:29 smithi064 podman[73445]: 2024-05-07 02:45:29.887357159 +0000 UTC m=+0.165151524 container create cd9d9d09816bcf9e37a35e233481706aae3af263320a58fb6bb74a1bebff3af7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-07T02:45:30.207 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:29 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:29.723+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.600018+0000 front 2024-05-07T02:44:52.600047+0000 (oldest deadline 2024-05-07T02:45:17.900188+0000) 2024-05-07T02:45:30.207 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:29 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:29.724+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:44:57.900629+0000 front 2024-05-07T02:44:57.900606+0000 (oldest deadline 2024-05-07T02:45:23.800730+0000) 2024-05-07T02:45:30.389 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:30 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:30.101+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:30.389 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:30 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:30.101+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.135339+0000 front 2024-05-07T02:44:54.135247+0000 (oldest deadline 2024-05-07T02:45:17.035496+0000) 2024-05-07T02:45:30.389 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:30 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:30.101+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.938208+0000 front 2024-05-07T02:45:02.938286+0000 (oldest deadline 2024-05-07T02:45:24.638420+0000) 2024-05-07T02:45:30.453 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:30 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:30.304+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:30.453 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:30 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:30.304+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:45:30.454 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:30 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:30.304+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:45:30.454 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:30 smithi064 podman[73445]: 2024-05-07 02:45:30.133318039 +0000 UTC m=+0.411112371 container init cd9d9d09816bcf9e37a35e233481706aae3af263320a58fb6bb74a1bebff3af7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-07T02:45:30.454 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:30 smithi064 podman[73445]: 2024-05-07 02:45:30.136751136 +0000 UTC m=+0.414545463 container start cd9d9d09816bcf9e37a35e233481706aae3af263320a58fb6bb74a1bebff3af7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-07T02:45:30.454 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:30 smithi064 podman[73445]: 2024-05-07 02:45:30.233219081 +0000 UTC m=+0.511013411 container attach cd9d9d09816bcf9e37a35e233481706aae3af263320a58fb6bb74a1bebff3af7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-07T02:45:30.454 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 07 02:45:30 smithi064 podman[73445]: 2024-05-07 02:45:30.454011115 +0000 UTC m=+0.731805437 container died cd9d9d09816bcf9e37a35e233481706aae3af263320a58fb6bb74a1bebff3af7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True) 2024-05-07T02:45:30.645 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:30 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-6[58135]: 2024-05-07T02:45:30.388+0000 7f242ea00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:51.829136+0000 front 2024-05-07T02:44:51.829028+0000 (oldest deadline 2024-05-07T02:45:17.129289+0000) 2024-05-07T02:45:30.646 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:30 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-6[58135]: 2024-05-07T02:45:30.388+0000 7f242ea00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.333280+0000 front 2024-05-07T02:45:02.333209+0000 (oldest deadline 2024-05-07T02:45:28.134185+0000) 2024-05-07T02:45:30.707 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:30 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:30.409+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:30.707 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:30 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:30.409+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:45:30.707 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:30 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:30.409+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:01.844911+0000 front 2024-05-07T02:45:01.844958+0000 (oldest deadline 2024-05-07T02:45:27.745107+0000) 2024-05-07T02:45:30.819 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:30 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:30.491+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.527515+0000 front 2024-05-07T02:44:52.527624+0000 (oldest deadline 2024-05-07T02:45:17.827549+0000) 2024-05-07T02:45:30.819 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:30 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:30.491+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.528686+0000 front 2024-05-07T02:45:02.528717+0000 (oldest deadline 2024-05-07T02:45:25.428840+0000) 2024-05-07T02:45:31.154 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@osd.5.service' 2024-05-07T02:45:31.201 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-07T02:45:31.202 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-07T02:45:31.202 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-07T02:45:31.202 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@osd.6 2024-05-07T02:45:31.207 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:30 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:30.736+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.600018+0000 front 2024-05-07T02:44:52.600047+0000 (oldest deadline 2024-05-07T02:45:17.900188+0000) 2024-05-07T02:45:31.207 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:30 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:30.736+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:44:57.900629+0000 front 2024-05-07T02:44:57.900606+0000 (oldest deadline 2024-05-07T02:45:23.800730+0000) 2024-05-07T02:45:31.327 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:31 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:31.063+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:31.328 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:31 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:31.063+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.135339+0000 front 2024-05-07T02:44:54.135247+0000 (oldest deadline 2024-05-07T02:45:17.035496+0000) 2024-05-07T02:45:31.328 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:31 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:31.063+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.938208+0000 front 2024-05-07T02:45:02.938286+0000 (oldest deadline 2024-05-07T02:45:24.638420+0000) 2024-05-07T02:45:31.540 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:31 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:31.313+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:31.540 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:31 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:31.313+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:45:31.540 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:31 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:31.313+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:45:31.645 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:31 smithi019 systemd[1]: Stopping Ceph osd.6 for 9ff84c16-0c17-11ef-bc97-c7b262605968... 2024-05-07T02:45:31.645 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:31 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-6[58135]: 2024-05-07T02:45:31.338+0000 7f242ea00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:51.829136+0000 front 2024-05-07T02:44:51.829028+0000 (oldest deadline 2024-05-07T02:45:17.129289+0000) 2024-05-07T02:45:31.645 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:31 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-6[58135]: 2024-05-07T02:45:31.338+0000 7f242ea00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.333280+0000 front 2024-05-07T02:45:02.333209+0000 (oldest deadline 2024-05-07T02:45:28.134185+0000) 2024-05-07T02:45:31.707 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:31 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:31.419+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:31.707 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:31 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:31.419+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:45:31.708 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:31 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:31.419+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:01.844911+0000 front 2024-05-07T02:45:01.844958+0000 (oldest deadline 2024-05-07T02:45:27.745107+0000) 2024-05-07T02:45:31.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:31 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:31.539+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.527515+0000 front 2024-05-07T02:44:52.527624+0000 (oldest deadline 2024-05-07T02:45:17.827549+0000) 2024-05-07T02:45:31.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:31 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:31.539+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.528686+0000 front 2024-05-07T02:45:02.528717+0000 (oldest deadline 2024-05-07T02:45:25.428840+0000) 2024-05-07T02:45:32.098 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:31 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-6[58135]: 2024-05-07T02:45:31.822+0000 7f2433200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-07T02:45:32.098 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:31 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-6[58135]: 2024-05-07T02:45:31.822+0000 7f2433200700 -1 osd.6 104 *** Got signal Terminated *** 2024-05-07T02:45:32.098 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:31 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-6[58135]: 2024-05-07T02:45:31.822+0000 7f2433200700 -1 osd.6 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-07T02:45:32.207 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:31 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:31.765+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.600018+0000 front 2024-05-07T02:44:52.600047+0000 (oldest deadline 2024-05-07T02:45:17.900188+0000) 2024-05-07T02:45:32.207 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:31 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:31.765+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:44:57.900629+0000 front 2024-05-07T02:44:57.900606+0000 (oldest deadline 2024-05-07T02:45:23.800730+0000) 2024-05-07T02:45:32.387 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:32 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:32.096+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:32.387 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:32 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:32.096+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.135339+0000 front 2024-05-07T02:44:54.135247+0000 (oldest deadline 2024-05-07T02:45:17.035496+0000) 2024-05-07T02:45:32.387 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:32 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:32.096+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.938208+0000 front 2024-05-07T02:45:02.938286+0000 (oldest deadline 2024-05-07T02:45:24.638420+0000) 2024-05-07T02:45:32.609 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:32 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:32.522+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.527515+0000 front 2024-05-07T02:44:52.527624+0000 (oldest deadline 2024-05-07T02:45:17.827549+0000) 2024-05-07T02:45:32.609 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:32 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:32.522+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.528686+0000 front 2024-05-07T02:45:02.528717+0000 (oldest deadline 2024-05-07T02:45:25.428840+0000) 2024-05-07T02:45:32.610 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:32 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:32.356+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:32.610 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:32 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:32.356+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:45:32.610 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:32 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:32.356+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:45:32.645 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:32 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-6[58135]: 2024-05-07T02:45:32.386+0000 7f242ea00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:51.829136+0000 front 2024-05-07T02:44:51.829028+0000 (oldest deadline 2024-05-07T02:45:17.129289+0000) 2024-05-07T02:45:32.646 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:32 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-6[58135]: 2024-05-07T02:45:32.386+0000 7f242ea00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.333280+0000 front 2024-05-07T02:45:02.333209+0000 (oldest deadline 2024-05-07T02:45:28.134185+0000) 2024-05-07T02:45:32.737 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:32 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:32.467+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:32.737 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:32 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:32.467+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:45:32.737 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:32 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:32.467+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:01.844911+0000 front 2024-05-07T02:45:01.844958+0000 (oldest deadline 2024-05-07T02:45:27.745107+0000) 2024-05-07T02:45:33.207 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:32 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:32.736+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.600018+0000 front 2024-05-07T02:44:52.600047+0000 (oldest deadline 2024-05-07T02:45:17.900188+0000) 2024-05-07T02:45:33.207 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:32 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:32.736+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:44:57.900629+0000 front 2024-05-07T02:44:57.900606+0000 (oldest deadline 2024-05-07T02:45:23.800730+0000) 2024-05-07T02:45:33.208 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:32 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:32.736+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:10.202683+0000 front 2024-05-07T02:45:10.202506+0000 (oldest deadline 2024-05-07T02:45:32.502386+0000) 2024-05-07T02:45:33.395 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:33 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:33.051+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:33.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:33 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:33.051+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.135339+0000 front 2024-05-07T02:44:54.135247+0000 (oldest deadline 2024-05-07T02:45:17.035496+0000) 2024-05-07T02:45:33.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:33 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:33.051+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.938208+0000 front 2024-05-07T02:45:02.938286+0000 (oldest deadline 2024-05-07T02:45:24.638420+0000) 2024-05-07T02:45:33.781 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:33 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:33.484+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:33.782 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:33 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:33.484+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:45:33.782 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:33 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:33.484+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:01.844911+0000 front 2024-05-07T02:45:01.844958+0000 (oldest deadline 2024-05-07T02:45:27.745107+0000) 2024-05-07T02:45:33.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:33 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:33.568+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.527515+0000 front 2024-05-07T02:44:52.527624+0000 (oldest deadline 2024-05-07T02:45:17.827549+0000) 2024-05-07T02:45:33.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:33 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:33.568+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.528686+0000 front 2024-05-07T02:45:02.528717+0000 (oldest deadline 2024-05-07T02:45:25.428840+0000) 2024-05-07T02:45:33.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:33 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:33.391+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:33.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:33 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:33.391+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:45:33.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:33 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:33.391+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:45:33.895 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:33 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-6[58135]: 2024-05-07T02:45:33.430+0000 7f242ea00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:51.829136+0000 front 2024-05-07T02:44:51.829028+0000 (oldest deadline 2024-05-07T02:45:17.129289+0000) 2024-05-07T02:45:33.895 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:33 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-6[58135]: 2024-05-07T02:45:33.430+0000 7f242ea00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.333280+0000 front 2024-05-07T02:45:02.333209+0000 (oldest deadline 2024-05-07T02:45:28.134185+0000) 2024-05-07T02:45:33.895 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:33 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-6[58135]: 2024-05-07T02:45:33.430+0000 7f242ea00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:08.135889+0000 front 2024-05-07T02:45:08.135099+0000 (oldest deadline 2024-05-07T02:45:32.835349+0000) 2024-05-07T02:45:34.207 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:33 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:33.781+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.600018+0000 front 2024-05-07T02:44:52.600047+0000 (oldest deadline 2024-05-07T02:45:17.900188+0000) 2024-05-07T02:45:34.207 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:33 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:33.781+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:44:57.900629+0000 front 2024-05-07T02:44:57.900606+0000 (oldest deadline 2024-05-07T02:45:23.800730+0000) 2024-05-07T02:45:34.207 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:33 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:33.781+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:10.202683+0000 front 2024-05-07T02:45:10.202506+0000 (oldest deadline 2024-05-07T02:45:32.502386+0000) 2024-05-07T02:45:34.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:34 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:34.049+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:34.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:34 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:34.049+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.135339+0000 front 2024-05-07T02:44:54.135247+0000 (oldest deadline 2024-05-07T02:45:17.035496+0000) 2024-05-07T02:45:34.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:34 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:34.049+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.938208+0000 front 2024-05-07T02:45:02.938286+0000 (oldest deadline 2024-05-07T02:45:24.638420+0000) 2024-05-07T02:45:34.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:34 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:34.049+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:08.741183+0000 front 2024-05-07T02:45:08.741806+0000 (oldest deadline 2024-05-07T02:45:34.041498+0000) 2024-05-07T02:45:34.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:34 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:34.549+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.527515+0000 front 2024-05-07T02:44:52.527624+0000 (oldest deadline 2024-05-07T02:45:17.827549+0000) 2024-05-07T02:45:34.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:34 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:34.549+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.528686+0000 front 2024-05-07T02:45:02.528717+0000 (oldest deadline 2024-05-07T02:45:25.428840+0000) 2024-05-07T02:45:34.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:34 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:34.406+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:34.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:34 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:34.407+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:45:34.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:34 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:34.407+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:45:34.895 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:34 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-6[58135]: 2024-05-07T02:45:34.472+0000 7f242ea00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:51.829136+0000 front 2024-05-07T02:44:51.829028+0000 (oldest deadline 2024-05-07T02:45:17.129289+0000) 2024-05-07T02:45:34.895 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:34 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-6[58135]: 2024-05-07T02:45:34.472+0000 7f242ea00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.333280+0000 front 2024-05-07T02:45:02.333209+0000 (oldest deadline 2024-05-07T02:45:28.134185+0000) 2024-05-07T02:45:34.896 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:34 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-6[58135]: 2024-05-07T02:45:34.472+0000 7f242ea00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:08.135889+0000 front 2024-05-07T02:45:08.135099+0000 (oldest deadline 2024-05-07T02:45:32.835349+0000) 2024-05-07T02:45:34.957 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:34 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:34.739+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.600018+0000 front 2024-05-07T02:44:52.600047+0000 (oldest deadline 2024-05-07T02:45:17.900188+0000) 2024-05-07T02:45:34.957 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:34 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:34.739+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:44:57.900629+0000 front 2024-05-07T02:44:57.900606+0000 (oldest deadline 2024-05-07T02:45:23.800730+0000) 2024-05-07T02:45:34.957 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:34 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:34.739+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:10.202683+0000 front 2024-05-07T02:45:10.202506+0000 (oldest deadline 2024-05-07T02:45:32.502386+0000) 2024-05-07T02:45:34.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:34 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:34.503+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:34.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:34 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:34.503+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:45:34.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:34 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:34.503+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:01.844911+0000 front 2024-05-07T02:45:01.844958+0000 (oldest deadline 2024-05-07T02:45:27.745107+0000) 2024-05-07T02:45:34.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:34 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:34.503+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:07.746120+0000 front 2024-05-07T02:45:07.745779+0000 (oldest deadline 2024-05-07T02:45:33.645623+0000) 2024-05-07T02:45:35.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:35 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:35.070+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:35.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:35 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:35.070+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.135339+0000 front 2024-05-07T02:44:54.135247+0000 (oldest deadline 2024-05-07T02:45:17.035496+0000) 2024-05-07T02:45:35.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:35 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:35.070+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.938208+0000 front 2024-05-07T02:45:02.938286+0000 (oldest deadline 2024-05-07T02:45:24.638420+0000) 2024-05-07T02:45:35.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:35 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:35.070+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:08.741183+0000 front 2024-05-07T02:45:08.741806+0000 (oldest deadline 2024-05-07T02:45:34.041498+0000) 2024-05-07T02:45:35.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:35 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:35.527+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.527515+0000 front 2024-05-07T02:44:52.527624+0000 (oldest deadline 2024-05-07T02:45:17.827549+0000) 2024-05-07T02:45:35.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:35 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:35.527+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.528686+0000 front 2024-05-07T02:45:02.528717+0000 (oldest deadline 2024-05-07T02:45:25.428840+0000) 2024-05-07T02:45:35.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:35 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:35.374+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:35.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:35 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:35.374+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:45:35.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:35 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:35.374+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:45:35.895 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:35 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-6[58135]: 2024-05-07T02:45:35.477+0000 7f242ea00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:51.829136+0000 front 2024-05-07T02:44:51.829028+0000 (oldest deadline 2024-05-07T02:45:17.129289+0000) 2024-05-07T02:45:35.895 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:35 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-6[58135]: 2024-05-07T02:45:35.477+0000 7f242ea00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.333280+0000 front 2024-05-07T02:45:02.333209+0000 (oldest deadline 2024-05-07T02:45:28.134185+0000) 2024-05-07T02:45:35.896 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:35 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-6[58135]: 2024-05-07T02:45:35.477+0000 7f242ea00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:08.135889+0000 front 2024-05-07T02:45:08.135099+0000 (oldest deadline 2024-05-07T02:45:32.835349+0000) 2024-05-07T02:45:35.957 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:35 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:35.721+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.600018+0000 front 2024-05-07T02:44:52.600047+0000 (oldest deadline 2024-05-07T02:45:17.900188+0000) 2024-05-07T02:45:35.957 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:35 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:35.721+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:44:57.900629+0000 front 2024-05-07T02:44:57.900606+0000 (oldest deadline 2024-05-07T02:45:23.800730+0000) 2024-05-07T02:45:35.957 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:35 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:35.721+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:10.202683+0000 front 2024-05-07T02:45:10.202506+0000 (oldest deadline 2024-05-07T02:45:32.502386+0000) 2024-05-07T02:45:35.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:35 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:35.485+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:35.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:35 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:35.485+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:45:35.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:35 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:35.485+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:01.844911+0000 front 2024-05-07T02:45:01.844958+0000 (oldest deadline 2024-05-07T02:45:27.745107+0000) 2024-05-07T02:45:35.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:35 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:35.485+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:07.746120+0000 front 2024-05-07T02:45:07.745779+0000 (oldest deadline 2024-05-07T02:45:33.645623+0000) 2024-05-07T02:45:36.398 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:36 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:36.100+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:36.399 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:36 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:36.100+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.135339+0000 front 2024-05-07T02:44:54.135247+0000 (oldest deadline 2024-05-07T02:45:17.035496+0000) 2024-05-07T02:45:36.399 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:36 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:36.100+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.938208+0000 front 2024-05-07T02:45:02.938286+0000 (oldest deadline 2024-05-07T02:45:24.638420+0000) 2024-05-07T02:45:36.399 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:36 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:36.100+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:08.741183+0000 front 2024-05-07T02:45:08.741806+0000 (oldest deadline 2024-05-07T02:45:34.041498+0000) 2024-05-07T02:45:36.793 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:36 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-6[58135]: 2024-05-07T02:45:36.525+0000 7f242ea00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:51.829136+0000 front 2024-05-07T02:44:51.829028+0000 (oldest deadline 2024-05-07T02:45:17.129289+0000) 2024-05-07T02:45:36.793 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:36 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-6[58135]: 2024-05-07T02:45:36.525+0000 7f242ea00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.333280+0000 front 2024-05-07T02:45:02.333209+0000 (oldest deadline 2024-05-07T02:45:28.134185+0000) 2024-05-07T02:45:36.794 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:36 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-6[58135]: 2024-05-07T02:45:36.525+0000 7f242ea00700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:08.135889+0000 front 2024-05-07T02:45:08.135099+0000 (oldest deadline 2024-05-07T02:45:32.835349+0000) 2024-05-07T02:45:36.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:36 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:36.543+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.527515+0000 front 2024-05-07T02:44:52.527624+0000 (oldest deadline 2024-05-07T02:45:17.827549+0000) 2024-05-07T02:45:36.860 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:36 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:36.543+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.528686+0000 front 2024-05-07T02:45:02.528717+0000 (oldest deadline 2024-05-07T02:45:25.428840+0000) 2024-05-07T02:45:36.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:36 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:36.360+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:36.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:36 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:36.360+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:45:36.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:36 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:36.360+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:45:36.957 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:36 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:36.720+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.600018+0000 front 2024-05-07T02:44:52.600047+0000 (oldest deadline 2024-05-07T02:45:17.900188+0000) 2024-05-07T02:45:36.957 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:36 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:36.720+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:44:57.900629+0000 front 2024-05-07T02:44:57.900606+0000 (oldest deadline 2024-05-07T02:45:23.800730+0000) 2024-05-07T02:45:36.957 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:36 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:36.720+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:10.202683+0000 front 2024-05-07T02:45:10.202506+0000 (oldest deadline 2024-05-07T02:45:32.502386+0000) 2024-05-07T02:45:36.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:36 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:36.522+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:36.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:36 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:36.522+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:45:36.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:36 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:36.522+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:01.844911+0000 front 2024-05-07T02:45:01.844958+0000 (oldest deadline 2024-05-07T02:45:27.745107+0000) 2024-05-07T02:45:36.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:36 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:36.522+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:07.746120+0000 front 2024-05-07T02:45:07.745779+0000 (oldest deadline 2024-05-07T02:45:33.645623+0000) 2024-05-07T02:45:37.396 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:37 smithi019 podman[114559]: 2024-05-07 02:45:37.287712428 +0000 UTC m=+5.608931479 container died de8d8c5971b026932f9171fc343bfa044ce089b6c1659f1933ad4a20974f9352 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True) 2024-05-07T02:45:37.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:37 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:37.074+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:37.397 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:37 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:37.074+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.135339+0000 front 2024-05-07T02:44:54.135247+0000 (oldest deadline 2024-05-07T02:45:17.035496+0000) 2024-05-07T02:45:37.397 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:37 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:37.074+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.938208+0000 front 2024-05-07T02:45:02.938286+0000 (oldest deadline 2024-05-07T02:45:24.638420+0000) 2024-05-07T02:45:37.397 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:37 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:37.074+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:08.741183+0000 front 2024-05-07T02:45:08.741806+0000 (oldest deadline 2024-05-07T02:45:34.041498+0000) 2024-05-07T02:45:37.757 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:37 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:37.477+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:37.757 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:37 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:37.477+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:45:37.758 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:37 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:37.477+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:01.844911+0000 front 2024-05-07T02:45:01.844958+0000 (oldest deadline 2024-05-07T02:45:27.745107+0000) 2024-05-07T02:45:37.758 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:37 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:37.477+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:07.746120+0000 front 2024-05-07T02:45:07.745779+0000 (oldest deadline 2024-05-07T02:45:33.645623+0000) 2024-05-07T02:45:37.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:37 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:37.519+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.527515+0000 front 2024-05-07T02:44:52.527624+0000 (oldest deadline 2024-05-07T02:45:17.827549+0000) 2024-05-07T02:45:37.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:37 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:37.519+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.528686+0000 front 2024-05-07T02:45:02.528717+0000 (oldest deadline 2024-05-07T02:45:25.428840+0000) 2024-05-07T02:45:37.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:37 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:37.519+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:11.230691+0000 front 2024-05-07T02:45:11.230829+0000 (oldest deadline 2024-05-07T02:45:37.130436+0000) 2024-05-07T02:45:37.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:37 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:37.363+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:37.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:37 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:37.363+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:45:37.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:37 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:37.363+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:45:37.885 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:37 smithi019 podman[114559]: 2024-05-07 02:45:37.847988564 +0000 UTC m=+6.169207609 container cleanup de8d8c5971b026932f9171fc343bfa044ce089b6c1659f1933ad4a20974f9352 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-6, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-07T02:45:37.886 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:37 smithi019 bash[114559]: ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-6 2024-05-07T02:45:38.207 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:37 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:37.756+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.600018+0000 front 2024-05-07T02:44:52.600047+0000 (oldest deadline 2024-05-07T02:45:17.900188+0000) 2024-05-07T02:45:38.207 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:37 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:37.756+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:44:57.900629+0000 front 2024-05-07T02:44:57.900606+0000 (oldest deadline 2024-05-07T02:45:23.800730+0000) 2024-05-07T02:45:38.207 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:37 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:37.756+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:10.202683+0000 front 2024-05-07T02:45:10.202506+0000 (oldest deadline 2024-05-07T02:45:32.502386+0000) 2024-05-07T02:45:38.396 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:38 smithi019 podman[114586]: 2024-05-07 02:45:38.074567955 +0000 UTC m=+0.771246475 container remove de8d8c5971b026932f9171fc343bfa044ce089b6c1659f1933ad4a20974f9352 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-6, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-07T02:45:38.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:38 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:38.040+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:38.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:38 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:38.040+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.135339+0000 front 2024-05-07T02:44:54.135247+0000 (oldest deadline 2024-05-07T02:45:17.035496+0000) 2024-05-07T02:45:38.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:38 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:38.040+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.938208+0000 front 2024-05-07T02:45:02.938286+0000 (oldest deadline 2024-05-07T02:45:24.638420+0000) 2024-05-07T02:45:38.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:38 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:38.040+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:08.741183+0000 front 2024-05-07T02:45:08.741806+0000 (oldest deadline 2024-05-07T02:45:34.041498+0000) 2024-05-07T02:45:38.609 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:38 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:38.502+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.527515+0000 front 2024-05-07T02:44:52.527624+0000 (oldest deadline 2024-05-07T02:45:17.827549+0000) 2024-05-07T02:45:38.609 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:38 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:38.502+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.528686+0000 front 2024-05-07T02:45:02.528717+0000 (oldest deadline 2024-05-07T02:45:25.428840+0000) 2024-05-07T02:45:38.609 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:38 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:38.502+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:11.230691+0000 front 2024-05-07T02:45:11.230829+0000 (oldest deadline 2024-05-07T02:45:37.130436+0000) 2024-05-07T02:45:38.610 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:38 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:38.355+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:38.610 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:38 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:38.355+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:45:38.610 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:38 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:38.355+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:45:38.957 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:38 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:38.714+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.600018+0000 front 2024-05-07T02:44:52.600047+0000 (oldest deadline 2024-05-07T02:45:17.900188+0000) 2024-05-07T02:45:38.957 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:38 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:38.714+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:44:57.900629+0000 front 2024-05-07T02:44:57.900606+0000 (oldest deadline 2024-05-07T02:45:23.800730+0000) 2024-05-07T02:45:38.958 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:38 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:38.714+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:10.202683+0000 front 2024-05-07T02:45:10.202506+0000 (oldest deadline 2024-05-07T02:45:32.502386+0000) 2024-05-07T02:45:38.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:38 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:38.508+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:38.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:38 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:38.508+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:45:38.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:38 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:38.508+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:01.844911+0000 front 2024-05-07T02:45:01.844958+0000 (oldest deadline 2024-05-07T02:45:27.745107+0000) 2024-05-07T02:45:38.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:38 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:38.508+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:07.746120+0000 front 2024-05-07T02:45:07.745779+0000 (oldest deadline 2024-05-07T02:45:33.645623+0000) 2024-05-07T02:45:39.280 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:38 smithi019 podman[114675]: 2024-05-07 02:45:38.832962455 +0000 UTC m=+0.042804394 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-07T02:45:39.281 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:38 smithi019 podman[114675]: 2024-05-07 02:45:38.975698577 +0000 UTC m=+0.185540502 container create 4bd5e1a1b859dd0debec45c70c84e6550c76f4b48fe1f0ac1d8e687613f26fc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-6-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240430, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-07T02:45:39.281 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:39 smithi019 podman[114675]: 2024-05-07 02:45:39.279634062 +0000 UTC m=+0.489476027 container init 4bd5e1a1b859dd0debec45c70c84e6550c76f4b48fe1f0ac1d8e687613f26fc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-6-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-07T02:45:39.281 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:39 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:39.012+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:39.281 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:39 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:39.012+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.135339+0000 front 2024-05-07T02:44:54.135247+0000 (oldest deadline 2024-05-07T02:45:17.035496+0000) 2024-05-07T02:45:39.281 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:39 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:39.012+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.938208+0000 front 2024-05-07T02:45:02.938286+0000 (oldest deadline 2024-05-07T02:45:24.638420+0000) 2024-05-07T02:45:39.281 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:39 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:39.012+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:08.741183+0000 front 2024-05-07T02:45:08.741806+0000 (oldest deadline 2024-05-07T02:45:34.041498+0000) 2024-05-07T02:45:39.646 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:39 smithi019 podman[114675]: 2024-05-07 02:45:39.292067971 +0000 UTC m=+0.501909933 container start 4bd5e1a1b859dd0debec45c70c84e6550c76f4b48fe1f0ac1d8e687613f26fc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-07T02:45:39.646 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:39 smithi019 podman[114675]: 2024-05-07 02:45:39.37827216 +0000 UTC m=+0.588114115 container attach 4bd5e1a1b859dd0debec45c70c84e6550c76f4b48fe1f0ac1d8e687613f26fc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-6-deactivate, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-07T02:45:39.724 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:39 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:39.404+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:39.724 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:39 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:39.404+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:45:39.724 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:39 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:39.404+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:45:39.724 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:39 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:39.494+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.527515+0000 front 2024-05-07T02:44:52.527624+0000 (oldest deadline 2024-05-07T02:45:17.827549+0000) 2024-05-07T02:45:39.725 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:39 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:39.494+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.528686+0000 front 2024-05-07T02:45:02.528717+0000 (oldest deadline 2024-05-07T02:45:25.428840+0000) 2024-05-07T02:45:39.725 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:39 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:39.494+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:11.230691+0000 front 2024-05-07T02:45:11.230829+0000 (oldest deadline 2024-05-07T02:45:37.130436+0000) 2024-05-07T02:45:39.957 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:39 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:39.733+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.600018+0000 front 2024-05-07T02:44:52.600047+0000 (oldest deadline 2024-05-07T02:45:17.900188+0000) 2024-05-07T02:45:39.957 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:39 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:39.733+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:44:57.900629+0000 front 2024-05-07T02:44:57.900606+0000 (oldest deadline 2024-05-07T02:45:23.800730+0000) 2024-05-07T02:45:39.958 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:39 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:39.733+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:10.202683+0000 front 2024-05-07T02:45:10.202506+0000 (oldest deadline 2024-05-07T02:45:32.502386+0000) 2024-05-07T02:45:39.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:39 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:39.489+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:39.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:39 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:39.489+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:45:39.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:39 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:39.489+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:01.844911+0000 front 2024-05-07T02:45:01.844958+0000 (oldest deadline 2024-05-07T02:45:27.745107+0000) 2024-05-07T02:45:39.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:39 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:39.489+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:07.746120+0000 front 2024-05-07T02:45:07.745779+0000 (oldest deadline 2024-05-07T02:45:33.645623+0000) 2024-05-07T02:45:40.040 INFO:journalctl@ceph.osd.6.smithi019.stdout:May 07 02:45:39 smithi019 podman[114675]: 2024-05-07 02:45:39.773675024 +0000 UTC m=+0.983516969 container died 4bd5e1a1b859dd0debec45c70c84e6550c76f4b48fe1f0ac1d8e687613f26fc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-07T02:45:40.395 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:40 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:40.038+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:40.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:40 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:40.039+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.135339+0000 front 2024-05-07T02:44:54.135247+0000 (oldest deadline 2024-05-07T02:45:17.035496+0000) 2024-05-07T02:45:40.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:40 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:40.039+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.938208+0000 front 2024-05-07T02:45:02.938286+0000 (oldest deadline 2024-05-07T02:45:24.638420+0000) 2024-05-07T02:45:40.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:40 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:40.039+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:08.741183+0000 front 2024-05-07T02:45:08.741806+0000 (oldest deadline 2024-05-07T02:45:34.041498+0000) 2024-05-07T02:45:40.708 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@osd.6.service' 2024-05-07T02:45:40.829 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-07T02:45:40.830 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-07T02:45:40.830 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-07T02:45:40.830 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@osd.7 2024-05-07T02:45:40.857 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:40 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:40.529+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:40.857 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:40 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:40.529+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:45:40.857 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:40 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:40.529+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:01.844911+0000 front 2024-05-07T02:45:01.844958+0000 (oldest deadline 2024-05-07T02:45:27.745107+0000) 2024-05-07T02:45:40.858 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:40 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:40.529+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:07.746120+0000 front 2024-05-07T02:45:07.745779+0000 (oldest deadline 2024-05-07T02:45:33.645623+0000) 2024-05-07T02:45:40.858 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:40 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:40.752+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.600018+0000 front 2024-05-07T02:44:52.600047+0000 (oldest deadline 2024-05-07T02:45:17.900188+0000) 2024-05-07T02:45:40.858 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:40 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:40.752+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:44:57.900629+0000 front 2024-05-07T02:44:57.900606+0000 (oldest deadline 2024-05-07T02:45:23.800730+0000) 2024-05-07T02:45:40.858 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:40 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:40.752+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:10.202683+0000 front 2024-05-07T02:45:10.202506+0000 (oldest deadline 2024-05-07T02:45:32.502386+0000) 2024-05-07T02:45:40.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:40 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:40.515+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.527515+0000 front 2024-05-07T02:44:52.527624+0000 (oldest deadline 2024-05-07T02:45:17.827549+0000) 2024-05-07T02:45:40.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:40 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:40.515+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.528686+0000 front 2024-05-07T02:45:02.528717+0000 (oldest deadline 2024-05-07T02:45:25.428840+0000) 2024-05-07T02:45:40.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:40 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:40.515+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:11.230691+0000 front 2024-05-07T02:45:11.230829+0000 (oldest deadline 2024-05-07T02:45:37.130436+0000) 2024-05-07T02:45:40.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:40 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:40.416+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:40.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:40 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:40.416+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:45:40.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:40 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:40.416+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:45:41.150 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:40 smithi027 systemd[1]: Stopping Ceph osd.7 for 9ff84c16-0c17-11ef-bc97-c7b262605968... 2024-05-07T02:45:41.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:41 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:41.055+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:41.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:41 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:41.055+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.135339+0000 front 2024-05-07T02:44:54.135247+0000 (oldest deadline 2024-05-07T02:45:17.035496+0000) 2024-05-07T02:45:41.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:41 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:41.055+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.938208+0000 front 2024-05-07T02:45:02.938286+0000 (oldest deadline 2024-05-07T02:45:24.638420+0000) 2024-05-07T02:45:41.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:41 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:41.055+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:08.741183+0000 front 2024-05-07T02:45:08.741806+0000 (oldest deadline 2024-05-07T02:45:34.041498+0000) 2024-05-07T02:45:41.456 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:41 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:41.150+0000 7fb27724e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-07T02:45:41.457 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:41 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:41.150+0000 7fb27724e700 -1 osd.7 104 *** Got signal Terminated *** 2024-05-07T02:45:41.457 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:41 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:41.150+0000 7fb27724e700 -1 osd.7 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-07T02:45:41.758 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:41 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:41.492+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:41.758 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:41 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:41.492+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:45:41.759 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:41 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:41.492+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:01.844911+0000 front 2024-05-07T02:45:01.844958+0000 (oldest deadline 2024-05-07T02:45:27.745107+0000) 2024-05-07T02:45:41.759 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:41 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:41.492+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:07.746120+0000 front 2024-05-07T02:45:07.745779+0000 (oldest deadline 2024-05-07T02:45:33.645623+0000) 2024-05-07T02:45:41.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:41 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:41.517+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.527515+0000 front 2024-05-07T02:44:52.527624+0000 (oldest deadline 2024-05-07T02:45:17.827549+0000) 2024-05-07T02:45:41.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:41 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:41.517+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.528686+0000 front 2024-05-07T02:45:02.528717+0000 (oldest deadline 2024-05-07T02:45:25.428840+0000) 2024-05-07T02:45:41.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:41 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:41.517+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:11.230691+0000 front 2024-05-07T02:45:11.230829+0000 (oldest deadline 2024-05-07T02:45:37.130436+0000) 2024-05-07T02:45:41.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:41 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:41.436+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:41.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:41 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:41.436+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:45:41.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:41 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:41.436+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:45:42.207 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:41 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:41.758+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.600018+0000 front 2024-05-07T02:44:52.600047+0000 (oldest deadline 2024-05-07T02:45:17.900188+0000) 2024-05-07T02:45:42.207 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:41 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:41.758+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:44:57.900629+0000 front 2024-05-07T02:44:57.900606+0000 (oldest deadline 2024-05-07T02:45:23.800730+0000) 2024-05-07T02:45:42.207 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:41 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:41.758+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:10.202683+0000 front 2024-05-07T02:45:10.202506+0000 (oldest deadline 2024-05-07T02:45:32.502386+0000) 2024-05-07T02:45:42.395 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:42 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:42.038+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:42.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:42 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:42.038+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.135339+0000 front 2024-05-07T02:44:54.135247+0000 (oldest deadline 2024-05-07T02:45:17.035496+0000) 2024-05-07T02:45:42.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:42 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:42.038+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.938208+0000 front 2024-05-07T02:45:02.938286+0000 (oldest deadline 2024-05-07T02:45:24.638420+0000) 2024-05-07T02:45:42.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:42 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:42.038+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:08.741183+0000 front 2024-05-07T02:45:08.741806+0000 (oldest deadline 2024-05-07T02:45:34.041498+0000) 2024-05-07T02:45:42.707 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:42 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:42.446+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:42.707 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:42 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:42.446+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:45:42.707 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:42 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:42.446+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:01.844911+0000 front 2024-05-07T02:45:01.844958+0000 (oldest deadline 2024-05-07T02:45:27.745107+0000) 2024-05-07T02:45:42.707 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:42 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:42.446+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:07.746120+0000 front 2024-05-07T02:45:07.745779+0000 (oldest deadline 2024-05-07T02:45:33.645623+0000) 2024-05-07T02:45:42.708 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:42 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:42.446+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:18.846956+0000 front 2024-05-07T02:45:18.846993+0000 (oldest deadline 2024-05-07T02:45:42.247711+0000) 2024-05-07T02:45:42.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:42 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:42.519+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.527515+0000 front 2024-05-07T02:44:52.527624+0000 (oldest deadline 2024-05-07T02:45:17.827549+0000) 2024-05-07T02:45:42.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:42 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:42.519+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.528686+0000 front 2024-05-07T02:45:02.528717+0000 (oldest deadline 2024-05-07T02:45:25.428840+0000) 2024-05-07T02:45:42.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:42 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:42.519+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:11.230691+0000 front 2024-05-07T02:45:11.230829+0000 (oldest deadline 2024-05-07T02:45:37.130436+0000) 2024-05-07T02:45:42.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:42 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:42.519+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:18.231863+0000 front 2024-05-07T02:45:18.231883+0000 (oldest deadline 2024-05-07T02:45:42.232389+0000) 2024-05-07T02:45:42.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:42 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:42.442+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:42.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:42 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:42.442+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:45:42.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:42 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:42.442+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:45:42.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:42 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:42.442+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:20.067840+0000 front 2024-05-07T02:45:20.067791+0000 (oldest deadline 2024-05-07T02:45:42.367749+0000) 2024-05-07T02:45:43.207 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:42 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:42.756+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.600018+0000 front 2024-05-07T02:44:52.600047+0000 (oldest deadline 2024-05-07T02:45:17.900188+0000) 2024-05-07T02:45:43.207 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:42 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:42.756+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:44:57.900629+0000 front 2024-05-07T02:44:57.900606+0000 (oldest deadline 2024-05-07T02:45:23.800730+0000) 2024-05-07T02:45:43.207 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:42 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:42.756+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:10.202683+0000 front 2024-05-07T02:45:10.202506+0000 (oldest deadline 2024-05-07T02:45:32.502386+0000) 2024-05-07T02:45:43.207 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:42 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:42.756+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:20.004752+0000 front 2024-05-07T02:45:20.004780+0000 (oldest deadline 2024-05-07T02:45:42.304890+0000) 2024-05-07T02:45:43.395 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:43 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:43.086+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:43.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:43 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:43.086+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.135339+0000 front 2024-05-07T02:44:54.135247+0000 (oldest deadline 2024-05-07T02:45:17.035496+0000) 2024-05-07T02:45:43.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:43 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:43.086+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.938208+0000 front 2024-05-07T02:45:02.938286+0000 (oldest deadline 2024-05-07T02:45:24.638420+0000) 2024-05-07T02:45:43.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:43 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:43.086+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:08.741183+0000 front 2024-05-07T02:45:08.741806+0000 (oldest deadline 2024-05-07T02:45:34.041498+0000) 2024-05-07T02:45:43.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:43 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:43.086+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:19.943463+0000 front 2024-05-07T02:45:19.943793+0000 (oldest deadline 2024-05-07T02:45:42.243512+0000) 2024-05-07T02:45:43.753 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:43 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:43.473+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:43.753 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:43 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:43.473+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:45:43.753 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:43 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:43.473+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:01.844911+0000 front 2024-05-07T02:45:01.844958+0000 (oldest deadline 2024-05-07T02:45:27.745107+0000) 2024-05-07T02:45:43.753 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:43 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:43.473+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:07.746120+0000 front 2024-05-07T02:45:07.745779+0000 (oldest deadline 2024-05-07T02:45:33.645623+0000) 2024-05-07T02:45:43.753 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:43 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:43.473+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:18.846956+0000 front 2024-05-07T02:45:18.846993+0000 (oldest deadline 2024-05-07T02:45:42.247711+0000) 2024-05-07T02:45:43.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:43 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:43.482+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.527515+0000 front 2024-05-07T02:44:52.527624+0000 (oldest deadline 2024-05-07T02:45:17.827549+0000) 2024-05-07T02:45:43.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:43 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:43.482+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.528686+0000 front 2024-05-07T02:45:02.528717+0000 (oldest deadline 2024-05-07T02:45:25.428840+0000) 2024-05-07T02:45:43.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:43 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:43.482+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:11.230691+0000 front 2024-05-07T02:45:11.230829+0000 (oldest deadline 2024-05-07T02:45:37.130436+0000) 2024-05-07T02:45:43.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:43 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:43.482+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:18.231863+0000 front 2024-05-07T02:45:18.231883+0000 (oldest deadline 2024-05-07T02:45:42.232389+0000) 2024-05-07T02:45:43.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:43 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:43.483+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:43.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:43 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:43.483+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:45:43.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:43 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:43.483+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:45:43.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:43 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:43.483+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:20.067840+0000 front 2024-05-07T02:45:20.067791+0000 (oldest deadline 2024-05-07T02:45:42.367749+0000) 2024-05-07T02:45:44.207 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:43 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:43.752+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.600018+0000 front 2024-05-07T02:44:52.600047+0000 (oldest deadline 2024-05-07T02:45:17.900188+0000) 2024-05-07T02:45:44.207 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:43 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:43.752+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:44:57.900629+0000 front 2024-05-07T02:44:57.900606+0000 (oldest deadline 2024-05-07T02:45:23.800730+0000) 2024-05-07T02:45:44.207 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:43 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:43.752+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:10.202683+0000 front 2024-05-07T02:45:10.202506+0000 (oldest deadline 2024-05-07T02:45:32.502386+0000) 2024-05-07T02:45:44.207 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:43 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:43.752+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:20.004752+0000 front 2024-05-07T02:45:20.004780+0000 (oldest deadline 2024-05-07T02:45:42.304890+0000) 2024-05-07T02:45:44.395 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:44 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:44.119+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:44.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:44 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:44.119+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.135339+0000 front 2024-05-07T02:44:54.135247+0000 (oldest deadline 2024-05-07T02:45:17.035496+0000) 2024-05-07T02:45:44.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:44 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:44.119+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.938208+0000 front 2024-05-07T02:45:02.938286+0000 (oldest deadline 2024-05-07T02:45:24.638420+0000) 2024-05-07T02:45:44.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:44 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:44.119+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:08.741183+0000 front 2024-05-07T02:45:08.741806+0000 (oldest deadline 2024-05-07T02:45:34.041498+0000) 2024-05-07T02:45:44.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:44 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:44.119+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:19.943463+0000 front 2024-05-07T02:45:19.943793+0000 (oldest deadline 2024-05-07T02:45:42.243512+0000) 2024-05-07T02:45:44.769 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:44 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:44.462+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:44.769 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:44 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:44.462+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:45:44.769 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:44 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:44.462+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:01.844911+0000 front 2024-05-07T02:45:01.844958+0000 (oldest deadline 2024-05-07T02:45:27.745107+0000) 2024-05-07T02:45:44.769 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:44 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:44.462+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:07.746120+0000 front 2024-05-07T02:45:07.745779+0000 (oldest deadline 2024-05-07T02:45:33.645623+0000) 2024-05-07T02:45:44.770 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:44 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:44.462+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:18.846956+0000 front 2024-05-07T02:45:18.846993+0000 (oldest deadline 2024-05-07T02:45:42.247711+0000) 2024-05-07T02:45:44.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:44 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:44.495+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.527515+0000 front 2024-05-07T02:44:52.527624+0000 (oldest deadline 2024-05-07T02:45:17.827549+0000) 2024-05-07T02:45:44.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:44 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:44.495+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.528686+0000 front 2024-05-07T02:45:02.528717+0000 (oldest deadline 2024-05-07T02:45:25.428840+0000) 2024-05-07T02:45:44.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:44 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:44.495+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:11.230691+0000 front 2024-05-07T02:45:11.230829+0000 (oldest deadline 2024-05-07T02:45:37.130436+0000) 2024-05-07T02:45:44.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:44 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:44.495+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:18.231863+0000 front 2024-05-07T02:45:18.231883+0000 (oldest deadline 2024-05-07T02:45:42.232389+0000) 2024-05-07T02:45:44.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:44 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:44.518+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:44.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:44 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:44.518+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:45:44.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:44 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:44.518+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:45:44.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:44 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:44.518+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:20.067840+0000 front 2024-05-07T02:45:20.067791+0000 (oldest deadline 2024-05-07T02:45:42.367749+0000) 2024-05-07T02:45:45.207 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:44 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:44.769+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.600018+0000 front 2024-05-07T02:44:52.600047+0000 (oldest deadline 2024-05-07T02:45:17.900188+0000) 2024-05-07T02:45:45.207 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:44 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:44.769+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:44:57.900629+0000 front 2024-05-07T02:44:57.900606+0000 (oldest deadline 2024-05-07T02:45:23.800730+0000) 2024-05-07T02:45:45.208 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:44 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:44.769+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:10.202683+0000 front 2024-05-07T02:45:10.202506+0000 (oldest deadline 2024-05-07T02:45:32.502386+0000) 2024-05-07T02:45:45.208 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:44 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:44.769+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:20.004752+0000 front 2024-05-07T02:45:20.004780+0000 (oldest deadline 2024-05-07T02:45:42.304890+0000) 2024-05-07T02:45:45.395 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:45 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:45.094+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:45.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:45 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:45.094+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.135339+0000 front 2024-05-07T02:44:54.135247+0000 (oldest deadline 2024-05-07T02:45:17.035496+0000) 2024-05-07T02:45:45.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:45 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:45.094+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.938208+0000 front 2024-05-07T02:45:02.938286+0000 (oldest deadline 2024-05-07T02:45:24.638420+0000) 2024-05-07T02:45:45.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:45 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:45.094+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:08.741183+0000 front 2024-05-07T02:45:08.741806+0000 (oldest deadline 2024-05-07T02:45:34.041498+0000) 2024-05-07T02:45:45.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:45 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:45.094+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:19.943463+0000 front 2024-05-07T02:45:19.943793+0000 (oldest deadline 2024-05-07T02:45:42.243512+0000) 2024-05-07T02:45:45.803 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:45 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:45.488+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:45.803 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:45 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:45.488+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:45:45.803 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:45 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:45.488+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:01.844911+0000 front 2024-05-07T02:45:01.844958+0000 (oldest deadline 2024-05-07T02:45:27.745107+0000) 2024-05-07T02:45:45.803 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:45 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:45.488+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:07.746120+0000 front 2024-05-07T02:45:07.745779+0000 (oldest deadline 2024-05-07T02:45:33.645623+0000) 2024-05-07T02:45:45.803 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:45 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:45.488+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:18.846956+0000 front 2024-05-07T02:45:18.846993+0000 (oldest deadline 2024-05-07T02:45:42.247711+0000) 2024-05-07T02:45:45.861 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:45 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:45.480+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.527515+0000 front 2024-05-07T02:44:52.527624+0000 (oldest deadline 2024-05-07T02:45:17.827549+0000) 2024-05-07T02:45:45.862 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:45 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:45.480+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.528686+0000 front 2024-05-07T02:45:02.528717+0000 (oldest deadline 2024-05-07T02:45:25.428840+0000) 2024-05-07T02:45:45.862 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:45 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:45.480+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:11.230691+0000 front 2024-05-07T02:45:11.230829+0000 (oldest deadline 2024-05-07T02:45:37.130436+0000) 2024-05-07T02:45:45.862 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:45 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:45.480+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:18.231863+0000 front 2024-05-07T02:45:18.231883+0000 (oldest deadline 2024-05-07T02:45:42.232389+0000) 2024-05-07T02:45:45.862 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:45 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:45.511+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:45.862 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:45 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:45.511+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:45:45.863 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:45 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:45.511+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:45:45.863 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:45 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:45.511+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:20.067840+0000 front 2024-05-07T02:45:20.067791+0000 (oldest deadline 2024-05-07T02:45:42.367749+0000) 2024-05-07T02:45:46.095 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:45 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:45.802+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.600018+0000 front 2024-05-07T02:44:52.600047+0000 (oldest deadline 2024-05-07T02:45:17.900188+0000) 2024-05-07T02:45:46.095 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:45 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:45.802+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:44:57.900629+0000 front 2024-05-07T02:44:57.900606+0000 (oldest deadline 2024-05-07T02:45:23.800730+0000) 2024-05-07T02:45:46.095 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:45 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:45.802+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:10.202683+0000 front 2024-05-07T02:45:10.202506+0000 (oldest deadline 2024-05-07T02:45:32.502386+0000) 2024-05-07T02:45:46.095 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:45 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7[52389]: 2024-05-07T02:45:45.802+0000 7fb273804700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:20.004752+0000 front 2024-05-07T02:45:20.004780+0000 (oldest deadline 2024-05-07T02:45:42.304890+0000) 2024-05-07T02:45:46.395 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:46 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:46.045+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:46.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:46 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:46.045+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.135339+0000 front 2024-05-07T02:44:54.135247+0000 (oldest deadline 2024-05-07T02:45:17.035496+0000) 2024-05-07T02:45:46.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:46 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:46.045+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.938208+0000 front 2024-05-07T02:45:02.938286+0000 (oldest deadline 2024-05-07T02:45:24.638420+0000) 2024-05-07T02:45:46.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:46 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:46.045+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:08.741183+0000 front 2024-05-07T02:45:08.741806+0000 (oldest deadline 2024-05-07T02:45:34.041498+0000) 2024-05-07T02:45:46.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:46 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:46.045+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:19.943463+0000 front 2024-05-07T02:45:19.943793+0000 (oldest deadline 2024-05-07T02:45:42.243512+0000) 2024-05-07T02:45:46.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:46 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:46.512+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:46.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:46 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:46.512+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:45:46.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:46 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:46.512+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:45:46.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:46 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:46.512+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:20.067840+0000 front 2024-05-07T02:45:20.067791+0000 (oldest deadline 2024-05-07T02:45:42.367749+0000) 2024-05-07T02:45:46.860 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:46 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:46.491+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.527515+0000 front 2024-05-07T02:44:52.527624+0000 (oldest deadline 2024-05-07T02:45:17.827549+0000) 2024-05-07T02:45:46.860 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:46 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:46.491+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.528686+0000 front 2024-05-07T02:45:02.528717+0000 (oldest deadline 2024-05-07T02:45:25.428840+0000) 2024-05-07T02:45:46.860 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:46 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:46.491+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:11.230691+0000 front 2024-05-07T02:45:11.230829+0000 (oldest deadline 2024-05-07T02:45:37.130436+0000) 2024-05-07T02:45:46.860 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:46 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:46.491+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:18.231863+0000 front 2024-05-07T02:45:18.231883+0000 (oldest deadline 2024-05-07T02:45:42.232389+0000) 2024-05-07T02:45:46.957 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:46 smithi027 podman[72910]: 2024-05-07 02:45:46.587479024 +0000 UTC m=+5.552264469 container died 0781010a37bdc7c5ef1fba4d8a4257f2cb45602a6187f8fa4395781ea15841fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-07T02:45:46.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:46 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:46.499+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:46.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:46 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:46.499+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:45:46.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:46 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:46.499+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:01.844911+0000 front 2024-05-07T02:45:01.844958+0000 (oldest deadline 2024-05-07T02:45:27.745107+0000) 2024-05-07T02:45:46.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:46 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:46.499+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:07.746120+0000 front 2024-05-07T02:45:07.745779+0000 (oldest deadline 2024-05-07T02:45:33.645623+0000) 2024-05-07T02:45:46.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:46 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:46.499+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:18.846956+0000 front 2024-05-07T02:45:18.846993+0000 (oldest deadline 2024-05-07T02:45:42.247711+0000) 2024-05-07T02:45:47.368 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:47 smithi027 podman[72910]: 2024-05-07 02:45:47.083725703 +0000 UTC m=+6.048511140 container cleanup 0781010a37bdc7c5ef1fba4d8a4257f2cb45602a6187f8fa4395781ea15841fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-07T02:45:47.369 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:47 smithi027 bash[72910]: ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7 2024-05-07T02:45:47.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:47 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:47.017+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:47.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:47 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:47.017+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.135339+0000 front 2024-05-07T02:44:54.135247+0000 (oldest deadline 2024-05-07T02:45:17.035496+0000) 2024-05-07T02:45:47.397 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:47 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:47.017+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.938208+0000 front 2024-05-07T02:45:02.938286+0000 (oldest deadline 2024-05-07T02:45:24.638420+0000) 2024-05-07T02:45:47.397 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:47 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:47.017+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:08.741183+0000 front 2024-05-07T02:45:08.741806+0000 (oldest deadline 2024-05-07T02:45:34.041498+0000) 2024-05-07T02:45:47.397 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:47 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:47.017+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:19.943463+0000 front 2024-05-07T02:45:19.943793+0000 (oldest deadline 2024-05-07T02:45:42.243512+0000) 2024-05-07T02:45:47.694 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:47 smithi027 podman[72938]: 2024-05-07 02:45:47.3687455 +0000 UTC m=+0.775460174 container remove 0781010a37bdc7c5ef1fba4d8a4257f2cb45602a6187f8fa4395781ea15841fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-07T02:45:47.694 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:47 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:47.545+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:47.695 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:47 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:47.545+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:45:47.695 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:47 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:47.545+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:01.844911+0000 front 2024-05-07T02:45:01.844958+0000 (oldest deadline 2024-05-07T02:45:27.745107+0000) 2024-05-07T02:45:47.695 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:47 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:47.545+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:07.746120+0000 front 2024-05-07T02:45:07.745779+0000 (oldest deadline 2024-05-07T02:45:33.645623+0000) 2024-05-07T02:45:47.695 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:47 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:47.545+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:18.846956+0000 front 2024-05-07T02:45:18.846993+0000 (oldest deadline 2024-05-07T02:45:42.247711+0000) 2024-05-07T02:45:47.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:47 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:47.517+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.527515+0000 front 2024-05-07T02:44:52.527624+0000 (oldest deadline 2024-05-07T02:45:17.827549+0000) 2024-05-07T02:45:47.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:47 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:47.517+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.528686+0000 front 2024-05-07T02:45:02.528717+0000 (oldest deadline 2024-05-07T02:45:25.428840+0000) 2024-05-07T02:45:47.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:47 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:47.517+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:11.230691+0000 front 2024-05-07T02:45:11.230829+0000 (oldest deadline 2024-05-07T02:45:37.130436+0000) 2024-05-07T02:45:47.860 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:47 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:47.517+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:18.231863+0000 front 2024-05-07T02:45:18.231883+0000 (oldest deadline 2024-05-07T02:45:42.232389+0000) 2024-05-07T02:45:47.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:47 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:47.472+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:47.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:47 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:47.472+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:45:47.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:47 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:47.472+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:45:47.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:47 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:47.472+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:20.067840+0000 front 2024-05-07T02:45:20.067791+0000 (oldest deadline 2024-05-07T02:45:42.367749+0000) 2024-05-07T02:45:47.957 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:47 smithi027 podman[73026]: 2024-05-07 02:45:47.694696306 +0000 UTC m=+0.104996979 container create bbf0958bef6600640415e974d1d11743fbaa5f709ae928497f968edec69a39c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-05-07T02:45:47.957 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:47 smithi027 podman[73026]: 2024-05-07 02:45:47.608155768 +0000 UTC m=+0.018456418 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-07T02:45:48.269 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:47 smithi027 podman[73026]: 2024-05-07 02:45:47.963762277 +0000 UTC m=+0.374062908 container init bbf0958bef6600640415e974d1d11743fbaa5f709ae928497f968edec69a39c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-07T02:45:48.269 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:47 smithi027 podman[73026]: 2024-05-07 02:45:47.967233198 +0000 UTC m=+0.377533829 container start bbf0958bef6600640415e974d1d11743fbaa5f709ae928497f968edec69a39c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-07T02:45:48.270 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:48 smithi027 podman[73026]: 2024-05-07 02:45:48.047184067 +0000 UTC m=+0.457484719 container attach bbf0958bef6600640415e974d1d11743fbaa5f709ae928497f968edec69a39c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430) 2024-05-07T02:45:48.395 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:48 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:48.045+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:48.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:48 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:48.045+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.135339+0000 front 2024-05-07T02:44:54.135247+0000 (oldest deadline 2024-05-07T02:45:17.035496+0000) 2024-05-07T02:45:48.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:48 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:48.045+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.938208+0000 front 2024-05-07T02:45:02.938286+0000 (oldest deadline 2024-05-07T02:45:24.638420+0000) 2024-05-07T02:45:48.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:48 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:48.045+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:08.741183+0000 front 2024-05-07T02:45:08.741806+0000 (oldest deadline 2024-05-07T02:45:34.041498+0000) 2024-05-07T02:45:48.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:48 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:48.045+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:19.943463+0000 front 2024-05-07T02:45:19.943793+0000 (oldest deadline 2024-05-07T02:45:42.243512+0000) 2024-05-07T02:45:48.541 INFO:journalctl@ceph.osd.7.smithi027.stdout:May 07 02:45:48 smithi027 podman[73026]: 2024-05-07 02:45:48.269748108 +0000 UTC m=+0.680048746 container died bbf0958bef6600640415e974d1d11743fbaa5f709ae928497f968edec69a39c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-07T02:45:48.808 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:48 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:48.591+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:48.809 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:48 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:48.591+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:45:48.809 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:48 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:48.591+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:01.844911+0000 front 2024-05-07T02:45:01.844958+0000 (oldest deadline 2024-05-07T02:45:27.745107+0000) 2024-05-07T02:45:48.809 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:48 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:48.591+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:07.746120+0000 front 2024-05-07T02:45:07.745779+0000 (oldest deadline 2024-05-07T02:45:33.645623+0000) 2024-05-07T02:45:48.809 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:48 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:48.591+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:18.846956+0000 front 2024-05-07T02:45:18.846993+0000 (oldest deadline 2024-05-07T02:45:42.247711+0000) 2024-05-07T02:45:48.833 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@osd.7.service' 2024-05-07T02:45:48.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:48 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:48.522+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.527515+0000 front 2024-05-07T02:44:52.527624+0000 (oldest deadline 2024-05-07T02:45:17.827549+0000) 2024-05-07T02:45:48.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:48 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:48.522+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.528686+0000 front 2024-05-07T02:45:02.528717+0000 (oldest deadline 2024-05-07T02:45:25.428840+0000) 2024-05-07T02:45:48.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:48 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:48.522+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:11.230691+0000 front 2024-05-07T02:45:11.230829+0000 (oldest deadline 2024-05-07T02:45:37.130436+0000) 2024-05-07T02:45:48.860 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:48 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:48.522+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:18.231863+0000 front 2024-05-07T02:45:18.231883+0000 (oldest deadline 2024-05-07T02:45:42.232389+0000) 2024-05-07T02:45:48.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:48 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:48.461+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:48.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:48 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:48.461+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:45:48.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:48 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:48.461+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:45:48.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:48 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:48.461+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:20.067840+0000 front 2024-05-07T02:45:20.067791+0000 (oldest deadline 2024-05-07T02:45:42.367749+0000) 2024-05-07T02:45:48.914 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-07T02:45:48.914 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-07T02:45:48.915 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-07T02:45:48.915 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@osd.8 2024-05-07T02:45:49.248 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:48 smithi064 systemd[1]: Stopping Ceph osd.8 for 9ff84c16-0c17-11ef-bc97-c7b262605968... 2024-05-07T02:45:49.248 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:49 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:49.247+0000 7f27611de700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-07T02:45:49.248 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:49 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:49.247+0000 7f27611de700 -1 osd.8 104 *** Got signal Terminated *** 2024-05-07T02:45:49.249 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:49 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:49.247+0000 7f27611de700 -1 osd.8 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-07T02:45:49.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:49 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:49.003+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:49.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:49 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:49.003+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.135339+0000 front 2024-05-07T02:44:54.135247+0000 (oldest deadline 2024-05-07T02:45:17.035496+0000) 2024-05-07T02:45:49.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:49 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:49.003+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.938208+0000 front 2024-05-07T02:45:02.938286+0000 (oldest deadline 2024-05-07T02:45:24.638420+0000) 2024-05-07T02:45:49.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:49 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:49.003+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:08.741183+0000 front 2024-05-07T02:45:08.741806+0000 (oldest deadline 2024-05-07T02:45:34.041498+0000) 2024-05-07T02:45:49.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:49 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:49.003+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:19.943463+0000 front 2024-05-07T02:45:19.943793+0000 (oldest deadline 2024-05-07T02:45:42.243512+0000) 2024-05-07T02:45:49.499 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:49 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:49.473+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:49.499 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:49 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:49.473+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:45:49.499 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:49 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:49.473+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:45:49.500 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:49 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:49.473+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:20.067840+0000 front 2024-05-07T02:45:20.067791+0000 (oldest deadline 2024-05-07T02:45:42.367749+0000) 2024-05-07T02:45:49.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:49 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:49.499+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.527515+0000 front 2024-05-07T02:44:52.527624+0000 (oldest deadline 2024-05-07T02:45:17.827549+0000) 2024-05-07T02:45:49.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:49 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:49.499+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.528686+0000 front 2024-05-07T02:45:02.528717+0000 (oldest deadline 2024-05-07T02:45:25.428840+0000) 2024-05-07T02:45:49.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:49 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:49.499+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:11.230691+0000 front 2024-05-07T02:45:11.230829+0000 (oldest deadline 2024-05-07T02:45:37.130436+0000) 2024-05-07T02:45:49.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:49 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:49.499+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:18.231863+0000 front 2024-05-07T02:45:18.231883+0000 (oldest deadline 2024-05-07T02:45:42.232389+0000) 2024-05-07T02:45:49.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:49 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:49.571+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:49.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:49 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:49.571+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:45:49.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:49 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:49.571+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:01.844911+0000 front 2024-05-07T02:45:01.844958+0000 (oldest deadline 2024-05-07T02:45:27.745107+0000) 2024-05-07T02:45:49.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:49 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:49.571+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:07.746120+0000 front 2024-05-07T02:45:07.745779+0000 (oldest deadline 2024-05-07T02:45:33.645623+0000) 2024-05-07T02:45:49.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:49 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:49.571+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:18.846956+0000 front 2024-05-07T02:45:18.846993+0000 (oldest deadline 2024-05-07T02:45:42.247711+0000) 2024-05-07T02:45:50.395 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:50 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:50.044+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:50.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:50 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:50.044+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.135339+0000 front 2024-05-07T02:44:54.135247+0000 (oldest deadline 2024-05-07T02:45:17.035496+0000) 2024-05-07T02:45:50.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:50 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:50.044+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.938208+0000 front 2024-05-07T02:45:02.938286+0000 (oldest deadline 2024-05-07T02:45:24.638420+0000) 2024-05-07T02:45:50.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:50 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:50.044+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:08.741183+0000 front 2024-05-07T02:45:08.741806+0000 (oldest deadline 2024-05-07T02:45:34.041498+0000) 2024-05-07T02:45:50.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:50 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:50.044+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:19.943463+0000 front 2024-05-07T02:45:19.943793+0000 (oldest deadline 2024-05-07T02:45:42.243512+0000) 2024-05-07T02:45:50.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:50 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:50.537+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.527515+0000 front 2024-05-07T02:44:52.527624+0000 (oldest deadline 2024-05-07T02:45:17.827549+0000) 2024-05-07T02:45:50.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:50 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:50.537+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.528686+0000 front 2024-05-07T02:45:02.528717+0000 (oldest deadline 2024-05-07T02:45:25.428840+0000) 2024-05-07T02:45:50.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:50 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:50.537+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:11.230691+0000 front 2024-05-07T02:45:11.230829+0000 (oldest deadline 2024-05-07T02:45:37.130436+0000) 2024-05-07T02:45:50.860 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:50 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:50.537+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:18.231863+0000 front 2024-05-07T02:45:18.231883+0000 (oldest deadline 2024-05-07T02:45:42.232389+0000) 2024-05-07T02:45:50.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:50 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:50.470+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:50.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:50 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:50.470+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:45:50.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:50 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:50.470+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:45:50.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:50 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:50.470+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:20.067840+0000 front 2024-05-07T02:45:20.067791+0000 (oldest deadline 2024-05-07T02:45:42.367749+0000) 2024-05-07T02:45:50.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:50 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:50.615+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:50.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:50 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:50.615+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:45:50.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:50 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:50.615+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:01.844911+0000 front 2024-05-07T02:45:01.844958+0000 (oldest deadline 2024-05-07T02:45:27.745107+0000) 2024-05-07T02:45:50.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:50 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:50.615+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:07.746120+0000 front 2024-05-07T02:45:07.745779+0000 (oldest deadline 2024-05-07T02:45:33.645623+0000) 2024-05-07T02:45:50.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:50 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:50.615+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:18.846956+0000 front 2024-05-07T02:45:18.846993+0000 (oldest deadline 2024-05-07T02:45:42.247711+0000) 2024-05-07T02:45:50.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:50 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:50.615+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:25.649723+0000 front 2024-05-07T02:45:25.649626+0000 (oldest deadline 2024-05-07T02:45:49.749582+0000) 2024-05-07T02:45:51.395 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:51 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:51.055+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:51.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:51 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:51.055+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.135339+0000 front 2024-05-07T02:44:54.135247+0000 (oldest deadline 2024-05-07T02:45:17.035496+0000) 2024-05-07T02:45:51.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:51 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:51.055+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.938208+0000 front 2024-05-07T02:45:02.938286+0000 (oldest deadline 2024-05-07T02:45:24.638420+0000) 2024-05-07T02:45:51.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:51 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:51.055+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:08.741183+0000 front 2024-05-07T02:45:08.741806+0000 (oldest deadline 2024-05-07T02:45:34.041498+0000) 2024-05-07T02:45:51.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:51 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:51.055+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:19.943463+0000 front 2024-05-07T02:45:19.943793+0000 (oldest deadline 2024-05-07T02:45:42.243512+0000) 2024-05-07T02:45:51.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:51 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:51.055+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:26.748390+0000 front 2024-05-07T02:45:26.748526+0000 (oldest deadline 2024-05-07T02:45:50.848641+0000) 2024-05-07T02:45:51.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:51 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:51.436+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:51.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:51 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:51.436+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:45:51.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:51 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:51.436+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:45:51.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:51 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:51.436+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:20.067840+0000 front 2024-05-07T02:45:20.067791+0000 (oldest deadline 2024-05-07T02:45:42.367749+0000) 2024-05-07T02:45:51.860 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:51 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:51.549+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.527515+0000 front 2024-05-07T02:44:52.527624+0000 (oldest deadline 2024-05-07T02:45:17.827549+0000) 2024-05-07T02:45:51.860 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:51 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:51.549+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.528686+0000 front 2024-05-07T02:45:02.528717+0000 (oldest deadline 2024-05-07T02:45:25.428840+0000) 2024-05-07T02:45:51.860 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:51 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:51.549+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:11.230691+0000 front 2024-05-07T02:45:11.230829+0000 (oldest deadline 2024-05-07T02:45:37.130436+0000) 2024-05-07T02:45:51.860 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:51 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:51.549+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:18.231863+0000 front 2024-05-07T02:45:18.231883+0000 (oldest deadline 2024-05-07T02:45:42.232389+0000) 2024-05-07T02:45:51.860 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:51 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:51.549+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:26.233890+0000 front 2024-05-07T02:45:26.233964+0000 (oldest deadline 2024-05-07T02:45:50.834328+0000) 2024-05-07T02:45:51.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:51 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:51.627+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:51.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:51 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:51.627+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:45:51.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:51 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:51.627+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:01.844911+0000 front 2024-05-07T02:45:01.844958+0000 (oldest deadline 2024-05-07T02:45:27.745107+0000) 2024-05-07T02:45:51.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:51 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:51.627+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:07.746120+0000 front 2024-05-07T02:45:07.745779+0000 (oldest deadline 2024-05-07T02:45:33.645623+0000) 2024-05-07T02:45:51.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:51 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:51.627+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:18.846956+0000 front 2024-05-07T02:45:18.846993+0000 (oldest deadline 2024-05-07T02:45:42.247711+0000) 2024-05-07T02:45:51.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:51 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:51.627+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:25.649723+0000 front 2024-05-07T02:45:25.649626+0000 (oldest deadline 2024-05-07T02:45:49.749582+0000) 2024-05-07T02:45:52.395 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:52 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:52.030+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:52.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:52 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:52.030+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.135339+0000 front 2024-05-07T02:44:54.135247+0000 (oldest deadline 2024-05-07T02:45:17.035496+0000) 2024-05-07T02:45:52.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:52 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:52.030+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.938208+0000 front 2024-05-07T02:45:02.938286+0000 (oldest deadline 2024-05-07T02:45:24.638420+0000) 2024-05-07T02:45:52.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:52 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:52.030+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:08.741183+0000 front 2024-05-07T02:45:08.741806+0000 (oldest deadline 2024-05-07T02:45:34.041498+0000) 2024-05-07T02:45:52.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:52 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:52.030+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:19.943463+0000 front 2024-05-07T02:45:19.943793+0000 (oldest deadline 2024-05-07T02:45:42.243512+0000) 2024-05-07T02:45:52.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:52 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:52.030+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:26.748390+0000 front 2024-05-07T02:45:26.748526+0000 (oldest deadline 2024-05-07T02:45:50.848641+0000) 2024-05-07T02:45:52.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:52 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:52.434+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:52.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:52 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:52.434+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:45:52.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:52 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:52.434+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:45:52.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:52 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:52.434+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:20.067840+0000 front 2024-05-07T02:45:20.067791+0000 (oldest deadline 2024-05-07T02:45:42.367749+0000) 2024-05-07T02:45:52.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:52 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:52.434+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:24.668989+0000 front 2024-05-07T02:45:24.669015+0000 (oldest deadline 2024-05-07T02:45:52.269691+0000) 2024-05-07T02:45:52.860 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:52 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:52.534+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.527515+0000 front 2024-05-07T02:44:52.527624+0000 (oldest deadline 2024-05-07T02:45:17.827549+0000) 2024-05-07T02:45:52.860 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:52 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:52.534+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.528686+0000 front 2024-05-07T02:45:02.528717+0000 (oldest deadline 2024-05-07T02:45:25.428840+0000) 2024-05-07T02:45:52.860 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:52 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:52.534+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:11.230691+0000 front 2024-05-07T02:45:11.230829+0000 (oldest deadline 2024-05-07T02:45:37.130436+0000) 2024-05-07T02:45:52.860 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:52 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:52.534+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:18.231863+0000 front 2024-05-07T02:45:18.231883+0000 (oldest deadline 2024-05-07T02:45:42.232389+0000) 2024-05-07T02:45:52.860 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:52 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:52.534+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:26.233890+0000 front 2024-05-07T02:45:26.233964+0000 (oldest deadline 2024-05-07T02:45:50.834328+0000) 2024-05-07T02:45:52.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:52 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:52.653+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:52.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:52 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:52.653+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:45:52.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:52 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:52.653+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:01.844911+0000 front 2024-05-07T02:45:01.844958+0000 (oldest deadline 2024-05-07T02:45:27.745107+0000) 2024-05-07T02:45:52.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:52 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:52.653+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:07.746120+0000 front 2024-05-07T02:45:07.745779+0000 (oldest deadline 2024-05-07T02:45:33.645623+0000) 2024-05-07T02:45:52.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:52 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:52.653+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:18.846956+0000 front 2024-05-07T02:45:18.846993+0000 (oldest deadline 2024-05-07T02:45:42.247711+0000) 2024-05-07T02:45:52.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:52 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:52.653+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:25.649723+0000 front 2024-05-07T02:45:25.649626+0000 (oldest deadline 2024-05-07T02:45:49.749582+0000) 2024-05-07T02:45:53.395 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:52 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:52.982+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:53.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:52 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:52.982+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.135339+0000 front 2024-05-07T02:44:54.135247+0000 (oldest deadline 2024-05-07T02:45:17.035496+0000) 2024-05-07T02:45:53.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:52 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:52.982+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.938208+0000 front 2024-05-07T02:45:02.938286+0000 (oldest deadline 2024-05-07T02:45:24.638420+0000) 2024-05-07T02:45:53.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:52 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:52.982+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:08.741183+0000 front 2024-05-07T02:45:08.741806+0000 (oldest deadline 2024-05-07T02:45:34.041498+0000) 2024-05-07T02:45:53.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:52 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:52.982+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:19.943463+0000 front 2024-05-07T02:45:19.943793+0000 (oldest deadline 2024-05-07T02:45:42.243512+0000) 2024-05-07T02:45:53.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:52 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:52.982+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:26.748390+0000 front 2024-05-07T02:45:26.748526+0000 (oldest deadline 2024-05-07T02:45:50.848641+0000) 2024-05-07T02:45:53.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:53 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:53.549+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:52.527515+0000 front 2024-05-07T02:44:52.527624+0000 (oldest deadline 2024-05-07T02:45:17.827549+0000) 2024-05-07T02:45:53.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:53 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:53.549+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.528686+0000 front 2024-05-07T02:45:02.528717+0000 (oldest deadline 2024-05-07T02:45:25.428840+0000) 2024-05-07T02:45:53.859 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:53 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:53.549+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:11.230691+0000 front 2024-05-07T02:45:11.230829+0000 (oldest deadline 2024-05-07T02:45:37.130436+0000) 2024-05-07T02:45:53.860 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:53 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:53.549+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:18.231863+0000 front 2024-05-07T02:45:18.231883+0000 (oldest deadline 2024-05-07T02:45:42.232389+0000) 2024-05-07T02:45:53.860 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:53 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8[53472]: 2024-05-07T02:45:53.549+0000 7f275978c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:26.233890+0000 front 2024-05-07T02:45:26.233964+0000 (oldest deadline 2024-05-07T02:45:50.834328+0000) 2024-05-07T02:45:53.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:53 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:53.452+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:53.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:53 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:53.452+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:45:53.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:53 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:53.452+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:45:53.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:53 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:53.452+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:20.067840+0000 front 2024-05-07T02:45:20.067791+0000 (oldest deadline 2024-05-07T02:45:42.367749+0000) 2024-05-07T02:45:53.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:53 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:53.452+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:24.668989+0000 front 2024-05-07T02:45:24.669015+0000 (oldest deadline 2024-05-07T02:45:52.269691+0000) 2024-05-07T02:45:53.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:53 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:53.631+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:53.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:53 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:53.631+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:45:53.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:53 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:53.631+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:01.844911+0000 front 2024-05-07T02:45:01.844958+0000 (oldest deadline 2024-05-07T02:45:27.745107+0000) 2024-05-07T02:45:53.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:53 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:53.631+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:07.746120+0000 front 2024-05-07T02:45:07.745779+0000 (oldest deadline 2024-05-07T02:45:33.645623+0000) 2024-05-07T02:45:53.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:53 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:53.631+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:18.846956+0000 front 2024-05-07T02:45:18.846993+0000 (oldest deadline 2024-05-07T02:45:42.247711+0000) 2024-05-07T02:45:53.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:53 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:53.631+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:25.649723+0000 front 2024-05-07T02:45:25.649626+0000 (oldest deadline 2024-05-07T02:45:49.749582+0000) 2024-05-07T02:45:54.395 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:54 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:54.017+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:54.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:54 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:54.017+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.135339+0000 front 2024-05-07T02:44:54.135247+0000 (oldest deadline 2024-05-07T02:45:17.035496+0000) 2024-05-07T02:45:54.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:54 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:54.017+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.938208+0000 front 2024-05-07T02:45:02.938286+0000 (oldest deadline 2024-05-07T02:45:24.638420+0000) 2024-05-07T02:45:54.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:54 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:54.017+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:08.741183+0000 front 2024-05-07T02:45:08.741806+0000 (oldest deadline 2024-05-07T02:45:34.041498+0000) 2024-05-07T02:45:54.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:54 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:54.017+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:19.943463+0000 front 2024-05-07T02:45:19.943793+0000 (oldest deadline 2024-05-07T02:45:42.243512+0000) 2024-05-07T02:45:54.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:54 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:54.017+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:26.748390+0000 front 2024-05-07T02:45:26.748526+0000 (oldest deadline 2024-05-07T02:45:50.848641+0000) 2024-05-07T02:45:54.802 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:54 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:54.427+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:54.803 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:54 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:54.427+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:45:54.803 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:54 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:54.427+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:45:54.803 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:54 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:54.427+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:20.067840+0000 front 2024-05-07T02:45:20.067791+0000 (oldest deadline 2024-05-07T02:45:42.367749+0000) 2024-05-07T02:45:54.803 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:54 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:54.427+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:24.668989+0000 front 2024-05-07T02:45:24.669015+0000 (oldest deadline 2024-05-07T02:45:52.269691+0000) 2024-05-07T02:45:54.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:54 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:54.587+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:54.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:54 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:54.587+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:45:54.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:54 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:54.587+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:01.844911+0000 front 2024-05-07T02:45:01.844958+0000 (oldest deadline 2024-05-07T02:45:27.745107+0000) 2024-05-07T02:45:54.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:54 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:54.587+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:07.746120+0000 front 2024-05-07T02:45:07.745779+0000 (oldest deadline 2024-05-07T02:45:33.645623+0000) 2024-05-07T02:45:54.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:54 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:54.587+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:18.846956+0000 front 2024-05-07T02:45:18.846993+0000 (oldest deadline 2024-05-07T02:45:42.247711+0000) 2024-05-07T02:45:54.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:54 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:54.587+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:25.649723+0000 front 2024-05-07T02:45:25.649626+0000 (oldest deadline 2024-05-07T02:45:49.749582+0000) 2024-05-07T02:45:55.057 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:54 smithi064 podman[73549]: 2024-05-07 02:45:54.802419147 +0000 UTC m=+5.675273514 container died 98e41fe2189337a0b32db790b81e5b906d8ab0542eba2284e51af9164070942e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-07T02:45:55.348 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:55 smithi064 podman[73549]: 2024-05-07 02:45:55.147769291 +0000 UTC m=+6.020623665 container cleanup 98e41fe2189337a0b32db790b81e5b906d8ab0542eba2284e51af9164070942e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-07T02:45:55.349 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:55 smithi064 bash[73549]: ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8 2024-05-07T02:45:55.395 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:54 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:54.987+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:55.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:54 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:54.987+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.135339+0000 front 2024-05-07T02:44:54.135247+0000 (oldest deadline 2024-05-07T02:45:17.035496+0000) 2024-05-07T02:45:55.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:54 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:54.987+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.938208+0000 front 2024-05-07T02:45:02.938286+0000 (oldest deadline 2024-05-07T02:45:24.638420+0000) 2024-05-07T02:45:55.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:54 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:54.987+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:08.741183+0000 front 2024-05-07T02:45:08.741806+0000 (oldest deadline 2024-05-07T02:45:34.041498+0000) 2024-05-07T02:45:55.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:54 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:54.987+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:19.943463+0000 front 2024-05-07T02:45:19.943793+0000 (oldest deadline 2024-05-07T02:45:42.243512+0000) 2024-05-07T02:45:55.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:54 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:54.987+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:26.748390+0000 front 2024-05-07T02:45:26.748526+0000 (oldest deadline 2024-05-07T02:45:50.848641+0000) 2024-05-07T02:45:55.609 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:55 smithi064 podman[73569]: 2024-05-07 02:45:55.349113171 +0000 UTC m=+0.541676742 container remove 98e41fe2189337a0b32db790b81e5b906d8ab0542eba2284e51af9164070942e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-07T02:45:55.609 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:55 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:55.467+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:55.609 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:55 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:55.467+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:45:55.609 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:55 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:55.467+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:45:55.610 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:55 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:55.467+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:20.067840+0000 front 2024-05-07T02:45:20.067791+0000 (oldest deadline 2024-05-07T02:45:42.367749+0000) 2024-05-07T02:45:55.610 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:55 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:55.467+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:24.668989+0000 front 2024-05-07T02:45:24.669015+0000 (oldest deadline 2024-05-07T02:45:52.269691+0000) 2024-05-07T02:45:55.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:55 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:55.545+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:55.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:55 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:55.545+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:45:55.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:55 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:55.545+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:01.844911+0000 front 2024-05-07T02:45:01.844958+0000 (oldest deadline 2024-05-07T02:45:27.745107+0000) 2024-05-07T02:45:55.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:55 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:55.545+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:07.746120+0000 front 2024-05-07T02:45:07.745779+0000 (oldest deadline 2024-05-07T02:45:33.645623+0000) 2024-05-07T02:45:55.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:55 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:55.545+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:18.846956+0000 front 2024-05-07T02:45:18.846993+0000 (oldest deadline 2024-05-07T02:45:42.247711+0000) 2024-05-07T02:45:55.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:55 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:55.545+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:25.649723+0000 front 2024-05-07T02:45:25.649626+0000 (oldest deadline 2024-05-07T02:45:49.749582+0000) 2024-05-07T02:45:56.109 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:55 smithi064 podman[73659]: 2024-05-07 02:45:55.67809362 +0000 UTC m=+0.018730382 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-07T02:45:56.109 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:55 smithi064 podman[73659]: 2024-05-07 02:45:55.808406734 +0000 UTC m=+0.149043532 container create dc082e5b13a48bc9e0ddf8f0052e47192e03dcee8a7f3a54bc12ba3285f1097e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a) 2024-05-07T02:45:56.109 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:56 smithi064 podman[73659]: 2024-05-07 02:45:56.002497487 +0000 UTC m=+0.343134229 container init dc082e5b13a48bc9e0ddf8f0052e47192e03dcee8a7f3a54bc12ba3285f1097e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-05-07T02:45:56.110 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:56 smithi064 podman[73659]: 2024-05-07 02:45:56.006077478 +0000 UTC m=+0.346714214 container start dc082e5b13a48bc9e0ddf8f0052e47192e03dcee8a7f3a54bc12ba3285f1097e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, ceph=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-07T02:45:56.389 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:56 smithi064 podman[73659]: 2024-05-07 02:45:56.236237562 +0000 UTC m=+0.576874321 container attach dc082e5b13a48bc9e0ddf8f0052e47192e03dcee8a7f3a54bc12ba3285f1097e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-07T02:45:56.389 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 07 02:45:56 smithi064 podman[73659]: 2024-05-07 02:45:56.31482107 +0000 UTC m=+0.655457816 container died dc082e5b13a48bc9e0ddf8f0052e47192e03dcee8a7f3a54bc12ba3285f1097e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-07T02:45:56.395 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:56 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:56.034+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:56.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:56 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:56.034+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.135339+0000 front 2024-05-07T02:44:54.135247+0000 (oldest deadline 2024-05-07T02:45:17.035496+0000) 2024-05-07T02:45:56.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:56 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:56.034+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.938208+0000 front 2024-05-07T02:45:02.938286+0000 (oldest deadline 2024-05-07T02:45:24.638420+0000) 2024-05-07T02:45:56.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:56 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:56.034+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:08.741183+0000 front 2024-05-07T02:45:08.741806+0000 (oldest deadline 2024-05-07T02:45:34.041498+0000) 2024-05-07T02:45:56.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:56 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:56.034+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:19.943463+0000 front 2024-05-07T02:45:19.943793+0000 (oldest deadline 2024-05-07T02:45:42.243512+0000) 2024-05-07T02:45:56.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:56 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:56.034+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:26.748390+0000 front 2024-05-07T02:45:26.748526+0000 (oldest deadline 2024-05-07T02:45:50.848641+0000) 2024-05-07T02:45:56.689 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:56 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:56.476+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:56.689 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:56 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:56.477+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:45:56.690 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:56 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:56.477+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:45:56.690 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:56 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:56.477+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:20.067840+0000 front 2024-05-07T02:45:20.067791+0000 (oldest deadline 2024-05-07T02:45:42.367749+0000) 2024-05-07T02:45:56.690 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:56 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:56.477+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:24.668989+0000 front 2024-05-07T02:45:24.669015+0000 (oldest deadline 2024-05-07T02:45:52.269691+0000) 2024-05-07T02:45:56.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:56 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:56.551+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:56.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:56 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:56.551+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:45:56.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:56 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:56.551+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:01.844911+0000 front 2024-05-07T02:45:01.844958+0000 (oldest deadline 2024-05-07T02:45:27.745107+0000) 2024-05-07T02:45:56.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:56 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:56.551+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:07.746120+0000 front 2024-05-07T02:45:07.745779+0000 (oldest deadline 2024-05-07T02:45:33.645623+0000) 2024-05-07T02:45:56.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:56 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:56.551+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:18.846956+0000 front 2024-05-07T02:45:18.846993+0000 (oldest deadline 2024-05-07T02:45:42.247711+0000) 2024-05-07T02:45:56.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:56 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:56.551+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:25.649723+0000 front 2024-05-07T02:45:25.649626+0000 (oldest deadline 2024-05-07T02:45:49.749582+0000) 2024-05-07T02:45:57.028 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@osd.8.service' 2024-05-07T02:45:57.067 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-07T02:45:57.067 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-07T02:45:57.067 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-07T02:45:57.067 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@osd.9 2024-05-07T02:45:57.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:57 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:57.010+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:57.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:57 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:57.010+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.135339+0000 front 2024-05-07T02:44:54.135247+0000 (oldest deadline 2024-05-07T02:45:17.035496+0000) 2024-05-07T02:45:57.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:57 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:57.010+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.938208+0000 front 2024-05-07T02:45:02.938286+0000 (oldest deadline 2024-05-07T02:45:24.638420+0000) 2024-05-07T02:45:57.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:57 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:57.010+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:08.741183+0000 front 2024-05-07T02:45:08.741806+0000 (oldest deadline 2024-05-07T02:45:34.041498+0000) 2024-05-07T02:45:57.397 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:57 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:57.010+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:19.943463+0000 front 2024-05-07T02:45:19.943793+0000 (oldest deadline 2024-05-07T02:45:42.243512+0000) 2024-05-07T02:45:57.397 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:57 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:57.010+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:26.748390+0000 front 2024-05-07T02:45:26.748526+0000 (oldest deadline 2024-05-07T02:45:50.848641+0000) 2024-05-07T02:45:57.397 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:57 smithi019 systemd[1]: Stopping Ceph osd.9 for 9ff84c16-0c17-11ef-bc97-c7b262605968... 2024-05-07T02:45:57.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:57 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:57.501+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:57.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:57 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:57.501+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:45:57.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:57 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:57.501+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:45:57.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:57 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:57.501+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:20.067840+0000 front 2024-05-07T02:45:20.067791+0000 (oldest deadline 2024-05-07T02:45:42.367749+0000) 2024-05-07T02:45:57.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:57 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:57.501+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:24.668989+0000 front 2024-05-07T02:45:24.669015+0000 (oldest deadline 2024-05-07T02:45:52.269691+0000) 2024-05-07T02:45:57.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:57 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:57.531+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:57.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:57 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:57.531+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:45:57.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:57 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:57.531+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:01.844911+0000 front 2024-05-07T02:45:01.844958+0000 (oldest deadline 2024-05-07T02:45:27.745107+0000) 2024-05-07T02:45:57.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:57 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:57.531+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:07.746120+0000 front 2024-05-07T02:45:07.745779+0000 (oldest deadline 2024-05-07T02:45:33.645623+0000) 2024-05-07T02:45:57.959 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:57 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:57.531+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:18.846956+0000 front 2024-05-07T02:45:18.846993+0000 (oldest deadline 2024-05-07T02:45:42.247711+0000) 2024-05-07T02:45:57.959 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:57 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:57.531+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:25.649723+0000 front 2024-05-07T02:45:25.649626+0000 (oldest deadline 2024-05-07T02:45:49.749582+0000) 2024-05-07T02:45:57.959 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:57 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:57.531+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-07T02:45:34.450853+0000 front 2024-05-07T02:45:34.450979+0000 (oldest deadline 2024-05-07T02:45:57.350626+0000) 2024-05-07T02:45:58.041 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:57 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:57.659+0000 7f3156000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-07T02:45:58.041 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:57 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:57.659+0000 7f3156000700 -1 osd.9 104 *** Got signal Terminated *** 2024-05-07T02:45:58.041 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:57 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:57.659+0000 7f3156000700 -1 osd.9 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-07T02:45:58.042 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:58 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:58.040+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:58.395 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:58 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:58.040+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.135339+0000 front 2024-05-07T02:44:54.135247+0000 (oldest deadline 2024-05-07T02:45:17.035496+0000) 2024-05-07T02:45:58.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:58 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:58.040+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.938208+0000 front 2024-05-07T02:45:02.938286+0000 (oldest deadline 2024-05-07T02:45:24.638420+0000) 2024-05-07T02:45:58.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:58 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:58.040+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:08.741183+0000 front 2024-05-07T02:45:08.741806+0000 (oldest deadline 2024-05-07T02:45:34.041498+0000) 2024-05-07T02:45:58.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:58 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:58.040+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:19.943463+0000 front 2024-05-07T02:45:19.943793+0000 (oldest deadline 2024-05-07T02:45:42.243512+0000) 2024-05-07T02:45:58.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:58 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:58.040+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:26.748390+0000 front 2024-05-07T02:45:26.748526+0000 (oldest deadline 2024-05-07T02:45:50.848641+0000) 2024-05-07T02:45:58.707 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:58 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:58.538+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:58.707 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:58 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:58.538+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:45:58.707 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:58 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:58.538+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:01.844911+0000 front 2024-05-07T02:45:01.844958+0000 (oldest deadline 2024-05-07T02:45:27.745107+0000) 2024-05-07T02:45:58.708 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:58 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:58.538+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:07.746120+0000 front 2024-05-07T02:45:07.745779+0000 (oldest deadline 2024-05-07T02:45:33.645623+0000) 2024-05-07T02:45:58.708 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:58 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:58.538+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:18.846956+0000 front 2024-05-07T02:45:18.846993+0000 (oldest deadline 2024-05-07T02:45:42.247711+0000) 2024-05-07T02:45:58.708 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:58 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:58.538+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:25.649723+0000 front 2024-05-07T02:45:25.649626+0000 (oldest deadline 2024-05-07T02:45:49.749582+0000) 2024-05-07T02:45:58.708 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:58 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:58.538+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-07T02:45:34.450853+0000 front 2024-05-07T02:45:34.450979+0000 (oldest deadline 2024-05-07T02:45:57.350626+0000) 2024-05-07T02:45:58.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:58 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:58.536+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:58.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:58 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:58.536+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:45:58.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:58 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:58.536+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:45:58.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:58 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:58.536+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:20.067840+0000 front 2024-05-07T02:45:20.067791+0000 (oldest deadline 2024-05-07T02:45:42.367749+0000) 2024-05-07T02:45:58.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:58 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:58.536+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:24.668989+0000 front 2024-05-07T02:45:24.669015+0000 (oldest deadline 2024-05-07T02:45:52.269691+0000) 2024-05-07T02:45:59.395 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:58 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:58.997+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:45:59.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:58 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:58.997+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.135339+0000 front 2024-05-07T02:44:54.135247+0000 (oldest deadline 2024-05-07T02:45:17.035496+0000) 2024-05-07T02:45:59.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:58 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:58.997+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.938208+0000 front 2024-05-07T02:45:02.938286+0000 (oldest deadline 2024-05-07T02:45:24.638420+0000) 2024-05-07T02:45:59.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:58 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:58.997+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:08.741183+0000 front 2024-05-07T02:45:08.741806+0000 (oldest deadline 2024-05-07T02:45:34.041498+0000) 2024-05-07T02:45:59.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:58 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:58.997+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:19.943463+0000 front 2024-05-07T02:45:19.943793+0000 (oldest deadline 2024-05-07T02:45:42.243512+0000) 2024-05-07T02:45:59.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:45:58 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:45:58.997+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:26.748390+0000 front 2024-05-07T02:45:26.748526+0000 (oldest deadline 2024-05-07T02:45:50.848641+0000) 2024-05-07T02:45:59.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:59 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:59.493+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:45:59.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:59 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:59.493+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:45:59.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:59 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:59.493+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:45:59.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:59 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:59.493+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:20.067840+0000 front 2024-05-07T02:45:20.067791+0000 (oldest deadline 2024-05-07T02:45:42.367749+0000) 2024-05-07T02:45:59.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:45:59 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:45:59.493+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:24.668989+0000 front 2024-05-07T02:45:24.669015+0000 (oldest deadline 2024-05-07T02:45:52.269691+0000) 2024-05-07T02:45:59.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:59 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:59.506+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:45:59.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:59 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:59.506+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:45:59.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:59 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:59.506+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:01.844911+0000 front 2024-05-07T02:45:01.844958+0000 (oldest deadline 2024-05-07T02:45:27.745107+0000) 2024-05-07T02:45:59.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:59 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:59.506+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:07.746120+0000 front 2024-05-07T02:45:07.745779+0000 (oldest deadline 2024-05-07T02:45:33.645623+0000) 2024-05-07T02:45:59.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:59 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:59.506+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:18.846956+0000 front 2024-05-07T02:45:18.846993+0000 (oldest deadline 2024-05-07T02:45:42.247711+0000) 2024-05-07T02:45:59.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:59 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:59.506+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:25.649723+0000 front 2024-05-07T02:45:25.649626+0000 (oldest deadline 2024-05-07T02:45:49.749582+0000) 2024-05-07T02:45:59.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:45:59 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:45:59.506+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-07T02:45:34.450853+0000 front 2024-05-07T02:45:34.450979+0000 (oldest deadline 2024-05-07T02:45:57.350626+0000) 2024-05-07T02:46:00.395 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:46:00 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:46:00.027+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:46:00.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:46:00 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:46:00.027+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.135339+0000 front 2024-05-07T02:44:54.135247+0000 (oldest deadline 2024-05-07T02:45:17.035496+0000) 2024-05-07T02:46:00.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:46:00 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:46:00.027+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.938208+0000 front 2024-05-07T02:45:02.938286+0000 (oldest deadline 2024-05-07T02:45:24.638420+0000) 2024-05-07T02:46:00.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:46:00 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:46:00.027+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:08.741183+0000 front 2024-05-07T02:45:08.741806+0000 (oldest deadline 2024-05-07T02:45:34.041498+0000) 2024-05-07T02:46:00.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:46:00 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:46:00.027+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:19.943463+0000 front 2024-05-07T02:45:19.943793+0000 (oldest deadline 2024-05-07T02:45:42.243512+0000) 2024-05-07T02:46:00.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:46:00 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:46:00.027+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:26.748390+0000 front 2024-05-07T02:45:26.748526+0000 (oldest deadline 2024-05-07T02:45:50.848641+0000) 2024-05-07T02:46:00.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:00 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:00.444+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:46:00.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:00 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:00.444+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:46:00.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:00 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:00.444+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:46:00.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:00 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:00.444+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:20.067840+0000 front 2024-05-07T02:45:20.067791+0000 (oldest deadline 2024-05-07T02:45:42.367749+0000) 2024-05-07T02:46:00.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:00 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:00.444+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:24.668989+0000 front 2024-05-07T02:45:24.669015+0000 (oldest deadline 2024-05-07T02:45:52.269691+0000) 2024-05-07T02:46:00.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:00 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:00.511+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:46:00.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:00 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:00.511+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:46:00.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:00 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:00.511+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:01.844911+0000 front 2024-05-07T02:45:01.844958+0000 (oldest deadline 2024-05-07T02:45:27.745107+0000) 2024-05-07T02:46:00.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:00 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:00.511+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:07.746120+0000 front 2024-05-07T02:45:07.745779+0000 (oldest deadline 2024-05-07T02:45:33.645623+0000) 2024-05-07T02:46:00.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:00 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:00.511+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:18.846956+0000 front 2024-05-07T02:45:18.846993+0000 (oldest deadline 2024-05-07T02:45:42.247711+0000) 2024-05-07T02:46:00.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:00 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:00.511+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:25.649723+0000 front 2024-05-07T02:45:25.649626+0000 (oldest deadline 2024-05-07T02:45:49.749582+0000) 2024-05-07T02:46:00.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:00 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:00.511+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-07T02:45:34.450853+0000 front 2024-05-07T02:45:34.450979+0000 (oldest deadline 2024-05-07T02:45:57.350626+0000) 2024-05-07T02:46:01.395 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:46:00 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:46:00.978+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:46:01.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:46:00 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:46:00.978+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.135339+0000 front 2024-05-07T02:44:54.135247+0000 (oldest deadline 2024-05-07T02:45:17.035496+0000) 2024-05-07T02:46:01.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:46:00 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:46:00.978+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.938208+0000 front 2024-05-07T02:45:02.938286+0000 (oldest deadline 2024-05-07T02:45:24.638420+0000) 2024-05-07T02:46:01.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:46:00 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:46:00.978+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:08.741183+0000 front 2024-05-07T02:45:08.741806+0000 (oldest deadline 2024-05-07T02:45:34.041498+0000) 2024-05-07T02:46:01.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:46:00 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:46:00.978+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:19.943463+0000 front 2024-05-07T02:45:19.943793+0000 (oldest deadline 2024-05-07T02:45:42.243512+0000) 2024-05-07T02:46:01.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:46:00 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:46:00.978+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:26.748390+0000 front 2024-05-07T02:45:26.748526+0000 (oldest deadline 2024-05-07T02:45:50.848641+0000) 2024-05-07T02:46:01.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:01 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:01.491+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:46:01.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:01 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:01.491+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:46:01.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:01 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:01.491+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:46:01.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:01 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:01.491+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:20.067840+0000 front 2024-05-07T02:45:20.067791+0000 (oldest deadline 2024-05-07T02:45:42.367749+0000) 2024-05-07T02:46:01.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:01 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:01.491+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:24.668989+0000 front 2024-05-07T02:45:24.669015+0000 (oldest deadline 2024-05-07T02:45:52.269691+0000) 2024-05-07T02:46:01.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:01 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:01.491+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-07T02:45:36.672575+0000 front 2024-05-07T02:45:36.672606+0000 (oldest deadline 2024-05-07T02:46:00.772356+0000) 2024-05-07T02:46:01.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:01 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:01.465+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:46:01.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:01 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:01.465+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:46:01.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:01 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:01.465+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:01.844911+0000 front 2024-05-07T02:45:01.844958+0000 (oldest deadline 2024-05-07T02:45:27.745107+0000) 2024-05-07T02:46:01.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:01 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:01.465+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:07.746120+0000 front 2024-05-07T02:45:07.745779+0000 (oldest deadline 2024-05-07T02:45:33.645623+0000) 2024-05-07T02:46:01.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:01 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:01.465+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:18.846956+0000 front 2024-05-07T02:45:18.846993+0000 (oldest deadline 2024-05-07T02:45:42.247711+0000) 2024-05-07T02:46:01.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:01 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:01.465+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:25.649723+0000 front 2024-05-07T02:45:25.649626+0000 (oldest deadline 2024-05-07T02:45:49.749582+0000) 2024-05-07T02:46:01.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:01 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:01.465+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-07T02:45:34.450853+0000 front 2024-05-07T02:45:34.450979+0000 (oldest deadline 2024-05-07T02:45:57.350626+0000) 2024-05-07T02:46:02.395 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:46:01 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:46:01.963+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:41.931041+0000 front 2024-05-07T02:44:41.931258+0000 (oldest deadline 2024-05-07T02:45:07.230234+0000) 2024-05-07T02:46:02.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:46:01 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:46:01.963+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.135339+0000 front 2024-05-07T02:44:54.135247+0000 (oldest deadline 2024-05-07T02:45:17.035496+0000) 2024-05-07T02:46:02.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:46:01 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:46:01.963+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.938208+0000 front 2024-05-07T02:45:02.938286+0000 (oldest deadline 2024-05-07T02:45:24.638420+0000) 2024-05-07T02:46:02.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:46:01 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:46:01.963+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:08.741183+0000 front 2024-05-07T02:45:08.741806+0000 (oldest deadline 2024-05-07T02:45:34.041498+0000) 2024-05-07T02:46:02.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:46:01 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:46:01.963+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:19.943463+0000 front 2024-05-07T02:45:19.943793+0000 (oldest deadline 2024-05-07T02:45:42.243512+0000) 2024-05-07T02:46:02.396 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:46:01 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9[64719]: 2024-05-07T02:46:01.963+0000 7f3151800700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:26.748390+0000 front 2024-05-07T02:45:26.748526+0000 (oldest deadline 2024-05-07T02:45:50.848641+0000) 2024-05-07T02:46:02.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:02 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:02.537+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:46:02.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:02 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:02.537+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:46:02.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:02 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:02.537+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:46:02.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:02 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:02.537+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:20.067840+0000 front 2024-05-07T02:45:20.067791+0000 (oldest deadline 2024-05-07T02:45:42.367749+0000) 2024-05-07T02:46:02.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:02 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:02.537+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:24.668989+0000 front 2024-05-07T02:45:24.669015+0000 (oldest deadline 2024-05-07T02:45:52.269691+0000) 2024-05-07T02:46:02.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:02 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:02.537+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-07T02:45:36.672575+0000 front 2024-05-07T02:45:36.672606+0000 (oldest deadline 2024-05-07T02:46:00.772356+0000) 2024-05-07T02:46:02.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:02 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:02.503+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:46:02.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:02 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:02.503+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:46:02.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:02 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:02.503+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:01.844911+0000 front 2024-05-07T02:45:01.844958+0000 (oldest deadline 2024-05-07T02:45:27.745107+0000) 2024-05-07T02:46:02.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:02 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:02.503+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:07.746120+0000 front 2024-05-07T02:45:07.745779+0000 (oldest deadline 2024-05-07T02:45:33.645623+0000) 2024-05-07T02:46:02.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:02 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:02.503+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:18.846956+0000 front 2024-05-07T02:45:18.846993+0000 (oldest deadline 2024-05-07T02:45:42.247711+0000) 2024-05-07T02:46:02.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:02 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:02.503+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:25.649723+0000 front 2024-05-07T02:45:25.649626+0000 (oldest deadline 2024-05-07T02:45:49.749582+0000) 2024-05-07T02:46:02.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:02 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:02.503+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-07T02:45:34.450853+0000 front 2024-05-07T02:45:34.450979+0000 (oldest deadline 2024-05-07T02:45:57.350626+0000) 2024-05-07T02:46:03.540 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:46:03 smithi019 podman[114780]: 2024-05-07 02:46:03.262481331 +0000 UTC m=+5.751243425 container died af4e5352dd9e5f8124bf9b827354552846fe4c38e95fa0b1166a19417f13fb3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-07T02:46:03.707 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:03 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:03.453+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:46:03.707 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:03 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:03.454+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:46:03.707 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:03 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:03.454+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:01.844911+0000 front 2024-05-07T02:45:01.844958+0000 (oldest deadline 2024-05-07T02:45:27.745107+0000) 2024-05-07T02:46:03.707 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:03 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:03.454+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:07.746120+0000 front 2024-05-07T02:45:07.745779+0000 (oldest deadline 2024-05-07T02:45:33.645623+0000) 2024-05-07T02:46:03.708 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:03 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:03.454+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:18.846956+0000 front 2024-05-07T02:45:18.846993+0000 (oldest deadline 2024-05-07T02:45:42.247711+0000) 2024-05-07T02:46:03.708 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:03 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:03.454+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:25.649723+0000 front 2024-05-07T02:45:25.649626+0000 (oldest deadline 2024-05-07T02:45:49.749582+0000) 2024-05-07T02:46:03.708 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:03 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:03.454+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-07T02:45:34.450853+0000 front 2024-05-07T02:45:34.450979+0000 (oldest deadline 2024-05-07T02:45:57.350626+0000) 2024-05-07T02:46:03.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:03 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:03.537+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:46:03.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:03 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:03.537+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:46:03.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:03 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:03.537+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:46:03.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:03 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:03.537+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:20.067840+0000 front 2024-05-07T02:45:20.067791+0000 (oldest deadline 2024-05-07T02:45:42.367749+0000) 2024-05-07T02:46:03.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:03 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:03.537+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:24.668989+0000 front 2024-05-07T02:45:24.669015+0000 (oldest deadline 2024-05-07T02:45:52.269691+0000) 2024-05-07T02:46:03.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:03 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:03.537+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-07T02:45:36.672575+0000 front 2024-05-07T02:45:36.672606+0000 (oldest deadline 2024-05-07T02:46:00.772356+0000) 2024-05-07T02:46:04.145 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:46:03 smithi019 podman[114780]: 2024-05-07 02:46:03.801262448 +0000 UTC m=+6.290024514 container cleanup af4e5352dd9e5f8124bf9b827354552846fe4c38e95fa0b1166a19417f13fb3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240430) 2024-05-07T02:46:04.146 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:46:03 smithi019 bash[114780]: ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9 2024-05-07T02:46:04.146 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:46:03 smithi019 podman[114800]: 2024-05-07 02:46:03.994363407 +0000 UTC m=+0.714752922 container remove af4e5352dd9e5f8124bf9b827354552846fe4c38e95fa0b1166a19417f13fb3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430) 2024-05-07T02:46:04.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:04 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:04.512+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:46:04.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:04 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:04.512+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:46:04.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:04 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:04.512+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:46:04.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:04 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:04.512+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:20.067840+0000 front 2024-05-07T02:45:20.067791+0000 (oldest deadline 2024-05-07T02:45:42.367749+0000) 2024-05-07T02:46:04.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:04 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:04.512+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:24.668989+0000 front 2024-05-07T02:45:24.669015+0000 (oldest deadline 2024-05-07T02:45:52.269691+0000) 2024-05-07T02:46:04.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:04 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:04.512+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-07T02:45:36.672575+0000 front 2024-05-07T02:45:36.672606+0000 (oldest deadline 2024-05-07T02:46:00.772356+0000) 2024-05-07T02:46:04.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:04 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:04.489+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:46:04.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:04 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:04.489+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:46:04.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:04 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:04.489+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:01.844911+0000 front 2024-05-07T02:45:01.844958+0000 (oldest deadline 2024-05-07T02:45:27.745107+0000) 2024-05-07T02:46:04.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:04 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:04.489+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:07.746120+0000 front 2024-05-07T02:45:07.745779+0000 (oldest deadline 2024-05-07T02:45:33.645623+0000) 2024-05-07T02:46:04.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:04 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:04.489+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:18.846956+0000 front 2024-05-07T02:45:18.846993+0000 (oldest deadline 2024-05-07T02:45:42.247711+0000) 2024-05-07T02:46:04.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:04 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:04.489+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:25.649723+0000 front 2024-05-07T02:45:25.649626+0000 (oldest deadline 2024-05-07T02:45:49.749582+0000) 2024-05-07T02:46:04.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:04 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:04.489+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-07T02:45:34.450853+0000 front 2024-05-07T02:45:34.450979+0000 (oldest deadline 2024-05-07T02:45:57.350626+0000) 2024-05-07T02:46:05.145 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:46:04 smithi019 podman[114885]: 2024-05-07 02:46:04.785147325 +0000 UTC m=+0.037556757 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-07T02:46:05.146 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:46:04 smithi019 podman[114885]: 2024-05-07 02:46:04.912316868 +0000 UTC m=+0.164726310 container create 3d09187adc47a3fd5347c243eb70dbd605cc105ec2743f91e1b4229814756df5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-07T02:46:05.645 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:46:05 smithi019 podman[114885]: 2024-05-07 02:46:05.16558977 +0000 UTC m=+0.417999216 container init 3d09187adc47a3fd5347c243eb70dbd605cc105ec2743f91e1b4229814756df5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-07T02:46:05.646 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:46:05 smithi019 podman[114885]: 2024-05-07 02:46:05.178029321 +0000 UTC m=+0.430438822 container start 3d09187adc47a3fd5347c243eb70dbd605cc105ec2743f91e1b4229814756df5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-07T02:46:05.646 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:46:05 smithi019 podman[114885]: 2024-05-07 02:46:05.281283732 +0000 UTC m=+0.533693243 container attach 3d09187adc47a3fd5347c243eb70dbd605cc105ec2743f91e1b4229814756df5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-07T02:46:05.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:05 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:05.491+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:46:05.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:05 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:05.491+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:46:05.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:05 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:05.491+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:46:05.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:05 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:05.491+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:20.067840+0000 front 2024-05-07T02:45:20.067791+0000 (oldest deadline 2024-05-07T02:45:42.367749+0000) 2024-05-07T02:46:05.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:05 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:05.491+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:24.668989+0000 front 2024-05-07T02:45:24.669015+0000 (oldest deadline 2024-05-07T02:45:52.269691+0000) 2024-05-07T02:46:05.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:05 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:05.491+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-07T02:45:36.672575+0000 front 2024-05-07T02:45:36.672606+0000 (oldest deadline 2024-05-07T02:46:00.772356+0000) 2024-05-07T02:46:05.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:05 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:05.462+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:46:05.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:05 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:05.462+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:46:05.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:05 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:05.462+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:01.844911+0000 front 2024-05-07T02:45:01.844958+0000 (oldest deadline 2024-05-07T02:45:27.745107+0000) 2024-05-07T02:46:05.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:05 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:05.462+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:07.746120+0000 front 2024-05-07T02:45:07.745779+0000 (oldest deadline 2024-05-07T02:45:33.645623+0000) 2024-05-07T02:46:05.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:05 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:05.462+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:18.846956+0000 front 2024-05-07T02:45:18.846993+0000 (oldest deadline 2024-05-07T02:45:42.247711+0000) 2024-05-07T02:46:05.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:05 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:05.462+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:25.649723+0000 front 2024-05-07T02:45:25.649626+0000 (oldest deadline 2024-05-07T02:45:49.749582+0000) 2024-05-07T02:46:05.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:05 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:05.462+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-07T02:45:34.450853+0000 front 2024-05-07T02:45:34.450979+0000 (oldest deadline 2024-05-07T02:45:57.350626+0000) 2024-05-07T02:46:05.996 INFO:journalctl@ceph.osd.9.smithi019.stdout:May 07 02:46:05 smithi019 podman[114885]: 2024-05-07 02:46:05.656532446 +0000 UTC m=+0.908941876 container died 3d09187adc47a3fd5347c243eb70dbd605cc105ec2743f91e1b4229814756df5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.build-date=20240430, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-07T02:46:06.368 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@osd.9.service' 2024-05-07T02:46:06.488 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-07T02:46:06.488 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-07T02:46:06.488 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-07T02:46:06.489 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@osd.10 2024-05-07T02:46:06.803 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:06 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:06.512+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:46:06.803 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:06 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:06.512+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:46:06.803 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:06 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:06.512+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:01.844911+0000 front 2024-05-07T02:45:01.844958+0000 (oldest deadline 2024-05-07T02:45:27.745107+0000) 2024-05-07T02:46:06.804 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:06 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:06.512+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:07.746120+0000 front 2024-05-07T02:45:07.745779+0000 (oldest deadline 2024-05-07T02:45:33.645623+0000) 2024-05-07T02:46:06.804 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:06 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:06.512+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:18.846956+0000 front 2024-05-07T02:45:18.846993+0000 (oldest deadline 2024-05-07T02:45:42.247711+0000) 2024-05-07T02:46:06.804 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:06 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:06.512+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:25.649723+0000 front 2024-05-07T02:45:25.649626+0000 (oldest deadline 2024-05-07T02:45:49.749582+0000) 2024-05-07T02:46:06.804 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:06 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:06.512+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-07T02:45:34.450853+0000 front 2024-05-07T02:45:34.450979+0000 (oldest deadline 2024-05-07T02:45:57.350626+0000) 2024-05-07T02:46:06.804 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:06 smithi027 systemd[1]: Stopping Ceph osd.10 for 9ff84c16-0c17-11ef-bc97-c7b262605968... 2024-05-07T02:46:06.804 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:06 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:06.802+0000 7f53734f0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-07T02:46:06.804 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:06 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:06.802+0000 7f53734f0700 -1 osd.10 104 *** Got signal Terminated *** 2024-05-07T02:46:06.804 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:06 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:06.802+0000 7f53734f0700 -1 osd.10 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-07T02:46:06.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:06 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:06.468+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:46:06.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:06 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:06.468+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:46:06.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:06 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:06.468+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:46:06.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:06 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:06.468+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:20.067840+0000 front 2024-05-07T02:45:20.067791+0000 (oldest deadline 2024-05-07T02:45:42.367749+0000) 2024-05-07T02:46:06.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:06 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:06.468+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:24.668989+0000 front 2024-05-07T02:45:24.669015+0000 (oldest deadline 2024-05-07T02:45:52.269691+0000) 2024-05-07T02:46:06.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:06 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:06.468+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-07T02:45:36.672575+0000 front 2024-05-07T02:45:36.672606+0000 (oldest deadline 2024-05-07T02:46:00.772356+0000) 2024-05-07T02:46:07.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:07 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:07.454+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:46:07.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:07 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:07.454+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:46:07.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:07 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:07.454+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:46:07.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:07 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:07.454+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:20.067840+0000 front 2024-05-07T02:45:20.067791+0000 (oldest deadline 2024-05-07T02:45:42.367749+0000) 2024-05-07T02:46:07.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:07 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:07.454+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:24.668989+0000 front 2024-05-07T02:45:24.669015+0000 (oldest deadline 2024-05-07T02:45:52.269691+0000) 2024-05-07T02:46:07.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:07 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:07.454+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-07T02:45:36.672575+0000 front 2024-05-07T02:45:36.672606+0000 (oldest deadline 2024-05-07T02:46:00.772356+0000) 2024-05-07T02:46:07.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:07 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:07.454+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6820 osd.7 since back 2024-05-07T02:45:44.773978+0000 front 2024-05-07T02:45:44.774143+0000 (oldest deadline 2024-05-07T02:46:07.073735+0000) 2024-05-07T02:46:07.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:07 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:07.513+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:46:07.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:07 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:07.513+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:46:07.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:07 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:07.513+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:01.844911+0000 front 2024-05-07T02:45:01.844958+0000 (oldest deadline 2024-05-07T02:45:27.745107+0000) 2024-05-07T02:46:07.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:07 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:07.513+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:07.746120+0000 front 2024-05-07T02:45:07.745779+0000 (oldest deadline 2024-05-07T02:45:33.645623+0000) 2024-05-07T02:46:07.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:07 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:07.513+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:18.846956+0000 front 2024-05-07T02:45:18.846993+0000 (oldest deadline 2024-05-07T02:45:42.247711+0000) 2024-05-07T02:46:07.959 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:07 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:07.513+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:25.649723+0000 front 2024-05-07T02:45:25.649626+0000 (oldest deadline 2024-05-07T02:45:49.749582+0000) 2024-05-07T02:46:07.959 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:07 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:07.513+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-07T02:45:34.450853+0000 front 2024-05-07T02:45:34.450979+0000 (oldest deadline 2024-05-07T02:45:57.350626+0000) 2024-05-07T02:46:08.817 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:08 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:08.446+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:46:08.817 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:08 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:08.446+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:46:08.817 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:08 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:08.446+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:46:08.818 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:08 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:08.446+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:20.067840+0000 front 2024-05-07T02:45:20.067791+0000 (oldest deadline 2024-05-07T02:45:42.367749+0000) 2024-05-07T02:46:08.818 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:08 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:08.446+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:24.668989+0000 front 2024-05-07T02:45:24.669015+0000 (oldest deadline 2024-05-07T02:45:52.269691+0000) 2024-05-07T02:46:08.818 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:08 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:08.446+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-07T02:45:36.672575+0000 front 2024-05-07T02:45:36.672606+0000 (oldest deadline 2024-05-07T02:46:00.772356+0000) 2024-05-07T02:46:08.818 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:08 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:08.446+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6820 osd.7 since back 2024-05-07T02:45:44.773978+0000 front 2024-05-07T02:45:44.774143+0000 (oldest deadline 2024-05-07T02:46:07.073735+0000) 2024-05-07T02:46:08.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:08 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:08.520+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:46:08.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:08 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:08.520+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:46:08.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:08 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:08.520+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:01.844911+0000 front 2024-05-07T02:45:01.844958+0000 (oldest deadline 2024-05-07T02:45:27.745107+0000) 2024-05-07T02:46:08.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:08 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:08.520+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:07.746120+0000 front 2024-05-07T02:45:07.745779+0000 (oldest deadline 2024-05-07T02:45:33.645623+0000) 2024-05-07T02:46:08.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:08 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:08.520+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:18.846956+0000 front 2024-05-07T02:45:18.846993+0000 (oldest deadline 2024-05-07T02:45:42.247711+0000) 2024-05-07T02:46:08.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:08 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:08.520+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:25.649723+0000 front 2024-05-07T02:45:25.649626+0000 (oldest deadline 2024-05-07T02:45:49.749582+0000) 2024-05-07T02:46:08.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:08 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:08.520+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-07T02:45:34.450853+0000 front 2024-05-07T02:45:34.450979+0000 (oldest deadline 2024-05-07T02:45:57.350626+0000) 2024-05-07T02:46:09.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:09 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:09.426+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:46:09.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:09 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:09.426+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:46:09.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:09 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:09.426+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:46:09.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:09 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:09.426+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:20.067840+0000 front 2024-05-07T02:45:20.067791+0000 (oldest deadline 2024-05-07T02:45:42.367749+0000) 2024-05-07T02:46:09.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:09 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:09.426+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:24.668989+0000 front 2024-05-07T02:45:24.669015+0000 (oldest deadline 2024-05-07T02:45:52.269691+0000) 2024-05-07T02:46:09.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:09 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:09.426+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-07T02:45:36.672575+0000 front 2024-05-07T02:45:36.672606+0000 (oldest deadline 2024-05-07T02:46:00.772356+0000) 2024-05-07T02:46:09.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:09 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:09.426+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6820 osd.7 since back 2024-05-07T02:45:44.773978+0000 front 2024-05-07T02:45:44.774143+0000 (oldest deadline 2024-05-07T02:46:07.073735+0000) 2024-05-07T02:46:09.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:09 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:09.558+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:46:09.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:09 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:09.558+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:46:09.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:09 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:09.558+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:01.844911+0000 front 2024-05-07T02:45:01.844958+0000 (oldest deadline 2024-05-07T02:45:27.745107+0000) 2024-05-07T02:46:09.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:09 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:09.558+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:07.746120+0000 front 2024-05-07T02:45:07.745779+0000 (oldest deadline 2024-05-07T02:45:33.645623+0000) 2024-05-07T02:46:09.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:09 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:09.558+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:18.846956+0000 front 2024-05-07T02:45:18.846993+0000 (oldest deadline 2024-05-07T02:45:42.247711+0000) 2024-05-07T02:46:09.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:09 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:09.558+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:25.649723+0000 front 2024-05-07T02:45:25.649626+0000 (oldest deadline 2024-05-07T02:45:49.749582+0000) 2024-05-07T02:46:09.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:09 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:09.558+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-07T02:45:34.450853+0000 front 2024-05-07T02:45:34.450979+0000 (oldest deadline 2024-05-07T02:45:57.350626+0000) 2024-05-07T02:46:10.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:10 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:10.441+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:46:10.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:10 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:10.441+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:46:10.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:10 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:10.441+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:46:10.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:10 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:10.441+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:20.067840+0000 front 2024-05-07T02:45:20.067791+0000 (oldest deadline 2024-05-07T02:45:42.367749+0000) 2024-05-07T02:46:10.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:10 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:10.441+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:24.668989+0000 front 2024-05-07T02:45:24.669015+0000 (oldest deadline 2024-05-07T02:45:52.269691+0000) 2024-05-07T02:46:10.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:10 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:10.441+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-07T02:45:36.672575+0000 front 2024-05-07T02:45:36.672606+0000 (oldest deadline 2024-05-07T02:46:00.772356+0000) 2024-05-07T02:46:10.861 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:10 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:10.441+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6820 osd.7 since back 2024-05-07T02:45:44.773978+0000 front 2024-05-07T02:45:44.774143+0000 (oldest deadline 2024-05-07T02:46:07.073735+0000) 2024-05-07T02:46:10.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:10 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:10.568+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:46:10.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:10 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:10.568+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:46:10.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:10 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:10.568+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:01.844911+0000 front 2024-05-07T02:45:01.844958+0000 (oldest deadline 2024-05-07T02:45:27.745107+0000) 2024-05-07T02:46:10.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:10 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:10.568+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:07.746120+0000 front 2024-05-07T02:45:07.745779+0000 (oldest deadline 2024-05-07T02:45:33.645623+0000) 2024-05-07T02:46:10.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:10 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:10.568+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:18.846956+0000 front 2024-05-07T02:45:18.846993+0000 (oldest deadline 2024-05-07T02:45:42.247711+0000) 2024-05-07T02:46:10.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:10 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:10.568+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:25.649723+0000 front 2024-05-07T02:45:25.649626+0000 (oldest deadline 2024-05-07T02:45:49.749582+0000) 2024-05-07T02:46:10.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:10 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:10.568+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-07T02:45:34.450853+0000 front 2024-05-07T02:45:34.450979+0000 (oldest deadline 2024-05-07T02:45:57.350626+0000) 2024-05-07T02:46:11.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:11 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:11.488+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:46:11.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:11 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:11.488+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:46:11.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:11 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:11.488+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:46:11.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:11 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:11.488+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:20.067840+0000 front 2024-05-07T02:45:20.067791+0000 (oldest deadline 2024-05-07T02:45:42.367749+0000) 2024-05-07T02:46:11.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:11 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:11.488+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:24.668989+0000 front 2024-05-07T02:45:24.669015+0000 (oldest deadline 2024-05-07T02:45:52.269691+0000) 2024-05-07T02:46:11.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:11 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:11.488+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-07T02:45:36.672575+0000 front 2024-05-07T02:45:36.672606+0000 (oldest deadline 2024-05-07T02:46:00.772356+0000) 2024-05-07T02:46:11.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:11 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:11.488+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6820 osd.7 since back 2024-05-07T02:45:44.773978+0000 front 2024-05-07T02:45:44.774143+0000 (oldest deadline 2024-05-07T02:46:07.073735+0000) 2024-05-07T02:46:11.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:11 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:11.600+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:44.243011+0000 front 2024-05-07T02:44:44.242847+0000 (oldest deadline 2024-05-07T02:45:08.942692+0000) 2024-05-07T02:46:11.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:11 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:11.600+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:53.043512+0000 front 2024-05-07T02:44:53.043581+0000 (oldest deadline 2024-05-07T02:45:17.743760+0000) 2024-05-07T02:46:11.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:11 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:11.600+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:01.844911+0000 front 2024-05-07T02:45:01.844958+0000 (oldest deadline 2024-05-07T02:45:27.745107+0000) 2024-05-07T02:46:11.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:11 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:11.600+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-05-07T02:45:07.746120+0000 front 2024-05-07T02:45:07.745779+0000 (oldest deadline 2024-05-07T02:45:33.645623+0000) 2024-05-07T02:46:11.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:11 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:11.600+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:18.846956+0000 front 2024-05-07T02:45:18.846993+0000 (oldest deadline 2024-05-07T02:45:42.247711+0000) 2024-05-07T02:46:11.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:11 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:11.600+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:25.649723+0000 front 2024-05-07T02:45:25.649626+0000 (oldest deadline 2024-05-07T02:45:49.749582+0000) 2024-05-07T02:46:11.958 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:11 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10[59514]: 2024-05-07T02:46:11.600+0000 7f536ba9e700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-07T02:45:34.450853+0000 front 2024-05-07T02:45:34.450979+0000 (oldest deadline 2024-05-07T02:45:57.350626+0000) 2024-05-07T02:46:12.595 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:12 smithi027 podman[73132]: 2024-05-07 02:46:12.322195078 +0000 UTC m=+5.631505165 container died e18a828b22f348c02a9c97fc87f4c0794578c923f50be765b343fef992f56b63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-07T02:46:12.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:12 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:12.517+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:46:12.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:12 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:12.518+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:46:12.861 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:12 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:12.518+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:46:12.861 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:12 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:12.518+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:20.067840+0000 front 2024-05-07T02:45:20.067791+0000 (oldest deadline 2024-05-07T02:45:42.367749+0000) 2024-05-07T02:46:12.861 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:12 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:12.518+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:24.668989+0000 front 2024-05-07T02:45:24.669015+0000 (oldest deadline 2024-05-07T02:45:52.269691+0000) 2024-05-07T02:46:12.861 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:12 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:12.518+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-07T02:45:36.672575+0000 front 2024-05-07T02:45:36.672606+0000 (oldest deadline 2024-05-07T02:46:00.772356+0000) 2024-05-07T02:46:12.861 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:12 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:12.518+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6820 osd.7 since back 2024-05-07T02:45:44.773978+0000 front 2024-05-07T02:45:44.774143+0000 (oldest deadline 2024-05-07T02:46:07.073735+0000) 2024-05-07T02:46:12.945 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:12 smithi027 podman[73132]: 2024-05-07 02:46:12.694267978 +0000 UTC m=+6.003578062 container cleanup e18a828b22f348c02a9c97fc87f4c0794578c923f50be765b343fef992f56b63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_CLEAN=True, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-07T02:46:12.945 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:12 smithi027 bash[73132]: ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10 2024-05-07T02:46:13.208 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:12 smithi027 podman[73150]: 2024-05-07 02:46:12.945371001 +0000 UTC m=+0.618590020 container remove e18a828b22f348c02a9c97fc87f4c0794578c923f50be765b343fef992f56b63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-07T02:46:13.590 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:13 smithi027 podman[73236]: 2024-05-07 02:46:13.178891714 +0000 UTC m=+0.018474377 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-07T02:46:13.590 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:13 smithi027 podman[73236]: 2024-05-07 02:46:13.320852919 +0000 UTC m=+0.160435584 container create 1ac2bb53e1a5df2d8e2ce371e2fa32227d269cfa2d9cc251f75ec0acf689ffaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-07T02:46:13.590 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:13 smithi027 podman[73236]: 2024-05-07 02:46:13.515194487 +0000 UTC m=+0.354777150 container init 1ac2bb53e1a5df2d8e2ce371e2fa32227d269cfa2d9cc251f75ec0acf689ffaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430) 2024-05-07T02:46:13.590 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:13 smithi027 podman[73236]: 2024-05-07 02:46:13.518278127 +0000 UTC m=+0.357860789 container start 1ac2bb53e1a5df2d8e2ce371e2fa32227d269cfa2d9cc251f75ec0acf689ffaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-07T02:46:13.591 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:13 smithi027 podman[73236]: 2024-05-07 02:46:13.590344598 +0000 UTC m=+0.429927269 container attach 1ac2bb53e1a5df2d8e2ce371e2fa32227d269cfa2d9cc251f75ec0acf689ffaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, ceph=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.schema-version=1.0) 2024-05-07T02:46:13.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:13 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:13.522+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:46:13.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:13 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:13.522+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:46:13.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:13 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:13.522+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:46:13.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:13 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:13.522+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:20.067840+0000 front 2024-05-07T02:45:20.067791+0000 (oldest deadline 2024-05-07T02:45:42.367749+0000) 2024-05-07T02:46:13.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:13 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:13.522+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:24.668989+0000 front 2024-05-07T02:45:24.669015+0000 (oldest deadline 2024-05-07T02:45:52.269691+0000) 2024-05-07T02:46:13.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:13 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:13.522+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-07T02:45:36.672575+0000 front 2024-05-07T02:45:36.672606+0000 (oldest deadline 2024-05-07T02:46:00.772356+0000) 2024-05-07T02:46:13.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:13 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:13.522+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6820 osd.7 since back 2024-05-07T02:45:44.773978+0000 front 2024-05-07T02:45:44.774143+0000 (oldest deadline 2024-05-07T02:46:07.073735+0000) 2024-05-07T02:46:13.957 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:13 smithi027 podman[73236]: 2024-05-07 02:46:13.830256048 +0000 UTC m=+0.669838711 container died 1ac2bb53e1a5df2d8e2ce371e2fa32227d269cfa2d9cc251f75ec0acf689ffaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-07T02:46:14.373 INFO:journalctl@ceph.osd.10.smithi027.stdout:May 07 02:46:14 smithi027 podman[73262]: 2024-05-07 02:46:14.360194852 +0000 UTC m=+0.524141392 container remove 1ac2bb53e1a5df2d8e2ce371e2fa32227d269cfa2d9cc251f75ec0acf689ffaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-10-deactivate, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-07T02:46:14.379 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@osd.10.service' 2024-05-07T02:46:14.419 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-07T02:46:14.420 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-07T02:46:14.420 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-07T02:46:14.420 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@osd.11 2024-05-07T02:46:14.709 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:14 smithi064 systemd[1]: Stopping Ceph osd.11 for 9ff84c16-0c17-11ef-bc97-c7b262605968... 2024-05-07T02:46:14.710 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:14 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:14.506+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:46:14.710 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:14 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:14.506+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:46:14.710 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:14 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:14.506+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:46:14.710 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:14 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:14.506+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:20.067840+0000 front 2024-05-07T02:45:20.067791+0000 (oldest deadline 2024-05-07T02:45:42.367749+0000) 2024-05-07T02:46:14.710 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:14 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:14.506+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:24.668989+0000 front 2024-05-07T02:45:24.669015+0000 (oldest deadline 2024-05-07T02:45:52.269691+0000) 2024-05-07T02:46:14.710 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:14 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:14.506+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-07T02:45:36.672575+0000 front 2024-05-07T02:45:36.672606+0000 (oldest deadline 2024-05-07T02:46:00.772356+0000) 2024-05-07T02:46:14.711 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:14 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:14.506+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6820 osd.7 since back 2024-05-07T02:45:44.773978+0000 front 2024-05-07T02:45:44.774143+0000 (oldest deadline 2024-05-07T02:46:07.073735+0000) 2024-05-07T02:46:14.711 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:14 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:14.708+0000 7f74268a8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-07T02:46:14.711 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:14 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:14.708+0000 7f74268a8700 -1 osd.11 104 *** Got signal Terminated *** 2024-05-07T02:46:14.711 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:14 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:14.708+0000 7f74268a8700 -1 osd.11 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-07T02:46:15.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:15 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:15.527+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:46:15.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:15 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:15.527+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:46:15.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:15 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:15.527+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:46:15.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:15 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:15.527+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:20.067840+0000 front 2024-05-07T02:45:20.067791+0000 (oldest deadline 2024-05-07T02:45:42.367749+0000) 2024-05-07T02:46:15.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:15 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:15.527+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:24.668989+0000 front 2024-05-07T02:45:24.669015+0000 (oldest deadline 2024-05-07T02:45:52.269691+0000) 2024-05-07T02:46:15.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:15 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:15.527+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-07T02:45:36.672575+0000 front 2024-05-07T02:45:36.672606+0000 (oldest deadline 2024-05-07T02:46:00.772356+0000) 2024-05-07T02:46:15.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:15 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:15.527+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6820 osd.7 since back 2024-05-07T02:45:44.773978+0000 front 2024-05-07T02:45:44.774143+0000 (oldest deadline 2024-05-07T02:46:07.073735+0000) 2024-05-07T02:46:15.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:15 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:15.527+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6820 osd.8 since back 2024-05-07T02:45:49.374470+0000 front 2024-05-07T02:45:49.374490+0000 (oldest deadline 2024-05-07T02:46:15.274395+0000) 2024-05-07T02:46:16.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:16 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:16.487+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:46:16.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:16 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:16.487+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:46:16.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:16 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:16.487+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:46:16.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:16 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:16.487+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:20.067840+0000 front 2024-05-07T02:45:20.067791+0000 (oldest deadline 2024-05-07T02:45:42.367749+0000) 2024-05-07T02:46:16.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:16 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:16.487+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:24.668989+0000 front 2024-05-07T02:45:24.669015+0000 (oldest deadline 2024-05-07T02:45:52.269691+0000) 2024-05-07T02:46:16.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:16 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:16.487+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-07T02:45:36.672575+0000 front 2024-05-07T02:45:36.672606+0000 (oldest deadline 2024-05-07T02:46:00.772356+0000) 2024-05-07T02:46:16.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:16 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:16.487+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6820 osd.7 since back 2024-05-07T02:45:44.773978+0000 front 2024-05-07T02:45:44.774143+0000 (oldest deadline 2024-05-07T02:46:07.073735+0000) 2024-05-07T02:46:16.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:16 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:16.487+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6820 osd.8 since back 2024-05-07T02:45:49.374470+0000 front 2024-05-07T02:45:49.374490+0000 (oldest deadline 2024-05-07T02:46:15.274395+0000) 2024-05-07T02:46:17.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:17 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:17.466+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:46:17.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:17 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:17.466+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:46:17.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:17 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:17.466+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:46:17.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:17 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:17.466+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:20.067840+0000 front 2024-05-07T02:45:20.067791+0000 (oldest deadline 2024-05-07T02:45:42.367749+0000) 2024-05-07T02:46:17.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:17 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:17.466+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:24.668989+0000 front 2024-05-07T02:45:24.669015+0000 (oldest deadline 2024-05-07T02:45:52.269691+0000) 2024-05-07T02:46:17.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:17 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:17.466+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-07T02:45:36.672575+0000 front 2024-05-07T02:45:36.672606+0000 (oldest deadline 2024-05-07T02:46:00.772356+0000) 2024-05-07T02:46:17.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:17 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:17.466+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6820 osd.7 since back 2024-05-07T02:45:44.773978+0000 front 2024-05-07T02:45:44.774143+0000 (oldest deadline 2024-05-07T02:46:07.073735+0000) 2024-05-07T02:46:17.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:17 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:17.466+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6820 osd.8 since back 2024-05-07T02:45:49.374470+0000 front 2024-05-07T02:45:49.374490+0000 (oldest deadline 2024-05-07T02:46:15.274395+0000) 2024-05-07T02:46:18.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:18 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:18.448+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:46:18.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:18 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:18.448+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:46:18.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:18 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:18.448+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:46:18.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:18 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:18.448+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:20.067840+0000 front 2024-05-07T02:45:20.067791+0000 (oldest deadline 2024-05-07T02:45:42.367749+0000) 2024-05-07T02:46:18.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:18 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:18.448+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:24.668989+0000 front 2024-05-07T02:45:24.669015+0000 (oldest deadline 2024-05-07T02:45:52.269691+0000) 2024-05-07T02:46:18.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:18 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:18.448+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-07T02:45:36.672575+0000 front 2024-05-07T02:45:36.672606+0000 (oldest deadline 2024-05-07T02:46:00.772356+0000) 2024-05-07T02:46:18.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:18 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:18.448+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6820 osd.7 since back 2024-05-07T02:45:44.773978+0000 front 2024-05-07T02:45:44.774143+0000 (oldest deadline 2024-05-07T02:46:07.073735+0000) 2024-05-07T02:46:18.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:18 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:18.448+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6820 osd.8 since back 2024-05-07T02:45:49.374470+0000 front 2024-05-07T02:45:49.374490+0000 (oldest deadline 2024-05-07T02:46:15.274395+0000) 2024-05-07T02:46:19.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:19 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:19.474+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-05-07T02:44:43.459970+0000 front 2024-05-07T02:44:43.459832+0000 (oldest deadline 2024-05-07T02:45:05.759911+0000) 2024-05-07T02:46:19.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:19 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:19.474+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6804 osd.1 since back 2024-05-07T02:44:54.362956+0000 front 2024-05-07T02:44:54.362988+0000 (oldest deadline 2024-05-07T02:45:16.063022+0000) 2024-05-07T02:46:19.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:19 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:19.474+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-07T02:45:02.465136+0000 front 2024-05-07T02:45:02.465305+0000 (oldest deadline 2024-05-07T02:45:27.165384+0000) 2024-05-07T02:46:19.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:19 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:19.474+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6812 osd.4 since back 2024-05-07T02:45:20.067840+0000 front 2024-05-07T02:45:20.067791+0000 (oldest deadline 2024-05-07T02:45:42.367749+0000) 2024-05-07T02:46:19.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:19 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:19.474+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-07T02:45:24.668989+0000 front 2024-05-07T02:45:24.669015+0000 (oldest deadline 2024-05-07T02:45:52.269691+0000) 2024-05-07T02:46:19.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:19 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:19.474+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-05-07T02:45:36.672575+0000 front 2024-05-07T02:45:36.672606+0000 (oldest deadline 2024-05-07T02:46:00.772356+0000) 2024-05-07T02:46:19.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:19 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:19.474+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.27:6820 osd.7 since back 2024-05-07T02:45:44.773978+0000 front 2024-05-07T02:45:44.774143+0000 (oldest deadline 2024-05-07T02:46:07.073735+0000) 2024-05-07T02:46:19.860 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:19 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11[60094]: 2024-05-07T02:46:19.474+0000 7f741ee56700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.64:6820 osd.8 since back 2024-05-07T02:45:49.374470+0000 front 2024-05-07T02:45:49.374490+0000 (oldest deadline 2024-05-07T02:46:15.274395+0000) 2024-05-07T02:46:20.467 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:20 smithi064 podman[73776]: 2024-05-07 02:46:20.208538951 +0000 UTC m=+5.588602730 container died b0efb800dae0ae4b9b1b53eff58e9bd7fc00db5dec2a35f5998890bcddf617ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD) 2024-05-07T02:46:20.834 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:20 smithi064 podman[73776]: 2024-05-07 02:46:20.56712922 +0000 UTC m=+5.947192999 container cleanup b0efb800dae0ae4b9b1b53eff58e9bd7fc00db5dec2a35f5998890bcddf617ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5) 2024-05-07T02:46:20.835 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:20 smithi064 bash[73776]: ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11 2024-05-07T02:46:21.109 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:20 smithi064 podman[73793]: 2024-05-07 02:46:20.834923916 +0000 UTC m=+0.627839096 container remove b0efb800dae0ae4b9b1b53eff58e9bd7fc00db5dec2a35f5998890bcddf617ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-07T02:46:21.454 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:21 smithi064 podman[73886]: 2024-05-07 02:46:21.144042226 +0000 UTC m=+0.104890889 container create d6527647493db6593012c76b3dda91578a6dc4c7ed897bdbdcbf275652c7cef7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11-deactivate, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-07T02:46:21.454 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:21 smithi064 podman[73886]: 2024-05-07 02:46:21.057708494 +0000 UTC m=+0.018557172 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-07T02:46:21.454 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:21 smithi064 podman[73886]: 2024-05-07 02:46:21.371446573 +0000 UTC m=+0.332295250 container init d6527647493db6593012c76b3dda91578a6dc4c7ed897bdbdcbf275652c7cef7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240430, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-07T02:46:21.454 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:21 smithi064 podman[73886]: 2024-05-07 02:46:21.37515123 +0000 UTC m=+0.335999895 container start d6527647493db6593012c76b3dda91578a6dc4c7ed897bdbdcbf275652c7cef7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a) 2024-05-07T02:46:21.455 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:21 smithi064 podman[73886]: 2024-05-07 02:46:21.454468355 +0000 UTC m=+0.415317030 container attach d6527647493db6593012c76b3dda91578a6dc4c7ed897bdbdcbf275652c7cef7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240430, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-07T02:46:21.859 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 07 02:46:21 smithi064 podman[73886]: 2024-05-07 02:46:21.67593923 +0000 UTC m=+0.636787899 container died d6527647493db6593012c76b3dda91578a6dc4c7ed897bdbdcbf275652c7cef7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-osd-11-deactivate, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-07T02:46:22.240 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@osd.11.service' 2024-05-07T02:46:22.279 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-07T02:46:22.279 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-07T02:46:22.279 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-07T02:46:22.280 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mds.a 2024-05-07T02:46:22.645 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 07 02:46:22 smithi019 systemd[1]: Stopping Ceph mds.a for 9ff84c16-0c17-11ef-bc97-c7b262605968... 2024-05-07T02:46:23.395 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 07 02:46:22 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-a[72008]: 2024-05-07T02:46:22.909+0000 7faa15000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-07T02:46:23.395 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 07 02:46:22 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-a[72008]: 2024-05-07T02:46:22.909+0000 7faa15000700 -1 mds.a *** got signal Terminated *** 2024-05-07T02:46:24.492 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 07 02:46:24 smithi019 podman[115001]: 2024-05-07 02:46:24.159484616 +0000 UTC m=+1.396246284 container died 34df7d493449bf07acead413e6c1d11e86f9a50a258a548b206839fb75c65669 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-a, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-07T02:46:24.846 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 07 02:46:24 smithi019 podman[115001]: 2024-05-07 02:46:24.619917792 +0000 UTC m=+1.856679477 container cleanup 34df7d493449bf07acead413e6c1d11e86f9a50a258a548b206839fb75c65669 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-07T02:46:24.846 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 07 02:46:24 smithi019 bash[115001]: ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-a 2024-05-07T02:46:25.145 INFO:journalctl@ceph.mds.a.smithi019.stdout:May 07 02:46:24 smithi019 podman[115014]: 2024-05-07 02:46:24.84563895 +0000 UTC m=+0.670828325 container remove 34df7d493449bf07acead413e6c1d11e86f9a50a258a548b206839fb75c65669 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-a, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240430) 2024-05-07T02:46:25.420 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mds.a.service' 2024-05-07T02:46:25.534 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-07T02:46:25.535 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-07T02:46:25.535 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-07T02:46:25.535 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mds.d 2024-05-07T02:46:26.145 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 07 02:46:25 smithi019 systemd[1]: Stopping Ceph mds.d for 9ff84c16-0c17-11ef-bc97-c7b262605968... 2024-05-07T02:46:26.645 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 07 02:46:26 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-d[70814]: 2024-05-07T02:46:26.181+0000 7f0439c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-07T02:46:26.646 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 07 02:46:26 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-d[70814]: 2024-05-07T02:46:26.181+0000 7f0439c00700 -1 mds.d *** got signal Terminated *** 2024-05-07T02:46:28.437 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 07 02:46:28 smithi019 podman[115119]: 2024-05-07 02:46:28.171918614 +0000 UTC m=+2.140054748 container died 0533f004eb397c67c5abe93ccee3219aa0eeebc954cf5e6b4ddb6919dcb5c251 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-d, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a) 2024-05-07T02:46:28.692 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 07 02:46:28 smithi019 podman[115119]: 2024-05-07 02:46:28.664850933 +0000 UTC m=+2.632987074 container cleanup 0533f004eb397c67c5abe93ccee3219aa0eeebc954cf5e6b4ddb6919dcb5c251 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-d, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-07T02:46:28.692 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 07 02:46:28 smithi019 bash[115119]: ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-d 2024-05-07T02:46:29.145 INFO:journalctl@ceph.mds.d.smithi019.stdout:May 07 02:46:28 smithi019 podman[115133]: 2024-05-07 02:46:28.841206289 +0000 UTC m=+0.652941053 container remove 0533f004eb397c67c5abe93ccee3219aa0eeebc954cf5e6b4ddb6919dcb5c251 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-d, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-07T02:46:29.326 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mds.d.service' 2024-05-07T02:46:29.436 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-07T02:46:29.437 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-07T02:46:29.437 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-07T02:46:29.437 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mds.g 2024-05-07T02:46:29.895 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 07 02:46:29 smithi019 systemd[1]: Stopping Ceph mds.g for 9ff84c16-0c17-11ef-bc97-c7b262605968... 2024-05-07T02:46:30.395 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 07 02:46:30 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-g[71593]: 2024-05-07T02:46:30.025+0000 7f604f800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-07T02:46:30.396 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 07 02:46:30 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-g[71593]: 2024-05-07T02:46:30.025+0000 7f604f800700 -1 mds.g *** got signal Terminated *** 2024-05-07T02:46:33.864 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 07 02:46:33 smithi019 podman[115239]: 2024-05-07 02:46:33.583873808 +0000 UTC m=+3.688513516 container died c5b8a079a2c9c679c2edd12c2d8cafe85454bb587b44e96714343d65c5607e02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-g, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True) 2024-05-07T02:46:34.118 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 07 02:46:34 smithi019 podman[115239]: 2024-05-07 02:46:34.090651267 +0000 UTC m=+4.195290968 container cleanup c5b8a079a2c9c679c2edd12c2d8cafe85454bb587b44e96714343d65c5607e02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-g, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-07T02:46:34.118 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 07 02:46:34 smithi019 bash[115239]: ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-g 2024-05-07T02:46:34.395 INFO:journalctl@ceph.mds.g.smithi019.stdout:May 07 02:46:34 smithi019 podman[115253]: 2024-05-07 02:46:34.27869814 +0000 UTC m=+0.677237311 container remove c5b8a079a2c9c679c2edd12c2d8cafe85454bb587b44e96714343d65c5607e02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-g, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a) 2024-05-07T02:46:34.766 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mds.g.service' 2024-05-07T02:46:34.879 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-07T02:46:34.879 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-07T02:46:34.879 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-07T02:46:34.879 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mds.j 2024-05-07T02:46:35.395 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 07 02:46:35 smithi019 systemd[1]: Stopping Ceph mds.j for 9ff84c16-0c17-11ef-bc97-c7b262605968... 2024-05-07T02:46:35.898 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 07 02:46:35 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-j[70358]: 2024-05-07T02:46:35.487+0000 7f4172c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-07T02:46:35.899 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 07 02:46:35 smithi019 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-j[70358]: 2024-05-07T02:46:35.487+0000 7f4172c00700 -1 mds.j *** got signal Terminated *** 2024-05-07T02:46:39.786 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 07 02:46:39 smithi019 podman[115359]: 2024-05-07 02:46:39.516050287 +0000 UTC m=+4.153824041 container died 3b7491ac7c35b9bb21cf4e34ecccb775a5f15b6b74392f2989456df2fc9bb8a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-j, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=) 2024-05-07T02:46:40.146 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 07 02:46:39 smithi019 podman[115359]: 2024-05-07 02:46:39.989225078 +0000 UTC m=+4.626998829 container cleanup 3b7491ac7c35b9bb21cf4e34ecccb775a5f15b6b74392f2989456df2fc9bb8a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-j, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-07T02:46:40.146 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 07 02:46:39 smithi019 bash[115359]: ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-j 2024-05-07T02:46:40.645 INFO:journalctl@ceph.mds.j.smithi019.stdout:May 07 02:46:40 smithi019 podman[115373]: 2024-05-07 02:46:40.173622661 +0000 UTC m=+0.639298366 container remove 3b7491ac7c35b9bb21cf4e34ecccb775a5f15b6b74392f2989456df2fc9bb8a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-j, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-07T02:46:40.728 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mds.j.service' 2024-05-07T02:46:40.842 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-07T02:46:40.843 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-07T02:46:40.843 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-07T02:46:40.844 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mds.b 2024-05-07T02:46:41.170 INFO:journalctl@ceph.mds.b.smithi027.stdout:May 07 02:46:40 smithi027 systemd[1]: Stopping Ceph mds.b for 9ff84c16-0c17-11ef-bc97-c7b262605968... 2024-05-07T02:46:41.444 INFO:journalctl@ceph.mds.b.smithi027.stdout:May 07 02:46:41 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-b[65159]: 2024-05-07T02:46:41.169+0000 7fcc11a6b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-07T02:46:41.444 INFO:journalctl@ceph.mds.b.smithi027.stdout:May 07 02:46:41 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-b[65159]: 2024-05-07T02:46:41.169+0000 7fcc11a6b700 -1 mds.b *** got signal Terminated *** 2024-05-07T02:46:44.336 INFO:journalctl@ceph.mds.b.smithi027.stdout:May 07 02:46:44 smithi027 podman[73341]: 2024-05-07 02:46:44.068421849 +0000 UTC m=+3.020653016 container died 92b7019a59cef8ce29cac6e03a37e63b48c1b781fc9a143bcdfe79df5cce0c22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-07T02:46:44.703 INFO:journalctl@ceph.mds.b.smithi027.stdout:May 07 02:46:44 smithi027 podman[73341]: 2024-05-07 02:46:44.435644153 +0000 UTC m=+3.387875306 container cleanup 92b7019a59cef8ce29cac6e03a37e63b48c1b781fc9a143bcdfe79df5cce0c22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-07T02:46:44.703 INFO:journalctl@ceph.mds.b.smithi027.stdout:May 07 02:46:44 smithi027 bash[73341]: ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-b 2024-05-07T02:46:44.865 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mds.b.service' 2024-05-07T02:46:44.901 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-07T02:46:44.901 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-07T02:46:44.902 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-07T02:46:44.902 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mds.e 2024-05-07T02:46:45.215 INFO:journalctl@ceph.mds.e.smithi027.stdout:May 07 02:46:44 smithi027 systemd[1]: Stopping Ceph mds.e for 9ff84c16-0c17-11ef-bc97-c7b262605968... 2024-05-07T02:46:45.707 INFO:journalctl@ceph.mds.e.smithi027.stdout:May 07 02:46:45 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-e[64547]: 2024-05-07T02:46:45.214+0000 7f96fd9f6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-07T02:46:45.708 INFO:journalctl@ceph.mds.e.smithi027.stdout:May 07 02:46:45 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-e[64547]: 2024-05-07T02:46:45.214+0000 7f96fd9f6700 -1 mds.e *** got signal Terminated *** 2024-05-07T02:46:48.407 INFO:journalctl@ceph.mds.e.smithi027.stdout:May 07 02:46:48 smithi027 podman[73472]: 2024-05-07 02:46:48.044338102 +0000 UTC m=+2.919472268 container died cbc890c768f0de80270e5d8364181b7005237b99c16be67d6915db34d7307b0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_CLEAN=True, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True) 2024-05-07T02:46:48.707 INFO:journalctl@ceph.mds.e.smithi027.stdout:May 07 02:46:48 smithi027 podman[73472]: 2024-05-07 02:46:48.505951138 +0000 UTC m=+3.381085315 container cleanup cbc890c768f0de80270e5d8364181b7005237b99c16be67d6915db34d7307b0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a) 2024-05-07T02:46:48.707 INFO:journalctl@ceph.mds.e.smithi027.stdout:May 07 02:46:48 smithi027 bash[73472]: ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-e 2024-05-07T02:46:48.911 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mds.e.service' 2024-05-07T02:46:48.944 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-07T02:46:48.945 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-07T02:46:48.945 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-07T02:46:48.945 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mds.h 2024-05-07T02:46:49.207 INFO:journalctl@ceph.mds.h.smithi027.stdout:May 07 02:46:49 smithi027 systemd[1]: Stopping Ceph mds.h for 9ff84c16-0c17-11ef-bc97-c7b262605968... 2024-05-07T02:46:49.707 INFO:journalctl@ceph.mds.h.smithi027.stdout:May 07 02:46:49 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-h[64849]: 2024-05-07T02:46:49.284+0000 7fe7d34bd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-07T02:46:49.707 INFO:journalctl@ceph.mds.h.smithi027.stdout:May 07 02:46:49 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-h[64849]: 2024-05-07T02:46:49.284+0000 7fe7d34bd700 -1 mds.h *** got signal Terminated *** 2024-05-07T02:46:52.208 INFO:journalctl@ceph.mds.h.smithi027.stdout:May 07 02:46:52 smithi027 podman[73593]: 2024-05-07 02:46:52.064673117 +0000 UTC m=+2.901555798 container died 6d86f8fb2e15f285adf1d9c48733a5de5b1cf227baecb429e3ad239471ca6429 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-h, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-07T02:46:52.602 INFO:journalctl@ceph.mds.h.smithi027.stdout:May 07 02:46:52 smithi027 podman[73593]: 2024-05-07 02:46:52.409479123 +0000 UTC m=+3.246361788 container cleanup 6d86f8fb2e15f285adf1d9c48733a5de5b1cf227baecb429e3ad239471ca6429 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-h, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.build-date=20240430) 2024-05-07T02:46:52.602 INFO:journalctl@ceph.mds.h.smithi027.stdout:May 07 02:46:52 smithi027 bash[73593]: ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-h 2024-05-07T02:46:52.833 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mds.h.service' 2024-05-07T02:46:52.853 INFO:journalctl@ceph.mds.h.smithi027.stdout:May 07 02:46:52 smithi027 podman[73607]: 2024-05-07 02:46:52.60247346 +0000 UTC m=+0.528528110 container remove 6d86f8fb2e15f285adf1d9c48733a5de5b1cf227baecb429e3ad239471ca6429 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-07T02:46:52.853 INFO:journalctl@ceph.mds.h.smithi027.stdout:May 07 02:46:52 smithi027 systemd[1]: ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mds.h.service: Deactivated successfully. 2024-05-07T02:46:52.853 INFO:journalctl@ceph.mds.h.smithi027.stdout:May 07 02:46:52 smithi027 systemd[1]: Stopped Ceph mds.h for 9ff84c16-0c17-11ef-bc97-c7b262605968. 2024-05-07T02:46:52.854 INFO:journalctl@ceph.mds.h.smithi027.stdout:May 07 02:46:52 smithi027 systemd[1]: ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mds.h.service: Consumed 40.935s CPU time. 2024-05-07T02:46:52.865 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-07T02:46:52.865 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-07T02:46:52.865 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-07T02:46:52.865 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mds.k 2024-05-07T02:46:53.207 INFO:journalctl@ceph.mds.k.smithi027.stdout:May 07 02:46:52 smithi027 systemd[1]: Stopping Ceph mds.k for 9ff84c16-0c17-11ef-bc97-c7b262605968... 2024-05-07T02:46:53.707 INFO:journalctl@ceph.mds.k.smithi027.stdout:May 07 02:46:53 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-k[64223]: 2024-05-07T02:46:53.210+0000 7fe08fc78700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-07T02:46:53.707 INFO:journalctl@ceph.mds.k.smithi027.stdout:May 07 02:46:53 smithi027 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-k[64223]: 2024-05-07T02:46:53.210+0000 7fe08fc78700 -1 mds.k *** got signal Terminated *** 2024-05-07T02:46:56.305 INFO:journalctl@ceph.mds.k.smithi027.stdout:May 07 02:46:56 smithi027 podman[73714]: 2024-05-07 02:46:56.051626601 +0000 UTC m=+2.959370933 container died 56b71961f80a3ff9337269477f56d400a3fcc51ff27f6e4e4ca47a567fece2d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS) 2024-05-07T02:46:56.591 INFO:journalctl@ceph.mds.k.smithi027.stdout:May 07 02:46:56 smithi027 podman[73714]: 2024-05-07 02:46:56.404970799 +0000 UTC m=+3.312715120 container cleanup 56b71961f80a3ff9337269477f56d400a3fcc51ff27f6e4e4ca47a567fece2d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-k, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, ceph=True) 2024-05-07T02:46:56.591 INFO:journalctl@ceph.mds.k.smithi027.stdout:May 07 02:46:56 smithi027 bash[73714]: ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-k 2024-05-07T02:46:56.816 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mds.k.service' 2024-05-07T02:46:56.848 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-07T02:46:56.848 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-07T02:46:56.849 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-07T02:46:56.849 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mds.c 2024-05-07T02:46:57.166 INFO:journalctl@ceph.mds.c.smithi064.stdout:May 07 02:46:56 smithi064 systemd[1]: Stopping Ceph mds.c for 9ff84c16-0c17-11ef-bc97-c7b262605968... 2024-05-07T02:46:57.440 INFO:journalctl@ceph.mds.c.smithi064.stdout:May 07 02:46:57 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-c[65500]: 2024-05-07T02:46:57.165+0000 7f5b1ab90700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-07T02:46:57.441 INFO:journalctl@ceph.mds.c.smithi064.stdout:May 07 02:46:57 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-c[65500]: 2024-05-07T02:46:57.165+0000 7f5b1ab90700 -1 mds.c *** got signal Terminated *** 2024-05-07T02:47:00.341 INFO:journalctl@ceph.mds.c.smithi064.stdout:May 07 02:47:00 smithi064 podman[73991]: 2024-05-07 02:47:00.054424609 +0000 UTC m=+3.009062966 container died a0ff8ef5c27e2d6a9aa5bb8ff64a4a70f27287736b67facc8cd17fdb5e500fef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0) 2024-05-07T02:47:00.609 INFO:journalctl@ceph.mds.c.smithi064.stdout:May 07 02:47:00 smithi064 podman[73991]: 2024-05-07 02:47:00.440618018 +0000 UTC m=+3.395256363 container cleanup a0ff8ef5c27e2d6a9aa5bb8ff64a4a70f27287736b67facc8cd17fdb5e500fef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-c, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-07T02:47:00.609 INFO:journalctl@ceph.mds.c.smithi064.stdout:May 07 02:47:00 smithi064 bash[73991]: ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-c 2024-05-07T02:47:00.885 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mds.c.service' 2024-05-07T02:47:00.922 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-07T02:47:00.922 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-07T02:47:00.922 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-07T02:47:00.923 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mds.f 2024-05-07T02:47:01.253 INFO:journalctl@ceph.mds.f.smithi064.stdout:May 07 02:47:01 smithi064 systemd[1]: Stopping Ceph mds.f for 9ff84c16-0c17-11ef-bc97-c7b262605968... 2024-05-07T02:47:01.609 INFO:journalctl@ceph.mds.f.smithi064.stdout:May 07 02:47:01 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-f[65192]: 2024-05-07T02:47:01.252+0000 7f7cdb454700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-07T02:47:01.609 INFO:journalctl@ceph.mds.f.smithi064.stdout:May 07 02:47:01 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-f[65192]: 2024-05-07T02:47:01.252+0000 7f7cdb454700 -1 mds.f *** got signal Terminated *** 2024-05-07T02:47:04.359 INFO:journalctl@ceph.mds.f.smithi064.stdout:May 07 02:47:04 smithi064 podman[74124]: 2024-05-07 02:47:04.052647267 +0000 UTC m=+2.907082472 container died 2dabcac5303a9e91fd2ad37b64d056546309fccb545277c671c211ce6f952769 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-f, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.build-date=20240430, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-07T02:47:04.778 INFO:journalctl@ceph.mds.f.smithi064.stdout:May 07 02:47:04 smithi064 podman[74124]: 2024-05-07 02:47:04.50241864 +0000 UTC m=+3.356853851 container cleanup 2dabcac5303a9e91fd2ad37b64d056546309fccb545277c671c211ce6f952769 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-f, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-07T02:47:04.779 INFO:journalctl@ceph.mds.f.smithi064.stdout:May 07 02:47:04 smithi064 bash[74124]: ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-f 2024-05-07T02:47:04.937 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mds.f.service' 2024-05-07T02:47:04.971 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-07T02:47:04.971 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-07T02:47:04.972 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-07T02:47:04.972 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mds.i 2024-05-07T02:47:05.298 INFO:journalctl@ceph.mds.i.smithi064.stdout:May 07 02:47:05 smithi064 systemd[1]: Stopping Ceph mds.i for 9ff84c16-0c17-11ef-bc97-c7b262605968... 2024-05-07T02:47:05.609 INFO:journalctl@ceph.mds.i.smithi064.stdout:May 07 02:47:05 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-i[65809]: 2024-05-07T02:47:05.297+0000 7f88f924e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-07T02:47:05.609 INFO:journalctl@ceph.mds.i.smithi064.stdout:May 07 02:47:05 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-i[65809]: 2024-05-07T02:47:05.297+0000 7f88f924e700 -1 mds.i *** got signal Terminated *** 2024-05-07T02:47:08.331 INFO:journalctl@ceph.mds.i.smithi064.stdout:May 07 02:47:08 smithi064 podman[74244]: 2024-05-07 02:47:08.062969454 +0000 UTC m=+2.868912027 container died 7d7e4fe2e2ccc47c9f87d42e2af9ed8f811a67db8bb7235f417eeec78c7e823d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-i, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, io.buildah.version=1.33.5) 2024-05-07T02:47:08.609 INFO:journalctl@ceph.mds.i.smithi064.stdout:May 07 02:47:08 smithi064 podman[74244]: 2024-05-07 02:47:08.414408238 +0000 UTC m=+3.220350821 container cleanup 7d7e4fe2e2ccc47c9f87d42e2af9ed8f811a67db8bb7235f417eeec78c7e823d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-05-07T02:47:08.609 INFO:journalctl@ceph.mds.i.smithi064.stdout:May 07 02:47:08 smithi064 bash[74244]: ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-i 2024-05-07T02:47:09.004 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mds.i.service' 2024-05-07T02:47:09.036 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-07T02:47:09.036 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-07T02:47:09.036 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-07T02:47:09.036 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mds.l 2024-05-07T02:47:09.359 INFO:journalctl@ceph.mds.l.smithi064.stdout:May 07 02:47:09 smithi064 systemd[1]: Stopping Ceph mds.l for 9ff84c16-0c17-11ef-bc97-c7b262605968... 2024-05-07T02:47:09.859 INFO:journalctl@ceph.mds.l.smithi064.stdout:May 07 02:47:09 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-l[64884]: 2024-05-07T02:47:09.368+0000 7fd9bf335700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-07T02:47:09.859 INFO:journalctl@ceph.mds.l.smithi064.stdout:May 07 02:47:09 smithi064 ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-l[64884]: 2024-05-07T02:47:09.368+0000 7fd9bf335700 -1 mds.l *** got signal Terminated *** 2024-05-07T02:47:11.359 INFO:journalctl@ceph.mds.l.smithi064.stdout:May 07 02:47:11 smithi064 podman[74364]: 2024-05-07 02:47:11.092196078 +0000 UTC m=+1.823663680 container died 62bcd6ca49bcc856ab4b8e82e1b96fec0dd6ae9d194a565d37423fd8c51d768e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-l, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.build-date=20240430) 2024-05-07T02:47:11.742 INFO:journalctl@ceph.mds.l.smithi064.stdout:May 07 02:47:11 smithi064 podman[74364]: 2024-05-07 02:47:11.39414444 +0000 UTC m=+2.125612031 container cleanup 62bcd6ca49bcc856ab4b8e82e1b96fec0dd6ae9d194a565d37423fd8c51d768e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-l, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True) 2024-05-07T02:47:11.742 INFO:journalctl@ceph.mds.l.smithi064.stdout:May 07 02:47:11 smithi064 bash[74364]: ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-l 2024-05-07T02:47:11.742 INFO:journalctl@ceph.mds.l.smithi064.stdout:May 07 02:47:11 smithi064 podman[74378]: 2024-05-07 02:47:11.619960702 +0000 UTC m=+0.527721249 container remove 62bcd6ca49bcc856ab4b8e82e1b96fec0dd6ae9d194a565d37423fd8c51d768e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-9ff84c16-0c17-11ef-bc97-c7b262605968-mds-l, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-07T02:47:11.756 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9ff84c16-0c17-11ef-bc97-c7b262605968@mds.l.service' 2024-05-07T02:47:11.787 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-07T02:47:11.788 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-07T02:47:11.788 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-07T02:47:12.115 INFO:teuthology.orchestra.run.smithi019.stdout:Deleting cluster with fsid: 9ff84c16-0c17-11ef-bc97-c7b262605968 2024-05-07T02:47:17.353 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-07T02:47:17.553 INFO:teuthology.orchestra.run.smithi027.stdout:Deleting cluster with fsid: 9ff84c16-0c17-11ef-bc97-c7b262605968 2024-05-07T02:47:20.076 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-07T02:47:20.274 INFO:teuthology.orchestra.run.smithi064.stdout:Deleting cluster with fsid: 9ff84c16-0c17-11ef-bc97-c7b262605968 2024-05-07T02:47:22.745 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-07T02:47:22.843 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-07T02:47:22.876 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-07T02:47:22.903 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-07T02:47:22.905 DEBUG:teuthology.misc:Transferring archived files from smithi019:/var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-07_01:13:22-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7695123/remote/smithi019/crash 2024-05-07T02:47:22.906 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/crash -- . 2024-05-07T02:47:22.996 INFO:teuthology.orchestra.run.smithi019.stderr:tar: /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-07T02:47:22.997 INFO:teuthology.orchestra.run.smithi019.stderr:tar: Error is not recoverable: exiting now 2024-05-07T02:47:23.000 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-07_01:13:22-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7695123/remote/smithi027/crash 2024-05-07T02:47:23.001 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/crash -- . 2024-05-07T02:47:23.051 INFO:teuthology.orchestra.run.smithi027.stderr:tar: /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-07T02:47:23.052 INFO:teuthology.orchestra.run.smithi027.stderr:tar: Error is not recoverable: exiting now 2024-05-07T02:47:23.053 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-07_01:13:22-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7695123/remote/smithi064/crash 2024-05-07T02:47:23.054 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/crash -- . 2024-05-07T02:47:23.099 INFO:teuthology.orchestra.run.smithi064.stderr:tar: /var/lib/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-07T02:47:23.099 INFO:teuthology.orchestra.run.smithi064.stderr:tar: Error is not recoverable: exiting now 2024-05-07T02:47:23.100 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-07T02:47:23.100 DEBUG:teuthology.orchestra.run.smithi019:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-07T02:47:23.214 INFO:tasks.cephadm:Compressing logs... 2024-05-07T02:47:23.214 DEBUG:teuthology.orchestra.run.smithi019:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-07T02:47:23.218 DEBUG:teuthology.orchestra.run.smithi027:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-07T02:47:23.221 DEBUG:teuthology.orchestra.run.smithi064:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-07T02:47:23.246 INFO:teuthology.orchestra.run.smithi064.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph.log 2024-05-07T02:47:23.247 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-05-07T02:47:23.247 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-osd.2.log 2024-05-07T02:47:23.247 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mds.f.log 2024-05-07T02:47:23.247 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-07T02:47:23.247 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-osd.5.log 2024-05-07T02:47:23.247 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mds.c.log 2024-05-07T02:47:23.247 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph.audit.log: /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mon.c.log 2024-05-07T02:47:23.252 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mgr.z.log 2024-05-07T02:47:23.252 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mon.c.log: /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mds.i.log 2024-05-07T02:47:23.252 INFO:teuthology.orchestra.run.smithi064.stderr: 87.6% -- replaced with /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-07T02:47:23.253 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-07T02:47:23.253 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mds.i.log: 90.4% -- replaced with /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-07T02:47:23.257 INFO:teuthology.orchestra.run.smithi027.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph.log 2024-05-07T02:47:23.258 INFO:teuthology.orchestra.run.smithi027.stderr:: No such file or directory 2024-05-07T02:47:23.258 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-07T02:47:23.258 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose --/var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph.log: /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-osd.7.log 2024-05-07T02:47:23.258 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mds.b.log 2024-05-07T02:47:23.258 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-osd.4.log 2024-05-07T02:47:23.259 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-osd.1.log 2024-05-07T02:47:23.259 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mgr.y.log 2024-05-07T02:47:23.259 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mds.h.log 2024-05-07T02:47:23.259 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-osd.10.log 2024-05-07T02:47:23.259 INFO:teuthology.orchestra.run.smithi027.stderr: 87.6%/var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mds.h.log: -- replaced with /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-07T02:47:23.259 INFO:teuthology.orchestra.run.smithi027.stderr: 90.4% -- replaced with /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-07T02:47:23.259 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-07T02:47:23.260 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mds.e.log 2024-05-07T02:47:23.260 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph.cephadm.log: 83.5% -- replaced with /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-07T02:47:23.260 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-07T02:47:23.261 INFO:teuthology.orchestra.run.smithi064.stderr: 92.2% -- replaced with /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mgr.z.log.gz 2024-05-07T02:47:23.262 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-osd.11.log 2024-05-07T02:47:23.262 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-osd.8.log 2024-05-07T02:47:23.262 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph.cephadm.log: /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-osd.11.log: 83.5% -- replaced with /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-07T02:47:23.262 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-07T02:47:23.264 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mds.e.log: 92.2% -- replaced with /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mgr.y.log.gz 2024-05-07T02:47:23.264 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mon.b.log 2024-05-07T02:47:23.274 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mds.l.log 2024-05-07T02:47:23.275 INFO:teuthology.orchestra.run.smithi019.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-07T02:47:23.275 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph.log 2024-05-07T02:47:23.276 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mds.k.log 2024-05-07T02:47:23.277 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-07T02:47:23.277 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mds.g.log 2024-05-07T02:47:23.279 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mds.j.log 2024-05-07T02:47:23.280 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-07T02:47:23.280 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-07T02:47:23.281 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-osd.3.log 2024-05-07T02:47:23.281 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mds.j.log: 87.6% -- replaced with /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-07T02:47:23.281 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-osd.0.log 2024-05-07T02:47:23.281 INFO:teuthology.orchestra.run.smithi019.stderr: 90.3% -- replaced with /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-07T02:47:23.282 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mgr.x.log 2024-05-07T02:47:23.283 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mon.a.log 2024-05-07T02:47:23.284 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mgr.x.log: 94.5% -- replaced with /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mds.g.log.gz 2024-05-07T02:47:23.284 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-osd.6.log 2024-05-07T02:47:23.285 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose/var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mon.a.log: -- /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-07T02:47:23.287 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mds.a.log 2024-05-07T02:47:23.288 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-07T02:47:23.289 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph.cephadm.log: 83.8% -- replaced with /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-07T02:47:23.291 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-07T02:47:23.292 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-07T02:47:23.293 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mds.d.log 2024-05-07T02:47:23.295 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mds.l.log: 94.3% -- replaced with /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mds.l.log.gz 2024-05-07T02:47:23.296 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-osd.9.log 2024-05-07T02:47:23.299 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-07T02:47:23.300 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-05-07T02:47:23.307 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/cephadm.log: 91.0% -- replaced with /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-07T02:47:23.310 INFO:teuthology.orchestra.run.smithi064.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-07T02:47:23.310 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/ceph-client.admin.log: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-05-07T02:47:23.313 INFO:teuthology.orchestra.run.smithi019.stderr: 90.9% -- replaced with /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-07T02:47:23.330 INFO:teuthology.orchestra.run.smithi019.stderr: 94.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-07T02:47:23.783 INFO:teuthology.orchestra.run.smithi027.stderr: 94.8% -- replaced with /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mds.k.log.gz 2024-05-07T02:47:23.917 INFO:teuthology.orchestra.run.smithi064.stderr: 91.7% -- replaced with /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mon.c.log.gz 2024-05-07T02:47:23.937 INFO:teuthology.orchestra.run.smithi019.stderr: 89.8% -- replaced with /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mgr.x.log.gz 2024-05-07T02:47:24.069 INFO:teuthology.orchestra.run.smithi027.stderr: 95.5% -- replaced with /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mds.e.log.gz 2024-05-07T02:47:24.147 INFO:teuthology.orchestra.run.smithi019.stderr: 95.0% -- replaced with /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mds.d.log.gz 2024-05-07T02:47:24.173 INFO:teuthology.orchestra.run.smithi019.stderr: 95.0% -- replaced with /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mds.j.log.gz 2024-05-07T02:47:24.319 INFO:teuthology.orchestra.run.smithi027.stderr: 91.9% -- replaced with /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mon.b.log.gz 2024-05-07T02:47:24.402 INFO:teuthology.orchestra.run.smithi064.stderr: 95.6% -- replaced with /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mds.f.log.gz 2024-05-07T02:47:24.933 INFO:teuthology.orchestra.run.smithi019.stderr: 90.4% -- replaced with /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mon.a.log.gz 2024-05-07T02:47:29.436 INFO:teuthology.orchestra.run.smithi064.stderr: 94.0% -- replaced with /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mds.c.log.gz 2024-05-07T02:47:29.613 INFO:teuthology.orchestra.run.smithi019.stderr: 91.6% -- replaced with /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-osd.6.log.gz 2024-05-07T02:47:29.872 INFO:teuthology.orchestra.run.smithi064.stderr: 91.6% -- replaced with /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-osd.11.log.gz 2024-05-07T02:47:30.294 INFO:teuthology.orchestra.run.smithi027.stderr: 91.8% -- replaced with /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-osd.4.log.gz 2024-05-07T02:47:30.304 INFO:teuthology.orchestra.run.smithi019.stderr: 91.3% -- replaced with /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-osd.9.log.gz 2024-05-07T02:47:30.462 INFO:teuthology.orchestra.run.smithi019.stderr: 91.8% -- replaced with /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-osd.3.log.gz 2024-05-07T02:47:30.498 INFO:teuthology.orchestra.run.smithi064.stderr: 92.1% -- replaced with /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-osd.8.log.gz 2024-05-07T02:47:30.653 INFO:teuthology.orchestra.run.smithi064.stderr: 91.6% -- replaced with /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-osd.2.log.gz 2024-05-07T02:47:30.793 INFO:teuthology.orchestra.run.smithi027.stderr: 91.9% -- replaced with /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-osd.1.log.gz 2024-05-07T02:47:31.020 INFO:teuthology.orchestra.run.smithi019.stderr: 92.1% -- replaced with /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-osd.0.log.gz 2024-05-07T02:47:31.097 INFO:teuthology.orchestra.run.smithi027.stderr: 92.0% -- replaced with /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-osd.10.log.gz 2024-05-07T02:47:34.229 INFO:teuthology.orchestra.run.smithi064.stderr: 93.9% -- replaced with /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mds.i.log.gz 2024-05-07T02:47:34.704 INFO:teuthology.orchestra.run.smithi027.stderr: 91.4% -- replaced with /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-osd.7.log.gz 2024-05-07T02:47:34.906 INFO:teuthology.orchestra.run.smithi064.stderr: 91.2% -- replaced with /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-osd.5.log.gz 2024-05-07T02:47:34.908 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-05-07T02:47:34.908 INFO:teuthology.orchestra.run.smithi064.stderr:real 0m11.672s 2024-05-07T02:47:34.908 INFO:teuthology.orchestra.run.smithi064.stderr:user 0m50.429s 2024-05-07T02:47:34.909 INFO:teuthology.orchestra.run.smithi064.stderr:sys 0m1.377s 2024-05-07T02:47:34.956 INFO:teuthology.orchestra.run.smithi019.stderr: 93.8% -- replaced with /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mds.a.log.gz 2024-05-07T02:47:34.960 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-05-07T02:47:34.960 INFO:teuthology.orchestra.run.smithi019.stderr:real 0m11.709s 2024-05-07T02:47:34.960 INFO:teuthology.orchestra.run.smithi019.stderr:user 0m40.434s 2024-05-07T02:47:34.960 INFO:teuthology.orchestra.run.smithi019.stderr:sys 0m2.967s 2024-05-07T02:47:35.244 INFO:teuthology.orchestra.run.smithi027.stderr: 93.9% -- replaced with /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mds.h.log.gz 2024-05-07T02:47:46.239 INFO:teuthology.orchestra.run.smithi027.stderr: 93.7% -- replaced with /var/log/ceph/9ff84c16-0c17-11ef-bc97-c7b262605968/ceph-mds.b.log.gz 2024-05-07T02:47:46.240 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-05-07T02:47:46.240 INFO:teuthology.orchestra.run.smithi027.stderr:real 0m23.004s 2024-05-07T02:47:46.241 INFO:teuthology.orchestra.run.smithi027.stderr:user 1m8.751s 2024-05-07T02:47:46.241 INFO:teuthology.orchestra.run.smithi027.stderr:sys 0m1.883s 2024-05-07T02:47:46.241 INFO:tasks.cephadm:Archiving logs... 2024-05-07T02:47:46.242 DEBUG:teuthology.misc:Transferring archived files from smithi019:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-07_01:13:22-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7695123/remote/smithi019/log 2024-05-07T02:47:46.242 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-07T02:47:51.688 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-07_01:13:22-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7695123/remote/smithi027/log 2024-05-07T02:47:51.689 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-07T02:48:00.646 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-07_01:13:22-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7695123/remote/smithi064/log 2024-05-07T02:48:00.647 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-07T02:48:06.914 INFO:tasks.cephadm:Removing cluster... 2024-05-07T02:48:06.915 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 --force 2024-05-07T02:48:07.256 INFO:teuthology.orchestra.run.smithi019.stdout:Deleting cluster with fsid: 9ff84c16-0c17-11ef-bc97-c7b262605968 2024-05-07T02:48:07.993 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 --force 2024-05-07T02:48:08.193 INFO:teuthology.orchestra.run.smithi027.stdout:Deleting cluster with fsid: 9ff84c16-0c17-11ef-bc97-c7b262605968 2024-05-07T02:48:08.530 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9ff84c16-0c17-11ef-bc97-c7b262605968 --force 2024-05-07T02:48:08.729 INFO:teuthology.orchestra.run.smithi064.stdout:Deleting cluster with fsid: 9ff84c16-0c17-11ef-bc97-c7b262605968 2024-05-07T02:48:09.046 INFO:tasks.cephadm:Removing cephadm ... 2024-05-07T02:48:09.046 DEBUG:teuthology.orchestra.run.smithi019:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-07T02:48:09.106 DEBUG:teuthology.orchestra.run.smithi027:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-07T02:48:09.122 DEBUG:teuthology.orchestra.run.smithi064:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-07T02:48:09.138 INFO:tasks.cephadm:Teardown complete 2024-05-07T02:48:09.138 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-07T02:48:09.149 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-07T02:48:09.150 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-07T02:48:09.155 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-07T02:48:09.164 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-07T02:48:09.313 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-07T02:48:09.324 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-07T02:48:09.324 DEBUG:teuthology.orchestra.run.smithi019:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-07T02:48:09.332 DEBUG:teuthology.orchestra.run.smithi027:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-07T02:48:09.335 DEBUG:teuthology.orchestra.run.smithi064:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-07T02:48:09.347 INFO:teuthology.orchestra.run.smithi027.stderr:bash: line 1: ntpq: command not found 2024-05-07T02:48:09.349 INFO:teuthology.orchestra.run.smithi064.stderr:bash: line 1: ntpq: command not found 2024-05-07T02:48:09.352 INFO:teuthology.orchestra.run.smithi027.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-07T02:48:09.352 INFO:teuthology.orchestra.run.smithi027.stdout:=============================================================================== 2024-05-07T02:48:09.352 INFO:teuthology.orchestra.run.smithi027.stdout:^* hv01.front.sepia.ceph.com 3 6 377 52 +386us[ +346us] +/- 47ms 2024-05-07T02:48:09.352 INFO:teuthology.orchestra.run.smithi027.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 50 -47us[ -47us] +/- 70ms 2024-05-07T02:48:09.352 INFO:teuthology.orchestra.run.smithi027.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 55 -462us[ -502us] +/- 60ms 2024-05-07T02:48:09.352 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-07T02:48:09.353 INFO:teuthology.orchestra.run.smithi064.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-07T02:48:09.353 INFO:teuthology.orchestra.run.smithi064.stdout:=============================================================================== 2024-05-07T02:48:09.353 INFO:teuthology.orchestra.run.smithi064.stdout:^* hv01.front.sepia.ceph.com 3 6 377 51 +325us[ +342us] +/- 47ms 2024-05-07T02:48:09.354 INFO:teuthology.orchestra.run.smithi064.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 51 -97us[ -81us] +/- 70ms 2024-05-07T02:48:09.354 INFO:teuthology.orchestra.run.smithi064.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 50 -524us[ -524us] +/- 60ms 2024-05-07T02:48:09.354 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-07T02:48:09.383 INFO:teuthology.orchestra.run.smithi019.stderr:bash: line 1: ntpq: command not found 2024-05-07T02:48:09.393 INFO:teuthology.orchestra.run.smithi019.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-07T02:48:09.393 INFO:teuthology.orchestra.run.smithi019.stdout:=============================================================================== 2024-05-07T02:48:09.393 INFO:teuthology.orchestra.run.smithi019.stdout:^* hv01.front.sepia.ceph.com 3 6 377 51 +369us[ +328us] +/- 47ms 2024-05-07T02:48:09.393 INFO:teuthology.orchestra.run.smithi019.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 52 -53us[ -94us] +/- 70ms 2024-05-07T02:48:09.393 INFO:teuthology.orchestra.run.smithi019.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 52 -468us[ -510us] +/- 60ms 2024-05-07T02:48:09.393 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-07T02:48:09.395 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-07T02:48:09.406 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-07T02:48:09.406 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-07T02:48:09.416 DEBUG:teuthology.orchestra.run.smithi019:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-07T02:48:09.658 DEBUG:teuthology.orchestra.run.smithi027:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-07T02:48:09.729 DEBUG:teuthology.orchestra.run.smithi064:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-07T02:48:09.799 DEBUG:teuthology.orchestra.run.smithi019:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-07T02:48:09.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-07T02:48:09.899 DEBUG:teuthology.orchestra.run.smithi027:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-07T02:48:09.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-07T02:48:09.929 DEBUG:teuthology.orchestra.run.smithi064:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-07T02:48:09.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-07T02:48:09.957 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-07T02:48:09.968 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-07T02:48:09.979 INFO:teuthology.task.internal:Duration was 2471.222621 seconds 2024-05-07T02:48:09.979 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-07T02:48:09.989 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-07T02:48:09.990 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-07T02:48:09.998 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-07T02:48:10.000 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-07T02:48:10.038 INFO:teuthology.orchestra.run.smithi027.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-07T02:48:10.039 INFO:teuthology.orchestra.run.smithi064.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-07T02:48:10.149 INFO:teuthology.orchestra.run.smithi019.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-07T02:48:10.732 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-07T02:48:10.732 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi019.front.sepia.ceph.com 2024-05-07T02:48:10.733 DEBUG:teuthology.orchestra.run.smithi019:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-07T02:48:10.798 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi027.front.sepia.ceph.com 2024-05-07T02:48:10.799 DEBUG:teuthology.orchestra.run.smithi027:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-07T02:48:10.820 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi064.front.sepia.ceph.com 2024-05-07T02:48:10.820 DEBUG:teuthology.orchestra.run.smithi064:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-07T02:48:10.843 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-07T02:48:10.844 DEBUG:teuthology.orchestra.run.smithi019:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-07T02:48:10.851 DEBUG:teuthology.orchestra.run.smithi027:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-07T02:48:10.863 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-07T02:48:10.938 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-07T02:48:10.938 DEBUG:teuthology.orchestra.run.smithi019:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-07T02:48:10.943 DEBUG:teuthology.orchestra.run.smithi027:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-07T02:48:10.946 DEBUG:teuthology.orchestra.run.smithi064:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-07T02:48:12.148 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-07T02:48:12.212 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-07T02:48:12.212 DEBUG:teuthology.orchestra.run.smithi019:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-07T02:48:12.308 DEBUG:teuthology.orchestra.run.smithi027:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-07T02:48:12.345 DEBUG:teuthology.orchestra.run.smithi064:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-07T02:48:12.386 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-07T02:48:12.463 DEBUG:teuthology.orchestra.run.smithi019:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-07T02:48:12.484 DEBUG:teuthology.orchestra.run.smithi027:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-07T02:48:12.494 DEBUG:teuthology.orchestra.run.smithi064:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-07T02:48:12.522 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern = core 2024-05-07T02:48:12.527 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = core 2024-05-07T02:48:12.580 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern = core 2024-05-07T02:48:12.625 DEBUG:teuthology.orchestra.run.smithi019:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-07T02:48:12.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-07T02:48:12.683 DEBUG:teuthology.orchestra.run.smithi027:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-07T02:48:12.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-07T02:48:12.712 DEBUG:teuthology.orchestra.run.smithi064:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-07T02:48:12.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-07T02:48:12.741 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-07T02:48:12.771 INFO:teuthology.task.internal:Transferring archived files... 2024-05-07T02:48:12.772 DEBUG:teuthology.misc:Transferring archived files from smithi019:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-07_01:13:22-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7695123/remote/smithi019 2024-05-07T02:48:12.772 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-07T02:48:12.965 DEBUG:teuthology.misc:Transferring archived files from smithi027:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-07_01:13:22-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7695123/remote/smithi027 2024-05-07T02:48:12.965 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-07T02:48:13.066 DEBUG:teuthology.misc:Transferring archived files from smithi064:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-07_01:13:22-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7695123/remote/smithi064 2024-05-07T02:48:13.066 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-07T02:48:13.172 INFO:teuthology.task.internal:Removing archive directory... 2024-05-07T02:48:13.173 DEBUG:teuthology.orchestra.run.smithi019:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-07T02:48:13.182 DEBUG:teuthology.orchestra.run.smithi027:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-07T02:48:13.185 DEBUG:teuthology.orchestra.run.smithi064:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-07T02:48:13.236 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-07T02:48:13.259 INFO:teuthology.task.internal:Not uploading archives. 2024-05-07T02:48:13.259 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-07T02:48:13.345 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-07T02:48:13.346 DEBUG:teuthology.orchestra.run.smithi019:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-07T02:48:13.356 DEBUG:teuthology.orchestra.run.smithi027:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-07T02:48:13.365 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-07T02:48:13.379 INFO:teuthology.orchestra.run.smithi027.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 7 02:48 /home/ubuntu/cephtest 2024-05-07T02:48:13.385 INFO:teuthology.orchestra.run.smithi064.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 7 02:48 /home/ubuntu/cephtest 2024-05-07T02:48:13.409 INFO:teuthology.orchestra.run.smithi019.stdout: 393247 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 7 02:48 /home/ubuntu/cephtest 2024-05-07T02:48:13.412 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-07T02:48:13.543 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-07T02:48:13.645 INFO:teuthology.nuke:Checking targets against current locks 2024-05-07T02:48:13.672 DEBUG:teuthology.nuke:shortname: smithi019 2024-05-07T02:48:13.673 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-07T02:48:13.698 DEBUG:teuthology.nuke:shortname: smithi027 2024-05-07T02:48:13.698 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-07T02:48:13.704 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi019.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-07_01:13:22-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7695123', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-07 01:57:12.361367', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMExTw6uTcQ/LDLck8NATCpkcpMwEYT+RWGfuCXeLD0RI0/uVnULJaFft0JA3DigNOV7Mv4W4PB3Ws7Rt1rEChc='} 2024-05-07T02:48:13.736 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi027.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-07_01:13:22-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7695123', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-07 01:57:12.359806', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNmuBCEQzBakC2p701jvGrHf2GNbVNRidlwag4DGI3+QDp6ivApD5TdsnF8kERWUrhuA0W2Q30cqf0HPhNJ4GZA='} 2024-05-07T02:48:13.737 DEBUG:teuthology.nuke:shortname: smithi064 2024-05-07T02:48:13.737 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-07T02:48:13.765 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi064.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-07_01:13:22-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7695123', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-07 01:57:12.362690', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJPPQB5lzfuaGlCVXiaJCgSaV7HuDbtLw4hpOTnLsSiabhRbvPXRoJWA8y2KFELtO4jPEtIaQup9Avy9rInFoJE='} 2024-05-07T02:48:13.780 INFO:teuthology.orchestra.console.smithi019:Power off 2024-05-07T02:48:13.780 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-07T02:48:13.806 DEBUG:teuthology.orchestra.console.smithi019:power off output: Chassis Power Control: Down/Off 2024-05-07T02:48:13.806 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-07T02:48:13.824 INFO:teuthology.orchestra.console.smithi027:Power off 2024-05-07T02:48:13.824 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-07T02:48:13.837 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is on 2024-05-07T02:48:13.856 DEBUG:teuthology.orchestra.console.smithi027:power off output: Chassis Power Control: Down/Off 2024-05-07T02:48:13.856 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-07T02:48:13.875 INFO:teuthology.orchestra.console.smithi064:Power off 2024-05-07T02:48:13.876 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-07T02:48:13.886 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is on 2024-05-07T02:48:13.918 DEBUG:teuthology.orchestra.console.smithi064:power off output: Chassis Power Control: Down/Off 2024-05-07T02:48:13.919 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-07T02:48:13.947 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is on 2024-05-07T02:48:17.838 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-07T02:48:17.888 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-07T02:48:17.949 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-07T02:48:17.961 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is on 2024-05-07T02:48:17.998 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is on 2024-05-07T02:48:18.062 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is on 2024-05-07T02:48:21.966 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-07T02:48:21.999 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-07T02:48:22.063 DEBUG:teuthology.orchestra.console.smithi064:pexpect command: ipmitool -H smithi064.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-07T02:48:22.077 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is off 2024-05-07T02:48:22.111 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is on 2024-05-07T02:48:22.173 DEBUG:teuthology.orchestra.console.smithi064:check power output: Chassis Power is off 2024-05-07T02:48:22.178 INFO:teuthology.orchestra.console.smithi019:Power off completed 2024-05-07T02:48:22.274 INFO:teuthology.orchestra.console.smithi064:Power off completed 2024-05-07T02:48:26.116 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-07T02:48:26.226 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is off 2024-05-07T02:48:26.327 INFO:teuthology.orchestra.console.smithi027:Power off completed 2024-05-07T02:48:26.434 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/suites/dbench}} duration: 2471.2226214408875 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi019 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=366c9130c497dee9e73178650cf710ed21269b18 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_pdonnell@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=c25603c46a8b4141989ee4e8e030368c status: fail success: false 2024-05-07T02:48:26.435 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-07T02:48:26.536 INFO:teuthology.run:FAIL